UNSOLVED

bmalone11

updated

14 years ago

B

bmalone11

33 Posts

0

213695

August 14th, 2012 10:00

vfoglight unbearably slow

Does anyone have any tips for improving vfoglight performance? We have a dedicated physical ms sql server hosting the db. The vfog server is a vm with lots of virtual resources. We've tried tweaking the java settings and removing all the community bundles but it's still so slow that at best we can use it for a few monits before it stops responding. Even when it works you have to wait at least 15 seconds between clicks to navigate the interface. Quest support has not been helpful at all with this and we are geting desperate.

  • 652

    0

    Posted August 14th, 2012 11:00

    Can you give me a little details about number of VMs and how long it has been running?  Do you have all of the default alarms running?  Also the first thing to do is go into the VM settings and reserve the full amount of memory.  We run java and need a contiguous memory block for our heap. 

  • 652

    0

    Posted August 14th, 2012 11:00

    What are the settings you have in your server.config?  Also run this script in the script console/editor and let me know the number of rows.

    import groovy.sql.Sql;

    import com.quest.nitro.service.util.JDBCHelper;

    msg = new StringBuilder();

    try{

    sql = new Sql(JDBCHelper.getDataSource());

    msg.append("Alarms Table Count: \n")

    msg.append(sql.rows("select count(*) from alarm_alarm"));

    }catch (Exception ex){

    msg.append("Exception"+ex);

    }

    return msg.toString();

  • bmalone11

    33 Posts

    652

    0

    Posted August 14th, 2012 11:00

    We have about 1000 vms and have been running around a year. We already memory for java and increase the cache size by editing the server.config file.

  • bmalone11

    33 Posts

    652

    0

    Posted August 14th, 2012 12:00

    We were told to add these lines to the server.config file

     

    server.vm.option0 = "-Xms12288m";

    server.vm.option1 = "-Xmx12288m";

    The script returned: 2377055

    server.vm.option2 = "-XX:ReservedCodeCacheSize=128m";

  • john_s_main

    132 Posts

    652

    0

    Posted August 14th, 2012 12:00

    Have hope.  We have 13000 VMs on a single server, with 1 - 2 second response time pulling up a table with all 13000 rows.

    Thomas is awesome, and will certainly be able to assist you.

    I'm surprised that support has not been as helpful as you would like, they are among the best support around.

    The reason that vBundle, in particular, needs to be removed is that it has some derived metrics which are known to cause performance issues.

    Other things which could be helpful:

    • VM configuration
      • how many cores,
      • how much vRAM allocated
      • screen shot of the vmExplorer page for the VM
    • ESX host configuration/performance
      • Core count
      • RAM
      • screen shot of the vmExplorer page for the ESX host
  • bmalone11

    33 Posts

    652

    0

    Posted August 14th, 2012 12:00

    I'm hesitant to remove vBundle as probably 9 of the 10 dashboards we use rely on this add-on. I'm not sure vFoglight is worthwhile for us without it.

  • john_s_main

    132 Posts

    652

    0

    Posted August 14th, 2012 12:00

    Understand, it has some useful tools, no doubt.

    Would simply be a diagnostic step, and would only need to disable the vBundle temporarily, not completely uninstall it.

    (Once determined that vBundle is part of the issue, we can then track down which part is the issue, and remediate that piece.)

    Did you get those screen shots?

  • bmalone11

    33 Posts

    652

    0

    Posted August 14th, 2012 13:00

    The vfog vm has 4 cores and 12gb ram

    1.png

    The host it's running on has 128 gb ram and 24 cores.

    2.png

  • dlarsen1

    15 Posts

    861

    0

    Posted August 14th, 2012 13:00

    First, I want to state that vBundle was last fully supported on vFoglight 6.5.1.

    vBundle 1.9b3 was the last TEST version, which was for vFoglight 6.6.0

    There are some definite issues with vBundle being used on 6.6 and later.  The derived metrics cause slowness and may contribute to churn.  I highly recommend you at least test disabling it.

    4 vCPUs is not overkill, so I'd consider adding a couple vCPUs

    Also:  You can adjust the resource share allocation of the vFoglight VM in VMware:

    Share Allocation.jpg

    Feel free to open a support case.  I don't see any vFoglight cases from you.

    Thanks,

    -dave

    vFoglight Support Engineer

  • 652

    0

    Posted August 14th, 2012 13:00

    So the issue with this is basically your alarms table.  The memory and everything else, imo, is far to large as well.  With a 1000VMs your heap shouldn't be larger then 6144m for XMS/XMX and I don't see a need for the code cache. 

    The first thing I would do is go and turn off all of the alarms in Rules Management that you aren't looking at or don't need.  As a rule, I turn off all but the agent alarms and then turn on only what is needed for a given environment.

    Secondly, the rules table needs to be cleared out.  The easiest way is via this support article https://support.quest.com/SolutionDetail.aspx?id=SOL40651  The basic problem is many screens look at alarm counts and with 2.7 million rows, the SQL statements take time to run.  Since there are alarm counts on almost all pages this can become an issue in larger environments where rules are not tuned.  Run this to purge all old alarms and your UI should almost instantly become better once it completes.