UNSOLVED

arosen_tca

updated

11 years ago

A

arosen_tca

1 Rookie

•

6 Posts

0

2424

April 1st, 2016 07:00

VNXe3200 Performance issues

I recently purchased an EMC VNXe 3200. I have deployed Vsphere 6.0 and several VMs. While things in general work ok, some of the access times are seeming slow. I have downloaded  HD Tune Pro and compared the VNXe to my 8-year old physical server, and the results are confusing me.

Some background on the VNXe system:

  • 900GB 10K RPM disks
  • The tested VMDK is sitting on a pool of 15 disks using 3x(4+1) RAID 5 sets
  • Thin provisioned VMDK sitting on a thin provisioned datastore
  • Activity in general is < 200 IOPS
  • Connectivity is 2x1GB w/jumbo frames

The physical server I tested has 2x146GB 15K drives in a RAID 1 setup

As you can see, the initial read on the VNXe starts off very slow for a while, then ramps up high. My concern is that general DB reads are going to be affected by this slow start since we do more of that than large sequential transfers. At times of what I would say is low useage, the Windows Disk sec/Read rate can be 10-20ms, and the Write can be 20-100ms.

Any thoughts on how I can improve my performance, and what would cause the graph displayed below?

VNXe:

01-April-2016_10-33.png

Physical:

01-April-2016_10-32.png

  • ZaphodB

    195 Posts

    1019

    0

    Posted April 2nd, 2016 11:00

    You network connectivity, 2 x 1Gb, gets you ~200MB/sec bandwidth.  Despite what HD Tune says, you aren't getting 1300MB/sec, or 839MB/sec. ... or anything close to that... over that connection.

  • kelleg

    6 Operator

    •

    4537 Posts

    1019

    0

    Posted April 7th, 2016 08:00

    Some thoughts:

    1. Comparing a standalone server with local hard disks to an array is not an direct comparison. The local server/disk has on-board caching, memory caching, disk caching, short data path and much high "single user" performance. An array is designed to be connected to multiple servers, has a longer data path and latency is affected by software driver layers. A standalone server with a local disk will always perform faster on most benchmarks designed for a standalone server/disk.

    2. As pointed out, a single 1Gb (100GB) ethernet connection will not provide 100MB/s performance - it's always less. With two 1Gb ports you will see something less than 200MB/s - probably more like 160-180MB/s is real production use. This is the overhead for ethernet.

    3. Each 10K RPM SAS disk can provide about 120 IO/sec - this is based on small block (less than 32KB IO Size), random IO's with a mix of reads/writes (80% Read and 20% Writes). With a 4+1 Raid 5, 10K disks can provide about 600 IOPS (5 * 120). With 15 disks you should be able to get about 1800 IOPS.

    4. Each 10K RPM SAS disk can provide about 12MB/s per disk - this is for IO Sized over 32KB. As IO Size increases, the amount of IO/sec decrease. For 15 disks, that would be 12MB/s * 15 = 180MB/s. Disks have two metrics for performances - Throughput (IIOPS) and Bandwidth (MB/s).

    glen

  • Rainer_EMC

    6 Operator

    •

    8645 Posts

    1019

    0

    Posted April 7th, 2016 09:00

    Also with multiple Ethernet port it depends on the protocol and network config whether both are utilized for one client / application

    Even if they are trunked a single client usually still uses only one port

  • arosen_tca

    1 Rookie

    •

    6 Posts

    1019

    0

    Posted April 18th, 2016 07:00

    Glen -

    Sorry for letting the response lag. I could have sworn I posted a reply the other week, but I guess I was dreaming or something.

    You are right, the comparison to the physical server is unfair given the differences in architecture. It's also really not the level of performance I am expecting. My main concerns is looking at Windows performance monitor and seeing the latencies hit too high on a pretty low use system (usually < 300 IOPS), where I am worried about degradation in SQL performance. And while I don't have a SSD cache layer on these system, I was expecting that the DRAM write cache would handle our low load better and keep the write latencies from being as high as I'm seeing.

    Unfortunately, I don't have detailed latency and throughput utilization from the old physical server prior to the migration to do a better comparison on. I am going more by perceived delays rather than concrete numbers, which makes troubleshooting issues that much hard.

    Adam

  • kelleg

    6 Operator

    •

    4537 Posts

    1019

    0

    Posted April 21st, 2016 14:00

    If you believe there are latency issues with the array, you could open a service request with EMC to check out the array.

    glen