Using ES (Enterprising Storage) NAS for Microsoft Exchange Server
Overview
This document provides solution with QNAP ES1640DC enterprise storage for Microsoft Exchange Server. We prepared Windows Server 2012 R2 virtual machine (VM) installed Exchange Server 2013 and test by Microsoft JetStress Utility to see IO performance.
Each Windows Server 2012 R2 VM has attached 4 iSCSI LUNs provided by QNAP ES1640DC to store databases and its’ logs. In addition, each Exchange Server has been set up Multipath I/O (MPIO) to have failover and load balancing capability during the messaging operation.
Infrastructure Component
System Network Configuration
| Primary Site Storage Network Settings | ||
|---|---|---|
| Role | IP | Description |
| SCA Management | 192.168.200.101 | Management IP in SCB |
| SCA Ethernet 1 | 10.1.1.101 | 10GBase-T Lan port 1 in SCA |
| SCA Ethernet 2 | 10.1.2.101 | 10GBase-T Lan port 2 in SCA |
| SCB Management | 192.168.200.102 | Management IP in SCB |
| SCB Ethernet 1 | 10.1.1.102 | 10GBase-T Lan port 1 in SCB |
| SCB Ethernet 2 | 10.1.2.102 | 10GBase-T Lan port 2 in SCB |
| Primary Site ESXi Cluster Network Settings | ||
|---|---|---|
| Role | IP | Description |
| ESXi 1 Management | 192.168.200.1 | Management IP in ESXi 1 |
| ESXi 1 Ethernet1 | 10.1.1.11 | 10G Data port 1 in ESXi 1 |
| ESXi 1 Ethernet2 | 10.1.2.11 | 10G Data port 2 in ESXi 1 |
| ESXi 2 Management | 192.168.200.2 | Management IP in ESXi 2 |
| ESXi 2 Ethernet1 | 10.1.1.12 | 10G Data port 1 in ESXi 2 |
| ESXi 2 Ethernet2 | 10.1.2.12 | 10G Data port 2 in ESXi 2 |
| Secondary Site Storage Network Settings | ||
|---|---|---|
| Role | IP | Description |
| SCA Management | 192.168.201.101 | Management IP in SCB |
| SCA Ethernet 1 | 10.1.3.101 | 10GBase-T Lan port 1 in SCA |
| SCA Ethernet 2 | 10.1.4.101 | 10GBase-T Lan port 2 in SCA |
| SCB Management | 192.168.201.102 | Management IP in SCB |
| SCB Ethernet 1 | 10.1.3.102 | 10GBase-T Lan port 1 in SCB |
| SCB Ethernet 2 | 10.1.4.102 | 10GBase-T Lan port 2 in SCB |
| Secondary Site ESXi Cluster Network Settings | ||
|---|---|---|
| Role | IP | Description |
| ESXi 1 Management | 192.168.201.1 | Management IP in ESXi 1 |
| ESXi 1 Ethernet1 | 10.1.3.11 | 10G Data port 1 in ESXi 1 |
| ESXi 1 Ethernet2 | 10.1.4.11 | 10G Data port 2 in ESXi 1 |
| ESXi 2 Management | 192.168.201.2 | Management IP in ESXi 2 |
| ESXi 2 Ethernet1 | 10.1.3.12 | 10G Data port 1 in ESXi 2 |
| ESXi 2 Ethernet2 | 10.1.4.12 | 10G Data port 2 in ESXi 2 |
QNAP ES1640dc Storage Configuration
- 4*200GB 2.5” SAS SSD for Cache Acceleration.
- 12*1TB SAS HDD with RAID1+0 pool.
- Add 4*250GB iSCSI LUNs for database, 2*64GB iSCSI LUNs for database log and enabled SSD Cache for each LUN.
ESXi Server Configuration
- Supermicro SYS-6028U-TR4T+ with Intel® Xeon® Processor E5-2620v3 and 256GB RAM
- VMware vSphere 6.0 Update 1b (ESXi ISO Build 3380124)
Exchange Server Configuration
- Exchange 2013 drivers (ESE version 15.00.1130.002)
- Microsoft JetStress Utility (JetStress version 15.01.0318.000).
- 4*Windows 2012 R2 Virtual machines with 4 vCPU & 16 GB RAM
- Attached 4*250GB iSCSI drives for database, 2*64GB iSCSI drives for database log and formatted with 64KB allocation unit size each Exchange server node.
- Configure VM vNIC adapter to VMXNET 3 for optimized network performance.
Solution description
Exchange Server Configuration
Number of DAG : 1
Number of Server per DAG : 4 (2 for primary site, 2 for secondary site)
Number of Exchange mailboxes : 4,000 mailboxes (1,000 mailboxes per server)
Estimated IOPS per mailbox : 0.168 (250 messages sent and received per mailbox per day)
Size of each Mailbox : 2GB
Number of database per Exchange server : 4 (2 active and 2 passive)
Copies of mailbox per database : 2 (2 active and 2 passive)
Note:
The Technet Blog Exchange Sizing Guide provide the table to compute the required IOPS per mailbox user (0.168) for 250 messages sent or received per mailbox per day.
Test results summary
Storage performance results
The JetStress are run in parallel across 4 VMs with 4 databases each Exchange server.
After JetStress test finished, it shows that one of Exchange Server’s Target transaction I/O per second is 168, and Achieved transaction I/O per second is about 232.
Conclusion
According to JetStress reports of all Exchange Server, the total Achieved transaction I/O per second in this DAG is above 1100 IOPS for serving 4000 mailbox users.
Appendix – Exchange-Server1 Performance Test Result Report
Test Summary
| Overall Test Result | Pass |
| Machine Name | JETSTRESS-TEST01 |
| Test Description | |
| Test Start Time | 1/18/2016 9:58:08 AM |
| Test End Time | 1/18/2016 12:11:09 PM |
| Collection Start Time | 1/18/2016 10:11:17 AM |
| Collection End Time | 1/18/2016 12:10:54 PM |
| Jetstress Version | 15.01.0318.000 |
| ESE Version | 15.00.1130.002 |
| Operating System | Windows Server 2012 R2 Standard (6.2.9200.0) |
| Performance Log | C:\JetStress_Report\HDD_RAID10_MP_1000_0.168_2GB(64k+SSD)\Tuning_2016_1_18_10_3_3.blg C:\JetStress_Report\HDD_RAID10_MP_1000_0.168_2GB(64k+SSD)\Performance_2016_1_18_10_11_1.blg |
Database Sizing and Throughput
| Achieved Transactional I/O per Second | 232.184 |
| Target Transactional I/O per Second | 168 |
| Initial Database Size (bytes) | 2098108301312 |
| Final Database Size (bytes) | 2098880053248 |
| Database Files (Count) | 4 |
Jetstress System Parameters
| Thread Count | 5 |
| Minimum Database Cache | 128.0 MB |
| Maximum Database Cache | 1024.0 MB |
| Insert Operations | 40% |
| Delete Operations | 20% |
| Replace Operations | 5% |
| Read Operations | 35% |
| Lazy Commits | 70% |
| Run Background Database Maintenance | True |
| Number of Copies per Database | 2 |
Database Configuration
| Instance900.1 | Log path: H:\log1 Database: D:\Jetstress001001.edb |
| Instance900.2 | Log path: H:\log2 Database: E:\Jetstress002001.edb |
| Instance900.3 | Log path: I:\log3 Database: F:\Jetstress003001.edb |
| Instance900.4 | Log path: I:\log4 Database: G:\Jetstress004001.edb |
Transactional I/O Performance
|
MSExchange Database ==> Instances |
I/O Database Reads Average Latency (msec) |
I/O Database Writes Average Latency (msec) |
I/O Database Reads/sec |
I/O Database Writes/sec |
I/O Database Reads Average Bytes |
I/O Database Writes Average Bytes |
I/O Log Reads Average Latency (msec) |
I/O Log Writes Average Latency (msec) |
I/O Log Reads/sec |
I/O Log Writes/sec |
I/O Log Reads Average Bytes |
I/O Log Writes Average Bytes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Instance900.1 | 14.404 | 2.363 | 37.194 | 20.644 | 33346.816 | 36932.486 | 0.000 | 1.063 | 0.000 | 5.325 | 0.000 | 20625.184 |
| Instance900.2 | 14.178 | 2.401 | 37.150 | 20.335 | 33388.017 | 36875.070 | 0.000 | 1.085 | 0.000 | 5.254 | 0.000 | 20607.456 |
| Instance900.3 | 14.044 | 2.423 | 37.402 | 20.907 | 33347.365 | 36822.408 | 0.000 | 1.079 | 0.000 | 5.319 | 0.000 | 20732.541 |
| Instance900.4 | 13.741 | 2.363 | 37.562 | 20.991 | 33315.952 | 36779.894 | 0.000 | 1.046 | 0.000 | 5.350 | 0.000 | 20513.353 |
Background Database Maintenance I/O Performance
|
MSExchange Database ==> Instances |
Database Maintenance IO Reads/sec |
Database Maintenance IO Reads Average Bytes |
|---|---|---|
| Instance900.1 | 8.993 | 261223.158 |
| Instance900.2 | 8.990 | 261295.253 |
| Instance900.3 | 8.997 | 261166.269 |
| Instance900.4 | 9.000 | 261228.101 |
Log Replication I/O Performance
|
MSExchange Database ==> Instances |
I/O Log Reads/sec |
I/O Log Reads Average Bytes |
|---|---|---|
| Instance900.1 | 0.468 | 179529.908 |
| Instance900.2 | 0.461 | 179571.373 |
| Instance900.3 | 0.470 | 181962.563 |
| Instance900.4 | 0.465 | 180502.970 |
Total I/O Performance
|
MSExchange Database ==> Instances |
I/O Database Reads Average Latency (msec) |
I/O Database Writes Average Latency (msec) |
I/O Database Reads/sec |
I/O Database Writes/sec |
I/O Database Reads Average Bytes |
I/O Database Writes Average Bytes |
I/O Log Reads Average Latency (msec) |
I/O Log Writes Average Latency (msec) |
I/O Log Reads/sec |
I/O Log Writes/sec |
I/O Log Reads Average Bytes |
I/O Log Writes Average Bytes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Instance900.1 | 14.404 | 2.363 | 46.187 | 20.644 | 77717.222 | 36932.486 | 2.539 | 1.063 | 0.468 | 5.325 | 179529.908 | 20625.184 |
| Instance900.2 | 14.178 | 2.401 | 46.141 | 20.335 | 77794.607 | 36875.070 | 2.883 | 1.085 | 0.461 | 5.254 | 179571.373 | 20607.456 |
| Instance900.3 | 14.044 | 2.423 | 46.399 | 20.907 | 77523.545 | 36822.408 | 2.720 | 1.079 | 0.470 | 5.319 | 181962.563 | 20732.541 |
| Instance900.4 | 13.741 | 2.363 | 46.562 | 20.991 | 77368.577 | 36779.894 | 2.757 | 1.046 | 0.465 | 5.350 | 180502.970 | 20513.353 |
Host System Performance
|
Counter |
Average |
Minimum |
Maximum |
|---|---|---|---|
| % Processor Time | 1.121 | 0.572 | 3.695 |
| Available MBytes | 14252.779 | 14203.000 | 14292.000 |
| Free System Page Table Entries | 16485011.017 | 16482346.000 | 16490963.000 |
| Transition Pages RePurposed/sec | 0.000 | 0.000 | 0.000 |
| Pool Nonpaged Bytes | 117907960.518 | 93839360.000 | 127930368.000 |
| Pool Paged Bytes | 164860169.086 | 164831232.000 | 164884480.000 |
| Database Page Fault Stalls/sec | 0.000 | 0.000 | 0.000 |
Test Log 1/18/2016 9:58:08 AM -- Preparing for testing ...
1/18/2016 9:58:12 AM -- Attaching databases ...
1/18/2016 9:58:12 AM -- Preparations for testing are complete.
1/18/2016 9:58:12 AM -- Starting transaction dispatch ..
1/18/2016 9:58:12 AM -- Database cache settings: (minimum: 128.0 MB, maximum: 1.0 GB)
1/18/2016 9:58:12 AM -- Database flush thresholds: (start: 10.2 MB, stop: 20.5 MB)
1/18/2016 9:58:17 AM -- Database read latency thresholds: (average: 20 msec/read, maximum: 100 msec/read).
1/18/2016 9:58:17 AM -- Log write latency thresholds: (average: 10 msec/write, maximum: 100 msec/write).
1/18/2016 9:58:17 AM -- Attaining prerequisites:
1/18/2016 10:03:03 AM -- \MSExchange Database(JetstressWin)\Database Cache Size, Last: 968945700.0 (lower bound: 966367600.0, upper bound: none)
1/18/2016 10:03:04 AM -- Performance logging started (interval: 5000 ms).
1/18/2016 10:03:04 AM -- Automatic tuning is starting ...
1/18/2016 10:03:36 AM -- Instance900.1 has 0.00 for read latency slope.
1/18/2016 10:03:36 AM -- Instance900.2 has 0.01 for read latency slope.
1/18/2016 10:03:36 AM -- Instance900.3 has 0.02 for read latency slope.
1/18/2016 10:03:36 AM -- Instance900.4 has 0.03 for read latency slope.
1/18/2016 10:05:43 AM -- 135 batch transactions/sec and 8 sessions have 270 IOPS.
1/18/2016 10:05:43 AM -- 8 sessions have actual 270 IOPS (target IOPS: 168)
1/18/2016 10:05:43 AM -- Instance900.1 has 20.3 for I/O Database Reads Average Latency.
1/18/2016 10:05:43 AM -- Instance900.2 has 20.1 for I/O Database Reads Average Latency.
1/18/2016 10:05:43 AM -- Instance900.3 has 19.7 for I/O Database Reads Average Latency.
1/18/2016 10:05:43 AM -- Instance900.4 has 19.6 for I/O Database Reads Average Latency.
1/18/2016 10:05:43 AM -- The process has average database read latencies higher than 20.000 msec.
1/18/2016 10:05:43 AM -- Instance900.1 has 2.0 for I/O Log Writes Average Latency.
1/18/2016 10:05:43 AM -- Instance900.1 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:05:43 AM -- Instance900.2 has 2.0 for I/O Log Writes Average Latency.
1/18/2016 10:05:43 AM -- Instance900.2 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:05:43 AM -- Instance900.3 has 1.5 for I/O Log Writes Average Latency.
1/18/2016 10:05:43 AM -- Instance900.3 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:05:43 AM -- Instance900.4 has 1.4 for I/O Log Writes Average Latency.
1/18/2016 10:05:43 AM -- Instance900.4 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:05:43 AM -- Operation mix: Sessions 4, Inserts 40%, Deletes 20%, Replaces 5%, Reads 35%, Lazy Commits 70%.
1/18/2016 10:06:15 AM -- Instance900.1 has 0.07 for read latency slope.
1/18/2016 10:06:15 AM -- Instance900.2 has 0.03 for read latency slope.
1/18/2016 10:06:15 AM -- Instance900.3 has 0.13 for read latency slope.
1/18/2016 10:06:15 AM -- Instance900.4 has 0.06 for read latency slope.
1/18/2016 10:08:22 AM -- 76 batch transactions/sec and 4 sessions have 166 IOPS.
1/18/2016 10:08:22 AM -- 4 sessions have actual 166 IOPS (target IOPS: 168)
1/18/2016 10:08:22 AM -- Instance900.1 has 15.5 for I/O Database Reads Average Latency.
1/18/2016 10:08:22 AM -- Instance900.2 has 15.3 for I/O Database Reads Average Latency.
1/18/2016 10:08:22 AM -- Instance900.3 has 15.7 for I/O Database Reads Average Latency.
1/18/2016 10:08:22 AM -- Instance900.4 has 15.1 for I/O Database Reads Average Latency.
1/18/2016 10:08:22 AM -- Instance900.1 has 1.2 for I/O Log Writes Average Latency.
1/18/2016 10:08:22 AM -- Instance900.1 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:08:22 AM -- Instance900.2 has 1.1 for I/O Log Writes Average Latency.
1/18/2016 10:08:22 AM -- Instance900.2 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:08:22 AM -- Instance900.3 has 0.8 for I/O Log Writes Average Latency.
1/18/2016 10:08:22 AM -- Instance900.3 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:08:22 AM -- Instance900.4 has 0.9 for I/O Log Writes Average Latency.
1/18/2016 10:08:22 AM -- Instance900.4 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:08:22 AM -- Operation mix: Sessions 5, Inserts 40%, Deletes 20%, Replaces 5%, Reads 35%, Lazy Commits 70%.
1/18/2016 10:08:53 AM -- Instance900.1 has 0.08 for read latency slope.
1/18/2016 10:08:53 AM -- Instance900.2 has 0.20 for read latency slope.
1/18/2016 10:08:53 AM -- Instance900.3 has 0.04 for read latency slope.
1/18/2016 10:08:53 AM -- Instance900.4 has 0.16 for read latency slope.
1/18/2016 10:11:01 AM -- 96 batch transactions/sec and 5 sessions have 196 IOPS.
1/18/2016 10:11:01 AM -- 5 sessions have actual 196 IOPS (target IOPS: 168)
1/18/2016 10:11:01 AM -- Instance900.1 has 15.9 for I/O Database Reads Average Latency.
1/18/2016 10:11:01 AM -- Instance900.2 has 15.2 for I/O Database Reads Average Latency.
1/18/2016 10:11:01 AM -- Instance900.3 has 15.2 for I/O Database Reads Average Latency.
1/18/2016 10:11:01 AM -- Instance900.4 has 15.4 for I/O Database Reads Average Latency.
1/18/2016 10:11:01 AM -- Instance900.1 has 1.2 for I/O Log Writes Average Latency.
1/18/2016 10:11:01 AM -- Instance900.1 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:11:01 AM -- Instance900.2 has 1.2 for I/O Log Writes Average Latency.
1/18/2016 10:11:01 AM -- Instance900.2 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:11:01 AM -- Instance900.3 has 1.5 for I/O Log Writes Average Latency.
1/18/2016 10:11:01 AM -- Instance900.3 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:11:01 AM -- Instance900.4 has 1.3 for I/O Log Writes Average Latency.
1/18/2016 10:11:01 AM -- Instance900.4 has 0.0 for I/O Log Reads Average Latency.
1/18/2016 10:11:01 AM -- Performance logging has ended.
1/18/2016 10:11:01 AM -- Automatic tuning succeeded.
1/18/2016 10:11:02 AM -- Operation mix: Sessions 5, Inserts 40%, Deletes 20%, Replaces 5%, Reads 35%, Lazy Commits 70%.
1/18/2016 10:11:02 AM -- Performance logging started (interval: 15000 ms).
1/18/2016 10:11:02 AM -- Attaining prerequisites:
1/18/2016 10:11:02 AM -- \MSExchange Database(JetstressWin)\Database Cache Size, Last: 1050984000.0 (lower bound: 966367600.0, upper bound: none)
1/18/2016 12:11:03 PM -- Performance logging has ended.
1/18/2016 12:11:03 PM -- JetInterop batch transaction stats: 22851, 22851, 22850 and 22850.
1/18/2016 12:11:03 PM -- Dispatching transactions ends.
1/18/2016 12:11:03 PM -- Shutting down databases ...
1/18/2016 12:11:09 PM -- Instance900.1 (complete), Instance900.2 (complete), Instance900.3 (complete) and Instance900.4 (complete)
1/18/2016 12:11:09 PM -- C:\JetStress_Report\HDD_RAID10_MP_1000_0.168_2GB(64k+SSD)\Performance_2016_1_18_10_11_1.blg has 479 samples.
1/18/2016 12:11:09 PM -- Creating test report ...
1/18/2016 12:11:13 PM -- Instance900.1 has 14.4 for I/O Database Reads Average Latency.
1/18/2016 12:11:13 PM -- Instance900.1 has 1.1 for I/O Log Writes Average Latency.
1/18/2016 12:11:13 PM -- Instance900.1 has 1.1 for I/O Log Reads Average Latency.
1/18/2016 12:11:13 PM -- Instance900.2 has 14.2 for I/O Database Reads Average Latency.
1/18/2016 12:11:13 PM -- Instance900.2 has 1.1 for I/O Log Writes Average Latency.
1/18/2016 12:11:13 PM -- Instance900.2 has 1.1 for I/O Log Reads Average Latency.
1/18/2016 12:11:13 PM -- Instance900.3 has 14.0 for I/O Database Reads Average Latency.
1/18/2016 12:11:13 PM -- Instance900.3 has 1.1 for I/O Log Writes Average Latency.
1/18/2016 12:11:13 PM -- Instance900.3 has 1.1 for I/O Log Reads Average Latency.
1/18/2016 12:11:13 PM -- Instance900.4 has 13.7 for I/O Database Reads Average Latency.
1/18/2016 12:11:13 PM -- Instance900.4 has 1.0 for I/O Log Writes Average Latency.
1/18/2016 12:11:13 PM -- Instance900.4 has 1.0 for I/O Log Reads Average Latency.
1/18/2016 12:11:13 PM -- Test has 0 Maximum Database Page Fault Stalls/sec.
1/18/2016 12:11:13 PM -- The test has 0 Database Page Fault Stalls/sec samples higher than 0.
1/18/2016 12:11:13 PM -- C:\JetStress_Report\HDD_RAID10_MP_1000_0.168_2GB(64k+SSD)\Performance_2016_1_18_10_11_1.xml has 478 samples queried.
