When creating a LUN, what is parity overhead and how is it related to record size?

In QES, RAID groups of type RAID 5, RAID 6, and RAID-TP use parity for disk failure tolerance. QES generates parity data by performing a calculation on the data on two drives, and then storing the results on a third drive. If any one of the three drives fails, QES is able to rebuild the lost data from the other two drives using parity. The storage space that QES requires to store parity data is determined by the sector size of the drive and the record size of the LUN.

Sector size is the size of a single unit of data written or read from a physical drive. In the past decade, disk manufacturers have been increasing sector size from 512 bytes to 4K bytes in order to provide better performance and error correction.

Record size specifies the maximum block size of data stored in a LUN. In QES, record size is determined by the LUN’s Performance Profile setting. When creating a storage pool, QES reserves space for parity data based on the assumption that the record size of all LUNs in the pool will be set to the default, which is 128K for a regular storage pool and 4K for an all-SSD storage pool with write coalescing enabled. When creating or modifying a LUN, setting a record size smaller than the default value will require extra space for storing parity data, called parity overhead.

Parity overhead example: We create a RAID 5 group consisting of 5 disks. Each disk has a sector size of 512 bytes. We then create a LUN on the RAID group with a record size of 4K bytes. If the OS writes 16K bytes of data to the LUN, the data will be laid out as follows:

The record size is 4K bytes, so the 16K bytes of data must be divided into four records of 4K bytes each, R1, R2, R3 and R4. RAID 5 uses striping, so data is spanned across the disks in the RAID group.

To write a single record of 4K bytes to disks with a 512 byte sector size requires 8 sectors. So each 4K bytes record now spans two rows and requires two parity blocks. Storing 16K bytes of data in this RAID 5 group actually requires 20K bytes, which is calculated as follows:

8(rows) * 5(disks) * 512 bytes(sector size) = 20K bytes

When a RAID 5 group consists of five disks, 20% or (1/5) is reserved for parity which means there is no parity space overhead with a 512 byte sector size.

Example 2: If the sector size of the disks is 4K bytes instead of 512 bytes, and the OS writes 16K bytes of data to the LUN, the data will be laid out as follows:

As with the previous example, the 16K bytes of data is divided into four records of 4K bytes each, R1, R2, R3 and R4. But this time, writing a single record of 4K bytes with a 4K bytes sector size only requires one sector. However, even if we are writing to a single sector, we have to write the parity data which will occupy another sector of 4K bytes. Therefore storing 16K bytes of data requires 32K bytes, which is calculated as follows:

8(sectors) * 4K bytes(sector size) = 32K bytes

When a RAID 5 groups consists of 5 disks, 20% or (1/5) is reserved for parity, which in this example totals 8K. However, in this example we have to use an additional 8K bytes to store the parity blocks, which means the party overhead is 8K.

Example 3: To avoid the parity overhead in example 2, we can set a larger record size. If the LUN’s record size is set to 16K bytes instead of 4K bytes and the OS writes 16K bytes of data to the LUN, the data will be laid out as follows:

Writing 16K bytes of data requires one record of 16K bytes, R1, four sectors of 4K bytes, and one parity block of 4K bytes. This means there is no parity overhead.

Note: Shared folders are also affected by parity overhead. However, parity overhead also depends on the application data pattern, which cannot be calculated before the data is actually written. So there will be no “Parity space” displayed when creating a shared folder.