This is an old revision of the document!
Table of Contents
NAS - Build a Linux NAS - Tune the System
There are many settings that can be tuned.
As there are many factors than can affect tuning performance, it is recommended to perform a number of tests using different figures to determine the optimum values:
- Create a test file that will be used for reading and writing tests.
- Perform each test using dd.
dd if=testfile of=/dev/null bs=8k
- Between each test, clear the OS disk cache.
sync;echo 3 > /proc/sys/vm/drop_caches
NOTE: Vary the size of the test file depending on how much RAM the system has.
- It is suggested to have the test file be large, such as 32G.
- Consider changing the blocksize used with dd to a larger value, say bs=16k, if there is a lot of RAM.
WARNING: Some changes may result in a bit of extra memory being used.
- After each command, the amount of memory being used can be checked with:
free -m
returns:
total used free shared buff/cache available Mem: 64326 22064 5354 1366 36907 40203 Swap: 975 315 660
Tuning stripe_cache_size
stripe_cache_size affects RAM used by mdadm for writing of data.
Ubuntu default value is 256.
Verify the current value:
cat /sys/block/md0/md/stripe_cache_size
Change it:
echo *number* > /sys/block/md0/md/stripe_cache_size or echo *number* | sudo tee /sys/block/md0/md/stripe_cache_size
NOTE: Test with different sizes.
- Test with stripe_cache_size=256
- Test with stripe_cache_size=512
- Test with stripe_cache_size=1024
- Test with stripe_cache_size=2048
- Test with stripe_cache_size=4096
- Test with stripe_cache_size=8192
- Test with stripe_cache_size=16384
- Test with stripe_cache_size=32768
At each test, check the Write performance, for example:
dd if=testfile of=/dev/null bs=8k 361680+0 records in 361680+0 records out 2962882560 bytes (3.0 GB) copied, 0.92471 s, 3.2 GB/s
Read speed test:
dd if=testfile of=/dev/zero
Write speed test:
dd if=/dev/zero of=testfile
Also check block devices:
blockdev --report
Tuning Read Ahead
Changing the read ahead value should impact read performance.
- Default read ahead value is 1536.
Change it::
blockdev --setra *number* /dev/md0
NOTE: Test with different sizes.
- Test with Read Ahead @ 1536
- Test with Read Ahead @ 4096
- Test with Read Ahead @ 32768
- Test with Read Ahead @ 262144
- Test with Read Ahead @ 524288
At each test, check the Read performance, for example:
dd if=testfile of=/dev/null bs=8k 361680+0 records in 361680+0 records out 2962882560 bytes (3.0 GB) copied, 0.92471 s, 3.2 GB/s
Read speed test:
dd if=testfile of=/dev/zero
Write speed test:
dd if=/dev/zero of=testfile
Also check block devices:
blockdev --report
Make these tuning changes permanent
There are many ways to allow this.
The suggested approach is to create a file /etc/profile.d/raid_tuning.sh:
- /etc/profile.d/raid_tuning.sh
echo *number* | sudo tee /sys/block/md0/md/stripe_cache_size blockdev --setra *number* /dev/md0 For example: echo 8192 > /sys/block/md0/md/stripe_cache_size echo 256 > /sys/block/sdb/queue/max_sectors_kb echo 256 > /sys/block/sdc/queue/max_sectors_kb echo 256 > /sys/block/sdd/queue/max_sectors_kb echo 256 > /sys/block/sde/queue/max_sectors_kb blockdev --setra 64 /dev/sd[bcde] blockdev --setra 16384 /dev/md0
NOTE: The files within /etc/profile.d/ are automatically run at boot, which will result in these tuning options being set at each boot.