Benchmark results, home machine

Home Machine Benchmark Results

So, how fast it? Can one truly tell the SCSI bigots and IDE mongoloids to "take that trash home?" See for yourself!

Configuration Notes

Soyo D6IBA motherboard

Dual P3/450 (Katmai, overclocked to 103/463) CPUs

512MB ECC memory, 3 CAS

Debian 2.2, kernel 2.4.0test9

QLogic 2100F/66 adapter, in a 32-bit 33MHz PCI slot

RAID setup is between 5 and 7 4G SCSI disks, across the two channels of the Adaptec AEC-7412B controller

All disks are 7200RPM: 4 IBM UW units, 3 Seagate Barracudas. (1 narrow, 2 wide, narrow on the first SCSI channel.)

All setups use a 64k block size on the Adaptec

Bonnie benchmark, unless noted run as "bonnie -s 2047 >~/results"

Pictures!

Adaptec controller, standalone 18GB disk with T-card, the completed SCSI-FC RAID stack and the obligatory screen shot.

pics/aec-guts-detail.jpg 156256 bytes 871 x 653	pics/aec-guts.jpg 98086 bytes 800 x 463	pics/aec-openbox.jpg 26812 bytes 383 x 490	pics/fc-connector.jpg 71430 bytes 734 x 621	pics/fc-finished-1.jpg 61300 bytes 808 x 650
pics/fc-intview.jpg 64000 bytes 783 x 511	pics/fc-raid.jpg 44640 bytes 425 x 824	pics/fc-success.jpg 106875 bytes 838 x 629	pics/fc-t-card.jpg 90896 bytes 932 x 699	pics/fc-up.jpg 78652 bytes 979 x 575

Results!

RAID 5

5 disks:
              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
       1*2047  7205 95.3 11109  9.4  5210  7.7  5770 70.4 12117 12.0 165.5  3.7
       
              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
       1*2047  6607 99.3 11115  9.4  5288  7.5  5584 76.4 11769 10.9 175.2  3.5


6 disks:
              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
       1*2047  7277 96.9 11267  9.7  5129  7.8  5927 72.6 11846 11.9 180.4  4.3

7 disks:
              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
       1*2047  6938 99.8 11780 10.6  5469  8.3  6872 92.9 16643 16.1 201.6  4.5

Same setup, using Bonnie++ v1.00g:
Version 1.00g       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
lethe            1G  6211  94 11671  11  4809   5  6305  93 16021   9 203.3   2
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16   377  98 +++++ +++  6979  96   385  99 +++++ +++  2432  99

RAID 4

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
       1*2047  7206 94.5 10966  9.4  5169  7.5  5924 72.2 11274 11.3 149.5  3.5

RAID 3

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
       1*2047  7146 95.1 11015  9.4  5192  7.3  5906 72.2 11332 11.1 156.0  3.5

If you forget to specify the size, bonnie uses 100MB by default, which easily fits into the cache. The results are rather impressive:

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
       1* 100  7577 98.2 23560 17.7 15107 18.9  8333 99.7 159760 101.4 13665.4 187.9

RAID 0

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
         2047  7706 98.6 17365 14.7  7226  9.1  5904 74.5 15962 13.6 171.0  2.1

Local disk: Fujitsu M2934SAU, 4G narrow

For comparison purposes, here is the same benchmark, run on my Linux drive.

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
       1*1024  4507 59.3  5655  4.8  2167  3.1  3196 38.2  4455  4.2  65.2  1.0

Local disk: Seagate Barracuda UW

Local disk on my Dell at work (same CPU, 256M memory, Adaptec 29160 and a Seagate 7200RPM 18G Barracuda disk. This is a good direct comparison, since many of the drive in the RAID array are Barracudas or comparable.

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
         2047  5698 85.4 13651 13.6  6407 13.9  5810 87.2 14725 11.9 123.4  2.1

Discussion

As expected, the RAID5 setup got faster as disks were added, though from watching as the program ran, the array was never keeping the disks 100% busy. Especially with the block transfers, we should easily been able to keep the disks saturated with requests. From talking to a colleague, the Adaptec controller should be capable of about 25MB/sec, so we have a bottleneck elsewhere.

Unfortunately, testing this requires faster drives, and I'm outta discretionary funding for the near future. Please feel free to donate drives my way!

In retrospect, I should have also run benchmarks of the other RAID setups at each array size, but the time required to rebuild the array with a new RAID level was too much. Hard science this isn't!

Conclusions

It's damned fast, but real speed requires a better RAID controller and some decent disks.

And damn if it isn't cool!

Roughly 16MB/sec sustained for block I/O, and the local cache provides excellent performance for seeking and small read/writes, meaning that it performs wonderfully under my typical Unix workload.

It's way faster than local SCSI

There are bottlenecks somewhere, which implies that if I can find them it'll get faster

It's cheap. I spent about $150 on disks, $149 on the controller, and have borrowed or been given the rest. Pretty cheap for a 24GB-usable disk!

It's quiet. Since I'm using fiber, the array is in a closet, and my PC only contains the SCSI CD-ROM. Man, it's quiet!

I've decided that FibreChannel rocks. It smokes SCSI, both in speed and expandability, the CPU overhead is tiny, and it's cheap to boot. What else could you ask for?

And if you're into multiplayer games, having a hard drive subsystem that gets you on the level before your competitors is certainly rather nice.

Please check out the introductory page for more information on how to get started.

Further Work

See the second results page!

Navigation Links

Introduction / HOWTO: Fibre Channel

FC at CDF page

FC at home page

FC at home, part two - chassis and tapes

Resources page

Back to home page