Impressive credentials for the Cheetah . . .

Quote:
Source:  The Best in Enterprise Hard Drives (April, 2007)

El_Pescador, best wishes for you and yours to awaken from your 'nightmare' in order to live your dreams.

StorageReview is quite adamant that its benchmarks are valid:

Quote:
Source:  Testbed3: Auditing IPEAK SPT

However, there is sufficient confusion about the benchmarks related to Cheetah drives specifically so as to cause one to pause and at least consider the possibility that these benchmarks are suspicious.

Confusion #1:  Concerning the reported benchmark performance of the Cheetah 15K.5 by StorageReview, note the contradictory information that is provided:  “When testing the previous generation of Seagate's SCSI drives (Savvio 10K.1, Cheetah 10K.7, and Cheetah 15K.5), StorageReview uncovered significant performance differences when these drives were set to different predefined cache segmentation strategies through Seagate's Seatools Enterprise utility.  The Cheetah 15K.5, however, returns the same scores regardless of whether the utility's ‘Performance Mode’ setting is toggled on or off” (see Seagate Cheetah 15K.5).

So, does the Cheetah 15K.5 show “significant performance differences when these drives were set to different predefined cache segmentation strategies” (desktop versus server mode), or does the Cheetah 15K.5 return “the same scores regardless of whether the utility's ‘Performance Mode’ setting is toggled on or off”?   If the latter, then one of two facts must be true: either (1) Seagate - a company known for its technical expertise - isn’t properly optimizing the operation of the firmware for either the server or desktop modes; or (2) the StorageReview benchmarks are not properly reflecting desktop versus server usage pattern differences.

Confusion #2:  The StorageReview benchmark methodology is described in this article:  The 2006 Desktop DriveMarks.  Note especially the graph “Relative Change from 2002 to 2006 DriveMark (Office)” that shows a dramatic decline (-24%) in performance by the Cheetah 15K.4 in desktop mode from 2002 to 2006.  In other words, something about the newer version of the benchmark has had the greatest negative impact upon the Cheetah drive specifically.  If there is that much change in the StorageReview benchmarks when the hardware is constant, can the benchmarks be trusted?

Confusion #3:  Additionally, the graph “StorageReview High-End DriveMark 2006” shows that a Cheetah 15K.4 in desktop mode outperforms a Raptor.

Conclusion:  Thus, I am less than confident in the information reported on the StorageReview website concerning the Cheetah drive.  Using this information to argue against the superiority of SCSI versus SATA (as I did in prior posts) may not be wise.

Rad, I think you have summarized the situation well.  I cannot comment upon the “integrity” of the StorageReview benchmarks, but something seems amiss.  What constitutes that “something” will probably remain unknown.

Another consideration, though, is that the hard disk performance information reported by other websites (e.g., Tom’s Hardware Guide) is based upon publicly available benchmarks (e.g., PCMark05) that anyone could use to confirm/refute the assertion.  However, with StorageReview the benchmarks are proprietary and are based upon a “playback of a trace file that captured 30 minutes of typical PC use by yours truly” (i.e., by the StorageReview author in the case of Office DriveMark 2002).  Thus, the StorageReview data is not subject to public scrutiny – an unfortunate condition that lessens its credibility, in my opinion.

Concerning the issue of reliability, a study by Carnegie Mellon University of 100,000 hard disk drives over five years found “little difference” in replacement rates between SCSI and SATA drives.

See:  Disk failures in the real world

Rad, the study referenced in the InfoWorld article (Reply #39) is the same Carnegie Mellon University research cited in Reply #38.

This research doesn’t say that SATA “performs better or as good as” SCSI (e.g., in terms of sustained transfer rates) – rather, it says that the reliability of the two are essentially the same.  Somewhat counterintuitive, yes; but, data are data, no?

By "SR," I think you were referencing the StorageReview website benchmarks.  In contrast, the Carnegie Mellon University research doesn't assert the claim that SATA is faster than SCSI (or the opposite); rather, it makes the observation that the reliability of SATA and SCSI hard disk drives are about the same.  In other words, this research is not about performance – but about reliability (i.e., the inverse of failure).

Additionally, the Carnegie Mellon University research does not involve "benchmark" comparisons.  They are based upon tabulations of the failure rate of 100,000 actual hard disk drives in use.

I do agree, though, that a 15K SCSI hard disk drive will – under most circumstances – outperform a 10K SCSI or a 10K SATA hard disk drive, although the latter two appear to have nearly the same performance characteristics, as far as I can determine, in a non-server environment.  Is that your understanding, too?

Concerning the 10K SATA versus 10K SCSI comparison, the manufacturer specifications of the Western Digital Raptor (WD1500AD) versus the Seagate Cheetah 10K.7 (ST3146707) support both of our perspectives:

Average latency (msec):  Raptor=3.0, Cheetah=3.0
Random read seek time (msec):  Raptor=4.6, Cheetah=4.7
Random write seek time (msec):  Raptor=5.2, Cheetah=5.3

Note that the Raptor is slightly faster than the Cheetah for read/write seek times, but on balance they are "the same."