SCSI Drives and RAID Arrays Functionality

SCSI Drives and RAID Arrays Functionality Nikola Zlatanov*

What is SCSI A computer is full of busses -- highways that take information and power from one place to another. For example, when you plug an MP3 player or digital camera into your computer, you're probably using an universal serial bus (USB) port. Your USB port is good at carrying the data and electricity required for small electronic devices that do things like create and store pictures and music files. But that bus isn't big enough to support a whole computer, a server or lots of devices simultaneously. For that, you would need something more like SCSI. SCSI originally stood for Small Computer System Interface, but it's really outgrown the "small" designation. It's a fast bus that can connect lots of devices to a computer at the same time, including hard drives, scanners, CD-ROM/RW drives, printers and tape drives. Other technologies, like serial-ATA (SATA), have largely replaced it in new systems, but SCSI is still in use. This article will review SCSI basics and give you lots of information on SCSI types and specifications.

SCSI Basics SCSI connector SCSI is based on an older, proprietary bus interface called Shugart Associates System Interface (SASI). SASI was originally developed in 1981 by Shugart Associates in conjunction with NCR Corporation. In 1986, the American National Standards Institute (ANSI) ratified SCSI (pronounced "scuzzy"), a modified version of SASI. SCSI uses a controller to send and receive data and power to SCSI-enabled devices, like hard drives and printers. SCSI has several benefits. It's fairly fast, up to 320 megabytes per second (MBps). It's been around for more than 20 years and it's been thoroughly tested, so it has a reputation for being reliable. Like Serial ATA and FireWire, it lets you put multiple items on one bus. SCSI also works with most computer systems. However, SCSI also has some potential problems. It has limited system BIOS support, and it has to be configured for each computer. There's also no common SCSI software interface. Finally, all the different SCSI types have different speeds, bus widths and connectors, which can be confusing. When you know the meaning behind "Fast," "Ultra" and "Wide," though, it's pretty easy to understand. We'll look at these SCSI types next.

Single Ended Parallel SCSI icon

SCSI Types

Different combinations of doubled bus speed, doubled clock speed and SCSI-3 specifications have led to lots of SCSI variations. SCSI has three basic specifications: SCSI-1: The original specification developed in 1986, SCSI-1 is now obsolete. It featured a bus width of 8 bits and clock speed of 5MHz. SCSI-2: Adopted in 1994, this specification included the Common Command Set (CCS) -- 18 commands considered an absolute necessity for support of any SCSI device. It also had the option to double the clock speed to 10 MHz (Fast), double the bus width from to 16 bits and increase the number of devices to 15 (Wide), or do both (Fast/Wide). SCSI-2 also added command queuing, allowing devices to store and prioritize commands from the host computer. SCSI-3: This specification debuted in 1995 and included a series of smaller standards within its overall scope. A set of standards involving the SCSI Parallel Interface (SPI), which is the way that SCSI devices communicate with each other, has continued to evolve within SCSI-3. Most SCSI-3 specifications begin with the term Ultra, such as Ultra for SPI variations, Ultra2 for SPI-2 variations and Ultra3 for SPI-3 variations. The Fast and Wide designations work just like their SCSI-2 counterparts. SCSI-3 is the standard currently in use. Different combinations of doubled bus speed, doubled clock speed and SCSI-3 specifications have led to lots of SCSI variations. The chart on this page compares several of them. Many of the slower ones are no longer in use -- we've included them for comparison.

SCSI controller In addition to the increased bus speed, Ultra320 SCSI uses packeted data transfer, increasing its efficiency. Ultra2 was also the last type to have a "narrow," or 8-bit, bus width. All of these SCSI types are parallel -- bits of data move through the bus simultaneously rather than one at a time. The newest type of SCSI, called Serial Attached SCSI (SAS), uses SCSI commands but transmits data serially. SAS uses a point-to-point serial connection to move data at 3.0 gigabits per second, and each SAS port can support up to 128 devices or expanders. All the different SCSI varieties use controllers and cables to interface with devices. We'll look at this process next. Controllers, Devices and Cables A SCSI controller coordinates between all of the other devices on the SCSI bus and the computer. Also called a host adapter, the controller can be a card that you plug into an available slot or it can be built into the motherboard. The SCSI BIOS is also on the controller. This is a small ROM or Flash memory chip that contains the software needed to access and control the devices on the bus. Each SCSI device must have an unique identifier (ID) in order for it to work properly. For example, if the bus can support sixteen devices, their IDs, specified through a hardware or software setting, range from zero to 15. The SCSI controller itself must use one of the IDs, typically the highest one, leaving room for 15 other devices on the bus. Internal devices connect to a SCSI controller with a ribbon cable. External SCSI devices attach to the controller in a daisy chain using a thick, round cable. (Serial Attached SCSI devices use SATA cables.) In a daisy chain, each device connects to the next one in line. For this reason, external SCSI devices typically have two SCSI connectors -- one to connect to the previous device in the chain, and the other to connect to the next device. The cable itself typically consists of three layers: Inner layer: The most protected layer, this contains the actual data being sent. Media layer: Contains the wires that send control commands to the device. Outer layer: Includes wires that carry parity information, which ensures that the data is correct. Different SCSI variations use different connectors, which are often incompatible with one another. These connectors usually use 50, 68 or 80 pins. SAS uses smaller, SATA-compatible connectors. Once all of the devices on the bus are installed and have their own IDs, each end of the bus must be closed. We'll look at how to do this next. Termination Some SCSI terminators are built into the SCSI device, while others may require an external terminator like this one. If the SCSI bus were left open, electrical signals sent down the bus could reflect back and interfere with communication between devices and the SCSI controller. The solution is to terminate the bus, closing each end with a resistor circuit. If the bus supports both internal and external devices, then the last device on each series must be terminated. Types of SCSI termination can be grouped into two main categories: passive and active. Passive termination is typically used for SCSI systems that run at the standard clock speed and have a distance of less than 3 feet (1 m) from the devices to the controller. Active termination is

used for Fast SCSI systems or systems with devices that are more than 3 feet (1 m) from the SCSI controller. SCSI also employs three distinct types of bus signaling, which also affect termination. Signaling is the way that the electrical impulses are sent across the wires. Single-ended (SE): The controller generates the signal and pushes it out to all devices on the bus over a single data line. Each device acts as a ground. Consequently, the signal quickly begins to degrade, which limits SE SCSI to a maximum of about 10 ft (3 m). SE signaling is common in PCs. High-voltage differential (HVD): Often used for servers, HVD uses a tandem approach to signaling, with a data high line and a data low line. Each device on the SCSI bus has a signal transceiver. When the controller communicates with the device, devices along the bus receive the signal and retransmit it until it reaches the target device. This allows for much greater distances between the controller and the device, up to 80 ft (25 m). Low-voltage differential (LVD): LVD is a variation on HVD and works in much the same way. The big difference is that the transceivers are smaller and built into the SCSI adapter of each device. This makes LVD SCSI devices more affordable and allows LVD to use less electricity to communicate. The downside is that the maximum distance is half of HVD -- 40 ft (12 m). An active terminator Both HVD and LVD normally use passive terminators, even though the distance between devices and the controller can be much greater than 3ft (1m). This is because the transceivers ensure that the signal is strong from one end of the bus to the other.

SCSI Command Protocol In addition to many different hardware implementations, the SCSI standards also include an extensive set of command definitions. The SCSI command architecture was originally defined for parallel SCSI buses but has been carried forward with minimal change for use with iSCSI and serial SCSI. Other technologies which use the SCSI command set include the ATA Packet Interface, USB Mass Storage class and FireWire SBP-2. In SCSI terminology, communication takes place between an initiator and a target. The initiator sends a command to the target, which then responds. SCSI commands are sent in a Command Descriptor Block (CDB). The CDB consists of a one byte operation code followed by five or more bytes containing command-specific parameters. At the end of the command sequence, the target returns a status code byte, such as 00h for success, 02h for an error (called a Check Condition), or 08h for busy. When the target returns a Check Condition in response to a command, the initiator usually then issues a SCSI Request Sense command in order to obtain a key code qualifier (KCQ) from the target. The Check Condition and Request Sense sequence involves a special SCSI protocol called a Contingent Allegiance Condition. There are 4 categories of SCSI commands: N (non-data), W (writing data from initiator to target), R (reading data), and B (bidirectional). There are about 60 different SCSI commands in total, with the most commonly used being: Test unit ready: Queries device to see if it is ready for data transfers (disk spun up, media loaded, etc.). Inquiry: Returns basic device information.

Request sense: Returns any error codes from the previous command that returned an error status. Send diagnostic and Receive diagnostic results: runs a simple self-test, or a specialized test defined in a diagnostic page. Start/Stop unit: Spins disks up and down, or loads/unloads media (CD, tape, etc.). Read capacity: Returns storage capacity. Format unit: Prepares a storage medium for use. In a disk, a low level format will occur. Some tape drives will erase the tape in response to this command. Read (four variants): Reads data from a device. Write (four variants): Writes data to a device. Log sense: Returns current information from log pages. Mode sense: Returns current device parameters from mode pages. Mode select: Sets device parameters in a mode page. Each device on the SCSI bus is assigned a unique SCSI identification number or ID. Devices may encompass multiple logical units, which are addressed by logical unit number (LUN). Simple devices have just one LUN, more complex devices may have multiple LUNs. A "direct access" (i.e. disk type) storage device consists of a number of logical blocks, addressed by Logical Block Address (LBA). A typical LBA equates to 512 bytes of storage. The usage of LBAs has evolved over time and so four different command variants are provided for reading and writing data. The Read(6) and Write(6) commands contain a 21-bit LBA address. The Read(10), Read(12), Read Long, Write(10), Write(12), and Write Long commands all contain a 32-bit LBA address plus various other parameter options. The capacity of a "sequential access" (i.e. tape-type) device is not specified because it depends, amongst other things, on the length of the tape, which is not identified in a machine-readable way. Read and write operations on a sequential access device begin at the current tape position, not at a specific LBA. The block size on sequential access devices can either be fixed or variable, depending on the specific device. Tape devices such as half-inch 9-track tape, DDS (4mm tapes physically similar to DAT), Exabyte, etc., support variable block sizes.

RAID SCSI is often used to control a redundant array of independent discs (RAID). Other technologies, like serial-ATA (SATA), can also be used for this purpose. Newer SATA drives tend to be faster and cheaper than SCSI drives. A is a series of hard drives treated as one big drive. These drives can read and write data at the same time, known as striping. The controller determines which drive gets which chunk of data. While that drive writes the data, the controller sends data to or reads it from another drive. Also improves fault tolerance through mirroring and parity. Mirroring makes an exact duplicate of one drive's data on a second hard drive. Parity uses a minimum of three hard drives, and data is written sequentially to each drive, except the last one. The last drive stores a number that represents the sum of the data on the other drives. What is RAID In 1987, Patterson, Gibson and Katz at the University of California Berkeley, published a paper entitled "A Case for Redundant Arrays of Inexpensive Disks (RAID)”. This paper described

various types of disk arrays, referred to by the acronym RAID. The basic idea of RAID was to combine multiple small, inexpensive disk drives into an array of disk drives which yields performance exceeding that of a Single Large Expensive Drive (SLED). Additionally, this array of drives appears to the computer as a single logical storage unit or drive.

The Mean Time Between Failure (MTBF) of the array will be equal to the MTBF of an individual drive, divided by the number of drives in the array. Because of this, the MTBF of an array of drives would be too low for many application requirements. However, disk arrays can be made fault-tolerant by redundantly storing information in various ways. Five types of array architectures, RAID-1 through RAID-5, were defined by the Berkeley paper, each providing disk fault-tolerance and each offering different trade-offs in features and performance. In addition to these five redundant array architectures, it has become popular to refer to a non-redundant array of disk drives as a RAID-0 array. Data Striping Fundamental to RAID is "striping", a method of concatenating multiple drives into one logical storage unit. Striping involves partitioning each drive's storage space into stripes which may be as small as one sector (512 bytes) or as large as several megabytes. These stripes are then interleaved round-robin, so that the combined space is composed alternately of stripes from each drive. In effect, the storage space of the drives is shuffled like a deck of cards. The type of application environment, I/O or data intensive, determines whether large or small stripes should be used. Most multi-user operating systems today, like NT, Unix and Netware, support overlapped disk I/O operations across multiple drives. However, in order to maximize throughput for the disk subsystem, the I/O load must be balanced across all the drives so that each drive can be kept busy as much as possible. In a multiple drive system without striping, the disk I/O load is never perfectly balanced. Some drives will contain data files which are frequently accessed and some drives will only rarely be accessed. In I/O intensive environments, performance is optimized by

striping the drives in the array with stripes large enough so that each record potentially falls entirely within one stripe. This ensures that the data and I/O will be evenly distributed across the array, allowing each drive to work on a different I/O operation, and thus maximize the number of simultaneous I/O operations which can be performed by the array. In data intensive environments and single-user systems which access large records, small stripes (typically one 512-byte sector in length) can be used so that each record will span across all the drives in the array, each drive storing part of the data from the record. This causes long record accesses to be performed faster, since the data transfer occurs in parallel on multiple drives. Unfortunately, small stripes rule out multiple overlapped I/O operations, since each I/O will typically involve all drives. However, operating systems like DOS which do not allow overlapped disk I/O, will not be negatively impacted. Applications such as on-demand video/audio, medical imaging and data acquisition, which utilize long record accesses, will achieve optimum performance with small stripe arrays. A potential drawback to using small stripes is that synchronized spindle drives are required in order to keep performance from being degraded when short records are accessed. Without synchronized spindles, each drive in the array will be at different random rotational positions. Since an I/O cannot be completed until every drive has accessed its part of the record, the drive which takes the longest will determine when the I/O completes. The more drives in the array, the more the average access time for the array approaches the worst case single-drive access time. Synchronized spindles assure that every drive in the array reaches its data at the same time. The access time of the array will thus be equal to the average access time of a single drive rather than approaching the worst case access time.

The different RAID levels RAID-0 RAID Level 0 is not redundant, hence does not truly fit the "RAID" acronym. In level 0, data is split across drives, resulting in higher data throughput. Since no redundant information is stored, performance is very good, but the failure of any disk in the array results in data loss. This level is commonly referred to as striping. RAID-1 RAID Level 1 provides redundancy by writing all data to two or more drives. The performance of a level 1 array tends to be faster on reads and slower on writes compared to a single drive, but if either drive fails, no data is lost. This is a good entry-level redundant system, since only two drives are required; however, since one drive is used to store a duplicate of the data, the cost per megabyte is high. This level is commonly referred to as mirroring. RAID-2 RAID Level 2, which uses Hamming error correction codes, is intended for use with drives which do not have built-in error detection. All SCSI drives support built-in error detection, so this level is of little use when using SCSI drives. RAID-3 RAID Level 3 stripes data at a byte level across several drives, with parity stored on one drive. It is otherwise similar to level 4. Byte-level striping requires hardware support for efficient use. RAID-4 RAID Level 4 stripes data at a block level across several drives, with parity stored on one drive. The parity information allows recovery from the failure of any single drive. The performance of a

level 4 array is very good for reads (the same as level 0). Writes, however, require that parity data be updated each time. This slows small random writes, in particular, though large writes or sequential writes are fairly fast. Because only one drive in the array stores redundant data, the cost per megabyte of a level 4 array can be fairly low. RAID-5 RAID Level 5 is similar to level 4, but distributes parity among the drives. This can speed small writes in multiprocessing systems, since the parity disk does not become a bottleneck. Because parity data must be skipped on each drive during reads, however, the performance for reads tends to be considerably lower than a level 4 array. The cost per megabyte is the same as for level 4.

Summary o

RAID-0 is the fastest and most efficient array type but offers no fault-tolerance.

o RAID-1 is the array of choice for performance-critical, fault-tolerant environments. In addition, RAID-1 is the only choice for fault-tolerance if no more than two drives are desired. o

RAID-2 is seldom used today since ECC is embedded in almost all modern disk drives.

RAID-3 can be used in data intensive or single-user environments which access long sequential records to speed up data transfer. However, RAID-3 does not allow multiple I/O operations to be overlapped and requires synchronized-spindle drives in order to avoid performance degradation with short records. o

RAID-4 offers no advantages over RAID-5 and does not support multiple simultaneous write operations. o

RAID-5 is the best choice in multi-user environments which are not write performance sensitive. However, at least three, and more typically five drives are required for RAID-5 arrays. o

Possible approaches to RAID Hardware RAID The hardware based system manages the RAID subsystem independently from the host and presents to the host only a single disk per RAID array. This way the host doesn't have to be aware of the RAID subsystems(s). • The controller based hardware solution DPT's SCSI controllers are a good example for a controller based RAID solution. The intelligent controller manages the RAID subsystem independently from the host. The advantage over an external SCSI---SCSI RAID subsystem is that the controller is able to span the RAID subsystem over multiple SCSI channels and by this remove the limiting factor external RAID solutions have: The transfer rate over the SCSI bus. • The external hardware solution (SCSI---SCSI RAID) An external RAID box moves all RAID handling "intelligence" into a controller that is sitting in the external disk subsystem. The whole subsystem is connected to the host via a normal SCSI controller and appears to the host as a single or multiple disks. This solution has drawbacks compared to the controller based solution: The single SCSI channel used in this solution creates a bottleneck. Newer technologies like Fiber Channel can ease this problem, especially if they allow to trunk multiple channels into a Storage Area Network. 4 SCSI drives can already completely flood a parallel SCSI bus, since the average transfer size

is around 4KB and the command transfer overhead - which is even in Ultra SCSI still done asynchronously - takes most of the bus time. Software RAID • The MD driver in the Linux kernel is an example of a RAID solution that is completely hardware independent. The Linux MD driver supports currently RAID levels 0/1/4/5 + linear mode. • Under Solaris you have the Solstice DiskSuite and Veritas Volume Manager which offer RAID-0/1 and 5. • Adaptecs AAA-RAID controllers are another example, they have no RAID functionality whatsoever on the controller, they depend on external drivers to provide all external RAID functionality. They are basically only multiple single AHA2940 controllers which have been integrated on one card. Linux detects them as AHA2940 and treats them accordingly. Every OS needs its own special driver for this type of RAID solution, this is error prone and not very compatible. Hardware vs. Software RAID Just like any other application, software-based arrays occupy host system memory, consume CPU cycles and are operating system dependent. By contending with other applications that are running concurrently for host CPU cycles and memory, software-based arrays degrade overall server performance. Also, unlike hardware-based arrays, the performance of a software-based array is directly dependent on server CPU performance and load. Except for the array functionality, hardware-based RAID schemes have very little in common with software-based implementations. Since the host CPU can execute user applications while the array adapter's processor simultaneously executes the array functions, the result is true hardware multi-tasking. Hardware arrays also do not occupy any host system memory, nor are they operating system dependent. Hardware arrays are also highly fault tolerant. Since the array logic is based in hardware, software is NOT required to boot. Some software arrays, however, will fail to boot if the boot drive in the array fails. For example, an array implemented in software can only be functional when the array software has been read from the disks and is memory-resident. What happens if the server can't load the array software because the disk that contains the fault tolerant software has failed? Software-based implementations commonly require a separate boot drive, which is NOT included in the array. A disk array controller is a device which manages the physical disk drives and presents them to the computer as logical units. It almost always implements hardware RAID, thus it is sometimes referred to as RAID controller. It also often provides additional disk cache. A disk array controller name is often improperly shortened to a disk controller. The two should not be confused as they provide very different functionality. Front-end and back-end side A disk array controller provides front-end interfaces and back-end interfaces. Back-end interface communicates with controlled disks. Hence protocol is usually ATA (a.k.a. PATA; incorrectly called IDE), SATA, SCSI, FC or SAS.

Front-end interface communicates with a computer's host adapter (HBA, Host Bus Adapter) and uses: • one of ATA, SATA, SCSI, FC; these are popular protocols used by disks, so by using one of them a controller may transparently emulate a disk for a computer • somewhat less popular protocol dedicated for a specific solution: FICON/ESCON, iSCSI, HyperSCSI, ATA over Ethernet or InfiniBand A single controller may use different protocols for back-end and for front-end communication. Many enterprise controllers use FC on front-end and SATA on back-end. Enterprise controllers In a modern enterprise architecture disk array controllers (sometimes also called storage processors, or SPs) are parts of physically independent enclosures, such as disk arrays placed in a storage area network (SAN) or network-attached storage (NAS) servers. Those external disk arrays are usually purchased as an integrated subsystem of RAID controllers, disk drives, power supplies, and management software. It is up to controllers to provide advanced functionality (various vendors name these differently): Automatic failover to another controller (transparent to computers transmitting data) Long-running operations performed without downtime Forming a new RAID set Reconstructing degraded RAID set (after a disk failure) Adding a disk to online RAID set Removing a disk from a RAID set (rare functionality) Partitioning a RAID set to separate volumes/LUNs Simple controllers A simple disk array controller may fit inside a computer, either as a PCI expansion card or just built onto a motherboard. Such a controller usually provides host bus adapter (HBA) functionality itself to save physical space. Hence it is sometimes called a RAID adapter. As of February 2007 Intel started integrating their own Matrix RAID controller in their more upmarket motherboards, giving control over 4 devices and an additional 2 SATA connectors, and totaling 6 SATA connections (3Gbit/s each). For backward compatibility one IDE connector able to connect 2 ATA devices (100 Mbit/s) is also present.

Conclusion While hardware RAID controllers were available for a long time, they always required expensive SCSI hard drives and aimed at the server and high-end computing market. SCSI technology advantages include allowing up to 15 devices on one bus, independent data transfers, hot-swapping, much higher MTBF. Around 1997, with the introduction of ATAPI-4 (and thus the Ultra-DMA-Mode 0, which enabled fast data-transfers with less CPU utilization) the first ATA RAID controllers were introduced as PCI expansion cards. Those RAID systems made their way to the consumer market, where the users wanted the fault-tolerance of RAID without investing in expensive SCSI drives.

ATA drives make it possible to build RAID systems at lower cost than with SCSI, but most ATA RAID controllers lack a dedicated buffer or high-performance XOR hardware for parity calculation. As a result, ATA RAID performs relatively poorly compared to most SCSI RAID controllers. Additionally, data safety suffers if there is no battery backup to finish writes interrupted by a power outage.

Resources [1] Serial Attached SCSI FAQ http://www.seagate.com/products/interface/sas/faq.html [2] NCR Collection (LSI Logic) at Smithsonian Museum". Smithsonianchips.si.edu. Retrieved 2014-05-11. [3] "Power Macintosh G3, G4: Differences Between Available SCSI Cards". Apple, Inc. 2010-0401. Retrieved 2014-07-07. [4] David Allen (2005-07-01). "The benefits of Serial Attached SCSI (SAS)". Retrieved 2015-0206. [5] "SCSI Standards & Cables for the "normal"* person". Ramelectronics.net. Retrieved 201405-11. [6] Arpaci-Dusseau, Remzi H.; Arpaci-Dusseau, Andrea C. (2014), Operating Systems: Three Easy Pieces [Chapter: RAID] , Arpaci-Dusseau Books [7] Patterson, David; Gibson, Garth A.; Katz, Randy (1988). A Case for Redundant Arrays of Inexpensive Disks (RAID) . SIGMOD Conferences. Retrieved 2006-12-31. [8] Randy H. Katz (October 2010). "RAID: A Personal Recollection of How Storage Became a System" . Eecs.umich.edu. IEEE Computer Society. Retrieved 2015-01-18. [9] Donald, L. (2003). "MCSA/MCSE 2006 JumpStart Computer and Network Basics" (2nd ed.). Glasgow: SYBEX. [10] Howe, Denis (ed.). Redundant Arrays of Independent Disks from FOLDOC. Free On-line Dictionary of Computing (Imperial College Department of Computing). Retrieved 2011-11-10. [11] Dawkins, Bill and Jones, Arnold. "Common RAID Disk Data Format Specification" [Storage Networking Industry Association] Colorado Springs, 28 July 2006. Retrieved on 22 February 2011. [12] "Adaptec Hybrid RAID Solutions". Adaptec.com. Adaptec. 2012. Retrieved 2013-09-07. [13] "Common RAID Disk Drive Format standard". SNIA.org. SNIA. Retrieved 2012-08-26. [14] Andrew S. Tanenbaum. Structured Computer Organization 6th ed. p. 95. [15] Hennessy, John; Patterson, David (2006). Computer Architecture: A Quantitative Approach, 4th ed. p. 362. ISBN 978-0123704900. [16] White, Jay; Lueth, Chris (May 2010). "RAID-DP: NetApp Implementation of Double Parity RAID for Data Protection. NetApp Technical Report TR-3298". Retrieved 2013-03-02. [17] Newman, Henry (2009-09-17). "RAID's Days May Be Numbered". Enterprise Storage Forum. Retrieved 2010-09-07. [18] "Why RAID 6 stops working in 2019". ZDNet. 22 February 2010. [19] Scott Lowe (2009-11-16). "How to protect yourself from RAID-related Unrecoverable Read Errors (UREs). Techrepublic.". Retrieved 2012-12-01.

[20] Vijayan, S.; Selvamani, S.; Vijayan, S (1995). "Dual-Crosshatch Disk Array: A Highly Reliable Hybrid-RAID Architecture". Proceedings of the 1995 International Conference on Parallel Processing: Volume 1. CRC Press. pp. I–146ff. ISBN 0-8493-2615-X. [21] Disk Failures in the Real World: What Does an MTTF of 1,000,000 Hours Mean to You? Bianca Schroeder and Garth A. Gibson [22] J.L. Hafner, V. Deenadhaylan, K. Rao, and J.A. Tomlin. "Matrix methods for lost data reconstruction in erasure codes. USENIX Conference on File and Storage Technologies, p1530, Dec. 13-16, 2005. [23] Art S. Kagel (March 2, 2011). "RAID 5 versus RAID 10 (or even RAID 3, or RAID 4)". miracleas.com. Retrieved October 30, 2014.

* Mr. Nikola Zlatanov spent over 20 years working in the Capital Semiconductor Equipment Industry. His work at Gasonics, Novellus, Lam and KLA-Tencor involved progressing electrical engineering and management roles in disruptive technologies. Nikola received his Undergraduate degree in Electrical Engineering and Computer Systems from Technical University, Sofia, Bulgaria and completed a Graduate Program in Engineering Management at Santa Clara University. He is currently consulting for Fortune 500 companies as well as Startup ventures in Silicon Valley, California.