An Algorithm for a Two-Disk Fault-Tolerant Array with (Prime 1) Disks

Size: px

Start display at page:

Download "An Algorithm for a Two-Disk Fault-Tolerant Array with (Prime 1) Disks"

Denis Blake
5 years ago
Views:

1 An Algorithm for a Two-Disk Fault-Tolerant Array with (Prime 1) Disks Sanjeeb Nanda and Narsingh Deo School of Computer Science University of Central Florida Orlando, Florida sanjeeb@earthlink.net, deo@cs.ucf.edu

2 Abstract In recent years commercial Redundant Arrays of Inexpensive Disks systems have gained considerable appeal due to their enhanced I/O bandwidths, capacities and economy. However, the continued demand for larger capacities at low cost, has led to the use of larger arrays with compromised disk quality. This has challenged RAID systems to offer better fault-tolerance without sacrificing performance or space. In this paper, we present an algorithm for recovery from two random disk failures in an array of (P 1) disks, where P is a prime number. The algorithm relies on exclusive-or parity bit-computed uniformly across all the disks. We give a constructive proof of the correctness of the algorithm and demonstrate the key steps with examples.

3 I. Introduction While CPU throughputs have doubled approximately every two years, enhancements to I/O bandwidth, in contrast, have been relatively conservative. In fact, I/O bottlenecks continue to restrict many applications from fully exploiting their processors. This fact motivated Patterson et al. to introduce Redundant Arrays of Inexpensive Disks (RAID) [5, 12, 13], which substitutes large expensive disks with arrays of inexpensive disks to provide greater cumulative I/O bandwidth. However, such arrays are probabilistically more susceptible to failure than their disks individually. Fortunately, several forms of RAID effectively solve this problem. Patterson originally defined RAID levels 0 through 5 [12] and the RAID Advisory Board subsequently added level 6 to the existing definitions. A RAID level 0 with N disks may partition each disk into equal sized blocks and order the blocks in a round-robin fashion. This enhances I/O throughput by permitting up to N blocks to be accessed concurrently when reading from, or writing to the array sequentially. Fig. 1 displays the order of the data blocks in a RAID level 0 with N disks and M data blocks per disk. But, RAID level 0 does not offer any redundancy. In contrast, levels 1 through 5 provide tolerance to the failure of exactly one disk in the array N N + 1 N + 2 N + 3 2N 2N + 1 2N + 2 2N + 3 3N (M 1)N + 1 (M 1)N + 2 (M 1)N + 3 MN Disk 1 Disk 2 Disk 3 Disk N Fig. 1. RAID level 0 with N disks and M blocks per disk, with the order of each block in the array indicated by an integer value. Since RAID levels 1 through 5 can recover from only one disk failure, their use is typically confined to smaller arrays where the probability of two or more disk failures is negligible. In fact, RAID levels 1 and 5 are common choices in such environments. RAID level 1 is simply a pair of disks with the data on any one

4 being mirrored on the other. Since this scheme does not use parity, it is computationally inexpensive and therefore, has relatively good I/O performance. However, its drawback is that, it consumes half the array s space for storing redundant data. In contrast, a RAID level 5 consists of at least three disks using parity to provide redundancy and various optimizations to enhance its performance [2, 6]. On the other hand, large arrays may need to tolerate two or more disk failures. A popular method that achieves this partitions an array of disks into RAID subsets each tolerant to a single disk failure and then creates another RAID level by using each aforementioned subset as a disk. In fact, practically every contemporary RAID controller offers RAID levels using this method. Some salient ones are RAID levels 10 and 50. RAID level 10 is obtained by creating a RAID level 0 using subsets that are each a RAID level 1. Similarly, RAID level 50 is obtained by creating a RAID level 0 using subsets that are each a RAID level 5. Fig. 2 displays a RAID 10 using three subsets of RAID level 1. Note that, identically ordered blocks in the array contain the same data M 2 3M 2 3M 1 3M 1 3M 3M Subset 1 (RAID 1) Subset 2 (RAID 1) Subset 3 (RAID 1) Fig. 2. A RAID level 10 with three virtual disks, each comprised of two disks, with the order of each block in the array given by an integer value. Note that, blocks in each pair of disks comprising a RAID 1 subset are identically ordered and therefore contain identical data. Although such arrays can tolerate as many disk failures as there are subsets, each failure is restricted to a unique subset. Thus, there is a compelling need for arrays that can tolerate multiple arbitrary disk failures. For most practical array sizes, the probability of occurrence of more than two simultaneous disk failures is generally small enough to be ignored, and a lesser number of failures can be tackled by the use of RAID systems resistant to two disk failures.

5 For this purpose, the RAID Advisory Board defined RAID level 6 as one that can tolerate two arbitrary disk failures. Numerous implementations of this type have been developed over the years, each with advantages and disadvantages. A well-known technique based on Reed-Solomon codes [14] has not gained much commercial favor because it uses Galois Field arithmetic to compute parities, which is computationally more expensive than using exclusive-or. Park proposed an algorithm [10] for evaluating the number of disks N and the configuration of data and parity, when given the ratio of disk space R used for storing parity. However, the solution is generally not optimal because RN is often greater than 2. The EVENODD technique proposed by Blaum et al. [1] allows non-prime values for the number of disks in the array only by artificially extending the actual size of the array to a prime number with imaginary disks containing 0 s as their contents. The latter scheme has greater computational complexity when the number of disks is non-prime and calculations to derive parity have to factor in the presence of the imaginary disks. In this paper, we present an algorithm to create an array of (P 1) disks, where P is prime, which can recover from the failure of any two arbitrary disks in the array. The algorithm generates a solution with (P 3)/2 blocks of data and one block of parity per disk, each block being of equal size. Each parity block s value is then obtained by computing the bit-wise exclusive-or of (P 3) data blocks that are specified by the algorithm. The solution is then extended for practical use by repeating the data and parity configuration of the (P 1)/2 data and parity blocks an integral number of times. The algorithm embodies numerous strengths that make it more attractive than competing techniques that provide tolerance to two arbitrary disk failures. First, the fraction of space used for storing parity in an array with (P 1) disks is 2/(P 1). This ratio is optimal because a solution to reconstruct two disk failures with a lesser value is not possible. Second, the algorithm distributes parity uniformly over all the disks, permitting data to be read from, and parity to be written to all disks concurrently. This maximizes cumulative I/O throughput to the disks. Third, the algorithm is computationally inexpensive because it calculates each parity block s value by using exclusive-or operations only. Finally, since (P 1) is even, this algorithm is well suited for many external storage enclosures that accommodate an even number of disks. Next, we shall state our algorithm and then, prove its correctness.

6 II. Algorithm To Create A 2-Disk Fault-Tolerant Array Of (P 1) Disks Let P 5 be a prime and M P = (m ij ) be a matrix with m ij = (2P i j 1) mod P, 0 i, j P 1. Let d 0, d 1, d 2, d P 2 be an array of P 1 disks, where P is prime. Each disk contains (P 3)/2 data blocks and a parity block. Each data block in disk d is assigned a unique unordered pair {x, y} as a data-block tag if m xy = m dd, and the parity block the singleton [d] as a parity-block tag, where 0 x, y P 2. The parity in the block with tag [d] is then calculated by taking the bit-wise exclusive-or of all data blocks with tags having d as an element. In other words, the data in the block with tag {x, y} is used to calculate the parities in the blocks with tags [x] and [y]. Fig. 3 displays the pseudo-code that implements the algorithm. In the pseudocode we represent the disks in an array by the variable d, and the equal sized blocks within each disk by the variable b. Fig. 4 displays the result of the algorithm applied to create a two-disk fault-tolerant array with six disks. for i 0 to (P 1) do for j i to (P 1) do m(i, j) (2P i j 1) mod P; for disk d 0 to (P 2) do { b 0; for i 0 to (P 3) do for j (i + 1) to (P 2) do if (m(i, j) = m(d, d)) then { block b in disk d is assigned data-block tag {i, j}; b b + 1; } } block b of disk d is assigned parity-block tag [d]; Fig. 3. Pseudo-code to implement the algorithm.

7 {3, 4} {0, 2} {1, 3} {2, 4} {0, 1} {0, 3} {2, 5} {4, 5} {0, 4} {1, 5} {3, 5} {1, 2} [0] [1] [2] [3] [4] [5] Fig. 4. Algorithm applied to create a six-disk system. III. Proof Of Correctness Of The Algorithm As defined earlier in our algorithm, P 5 is a prime and M P = (m ij ) is a matrix with m ij = (2P i j 1) mod P, 0 i, j P 1. Fig. 5 illustrates M 7 as an example Fig. 5. The matrix M 7. Now, we state and prove some properties needed to prove the correctness of our algorithm. Property 1 Let S(x) = {(i, j) m ij = x} and S(y) = {(i, j) m ij = y}, 0 x, y P 1. Let, r S(x) S(y) be a function, with r(i, j) = (i, k). Similarly, let c S(y) S(x) be a function, with c(i, j) = (k, j). Note that r and c are isomorphic. Thus, r represents a move from an entry with value x to an entry with value y on the same row, and c represents a move from an entry with value y to an entry with value x on the same column. Then, given

8 elements of the form (i, i) S(x) and (j, j) S(y), r (c r) ½(P 1) (i, i) = (j, j). In other words, exactly P total moves, comprised of ½(P + 1) moves of the form r interleaved with ½(P 1) moves of the form c, are required to start from m ii = x and reach m jj = y. For example in M 7, let r S(6) S(2), and c S(2) S(6). Then, exactly seven total moves comprised of r alternating with c are needed to start from the (0, 0) entry on the major diagonal, having value 6, and reach the (2, 2) entry on the major diagonal, having value 2. The sequence of coordinates traversed is as follows (0, 0), (0, 4), (3, 4), (3, 1), (6, 1), (6, 5), (2, 5), (2, 2) Proof Let, r S(x) S(y), with r(i, j) = (i, k). Since 2P i j 1 x (mod P) and 2P i k 1 y (mod P), it follows that k j = x y; hence k = j + x y. Thus, a move of the form r takes us from the (i, j) entry to the (i, j + x y) entry. Similarly, let c S(y) S(x), with c(i, k) = (l, k). Since, 2P i k 1 y (mod P) and 2P l k 1 x (mod P), it follows that l i = y x; hence l = i + y x. Thus, a move of the form c takes us from the (i, k) = (i, j + x y) entry to the (l, k) = (i + y x, j + x y) entry. Then, the composition (c r) S(x) S(x) takes us from the (i, j) entry to the (i + y x, j + x y) entry. Now, suppose we start from the (i, i) entry with m ii = x, then (c r) S(x) S(x) takes us from the (i, i) entry to the (i + y x, i + x y) entry. It is then easy to see that the (i + y x, i + x y) entry is not on the major diagonal, for else we would have i + y x i + x y (mod P), that is, 2(x y) 0 (mod P). A contradiction. Using the same argument we can show that, for any integer n 1, (c r) n S(x) S(x) will take us from the (i, i) entry to the (i + n(y x), i + n(x y)) entry, and that, the latter cannot be on the major diagonal. However, r (c r) ½(P 1) S(x) S(y) will take us from the (i, i) entry to the (i + ½(P 1)(y x), i + ½(P 1)(x y) + x y) entry. It is then easy to see that the latter is an entry on the major diagonal since, i + ½(P 1)(y x) (i + ½(P 1)(x y) + x y) = P(y x) 0. Thus, ½(P + 1) moves of the form r interleaved with ½(P 1) moves of the form c are required to start from m ii = x and reach m jj = y, for a total of exactly P moves.

9 Let d 0, d 1, d 2, d P 2 be an array of (P 1) disks with data-block tags. A sequence of data-block tags {{u 0, v 0 }, {u 1, v 1 }, {u 2, v 2 }, {u K, v K }} from both disks is called a path if {u i, v i } {u i + 1, v i + 1 } = 1, where 0 i < K, and K P 4. Such a path is a cycle if {u k, v k } {u 0, v 0 } = 1. Property 2 Proof No pair of disks in d 0, d 1, d 2, d P 2 can have a cycle of data-block tags. Suppose to the contrary that a pair of disks d x and d y has a cycle C = {{u 0, v 0 }, {u 1, v 1 }, {u 2, v 2 }, {u K, v K }} of data-block tags, K P 4. Note that, each tag {u i, v i } in the sequence is obtained from the coordinates of an entry in M P with value m xx or m yy. Furthermore, if a tag in C corresponds to the coordinates of an entry with value m xx then its preceding and succeeding tags in C correspond to the coordinates of an entry with value m yy. Thus, the cycle C describes a sequence of alternating horizontal and vertical moves in M P that starts, without loss of generality, at an entry with value m xx and moves horizontally to an entry with value m yy, followed by a vertical move to an entry with value m xx, and so on, until we arrive back at the starting entry with value m xx with a vertical move. Let S (x) = {(i, j) m ij = m xx } and Let S (y) = {(i, j) m ij = m yy }. Let, r S (x) S (y) be a function with r(i, j) = (i, k), and let c S (y) S (x) be a function with c(i, j) = (k, j). Then, using arguments from the proof of Property 1, we can show that, (c r) n S (x) S (x) takes us from the (i, j) entry to the (i + n(y x), j + n(x y)) entry, for any integer n 1. Hence, the presence of C implies that, i + n(y x) i (mod P), and j + n(x y) j (mod P). Since x y, n must equal P. This is a contradiction, since K P 4; and therefore, n (P 3)/2. A path of data-block tags T = {{u 0, v 0 }, {u 1, v 1 }, {u K, v K }} in a pair of disks d x and d y is a maximal path if there does not exist any tag {u, v} in d x or d y such that, {u, v} T, and either {u 0, v 0 } {u, v} = 1 or {u K, v K } {u, v} = 1. Then, we observe that every data block tag in disks d x and d y must lie on a maximal path. Let us now prove the following theorem. Theorem The array of disks d 0, d 1, d 2, d P 2 can tolerate two disk failures.

10 Proof Assume disks d x and d y have failed, where x, y {0, 1, P 2}. Let T = {{u 0, v 0 }, {u 1, v 1 }, {u K, v K }} be a maximal path of data-block tags in d x and d y, K P 4 Assume without loss of generality that {u 0, v 0 } {u 1, v 1 } = {v 0 } and {u K 1, v K 1 } {u K, v K } = {u K }. Then, the parity-block tags [u 0 ] or [v K ] cannot both lie on the failed disks. Suppose to the contrary that parity-block tags [u 0 ] and [v K ] both lie on the failed disks. Then, the tag [u 0 ] must lie on the same disk as the data-block tag {u K, v K }, and [v K ] must lie on the same disk as the data-block tag {u 0, v 0 }, because by construction, a data block on a disk does not contribute to the parity in that disk s parity block. Now, the sequence of tags [u 0 ], {u 0, v 0 }, {u 1, v 1 }, {u K, v K }, [v K ] corresponds to the sequence of coordinates in M P starting with (u 0, u 0 ) followed by coordinates (u 0, v 0 ), (u 1, v 1 ), (u K, v K ), and ending with the (v K, v K ). A traversal through the aforementioned sequence of coordinates would require exactly P moves, by Property 1. This is a contradiction for K P 4, and therefore the sequence [u 0 ], {u 0, v 0 }, {u 1, v 1 }, {u K, v K }, [v K ] contains at most P 1 tags describing P 2 moves. Hence, the tags [u 0 ] or [v K ] cannot both lie on the failed disks. Suppose [u 0 ] does not lie on a failed disk. Then, we can reconstruct the failed disks starting with the data block with tag {u 0, v 0 } using its parity in parity block with tag [u 0 ] and the remaining data blocks contributing to that parity, none of which can be on disk d x or d y for the following reason. If there is a data block on disks d x or d y different from the one with tag {u 0, v 0 } contributing to the parity in the parity block with tag [u 0 ], then that data block would have a tag of the form {u 0, v} or {v, u 0 } and could be added to the maximal path T if it is not already in T. This would be a contradiction for T is maximal. On the other hand, if {u 0, v} or {v, u 0 } is already in T, then T would form a cycle. This too would be a contradiction, for by Property 2 no path in disks d x and d y can have a cycle. Once the data block with tag {u 0, v 0 } has been reconstructed, we can next reconstruct the data block with tag {u 1, v 1 } by observing that {u 0, v 0 } {u 1, v 1 } = {v 0 } and therefore, all the remaining data blocks contributing to the parity in the parity block with tag [v 0 ] are intact or have been reconstructed. In this manner we can iteratively reconstruct all data blocks whose tags occur in the maximal path T. Furthermore, since each data block tag lies on some maximal path, we can reconstruct all corresponding data blocks. Thereafter, the reconstruction of the blocks containing parity is trivial.

11 References [1] M. Blaum, J. Brady, J. Bruck, and J. Menon, EVENODD An efficient scheme for tolerating double disk failures in RAID architectures, IEEE Transactions on Computers, vol. 44, no. 2, pp , [2] P. M. Chen and E. K. Lee, Striping in a RAID level 5 disk array, Proceedings of the 1995 ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems, pp , [3] P. M. Chen and D. A. Patterson, Maximizing performance in a striped disk array, Proceedings of the 17 th IEEE Annual International Symposium on Computer Architecture, pp , [4] J. N. Gray, et al., Parity striping of disk arrays Low cost reliable storage with acceptable throughput, Proceedings of the 16 th International Conference on Very Large Data Bases, pp , [5] R. H. Katz, D. A. Patterson and G. A. Gibson, Disk system architectures for high performance computing, Proceedings of the IEEE, vol. 77, no. 12, pp , Dec [6] E. K. Lee and R. H. Katz, The Performance of Parity Placements in Disk Arrays, IEEE Transactions on Computers, vol. 42, no. 6, pp , Jun [7] W. Litwin and T. Schwarz, Lh*rs A high availability scalable data structure using Reed Solomon codes, Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp ACM Press, [8] R. R. Muntz, and J. C. S. Lui, Performance analysis of disk arrays under failure, Proceedings of the 16 th International Conference on Very Large Data Bases, pp , [9] S. W. Ng, Improving disk performance via latency reduction, IEEE Transactions on Computers, vol. 40, no. 1, pp , [10] C.-I. Park, Efficient placement of parity and data to tolerate two disk failures in disk array systems, IEEE Transactions on Parallel and Distributed Systems, pp , [11] C.-I. Park and T.-Y. Choe, Striping in Disk Array RM2 Enabling the tolerance of double disk failures, Supercomputing

12 [12] D. A. Patterson, G. A. Gibson, and R. H. Katz, A case for redundant arrays of inexpensive disks, Proceedings of the ACM SIGMOD, pp , Jun [13] D. A. Patterson, P. M. Chen, G. Gibson, and R. H. Katz, Introduction to redundant arrays of inexpensive disks, Proceedings of the IEEE COMP- CON, pp , Spring [14] J. S. Plank, A tutorial on Reed-Solomon coding for fault-tolerance in RAID-like systems, Software - Practice and Experience, vol. 27, no. 9, pp , [15] P. Scheuermann, G. Weikum and P. Zabback, Data partitioning and load balancing in parallel disk systems, VLDB Journal, vol. 7, no. 1, pp , [16] H. Simitci and D. A. Reed, Adaptive disk striping for parallel input/output, Proceedings of the Seventh NASA Goddard Conference on Mass Storage Systems, IEEE Computer Society Press, pp , [17] P. Triantafillou and C. Faloutsos, Overlay striping and optimal parallel I/O for modern applications, Parallel Computing, vol. 24, no. 1, pp , 1998.

Secure RAID Schemes from EVENODD and STAR Codes

Secure RAID Schemes from EVENODD and STAR Codes Wentao Huang and Jehoshua Bruck California Institute of Technology, Pasadena, USA {whuang,bruck}@caltechedu Abstract We study secure RAID, ie, low-complexity