2014 IEEE International Conference on Big Data
Repair Efficient Storage Codes via Combinatorial Configurations Bing Zhu and Hui Li Institute of Big Data Technologies Shenzhen Eng. Lab of Converged Networks Technology Peking University Shenzhen Graduate School
[email protected],
[email protected]
Abstract—Fractional repetition (FR) codes are a special class of regenerating codes characterized by the exact and uncoded repair property. In this work, we propose an explicit method to construct FR codes from combinatorial configurations. The proposed construction gives FR codes with parameters that are not covered by prior approaches. Keywords-Distributed storage; regenerating codes; uncoded repair; combinatorial designs
Figure 1. An FR code with repetition degree ρ = 2 for a storage system with n = 4 nodes. A file consisting of 5 data packets is first encoded using an outer (6, 5) MDS code. Each coded block is then replicated on 2 distinct nodes, where N1 , . . . , N4 represent the storage nodes in the system.
I. I NTRODUCTION Distributed storage systems provide an efficient solution to store the large amount of online data that produced anytime and anywhere. However, in a realistic storage environment, node failure is a routine rather than an exception due to the commodity nature of physical storage devices [1]. In order to achieve the desired level of data reliability, practical storage systems are increasingly using erasure coding techniques to against the frequent failures [2]. In a typical coding scheme, a data file is encoded and distributed over n storage nodes such that any k out of the n nodes are sufficient to retrieve the source file. Although traditional erasure codes can reduce the storage overhead as compared to replication, they suffer from the drawback that repairing a lost block requires large amounts of network traffic. To lower repair bandwidth, regenerating codes are introduced in [2] with the capability to recover a failed node by connecting to a subset of d surviving nodes. However, upon failure of a node, each helper node needs to read all the data it stored from memory and transfer a linear combination of the retrieved data to the replacement node, which entails a large number of input/output operations. In this work, we consider a family of erasure codes that provide exact and uncoded repair at the minimum bandwidth regenerating point [3]. The encoding scheme is formed by concatenating an outer maximum distance separable (MDS) code and an inner fractional repetition code. Specifically, a file consisting of κ data packets is first encoded by a (µ, κ) MDS code with an output of µ coded blocks. Each block is then replicated ρ times and distributed over n storage nodes according to the inner fractional repetition code.1 Figure 1
illustrates an FR code with repetition degree ρ = 2. We note that there exist many available constructions of MDS codes in the literature, the main challenge of FR codes lies in the design of the inner fractional repetition code [4]–[7]. Contribution: We present an explicit construction of FR codes by utilizing the combinatorial structure known as configurations. The proposed technique allows us to construct FR codes that are not covered by those from prior methods. II. F RACTIONAL R EPETITION C ODES FROM C OMBINATORIAL C ONFIGURATIONS A. Configurations We start by providing a brief definition of a configuration. A configuration (vr , bt ) is a finite incidence structure of v points and b lines such that each line contains t points, each point lies on r lines, and two different points are connected by no more then one line. If v = b (and hence, r = t), the configuration is called symmetric and is denoted by vt . Consider a point set {1, . . . , 7} and 7 distinct lines. Each line contains exactly three points: {1, 2, 4}, {2, 3, 5}, {3, 4, 6}, {4, 5, 7}, {1, 5, 6}, {2, 6, 7}, {1, 3, 7}. This structure is a configuration 73 by definition, which is known as the Fano plane. We note that Fano plane is adopted in the construction of FR code due to the fact that Fano plane can also be viewed as a special case of Steiner systems [3]. The necessary conditions for the existence of a configuration (vr , bt ) are 1) v ≤ b and t ≤ r, 2) vr = bt, and 3) v ≥ r(t − 1) + 1. When t = 3, the necessary conditions are also sufficient. We refer the readers to [8] for more details about configurations.
1 We consider a homogeneous system model in which each node contains the same number of blocks.
978-1-4799-5666-1/14/$31.00 ©2014 IEEE
Kenneth W. Shum Institute of Network Coding The Chinese University of Hong Kong Shatin, Hong Kong
[email protected]
80
Figure 2.
III. C ONCLUSION AND F UTURE W ORK We present an explicit construction of FR codes, which is based on combinatorial configurations. We thus extend the construction of FR codes with more available parameters. Future work would include the implementation of FR codes in practical storage systems. While original FR codes are proposed assuming that each node stores the same number of packets, recent studies have included the extension of FR codes in heterogeneous distributed storage systems [9], [10]. It would be an interesting task to explore more constructions for such codes. ACKNOWLEDGMENT This work is supported in part by National Basic Research Program (973 Program under Grant 2012CB315904), Natural Science Foundation of Guangdong Province under Grant No. S2013020012822, and Shenzhen Basic Research Project under Grant JCYJ20130331144502026.
An FR code from the Desargues configuration 103 .
B. Constructions We present now the formal construction of FR codes from configurations. Construction: Consider a configuration (vr , bt ). Let each line represent a storage node and the points on the lines represent the indices of the coded blocks. An FR code from the configuration (vr , bt ) can be obtained by setting µ = v and ρ = r. Further, we refer to the number of blocks that a node contains as the storage capacity of this node, denoted by d. In this case, the above construction can be employed in a system with n = b nodes, each having a capacity d = t. Note that if we view the points as nodes and the lines as the indices of the coded blocks, we can generate an FR code with µ = b and ρ = t. Correspondingly, the parameters of the storage system are given by n = v and d = r. Example: Suppose that M = (m1 , . . . , m6 ) is a file that needs to be stored on the system. We adopt a (10, 6) MDS code which outputs 10 coded blocks. Consider the Desargues configuration 103 as shown in Figure 2. Based on the above construction, we can generate an FR code with µ = 10 and ρ = 3 for a storage system with n = 10 nodes N1 , . . . , N10 . It can be checked that there are at least 6 distinct points in any collection of 3 different lines. Therefore, a data collector can recover the source file M by connecting to any 3 nodes. In other words, the storage system can tolerate up to 7 node failures without data loss. In the presence of node failures, we replace the failed node by introducing a new node. The content of the new node is obtained by contacting some helper nodes and downloading data from them. For instance, suppose that node N1 fails, one block each can be downloaded from three nodes N2 , N4 , N6 . We note that the repair process is exact and uncoded, which implies that the contacted nodes read and transfer exactly the same amount of data without extra operations. Further, the property that each point lies on more then one line provides the flexibility in finding helper nodes during the repair. For example, node N1 can also be regenerated by connecting to nodes N5 , N7 , N9 alternatively.
R EFERENCES [1] S. Ghemawat, H. Gobioff, and S.-T. Leung, “The Google file system,” in Proc. 19th ACM Symp. on Operating Systems Principles (SOSP), pp. 29–43, Oct. 2003. [2] A. G. Dimakis, P. B. Godfrey, Y. Wu, M. Wainwright, and K. Ramchandran, “Network coding for distributed storage systems,” IEEE Transactions on Inf. Theory, vol. 56, no. 9, pp. 4539–4551, Sept. 2010. [3] S. El Rouayheb and K. Ramchandran, “Fractional repetition codes for repair in distributed storage systems,” in Proc. 48th Allerton Conference, pp. 1510–1517, Oct. 2010. [4] S. Pawar, N. Noorshams, S. El Rouayheb, and K. Ramchandran, “Dress codes for the storage cloud: simple randomized constructions,” in Proc. IEEE Int. Symp. on Inf. Theory (ISIT), pp. 2338–2342, Aug. 2011. [5] J. Koo and J. Gill, “Scalable constructions of fractional repetition codes in distributed storage systems,” in Proc. 49th Allerton Conference, pp. 1366–1373, Sept. 2011. [6] O. Olmez and A. Ramamoorthy, “Repairable replicationbased storage systems using resolvable designs,” in Proc. 50th Allerton Conference, pp. 1174–1181, Oct. 2012. [7] B. Zhu, K. W. Shum, H. Li, and S.-Y. R. Li, “On low repair complexity storage codes via group divisible designs,” in Proc. 19th IEEE Symp. on Comput. and Commun. (ISCC), Jun. 2014. [8] T. Beth, D. Jungnickel, and H. Lenz, Design Theory, Second Edition, Cambridge University Press, 1999. [9] B. Zhu, K. W. Shum, H. Li, and H. Hou, “General fractional repetition codes for distributed storage systems,” IEEE Commun. Lett., vol. 18, no. 4, pp. 660–663, Apr. 2014. [10] Q. Yu, C. W. Sung, and T. H. Chan, “Irregular fractional repetition code optimization for heterogeneous cloud storage,” IEEE J. Sel. Areas Commun., vol. 32, no. 5, pp. 1048–1060, May 2014.
81
