1204
IEEE Transactions on Consumer Electronics, Vol. 51, No. 4, NOVEMBER 2005
Shared Key Encryption of JPEG Color Images Subramania Sudharsanan, Senior Member, IEEE
Abstract — Several methods have been proposed for encrypting images by shared key encryption mechanisms since the work of Naor and Shamir. All the existing techniques are applicable to primarily non-compressed images in either monochrome or color domains. However, most imaging applications including digital photography, archiving, and internet communications nowadays use images in the JPEG compressed format. Application of the existing shared key cryptographic schemes for these images requires conversion back into spatial domain. In this paper we propose a shared key algorithm that works directly in the JPEG domain, thus enabling shared key image encryption for a variety of applications. The scheme directly works on the quantized DCT coefficients and the resulting noise-like shares are also stored in the JPEG format. The decryption process is lossless preserving the original JPEG data. The experiments indicate that each share image is approximately the same size as the original JPEG image retaining the storage advantage provided by JPEG compression standard. Three extensions, one to improve the random appearance of the generated shares, another to obtain shares with asymmetric file sizes, and the third to generalize the scheme for n>2 share cases, are described as well. 1. Index Terms — Image Encryption, JPEG Images, Shared Key Encryption, Visual Cryptography.
I. INTRODUCTION Visual cryptography based on secret sharing has received considerable attention since the publication of [1] by Naor and Shamir. A secret sharing encryption scheme creates several (n) shares of the original information and distributes to n participants. The decryption is carried out by using a prescribed number (k, k≤n) of subset shares. Fewer than k shares are insufficient to reconstruct the original data. The scheme proposed in [1] generated two shared key images from a given binary image as a printed page and a transparency of the same size. When the transparency is stacked on top of the printed page, the original image is formed. The two keys are generated in such a way that the share images are “random" looking with no semblance to the original image. It is a highly secure mechanism since the decryption is being performed by the human visual system when both shares are brought together. This is a 2-out-of-2 or {2,2} secret sharing method. An extension to k out of n secret sharing where k stacked 1
This work was supported in part by Sun Microsystems, Inc.
S. Sudharsanan is with the Department of Electrical and Computer Engineering, Queen’s University, Kingston, ON K7K 7L4, Canada (e-mail:
[email protected]). Contributed Paper Manuscript received September 19, 2005
images are needed to reconstruct the clear image has also been developed in [1]. Several publications that followed this development extended the basic visual cryptography using concepts from digital halftoning [2, 3] to address gray scale images and color pictures [4,5]. In many of these schemes, the encryption-decryption process is lossy. The original image is transformed into a haltoned image before deriving the share images. Halftoning is a lossy process. Furthermore, in many of the proposed schemes the reconstruction from the k shared images create a lossy version of the half-toned image itself. This led to several researchers’ efforts on improving contrast quality of the decrypted images. However, in some of the recent publications [6, 7] the original intent of performing the decryption using the human visual system with a simple mechanical system of stacking transparencies is lost by requiring computer processing to reconstruct the image. Though they lose the ability of decryption by visual means, such encryption schemes have a large number of potential applications in the Internet world [6]. The computer-based decryption also opens an avenue to a wide variety of shared image encryption systems. This has also lead to development of methods that apply to color images. In addition, the methods described in [6] and [7] also address computational complexity issues by designing schemes that require simpler bit-level arithmetic operations during decryption. In this paper, we describe a method similar in spirit to these schemes but addressing a larger domain of images stored in compressed digital formats. Large number of imaging applications including digital photography, archiving, and internet communications primarily use images stored in the JPEG format. Most of the digital cameras in the market use JPEG. Application of the existing shared key cryptographic schemes for these images requires conversion back into spatial domain. When transmission and storage of the shares are concerned, one may not necessarily be able to apply lossy compression techniques since the loss may result in an inability to decrypt. Hence, encryption techniques applicable in the compressed domain are needed. In this paper we propose a shared key algorithm that works directly in the JPEG domain, thus enabling shared key image encryption for a variety of applications. The scheme manipulates the quantized DCT coefficients and the resulting noise-like shares are also stored in the JPEG format. The decryption process that combines the shares is lossless and hence the original JPEG file’s fidelity is preserved. Our experiments indicate that each share image is approximately the same size as the original JPEG retaining the storage advantage provided by JPEG. We describe the key generation and the decryption schemes in detail and show that both encryption and decryption require only small modifications to
0098 3063/05/$20.00 © 2005 IEEE
S. Sudharsanan: Shared Key Encryption of JPEG Color Images
standard JPEG computational procedures. This in turn implies an easy adaptation for a hardware implementation which is particularly useful for digital camera applications. In the next section, we describe the basics of the proposed encryption mechanism. Section 3 details some software constructs to perform required experiments. The software is based on an open source implementation of JPEG [8]. We provide results from several important experiments and discuss the results. From the results, three extensions are identified and addressed in Section 4. Finally, we provide a summary.
1205
magnitude in binary. This is given by n
Xi can be represented in two’s complement form as
X inb = bin ( N )bin ( N − 1) bin (0)
(1)
bin (l ), l = 0,1, N is a binary value, either zero
where each
n
or one. Hence, the value of Xi can be obtained as [10] N
X in = −bin ( N ) × 2 N + ¦ bin ( N − l ) × 2 N −l.
(2)
l =1
II. PROPOSED SCHEME A. Monochrome Images The lossy version of JPEG image compression uses discrete cosine transforms (DCT) [9]. A monochrome image is first split into 8×8 non-overlapping blocks of pixels. An 8×8 DCT is applied to each block and the resulting coefficients are scalar quantized using a quantization matrix. The quantized coefficients are then converted from a two-dimensional representation to a one-dimensional vector by a process known as zig-zag scanning and sent to an entropy coder that uses either Huffman or arithmetic coding. The process is shown in Fig.1.
N = ª«log 2 X in º» . Now,
Note that N is dependent of both i and n and we have not explicitly shown the dependency to reduce clutter. Now, let us consider an encryption system where the image to be encrypted is given in the JPEG format. Let the image be decoded partially to obtain the zig-zagged, quantized
X in , i = 0,1, 63 corresponding to block Bn. At
coefficients
nb
n
this stage, the two’s complement representation Xi of Xi as indicated in the above equation is available. Now, two n1
n2
versions, Xi and Xi , are generated using a {2,2} shared secret key encryption method based on random assignment. We use an assignment scheme based on the one proposed by n
[7]. Each bit bi (l) is split into two shares using the following scheme:
°{[0,1] , [1, 0]} if bin (l ) = 1 [b (l ) b (l )] ∈ ® . n °¯{[0, 0] , [1,1]} if bi (l ) = 0 n1 i
n2 i
(3) nb
The above procedure is applied to all N+1 bits of Xi obtain
n1b Xi
and
n2b Xi
whose values
n1 Xi
and
n2 Xi
to
respectively
are calculated as N
Figure 1 JPEG Encoding Process
X in1 = −bin1 ( N ) × 2 N + ¦ bin1 ( N − l ) × 2 N −l
(4)
l =1
The zig-zag transformation transforms 8x8 two 2D values into 64 one dimensional quantized coefficients. For an 8×8 n
block B with an index n, let the quantized coefficients be
and N
X
n
denoted as , where X0 corresponds to the DC coefficient and the rest to the AC coefficients. The index n can be obtained as the standard scanning order of blocks of the image as defined in the JPEG standard [9]. We will also use the notation of
X = [ X , X , , X ] . According to baseline JPEG n
n 0
n 1
n 63
specification, the input image components are represented by 8-bit pixel values and the quantized samples are limited to at most 15 bits of magnitude and a sign bit. For gathering the n
statistics for entropy coding purposes, Xi values are placed in bins according to the number of bits needed to represent its
n2 i
= −b ( N ) × 2 + ¦ bin 2 ( N − l ) × 2 N −l . n2 i
N
(5)
l =1
n1
Using the above procedure, we obtain two sequences, X
and
n2
X , that represent zig-zagged, quantized coefficients for 8×8 n
blocks for the entire image based on the original sequence X . These two sequences can further be entropy coded using either Huffman or arithmetic coding techniques to produce the two share images in the JPEG format. n
nb
The sequence Xi and the binary representation Xi can be reconstructed simply by a bit-wise XOR operation ( ⊕ ) of the N+1-bit shares. For each block n,
1206
IEEE Transactions on Consumer Electronics, Vol. 51, No. 4, NOVEMBER 2005
X ibn (l ) = ª¬ ( bin1 ( N ) ⊕ bin 2 ( N ) ) ( bin1 (0) ⊕ bin 2 (0) )º¼ i = 0,1, , 63 The procedure is fairly simple to implement with the only possible complexity arising from a random number generator. A pictorial depiction of the encryption procedure is given in Fig. 2. From an implementation point of view, the changes required affect only the entropy coding part. As shown in Fig. 2, the shares are generated using the quantized coefficients and then two entropy encoders are used to generate the final JPEG share images. The scheme is independent of the type of entropy coder used. In fact, the shares could be generated using different coders such as a pair of an arithmetic coder and a Huffman coder. Hardware complexity required to construct the above encryption mechanism is dominated by the need for a strong random number generator. Strong random number generators that are acceptable in cryptography environments are becoming ubiquitous in many processor implementations [11]. The additional entropy encoder is relatively inexpensive. The bit decision process is described as serial but parallelization of it is relatively straightforward.
B. Color Images and JPEG Modes Just as with the JPEG compression, the proposed method is color blind. JPEG uses a component model for applications to enable coding of color images [9]. Typically, most applications use three components in the YCbCr color space [9]. The chrominance data is sub-sampled for better compression efficiency. The proposed encryption scheme uses the same JPEG approach to handle color images. Since the resulting image shares are JPEG images, any color space that can be handled by JPEG is also suitable for our application. JPEG supports up to 255 components in one image and hence support for a large variety of image formats. The description of the encryption technique in the previous subsection relies on a standard baseline mode of JPEG. JPEG also has a number of other modes such as progressive DCT, hierarchical, and lossless [9]. The progressive DCT mode by multiple scans of the quantized coefficients allows incremental transmission of the image so that the picture could be reconstructed sequentially with increasing fidelity. There are two modes for performing progressive DCT: spectral selection and successive approximation. In spectral selection, the AC coefficients are sent after sending the DC coefficients. The successive approximation uses a lower precision at the start and improves the precision in the subsequent scans. A mixed mode by combining the two schemes is also possible. For the progressive-DCT JPEG encoding, the proposed encryption scheme applies directly. Once the shares are created, the progressive-DCT based scans could be applied to generate two JPEG shares. During the decode, the shares have to be n1
completely decoded to produce X n
Figure 2 Shared-Key Encryption Process
The decoding procedure takes two JPEG coded shares and creates the quantized coefficients corresponding to the shares. The coefficients are simply sent through an XOR circuit to obtain the quantized coefficients of the clear image. The process is shown in Fig. 3. The additional complexity brought in by two parallel entropy decoder is small.
Figure 3 Decryption Process
n2
and X . XOR operation
on these values produce the X sequence which can further be processed to obtain the clear picture. JPEG also has a lossless mode. The lossless coding uses differential pulse code modulation (DPCM) and Huffman or arithmetic coding. The proposed encryption method does not directly apply to lossless mode but a scheme described in [7] could be used for this purpose. The hierarchical mode in JPEG uses multi-scale approach to compression. While the proposed scheme can potentially work in the hierarchical mode with DCT-based coding, it will not work in the lossless hierarchical mode.
S. Sudharsanan: Shared Key Encryption of JPEG Color Images
1207
III. EXPERIMENTAL RESULTS To conduct experiments, we made use of the JPEG implementation from the independent JPEG group (IJG)[8]. The code from IJG is written in C programming language and is highly portable to many different systems. The code is reasonably documented and amenable for making modifications. We ported the code on a Sun Blade 2000 Workstation and modified the JPEG encoder and the decoder to include the parts that generate the shares and perform decryption. The shares were generated only for the blocks that n
had non-zero X vectors. The pseudo random number generator we used is lrand48() which is available in the Sun Solaris platform. Better random number generators [13] are freely available, however lrand48() is used for the sake of convenience. The scheme for share generation is listed below. •
Read the image in JPEG format and perform variable-length decode to obtain the quantized n
•
Figure 4 Original JPEG File of Image 1
n
DCT coefficients, X , for each block B . Using (4) and (5), and random numbers from a pseudo-random number generator, produce the n1
n2
sequences X and X . • Perform entropy coding for the two sequences to produce two JPEG share images with the same dimensions and color space. The decryption procedure is simply the reverse of it, where the decoder has also been modified to read two JPEG images and perform entropy decoding to get two share sequences of the quantized DCT coefficients. The sequences are XORed to produce the decrypted sequences and the rest of the decode operations, dequantization and inverse transform, are continued to produce the decrypted image. We performed the experiments on several types of images and compared its performance. We show the results corresponding to an image, one from [14] (Image 1, size 768x512). Image 1 compressed with a JPEG quality setting of 90 (small quantizers) in the baseline mode using IJG’s JPEG encoder [8]. The shares were generated using the proposed method. Fig. 4 shows the original JPEG image, Image 1. Fig. 5 shows Share 1 as generated by the proposed scheme. Fig. 6 shows Share 2. The decrypted image is exactly same as the original, Fig. 4.
Figure 5 JPEG Share 1 of Image 1
Figure 6 JPEG Share 2 of Image 1
From Figs. 5 and 6, it is clear that the generated shares have no visual resemblance to the originals and are “random" looking. The randomness is slightly modulated by the underlying image shapes. The JPEG file size of Image 1 is 165,403 bytes. The share images have the size of 154,489 and 154,951 bytes. Also, if the images are represented in an uncompressed format such as TARGA, the storage requirement for Image 1 will be 1,179,666 bytes bytes. Shares developed using techniques in [7] will be of those sizes as well. Furthermore, the shares developed through methods
1208
IEEE Transactions on Consumer Electronics, Vol. 51, No. 4, NOVEMBER 2005
using digital halftoning [6] will be much larger than these values. Hence, the storage and transmission gains obtained by JPEG are directly applicable to the proposed method. These gains, depending on the resulting image quality and the source images, range between five and 40 times over the corresponding non-compressed image file sizes. Such gains cannot be achieved with previous shared encryption techniques for images. We have also conducted several experiments over a wide variety of images at various JPEG quality settings and observed similar results with respect to the size of the shares. IV. EXTENSIONS AND FURTHER EXPERIMENTS The technique proposed above can be further extended for several improvements. In this section, we discuss three such extensions. One is improvement of appearance of randomness in the shares when needed. The method as described so far is a {2,2} sharing scheme. We will point out how this can be extended to a {n,k} sharing scheme with relatively simpler additional steps. We will also detail a method for generating asymmetric shares such that the size of one share is only a fraction of the other’s or that of the original. In the following subsections, these extensions are described with corresponding experimental results. With these extensions, the application possibilities of the proposed method significantly improve. A.
Insertion of Coefficients If we needed the ability to change the appearance of the resulting shares, we should be able to control the quantized DCT coefficients and their distributions. To achieve that, we make an important observation. A quantized, DCT coefficient with zero value can produce non-zero valued shares according to (3) by assigning the same values for both shares. Using this technique, it is possible to introduce new coefficients at various different frequency locations where there are zero coefficients in the original image. When introducing the coefficients, the only constraints we impose are that the shares have to have the exact same bit pattern for these inserted coefficients and that the inserted values do not exceed the 12 bits (baseline JPEG restriction). For images with smooth regions with dominant chroma components and high contrast regions, the base scheme produces shares that show some envelope of the underlying picture. While this may be acceptable for many applications, some applications may demand unrecognizable noise-like structures for the shares. To arrive at noise-like shares, we use the insertion mechanism as described above. We insert new coefficients in the general region of spatial frequency points where the human visual sensitivity is high [12]. For the luminance components, the sensitivity starts increasing at the DC location and peaks at around the first few coefficients of the zig-zag scanned AC coefficient. For chroma components, this region includes the DC and a small region surrounding it [9]. Given such sensitivity patterns, we introduce new nonzero coefficients in the shares where the original has zero
values in the regions closer to the DC component. The amplitudes and signs of the introduced coefficients were chosen randomly. We selected the blocks that have their total energy below a threshold for candidates for insertion of coefficients. If 63
E = ¦ X in < Threshold ,
(6)
i =0
n
then the corresponding block B is chosen for insertion of coefficients. For experiments presented in this paper, we determine the value of Threshold using experimental evidence. The number of inserted coefficients is also chosen in an ad hoc manner using several experiments. An extension of this method may use information from the standard human visual studies as an aid to automatically determine which blocks need to be inserted with coefficients. To test the effectiveness of insertion of coefficients, an image with sharp contrasts as shown in Fig. 7 was utilized for experiments. Under the share generation mechanism considered so far, the resulting Share 1 is as shown in Fig. 8. The contours of the image have a strong presence in the share. Share 2 (not shown here) exhibits similar characteristics. We then carried out an experiment for coefficient insertion. The threshold value was set at 100. For each block that resulted in energy values lower than the threshold, as calculated in (6), it was injected with four randomly generated values (maximum of 12 signed bits). The resulting Share 1 image is shown in Fig. 9. The image appears as a random one. The file size of Share 1 with no insertions was 30,046 bytes and that of the coefficient inserted Share 1 was 73,616 bytes. Both file sizes are far less than that of the non-compressed, 24-bit Targa file size of 589, 942 bytes for the same resolution. But by introducing coefficients, the JPEG share sizes are increased. If randomness is desired, for the additional size penalty, the insertion scheme is a good choice. Again, varying degrees of coefficient insertions are possible based on standard human visual system engineering principles and reasonable compromised algorithms can be developed for this problem.
Figure 7 Test Image for Coefficient Insertion
S. Sudharsanan: Shared Key Encryption of JPEG Color Images
1209
14,660 bytes and Share 2 has a file size of 71,688 bytes. This asymmetry is desirable in applications where Share 1 needs to be stored in small capacity smart cards [15] and the contours shown in Share 2 are acceptable in such usage scenarios.
Figure 8 Share 1 with No Coefficient Insertions
Figure 10 Test Image for Asymmetric Share Generation
Figure 9 Share 2 with Coefficient Insertions
B. Asymmetric Shares The base mechanism we discussed in Section II-A results in producing shares with file sizes roughly equal to the original file size. Some applications may prefer an asymmetry in the resulting file sizes for security or communication bandwidth reasons. The file sizes depend on the number of coefficients and their distribution in each DCT block. Again, the DC component and the AC coefficients near it contain the bulk of the information in a picture and by selecting a few of the lower frequency components for share generation, shares with asymmetric sizes can be obtained. We again describe the results using some experiments. The image shown in Fig. 10 produces two asymmetric shares shown in Fig. 11 (Share 1) and Fig. 12 (Share 2). Share 1 was generated considering only the DC and the first AC coefficient alone. Share 2 contains the other shares of these coefficients and the rest of the AC coefficients unmodified from the original image. Share 1 contains less information and does not reveal anything significant regarding the original image. Share 2 however retains the high frequency information and hence shows some contours of the image. Share 1 has a file size of
Figure 11 Share 1, Smaller Share
Figure 12 Share 2, Larger Share
1210
IEEE Transactions on Consumer Electronics, Vol. 51, No. 4, NOVEMBER 2005
C. {n,n} and {n,k} Shared-key Methods The {2,2} secret sharing method we discussed could easily be extended to {n,n} and {n,k}, n,k>2 schemes using a binary tree since the base share generation method is lossless. The extension mechanism is illustrated in Fig. 13. An {n,n} scheme can be obtained by considering all the leaf nodes of binary trees, expanded to contain n leafs. An {n,k} scheme could be obtained by considering both leaf and non-leaf nodes by assigning k nodes that are sufficient to decrypt the image. For the example case shown in Fig. 13, a {5,5} scheme is obtained 2 2 2 2
0
by the five shares of S1,S2,S3,S4, and S2. A {5,3} secret 2 2 1 1
sharing method can be obtained considering S1,S2,S1,S2, and 0
S2. The original can be reconstructed using three shares
S11 , S21 , and S20 instead of all five.
Similarly, the decryption complexity is also augmented by an additional entropy decoder when compared to JPEG decoding. The hardware architecture of the encryption and the decryption processes can easily be obtained by modifying the standard JPEG pipelines. (A software implementation based on the IJG’s JPEG code [8] is available from the author.) It was also shown that the sizes of the resulting shares are comparable to that of their straight JPEG representations. We also showed several extensions to the method: insertion of coefficients for improved apparent randomness of the image, asymmetrically-sized share generation, and an {n,k} secret sharing. The proposed method can be extended to any other transform or wavelet domain techniques for image coding. An extension to JPEG 2000 [16] is under development. Currently, we are looking into application of this scheme for secure smartcards and digital rights management. REFERENCES [1]
[2]
[3]
[4] [5]
[6]
[7] Figure 13 {n,n} and {n,k} Shared-key Generation
V. SUMMARY AND FUTURE WORK {2,2} shared encryption schemes for images have been a growing concern for researchers. The original visual cryptography results for binary and gray-scale images have now been improved to include color images as well. However, a large portion of the digital image collections in practice are compressed using JPEG compression standard and an encryption scheme for them have been lacking. We proposed a new {2,2} shared encryption method for JPEG images. The proposed method can take in JPEG or other image formats and create two shares in the JPEG format. The shares are generated using the quantized DCT coefficients in the JPEG representation. The shares are used to reconstruct the original quantized DCT coefficients during decryption. The proposed scheme is applicable to color and monochrome images and offers all the compression advantage of JPEG to all the share images. The computational complexity of the encryption process is comparable to that of a standard JPEG encoder, an additional bit stream entropy encoder and a random number generator.
[8] [9] [10] [11] [12] [13]
[14]
[15]
[16]
M. Naor and A. Shamir, in: A. de Santis (Ed.), Visual Cryptography, Advances in Cryptology: Eurocrypt ’94, Lecture Notes in Computer Science, vol. 950, pp. 1-12, 1995. G. Ateniese, C. Blundo, A. de Santis, and D. Stinson, “Visual cryptography for general access structures," Information and Computation, vol. 129, no. 2, pp. 86-106, 1996. C-C. Lin and W-H. Tsai, “Visual cryptography for gray-level images by dithering techniques," Pattern Recognition Letters, vol. 24, pp. 349358, 2003. Y-C. Hou, "Visual cryptography for color images," Pattern Recognition, vol. 36, pp. 1619-1621, 2003. Y.C.Hou, C.Y. Chang, and F. Lin, "Visual cryptography for color images based on color decomposition," Proceedings of the Fifth Conference on Information Management, pp. 584-191, 1999. C-C. Chang, T-X, Yu, "Sharing a secret gray image in multiple images," Proceedings of the First IEEE International Symposium on Cyber Worlds, pp. 230-237, 2002. R. Lukac and K. N. Plataniotis, "Cost-effective encryption of natural images," Proceedings of 22nd Queen’s Biennial Symposium on Communications, pp. 89-91, 2004. (Also from the URL http://www.ece.queensu.ca/symposium/papers/2C_1. pdf) Independent JPEG Group, http://www.ijg.org. W. Pennebaker and J. Mitchell, JPEG Still Image Data Compression Standard, New York: Van Nostrand Reinhold, 1993. R. Katz, Contemporary Logic Design, Redwood City, California: Benjamin Cummings, 1994. Cryptography Research, Inc. “Evaluation of via c3 nehemiah random number generator," San Francisco, 2003. G. E. Legge and J. M. Foley, “Contrast masking in human vision," J. Opt. Soc. Am., vol. 70, no. 12, pp. 1458-1470, 1980. Mathematics Department, University of Salzburg, pLab: Theory and Practice of Random Number Generation, http://random.mat.sbg.ac.at/. http://www.cryptography.com/resources/whitepaper s/VIA_rng.pdf Efficient Content-based Retrieval Group, University of Washington, http://www.cs.washington.edu/research/imagedatab ase/groundtruth/. D. Scheuermann, “The smartcard as a mobile security device,” IEE Journal on Electronics & Communication Engineering,”,Êvol. 14, no. 5,Êpp. 205 - 210, Oct. 2002. Michael W. Marcellin, Michael J. Gormish, Ali Bilgin, Martin P. Boliek: An Overview of JPEG-2000. Proceedings of the Data Compression Conference 2000, pp. 523-544, 2000.
S. Sudharsanan: Shared Key Encryption of JPEG Color Images
Subramania Sudharsanan (M’91-SM’00) has been an Associate Professor at the Department of Electrical and Computer Engineering at Queen’s University, Ontario, Canada since July 2003. Prior to that he held industrial positions at several leading companies including IBM, Digital Equipment Corporation, and Sun Microsystems. His research interests are in multimedia processing and computer architecture. He received his Ph.D. from the University of Arizona in 1990. He is a co-inventor of 16 US and eight European and Japanese patents.
1211
