Enabling user mobility for optical camera communication using mobile

Vol. 26, No. 17 | 20 Aug 2018 | OPTICS EXPRESS 21762

Enabling user mobility for optical camera communication using mobile phone JIN SHI, JING HE,* JING HE, ZHONGWEI JIANG, YUDONG ZHOU, AND YAOQIANG XIAO College of Computer Science and Electronic Engineering, Hunan University, Changsha 410082, China *[email protected]

Abstract: We propose and experimentally demonstrate an optical camera communication (OCC) supporting user mobility. A mobile test platform is designed to emulate user mobility. In the mobile scenario, dynamic column matrix selection algorithm is proposed to select an appropriate column matrix with high extinction-ratio (ER) while avoiding the blooming effect. The mobile phone is placed on the moving track to receive the visible light at a vertical distance of 60 cm. By varying the moving speed at 20, 40, 60, 80, and 100 cm/s and lateral distance at 50 and 70 cm respectively, the system performance using the proposed algorithm is investigated. The experimental results show that with the increase of lateral distance (far from the light source) and user moving speed, the system performance gets degraded. Moreover, it demonstrates that the mobile system can achieve a throughput of 4.08 kbps under a low illuminance of 275 lx. © 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement OCIS codes: (230.3670) Light-emitting diodes; (060.4510) Optical communications; (060.4080) Modulation.

References and links 1.

D. Tsonev, S. Videv, and H. Haas, “Towards a 100 Gb/s visible light wireless access network,” Opt. Express 23(2), 1627–1637 (2015). 2. J. Shi, Y. Hong, J. He, R. Deng, and L.-K. Chen, “Experimental demonstration of OQAM-OFDM based MIMONOMA over visible light communications,” in Optical Fiber Communication Conference (OFC 2018), paper M2K.3. 3. Y. Hong and L.-K. Chen, “Toward user mobility for OFDM-based visible light communications,” Opt. Lett. 41(16), 3763–3766 (2016). 4. Y. Wang, L. Tao, X. Huang, J. Shi, and N. Chi, “8-Gb/s RGBY LED-Based WDM VLC system employing high-order CAP modulation and hybrid post equalizer,” IEEE Photonics J. 7(6), 7904507 (2015). 5. A. Sewaiwar, S. V. Tiwari, and Y. H. Chung, “Mobility support for full-duplex multiuser bidirectional VLC networks,” IEEE Photonics J. 7(6), 7904709 (2015). 6. J. Shi, Y. Wang, X. Huang, and N. Chi, “Enabling mobility in LED based two nodes VLC network employing self-adaptive STBC,” in Optical Fiber Communication Conference (OFC 2018), paper Th2A.23. 7. W.-C. Wang, C.-W. Chow, L.-Y. Wei, Y. Liu, and C.-H. Yeh, “Long distance non-line-of-sight (NLOS) visible light signal detection based on rolling-shutter-patterning of mobile-phone camera,” Opt. Express 25(9), 10103– 10108 (2017). 8. C. W. Chow, C. Y. Chen, and S. H. Chen, “Enhancement of signal performance in LED visible light communications using mobile phone camera,” IEEE Photonics J. 7(5), 7903607 (2015). 9. Y. Liu, C.-W. Chow, K. Liang, H.-Y. Chen, C.-W. Hsu, C.-Y. Chen, and S.-H. Chen, “Comparison of thresholding schemes for visible light communication using mobile-phone image sensor,” Opt. Express 24(3), 1973–1978 (2016). 10. J. Shi, J. He, J. He, R. Deng, Y. Wei, F. Long, Y. Cheng, and L. Chen, “Multilevel modulation scheme using the overlapping of two light sources for visible light communication with mobile phone camera,” Opt. Express 25(14), 15905–15912 (2017). 11. R. Deng, J. He, Y. Hong, J. Shi, and L. Chen, “2.38 Kbits/frame WDM transmission over a CVLC system with sampling reconstruction for SFO mitigation,” Opt. Express 25(24), 30575–30581 (2017). 12. K. Liang, C.-W. Chow, and Y. Liu, “Mobile-phone based visible light communication using region-grow light source tracking for unstable light source,” Opt. Express 24(15), 17505–17510 (2016).

1. Introduction Visible light communication (VLC) is a favorable option for future optical wireless access and considered as promising supplement to radio frequency (RF) communication [1]. VLC #335094 Journal © 2018

https://doi.org/10.1364/OE.26.021762 Received 13 Jun 2018; revised 24 Jul 2018; accepted 24 Jul 2018; published 7 Aug 2018

Vol. 26, No. 17 | 20 Aug 2018 | OPTICS EXPRESS 21763

using light-emitting diodes (LEDs) has gained great attention, which can enable communication and illumination simultaneously. Meanwhile, it can offer some advantages such as immunity to electromagnetic interference, low power consumption, simple installation and large spectral availability [2–4]. To detect the signal from visible light, the positive-intrinsic-negative (PIN) and the avalanche photodiode (APD) are widely used as receiver in VLC systems [5,6]. Recently, optical camera communication (OCC) using Complementary Metal-OxideSemiconductor (CMOS) camera for VLC system has received extensive attention due to its flexibility and low-cost [7,8]. Enabled by the rolling shutter effect, the CMOS camera can capture one frame line-by-line instead of capturing one frame in a shot by using global shutter effect. Hence the transmitted data are represented by bright and dark fringes in the image frame, resulting in a higher achievable throughput than the frame rate of CMOS camera. Moreover, it is common that each mobile phone is equipped with a built-in CMOS camera. Therefore, using mobile phone as receiver is available for OCC. To enhance the system throughput, many technologies have been proposed, such as blooming mitigation scheme, thresholding scheme, multilevel modulation and wavelength division multiplexing (WDM) technology [8–11]. However, the blooming mitigation scheme in [8] can only work well when light source is positioned at the center of received images. Meanwhile, these experiments are only performed at fixed points in stationary scenario. It means the position of the receiver always keeps fixed with respect to light source. In addition, in stationary scenario, the fixed column matrix can be applied for all received image frames to obtain a column of grayscales with high extinction-ratio (ER) while avoiding the “blooming effect” due to the pixel saturation, leading to the bright fringe getting wider. Compared with stationary scenario, it is preferable that the OCC system can support user mobility as shown in Fig. 1. However, for mobile OCC system, the receiver position always varies along with user mobility, resulting in the aforementioned fixed column matrix selection (CMS) scheme inapplicable. In [12], a tracking moving light source scheme using region-grow algorithm for CMS has been reported considering the light source is unstable. But the light source may disappear in the received images for mobile OCC and the scheme has high computational complexity. Moreover, the distance between light source and mobile receiver will affect the system performance due to light intensity fluctuation. In addition, compared to the stationary OCC, the user mobility introduces severe inter-symbol interference (ISI), which means each row of pixels suffer more interference from neighboring pixels. Thus, mobile OCC system is worth investigating for practical consideration.

Fig. 1. The scenario of mobile OCC system with user mobility.

To the best of our knowledge, we propose and demonstrate a mobile OCC system enabling user mobility by using a commercial mobile phone camera for the first time. To select an appropriate column matrix with high ER, dynamic CMS algorithm is proposed for the mobile OCC system. The mobile platform with controllable lateral distance and moving speed is implemented to emulate user mobility. At the different moving speed of 20, 40, 60, 80 and 100 cm/s and lateral distance of 50 and 70 cm, the proposed system is evaluated in terms of bit error rate (BER) performance. The experimental results show that, with the

Vol. 26, No. 17 | 20 Aug 2018 | OPTICS EXPRESS 21764

increase of moving speed and lateral distance, the system performance gradually gets degraded. At the moving speed of 100 cm/s and the lateral distance of 70 cm, the mobile OCC system can achieve a throughput of 4.08 kbps with a BER of 7.4 × 10−3 while keeping the illuminance below 275 lx. 2. Experiments and algorithm A proof-of-concept experimental setup of mobile OCC system enabling user mobility with mobile phone camera is shown in Fig. 2. The transmitted data modulated by On-Off Keying (OOK) are offline generated from personal computer (PC), then fed into a single port readonly memory (ROM) of field-programmable gate array (FPGA) from Xilinx Spartan 6 series (xc6slx45). The general-purpose input/output (GPIO) pin outputs logic high or logic low representing the data bit “1” or “0” stored in ROM respectively. It is used to control the “ON” or “OFF” state of LED. In order to increase the current drive and control the brightness of the light, the LED driver circuit is used to drive a white-light LED (Cree XLamp XR-E) for signal transmission. After free space transmission, a mobile phone (iPhone 6s) is mounted on the mobile platform in order to receive the signal and to emulate user mobility. The lateral distance of mobile platform can be set to d cm with a maximum distance of 100 cm. The moving speed can also be adjusted with a controlled speed of v cm/s. In each case, a 3-minute video is recorded using mobile phone camera. The video is recorded at the frame rate of 60 fps and the resolution of 1080 x 1920. Moreover, the exposure time is manually fixed. It is worth noting that the smaller exposure time has clearer stripe, whereas the brightness of image will be very low due to the short time letting light in. Hence, the trade-off exposure time is set to 1/15400. Besides, to increase the receiver sensitivity to light, the camera sensor sensitivity (ISO) is set to its maximum at 735. The recorded video is then loaded into Matlab for further signal processing. In addition, a digital light meter (Benetech-GM1020) is used to measure the intensity of received signal. (OOK) 60 cm

Packet 1

FPGA

LED Driver

Packet 2

Smartphone

(OOK)

1 2

Distance: d cm Speed: v cm/s

PC

GPIO

LED

3

PC 1 Mobile Phone 3

2 Mobile Platform

4 1

Packet N

2 111000 111010

Header

Payload

111000 111010

Payload

111000 111010

7

Payload

The structure of data packet

6

3 DC Power Supply 4 LED Light Source 5 FPGA 6 LED Driver 7 LED

5

Fig. 2. Experimental setup of mobile OCC using mobile phone. The insets are the structure of data packet (left) and the photo of experimental setup (right).

The structure of data packet is illustrated in the inset of Fig. 2. Each packet consists of header and payload. A 12-bit header is used for synchronization and clock recovery. 68 data bits are used for payload. It is worth noting that the frame processing time as “blind time” is about 38% of frame duration. Hence, the header and payload transmit three times successively to make each frame include a complete data packet. The transmitted baud rate is 14.4 Kbaud/s ((12 + 68)*3 times *60 fps). The system process diagram is shown in Fig. 3. The recorded video is firstly extracted to image frames according to the frame rate. Then, grayscale conversion is used to convert the received image into grayscale image with pixel grayscale value ranging from 0 to 255. Subsequently, the CMS algorithm is used to select an appropriate column of grayscale values for signal demodulation as shown in Fig. 3(a). The fixed CMS scheme is usually adopted for stationary OCC system, while it is not appropriate for mobile OCC system due to the varying of the receiver position in mobile OCC system. Thus, in the paper, we propose the dynamic

Vol. 26, No. 17 | 20 Aug 2018 | OPTICS EXPRESS 21765

CMS algorithm for mobile OCC system. Then, a low pass filter (LPF) is utilized to smooth the noise as shown in Fig. 3(b). And to enhance the ER, the histogram equalization is performed as shown in Fig. 3(c). After finding the header of data packet, the transmitted signal can be recovered according to the 4-order polynomial thresholding scheme as the red line illustrated in Fig. 3(d). Finally, the system performance is evaluated in terms of the BER measurement.

Fig. 3. System process diagram of OCC system using mobile phone camera. Grayscale values (a) after column selection, (b) after LPF, (c) after histogram equalization, and (d) using thresholding scheme. START Pixel values of image p(i,j)

Dynamic CMS for Mobile OCC (b) ROI (c)

ROI

(a)

Obtain the row vector c(j) containing the maximum element from each column

ROI

(e)

(d)

Find the maximum value M and the index If of the first one found

If M>240

(c) No

Yes If If >1920/2 Yes

No

(a)

Joptimal =If – Constant Offset

Find the index Il of the last maximum value found (b) Joptimal =Il+ Constant Offset

Joptimal =If

END

Fig. 4. Flowchart of dynamic CMS for mobile OCC system. (a), (b) and (c) are the image frames, respectively corresponding to different cases on the flowchart. The column matrix of grayscale values (d) using dynamic CMS algorithm, and (e) using fixed CMS scheme.

In order to select an appropriate column of grayscale values with high ER while avoiding the blooming effect for mobile OCC, for the first time, the dynamic CMS algorithm is proposed and the flowchart of dynamic CMS algorithm is illustrated in Fig. 4. Firstly, let the p(i, j) denote the pixel grayscale value in the i (< = 1080) row and j (< = 1920) column of the grayscale image. Then, obtain the row vector c(j) containing the maximum element from each column. Subsequently, record the maximum pixel value M from c(j) and the index If of the first one found. On the one hand, when M is larger than 240 which is considered as the

Vol. 26, No. 17 | 20 Aug 2018 | OPTICS EXPRESS 21766

boundary grayscale value of the blooming region in our experiment. There are multiple maximum pixel values in the different column in the blooming region. For instance, as shown in Fig. 4(a), the pixel grayscale value in the blooming region (purple circle) always has an intensity of 255. When the If is larger than the half of the total column number, i.e., (1920/2), it means that the region of interest (ROI) is mainly located in the right region of the image. To avoid the blooming effect, the column matrix for signal demodulation, as the yellow dashed vertical line shown in the Fig. 4(a), is selected a little away from the column of blooming boundary. Hence, a constant offset is introduced and manually set to 180 to mitigate the blooming effect while minimizing the fluctuation of grayscale values, leading to better thresholding decision. In this case, the optimal column can be written as Joptimal = If - 180. On the contrary, when the If is smaller than the half of the total column number, as illustrated in the Fig. 4(b), the ROI is mainly located in the left region of the image. In this case, if we still set the optimal column like Joptimal = If - 180, the optimal column will probably be beyond the left boundary of the image. To avoid this situation, the index Il of the maximum pixel value M the last one found is used when If is smaller than the half of the total column number. And the optimal column is set to Joptimal = Il + 180. On the other hand, when the maximum grayscale value M is smaller than 240, there is no blooming region existed in the ROI as shown in Fig. 4(c). Hence, the optimal column is set to Joptimal = If. The yellow dashed vertical lines shown in Figs. 4(a)-4(c) are used to select the appropriate column with the help of dynamic CMS algorithm. Meanwhile, the fixed CMS as red dashed vertical lines shown in Figs. 4(a)-4(c) is used for comparison. Figures 4(d) and 4(e) are the column matrix of grayscale values employing the dynamic CMS algorithm and the fixed CMS scheme with respect to the Fig. 4(b), respectively. It is evident that the selected column matrix of grayscale values using the dynamic CMS algorithm is better for thresholding decision than that of using the fixed CMS scheme for mobile OCC system. 3. Results and discussion

Fig. 5. The BER performance and the illuminance versus the offset to center of the track.

The BER performance and the illuminance for different offsets of the receiver in stationary OCC system are firstly analyzed. The mobile phone camera is fixed at some positions with different offsets to center of the track. Figure 5 illustrates the BER performance and illuminance with respect to different offsets to center of the track. It can be seen that the maximum illuminance at the center position can be achieved at 275 lx. The illuminance distribution is symmetrical to the center of the track. The illuminances at the offsets of 30 cm and −30 cm are 154 lx and 147 lx, respectively. It can be observed that the BER performance is proportional to the illuminance. It can achieve superior BER performance with the increase of illuminance due to the enhancement of the received power. Hence, the minimum BER of 4.7 × 10−4 can be achieved at the center position. At the offsets of 30 cm and −30 cm, the BERs are 7.4 × 10−3and 7.9 × 10−3, respectively. It is worth noting that dynamic CMS algorithm is adopted in the case of stationary OCC with mobile phone camera fixed at some

Vol. 26, No. 17 | 20 Aug 2018 | OPTICS EXPRESS 21767

positions. It indicates that the proposed dynamic CMS algorithm is capable of supporting the stationary OCC system.

Fig. 6. The BER versus the moving speed at the lateral distance of (a) 50 cm and (b) 70 cm.

Furthermore, the impact of moving speed and lateral distance on the BER performance is investigated for the proposed mobile OCC system. The BER performance versus the moving speed at the lateral distance of 50 cm and 70 cm are shown in Figs. 6(a) and 6(b), respectively. It can be seen that the BER performance gradually gets deteriorated with the increasing of moving speed. It is because that the increasing of moving speed will introduce more ISI and noise to the received signal. Hence, to avoid the appearance of sharp noise and smooth the data, a LPF with filter coefficient h = [1/2,1/2] is adopted for the selected column matrix. Under the moving speed of 80 cm/s, as shown in Fig. 6(a), the BER can be improved from 6 × 10−3 to 3.29 × 10−3 by using the smoothing process for the case of lateral distance of 50 cm, and as shown in Fig. 6(b), the BER can be improved from 1.1 × 10−2 to 7.5 × 10−3 by using the smoothing process for the case of lateral distance of 70 cm. In addition, when the lateral distance is 70 cm and the moving speed is 100 cm/s, the proposed system can still achieve a BER of 7.4 × 10−3 which is below the 20% FEC limit. It can be found that the overall performance with the lateral distance of 50 cm achieves better BER performance than that of 70 cm. It is because that the increase of lateral distance leads to the descending of average illuminance, resulting in the lower average signal noise ratio (SNR). In addition, the proposed mobile OCC using mobile phone can achieve a data rate of 4.08 kbps (68 bits/frame*60 frames/s) under an illuminance of 275 lx. The lateral distance can be further enhanced when wider coverage area is provided by using multiple LEDs. Besides, the BER performance can be further improved by increasing the illuminance. The experimental results demonstrate that the OCC using mobile phone is capable of supporting user mobility at different moving speed and lateral distance. 4. Conclusion In this paper, mobile OCC system using mobile phone is firstly proposed and experimentally demonstrated and its experimental characterization of user mobility by varying moving speed and lateral distance is investigated. The dynamic CMS algorithm is proposed for the mobile OCC system to select an appropriate column matrix while avoiding the blooming effect. The smoothing process is utilized to improve the BER performance. The experimental results show that the BER performance is proportional to the moving speed and lateral distance and it will get degraded with the increase of moving speed and the increase of lateral distance. Furthermore, the proposed mobile OCC system using mobile phone can achieve a data rate of 4.08 kbps under the low illuminance of 275 lx. Funding National Natural Science Foundation of China (Grant 61775054, 61377079); the Science and Technology Project of Hunan Province (2016GK2011); Natural Science Foundation of Hunan Province of China (Grant No. 2017JJ2047).