Google Glass in Medical Environment Mohit Kumar Sinha Asia Pacific Institute of Information Technology, Faridpur Road, G.T. Road Karnal Road Side, Panipat-132103
[email protected] Abstract— Wearable computing devices are starting to change
an Android device, Google Glass are able to connect to the
the way users interact with computers and the Internet. Among
Internet through Wi–Fi or Bluetooth and they are capable to
them, Google Glass includes a small screen located in front of the
understand spoken commands and read text aloud. At the
right eye, a camera filming in front of the user and a small
moment, only 8000 pairs of Google glass are available
computing unit.
worldwide through the Google Explorer Program.
In this paper, we developed a Google Glass application able to take a photo and send it to a medical image retrieval system along with keywords in order to retrieve similar cases. As a preliminary assessment of the usability of the application, we tested the application under three conditions (images of the skin; printed CT scans and MRI images; and CT and MRI images acquired directly from an LCD screen) to explore whether using Google Glass affects the accuracy of the results returned by the medical image retrieval system. The preliminary results show that despite minor problems due to the relative stability of the
Fig 1: The Google Glass and their components.
Google Glass, images can be sent to and processed by the medical
As a new wearable device, Google Glass uncovers many new
image retrieval system and similar images are returned to the
applications, also in the medical domain, that can ease the
user, potentially helping in the decision making process.
work of medical staff from paramedics, nurses to surgeons.
I. INTRODUCTION With the increasing availability of smartphones, users are
Research papers involving Google Glass for medical applications already exist in the literature. Researchers used
getting accustomed to access information from anywhere.
Google glass during 4 weeks in a hospital. They focused on a
Now, wearable devices are getting on the market allowing
limited usages of the built-in capabilities of Google Glass.
users to access information in an even more interactive and
They tested voice recognition as well as latency, lag time and
less disruptive way. For example, smart watches show
visual quality of local and transatlantic videoconferencing.
notifications or messages directly on their screens, letting
Their findings demonstrate that Google Glass could be useful
users get information without looking at their phones on any
in various medical tasks. However, the current version of
additional device. As another wearable device, Google
Google Glass lacks battery life and Internet connection
introduced Google Glass able to interact with different online
stability. Similarly, other researchers used Google Glass for
services. As illustrated in Fig. 1, Google Glass features a
documenting medical findings during forensic examinations.
camera taking photos at 5 Mega Pixels and recording video
They created their own small application for Google Glass to
with 720p, a prism in front of the right eye, a touchpad on the
take pictures of body parts hands– free. Using these pictures,
right side of the frame as well as speakers and microphone. As
they compared the quality of the results to pictures acquired
by the state–of–the–art procedure using a DSLR camera.
using Google Glass influences the results returned by the
Their results show multiple interesting findings. First, the
CBIR and can be used without problems.
Google Glass procedure was significantly slower than the state-of-art procedure. Second, pictures taken by Google Glass were not good enough for documenting medical findings compared to the picture taken by a DLSR camera. Still, some of these points can simply be avoided by users having experience with the device and applications being really adapted to the user groups. Despite a few limitations of Google Glass, we believe that these augmented glasses have the potential to facilitate handling of patients in different stages of treatment. For instance, connected to a content–based image retrieval (CBIR) system, such devices can allow medical staff to search for information or to go through medical history while discussing with a patient. CBIR systems in the medical field give access to large amounts of medical images of cases for later use such as for teaching, research or in diagnosis.
II. LITERATURE SURVEY Google glass (June, 2012) is one of the latest innovations based on augmented reality and ubiquitous computing. It is a wearable technology which is being projected as the affordable future technology for healthcare applications. Though the device has just hit the market (February, 2014), it carries the potential for disruptive innovation. The concept behind this glass technology is to integrate many types of information and defining a new path to this information to help healthcare personnel deliver an organized and more effective patient care. Making use of the Eye tap technology in Fig. 2, it can record the scene available in the field of vision and superimposes computer generated imaginary on the scene available to eye for facilitating surgeries. This technology carries applications in many departments of the hospitals. The study is based on the data drawn from secondary sources like Internet, newspapers, magazines and research papers.
Fig 2: Mechanism of CBIR. Research has shown that CBIR can facilitate diagnosis for radiologists, especially if they are less experienced. Despite a few limitations, Google Glass could allow medical staff to conduct a diagnosis in a non–disruptive way. A medical doctor can talk to a patient while retrieving data from a CBIR system. Google Glass has raised some concerns about patient
Fig 3: Eye Tap Technology.
confidentiality and privacy. In the current case, both patient’s
The Google Glass technology applications can bring
consent and protection of patient’s data on Internet are
significant
achievable. To the best of our knowledge, there are currently
accelerating social welfare especially for poor and deprived
no publications on the combination of Google Glass with a
segment of society through public healthcare system. The
CBIR system. Therefore, this paper presents a novel interface
health sector witnessed the use of Google glass technology for
to CBIR allowing medical staff to search for images,
the first time in June 2013, when a Spanish surgeon Guilin, Dr.
potentially aiding the decision making. As preliminary
Pedro wore Google Glass while performing a chondrocyte
assessment, we conducted three tests to examine whether
transplant (a surgery conducted on the knee to repair cartilage
changes
in
healthcare
delivery,
surgeries
injuries) and live streaming it on the internet in Fig. 3.
decision making. The traditional live surgery requires a
Kaeding (August 2013) used Google Glass to consult with a
cameraman to video graph from different angles and thus
colleague in a distant part of Columbus. In India, first time Dr.
increase the chances of infection very remote. It can be an
JS Rajkumar in September 2013 used Google Glass
important teaching tool by displaying the video footage and
technology to perform surgeries of a 43 year old patient for
converting it into an education, instructional video for the
severe heart burn and 46 year old woman for hernia wearing a
students. In another vital application, the glasses can well
pair of Google Glass and live streamed it. In India 83% of
serve in disaster management as a tool with the information
operations occur in rural area and small towns. A surgeon
that can be shared, video and photos can be accessed and
with the help of Google Glass can remain in touch with other
navigated for the dispatch purposes.
surgeons to see the critical moments during the procedure and
This technology can indeed change our leisure activities to an
prevent distractions during surgery. Parekh (2014) conducted
exciting time as it facilitates the communication, people can
the foot and ankle surgery during Indo US conference using
use the internet facility and with the help of application like
Google Glass in Jaipur, which was broadcasted live on Google
Google map, Google plus etc., people can be brought closer. It
website. The technology is useful in education as it facilitates
can bring a radical change in the education system also.
communication with a patient’s family, can teach students and
Students can record their lectures; capture the necessary notes
communicate with other doctors during surgery.
in real time for future reference. And the Glass will allow student/teacher
to
stay
connected
to
an
interactive
environment and thus the distant learning also becomes easier. The Google Glass can be used in E-Commerce. The tool helps finding the best suited products as one can easily compare the alternatives, features, and price and hence can select the most suited. While discussing another important application of Google Glass, a health personnel gets interactive environment that provides real time analysis of patient health and hence its Fig 4: Dr. Pedro performing a chondrocyte transplant.
application widens from primary care in OPD to a specialized
Google Glass works by making use of the Eye tap, bone
care in Operation theatre and ICU. The surgeon can command
conduction & Smart clothing technology. It records the scene
and click pictures and captures videos while using a hands
available in the field of vision and superimposes computer
free device, it has improved the patient consultant relationship
generated imaginary on the scene available to eye by eye tap
to a certain extent.
technology. The bone conduction technology helps the users to access the web on glass using specific transmission of voice commands. The glass make use of the smart clothing technology in which the clothing is installed with digital devices which are worn by the bearer. Physicians can use glass to receive the incoming notifications regarding the patient’s history, medical reports simultaneously interacting with the patients and thus save time and cost. This empowers the specialists in quick and right
Fig 5: Doctors performing operation live on the internet.
There are various indications in different department of
The usage of mobile devices such as smart phones and
hospital where the discussed technology can be applied. The
wearable devices has increased dramatically the last decade.
Google Glass has the potential to transfer the health care industry from the point of view of physician as well as patients. In outpatient department, it can improve the direct doctor patient communication as the doctor would not need to take his eyes off from the patient while seeing his record. As soon as the patient would make an entry to the chamber, a doctor can navigate the patient’s information by swiping or tapping the device’s frame. In an emergency case with the help of live streaming the occupancy status of the department can be identified. The latest innovation by Google and its attempt on product differentiation with styled frames and prescription lens can bring about instrumental change in the healthcare industry. With increase demand for improvised standards in healthcare, this innovation might well serve the demand and raise the bar. Field like telemedicine, tele mentoring, education and research could see new highs in delivering the excellence. Conclusively, Glass is an affordable technology which carries immense potentiality of applications in revolutionizing healthcare delivery. But there is a need of research in this area as it is still in its infancy. This Glass technology may be the next big thing for healthcare business world in coming decades.
Fig 6: Health Condition of a Patient in Google Glass. As a consequence of this, researcher have a lower threshold to translate the analogue world to a digital environment. Nowadays personnel have to keep track of every patient by printing out their medical record and manual revising the required information. In this paper we present a proof-ofconcept system that is developed for physicians and nurses to automatically perform a revision of patient data through a natural user interface through Google Glass. Also real-time check-ups of the patient parameters are possible. To do all this there are a couple of important factors in designing a tour guide system for wearable devices, namely hardware capabilities and communication. Recent developments in hardware allow for significant processing power in these devices, in our case, Google Glass incorporates a dual core 1GHz Processor with 682 MB of RAM and a 5MP camera.
III. OWN CONTRIBUTION In the development of our system, we considered using different technologies towards achieving our goals. By comparing all of these technologies we have opted for the most accurate, yet practical solution to implement into our system. Technologies such as proximity sensors to do patient identification, camera based heart rate detection to create a contactless sensor and different visualization techniques to give the user the natural user interface provide by the Google Glass. On this natural interface all of the previous data will be shown. This paper discuss the possibilities of usages of a Google Glass in a medical environment, by taking everyday task of personnel in a hospital to a new dimension.
Patient Localization To automate the process of determining the patient’s location and give the user the corresponding medical information of that particular patient, a proximity–based system is used. By this system the location of the patient and doctor can be crossreferenced so that the proximity between two persons can be determined. Continuing on this principle the user will get a notification and will be able to review the patient’s medical parameters. By combining the camera of the Google Glass with the Augmented Reality capabilities of the wikitude SDK, the user will be able to see all the patient in a specific radius from his point of view. Having this option on the Glass,
doctors as well as nurses can keep a real-time overview of
the data repository. This all because of the speech-to-text
each and every patient in an area of interest.
features provided on the Glass. IV. RESULT AND CONCLUSION Table I and Table II display the results for the second and third test, respectively. Under these two tests, the effect of taking a photo with Google Glass can be observed in comparison to using a digital version of the original image sent directly to CBIR. In both tests, sending images with no keyword yielded consistently lower results, whereas, using keywords significantly increased the accuracy of the results in
Fig 7: Augmented Reality (AR) in Google Glass.
all conditions, as shown in Table I and Table II. Taking a
Visualization approaches
photo of a printed image did not affect the number of accurate
To visualize the specific medical data of a patient a number of
results returned by CBIR compared to the original photo
different techniques are used by using a different approach for
condition. However, taking a photo of an image on an LCD
each dataset the maximum amount of data can be displayed on
screen produced a smaller number of accurate results
the most efficient way. This is necessary because of the
compared to the two other conditions.
limited amount of resolution of the display. For example when a large set of data has to be presented to the user, the data is converted to a list where it is possible for the user to scroll through the list by tilting his head.
Fig. 5: An example of a retrieved image and caption in the Google Glass.
Fig 8: Virtual Approach by Doctors. Data Repository Link Not only the visualization of the data is important but also a possibility of uploading new data to the data repository needs to be implemented. When a doctor checks up on a patient his spoken assessments are converted to text and then uploaded to
Conclusion: Glass was efficient for acquiring images for documentation in forensic medicine, but the image quality was inferior compared to a DSLR camera. Images taken with
4. Henning M¨uller, Nicolas Michoux, David Bandon, and Antoine Geissbuhler, “A review of content–based image retrieval systems in medicine–clinical benefits and future directions,” International Journal of Medical Informatics, vol. 73, no. 1, pp. 1–23, Accessed on 1st Oct 2015.
Glass received significantly lower ratings for all 4 categories
5. Ceyhun Akg¨ul, Daniel Rubin, Sandy Napel, Christopher Beaulieu, Hayit
in an autopsy setting and for region of interest and brightness
Greenspan, and Burak Acar, “Content–based image retrieval in radiology:
in postmortem examination. The effort necessary for
Current status and future directions,” Journal of Digital Imaging, vol. 24, no. 2, pp. 208–222, Accessed on 5th Oct 2015.
achieving the objectives was higher when using the device
6. Alex M. Aisen, Lynn S. Broderick, Helen Winer-Muram, Carla E. Brodley,
compared to the DSLR camera thus extending the postmortem
Avinash C. Kak, Christina Pavlopoulou, Jennifer Dy, Chi- Ren Shyu, and
examination duration. Its relative high power consumption
Alan Marchiori, “Automated storage and retrieval of thin–section CT images
and low battery capacity is also a disadvantage. At the current
to assist diagnosis: System description and preliminary assessment,” Radiology, vol. 228, no. 1, pp. 265–270, Accessed on 4th Oct 2015.
stage of development, Glass may be an adequate tool for
7. Alba Garc´ıa Seco de Herrera, Jayashree Kalpathy-Cramer, Dina Demner
education. For deployment in clinical care, issues such as
Fushman, Sameer Antani, and Henning M¨uller, “Overview of the
hygiene, data protection, and privacy need to be addressed and
ImageCLEF 2013 medical tasks,” in Working Notes of CLEF 2013 (Cross Language Evaluation Forum), September 2013. Accessed on 5th Oct 2015.
are currently limiting chances for professional use.
8. Alba Garc´ıa Seco de Herrera, Dimitrios Markonis, Ivan Eggel, and Henning M¨uller, “The medGIFT group in ImageCLEFmed 2012,” in
REFERENCES 1. Oliver J. Muensterer, Martin Lacher, Christoph Zoeller, Matthew
Working Notes of CLEF 2012, Accessed on 4th Oct 2015.
Bronstein, and Joachim Kbler, “Google glass in pediatric surgery: An
9. Henning M¨uller, Alba Garc´ıa Seco de Herrera, Jayashree Kalpathy-
exploratory study,” International Journal of Surgery, vol. 12, no. 4, pp. 281 –
Cramer, Dina Demner Fushman, Sameer Antani, and Ivan Eggel, “Overview
289, Accessed on 2nd Oct 2015.
of the ImageCLEF 2012 medical image retrieval and classification tasks,” in
2. Urs-Vito Albrecht, Ute von Jan, Joachim Kuebler, Christoph Zoeller,
Working Notes of CLEF 2012 (Cross Language Evaluation Forum), Accessed
Martin Lacher, J. Oliver Muensterer, Max Ettinger, Michael Klintschar, and
on 5th Oct 2015.
Lars Hagemeier, “Google glass for documentation of medical findings:
10. Dimitrios Markonis, Alba Garc´ıa Seco de Herrera, Ivan Eggel, and
Evaluation in forensic medicine,” J Med Internet Res, vol. 16, no. 2, pp. e53,
Henning M¨uller, “The medGIFT group in ImageCLEFmed 2011,” in
Accessed on 5th Oct 2015.
Working Notes of CLEF 2011, Accessed on 3rd Oct 2015.
3. L. Rodney Long, Sameer Antani, Thomas M. Deserno, and George R. Thoma,
“Content-based
image
retrieval
in
medicine:
Retrospective
assessment, state of the art, and future directions,” International Journal of Healthcare Information Systems and Informatics, vol. 4, no. 1, pp. 1 16, Accessed on 6th Oct 2015.
