2011 Sixth IEEE International Symposium on Electronic Design, Test and Application
Design, Implementation and Testing of Mobile Phone Application for Pleasant Wake Up
Ondrej Krejcar
Jakub Jirka
Dept. of Measurement and Control, Center for Applied Cybernetics, Faculty of Electrical Engineering and Computer Science, VSB Technical University of Ostrava Ostrava-Poruba, Czech Republic
[email protected]
Dept. of Measurement and Control Technical University of Ostrava, Faculty of Electrical Engineering and Computer Science Ostrava-Poruba, Czech Republic
[email protected]
This substance is produced in our brain and some scientists believe that it is also cause of a metabolism slowdown before falling asleep. Melatonin secretion leads to the body temperature reduction, blood flow towards brain limitation and muscles slackness.
Abstract — Human life is divided into day periods as a main periods which consist of a day and night phase. Most of humans need to sleep a eight hours in a minimum to have their body and all the body function ready in day phase of a day. The second very important thing (instead of 8 hour sleep) is a phase of sleep in which a human is wake up. Waking up of a human in deep sleep phase lead to a very “unhappy” morning and not so nice spend of day. We are trying to develop algorithms to detect mild sleep stages which are the most suitable to human wake up. Actually we use microphone input to record and process sound near the monitored user. All the knowledge is implemented to mobile phone (windows mobile based device) as an intelligent alarm application. Whole application is designed as multithread Real-Time application based on .NET Compact Framework platform in C# language. Filtration and detection algorithms used special Math library Math.NET to process a recorded sound signal. This paper consists of several chapters which describes design, implementation and testing phases of this mobile application development. Actual testing results shows that our developed application detect correctly more than 70 % of mild sleep stages of all tested persons.
A. Sleep stages Approximately two hours after we fall asleep our eyes starts to move back and forth irregularly. Based on this fact scientists divided sleep stages into two main stages REM sleep with (Rapid Eye Movement) and NREM sleep stage (Non Rapid Eye Movement). NREM sleep is divided into another four sub-stages, when with increasing number the sleep is more and more deeper. During healthy night time sleep REM and NREAM stages changes a few times. Most of the dreams are happening in REM stage. Body muscles are completely loosened and thanks’ to this fact one is awaken refreshed. During deep (NREM 3 and 4) sleep stages blood pressure is decreasing which lowers chance of cardiovascular danger. Also growth hormone is produced in its maximum in adolescent age.
Keywords: Wake Up, Windows Mobile, User Adaptive System, FFT Analysis
I.
Sleep stages: a) Wake (Awake) b) REM – we dream in this stage c) NREM1 – falling asleep d) NREM2 – light sleep e) NREM3 – deep sleep f) NREM4 – deepest sleep
INTRODUCTION
Sleep is a complex process regulated with our brain and as such is driven by 24 hour biological rhythm. With ages coming our sleeping habits change rapidly. A new-born baby is sleeping in short periods for 18 hours a day. According to studies 3 hours of sleep is enough for some people but some needs 8 or even 10 hours of sleep to feel rested. Recent research shows that differences in biological rhythm of people can explain why some teenagers have such difficulty of waking up. It seems that during puberty this rhythm is being shifted and adults tend to go to bed much later and wake up much later as well. This sleep shift is common and during adolescence it vanishes. Our biological clocks are controlled by chemical substances that are mostly known to us. One of them is hormone called melatonin which is suspected to make us feel sleepy. 978-0-7695-4306-2/11 $26.00 © 2011 IEEE DOI 10.1109/DELTA.2011.51
To make description and overview of processes in human body during individual sleep stages complete, table with description and specification follows (Table 1). This table shows relations between biological manifestations in different sleep stages. This table and it´s information are taken into account in wakeNsmile detection algorithm.
248 242
II.
TABLE E I. BIOLOGICAL M MANIFESTATIONS IN SLEEP STAGES Physiologi cal Process
During NRE EM deecreases w wakefulness
heart rate
sllows from wakefuulness
blood pressure
deecreases from w wakefulness do oes not changee from w wakefulness in most reegions
blood flow to brain respiration airway resistance body temperatur e muscle tension sexual arousal
deecreases w wakefulness
from
from
inncreases from w wakefulness iss regulated at low wer set po oint than wakeffulness; shhivering initiatted at loower temperaturre than du uring wakefulnesss deecreasing with inncreasing stagge of N NREM occcurs infrequentlyy
The final gooal of this projject is a develloping of winddows mobile applicaation, which will implement algorithm m ms to deetect sleep stagges. Currentlyy there are som me similar proojects avvailable to download and rrun onto vario ous mobile deevice platforms from Nokia, Applee iPhone, Win ndows Mobilee etc.. Hoowever nobody providee info whaat algorithmss is im mplemented annd how. Whaat level of succcessfully deteected sleeep stages annd finally w waked up useers is possiblle to acchieve. N and iP Phone First projectt from [1] is available to Nokia m mobile platform ms. HappyWakkeUp does no ot try to wakee you upp when you arre in deep and calm sleep off if you are in REM R sleeep (sleep phhase with dreaaming). Thesee are the mom ments whhen it is most difficult to waake up [1]. d on principlle of HappyWakeeUp applicatiion is based acctively monitoors users sleepp using the microphone m o the of m mobile phone. It I is trying to ddetect user mo ovements in a bed. Thhis is the reasoon, why user’’s mobile deviice (mobile phhone) m must be placeed near to user. u Applicaation then makes m staatistical analyysis of the quaality and cyclees of user sleepp. As auuthors describbe, applicationn is developeed to detect only staatistically siggnificant movvements as arousals, beccause duuring these moments m userss are actuallyy awake or allmost aw wake [1]. Authhors howeverr don’t publish h any other deetails likke algorithms or techniquess, so it is impossible to evaaluate thheir software according a thesee principles. It is also possible p to fi find other sim milar projects like HaappyWakeUp. For examplee at Macjek Drrejak Labs devvelop a Sleep Cycle mobile appliication. Their application Sleep S h a #1 paid d app Cyycle has beccome a huge success with poosition in manny countries, including Germany, Japan n and Ruussia [2]. Innstead of H HappyWakeUpp a Sleep Cycle C appplication mo onitor users m move by usiing of embeedded acccelerometers which are aactually equip pped with moodern m mobile devicess (like iPhonees or some HTC H types). User m make during diifferent sleep pphases differeent moving inn bed, whhich is possibble to determiine by Sleep Cycle applicaation. Reemainder phaases of signaal processing stay withoutt any chhanges. p of this paaper deal with h an algorithmss and Following parts thheir implementtation to real aapplication to detect sleep stages byy using of miccrophone inpuut. Using of accelerometers a s will bee investigated in near future.
During REM
brain activity
RELLATED WORK
increaases in motor and sensorry areas, while other areas are a similar to NR REM increaases and v varies compaared with NREM M increaases (up to 30 perrcent) and vaaries from NREM M increaases by 50 to 200 percennt from NR REM, depennding on brain reggion increaases and varies from NREM M, but may show w brief stoppaages (appnea); coughhing suppressed increaases and varies from wakeffulness is not n regulated; no shiverring or sweaating; tempeerature drifts tooward that of the local enviroonment increaases from NREM increaases from NREM M (in both males m and females)
B. Hypnograam Hypnograam is a graph, which visualiizes sleep stagges in a time. In real itt is a series off individual staages (Figure 1).
III.
YSIS DETECTTING OF SLEEPP STAGES BY SIGNAL ANALY
Sleep stage analysis algoorithm is based d upon knowlledge abbout specific sleep s stages, thhat are summarized in (Tabble 1) annd empirical teesting on two subjects. Recording itself is trigggered 20 minnutes (by deffault) beefore the wake up time set by the user before b sleep. If an errratic movem ment (or shiveering) is dettected within this innterval user is awaken a in thee time of erratiic moves. This time represent r REM M stage of sleep, when wee are alm most awake. If no moveement is dettected during this 20 minnutes wiindow user is awaken on seet up time.
Fiigure 1. Typicall healthy person hypnogram h
Typical hypnogram h (F Figure 1) (dduration - 8 hours, obvious perioodical changes of deep sleep stages andd REM sleep stages). mation provvided in prrevious Accordingg to inform paragraphs opptimal time to t wake up are a periods of o mild sleep. The moost optimal period of time is of course after 8 hours of sleep p.
243 249
Size of a wake up window can be adjusted by the coder by means of WAKE_WINDOW_LENGTH constant, found in Analyzer class. Regularly increased levels of sound amplitude on the input ale filtered out because they may represent biological events like breathing, with higher respiration resistance or snoring by that means. On next image we may observe breathing with higher respiratory resistance (snoring). Another interesting information gained from detail analysis of a signal is the presence of dominant frequencies in amplitude - frequency domain signal spectrum. Analysis was performed with Fast Fourier Analysis method. More specifically with FFT (Fast Fourier Transformation) algorithm. Signal auto-correlation shows then certain signal peaks, which can be classified and taken into account with final decisions whether to wake or not to wake. Algorithm itself is based on time-series analysis. It is based on sound wave differences (period, amplitude), that are abnormal and above isoline. Isoline is computed by linear moving average algorithm. Waves are likely to be more similar within biological manifestations of sleep (breath, snoring). Waves are much more similar in means of period and amplitude. These waves as biological manifestation during sleep are taken as false deviations from isoline as they most likely represent deeper NREM stage of sleep and thus are not positive to wake user up and are taken in final statistical analysis as negative sleep anomaly. On the other hand if erratic waves (different in period and amplitude) are registered at least DETECT_ANOMALIES_COUNT they are taken into final statistical analysis as positive sleep anomaly. These erratic waves in final analysis are most likely to represent user movement in a bed which is manifestation of brain-motor interconnection that occurs during REM stage of sleep, which is very positive to wake user up. Algorithm described in previous paragraphs is adjustable for coder in many ways. Every constant that can adjust sensitivity of algorithm in any aspect described is shown below. These constants are saved in Analyzer.cs file.
Figure 2. UML activity diagram of sleep stages detection algorithm.
IV.
IMPLEMENTATION OF ALGORITHMS TO MOBILE PHONE APPLICATION
wakeNsmile application (Figure 3) is written in C# programming language and uses .NET Compact Framework version 3.5 [4], which is a special derivative of .NET Framework for mobile devices. Application was developed in Visual Studio 2008 Team Edition on Windows Mobile 6.5 emulator and tested on a Hewlett Packard mobile device with Windows Mobile 6.5 operating system. Minimal requirements are mobile device with Windows Mobile 6 and higher and .NET Compact Framework 3.5 and higher. wakeNsmile application uses user control called Alarm, that has been created as a part of this project. Application is using Math.NET neodym library [4] for FIR filter design and WaveIn and WaveOut libraries [6] for mobile device sound interface communication.
244 250
F Figure 3. wakeN Nsmile application n example
As has alrready been saiid application is using user control alarm created d as a part off this project.. This component is reusable and can be added d to a newly created projeect and adjusted by neeeds of a proggrammer. User conttrol is using nnumber of oth her classes th hat take care of signal recording andd analysis. • Recorder class • Analyyzer class • Wavee, WaveIn, W WnsWaveIn, WaveOut W classs • Playeer class F Figure Nsmile class diag gram 4. wakeN
A. Application Multithreadinng Recorder, Analyzer A and d Alarm (useer control) classes arre all running in i their own thhreads so that signal recordiing is no ot influenced by b signal anaalysis and userr is always ab ble to in nteract with program p evenn during sign nal recording and an nalysis. Recorder class c is using WnsWaveIn W class c and savees the so ound recorded from the deviice microphon ne input. Recorder class is using ssingleton desig gn pattern beccause off the fact that we w can recordd from only on ne device inpu ut in a tim me. Recorder class c is runninng in its own n thread. Thu us the grraphical user interface Alaarm (user con ntrol) is ready y and av vailable to reeceive commaand at any time t even du uring recording and analysis a proceess and Analyzzer class is ab ble to nalyze data saample at anoother sample recording without an recording interrruption.
245 251
this._wakeUpTime = this._alarmUI.wakeUpTime; this._processData = true; this._analyzeThread = new Thread(new ThreadStart(analyzeWorker)); this._analyzeThread.Priority = ThreadPriority.AboveNormal; this._analyzeThread.Start(); }
.NET compact framework does not offer any classes or interfaces to communicate with audio interface on mobile device. Thus WaveIn and WaveOut classes are used [6]. These classes creates interface with coredll.dll library which is a in/out device provider on mobile device. (An alternative to windowsNT and higher kernel32.dll) B. Recorder class Recorder class contains important constant MAX_REC_TIME, which defines time range of record before is processed and analyzed. By default it is set to 50ms. That means that 50ms worth data samples are recorded and then handed over to Analyzer class that starts signal analysis in its own thread immediately. This constant can be changed by programmer if needed. The shortness program code of recorder class is like following:
private void analyzeWorker() { while (this._processData) { Recorder.processData.WaitOne(); //Wait until recorder thread signals you this.ProcessData(); } }
public class Recorder { public void Start() { this._analyzer.alarmUI = this._alarmUI; this._recThread = new Thread(new ThreadStart(RecWorker)); this._recThread.IsBackground = true; this._recThread.Priority = ThreadPriority.Highest; this._analyzer.Start(); this._recThread.Start(); this._recThreadRunning = true; }
D. Player class Last but not least Player class as wrapper around WaveOut class that plays alarm sound in a loop until stopped by user. Instance of this singleton patter designed class is running in its own thread as well. V.
wakeNsmile application was tested for functionality and for sleep stage detection level. We create a testing methodology to achieve the most accurate and reliable results as possible. Every test person was requested to fill out a questionnaire before the test was performed. Such questionnaire collects several simple questions to detect a category to which a test person belongs to. This is a very important phase of testing methodology because it helps us to disqualify persons with pathological hypnogram (these persons are impossible to wake up before the adjusted alarm time due to the impossible detection of theirs sleeps stages). After first questionnaire phase of testing methodology a real testing was started only with collection of healthy hypnogram. In this collection a sufficient number of tests (more than 10 of successfully finished tests for every test person of collection) were executed to get a valuable collection of results. The achieved level of successfully detected sleep stages resulted in value of 71 % of all tests (without people separation) and in 73 % as median value for every person.
C. Analyzer class Analyzer class is class intended for recorded signal with MAX_REC_TIME analysis. Instance of this class is running in its own thread. But this thread is started immediately after the recording has started and it is not waiting to be created by Recorder class instance after MAX_REC_TIME interval. This is arranged like so because of unwanted delay that we want to get rid of. Analyzer thread is in waiting state until is unblocked by the Recorder thread after samples of data are recorded and buffered. The shortness program code of recorder class is like following: class { const const 2000;
TESTING OF DEVELOPED SOLUTION
Analyzer int SAMPLE_RATE = 22050; int FIR_FILTER_LOWPASS_CUTOFF = // In Hz
A. Founded problems during testing period Only problem which were detected during our initial test period was the disturbance during the tests which resulted sometimes in wrong sleep stage detection. This problem grows in a way how the signal is measured by microphone. To solve this problem we select only these test persons which are sleeps alone in a room. Of course this is a huge limitation for real appointment, but we are hard working on pass over this limitation.
public void Start() { this._firFilter = new OnlineFirFilter(FirCoefficients.LowPass( SAMPLE_RATE, FIR_FILTER_LOWPASS_CUTOFF, 80));
246 252
VI.
PROACTIVE USER ADAPTIVIVITY
[3]
wakeNsmile application is developed to react on users declared request in form of happy wake up at predefined time. The time defined for alarm is however the latest possible time to wake up of user. We are trying to detect a body state in which the user is most able to wake up with a smile. Time period for detection analysis of state phases is declared to 30 minutes. Created application is an example of user adaptive solution for mobile devices [8]. Currently a single application is developed, but a distributed architecture version with a neural network analysis and people database is planned for future steps to be a completely embedded solution at Mobile UAS Framework which grows from several previous project conjunctions [13], [14], [15], [16], [17]. Actual tests provide very promising results with more than 70 % of successful happy wake up of test persons at morning. Developed application act as a proactive solution is sense of wake up of users in most suitable time. User adaptivity can be however sustained by user’s inputs collection store (user manual sets along with successfully detected mild sleep stages) to achieve a higher level of user adaptivity based on them. Similar adaptation is used by next project of intelligent alarm called “Gently Alarm” [7]. In wide context a limited type of green pervasive computing system can grown from developed application as a base for such system inputs in form of users knowledge or users mental state during a day after a successful or unsuccessful happy wake up [18], [19].
[4]
[5] [6]
[7] [8]
[9]
[10]
[11] [12]
[13]
VII. CONCLUSION It is well known that human life is divided into day and night phase. However less known thing is that for pleasant day is important a phase of sleep in which a human is wake up. Waking up of a human in deep sleep phase lead to a very “unhappy” morning and not so nice spend of day. We develop an intelligent alarm application to provide user with happy wake up. Actual testing results shows that our developed application detect correctly more than 70 % of mild sleep stages of all tested persons. Also there still exists some space for algorithm improvements to over pass some limitations (e.g. disturbances at room space during users monitoring) which we would like to solve in near future.
[14]
ACKNOWLEDGMENT
[17]
[15]
[16]
This research has been carried out under the financial support of the research grants “Centre for Applied Cybernetics“, Ministry of Education of the Czech Republic under Project 1M0567.
[18]
[19]
REFERENCES [1] [2]
HappyWkaUp (http://www.happywakeup.com/en/) smart alarm clock application for Nokia mobile phones. 2010 Sleep Cycle mobile application (http://www.mdlabs.se/sleepcycle/)
247 253
Phan, J., WakeMake Analyzes Your Sleep Cycle to Wake You Up Refreshed (http://www.sync-blog.com/sync/2010/01/analyze-yoursleep-cycle-wake-up-refreshed.html) Math.NET documentation (http://mathnet.opensourcedotnet.info/doc/), OpenSource .NET Projects 2002 – 2010 Microsoft developer network MSDN (http://msdn.microsoft.com/), Microsoft 1985-2010 Recording and Playing Sound with the Waveform Audio Interface ( http://msdn.microsoft.com/en-us/library/aa446573.aspx ), Seth Demsey – Microsoft (2004) Gentle Alarm iPhone Application developed by Craft mobile company (http://gentle-alarm.com/) 2010 Krejcar, O., Jirka, J., Janckulik, D., Proactive User Adaptive System for Windows Mobile Devices – Processing of Sound Input Signal for Sleep State Detection, In Proceedings of 2nd International Conference on Mechanical and Electronics Engineering, ICMEE 2010, 01. 03. August 2010, Kyoto, Japan, Vol. 1, pp. 374-378, DOI 10.1109/ICMEE.2010.5558525 (2010) Carskadon, M.A., Acebo C., Richardson, G.S., Tate, B.A., and Seifer, R. 1997. An approach to studying circadian rhythms of adolescent humans. Journal of Biological Rhythms, Vol. 12: pp. 278–289. Szeinberg, A., Borodkin, K., Dagan, Y., Melatonin treatment in adolescents with delayed sleep phase syndrome, In Clinical Pediatrics Journal, Vol. 45, Iss. 9, pp. 809-818, (2006) Rechtschaffen, A. 1998. Current perspectives on the function of sleep. Perspectives in Biological Medicine, 41: 359–390. Krejcar, O., “Problem Solving of Low Data Throughput on Mobile Devices by Artefacts Prebuffering”, In EURASIP Journal on Wireless Communications and Networking, Article ID 802523, 8 pages. Hindawi publishing corp., New York, USA, DOI 10.1155/2009/802523 (2009) Krejcar, O., Janckulik, D., Motalova, L., Kufel, J., “Mobile Monitoring Stations and Web Visualization of Biotelemetric System - Guardian II”. In EuropeComm 2009. LNICST vol. 16, pp. 284-291. Springer, Heidelberg DOI 10.1007/978-3-642-11284-3_29 (2009). Krejcar, O., Janckulik, D., Motalova, L., “Complex Biomedical System with Mobile Clients”. In The World Congress on Medical Physics and Biomedical Engineering 2009, WC 2009, September 0712, 2009 Munich, Germany. IFMBE Proceedings, Vol. 25/5. O. Dössel, W. C. Schlegel, (Eds.). Springer, Heidelberg. DOI 10.1007/978-3-642-03904-1_40 (2009) Krejcar, O., Janckulik, D., Motalova, L., Frischer, R., “Architecture of Mobile and Desktop Stations for Noninvasive Continuous Blood Pressure Measurement”. In The World Congress on Medical Physics and Biomedical Engineering 2009, WC 2009, September 07-12, 2009 Munich, Germany. IFMBE Proceedings, Vol. 25/5. Springer, Heidelberg. DOI 10.1007/978-3-642-03904-1_39 (2009) Penhaker, M., Cerny, M., Martinak, L., Spisak, J., Valkova, A., “HomeCare - Smart embedded biotelemetry system”, In World Congress on Medical Physics and Biomedical Engineering, Vol 14, PTS 1-6, Aug 27-Sep 01, pp. 711-714, Seoul, South Korea (2006) Cerny, M., Penhaker, M., “Biotelemetry”, In 14th Nordic-Baltic Conference an Biomedical Engineering and Medical Physics, IFMBE Proceedings, Vol 20, Jun 16-20, pp. 405-408, Riga, Latvia (2008) Chilamkurti, N., Zeadally, S., Jamalipour, S. and Das, S.K., “Enabling Wireless Technologies for Green Pervasive Computing,” In EURASIP Journal on Wireless Communications and Networking, vol. 2009, Article ID 230912, 2 pages. doi:10.1155/2009/230912 (2009) Chilamkurti, N., Zeadally, S. and Mentiplay, F., “Green Networking for Major Components of Information Communication Technology Systems,” In EURASIP Journal on Wireless Communications and Networking, vol. 2009, Article ID 656785, 7 pages. doi:10.1155/2009/656785 (2009)
