Vaidya: A Spoken Dialog System for Health Domain Prathyusha Danda, Brij Mohan Lal Srivastava, Manish Shrivastava {prathyusha.danda,brijmohanlal.s}@research.iiit.ac.in,
[email protected] LTRC, International Institute of Information Technology, Hyderabad
About Introducing Vaidya, a spoken dialog system built for the purpose of solving basic healthcare requirements in Indian context. Objectives: Avoid network dependence by keeping the complete system in device. 2 Support multiple languages 3 Low-resource dialog modeling 1
Approach & Implementation • Each
symptom is mapped against a disease from UMD ontology • To create the dialog flow language-independent, symptoms are mapped to a concept ID which is referred to based on concept resolution. • A symptom × disease (S) binary matrix is created to shortlist diseases based on input symptom as speech. Probable diseases are picked by performing AND operation over all the rows corresponding to input symptoms. • Disease matrix D is created as transpose(S) to shortlist possible symptoms to enquire from user. System must pick an optimal symptom from the new set of possible symptoms which reduces the disease search space to half the previous state. • Symptoms with high disease correlation are avoided to overfit the result. An optimum symptom is chosen such as to optimize the number of steps for diagnosis.
Progress • Developed
low-resource language-independent keyword spotting • Developed highly-scalable language identification • Provided support for three languages, namely, Hindi, English and Telugu
System architecture
Resources used • Human
disease ontology by Univ. of Maryland Medical Center • Pocketsphinx speech recognition toolkit • Android framework
Speech analysis • Collected
30 hours of Telugu speech and transcribed • Analyzed 15 hours of Doctor-Patient conversations for modeling dialog state transition • Learnt robust articulatory gesture based representation for multilingual speech recognition
Subjective evaluation results
Domain agnostic dialog flow
