The Role of Chatbots in Formal Education

SISY 2018 • IEEE 16th International Symposium on Intelligent Systems and Informatics • September 13-15, 2018, Subotica, Serbia

The Role of Chatbots in Formal Education György Molnár*, Zoltán Szűts** *

Budapest University of Technology and Economics/ Department of Technical Education, Budapest, Hungary ** Budapest University of Technology and Economics/ Department of Technical Education, Budapest, Hungary [email protected]; szutszoltan@ eik.bme.hu ABSTRACT Chatbots appeared in large numbers at the beginning of the current decade. Interactive technology, often combined with artificial intelligence, has rapidly invaded and occupied the world of online chat. Chatbots are not just elements of virtual assistants, but are used by organizations and governments on websites, in applications, and instant messaging platforms to promote products, ideas or services. In this paper, the authors firstly present a theoretical and historical background, then discuss the issues of using chatbots as educational assistants, and finally describe the basic steps and challenges of programming a bot. DEFINITIONS The term chatbot – chat(ter)bot was coined by Michael L. Mauldin. [1]. According to the simplified definition offered by Shawar and Atwell chatbots are chat applications supported by artificial intelligence whose functions range from answering simple questions to taking part in complex conversations [2]. Depending on their type, chatbots can participate in voice and text-based conversations; in this study, the authors focus on the latter. Chat applications are capable of giving different responses to requests or questions from different users [11]. In the beginning, chatbots were not very intelligent; to some preprogrammed questions they would give specific, predetermined responses. In most cases, they were of no help to users, and often actually discouraged them from interacting. With the help of artificial intelligence, however, they stared learning and often assuming the role of human operators in several areas, including some basic communication tasks in education [15], [16]. According to Britz [3], chatbots are computer programs capable of conducting conversations similar to those between people. Chatbots are often used in order to automate or optimize a business process. The types of chatbots range from simple to complex – in the latter case, the aim is to exploit a wide spectrum of artificial intelligence. Simple bots handle basic messages and requests from users. When communicating with users, these algorithms give pre-programmed responses to a given input as outputs, and their communication style or language use is not sophisticated or highly differentiated. For example to a question about a product, a link to the information requested can be offered. More complex

978-1-5386-6841-2/18/$31.00 ©2018 IEEE

chatbots are more efficient and able to engage in more complex discourses. Through the use of machine learning, these are programmed to learn from their previous conversations. This learning process often takes place under active development supervision [4]. Chatbots are usually used in dialog systems, for example, in customer service, where information is exchanged. The authors believe that chatbots could also prove themselves useful/effective/helpful as educational assistants in roles which do not require them to answer overly complex practical questions. In basic cases, simpler systems can recognize keywords by identifying them in their database, while more sophisticated operations require natural language processing or understanding (NLP). Britz points out that natural language processing or understanding allows chatbots to understand the intentions and complex demands of users. Chatbots can be used, for example, in answering frequently asked questions - but, as we will see, there are many conditions which must be met for this to succeed [3]. HISTORY OF CHATBOTS To present the use of chatbots as teaching assistants, their history should be summarized. AOL Instant Messenger, developed by AOL, was released in 1997. Like other chat applications, users could talk to each other. In the AIM environment StudyBuddy and SmarterChild (see Figure 1), capable of interacting with users and assisting them with non-formal learning, were introduced. In addition to interpersonal communication, the application also supported chat rooms, in which members of the online community could talk [5].

Figure 1.

StudyBuddy and SmarterChild, own screenshot

Although the AIM and Facebook chatbots are the most popular in scientific discourse, many chat applications have been set up since their advent to chat with users. The classic example is ELIZA (see Figure 2), which played the role of a psychotherapist in 1966, followed by Parry, which was developed in 1972 [6]. Chatbot ALICE (Artificial Linguistic Internet Computer Entity) worked between 1995 and 2000 and was also based on ELIZA. Its teaching took 5 years [2].

000197

Gy. Molnár, Z. Szűts • The Role of Chatbots in Formal Education

ALICE was also based on the Artificial Intelligence Markup Language (AIML), which is largely similar to the structure of today’s modern solutions. [7]

Figure 2. ELIZA

A qualitative leap occurred in 2003, when AIM users could interact with ZoeOnAOL. After setting up Zoe on the list of acquaintances, they could ask questions. Most of the questions were about information that could be obtained through search engines. At that time, users were spending a lot of time chatting, and the chatbot made it possible for them not to leave the chat window, interrupting the interaction with other contacts momentarily. Zoe recognized the keywords and run the searches, such as ‘what is the weather in Dallas?” and immediately gave the correct answer. As Zoe could take part only in basic interactions, users were always aware of the fact that they were communicating with a machine then. SmarterChild, running on AOL, Yahoo, or MSN, worked on a similar principle, and users often asked for information about sports events or current share prices. Since the early 2000s, advertisers and marketers have recognized the potential of chatbots, so eBay, Warner Bros. Records, and Capital Records have operated simple messaging bots in messaging systems for marketing purposes.

decision-tree structures that lead users to predetermined dialogs. B. Generative models Generative models based on comprehension are much more advanced because they do not rely on predetermined answers. Basically, they use machine translation technology. However, this translation does not mean transposition from one language to another, but rather it produces output from the input. Both approaches have their strengths and weaknesses. For example, thanks to their use of pre-generated responses, tree-based chatbots do not commit grammatical and linguistic errors that frustrate users. However, sometimes the system cannot find pre-defined responses. The generative model, on the other hand, is “smarter”. It refers back to previous information, so the perception can be that the user is talking to a real person. These types of chatbots are difficult to program, teach, and grammatical mistakes are frequent, especially in long sentences, during the conversation. If there are too many mistakes, the users can easily become frustrated. Deep learning technology, artificial intelligence and machine learning can be successfully applied to both models. Architectures such as “Sequence to Sequence” are suitable for generating texts, but tree systems are more reliable in their present state of simplicity. [3] CHATBOTS IN EDUCATION The authors believe that education is a very important field of application for chatbots. Nowadays, students receive a significant part of their information about studies, curriculum and tasks online. Chat programs therefore provide significant help in learning and learning processes, for example, NerdyBot. (See Figure 3).

TAXONOMY OF CHATBOTS Babar et al. distinguish two types of chatbots, those employing the tree structure and the generative model [9]. A. Retrieval-based models The tree structure, retrieval-based models use repositories of various sizes, and to some degree the heuristic imitation of human memory to respond. This heuristic can be simple: answers are given by simple pairing, while in complicated conversations machine learning (we don’t use it here in the way Hungarian can)plays a role. These systems are based on predefined databases. When using natural language processing, developers have at their disposal .NET or Java programming languages. Using these they can create

000198


• The total number of users accessed • The length of the conversations (time and question number) • The number of conversations with individual users • The number of clicks on links • The response time BEST PRACTICES

Figure 3.

A good example of the successful application of chatbots as educational assistants is a chatbot, Jill, developed by Ashok Goel and applied at Georgia Tech. More than 400 students attend the online courses of Ashok Goel every semester. These students use e-learning study independently of space and time, but naturally, many of them have questions. In the course of the semester - as there is no real encounter between the teacher and the students according to e-learning rules - more than 10,000 questions were received. Even though several teaching assistants were involved in the work, it became difficult – impossible – to answer the questions over time. That’s why Goel developed a teaching assistant chatbot. [10] The questions answered ranged over a wide scale. The students were interested in the format of the papers required and the topics and deadlines of the tasks to be completed during the semester. Teachers responded to complex questions, while the simple ones were answered by the algorithm. The chatbot was trained on specific questions (40,000 items) that came from a variety of sources in the early years. When answers showed 97% accuracy, the test mode was cancelled and the bot went online. Jill was created using an IBM Bluemix platform [11].

NerdyBot

These programs already offer a native communication channel for the Z generation. In such an environment, every student becomes a member of a virtual learning group. The use of chatbots allows the sending of reminders about exams, and generative systems can even help with understanding the curriculum. The ongoing developments in NLP make it possible for systems to understand students’ questions. These students are more likely to believe in information received from chatmates rather than search engines. Therefore, knowledge must be disseminated in an environment in which the students spend a lot of time.

PROPOSITION At the end of this paper the authors present a short proposition on the use of chatbots as teaching assistants. There are several platforms where HEI-s can start to build a conversation app. In this proposition, https://wit.ai was used. This chatbot can easily be integrated into Facebook messenger. After registration, the bot has to be programmed. The first task was to recognize interest. As a first step in the Understanding tab on the Console we typed in the expression field: > What should the length of the essay be? (see Figure 4).

ISSUES OF APPLYING CHATBOTS When an educational institution uses a chatbot to communicate with students, the error rate at which the application works is initially very high. The most effective way to apply it is if it is initially implemented in some of its predefined topics. The ability of chatbots will evolve over time through their evaluating chat conversations in the course of the learning process. The more conversations a chatbot has, the more intelligent it becomes. In order for this to happen, the following are required:

000199

Figure 4. Expression field


In the Add a new entity dropdown we defined the “intent” as “paper format”. (This is called a trait entity, in which the sentence as a whole means intent=paper format instead of one particular word) (Figure 5).

Using this logic, the most typical questions can be answered by the teaching assistant chatbot. Wit.ai provides the opportunity to use several languages, including Hungarian. This can be useful in e-learning classes at Budapest University of Technology and Education. CONCLUSION

Figure 5. Paper format

The next step was the implementation of the bot into our Facebook Messenger profile. In this step Natural Language Processing (NLP) was used. NLP allowed an understanding and the extraction of meaningful information (the entities) from the messages students would send during the semester. These entities could then be used to identify intent and at the same time automate some of the replies, route the conversation to a human teaching assistant via livechat, and collect audience data. During the implementation, in Facebook app’s 'Messenger Settings’ page we toggled the „on off” switch to enable/disable built-in NLP for our app. [12]. For each message, the Messenger Platform will return a mapping of the entities that were captured alongside their structured data. The key pieces of information here are the degree of confidence and the value for each entity. Confidence is a value between 0 and 1 and indicates the probability the parser thinks its recognition is correct (See Figure 6). function firstEntity(nlp, name) { return nlp && nlp.entities && nlp.entities[name] && nlp.entities[name][0]; } function handleMessage(message) { // check paper format is here and is confident const paperFormat = firstEntity(message.nlp, 'paper format'); if (paperFormat && paperFormat.confidence > 0.8) { sendResponse('See format description at the link'); } else { // default logic } } Figure 6. Recognition process

There are a number of areas in which, in authors’ opinion, chatbots cannot be used in education. In such contexts, they are more likely to be able to disclose supplementary information than to solve content issues. Lastly, chatbots can not only simplify the administrative work of teachers and increase trust; if, for example, they are not efficient enough or do not understand the users’ requests, they could also cause frustration originating from unsuccessful communication. REFERENCES [1]

Mauldin, Michael L.,” CHATTERBOTS, TINYMUDS, and the Turing Test: Entering the Loebner Prize Competition.” Proceedings of the 12th National Conference on Artificial Intelligence (Seattle, WA, USA, July 31 – August 4, 1994), Vol. 1. p. 16–21. [2] Shawar, Bayan Abu and Eric Atwell, „Using dialogue corpora to train a chatbot”. Proceedings of the Corpus Linguistics 2003 conference (Lancaster, 28-31 March 2003) Lancaster University, 2003, p. 681–690. [3] Britz, Denny, „Deep Learning for Chatbots. Part 1”. WILDML. Artificial Intelligence, Deep Learning, and NLP. 6 April 2016, http://www.wildml.com/2016/04/deep-learning-for-chatbots-part1-introduction/ [4] Bradford, Laurence, „How Chatbots Are About To Change Communication”, Forbes.com, 25 July 2017, https://www.forbes.com/sites/laurencebradford/2017/07/24/howchatbots-are-about-to-change-communication/#258058164aa8 [5] Petronzio, Matt, „A Brief History of Instant Messaging”, Mashable.com, 25 October 2012, http://mashable.com/2012/10/25/instant-messaginghistory/#XnMxDUiwxPqR [6] Weizenbaum, Joseph, „ELIZA – a Computer Program for the Study of Natural Language Communication Between Man and Machine”, Commun. ACM, Vol. 9., (1966), No. 1., 36–45. [7] Wallace, Richard, „The elements of AIML style”, 2003, http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.693.366 4&rep=rep1&type=pdf [8] Olsen, Stefanie, „AOL tries out new IM chat bot”, CNET.com 27 January 2003, https://www.cnet.com/news/aol-tries-out-new-imchat-bot/ [9] Babar, Zia, Alexei Lapouchnian, Eric Yu, Chatbot Design – „Reasoning about design options using i* and process architecture.” iStar Workshop, 2017. http://ceur-ws.org/Vol1829/iStar17_paper_7.pdfCopeland, Jack and Diane Proudfoot, „Turing’s test: A philosophical and historical guide”, in Robert Epstein, Gary Roberts and Grace Beber (eds.), Parsing the turing test: Philosophical and methodological issues in the quest for the thinking computer, Springer, New York, 2008, pp. 119–133 [10] Goel, Ashok, et al. "Using watson for enhancing human-computer co-creativity." 2015 AAAI Fall Symposium Series. 2015. [11] Joyner, David. "Squeezing the limeade: policies and workflows for scalable online degrees." Proceedings of the Fifth Annual ACM Conference on Learning at Scale. ACM, 2018. [12] Warwick, Kevin and Huma Shah (2016) Can machines think? A report on Turing test experiments at the Royal Society, Journal of

000200


Experimental & Theoretical Artificial Intelligence, 28:6, 9891007. [13] Copeland, Jack and Diane Proudfoot. (2008). Turing’s test: A philosophical and historical guide. In Robert Epstein, Gary Roberts and Grace Beber (eds.), Parsing the turing test: Philosophical and methodological issues in the quest for the thinking computer (pp. 119–138). USA: Springer. [14] Tibor Ujbanyi, Gergely Sziladi, Jozsef Katona, Attila Kovari, ICT Based Interactive and Smart Technologies in Education Teaching Difficulties, INTERNATIONAL JOURNAL OF

MANAGEMENT AND APPLIED SCIENCE 3.:(10.) pp. 72-77. (2017) [15] Peter Toth, Imre Rudas, Web-based Learning and Web Mining, In: Shamim Ali et al (ed.) The Asian Conference on Technology in the Classroom: The Impact of Innovation: Technology and You, 2013, .pp. 101-113. [16] E. Gogh, A. Kovari, Examining the relationship between lifelong learning and language learning in a vocational training institution, Applied Technical and Educational Sciences, Vol 8 No 1 (2018): 2018/1.pp 52.

000201


000202