International Journal of Applied Engineering Research ISSN 0973-4562 Volume 9, Number 24 (2014) pp. 26957-26970 © Research India Publications http://www.ripublication.com
RoBaJe – A Simulated Computational Model for Human Memory to Illustrate Encoding and Decoding of Information K.Roshini 1 B.Bavya2 G.Jeyakumar3 Department of Computer Science and Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Ettimadai, Coimbatore – 641112. 1
[email protected] ,
[email protected],
[email protected]
Abstract: Modeling a human brain plays a vital part in the simulation of human being behavior. To model human behavior we need to understand the thought process of individuals. Thought process is influenced by the factors that affect the permanent storage of memory. Objective of this paper is to present the details of the experiments to create a simulation of human brain, in particular the storage and retrieval of memory with visual perceptions. It is critical to understand how the human memory works, without a suitable memory model. Currently there are inadequate models to simulate and understand the human brain. This paper presents design and implementation of a simple simulation model for human brain, in particular to simulate the memory process with visual perception. This simple model of human brain will serve as a basic prototype for future enhancements. Keywords: Brain Simulation, Memory Encoding, Memory Decoding, Recall
1. Introduction The processes that are involved in acquiring, storing, retaining and retrieving information is referred to as memory. Perception is the beginning phase of creation of memory. The three major processes involved in memory are encoding, storage and retrieval. Encoding is the process in which information is changed into a usable form to create new memories. After successful encoding of information, it is necessary to store it in our memory for later use. Much of this stored memory lies outside of our
Paper code: 29214 - IJAER
26958
K.Roshini et al
awareness most of the time, except when we actually need to use it. The retrieval process allows us to bring the stored memories into conscious awareness. The registration of information during perception occurs in the sensory storage. It lasts only a fraction of a second. Our sensory memory allows a perception such as a visual pattern to linger for a brief moment after the stimulation is over. After that first flicker, the sensation is stored in short-term memory. Shortterm memory can hold about seven items at a time for no more than 20 or 30 seconds. We can increase this capacity by using various memory strategies. For example (Mohs, R.C., 2007), a ten-digit number such as 8005840392 may be too much for our short-term memory to hold. But divided into chunks, as in a telephone number, 800584-0392 may actually stay in our short-term memory long enough for us to dial the telephone. Likewise, by repeating the number to ourselves, we can keep resetting the short-term memory clock (Mohs, R.C., 2007). Gradually, important information will be transferred from short-term memory into long-term memory. If the information is frequently used or recalled, it is more likely to end up in long-term memory, or be retained. Long-term memory can store unlimited amounts of information whereas sensory and short-term memory is limited and decay rapidly. People tend to remember subjects more easily when they are familiar with the content because that information is connected to the related information in their long-term memory. That is why we are able to remember information about any subject. Distractions that occur while trying to register information in memory affect the encoding process. This may lead to inefficient and poor encoding. The person may ‘forget’ memories because of trouble in retrieving the poorly encoded memory. These distractions are termed as noise. Our brain does not modify its structure. But the connections between the nerve cells change as we learn. Our synapses get reinforced and connections get strengthened. But ageing causes synapses to falter. This affects our retrieval of memories. Aging also affects some parts in brain which are essential for memory. Also, as we age, our brain shrinks and becomes less efficient. Inherited unhealthy genes, exposure to poisons, alcohol or smoke speed up this decline.
2. Related works The fundamental concept of cognitive science is that "thinking can best be understood in terms of representational structures in the mind and computational procedures that operate on those structures”. In mid-1950s researchers started developing theories of mind, based on various representation structures and procedures. Henry, L. (2012), developed an influential theory as a working model to understand how thinking and reasoning manipulates the information in memory. It consists of four components: the phonological loop (speech material), the visuospatial sketch pad (visual information), central executive (overall control), episodic buffer (integrating information). Marr, D. (1982) reported the importance of computational theory for understanding the information processing in brain. He suggested that the task of any information processing system can be understood in three levels: computational theory, algorithm
RoBaJe – A Simulated Computational Model
26959
and representation and hardware implementation. Laughery, K.R. and Allen L. (1969), formulated a simulation model for human short-term memory. Pew, R.W. and Mavor, A.S. (1998) conducted a review on the human modeling literature. An overview of the working of human memory which includes the role of consciousness in learning and memory is covered in (Baddeley, A. 1997). Foltz, P.W. (1991) concluded that models of human memory and computer information retrieval have many similarities in the methods they use for representing information and accessing the information. In (Salvioni, P et al., 2013), the authors say that time is a key feature of any sensory experience. Sensory events unfold in time and the way we perceive this temporal unfolding is crucial for our understanding of these events. A general outline for the theoretical framework of human memory was presented in (Atkinson, R.C. and Shiffrin, R.M., 1968). Cognitive Science thrives to understand mind and its operations. Mind is nothing but what our brain does. So, simulating the mind is the basis for brain simulation. It is, however, not necessary to mimic human brain in its total but enough to figure out how the software parts of brain work (Dvorsky. G, 2012). It is sufficient to mimic how the human mind works. What is required is a functional understanding of all the necessary low level information about the brain, mind and knowledge (Dvorsky. G, 2012). A team of Japanese and German researchers performed a simulation of one second brain activity on the K Computer (ranked few years ago as the world’s fastest computer). It required 82,944 processors to do it. This shows that we are still quite a ways off from being able to match the computational power of the human brain. The aim of this project is to propose a model with cognitive abilities similar to those of human brain. As a preliminary attempt, we focused into developing the representation structures of mind and computational procedures that operate on those structures, for encoding and decoding of the information. Cognitive theorists propose that the mental representations can be modeled as logical prepositions, rules, concepts, images and analogies. The mental procedures can be modeled by deduction, search, matching and retrieval (Zalta, E.N, 2014). The basic mechanisms involved in simulating the memory of a human brain are encoding and decoding. In (Craik, F.I.M and Lockhart, R.S, 1972), the author suggests that memory was actually dependent upon the level of processing of the information, rather than being in different stores with different features. The authors, in (Dennis S. et al., 2005), conclude that the first step in memory retrieval in a free recall task is the unloading of the active buffer (items currently in the short term buffer). As different approaches are suggested and different programming languages can be used, there can be numerous computational models possible for the mind. With the objective of making the model simple and scalable, we designed database tables for the representation and computational algorithms for the procedures. The algorithms, framed by us, are presented in figure 4 and figure 5 for the encoding and decoding processes, respectively.
3. The Methodology Priorities, emotions attached to thoughts, relationships with permanent thoughts all
26960
K.Roshini et al
influence storage of memory in various sections namely the sensory, short term and long term memory. Every thought first enters sensory memory. It eventually enters short term memory and long term memory depending on factors that affect storage in various sections of memory. Forming relationships of new thoughts with already existing permanent thoughts gives, the information from the thought more meaning. This can be linked to related thoughts in the memory. As and when a thought in the memory is recalled, it has more chances of getting into the permanent memory as it is used often. Recalling process is affected by many factors such as ageing and noise. We need to find ways to slide important information residing in short term memory to long term memory as the latter has no limit in capacity or time. The simulation basically deals with encoding and decoding of thoughts. For the sake of simplicity we simulate human memory for visual perception. The properties of the perceived objects are stored in the memory by forming relations with an already existing thought and setting priorities to them. Priorities and other factors such as frequency of recollection decide the storage area of thoughts (i.e. either in the sensory memory, short term memory or long term memory). Thoughts are retrieved from the memory on exact or related object match. Retrieval of thoughts is influenced by factors such as noise during storage and ageing. Mind wavering is a special case of retrieval of thoughts. 3.1 Encoding Thoughts In this simulation model the thoughts are considered to be in the form of episodes. Each episode can be considered as a sequence of images that are captured by our eyes at an interval of 1/24th of a second. Using image processing methods (object recognition), these images can be analyzed to recognize distinct objects. Each object can be uniquely described with a set of finite properties, namely: Form, movement and color. Form describes the perceived object’s shape, size, capacity and texture. Movement describes the variation in position of the parts of the object with respect to time. Color parameter describes the intensity and gradient of color in each pixel of the object. Since we consider to simulate the brain for the visual perception, we use the ‘object’ here as the ‘hand gesture’. 3.1.1 Forming Relations Relations between objects can be formed based on: 1. Match in properties between various objects. Eg: Consider we are trying to store the hand gesture for a “Ball” by showing a red ball, the machine recognizes a blue ball also as “Ball” due to the match in the properties between the two balls. 2. Actions performed by the object during input. Eg: If the hand gesture for “Man sits on chair” is stored in the machine then a relation “Sits” is formed between “Chair” and “Man”.
RoBaJe – A Simulated Computational Model
26961
3. Explicit details of the object fed by a third person to the person whose memory is being encoded. Eg: Consider we are trying to store the hand gesture for “Mother” and “Child” in the memory of our learning machine. We will also explicitly teach the machine that the relation between the objects is “Parent”. 3.1.2 Setting Priority to thoughts The priority for a particular thought / object is based on the frequency of recollection, usage and also the emotions felt while perceiving the thoughts. The prioritization of emotions (Levine, L.J. and Burgess, S.L., 1997) based on how they affect our memory has been shown in Table 1. 3.1.3 Handling Redundant Input Consider an object is already encoded and stored in the memory. If this object is perceived again, the extra properties that are perceived in addition to the properties that were already stored in the memory are added to the object. The object is not created again in the memory. Table 1 Prioritization of emotions Emotions Priority Value Happy 62 Angry 35 Sad 41 Neutral 85
3.2 Storage Area The storage area can be divided into three sections: the sensory memory, the short term memory and the long term memory. A threshold value of priority is fixed for storage in various sections of memory. Table 2 displays the threshold value for entry of thoughts into various sections of memory. For example, if the priority value of an object exceeds 80, it can be moved to the long term memory. Table 2 Categorization of storage area based on priority Storage Area Threshold Value of priority Long Term Memory 80 Short Term Memory 50 Sensory Memory 0
3.2.1 Sensory Memory
26962
K.Roshini et al
Certain properties help us place a thought in sensory memory stage. The properties are as follows: 1. Frequency of recollection of thought should be less. 2. Recollection time: should be in a few seconds (200-500ms) after storage of thought. 3. The priority of the thought should be less. The particular content of sensory memory is deleted, if the priority has not been increased or recollection has not happened within the given threshold timestamp. Mostly high attention span increases the chances for recording the minutest fragments of memory and these thoughts, could, in turn, enter the Short Term Memory. 3.2.2 Short term Memory The properties that help us place a thought in short term memory are as follows: 1. The last retrieval of thought from memory should be recent. 2. Frequency of recollection must be quite high. 3. Recollection time: should be able to recollect the thought between 10-15 seconds after the storage of the thought. 4. The priority of the thought should be high i.e. more relations should be formed compared to the volatile Sensory Memory. The particular content of short term memory is deleted, if the priority is decreased or recollection has not happened within the given ‘Recollection time’. If frequency of recollection increases it enters the Long Term Memory. 3.3.3 Long term Memory Few properties help us place a thought in long term memory. The properties are as follows: 1. The last retrieval of thought from memory may not be recent. 2. Recollection time: should be able to recollect the thought from a few months to years after the storage of the thought. 3. The priority of the thought should be high i.e. more relations should be formed compared to the short term memory. The particular content of long term memory is deleted, if the priority is decreased or recollection has not happened within the given ‘Recollection time’. 3.3 Memory Decoding The retrieval of thoughts is based on relations formed by the object. The decoding is priority based. It is based on exact object match or related object match. Mind Wavering: Wavering of thoughts is based on random retrieval of objects connected through relations. Noise: In general, our mind can focus on seven objects at once. If an episode contains more than one thought at a particular timestamp (distractions during perception of thoughts) then noise occurs in the sequence of recollection
RoBaJe – A Simulated Computational Model
26963
because of overlap in timestamp of perception. The retrieval is then based on priority of thoughts during the overlapping time period. Ageing: When a person gets older, the relations between objects are deleted in random after every decade. This closely resembles the neuron synapses getting weakened or broken.
4 Algorithms for Simulation The overall flow of how memory is encoded and decoded has been shown in Figure 1. Objects in the environment are perceived through the sensory organs. These are stored in various sections of memory considering the factors that affect storage. The flow of thoughts during encoding is shown in Figure 2. Memory flow is only unidirectional. That is, it can flow from sensory to long term but not vice-versa. But objects in memory are prone to deletion due to ageing, analogous to weakening of synapses in brain. The detailed algorithmic description of the encoding process used for our simulation study is presented in Figure 4. Decoding takes place when objects are retrieved from memory. Whatever we try to recall is taken as an input to the system (brain). These are then recalled partially or completely. Memory retrieval is affected by factors such as ageing or noise (distractions) during storage. If they are affected by such factors, only partial retrieval of object takes place. Otherwise, they are retrieved completely. Mind wavering does not follow a specific path. It can choose any random connected path and retrieve objects from memory. The flow of thoughts in memory decoding is shown in Figure 3. The detailed algorithmic description for the decoding process is shown in Figure 5.
Figure 1 Overall flow of encoding and decoding of thoughts in memory
26964
K.Roshini et al
Figure 2.The flow of encoding process
Figure 3. The flow of decoding process
RoBaJe – A Simulated Computational Model
26965
PSEUDOCODE Procedure: Encoding thoughts Input: Image perceived by eye (Visual Perception) begin PROCESS 1: Retrieve components from the image using Image Processing Techniques 2: Each component is labeled with a distinct object name 4: if (PERCEIVED OBJECT IS NOT REDUNDANT) then 3: { 4: Objects are analyzed for their properties (Form, Movement and Color) 5: Form should contain description of the object’s shape, size, capacity and texture 6: Movement should contain position of object and the time when the object was perceived to determine the variation in position 7: Color should contain the intensity and gradient of color of object 9: These properties are added to the database for the respective objects 10: } 11: else 12: Only those properties that have changed needs to be added to the database 13: Emotions felt while perceiving the object is stored and the corresponding (emotional) integer value is taken from Table 1 14: The value of priority is calculated as an average of emotional quotient and the attention span while perceiving the object 15: Based on the value of priority the area of storage in brain is determined 16: if (0
