Fuzzy cognitive mapping of LMS usersÐ²â¬â¢ Quality of

Expert Systems with Applications 42 (2015) 7399–7423

Contents lists available at ScienceDirect

Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa

Fuzzy cognitive mapping of LMS users’ Quality of Interaction within higher education blended-learning environment Sofia B. Dias a,⇑, Sofia J. Hadjileontiadou b, Leontios J. Hadjileontiadis c, José A. Diniz a a

Faculty of Human Kinetics, University of Lisbon, 1499-002 Cruz Quebrada, Lisbon, Portugal Hellenic Open University, Praxitelous 23, 10562 Athens, Greece c Department of Electrical and Computer Engineering, Aristotle University of Thessaloniki, GR-54124 Thessaloniki, Greece b

a r t i c l e

i n f o

Article history: Available online 29 May 2015 Keywords: Blended learning Moodle Learning Management System Fuzzy Cognitive Maps (FCMs) Quality of Interaction (QoI) FCM-Viewer application

a b s t r a c t Learning Management Systems (LMSs) under blended (b-) learning modality can efficiently support Online Learning Environments (OLEs) at Higher Education Institutions (HEIs). Mining of LMS users’ data, involving artificial intelligence and incertitude modeling, e.g., via fuzzy logic, is a fundamental challenge. This study addresses the hypothesis that the structural characteristics of a Fuzzy Cognitive Map (FCM) can efficiently model the way LMS users interact with it, by estimating their Quality of Interaction (QoI) within a b-learning context. This study introduces the FCM-QoI model (combined with a model visualizer) consisting of 14 input-one output concepts, dependences and trends, considering one academic year of the LMS use from 75 professors and 1037 students at a HEI. The experimental findings have shown that the proposed FCM-QoI model can provide concepts interconnection and causal dependencies representation of Moodle LMS users’ QoI, revealing perspectives of its evolution both at a micro and macro analysis level. Moreover, the FCM-QoI model significantly adds to the evaluation and analysis of the QoI influential concepts’ contribution to self-sustained cycles (static analysis) and their alterations, when the time period of the LMS use is considered (dynamic analysis), showing potential to increase the flexibility and adaptivity of the QoI modeling and feedback approaches. Clearly, based on the FCM-QoI model, pedagogical instructors and decision-makers of HEIs could be assisted to holistically visualize, understand and assess OLEs stakeholders’ needs within the teaching and learning practices. Ó 2015 Elsevier Ltd. All rights reserved.

1. Introduction Education can be seen as a set of processes designed to transmit knowledge, skills and values to develop individual and collective abilities. At the same time, Learning Management Systems (LMSs), such as Modular Object-Oriented Dynamic Learning Environment (Moodle), within an Online Learning Environment (OLE), can provide educators an environment to place their online course materials and for students to receive that education while interacting with other students/teachers; however, students’ interactions, attention and communications are seen as relatively low in the LMSs (Musbahtiti & Muhammad, 2013). Nevertheless, it seems

⇑ Corresponding author at: Faculty of Human Kinetics, Estrada da Costa 1499-002 Cruz Quebrada-Dafundo, Lisbon, Portugal. Tel.: +351 214 149 151; fax: +351 214 153 095. E-mail addresses: [email protected] (S.B. Dias), shadjileontiadou@gmail. com (S.J. Hadjileontiadou), [email protected] (L.J. Hadjileontiadis), jadiniz@fmh. ulisboa.pt (J.A. Diniz). http://dx.doi.org/10.1016/j.eswa.2015.05.048 0957-4174/Ó 2015 Elsevier Ltd. All rights reserved.

fair to say that Higher Education Institutions (HEIs) are facing the need of constant monitoring of users’ interaction with LMS, in order to identify key areas for potential improvement. This could be expressed in terms of Quality of Interaction (QoI) through the LMS use within a blended (b-)learning environment (López-Pérez, Pérez-López, & Rodríguez-Ariza, 2011; Miyazoe & Anderson, 2010; Oliver & Stallings, 2014; Woltering, Herrler, Spitzer, & Spreckelsen, 2009). In fact, the lack of interaction is considered an important obstacle to online learning (Parker, 1999); however, several improvements in Information and Communication Technologies (ICTs) over the last years have enhanced the QoI in online environments. In this vein, there are different types of interactions that educators can consider into the curriculum, more specifically: learner to instructor, learner to learner, learner to content, learner to tools, and learner to environment (Bastedo & Vargas, 2014). Considering the aforementioned perspectives, QoI can be extended from individuals to groups/networks, from closed to open learning environments, and from small groups to massive

7400

Table 1 Research studies regarding to the FCMs use for educational scenarios along with corresponding objective(s) and problem solving areas. Research Study

Tsadiras and Stamatis (2008)

Salmeron (2009) Hossain and Brooks (2008) Yang, Li and Lau (2011)

Yesil, Ozturk, Dodurka and Sahin (2013) Barón, Crespo, Espada and Martínez (2014)

Peña-Ayala and Sossa-Azuela (2014) Baykasog˘lu and Gölcük (2015)

Georgiou and Botsios (2008) Georgopoulos, Chouliara and Stylios (2014)

Takács, Rudas and Lantos (2014)

Chrysafiadi and Virvou (2013) van Vliet, Kok and Veldkamp (2010) Luo, Wei and Zhang (2009) Anusha and Ramana (2015)

To examine the use of FCMs for planning network learning and for enhancing the success of learning programs To model critical success factors in LMS To model educational software adoption at UK secondary schools based on stakeholders’ perceptions To propose a set of student’s progress indicators based on the FCM and describe his/her progress on various aspects and their causal relationships To identify the causalities of the education management of the Thailand science-based technology school To model the control engineering educational critical success factors To present a learning assessment system that uses multivariate analysis based on structural equation modeling, facilitating the assessment of learning on interactive environments To model the teaching–learning environment and simulate the bias exerted by authored lectures on the student’s learning To propose a new model, that integrates the technique for order preference by similarity to ideal solution and FCMs for modeling and solving decision problems related to industrial engineering To identify learning styles in adaptive educational hypermedia systems To expand the utilization of FCM based on medical decision support systems for learning and educational purposes using a scenario-based learning approach To introduce a novel FCM based algorithm for the calculation of the values of the interrelation levels between the factors in a system for the student’s grade evaluation To contribute to the improvement of the navigation support in an e-learning adaptive system, for teaching computer programming To describe an updated framework for participatory scenario, bridging the gap between storylines and models To design a game-based learning system To analyze the symptoms of stress on college students using combined disjoint block fuzzy cognitive maps model

Problem solving Modeling

Decisionmaking

Planning

Prediction

U

U

U

U

U U

U

Factors analysis

Knowledge representation

Progress indicators

Framework setting

Learning

Control and guiding

AnalysisEvaluation

U U

U

U

U

U

U

U

U

S.B. Dias et al. / Expert Systems with Applications 42 (2015) 7399–7423

Nownaisin, Chomsuwan and Hongkrailert (2012)

Objective(s)

U

U

U

U

U

U

U

U

U

U

U

U

U

U

U

U

U

U U


opportunities. The QoI is usually estimated by employing statistical analysis of LMS data, combined with transcripts of the discussions and exchanges of teacher and learners within the online forums, specifically investigating the dimension, depth and category of exchanges occurred (Ping, Cheng, & Manoharan, 2010). An alternative approach has been proposed by Dias and Diniz (2013), who introduced for the first time a fuzzy logic (FL)-based model, namely FuzzyQoI, to estimate the QoI, taking into account the users’ (professors’ and students’) interactions, as expressed through the LMS usage within a b-learning environment. Note that for a state-of-the-art review upon the use of FL in education the reader could refer the work of Dias and Diniz (2013) and Dias, Diniz, and Hadjileontiadis (2014a). In the FuzzyQoI model, the knowledge of the experts in the field was translated into fuzzy constructs, and, through five nested fuzzy inference systems (FISs), an estimation, in a quantitative way, of a normalized index of the users’ QoI was achieved (Dias & Diniz, 2013). In the FuzzyQoI model, the input (users’ LMS metrics)–output (normalized index of users’ QoI) mapping was integrated into a system as a quantitative map, yet, internally it can be considered as a set of qualitative linguistic rules that could be used to model the type of vague or ill-defined systems that are difficult to handle using conventional binary valued (crisp) logic (Tsoukalas & Uhrig, 1996). Stemming from the FL concept proposed in FuzzyQoI model (Dias & Diniz, 2013), an alternative approach, yet within the field of FL, is proposed here, based on the concept of Fuzzy Cognitive Map (FCM) (Kosko, 1986), namely FCM-QoI model. In general, FCMs can model causal relationships in complex systems that evolve with time. From this line, several studies have shown how FCMs can provide an understanding of problematic domains or systems and/or knowledge for strategic purposes in terms of maximization of benefits, minimization of risks and management issues (e.g., Glykas, 2013; Jetter & Sperry, 2013; RodriguezRepiso, Setchi, & Salmeron, 2007; Sharif & Irani, 2006). The adoption of the FCM approach within the FCM-QoI model is motivated by the fact that they are: (i) easy to use, create, visualize and parameterize, (ii) flexible in representation/visualization, (iii) understandable for nontechnical experts and for different stakeholders groups, (iv) able to handle with complex and dynamic issues, and (v) capable of providing an efficient feedback mechanism that combines information drawn from distinct resources to create an enrich body of knowledge. In line to the above, the use of FCMs in modeling the QoI of the LMS users within b-learning environment aims at providing a ‘‘strategic mapping’’ to better understand the particular domain of HE in an interconnected way of thinking. In fact, the theory of FCM is mainly concerned with modeling factors and their interrelationships within complex domains/systems. Knowing that the adoption of LMSs in HEIs represents a complex domain with multiple influencing aspects, FCMs seem quite suitable to model this particular domain; simultaneously, considering that FCMs have the ability to model dynamic systems (Kosko, 1992), they are promising tools to capture the (internal/external) dynamic aspect of the LMSs. Furthermore, the FCM-based model could present a predictive power for estimating the QoI, involving concepts, dependences and trends, considering the time period of the LMS use. From this perspective, experimental validation using real LMS Moodle data drawn from a large database from a HEI, could reveal the efficiency of the proposed FCM-QoI model under the LMS-based b-learning approach. To accomplish the aforementioned goals, this study addresses the hypothesis that the structural characteristics of a FCM can efficiently model the way LMS users interact with it and, further, evaluate it by estimating their QoI within a b-learning context.

7401

2. Literature review The literature review is divided into two major sections. The first section reviews research done on the use of FCM in the educational context, while the second one focuses on literature review dealing with improvements in the FCM functionality.

2.1. FCM within the educational context Originated from the theories of FL, neural networks, soft computing and computational intelligence techniques, FCMs can be understood as a modeling methodology based on exploiting knowledge and experience (Kosko, 1986, 1992; Papageorgiou & Stylios, 2008). Nowadays, FCMs have played an important role in different scientific fields, such as behavioral sciences (e.g., Carvalho, 2013), medicine (e.g., Bourgani, Stylios, Manis, & Georgopoulos, 2014), engineering (e.g., Vašcˇák & Reyes, 2014), management and business (e.g., Campaña, Medina, & Vila, 2014; Glykas, 2013), environment (e.g., Wildenberg, Bachhofer, Isak, & Skov, 2014), and information systems (e.g., Irani, Sharif, Kamal, & Love, 2014). Regarding to the FCMs use for educational scenarios/issues, examples of research studies (spanning from 2008 up to 2015) are tabulated in Table 1, where their objective(s) and related problem solving are listed, with the latter being the basis of their order of appearance. Table 1 reveals that most of the listed works use the FCM as a basis for modeling, decision-making, factors analysis, learning and prediction, while some of them focus on knowledge representation, derivation of progress indicators, framework setting, control and guiding, along with analysis and evaluation. The variety of these problem solving categories reveals the ample space for applicability of the FCM in the area of education. In this vein, the related findings show that: (i) insight into the context of educational software adoption in schools could be achieved, which can guide both educational decision-makers and software developers in terms of more appropriate software development efforts (Hossain & Brooks, 2008), (ii) support to the online learning community could be provided, by allowing prediction comparisons to be made between numerous tools measured by multiple factors and its relations, so decision makers can be helped to efficiently/effectively select e-learning technologies (Salmeron, 2009), (iii) causalities of the education management could be easily understood by linked graph representation (Nownaisin, Chomsuwan, & Hongkrailert, 2012), (iv) the success factors of educational organizations could better be understood (Yesil, Ozturk, Dodurka, & Sahin, 2013), (v) the assessment of learning on interactive environments could be facilitated (Barón, Crespo, Espada, & Martínez, 2014), (vi) learning style could be recognized, by handling the uncertainty and fuzziness of a learning style diagnosis in an efficient way (Georgiou & Botsios, 2008), (vii) game-based learning could be promoted (Luo, Wei, & Zhang, 2009), (viii) highly participatory scenario frameworks, which involve a blend of qualitative, semiquantitative, and quantitative methods, could be established, linking stakeholders and modelers in scenario studies (van Vliet, Kok, & Veldkamp, 2010), (ix) the domain knowledge could be represented in a more realistic way, allowing the adaptive and/or personalized tutoring system to dynamically deliver the learning material to each individual learner, taking into account his/her learning needs and his/her different learning pace (Chrysafiadi & Virvou, 2013), (x) decision-making services could be provided by an intelligent and adaptive web-based educational system, provoking learners’ traits to adapt lectures to enhance their apprenticeship (Peña-Ayala & Sossa-Azuela, 2014), (xi) critical decisionmaking in medical education could be supported, by exploring extensive ‘‘what-if’’ scenarios in case studies and preparing for

7402


dealing with critical adverse events (Georgopoulos, Chouliara, & Stylios, 2014), (xii) student’s grade evaluation and prediction the forthcoming semesters could achieved (Takács, Rudas, & Lantos, 2014), (xiii) stress factors during learning could be identified (Anusha & Ramana, 2015), (xiv) making decisions concerning networked learning could be supported from both a static and dynamic perspective (Tsadiras & Stamatis, 2008), (xv) a better understanding on students’ progress could be offered both to teachers and students via appropriate indicators (Yang, Li, & Lau, 2011), and (xvi) modeling and solving decision problems with multiple, conflicting, and interdependent attributes under complex, poorly defined and uncertain environments, such those identified in some educational contexts, is achievable (Baykasog˘lu & Gölcük, 2015).

2.2. Improving FCM performance In general, FCM has the ability to analyze and depict human perception of a given system. This is due to the inherent production of a conceptual model, which is not limited by exact values and measurements, and thus, is well suited to represent relatively unstructured knowledge and causalities expressed in imprecise forms (Glykas, 2010; Papageorgiou, 2014). Usually, there is a manual construction of FCMs, and, thus, they cannot be applied when dealing with large number of variables. In such cases, their development could be significantly affected by the limited knowledge and skills of the knowledge holder; thus, it is vital to use learning algorithms to accomplish this task (Papageorgiou, 2014). In order to improve the performance of FCMs, several methodologies have been explored; for instance, Peña, Sossa, and Gutierrez (2007) proposed a model for the design and development of ontology agents to manage rule-based FCMs (i.e., RB-FCM), taking into account the ontology agents and the FCMs represented by fuzzy rule bases. Based on a theoretic analysis, Pedrycz (2010) presented the synergy of granular computing and evolutionary optimization to design efficiently FCMs. On the other hand, Song, Miao, Roel, Shen, and Catthoor (2010) proposed a fuzzy neural network to improve the learning ability of FCMs, incorporating the inference mechanism of conventional FCMs with the determination of

membership functions and the quantification of causalities. Cai, Miao, Tan, Shen, and Li (2010) developed the Evolutionary Fuzzy Cognitive Map (E-FCM), as an extension of the FCMs. Basically, the E-FCM approach was used to model dynamic concepts and their causal relationships in the virtual world, including modeling characters and contexts. In this way, the virtual world can become more dynamic, with plausible characters/contexts, enhancing player’s experience in the interactions with the environment. Compared with the FCM, the E-FCM presented three main improvements, in particular: (i) it allowed a different update time schedule for each variable; (ii) it enabled self-mutation of the context variables, showing the dynamics of the world variables as evolving behavior; and (iii) it involved the probabilistic causality among the variables, reflecting realistic relationships among the concepts and adding more dynamics to the character. Papageorgiou (2011) proposed a methodology to design augmented FCMs combining knowledge from experts and from different data sources in the form of fuzzy rules generated from rule-based knowledge discovery methods. In another study (Acampora, Loia, & Vitiello, 2011), a FCM-based emotional agent in an Ambient Intelligence (AmI) environment was developed, in order to provide advanced services and to enhance the quality of life. In general, this approach tried to maximize the system’s usability in terms of efficiency, accuracy and emotional response. Chunying, Lu, Dong, and Ruitao (2011) proposed the Rough Cognitive Maps (RCM) as an FCM extension to model the whole set of relations between two concepts in real world, in order to improve the simulation capabilities of the FCM. In general, they concluded that RCM is more flexible and robust than FCM, simulating the real world in a better way. Finally, more recently, Štula, Stipanicˇev, and Šeric´ (2012) investigated the possibility of multi-agent systems (MAS) application in distributed computation by using FCM technique. For a thorough description of the literature review upon the FCM and its use the reader could refer to Papageorgiou (2012), Papageorgiou (2014), Papageorgiou and Salmeron (2013) and to the online FCM Community (http:// joomla.epapageorgiou.com/index.php/about-fuzzy-cognitive-maps). The way FCM is used here it is explained in the methodology section. 2.3. Contribution of the proposed approach Considering the research presented in the previous subsections, this paper contributes to the literature in the following ways:

Fig. 1. A schematic representation of a FCM model with four concepts (i.e., C1; . . . ; C4), corresponding to the interconnection weight matrix W of (1).

1. Bridges the field of FCM with the QoI of LMS users within the b-learning context. 2. Introduces a new model that extends the FL-based modeling of the QoI of the LMS users further, by providing FCM-based structural maps that reveal the causal interconnections of the important factors within the complex domain of LMS-based online learning. 3. Contributes to the QoI estimation, due to the predictive power of the proposed FCM-QoI model, taking into account concepts, dependences and trends. 4. Provides FCM structural approaches based on the time (in)dependencies, that could reveal bilateral perspectives in the evolution of the LMS users’ QoI, both at a micro and macro analysis level. 5. Evaluates the role of the influential factors of the QoI and analyzes their contribution to self-sustained cycles (static analysis) and their alterations, when the time period of the LMS use is considered (dynamic analysis). 6. Assists the decision-making of pedagogical planners to (re)adjust online tools, in order to maximize the use of the LMS Moodle within the teaching and learning practices.

7403


Fig. 2. A schematic representation of the proposed FCM-QoI model, with the LMS Moodle user’s interaction metrics (M1 ; . . . ; M 110 Þ categorized into 14 input parameters (C1; . . . ; C14Þ fed to (a) the FuzzyQoI model (Dias and Diniz, 2013) outputting the estimated QoIFIS and (b) the FCM-QoI model as input concepts interconnected with the FCMQoI as an output concept to estimate the QoIFCM . Note that the estimated QoIFIS is fed to the FCM-QoI model during the training phase only, to correctly adjust the interconnection weights of the latter towards the minimization of the error in the knowledge representation process.

7. Provides specific structural numerical data (e.g., estimated weight matrices), validated across a large amount of LMS Moodle logs under different scenarios, taking into account the user’s type (i.e., professor/student), ready to be used by other researchers in the field and/or LMS Moodle developers, exhibiting potential to be integrated within LMS users’ QoI modeling and feedback schemes, towards increased flexibility and adaptivity; and 8. Introduces a new FCM visualization tool. The aforementioned contributions are justified by the analysis presented in the subsequent sections. 3. Methodology 3.1. FCM functional structure A FCM is depicted as a fuzzy causal graph, where nodes represent concepts, whereas directed edges between the concepts denote causal relationships present between them (Kosko, 1986). Under the FCM approach, a given system is defined as a collection of concepts (e.g., events, actions, values, goals, etc.) that possess influential interconnections, expressing cause-effect relationships, which are quantified through a weighting correspondence, usually normalized to the [1, 1] interval. In the latter, positive values of weights describe promoting effect, whereas negative ones describe inhibiting effect; other values correspond to different intermediate

levels of the causal effect. A generic representation of the FCM model is presented in Fig. 1, consisting of four concepts, i.e., C1; . . . ; C4, interconnected with the following 4 4 weight matrix W:

2

0

1

0

6 0 0 6 W ¼6 4 0:5 0:1

0

1

0:2

0

0

1

3

1 7 7 7: 0:3 5

ð1Þ

0

In general, a FCM is defined using a 4-tuple fC; W; A; f g, where C ¼ fC1; C2; . . . CNg denotes the set of N concepts of the graph, W : ðCi; CjÞ ! wij is a function that associates a causal value wij 2 ½1; 1 to each pair of nodes of the connection matrix, denoting the weight of the directed edge from Ci to Cj, representing the causality degree between the interconnected concepts; hence, the weight matrix W NN gathers the system causality, usually determined by knowledge experts. Every label Ai 2 C (in the example of (1) i ¼ 1; . . . ; 4Þ is mapped to its activation value Ai 2 ½0; 1, i.e., A : ðCiÞ ! Ai , with 0 and 1 corresponding to no and full activation, respectively. The activation value of concepts is also fuzzy in nature and regularly takes values in the range [0, 1], although the scale [1, 1] is also allowed (Nápoles, Bello, & Vanhoof, 2013). In the proposed FCM-QoI model, all activation values refer to parameters that lie within the [0, 1], as they express FIS inputs/outputs from the FuzzyQoI model (Dias & Diniz, 2013). The higher the activation value, the stronger the influence of the concept over the system,

7404


which helps to understand the modeling. In fact, A is a function that associates the activation degree Ai 2 R to each concept Ci at the iteration k(k = 1, 2, . . . K). Finally, a transformation (or threshold) function f : R ! ½0; 1 is employed to keep the activation value of concept within the interval ½0; 1. The most frequently used threshold functions include the bivalent function, the trivalent function and the sigmoid variants (Bueno & Salmeron, 2009). The labels from C can be interpreted as linguistic terms that point to fuzzy sets (Glykas, 2010; Kosko, 1986), meaning that each Ai is interpreted as the value of fuzzy membership function that measures the degree in which an observed value belongs to the fuzzy set pointed by the related term. Moreover, the Ci labels (in the example of (1) i ¼ 1; . . . ; 4Þ) denote the real valued variables, with their domains assumed to be normalized into the ½0; 1 interval (Papageorgiou, 2014). For the manual construction of the FCM, the number and kind of concepts (e.g., input or output) are determined by a group of knowledge experts and each interconnection is described by either with an IF–THEN rule that infers a fuzzy linguistic variable from a determined set or with a direct fuzzy linguistic weight, which associates the relationship between the two concepts and determines the grade of causality between the two concepts. Then, all the linguistic variables suggested by the knowledge experts are aggregated using the SUM method (Kosko, 1986) and an overall linguistic weight is produced, which is then transformed into a numerical weight wij , belonging to the interval ½1; 1, by using a defuzzification method (e.g., the center of gravity (Kosko, 1986)), and, finally, a numerical weight for wij is calculated. In this way, all the weights of the FCM model are inferred. As it was mentioned above, an automated estimation of the W can be achieved, overcoming the dubious effort and potential subjectivity of the knowledge expert in his/her involvement in its definition process, by adopting learning algorithms (Hebb, 1949). Towards such endeavor, adaptive Hebbian-based learning algorithms, the evolutionary-based (e.g., genetic algorithms) and the hybrid approaches (composed of Hebbian type and genetic algorithm) (Papageorgiou, Stylios, & Groumpos, 2003; Stach, Kurgan, Pedrycz, & Reformat, 2005; Glykas, 2010; Papageorgiou, 2012, 2014), have been proposed and widely used as the most efficient methods for training FCMs. Once the FCM is constructed, it can receive data from its input concepts, perform reasoning and infer decisions as values of its output concepts. During reasoning, the FCM iteratively updates

its state by multiplying the causal weight matrix by the current state vector until convergence is reached or a stopping condition is met (e.g., the K number of iterations has been reached). Note that, in the case of the involvement of a learning algorithm within the construction of the FCM, apart from the Ai , the W matrix is also updated at each iteration.

3.2. The proposed FCM-QoI modeling approach A schematic representation of the proposed FCM-QoI model is depicted in Fig. 2. As it is clear from the Fig. 2(a), the user interacts with the LMS Moodle and 110 metrics are acquired, exactly the same as the ones used in the FuzzyQoI model (Dias & Diniz, 2013, see also Appendix A – Table A.1). These metrics are then categorized into 14 categories, denoted as C1; C2; . . . ; C14, corresponding to the 14 categories used in the FuzzyQoI model (Dias & Diniz, 2013), i.e., C1: {Journal/Wiki/Blog/Form (J/W/B/F)}, C2: {Forum/Discussion/Chat (F/D/C)}, C3: {Submission/Report/Quiz/Fe edback (S/R/Q/F)}, C4: {Course Page (CP)}, C5: {Module (M)}, C6: {Post/Activity (P/A)}, C7: {Resource/Assignment (R/A)}, C8: {Label (L)}, C9: {Upload (UP)}, C10: {Update (U)}, C11: {Assign (A)}, C12: {Edit/Delete (E/D)}, C13: {Time Period (TP)}, and C14: {Engagement Time (ET)}. These 14 concepts are considered the inputs of the FCM within the FCM-QoI model and the additional FCM-QoI concept is considered its output (see Fig. 2(b)). The FuzzyQoI model (Fig. 2(a)) outputs the QoIFIS , which is used in the training phase of the FCM within the FCM-QoI model (Fig. 2(b)). In the proposed FCM-QoI model, the Nonlinear Hebbian Rule (NHR)-based learning algorithm (Papageorgiou et al., 2003) is adopted. For the sake of completeness, a mathematical description of the adopted FCM updating process (both of Ai

Table 2 The scenarios and their phases per user (Professors, Students) involved in the FCMQoI model. Users’ Category

SCENARIOS Time dependent

Time independent

Professor

Training phase Training phase

Training phase Training phase

Student

Testing phase Testing phase

Testing phase Testing phase

Fig. 3. A schematic representation of the time-period segmentation of the examined academic year 2009/2010, spanning 51 weeks, involving three distinct periods, i.e., Lecture, Interruption (from left to right: Christmas, Carnival, Easter, Summer) and Exam, corresponded to specific weeks, for the first (soft-stems) and the second (heavystems) semester.


and W) is presented in Appendix B. During the iterative learning ðkÞ

process (k ¼ 1; 2; . . . ; K maximum iterations), the weights wji , ðkþ1Þ

, are updated, using along with the corresponding labels Ai Eqs. (A.3) and (A.5), respectively, towards the satisfaction of Eq. (A.6) (see Appendix B). When the latter is met, the FCM is considered trained and the updated weights (wup ji ) are used in the testing phase of the FCM-QoI model using Eq. (A.8), considering the satisfaction of Eq. (A.7) (see Appendix B). The fulfilment of the latter denotes the final estimation of the QoIFCM output of the FCM-QoI model. As it is clear from the aforementioned description of the FCM-QoI model, the involved FCM plays the role of a system representation, dismantling the 14 inputs-1 output relations, as represented by the estimated W up matrix. This actually reflects the

7405

expert’s knowledge representation hidden in the IF/THEN fuzzy rules of the FuzzyQoI model, yet in a more quantitative way, i.e., in the form of the interconnection weight values. The efficiency of the proposed FCM-QoI model has been evaluated through its application to the same data used in the FuzzyQoI model (Dias & Diniz, 2013), drawn from a real-life LMS Moodle use case from higher education (HE), involving both professors and students, as thoroughly described in the succeeding section. 4. Experimental validation of the FCM-QoI model 4.1. Data characteristics The LMS Moodle data for the validation of the proposed FCM-QoI model were drawn from a b-learning environment

FCM Fig. 4. (a) The estimated P QoIFCM minRMSE (dense black line), which corresponds to the iteration 291 that exhibits the minimum RMSE, and (b) the estimated P QoImaxRMSE (dense black line), which corresponds to the iteration (415) that exhibits the maximum RMSE. In both subplots, the corresponding P QoIFIS (light gray line) derived from the FuzzyQoI model (Dias and Diniz, 2013) and used for the time-dependent training phase of the FCM-QoI model is superimposed for comparison reasons. The vertical lines illustrate the time-period segmentation of Fig. 3.

7406


related to five undergraduate courses (Sport Sciences, Ergonomics, Dance, Sport Management and Psychomotor Rehabilitation) offered by the first Author’s affiliated HEI. As it was mentioned before, the data were identical to the ones used for the validation of the FuzzyQoI model (Dias & Diniz, 2013) for comparison reasons. LMS Moodle data from both professors [75 in total, 49% male, aged from 24 to 54 yrs (mean ± std = 47.19 ± 8.8 yrs)] and students [1037 in total, 45% male, aged from 18 to 48 yrs (mean ± std = 25.05 ± 5.9 yrs)] were used. All of them started to use LMS Moodle in the 2009/2010 academic year. The time-period involved spanned 51 weeks (August 26, 2009–August 18, 2010), including 610,775 LMS users’ interactions in total (94,288 from professors and 516,487 from students). The number of the daily data loggings belonging to the same category per user was summed across the duration of one week (the analysis time-unit), and all derived input variable values per week were normalized to the corresponding maximum value across the analyzed total time-period, i.e., 51 weeks, for each user. Furthermore, some distinct dates across the whole examined time-period were taken under consideration to segment the latter; this segmentation is depicted in Fig. 3, involving Lectures, Interruptions and Exams time-period sections for the first (soft-stems) and the second (heavy-stems) semesters.

and a more abstract (per academic year) perspective are examined, showing the potentialities of the FCM-QoI model at both microand macro-levels. In particular, in the time-dependent training scenario, 1000 randomized selections of the 75% of the training set were considered and, for each random selection, the mean value across the users per input/output per week was estimated, setting, in this

4.2. Training and testing dataset configuration

the initial data) to infer the QoIFCM per week and per academic year, respectively. In all cases, the evaluation of the performance of the FCM-QoI model was realized via the estimation of the Root Mean Squared Error (RMSE) and the correlation coefficient between the estimated

The whole dataset of the 14 input–1 output data from the FuzzyQoI model was randomly split into 75% as a training dataset and 25% as a testing dataset for the FCM-QoI model. The latter was kept separate from the former one, in order to test the generalization power of the FCM model. Table 2 summarizes the analysis scenarios/phases per users’ category adopted. In particular, a bilateral approach was realized via the time-dependent and time-independent scenarios, both including a training and a testing phase, accordingly. In the time-dependent scenario, the time unit of analysis of one week is taken into account, in order to obtain the best estimation QoIFCM per week, whereas in the time-independent scenario, the time unit of analysis of the whole academic year was considered, so to obtain the best estimation QoIFCM per academic year. In this way, a fine-grained (per week)

ð1Þ

way, the initial values of the Ai ; i ¼ 1; 2; . . . ; 15, per week in the updating equation of Eq. (A.5) (see Appendix B). The initial values ð0Þ

of the W matrix (wji ; j – i in (Eq. (A.3))) per week were randomly selected from the range of ½1; 1 (apparently diagonalðWÞ ¼ 0Þ. Consequently, the training process in the time-dependent scenario outputted a W up per week ðW up;w Þ. In the time-independent training scenario, the same process as in the case of the time-dependent scenario was followed, yet here, for each random selection of the 1000 randomized selections of the 75% of the training set, the mean value across the users per input/output and across the weeks was estimated. In this way, the training process in the time-independent scenario outputted a W up per academic year ðW up;y Þ. In the time-dependent and time-independent testing scenarios, the estimated W up;w and W up;y were used in the Eq. (A.8), respectively, conducted upon the initially selected testing set (25% of

QoIFCM and the QoIFIS derived from the FuzzyQoI model.

4.3. Implementation and visualization issues The implementation of the whole analysis of the FCM-QoI model was carried out in Matlab 2014a (The Mathworks, Inc., Natick, USA), using custom-made programming code. The archived data in the LMS Moodle repository were exported from .xml to .xlsm (Microsoft Excel format) and imported to the Matlab environment and archived as .mat files. The values used for the

FIS Fig. 5. The estimated professors’ P QoIFCM ðPW up;w ;testingÞ (thick grey line) during the testing phase of the time-dependent scenario, along with the corresponding P QoI testing (light black line) derived from the FuzzyQoI model (Dias and Diniz, 2013) and used for the time-dependent testing phase of the FCM-QoI model. The vertical lines illustrate the time-period segmentation of Fig. 3.

7407


updating process of the FCM were selected as the optimum ones that minimize the number of iterations for meeting the termination criteria of Eqs. (A.6) and (A.7) (see Appendix A). In particular, K ¼ 20; 000; gk ¼ 0:01; 8k (see Eq. (A.3)), fk ¼ 2:0, ltraining ¼ 1; ltesting ¼ 0:5g (see Eq. (A.4)), fv ¼ 1; rtraining ¼ 0; rtesting ¼ 0:5g (see Eq. (A.5)), e ¼ 0:001 (see Eqs. (A.6) and (A.7)). The visualization of the estimated FCM graphs was carried out via an especially programmed Matlab application, namely FCM-Viewer (see Appendix B). It should be noted that FCM-Viewer is freely available from the authors upon request. The results from the analysis of the aforementioned LMS Moodle data with the proposed FCM-QoI model scheme are described and discussed in the succeeding subsections.

comparison. Moreover, the vertical lines illustrate the time-period segmentation according to Fig. 3. As it can be seen FCM from Fig. 4, the estimated P QoIFCM minRMSE and P QoImaxRMSE exhibit

a very good phase synchronization with the P QoIFIS , denoting a successful training outcome (even in the case of the maximum RMSE, Fig. 4(b)). This is justified by the corresponding correlation coefficients, i.e., r fPQoIFCM ;PQoIFIS g ¼ 0:9311 ðp 0:01Þ and minRMSE

rfPQoIFCM

maxRMSE ;PQoI

FIS

g

¼ 0:9608 ðp 0:01Þ.

The higher correlation seen in the case of maximum RMSE, compared to the case of minimum RMSE, is justified by Fig. 4, where the P QoIFCM maxRMSE (Fig. 4(b)) follows more uniformly, compared to the P QoIFCM minRMSE (Fig. 4(a)), the phase alterations of P QoIFIS , yet exhibiting an underestimation of the true values of

5. Results and discussion 5.1. Professors’ QoI 5.1.1. Time-dependent scenario: training phase The mean RMSE between the estimated P QoIFCM and the P QoIFIS across 1000 iterations during the time-dependent training phase of the FCM-QoI model for the professors’ case (denoted by the prefix of P) was found equal to 0.0201, spanning between 0.015 and 0.024, and showing a similar behavior across the 1000 iterations. This denotes a consistency in the behavior of the FCM-QoI model during the time-dependent training phase, sustaining the RMSE in low values. The latter is reflected in Fig. 4, where the estimated P QoIFCM minRMSE (dense black line), which corresponds to the iteration (291) that exhibits the minimum RMSE, is illustrated in Fig. 4(a), whereas the estimated P QoIFCM maxRMSE (dense black line), which corresponds to the iteration (415) that exhibits the maximum RMSE, is shown in Fig. 4(b). In both subplots, the corresponding P QoIFIS (light grey line) derived from the FuzzyQoI model (Dias & Diniz, 2013) and used for the time-dependent training phase of the FCM-QoI model is superimposed to facilitate the

P QoIFIS ; this is reduced in the case of P QoIFCM minRMSE , justifying the corresponding minimum RMSE value. As it can be seen from Fig. 4(a), the P QoIFCM minRMSE shows its main phase mismatch with the P QoIFIS at weeks 17 and 20 of the first semester, whereas in the second semester it clearly follows the changes of P QoIFIS . The corresponding P W up;w matrix exhibits a degree of sparseness with sufficient balance between the negative and positive weight values, showing an increase in the latter ones around week 32 (just after the Carnival interruption at week 31). Obviously, the complexity of the 4D-matrix of P W up;w does not allow for easy visualization of the corresponding FCM per week across the academic year. Nevertheless, specific weeks in distinct time-periods are isolated and the corresponding FCMs are visualized in the forthcoming Time-Period Dependences subsection.

5.1.2. Time-dependent scenario: testing phase Fig. 5 illustrates the results from the testing phase of the time-dependent scenario of the FCM-QoI model. As it was noted before, the data used in the testing phase were totally independent from the ones involved in training phase of the time-dependent scenario. Apparently, the P W up;w was used in the construction

Fig. 6. The estimated P QoIFCM (diamonds) along with the P QoIFIS (dots), corresponding to the double averaged (across the professors and across the weeks) P QoIFIS values estimated from the FuzzyQoI model (Dias and Diniz, 2013), across the 1000 iterations of the training phase of the time-independent scenario. The solid lines correspond to the mean½P QoIFCM ¼ 0:1529 and mean½P QoIFIS ¼ 0:1768, respectively.

7408


of the FCMs per week for the estimation of the P QoIFCM ðPW up;w ;testingÞ (thick grey line), as it corresponds to the minimum RSME form the training phase of the time-dependent scenario. For comparison QoIFIS testing

reasons, the P (light black line), which corresponds to the QoI estimated from the FuzzyQoI model across the same professors that are incorporated in the construction of the testing data of FCM under the time-dependent scenario, is superimposed. The RMSE and the correlation coefficient between FIS the P QoIFCM ðPW up;w ;testingÞ and P QoI testing are equal to 0.0368 and

0.4045 (p ¼ 0:0032), respectively. Clearly, the derived RMSE is quite low, showing an efficient generalization of the FCM-QoI model in predicting the QoI after a training procedure based on historical data. The estimated correlation coefficient shows a good phase synFIS chronization of the P QoIFCM ðPW up;w ;testingÞ with P QoItesting , yet not

in all weeks across the academic year. From Fig. 5 it can be seen that the estimated P QoIFCM ðPW up;w ;testingÞ matches better the phase alterations of P QoIFIS testing after week 26, i.e., in the second semester, in comparison to the ones in the first semester, with the exception of the underestimation of the P QoIFIS testing peak at week 25. In general, however, the P QoIFCM ðPW up;w ;testingÞ is in good agreement with the P QoIFIS testing , showing a satisfactory performance of the FCM-QoI model to the QoI estimation for the case of professors.

5.1.3. Time-independent scenario: training phase The mean RMSE between the estimated P QoIFCM and the P QoIFIS across 1000 iterations during the time-independent training phase of the FCM-QoI model for the professors’ case was

Table 3 The concept interconnection weights of the FCM-QoI model corresponding to the P W up;y embedded within the FCM of Fig. 7.

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 P QoIFCM

C1

C2

C3

C4

C5

C6

C7

C8

C9

C10

C11

C12

C13

C14

P QoIFCM

0 0.13 0 0 0.13 0.19 0 0 0 0 0 0.13 0 0 0

0 0 0 0.24 0 0 0 0 0.21 0 0 0.13 0 0 0.14

0 0.13 0 0.11 0 0.1 0.17 0 0.22 0 0 0 0 0 0

0.10 0.11 0 0 0 0 0 0.11 0.13 0 0.22 0.14 0.24 0.23 0

0 0.1 0 0 0 0.11 0 0 0 0 0 0 0 0 0.11

0 0 0 0.11 0 0 0 0.16 0 0 0.24 0 0.11 0 0

0 0.12 0 0 0.11 0 0 0 0.21 0.25 0 0 0.17 0 0

0.13 0 0 0 0.23 0 0 0 0.21 0 0.11 0 0 0 0

0 0 0 0 0 0 0.11 0 0 0 0.13 0 0.1 0 0.12

0.13 0 0 0.15 0 0 0 0 0 0 0.11 0 0.14 0 0

0 0.17 0 0 0 0.14 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0.11 0

0 0 0 0 0 0 0 0 0.13 0 0 0.16 0 0 0

0 0 0 0 0 0 0.12 0.11 0 0 0 0 0.1 0 0

0.24 0 0 0.23 0.17 0.13 0.12 0 0 0 0 0.29 0 0.14 0

Fig. 7. The FCM corresponding to the interconnection weights matrix P W up;y between the 15 concepts [14 input concepts and P QoI as the output concept of the FCM-QoI model], shown in Table 1. The Ci; i ¼ 1; 4; 5; 6; 7; 12; 14, enclosed by thick circle border, correspond to the concepts that are directly connected to the output P QoI concept.


found equal to 0.024, spanning between 105 and 0.0316, and showing a similar behavior across the 1000 iterations mostly between 0.005 and 0.0316 RMSE values. Unlike the case of the time-dependent scenario, where the week is the time-analysis unit, in the case of time-independent scenario the academic year serves as the analysis unit; hence, the results of the training phase under this scenario are depicted across the 1000 iterations. This is illustrated in Fig. 6, where the estimated P QoIFCM (diamonds) along with the P QoIFIS (dots), corresponding to the double averaged (across the professors of the training phase and across the 51 weeks of the academic year) P QoIFIS values estimated from the FuzzyQoI model (Dias & Diniz, 2013), are depicted for the 1000 iterations of the training phase of the time-independent scenario. Moreover, the mean½P QoIFCM ¼ 0:1529 and mean½P QoIFIS ¼ 0:1768 are also shown as horizontal solid lines, respectively. As it can be seen from Fig. 6 and deduced from the corresponding mean values, the FCM-QoI model exhibits a kind of underestimation of the QoI values per academic year derived from the FuzzyQoI model (training set). Table 3 presents the estimated interconnection weight matrix P W up;y between the 15 concepts (14 input concepts and P QoI as the output concept) of the time-independent training phase of the FCM-QoI model, which corresponds to the iteration (597) that exhibits the minimum RMSE (105). As it is shown in Table 3, the estimated interconnection weights lie within the range of ½0:29; 0:24 and P W up;y exhibits about 73% of sparseness. This is reflected into the FCM that corresponds to P W up;y depicted in Fig. 7. In the latter, the way the 14 input concepts and the output concept (within the red border) P QoI of the FCM-QoI model are interconnected is visualized via the FCM-Viewer (see Appendix B). Note that Ci; i ¼ 1; 4; 5; 6; 7; 12; 14, enclosed by thick border, are those concepts that are directly connected to the output P QoI concept.

7409

data, the output of the FCM-QoI model, i.e., P QoIFCM , is a single value. In this vein, for the specific testing data it was found that P QoIFCM ¼ 0:1622, close enough to the P QoIFIS ¼ 0:1607. Moreover, although the time-independent scenario does not involve week-based analysis in the training phase, an effort to test the generalization power across the academic year (i.e., per week analysis) was attempted, by predicting the P QoIFCM per week, yet based on the P W up;y matrix, rather than on a P W up;w one. The results from this approach are shown in Fig. 8, which illustrates the estimated professors’ P QoIFCM ðPW up;y ;testingÞ (thick grey line) during the testing phase of the time-independent scenario, along with the corresponding P QoIFIS testing (light black line) derived from the FuzzyQoI model (Dias & Diniz, 2013) and used for the time-independent testing phase of the FCM-QoI model; the vertical lines illustrate the time-period segmentation of Fig. 3. The P W up;y matrix of Table 3 was used here, derived from the training phase of the time-independent scenario. The RMSE and the correlation coefficient between the P QoIFCM ðPW up;y ;testingÞ and P QoIFIS testing are equal to 0.0346 and 0.84 (p 0:01), respectively. The derived RMSE supports the generalization efficiency of the FCM-QoI model in predicting the QoI after a training procedure based on historical data. Moreover, the estimated correlation coefficient shows a good phase synchronization of the FIS P QoIFCM ðPW up;y ;testingÞ with P QoI testing , yet without providing a good

estimation of the amplitude of the P QoIFIS testing in all weeks across the academic year. Consequently, from Fig. 8 it can be seen that the estimated P QoIFCM ðPW up;y ;testingÞ identifies better the morphology of the P QoIFIS testing per week and provides, mainly, the trend of the latter, rather than an absolute matching of the P QoIFIS testing values. 5.2. Students’ QoI

5.1.4. Time-independent scenario: testing phase Since this scenario involves the double averaged values of the

5.2.1. Time-dependent scenario: training phase The same analysis trajectory as in the case of professors was followed for the case of students. The mean RMSE between the esti-

P QoI, i.e., P QoIFIS , of the professors participating in the testing

mated S QoIFCM and the S QoIFIS across 1000 iterations during

FIS Fig. 8. The estimated professors’ P QoIFCM ðPW up;y ;testingÞ (dense black line) during the testing phase of the time-independent scenario, along with the corresponding P QoItesting (light grey line) derived from the FuzzyQoI model (Dias and Diniz, 2013) and used for the time-independent testing phase of the FCM-QoI model. The vertical lines illustrate the time-period segmentation of Fig. 3.

7410


the time-dependent training phase of the FCM-QoI model for the students’ case (denoted by the prefix of S) was found equal to 0.013. Similarly to the case of professors, the estimated RMSE exhibits a similar behavior across the 1000 iterations, covering a range between 0.0085 and 0.0175, showing a good consistency, as justified via the estimated S QoIFCM minRMSE depicted in Fig. 9(a) as dense black line (iteration 87) that exhibits the minimum RMSE, along with the estimated S QoIFCM maxRMSE shown in Fig. 9(b) as dense black line (iteration 554) that exhibits the maximum RMSE. In both subplots, the corresponding S QoIFIS (light grey line) is superimposed to facilitate the comparison. As usually, the vertical lines illustrate the time-period segmentation according to Fig. 3. As Fig. 9 shows, FCM the estimated S QoIFCM minRMSE and S QoImaxRMSE exhibit an efficient

phase synchronization with the S QoIFIS , even in the case of the maximum RMSE (Fig. 9(b)), justified by the corresponding correlation coefficients, i.e., r fSQoIFCM ;SQoIFIS g ¼ 0:8496 ðp 0:01Þ and minRMSE

rfSQoIFCM

maxRMSE ;SQoI

FIS

g

¼ 0:8405 ðp 0:01Þ. As it can be seen from

Fig. 9(a), the S QoIFCM minRMSE shows its main phase mismatch with the S QoIFIS between weeks 2 and 16 of the first semester, whereas in the second semester it clearly follows the changes of S QoIFIS . The corresponding S W up;w matrix is characterized by some sparseness with sufficient balance between the negative and positive weight values, with the latter ones increasing around weeks 18–22 and 38–44, coinciding within the exam periods of the first and second semester, respectively. The alteration of the corresponding FCMs across specific time periods is further examined in the forthcoming Time-Period Dependences subsection. 5.2.2. Time-dependent scenario: testing phase Fig. 10 depicts the results from the testing phase of the time-dependent scenario of the FCM-QoI model for the students’

FCM Fig. 9. (a) The estimated S QoIFCM minRMSE (grey line), which corresponds to the iteration 87 of Fig. 12 that exhibits the minimum RMSE, and (b) the estimated S QoI maxRMSE (grey line), which corresponds to the iteration (554) that exhibits the maximum RMSE. In both subplots, the corresponding S QoIFIS (black line) derived from the FuzzyQoI model (Dias and Diniz, 2013) and used for the time-dependent training phase of the FCM-QoI model is superimposed for comparison reasons. The vertical lines illustrate the timeperiod segmentation of Fig. 3.


7411

FIS Fig. 10. The estimated students’ S QoIFCM ðSW up;w ;testingÞ (grey line) during the testing phase of the time-dependent scenario, along with the corresponding S QoItesting (black line) derived from the FuzzyQoI model (Dias and Diniz, 2013) and used for the time-dependent testing phase of the FCM-QoI model. The vertical lines illustrate the time-period segmentation of Fig. 3.

case. The S W up;w that corresponds to the minimum RSME form the training phase of the time-dependent scenario was used in the construction of the FCMs per week for the estimation of the S QoIFCM ðSW up;w ;testingÞ (thick black line), which compared with the superimposed S QoIFIS testing (light grey line). The RMSE and the correlation

coefficient

between

the

S QoIFCM ðSW up;w ;testingÞ

and

S QoIFIS testing are equal to 0.0264 and 0.5363 (p 0:01), respectively. Clearly, the derived RMSE is quite low (even lower

compared to the corresponding one in the professors’ case), justifying the generalization power of the FCM-QoI model, whereas the estimated correlation coefficient (higher compared to the corresponding one in the professors’ case), shows a good phase synchroFIS nization of the S QoIFCM ðSW up;w ;testingÞ with S QoI testing , yet with a

phase mismatch occurring mainly at the beginning (weeks 1–4) and the end (weeks 44–51) of the academic year. Nevertheless, the FCM-QoI model exhibits overall a satisfactory QoI estimation performance, even higher than that seen in the case of professors.

Fig. 11. The estimated S QoIFCM (diamonds) along with the S QoIFIS (dots), corresponding to the double averaged (across the students and across the weeks) S QoIFIS values estimated from the FuzzyQoI model (Dias and Diniz, 2013), across the 1000 iterations of the training phase of the time-independent scenario. The solid lines h i correspond to the mean S QoIFCM ¼ 0:131 and mean S QoIFIS ¼ 0:148, respectively.

7412


5.2.3. Time-independent scenario: training phase During the time-independent training phase of the FCM-QoI model for the students’ case, the mean RMSE between the estimated S QoIFCM and the S QoIFIS across 1000 iterations was found equal to 0.017 (range 8 106 to 0.0316; similar behavior across the 1000 iterations mostly between 0.0022 and 0.0316). Similarly to the corresponding results in the professors’ case, the estimated S QoIFCM (diamonds) along with the S QoIFIS (dots), the mean½S QoIFCM ¼ 0:131 and mean½S QoIFIS ¼ 0:148 (horizontal solid lines) are all illustrated in Fig. 11. The latter shows that the FCM-QoI model underestimates the QoI values per academic year derived from the FuzzyQoI model (training set), yet less when compared to the corresponding case of the professors (Fig. 6). This is

also

reflected

in

the

estimated

mean½S QoIFCM

and

The RMSE and the correlation coefficient between the FIS S QoIFCM ðSW up;y ;testingÞ and S QoI testing are equal to 0.0274 and

0.8047 (p 0:01), respectively. As in the case of professors, the estimated RMSE supports the generalization efficiency of the trained FCM-QoI model in predicting the QoI, exhibiting a good phase S

synchronization

QoIFIS testing ,

between

the

S QoIFCM ðSW up;y ;testingÞ

and

yet, similarly to the professors’ case, without providing

an efficient amplitude estimation of the S QoIFIS testing in all weeks across the academic year. Consequently, from Figs. 11 and 13 it can be deduced that the estimated P; S QoIFCM ðP;SW up;y ;testingÞ better identify the morphology of the corresponding P; S QoIFIS testing per week, rather than their actual amplitude, mainly providing an estimation of their trends across the whole academic year.

FIS

mean½S QoI values, since their distance is less than that between the mean½P QoIFCM and mean½P QoIFIS values. The S W up;y of the time-independent training phase of the FCM-QoI model, which corresponds to the iteration (954) that 6

exhibits the minimum RMSE (8 10 ), is tabulated in Table 4; S W up;y exhibits about 71.5% of sparseness and its weights lie within the range of ½0:25; 0:26. The corresponding FCM (visualized via the FCM-Viewer, see Appendix C) that incorporates the S W up;y is depicted in Fig. 12. From the latter, it can be seen that from the 14 input concepts only the Ci; i ¼ 2; 3; 5; 7; 14, enclosed by thick border, are directly connected to the output S QoI concept. Compared to the case of professors, the Ci; i ¼ 5; 7; 14 concepts are common to the both user’s type, with regard to their direct connection with the corresponding output concept (i.e., P; S QoI). 5.2.4. Time-independent scenario: testing phase As in the case of professors, for the specific testing data a single valued output was estimated, i.e., S QoIFCM ¼ 0:1551, comparable to the S QoIFIS ¼ 0:1480. Moreover, the generalization power of the FCM-QoI model across the academic year (i.e., per week analysis) for the case of students was tested by predicting the S QoIFCM per week, yet based on the S W up;y matrix derived during the training phase of the time-independent scenario (Table 4), rather than on a S W up;w one. The related findings are shown in Fig. 13, which illustrates the estimated students’ S QoIFCM ðSW up;y ;testingÞ (dense black line) and the corresponding S QoIFIS testing (light grey line); as before, the vertical lines illustrate the time-period segmentation of Fig. 3.

5.3. Static analysis of the derived main FCMs In an effort to further deep in the derived results, a static analysis of the derived main FCMs under the time-independent scenario for both users’ categories (Figs. 7 and 12, respectively) was carried out. As a first level of this analysis, the causal relationships between the concepts that are directly connected with the P; S QoIFCM , illustrated as bold circles in Figs. 7 and 12, respectively, are interpreted. In particular, focusing at the professors’ case (Fig. 7 and Table 3), the main concepts are C1; C4; C5; C6; C7; C12 and C14, which are directly connected with the P QoIFCM . Based on the corresponding weights (Table 3), C12: {Edit/Delete (E/D)} is the concept that negatively affects most the P QoIFCM , whereas C1: {Journal/Wiki/Blog/Form (J/W/B/F)} is the one that positively affects most the P QoIFCM , closely followed by C4: {Course Page (CP)}. From an overall perspective across the whole academic year and despite the moderate values of weights, these findings allow for some guidance to the Professors, i.e., the alterations (E/D) they perform during their interaction with the LMS should be taken with care, while the use of J/W/B/F and CP should further be encouraged. With regard to the Students’ case (Fig. 12 and Table 4), the main concepts are C2; C3; C5; C7 and C14, which are directly connected with the S QoIFCM . According to the corresponding weights of Table 4, C3: {Submission/Report/Quiz/Feedb ack (S/R/Q/F)} is the concept that negatively affects most the S QoIFCM , whereas C7: {Resource/Assignment (R/A)} is the one that positively affects most the S QoIFCM . Similarly to the case of professors, these findings allow for some guidance to the students, indicating the need for attention in the way they perform the

Table 4 The concept interconnection weights of the FCM-QoI model corresponding to the S W up;y embedded within the FCM of Fig. 12.

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 S QoIFCM

C1

C2

C3

C4

C5

C6

C7

C8

C9

C10

C11

C12

C13

C14

S QoIFCM

0 0 0 0.17 0 0 0 0.13 0 0 0 0.13 0.17 0 0

0 0 0 0 0 0 0 0 0.25 0.12 0 0 0 0.13 0

0 0.13 0 0 0 0 0 0.1 0 0 0 0 0 0 0.11

0 0 0.11 0 0.19 0 0 0 0 0.12 0.23 0 0.12 0 0

0 0 0 0 0 0.11 0 0 0 0 0 0.13 0.2 0 0.14

0.17 0 0 0.2 0 0 0.16 0.12 0 0 0 0 0 0 0

0 0 0 0.11 0 0 0 0.18 0 0 0.15 0.12 0 0 0

0 0 0 0.1 0 0 0.15 0 0 0.1 0 0.13 0 0 0

0.12 0 0.1 0.15 0 0.15 0 0 0 0 0 0.13 0 0 0.13

0.11 0.13 0 0.13 0 0.15 0.19 0.12 0 0 0 0 0.14 0.26 0

0 0.12 0 0.2 0 0 0.18 0 0 0 0 0.17 0 0 0

0 0 0.18 0 0 0 0 0 0 0.15 0 0 0.1 0 0.26

0 0 0.25 0 0.14 0 0.15 0 0.18 0 0 0 0 0 0

0 0 0 0 0 0 0.17 0 0.15 0 0 0 0 0 0

0 0.11 0.18 0 0.13 0 0.2 0 0 0 0 0 0 0.14 0


7413

Fig. 12. The FCM corresponding to the interconnection weights matrix S W up;y between the 15 concepts [14 input concepts and S QoI as the output concept of the FCMQoI model], shown in Table 2. The Ci; i ¼ 2; 3; 5; 7; 14, enclosed by thick circle border, correspond to the concepts that are directly connected to the output S QoI concept.

FIS Fig. 13. The estimated students’ S QoIFCM ðSW up;y ;testingÞ (grey line) during the testing phase of the time-independent scenario, along with the corresponding S QoItesting (black line) derived from the FuzzyQoI model (Dias and Diniz, 2013) and used for the time-independent testing phase of the FCM-QoI model. The vertical lines illustrate the timeperiod segmentation of Fig. 3.

S/R/Q/F, while their engagement with R/A should further be encouraged. However, as there are a number of interrelationships between the concepts within the FCMs of Figs. 7 and 12, there is a potential

for feedback cycles to exist within these FCMs, whereby a change in one concept can have a cumulative and/or self-sustaining effect. In this way, similarly to the work of Hossain and Brooks (2008), the second level of the static analysis adopted was based on studying

7414


the characteristics of the weighted directed graph that represent each derived FCM, using graph theory techniques (Tsadiras, Kouskouvelis, & Margaritis, 2001). In particular, the main feedback cycles were identified within the estimated FCMs of Figs. 7 and 12, structured by the concepts that are directly connected with the P; S QoIFCM , illustrated as bold circles in Figs. 7 and 12. The identified main feedback cycles are tabulated in Table 5 for both users’ categories (Professors/Students). Each identified feedback cycle was allocated a positive or negative sign (last column of Table 5), which was determined by multiplying the signs of the arcs present in each feedback cycle (Tsadiras et al., 2001). In fact, positive (negative) feedback cycle behavior is that of amplifying (counteracting) any initial change, leading to a constant increase (decrease) in case an increase is introduced to the system (Tsadiras et al., 2001). For example, the P5 cycle (Table 5, Professors’ case) involves the C6: {Post/Activity (P/A)}, P QoIFCM , C5: {Module (M)}, C1: {J/W/B/F}, and C4: {CP} concepts, showing that P/A affects P QoIFCM , which affects M, which affects J/W/B/F, which affects CP, which then affects back P/A. This cycle could interpreted as follows: the involvement of professors in P/A could affect the P QoIFCM and then lead to the engagement of M, which could evoke J/W/B/F activity in the related area; hence, involvement of CP, leading back to more P/A. This cycle exhibits a positive sign (Table 5, last column) and shows how the feedback of P QoIFCM could contribute in fueling a self-sustaining action (C6), which has a cumulative effect on other actions, i.e., C5; C1 and C4 within the derived FCM. As a second example, the S5 cycle (Table 5, Students’ case) involves the C7: {R/A}, S QoIFCM , and C12: {E/D}, implying that R/A affects the S QoIFCM , which then influences the content alterations (E/D), which could affect back the way Students could handle the R/A. The negative sign of S5 (Table 5, last column) indicates that in this cycle, the increase in R/A will not necessary lead to a self-sustained action, but as an overall, it would negatively be affected by the changes in S QoIFCM and C12; hence, careful design in the students’ R/A activities should be considered, without boosting the need for any unnecessary alterations (E/D). From an overall perspective of the cycles in Table 5, it seems that there is a balance between the number of cycles with positive and negative sign, in both users’ categories (Professors/Students). This means that the involvement of P; S QoIFCM within these cycles could affect them in a bilateral way, either by boosting or

by decreasing them. From a cross-users’ perspective, it is interesting to notice that P; S QoIFCM affect in the same way the cycles initiated by the concepts that are common between the two users’ cases (Professors/Students), i.e., P4/S4 ðC5; Þ, P6-P7/S5 ðC7; Þ, and P9/S6 (C14; þ), with the exception of S7 (C14; Þ. This finding, however, indicates that the interconnection of these common concepts with other ones and the corresponding output of the FCM-QoI model follows a similar pattern, independently of the user’s category; hence, they should be considered as factors that affect in a similar way both main stakeholders’ (i.e., Professors/Students) QoI. 5.4. Time-Period Dependences As it was shown in the previous two subsections, the structure of the FCM-QoI model supports the monitoring of the dynamics in the change of the QoI per week across the academic year, both for professors and students. This provides the opportunity to focus upon some specific time-periods (see Fig. 3) and compare the changes in the FCMs of the FCM-QoI model when shifting from one specific time-period to another. Apparently, all discontinuities in Fig. 3 could be considered as points of interest for Time-Period Dependences; yet, due to space limitations, the most profound ones that involve the main distinct periods of Fig. 3 are analyzed here. Obviously, the same approach could be easily expanded to the rest of the transitional time-periods. In particular, the week-pairs of PR1 = {15, 16}, PR2 = {17, 18}, PR3 = {22, 23} and PR4 = {37, 38} were selected, as they represent the transition from Lecture (1st semester) to Interruption (Christmas), Interruption (Christmas) to Exam (1st semester), Exam (1st semester) to Lecture (2nd semester) and Lecture (2nd semester) to Exam (2nd semester), respectively. Fig. 20 summarizes the corresponding FCMs (derived from the training phase of the time-dependent scenario) to the aforementioned week-pairs for the case of professors (Fig. 14(a), P-PR1, P-PR2, P-PR3, P-PR4) and students (Fig. 14(b), S-PR1, S-PR2, S-PR3, S-PR4), using the FCM-Viewer (see Appendix B). In all FCMs of Fig. 14, the concept interconnection weights have been omitted for better visualization of the changes in the FCMs structure across the week-pairs (note that all numerical material corresponding to the FCM-QoI model parameters are freely available upon request from the authors). Moreover, the concepts of each FCM that are directly connected to the corresponding output

Table 5 The main feedback cycles of the estimated FCMs for the cases of Professors and Students under the time-independent scenarios (see Figs. 7 and 12, respectively). Note that these feedback cycles are structured by the concepts that are directly connected with the P=S QoIFCM , illustrated as bold circles in Figs. 7 and 12, respectively; in the case of Students, however, some ‘auxiliary’ concepts (denoted within brackets) are needed to close such feedback cycles. Users Professors (Fig. 7)

Students (Fig. 12)

No.

Main Feedback Cycles FCM

Cycle Sign

P1

C1 ! P QoI

P2

C1 ! C4 ! P QoIFCM ! C5 ! C1

ðÞ negative ðÞ negative

! C5 ! C1

ðÞ negative

P3

C4 ! P QoIFCM ! C5 ! C1 ! C4

P4

C5 ! C1 ! P QoIFCM ! C5

ðÞ negative

P5

C6 ! P QoIFCM ! C5 ! C1 ! C4 ! C6

ðþÞ positive

P6

C7 ! P QoIFCM ! C5 ! C7

ðÞ negative

P7

C7 ! C14 ! C12 ! C1 ! P QoIFCM ! C5 ! C7

ðÞ negative

P8

C12 ! P QoIFCM ! C5 ! C7 ! C14 ! C12

ðþÞ positive

P9

C14 ! C12 ! P QoIFCM ! C5 ! C7 ! C14

ðþÞ positive

S1

C2 ! C3 ! S QoIFCM ! ½C9 ! C2

ðþÞ positive

S2

C2 ! S QoIFCM ! ½C9 ! C2

ðÞ negative

S3

C3 ! S QoIFCM ! ½C9 ! C2 ! C3

ðþÞ positive

S4


ðÞ negative

S5


ðÞ negative

S6

C14 ! S QoIFCM ! ½C9 ! C14

ðþÞ positive

S7

C14 ! C2 ! S QoIFCM ! C3 ! ½C9 ! C14

ðÞ negative


7415

Fig. 14. The estimated FCMs from the training phase of the time-dependent scenario that correspond to the week-pairs of (a) professors (P-PR1, P-PR2, P-PR3, P-PR4) and (b) students (S-PR1, S-PR2, S-PR3, S-PR4), visualized using the FCM-Viewer (see Appendix B). For both cases, PR1 = {15, 16}, PR2 = {17, 18}, PR3 = {22, 23} and PR4 = {37, 38}, representing the transition from Lecture (1st semester) to Interruption (Christmas), Interruption (Christmas) to Exam (1st semester), Exam (1st semester) to Lecture (2nd semester) and Lecture (2nd semester) to Exam (2nd semester), respectively. In (a) and (b), the Ci concepts with thick border represent those that are directly connected to the corresponding output concept, i.e., P; S QoIðw#Þ, where # denotes the current week. Note that in all FCMs, the interconnection weights have been omitted to facilitate the visualization of the FCMs.

(i.e., P; S QoI) are visualized as circles with thicker border than the rest ones. As it can be seen from Fig. 14(a) (professors’ case), the number of concepts that are directly connected to P QoI is decreased from six to three within the P-PR1 and increased from three to eight within the P-PR2; moreover, it is increased from four to six within P-PR3 and from two to 10 within P-PR4. In this context, the concepts C5 ({Module (M)}), C11 ({Assign (A)}) and C12 ({Edit/Delete (E/D)}) appear in five out of eight weeks of the P-PR1:PR4 week-pairs, leaving behind the rest ones that appear in one up to three weeks of the week-pairs. In an analogy, from Fig. 14(b) (students’ case), it is deduced that the number of concepts that are directly connected to S QoI is decreased from eight to three within the S-PR1 and increased from seven to 10 within the S-PR2; furthermore, it is increased from six to eight within S-PR3 and decreased from nine to six within S-PR4. Moreover, the concept C3 ({Submission/Report/Quiz/Feedback (S/R/Q/F)}) appears in seven out of eight weeks of the S-PR1:PR4 week-pairs, concepts C1 ({Journal/Wiki/Blog/Form (J/W/B/F)}) and C5 ({Module (M)}) appear in six out of eight weeks, concepts C8 {Label (L)} and C14 ({Engagement Time (ET)}) appear in five out of eight weeks, while the rest ones appear in one up to four weeks of the week-pairs. These characteristics, reflected in Fig. 14, show that time-period dependency alters the number and the type of concepts of FCM-QoI model that directly affect the P; S QoI.

More specifically, in the case of professors, it seems that entrance to Lecture or Exam time-periods increases the number of concepts that directly influence P QoI. This is also observed, in general, in the case of students, with the exception of S-PR4, where the transition from Lecture (2nd semester) to Exam time-period decreases the number of concepts that directly influence S QoI. This time-period dependency seen in the structural characteristics of the FCM-QoI model actually translates the changes in the users’ interaction with the LMS due to the temporal changes of the educational context (e.g., transition from the Lecture to Exam time-period) into role changes of the concepts involved, by enabling them to be more or less direct effective on the FCM-QoI output. Consequently, the latter is involved in more complex or simpler feedback cycles, as those examined in Section 5.3, yet in a dynamic way. This information could be useful in a dynamic design of the online part of the b-learning, by considering which concepts across time become more important and can contribute to self-sustained cycles that positively affect the users’ QoI; hence, encouragement of users’ interaction with these concepts could be promoted by the LMS and the curriculum of the b-learning, in general. Moreover, it seems that the nature of the aforementioned concepts with high appearance frequency, that have a direct impact on the estimation of P; S QoI within the examined week-pairs, comply with the corresponding role of the user’s type; that is, concepts

7416


C5, C11 and C12 refer to main activities of professors in the area of information addition and alteration (Dias & Diniz, 2013), reflecting the teaching role, whereas concepts C3, C1, C5, C8 and C14 (sorted from high to low appearance frequency) express students’ LMS interactions in the area of information view, addition and engagement (Dias & Diniz, 2013), potentially reflecting main activities of the learning process. Apparently, the structure of the proposed FCM-QoI model allows for zooming into specific time-dependent characteristics of the users’ interaction with the LMS Moodle, capturing the internal changes that appear in the dynamics of the interconnected concepts that directly influence the final P; S QoI across the academic year. 5.5. Overall perspective 5.5.1. Hypothesis justification The proposed FCM-QoI model, when placed within the panorama of the works that combine FCMs with the educational context (see Literature Review section), fills a gap that relates to the way the users interact with LMS within a b-learning context. In this perspective, it justifies the hypothesis adopted (see Introduction section), by providing an estimation of the LMS users’ QoI, complying with the findings of Hossain and Brooks (2008) and further contributing to shed light into the way LMS is used in OLEs; hence, providing valuable information to both educational decisionmakers and software developers towards more appropriate software development efforts. Moreover, the FCM-QoI model counterparts the CSFs of Salmeron (2009) and Yesil et al. (2013), providing quantitative measure (i.e., the QoI) for the LMS users’ interaction attitude, assisting to more effective categorization of the main activities that are essential towards optimization of the OLEs functionality. Similarly to the work of Nownaisin et al. (2012), the FCM-QoI model identifies the causalities within the different types of users’ interactions with the LMS that influence their QoI, presented in an easy to use and easy to understand interlinked graph representation. In accordance to the aims of Chrysafiadi and Virvou (2013), the FCM-QoI model allows dynamic monitoring of LMS users’ QoI across the academic year; hence, contributing to the perspective of OLEs from a dynamic rather static view, taking into account the alterations in the QoI of the LMS users with time. Furthermore, since the FCM-QoI model identifies the different LMS interaction attitude between professors and students, in terms of their QoI, could be combined with the findings of Peña-Ayala and Sossa-Azuela (2014), for more holistically approaching their modeling of the teaching–learning processes within OLEs. 5.5.2. Comparison with the FuzzyQoI model Apparently, the FCM-QoI model stems from the FuzzyQoI one (Dias & Diniz, 2013), since its training phase is based on the QoIFIS data provided by the FuzzyQoI model. Nevertheless, when looking from an overall perspective at these two models, the following could be identified: The knowledge representation in FuzzyQoI model was realized through 600 IF/THEN rules provided by an expert in the field (Dias & Diniz, 2013). In the case of FCM-QoI model, however, up;y this knowledge is reflected in the W up;w ½151551 or W ½1515 matrices, automatically derived after the training process. In the FuzzyQoI model, the way the estimated QoI output is interconnected with all 14 categorized LMS input metrics is not explicitly tangible, as it is implied through the 600 expert’s defined IF/THEN fuzzy rules (Dias & Diniz, 2013). On the contrary, in the FCM-QoI model, there is a direct and explicit representation of the way each one of the 14 categorized LMS

input metrics is interconnected with the others and the estimated QoI output, visualized with an easy and comprehensible way via the FCM. Unlike the FuzzyQoI model, the FCM-QoI one incorporates a training process, since it draws its inference upon QoI after learning from historical data. Dissimilarly to the FuzzyQoI model that remains the same for both professors and students, the FCM-QoI one exhibits a different structure, in terms of the concept interconnection weight values, for the professors and the students. This provides a higher degree of adaptation to the FCM-QoI model compared to the FuzzyQoI one, according to the characteristics of each user’s type. Both FCM-QoI and FuzzyQoI models share the same 14 inputs 1 output scheme, handling the LMS users’ interaction data via the same metrics categorization. This makes both models comparable in terms of data handling, offering alternative ways of approaching and analyzing the same LMS data. 5.5.3. FCM-QoI model limitations Considering the structure and the functionality of the proposed FCM-QoI model, some limitations could be identified. In particular, the need for historical data to achieve its training phase reduces its flexibility to adapt to other educational scenarios where there is a lack of available data. In these cases, however, the interconnection weight matrix could be manually formed, based on the fusion of the experts’ opinions (e.g., by forming a FL-based fusion schema). Moreover, according to the presented results, the use of the W up;w provides more efficient estimation of the QoI compared to use of the W up;y . This, however, increases the complexity of the FCM-QoI model, as the former case incorporates a ½15 15 51 matrix rather than the latter case, which incorporates just a ½15 15 matrix. Finally, the FCM-QoI model was trained using the mean values of QoIFIS across users provided by the FuzzyQoI model; this, however, merges the specific characteristics of each user to an average behavior. Apparently, following the course of each user across the academic years (or semesters), a ‘‘personalized’’ set of FCMs per user could be formed, using as training QoIFIS data the ones from the first academic year (or semester) and estimating the QoIFCM at the next academic years (or semesters). 5.5.4. FCM-QoI model implications As mentioned before, recent research has demonstrated that the users’ interactions are relatively low in the LMSs. Considering the impact and benefits/opportunities of LMSs that have been created in the last decade, it is essential to ensure their successful implementation/adoption and development. From this perspective, the proposed FCM-QoI model arising from this study, based on QoI of the LMS users, clearly constitutes a practical implication to the domain of FCM research in the HE context. In the latter, managerial implications could be traced; actually, based on the concepts and the causal dependencies, the proposed FCM-QoI model can help pedagogical planners/instructors to holistically visualize, capture, understand, and assess stakeholders’ needs and their interdependencies. In other words, the FCM-QoI approach, here, can be used as ideal mechanism/feedback to support different stakeholder groups (including department heads, teachers, administrators, technical support staff, and learners) in the domain of HE, combining knowledge by mathematically aggregating individual/collective FCM models. Moreover, the causal relationships within the model are weighted in the sense that the strength of the influence of a causal factor on an effect factor can be assigned. Objectively, the assigned strength values to causal relationships in the proposed FCM-QoI model can reveal which factors are believed to


have a greater effect on the adoption of online tools in the LMS Moodle, in a kind of simplified and global visualization for all combinations of inputs and output. Generally speaking, each concept can be examined, not only on its own values but also on the associated ones interconnected to it through the causal fuzzy weights. In this way, the analysis of a particular model can allow the expert to view the holistic picture, as the identified factors evolve, allowing the incorporation of an eclectic (strategic) perspective. Under this line, the fuzzy weights express the magnitude of change that a variable may undertake due its causal relationship with other variables. Seen as a holistic and dynamic model, the FCM-QoI approach has the potential to explore possibilities and scenarios from different perspectives; for instance, pedagogical planners and decision makers can (re)adjust online tools, towards maximum use of the LMS Moodle within the teaching and learning practices. To exemplify such capabilities, the five undergraduate courses that data come from could be considered as an application space. Although each course was not considered the unit of analysis here, as in the case of Dias, Diniz, and Hadjileontiadis (2014b), Dias, Hadjileontiadis, and Diniz (2014c), some useful general directions, instructions and directives could be identified. From one hand, as it was shown in Sections 5.3 and 5.4, some concepts contribute more than others to self-sustained feedback cycles within the FCM-QoI model, whereas the complexity of these cycles, in terms of the number of concepts involved that directly affect the FCM-QoI output, varies according to the time-period. From the other hand, the learning goals of the b-learning courses are pursued within the same time-period; hence, an optimum direction to be followed would be the adaptation of the online part of b-learning towards the boosting of these important concepts identified by the FMC-QoI model, resulting in the users’ increased QoI with the LMS, thus, contributing in their effort to achieve such learning goals. Taking for example the course of dance, from a pedagogical planning the aims are set in the construction of a place of experience and experimentation with different materials, choreographic and contextual, along with specific techniques for analysis, leading to ‘‘know-how’’ and the enlargement and consolidation of formal and expressive repertoire of the students. Moreover, it is intended that students will be able to play with fluency and accuracy through demonstration sequences danced in technical context; cooperate with colleagues in group tasks; interact with faculty and/or colleagues by actively participating in the tasks; assess their technical performance and that of others and their participation in groups (Dias, Diniz, & Hadjileontiadis, 2014c). Considering the concepts involved in the main self-sustained feedback cycles (e.g., S3 in Table 5), it could be instructed that a shift towards more interactive and appealing online resources (e.g., quizzes, discussion forums, blogs, videos, e-portfolios), triggered at specific time-periods, would increase the interest to LMS interaction, seeing the latter as a more enhanced source of information, which could accompany the face-to-face interaction and complement the effort towards multifaceted way of learning. The aforementioned exemplifies how the FCM-QoI model could attach a dynamic character to the role of LMS towards an ‘intelligent’ LMS, assisting both teachers and students to enhance the quality of the instructional environment. The main challenge in this endeavor, however, is the integration of different learning settings, different learner groups and different sources, along with the challenge to provide a dynamic and flexible interface to set up and maintain the list of external resources. In this vein, the FCM-QoI model could extend the view of the LMS within the knowledge society, which demands competencies and skills that require innovative, personalized educational practices, based on open sharing and the evaluation of ideas, fostering creativity and teamwork

7417

among the learners. Towards such endeavor, new approaches to content and network analysis, technical integration of different LMS, equipped with flexibility and adaptivity, should be considered. In addition, a large real-life database was used to illustrate the power and applicability of the method adopted; in fact, very few organizations/HEIs are far along the maturity curve in dealing with the process of discovering useful patterns and trends in large data sets (i.e., big data); however the ability to address big data is going to be the most important infrastructure change for information technology in the next decade, with major implications for knowledge management (Larose & Larose, 2014). Furthermore, a relatively good visualization of the LMS use in the form of FCM can be obtained in a short time, giving a clear ‘‘mental landscape’’ of the system and revealing some of the assumption of the ‘‘mental modeling’’ behind the stakeholders’ perspectives. This could assist the pedagogical instructors of HEIs to manage the constant change and development that we are likely to see over the next few years, not just in the application of new technologies to learning, but in the overall approach to teaching. In this vein, HEIs will need to have in place clear strategic directions. If innovation is to be a key strategy, then HEIs will need to have in place strong governance mechanisms for decision-making around the choice and use of technology, to ensure consistency and quality, while allowing as much freedom as possible for instructors to choose the technologies and learning environments that best fit the needs of their students. Given the inexorable development of new technologies, it is not really possible to isolate the HEIs from the changes going on all around it. ICT and teaching will continue to bounce off one another over time. Technology feeds and amplifies this trend towards change. These changes are not necessarily good (or bad), but HEIs, instructors and students do need strategies to prepare themselves for a continuously and rapidly changing environment for teaching and learning. The scalability of the FCM-QoI model to capture both fine-grained and more abstract characteristics of the users’ QoI with LMS, along with its adaptivity to time-dependencies, provide pedagogical instructors of HEIs with different levels of ‘analysis resolution’ of the concepts involved and their functional interconnections (causality) that promote the users’ QoI. This allows for better understanding, quantitative assessment of transitional and dynamic changes that could not be accessed before (e.g., access to and evaluation of the role of specific concepts within feedback cycles at specific time-periods), allowing for flexible regulations, targeted enhancements and directed scenarios that accommodate the needs of the stakeholders involved within the b-learning context.

6. Conclusions and future work An original approach to the estimation of the QoI of both professors and students when interacting with the LMS Moodle within a b-learning context has been proposed in the present work, introducing the FCM-QoI modeling approach. The FCM-based structure of the FCM-QoI model provided an easy way to represent LMS Moodle academic community understanding, in a form of scaled up mental modeling, as a kind of internal representation of external reality. The efficient performance of the FCM-QoI model was validated on real data, incorporating adequate number of users and LMS data logs from a HEI. Combined with the FuzzyQoI model (Dias & Diniz, 2013), the FCM-QoI one extends further the potentiality of the FL to represent the LMS users’ attitude in terms of their QoI. As an expert and intelligent system, the presented FCM-QoI model resembles the form of adaptive inferencing about QoI, since it represents a recursive ‘learning’ procedure for gradually modifying connection strengths driven by the available data,

7418


up to a satisfactory RMSE, shaping the inferential mechanisms housed by the finally estimated connection weights matrix. The latter translates the expert’s direct or indirect knowledge acquisition from observation, from sensing and, perhaps, from measuring the flux of experience. This, however, is a significantly complex task for humans, especially when multiple analysis dimensions are involved, like the ones examined here. Most decision tree induction methods used for extracting knowledge in classification problems do not deal with cognitive uncertainties, such as vagueness and ambiguity associated with human thinking and perception (Glykas, 2010). The FCM equipped with the NHR-based learning algorithm embedded within the FCM-QoI model, in turn, allows for adaptiveness and intelligence, representing the classification knowledge more naturally to the way of human thinking; hence, the FCM-QoI model constructs a manageable representation of the system that governs the LMS-based QoI and unlocks the ‘black-box’, by offering access and control to its elements (concepts), unveiling the way they influence the final output (i.e., QoI). Probing to the future, some additional aspects related to the FCM-QoI model could be considered. In particular, here, the five courses involved in the training and testing data were unified; in this way, a course effect analysis could be performed and related with the estimated FCM-QoI outputs, so to examine how the course content affects the users’ QoI. Since the structure and functionality of the proposed FCM-QoI model are not related to a specific educational context, it could easily be expanded to the analysis of LMS data from various fields, e.g., Social Sciences and/or Engineering Education (Lawton et al., 2012), exploring possible dis/similarities and correlations in the LMS users’ QoI, from an institutional perspective. Moreover, the data used here refer to the LMS Moodle usage from one year only; the FCM-QoI model, however, could also be applied to similar data from consequent academic years (e.g., 2010/2011–2013/2014), revealing possible causal dependencies, converged and/or dispersed interaction trends, all reflected at the FCM-QoI model response, in order to promote a more objective interpretation of the way LMS Moodle-based b-learning environments could be realized within the online learning context. Finally, a hybrid model (e.g., FISFCM-QoI) could also be examined, by considering the intermediate variables outputted by the FIS1–FIS3 of the FuzzyQoI model, i.e., View, Addition, Alteration, respectively, along with the two inputs to the FIS5, i.e., Time Period and Engagement Time (Dias & Diniz, 2013), as five-input

concepts to the FISFCM-QoI, keeping the QoI as its desired output. In this way, the number of concepts will be reduced from 15 of the FCM-QoI model down to six of the hybrid FISFCM-QoI one, reducing, simultaneously, the computational complexity. The exploratory study presented here aims at providing a framework for possible rethinking of the LMS-based b-learning modeling, extending its views as a more fruitful source of information about the attitude of its users’ interaction.

Acknowledgments The first author has been supported by the Foundation for Science and Technology (FCT, Portugal) (Postdoctoral Grant SFRH/BPD/496004/2013). Moreover, this work was realized within the framework of the EU FP7-ICT-2011-9-ICT-2011.8.2, under the grant agreement n° 600676: ‘‘i-Treasures’’ Project (http://i-treasures.eu). Finally, the authors would like to thank Dr. C. Ferreira of the Faculty of Human Kinetics, University of Lisbon, Portugal, for his assistance in the retrieval of the LMS Moodle data.

Appendix A. LMS Moodle metrics categorization A typical LMS Moodle includes 110 metrics that relate with the user’s activity in the system (both as ‘Action’ and combination of ‘Module’ and ‘Action’) and theoretically could all be used as useful input variables to the modeling approach, since they convey information for all the facets of the user’s activity, expressed via his/her actions in the LMS Moodle environment. Categorization of such LMS Moodle metrics into 12 basic categories, in which the summation of the metrics belonging to the same category defines the value of the corresponding variable, could significantly reduce the number of the input parameters involved in the modeling design. This approach allows all metrics to be employed in the inference process and it was initially proposed by Dias and Diniz (2013) in their FuzzyQoI model. Table A.1 tabulates these 110 metrics used as inputs to the FuzzyQoI model (Dias & Diniz, 2013) and the corresponding categories (Ci; i ¼ 1; 2; . . . ; 12Þ, also adopted by the proposed FCM-QoI model. Note that, like in (Dias & Diniz, 2013), two additional categories, i.e., C13: {Time Period (TP)}, and C14: {Engagement Time (ET)}, were also used.

Table A.1 Categorization of all LMS Moodle metrics (in their original appearance) used as inputs to the FuzzyQoI model (Dias and Diniz, 2013) to the corresponding categories (Ci; i ¼ 1; 2; . . . ; 12Þ, also adopted by the proposed FCM-QoI model. Note that, like in Dias and Diniz (2013), two additional categories, i.e., C13: {Time Period (TP)}, and C14: {Engagement Time (ET)}, were also used (not included in the list below). LMS Moodle metrics ‘Action’

Combination of ‘Module’ and ‘Action’ ‘Module’

View all View discussion View form View forum(s) View grade View graph View report View responses View submission View subscribers Preview Info Links Search Mail error

C#

Category name

4 2 1 2 3 3 3 2 3 1 4 1 1 4 2

Course Page (CP) Forum/Discussion/Chat (F/D/C) Journal/Wiki/Blog/Form (J/W/B/F) Forum/Discussion/Chat (F/D/C) Submission/Report/Quiz/Feedback Submission/Report/Quiz/Feedback Submission/Report/Quiz/Feedback Forum/Discussion/Chat (F/D/C) Submission/Report/Quiz/Feedback Journal/Wiki/Blog/Form (J/W/B/F) Course Page (CP) Journal/Wiki/Blog/Form (J/W/B/F) Journal/Wiki/Blog/Form (J/W/B/F) Course Page (CP) Forum/Discussion/Chat (F/D/C)

‘Action’

(S/R/Q/F) (S/R/Q/F) (S/R/Q/F) (S/R/Q/F)

7419


Table A.1 (continued) LMS Moodle metrics ‘Action’


Mark read Templates view Course Glossary Resource Assignment Feedback User Choice Wiki Lesson Blog Notes Quiz Scorm Data Flashchat Hotpot Chat Journal

View View View View View View View View View View View View View View View View View View

Resource Chat Forum Label Assignment Feedback Choice Calendar Wiki Glossary Data Lesson Flashchat Hotpot Journal Quiz Scorm Survey

Add Add Add Add Add Add Add Add Add Add Add Add Add Add Add Add Add Add

Add mod Add post Add entry Add discussion Add comment Choose Choose again Start Start complete End Fields add Report Report live Report log Report outline Report participation Report stats Submit Subscribe Subscribe all Talk Attempt Continue attempt User report Bogus Set page flags

Assign Unassign Enrol Unenrol Unsubscribe Upload Attachment Update

C#

Category name

2 4 4 1 4 3 3 4 3 1 1 1 1 3 3 4 2 3 2 1 5 6 7 6 6 6 6 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 7 6 6 8 6 6 6 6 6 6 5 5 6 6 6 6 5 5 11 12 11 12 12 9 9 10

Forum/Discussion/Chat (F/D/C) Course Page (CP) Course Page (CP) Journal/Wiki/Blog/Form (J/W/B/F) Course Page (CP) Submission/Report/Quiz/Feedback Submission/Report/Quiz/Feedback Course Page (CP) Submission/Report/Quiz/Feedback Journal/Wiki/Blog/Form (J/W/B/F) Journal/Wiki/Blog/Form (J/W/B/F) Journal/Wiki/Blog/Form (J/W/B/F) Journal/Wiki/Blog/Form (J/W/B/F) Submission/Report/Quiz/Feedback Submission/Report/Quiz/Feedback Course Page (CP) Forum/Discussion/Chat (F/D/C) Submission/Report/Quiz/Feedback Forum/Discussion/Chat (F/D/C) Journal/Wiki/Blog/Form (J/W/B/F) Module (M) Post/Activity (P/A) Resource/Assignment (R/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Module (M) Module (M) Module (M) Module (M) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Resource/Assignment (R/A) Post/Activity (P/A) Post/Activity (P/A) Label (L) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Module (M) Module (M) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Post/Activity (P/A) Module (M) Module (M) Assign (A) Edit/Delete (E/D) Assign (A) Edit/Delete (E/D) Edit/Delete (E/D) Upload (UP) Upload (UP) Update (U)

‘Action’

(S/R/Q/F) (S/R/Q/F) (S/R/Q/F)

(S/R/Q/F) (S/R/Q/F)

(S/R/Q/F)

(continued on next page)

7420


Table A.1 (continued) LMS Moodle metrics ‘Action’


Update comment Update entry Update feedback Update grades Update mod Update post Fields update Recent Move discussions Templates saved Edit Edit section Edit questions Delete Delete all Delete attempt Delete comment Delete discussion Delete entry Delete mod Delete post Change password Review

Appendix B. FCM parameter updating process

ðkþ1Þ

Ai

Adopting the general FCM 4-tuple fC; W; A; f g defined in Methodology section, the FCM parameter updating process involved in the proposed FCM-QoI model is realized through the Nonlinear Hebbian Learning (NHL) (Papageorgiou et al., 2003) as follows. Under the view of NHL, the value of each concept at the iteration ðk þ 1Þ is calculated taking into account the influence of the other concepts with which it is interconnected via the updated interconnection weights, i.e.: ðkþ1Þ

Ai

¼ f ðAki þ

N X ðkÞ ðkÞ Aj wji Þ;

C#

Category name

10 10 10 10 10 10 10 10 10 9 12 12 12 12 12 12 12 12 12 12 12 12 12

Update (U) Update (U) Update (U) Update (U) Update (U) Update (U) Update (U) Update (U) Update (U) Upload (UP) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D) Edit/Delete (E/D)

‘Action’

k ¼ 1; 2; . . . ; K

ðA:1Þ

¼ f ððmAki rÞ þ

N X ðkÞ ðkÞ ðmAj rÞ wji Þ;

k ¼ 1; 2; . . . ; K

where m; r are constants. The Eq. (A.5) is employed in order to remove the spurious influence of inactive concepts (with Ci ¼ 0) on other concepts, and avoids the conflicts emerge in cases where the initial values of concepts are 0 or 0.5 (Papageorgiou et al., 2012). In the FCM-QoI model, the iterative process is terminated when a criterion is satisfied. The latter depends upon the phase of the FCM, and has the form of:

Training phase criterion : jQoIFCM

ðkþ1Þ

QoIFIS

ðkþ1Þ

j–i j¼1

where N is the number of concepts involved in the FCM, K is the ðkþ1Þ

ðkÞ

concept at the iteration ðk þ 1Þ, Aj

denotes the value of the C i

ðkÞ

interconnection between C j and C i concepts at the previous iteration ðkÞ, whereas f is the sigmoid threshold function, i.e.,

1 : 1 þ ekx

ðA:2Þ ðkÞ

The updating equation of the wji

in Eq. (A.1) is given by

(Papageorgiou et al., 2003): ðkÞ

ðk1Þ

ðkÞ

ðk1Þ

þ gk Aj ðAi Aj wji

Þ;

k ¼ 1; 2; . . . ; K;

ðA:3Þ

where the coefficient gk is a very small positive scalar factor, namely learning parameter. As Tsadiras (2008) demonstrated, the inference capability of FCM may be strongly influenced by the selection of the concept transformation function. To this end, Eq. (A.2) can take the more general form of:

f ðxÞ ¼

1 ; 1 þ ekðxlÞ

Testing phase criterion : jQoIFCM

ðkþ1Þ

ðA:4Þ

where l is a constant. Moreover, as Papageorgiou, Oikonomou, and Kannappan (2012) suggest, if convenient, the Eq. (A.1) could be used in a rescaled form, like:

ðkÞ

QoIFCM j < e; k ¼ 1;2; .. .; K; ðA:7Þ

is the value of the C j concept

at the previous iteration ðkÞ, wji corresponds to the weight of the

wji ¼ wji

j < e; k ¼ 1;2;...;K; ðA:6Þ

maximum number of iterations, Ai

f ðxÞ ¼

ðA:5Þ

j–i j¼1

where eð0 < e 1Þ is a tolerance level keeping the variation of the values in Eqs. (A.6) and (A.7) as low as possible. Apparently, if Eq. (A.6) or Eq. (A.7) is not met during the iterative procedure, the ðKÞ

QoIFCM is adopted as the output of the FCM-QoI model. In the training phase, when the termination conditions are met, the final weight matrix W up : fC j ; C i g ! wup ji ; j; i ¼ 1; 2; . . . N; is derived and used as the one that expresses the concepts interconnection knowledge within the FCM of the FCM-QoI model. Consequently, the Eq. (A.5) during the testing phase of the FCM-QoI model takes the form of ðkþ1Þ

Ai

¼ f ððmAki rÞ þ

N X ðkÞ ðmAj rÞ wup ji Þ;

k ¼ 1; 2; . . . ; K: ðA:8Þ

j–i j¼1

Appendix C. The FCM-Viewer application Fig. A.1(a) illustrates the interface of the developed FCM-Viewer application. As it can be seen from the latter, a series of user-selection categories is available. In particular, the requirement for the FCM-Viewer to function is the loading of the concept interconnection weights matrix. This could be realized in two


7421

Fig. A.1. (a) The interface of the FCM-Viewer developed application with the main user-selection categories, including: loading of the weights matrix, selection of the layout type, nodes auto size and scaling, concept navigation, concept layout characteristics selection, concept settings saving, arrows characteristics selection, weights layout, default use, help recall, exporting/loading visual object handles and, finally, creation and visualization of the FCM in a new window, as shown in (b), with additional menu and node right-click functionalities of the FCM.

ways, i.e., either from a Matlab format (.mat) file or as a variable from the Matlab workspace. As soon as the weights matrix is loaded, a series of concept layout parameters becomes available.

With the navigation buttons (PREVIOUS-NEXT-GO TO), the user could navigate to each concept and alter/save its characteristics or use the defaults button that directly corresponds the default

7422


values to all variables. Moreover, the user could set the weights layout, select the FCM layout type, enable/disable nodes auto size, set the scaling factor (values greater (smaller) than 1 shrink (expand) the FCM representation), select the arrows characteristics and the weights layout. Via the HELP button, a help video is automatically triggered that visually explains all the functionalities of the FCM-Viewer. When the aforementioned parameters are set, the FCM could be created and visualized in a new window (Fig. A.1(b)) with available menu functionalities of saving, exporting, moving, zoom in/out, and node right-click functionalities, such as highlighting of the ancestors/descendants or both, along with the node properties. The visualized FCM could also be exported as a visual object handle and loaded again as a separate object (see Fig. A.1(a)-bottom), to facilitate the easy access to existing FCMs. It should be noted that the FCM-Viewer requires prior installation of the Matlab Bioinformatics Toolbox; hence, with the initiation of the FCM-Viewer application, the user is automatically informed about the existence or not of the specific toolbox in the currently running Matalb version (>R2011 version is preferable). Finally, a mouse-over help is available for each button of the FCM-Viewer, whereas a series of warnings/information messages or returns to previous states is evoked when a wrong selection is performed by the user.

References Acampora, G., Loia, V., & Vitiello, A. (2011). Distributing emotional services in ambient intelligence through cognitive agents. Service Oriented Computing and Applications, 5(1), 17–35. Anusha, G., & Ramana, P. V. (2015). A study on symptoms of stress on college students using combined disjoint block fuzzy cognitive maps (CDBFCM). International Journal of Applied Mathematics and Mechanics, 2(3), 177–182. Barón, H. B., Crespo, R. G., Espada, J. P., & Martínez, O. S. (2014). Assessment of learning in environments interactive through fuzzy cognitive maps. Soft Computing. http://dx.doi.org/10.1007/s00500-014-1313-x. Bastedo, K., & Vargas, J. (2014). Assistive technology research, practice, and theory. In B. DaCosta & S. Seok (Eds.), Assistive technology and distance learning: Making content accessible (pp. 233–251). Hershey, USA: IGI Global. _ (2015). Development of a novel multiple-attribute Baykasog˘lu, A., & Gölcük, I. decision making model via fuzzy cognitive maps and hierarchical fuzzy TOPSIS. Information Sciences, 301, 75–98. Bourgani, E., Stylios, C. D., Manis, G., & Georgopoulos, V. C. (2014). Time dependent fuzzy cognitive maps for medical diagnosis. In A. Likas, K. Blekas, & D. Kalles (Eds.), Artificial Intelligence. Methods and Applications. Lecture notes in computer science (Vol. 8445, pp. 544–554). Switzerland: Springer International Publishing. Bueno, S., & Salmeron, J. L. (2009). Benchmarking main activation functions in fuzzy cognitive maps. Expert Systems with Applications, 36, 5221–5229. Cai, Y., Miao, C., Tan, A. H., Shen, Z., & Li, B. (2010). Creating an immersive game world with evolutionary fuzzy cognitive maps. IEEE Computer Graphics and Applications, 30(2), 58–70. Campaña, J. R., Medina, J. M., & Vila, M. A. (2014). Semantic data management using _ fuzzy relational databases. In O. Pivert & S. Zadrozny (Eds.). Flexible approaches in data, information and knowledge management, studies in computational intelligence (Vol. 497, pp. 115–140). Switzerland: Springer International Publishing. Carvalho, J. P. (2013). On the semantics and the use of fuzzy cognitive maps and dynamic cognitive maps in social sciences. Fuzzy Sets and Systems, 214, 6–19. Chrysafiadi, K., & Virvou, M. (2013). A knowledge representation approach using fuzzy cognitive maps for better navigation support in an adaptive learning system. SpringerPlus, 2(1), 1–13. Chunying, Z., Lu, L., Dong, O., & Ruitao, L. (2011). Research of rough cognitive map model. In G. Shen & X. Huang (Eds.). Advanced research on electronic commerce, web application, and communication, communications in computer and information science (Vol. 144, pp. 224–229). Berlin Heidelberg: Springer. Dias, S. B., & Diniz, J. A. (2013). FuzzyQoI model: A fuzzy logic-based modelling of users’ quality of interaction with a learning management system under blended learning. Computers & Education, 69, 38–59. Dias, S. B., Diniz, J. A., & Hadjileontiadis, L. J. (2014b). Exploring B-learning scenarios using fuzzy logic-based modeling of users’ LMS quality of interaction in ergonomics and psychomotor rehabilitation academic courses. Lecture notes in computer science, Vol. 8512, 233–243. Dias, S. B., Diniz, J. A., & Hadjileontiadis, L. J. (2014a). Towards an intelligent learning management system under blended learning: trends, profiles and modelling perspectives. In J. Kacprzyk & L. C. Jain (Eds.). Intelligent systems reference library (Vol. 59). 978-3-319-02077-8. Berlin/Heidelberg: Springer-Verlag.

Dias, S. B., Hadjileontiadis, L. J., & Diniz, J. A. (2014). On enhancing blended-learning scenarios through fuzzy logic-based modeling of users’ LMS quality of interaction: The rare & contemporary dance paradigms. In Proceedings of the ninth international joint conference on computer vision, imaging and computer graphic (VISIGRAPP/IAMICH 2014) (Vol. II, pp. 765–772). Georgiou, D. A., & Botsios, S. D. (2008). Learning style recognition: A three layers fuzzy cognitive map schema. In Proceedings of the IEEE international conference on fuzzy systems, Hong Kong, China (pp. 2202–2207). Georgopoulos, V. C., Chouliara, S., & Stylios, C. D. (2014). Fuzzy cognitive map scenario-based medical decision support systems for education. In Proceedings of the 36th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC) (pp. 1813–1816). Chicago, IL: IEEE. Glykas, G. (2010). Fuzzy cognitive maps: Advances in theory, methodologies, tools and applications (1st ed.). Heidelberg: Springer-Verlag. Glykas, M. (2013). Fuzzy cognitive strategic maps in business process performance measurement. Expert Systems with Applications, 40(1), 1–14. Hebb, D. O. (1949). The organization of behavior. New York: Wiley & Sons. Hossain, S., & Brooks, L. (2008). Fuzzy cognitive map modelling educational software adoption. Computers & Education, 51(4), 1569–1588. Irani, Z., Sharif, A., Kamal, M. M., & Love, P. E. (2014). Visualising a knowledge mapping of information systems investment evaluation. Expert Systems with Applications, 41(1), 105–125. Jetter, A. J., & Sperry, R. C. (2013). Fuzzy cognitive maps for product planning: using stakeholder knowledge to achieve corporate responsibility. In Proceedings of the 46th Hawaii international conference on system sciences (HICSS), Maui, Hawaii, (pp. 925–934). Kosko, B. (1986). Fuzzy cognitive maps. International Journal of Man-Machine Studies, 24(1), 65–75. Kosko, B. (1992). Fuzzy associative memory systems. In A. Kandel (Ed.), Fuzzy expert systems (pp. 135–162). Boca Raton, FL: CRC Press. Larose, D. T., & Larose, C. D. (2014). Discovering knowledge in data: an introduction to data mining (2nd ed.). NJ, USA: John Wiley & Sons. Lawton, D., Vye, N., Bransford, J., Sanders, E., Richey, M., French, D., et al. (2012). Online learning based on essential concepts and formative assessment. Journal of Engineering Education, 101(2), 244–287. López-Pérez, M., Pérez-López, M. C., & Rodríguez-Ariza, L. (2011). Blended learning in higher education: Students’ perceptions and their relation to outcomes. Computers & Education, 56(3), 818–826. Luo, X., Wei, X., & Zhang, J. (2009). Game-based learning model using fuzzy cognitive map. Proceedings of the first ACM international workshop on multimedia technologies for distance learning, Beijing, China (pp. 67–76). Miyazoe, T., & Anderson, T. (2010). Learning outcomes and students’ perceptions of online writing: Simultaneous implementation of a forum, blog, and wiki in an EFL blended learning setting. System, 38(2), 185–199. Musbahtiti, K., & Muhammad, A. (2013). Improvement quality of LMS through application of social networking sites. International Journal of Emerging Technologies in Learning, 8(3), 48–51. Nápoles, G., Bello, R., & Vanhoof, K. (2013). Learning stability features on sigmoid Fuzzy Cognitive Maps through a Swarm Intelligence approach. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications (pp. 270–277). Berlin Heidelberg: Springer. Nownaisin, P., Chomsuwan, K., & Hongkrailert, N. (2012). Utilization of fuzzy cognitive map in modeling of Thailand science-based technology school. In Proceedings of the international conference on teaching, assessment and learning for engineering (TALE), Hong Kong, China (pp. T2D5–T2D7). Oliver, K., & Stallings, D. (2014). Blended learning: Core concepts and evaluation strategies. Journal of Technology and Teacher Education, 22(1), 57–81. Papageorgiou, E. I. (2011). A new methodology for decisions in medical informatics using fuzzy cognitive maps based on fuzzy rule-extraction techniques. Applied Soft Computing, 11(1), 500–513. Papageorgiou, E. I. (2012). Learning algorithms for fuzzy cognitive maps – A review study. IEEE Transactions on Systems, Man, and Cybernetics – Part C: Applications and Reviews, 42(2), 150–163. Papageorgiou, E. I. (Ed.). (2014). Fuzzy cognitive maps for applied sciences and engineering. Intelligent systems reference library (Vol. 54). http://dx.doi.org/ 10.1007/978-3-642-39739-4. Papageorgiou, E. I., Oikonomou, P., & Kannappan, A. (2012). Bagged nonlinear hebbian learning algorithm for fuzzy cognitive maps working on classification tasks. In I. Maglogiannis, V. Plagianakos, & I. Vlahavas (Eds.), Artificial intelligence. theories and applications. Lecture notes in computer science (Vol. 7297, pp. 157–164). Berlin Heidelberg: Springer. Papageorgiou, E. I., & Salmeron, J. L. (2013). A review of fuzzy cognitive maps research during the last decade. IEEE Transactions on Fuzzy Systems, 21(1), 66–79. Papageorgiou, E. I., Stylios, C., & Groumpos, P. (2003). Fuzzy cognitive map learning based on nonlinear Hebbian rule. In T. D. Gedeon & L. C. Fung (Eds.). AI 2003: Advances in artificial intelligence (Vol. 2903, pp. 256–268). Heidelberg: Springer. Papageorgiou, E. I., & Stylios, C. D. (2008). Fuzzy cognitive maps. In W. Pedrycz, A. Skowron, & V. Kreinovich (Eds.), Handbook of granular computing (pp. 755–774). Chichester, England: John Wiley & Son Ltd, Publication Atrium. Parker, A. (1999). Interaction in distance education: The critical conversation. AACE Journal, 1(12), 13–17. Pedrycz, W. (2010). The design of cognitive maps: A study in synergy of granular computing and evolutionary optimization. Expert Systems with Applications, 37(10), 7288–7294.

S.B. Dias et al. / Expert Systems with Applications 42 (2015) 7399–7423 Peña, A., Sossa, H., & Gutierrez, F. (2007). Ontology agent based rule base fuzzy cognitive maps. In N. T. Nguyen, A. G. Adam, R. J. Howlett, & L. C. Jain (Eds.), Agent and multi-agent systems: Technologies and applications. Lecture notes in computer science (Vol. 4496, pp. 328–337). Berlin Heidelberg: Springer. Peña-Ayala, A., & Sossa-Azuela, J. H. (2014). Decision making by rule-based fuzzy cognitive maps: an approach to implement student-centered education. In E. I. Papageorgiou (Ed.), Fuzzy cognitive maps for applied sciences and engineering. Intelligent systems reference library (Vol. 54, pp. 107–120). Berlin Heidelberg: Springer. Ping, T. A., Cheng., A. Y., & Manoharan, K. (2010). Students’ interaction in the online learning management systems: A comparative study of undergraduate and postgraduate courses. In Proceedings of the AAOU-2010 annual conference (1–14), Hanoi, Vietnam. Rodriguez-Repiso, L., Setchi, R., & Salmeron, J. L. (2007). Modelling IT projects success with fuzzy cognitive maps. Expert Systems with Applications, 32(2), 543–559. Salmeron, J. L. (2009). Augmented fuzzy cognitive maps for modelling LMS critical success factors. Knowledge-Based Systems, 22(4), 275–278. Sharif, A. M., & Irani, Z. (2006). Exploring fuzzy cognitive mapping for IS evaluation. European Journal of Operational Research, 173(3), 1175–1187. Song, H., Miao, C., Roel, W., Shen, Z., & Catthoor, F. (2010). Implementation of fuzzy cognitive maps based on fuzzy neural network and application in prediction of time series. IEEE Transactions on Fuzzy Systems, 18(2), 233–250. Stach, W., Kurgan, L., Pedrycz, W., & Reformat, M. (2005). Genetic learning of fuzzy cognitive maps. Fuzzy Sets and Systems, 153(3), 371–401. Štula, M., Stipanicˇev, D., & Šeric´, L. (2012). Multi-agent systems in distributed computation. In G. Jezic, M. Kusek, N.-T. Nguyen, R. J. Howlett, & Lakhmi C. Jain (Eds.), Agent and multi-agent systems. Technologies and applications. Lecture notes in computer science (Vol. 7327, pp. 629–637). Berlin Heidelberg: Springer. Takács, M., Rudas, I. J., & Lantos, Z. (2014). Fuzzy cognitive map for student evaluation model. In Proceedings of the international conference on system science and engineering (ICSSE) (pp. 284–288). Shanghai: IEEE.

7423

Tsadiras, A. K. (2008). Comparing the inference capabilities of binary, trivalent and sigmoid fuzzy cognitive maps. Information Science, 178, 3880–3894. Tsadiras, A., & Stamatis, D. (2008). Decisions on networked learning based on fuzzy cognitive maps. In Proceedings of the sixth international conference on networked learning (NLC2008), Halkidiki, Greece (pp. 376–383). Tsadiras, A. K., Kouskouvelis, I., & Margaritis, K. G. (2001). Making political decisions using fuzzy cognitive maps: The FYROM crisis. In Proceedings of the eigth panhellenic conference on informatics (Vol. 2, pp. 501–510). van Vliet, M., Kok, K., & Veldkamp, T. (2010). Linking stakeholders and modellers in scenario studies: The use of fuzzy cognitive maps as a communication and learning tool. Futures, 42(1), 1–14. Vašcˇák, J., & Reyes, N. H. (2014). Use and perspectives of fuzzy cognitive maps in robotics. In Fuzzy cognitive maps for applied sciences and engineering. In E. I. Papageorgiou (Ed.). Intelligent systems reference library (Vol. 54, pp. 253–266). Berlin Heidelberg: Springer. Wildenberg, M., Bachhofer, M., Isak, K. G., & Skov, F. (2014). Use and evaluation of FCM as a tool for long term socio ecological research. In Fuzzy cognitive maps for applied sciences and engineering. In E. I. Papageorgiou (Ed.). Intelligent systems reference library (Vol. 54, pp. 221–236). Berlin Heidelberg: Springer. Woltering, V., Herrler, A., Spitzer, K., & Spreckelsen, C. (2009). Blended learning positively affects students’ satisfaction and the role of the tutor in the problembased learning process: results of a mixed-method evaluation. Advances in Health Sciences Education, 14(5), 725–738. Yang, F., Li, F. W., & Lau, R. W. (2011). Fuzzy cognitive map based student progress indicators. In H. Leung, E. Popescu, Y. Cao, R. W. H. Lau, & W. Nejdl (Eds.), Advances in web-based learning (ICWL 2011). Lecture notes in computer science (Vol. 7048, pp. 174–187). Berlin Heidelberg: Springer. Yesil, E., Ozturk, C., Dodurka, M. F., & Sahin, A. (2013). Control engineering education critical success factors modeling via fuzzy cognitive maps. In Proceedings of the 12th international conference on information technology based higher education and training (ITHET), Antalya, Turkey (pp. 1–8).

Fuzzy cognitive mapping of LMS usersÐ²â¬â¢ Quality of

Fuzzy cognitive mapping of LMS usersÐ²â¬â¢ Quality of

Suggest Documents