A Survey on Dynamic Mobile Malware Detection

A Survey on Dynamic Mobile Malware Detection Ping Yan1, Zheng Yan1, 2 (*) 1

The State Key Laboratory on Integrated Services Networks, School of Cyber Engineering, Xidian University, Xi’an 710071, China 2 Department of Communications and Networking, Aalto University, Espoo 02150, Finland

[email protected],[email protected]

Abstract. The outstanding advances of mobile devices stimulate their wide usage. Since the mobile devices are coupled with third-party applications, lots of security and privacy problems are induced. However, current mobile malware detection and analysis technologies are still imperfect, ineffective and incomprehensive. Due to the specific characteristics of mobile devices such as limited resources, always network connectivity, user activities and location sensing, and local communication capability, mobile malware detection faces new challenges, especially on dynamic runtime malware detection. Many intrusions or attacks could happen after a mobile app is installed or executed. The literature still expects practical and effective dynamic malware detection approaches. In this paper, we give a thorough survey on dynamic mobile malware detection. We first introduce the definition, evolution, classification, and security threats of mobile malware. Then, we summarize a number of criteria and performance evaluation measures of mobile malware detection. Furthermore, we compare, analyze and comment existing mobile malware detection methods proposed in recent years based on the evaluation criteria and measures. Finally, we figure out open issues in this research field and motivate future research directions. Keywords: mobile malware; dynamic malware detection; security threats; classification algorithm; evaluation criteria

1

Introduction

With the fast development of concurrent software, ubiquitous networking, wireless communications, and enhanced sensing capabilities, mobile devices have rocketed lately, especially smartphones, wearable devices and portable tablets. According to the report of Gartner [27], smartphone users reached more than 655 million in China in the third quarter of 2015. Based on the statistics of mobile operating systems (OSs) in 2015, Android system accounted for 77% among them, while iOS obtained 18% ranking the second and windows ranked the third with 3%. A mobile device becomes an open concurrent software platform that can run various mobile apps developed by not only mobile device manufactures, but also many third parties. However, the third app development parties cannot ensure the security and integrity of their shipped apps.

At the same time of the fast growth of mobile apps, mobile malware is developing quickly as well. Mobile malware is a malicious program targeting mobile devices. Based on Nokia Threat Intelligence Report [30], the total growth of Android malware samples in their database was 342% in 2015, so as the growth of iOS and Windows malware. Mobile malware holds similar purposes to computer malware and intends to launch attacks to a mobile device to induce various threats, e.g., system resource occupation, user behavior surveillance, and user privacy intrusion. Mobile malware pays special attention to the typical properties of the mobile devices, such as mobility and network connectivity, to gain specific profits, e.g., tracking user trajectory, disturbing or blackmailing users via short message fraud, forcing users to pay for extra mobile service fees and disclosing user credentials. Mobile malware is evolving quickly in recent years. At beginning, Khan et al. [5] showed that mobile malware mimicked the strategies used by Personal Computer (PC) malware, which is apt to cause system corruption or divulge user private information. Later on, when multifunctional mobile devices become a mainstream in the market, mobile malware evolves accordingly. For example, with the ability of wireless networking, attackers can easily intrude a mobile device via air interfaces instead of physical connection. Mobile malware could make use of mobile devices to send premium SMS messages to increase profit and subscribe extra paid mobile services secretly [41]. In recent years, the mobile devices enhanced with sensing and networking capabilities face with novel threats and malware, which can gained a super privilege to manipulate user information, e.g., getting access to accelerometers and gyroscopes and leaking user private information to a remote server. Nowadays, malware can rely on advanced camouflage techniques to produce metamorphoses and heteromorphic versions. It also uses evade techniques to circumvent regular detection. Besides, it can broadcast itself using social networks based on social engineering attacks by making use of curiosity and credulity of mobile users, which is relatively difficult to prevent. Company Proofpoint pointed out that in 2015 users unknowingly download over two billion times regarding malicious applications with data stealing functions [28]. It is undeniable that with smart wearable devices, hospital portable devices [51] and other devices emerging, there will be more security threats targeting at mobile devices. In a word, attackers are sensitive to grasp unique mobile ecosystem opportunities to develop malware. In the literature, existing malware detection methods encompass two different approaches while collecting features: static and dynamic. The static method aims to find malicious characteristics or suspicious code segments without executing applications, such as the methods proposed by Fan et al. [17], Maetinelli et al. [52] and Su et al. [59]; while the dynamic approach focuses on collecting an app’s behaviors information and proofs during its runtime like all methods listed in Section 4. The static methods can be applied to detect known malware with high accuracy and speed. On the contrary, the dynamic methods can identify zero-day attacks companied with a relatively high false positive rate and heavy resource consumption. However, the static methods can do nothing facing with camouflage and evasion techniques or zerodays vulnerabilities. In this paper, we focus on dynamic detection of malware.

Malware detection methods and techniques are developing and evolving all the time. Current mobile malware detection and analysis technologies are still imperfect, ineffective and incomprehensive. Existing defenses in mobile devices, such as restriction of privileges and installation of virus scanners, can prevent some known intrusions and attacks. But thriving and intensive malware gets its way to devices with new methods, e.g., persisting in devices and intruding them after execution for a while [31]. This is difficult to expose and clear. Due to the specific characteristics of mobile devices such as limited resources and always connectivity, mobile malware detection introduces new challenges, especially on dynamic runtime malware detection. Many intrusions or attacks could happen after a mobile app is installed or executed or when mobile network is connected. The literature still expects practical and effective approaches that can dynamically detect malware at app runtime. This kind of detection method is also effective to figure out the metamorphoses and heteromorphic versions of malware, thus very useful in practical usage. In this paper, we give a thorough survey on dynamic mobile malware detection. We first give an overview of mobile malware with regard to its taxonomy, security threats and prosecution evidence. Then, we summarize a number of performance evaluation measures and criteria for mobile malware detection. We further compare, analyze and comment existing mobile malware detection methods based on the performance evaluation measures and criteria. We finally figure out open issues in this research field and motivate future research directions. Unlike existing surveys about malware detection, the survey presented in this paper is unique and valuable. Suarez-Tangil et al. [18] reviewed malware detection methods of smart devices published between 2010 and 2013, which covers both static and dynamic approaches. Faruki et al. [53] conducted a comprehensive survey on Android security, including Android security mechanisms and threats, penetrating methods of Android malware and their countermeasures between 2010-2014. Sufatrio et al. [54] seriously studied Android system by exploring its weaknesses and classifying existing solutions to overcome it weaknesses. Tam et al. [55] and Xu et al. [56] focused on malware detection in Android. Due to the importance to figure out mobile malware in a dynamic way at mobile app runtime, our survey focuses on the dynamic malware detection methods published from 2013 to 2017 and campares them with newly proposed criteria that were not considered in the previous work. We concern all operating systems for most portable devices. Specifically, the contributions of this paper are summarized below: • We introduce the flourishing threats and behaviors of newest mobile malware in recent years and summarize most-frequently used features in mobile malware detection; • We summarize the criteria and measures that are used for evaluating the quality and performance of mobile malware detection and apply them as a research model to study the pros and cons of the existing methods for dynamic mobile malware detection; • We review the recent advance in the literature of dynamic mobile malware detection. We compare and analyze existing methods according to the proposed criteria

and measures for evaluating their performance and discussing their advantages and disadvantages; • We further come up with future mobile malware development and the requirements of mobile malware detection to figure out open issues and motivate promising future research directions. The rest of this paper is organized as follows. Section 2 gives an overview of mobile malware, including its taxonomy, security threats and prosecution evidence. In Section 3, we summarize the criteria of dynamic mobile malware detection and the measures to evaluate detection performance. We review and compare the existing dynamic mobile malware detection methods propoed since 2013 in Section 4 based on the measures and criteria proposed in Section 3. In Section 5, we further point out a number of open issues and suggest several promising directions for future research. Finally, a conclusion is provided in the last section.

2

Overview of Mobile Malware

2.1

Taxonomy of Mobile Malware

Mobile malware can be theoretically divided into several classes according to their malicious goals and behaviors. As a supplementary condition, distribution technique is another reference standard. Normally, there are two main distribution strategies: self-propagating and social engineering. The first approach uses different strategies to automatically install malware into mobile devices, like worms, while the second one takes advantage of user curiosity and unawareness of security to allure them to manually install apps (e.g., adwares). Herein, we summarize several basic types of mobile malware based on their malicious behaviors, as shown in Table 1. Except for these mainstream categories, there are many non-mainstream malware, for example, piggybacked apps mentioned in [17] are originated from benign apps, and then become botnet or malicious apps by being injected into malicious payloads. Table 1. Taxonomy and examples of mobile malware Category

Threats

Example

Adware

Bundled with unknown software via pop-up ads or by any other means to do commercial advertisements without the permission of users; A faked antivirus protector or an app that misleads a user to pay for or download contents from networks, aiming at earning profits; Spy on any actions of mobile device users; Mobile ecosystem adaptive Trojans to achieve malicious mobile-based goals, such as Banking Trojans and SMS Trojans;

UAPush [30]

Scareware

Mobile Spyware Mobile Trojans

Koler [30] FakeFlash [30] Infostealer [31] TrojanBanker.AndroidOS.Gugi.c [47] Kasandra.B [31]

Mobile Botnet

Mobile Virus Mobile Worm

A number of inter-connected compromised mobile devices execute tasks and clear their actions based on a C&C server or by passing messages; Adapt to a mobile cellular environment and mobile device properties to help malicious programs propagate; Exploit weakness of an app or a system, spread and active themselves on their own.

NotCompatible [31]

Ghost push [32] EPOC cabir [47]

According to the malware reports of Kaspersky, McAfee, Nokia and Proofpoint [30, 31, 32, 33] in recent one year, the most popular mobile malware in accordance with market share include DangerouObject, Trojans, Backdoor, Spyware and Adware in turn. In detail, Banking Trojans rank number one in Trojan family, which is a specific trojan type aiming at mobile banking activities and services. It can collect user personal information and important credentials and save them in an unaware sector, and uploads them to a C&C server once a network connection is available. The Banking Trojans were developed with the transition of banking from personal computers to personal mobile phones. They intercept validation information and digital certificates between banks and phones to make profits. SMS Trojans rank second to lure premium SMS attack. To control SMS spam or attacks, Liang et al. [50] proposed a trust management-based SMS spam intercept system namely TruSMS. This method learns spam behaviors and SMS related traffic flow, and calculates the trust values of SMS sources to detect and control SMS spam. To classffy malware in practice, Fan et al. [23] proposed and implemented a frequent sungraph based malware classifier. It extracts common malicious behaviors for the same malware family (like Adware) to construct a function call graph based on different sensitive API calls that malware invokes. Then a TF-IDF-like method is applied to calculate the similarity of each graph, so the weights of API calls can be determined. Finally, similarity calculation can be conducted to classify malware into different families. This method helps us calssify malware and gives us a good view on how it behaves while threatening user devices. 2.2

Malware Threats and Prosecution Evidence

Mobile devices are different from personal computers in some aspects. i) Restrained resources: although the computing and storage capabilities of mobile devices have improved a lot, the capacities of their battery, RAM, CPU and memory are still very limited; ii) Mobile networking service: mobile devices are served by Mobile Network Operators (MNOs), which could cause a specific kind of attacks aiming at toll SMS and calls; iii) Interfaces: mobile devices support wireless and cellular networking and also have a Bluetooth data-exchange interface, which could be a security weakness that is easily intruded; iv) Sensing capability: there are lots of sensors on the board of a mobile device, which can provide various data related to its user, but also could be a security vulnerability. All above differences make mobile malware somehow different from computer malware. These differences should be paid attention while detecting mobile malware. In this section, we analyze and summarize main security threats

caused by mobile malware and their characteristics. We aim to figure out the corresponding features of each type of threats in order to explore mobile malware detection evidence. Threat Types and Characteristics Nowadays, attackers apply many advanced algorithms and camouflage tactics to evade detection and clearance. They also make use of the newest and commonest vulnerabilities to achieve their goals. Here we briefly review mobile malware attack types. We aim to extract the useful features that can be used as evidence while detecting mobile malware. • Hardware-based attacks. Hardware-based attacks are serious and dangerous physical attacks because once hardware is manipulated, there is no way to repair but to replace it [35]. This kind of attacks has two-fold intentions. One is that attackers attempt to use specific commands or operations to crash hardware or insert firmware. Firmware makes it easy for malware to gain super-user or administrator privileges. Thus the goal of sabotage and espionage can be easily achieved. Another intention is to modify hardware artificially to make it abnormal or even malicious. Usually this kind of attacks is latent and everlasting to execute harmful tasks for attackers. The motivations and threats caused by this kind of attacks are summarized below: ─ Creating a backdoor. The backdoor allows malware to gain super-user or administrator privileges, which helps attackers to execute in a penetrative way and perform malicious operations. ─ Intercepting Data. Intercepting data helps malware get access to secure storage and eavesdrop personal information without notice. Intercepted data are sent out through physical radiation or audio emission methods. Intercepting mobile smartcard communications and sniffing sensors are typical examples. ─ Manipulating hardware and inserting firmware. Hardware attacks try to tamper circuits slightly so that they can insert malicious firmware to elude and escape from detection. For example, a hardware attack technology “Rowhammer” overwrites one transistor in DRAM to force bits to change [35]. ─ Inducing normal behaviors suspended. Sometimes, infected devices halt or cannot do any operations due to hardware attacks. Such as, any successive requests and operations could cause device crash. Mobile applications for healthcare, as proposed by Carlos et al. [51], are easy to be halted by a third party once the mobile device suffers from hardware attack. ─ Cloning hardware and services. By breaking official services, attackers replicate hardware and services to prepare for malicious actions. • Software-based attacks. Software-based attacks are traditional and popular ways to attack mobile devices as what computer malware does [18]. These attacks make use of existing mobile software to achieve malicious goals. Due to the flourish of third-party mobile app markets, especially for Android devices, attackers modify official and benign apps into malware and upload to the market for download. Lots

of scareware, Banking Trojans and spyware apply this method with advanced technologies to infect victims. The main motivations of software-based attacks are listed below: ─ Intercepting Data. Similar to eavesdropping data in hardware attacks, softwarebased attacks could monitor and steal user private information and send it to a malicious C&C server. In this process, malware colludes with other apps to build up mobile apps collusion, i.e., several apps cooperate to steal, store and send data via inter-app communications such as covert channels [12], data interchange, etc. ─ Creating botnet. Cybercriminals recruit and organize compromised mobile devices into a group to help them share attack payload. They always launch attacks without user attention, such as sending spam, stealing information, overloading servers, doing billing fraud, performing Advanced Persistence Threats (APT) attacks, etc. For example, personate Android app Android.DDoS.1.origin uses worldwide compromised mobile phones to launch DDoS attacks [33]. ─ Gaining pecuniary benefits. Software-based attacks could cause abnormal behaviors of apps, e.g., sending premium SMS messages and inducing apps to click chargeable fraud links. Some even blackmail victims with collected sensitive data or through the vulnerabilities of devices. • Firmware-based attacks. Firmware is the program stored in a non-volatile rewritable memory. Its functions provide basic but significant support for electric smart devices, such as providing connection interfaces, controlling functions and so on. Firmware-based attacks happen to all the devices containing the firmware [5], such as mobile devices, printers, personal portable assistants, wearable devices and even industrial systems and Europay Mastercard Visa (EMV) cards. They modify or change programs so that they can crash or control devices [31]. For example, many attackers exploit EMV smart cards’ flaws to attack their users [32]. The main purposes of firmware-based attacks are listed below: ─ Obtaining control privilege. For example, boot integrity attacks can change a boot program and help attackers gain the control privilege of a victim system. E.g., Infostealer can obtain and breach user information easily with the help of firmwarebased attacks [30]. ─ Creating backdoors. Not only in software and hardware, firmware can be inserted backdoors as well. • Infrastructure and protocol attacks. Actually, many kinds of attacks aim at communication protocols and basic infrastructures. By taking the advantages of vulnerabilities founded in communication protocols and basic infrastructures, attackers can introduce troubles to mobile devices. E.g., the weakness of Mobility Management Entity (MME) infrastructure could drain battery fast. The main motivations of infrastructure and protocol attacks are: ─ Network attacks. By making use of the vulnerabilities of existing protocols and infrastructures, attackers can launch network attacks, such as DoS/DDoS attacks,

source address and DNS spoofing, etc. They are prone to break down and cheat network servers. Synchronize flood attack is a typical example. ─ Intercepting information and gaining pecuniary benefits. Man in the Middle (MITM) attacks like IMSI catcher personate a proxy server to hijack sessions, monitor network traffic and sniff and modify session contents. Bai et al. [14] mentioned that the Apple ZeroConf property could incur MITM attacks, for example, attackers make use of QQ or AirDrop to intercept messages and information. What is more, MITM attackers may pretend to dial calls to cause huge cost of victims. MITM attacks cause the most dangerous security threat to online games and online banking. ─ Sniffing infrastructure. Attackers analyze the features of a system through user feedback, which helps them launch corresponding attacks. • Other attacks. Except for aforementioned attacks, there are many other kinds of attacks, for instance, social engineering based attacks. Social engineering utilizes human curiosity and human nature to cause cognition biases while making decision. Many types of attacks are generated based on this, such as pishing, pretexting and diversion theft. Pretexting creates a faked scenario and gain victims’ trust, and then utilize psychological manipulation to get access to user sensitive data like password or bank account details. Diversion theft means that hackers entice a third legitimate party to deliver malicious packages to wherever consignments request. This technique is widely used in fraud, and is easy to implement, but difficult to prevent. It is estimated that hardware-based attacks [35] and social engineering attacks [27] could dominate the mobile malware filed. The first type of attacks is developing fast and fetal to victims. The second type is easy to implement, because humanity is full of curiosity and greed, which could be made used by malicious attackers. Features Used as Prosecution Evidence Hardware

Battery, CPU, Memory, I/O Interfaces, Sensors, Camera, Screen, IMEI ···

Software

Permissions, Privileges, Network Traffic, Data Access, Information Flow, Covert Channel, Package ···

Firmware

Operating System, System Calls, IPL, API, Library, Pre-defined Function ···

Others

Ibt, Topology Graph, Protocols, Inconsistency Semantic Content, Privilege Escalation ···

Fig. 1. Features used as prosecution evidence

According to the analysis of threats and their characteristics, in this part, we summarize the specific features that can be used in dynamic mobile malware detection in order to show the existence of mobile malware and malicious behaviors. Dynamic mobile malware detection focuses on investigating the interaction between suspicious apps and operating systems or the Internet. Only if the mobile malware wants to achieve its malicious goals, its interaction with system resources and network access could happen. Refer to Fig. 1, we divide the specific features of mobile malware into a number of categories as described below. • Hardware features. Hardware components are the foundation of a device. Each operation needs them to support. Hardware features include device information (e.g., IMEI number) and the states of hardware components, such as battery usage applied in [19] to reveal SMS and location malware, and battery, memory, and CPU related features used in [26] and [38] to detect covertly operations of malware. I/O interfaces and sensor data availability can help examine app privileges. Except for these, the hardware features also include the speed of device battery draining that can expose the running of malware and its occupation of CPU to reveal covert malicious behaviors [12]. In addition, we can apply the frequency of memory reading and writing to show the information exchange between the operating system and apps [3,4], and use the data about access to camera, screen and sensor to examine whether apps own privileges to do so. • Software features. The malicious behaviors of apps are the most important evidence to detect malware, especially at app runtime. In general, software features include apps behaviors. For example, in [3, 4, 11, 15], the permission requests and privilege acquirement of an app are monitored to detect privacy leakage and abuse of permissions. Network traffic is another evidence since some malware makes use of a network to transmit privacy data to a remote server. Spying on network packages was proposed in [1, 6] to expose this malicious behavior. Additionaly, the usage of phone calls and SMS services, and information flows between or within apps are applied to detect abnormal information exchange behaviors [7, 20]. This is because some software colludes with each other to steal and share private information with covert channels or by other means [9]. • Firmware features. Firmware is the most rudimentary supportive ‘software’ in a mobile device. It provides cooperation and communication means for software and hardware, such as the operating systems and Initial Program Loader (IPL) in smartphones. All mobile devices own their own firmware. In those programs, there are many features used in malware detection. For example, Wei et al. designed a cross-platform detection method based on underlying libraries [44]. API interfaces and system calls are the most frequently used features in the current existing methods [15-17, 21, 22]. The patterns of the system calls of malware and the predefined functions being manipulated by malware can be applied to contribute to dynamic malware detection. • Other features. Except for the aforementioned features, there are some other features that are very useful for mobile malware detection, e.g., suspicious server access (e.g., a C&C server connection) [12], irrelevant bad terms (ibt) [45], Topolo-

gy Graph (TG) [10], inconsistent semantic contents comparison [13], privilege escalation [46], and so on.

3

Detection Performance Evaluation Measures and Criteria

In order to protect mobile devices and resist threats mentioned in Section 2, we summarize the requirements that a mobile malware detection method should satisfy and the measures for evaluating detection performance. Criteria are the requirements that a malware detection method should satisfy. Measures refer to the evaluation metrics that is used to evaluate the performance of a detection method. Both of them are needed for commenting and evaluating a mobile malware method. We can improve and optimize the method by using the criteria and measures to achieve high performance and reach design expectation. 3.1

Criteria of Dynamic Mobile Malware Detction

A number of criteria should be applied to assess the performance of a dynamic mobile malware detection method. We need to consider many aspects with regard to detection performance requirements, such as the complexity of a detection algorithm, applicable platforms and ability of adaption, etc. Herein, we propose a number of emphasized criteria as described below: • Detection accuracy. Detection rate or detection accuracy is a basic criterion to evaluate the performance of a method. The higher the detection rate, the better the method performs. In section 3.2, we summarize a number of measures that are used to measure the detection accuracy. • Real-time detection support. Real-time detection refers to that a detector can identify mobile malware by collecting and analyzing app runtime data continuously without impacting app execution performance. An ideal solution is that the identification and analysis are performed inside the mobile device. The detection should be fast and efficient. Constrained by the energy and storage in mobile devices, most of the existing methods outsource collected data to remote servers or a cloud platform to perform malware detection. Obviously, malware detection based on this method is delayed, which could cause serious loss. Therefore, real-time detection support becomes crucial in dynamic mobile malware detection. • Privacy preservation. As mentioned above, outsourcing data from a mobile device to a third party for malware detection could induce information leakage or malicious data manipulation. It may also intrude the privacy of mobile device users. In [8], the authors proved that the outsourcing method could greatly save mobile device energy. However, preserving user privacy and the security of data because essential in this kind of methods. • Ability to identify unknown apps. A dynamic mobile malware detection method should be able to identify unknown apps and zero-day attacks.

• Economic resource consumption. Resource consumption is a big problem in mobile malware detection. Resource-intensive methods cannot be widely used in practice even with a high detection rate. • Proper choice of classification algorithms. Classification algorithms used to identity mobile malware influence the efficiency and accuracy of a detection method. Previous work proved that the performance of different classification algorithms is quite different with regard to malware detection [2-4, 38]. Thus, properly choosing or designing a classification algorithm could impact detection performance. 3.2

Measures of Detection Performance

In the statistics, there are many measures that can be used to evaluate the performance of a detection method in order to show its detection accuracy, as summarized in Table 2. Table 2. Measures used for detection performance evaluation Measures True Positive (TP) False Positive (FP) True Negative (TN) False Negative (FN) Recall: True Positive Rate (TPR) True Negative Rate (TNR)

Explanation Malicious programs correctly identified as malicious, i.e., True Positive = correctly identified; Benign programs incorrectly identified as malicious, i.e., False Positive = incorrectly identified; Benign programs correctly identified as benign, i.e., True Negative = correctly rejected; Malicious programs incorrectly identified as benign, i.e., False Negative = incorrectly rejected; 𝑇𝑃𝑅 =

𝐹𝑃𝑅 =

Precision: Positive Prediction Value

𝑃=

Accuracy = Detection Rate Receiver Operating Characteristic (ROC) Area Under the Curve (AUC)

, i.e., sensitivity or recall means the benefits we gain;

!"

𝑇𝑁𝑅 =

False Positive Rate (FPR)

F-score (F-measure)

!" !"!!"

!"!!" !" !"!!"

!" !"!!"

= 1 − 𝐹𝑃𝑅, i.e., specificity means the costs we spend;

, i.e., false alarm rate;

;

 𝐹 − 𝑠𝑐𝑜𝑟𝑒 = 𝐹 − 𝑚𝑒𝑎𝑠𝑢𝑟𝑒 =

!!! ! !×!"# ! ! !!!"#

, 𝛼 is a pre-defined parame-

ter;  𝐴𝑐𝑐 =

!"!!" !"!!"!!"!!"

;

Defined by FPR and TPR as x and y axes respectively, it help us determine trade-offs between True Positive and False Positive, in other words, the benefits and costs;  𝐴𝑈𝐶 =

!! 𝑇𝑃𝑅 !!

𝑡 𝐹𝑃𝑅(𝑡)𝑑𝑡.

Sensitivity and specificity are statistical measurements. TPR, the hit rate, represents sensitivity that stands for the proportion of correctly identified positives. TNR represents specificity that measures the proportion of correctly identified negatives.

Besides, Detection Rate (DR) is the accuracy that represents the correct classification rate of all detection samples. Precision and recall are not adequate for showing the performance of detection even contradictory to each other, because they do not include all the results and samples in their formula. F-score (i.e., F-measure) is then calculated based on precision and recall in order to compensate this disadvantage. Receiver Operating Characteristic (ROC) is a statistical plot that depicts a binary detection performance while its discrimination threshold setting is changeable. The ROC space is supposed by FPR and TPR as x and y axes, respectively. It helps us determine trade-offs between TP and FP, in other words, the benefits and costs. Since TPR and FPR are equivalent to sensitivity and (1-specificity) respectively, each prediction result represents one point in the ROC space. The point in the upper left corner or coordinate (0, 1) of the ROC curve stands for the best detection result, representing 100% sensitivity and 100% specificity. This point is also called perfect detection. An Area Under the Curve (AUC) is usually between 0.5-1.0, the bigger, the better the detection is. Different measurements could be contradictory with each other. It is hard to meet with high precision and recall at the same time. We need to make a trade-off to balance them. Thus, F-measure i.e., F-score is often used to indicate detection performance.

4

Dynamic Mobile Malware Detection Methods

In this section, we review dynamic mobile malware detection methods that have been proposed since 2013. We mainly retrospect related papers from several databases: ACM, Springer, IEEE Explorer libraries, Elsevier Science and ScienceDirect, by searching key word: dynamic malware detction, malware classifier, mobile malware, smartphone malware, Android malware detection and so on. It is a challenge to detect malware dynamically in mobile devices, especially, when malware designers use encryption algorithms and evading strategies [42]. In general, the evolution of detecting mobile malware is stimulated by the techniques used by mobile malware. From static code segments analysis, to semantic analysis; from system calls to calling graph, from codes to behaviors, the detection techniques of mobile malware progress fastly. We organize this section as follows: we briefly divide the dynamic detection methods into three types: Anomaly-based Detection, Specification-based Detection, and Signature-based Detection according to the type of identification. For each type, we review the current literature and perform a comprehensive comparison, as shown in Table 4, in terms of target operating systems, technique categories of detection, applied classification algorithms, the features used in detection, the place of detection analysis, real-time detection support, privacy preservation support, threats that can be revealed, detection performance evaluation measures, and performance test results. We also comment their pros and cons accordingly. Before the literature review on the dynamic detection methods, we first briefly introduce classification algorithms that

play an important role in malware detection by classifying app samples into such classes as malicious or benign. 4.1

Classification Algorithms

Classification algorithms aim at classifying unknown samples with proper labels, such as malicious or benign. They serve as the most essential part of malware detection together with the features. The most popular technique used in classification is machine learning, accompanied with data mining methods. Data mining mainly uses statistical methods and programming methods to find patterns of features, which can be applied into machine learning to build classification models. Most classification algorithms fall into the scope of machine learning. Bazrafshan et al. [29] reviewed several heuristic malware detection techniques based on various classification algorithms. In Table 3, we briefly introduce several popular machine-learning-based classification algorithms. For deep and comprehensive understanding on them, refer to [36, 37, 38]. Choosing a proper classification algorithm according to the goal of detection is very important since it impacts detection accuracy and performance. When we have a small training set, we normally choose Naive Bayes or K means. If we need to observe accurate probabilities of each class, it is better to choose Logistic Regression. If we have many types of features to consider, Random Forest could be a good candidate. Notably, lots of experiments are normally needed for setting proper parameters for achieving the most effective results. The algorithms such as K-means, K Nearest Neighbors (KNN), and Random Forest, need to predefine some parameters for detecting malware. And the values of parameters greatly influence detection performance. Table 3. Frequently used classification algorithms Algorithms

Advantages

Disadvantages

Naive Bayes

High speed, insensitive to irrelevant feature data, simple and mature algorithms; High precision and accuracy, nonlinear classification, no assumption of features; Irrelevant feature dealing capability, high precision, training sample can be small; High accuracy and speedy, deal with high dimensional and non-linear problems; Overcome overfitting, deal with high dimensional data; Rapid to converge, self-optimize capability;

Require assumption of mutual independence of features;

K Nearest Neighbors Decision Trees

Support Vector Machine-SVM Random Forest K-means

Sensitive to unbalanced sample set, heavy computing burden; Easy to overfitting;

Heavy storage burden, require to properly choose kernel function and repeatly call parameters; Accuracy depends on tree number, sensitive to unbalanced sample set. Required to predefine value of K, sensitive to outlier;

Adaboosting Logistic Regression C4.5 (=J48)

4.2

High precision, overcome overfitting, simple; Nice probabilistic interpretation, model updating capability, high speed; Easy to understand generated rules, high accuracy, overcome overfitting.

Sensitive to outlier; Prone to overfitting, weak to handle high dimensional data; Multi-visit to data lead to low efficiency.

Dynamic Anomaly-based Detection

Anomaly-based detection aims at building a model that contains apps’ normal behaviors, which are used to classify an app’s maliciousness or benignancy. This technique has potential to detect previously unseen malware accompanied by a high omission rate, but it is a classical method. To ensure correctly recognize normal behaviors and identify samples, the anomalybased detection approach contains two phases: the training phase where a profile model of normal behaviors is built according to the normal behaviors of a host and inspection apps; the testing phase where the behavior information monitored during the execution of query samples is compared with the profile model. In the process, machine learning algorithms can be applied to detect anomalies, such as artificial intelligence algorithms, data mining methods and so on. Suarez-Tangil et al. [8] suspected whether the cloud or server based detection methods (i.e., outsourcing detection) is more power saving than on-device detection methods (i.e., detecting malware at mobile devices). They proposed a power model to compare the energy consumption of both types of methods. They testified 3 classification algorithms: J48, Naïve Bayes and K-means. Regarding the effectiveness of the three algorithms. J48 turned out to be time-saving and more effective than the other two algorithms. Particularly, they found that the outsourcing detection method is a better choice to save power. But performing malware detection outside a personal device could cause a series of user privacy and data security issues. Cui et al. [2] implemented a real-time mobile malware detection system based on cloud computing. It analyzes network packets by applying data mining and machine learning algorithms. The system first captures packets that a mobile device sends at an operator gateway and uses several mining algorithms to cut down subordinate branches of packets according to back up knowledge. Then, it sends left packages to a core model called SMMDS to identify malware. Only if the packet is malicious, it will be added into a blacklist as prior knowledge. SMMDS applies decision trees, K-means, center distance and Naive Bayes to learn sample features. The whole detection process is conducted in a cloud server. Detection results are uploaded to a call center, which allows an operator to automatically send its subscribed mobile devices an alarm message to alert users. The advantage of this implementation is it makes use of cloud computing to save local resources of mobile devices and applies data mining methods to decrease the size of data sets. But its shortcomings are also obvious. High accuracy accompanies with a high false alarm rate if they do not properly choose algorithms. Only the method based on decision trees performs well with 99% accura-

cy and 2.5% false alarm rate. Notably, it is essential to choose a proper classification algorithm in order to achieve good detection performance. Table 4. Comparison of dynamic mobile malware detection methods Ref

Technique categories, classification algorithms, features and measures used, performance, pros and cons, etc. Plat

ToD

CA

Fea

P.A

R

PP

Threat

Ser

×

×

All kinds

Mea

Value

Pros & Cons

Acc

85.94%

Apply popular features, low accuracy, no privacy protection, heavy computation burden.

[1]

All

SIG

NB

Net. B

[2]

All

A

DT, NB, K.m

Pac

Clo

×

×

Botnet, malicious networking

Acc, FPR

99%, 2.5%

Alleviate computation burden, high accuracy dependent on algorithms, no privacy preservation.

[3]

And

SIG

A simple system, no exact criteria to evaluate the system, it is not complete and unable to detect new malware families.

[4]

And

SPE

[6]

All

[7]

All

K.m, NB NB, J48, SVM

Per

Dev, Ser

×

×

Abuse app permission

N/A

N/A

Per, API SC

Ser

×

×

Privacy leakage

TPR, Fscore

98.4%, 0.983

A

N/A

Pac

Ser

×

×

Privacy leakage

N/A

N/A

A simple software-based method, ineffective without evaluation, no privacy protection.

SPE

DT

IF

Ser

√

×

Privacy leakage

N/A

N/A

Android virtual execution for real-time detection, high resource consumption.

Bat

Ser

×

×

Privacy concern

N/A

N/A

Aware of privacy, testify effectiveness of outsourcing methods, without new idea, no solution to privacy protection.

A novel fusion method and good performance, heavy computation burden.

[8]

All

A

J48, NB, k.m

[20]

And

SPE

N/A

IF

Dev, Ser

×

×

All kinds

TPR

100%

Sample set is too small, high TPR, extra human operation needed.

[21]

And

SIG

M

API, SC

Ser

×

×

All kinds

Acc

96%

Unable to list all sensitive behaviors, but high detection accuracy, collected data set is big.

[12]

And

SPE

NEU, DT

CC

Ser

×

×

Malware collusion

Acc

>90%

Novel features and methods used to detect malware, but the features seem not sufficient to reveal all malware.

[11]

All

A

Gau

Bat CPU Me

Ser

×

×

Anomaly behaviors

AUC

~1

Monitor apps behaviors, i.e., hardware features to construct a Gaussian mixture model to detect malware with high AUC, but heavy computation burden.

[19]

And

A

M

Bat

Dev, Ser

×

×

All kinds

FPR