Merging Permission and API Features for Android Malware Detection

2016 5th IIAI International Congress on Advanced Applied Informatics

Merging Permission and API Features for Android Malware Detection Mengyu Qiao1, Andrew H. Sung2, and Qingzhong Liu3 1

Department of Mathematics and Computer Science, South Dakota School of Mines and Technology, Rapid City, SD 57701, USA 2 School of Computing, The University of Southern Mississippi, Hattiesburg, MS 39406, USA 3 Department of Computer Science, Sam Houston State University, Huntsville, TX 77341, USA Android is on the majority of smartphones in most countries in the world with a share of 82.8% [2]. As of July 2015, Google Play is primary and official Android application store, which provides about 1.6 million apps for Android users [3], and has reached about 50 billion application downloads in the past 12 months [4].

Abstract—the prosperity of mobile devices have been rapidly and drastically reforming the use pattern and of user habits with computing devices. Android, the most popular mobile operating system, has a privilege-separated security system through a sophisticated permission control mechanism. Android Apps need to request permissions to access sensitive personal data and system resources, but empirical studies have found that various types of malicious software could obtain permissions and attack systems and applications by deceiving users and the security mechanism. In this paper, we propose a novel machine learning approach to detect malware by mining the patterns of Permissions and API Function Calls acquired and used by Android Apps. Based on static analysis of source code and resource files of Android Apps, binary and numerical features are extracted for qualitative and quantitative evaluation. Feature selection methods are applied to reduce the feature dimension and enhance the efficiency. Different machine learning methods, including Support Vector Machines, Random Forest and Neural Networks, are applied and compared in classification. The experimental results show that the proposed approach delivers accurate detection of Android malware. We deem that the proposed approach could help raise users’ awareness of potential risks and mitigate malware threats for Android devices.

Android is a privilege-based operating system with security provided through a complex permission mechanism. Android apps need to request permissions to access sensitive personal data and system resources through privacy- and security-related API. Android require each Application to explicitly specify upfront what permissions it requires to access hardware, software, and data sources. Developers must define the permissions that their applications will use in a configuration file, which will be reviewed and approved by the user during Application installation. This permission mechanism gives the user the power to control integrity and privacy of their Android devices. If a user does not want to grant a permission to an application, he or she can cancel the installation process. From the study on permission evolution [5], researchers found that the set of Android permissions is growing. Rather than providing finer grained permissions, the growth aims at offering access to new hardware features, which makes the permission mechanism hard to understand and use by ordinary users. Empirical studies have found that various types of malicious software could obtain permissions and attack systems and applications by deceiving users and security mechanism,

Keywords— Android App, Android Permission, Malware, Risk Assessment, Support Vector Machines, Neural Networks, Random Forest

I.

INTRODUCTION

The popularity of the Android devices and the fast growing mobile application market have attracted the malware developers. Recent research shows that the number of mobile malware in 2014 got 391 percent increase from 2013 alone, and 97 percent of all mobile malware developed is targeting Android devices [6] There are a few reasons why Android becomes a frequent victim of malware. Firstly, Android has a loosely control ecosystem, which primarily relies on the Android community. Thus, many developers and users choose to publish to and download apps from third party application stores. Secondly, many manufacturers are licensed to produce a wide variety of smartphones and tablets, which make a large and complex family of different Android versions and hardware configurations. The lack of uniformity in operating systems, software and hardware makes the support and maintenance

In the past decade, the use of production and usage of mobile devices has significantly increased due to the advance in high performance and low power consumption mobile processor and storage chips, high-resolution touchscreen, and high-speed wireless networks. In the meantime, rich-content mobile applications further penetrate daily life beyond the basic communication use, which make usage pattern shift from desktop to mobile devices at a vigorous pace and make mobile devices and applications become an indispensable part of our daily life. Since 2014, Mobile devices have been quickly surpassing desktop PC in computing devices market in multiple aspects including hardware shipment, number of users, time of usage, and network traffic [1]. Among the leading mobile OS platform,

978-1-4673-8985-3/16 $31.00 © 2016 IEEE DOI 10.1109/IIAI-AAI.2016.237

566

inefficient and insufficient. Thirdly, most Android apps on are developed using java, which is sometimes described as a “semicompiled” language. It means the first part of the compilation is carried out by the programmer and the second part of the compilation is carried out on the client machine. This feature makes these applications relatively easy to decompile and modify, and the modification could be without profound skills, so that a large portion of Android malware is repackaged normal apps with injected malicious code. Moreover, the vulnerabilities in Dalvik Virtual Machine may also be exploited by some malware.

user ID and group ID. Android system use a default denial approach to control application executions, which means all operations are denied unless a specific permission is explicitly granted. A collection of permissions are defined and aligned to cover a variety of system resources. Each application must declare permissions in a manifest file in order to use corresponding APIs. The declaration of permissions is statistically done, so cannot be done dynamically at runtime. Android system relies on users to decide whether permissions need to be granted to an application, which is done by prompting the user for consent at the time the application is installed. In Android 5.1 (API level 22) or lower version, once an application is installed and all required permissions are granted, there is no further checks with the user while an application is running. Any feature related to granted permission can be used by the application as desired, and any other features beyond the granted permissions will fail without prompting the user.

In order to address the security concerns in Android apps, we propose a novel machine learning approach to detect malware by mining the patterns of Permissions and function calls acquired and used by Android apps. Our approach is based on static analysis of source code and resource files of Android applications to obtain binary and numerical features from qualitative and quantitative examination. Due to the large amount of permissions and related functions in Android API, feature selection methods are applied to reduce overhead and enhance the accuracy of results. Different machine learning methods, including Support Vector Machines, Random Forest and Neural Networks, are used for classification.

Similar to Linux systems, only the kernel and a small subset of the core applications (usually the pre-installed applications) run with root permission, which is the highest privilege and capable to modify the operating system, kernel. Application with root permission also has full access to all application and all application data.

The rest of this paper is organized as follows. Section II briefly reviews related work on Android malware detection. Section III provides an introduction to security mechanism in Android system. Section IV describes the conditions and development of malware on Android platform. Section V presents the proposed approach for Android malware detection. Section VI presents experiments with discussions followed by conclusions in Section VII. II.

In order to help users and developers understand the importance of permissions, Android has four levels of permission based on their intended use. 1. Normal permissions that may have minor impact on usage, but present minimal risk to the user's privacy or security, and thus will be granted automatically without prompting the user. For example, let the device vibrate upon receiving a message.

RELATED WORK

2. Dangerous permissions that control access to the sensitive data or resources that involve the user's private information, or could potentially affect the user's stored data or the operation of other apps. For example, SEND_SMS allows an application to send SMS messages.

As the fundamental security mechanism in Android OS, the permission were initially investigated and used to identify malicious activities. Enck et al. studied the combination of critical permission, and proposed several risk signals according to the permissions claimed by an application [7]. In [8], Peng et al. proposed different risk scoring schemes based on probabilistic generative models. In [5], Wei et al. showed the growing trend of the Android permission set, and found out third-party and pre-installed apps tend to use more permissions than they actually need and violate the principle of least privilege. Felt et al. proposed a web-based tool, called “Stowaway”, to detect over-privilege in apps [9]. Tam et al. proposed an automatic VMI-based dynamic analysis system to reconstruct OS- and high-level Android-specific behaviors, and identify malware from behavior portfolio [10]. Muñoz et al. collected the meta-data from Google play, and applied a machine learning approach to select predictive features and detect malware [11]. Elish et al. designed data-flow features from API call invocation and achieve good accuracy and discovered new malware on Google application store [12]. III.

3. Signature permissions that have highest privilege, and can be granted only if the application is signed with the device manufacturer’s certificate. 4. SignatureOrSystem permissions are only granted to apps that are signed or installed in a special system folder. Usually the per-installed apps have this kind of permission. Third-party apps can only use Normal and Dangerous permissions. When they request Signature or SignatureOrSystem permissions, they will be ignored. Staring from Android 6.0 (API level 23), the application needs to request permissions from the user at run-time, and the user can revoke granted permission at any time. The application needs to have then permissions every time when it executes. This change may raise the user’s awareness and caution at runtime.

ANDRORD PERMISSION

Android is a privilege-separated system, which employs mandatory access control over all processes. Each Android application has a unique identity in the system, which comprises

567

IV.

ANDRORD MALWARE

some malicious activities after installation. As important features to characterize applications, we extract requested permissions from the manifest file to reveal the capability of using system resources.

Even though the Android systems has enhanced security mechanism, Android devices are still considered more vulnerable to malware than their Apple peers.

Since the permission feature alone cannot precisely describe the behavior of the application, we also need to inspect the source code of the application to characterize its behavior by analyze the actual calls to permission-controlled functions. We decompile the Dex bytecode to Java source code and compute statistical features through static analysis of source code.

Google has built an application scan service, Google Bouncer, to help get rid of malware from Google Play. This mechanism largely prevent publishing and propagating malware through the official Android market. But unfortunately, it also has limitations, so sometimes allows faked and infected application to be published [13]. Besides Google Play, there are many third-party App Stores in the open Android ecosystem, which have little control on application publishing, which may cause a greater concern on security.

We define the set of all Android permission as P and the set of function as F.

Recent report shows that the number of Android malware samples has been growing at vigorous speed, and has reach the size about 14 times the number of 2011. The top 4 types of Android malware have been found are [14]:

ܲ ൌ ሼ‫݌‬ଵ ǡ ‫݌‬ଶ ǡ ‫݌‬ଷ ǡ ǥ ǡ ‫݌‬ଵ଴ସ ሽ

(1)

‫ ܨ‬ൌ ሼ݂ଵ ǡ ݂ଶ ǡ ݂ଷ ǡ ǥ ǡ ݂଺ହଶ ሽ

(2)

Based on the provided by Android API documents, we define the association map between permission and function as A.

1) Data Stealers: The purpose of this type of malware is to get system and user information such as photos, messages, personal files, OS version, product ID, International Mobile Equipment Identity (IMEI) number, and International Mobile Subscriber Identity (IMSI) number from the infected device and send it to a third party. 2) Rooting-capable: This type of malware infects mobile devices to take the complete control over mobile devices and functions. Root privilege also helps malware to self-replicate, which cannot be easily detected and deleted. 3) Premium Service Abusers This type of malware sends text messages to predetermined premium numbers, which cause users being charged for services that they did not sign up for. 4) Mobile Device Spies It is a malware that secretly track user’s behavior and use pattern, such as GPS location and calling log. And then it sends the stolen data to a third party. . There are lots of antivirus software on Android platform such as those provided by Symantec, AVG, Kaspersky Lab, Trend Micro, ESET, Lookout, Webroot and Dr. Web. Most antivirus software works by checking potential malware against a list of known “signatures”, or essential lines of code that can help identify a program’s functions. Due to the limitation in Android platform, the antivirus software itself is also an application which runs in the sandbox, so that it is easily fooled by simple tricks, such as polymorphic or metamorphic malware [15]

‫ ܣ‬ൌ ሼሺ‫݌‬ǡ ݂ሻȁ‫ܲ א ݌‬ǡ ݂ ‫ܨ א‬ǡ ݂ܽ݉݀݅‫݌ݕܾ݈݈݀݁݋ݎݐ݊݋ܿݏ‬ሽ Then, we compute the binary API features as the existence of call to each permission-control function, denoted as BC, and the numerical API features as the count of call to each permission-control function, and denoted as NC.

‫ ܥܤ‬ൌ ሼܾܿଵ ǡ ܾܿଶ ǡ ܾܿଷ ǡ ǥ ǡ ܾܿ଺ହଶ ሽ ܾܿ௜ ൌ ൜

ͳǡ݂݅‫݂׌‬௜ Ͳǡ݂݅‫݂׍‬௜

(3) (4)

ܰ‫ ܥ‬ൌ ሼ݊ܿଵ ǡ ݊ܿଶ ǡ ݊ܿଷ ǡ ǥ ǡ ݊ܿ଺ହଶ ሽ

(5)

݊ܿ௜ ൌ σ ߜሺ݂௜ ሻ

(6)

Compared to claimed permissions, the actually utilized permissions are more precisely reveal the behavioral pattern of the application. Therefore, we compute the binary Permission features as the existence of call to any function controlled by each permission, denoted as BP, and the numerical Permission features as the count of calls to all functions controlled by each permission, and denoted as NP.

‫ ܲܤ‬ൌ ሼܾ‫݌‬ଵ ǡ ܾ‫݌‬ଶ ǡ ܾ‫݌‬ଷ ǡ ǥ ǡ ܾ‫݌‬ଵ଴ସ ሽ ܾ‫݌‬௜ ൌ ቊ

ͳǡ݂݅‫׌‬൫݂௝ ȁ൫‫݌‬௜ ǡ ݂௝ ൯ ‫ܣ א‬൯ Ͳǡ݂݅‫׍‬൫݂௝ ȁ൫‫݌‬௜ ǡ ݂௝ ൯ ‫ܣ א‬൯

(7) (8)

ܰܲ ൌ ሼ݊‫݌‬ଵ ǡ ݊‫݌‬ଶ ǡ ݊‫݌‬ଷ ǡ ǥ ǡ ݊‫଺݌‬ହଶ ሽ(9)

V. PROPOSED APPROACH Android application is distributed and installed using Android application package (APK) file, which is an all-inclusive bundle containing bytecode, configuration, precompiled library and resource files. In order to perform static analysis on Android applications, we first we unpack the package and extract the Android Manifest and Dex bytecode files.

݊‫݌‬௜ ൌ σ ߜ ቀ൫݂௝ ȁ൫‫݌‬௜ ǡ ݂௝ ൯ ‫ܣ א‬൯ቁ

(10)

After collected all the data, the follow step is data analysis. Figure 1 and Figure 2show the top ten used API calls and Permissions used in benign and malware applications. The Permission set contains 104 binary features and 104 numerical features, while the API contains 654 binary features and 654 numerical features set. Then, we combine binary API and permission to form binary feature set and combine numerical API and permission to form numerical feature set.

The AndroidManifest.xml file contains the permissions the application requested from the system, but the claimed permission may or may not be used by the application. If an application request excess permission, which is known as overprovide problem, the application may be utilized to carry out

568

Embedded methods are built into adaptive systems, and simultaneously process feature selection with a learning classifier. To deal with the dimensionality problem in malware detection, we compare two feature selection methods: one-way analysis of variance (ANOVA) [16], which is a filter method, and Support Vector Machine—Recursive Feature Elimination (SVM-RFE) [17], which is a wrapper method. In the last step, we apply three machine learning algorithms, Support vector machines (SVM) [18] with RBF kernels, Random Forest [19], Artificial Neural Networks [20] to classify benign and malicious application. (a)

(b)

VI. Figure 1. Top ten used API calls (a) and Permissions (b) in benign applications

(a)

EXPERIMENTS AND DISCUSSIONS

A. Data Collection Google play is official and most reliable Android application market, which have the largest variety of Android application in several categories. Google currently use a build-in automated anti-virus system known as Google Bouncer to find and remove malicious apps been uploaded to the Google Play Store. Even some research found that this security mechanism doesn’t provide 100% guarantee on malware detection, but it is still the most reliable source to collect benign applications. We developed a crawler tool to automatically download applications from Google Play. In total, we collected 5000 popular applications from 25 categories, such as books, business and Entertainment, etc. The Android malware applications used in our experiments are from the Android Malware Genome Project [21]. This dataset consists of 1260 Android malware samples in 49 families from different markets.

(b)

Figure 2. Top ten used API calls (a) and Permissions (b) of malware applications

To achieve better performance in detection, we combine the Permission and API Call feature sets. However, with more features being included in the feature set, the increasing feature dimension and feature redundancy compromise the performance and efficiency of classification. Feature selection is the common process to get improved performance by eliminating features with little discriminative information. Therefore, for a comprehensive approach, feature selection should be a useful solution to further enhance the classification accuracy as well as reduce the overhead.

B. Setup Totally, we have six different data sets to experiment on, which are binary Permission, binary API, numerical Permission, numerical API, combined binary, and combined numerical sets. We experiment with multiple output dimensions of feature selection to search the optimal dimension for classification. In classification, 80 percent of the samples were used to train the classification model, and the rest 20 percent of the samples were used for testing. For every experimental setting, we ran the experiment 10 times, and the training and testing sets were randomly divided every time. The classification returned results consisting of true positive (TP), true negative (TN), false positive (FP), and false negative (FN). The testing accuracy is calculated as:

Most widely used feature selection methods could be categorized into filter, wrapper and embedded methods. Filter methods select feature subsets based on performance evaluation metric extracted from feature set, and work with no dependency on reference to learning algorithms. Filter methods are generally less expensive than wrapper and embedded methods. However, filter methods consider the features as independent individuals and ignore the possible interaction of features among features. The combination of features does not guarantee to provide combination effect according to the performance evaluation of individual features. Moreover, filter methods intend to select features corresponding to higher evaluation scores, which might generate more redundant but less informative feature subsets. Wrapper methods wrap around particular learning algorithms that can assess the selected feature subsets in terms of the estimated classification errors and then build the final classifiers.

ܵ݁݊‫ ݕݐ݅ݒ݅ݐ݅ݏ‬ൌ  ܵ‫ ݕݐ݂݅ܿ݅݅ܿ݁݌‬ൌ 

்௉ ்௉ାிே ்ே ்ேାி௉

(11) (12)

‫ ݕܿܽݎݑܿܿܣ‬ൌ ‫ ݓ‬ൈ ܵ݁݊‫ ݕݐ݅ݒ݅ݐ݅ݏ‬൅ ሺͳ െ ‫ݓ‬ሻ ൈ ܵ‫ݕݐ݂݅ܿ݅݅ܿ݁݌‬

(13)

Where w is a weighting factor at the range of [0, 1]. Without losing generality, w is set to 0.5 in our experiments.

569

C. Results Table I list the classification results from the experiments with binary Permission features, binary API features, numerical Permission features, and numerical API features separately without feature selection using Support Vector Machines, Random Forest, and Artificial Neural Networks. Besides, combined binary Permission and API features, as well as combined numerical Permission and API features are also experimented for comparison. TABLE I.

RF ANN SVM

RF ANN SVM

RF ANN SVM

RF ANN SVM

RF ANN SVM

RF ANN SVM

top 80 features from permission set, which achieve higher efficiency without significate impact on classification accuracy.

CLASSIFICATION RESULTS WITHOUT FEATURE SELECTION Binary Permissions Accuracy Sensitivity 87.84% 86.14% 92.12% 91.08% 87.72% 85.70% Binary API Accuracy Sensitivity 90.10% 91.18% 93.48% 93.04% 88.40% 87.04% Binary All Accuracy Sensitivity 94.08% 94.00% 96.00% 95.60% 91.48% 90.46% Numerical Permissions Accuracy Sensitivity 92.52% 95.06% 92.52% 92.40% 79.68% 65.20% Numerical API Accuracy Sensitivity 94.34% 97.36% 94.18% 93.86% 78.40% 56.72% Numerical All Accuracy Sensitivity 94.98% 97.56% 94.66% 93.96% 81.68% 66.72%

Specificity 89.50% 93.18% 89.74%

Figure 3. API calls with top ten F-statistic value

And Figure 4 shows the same result for permission data.

Specificity 89.02% 93.94% 89.72% Specificity 94.14% 96.40% 92.48% Specificity 90.00% 92.62% 94.18% Figure 4. Permissions with top ten F-statistic value Specificity 89.90% 94.42% 97.74%

TABLE II.

Specificity 92.34% 95.40% 96.68%

RF ANN SVM

From the comparison, we found that the API features outperform Permission features in both binary and numerical representations, which conforms to our hypothesis about the better characterization of the application’s behavior from API calls. The combined feature set achieves higher accuracy than separate feature sets. By comparing binary and numerical features, we found that quantitative features provide better classification accuracy than qualitative features in general.

RF ANN SVM

RF ANN SVM

Even though the combined feature set provide more accuracy classifications, it also introduce great overhead in computation and possibly some redundancies in features. To enhance the efficiency the classification process, we apply two feature selection methods, ANOVA, and SVM-RFE to rank and filter features. Figure 3 and 4 show the top ten features with highest F-statistic values, which are considered as more discriminative features. After experimentation with different feature dimensions, we pick top 300 features from API set and

RF ANN SVM

CLASSIFICATION RESULTS WITH FEATURE SELECTION ANOVA Selected Binary All Accuracy Sensitivity Specificity 91.36% 94.36% 88.49% 93.78% 93.87% 93.46% 90.61% 91.00% 89.62% SVMRFE Selected Binary All Accuracy Sensitivity Specificity 91.52% 94.23% 88.78% 93.61% 93.67% 93.67% 90.44% 91.90% 88.90% ANOVA Selected Numerical All Accuracy Sensitivity Specificity 94.41% 97.69% 91.00% 93.95% 94.12% 93.87% 82.20% 71.91% 91.00% SVMRFE Selected Numerical All Accuracy Sensitivity Specificity 94.41% 97.69% 91.00% 94.02% 94.00% 93.50% 82.50% 71.09% 93.40%

Table II shows the results by using the combination of selected Permission and API features. In the comparison

570

[8]

between selected and all features, numerical features show more stable performance than binary features. For binary features, the classifiers lose roughly 1%~3% accuracy after feature selection. For numerical features, no significant performance downgrade occurs, and support vector machines get some improvements in classification after feature selection.

[9]

VII. CONCLUSIONS

[10]

In this paper, we propose an a novel machine learning approach to detect malware by mining the patterns of Permissions and API Function Calls acquired and used by Android applications. Different features are combined, selected and experimented with machine learning algorithms. Experimental results show that our approach obtains good accuracy in the detection of different types of Android malware, and show great potentials to mitigate malware threats for Android devices.

[11]

[12]

[13]

[14]

REFERENCES

[15] [1]

[2] [3]

[4] [5]

[6] [7]

“Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2015–2020 White Paper”, http://www.cisco.com /c/en/us/solutions/collateral/service-provider/visual-networking-indexvni/mobile-white-paper-c11-520862.html, 2016-03-06. “Smartphone OS Market Share, 2015 Q2”, http://www.idc.com/prodserv /smartphone-os-market-share.jsp, 2016-03-06. “Number of apps available in leading app stores as of July 2015”, http://www.statista.com/statistics/276623/number-of-apps-available-inleading-app-stores/, 2016-03-06. “Statistics and facts about App Stores”, http://www.statista.com/topics /1729/app-stores/. Wei, Xuetao, Lorenzo Gomez, Iulian Neamtiu, and Michalis Faloutsos. "Permission evolution in the android ecosystem." In Proceedings of the 28th Annual Computer Security Applications Conference, pp. 31-40. ACM, 2012. “Mobile threat report - F-Secure”, https://www.f-secure.com/documents /996508/1030743/Mobile_Threat_Report_Q3_2013.pdf, 2016-03-06. Enck, William, Machigar Ongtang, and Patrick McDaniel. "On lightweight mobile phone application certification." In Proceedings of the 16th ACM conference on Computer and communications security, pp. 235-245. ACM, 2009.

[16] [17]

[18] [19]

[20] [21]

571

Peng, Hao, Chris Gates, Bhaskar Sarma, Ninghui Li, Yuan Qi, Rahul Potharaju, Cristina Nita-Rotaru, and Ian Molloy. "Using probabilistic generative models for ranking risks of android apps." In Proceedings of the 2012 ACM conference on Computer and communications security, pp. 241-252. ACM, 2012. Felt, Adrienne Porter, Erika Chin, Steve Hanna, Dawn Song, and David Wagner. "Android permissions demystified." In Proceedings of the 18th ACM conference on Computer and communications security, pp. 627638. ACM, 2011. Tam, Kimberly, Salahuddin J. Khan, Aristide Fattori, and Lorenzo Cavallaro. "CopperDroid: Automatic Reconstruction of Android Malware Behaviors." In NDSS. 2015. Munoz, Alfonso, Ignacio Martin, Antonio Guzman, and Jose Alberto Hernandez. "Android malware detection from Google Play meta-data: Selection of important features." In Communications and Network Security (CNS), 2015 IEEE Conference on, pp. 701-702. IEEE, 2015. Elish, Karim O., Xiaokui Shu, Danfeng Daphne Yao, Barbara G. Ryder, and Xuxian Jiang. "Profiling user-trigger dependence for Android malware detection." Computers & Security 49 (2015): 255-273. “Infected Fake Versions of Arcade Games on Google Play Threatened Players with Nasty Trojans”, http://www.eset.com/me/about/press /articles/article/android-trojan-drops-in-despite-googles-bouncer/, 201603-06. “Snapshot of Android Threads”, http://blog.trendmicro.com/trendlabssecurity-intelligence/snapshot-of-android-threats/, 2016-03-06. Scharr, Jillian, "Android Anti-Virus Software Easily Fooled", http://www.technewsdaily.com/17982-android-antivirus-seriousweakness.html, 2013-05-07. Iversen, Gudmund R., and Helmut Norpoth. Analysis of variance. No. 1. Sage, 1987. Guyon, Isabelle, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. "Gene selection for cancer classification using support vector machines." Machine learning 46, no. 1-3 (2002): 389-422. Cortes, Corinna, and Vladimir Vapnik. "Support-vector networks." Machine learning 20, no. 3 (1995): 273-297. Ho, Tin Kam. "The random subspace method for constructing decision forests." Pattern Analysis and Machine Intelligence, IEEE Transactions on 20, no. 8 (1998): 832-844. Werbos, Paul. "Beyond regression: New tools for prediction and analysis in the behavioral sciences." (1974). Zhou, Yajin, and Xuxian Jiang. "Dissecting android malware: Characterization and evolution." In Security and Privacy (SP), 2012 IEEE Symposium on, pp. 95-109. IEEE, 2012.