Great data storage and manipulation .... Data Mining. Data Science. (aka Big Data!) Machine ... Information Retrieval, N
Learning Machine Learning for the Future Dr. Fayyaz ul Amir Afsar Minhas PIEAS Biomedical Informatics Research Lab Department of Computer and Information Sciences Pakistan Institute of Engineering & Applied Sciences PO Nilore, Islamabad, Pakistan http://faculty.pieas.edu.pk/fayyaz/
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
Introduction • Objective – What are some possibilities with Machine Learning?
• Download the presentation – http://goo.gl/CoMcW9
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
2
Ways of obtaining knowledge • Observation • Experience • Reason or Logic
• Testimony
Man is essentially ignorant, and becomes learned through acquiring knowledge. (Ibn Khaldun)
• Revelation Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
3
Intelligent Computers • Computers are ______________ – Powerful – Great data storage and manipulation devices – Dumb!
• The science of making computers intelligent is called – Artificial Intelligence – Replicating ways of acquiring knowledge in the computer
• Examples? Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
4
What is Machine Learning?
Apples
Learning Machine Learning for the Future
Oranges
PIEAS Biomedical Informatics Research Lab
5
What is this?
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
6
Paintings by two different painters
Escher
Dali
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
7
Whose painting is this?
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
8
And this?
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
9
How many categories (clusters) are there?
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
10
Find the odd one out!
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
11
Predict the series • 1,1,2,3,5,8,13,…
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
12
Question? • Consider the vectors – X1=[1 2 1 4]T – X2=[2 4 2 4]T – X3=[0 0 0 4]T – X4=[3 6 3 4]T – X5=[4 8 4 4]T
• To store each vector, how many dimensions (or variables) do we need?
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
13
Learning to write
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
14
Questions • How were you able to recognize that the object shown was indeed an apple? • How were you able to discriminate between the paintings from two different painters? • How were you able to find out the different types of apples in the picture? • How did you manage to find the next number in the series? • How were you able to find which dimension was redundant? • How were you able to find the odd one out? • Learning to drive / write?
Learning Machine Learning for the Future
Classification
Classification Clustering Regression Dimensionality Reduction Anomaly Detection Reinforcement learning
PIEAS Biomedical Informatics Research Lab
15
What is Machine Learning? • Computers are ___________. – Dumb
• Making a machine (computer) perform the same tasks which you have just done is called ______________ – Artificial Intelligence
• If you learn to do these tasks using existing data, then this is called ____________ – Machine Learning Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
16
How to use Data to Produce Knowledge?
Real world Phenomenon
Sensor
Feature extraction mechanism
Decision
Machine Learning Existing Data
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
17
Example H
W
• Objective: Make good predictions not only on known data but previously unseen one – Generalization Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
18
Classification Example • Making a prediction rule – Nearest Neighbor – Linear Discriminant – Support Vector Machine
Feature-2
?
• Margin
– Non-Linear Boundaries Learning Machine Learning for the Future
Feature-1
PIEAS Biomedical Informatics Research Lab
19
What is machine learning? • Learning from observations, experience or Inductive Reasoning AI
Data Science (aka Big Data!)
Machine Learning
The Machine Learning Centric View of AI (not to scale)
Data Mining
CI Related Areas
PR
Statistics Linear Algebra Calculus, Optimization Techniques High Performance Computing Algorithms, Data structures and Programming Information Retrieval, NLP, Computer Vision, Signal Analysis Learning Machine Learning for the Future
Machine Learning is the Construction of algorithms that can learn from data to “explain” the data and make predictions
PIEAS Biomedical Informatics Research Lab
20
When to Apply Machine Learning? • Information Explosion in the Global Village • Machine Learning is particularly suited for areas that can exploit “The Unreasonable Effectiveness of Data”
The Economist. 2010. “Data, Data Everywhere,” February 25, 2010. http://www.economist.com/node/15557443.
– When making rules or theories about a phenomenon is hard or impossible? – Examples Halevy, Alon, Peter Norvig, and Fernando Pereira. “The Unreasonable Effectiveness of Data.” IEEE Intelligent Systems, 2009. Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
21
Applications
http://heli.stanford.edu/
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
22
Handwriting Recognition / OCR
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
23
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
24
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
25
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
26
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
27
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
28
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
29
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
30
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
31
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
32
OCR Accuracy
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
33
Gmail: ML in NLP
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
34
Facebook Friends Tagging
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
35
Applications of PR
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
36
Recommender Systems
• Recommend movies based on user preferences, interests and likes • Similar ideas for facebook… – Find friends that share your interests Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
37
PIEAS Bio-Medical Informatics Lab • Objective – Development of Intelligent Computational Solutions to problems in Biology and Medicine
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
38
BMI Lab Projects • Predicting Liver Disorders – Given: Liver ultrasound Images – Output: Diagnose surface & textural irregularities
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
39
BMI Lab Projects • Detecting cells – Input: Histopathology Images – Output: Identifying location and types of cells
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
40
BMI Lab Projects • Predicting ECG Abnormalities – Input: ECG Recording
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
41
BMI Lab Projects • Predicting Protein Binding Sites – Input: Protein Sequences or 3D structures – Output: Identifying interfaces
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
42
BMI Lab Projects • Identifying Molecular Causes of Disease
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
43
BMI Lab Projects • Predicting Prion Proteins – Input: Protein Sequences – Output: Whether this protein can form prions
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
44
BMI Lab Projects • Predicting Chemical Compounds in MassSpectrometry Data – RAMClust
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
45
BMI Lab Projects • Development of opensource machine learning tools and packages – PyLemmings: Python Based Large Margin Multiple Instance Learning System – CAFÉ-Map: Context Aware Feature Mapping Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
46
BMI Lab Projects: Biometrics
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
47
Modern Issues in Machine Learning • Extracting Features – Feature Engineering Takes a Long Time and Effort – Deep Learning – Graphical Processing Units
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
48
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
49
Modern Issues in Machine Learning • Labeling Data – Getting labeled data is hard • Easier to obtain a large amount of unlabeled or partially labeled data
• Develop machine learning models that can learn from unlabeled or ambiguously labeled data – Multiple Instance Learning – Active Learning – Semi-Supervised Learning – Self-Taught Learning Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
50
Modern Issues in Machine Learning • Scalability and “Big Data” – GPUs – Cluster and Cloud Computing – Machine Learning as a service
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
51
Modern Issues in Machine Learning • Tall Data – Large number of dimensions • Many dimensions are unrelated
– Small number of examples • Curse of Dimensionality
• Application areas – Bioinformatics
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
52
Example
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
53
Modern Issues in Machine Learning • Structured Outputs – Required output is not a simple decision • 𝑓: 𝑋 → 𝑦
– Rather a complex data object • 𝑓: 𝑋 → 𝑌
• Unstructured data – For example webpages or documents Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
54
Modern Application Areas: Bioinformatics • • • •
The cost of DNA sequencing has come down Large amounts of data Few people to fill the gap Impactful Applications
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
55
How do I get started? • Learn to program – Python – PIEAS Offering courses on Python Programming
• Take online courses or attend University ones – Coursera
• University Courses
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
56
How to Join the Lab? • Passion for Biological and Medical Informatics • Good Programming Skills • Good Mathematics • Need to know – Cross-Disciplinary Area – Application Oriented
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
57
References • Publications – http://faculty.pieas.edu.pk/fayyaz/pubs.html – http://faculty.pieas.edu.pk/fayyaz/bmi.html
• Interesting Machine Learning Papers – Jordan, M. I., and T. M. Mitchell. 2015. “Machine Learning: Trends, Perspectives, and Prospects.” Science 349 (6245): 255– 60. doi:10.1126/science.aaa8415. – Domingos, Pedro. 2012. “A Few Useful Things to Know About Machine Learning.” Commun. ACM 55 (10): 78–87. doi:10.1145/2347736.2347755. – Wagstaff, Kiri. 2012. “Machine Learning That Matters.” arXiv:1206.4656 [cs, Stat], June. http://arxiv.org/abs/1206.4656. Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
58
We want to make a machine that will be proud of us. - Danny Hillis
Learning Machine Learning for the Future
PIEAS Biomedical Informatics Research Lab
59