Dec 18, 2016 - bution Machine (LDM) [13] and ELM [5]. The global optimality of ELDM is proved subsequently. An efficiently multi-class extension of ELDM.
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Extreme Large Margin Distribution Machine and Its Applications for Biomedical Datasets Zhiyong Yang, Jingcheng Lu, Taohong Zhang University of Science and Technology Beijing
Presented by Zhiyong Yang December 18th 2016
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 1/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Outline
1 Introduction 2 A brief introduction to ELM 3 Proposed methods 4 Empirical study 5 Future works 6 Reference
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 2/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Outline
1 Introduction 2 A brief introduction to ELM 3 Proposed methods 4 Empirical study 5 Future works 6 Reference
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 3/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Introduction
Classification methods have become increasingly popular for biomedical and bioinformatical data analysis[9, 6, 8, 11, 7]. However, due to the difficulty of data acquisition, sometimes we could only obtain small-scale datasets which may lead to unreasonable generalization performances due to overfitting. We could resort to large margin theory to find out solutions for such dilemma. This paper aims at providing a large margin formation for Extreme Learning Machine(ELM), an efficient training algorithm for single hidden layer feedforward neural networks.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 4/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Our contributions
A novel algorithm called Extreme Large Margin Distribution Machine (ELDM) is proposed by bridging Large margin Distribution Machine (LDM) [13] and ELM [5]. The global optimality of ELDM is proved subsequently. An efficiently multi-class extension of ELDM For general propose, we evaluate the performance of our proposed algorithm on 7 benchmark datasets. While for biomedical applications, we evaluate our proposed algorithm on 5 small disease diagnosis datasets.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 5/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Outline
1 Introduction 2 A brief introduction to ELM 3 Proposed methods 4 Empirical study 5 Future works 6 Reference
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 6/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Illustration
Figure: An illustration of ELM
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 7/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Basic ELM I 1
d Given a training set D = {Xi , Yi }m i=1 , where Xi ∈ R is the ith input vector, Yi ∈ Rnc , is the ith target vector, i = 1, 2, · · · , m and a piecewise continuous activation function g (·)
2
randomly generate input layer weight matrix W ∈ Rd×nh and the hidden layer bias b ∈ Rnh , where nh is the number of hidden layer neurons.
3
Calculate the hidden layer features h1 (X1 ) · · · hnh (X1 ) h(X1 ) .. .. ··· H = ... = . . h(Xm ) h1 (Xm ) · · · hn (Xm )
(1)
h
where hi (Xj ) = g W (i)> Xj + bi and W (i) is the ith column of W
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 8/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Basic ELM II
4
Estimate the output layer weight β by following equation ˆ = H† Y = (H> H + I )−1 H> Y β C
(2)
where Y is the target vector (Y1 , · · · , Ym )> .
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 9/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Basic ELM III
From Eq.(2), β could be also regraded as the solution of the following least square problem : Problem P (P) : argmin β
1 1 tr (β > β) + Ctr (Hβ − Y)> (Hβ − Y) 2 2
(3)
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 10/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Advantages of ELM
advantages Iteration-free Easy to be implemented Efficient Theoretical supports [5, 4, 3]
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 11/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
A problem with ELM
ELM forces all the functional margins yi h(xi )> β to be close to 1 via the square loss.
m m X X (Hβ − Y)> (Hβ − Y) = (yi − h(xi )> β)2 = (1 − yi h(xi )> β)2 i=1
i=1
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 12/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
how to push the instances further away from h> β = 1 or h> β = −1?
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 13/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Outline
1 Introduction 2 A brief introduction to ELM 3 Proposed methods 4 Empirical study 5 Future works 6 Reference
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 14/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
The margin distribution on the training dataset In order to maximize the functional margins on average, we could optimize their distribution. ([1], [10]) The mean and variance of the functional margin on D: mean m
1 X 1 γ1 = yi β > h(Xi ) = (H> Y)> β m m
(4)
i=1
variance
γ2 =
m m 2 1 XX (yi β > h(Xi ) − yj β > h(Xj )) 2 m i=1 j=1
(5)
2 > > 1 = β H Hβ − ( )β > H> YY> Hβ m m
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 15/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
ELDM & LDM
Formulation of LDM [13, 14, 12] : merge γ1 and γ2 into the objective function of SVM. Formulation of ELDM (ours): merge γ1 and γ2 into the objective function of ELM. Though seemingly similar these algorithms may be, they adopt completely different solutions.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 16/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Formulation of ELDM I
1
A good distribution of function margin must have a large mean with a small variance.
2
Add λ2 γ2 − λ1 γ1 into the original objective function of Problem P, the result of which is denoted Problem Q through out this paper. Then our proposed algorithm, Extreme Margin Distribution Machine, gives the closed-form solution of Problem Q.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 17/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Formulation of ELDM II
Problem Q (Q) : minimize :
1 > 1 β β + C (Hβ − Y)> (Hβ − Y) + 2 | {z } |2 {z } l1
l2
(6)
λ2 γ2 − λ1 γ1 | {z } l3
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 18/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Formation of ELDM III
Explanations l1 : a regularizer to punish high model complexity l2 : a weak constraint that require all the functional margins to be close to one and thus be positive l3 : objective function for an optimal margin distribution
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 19/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Solution of ELDM I
Theorem A global minimum solution of problem Q could be represented as : β∗ = where k1 =
−1 I k2 + k1 H> H − hh> (k3 h) C m
(7)
4λ2 λ1 4λ2 + 1, k2 = , k3 = + 1 and h = H> Y Cm Cm Cm
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 20/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Solution of ELDM II
Proof The objective function of Problem Q could be rearranged as follows L=
λ 1 > 1 λ1 2 β β+ (Hβ − Y)> (Hβ − Y) + γ2 − γ1 2C 2 C C
(8)
By setting the partial derivative of L with respect to β to zero we have 4λ2 4λ2 λ1 > I +( + 1)H> H − 2 hh> β − H> Y − H Y = 0 (9) C Cm m Cm Up to now we have reached an optimal solution β ∗ of Q
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 21/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Solution of ELDM III Proof −1 k2 I + k1 H> H − hh> (k3 h) (10) C m Since problem Q is an unconstrained quadratic programming with respect to β, to prove the global optimality of β ∗ , we only need to prove that the Hessian matrix of L, which is denoted as He, is positive semi-definite. We have β∗ =
I + H> H + (k1 − 1)H> H C 1 − (k1 − 1) H> YY> H m I 1 > = + H H + (k1 − 1)H> (I − YY> )H C m
He =
(11)
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 22/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Solution of ELDM IV
Proof Obviously, the first two terms of He are all semi positive-definite. For > 1 the last term, it is easy to notice that I− YY> is an idempotent m matrix. As a result, the last term of He could be expressed as (k1 − 1)H> (I −
1 YY> )H = KK> m
(12)
√ 1 where K = k1 − 1(I − YY> )H. Then by Eq.(12), He must be m positive semi-definite.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 23/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Solution of ELDM V
When nh < 2m the computing the solution proposed in the theorem above takes O(8nh3 + 2m2 nh ). When nh > 2m and that is exactly the case for small datasets, we could further simplify the solution as follows. ˜> I + H ˜H ˜ > −1 Hh ˜ β ∗ = Ck3 h − Ck3 H C
(13)
˜ = [H> K> ]> According to Eq.(13) computational where H complexity for calculating the inversion is O(8m3 + 2nh2 m), which is much faster than Eq.(7) for small datasets.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 24/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Speed up multi-class ELDM I Since LDM is, in nature, proposed for binary class problems. We only consider the One vs. All scheme for multi-class extensions.
While naively solving this problem forces us to repeat the binary case algorithm nc , is there a more efficient solution?
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 25/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Speed up multi-class ELDM II
Now considering the ith subproblem, which use the ith column of Y, i.e. Y(i) as its target output. Following Theorem 1, we have ∗
β (i) =
−1 I k2 + k1 H> H − hi h> (k3 hi ) i C m
(14)
where hi = H> Y(i) , β (i)∗ is the ith column of output layer parameter β.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 26/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Speed up multi-class ELDM III
Given the following notations notations ∗
−1 I k2 + k1 H> H − hi h> (k3 hi ) i C m k2 I Ai = + k1 H> H − hi h> i C m Bi = k3 hi
β (i) =
A=
I + k1 H> H C
(15) (16) (17) (18)
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 27/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Speed up multi-class ELDM IV
Then following Woodbury matrix identity [2], we have :
A−1 i
k2 > vi vi m = A−1 + k2 1 − hi > A−1 hi m
(19)
where vi = A−1 hi . Thanks to Eq.(19), we could reducethe 3 2 complexity for computing A−1 to i s from O nc nh + mnh O nh3 + mnh2 .
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 28/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Outline
1 Introduction 2 A brief introduction to ELM 3 Proposed methods 4 Empirical study 5 Future works 6 Reference
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 29/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
datasets I 7 libsvm datasets and 5 biomedical datasets from Kent Ridge Bio-medical Dataset are employed to validate the effectiveness of our proposed method. Table: Basic Information of the employed datasets
austrilian diabete german heart glass svmguide2 wine
#instance
#feature
#class
690 768 1000 270 214 391 178
14 8 24 13 9 20 13
2 2 2 2 6 3 3
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 30/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
datasets II
Table: Summery of Biomedical Datasets
DLBCL Harvard Outcome DLBCL Stanford LungCancer Ontario Leukemia ALLAML Leukemia MLL
#instance
#feature
#class
58 47 39 72 72
6817 4026 2880 7129 12582
2 2 2 2 3
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 31/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Experiment setting I
For each dataset, a 5 fold cross-validation is repeated twice in search of the best performance, and the best average accuracies are recorded. ELDM : tanh is selected as the hidden layer activation function, C is selected from range 2[−3:3] , both λ1 and λ2 are chosen from range 2[−8:−3] . For binary classifications, nh is selected from {50, 100, 150, 200, 300}. For multi-class classifications, nh is selected from {50, 100, 150, 200, 300, 1000}. For biomedical datasets nh is selected from {100, 150 ,200, 300, 500, 1000, 2000, 3000}.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 32/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Experiment setting II
ELM : the same as ELDM except λ1 and λ2 . LDM : the parameter settings for nh , C , λ1 and λ2 are the same as those of ELDM. The Gaussian kernel function is employed for LDM. And the corresponding parameter is chosen from 2[−5:5] .
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 33/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Results for libsvm datasets I
Table: Accuracy Comparisons
austrilian diabete german heart glass svmguide2 wine
ELDM
ELM
LDM
0.8658±0.0148 0.7768±0.0262 0.7384±0.0137 0.8200±0.0239 0.6961±0.0693 0.8439±0.0527 0.9972±0.0088
0.8655±0.0139 0.7701±0.0305 0.734±0.0107 0.8037±0.0232 0.6891±0.0656 0.8273±0.0484 0.9916±0.0135
0.8580±0.0108 0.7734±0.0212 0.7446±0.0108 0.8304±0.0303 0.7024±0.0721 0.8413±0.0413 0.9887±0.0199
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 34/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Results for libsvm datasets II
Table: Efficiency Comparisons for benchmark datasets
austrilian diabete german heart glass svmguide2 wine
ELDM
ELM
LDM
0.0081 0.0080 0.0100 0.0067 0.0063 0.0050 0.0047
0.0074 0.0063 0.0088 0.0016 0.0021 0.0031 0.0017
0.4447 0.5368 1.7182 0.0265 0.3613 0.9113 0.0913
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 35/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Results for libsvm datasets III
1
First of all, ELDM could always outperform ELM on all of the 7 libsvm datasets. Hence ELM could benefit from margin distribution optimization.
2
Secondly, compared to LDM, ELDM could slightly outperform LDM on 4 out of the 7 benchmark datasets. However, if we further comparing the efficiency of ELDM and LDM, it is obvious to see that the running time results of LDM are much slower than that of ELDM for all 7 datasets.
3
As a result, ELDM is at least no worse than LDM for most of the tested datasets with higher efficiency.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 36/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Results for biomedical datasets I
Table: Accuracy Results of the biomedical datasets
DLBCL Harvard Outcome DLBCL Stanford LungCancer Ontario Leukemia ALLAML Leukemia MLL
ELDM
ELM
LDM
0.6818±0.1090 0.9256±0.0873 0.7607±0.1360 0.7448±0.1148 0.8538±0.062
0.6470±0.0803 0.9044±0.0783 0.7339±0.1462 0.7219 ±0.1268 0.7929 ±0.0686
0.6364±0.1037 0.9044±0.1028 0.7339±0.1377 0.6800±0.1437 0.8757±0.0519
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 37/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Results for biomedical datasets II 0.65
0.7
0.6
ldm
0.55 0
1000
2000
0.65 0.6 0.55 -4
3000
accuracy
eldm elm
accuracy
accuracy
0.7 0.65
eldm elm ldm
-2
nh
0
2
0.6 0.55 0.5 -5
4
0
5
log2 γ
log2 C
Figure: Parameter Analysis for DCBL Harvord Outcome dataset
1
0.9 0.88 0.86 0
1000
2000
nh
3000
1 eldm elm ldm
0.8
0.6 -4
-2
0
log2 C
2
ldm
accuracy
eldm elm
0.92
accuracy
accuracy
0.94
4
0.8
0.6 -5
0
log2 γ
5
Figure: Parameter Analysis for DCBL Stanford dataset
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 38/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Results for biomedical datasets III
0.74 0.72 0.7 0
1000
2000
0.74 0.72 0.7 -4
3000
ldm
eldm elm ldm
0.76
accuracy
eldm elm
0.76
accuracy
accuracy
0.75
0.78
0.78
-2
nh
0
2
0.7 0.65 0.6 -5
4
log2 C
0
5
log2 γ
Figure: Parameter Analysis for LungCancer Ontario dataset
0.75
0.72 0.7 0.68 0
1000
2000
nh
3000
0.68 ldm
accuracy
eldm elm
0.74
accuracy
accuracy
0.76
0.7
0.65 -4
eldm elm ldm
-2
0
log2 C
2
4
0.66
0.64 -5
0
log2 γ
5
Figure: Parameter Analysis for Leukemia ALLAML dataset
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 39/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Results for biomedical datasets IV
1
1
0.6
eldm elm
0.5 0
1000
2000
nh
3000
accuracy
0.7
accuracy
accuracy
ldm
0.8
0.8 0.6 0.4 -4
-2
0
log2 C
eldm elm 2 ldm 4
0.8 0.6 0.4 -5
0
5
log2 γ
Figure: Parameter Analysis for Leukemia AML dataset
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 40/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Results for biomedical datasets V
1
ELDM could outperform both of its prototypes on 4 out of 5 employed datasets. For Leukemia MLL, though not better than LDM, ELDM performs significantly better than ELM.
2
ELDM could significantly outperform ELM for most of the involved parameters.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 41/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Discussion I
1
For small biomedical datasets, the improvements of ELDM on generalization performance are higher than that of the libsvm benchmark datasets.
2
The variances for libsvm benchmark datasets are much smaller than that of the biomedical datasets.
3
Correspondingly, for benchmark datasets, overfitting may not be of a much issue. In such cases, ELDM and LDM get similar performance. However, due to the high variance and limited instances, the 5 small biomedical datasets may probably suffer from overfiting. In such cases, ELDM could outperform LDM with the following possible reason:
4
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 42/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Discussion II
LDM adopts an iterative manner to update its parameter, the stopping condition is partially (the maximum number of iterations could also control stopping condition) based on the performance of training set. When nh is small, ELM has much lower model complexity.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 43/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Outline
1 Introduction 2 A brief introduction to ELM 3 Proposed methods 4 Empirical study 5 Future works 6 Reference
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 44/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Future works
Future Works Hierarchical ELDM & Deep ELDM Multi category ELDM Efficient hyper-parameter Optimization Efficient extension toward big data analyses
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 45/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Outline
1 Introduction 2 A brief introduction to ELM 3 Proposed methods 4 Empirical study 5 Future works 6 Reference
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 46/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
References I [1] Wei Gao and Zhi-Hua Zhou. On the doubt about margin explanation of boosting. Artif. Intell., 203:1–18, 2013. [2] William W. Hager. Updating the inverse of a matrix. SIAM Review, 31(2):221–239, 1989. [3] Guang-Bin Huang. An insight into extreme learning machines: Random neurons, random features and kernels. Cognitive Computation, 6(3):376–390, 2014. [4] Guang-Bin Huang, Lei Chen, and Chee-Kheong Siew. Universal approximation using incremental constructive feedforward networks with random hidden nodes. Neural Networks, IEEE Transactions on, 17(4):879–892, July 2006. [5] Guang-Bin Huang, Qin-Yu Zhu, and Chee-Kheong Siew. Extreme learning machine: Theory and applications. Neurocomputing, 70(1–3):489 – 501, 2006. Neural NetworksSelected Papers from the 7th Brazilian Symposium on Neural Networks (SBRN ’04)7th Brazilian Symposium on Neural Networks. [6] Jyrki L¨ otj¨ onen, Robin Wolz, Juha Koikkalainen, Valtteri Julkunen, Lennart Thurfjell, Roger Lundqvist, Gunhild Waldemar, Hilkka Soininen, and Daniel Rueckert. Fast and robust extraction of hippocampus from MR images for diagnostics of alzheimer’s disease. NeuroImage, 56(1):185–196, 2011. [7] Yun Niu and Yuwei Wang. Protein-protein interaction identification using a hybrid model. Artificial Intelligence in Medicine, 64(3):185–193, 2015.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 47/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
References II [8] S. C. Saxena, V. Kumar, and S. T. Hamde. Feature extraction from ECG signals using wavelet transforms for disease diagnostics. Int. J. Systems Science, 33(13):1073–1085, 2002. [9] Harsh Shrivastava, Vijay Huddar, Sakyajit Bhattacharya, and Vaibhav Rajan. Classification with imbalance: A similarity-based method for predicting respiratory failure. In 2015 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2015, Washington, DC, USA, November 9-12, 2015, pages 707–714, 2015. [10] Liwei Wang, Masashi Sugiyama, Zhaoxiang Jing, Cheng Yang, Zhi hua Zhou, Jufu Feng, and Manfred Warmuth. A refined margin analysis for boosting algorithms via equilibrium margin. Journal of Machine Learning Research, 2011. [11] Zhisen Wei, Ke Han, Jing-Yu Yang, Hong-Bin Shen, and Dong-Jun Yu. Protein-protein interaction sites prediction by ensembling SVM and sample-weighted random forests. Neurocomputing, 193:201–212, 2016. [12] Cuihong Wen, Jing Zhang, Ana Rebelo, and Fanyong Cheng. A directed acyclic graph-large margin distribution machine model for music symbol classification. Plos One, 11(3), 2016.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 48/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
References III
[13] Teng Zhang and Zhi-Hua Zhou. Large margin distribution machine. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, New York, NY, USA - August 24 - 27, 2014, pages 313–322, 2014. [14] Yu-Hang Zhou and Zhi-Hua Zhou. Large margin distribution learning with cost interval and unlabeled data. IEEE Trans. Knowl. Data Eng., 28(7):1749–1763, 2016.
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 49/50
Introduction A brief introduction to ELM Proposed methods Empirical study Future works Reference References
Thank you for your attention!
Zhiyong Yang, Jingcheng Lu, Taohong Zhang — Extreme Large Margin Distribution Machine and Its Applications for Biomedical Da 50/50