Random FieldsCRF[2]. Support Vector Ma-. chinesSVM[3]. Collins. [4] .... Vol. 49, No. 11, pp. 3765â3776, 2008. [2] . S
Factorization Machines
{hirata-ai@ed, komachi@}tmu.ac.jp
1 Collins Named Entity Recognition
[4]
co-training
Primadhanty
[5]
SVD L1 L2 [1]
Factorization Machine
3 Factorization Machines
Factorization Machines
Factorization Machines [6] Support Vector Machine (SVM) (Matrix Factorization)
d=2
2 Random Fields CRF [2]
Factorization
Machines Conditional Support Vector Ma-
yˆ(x) := w0 +
chines SVM [3]
n !
w i xi +
i=1
1 i w ∈ Rn
1
n n ! !
i=1 j=i+1
n w
w0 ∈ R
⟨vi , vj ⟩xi xj
(1)
xi
x
2
wi
1
xi
3 k
vi , vj
2 k !
⟨vi , vj ⟩ := vi k
f =1
vi,f · vj,f
(2)
V ∈ Rn×k
i
2 Machines
1: Primadhanty
⟨·, ·⟩
PER
6,516 (3,489)
1040 (762)
1,342 (925)
LOC ORG MISC
6,159 ( 987) 5,721 (2,149) 3,205 ( 760)
176 (128) 400 (273) 177 (142)
246 (160) 638 (358) 213 (152)
O
36,673 (5,821)
951 (671)
995 (675)
Factorization
2
4.2 Markov Chain Stochastic Gradient De-
Monte Carlo (MCMC)
Primadhanty 12
scent (SGD) 2
V n×k
4.3
SVM Factorization Machines
scikit-learn
SVM Fac-
Version 0.17
torization Machine libFM Version 1.4.2 [7] SVM Factorization Machines one-vsFactorization
Factorization Machines all Machines
d=2 k
4 4.4 precision
Primadhanty
recall
F1
O 4
SVM Factorization Machine
4.5
4.1
3
Primadhanty Primadhanty
CoNLL-2003
1
Primadhanty 4
1
precision-recall Factorization Machines
PER ORG O
LOC
Primadhanty precision
MISC
recall
1
1 F1
1
Factories 1
1 Primadhanty
POS
2: cap=1, cap=0 all-low=1, all-low=0 all-cap1=1, all-cap1=0 all-cap2=1, all-cap2=0 num-tokens=1, num-tokens=2, num-tokens>2
1
2
dummy Primadhanty
k
3: Primadhanty
k=5 precision
recall
F1
49.75
44.50
46.75
53.75 62.03
50.67 53.92
51.94 55.88
60.93
55.10
57.27
SVM [5] Factorization Machine
F1 k=1
57.1 k=5
k=8 F1
k k=5
Primadhanty
40 Factorization Machines
Machines
10 Factorization Machines Primadhanty
5 “ORGANIZATION”
40
Factorization Machines
“ORGANIZATION” “Vice-
“OTHER”
Primadhanty
President”
Factorization Machines
“Vice-President”
2 SVD Factorization Machines
“LOCATION” “PERSON” 2
6 PERSON
Factorization Machines Primadhanty Factorization Machines Factorization
Machines Primadhanty 2
Factorization Machines
4: PERSON SVM [5] Factorization Machine
LOCATION
MISC
P
R
F1
P
R
F1
P
R
F1
P
R
F1
86.45
72.28
78.73
31.35
38.62
34.61
62.54
59.40
60.93
34.67
32.39
33.50
73.83 84.36
90.84 80.40
81.46 82.33
64.96 39.49
36.18 50.41
46.48 44.29
72.11 70.88
44.98 55.33
55.41 62.15
37.20 48.99
43.66 34.27
40.17 40.33
1:
.
ORGANIZATION
[1]
,
[2]
. Semi-Markov conditional random fields . , , 2006.
[3]
,
precision-recall
. , Vol. 49, No. 11, pp. 3765–3776, 2008.
, .
. Support vector machine , Vol. 43, No. 1,
pp. 44–53, 2002. [4] Michael Collins and Yoram Singer. Unsupervised models for named entity classification. In EMNLP, pp. 100–110, 1999. [5] Audi Primadhanty and Xavier Carreras Ariadna Quattoni. Low-rank regularization for sparse conjunctive feature spaces: An application to named entity classification. In Proceedings of ACL-IJCNLP, pp. 126–135, 2015. [6] Steffen Rendle. Factorization machines. In Proceedings of ICDM, pp. 995–1000, 2010. [7] Steffen Rendle. Factorization machines with libFM. ACM Trans. Intell. Syst. Technol., Vol. 3, No. 3, pp. 57:1–57:22, 2012.
2: Factorization Machines
k