Eyewitness identification of the suspect / offender during criminal proceedings. â. Most common case: select suspect among other persons (fillers). â. Either âin ...
Towards Recommender Systems for Police Photo Lineup Ladislav Peškaa and Hana Trojanováb a Department
of Software Engineering, b Department of Psychology, Charles University, Prague, Czech Republic
Recommender Systems
Propose relevant items to the right persons at the right time
Successful on many various domains
Multimedia E-commerce POIs Events Recepies …
However, some novel (relevant, important) tasks are yet to discover
DLRS@RecSys2017, Como
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
2
Police Photo Lineup
Eyewitness identification of the suspect / offender during criminal proceedings Most common case: select suspect among other persons (fillers) Either „in natura“ or based on photographies
DLRS@RecSys2017, Como
-
Suspect is positioned at random
-
3-7 fillers are added, witness should not know them
-
Wittness is instructed that the offender may or may not be present
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
3
Errors in Photo Lineup 1) Observe the event
2) Identify the offender - among other candidates
DLRS@RecSys2017, Como
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
4
Errors in Photo Lineup 1) Observe the event. - numerous sources of error (poor lighting, large distance, short time, fear, fatigue…)
- affects memory and recognition processes - decrease witness certainty and reliability
2) Identify the offender - decreased certainty may lead to incorrect identification - and conviction of innocent persons
DLRS@RecSys2017, Como
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
5
Assembling Fair Photo Lineups System should be set to eliminate testimony of uncertain witnesses
Double-blind administration Fair (unbiased) assembling of lineups
Out of the dataset of candidates, lineup administrator should select fillers similar to the suspect
Currently, only feature-based filtering systems are available Better automatization in content processing is necessary
How to better approximate human-perceived similarity of persons?
Convolutional Neural Networks?
DLRS@RecSys2017, Como
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
6
Assembling Fair Photo Lineups First approach: item-based recommendations
Propose top-k candidates for each suspect CB-RS, cosine similarity of candidates’ and suspect’s CB features
Nationality, Age, Appearence features Baseline (mimic feature-based filters)
Visual-RS, cosine similarity of candidates’ and suspect’s VGGFace probability layers
Approximate the similarity of a candidate and the suspect through their similarities with other persons
VGG-Face1 network
1Parkhi,
VGG network for facial recognition problem Trained as a multi-classification problem Dataset of 2,622 celebrities with 1,000 images each O. M., Vedaldi, A., & Zisserman, A. (2015). Deep Face Recognition. British Machine Vision Conference.
DLRS@RecSys2017, Como
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
7
Research Questions Is candidates recommendation (based on VGG-Face’s features) suitable for the lineup assembling?
How well is the human-perceived similarity approximated within top-k candidates? What is the role of diversity in lineup assembling? Is there a space for personalized recommendations (i.e. personalized per lineup administrator)? Can there be „too similar“ candidates?
DLRS@RecSys2017, Como
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
8
Evaluation – Protocol Dataset of missing and wanted persons in Czech Republic
4423 males
User study, 30 different lineups, 7 forensic technicians
For each lineup, both CB-RS and Visual-RS proposed top-20 candidates Merged into one list, forensic technician should select relevant fillers for particular suspect Follow-up questionaire and discussion
DLRS@RecSys2017, Como
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
9
Results In total 202 completed lineups, 800 selected candidates
Recommendations were seemingly relevant in most cases Visual-RS considerably outperformed CB-RS The performance difference was much smaller for suspects from outside of central Europe
Effect of different ethnicity? (Known in the forensic psychology literature). Total lineups Selected candidates
Level of agreement
All lineups Visual-RS CB-RS
202
466 (58%)
0.178
298 (37%)
0.138
Lineups with suspects from countries outside Central Europe Visual-RS CB-RS DLRS@RecSys2017, Como
53
105 (51%)
0.128
82 (40%)
0.205
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
10
Results II. Very low intersection (1.8%) of top-20 CB-RS and Visual-RS candidates
Potential to apply some combination in the future work
Average position of selected candidate
CB-RS: 8.9 (+/- 5.2); Visual-RS: 8.2 (+/- 5.7)
Ordering seems relevant even in top-20, however can be improved
Low level of agreement among participants on the selected candidates (0.178, 0.138)
Limits of the global ordering? Potential for personalized recommendation?
Less diversity is better
Participants agreed on the need for providing homogeneous lineups Dynamical recommendation based on the selected candidates?
No seemingly too similar candidates
However, the dataset contained some duplicates Should be addressed in the future work
DLRS@RecSys2017, Como
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
11
Conclusions
Recommending candidates based on CNN’s features seems plausible in police photo lineup
Item-based recommendations seems like a good first approach There is a potential for: Session-based personalized recommendations (need for uniformity of the final list) Long-term preference learning (due to the level of disaggreeement among lineup administrators) Approaches combining CNN’s features with content-based attributes
Future work
Move from recommended candidates to recommended lineups Evaluation through mock witnesses Deployable demo
DLRS@RecSys2017, Como
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
12
Thank you! Questions, comments?
Supplementary materials: http://github.com/lpeska/lineups, https://tinyurl.com/yat3rtr2 Slides: https://www.slideshare.net/LadislavPeska/towards-recommender-systems-for-police-photo-lineup DLRS@RecSys2017, Como
Ladislav Peska & Hana Trojanova: Towards Recommender Systems for Police Photo Lineup
13
