Region-based Object Categorisation using Relational ...

Region-based Object Categorisation using Relational Learning Reza Farid, Claude Sammut

Domains    

Machine Learning: fundamental problem in Artificial Intelligence Computer Vision Robotics Object classification

2

The 13th Pacific Rim International Conference on Artificial Intelligence (PRICAI) , Gold Coast, December 1-5, 2014

Domains    

Machine Learning: fundamental problem in Artificial Intelligence Computer Vision Robotics Object classification

3


Urban Search and Rescue

4


Rescue arena

5


Rescue arena

6


Related work Shanahan 2002  Using a logic program as a relational representation for 3D objects in 2D line drawings  Using abduction for object recognition Bo et al. 2011  Local features

7


Explore, Map and Find victims

8


Robot Planning

9


Platform

10


Range image and Point Cloud

11


Summary of Previous work •

Segmentation  Fit planes to surfaces  ... may also use other geometric objects

•

Feature extraction  Extract features of planes  Find relations between planes

•

Training  Label set of planes as belonging to an example of an object class

•

Learning Evaluation  10-fold cross validation

12


Segmentation  

Point cloud segmentation Using Planes as Primitives

13


Segmentation    

Point cloud segmentation Using Planes as Primitives Represent each region’s boundary by a convex hull Using Plane’s normal vector for orientation

14


Segmentation 

More examples

15


Feature extraction 



Two sets of features  Properties of individual planes  Relationships between pairs of planes Represented as PROLOG predicates

16






plane(pl1). plane(pl3).

plane(pl2). plane(pl4).

distributed_along(pl1,axisX). distributed_along(pl3,axisX). distributed_along(pl5,axisY).

17

plane(pl5). distributed_along(pl2,axisX). distributed_along(pl4,axisX).





Two sets of features  Properties of individual planes  Relationships between pairs of planes Represented as PROLOG predicates // Convex Hull Ratio // Diameter/Width ch_ratio(pl1,'4.0±0.25'). ch_ratio(pl2,'2.5±0.25'). ch_ratio(pl3,'3.5±0.25'). ch_ratio(pl4,'2.0±0.25'). ch_ratio(pl5,'1.5±0.25').

18






// Region’s normal vector in spherical coordinates normal_spherical_theta(pl1,'-90±15'). normal_spherical_phi(pl1,'135±15'). … normal_spherical_theta(pl5,'-135±15'). normal_spherical_phi(pl5,'112±15'). 19






// Angle between two regions angle(pl1,pl2,'90±15'). angle(pl1,pl3,'45±15'). angle(pl1,pl4,'90±15'). angle(pl1,pl5,'45±15'). angle(pl2,pl3,'90±15'). angle(pl2,pl4,'0±15'). angle(pl2,pl5,'90±15'). angle(pl3,pl4,'90±15'). angle(pl3,pl5,'90±15'). angle(pl4,pl5,'90±15'). 20


Binning 

Angle bins

21


Learning Object classes 

Object classes (positive/negative examples)  Step  Staircase

197/718

 Wall  Box

105/803

237/656 143/771

 Pitch/roll ramp 131/201 

Training by labelling  Segmented point cloud as planes  User interface to group extracted planes into objects  Label each selected set as positive example of an object class and negative example for other object classes

22


Learning Object classes 

Positive and negative example for each object class



The result of labelling as PROLOG predicates



Using ALEPH to construct one classifier for each type of object

23


Evaluation

(Farid and Sammut, 2014)

24


Description of a Staircase


25


Cumulative Learning


26


Comparison with a Non-Relational Object Classifier 

Bo et al., 2011 Depth kernel descriptors for object recognition • Gradient kernel descriptors (Gradient KDES) • Local binary pattern kernel descriptors (LBP KDES)

27



Using RGB-D dataset (Lai et al., 2011), 51 classes

28



Chosen sub-set

29


Planar Segmentation 

The number of planes for curved surfaces

30


Evaluation

(Farid and Sammut, 2014) 31


Summary of Extended work •

Segmentation  Fit regions to surfaces

•

Feature extraction  Extract features of regions  Find relations between regions

•

Training  Label set of regions as belonging to an example of an object class

•

Learning Evaluation  10-fold cross validation

32


Segmentation •

Primitives: Planes, Cylinders, Spheres

•

Point Cloud Library

33


Segmentation •

Using RGB-D dataset

34





Three sets of features  Properties of individual Regions • Updating normal vector definition for cylinder/sphere • Adding Oriented Bounding Box • Adding Radius to OBB ratio for cylinder/sphere primitives  Relationships between pairs of Regions  Properties of the Region-set (Regions forming an instance) Represented as PROLOG predicates

35





Three sets of features  Properties of individual Regions • Updating normal vector definition for cylinder/sphere • Adding Oriented Bounding Box • Adding Radius to OBB ratio for cylinder/sphere primitives  Relationships between pairs of Regions  Properties of the Region-set (Regions forming an instance) Represented as PROLOG predicates

36





Three sets of features  Properties of individual Regions  Relationships between pairs of Regions • Relative difference between two centres of mass and their normals (three angular values)  Properties of the Region-set (Regions forming an instance) Represented as PROLOG predicates

37





Three sets of features  Properties of individual Regions  Relationships between pairs of Regions  Properties of the Region-set (Regions forming an instance) • Number of regions • OBB ratios Represented as PROLOG predicates

38


Feature extraction

39


Evaluation (Comparing with Planar Approach) 

Reduction in number of segmented regions

40


Evaluation (Comparing with Planar Approach)  

Reduction in number of segmented regions Reduction in size of extracted features

41


Evaluation (Comparing with Planar Approach) 

No loss in accuracy

42


Evaluation (Comparing with Planar Approach) No loss in accuracy  Significant reduction in training time (hour:minute) (two orders of magnitude) 

43


More Classes and Larger Datasets  

The number of classes (Doubled) The number of instances for each class (Five times)

44


More Classes and Larger Datasets  

The number of classes (Doubled) The number of instances for each class (Five times)

45


Conclusion   





Extension of plane-based object classification using relational learning Primitives: planes, cylinders, spheres Updating features – Single Region Features – Region Relationships – Region-Set Features Comparison – No loss in accuracy – Reduction in the size of features – Significant reduction in training time Covering more classes and instances

46


References 













Bo, L., Ren, X., Fox, D.: Depth kernel descriptors for object recognition. In: Proc. Of IROS 2011 (2011) Farid, R., Sammut, C.: Plane-based object categorisation using relational learning. Machine Learning 94(1), 1–21 (2014) Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view RGB-D object dataset. In: Proc. of ICRA 2011 (2011) Rusu, R.B., Cousins, S.: 3D is here: Point Cloud Library (PCL). In: Proc. of ICRA 2011, pp. 1–4 (2011) Shanahan, M.: A logical account of perception incorporating feedback and expectation. In: Proc. of 8th Int. Conf. on Principles of Knowledge Representation and Reasoning, Toulouse, France, pp. 3–13. Morgan Kaufmann (2002) Srinivasan, A.: The Aleph Manual (Version 4 and above). Technical report. University of Oxford (2002) Vince, J.A.: Geometry for Computer Graphics: Formulae, Examples and Proofs. Springer (2005) 47


Questions

?

48


Region-based Object Categorisation using Relational ...

Region-based Object Categorisation using Relational ...

Suggest Documents

Plane-based Object Categorisation using Relational Learning

Plane-based Object Categorisation using Relational Learning ...

Plane-based Object Categorisation using

Plane-based Object Categorization using Relational Learning

Distributed Model-Based Diagnosis using Object-Relational ...

Subordinate class recognition using relational object models

Subordinate class recognition using relational object models

Object-Relational Mapping (ORM)

An Object Relational Approach

Converting Relational Databases into Object- relational Databases

Ontology Based Object Categorisation for Robots

Object-Relational Mapping (ORM) Hibernate

Object/Relational Mapping with Hibernate

Towards Object-Relational Data Mining

PENERAPAN OBJECT RELATIONAL MAPPING - Institutional ...

Recursive Queries Using Object Relational Mapping - Semantic Scholar

Using Object-Relational Mapping to Create the Distributed Databases ...

Using a Fuzzy Object-Relational Database for Colour Image Retrieval

Recursive Queries Using Object Relational Mapping - Semantic Scholar

Steps of Designing Biodiversity Data Model Using Object-Relational ...

Subordinate class recognition using relational object ... - Google Sites

A Case Study of Using an Object-Relational

Implementation of Object-Relational DBMSs in a Relational ...

Object-Oriented and Object- Relational Database Systems Basic ...