WoTSF: A Framework for Searching in the Web of ...

M. Younan, S. Khattab, and R. Bahgat, "WoTSF: A Framework for Searching in the Web of Things," in INFOS 2016 : The 10th International Conference on Informatics and Systems, ACM, Cairo , Egypt, May, 2016.

WoTSF: A Framework for Searching in the Web of Things M. Younan, S. Khattab, and R. Bahgat

May 11th 2016


Agenda 1

• Introduction

2

• Related Work

3 • The Architecture of the WoTSF

4 • The Implementation of the WoTSF

5 • Experimental Evaluation

6 • Conclusion and Future Work


INTRODUCTION


Introduction (WSN -> WoT) • Wireless Sensor Network (WSN) • Daily life integration of embedded devices – Convert things to Smart Things (SThs) – No.of devices reaches order of billions in 2020. – No.of users < no.of devices.

• The Internet of Things (IoT). – Huge Sensory Data (data stream).

• The Web of Things (WoT). – Current web tools and services.

• Contribution Idea: – Searching in the WoT increases its popularity


Introduction • Features of the WoT (Searching) – – – –

Different formats. Non-standardized naming Huge and dynamic Sensory Data. Daily life integration (normal user)

Crawling

• Simple query language • Real time queries • Interested in high level knowledge (summary)

Indexing and Searching

• Main Points: – Crawling • LWoTSEs’ Data –> following features of the WoT dataset presented in [1][2].

– Indexing • WoTSF: a Framework for searching in the WoT (High level indices)


RELATED WORK


Related Work (Selection) 

Traditional Search engines o Google Optimizations [3]

o AJAX Crawler [4] 

DiscoWoT (Mayer and Dominque. 2011)[5].



Shodan SE (www. Shodanqh.com,2015)[6] .



Dyser SE (Ostermaier et al.- 2010)[7].



Zhang et al. (2015) present a framework for a distributed range-query search[8]


Dyser • Dyser uses periodic patterns for each sensor (L,O,W,P), where L: period length O: offset

W: sensor output P: probability value

E.g., (7-Days, 2, empty, 0.5)

• Expansion of such periodic pattern is (K*L +O) For Example: If crawling process starts in 20-4, then Index record: (Occ., ID_01, 25-4-2016, empty, 0.5)

Su

Mo

Tu

We

Th

Fr

Sa

17

18

19

20

21

22

23

24

25

26

27

28

29

30

# recorded patterns in Dyser index will be >= # SThs


Dyser Index-> WoTSF Index 1

2

3

4 5

6 7

S01: (7-Days, 1, empty, 0.8) S02: (7-Days, 1, empty, 0.8) S03: (7-Days, 2, empty, 0.5) S04: (7-Days, 2, empty, 0.9) S05: (7-Days, 2, empty, 0.6) S06: (7-Days, 3, empty, 0.7) S07: (7-Days, 3, empty, 0.2) S08: (7-Days, 3, empty, 0.4) S09: (7-Days, 3, empty, 0.9) S10: (7-Days, 4, empty, 0.3) S11: (7-Days, 5, empty, 0.8) S12: (7-Days, 6, empty, 0.6) S13: (7-Days, 7, empty, 0.5) S14: (7-Days, 7, empty, 0.7)

WoTSF: High Level Index

WoT01: (7-Days, 1, empty, 0.8, S01) WoT01: (7-Days, 1, empty, 0.8, S02) WoT01: (7-Days, 2, empty, 0.9, S04) WoT01: (7-Days, 3, empty, 0.9, S09) WoT01: (7-Days, 4, empty, 0.3, S10) WoT01: (7-Days, 5, empty, 0.8, S11) WoT01: (7-Days, 6, empty, 0.6, S12) WoT01: (7-Days, 7, empty, 0.7, S14) Using aggregation function: ‘Max.’


THE PROPOSED WOTSF ARCHITECTURE


The WoTSF Architecture Crawling (WoT)

Google Opt. (server-root file)

Save time:

crawl less no.of pages

parse less no. of pages (Union format)

(High Level)

Query life cycle of the WoTSF


The WoTSF Architecture (Query Processing) Query

• Generate sub-queries

q2

• Select WoTSEs APIs

Master Indices

a

q2 WoT_1

• Send sub-queries

b

WoT_1 M.Ds

c

Ind.

Ind.

Ind.

d

DS_1

DS_2

DS_3

…

WoT_2 M.Ds

…

WoT_3 M.Ds

Key: sensor type


The WoTSF Architecture (Index Structure)


The WoTSF Architecture (Index Structure) WoT-Level (WoTSF)

SThs-Level (individual WoTSEs)

mi mi MI

SI

D D

A

R

Master Indices

SI SI

D D

WWW WWW WWW

S

S A

Secondary Indices

D

Dynamic Pages R

Storage


The WoTSF Implementation (Ranking) • Sensor State probability  𝑆𝑆 𝑖 = 𝐾 ∗ 𝐶𝑆 𝑖 + 1 − 𝐾 𝑃𝑖 K-> constant (0:1), CS-> current state, P->Probability

• Entity State probability  𝐸𝑆 𝑖 =

1 𝑛

𝑛 𝑗=1 𝑆𝑆

𝑗

N-> number of sensors

 𝐸𝑆(𝑖) =

𝑛 𝑗=1 𝐷(𝑗)

∗ 𝑆𝑆(𝑗)

D(j) -> impact factor of sensor j on ES(i)


THE WOTSF IMPLEMENTATION


The WoTSF Implementation (Crawling) Crawling Levels: – WoTSE – WoTSF

Start Listen to Sensors Receive new_value

Start

Wait Period (T)

SrVal_Stability (Sr,new_value) = (cur_rec_time - last_rec_time(Sr))/interval Calculate Entity State

Read Cur_Value

newChange (EoI, State)

Compare Cur_value with previous value (range)

Change

Yes

Send Cur_Value

(a) Sensor

Add new_value

no.of records >N

no.of. consecutive changes > M

Update (EoI, State)

Replace (oldest_value, new_value

Update prediction model (Sr)

(b) Gateway


The WoTSF Implementation (Indexing) • The WoTSF is evaluated using Real dataset: We use Kth percentile (historical data) -> single record per time unit. e.g. Suppose, a temperature sensor reads a set of values {5, 20, 21, 20, 20, 21, 23, 20, 21, 23} then, average=19.4 50th = 20


The WoTSF Implementation (Indexing) • The WoTSF is evaluated using random values: – prediction model (‘7-Days’) – SThs of type ‘occupancy’ in 10 WoT networks, -> 10,000 SThs – aggregation function ‘Max’ (empty room). – expanding the quadruple predictions

The prediction model type -> periodical crawling processes.


EXPERIMENTAL EVALUATION


The WoTSF Implementation (Indexing)

(a)

• WoTSF index (b)

– Count -> Max reading (WoT level) (b)

In case of distinct SThs values • Dyser Index (a)

– Count -> Max reading (STh level)

Local search engine WoTSF


The WoTSF Implementation (Searching)

(a)

(b)

(c)

• The WoTSF prototype: – implements autosuggestions like dyser. – filters buildings (static part) -> WoTSEs’ APIs – Evaluation is done on the dynamic part of the query (WoTSEs) using: • Building - Level (Master Indices) • SThs – Level (Secondary Indices)


The WoTSF Evaluation (Index size)


Experimental Evaluation (WoTSF & Dyser)

• Index Size


Experimental Evaluation (WoTSF & Dyser)

• Processing Time - WoTSF indices (High Level) - Dyser Indices


Experimental Evaluation (WoTSF & Dyser) • Result Accuracy Accuracy = Consistency =

𝐶𝑜𝑟𝑟𝑒𝑐𝑡 𝑉𝑎𝑙𝑢𝑒𝑠 (𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠) 𝑇𝑜𝑡𝑎𝑙 𝑅𝑒𝑡𝑟𝑖𝑒𝑣𝑒𝑑 (𝑓𝑎𝑙𝑠𝑒 𝑎𝑛𝑑 𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠)

𝐼𝑛𝑡𝑒𝑛𝑑𝑒𝑑 𝑅𝑒𝑡𝑟𝑖𝑒𝑣𝑒𝑑 (𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠) 𝑇𝑜𝑡𝑎𝑙 𝐼𝑛𝑡𝑒𝑛𝑑𝑒𝑑 (𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠+𝑓𝑎𝑙𝑠𝑒 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠)

Each building represents a single WoT network and hosts an occupancy sensor in each room Building ID

100

200

Room ID

101 102 103 201 202 203

State

Prediction Value

Empty

0.8 0.4 0.7 0.3 0.5 0.4

Search Engine

Index Size

WoTSF

2

Dyser

6


The WoTSF Evaluation (Index size) WoTSF and DSE search results according to different query types. Search Engine

Number of Results First K-Result = 4 Probability >= 50% 2 2 4 3

all 2 6

WoTSF DSE

WoTSF and DSE searching results (list of rooms) according to different query types. Search Engine WoTSF

DSE

Number of Results Results Room 101 Room 202 Room 101 Room 103 Room 202 Room 102 Room 203 Room 201

all

0.8 0.5 0.8 0.7 0.5 0.4 0.4 0.3

       

First K-Result = 4      

Probability >= 50%     


The WoTSF Evaluation (Summary) Criteria

DSE

Granularity of Device (STh) level master index

WoTSF

Network (e.g., building) level

     Indices and accuracy: based on  prediction models.    Less network overheads.   More consistent results

 All available results Case: Speed Search  More time for ranking. Case: Accurate Results Pros

  Cons



Larger indices. Harder to keep indices up-to date. More time for crawling, parsing, and indexing

top values per WoT network. Faster search Indices more up-to-date Consumes more time Indices: up-to-date. Accuracy: high Small and semi-dynamic indices. Less time for (crawling, parsing, indexing).

Tradeoff between search speed and result accuracy


CONCLUSION AND FUTURE WORK


Conclusion • The WoTSF:     

works on the top of Dyser supports simple query language saves time consumed for crawling, parsing and indexing builds High level indices Supports two types of search: • Search in high level indices (speed search). • Search in secondary indices using LWoTSEs’ APIs (Accurate search)

 increasing accuracy indices are up-to-date  keeping individual LWoTSEs handle their network.


Future Work • WoTSF has limitations such as: – It consumes more time in accurate search type. • Network overhead • Merging and Ranking results.

– Scheduling the crawling processes • to balance network overhead and result accuracy

– Using semantic technology: will be helpful for interoperability. – Considering other aggregation functions

• LWoTSE has limitations such as: – Extracting prediction models from SThs historical data – Dynamic discovery. – Solving problem of using multiple formats partially (crawling) but not by meaning -> (ontologies)


References (Selected) [1] M. Younan, S. Khattab, and R. Bahgat, "An Integrated Testbed Environment for the Web of Things," in ICNS 2015 : The Eleventh International Conference on Networking and Services, ISBN: 978-1-61208-404-6, Rome, Italy, May, 2015, pp. 69-78. [2] M. Younan, S. Khattab, and R. Bahgat, "Evaluation of An Integrated Testbed Environment for the Web of Things ", in IntSys15v8n34 : International Journal On Advances in Intelligent Systems, v 8 n 3&4, December, 2015. [3] Google. (2010, Jan.) Search Engine Optimization (SEO) - Starter Guide. [4] P. Suganthan G C, "AJAX Crawler," in Data Science & Engineering (ICDSE), International Conference on. IEEE, Cochin, Kerala, July 2012, pp. 27-30. [5] S. Mayer, D. Guinard, "An Extensible Discovery Service for Smart Things," in in Proceedings of the 2nd International Workshop on the Web of Things (WoT 2011), ACM, San Francisco, CA, USA, June, 2011, pp. 7-12. [6] (2015, Jan.) shodan search engine. [Online]. www.shodanhq.com [7] B. Ostermaier, K. Romery, F. Mattern, M. Fahrmairz, and W. Kellererz, "A Real-Time Search Engine for the Web of Things," in The 2nd IEEE International Conference on the Internet of Things (IoT), Tokyo,Japan, November. 2010, pp. 1-8. [8] C. Zhang, T. Zhang, and M.Wang, "A Distributed Range Query Framework for the Internet of Things," in 18th International Conference on Intelligence in Next Generation Networks (ICIN 2015), IEEE 2015, ISBN 978-1-4799-1866-9, Paris, France, February, 2015, pp. 83-88.


THANKS QUESTIONS ?

WoTSF: A Framework for Searching in the Web of ...

WoTSF: A Framework for Searching in the Web of ...

Suggest Documents

WoTSF: A Framework for Searching in the Web of ...

Framework for Searching in the Web of Things

WoTSF

A Hybrid Approach for Searching in the Semantic Web - IW3C2

Searching for digital images on the web

PROMS: A Web-based Tool for Searching in Polyphonic ... - CiteSeerX

Web Design Issues when Searching for Information in a Small ...

Searching the Workplace Web - CiteSeerX

WapSearch: A system for searching the Web using WAP enabled ...

a web server for batch searching the Pfam database - BioMedSearch

A domain framework for representation of Web

COGNITIVE STRATEGIES IN WEB SEARCHING - Department of ...

An Operational Metadata Framework for Searching ... - CiteSeerX

Ontology-based Searching Framework for Digital Shapes

Web searching for systematic reviews: a case study of reporting ...

Utilization of the PICO framework to improve searching PubMed for

Searching the 'Web of Things'

Basque web searching

WebPlug: A Framework for the Web of Things - The Distributed ...

A Framework for Establishing a Semantic Web in a ...

A Framework for Establishing a Semantic Web in a ...

Searching ... in a Web - Department of Computer Science - University ...

ISART: A Generic Framework for Searching Books with Social ...

Sorting out searching: a user-interface framework for text searches