IPTV Statistic Data Collection, Processing and ... - Science Direct

Available online at www.sciencedirect.com

ScienceDirect Procedia Computer Science 77 (2015) 221 – 226

ICTE in Regional Development

IPTV statistic data collection, processing and preparation for use in a modeling system Vjaceslavs Dubovskisa*, Artis Teilansa, Nikolajs Visockisb a

Rezeknes Augstskola, Atbrivosanas st. 115, Rezekne, LV-4601, Latvia b SIA Microlines, Atbrivosanas st. 98, Rezekne, LV-4601, Latvia

Abstract Today most people get information from a TV. Trust level for television is very high and this kind of media can strongly influence public opinion. To research content watched on TV or public opinion, questioning and other methods are used today and respondents know about the research process. This knowledge forces people to give untruthful answers, because sometimes they don’t want to share their thoughts. This kind of research result is not satisfactory and conclusions can create misconceptions. Fast development of IPTV gives new opportunities for research using collected statistics. To make research legal, all statistical data must be anonymous. If a TV watcher doesn’t think that he has to make a choice, he will watch TV content he is interested in. At the moment there is no one united standard for collecting and processing IPTV statistical data. Each vendor handles data differently. Some solutions do not allow collecting data about watched TV channels and programs. In this paper the author will present a possible, universal method for data collection in a variety of IPTV networks, and different types of streaming and Middleware. © Published by Elsevier B.V.B.V. This is an open access article under the CC BY-NC-ND license © 2015 2016The TheAuthors. Authors. Published by Elsevier (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the Sociotechnical Systems Engineering Institute of Vidzeme University of Applied Sciences. Peer-review under responsibility of the Sociotechnical Systems Engineering Institute of Vidzeme University of Applied Sciences Keywords: IPTV; Middleware; Statistic.

1. Introduction Over the last decade, the growth of satellite service, the rise of digital cable, and the birth of HDTV (Highdefinition television) have all left their mark on the television landscape. Now, a new delivery method threatens to

* Corresponding author. E-mail address: [email protected]

1877-0509 © 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the Sociotechnical Systems Engineering Institute of Vidzeme University of Applied Sciences doi:10.1016/j.procs.2015.12.376

222

Vjaceslavs Dubovskis et al. / Procedia Computer Science 77 (2015) 221 – 226

shake things up even more powerfully. Internet Protocol Television (IPTV) has arrived, and backed by the deep pockets of the telecommunications industry, it's poised to offer more interactivity and bring a hefty dose of competition to the business of selling TV1. There were 117.39m IPTV subscribers worldwide at the end of Q4 2014 according to Point Topic’s latest research. Strong growth continued with a second consecutive quarterly increase of 4.1% recorded. There were less 100m IPTV subscribers worldwide at the end of Q4 20132.

Fig. 1. Global IPTV trends2.

IPTV users generate billions of records of statistic data every moment. This statistic shows which TV channels customers watch, what media content, what kind of TV programs, how long and how often and many other parameters. The usage of these statistics is not a popular practice the moment, but this data can provide a unique opportunity to know more about people’s interests and thoughts. At the moment there is no one united standard for collecting and processing IPTV statistical data. Each vendor handles data differently. Some solutions do not allow collecting data about watched TV channels and programs. This article presents a possible, universal method of data collection in a variety of IPTV networks, and different types of streaming and Middleware. 2. Methods for collecting IPTV data IPTV solution includes many nodes. There are streamers/encoders, Middleware, a billing solution, video-ondemand (VoD), content access system/digital rights management and others on the IPTV end (fig. 2). The key point for the collection of statistical data can be IPTV Middleware. Every client device must communicate with it to receive a TV channel list and other multimedia content availability.

Fig. 2. The IPTV solution diagram3.


223

The IPTV middleware also provides some specific functions: x resource management function, a functional module to manage system resources in IPTV terminal devices; x application management function, a functional module to manage the life cycle of the applications and interaction operations between them4. There are different IPTV solutions in the world. They can have many features and characteristics. Some Middleware solutions have many features, others offer poor functionality, Critical statistic feature distinctions of IPTV Middleware involve the following: x Web-based terminal middleware has one central middleware which orchestrates various applications. This orchestrating middleware, generally called "browser" or "user agent", processes a structured document and an interpretive language, usually called a "script", to enable various services5. Middleware sets TV channels and multimedia content list for client STB (set-top-box, IPTV terminal device6) device and research all user activities, log TV channels switching and other5; x Middleware sets TV channels and multimedia content list for a client’s STB device, but doesn’t follow and log user activities, doesn’t comply with Recommendation ITU-T H.760. IPTV streaming can use different transmission types: multicast and unicast. Transmission types are chosen by the operator and this option depends on the condition of the network. NB! In accordance with the European Union Directive for the processing of personal data a customer ID can only be used in anonymous format7. Customer ID is applicable only for STB interface language detection or approximate location (town, region). Statistical data can only be used for creating generalized models. 2.1. Middleware with statistical data export options IPTV Middleware with two-way communication options, when Middleware and STB constantly exchange data, log all data in the database3. This Middleware classifies as Middleware with statistic data export options. Information about user activities gets received (TV channels switching, STB status, multimedia content access and other) and carried out two ways: 1. STB asks access conditions or/and link to multimedia content from Middleware8. 2. STB sends reports to Middleware9. From the Middleware database statistical data gets exported to another database (DB)8. Mainly the statistics DB needs following data: 1. 2. 3. 4. 5.

The name of TV channel. TV channel status: on or off. Unique ID for each Set Top Box (IP address). Event time. Set top box status (on/off).

2.2. Middleware without statistical data export options in unicast streaming But it is possible that the Middleware does not constantly exchange with data STB, or Middleware does not have a database connection. Some budget solutions provide only a playlist set on the STB, but do not support statistical data collection. In this case a Streamer was used as a node for IPTV data collecting.

224


It is possible if a network can provide an IP unicast connection. The streamer is on one side connected to other multimedia content servers: CatchUp (TVoD, TV on demand), VoD, nVod (near video on demand), DVB (digital video broadcasting) to IP gateways, re-streamers, encoders and other, and to the distribution network on the other side. Streaming software uses Apache or Nginx logs10. All statistic data is included. The name of TV channel is shown as a source link or IP address, in this instance a script needs to be made to convert log data to the required format. 2.3. Middleware without statistical data export options in multicast streaming In networks where the stream transmission type is multicast and the Middleware does not constantly exchange data with STB, statistic data collection gets more difficult. IP multicasting is the transmission of an IP datagram to a "hostgroup", a set of zero or more hosts identified by a single IP destination address. A multicast datagram is delivered to all members of its destination host group with the same "best-efforts" reliability as regular unicast IP datagrams, i.e., the datagram is not guaranteed to arrive intact at all members of the destination group or in the same order relative to other datagrams11. It means that IPTV elements do not control stream delivery when the network uses multicast transmission. In this case only IGMP traffic collection can provide statistic data about linear TV channel viewing. The Internet Group Management Protocol (IGMP) is used by IP hosts to report their multicast group memberships to any immediately-neighbouring multicast routers12. An IGMP message refers to a multicast router or multicast traffic source. It means that the IGMP traffic collecting node will be in multicast routed and non-routed networks. In routed networks it is the last switch before the router and in non-routed networks it is the last switch before the multicast source (Fig. 3).

Fig. 3. (a) IGMP collecting point in multicast routed networks; (b) IGMP collecting point in multicast not routed networks;

Two key IGMP messages involve the following: x "Join group" message occurs when the host decides to join the group on the interface. It may occur only in the Non-Member state. x "Leave group" occurs when the host decides to leave the group on the interface. It may occur only in the Delaying Member and Idle Member states12. IGMP messages content showed in Fig.3. It includes information about IGMP message source, group number and time when it was sent. IGMP message source is the IP address of the device and it can be equal to the client’s ID. Group number is unique TV channel ID and it can correspond to the TV channel’s name.


225

Fig. 4. (a) IGMP v2 membership report – join group; (b) IGMP v2 membership report – join group.

Linux sniffing software can collect IGMP messages and parse key statistical data to the special database. 3. Structure of the IPTV statistics database The IPTV statistics database are shown on Table 1. There are many fields included. Some were inspected in this article, but the fields EPG and EPG_GENRE were not mentioned. EPG is Electronic program guide, the name of the TV channel program. EPG_GENRE is the genre of the TV channel program. These fields can be filled using a web parser and parsing it from a TV channel web site or from special XMLTV files. The XMLTV format used differs from most other XML-based TV listings formats in that it is written from the user's point of view, rather that the broadcaster's. It doesn't divide listings into channels; instead all the channels are mixed together into a single unified listing13. Table 1. Table stat in statistics database. The name of field

The meaning of field

Type of the variable

EVENT_ID

This field signs what type of IPTV content; TV channel switch on or off

INTEGER

CHANNEL_TAG

The name of TV channel

VARCHAR2

SUBSCRIBER_ID

Unique number for each subscriber

INTEGER

DEVICE_ID

Unique number for each Set Top Box (IP address)

INTEGER

TIMESTAMP

The time when TV content starts

DATETIME

EPG

Electronic program guide information

VARCHAR2

EPG_GENRE

Genre of content

VARCHAR2

INSERT_DATE

Date and time when the record is inserted into MySQL database

DATETIME

STB_STAT

Set top box status (1 – on, 0 – off)

INTEGER

4. Conclusions IPTV services offer great opportunities for service providers to diversify their revenue stream and move aggressively into the potentially lucrative broadcast TV and emerging interactive video markets. At the same time, the deployment and assurance of IPTV service incorporates enormous challenges in the management of complex

226


multilayer technology to provide a service that has historically been associated by customers with high quality and availability14. Despite the increasing number of IPTV user worldwide2, the use of statistics that generate subscribers is still undervalued. At the moment this statistic data doesn’t get collected in most of networks, but it can offer a lot of useful information for research. Most IPTV solutions don’t provide separate functionality for the export of statistics data. This article proposed IPTV statistical data collection, processing and preparation methods for use in a modelling system. Using methods presented in this article, it is possible to export statistics for almost any IPTV solution and network type.

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.

Anderson, Nate. "An introduction to IPTV." Arstechnica, Mar 12, 2006. Point Topic’s , “Global Broadband Statistics”, 2015. Netup.tv, “IPTV Solution for Medium to Large Deployments”, 2015. International Telecommunication Union, “ITU-T J.701: Broadcast-centric IPTV terminal middleware”, October 2010. International Telecommunication Union, “ITU-T H.730: Web-based terminal middleware for IPTV services”, June 2012. International Telecommunication Union, “ITU-T Y.1901: Requirements for the support of IPTV services”, January 2009. Directive 95/46/EC of the European Parliament and of the Council of 24 October 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data. Wiki.infomir.eu, “Technical documentation: Middleware configuration”, 2013. Tanovic, A., I. Androulidakis, and F. Orucevic, "Analysis of IPTV Channels Watching Preferences in Bosnia and Herzegovina.", Engineering, Technology & Applied Science Research 1.5, 2011, 105. Wowza Media Systems, “Wowza Streaming Engine: User's Guide Version: 4.3”, 2015. RFC 1112, “Host Extensions for IP Multicasting”, 1989. RFC 2236, “Internet Group Management Protocol, Version 2”, 1997. Wiki.xmltv.org, “XMLTV File format”, 2008. Kerpez, Ken, et al. "IPTV service assurance.", Communications Magazine, IEEE 44.9, 2006, 166-172.

Vjaceslavs Dubovskis is Rezekne University College PhD student since 2015. He received his first M.S. in social science from Turiba school of business administration in 2009, his second M.S. in engendering science from Rezekne University College. He has over 13 years served in the role of Senior technical manager and technology expert for Microlines Ltd. and Latnet TV Serviss Ltd. and over 5 year has position of director for Telecom businessman association of Latvia. His current research is on IPTV statistic data collection and using for public opinion modulating. E-mail: [email protected]. Artis Teilans received his first M.S. in system technics engineering from Riga Technical university in 1990, his second M.S. in computing science from University of Latvia in 1996 and his Ph.D. in engineering science from Riga Technical university in 1999. He has been with Rezekne University College since 2002, where he currently holds a professor position and managing director in PhD program in Social and technical system modulation. He is the author of numerous technical papers in system analytic, modulation and imitation theme. E-mail: [email protected]. Nikolajs Visockis has served the role of senior system architect and technology expert for Microlines Ltd. and Latnet TV Serviss Ltd. for a variety of technologies such as IPTV, MPLS VPNs, Ethernet and optical services, and VoIP. He is senior expert in Telecom businessman association of Latvia in IPTV and next-generation network configuration and management. He received M.S. in engineering science from Riga Technical university in 1993. E-mail: [email protected].