Understanding the latest trends of BLOB Storage
Issue 7: February 2014
Understanding the latest trends of BLOB Storage (SQL Server 2008 R2 & Mongo DB NoSQL Database)
Information gets power from the live data and data is considered as live if they are connected with each other in more meaningful way. The world of information is consisting of basically two type data known as structured and unstructured data. When we speak about the structured and unstructured data then definitely the question comes to mind that what parameters makes the data to be called as structured and unstructured.
Data is considered to be unstructured if the format of the data can’t be presumed before it physically become available but this is not always the case to define the unstructured data. When data is not in machine readable form and can’t be linked with each other to make more meaningful information then also we may consider the data as unstructured data. “*.Pdf” files, media files as well as other binary files can be considered as unstructured data. Normally this category of data is used to be stored in database as binary format so they are popularly known as BLOB- binary Large OBject. On the basis of these two categories of data, there are basically two different kind of data engineering concepts are evolving. Data ware housing, Data Mining, BI tools developments are the latest trends to play with structured data where as the BLOB (Binary Large OBject) storage and BLOB processing become another area of evolution. Now we will try to understand that why BLOB storage management is so important topic of the current time and what are the different challenges companies are facing today as well as we will discuss about the solutions for those challenges in SQL & NoSQL technologies. Two big giants in the industry of database management are SQL & NoSQL and both these technologies are having very good support to deal with structure data as well as unstructured data in different way but many developers treat both types of data in similar fashion which makes the system less tolerable to availability o CIA pyramid. In this article we will see how SQL Server 2008 R2 provides features to deal with BLOB. Examples are always the conventional way to understand many critical topics. So lets take example of a scenario where we have design a system which takes both structure as well as unstructured data at the same time and provide storage as well as retrieval system.
Page No. 2
Issue 7: February 2014
Understanding the latest trends of BLOB Storage
Name of the Applicant: Father’s Name: Date of Birth:
dd/MM/yyyy
Upload scan documents of educational qualifications: File System
Browse
STORAGE
OR
Store reference of the unstructured data in table
Store unstructured data in binary field
Pros: Less work load on the database server as the server only stores the reference to the file only.
Pros: Single DML Sql insert command to store both the structured as well as unstructured data is used so Atomicity is always maintained during transactions.
Conversion to/from binary format is not required during storage and retrieval. Cons: 1. Extra programming overhead to upload files to the file system along with the DML Sql insert command. 2. Atomicity is not maintained due to multiple steps involved to store structured data (name, father’s name, date of birth) and unstructured data like the scanned file being uploaded. 3. Back up need to be taken separately for database as well as file system.
Back up need to be taken only from one database and no need to have separate back up from file system as the unstructured data is stored along with the structured data in the same database. Cons: 1. As the unstructured data is to be stored along with structured data so retrieval may become very performance hungry if the unstructured data (BLOB data) size is huge. 2. This mechanism affects the server workload heavily.
Page No.3
Understanding the latest trends of BLOB Storage
Issue 7: February 2014
After facing the performance problem in both type of BLOB storage, SQL Server 2008R2 came with a hybrid solution of the above two methodologies. This feature in SQL Server 2008 R2 and above is popularly known as “File Stream”. In File Stream mechanism of SQL Server, the unstructured BLOB data is inserted into/from the database storage along with other structured data using a single DML SQL Insert command keeping the Atomicity intact. But the SQL Server stores the reference to the BLOB data in the column of a table along with other structured data and puts the real BLOB in the predefined file system. Now the responsibility of storing and retrieving the BLOB to/from the file system is taken care by SQL Server. As the real BLOB data is stored in the file system with the reference stored in the table and the entire mechanism is taken care by the SQL Server itself, so it became the best possible way to store BLOB in SQL Server as per the current technology is concerned. Configuring SQL Server 2008R2 for File stream: Step1: Right Click on the database engine and select the property. From the Advanced menu select “Filestream Access Level” to Full access enabled.
Page No. 4
Understanding the latest trends of BLOB Storage
Issue 7: February 2014
Step2: Right click on the database menu and select the option to create a new Database. From the right side menu select the Filegroups option and in the Filestream tab add one Filestream name (Example: FS_g. Then press OK. Step3: Go to the general Tab once and give Database name and click on Add. Give Logical name as “fs”. Select File Type to Filestream and Filegroup. to “FS_g” . Step4: Select the “Path” value of the “fs” to some file location and then click on OK.
Insert unstructured data into the table of Database: Before inserting a BLOB file, we first need to create a table with column set as filestream so that the BLOB file storage management will be handled by the SQL Server database itself.
Insert a “*.jpg” using the sql command from the “G:\” drive.
Retrieval of the unstructured data from the table using C#.net: Asp.net is a very popular front end for SQL Server as backend database. So we are here presenting C# code to access the JPG file we have inserted in to the database in our previous example. The code below is fetching the JPG file using SQL Select command and then places that stream in the HTTP response.
Page No.5
Understanding the latest trends of BLOB Storage
Issue 7: February 2014
Now we will see how the most popular NoSQL technology deals with the BLOB storage management using it’s unique GRID FS feature. The popular MongoDB database has been taken for technology demonstration purpose in the next section.
Page No. 6
GridFS – Rescue for LARGER FILES
Issue 8: February 2014
GridFS is a specification for storing and retrieving files that exceed the BSONdocument size limit of 16MB. Instead of storing a file in a single document, GridFS divides a file into parts, or chunks, and stores each of those chunks as a separate document. By default GridFS limits chunk size to 256k. GridFS uses two collections to store files. One collection stores the file chunks, and the other stores file metadata. When you query a GridFS store for a file, the driver or client will reassemble the chunks as needed. You can perform range queries on files stored through GridFS. You also can access information from arbitrary sections of files, which allows you to “skip” into the middle of a video or audio file. GridFS is useful not only for storing files that exceed 16MB but also for storing any files for which you want access without having to load the entire file into memory.
When should I use GridFS? For documents in a MongoDB collection, you should always use GridFS for storing files larger than 16 MB. Say you want to store larger files (maybe a 2GB video) when you perform a query on that file all 2GB needs to be loaded into memory … say you have a bigger file, 10GB, 25GB etc … it‟s quite likely you‟d run out of usable RAM or not have that much RAM available at all! So, GridFS solves this problem by streaming the data back (in chunks) to the client … this way you‟d never need to use more than 4MB of RAM. The following are other specialities to consider GridFS over normal File system.
If you are using replication or autosharding your GridFS files will be seamlessly sharded or replicated for you. Since MongoDB datafiles are broken into 2 GB chunks MongoDB will automatically break your files into OS manageable pieces. You won‟t have to worry about OS limitations like „weird‟ filenames or a large number of files in one directory, etc. MongoDB will auto generate the MD5 hash of your file and store it in the file‟s document. This is useful to compare the stored file with it‟s MD5 hash to see if it was uploaded correctly, or already exists your database.
Note: Furthermore, if your files are all smaller the 16 MB BSON Document Size limit, consider storing the file manually within a single document. You may use the BinData data type to store the binary data.
Page No. 8
Issue 8: February 2014
GridFS – Rescue for LARGER FILES Implement GridFS To store and retrieve files using GridFS, use either of the following:
A MongoDB driver. See the drivers documentation for information on using GridFS with your driver. The mongofiles command-line tool in the mongo shell.
GridFS Collections GridFS stores files in two collections:
chunks stores the binary chunks. files stores the file‟s metadata.
GridFS places the collections in a common bucket by prefixing each with the bucket name. By default, GridFS uses two collections with names prefixed by fs bucket:
fs.files fs.chunks
You can choose a different bucket name than fs , and create multiple buckets in a single database. Each document in the chunks collection represents a distinct chunk of a file as represented in the GridFS store. unique ObjectID stored in its _id field.
Each
chunk
is
identified
by
its
The chunks Collection Each document in the chunks collection represents a distinct chunk of a file as represented in the GridFS store. The following is a prototype document from the chunks collection.:
{ "_id" : , "files_id" : , "n" : , "data" : } A document from the chunks collection contains the following fields: chunks._id
Page No. 9
GridFS – Rescue for LARGER FILES
Issue 8: February 2014
The unique ObjectID of the chunk. chunks.files_id
The _id of the “parent” document, as specified in the files collection. chunks.n
The sequence number of the chunk. GridFS numbers all chunks, starting with 0. chunks.data
The chunk‟s payload as a BSON binary type. The chunks collection uses a compound index on files_id and n , as described in GridFS Index.
The files Collection Each document in the files collection represents a file in the GridFS store. Consider the following prototype of a document in the files collection:
{ "_id" : , "length" : , "chunkSize" : "uploadDate" : "md5" :
"filename" : , "contentType" : , "aliases" : , "metadata" : , } Documents in the files collection contain some or all of the following fields. Applications may create additional arbitrary fields: files._id
Page No. 10
Issue 8: February 2014
GridFS – Rescue for LARGER FILES
The unique ID for this document. The _id is of the data type you chose for the original document. is BSON ObjectID.
The
default
type
for
MongoDB
documents
files.length
The size of the document in bytes. files.chunkSize
The size of each chunk. GridFS divides the document into chunks of the size specified here. The default size is 256 kilobytes. files.uploadDate
The date the document was first stored by GridFS. This value has the Date type. files.md5
An MD5 hash returned the String type.
from
the
filemd5
API.
This
value
has
files.filename
Optional. A human-readable name for the document. files.contentType
Optional. A valid MIME type for the document. files.aliases
Optional. An array of alias strings. files.metadata
Optional. Any additional information you want to store.
GridFS Index GridFS uses a unique, compound index on the chunks collection for he files_id and n fields. The files_id field contains the _id of the chunk‟s “parent” document. The n field contains the sequence number of the chunk. GridFS numbers all chunks, starting with 0. For descriptions of the documents and fields in the chunks collection,
Page No. 11
GridFS – Rescue for LARGER FILES
Issue 8: February 2014
The GridFS index allows efficient retrieval of chunks using the files_id and n values, as shown in the following example:
cursor = myFileID}).sort({n:1});
db.fs.chunks.find({files_id:
See the relevant driver documentation for the specific behavior of your GridFS application. If your driver does not create this index, issue the following operat ion using the mongo shell:
db.fs.chunks.ensureIndex( { files_id: 1, n: 1 }, { unique: true } );
Command Line: mongofiles An easy way to get started and see how GridFS works is the use he mongofiles command line utility. To make things easy, mongofiles accepts RESTful looking commands, for example … $ ./mongofiles -d myfiles put 03-smbd-menu-screen.mp3 connected to: 127.0.0.1 added file: { _id: ObjectId('4ce9ddcb45d74ecaa7f5a029'), filename: "03-smbd-menu-screen.mp3", chunkSize: 262144, uploadDate: new Date(1290395084166), md5: "7872291d4e67ae8b8bf7aea489ab52c1", length: 1419631 } done! This uploaded (PUT) the 03-smbd-menu-screen.mp3 file to a database called myfiles (it could be any database.) This file now resides in the myfiles DB in the fs.files Collection. We can confirm this by passing the list command. $ ./mongofiles -d myfiles list connected to: 127.0.0.1 03-smbd-menu-screen.mp3 1419631
Page No. 12
Issue 8: February 2014
GridFS – Rescue for LARGER FILES
Hurrah! We have our files in there …you can also query it via the MongoDB Shell like so … > use myfiles; > db.fs.files.find({}); { "_id" : ObjectId("4ce9ddcb45d74ecaa7f5a029"), "filename" : "03-smbd-menu-screen.mp3", "chunkSize" : 262144, "uploadDate" : "Mon Nov 22 2010 03:04:44 GMT+0000 (UTC)", "md5" : "7872291d4e67ae8b8bf7aea489ab52c1", "length" : 1419631 } Note: the size, upload date & md5 are all produced for you which is pretty handy.
Uploading a File (or Data) via MongoDB Driver Likely a more realistic way of storing files in GridFS will be via one of the many available language drivers. Each driver handles GridFS a little differently but the concepts are the same. The first thing you need to sort is out is are you going to upload actual files or are you going to create files from strings of data? For example an application that allows a user to upload a video directly from their computer to your application would use the file method … however an application that would take a profile image (for example) and compress and resize it for use in your application would likely use the string of data method.
The File Method In this example we’ll assume the file is already in your filesystem in the /tmp/ dir, but the file could be wherever your web-server/PHP is configured to access. To work with GridFS files in PHP you use MongoGridFS class, more information can be found in the documentation. We
will
use MongoGridFS::storeFile but
you
could
also
use MongoGridFS::put (which works like the command line example.)
Page No. 13
GridFS – Rescue for LARGER FILES
Issue 8: February 2014
The String of Data Method String of data is very similar only we’ll pass a string instead of a file/path so use the code above but use storeBytes instead. $storedfile = $grid->storeBytes("This is test file data!", array("metadata" => array("filename" => $filename), "filename" => $filename)); You could of course pass any string, string representation of a images (or an encoded file via a string) where we’ve put “This is test file data!” …
A Little About Metadata For PHP it doesn’t really matter but since other drivers handle things slightly differently it’s best to write any metadata to its own metadata field as well as a separate filename field as we have done in the example above.
Page No. 14
Issue 8: February 2014
GridFS – Rescue for LARGER FILES You
can
put
any
file
metadata
that
makes
sense
for
your
use
in
the metadata field.
Stream Back Files Now that our file or files are loaded into GridFS streaming back the file is farily simple …
Connect to MongoDB
Do a findOne() on the file
Load it into memory using getBytes()
Set the proper headers
Stream the file back to the browser
So, here is how we’d stream back an image in PHP …
Stream an Image from GridFS to the Browser Warning: this will load the file into memory. If the file is bigger than your memory, this will cause problems!
Page No. 15
Issue 8: February 2014
GridFS – Rescue for LARGER FILES
With a little adjustment you could stream back an mp3, or video, or prompt for a file download, etc.
Other Ways to Search for GridFD Files You could also use the Document’s ID … $image = $gridFS->findOne( array("_id" => new MongoId("4ceb167810f1d50a80e1c71c")) ); That will likely be how your application would look up a file in a real world system. You can use any valid MongoDB findOne() query in its place as well, or use find() to get back a GridFS cursor of files.
Deleting Files You delete files in the same way, there are actually a couple ways to remove GridFS files, but we’ll just use one of the easiest … Be really careful about passing the correct query to remove or you might just find yourself removing all your files! You can also use MongoGridFS::delete and pass the Document’s ID only.
Wrap Up Hopefully you can now get started with GridFS and see if it will work well for your application … remember if you stream back (using the image example above) the files they will be loaded into memory and not streamed in 4MB chunks …so be careful! Have fun with your LARGER datas.
Hands-on with the CSharp Driver Downloading the C# Driver The simplest way to get started is by using the nuget package. The package page will provide details on using nuget. You can also download the C# driver here: https://github.com/mongodb/mongo-csharp-driver/releases If you downloaded the .zip file, simply unzip it and place the contents anywhere you want. If you downloaded the .msi file, double click on the .msi file to run the setup program, which will install the C# driver DLLs in the C:\Program Files (x86)\MongoDB\CSharp Driver 1.x directory (the exact path may vary on your system).
Add a Reference to the C# Driver DLLs Right click on the References folder in Visual Studio‟s Solution Explorer and select Add Reference... . Navigate to the folder where the C# driver DLLs were installed and add a reference to the following DLLs: 1. MongoDB.Bson.dll 2. MongoDB.Driver.dll As an alternative you could use the NuGet package manager to add the C# driver package to your solution.
Page No. 17
GridFS – Rescue for LARGER FILES
Issue 8: February 2014
Add Required using Statements As a minimum you will need the following using statem ents:
using MongoDB.Bson; using MongoDB.Driver; Additionally, you will frequently add one or more of these using statements:
using MongoDB.Driver.Builders; using MongoDB.Driver.GridFS; using MongoDB.Driver.Linq; There are additional namespaces that would only be required in special cases.
Get a Reference to the Client Object The easiest way to get a reference to a client object is using a connection string:
var connectionString = "mongodb://localhost"; var client = new MongoClient(connectionString); If you want to store the client object in a global variable you can. MongoClient is thread-safe.
Get a Reference to a Server Object To get a reference to a server object from the client object, write this:
var server = client.GetServer();
Get a Reference to a Database Object To get a reference to a database object from the server object, write this:
var database = server.GetDatabase("test"); // "test" is the name of the database If you use more than one database, call GetDatabase again for each database you want to use.
You Do NOT Need to Call Connect or Disconnect
Page No. 18
GridFS – Rescue for LARGER FILES
Issue 8: February 2014
The C# driver has a connection pool to use connections to the server efficiently. There is no need to call Connect orDisconnect ; just let the driver take care of the connections (calling Connect is harmless, but calling Disconnect is bad because it closes all the connections in the connection pool).
Full Sample Program using using using using using using using using using using using
System; System.Collections.Generic; System.Linq; System.Web; System.Web.UI; System.Web.UI.WebControls; MongoDB.Bson; MongoDB.Driver; System.IO; MongoDB.Driver.Builders; MongoDB.Driver.GridFS;
public partial class _Default : System.Web.UI.Page { string file_id; protected void Page_Load(object sender, EventArgs e) { } //Upload the file into MongoDB protected void Button1_Click(object sender, EventArgs e) { var server = MongoServer.Create("mongodb://localhost:27017"); var database = server.GetDatabase("Organization"); var fileName = "D:\\test\\test3.pdf"; var fs = new FileStream(fileName, FileMode.Open); var gridFsInfo = database.GridFS.Upload(fs, fileName); file_id = gridFsInfo.Id.ToString(); fs.Close(); } //Download the file through stream protected void Button2_Click(object sender, EventArgs e) { var server = MongoServer.Create("mongodb://localhost:27017"); var database = server.GetDatabase("Organization"); ObjectId oid = new ObjectId("52a98d4c14747c2724f38eb9"); var file = database.GridFS.FindOne(Query.EQ("_id", oid)); using (var stream = file.OpenRead()) { var filesize = stream.Length; byte[] bytes = new byte[filesize]; while ((stream.Read(bytes, 0, bytes.Length)) > 0) { Response.ContentType = "pdf/application"; Response.AddHeader("content-disposition", "attachment;filename=First_PDF_document.pdf"); Response.OutputStream.Write(bytes, 0, bytes.Length); Response.Flush(); } } }
Page No. 19
GridFS – Rescue for LARGER FILES
Issue 8: February 2014
//Download the file without the stream protected void btn_download2_Click(object sender, EventArgs e) { var server = MongoServer.Create("mongodb://localhost:27017"); var database = server.GetDatabase("Organization"); ObjectId oid = new ObjectId("52a98d4c14747c2724f38eb9"); var fileinfo = database.GridFS.FindOne(Query.EQ("_id", oid)); FileStream fs = new FileStream("D:\\test\\new_test3.pdf", FileMode.Create); database.GridFS.Download(fs, fileinfo); fs.Close(); } }
REFERENCES: http://docs.mongodb.org/manual/core/gridfs/ http://docs.mongodb.org/manual/faq/developers/#faq-developers-when-to-usegridfs http://docs.mongodb.org/manual/reference/gridfs/
Page No. 20
ANDROID APPLICATION DEVELOPMENT
Issue 7: February 2014
By ANUNAY RAJ
[email protected]
These days the demand of smart phone is being increased and we have different types of client e.g. Touch Phone, tables, Note etc. The demands of the applications for these smart clients are increasing and we have several operating systems for these smart clients e.g. Android, IOS and Windows etc. Android is a software platform and operating system for mobile devices based on the Linux operating system and developed by Google and the Open Handset Alliance. We are saying based on Linux operating system because the core of operating system is the Linux kernel. Android devices are in great demand in market more than 300 million devices has been activated. As it is open source and Google releases the source code, this open-source code and permissive licensing allows the software to be freely modified and distributed by device manufacturers and developer. The market is beginning to flood with Android devices of all shapes and sizes. So it is a good investment to become an android application developer. Android application development is the process by which new applications are created for the Android operating system. Applications are usually developed in the Java programming language using the Android Software Development Kit. It provides great opportunities for developer as it is developed using JAVA. By using java concept along with the android inbuilt libraries and API which makes it powerful to develop application for any hardware configuration supporting android operating System. All the android Libraries and APIs come in a single package provided by Google popularly known as Android Software Development Kit (SDK). Android developers are free to write applications that take full advantage of increasingly powerful mobile hardware and distribute them in an open market. As it uses java features we can use all the features of java like inheritance, abstraction and many more oops features. As the need of time is to develop apps, so there are IDEs to prevent developers from writing the long codes. For android application development the best IDE is eclipse. The eclipse is a good environment to build the apps by simple doing drag and drop for items to design the GUI. The Android software development kit (SDK) includes a comprehensive set of development tools. These include a debugger, libraries, a handset emulator based on QEMU, documentation, sample code, and tutorials. Currently supported development platforms include computers running Linux (any modern desktop Linux distribution), Mac OS X 10.5.8 or later, Windows XP or later; for the moment one can develop Android software on Android itself by using [AIDE - Android IDE - Java] app and [Android java editor] app. The officially supported integrated development environment (IDE) is Eclipse using the Page No. 21
ANDROID APPLICATION DEVELOPMENT
Issue 7: February 2014
Android Development Tools (ADT) Plugin, though IntelliJ IDEA IDE (all editions) fully supports Android development out of the box, and NetBeans IDE also supports Android development via a plugin. Additionally, developers may use any text editor to edit Java and XML files, then use command line tools (Java Development Kit and Apache Ant are required) to create, build and debug Android applications as well as control attached Android devices (e.g., triggering a reboot, installing software package(s) remotely). Coming to android application lifecycle every application runs in its own process, which gives better performance in security, protected memory and other benefits. Therefore, Android is responsible to run and shut down correctly these processes when it is needed. It is important that application developers understand how different application components (in particular Activity, Service, and BroadcastReceiver) impact the lifetime of the application's process. Not using these components correctly can result in the system killing the application's process while it is doing important work.
Figure: Application Lifecycle Now coming to android application project package, it contain a list of files and folders which are auto generated after creating a project but the important are the .java file which resides in src folder where the coding is done .xml file which reside in layout folder contains the layout of the application. There is also some mysterious files which are R.java which resides in gen folder of package we can see a Java class that contains nested Java classes. These nested Java classes have the same names as some of the res folders defined in the preceding res section. Under each of those subclasses, you can see members that have the same names as the resources in their respective res folders (excluding their file extensions). I’m happy to tell you that we don’t need to worry about it.
Page No. 22
ANDROID APPLICATION DEVELOPMENT
Issue 7: February 2014
The Android Development Tool (ADT) generates this code for you so that we don’t have to worry about what’s happening behind the scenes. As we add resources and the project is rebuilt, Android Development Tool (Eclipse) regenerates the R.java file. This newly generated file contains members that reference your recently added resources. And the other is manifest file named as androidmanifest.xml it keeps track of everything application needs, requests, and has to use to run. The Android manifest file is stored at the root of your project and is namedAndroidManifest.xml. Every application must have an Android manifest file in its root directory. The application manifest file provides all the essential information to the Android system — information that it must have before it can run any of your application’s code.
The application manifest file also provides the following:
The name of Java package for the application, which is the unique identifier for application in the Android system as well as in the Android Market
The components of the application, such as the activities and background services
The declaration of the permissions your application requires to run
The minimum level of the Android API that the application requires All the android application has a common extension or what we call has as file type
that is .apk. Any android application is recognized by .apk as its extension. Android runtime contain a very light weight machine like Java has JVM to run the application that is Dalvic machine. It was designed specifically for Android running in limited environment, where the limited battery, CPU, memory and data storage are the main issues. Android gives an integrated tool “dx”, which converts generated byte code from .jar to .dex file, after this byte code becomes much more efficient to run on the small processors.
Figure: Conversion from .java to .dex file
As the result, it is possible to have multiple instances of Dalvik virtual machine running on the single device at the same time. The Core libraries are written in Java language and contains of the collection classes, the utilities, IO and other tools.
Page No. 23
ANDROID APPLICATION DEVELOPMENT
Issue 7: February 2014
With all upcoming applications and mobile services Google Android is stepping into the next level of Mobile Internet. Android participates in many of the successful open source projects. That is, architect the solution for participation and the developers will not only come but will play well together. This is notable contrast with Apple and other companies, where such architecture of participation is clearly belated.
Thus, some operating systems are designed to be convenient, others to be efficient, and others some combination of the two and developing the application in android platform is the good decision to compete with the present scenario of the global market.
Page No. 24
CYBER SECURITY NEWS AND EVENTS ISSUE8, FEBRUARY 2014 A controversial US surveillance programme that sweeps internet usage data had 700 snooping servers installed at 150 locations around the world, including one in India, according to a report. The XKeyscore programme, run by the National Security Agency (NSA), allowed analysts to search through vast databases containing emails, online chats and browsing histories of millions of individuals, Guardian reported, citing documents provided by whistleblower Edward Snowd en.
Android phones using Bing under threat Cyber security sleuths have alerted Android software based mobile phone users against a suspicious activity being detected in the commonly used search engine 'Bing' on Internet-enabled communication devices. Terming the severity of the malfunction as "medium", the Indian cyberspace security agencies have alerted users that a possible attack of the virus could "compromise" sensitive information of the owner stored on the phone. The system that is found affected is the 'Microsoft Bing version 4.2.0 and prior on Android'. "Arbitrary code execution vulnerability has been reported in Microsoft Bing for Android which could allow remote attackers to execute arbitrary code and install arbitrary APK (Android application package file) on Android devices," an advisory issued by the Computer Emergency Response Team-India (CERT-In) said. The CERT-In is the nodal agency to combat hacking, phishing and to fortify security-related defences of the Indian internet domain. Describing the abnormal activity, the agency in its latest advisory said "a flaw has been reported in Microsoft Bing for Android which could trigger while handling DNS (domain name system) responses on an insecure network." "An attacker could leverage this issue to executing arbitrary code within the context of the application. Successful exploitation of this vulnerability could allow an attacker to install arbitrary APK files via vectors involving a crafted DNS response, leading to the compromise of the device and resulting in information disclosure," the agency said. Cyber security experts said Android phone users taking the help of Bing to surf the Internet should update their existing versions and deploy proper anti-virus on their systems.(Source : Zee news)
Bit9's new Carbon Black sensor to be integrated with Check Point firewalls Days after merging its operations with tiny start up Carbon Black, US security firm Bit9 has announced a new deal that will see its technology integrated with Check Point firewalls. The partnership goes some way to explain why Carbon Black found itself on Bit9’s radar. The latter’s innovation is an endpoint sensor that will feed alert data back to Check Point’s Threat Emulation layer, giving an integrated attack detection and incident response in a one-er. Although rival security vendors downplay firewalls as almost vestigial devices, in Check Point’s Next Generation Firewall universe they remain the central point of policy setting and control for network security. It's a model in which remote sensors generate information that is fed back to the firewall to co-ordinate a response. The firms will work to complete the integration between these technologies by the first half of 2014, a fascinating admission; Bit9 has only had Carbon Black in its stable for a week and already getting the Check Point partnership up and (Continued to page 2………………. Page
CYBER SECURITY NEWS AND EVENTS ISSUE8, FEBRUARY 2014 A controversial US surveillance programme that sweeps internet usage data had 700 snooping servers installed at 150 locations around the world, including one in India, according to a report. The XKeyscore programme, run by the National Security Agency (NSA), allowed analysts to search through vast databases containing emails, online chats and browsing histories of millions of individuals, Guardian reported, citing documents provided by whistleblower Edward Snowden.
India investigates alleged Huawei 'hacking' of mobile base station Chinese equipment maker Huawei is reportedly being investigated by the Indian Government over alleged „hacking‟ of equipment belonging to state-run telecoms firm Bharat Sanchar Nigam Ltd (BSNL). The concerns are believed to be connected to media reports of interference with a mobile base controller belonging to BSNL in the state of Andhra Pradesh by engineers working for Huawei. "An incident about the alleged hacking of Bharat Sanchar Nigam Ltd (BSNL) network by Huawei has come to [our] notice," junior minister for communications and information technology Killi Krupa Rani was quoted as saying by Reuters in a written reply to a parliamentary question. "The government has constituted an inter-ministerial committee to investigate the matter," she added without elaborating on details. What „hacking‟ means in this context is open to interpretation but the equipment is believed to have been installed by Huawei‟s Chinese rival, ZTE, in 2012. The issue could be local to a piece of equipment or of wider significance. India has a history of suspicion over the Chinese equipment vendors building the country‟s communications systems, and like their US and European counterparts, the Indian intelligence services are reported to monitor Huawei‟s behaviour. "Huawei India denies such alleged hacking and continues to work closely with customers and governments in India to address any network security issue that may arise in technical and business operations," a company spokesperson was quoted by Reuters as saying. (Source: Techworld)
Bit9's new Carbon Black sensor to be integrated with Check Point firewalls (……………Continued from page 1) running is a driving influence for the sensor. Bit9 and Check Point are also working to integrate the NGF with Bit9‟s own security platform. “Integrating the Check Point Threat Emulation Service with the Bit9 Security Platform, now with Carbon Black, extends real-time malware prevention, detection, analysis and response to every endpoint and server,” said Bit9‟s vice president of product management, Brian Hazzard. “The best protection is to secure as many endpoints and servers as possible and put the rest in a „detectdetonate-deny‟ posture that allows for real-time security policy enforcement as threats appear.” The emergence of these systems is a belated response to the incredible vulnerability of the PC, which firewall vendors used to ignore as someone else‟s problem. Increasingly, endpoint defence is about protecting these vulnerable devices from the network or cloud level. (Source: Techworld) Page
MACHINE INTELLIGENCE NEWS AND EVENTS ISSUE: 8, FEBRUARY 2014
Battery-free technology recognition to all devices
brings
gesture
University of Washington computer scientists have built a lowcost gesture recognition system that runs without batteries and lets users control their electronic devices hidden from sight with simple hand movements. The prototype, called "AllSee," uses existing TV signals as both a power source and the means for detecting a user's gesture command. "This is the first gesture recognition system that can be implemented for less than a dollar and doesn't require a battery," said Shyam Gollakota, a UW assistant professor of computer science and engineering. "You can leverage TV signals both as a source of power and as a source of gesture recognition." The technology is set to appear April 2-4 at the Symposium on Networked Systems Design and Implementation conference in Seattle. The researchers built a small sensor that can be placed on an electronic device such as a smartphone. The sensor uses an ultralow-power receiver to extract and classify gesture information from wireless transmissions around us. When a person gestures with the hand, it changes the amplitude of the wireless signals in the air. The AllSee sensors then recognize unique amplitude changes created by specific gestures. Sensors use three to four times less power than existing gesture recognition systems by harvesting power from wireless transmissions. This allows for mobile devices to always have the gesture technology on and enabled. Gesture recognition already is possible on some mobile devices, including the Samsung Galaxy S4 smartphone. But users have to first manually enable the feature and be able to see the device for the gesture technology to work, and if left on, the gesture system quickly drains the phone's battery. In contrast, AllSee consumes only tens of microwatts of power and can always be left on. The user could gesture at the phone in a pocket or handbag to change the volume or mute the phone without having to touch or see the phone. This technology could allow sensors to be attached to household electronics, making it possible to interact with everyday objects using gestures and also connect them to the Internet and to each other in an "Internet of Things" world. "Beyond mobile devices, AllSee can enable interaction with Internet of Things devices. These sensing devices are increasingly smaller electronics that can't operate with usual keypads, so gesture-based systems are ideal," said Bryce Kellogg, a UW doctoral student in electrical engineering. The UW team tested AllSee's capabilities on smart phones and battery-free sensors using eight different hand gestures such as pushing or pulling to zoom in and out. (Continued to page 2……………………….
Bats inspire 'micro air vehicle' designs: Small flying vehicles complete with flapping wings, may now be designed More than 1,000 species of bats have hand membrane wings, meaning that their fingers are essentially "webbed" and connected by a flexible membrane.But understanding how bats use their wings to manipulate the air around them is extremely challenging -primarily because both experimental measurements on live creatures and the related computer analysis are quite complex.In Virginia Tech's study of fruit bat wings, the researchers used experimental measurements of the movements of the bats' wings in real flight, and then used analysis software to see the direct relationship between wing motion and airflow around the bat wing. They report their findings in the journal Physics of Fluids. "Bats have different wing shapes and sizes, depending on their evolutionary function. Typically, bats are very agile and can change their flight path very quickly – showing high manoeuvrability for midflight prey capture, so it's of interest to know how they do this," explained Danesh Tafti, the William S.Cross professor in the Department of Mechanical Engineering and director of the High Performance Computational Fluid Thermal Science and Engineering Lab at Virginia Tech. To give you an idea of the size of a fruit bat, it weighs roughly 30 grams and a single fully extended wing is about 17 x 9 cm in length, according to Tafti. Among the biggest surprises in store for the researchers (Continued to page 2…………….. Page
MACHINE INTELLIGENCE NEWS AND EVENTS ISSUE: 8, FEBRUARY 2014
Battery-free technology brings gesture recognition to all devices................continued from page 1)
Bats inspire 'micro air vehicle' designs: Small flying vehicles complete with flapping wings, may now be designed............... Continued from page 1)
was how bat wings manipulated the wing motion with correct timing to maximize the forces generated by the wing. "It distorts its wing shape and size continuously during flapping," Tafti noted.
The prototype could correctly identify the gestures more than 90 percent of the time while performed more than 2 feet away from the device. Researchers have tested the technology for response time and whether it can distinguish between other motions and those directed at it. They found that the technology's response time is less than 80 microseconds, which is 1,000 times faster than blinking an eye."This enables a seamless and interactive experience for the user," said Vamsi Talla, a UW doctoral student in electrical engineering. The researchers also designed a wakeup gesture that allows the system not to confuse unintentional motions for actual gestures. This technology builds on previous work by Gollakota on leveraging Wi-Fi signals around us for gesture recognition around the home. Prior wireless gesture recognition techniques, however, consume tens of watts of power and aren't suitable for mobile or Internet of Things devices. The research is funded by a Google Faculty Research Award and the Washington Research Foundation. (Source: Science Daily)
For example, it increases the area of the wing by about 30 percent to maximize favorable forces during the downward movement of the wing, and it decreases the area by a similar amount on the way up to minimize unfavourable forces. The force coefficients generated by the wing are "about two to three times greater than a static airfoil wing used for large airplanes," said Kamal Viswanath, a co-author who was a graduate research assistant working with Tafti when the work was performed and is now a research engineer at the U.S. Naval Research Lab's Laboratories for Computational Physics and Fluid Dynamics. This study was just an initial step in the researchers' work. "Next, we'd like to explore deconstructing the seemingly complex motion of the bat wing into simpler motions, which is necessary to make a bat-inspired flying robot," said Viswanath. The researchers also want to keep the wing motion as simple as possible, but with the same force production as that of a real bat . (Source: Science Daily)
Page
