Unity 3D Tutorials - Marcelo Santos

Unity 3D Tutorials Augmented Reality using Unity 3D and FaceAPI

By Sriram.A.S. Date: 3rd Dec, 2010 [email protected] Originally Posted in: http://mypersonalsoft.blogspot.com/ (Keep checking the blog for updates)

About this Tutorial This tutorial is about implementing Augmented/Virtual Reality using Unity 3D and FaceAPI. Demo video about this tutorial is posted in YouTube. http://www.youtube.com/watch?v=yN_YEnDs2uk

Requirements  Unity 3D (http://unity3d.com)  FaceAPI Non Commercial Edition (http://www.seeingmachines.com/product/faceapi)  6dofstreamer (http://code.google.com/p/6dofstreamer) Download faceapistreamer___.zip file only  The main script (as Unity Package) http://megaupload.com/?d=YKX2B382 A little knowledge of C# and Sockets is absolutely required Tutorials 1. Install the FaceAPI Non Commercial Edition, file named setup_FaceTrackingAPI_NC___.exe 2. Extract the 6dofstreamer zip file, something named faceapistreamerxxx.zip

3. Open Unity 3D and create a new project (FileNew Project) 4. Import the file by AssetsImport Package 5. This is what you see after that

As you see that, a new scene s1 is added and script ‘script’ is also added 6. Click the ‘camera’ element and you can notice that the script ‘script’, which is a C# script is attached to this camera element

7. Now that is all, we are going to open the script and understand that. The script may look a bit fuzzy, but believe me its very easy. Open the script ‘script’ (more preciously script.cs 8. This is the script and its explanation: using UnityEngine; using System.Collections; using System; using System.Net; using System.Net.Sockets; using System.Text;

public class script: MonoBehaviour { //This is the port that 6dofstreamer sends the face details. They use UDP //sockets. The string output from the streamer may look like //A B C D E F G //I don’t remember the format order much, check its documentation. But //it just sends three rotations and three translations in three axes //It is to be noted that we are interested in 4th and 5th words which //describes one rotation and one translation private const int listenPort=29129; UdpClient listener = new UdpClient(listenPort); IPEndPoint groupEP = new IPEndPoint(IPAddress.Any, listenPort); String op; float pz,ry,ry_old=0; Vector3 newpos; //Note the update function void Update () { byte[] receive_byte_array= listener.Receive(ref groupEP); //op contains the string received by the 6dofstreamer //Live streaming is done, so each time when the frame reloads, we want //our update function to get the fresh string from the streamer op=Encoding.ASCII.GetString(receive_byte_array, 0, receive_byte_array.Length); //We form an array by splitting the string string[] words=op.Split(' '); //We are in particular about the 4th and 5th as I told before //Now we work with a bit of mathematics //ry for rotation y and pz for translation x pz=(float)Convert.ToDouble(words[3]); ry=(float)Convert.ToDouble(words[4]); //This is my simple mathematical model, for much more smoother transition, I recommend you to use your own mathematical mapping of //the STRING FROM THE STREAMER TO ANY CHANGE IN ORIENTATION OF //OUR OBJECT if(pz-0.4) pz=-50; else if(pz-0.3) pz=-40; else if(pz-0.2) pz=-30;

else if(pz-0.1) pz=-20; else if(pz0) pz=-10; else if(pz=0) pz=0; else if(pz>0 && pz0.1 && pz0.2 && pz0.3 && pz0.4 && pz0 && ry0.1 && ry0.2 && ry0.3 && ry0.4 && ry