Matthieu Deru's Homepage

I read that a lot of people are having problems with the Unity WebCamTexture and Android. Indeed the problem comes from the fact that the texture streamed from the webcam is always flipped by 180 degrees and you will always get a wrong rotated picture of the camera (even if you turn your device (see picture)). This is really annoying as you would expect that the WebcamTexture will show you a correctly orientated picture of the camera. Obviously the API of WebCamTexture doesn’t help a lot doing this and you’re ready to scratch your head…because GUITextures can theoretically not be rotated (ok, yes they can but by using GUIUtility.RotateAroundPivot() or by playing with the Matrix functions)… but a little overkill for just showing our camera stream isn’t it? How to solve this? The solution is really easy, no need to use complex maths or pixel copying. Note: this solution might also apply for iPhone or iPad (not tested, but the problem seems to exist). Please also read the update section at the end of this post.

Step 0: Our problem: the camera picture is always inverted

Ouch, our poor Android robot has its head down no matter how we turn our device. We need a solution!

Step 1: Setup your Unity scene

Add a Camera and a GUITexture. You can leave the default “Unity Logo Texture” or use your own texture. It doesn’t matter, as this texture will be used as a holder for the camera’s streamed picture. You can also change the name of UnityWatermark. For the purpose of this tutorial I’ve changed the name into MyCameraPicture. In the Pixel Inset Box, set X,Y, Width and Height to the value 0. Your GUITexture shouldn’t be visible no more, but don’t worry it’s normal.

Step 2: WebCamScript.cs

Create a new C# script and name it WebCamScript and drag’n’drop it on the GUITexture named MyCameraPicture placed in your Hierarchy view.

using UnityEngine;
using System.Collections;
using System.IO;
using System;

public class WebCameraScript : MonoBehaviour {

    public GUITexture myCameraTexture;
    private WebCamTexture webCameraTexture;

    void Start () {

        // Checks how many and which cameras are available on the device
        for (int cameraIndex = 0; cameraIndex < WebCamTexture.devices.Length; cameraIndex++)
        {
            // We want the back camera
            if (!WebCamTexture.devices[cameraIndex].isFrontFacing)
            {
                webCameraTexture = new WebCamTexture(cameraIndex, Screen.width, Screen.height);

                // Here we flip the GuiTexture by applying a localScale transformation
                // works only in Landscape mode
                myCameraTexture.transform.localScale = new Vector3(-1,-1,1);
            }

        }    

        // Here we tell that the texture of coming from the camera should be applied 
        // to our GUITexture. As we have flipped it before the camera preview will have the 
        // correct orientation
        myCameraTexture.texture = webCameraTexture;

        // Starts the camera
        webCameraTexture.Play();
    }

    public void ShowCamera()
    {
        myCameraTexture.guiTexture.enabled = true;
        webCameraTexture.Play();
    }

    public void HideCamera()
    {

        myCameraTexture.guiTexture.enabled = false;
        webCameraTexture.Stop();

    }

    void OnGUI()
    {
        // A button to demonstrate how to turn the camera on and off, in case you need it
        if(GUI.Button(new Rect(0,0,100,100),"ON/OFF"))
        {
            if(webCameraTexture.isPlaying)
                this.HideCamera();
            else
                this.ShowCamera();
        } 
    } 
}

The trick is all in this single line:

myCameraTexture.transform.localScale = new Vector3(-1,-1,1);

We apply a negative scale to the target texture that is the equivalent to make a 180° rotation of our picture.

Note that these transform.localScale parameters may vary if you have another phone (perhaps your Android or iPhone mobile phone delivers the camera picture with the correct orientation), in this case you can apply this line:

 myCameraTexture.transform.localScale = new Vector3(1,1,1);

You should have following screen at the end of the two steps:

Screenshot of Unity 3D showing the attached WebCameraScript.cs and the parameteres for the GUITexture.

Step 3: Deploy and test it

Before deploying don’t forget to change the orientation mode (Landscape Left/Right)

Step 4: The robot is now displayed correctly (and is happy)

The problem is solved 🙂 The camera picture is shown with the correct orientation.

If you are not completely happy with this solution or need another orientation angle for your Camera Picture here is an explanation how to solve this problem in another manner:

http://answers.unity3d.com/questions/11022/how-to-rotate-gui-textures.html

Update #1

I’ve received a lot of comments and emails stating that some devices show a black screen instead of the camera’s picture. The following tutorial was realized by using Unity 3.5 f5 and the minimum API Level is Android 2.0.1. For your testing purpose, here is a screenshot of the Publish settings of the tutorial’s project.

I’ve also tested it successfully on a Dell Streak 5 (see picture) with Android 2.2.2 (which is quite old). On the Dell Streak 5 you must comment the following line to flip the camera’s picture correctly.

myCameraTexture.transform.localScale = new Vector3(-1,-1,1);

Test with a Dell Streak 5 : the WebcameraTexture is displayed correctly.

On Jelly Bean (Android 4.1 over Cyanogenmod) on an Acer Iconia Tab the camera’s picture remains black… 🙁 The same issue with the brand new Nexus 7. Seems as if Jelly Bean is not correctly supported.
So what is the solution? I suspect that the internal Unity JNI calls to get the camera’s picture (you can check these calls over DDMS and the Logs) are buggy or simply not working and issuing internal errors on some Android versions. It’s in fact a curious problem. So to help others it would be nice to tell in the comments which Android version you are using and if the Log has written a specific error message.

But is there a real solution to solve this problem? Yes, here is the conceptual idea: Over an Unity Android Plug-In (JNI) you can get the camera’s raw picture data (over the Camera.PictureCallback). You can then send this raw data (actually a byte array) to Unity. The C# part in Unity must then decode this raw data. At this step you can then recreate the picture with the Texture3D.SetPixel() function and apply this texture to any GameObject or GUITexture … but it’s not an easy thing to implement!

Update #2

Here are the information concerning my Samsung Galaxy S3 (version used in this tutorial), as some visitors have difficulties to run the code on their own phone.