Saturday, January 2, 2016

Working with Kinect Camera

App Type     : WPF
Language     : C#
Kinect SDK : v1.8

In this post we are going to create a simple cam application.
Within this cam application we are going to develop the following feature:
1. Start and Stop the Kinect sensor
2. Screen to display the current view of the Kinect Camera.
3. Capture the current view of the Kinect Camera

We are going to create this application which has the bellow design (You can change the design according to your imagination)


XAML design:

<Window x:Class="KinectInfoBox.KinectCam.CamWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="Kinect Cam" Height="500" Width="800" Closed="Window_Closed_1">
    <Grid>
        <Image Name="CamControl" Stretch="Fill" HorizontalAlignment="Left" Height="213" Margin="200,60,0,0" VerticalAlignment="Top" Width="400" />
        <Button Content="Start" HorizontalAlignment="Left" Margin="200,300,0,0" VerticalAlignment="Top" Width="100" Click="Button_Click_2" />
        <Button Content="Stop" HorizontalAlignment="Left" Margin="350,300,0,0" VerticalAlignment="Top" Width="100" Click="Button_Click_1"/>
        <Button Content="Capture" HorizontalAlignment="Left" Margin="500,300,0,0" VerticalAlignment="Top" Width="100" Click="Capture_Clicked"/>
        <TextBox Name="txtCamAngle" HorizontalAlignment="Left" Height="23" Margin="350,350,0,0" TextWrapping="Wrap" Text="" VerticalAlignment="Top" Width="100" />
        <Button Name="btnSetAngle" Content="Ok" HorizontalAlignment="Left" Margin="500,350,0,0" VerticalAlignment="Top" Width="100" Click="btnSetAngle_Click"/>
    </Grid>
</Window>

.cs file:

namespace KinectInfoBox.KinectCam
{
    public partial class CamWindow : Window
    {
        private KinectSensor Sensor;

        public CamWindow()
        {
            InitializeComponent();
        }

        private void StartKinectCam()
        {
            if (KinectSensor.KinectSensors.Count > 0)
            {
                Sensor = KinectSensor.KinectSensors.FirstOrDefault(sensorItem => sensorItem.Status == KinectStatus.Connected);
                if (Sensor != null && !Sensor.IsRunning)
                {
                    Sensor.Start();
                }
                Sensor.ColorStream.Enable();
                Sensor.ColorFrameReady += Sensor_ColorFrameReady;
            }
            else
            {
                MessageBox.Show("Kinect is not ready!", "Message");
            }
        }

        void Sensor_ColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
        {
            byte[] pixeldata;

            using (ColorImageFrame imageFrame = e.OpenColorImageFrame())
            {
                //check if the incomming frame is not null
                if (imageFrame == null)
                {
                    return;
                }
                else
                {
                    //Get the pixel data in the byte array
                    pixeldata = new byte[imageFrame.PixelDataLength];

                    //Copy the pixel data
                    imageFrame.CopyPixelDataTo(pixeldata);

                    //calculate the stride
                    int stride = imageFrame.Width * imageFrame.BytesPerPixel;

                    //assign the bitmap image source into image control
                    CamControl.Source = BitmapSource.Create(imageFrame.Width, imageFrame.Height, 96, 96, PixelFormats.Bgr32, null, pixeldata, stride);
                }
            }
        }

        private void Window_Closed_1(object sender, EventArgs e)
        {
            if (Sensor != null && Sensor.IsRunning)
            {
                Sensor.Stop();
            }
        }

        //Save image in the projects' bin -> debug
        private void SaveImage()
        {
            using (FileStream fileStream = new FileStream(string.Format("{0}.jpg", Guid.NewGuid().ToString()), System.IO.FileMode.Create))
            {
                BitmapSource imageSource = (BitmapSource)CamControl.Source;
                JpegBitmapEncoder jpeegEncoder = new JpegBitmapEncoder();
                jpeegEncoder.Frames.Add(BitmapFrame.Create(imageSource));
                jpeegEncoder.Save(fileStream);
                fileStream.Close();
            }
        }

        private void Capture_Clicked(object sender, RoutedEventArgs e)
        {
            if (Sensor.IsRunning && Sensor.ColorStream.IsEnabled)
            {
                SaveImage();
            }
        }

        private void btnSetAngle_Click(object sender, RoutedEventArgs e)
        {
            int angle = int.Parse(txtCamAngle.Text);
            if (angle > Sensor.MinElevationAngle && angle < Sensor.MaxElevationAngle)
            {
                Sensor.ElevationAngle = angle;
            }
            else
            {
                MessageBox.Show("Invalid value. Enter value between -27 and 27", "Error");
            }

        }

        private void Button_Click_1(object sender, RoutedEventArgs e)
        {
            if (Sensor.IsRunning)
            {
                Sensor.Stop();
                CamControl.Source = null;
            }
        }

        private void Button_Click_2(object sender, RoutedEventArgs e)
        {
            StartKinectCam();
        }
    }
}

Friday, December 11, 2015

Kinect v1 with ASP.Net MVC 4: Building MVC App to get Kinect Information

SDK Version: 1.8

Usually Kinect applications  are develop as stand alone applications. In this blog I'll describe how to develop web based application with Kinect.

In this post I'm going to create an MVC 4 application which show some basic Kinect Information.

KinectInfo Model

This model use to set and get the Kinect Information.
Define this class in the Model section in your ASP.Net MVC application.

public class KinectInfo
{
        public string ConnectionId { get; set; }
        public string UniqueDeviceId { get; set; }
        public string Status { get; set; }
        public bool ColourStream { get; set; }
        public bool DepthStream { get; set; }
        public bool SkeletonTracking { get; set; }
        public int SensorAngle { get; set; }
}

We are going to create a model helper for set the values getting from the KinectSensor class in the Kinect SDK.

public KinectInfo SetKinectInfo(KinectSensor kinect)
{
     KinectInfo _infoModel = new KinectInfo()
     {
          ConnectionId = kinect.DeviceConnectionId,
          UniqueDeviceId = kinect.UniqueKinectId,
          Status = kinect.Status.ToString(),
          ColourStream = kinect.ColorStream.IsEnabled,
          DepthStream = kinect.DepthStream.IsEnabled,
          SkeletonTracking = kinect.SkeletonStream.IsEnabled,
          SensorAngle = kinect.ElevationAngle

     };
     return _infoModel;

}

Controller class

This is the controller class of our MVC Application.


public class HomeController : Controller
{
        private static KinectSensor kinect;

        public ActionResult Index()
        {
            InitializeSensor();
            return View();
        }


        private void InitializeSensor()
        {
            try
            {
                if (KinectSensor.KinectSensors.Count > 0)
                {
                    kinect = KinectSensor.KinectSensors[0];
                    StartKinect();
                    kinect.DepthStream.Enable();
                    kinect.ColorStream.Enable();
                    kinect.SkeletonStream.Enable();
                }
            }
            catch (Exception e)
            {
                ViewBag.Msg = string.Format("Error: {0}", e.ToString());
            }
        }






This method define to Initialize the Kinect sensor.

This assume only one Kinect is attached to your system.
        kinect = KinectSensor.KinectSensors[0];

 Then we enable the data streams that we want. (ex: skeleton stream to capture the body movements)       
        private ActionResult StartKinect()
        {
            if (kinect != null && !kinect.IsRunning)
            {
                kinect.Start();
                ViewBag.Msg = "Kinect Started";
            }
            return View();
        }

 
The above method allows to start the Kinect sensor.
If the kinect sensor is not started we can't use it functionality.
       
        public ActionResult StopSensor()
        {
            if (kinect != null && kinect.IsRunning)
            {
                kinect.Stop();
                ViewBag.Msg = "Kinect Stoped";
                kinect = null;
            }
            return View();
        }


The Kinect stop method allows to stop the kinect sensor when we done with the work and when we want to close the application. 

        public ActionResult ShowKinectInfo()
        {
            KinectInfoHelper helper = new KinectInfoHelper();
            var infoModel = helper.SetKinectInfo(kinect);
            return View(infoModel);
        }


This action result use to get the Kinect infomation and pass it to the view.  

}

ShowKinectInfo view

This is the front end for our application.
We bind the KinectInfo model with this view.

@model KinectWebMvc.Models.KinectInfo

@{
    Layout = null;
}

<!DOCTYPE html>

<html>
<head>
    <meta name="viewport" content="width=device-width" />
    <title>ShowKinectInfo</title>
</head>
<body>
    <div>
        <table>
            <tr>
                <td>Connection Id: </td>
                <td>@Html.DisplayTextFor(m => m.ConnectionId)</td>
            </tr>
            <tr>
                <td>Unique Device Id: </td>
                <td>@Html.DisplayTextFor(m => m.UniqueDeviceId)</td>
            </tr>
            <tr>
                <td>Status: </td>
                <td>@Html.DisplayTextFor(m => m.Status)</td>
            </tr>
            <tr>
                <td>Colour Stream: </td>
                <td><input type="checkbox" checked="@Html.CheckBoxFor(m => m.ColourStream)"/></td>
            </tr>
            <tr>
                <td>Depth Stream: </td>
                <td><input type="checkbox" checked="@Html.CheckBoxFor(m => m.DepthStream)"/></td>
            </tr>
            <tr>
                <td>Skeleton Tracking: </td>
                <td><input type="checkbox" checked="@Html.CheckBoxFor(m => m.SkeletonTracking)"/></td>
            </tr>
            <tr>
                <td>Sensor Angle: </td>
                <td>@Html.TextBoxFor(m => m.SensorAngle)</td>
            </tr>
        </table>
    </div>
</body>
</html>


Now you are ready to go.
Run the application and have fun.

Friday, November 6, 2015

ForceInfraredEmitterOff giving compile error

We all need to debug our applications. When developing applications with Kinect devices also we are going to debug our application. But when when we stop debugging using the VS (Visual Studio) the IR (Infrared) light of our Kinect will be still on.
In order to stop this there is a property called 'ForceInfraredEmitterOff'. This accept boolean value (true or false). Read more on this from here
We can set this to 'true' to forcefully turn off the IR light when we stop the debugging in VS.

But some of you like me will experience the application will not compile and it will give a compilation error when this property is defined.
The reason for this is you are using a Kinect device which use for XBOX 360.
The XBOX 360 Kinect device not support this property and this property is for Kinect for Windows devices.

So if your XBOX 360 Kinect device's IR light is still on when you stop the debugging, you can run the program again and close it (using the application close button). This will then turn off the IR light with the other pipelines as well.