Introduction of Our Project

This project we will study on body movement and motion capture.

In this project, Kinect and leap motion are used by us to capture the characteristics of depth information.Using depth image and motion capture we are able to explore more possibilities behind actions. An action may contain many information, such as motion, track and emotion.

We made those actions present in a richer visual form, also combining with music in order to reach our goal of interaction.

In this project, we use Kinect and Leap Motion as the hardware; Completing the code with Processing; Using Open Kinect library and Syphon library. Other software we used in this project are Resolume Arena, Cinema 4D, Blender and After effect.

Process Step 1：Depth Capture

Using Kinect in order to capture depth image and then written code by using processing to achieve a mask of movement.

We use Kinect to get the depth image and use the Processing Open Kinect library to get a certain depth range of pixels.

Step 2：Visualization

By using Processing to control particles or color with depth images.

Step 3：Leap Motion

Through using Leap Motion to capture fingers position and use this information to control elements, such as movement or color.

Step 4：Real Time Effect

First, using Kinect and Processing in order to get depth image. Then, use Syphon library to transmit the video into a VJ software Rseolume Arena in real time. Finally, use Rseolume Arena to achieve real-time effect with music to complete this interactive project.

Step 5：Skeleton Motion Capture and 3D Animation

Using Kinect at first to get information of image from people's bodies that in front of the sensor, then plus using NI mate to make skeleton motion captures. Import captures into blender and use Cinema 4D, we can finish a 3D animation.

Source Code

You can download our source codes from Github.

source code

Final Video

A final video of our project.

Thanks and Others

It's the namelist we are grateful for.

  • Special Thanks
  • Miguel ELIZALDE PEDROUSO
  • Daniel Shiffman
  • Levente Sandor
  • Raven Kwok
  • Greyscalegorilla
  • Ken Ottmann
  • -
  • Music
  • Intuition (Instrumental) by Get Better
  • Hardware
  • Kinect
  • Leap Motion
  • Digital Camera
  • -
  • Program
  • Processing
  • -
  • Library
  • Open Kinect for Processing
  • Leap Motion for Processing
  • Syphon
  • Software
  • Kinect Studio
  • Rseolume Arena (Real time VJ)
  • NI mate（Motion capture）
  • Blender（3D animation）
  • Cinema 4D（3D animation）
  • Mixamo（Model and binding）
  • Turblence fd（Smoke effect）
  • Octane renderer
  • Krakatoa renderer
  • After Effect（Compose）
  • Final cut pro（Edit）