Posted on May 3, 2017

This simulation was developed as part of the final class project for the course Technical Character Animation.

The simulation showcases a small desktop VR game featuring a cartoon forest with giant marshmallows. The player is tasked with roasting and consuming as many giant marshmallows as possible in a duration of 5 minutes. The player is represented using a 3d character model and can move around in the scene using the Microsoft Kinect version 2 sensor. The player can make punch gestures in front of the Kinect sensor to shoot fire out of his/her hands and roast the Marshmallows before consuming them to gain more points. The simulation uses the Unity game engine for the environment and the Kinect sensor for body tracking and gesture recognition to produce fire effects in the environment.

A custom gesture recognition algorithm was written to identify a punch gesture. The algorithm calculates the direction of the punch and an associated force value based on the speed. The fire effects produced in the game as a result of the gesture recognition use the punch direction and the force value to produce fire effects proportional to the intensity of the punch. The character movement in the environment was achieved using a technique similar to the Human Joystick technique.

The upper body movements of the player were mapped exactly onto the avatar in the game but the lower body of the avatar had walking animations based on which direction the player was stepping in in front of the Kinect sensor.

The video below demonstrates the final product.