Kamehameha!

Dr. Remy and I recently discussed our long term goals for the Point Cloud Web Service project and our destination of the "jump rope" project. He wanted me to come up with a design for it and we discussed parameters such as the frequency of the rope's "jump", how it would actually move, and how we would determine a passing jump. It was definitely decided that the jump rope's swing should have a repeating pattern or period rather than swing at random intervals. This would aid the user with jumping over it successfully as without a prototype, we don't know how well the spatial audio will portray the moving rope in just one pass. Likewise we needed to choose between actually have the rope swing down below the feet and back up again or just move horizontally at a constant height. Would the user be able to tell the difference between the two methods if implemented with the Web Audio API through the browser? And would they be able to track the location of the rope more accurately using one or the other? Questions like these can only be answered with a prototype, but we can form hypotheses about them based off the Skeleton Tracking Audio program I've already implemented.

Dr. Remy also brought up one of my other ideas, basketball. We discussed how the user would locate the  ball, hold and move it around, and actually shoot it. My original idea was to have the ball be "invisible" and "silent" and for a shooting motion to initiate the sound of a shot, or a wooshing ball as it moves towards the basket and then a noise from the basket to indicate the accuracy and power of the shot versus how on target it was. We also pondered about allowing the user to control the ball before the shot and perhaps pass it around between each hand. This would aid with getting a feel for how your virtual ball behaved before you shot it as well as assist accuracy during the actual shot. This would be slightly more difficult though as it involves using the Kinect data to tell the difference between flinging the ball side to side versus a full shot straight forward on a perpendicular axis. We determined that perhaps a better method was to allow the user to hold  both hands together to "initialize" the ball and sound associated with it and then let them shoot at their leisure. If their hands move apart, the ball and sound disappear. You can't play around with the ball physics prior to the shot but you would still be able to see/hear how.

Talking about this design made me think of someone else's Kinect project that I had seen on youtube where they juggled several balls. Unfortunately I forgot to add it to my early blog post on example OpenKinect projects but as it turns out, it was done with OpenNI so that may have been why I forgot it. I've gone back and added it and here is a link to the juggling video. In the description is a link to his code as well. In his video it appears that he is using acceleration vectors from the motion of his hands to determine when/where to throw the balls. Otherwise when a ball collides with his hand it automatically sticks to it. Also, it appears that placing his hand out of sight and bringing it back in places a new ball in his hand. When balls move off screen they are removed and the color of a new ball is based on how many are on the screen at a given time. Interestingly the balls move in three dimensions as can be seen around the 50s mark in the video when he throws them at the camera and they grow largely in size (and at the beginning of the video when he demonstrates it). It's hard to determine if he must match the z-axis of his hand to the ball however in order to catch it as this would make catches much more difficult.

Out of curiosity, I followed the link in the description of his video to his "inspiration" which is this Kinect project that imitates a Kamehameha from Dragonball-Z. We were both intrigued by this other project as it is quite different from the more common "ball" or "object" tossing Kinect program. We also believe it might have some interesting applications to our work with point cloud web services and would be a better fit than jump rope, basketball, or juggling. The Kamehameha uses the same skeleton tracking API I was using (Microsoft Kinect for Windows SDK). The program highlights the persons outline with an "aura" and gives them the classic spiky hair from Dragonball-Z. When the user puts their hands together, a shiny orb is formed. Crouching and bringing the arms into a certain position makes the orb grow and when it is large enough, shifting your arms forward shoots the orb and results in a large beam of light (with some other fun side effects). Something to note is that it is very important the Kinect see your entire skeleton. The legs are very significant and without the entire view of your legs as you crouch, the orb won't grow very much and is hard to trigger.