Web-Kamehameha with Beam

After receiving help on the geometry necessary to draw the beam from Dr. Remy, I was successfully able to write code to incorporate it into Web-Kamehameha. The below video shows the final product.

The solution to drawing the beam was to use the atan2 function (featured in almost every programming language) to get the angle of the center of the beam from the two given points. Then, using a factor of energy level and radians, I adjusted that angle to get the angle of the either side of the beam. From the newly computed angle, I determined which edge of the canvas the beam would hit (top, bottom, left, or right). Then, depending on the edge, I could determine the X or Y coordinate of the point to connect the base of the beam to. Given that X or Y coordinate, I could calculate it's distance from the opposite axis and use the tan() function and the basic formula tan = opposite / adjacent to find the length of the missing side, which would be directly equal to the missing coordinate. Below is my code.

function drawBeamSide(ballBase, angle) {
 var adjacent = null;
 var opposite = null;
 var newX = ballBase[0];
 var newY = ballBase[1];

 // transpose the angle to the range [0, 2PI)
 if (angle < 0) {
  angle += 2 * Math.PI;
 }

 // determine which edge this beam should hit as it
 // will affect the calculations
 if (angle < 1 / 4 * Math.PI || angle > 7 / 4 * Math.PI) {
  // Hits right edge
  adjacent = ballBase[0] - 500; // distance from point to right edge
  opposite = Math.tan(angle) * adjacent; // use trig to find Y coordinate
 } else if (angle <= 3 / 4 * Math.PI) {
  // Hits top edge
  opposite = 0 - ballBase[1];
  adjacent = opposite / Math.tan(angle);
 } else if (angle <= 5 / 4 * Math.PI) {
  // Hits left edge
  adjacent = ballBase[0];
  opposite = Math.tan(angle) * adjacent;
 } else {
  // Hits bottom edge
  opposite = 500 - ballBase[1];
  adjacent = opposite / Math.tan(angle);
 }

 newX -= adjacent;
 newY += opposite;
 
 drawLine(ballBase[0], ballBase[1], newX, newY);
}

The function's first argument is a two element array representing the X and Y coordinates of the ball's base (which is the same location for the beam's base) and then the angle (from the x-axis) of the beam to draw. Logic outside of this function already took care of changing the angle's and then calling this function to produce two sides to the beam.

The tricky thing to note is that the coordinate system for the HTML5 has a positive increasing Y value as you move down the image. The top of the image is Y = 0 while the bottom is Y = HEIGHT (or 500 in my case). The left is X = 0 and the right X = WIDTH.

Some interesting things to note about the program is that the depth appears to affect the accuracy or straightness of the beam. Tomoto's code, and what I've modified and put into my program, is simply drawinga vector along the line from the shoulder to the hand, however in the video below you will see that even when both are on the same Y level, the angle of the beam is quite off (too low). I noticed this effect increases with depth. It is not noticeable in the previous video since I crouched to increase the energy level. If I stand up straight and release a Kamehameha however you can see the problem.

In the below video I demonstrate this. I start off by showing the hand-distance threshold for ball generation. Then I release two crouched Kamehameha's for which the angle appears to be accurate. Then I release one standing and you can see that it shoots downward despite my arm being straight. I can even raise one hand to show the path of the beam, despite not having any energy, and you can see it is inaccurate.

Web-Kamehameha

The bare-bones web/claude based version of Kamehameha is almost complete. There is just one key piece of functionality missing, the energy beam! In the video below you will see that the user skeleton is being rendered and that when the hands come close enough together, an energy ball starts to grow. Things unnecessary to the core functionality, like dynamic lighting and the glowing auro, have been removed in favor of getting an example that simply proves the browser can be used to mediate the flow of data from the Kinect, to Kamehameha, to the browser.

When the energy ball is shot, by straightening the arms outward, currently all I have is a line showing the direction the beam should be going. The line is formed by the point at the center of the energy ball and what appears to be an arbitrarily chosen secondary point that forms a direction vector. I can't tell from Tomoto's code how the second point is calculated, however I haven't looked to far into it. His code also has no comments and is rather hard to follow. My idea for the beam is to have an angle (theta) that is a direct function of the energy level and represents how far a pair of lines is from the initial direction vector seen in the video. The lines can be of infinite length (to the edge of the screen) but they should each have a base at the ball center and one should be angle theta above the direction vector and the other theta below. I whipped up the below example in paint.

I thought I would be able to use basic geometry and trigonometry to find the X, Y, Z coordinates of the two lines that form the isosceles triangle to represent the beam, but it turned out to be harder than I thought. I basically need to use an angle and slope to find the slope of the lines for the beam, but then use that slope to calculate actual points. This problem can be thought of as 2-Dimensional since I am not using the Z dimension in my HTML5 rendering and it doesn't have any bearing on the visualization of the beam. But that didn't make it any easier as I couldn't figure out how to calculate the slope of a line theta degrees away from another line (sharing the circle center as a common base) nor how to take the slope of a line and given a pair of X, Y coordinates, find another pair of X, Y coordinates N units away from the first pair (where N is large enough to make the line go off screen). Or even more appropriately make the second pair of coordinates 0, Y where Y is a value such that 0, Y falls on that line, thus only drawing as much of the line as necessary.

Putting Force Data Online

To answer some questions from Dr. Remy's comment on the previous post, each Tactonic tile is 2'x2' with 24x24 sensels. Each sensel is one inch away from all adjacent sensels on the same panel. In the force array that the devices generate, the coordinates increase as follows:

The arrows indicate an increasing positive coordinate. It should be noted that when the array is printed, the results are not always consistent with this

I implemented a function to calculate coordinates of the center of pressure, as well, and it seems consistent with the pressure placed on the devices, though I have not had the chance to test this rigorously yet. The force data is also now printed to my localhost using Mongoose by Cesanta. The call to Mongoose prints a snapshot of the force array, and loading the page prints the force array stored at load time. Together, this data looks like so:

This is a record of me standing on the upper- and lower-middle-left of the master device (directions as indicated in above picture) with only the master device recording.

The program is set to run 3000 times, and the print function is only called when an HTTP Request is made. The print function prints out the run that the device is currently on, the force array for that run, and the center of pressure of that force array. The center of force is calculated when the print function is called.

Device 3A is currently not working, and when it is attached to the chain it causes a mirroring effect between 0A and 1A and also between 2A and 3A. The latter was discovered when I only put pressure on device 3A but pressure was recorded on 2A, which tells me that 3A can still determine force exerted upon it.

Mongoose Setup

The embeddable Mongoose server is distributed as mongoose.c and mongoose.h. All that is needed to use it is an #include "mongoose.h" in the file that uses the Mongoose API, and then include mongoose.c in the compile command. The server can then be started within a C or C++ program by calling the mg_start() function and passing it configuration parameters and an HTTP event handler callback function.

The mg_context struct holds the connection to the server once that connection is opened. The C-string array contains the configuration options needed to display the results of the Tactonic program. Currently, the port is set to 8080, the default for Mongoose, and the server root is being redirected to the location of the folder containing the program that calls the server (pardon the filepath names). C++ complains about the escaped spaces, but that is how spaces in the filepath must be represented on OSX.

Currently, the event callback just calls the printTouchRegistry() function, which has been altered to output a const char pointer instead of writing to file and to calculate the center of pressure when it is called.

The final piece is a call to mg_start().

This starts the server with the startup options and binds the event callback.

Currently, the server only posts a snapshot of the pressure data, as previously mentioned. If needed an asynchronous function can be made to update the pressure data in realtime. Mongoose works with Lua Server Pages, which could conceivably be used in conjunction with AJAX to update asynchronously.

Chrome vs. Firefox Performance

While working on the web version of Kamehameha, I've noticed a distinct performance difference between Chrome and Firefox. I don't know exactly what is causing the difference but the results can seen in this video. (Firefox on the left, Chrome on the right).

I am using the HTML5 canvas to render the skeleton however in the data for the joint locations come from GET requests fired every 50ms. It could be the HTML5 rendering engine, web request efficiency, or possibly something else that is accounting for the difference. To clarify, while the motion of my skeleton in firefox is smooth and consistent, in chrome it is very choppy and laggy. At times, chrome would run smooth too until it had been running for about 30s, then chrome would start to lag but firefox would still maintain its performance. Also note that the flashing mess of lines that appears in the lower left corner of each browser is the Kinect confusing my jacket, which is hanging on a chair in the background, for a skeleton.

Pressure Output

The output issue was solved today, and pressure output is now recorded from the Tactonic device. As of now, it writes the values to a text file whenever a nonzero force is recorded at every 20th call to the device. The output looks like this:

This is a record of me standing on two of the tiles.

The code for this operation doesn't do anything unusual: it grabs the recorded pressure values and outputs them to a file specified at the command line.

 

Whenever the device's touch callback is called, it calls the registerTouches() then check the result of that function and the number of times registerTouches() was called and runs printTouchRegistry() if writeFlag was true and the function has run some multiple of 20 times.

Segfault Solved

The major roadblock I encountered writing code to extract raw data from the Tactonic device was a segmentation fault in the startup phase. When the code attempted to assign a device to the custom visualizer class, execution would immediately stop due to an EXC_BAD_ACCESS error being thrown (see debug output below).

The instance of the custom viewer was originally a viewer pointer, but after changing it to a non-pointer instance the EXC_BAD_ACCESS error was no longer thrown, and the device paired with the viewer normally. I managed to output the force matrix generated by the Tactonic device, and it now gets written to a text file supplied as an argument. I'm working on a mainloop that will output the force matrix at every x calls to change it, but for now the program outputs this:

When running through the command line, a record of changing pressures was left, showing that the device is still recognizing and outputting nonzero pressures.

Once the mainloop is finished, we'll have a better idea of what needs to be done to get meaningful data from the device.

Video Reaction

The demo video of Tactonic Technologies' Tactonic Sensors was enlightening despite the flaws in the presentation itself. Their product is a malleable pressure sensor that has nodes spaced at 1/2", yet they only created a visualizer to see pressure patterns on the sensor. I agree that this has its uses--they mentioned medical and commercial applications, and one that comes to mind for me is determining the shape of and weight distribution across a person's foot--but they did not provide a set of functions for extracting raw data from the pressure sensor. The presentation also did not talk about the technical side of the sensors much, but this video seems to have been more of a sales pitch than a tech talk so that's understandable.

The first complaint above (not providing a means of extracting raw data) is the subject of my first investigation. I will be attempting to create a library that allows users to get raw data instead of images when they use the pressure sensors. The majority of the work will be in interpreting the provided source classes to see how they gather data.