Mongoose and Tactonic Modifications Updated

As of today, the information provided by the modifications to Tactonic's sensor source code is available in three different formats. Mongoose was recently updated to version 5, which introduced struct mg_server to handle all server requests in place of struct mg_context. This update allows a bit more flexibility in handling user requests, or maybe just makes it simpler, and this week was spent updating existing code and making sure it worked.

This short block of code added to TactonicNums.cpp, along with a later call to <code>mg_poll_server(milliseconds_per_poll)</code> in an infinite loop, sets up a server, sets its hosting information,  and adds three request handlers that allow us to work with different URIs from within the file calling the server itself.

This is the result.

Human-Readable Data

Machine-Readable Data

Center of Pressure

These three are accessed by navigating to the server's IP and listening port while it is running.
They are, respectively

1.  <ServerIP>:<ListeningPort>/
2.  <ServerIP>:<ListeningPort>/data
3. <ServerIP>:<ListeningPort>/center

As we do not have a dedicated server machine that can run the Tactonic sensors yet, I've continued testing on localhost,  but Daniel was able to connect to the server prior to these modifications, and I connected using one of the lab computers after modifying.

A significant issue so far has been response time. Requests can take up to 30 seconds to complete, and that performance is not good enough for our needs. Cesanta has given Mongoose its own threading API, so that can be implemented to make a concurrent server if necessary.
  
The updated files are zipped in Box with one big improvement: a Makefile! No longer is some huge compilation command involving no less than 7 filenames necessary!

JavaScript setInterval and asynchronous ajax

As a part of performance testing for Kamehameha and Claude, Dr. Remy and I wanted to look at not only network latencies, but also what kind of difference asynchronous vs. synchronous calls would make. Since the ajax requests that update the skeleton data in Kamehameha are called by a JavaScript timer, it complicates what "asynchronous" and "synchronous" really mean in terms of the ajax call and the rest of the JavaScript.

JavaScript is completely serial; there is no true concurrency within the language itself. However, after an ajax call is made JavaScript can either halt script execution until the response is received or it can continue execution and handle the data returned by the call at a later time using event handlers. All my work with Kamehameha up to this point had been using synchronous ajax calls. The benefit in this is a guarantee that data will be received in order. You don't really want the "next" set of data until the previous call has finished anyways so serial ajax calls meets our needs. But out of curiosity, I wanted to see the effects of asynchronous calls. Additionally, I had been testing Kamehameha at a rate of one call per 50ms. What would happen when I make that rate quicker? What about 1ms? And how would the two different types of ajax calls handle it?

One other thing that tricked me at first is how JavaScript setInterval works. Even though I was making the ajax calls serially, what did that mean for the 50ms timer I created on the page? If the ajax call took longer than 50ms would the timer still fire the getData function again? The answer is no. Because the ajax call was synchronous, JavaScript did not consider the getData function to have finished until the ajax call was done. JavaScript's timers (setInterval) also will not call a function again if the last call to it has not finished. So as a result, past a certain threshold of calls per ms, the getData function would reach a maximum performance and calls would simply happen as fast as the ajax response would allow it.

Now changing the ajax calls to asynchronous causes more interesting things to happen. JavaScript no longer has to wait for the ajax request to be over to consider the entire getData function to be done. The getData function is done as soon the as the ajax call is started. There is no need for it to wait for the response. This means that the limit on the number of ajax calls is now determined by how quickly an ajax call can be initialized and sent, no wait time included. This should theoretically perform better than synchronous ajax calls. The network link between Kamehameha and Claude, the server hosting the data, can now be though of a pipeline. Rather than sending something through and waiting for it to come all the way back, multiple things can be sent in one after the other and responses will come back at, hopefully, the same intervals apart as they were sent. Due to the way networks transfer packets, the delay between ajax responses will definitely not be the same as it went in but it should at least be smaller than the delay between synchronous ajax responses.

I have updated the Kamehameha code to include a performance html file. It is a replica of the original kamehameha html/javascript page but with additional code to print some debug information. I haven't added time metrics yet but that will be a small change. Currently each ajax call is given an id that is simply the ajax call count up to that point. At the time the ajax call is started, a debug message is printed to indicate so. Then, when the ajax call finally ends, another message is printed also including the id. For synchronous calls, the ids are always in order and one id must start and finish before the next occurs. But for the asynchronous  calls pretty much anything can happen. The only guarantee is that the starting calls will be in chronological order. However, the end time for a call can happen much later or much sooner than other starting and ending calls.

Maxing out the timer interval (or rather, minimizing it) for asynchronous ajax calls did not produce any visible differences to the skeleton rendering, despite the fact that many calls were coming out of order (as seen by the debug output). I have yet to come up with a metric for the "accuracy" of the skeleton rendering, that is to measure how badly the ordering of ajax responses is of the asynchronous version to the performance boost it provides. Additionally, once I add code to get the actually timing, tests will be conducted on the local network and over the internet.

2014 Early Work and Short- and Long-Term Goals

To start the year out, I have taken a step back from code while waiting for things to get up and running again and have spent time working out logistics for the future of Tyler, our floor tile pressure sensors.

The last post demonstrated the differences in Tyler's pressure-sensing abilities with and without a foam-tile covering. The foam covering actually helped to make areas of pressure surrounding a center more distinct, which will be beneficial moving forward. Once we receive confirmation of reimbursement, I will purchase eight foam-tiles (two four-packs) for ~$40. The tiles I'm looking at have the same area as Tyler (each tile is 2 ft^2), meaning we will get ample coverage on which to run future tests.

Unfortunately, of the eight tiles available, we can only make use of two at a time. There is an issue with power not flowing to the third tile in a chain and onward. The tiles are ordered, and Tactonic (the company that makes these tiles) claims that the tiles must be connected in that order, but I have been able to power and gather data from two tiles that are not necessarily in that order.

Despite the powering issue, I have produced a video of myself walking across the two available tiles. This demonstrates in a small space what the pattern of a walking gait can look like.

Looking forward over the next year, one goal is to take the data from Tyler and perform pattern-matching to recognize different foot orientations, and then to associate sounds with different patterns. The end goal is, in lieu of having a prosthetic foot and a foot-shaped pressure sensor, to use the pressure patterns observed by Tyler to create an auditory adaptation of a foot's sense of touch.

The first stage in this is to create a strategy for 1) determining a single group of points that represents pressure being exerted on multiple sensels (such as the ball or heel of the foot) and 2) determining whether or not two or more groups make up a single object exerting pressure on the sensel grid (such as the ball and heel of one foot). This will likely occupy the next few weeks of my work.

The other long-term goal is facilitating a move to distributed processing of Dr. Remy's evolutionary algorithm. Over the next few weeks, I will brush up on SSH, Pythons multiprocessing library, and the workings of evolutionary algorithms.

Comparison of Sensor Data With and Without a Foam Cover

This image shows a side-by-side comparison of pressure sensor visualization with and without a foam covering over the sensor. I tried to stand the same way twice, but no two poses can be exactly the same. The pressure tapers off more quickly with the foam cover, more accurately pinpointing areas of higher pressure; without the cover, more detail can be seen, for instance the big toe of each of my feet shows up. As long as the foam covers are removable, I recommend getting some to place over the floor tiles, if for no other reason than upkeep.

Songs of Diridum Revisited

 To further improve our own attempts at using the WebAudio API’s spatial audio features, I have been looking more closely at the Songs of Diridum demo and all of the technologies behind it. Firstly I wanted to ensure that it was solely the WebAudio API they were using and that I wasn’t missing some other piece of software that was making their spatial audio sound so realistic. Furthermore, I know they used Goo as a game engine but then does that mean the WebAudio API is integrated with Goo? Or did Songs of Diridum use Goo for graphics and separately use WebAudio API for audio?

It appears to be the latter. Here, in the original article I was reading, is a description of Goo.

“Goo is a HTML5 and WebGL based graphics development platform capable of powering the next generation of web games and apps. From the ground up, it’s been built for amazing graphics smoothness and performance while at the same time making things easy for graphics creators.”

Additionally, Goo Technologies has produced Goo Create, and web-based 3D editor. Both the Goo engine and creator are really cool pieces of technology built on top of the WebGL library. Here is another excerpt from the article detailing WebGL.

“We wanted to build Songs of Diridum as a HTML5 browser game, but how to do that in 3D? The answer was WebGL. WebGL is a new standard in HTML5 that allows games to gain access to hardware acceleration, just like native games.”

WebGL is a library that provides access to the GPU without the need to install any additional plug-ins (also see the wiki article). Goo engine is a library built on top of WebGL to simplify its use, similar to the way most popular C graphics libraries abstract from OpenGL.

The WebAudio API is completely separate from WebGL, Goo, and even HTML5. Though it can work closely with HTML5 (see “audio” tag) it is its own “high-level JavaScript API”. The W3C standard for it can be found here.

So how does this help our research and quest for better quality audio spatialization? Well, it tells us it should be possible with what we already have. We don’t need Goo or any other HTML5 technologies, just the WebAudio API.

The quality of the mp3 we were using (the boiling cauldron sound) may share a part in the poor ability to spatialize it. Furthermore, the article linked above includes a small excerpt of code presumably similar to what was used in Songs of Diridum. It is quite similar to what I’ve done with the WebAudio API in attempts to spatialize our sounds however there are a few differences which I could apply to see if better results can be yielded.