Introducing THEA and Server Orientation

This is THEA, the Tactile Hand Evaluation Apparatus, a cousin of Tyler's from Tactonic Technologies. Tyler is experiencing frequent technical difficulties these days, so THEA is filling in while we await a new USB adapter.

THEA, like Tyler, is a pressure sensor made up of sensels. Unlike Tyler, THEA's sensels are packed much more tightly together. Each of Tyler's sensels is spaced about 1in. from every adjacent sensel, with 24x24 sensels in one panel. THEA, on the other hand, packs 34x44 sensels into a single panel, each spaced about 1/4in. from every adjacent sensel.

Due to Tyler's injury, I'll be using THEA to move forward with the Sensory Translation project. The first phase is to make a simple keyboard (as in piano) by dividing THEA's sensels into groups that each emit a distinct tone. I'll be implementing this using the WebAudio API and data from the Mongoose server I've been working on.

THEA has already helped in adding features to the server: now, with a query string, users can specify a tile orientation in one of four directions. Since the tiles are only usable in straight lines, swapping orientations was not difficult, but being able to rapidly test the code with THEA saved quite a bit of time.

Now, if a user wants to specify an orientation, they just add

?orientation=x 

where x may be any 32-bit integer value. The value is taken mod(4) if it is greater than three, then passed into the function that transforms the force grid into a string. That function has been modified to be:

std::string NumViews::getForceGrid(TactonicFrame *frame, int orient, bool usespacer) {
    std::string out = "";
    int xmax, ymax, f;
    uint8_t xref, yref;

    ymax = (!(orient & 1)?device.rows:device.cols) - 1;
    xmax = (!(orient & 1)?device.cols:device.rows) - 1;

    yref = (orient == 0 || orient == 3?0:1);
    xref = (orient == 0 || orient == 1?0:1);


    for (int i = 0; i <= ymax; i++) {
        for (int j = 0; j <= xmax; j++) {
            std::string spacer = "";

            if (orient == 0 || orient == 2) {
                f = frame->forces[(yref&1?(ymax-i):i) * device.cols + (xref&1?(xmax-j):j)];
            }
            else {
                f = frame->forces[(yref&1?(ymax-i):i) + (xref&1?(xmax-j):j) * device.cols];
            }

            out.append(std::to_string(f));
            if (!usespacer && j < xmax) {
                spacer = " ";
            }
            else if (!usespacer){
                spacer = "";
            }

            else {
                spacer = (f>=1000?"":f>=100 && f < 1000?" ":f>=10 && f < 100?"  ":"   ");
            }


            out.append(spacer);
        }
        if (i < ymax) {
            out.append("\n");
        }
    }
    return out;
}

Essentially, what has happened is the frame of reference for iterating over the force grid is changed depending upon the orientation. I'll explain the new parts piece-by-piece.

Deciding Which Coordinate Represents Rows vs. Columns

    ymax = (!(orient & 1)?device.rows:device.cols) - 1;
    xmax = (!(orient & 1)?device.cols:device.rows) - 1;

In orientation 0 or 2, the grid is iterated over with x as the column dimension and y as the row dimension. However, in orientation 1 or 3, this is swapped. This is then used in the nested for loops to set the bounds for iterating over each dimension.

    for (int i = 0; i <= ymax; i++) {
        for (int j = 0; j <= xmax; j++) {

This allows the parameters of the loop to change without having to do any work in the loop setup or having to create another set of loops.

Deciding Which Point is Each Dimension's Origin

    yref = (orient == 0 || orient == 3?0:1);
    xref = (orient == 0 || orient == 1?0:1);

To change the orientation, we also need to know if each dimension starts at the beginning or end of its range of values. xref and yref are just flags that indicate whether or not the iterator must read from front-to-back or back-to-front over a given dimension.

Pulling the Correct Values from the Array

 for (int i = 0; i <= ymax; i++) {
        for (int j = 0; j <= xmax; j++) {
            std::string spacer = "";

            if (orient == 0 || orient == 2) {
                f = frame->forces[(yref&1?(ymax-i):i) * device.cols + (xref&1?(xmax-j):j)];
            }
            else {
                f = frame->forces[(yref&1?(ymax-i):i) + (xref&1?(xmax-j):j) * device.cols];
            }

The final piece of this data is to pull the correct values out of the array. This was an interesting case because the array is 1-dimensional, so all of the math had to be done in one shot. Essentially, the same thing is happening in both cases:

index = (current_row * column_count) + current_column

The first case deals with orientations 0 and 2, which are 180-degree rotations of each other. The inline conditionals check the flags to see whether or not they are set. If so, the dimension is the difference between the maximum index for that dimension and the current iterator value (read from the back); if not, the value is the current iterator value (read from the front).

The second case deals with orientations 1 and 3, which are 90- and 270-degree rotations of orientation 0, respectively. The conditionals do the same thing as in the first case, but here the x- and y-dimensions are swapped in the for loops, making the loops iterate over all y for a particular x before moving on to the next x.

The output of this code is as follows:

Orientation 0

Orientation 1

Orientation 2

Orientation 3

 These four screenshots are all of my left hand. I did not move my hand between screenshots. The image looks like my right hand because of the location of (0, 0) on THEA's sensor grid and the fact that it's more convenient to put THEA to my left on my desk.

Essentially, I've made an in-place row/column-major determining function. Careful observers will notice that the first indexing case above may actually be done with one check, since in orientation 2 both flags are set to 1. However, by leaving it as-is, we actually allow for four other orientations to be implemented, if desired, by swapping the flags of orientations 0 and 1 and orientations 2 and 3. These will produce mirror-images of the currently implemented orientations across their major axis, so I'm leaving it as-is in the event that somebody wants to use them in the future.

Currently, no drop in performance has been observed as a result of these changes, and since Daniel has been making AJAX calls to the server using this new setup, we should have seen some slow-down in the tests if this was going to be an issue.

The completion of this task means that my work with the code that interprets Tyler's and THEA's data is complete until somebody needs new functionality, so I'm moving on to other tasks.