Showing posts with label Pressure Translation. Show all posts
Showing posts with label Pressure Translation. Show all posts

Friday, April 18, 2014

Generating Audio from Arbitrary Pressure Groups

Today, I wrote the last leg of this project: generating audio from arbitrary pressure data. The purpose of this is not only to take some random data and make audio from it--that's easy to do. By simply throwing numbers at WebAudio I can make random, meaningless noise. Now I'm searching for some means of making the sound meaningful in some way, for instance making the tone sharper or flatter as the pressure group moves up or down the x-axis. If changes in the tone are made meaningful, then this project has applications in both human and machine learning, assuming that the agent learning from the device knows how to interpret the changes in tone.

I'll post information on the process of actually making the tone meaningful as that happens, but for now I have something basic that accomplishes this imperfectly:

freq = 110 * (Math.pow(2, max_data / (1000 * area)) + (center_y) + (center_x / 10))

This takes the frequency of an A2 (110), scales it up by 2 to the power of the local pressure maximum divided by the area adjusted to be more even with the max_data point, then offsets the resulting value by the center coordinates. The multiplication by a power of 2 was chosen because the same note in different octaves represents the frequency of that note in octave 1 * 2^the current octave number (e.g. A1 = 55Hz, A2=110Hz, A3=220Hz, etc.).

This method, as mentioned above, needs tweaking. Many different groups sound similar by virtue of the
 
max_data / (1000 * area) 

calculation, and THEA is actually so sensitive to pressure input that even a small change in pressure can cause the max_data variable to skyrocket, resulting in high-frequency spikes.

On the Javascript end of today's code, and if anyone from W3C sees this, WebAudio's AudioBufferSourceNode object could benefit from a function to check whether or not the node is in start() or stop() state. Javascript throws up an error when a node is started or stopped twice while in the each respective state, and while it isn't hard to implement a check using boolean flags, I was surprised to see that an internal state is not set when start or stop is called. Perhaps it is and that state is simply kept private. In any case, that's a minor issue at best.

Monday, April 14, 2014

Fixed Static Parameters Version of the Server, Set Up Tests of Concave Hull

The version of the server that passes parameters by value instead of reference (mentioned in the last post) is ready but not yet uploaded to Buffet. The Buffet server isn't loading pages, so there must be maintenance, development, or a server issue preventing its use. The repos will be updated as soon as possible.

Also, minor testing of the concave hull groups has been set up, and I'll post the results later this week. The algorithm is as follows.

Given a set of adjacent points (a Group as defined in earlier posts), for each point in the group:
  • Check all adjacent points, including diagonal points. Record the coordinates of all 0-datapoints adjacent.
  • If there are only one or two adjacent 0-datapoints, or more than three, add the point being evaluated to the list of hull points.
  • Else, if there are three 0-datapoints, check to see if the set of adjacent 0-datapoints is linear. If it is not linear, add the point being evaluated to the list of hull points. Otherwise, it is part of the hull but is unnecessary to determine the shape of the hull, so do not add it to the point list.
  • Else, if there are no adjacent 0-datapoints, do not add it to the point list.

The implementation has not been tested yet, and improvements are probably necessary, but having worked this out by hand on several data sets, it seems to be a sufficient starting point. In addition, it can readily be extended to higher dimensions with a concrete definition of adjacency in that dimension.

For two dimensions, the current C++ implementation is as follows.


void NumViews::addHullGroup(TactonicFrame *frame, uint8_t * checkedArray, int x, int y) {
    Group * g = new Group();
    Point * p = NULL;
    int zeropoints[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
    std::queue<Point *> * q = new std::queue<Point *>();
    int cols = device.cols, rows = device.rows, data, zerocount;

    q->push(new Point(x, y, frame->forces[y * cols + x]));

    while (q->size() > 0) {
        p = q->front();
        q->pop();

        x = p->getX();
        y = p->getY();

        if (!checkedArray[y * cols + x]) {
            data = p->getData();
            zerocount = 0;
            checkedArray[y * cols + x] = 1;



            if (x - 1 >= 0 && frame->forces[y * cols + (x - 1)] > 0 && !checkedArray[y * cols + (x - 1)]) {
                q->push(new Point(x - 1, y, frame->forces[y * cols + (x - 1)]));
            }
            else {
                zeropoints[zerocount] = x - 1;
                zeropoints[8 + zerocount++] = y;
            }

            if (x - 1 >= 0 && y - 1 >= 0 && frame->forces[(y - 1) * cols + (x - 1)] > 0 && !checkedArray[(y - 1) * cols + (x - 1)]) {
                q->push(new Point(x - 1, y - 1, frame->forces[(y - 1) * cols + (x - 1)]));
            }
            else {
                zeropoints[zerocount] = x - 1;
                zeropoints[8 + zerocount++] = y - 1;
            }

            if (y - 1 >= 0 && frame->forces[(y - 1) * cols + x] > 0 && !checkedArray[(y - 1) * cols + x]) {
                q->push(new Point(x, y - 1, frame->forces[(y - 1) * cols + x]));
            }
            else {
                zeropoints[zerocount] = x;
                zeropoints[8 + zerocount++] = y - 1;
            }

            if (x + 1 < cols && y - 1 >= 0 && frame->forces[(y - 1) * cols + (x + 1)] > 0 && !checkedArray[(y - 1) * cols + (x + 1)]) {
                q->push(new Point(x + 1, y - 1, frame->forces[(y - 1) * cols + (x + 1)]));
            }
            else {
                zeropoints[zerocount] = x + 1;
                zeropoints[8 + zerocount++] = y - 1;
            }

            if (x + 1 < cols && frame->forces[y * cols + (x + 1)] > 0 && !checkedArray[y * cols + (x + 1)]) {
                q->push(new Point(x + 1, y, frame->forces[y * cols + (x + 1)]));
            }
            else {
                zeropoints[zerocount] = x + 1;
                zeropoints[8 + zerocount++] = y;
            }

            if (x + 1 < cols && y + 1 < rows && frame->forces[(y + 1) * cols + (x + 1)] > 0 && !checkedArray[(y + 1) * cols + (x + 1)]) {
                q->push(new Point(x + 1, y + 1, frame->forces[(y + 1) * cols + (x + 1)]));
            }
            else {
                zeropoints[zerocount] = x + 1;
                zeropoints[8 + zerocount++] = y + 1;
            }

            if (y + 1 < rows && frame->forces[(y + 1) * cols + x] > 0 && !checkedArray[(y + 1) * cols + x]) {
                q->push(new Point(x, y + 1, frame->forces[(y + 1) * cols + x]));
            }
            else {
                zeropoints[zerocount] = x;
                zeropoints[8 + zerocount++] = y + 1;
            }

            if (x - 1 >= 0 && y + 1 < rows &&frame->forces[(y + 1) * cols + (x - 1)] > 0 && !checkedArray[(y + 1) * cols + (x - 1)]) {
                q->push(new Point(x - 1, y + 1, frame->forces[(y + 1) * cols + (x - 1)]));
            }
            else {
                zeropoints[zerocount] = x - 1;
                zeropoints[8 + zerocount++] = y + 1;
            }



            if (zerocount == 3) {
                for (int i = 0; i < zerocount - 1; i++) {
                    if (zeropoints[i] != zeropoints[i + 1] || zeropoints[i + 8] != zeropoints[i + 9]) {
                        g->addPoint(p, 'x');
                    }
                }
            }
            else if (zerocount != 0) {
                g->addPoint(p, 'x');
            }
        }
        else {
            std::cerr << "Deleting a point in NumViews::addHullGroup" << std::endl;
            delete p;
        }

        p = NULL;
    }
    std::cerr << "Deleting q in NumViews::addHullGroup" << std::endl;
    delete q;

    if (g->size() > 0) {
        g->calculateCenter();
        pressure_groups.push_back(g);
    }
    else {
        delete g;
    }
}

Friday, February 7, 2014

Translating Pressure Data to Audio Output: The Plan

An avenue that I am currently pursuing is taking the pressure data from Tyler by means of my modifications to Tactonic's source code and translating it into audio output. The end-goal here is to use that program to show that a unique signal can be produced from a particular set of pressure data.

Dr. Remy has suggested that I implement this in Python, and the current work on this is being written using Python 2.7.3.

The major problems that this code must solve are:
  • Distinguishing between and grabbing pressure groups
  • Modulating audio output based on pressure data
As of now, pressure data is being received in a (w * h)n array of integers, where n is the number of pressure sensors and w and h represent the number of sensors in a row or column of sensors, respectively. The test data currently being used is:


The array generated by Tyler is presented as a single string where elements are delimited by either a single space or a newline character. This data allows for a not-quite-rigorous test of group-grabbing code because, despite its appearance, it actually contains three groups.

How is a group defined?

A group, as I mean here, is a contiguous set of nonzero data points adjacent to each other on a Cartesian plane. This definition presents its own challenges. A group can be of any shape so long as each point is adjacent to at least one other point in the group. Any nonzero point that is not adjacent to another point is in its own group. This means that a group can consist of a single point. In the future, more constraints may be added, but for the time being this definition is sufficient for generating code that can handle grabbing groups out of an array.

One example of a decision to make concerning this definition is determining whether or not elements that are "diagonal" will count as adjacent. For the time being, adjacency is limited to points (x - 1, y), (x + 1, y), (x, y - 1), and (x, y + 1).

Implemented in Python, this looks like:

class Group:
"""Group represents a set of points with some relation to one another. In
addition to the point list, it calculates the coordinates of the center of
the group."""
def __init__(self, cx=0.0, cy=0.0):
self.point_list = []
self.center = [cx, cy]

The class stores a list of Point objects and a list of coordinates representing the center of the group.

The Point class is simply a storage class that keeps a pair of coordinates and the data stored at those coordinates.

class Point:
"""Point represents one unit of data on a Cartesian plane. In addition to
coordinate information, it stores the data recorded at the point"""
def __init__(self, x=0, y=0, d=0):
self.coords = (x,y) #These do not need to change
self.data = d #This is expected to change



What information must a group contain?

Per the definition above, a Group storage type must contain all points within the group and their force data. In addition, groups will contain the coordinates of their center of pressure. This is determined by

CenterX = sum(x * force(x)) / sum(force)
CenterY = sum(y * force(y)) / sum(force)

This is calculated by iterating over the entire group and summing the necessary components at each point. This operation has been rewritten for Python as:

def calculate_center(self):
force = 0.0

for p in self.point_list:
force += p.data
self.center[0] += float(p.data) * float(p.coords[0])
self.center[1] += float(p.data) * float(p.coords[1])

self.center = [_ / force for _ in self.center]


Looking Forward

The main concern at this point is writing an efficient group-grabbing algorithm. A successful algorithm will be able to pull a group from the array in a single pass with no information about the shape of a group, and it will create no duplicate groups or groups whose points overlap.

As of now, the code pulls information from a text file, but the eventual goal is to be able to read from an array posted online. This functionality will need to be implemented once Tyler is able to interface with the Raspberry Pi.

I have written tentative code for both classes mentioned above. The entirety of the code for those classes as of now is as follows:

class Point:
"""Point represents one unit of data on a Cartesian plane. In addition to
coordinate information, it stores the data recorded at the point"""
def __init__(self, x=0, y=0, d=0):
self.coords = (x,y) #These do not need to change
self.data = d #This is expected to change

def compare(self, p):
return (self.coords == p.coords and self.data == p.data)

def print_point(self):
print '{0:3d} {1:3d} {2:4d}\n'.format(self.coords[0], self.coords[1],
self.data)

class Group:
"""Group represents a set of points with some relation to one another. In
addition to the point list, it calculates the coordinates of the center of
the group."""
def __init__(self, cx=0.0, cy=0.0):
self.point_list = []
self.center = [cx, cy]

def add_point(self, x, y, d):
self.point_list.append(Point(x, y, d))

def calculate_center(self):
force = 0.0

for p in self.point_list:
force += p.data
self.center[0] += float(p.data) * float(p.coords[0])
self.center[1] += float(p.data) * float(p.coords[1])

self.center = [_ / force for _ in self.center]


def contains(self, x, y, d):
p = Point(x, y, d)
for point in self.point_list:
if p.compare(point):
del p
return True

del p
return False

def print_group_data(self):
for point in self.point_list:
point.print_point()
print "Center {}".format(self.center)