Part II

This Fall I'll be moving on to work on a new project.  I'll be working on a paper.  Expanding off the work other students have done in the past, I will be moving towards publishing a paper covering the topic of running Genetic Algorithms over a network, and how they perform.  I'll be running my own tests of a GA implementation and checking my results against those formerly recorded, and updating the paper based on this process.

New Features

I've recently updated the ball plate testing system to include a practice stage before the experiment begins.  This allows a participant to get experience with the system before being tested for trust.  From here a simple button press will take a participant to the normal test, which now allows for the input of their experiment ID and has a button to start the experiment, which includes starting the remote participant and the scoring script.

A strange occurrence

I recently wrote my way into an interesting bug as I was updating my python script for logging collisions.  The aim was to add in collisions for bins as well as netcode to get those bins from the web server.

Upon testing the new code I had produced, I found that some of the locations of the balls were getting set to 0 right before I tried to calculate whether they were colliding with a bin.  I ended up fixing this problem when I noticed that I was trying to collect the information on 5 bin types 30 times.  I only needed to talk to the server 5 times to determine where all the bins were, but it was getting this data 30 times, and continuing to append it onto a list.

Somewhere in this process, the over-sized list was causing a problem with the ball locations, and fixing the lists led to the disappearance of this bug.  I am still not altogether sure why this change fixed this problem, but for now, I have functional python behaving how I expect.

Almost Easy

The last couple of days I have spent trying to run a genetic algorithm to get feedback on how to tune the robots that will collaborate with the human.  These tests have not necessarily worked out.  The initial runs were using a remote server that has given us trouble in the past, and fared no better when I attempted it.  Today's attempts have taken significantly longer than expected, and have not even been completed, which is confounded by the fact that the data returned in the log files looks suspicious and maybe not meaningful.

I have also been working to improve the ability for the server to define the locations of target regions, and for the other programs to use this data to operate.  So far progress has been made, but has again stalled out due to what is most likely a very simple error in code logic in the javascript. As it stands now, though, the data coming in from the server is arriving correctly, but is being stored poorly, and all of the incoming data is being overwritten by the most recent data point, which obviously causes serious problems.

Making the same squares better

Today I've been working on completing the updates to allow the backend to handle the location of target regions.  It's interesting trying to determine the best system for all of the moving parts to communicate about this information consistently and make sure everything has all the information it needs.  In the end things will still look very similar but will allow us to make important changes much more easily for the experiment.  Unfortunately I am having a problem with the javascript that is contacting the server.  I am getting a 404 response code, but the same url when sent through my browser returns the expected data.

Paper with LaTeX

Last week I put together a paper and a presentation to go over the work I've done so far.  To write the paper, I used a really neat tool called LaTeX, which is like a small scripting language almost that allows for the formatting of documents.  With this tool, I was able to do a very minimal amount of work on format and simply link the file to a template created by acm which made everything appear nice and in its proper place.  I could also very easily change this template that is being used to have the paper show up as it would in any other format for other conferences etc.

Bringing it Together

I managed to meet with the psychologist, Dr. Pak, who will be running experiments on the platform we are developing.  We had a great meeting and were able to clearly define a lot of new objectives for the platform to help the experiment run as well as possible, with as much good data as possible.

We have got plans to improve the amount of interaction that participants will necessarily have, improve the clarity of the system to participants, increase participants' motivation to perform well, and increase the potential data we have to determine trust levels.

As we begin to implement some of these plans, we will be looking to get the experiment in a good place to be run early by Dr. Pak before we continue updating all of the possible pieces of the code.  Hopefully this will start giving us some very good results soon and meaningful data that we can use to help understand how people work with robots.

Performance Problems

This past week we've been largely trying to deal with performance problems in the system.  As we were moving into trying to do testing, we began to run into problems with low frame rate and a whole array of problems caused by the slower performance.  We were experiencing keyboard events that were missed by the program, and seemingly unexpected and inconsistent behavior when it came to the ball movements.

It took a while to further narrow down some of these problems since the testing environment mattered so much (being a performance issue).  Now that we think we have it pretty well narrowed down, we will be looking into doing testing in a controlled environment that we know will avoid issues with performance.  We will also be working on implementing multi-threading and looking into other options to improve performance so that we can scale up the experiment.

Prepared

The test platform has largely come together recently.  The newer html rendering has been largely finished and given a brief test.  There is a definite problem with control, where some of the ball-plates are behaving erratically, giving extreme and unexpected motion.  This problem will hopefully quickly be resolved though, as it hadn't persisted in the past system, and a similar fix can likely be implemented. 

The first interactive test taught me a lot about how I can expect this research to continue.  In some ways it was much more difficult than anticipated, since it is easy to get impatient and continue to update the orders of the ball plates while they are still in motion, so the movement of a ball you are controlling may not always be predictable.  This will definitely give us a good glimpse at how people may respond to a collaborative environment when they need to be cautious and may not feel entirely in control.  That being said, it also seemed like an easy test, in some ways, as many of the simultaneous movements were on opposite regions of the test space, which made it feel as though there wasn't serious interaction and collaboration at that point.  It was still interesting to see near-collisions play out without direct communication, as both participants need to guess what the other will do in reaction to the situation.

I've been thinking about how to score a pairing of participants on how well they can complete their task, and I don't know what numbers are best to use without more testing, it seems like a fairly simple algorithm will be best.  I think a pairing will be assigned some number of points based on expected time of completion.  Beating this time will give a large positive score, while failing at this time should give a small positive score.  Any collisions between balls, or entry into the wrong target areas will further reduce the points. 

Communication

Yesterday I managed to finish taking care of any problems with visualization, and get the new html file functioning as rendered by the python scripts.  It is successfully communicating with the python server and updating ball positions based on the server's status.  I've still got some confusion about some of the math going on, and what the ratio is of the python view of the plate area is, and the html version, but the numbers from the old html work out for now, and make everything appear to be working great.

From here I will need to make sure users can input their commands to the server using keyboard and mouse as an interface.  This will allow us to begin testing how well humans can collaborate, and hopefully test with machines soon, as well.

On Display

This week I've been working on finding a new way to show our visuals for the project, since the old method caused problems on firefox.  After digging around for a while I came up with a new system for display that should provide better browser compatibility.  The new solution has worked great for getting rid of the constant flashing that was a result of refresh-rate issues on firefox, but since then the task has been to get this simple proof-of-concept for the visuals fully integrated with the existing code base.

This process has seen a lot of growing to better understand the system as it exists, as well as webpy, javascript, and html in general.  Indeed, it took me far too long just realizing the way webPy is rendering the html pages from the python scripts at runtime.  Now I've got things largely in position, but am facing a new problem with display.  The balls I'm putting on screen are not showing up where they are supposed to, or certainly not where I expect them.  I have kept in tact the same function to draw with, but it is now being called from a different place, after communicating with the python scripts.  I will have to look into the possibility that information exchanged with the python code is changing my outputs, but for now am not sure where my problem lies.

The Project

The research project is a multidisciplinary study on how humans and machines interact.  We are trying to determine how well they trust each other, and also to find some way to measure and quantify levels of trust in a collaborative environment.  We are utilizing a simulated ball and plate system as the work environment.  This system allows us to have multiple collaborators (i.e. a human, machine pair) working on the system via controllers. This system will allow us to make machines of varying levels of competency, and gauge how well humans are trusting their machine partners based on the accuracy of a given machine.  We will also be studying the affects of the network on this system, and how slow connections to a remote site might alter the perceptions of trust and competency. 

Introduction

I am a senior at Clemson University studying computer science.  I enjoy puzzling through challenges in computing, and have only continued to enjoy the field more as I've gotten into it.  My dream job would be getting to apply my skills towards working in the video game industry, and helping the next generation of young gamers have the exciting moments that I had so much fun with growing up.  I love reading fantasy novels, gaming, and spending weekends in the mountains.

I've always loved staying up late and trying to stay up late and spend time thinking about all of life's greatest question, and I can be quoted on saying that I'll start a new school of philosophic thought one day.  Realistically, it's probably just always going to be idle thoughts over the perfect cup of coffee, but a man can dream, I like to think.

Python Anywhere - What it is and how can it be used

PythonAnywhere is an online IDE with the options of Python 2.7-3.4, iPython, PyPy 2.7, Bash terminal, and MySQL terminal. For a free account, you have the ability to create a python web-app that's hosted on the main site, up to two shells, and the ability to share your console with anyone. There's a private file storage of up to 512mb which for CS students is more than enough.

Advantages:
-This website has a python shell and a bash terminal. This can open up students gently to the idea of using a shell (then using vim, pico, etc).
-Students can use the online text editor which has a push-button option to run code.
-Accounts have file management, so files can be stored and downloaded to local machines.
-Consoles can be shared, which opens up opportunities for teamwork or student/teacher sessions.
-Has tons of modules

Disadvantages:
-Consoles are limited based on processor time used (100 seconds for a free user)
-Site server performance is a concern, but not much of an issue.
-Online only.

Brython - Adaption to class tool

Before going into specifics on how best to utilize Brython, I'll be covering some advantages/disadvantages on the onset of using this as a classroom tool.

Advantages:

Disadvantages:
-Useless if offline. Python's native GUI and countless other applications make this inadequate.
-No file sharing. If you want to publish something, you will have to do it on your own server.
-The constant need to build a HTML webpage in order to actually use Brython.
-If you're not familiar with ECMAscript, brython under the hood will seem weird.
-Limited to its own libraries (Python 3.x) and its ECMAscript libraries.
-Suspect translation of python to ECMAscript. 
-Questionable speeds due to python to ECMAscript translations.
-Python 3 maybe not be the standard for many applications.
-If being taught, would need to implement HTML template source code. 
-There's not immersive way to use this as a teaching tool.

Observing a computer science 101 class

Observed on Oct 30, 2014 - a CPSC1010 class

This professor used a projector setup with her own personal laptop.
The subject language was C and a swap function with pointers was being dissected and discussed. I noted that she was using Word to review code in C.
Next lesson was a quiz review. She went over the questions and answers using the above technique.

At a particular point in the classroom she had to use the faculty computer which was a Windows machine which necessitated the use of SSH. The SSH program for Windows might be difficult to see because of the white background. Unsure if there are program options to change the text font or size.

Using a SSH shell, she demonstrated the use of command line arguments with argc and argv. She would alternate between using VI and then using the command line compiler (GCC) to compile source and observe output.

I noticed most of the students (save a few for Mac users) were using SSH/Putty to use the compiler on school computers. This would be an excellent use of an online IDE as it would make it a lot easier to students/teachers to view/edit/compile source code. Instead of using the cumbersome SSH program, class exercises could easily be done using koding or even cloud9. There was also a special needs student and I wondered how this affected the teacher's methods.

Personal Thoughts:
I have discussed using cloud-based IDEs to help demonstrate programming exercises with the professor. However, I had to be mindful because techniques cannot be easily changed overnight and many teachers have a method to how they teach (e.g. using Word to go over C code) and their personal preferences. I personally think programming code would be easier to view/learn from in terms of it being shown in an IDE environment or an application where a teacher could change the font/size. Students could vastly benefit from using cloud-based IDEs as well. For those who don't know exactly (or yet) how to use command line based editors or SSH/Putty on Windows, they could easily use koding to type out and compile example code. This eliminates the hassle of having to log on to a specific computer (which can be confusing to the first year student). This also effectively cuts the users who are signed on to school workstations thus eliminating extemporaneous workloads on said workstations that could be utilized by later year students.

Summary of powerpoint presentation: Web Based IDEs and their place in the classroom

Overview of IDEs:

Brython:

• Designed to replace JaveScript as the web scripting language.
• Pros:
  Can be used to create custom, interactive lessons in python.
  Can be used to make websites documenting the power of Python as a web script.
• Cons:
  Its scope is limited to being a script object in HTML.
  Limited by its JavaScript library.
  Can be utilized offline. However, if teaching Python offline, perhaps it’s better to use the official Python Idle.

PythonAnywhere:

• PythonAnywhere is an online IDE/Web Host based on Python.
• Pros:
  Implements many versions of python, bash, and even MySQL
  Accounts are free and have 512mb of storage
  Console can be shared -- cooperative work environment
  Python scripts can be saved on the server.
  Many python modules supported.
  On-site script editor alongside python interpreter.
• Cons:
  Dropbox integration is down.
  Use is limited by CPU allowance (processor time used).

koding:

• Koding is an online IDE that allows real-time collaboration alongside access to a full Ubuntu linux terminal that supports many languages.
• Pros:
  A free Ubuntu terminal with many developer tools.
  Extensive help section alongside other users that can answer questions.
  Can get files from Google Drive, BitBucket, DropBox, etc.
  Ability to collaborate in real time plus integrated Google hangouts
• Cons:
  Downtime of terminals (sometimes unscheduled)

Student use:
As a student, it’s simple to log into PythonAnywhere or Koding.
Brython might require the student to be familiar with HTML just to program in python. The student can follow along any online course (youtube, udemy, coursera, etc.) and program in the same window. Collaboration and the ability to save files onto site host servers makes it easy for students to work from home and school on different machines. To turn in homework or assignments, students can invite the teacher into a workspace so he/she can explain the program in real time. No need to install software; less headache about hardware limitations. Internet and a compatible web browser is only needed.

Teacher use:
No administrative access to websites so students will have need a system to turn in completed work. PythonAnywhere and koding can be used to demonstrate concepts while in class. While using a centralized IDE, students will worry less how to do things in the IDE and concentrate on actual programming.
Teachers could write half-completed programs as activities and have students fill in the blanks. Teachers could plan and utilize Brython and create a webpage that demonstrates Python concepts. No need for books or if books are necessary students can follow along and be less distracted by switching between two windowed applications.

Areas to improve for these IDEs (in terms of teaching/learning):
Compiler:

• Providing a compiler that compiles as code is being written.

• A compiler with clearer error messages (more similar to clang)

• If the compiler is providing hints and the student still doesn’t understand what’s wrong, maybe a compiler that just provides answers

Drag and Drop:
Allowing the student to drag and drop things. For instance, if compiler says “missing semi-colon” then student can drag and drop a semi-colon. Or drag and drop variables and initializations to these variables. (Basically like Scratch) (A survey of literature on the teaching of introductory programming)

Help Guide/Forum:
If need help finding certain function names, or functions. Or need help figuring out how to actually do something in code. This can be a forum, where they ask people, or be like a huge database in the system, that just so happens to be intelligent enough to do this.

Built In Code:
Providing code for the student to play around with. Code can be specific examples of if statements, loops, recursion, etc. that encourages the student to change things around, asks them specific questions about what they’re doing and what the code is doing.

Brython -- an introduction

In this blog post, I am going to briefly do an overview on Brython, how to install several versions of it, and ways to implement it as a student.

What is Brython?
Brython is a way to use python instead of javascript for browsers.
However, the only way to implement brython on web pages is to source it from a JavaScript file. The convenience is able to write python script inside a HTML file or to have a python program (.PY file) executed by HTML controls.

How do I use Brython?
Taken from the github repository of Brython:
To use Brython, all there is to do is .

1. load the script brython.js.
2. run the function brython() on page load.
3. write Python code inside tags <script type="text/python">.

Note that Brython implements Python 3 so several 2.7 methods/techniques won't be applicable.

Where to download Brython:
https://github.com/PierreQuentel/brython/releases

There are two different files:
The latest version of brython is the necessary javascript files to code Python into a html file.
The site mirror file contains a clone of the brython.info website that includes documentation and an online script editor console (similar to CPython).

How does Brython work?
Python scripts are converted to JavaScript code, which might explain why it compiles 3-5 times slower. In the FAQ (http://brython.info/doc/en/index.html) , the developers explain circumstances concerning run-time:

• the time to translate Python into Javascript, on the fly in the browser. To give an idea, the module datetime (2130 lines of Python code) is parsed and converted to Javascript in 0,5 second on an ordinary PC
• JavaScript code generated by Brython must be compliant with the specifications of Python, including the dynamic nature of the search attributes, which leads to unoptimized Javascript code

Infrastructure as a Service as a teaching tool

On upcoming blog posts I will be documenting (technically and figuratively) several website that can be used in Computer Science classes. These websites have been chosen for their ability to emulate a Unix environment (primarily for python scripting use) and the ability to share and collaborate the code with instructors and other students.

The following websites:
www.koding.com
brython.info
Cloud9
IDEone
Python Anywhere

Of these three websites, I will document and assess their ability for use for beginning software developer students.

The criteria for assessing these abilities:

• Ease of use in usage.
• Ease of use in installing (brython).
• The capabilities of each website in terms of development environments.
• Reliability of each website.
• The ability to collaborate and/or evaluate source code.
• The aesthetic design of each website.
• The overall usability for each website.

The websites will then be compared with their strengths and weaknesses and it will ultimately be up to the reader to decide which platform best fits their needs in a classroom.

When Benchmarking Goes Bad

For some reason, the benchmarking utility has started going into an infinite loop whenever I run it. Any fixes I try mess up the sample collection (e.g. out of 5000 successful XHRs in one loop I got 12 data points). I don't exactly know what changed--I certainly haven't touched the benchmarking code (though I re-downloaded it for good measure), and I got two successful benchmarks out of it (read: two on each client for a total of six) after the last change I made to the server, meaning it was definitely working. The oddest part is it's still receiving good data from the server. I've tracked the received data several times, and it's dynamic with what the sensor is seeing.

Whatever is happening here, it's clear that I can't be certain of the reliability of any of my previous benchmarks until I figure out what's wrong.


UPDATE:
Apparently I've found some means of controlling the number of requests again. Everything seems to be an order of magnitude slower now, though, which is moderately irritating.

UPDATE 2:
Apparently this is only really an issue over localhost. Walter slowed some, but that's more likely to be a result of network traffic in our lab than anything else (~2ms slower than usual). Dragon had the same time as usual. It's only on localhost that I'm forced to sit through ~6000 requests with a slowdown of an order of magnitude--meaning that the Air on localhost is about as slow as Walter concerning asynchronous sever calls. I'm going to keep looking into this.