I have been working on a USB HID Mouse for the Nanoboard, I started a wiki in order to explain this project as well as some of my other projects.
here is a video demonstrating the NB3K USB HID Mouse
Here is the Altium project zipped up.
Note: The project wont run unless you open up the debug soft terminal in the “Devices” view.
I haven’t been working on Android GPI (was called Android UCS) for a little while, and I thought it probably would be better in the hands of hobbyist.
I threw it up last night along with a quick tutorial I hope it can find a home in some quirky project somewhere.
The eventual goal was to attach it to the Android Scripting Environment. If your interested in a windows manager for android, graphical sensor interface, or an Android DLCM port give it a go
http://code.google.com/p/android-gpi/
or you can just download the an APK
http://android-gpi.googlecode.com/files/android-gpi.apk
I would suggest using it on the real phone, and taking a look at the tutorial, I’m not a user interface guy, so the design is not as intuitive as it could be
http://code.google.com/p/android-gpi/downloads/detail?name=GPI%20Tutorial%201.pdf&can=2&q=
If your interested in working on the code send me a message.
I thought Mikes Function Generator was so cool, I modified it, and made a function generator that displayed results locally as well as broadcasting them, then I thought this is silly I should just make an application that uses both of these great libraries JEval, and JChart2D, so here’s a little app, that plots out what you put into it in real time, there is a lot of work that can go into this, but its a good starting point for anyone wanting to develop a real time math charting tool.
Here is the source code, and the actual distribution, it includes all the library, and if you have java 1.6 you can give it a go by
Downloading the dist.zip
go to that directory, unzip it, and type in
“java –jar JEvalCharter.jar”
Nothing groundbreaking but its cool seeing two libraries written by different groups interacting with each other.
Here are the files:
Mike, the other project owner of SubServ has created a function generator widget
I’ve been playing with it for the last hour and a half, its oddly cathartic to watch a tangent wave.
Its up on the project page right now, but it took some maneuvering to get the jeval library he used to work with SubServ. Simplifying the user experience might be our next move.
My task was to build an LCM demo for the Altium Nanoboard 3000. I’ve had a very difficult time. My main issue was communication. In the Nanoboard I had a soft processor that was running at 50MHz, and my desktop is running something around 2 Gigahertz or so. Both computers are fast enough but the pipe used to send data has been the bottleneck. I had ported LCM over to the Nanoboard, and then initially developed a provider (or a means of communication for LCM) that worked with RS232. It worked, but I never pushed it faster than 9600 baud, for getting a proof of concept out, I wasn’t worried about the speed, more about if it could get done.
Since the SubServ application has been working so well, I wanted to get my embedded design to work with it, Unfortunately you have to jump through a lot of hoops in order to get Java to communicate over Serial, so I’ve switched gears and worked on a TCP provider for the embedded LCM. The first one was done about 3 days before my last post. But my data flow was painfully slow. I could send data back and forth, communicate with the PubSubs, and basically prove the fundamental functionality of DLCM, but once I sent more than 1 packet of data every 100ms my embedded code would crash. I wrote the previous post as my initial attempt at remedying the problem, but still was limited to a painfully slow rate.
I searched the Altium forums for a solution, and found that other people had had similar issue with the Lightweight IP TCP/IP stack that is used. So I started working on a port of the simpler uIP TCP/IP stack. Porting uIP went relatively fast because it was written so well, and relied on the standard C library, and some driver code. I modified the emac driver code to utilize true blocking and successfully started communicating between my desktop and the Nanoboard again. This time I could send and receive a lot faster but only because I added a lot of workarounds for freezing, for example, when I was waiting for a posix interrupt, I would timeout after 10ms and poll the information. Unfortunately the uIP stack kept repeating itself when it would respond, so when I used Wireshark to packet sniff, I ended up with a bunch of warnings and errors on the TCP, and ICMP level. I rewrote the driver to be cleaner, and faster, and the system responded a little worse. At about 3am the morning I finished rewriting the emac driver for the second time I get an email from a forum thread I had subscribed to for Altium notifying everyone of another bug in the Altium low level functions. It basically stated that there was a chance that the interrupts could be completely disabled when switching threads, or when an interrupt occurs. Unfortunately that occurs most when you are working with multiple threads, and interrupts. I was working with 8 threads at the time, and using an interrupts. So for now I am taking a break from the Nanoboard port until the update to fix this problem comes along.
I have been focusing on the Android port of SubServ, its coming along pretty fast, so far so good, its actually a continuation of the UCS I had been working for android. Its behaving a lot like the Java version, except its more graphical due to the limitation of user input.
Here is a shot of the control canvas, or where you hook things up, sorry about the icons being the same for the integer function generator, and the integer graph widget
The image doesn’t portray the pulsing of the Integer output, the pulsing is so that the user understand that data is flowing out. In order to view the data, you need a widget, which is the icon on the right, here is a view of the manifestation layer, or the layer used to display data.
since the widget isn’t attached yet, it just displays “No Data”, but if I attach the two you can see that the widget is reading the integers that are coming in as a sine wave
my next task is getting Android to communicate with the PubSubs on the desktop.
In order to demonstrate the flexibility of DLCM, SubServ, and LCM I am working on an embedded design which communicates with SubServ over Ethernet. I’m using the NanoBoard3000 from Altium.
Simply put the NanoBoard 3000 is a playground for a HDL/Software engineer. I couldn’t do it justice describing it, but here the main wiki page for it.
The great part about the NanoBoard and Altium is that there is a lot of support for peripherals such as Ethernet, Serial, and USB, but they do need some work to get them to do what you want. Right now I’m creating a TCP LCM Provider, or a means in which LCM can communicate over Ethernet. Its a tricky process of jumping between the included examples, and Google searches, but I got one up and running. I based it off of my LCM serial provider. Unfortunately the provider right now has been restricted to a painfully slow speed. My mistake was attempting to process my data in the same thread that captured my data from Ethernet. I’ve been trying to devise a way to receive data, and process it fast enough. I’ve come up with a triple buffering technique.
Here is the problem.
Data arrives over the Ethernet asynchronously, meaning I don’t know when it comes. The data comes fast enough that if I try to read the data, and then process the data in the same function, I may receive new data while the processing is occurring, if the input buffer overflows then my connection drops, and I loose data.
Here is my approach
Two Threads, one dedicated to reading data from the Ethernet port as fast as possible (Data Thread). The other thread for processing data (Processing Thread). Data Thread always needs space to put information in, while Processing Thread is always looking for new data to work on. Since Data Thread is responding to asynchronous signals, it needs the highest priority.
Three buffers: Buffer A, B, and C.
Data Thread reads a packet and puts it into Buffer A immediately, and sends a notification to Process Thread to wake up and work on any available buffer that its not using. It then switches to Buffer B, if any data is arrives, it will write into that buffer.
Meanwhile Process Thread wakes up and starts working on Buffer A. When its done it releases Buffer A, saying its available to read new data.
Three scenarios can happen while the Process thread is working.
Now when the Process Thread is finished reading buffer A it looks for any new data. If there is nothing available like in scenario 1 then the processing thread goes to sleep, In scenario 2 and 3 Processing Thread has buffer B to work on. This way data is always being processed, and read.
I’m still debugging the provider right now, but I believe this approach will help out significantly. Once I get the provider finished I’m going to write the 3rd tutorial as well as a video for DLCM/SubServ which will demonstrate the communication between SubServ, and LCM.
I have created a Google code page to host DLCM, and SubServ.
DLCM is a java library. It is similar to LCM in that it is made for lightweight communication, and marshalling, but it is different in that a lot of functionality that was done at compile time for LCM is done at runtime.
SubServ is an application that uses DLCM and LCM to connect with pre-existing LCM programs, and other DLCM structures. SubServ is equipped with “Widgets” for data manipulation and viewing.
Main SubServ Controller, from here you can generate new lcm structures, PubSub (Publisher/Subscribers) with a given provider.
Here is a screen with two PubSubs communicating, and using the Paint Widget
Here is a screen with the same PubSubs as well as a VirtualBox Ubuntu PubSub
I wrote a couple of tutorials. I am not experienced distributing code so I’m not sure how to create easily downloadable distribution, but the most recent java release is here:
SubServ was written for Java 1.6, I am planning on making it work with 1.5 in order to support Macs.
To get it started, go to the downloaded directory and type:
/download/directory/dist/java -jar SubServ.jar
There are a lot of bugs, and some of the functionality isn’t all working but it is usable right now. I wrote a couple of tutorials in order to get started, and I am working on a Tutorial that will illustrated DLCM and LCM communication.
Here are the links to the Tutorials:
Tutorial 2: Working with widgets
Here are some additional screens of other Widgets:
“Number\Slider Widget” in Ubuntu controlling a “JChart Widget” in Windows
Sending a String with the “Text Box Widget”
Dave
UCS
I’ve been working on the UCS primarily, and have been getting great results, unfortunately I follow the pattern where I work on something for a few weeks, get my head wrapped around it, and realize it would be better written another way. Its a fun experience, and it makes the hours go by and I think by the end of January I might have an alpha release.
I enjoy working in Android because the mixture of Java, and the Android methodology is powerful enough to easily get a GUI going, but flexible enough to allow me to create a new user interface. I know I’m starting to diverge from the main application format for Android, mostly because I wanted to create a user interface that is intuitive enough for a “pickup and play” like the ipod, yet powerful enough to talk to new devices. In the course of thinking about how I was going to preset the information read from an arbitrary dive such as a local GPS, or a LCM connected ADC I thought of the manifest layer that a lot of audio software uses for controlling and displaying information. Another example of this duality would be LabView.
http://www.ni.com/labview/whatis/
UCS would differentiate from LabView in that it would be more plug and play, as apposed to program, plug and play.
LCM
I bought an Altium Nanoboard 3000, and it will be my first LCM victim. It’s also an excuse to optimize, and clean up my LCM control code and then connect to all the interesting peripherals. After considering what I want to do with LCM I realizing a new problem. LCM is a means in which one can send, and receive information across two mediums (An embedded device, such as a GPS Linux box, or my FPGA, to some sort of information gatherer) The main problem I see is that both sides already have to agree on the format of the information that is sent between the two. Its not a big deal for a closed system like the projects in which the protocol was originally designed for, but when it comes to something more dynamic, and direct like my project pre writing the data structure doesn’t make sense. To that end I was thinking of adding a new layer onto LCM, a flexible layer that adds a few or many controls. It would allow one program to query a device for its capabilities and then dynamically setup its data structures to receive it. I think it can be based on the lcm generation program used to create LCM data structures external to the final program, and port it over to java, which, instead of creating a separate data structure class will utilize a generic hash algorithm, and then use either an ArrayList, or HashTable to store data. The main issue I feel the most unsure of is going to be computing the data integrity hash arguments that LCM utilizes to validate incoming/outgoing data. Its a little way down the road, but embedded lcm, and dynamic lcm are on the horizon.
Porting Eclair
I received a message from someone interested in getting Android Eclair running. I thought it would be a simple matter of applying the patches I wrote for Android Donut, but the first patch I tried to apply to the InputDevice.java wasn’t happy. Its not a very big deal to work with Donut as apposed to Eclair, but Eclair feels a lot more polished, even giving a quick glance to the InputDevice.java there has been more attention paid to smaller details. It seems like Android is starting to mature enough so that the developers can work on some finer details. I’m very happy that I have Android running, but I sort of need some hacks to handle the higher screen resolution on Devkit for Donut. The support for the higher screen resolution doesn’t exist for API level 3, so every project I start for Android involves a strange voodoo dance to trick it into thinking its a level 4 (Eclair) program running on a level 3 device. Otherwise my screen is only 50% utilized.
I needed a way to exchange information with my FPGA’s soft-processor and my development board running Android. I found the term I was looking for was marshaling:
“In computer science, marshalling (similar to serialization) is the process of transforming the memory representation of an object to a data format suitable for storage or transmission. It is typically used when data must be moved between different parts of a computer program or from one program to another.” -http://en.wikipedia.org/wiki/Marshalling_(computer_science)
After doing some research on marshalling I found CORBA, and Ice. CORBA has been around the longest, but it lacks cohesive support because although the protocol was written by a committee, the actual implementation varies, and one implementation could potentially (and normally) not be compatible with another. Ice is another marshalling protocol, which has the benefit of being centralized in its design, and implementation. It even has a version for Android, but it requires an operating system to run correctly. My FPGA’s soft-processor allowed me to use a micro-kernel to handle scheduling, thread management, IPC and a lot of other low-level systems, but it lacked a lot of things that Ice required. My friend Andrew turned me on to LCM. A protocol that he uses at his school in order to facilitate communication for robotic based projects. Here is a quote from LCM’s Google code page:
“LCM is a library for message passing and data marshalling targeted at real-time systems where high-bandwidth and low latency are critical.” – http://code.google.com/p/lcm/
Originally I envisioned a function called by android calling a function within the FPGA, and returning the results. LCM is not designed to work like this, it is based on a “publishing” “subscribing” methodology. I plan to implement a pseudo function with this, and eventually expand on the protocol to have a dynamic channel, and type adding feel to it (think jini in java), but that will have to wait until the base UCS is up and running.
Custom LCM for Altium
I believe that LCM was originally intended to sit on an operating system like Ice, and CORBA. In order to get the LCM runtime started on my device I had to work around two libraries that LCM requires: glib, and regex.
regex was primarily used to parse data in a the initialization portion of LCM, I was able to get around it with the use of strtok, and strcmp. They are not as easy to use as regex, but it was enough to get the job done.
The replacement of glib was much more involved than regex. The two parts that needed to be circumvented was the glib hash table, and glib threading control. Threading control was accomplished by utilizing the lower level functions that pthread offered, and the GHashTable was replaced by a custom list. The list is nothing special. The main performance hit will be the inability to utilize the name of the channel as a way to quickly retrieve a handle using a hash function. In order to accomplish the same ends as GHashTable hash lookup I simply traversed a list and found a matching string “tag”. For a small amount of channels. This performance hit will not be noticeable, but if any large amount of handlers are required, other methods should be developed.
Serial Provider
My FPGA development board came with many things. But one thing that was not included was an Ethernet phy. It would have made the project much easier due to lwip (light weight ip) that was included in Altium, but since the IDE required an Ethernet phy hdl implementation before I could use the rich features (without a lot of hacking) I turned to the serial port.
Inheritance doesn’t exist within C, but the team at LCM utilized function pointers to accomplish the same ends as an inherited class would. Basically implementing the functions designated by the function pointers was all that was needed in order to get the system up and running. In order to use two way communication on a the serial port was another story. I’ve included my serial provider in hopes that it can be adapted towards a more general platform. I used the mqueue.h (message queues) provided by Altium, and unfortunately I think those libraries are platform specific (EDIT, thanks for the feedback Woodrow Douglass, mqueue.h isn’t specific to the platform). I believe they can be easily replaced with pipes, which isn’t part of Altium’s software IPC repertoire.
I am very new to threading, and I wasn’t able to get my read thread to sleep because for some reason the fread function wouldn’t block, and this put the thread to sleep. I needed to force the thread to sleep after every packet of data I received. I believe that this could effect performance a little, but I didn’t understand threading well enough to implement a better solution.
LCM For Android
LCM for android didn’t take nearly as long as the soft processor. I basically dropped the package into a Android project, and let it scream at me until I was able to get rid of the specific parts that didn’t work. Unfortunately, I think the Android developer team hadn’t implemented multicasting on Android until the most recent release. This is unfortunate because my development board, and phone go up to Android 1.6, and I don’t think either of them can support the 2.0 version. This means that UDP (Which I believe is the primary provider of LCM) is out for now. I’ve included my source code for a running android implementation. The serial provider for Android is a work around more than a true serial provider.
Android doesn’t have any support for serial, The previous time I got Java talking to a serial port I had a lot of hoops to jump through, so I understood that getting serial communication was not an easy task. I wrote a daemon that works in the native GNU/Linux environment that Android sits on top of. It must be run as root, and it opens two serial ports on one end, and three sockets on the other. The serial provider in Android simply communicates through a raw socket, and that information is relayed through the serial port to the FPGA.
Extra Notes
My code for the lcm.c, and lcm_serial.c is riddled with a lot of comments. I plan on cleaning it up a lot, but I left it in so that anybody wishing to use my work can see where I modified the original code.
For both LCM implementation lcm-gen works unmodified, so far. A lot more testing must be done before I will be confident in it.
File Links:
I shouldn’t really call this an update, since I have been working on this more more than 7 months. Here are some things that needed to be written down.
The Story of Android
I got the Devkit8000 (a clone of the beagleboard development board) and started playing around with the different packaged operating systems, and one of them was Android. I was elated to find it had already been ported onto this device. The excitment was short lived when I realized that the port didn’t include any source. Since the Devkit was so close to the Beagleboard, I joined the beagleboard mailing list, and started searching for the differences between the two. For the most part they were quite miniscule, in fact you can almost boot up Android for Beagleboard but you need to tell the kernel that it was okay to boot on the Devkit8000. Initially a work around for this was to disable the board check, but After searching around the kernel porting scene I found a board number had already been assigned to Devkit. As well as the board number some other notable differences include: On board Ethernet, some uart ports were exposed in different ways, and a experimental USB Interface.
In order to modify the kernel to support a different device, there are a few things that need to be modified:
I purchased the 7 inch touch screen option when I bought the Devkit, and in order to get it working I needed to learn about the hardware display support for the OMAP3530 called the display sub system (DSS). Basically the DSS has four main parts:
1: static struct omap_panel dk8k7_panel = {
2: .owner = THIS_MODULE,
3: .name = "panel-dk8k7",
4: .init = dk8k7_panel_init,
5: .enable = dk8k7_panel_enable,
6: .disable = dk8k7_panel_disable,
7: .suspend = dk8k7_panel_suspend,
8: .resume = dk8k7_panel_resume,
9:
10: .timings = {
11: /* 800 x 480 @ 60 Hz Reduced blanking VESA CVT 0.31M3-R */
12: .x_res = 800,
13: .y_res = 480,
14: .pixel_clock = 36000,
15: .hfp = 1,
16: .hsw = 48,
17: .hbp = 1,
18: .vfp = 12,
19: .vsw = 3,
20: .vbp = 25,
21: },
22:
23: .config = OMAP_DSS_LCD_TFT,
24: };
The touch screen was difficult because I couldn’t find any way apply a configuration file within Android. I suppose that capacitive touch screens were the only types of touch screens that people thought were going to be used with Android. I found a post about this problem from someone who figured out how to modify the java input file to allow a configuration file to modify the input signals. I’m still not sure if this is the best solution, but I followed his post, and was able to add a configuration file that InputDevice.java read, and change the inputs basically rotating all screen touches to 180 degrees clockwise.
aaaanndd Android is up and running. interestingly enough as soon as I plugged the USB slave of my running android system the Android Development Tool recognized it, and I was able to download programs directly to it, as well as debug.
The Story of the Digilent Nexys2, and the TSK3000
The UCS will interface a FPGA with Android. In order to accomplish this I needed an FPGA. I started using a FPGA development board I got from SparkFun, but I ran into some difficulties with it, so I purchased the Nexys2 from Digilent. An amazing deal as far as FPGA’s go: Nexys2
My father and I purchased Altium Designer in order to develop hardware. It is an incredible piece of software, that not only integrates schematic capture, and hardware layout, it is capable of capturing hdl source, schematics. The most applicable for me now is the ability to set up a soft processor inside the FPGA. Its like a virtual processor. The software suite also comes with an integrated development environment that interfaces with the soft processor seamlessly.
Altium didn’t have any inherent support for my board, but after modifying the constraints of the FPGA, and choosing a place for a soft jtag I initialially started building a normal HDL schematic for my design. (An old example of my modified Altium video capture HDL can be seen here. I didn’t capture the entire thing, I only added the top left module, and the underlying Verilog to control the Sparkfun camera Sparkfun Video Capture Schematic) unfortunately the enhanced software interface that was added to Altium Winter 2009 doesn’t recognize schematic components, and encourages developers to use the new Open Bus. I have fought it tooth and nail since it came out, because it doesn’t feel like developing HDL, but in the end I submitted because the IDE can speak with Open Bus designs, and automatically add support for Serial Ports, Soft Processors, Ethernet, etc… Here is my Open Bus schematic, its pretty simple: 
In the next post I’ll talk about the some of the Software that I needed to build in order to get the two boards communicating, including the software/socket daemon, Android Java, and the porting of Lightweight Code Marshalling (LCM).
I started to thinking about all the common problems I had whenever I worked on a project, and found that one of the most frustrating things I had to deal with when designing anything is the user interface. It is something relatively common to all projects… we need a way to interface with our design.
Three technologies that need consolidation
Altium: Create HDL for an FPGA faster, and more reliably than before.
Beagleboard (or the clone Devkit8000): A hardware embedded development board that is capable or running GNU/Linux.
Android: A simple GUI OS that sits on top of GNU/Linux that allows a user to interface with their embedded system in an intuitive way, while using the power of Java.
The UCS will combine all three of these to harness the power, and flexibility of FPGA’s with the convenience of Android, and Java.