The Iron Girl Project - HUD, Part 4

Howdy all!

The Unity version of the HUD is coming along nicely - I've finished the code to pull in all the data from the IMU to one of the information panels, and I've been tweaking the code here and there to make sure the readings are accurate. The only (noteworthy) issue I've run into has been Unity's update() function, which runs once per frame. The application is (currently) so lightweight that the framerate can exceed the read/parse process of the serial data stream. I've implemented a crude way of temporarily solving that problem (execute this process every x number of frames), but framerate will still vary widely in the editor. However, you can put a frame limiter on applications that have been built, so that'll probably be the way to go. 60 or 120 FPS ought to be a good number to base all of the functions off of.

Below is a screenshot of the application running with all the IMU variables in the right info panel, which can be animated/moved around at will (SO much easier than Processing). On the far right, the raw serial data is shown, while the data that gets barfed out for debugging is in the lower left-hand corner.

The new build will be available soon - it's actually already ready to go, but since I've modified the original Adafruit 10-DOF example code to streamline the ReadLine() operation, I'll need to put the Arduino code up with it, and potentially a copy of the 10-DOF library, so folks can compile the new code. Sooo....there isn't much point quite yet, as the only changes have been on the serial side of things.

Cheers,
~Lexikitty

The Iron Girl Project - Systems Report and HUD Updates

Howdy y'all! Some interesting updates have happened with the HUD, so I'll dive right in.

I've swapped Processing out for the Unity game engine, both for performance and ease of manipulation in 3D space. Unity works much like a regular 3D application, whereas Processing requires you to place objects via a coordinate system. Attempting to animate any elements in Processing quickly became a nightmare, and adding and placing new components was time-consuming. Here's an example of the first copy of the Unity HUD:

Since Unity supports C#, a JavaScript-ish thing called UnityScript, and Boo (a Python variant), it means there's a nice variety of coding choices for different elements. For instance, the scripts that run the dynamic text on the screen are in JS, while the script that pulls in data from the serial port (which is fed numbers by an IMU hooked up to an Arduino) is in C#. Below is an example of the working serial feed.

After a lot of blundering about and experimenting, I took a few weeks to got some actual training in Unity (from a company called Digital-Tutos), after which I redid the HUD in GUI elements, rather than TextMesh objects mapped to 3D assets. This way, no matter what, elements on the HUD will resize to fit the screen resolution and aspect ratio of the target system. (The end goal of this is to achieve stereoscopic rendering via Google Cardboard's SDK, but shhh - don't tell anyone, that'll be a surprise). Examples of dynamic GUI elements:

Test builds are now available for both Windows and OS X over on the download page. Please download them, and comment below/send me a message on your experiences. I have a decent number of Linux and Windows test machines, but I have no testing platforms for OS X. I also don't have any way of testing multiple camera setups, as I only have 3 or 4 at home. If you put something on COM3 at 115200kbs, it'll write it to the Debug Log.

Have fun with it, and let me know the results!

In other news, I'm currently delaying work on fabrication (and to a lesser extent, design) of physical pieces for the IGP suit. I may be moving in the next few months to a larger space, which should greatly increase my ability to produce parts, but until then, I'd rather not prototype a bunch of stuff that I'll then have to carefully box up and move. So expect HUD-focused stuff for the next few months. Around September, things should settle down again and I'll be able to switch to physical production.

Have a lovely evening, and go make something awesome!

Cheers,
~Lexikitty

The Iron Girl Project - HUD, Part 3

Part 3 of the HUD! Yay! This'll be a short update, as not much has changed except graphical improvements.

After the previous migration to Processing (with the hopeful RPi port), and a few tinkers with .OBJ files for the horizon, I decided to stick to Processing-based assets as much as possible, and only importing images where needed (because drawing outlines by vectors is sometimes stupid. The end results/evolution is shown below.

And the current state:

Data still comes in over the I2C bus from the 10-DOF sensor, but I've added in a few more functional variables, like the Repulsor charge indicators (with moving bar graphs), system uptime, and the compass indicator up top. CPU usage is low with this rig so far, so as long as I don't introduce face-tracking, I should be okay to still run this setup on an RPi.

I'll be taking a bit of a break from the HUD to focus on the underskeleton 3D models, so I can actually start nailing tech to my body.

Oh, and other exciting news! The code for the Mk 0.4 version of the IGP will be up on GitHub within the next few days. This will include the Processing and Arduino sketches I'm using. I'm under the name Lexikitty (what else?) and the repository will be Iron_Girl_Mk_0.4.

Go have an awesome day, and build something neat!
~Lexikitty

The Iron Girl Project - HUD, Part 2

So here's a few updates on the HUD and AHRS (Altitude Heading and Reference System).

I recently got an Adafruit 10-DOF IMU to handle all the head tracking and movement data for the "horizon lock" effect seen in the IM Mark II display. Combining this with the sample code and a few modifications, I managed to get something along the lines of this:

Which doesn't look too shabby at all, at least for a proof-of-concept. After fiddling about with calibration settings and how to display data on-screen in Processing (I'm still somewhat new to it), I managed to get a slightly more informative display up and running. I also added a "horizon lock", since the real Mk II showed that on it's maiden flight.



Still a long way from the original. But still, this is the work of just one girl (who has a full-time job and a girlfriend), and I'm still aiming to get as many of the systems seen int eh IM movies (particularly the initial Mark II test sequence, below) as functional as realistically possible.



It remains to be seen how much of this will be able to carry over to the RPi - if I can at least get the OBJ loader to work, I may be able to simply black out everything else and "fake" keyframing with the opacity switch. Remains to be seen. For now, though, I'm off to model a real HUD model, since that last one was crappy alpha at best.

Cheers,
~Lexikitty

The Iron Girl Project - HUD, Part 1

When I was looking into the Iron Girl project, the most intriguing - and scary - part by far was the HUD. The HUD on the Mark II is magnificent, and it only gets better with each suit in the movie series. If we skip the holographic and multi-layer effects (again, tech not financially feasible), we’re left with the concept alone:

A video feed from the outside world, as well as:
Numerical and sensory data about the suit
Corresponding visual references to video feed images (heading, bearing, horizon line)
So originally, I wanted to use some sort of USB webcam > OpenCV (computer vision application), and have the Pi do all the processing. Turns out both of the webcams I had lying around had issues with the Pi - one needed additional power, and the other gave me about 4FPS tops (without any image processing). So I ordered the standalone Raspberry Pi camera board, hoping that would have a better framerate. It definitely did, but the issue there was that it injects video directly to the framebuffer - meaning that the operating system/Python/X has no way of knowing what the heck the camera is seeing. The camera feed just fills the entire screen right over whatever it was that you were doing. Rude, but useful in some circumstances.

Using the raspivid command-line arguments, you can specify x and y coordinates for where you want the video. In doing this, you can kind of fake an X window at specific coordinates or use a non-resizable GUI to get the effect. However, the mouse pointer will always disappear behind this, and for the HUD purposes it didn’t make sense to shrink the video to the center of the screen to display edge-HUD elements.

There is, however, a raspivid command-line option labeled -op for “opacity”. This gives you a range of transparency (0-255) on the video itself, so you can actually have highly visible elements BEHIND the video feed and they’ll show up okay. Color mixing is a bit off, and it’s not the prettiest solution, but short of delving deep into OpenGL and MMAL just for the sake of this one project, it was good enough for me.

Booting into Wheezy, I pulled up a sample HUD image in the picture viewer, made it full-screen and zoomed in till I got the edges to touch the screen. I then started raspivid and played for a while with the command-line. At an opacity level of about 110, I managed to get something reasonable:

So that worked okay, and decently for my proof-of concept. I ended up adjusting the settings a bit since I would be working with a pair of video glasses on the composite output of the Pi, not the HDMI port with a nice monitor. The glasses only have a resolution of 720x480. The specific ones I used were based off the Olympus Eye-Trek FMD-150W, but actually removed from an old Enhanced Visions JORDY unit that had cracked on the side and lost both earpiece hinges:

I switched the output to composite on the RPI and had to adjust the contrast settings and brightness to make it look anywhere near decent, but I manged to get something workable with my test HUD image and the video feed overlay:

So far, so good.

For anybody looking for cheapish HMD’s, the JORDY might not be an entirely bad option. Just make sure you look for a version 2 if you need widescreen. The version 1 has a black cord linking the controller unit and the glasses, and has a notch between the eyepieces (right above the nosepiece). The version 2 has a clear/braided cord and no notch above the nose piece. The only real difference is the resolution of the internal screens - the version 1 is based off the FMD-150 and has a resolution of 640x480 (to the best I can tell). The V2 is the FMD-150W with the 720x480 resolution. I manged to find a V1 for about 60 bucks on eBay at the time of this writing. Adafruit.com also has 320x240 HMD’s for $109, and you won’t have to do any salvage.

For the rest of the overlay, I decided to use Pygame to load all the data off the I2C bus, but that’s for another post.

Cheers for now,

~Lexikitty