Day 38: OpenGL bugs, new PSU, 123d-AR plan, documentation

Plan of the 123D-AR demo

How it's all going to work! Used OneNote to splash out the basic draft of how everything works and sent it to Toby for review. It's a little muddled but hopefully sensical enough.

123D

Saw this 123D capture from Animation13, which interestingly managed to grab the television screen:

Good one

Interesting. Anyway..

I found you can actually upload multiple 123D captures simultaneously, just run multiple instances of the program and they all work independently. This will save a lot of time and allow us to get through many more 123D -> AR capture processes at events.

You just need to remember to make sure you are signed in on all of the 123D programs running simultaneously. If you aren't signed in you can't export models.

OpenGL 3.3

Helped Georgiana fix a bug issue with OpenGL 3 today, she's been porting the older OpenGL 1 lab code to the newer version to make it a little more up to date. Previously, the code to draw primitives (like a circle) would take points sequentially so you could have a loop that generated the next point to draw on every iteration and passed it to the function. The new version of OpenGL however, takes an array of complete points at once (and as per standard C, you tell it how many points from the array to wish it to draw).

Attempts to pass points into the new functions by iterating a loop was the root of the problem, and was soon fixed. Now we get correctly drawn circles and primitives! The change means we need to make an array to hold all the points before they are passed to the drawing function, which requires an extra variable in memory.

Donald's new PSU

... is installed and running very nicely indeed. Astoundingly quiet and reliable device, a BeQuiet! L8 600W. Should outlast the rest of the system by a good few years I hope.

Documentation

Started making the user manual for the AR. Will have a sheet with instructions for the 123D operator, and another one with instructions for the AR operator, which include instructions, FAQs,  general guidelines and whatnot.

Will also make documentation for how to reinstall AR - 123D on  new machines.

Day 37: Polishing ExportJPEGs

Added lots of exception handling to ExportJPEGs, now it will rarely crash from I/O file and directory issues, but rather print a hopefully useful error message (see below) and continue provided the problem isn't fatal.

Added a bunch of methods for handling the red label text and using it to show errors and messages when the program is running:
setLabel(String text)
setPersistentLabel(String text, int time) <- time is how long the label persists
setPersistentLabel(String text) <- time defaults to 5 seconds if not present
renderLabel()

This affects the large red text at the top of the screen, which prints useful messages and prompts when necessary. Via these methods you can set the text to anything you like, normally once it is set to something the label stays like that until something else sets it to something different. Now you can set a persistent label where it will not allow anything to change it until a certain time has passed (except another persistent label call). This is useful for e.g. printing error messages that you want to make absolutely sure will stay on the screen for a number of seconds.

Also the render label method splits long text over multiple lines and renders it. Previously if the label was too long it simply went off the end of the screen, now you can actually see the whole message.

Also added functionality to load all existing models at startup. If when the program starts up, the incoming folder has any models in it then the program will prompt the user whether to load existing models or ignore them. The user can then choose to load the existing models or ignore them. It's all done quite nicely and intuitively in the GUI. So now if there are a bunch of heads we want to show up from the get go, or if the program shuts down and has to restart, you can safely get all the heads back as long as you have the original models. On that train of thought, I also set ExportJPEGs to clean out the built heads folder on startup. So if heads had previously been built before starting the program now, they won't get loaded up again. This prevents LowryDemo from auto-loading old heads and from wasting storage space. Hmm actually I think I can improve this.. Okay we'll see how this goes.

Now making a separate background thread to check for files on the server every so often, this way when downloading files it won't freeze the GUI.

Also set the upload gif part of the program to always use unique gif names. In this way no gifs can possibly be overwritten.

The way the server file sync thing will work is that the push script on the 123D machine will have an exact representation of the server folder. So anything not in the folder in the push (outgoing) directory will be deleted off the server. In this way the 123D people have full control over the models that are on the server and can replace them. If a model is replaced, the AR/ExportJPEGs program will notice and build it again (it won't replace it's old model on the list though, once a model is built it's there for good). At the end of the day, if they want to clean up the server folder just empty the outgoing directory and wait for the script to update (which will probably be set to do so once a minute).

Importantly, the 123D people MUST NOT EDIT OR RENAME TEXTURES AFTER THEY HAVE BEEN EXPORTED from 123D. Just export and send them as is, if any texture is renamed after being exported then it won't work. You can rename models though but I strongly suggest just exporting and sending as is. It's best the 123D people export directly into the outgoing folder.


Anyway, all these details plus anything important will be packed into a little user manual shortly, for the future operators of the wondrous 123D-AR demo.

Day 36: Exceptions, LowryDemo, etc

No more Cygwin

I learnt some fantastic stuff today - I can remove the need for cygwin from the AR completely and make it a fully native and portable windows executable program. Well you may need to install some visual c++ redistributables, and I'm not sure if it will work without XNA installed... but it's pretty standalone now! I have a few required .exe's that need to go around with the program, but that's basically it.

First off a big thank you to the University of Manchester for making certain file transfer protocols unusable over wifi. If it wasn't for the fact that lftp (and ftp and rsync and everything else I painstakingly tried and tested) didn't connect over wifi I would never have even looked for alternatives and learnt all the wonderful gadgetry I learnt today.

So first off, I learnt that WinSCP not only has a synchronize feature to rival lftp mirror, it also comes as a .NET assembly and a portable executable - and it works over the University wifi to boot.

Now, the fact that I can use WinSCP from inside the ExportJPEGs program (remember the .NET assembly is available) means that I don't need to use any cygwin scp or ssh or lftp or ftp or any of that gubbins. Which also means that now the only reason I have cygwin is to use the ported image magick convert tool. After a little research I found that image magick also offer a portable version that works on Windows, which comes with a little convert.exe that, after testing, works exactly as intended. Which means I suddenly have no use for cygwin at all.

Well one thing's for sure, I've completely given up on making this program portable. That's probably a good thing because I don't fancy working out how to manually port all of Goblin's code onto Linux and Mac (XNA has been ported to Mac via MonoGame and MonoDevelop, but no one has bothered with the Goblin wrappers). So from here on out, the University of Manchester AR and 123D wonder is a completely Windows based offering.

In other news I managed to destroy LowryDemo because I messed up with using the git repository and ended up with ridicuous "<<<<<<<< HEAD" lines in random files that prevented the whole thing from loading or building. It didn't even tell me which files had been victimised, so the only solution was to just revert to my backed up copy - luckily I have a few. Then I had to redo all the content optimizations I had made, moving textures and models to a central folder, setting relative pathnames for most of the models (converting them to binary first), etc.

First automated draft complete

That's right. I finished implementing the automated loader code in LowryDemo. Now it automatically loads up every single precompiled head in the heads directory, then waits for new ones. As ExportJPEGs compiles new heads, it automatically loads and adds them to the latest free model set.

It's all working!

At one point I got totally lost because WinSCP doesn't copy files the natural way. In fact it makes a placer file, then renames the file once the copying is complete. While this is a great idea, it meant that my file watcher (which watches for newly created .fbx files) completely missed the renames until I told it to watch for renamed .fbx files as well. A simple problem but it had me lost for an hour or two in despair thinking all was broken.

Tomorrow I improve it, add exception handling, etcetera.

Day 35: Donald's PSU

Doing what I do best, troubleshooting faulty PC's

Donald wouldn't start up today. Power comes through, everything starts whirring, but nothing comes on screen. At first I thought it was the fact that I hadn't used all 9 motherboard standoffs when installing the motherboard, and now some parts were shorting out. So I removed the mobo, stole some standoffs from an old Pentium IV PC nearby (I'm sure no one is going to use it...) and reinstalled the motherboard with standoffs all in place. But it still didn't work.

I checked RAM, checked drives, checked graphics card, and noticed that the red CPU led was lighting up and staying lit. Normally it would go off when the mobo gets past the CPU test. These little led's are a wonderful feature of this Asus M5A97 motherboard, allowing the user to instantly know where the board is failing as the PC boots up. There are led's for CPU, RAM, VGA (graphics card), and boot device.

Anyhow the CPU was either broken or not getting enough power. I checked the power cables to the CPU and to the motherboard but still didn't work.

I then removed the entire motherboard and removed absolutely everything except the RAM, the CPU, and two power cables from the PSU - the 8 pin CPU power and the 24pin ATX motherboard power. Didn't work, same problem with CPU led lighting up.

I then opened Walter and used the 8pin and 24pin cables from his PSU, and boom it all worked!

So we ordered a new BeQuiet! Purepower L8 600W (only one of the best most reliable PSU's you can buy) from Scan which should be here soon.

Getting AR up on Walter

Moved the development to Walter in the meantime, luckily all the code was uploaded to our BitBucket version control system so I had the latest code ready to go.

To install the AR on a new PC it involves installing all the component software like Visual Studio, XNA, Alvar, OpenCV etc, and also Cygwin (for the shell scripting part of it).

Cygwin, installed openssh, imagemagick (both), and lftp.

Aaaaaaand it looks like lftp doesn't work over WiFi. Oooooh deary.

Welp, there goes the fantastic lftp mirror command I liked so much!

Finished off the Automated GUI and it works great! Only thing left now is to get Lowry Demo to automatically load new heads, should be a cinch! Oh and also to find a replacement for lftp mirror. It really is a shame that it doesn't work over WiFi, because it is just so ideal for what we want.

And I also need to add exception handling for when a build fails. (VERY IMPORTANT HAMZA DON'T FORGET)

LaRue

You guessed it, a new PC! It's an old Dell T3400 workstation called Atlas50 or something, which for some reason has 8GB's of RAM in it (!). It has a decent dual core CPU and an 80GB hard drive so it isn't ancient. It's a real pain trying to get drivers to install on it though, the darn thing doesn't recognise USB devices (because the chipset drivers hasn't been installed) and I can't connect to the internet to download them (because the ethernet driver hasn't been installed). Gads. Well it shall have to be done on Monday then.

Till Monday!

Day 34: Walter arrives, MEng report, Vacation Extension and a dual-list GUI

Walter arrives

I now have two PC's on my desk! I do like having two PC's on my desk.

Reinstalled Walter with the oncampus windows 7, he's now activated correctly, finished installing everything up to date and got the AR Demo with version control all sorted out and set up. There are massive amounts of backups on Walter, on Donald, and with a full set taking up 300 or so GB on the 1TB external hard drive.

ExportJPEGs and models

So it looks like having a scrollable list of heads and allowing the user to select between them is going to be a little difficult (i.e. I have no clue how to finish the implementation of ScrollPane and it would waste too much time to try).

So as an alternative I could have one list with space for 10 head names, and then allow the user to press a button to go the next set of 10 names, and the next and so on and so forth. Two buttons in fact, to go back and forward between sets of heads. Then the user can export them as they see fit.

Implemented it, ended up having two adjacent lists on the GUI because that means I can fit 20 heads on one screen instead of just 10 which is great.

Some extremely complicated programming was involved, so I used a bit of good old abstraction to make it all easy on the brain. Basically trying to decide when the forward and back buttons should be enabled and disabled, and working out when to initialize a new list for each of the two lists, and when I can assign an initialized list to one of the visible lists and how to determine which model to load when all I have is a list index that could be from any of the two lists and all this stuff is nigh on impossible. So, I simply made a bunch of wrapper methods around the two lists that make them behave like a single, simple list.

Namely:
makeNewHeadList() - which actually makes a pair of lists but the user doesn't see that.
setHeadList(int whichListToShow) - which shows on the GUI the correct pair of lists, indexed 0, 1, 2 etc, each index referring to a pair of lists.
getModelIndex() - which takes a list selection (i.e. after a user selects an item in one of the lists on the GUI) and returns the index of the corresponding model so the program can load whichever model was clicked.
addModelToList() - takes a model name, and adds it to the latest slot in whichever of the two lists it needs to go.

(remember each list is showing 10 names of models, the user clicking on a name makes that model show up in the GUI)

Here's a little taster of what it's looking like:

ExportJPEGs running - the heads will show up in the list as they are added to the watch folder

Placement Report

Got an email from the MEng manager Nick Filer informing us that we will need to write a full-on 35 page report regarding the projects us MEng students have been doing over this summer - worth 15 credits alone! I'm so glad Toby had be blog everything, because this is just great - I have all the information I need all here in this blog and all I need to do is write it out into a report. I would have loved to do this Augmented Reality stuff as a third year project (not in any way belittling the project I actually did, a SpiNNaker demonstration, because that too was awesome), and now I can finally live the dream with a lovely little report and a seminar all about the work I've been doing.

This is quite probably the first time the prospect of writing a report hasn't been met with unadulterated despair bordering on utter doom - what a nice change :-)

The vacation experience

So, in general, this summer has been absolutely fantastic. I've enjoyed every bit of it, from the copious random tasks to the huge events to the rebuilding some ultra-extreme-cutting-edge PCs to the difficult programming to the combining of Visual C#, .Net, XNA, 123D, shell scripting and various other technologies in order to make one heck of an Augmented Reality program. So much so that Toby is actually pursuing an extension to my work placement! If that goes through it would just be great. I can spend three more weeks doing what I do best and having a great time doing it, and as a result we should see a super polished and refined Augmented Reality experience by the time this summer is done. Count on it.

Day 33: Results of hard labour

Stuffage

Today I did the following:

• got the FileSystemWatcher working in ExportJPEGs (calls an event handler whenever a file is added to the folder)
• Implemented ExportJPEGs to do everything up to the uploading of the gif, when user clicks export button
• Implemented ExportJPEGs to automatically run lftp monitor upon starting
• Implemented ExportJPEGs to fix model and rename regex inside fbx files when file is added to watch directory
• Implemented ExportJPEGs to automatically build models after the files are ready

Now I basically want a system where new heads that are imported from the webserver show up in the GUI.

So the new files get built automatically and sent to the common folder (that LowryDemo and this ExportJPEGs can access).

ExportJPEGS displays a list of all head files that are ready for loading (the pre-load building is happening automatically, after files are built they are shown in this list).

The user can select a head from the list and click a button to load it, which brings it into the view so they can move it about and export it as a gif as and when required. In this way the user can switch between heads back and forth and re-make-and-upload gifs.

Update: Just found out GoblinXNA has only a half-finished version of a ScrollPane. Its basically disabled, but in the source code you can see a mostly-complete-looking implementation of a ScrollPane with the comment "IMPLEMENTATION NOT YET FINISHED" at the top.

Without a ScrollPane our list won't be able to scroll up or down, so when we have 50 or so heads in they will just overflow the view and run off the end.

Well... I've been having a crack at fixing scroll pane myself. Joy.

Day 32: GUIs and exporting, Animation13 web gallery

Batch Photo Processing - sed and imagemagick

Got my CS password reset and now I can successfully get into babbage.

Image magick has a tool called mogrify which is used to batch convert images, but I simply hacked together a shell script or two to get the job done with the renames and everything in tow.

This is the basis:

for infile in *; do

outfile=$(echo $infile | sed -n 's/\./.preview./p');

test -z $outfile && continue;

convert $infile -strip -density 72 -thumbnail 300x300^ -gravity center -extent 300x300 $outfile;

done

Which took all the images called img_xxxx.jpg and made previews according to the required specification. A similar command was run before this to turn the raw images into suitable main images of dpi 72 and resolution 1200 by whatever proportional y value.

Results are good: http://animation.cs.manchester.ac.uk/gallery/#/Gallery/Animation13

And then back to automating ExportJPEGs

Coding and GUIs

So the plan for the GUIs is to have 2 sliders, one to control the vertical height of the model and the other to control its size. There will also be a checkbox to flip the head upside down (because some heads are simply upside down).

All done, and it works nicely!

Easiest to use JPEG exporter on the market!

Now I need to manage loading content into the program, so let's do a bit of thinking..

I know I can build XNB files outside of the program. I can manually press a key (or a button on the GUI) to load the newly built files in and display them in the GUI once they have been built (so the user can export JPEGs). I wonder if this can be automated..

Found that you can watch a directory for file changes using a FileSystemWatcher, details here: http://www.codeproject.com/Articles/26528/C-Application-to-Watch-a-File-or-Directory-using-F

So perhaps I can implement a system that monitors for files and displays them in a drop down list on the GUI? And when a new head model arrives the user can click the head in the drop down list, and the program will build it, load it so the user can export jpegs, and send it to the lowry demo program.

This will completely remove the need for shell scripting to manage and move files about, but we will still use shell scripts to monitor the web server and grab files, and to make and upload gifs after JPEGs have been exported.

Now the final step if I do implement this would be to tell LowryDemo to load the next head once the head is available. Again, this can presumably be done with the FileSystemWatcher - watch the content directory for new .xnb files coming in, when they do come in then load them and add to the head model set.

FileSystemWatcher... what an utterly fantastically useful tool!

The answer has been comprehended

And now I've discovered you can run bash shell scripts from Visual Studio C# applications (actually you can run them from an MS Dos batch file (.bat), and you can run such batch scripts from C# programs). So in other words I can automate the system so that all the user has to do is to run this exportJPEGs program, and the program will automatically do just about everything:

1. run LowryDemo
2. run the lftp monitoring client to grab new heads from the central file server
3. fix the model and texture files, rename them etc
4. build them
5. have LowryDemo automatically sense newly built files, load them and add them to the AR program
6. load the files into this ExportJPEGS program, display them in the GUI for the user to set the view etc
7. Let the user click the export button, which will and that will not only export JPEGs but also convert them into a gif AND upload the gif to the web server.

So, now let's see if I can get all the parts of this to work together nicely.

Automation tasklist:

• Get FileSystemWatcher working in ExportJPEGs (return event when file added)
• Implement ExportJPEGs to do everything up to the uploading of the gif, when user clicks export button
• Implement ExportJPEGs to automatically run lftp monitor upon starting
• Implement ExportJPEGs to fix model and rename regex inside fbx files when file is added to watch directory
• Implement ExportJPEGs to automatically build models after the files are ready
• Get FileSystemWatcher working in LowryDemo, load new head after content is built, manage model sets etc