Day 13: The future (of 3D vision techniques), today

123D Catch

Few important things happened today, firstly we tested 123D catch and did some pretty crazy stuff with it. It's a recent software developed by Autodesk that in essence allows anyone to upload a bunch of pictures of an object taken from all angles, and have a 3D model generated from those images automatically.

The processing is all done by uploading the images to the cloud, so you don't need a special computer - or any computer actually, you can even do it directly from your smartphone, tablet, etc which is pretty clever. This kind of stuff is what we were researching as part of the Advanced Computer Vision module earlier this year! Very enlightening to see a company has already made this ongoing research subject into a publicly available reality, so of course we just had to put it to the test. Actually we put it to the test because it sounds like an awesome demo to show off to the kids in Animation 13, but the first reason makes us look far more dedicated.

At the end of it you get a model that you can edit in the 123D catch program, or you can export it as an FBX, OBJ, etc to do stuff with it in other programs.

Attempt 1 was taking pictures of a model (me) in a swivel chair and swivelling after each photo. We did 20 photos indoors with flash against a gray background. The pics were of the head and a small amount of shoulders.

Result was terrible, the model received was garbage, we did a bit of troubleshooting and Toby discovered a few issues we hadn't realised: you aren't supposed to move the model, and you aren't supposed to use flash. Just goes to show we should sure as shooting go ahead and read the documentation before using the software next time. Welp! On we went..

Attempt 2 was taken outside, model (me again) sat in a chair and the camera Meister (Toby) moved around snapping shots at all angles with a lovely Canon SLR. We did one round from a highish angle and another quick round from a lower stance, total 39 photos.

Result was far better, it had the front of my face down very well, small mistakes but nothing significant and a very impressive result overall. The sides of my head and the back (basically all the bits that were overflowign with my long hair) were a problem for the software though, it didn't even have a mesh for the back because for all the pics with too much hair in them it couldn't find matching points (everything looks black-ish).

You can manually add matching points in the software after processing, and send the pics back for reprocessing, which turned out quite well but I still don't have a back of a head.

I think part of the problem is that the pics of the back of my head simply don't have enough interesting points as the entire view is taken up by my hair, so a possible solution is to zoom out in the pictures so we get some background as well as the back of my head, and take pictures with enough of the background overlapping in adjacent pics so that the software can match points in the background and work out whereabouts the camera is in relation to the model. That was a long sentence.

The position of the camera is calculated for every photo, so that should have significance in giving me a proper cranium.

What I plan to do now with 123D Catch is seeing how the models export, seeing how they work out in the AR demo, and getting the exported objects to load properly in GLC Player (and the AR program). Currently the OBJ export loads in GLC but the texture comes up completely scrambled.

Donald

I installed the new parts from Scan into Donald, and he now has a 6 core 3.5GHz monster of a CPU, 8GB of RAM, and a very impressive new 970FX motherboard. He's fast. Very.

Meet Donald... or something like that.

Out of the bargain we got a faulty P5N-E SLI motherboard, fully working 4GB of OCZ DDR2 RAM and a Q6600 quad core 2.6GHz CPU. The ram and CPU will likely be used to give Katy a boost - we don't like to waste good hardware and Katy really needs some more RAM after upgrading to Windows 7.

In fact, in order to make use of the extended RAM, I plan to reinstall Katy with the 64bit version of Windows 7, as the current 32bit version can only recognize 3GB. This wasn't an issue yesterday when we didn't know we were going to increase Katy's RAM, but obviously now that Katy is going to get about 6GB it's the obvious decision. At least now I know exactly how and what to install from the get go, to get Katy, Donald and the new Lenovo Z500 AR laptop all set up as fast as possible without wasting productive time.

AR demo with new software: XNA 4.0 and Visual Express C# 2010

I tested out the AR demo program on the newly installed updated software on Katy (XNA 4 and Visual Studio 2010), but unfortunately it didn't work straight out. However I'm confident it can be coaxed into being reasonable after a little effort. First it told me it needs to convert the current AR program to work with the new version of VS, claiming it's not backwards compatible. After allowing it to have its way with the precious code, when I attempted to compile it it threw an error telling me it can't find the skinningmodel or some such ridiculously childish claim, I plan to investigate further while Donald and Katy are being set up tomorrow.

If worst comes to the worst I will simply have to source out Visual Express 2008 and XNA 3.1 and just make do with the outdated software, but that is a last resort and to be honest I'm certain I can iron out the errors once I get cracking with it!

In other news I took 2 hours off work today to start making up for the extra hours spent last Saturday for the Open Day so there wasn't as much going on, but of course tomorrow productivity shall shoot up as expected with a good ol' 7-hour working day. I'll probably take the remaining two and a half hours off next Wednesday.

To the laboratory!


Oh and one last thing, I was checking the stats for this blog, and saw this amazosity:

It's me. I'm dead famous.

It's probably a glitch, but that many +1's is certainly nice to see!