Latest fun thing in the office: a Movidius Neural Compute Stick

A Movidius Neural Compute Stick just turned up in a delightfully retro style box. Won’t have time to do anything with it until the weekend unfortunately but very interested to see what it can do. It’s another enabler of the movement to add inference to low power mobile devices without relying on a cloud server.

Second version of HoloLens HPU – separating mixed reality from the cloud

Some information from Microsoft here about the next generation of HoloLens. I am a great fan of only using the cloud to enhance functionality when there’s no other choice. This is especially relevant to MR devices where internet connectivity might be dodgy at best or entirely non-existent depending on the location. Putting some AI inference capability right on the device means that it can be far more capable in stand-alone mode.

There seems to be the start of a movement to towards putting serious but low power-consuming AI capability in wearable devices. The Movidius VPU is a good example of this kind of technology and probably every CPU manufacturer is on a path to include inference engines in future generations.

While the HoloLens could certainly use updating in many areas (WiFi capability, adding cellular communications, more general purpose processing power, supporting real-time occlusion), adding an inference engine is certainly extremely interesting.

ZenFone AR – Tango and Daydream together

The ZenFone AR is a potentially very interesting device, combining both Tango for spatial mapping and Daydream capability for VR headset use all in one package. This is a step up from the older Phab 2 Pro Tango phone in that it can also be used with Daydream (and looks like a neater package). Adding Tango to Daydream means that it is possible to do inside-out spatial tracking in a completely untethered VR device. It should be a step up from ARKit in its current form which relies on just inertial and VSLAM tracking from what I understand. Still, the ability for ARKit to be used with existing devices is a massive advantage

Maybe in the end the XR market will divide up into those applications that don’t need tight spatial locking (where standard devices can be used) and those that do require tight spatial locking (demanding some form of inside-out tracking).

Using the HoloLens to aid back surgery

Fascinating video of a HoloLens being used in a real back surgery – presumably the video was mostly shot using Spectatorview or something similar. I have seen other systems where mocap type technology is used to get more precision in the pose of the HoloLens but this system doesn’t seem to do that. Not that I am a surgeon but I doubt that the HoloLens can replace the usual fluoroscope since that gives real time feedback on the location of things like needles with respect to the body (yes, I have been on the literal sharp end of this!). However, if the spatial stability of the hologram is good enough, I am sure that it greatly helps with visualization.

As one of the many people with dodgy backs, I am always interested in anything that can improve outcomes and minimize risk and side-effects. If the HoloLens can do that – brilliant!

Raspberry Pi based outdoor camera

It was time to replace one of my old outdoor Panasonic network cameras. They get damaged by the sun – the plastic bubble over the lens gets really nasty and the video ends up looking like there is perma-fog outside.

I have a few of these wide angle webcams around for other projects and it seemed like they might be ideal for this purpose. Since they are not weather-proof I needed a suitable housing. This one from Monoprice was the smallest that I could find. It’s still ludicrously large for this but never mind. I decided just to put the webcam in the housing (attached with hot glue – my favorite engineering material) and pass the USB cable through to the inside of the building to connect to a Raspberry Pi. This eliminates the need to ever open the housing again and reduces the number of wires from two to one (power and ethernet versus USB).

The result is not bad at all considering the cost and it certainly looks like a serious piece of equipment!

The image quality in the corners isn’t spectacular but at least gives good coverage. The housing does intrude at the top left and right but that’s not a major problem.

Yes, there are all kinds of outdoor cameras that you can buy at very reasonable cost. However, I certainly don’t want anything that relies on or involves a cloud service – I want my video data to remain on site at all times. Plus, this is a fully open system so that I can do whatever I want with the data without having to battle proprietary SDKs (even if they are available). For example, I do custom motion detection and multiple resolution stream generation in the camera itself which fits nicely with all the other components of my system.