Falcon Heavy and Starman

Elon Musk is having a lot of fun with this.


DroNet – flying a drone using data from cars and bikes

Fascinating video about a system that teaches a drone to fly around urban environments using data from cars and bikes as training data. There’s a paper here and code here. It’s a great example of leveraging CNNs in embedded environments. I believe that moving AI and ML to the edge and ultimately into devices such as IoT sensors is going to be very important. Having dumb sensor networks and edge devices just means that an enormous amount of worthless data has to be transferred into the cloud for processing. Instead, if the edge devices can perform extensive semantic mining of the raw data, only highly salient information needs to be communicated back to the core, massively reducing bandwidth requirements and also allowing low latency decision making at the edge.

Take as a trivial example a system of cameras that read vehicle license plates. One solution would be to send the raw video back to somewhere for license number extraction. Alternately, if the cameras themselves could extract the data, then only the recognized numbers and letters need to be transferred, along with possibly an image of the plate. That’s a massive bandwidth saving over sending constant compressed video. Even more interesting would be edge systems that can perform unsupervised learning to optimize performance, all moving towards eliminating noise and recognizing what’s important without extensive human oversight.

The end of the taxiway for the 747 (in the US at least)

Nice photos and story here about the final flight by a US airline of a 747. This brought back memories because, during the 90s, I spent a lot of time on Virgin Atlantic 747s between LHR and JFK (and occasionally BOS). I remember some of the old Virgin aircraft names – Spirit of Sir Freddie, Ruby Tuesday (above) and Lady Penelope for example. One of the things I would do to alleviate the boredom was to try to get off the aircraft first. If you were sitting in the correct seat on the upper deck and managed to get down the stairs before anyone else, there was always a good chance!

The best time was when I managed to get in the cockpit jump seat for a Virgin Atlantic 747 landing at SFO (yes, this was most certainly pre 9/11). It was great to see the crew handle the aircraft and air traffic control and just confirmed something that I already knew – that this kind of stuff was best left to professionals (I was a terrible pilot!).

Virgin Atlantic gradually replaced the 747s with A340s which were just not the same at all but, by then, I had mostly stopped flying across the Atlantic on a regular basis.

The Bitcoin Lightning Network

Just started finding out about the Lightning system that is designed to avoid some of the key inefficiencies of Bitcoin. I’ve always thought the excitement about Bitcoin to be a bit strange given its seven transactions per second throughput limit, absurd mining energy requirements, the need to effectively bribe miners with a fee in order to get a transaction included in the blockchain and the ever increasing (and already quite large) blockchain itself.

As described here, Lightning helps solve the scaling problem by implementing a network of micropayment channels that bypass the blockchain. Something that I have been interested in is how to use blockchain technology in large IoT networks that can generate large amounts of data in real time. The benefit would be an immutable (append-only) record of everything that happened that could be relied upon for accuracy and not amenable to after the fact modification. This record could be of evidential quality. Whether Lightning (or similar technology) can help with this is the question.

Years ago I was working on a tamper-proof video system for surveillance cameras using steganography – basically every frame included embedded overlapping error correcting codes that could survive compression and indicate on a grid where a frame had been modified by checking the syndromes. By using the frame timestamp as the data in the code, encrypted with a private key embedded in the source camera, completeness of the video record could also be determined. What’s interesting is whether blockchain technology can be similarly leveraged to solve this and related problems.

Google’s WorldSense

Lenovo just announced the Mirage Solo VR headset with Google’s WorldSense inside-out tracking capability. The result is an untethered VR headset which presumably has spatial mapping capabilities, allowing spatial maps to be saved and shared. If so, this would be a massive advance over ARKit and ARCore based AR which makes persistence and collaboration all but impossible (the post here goes into a lot of detail about the various issues related to persistence and collaboration with current technology). The lack of a tether also gives it an edge over Microsoft’s (so-called) Mixed Reality headsets.

Google’s previous Tango system (that’s a Lenovo Phab 2 Pro running it above) did have much more interesting capabilities than ARCore but has fallen by the wayside. In particular, Tango had an area learning capability that is missing from ARCore. I am very much hoping that something like this will exist in WorldSense so that virtual objects can be placed persistently in spaces and that spatial maps can be shared so that multiple headsets see exactly the same virtual objects in exactly the same place in the real space. Of course this isn’t all that helpful when used with a VR headset – but maybe someone will manage a pass-through or see-through mixed reality headset using WorldSense that will enable persistent spatial augmentation using a headset with hopefully reasonable cost for ubiquitous use. If it was also able to perform real time occlusion (where virtual objects can get occluded by real objects), that would be even better!

An interesting complement to this is the Lenovo Mirage stereo camera. This is capable of taking 180 degree videos and stills suitable for use with stereoscopic 3D displays, such as the Mirage headset. Suddenly occurred to me that this might be a way of hacking a  pass-through AR capability for Mirage before someone does it for real :-). This is kind of what Stereolabs are doing for existing VR headsets with their ZED mini except that this is a tethered solution. The nice thing would be to do this in an untethered way.

The disaggregated smartphone and the road to ubiquitous AR

Nearly five years ago I posted this entry on a blog I was running at the time:

Breaking Apart The Smartphone

…it’s not difficult to imagine a time when the smartphone is split into three pieces – the processor and cellular interface, the display device and the input device. One might imagine that the current smartphone suppliers will end up producing the heart of the system – the display-less main CPU, the cellular interface, Bluetooth, WiFi, NFC etc. However, it will open up new opportunities for suppliers of display and input devices. It’s pretty safe to assume that Google Glass won’t be the only show in town and users will be able to pick and choose between different display devices – and possibly different display devices for different applications. Likewise input devices will vary depending on the environment and style of use.

Maybe we’ll look back on the current generation of smartphones as being inhibited by their touchscreens rather than enabled by them…

I was only thinking vaguely of AR at the time but now it seems even more relevant. A key enabling technology is a low power wireless connection between the processing core and the display. With this implemented in the smartphone core, things change tremendously.

For example, I have a smartphone that is pocketable in size, an iPad for things where a bigger screen is required, a smartwatch for when I am too lazy to get the phone out of my pocket etc. I only have a SIM for the smartphone because even having one cellular contract is annoying, let alone one per device. How would having a wireless display capability change this?

For a start, I would only have one smartphone core for everything. This would have the one and only SIM card. When I want a tablet type presentation, I could use a suitable size wireless display. This would be light, cheap and somewhat expendable, unlike the smartphone itself. However, in this concept, the smartphone can always be kept somewhere safe – expensive screen replacements would be a thing of the past, especially if the smartphone core doesn’t even have a screen. I like to ride a bike around and it would be nice to have easy access to the smartphone while doing so and in all weathers. You can get bike bags that you can put a smartphone in but they are pretty lame and actually quite annoying in general. Instead, I could have a cheap waterproof display mounted on the bike without any need for waterproof bags.

Since the display is remote, why not have a TV sized screen that connects in the same way? Everything streamable could be accessed by the smartphone and displayed on the TV without a need for any other random boxes.

Finally, AR. Right now AR headsets kind of suck in one way or another. I am intrigued by the idea that, one day, people will wear AR type devices most of the time and what that means for, well, everything. The only way this is going to happen in the near future is if the headset itself is kept as small and light as possible and just acts as a display and a set of sensors (inside out tracking, IMU, depth etc). Just like the other displays, it connects to a smartphone core via a wireless link (I believe that any sort of tethered AR headset is unacceptable in general). The smartphone core does all of the clever stuff including rendering and then the output of the GPU is sent up to the headset for display. An AR headset like this could be relatively cheap, waterproof, dustproof and potentially worn all day.

What does a world with ubiquitous AR actually look like? Who knows? But if people start to assume that everyone has AR headsets then “real world” augmentation (decoration, signage etc) will give way to much more flexible and powerful virtual augmentations – anyone not using an AR headset might see a very bland and uninformative world indeed. On the other hand, people using AR headsets might well see some sort of utopian version of reality that has been finely tuned to their tastes. It’s definitely Black Mirror-ish but not all technology has to have horrendous outcomes.

Rigetti Computing’s Forest and pyQuil

Rigetti Computing seems to have a very nice system for testing out quantum algorithms. I haven’t tried it out myself yet but it consists of Forest, a cloud-based API with access to up to 26 simulated qubits. There’s also access to 19 real qubits apparently although that’s on request. To go with it, pyQuil is a Python environment that provides a nice way of driving the cloud API and is where the quantum algorithms are defined.

One nice advantage over Microsoft’s Q# is that (as I understand it) qubits in Q# are opaque which means that the state of qubit can only be accessed via measurement whereas in pyQuil the register state can be accessed (using the wavefunction keyword). While this opacity in Q# reflects the reality of a physical quantum processor, a simulator should permit access to the internal state in order to help with understanding what’s going on.

However, both certainly have a role to play in main-streaming quantum computing and helping us all get ready for a time when real quantum processors become as available as GPUs (maybe).