Category Archives: Techniques

Using C++ REST SDK as a static library

cpprestsdk is a handy library for implementing REST interfaces in C++ code. On Windows it is conveniently available as a NuGet package. However, it is a dynamic link library (DLL). Normally that’s not a problem except for the fact that I am trying to combine it with WebRTC which builds as a static library and mixing the two causes strange crashes. I don’t know if it is possible to compile WebRTC as a DLL – the ninja build is very complicated :-(.

I found this very helpful post which provides life-saving instructions on how to build and use cpprestsdk as a static linked library. In fact the cpprestsdk Visual Studio solution now includes a project to build a static library so that part is mostly taken care of. It is still necessary to change the Runtime Library option to /MTd (debug) and /MT (release) in the casablanca140.static project however.

To use the library, the preprocessor symbol _NO_ASYNCRTIMP needs to be defined in the project as described in the post. You also have to add winhttp.lib, crypt32.lib, bcrypt.lib and zlibstatic.lib as additional linker dependencies. The library for the latter needs to copied somewhere that the project can find – it can be obtained from the packages in the cpprestsdk folder for zlib.

Another thing is that the static version of cpprestsdk needs to be built in Debug and Release modes if the new project is to run in both modes. With some things you can use Release for both – this doesn’t seem to work for cpprestsdk.

The result now is that I can use REST for signaling in my WebRTC project – not as easy as I expected!

Using a webcam with a HoloLens SpectatorView rig

Following on from my previous post regarding HoloLens SpectatorView, I had been wondering if it was possible to use a webcam instead of a DSLR. It changes the mounting concepts but, just for testing, it wasn’t hard to place a Logitech C920 webcam on top of the HoloLens and get it aligned enough physically so that the calibration data numbers looked reasonable.

An immediate problem was that the code was not setting the webcam’s frame size. A quick look at OpenCVFrameProvider.cpp showed the problem. The code was trying to set the frame width and height before opening the capture object which doesn’t work. This is the original:

The fix is put line 44 before line 41. Then it works fine. The preview window in the calibration code has red and blue swapped but the processed images are correct. Once it was calibrated, I could go on and run the Unity app and look at the composite output – now pretty decent 1080p video.

Using a webcam like the C920 is far from perfect however. The field of view was measured at 75 degrees by the calibration software which really isn’t enough to be useful. Another problem is the autofocus which causes frequent focus breathing. And then there’s the challenge of proper mounting but at least the C920 does have a 1/4 inch thread so there are possibilities.

A decent DSLR (this would be my choice as it can output clean 4k 4:2:2 video over HDMI apparently at a decent price and I have all the lenses 🙂 ) is going to give better results for sure. On the other hand, there may be many applications where a webcam is just fine and you can’t argue with the price.

Null pointer crashes

Interesting article here about eliminating null pointer crashes at the language level. I was intrigued by this since the other day I fixed an incorrect and unsafe cross-thread call situation that resulted in…a null pointer crash.

Putting together a HoloLens SpectatorView rig

Having sorted out a way of mounting the HoloLens on a camera using the alternate rig described here, it was then time to put the rest of the system together. First thing was the Blackmagic Intensity Pro 4K capture card. Hardware and software installation was very straightforward and nicely captured video from the camera’s HDMI port. Next up was the SpectatorView software itself.

The first step is to get the calibration software working – instructions are here. The OpenCV link doesn’t work – use this instead. I am actually using VS2017 but had no problems apart from being asked if it was ok to upgrade things.

To calibrate the rig, a pattern is needed. This is my attempt:

Seemed to work ok. The next thing is to build the Compositor. It needs to be built for x86 and x64 in Release mode (x86 for the HoloLens app, x64 for the Unity Editor app). I think I had to force it to build SpatialPerceptionHelper in x86 mode. Anyway, once all that’s done, the DLLs need to be copied into the sample app (I was using the sample which is the Shared Holograms tutorial code).

It took me a while to realize that the CopyDLL.cmd needs parameters for it to work with the sample app. Comments in the code tell you what to do but it is basically this:

CopyDLL "%~dp0\Samples\SharedHolograms\Assets\"

Time to fire up Unity using the sample app. Double click on the Sharing scene to kick it in. Then, click on the SpectatorViewManager object and look at the inspector. The Spectator View IP address needs to be set to the IP address of the HoloLens in the SpectatorView rig. Took me a while to work that out :-(. The Sharing Service IP field needs to be set to the address of the machine running the sharing server. The sharing server can be kicked off from Unity using the menu bar with HoloToolkit->SharingServer->Launch Sharing Service. The Sharing prefab also needs to be configured with the address of the sharing server. Once that’s done, it’s pretty much ready to deploy to the SpectatorView HoloLens and any others in the system.

The app needs to be run in the Unity Editor and then, using the menu bar again, kick off Spectator View->Compositor. This will shows a window with the combined live video from the camera and the virtual objects mixed in. This window also provides buttons to save video and snapshots.

Unfortunately, I only have one HoloLens to hand so I couldn’t really test the system. I did build a little test app that seemed to work ok as well as far as I could test it.

The biggest issue was my inadequate camera. I was hoping to find a way to use my Canon 6D for this, even though it does not fill the 1920 x 1080 output frame via its live HDMI port. I figured an OpenCV hack could deal with that. The bigger problem is that output is interlaced and causes horrible horizontal tearing in the composed video if anything in the scene is moving. I think it’s the end of the line for the 6D and SpectatorView.

Time for a proper 1080p/4K camera.

Obtaining high quality Unity textures from image files

The default settings for image import in Unity produce poor quality textures, especially when the image contains text that needs to look good in the scene. To fix this, click on the image file (not the generated material file) to get the Import Settings inspector. Deselect Generate Mip Map and change Filter mode to Point (no filter). The result should look like that in the screen capture below and the material will update automatically.

Connecting a webcam to a VirtualBox guest OS

I am running Ubuntu 16.04 in a VirtualBox VM on a Windows 10 machine and wanted to access the laptop’s webcam from a Python script running in the Ubuntu VM. The trick (as described here) is to enter this line on the host while the VM is running:

VboxManage controlvm "vmname" webcam attach .0

where vmname is the name of the VM to be modified.

There doesn’t seem to be any need to add a USB filter for the webcam – doing that doesn’t seem to help at all.

The only problem with this is that the change isn’t permanent – it has to be run each time the VM is started. Simplest way to deal with that is to start the VM from a batch file:

cd "c:\Program Files\Oracle\VirtualBox"
VboxManage startvm "vmname"
VboxManage controlvm "vmname" webcam attach .0

Incidentally, this attaches the default webcam. Individual ones can be specified using .1, .2 etc. Use:

VboxManage list webcams

to get a list of webcams and aliases.

Configuring Hyper-V networking for the HoloLens emulator

emulatorThe HoloLens emulator is a handy tool for quickly checking out code destined for the HoloLens itself. Apart from requiring Windows Pro for Hyper-V support, the only problem encountered was to give the HoloLens emulator access to some sort of network. I basically followed the instructions here but still encountered issues with getting it to connect to the sharing server, probably because I wasn’t thinking about what was happening.

The HoloLens emulator automatically creates the Emulator NAT Network Adaptor and Emulator Internal Adaptor when it starts up if they aren’t already there. When you manually create the new switch and add it to the Emulator NAT Network Adaptor, this couples things to the physical network adaptor. That adaptor will get an address on the external network (e.g. 192.168.1.23). However, the emulator is connected internally and (in my case at least), gets an address of 172.16.80.2 while the host PC gets an address of 172.16.80.1 on that interface.

So, if you run the sharing server on the host PC, the emulator needs to be pointed to the server at 172.16.80.1 rather than the 192.168.1.23 address that a real HoloLens would have to use. Once I realized that, things started working.

Another problem with my initial configuration I suspect was that the host PC originally had a static IP address. This seemed to cause strange effects and changing to dynamic IP let things work properly.