Using JSON for streaming data flows is very convenient. It allows pipeline processing elements (PPEs) to insert extracted features that enhance the value of the data in a very natural manner – just adding new JSON fields to the message.
JSON does have one big limitation though: it has no way to transport binary data natively. I have been using base64 encoding for high rate sensor data (such as video and audio) but it always seemed ugly. So all of rtndf, with support from Manifold, now supports a joint JSON + native binary transport.
There’s no great rocket-science here, the payloads just look like this:
<Serialized JSON message length (4 bytes)> <Binary data length (4 bytes)> <Serialized JSON message> <Binary data>
Simple but effective in eliminating wasteful base64 encodes/decodes. I know that there are some binary JSON-like libraries but this seems like a better solution for this specific application where typically there is only one big binary field that needs to be transported.
Another reason for being concerned about base64 encoding is that I want to be able to move uncompressed video frames around using shared memory links when the pipeline section is running within one machine. Right now, video frames move as JPEGs internally to keep data rates manageable but this uncompressed transport concept could eliminate a lot of wasteful JPEG encode/decodes too.