JsonURI: json serialization and deserialization for logging

May 28, 2016 by guidj

A while back, while working on the infrastructure of a ecommerce recommendations service provider, we ran into problems handling traffic from our clients in real time. As a simple solution, we decided to send data to logs through AWS S3, by appending HTTP URL parameters to GET requests to a tiny image file; something that should only be done when you’re not dealing with sensitive data. However, we had a minor issue: we had complex nested JSON objects, with objects inside fields; however, JavaScript and JQuery standard libraries only supported serialization of flat JSON objects. So, to circumvent that, I wrote a procedure to serialize complex JSON objects into plain HTTP URL parameters, and deserialize the result back to JSON. I wrote packages in Python and in JavaScript to have JavaScript client libraries sending data, and a Python app processing the logs.

Now, after being battle tested (for a long period), the first stable version of JsonURI has been released, and you can read about it on its github page