At work, the adoption of python 3 was finally moving at warp speed - the end of its support might have had something to do with it. As a result, there was a lot of code to migrate over. One of the things I did during this migration was add type hinting as well as linter checks to the codebase. And I... was not... ready for that! When I first read about type hinting I thought it would be a neat thing to help people new to the language and existing users navigate through code. ... Read more

In this post I an encoding behavior in python 2 and their differences in python 3 If you’re still using python 2, which many folks are, you may run into encoding issues when processing data. Let’s say we have a file called translations.txt that contains translations between English and Mandarin (from the Oxford dictionary): "The book has 500 pages of text.","这本书正文有500页。" "I'll send you a text as soon as I have any news. ... Read more

In my daily work, I often need to write data pipelines to produce metrics, create datasets for machine learning models, or just clean up logs. I had usually done testing the traditional way, verifying my code does what I expect by checking for normal and edge cases I could think of. But over the past couple of months, I started using property based testing, and I feel like my code quality has improved dramatically. ... Read more

The first programming language I learned was Pascal. I was in my math class in secondary school, and my teacher told us about a programming competition that was held every year between schools. At that point, I everything I knew about programming was a pure construct of my imagination. I knew that you were suppose to type something into a machine and it would do things. But I was into computer hardware at the time, so I figured why not. ... Read more

I have been thinking about excellence for the past couple of months, since I’ve starting working full time again. Excellence in writing code, designing systems, and formulating problems. Last week I had the opportunity to attend a talk titled Relatively Painless Technical Excellence from J.B. Reinsberger. It was one of the best talks I’ve heard on agile and software engineering. I have to start by saying I didn’t have much expectations about it, and I was positively surprised by the content. ... Read more

A while back, while working on the infrastructure of a ecommerce recommendations service provider, we ran into problems handling traffic from our clients in real time. As a simple solution, we decided to send data to logs through AWS S3, by appending HTTP URL parameters to GET requests to a tiny image file; something that should only be done when you’re not dealing with sensitive data. However, we had a minor issue: we had complex nested JSON objects, with objects inside fields; however, JavaScript and JQuery standard libraries only supported serialization of flat JSON objects. ... Read more

The art of writing was invented, I suppose, so that we could communicate with the future, i.e. record the past and present, and in the process create history. Software programs, on the other hand, are written for the purpose of defining the future. One that is meant to be interpreted by machines. The tapping of a keypad turns a blank page into a blueprint. It starts with one file, and can quickly grow larger. ... Read more