Observability's Missing Pillar (draft)

Interactive drill down is an underappreciated capability in understanding and debugging distributed systems.

From logs to insights

One of my projects when I was working at Parse was called (quite creatively) logtailer. Logtailer was a software system that took the relatively unstructured logs that various parts of Parse’s infrastructure were generating and made them useful to understand in aggregate. Putting aggregate event data in front of the developers and operators at Parse changed how we understood, improved, and debugged our systems.

Ultimately the heavy lifting was conducted by Scuba, a database and visualization tool. It’s one of the tools that former Facebook employees say they miss the most.

Logging, Metric, and Tracing