As a data scientist, I frequently find myself in need of domain-specific tools optimized for constrained research environments, tools which all-too-often do not yet exist. Therefore, I started to build (and share) my own tools. Below is an overview of some of the projects I have made public along with the philosophy that helped develop them.
I build tools under three governing ideas:
- pragmatic minimalism (“Do everything necessary to do one thing well”),
- literate programming (inspired by Knuth, but adapted to the AI era), and
- highly-structured collaboration with AI agents (see the Working Agreement).
- To minimize costs on rented machines, many tools needed to be pure client-side. A GET-only server can solve that problem without a large attack surface. Servette is a single-file pure-Python secure static site server.
- Some of my quantitative research required an embeddable database that could both ingest and compute quickly on ordered numeric data. TallyDB is an append-optimized SQL database with zero-copy compute access.
- Faster in-database compute led to embedding Lua, which required a serious numeric stack. MatLua is a NumPy-shaped array and linear algebra library for Lua.