Optimizing Firestarter: From Node.js to Bun

In a quest to enhance the performance of our latency-sensitive service, Firestarter, we transitioned from Node.js to Bun, achieving a remarkable 5x increase in throughput. Initially, Firestarter, which manages long-poll HTTP connections for task execution, was plagued by high CPU usage. Profiling revealed that 31% of CPU time was consumed by a SQLite query engine, prompting us to replace it with a more efficient composite-key Map structure. This change alone doubled our throughput and improved latency metrics significantly.

Next, we tackled the overhead in Node.js's HTTP stack by switching to Bun, which offered a native routing API. This transition further doubled our throughput and halved latency. Profiling Bun with its markdown-friendly CPU profiling tool revealed hotspots in Zod schema validation, header conversion, and debug logging. By optimizing these areas, we reduced CPU usage by 40%.

We then compiled our Bun application into a single binary, reducing the container image size from 180MB to 68MB and increasing throughput by 14%. However, we encountered a memory leak due to Bun's different HTTP model, where unresolved promises in client disconnect scenarios led to memory retention. By resolving these promises with a 499 status, we eliminated the leak, stabilizing memory usage.

Our benchmarks showed a 5x increase in throughput and a dramatic reduction in latency, with the compiled Bun version outperforming Node.js in every metric. Despite a slight CPU increase after fixing the leak, the overall performance and stability gains in production were substantial. We learned the importance of profiling, understanding Bun's unique HTTP model, and the benefits of compiling binaries for performance gains. Our debugging efforts also highlighted the need for custom metrics and careful handling of AbortSignal events in Bun.