Case Study · Performance

Cutting API Latency 75%: 800ms → <200ms

A diagnostic walkthrough of how I traced and eliminated bottlenecks across the full request path.

Role: SDE-III · Stack: Node.js · MySQL · Redis · RabbitMQ · Gearman · APM tooling

Latency distribution before and after — the long tail collapses, p95 drops below 200ms, and p99 follows.

The Problem

Core marketplace APIs were averaging 800ms response times, with p95s much worse. This was hurting checkout conversion and Google's Core Web Vitals scores — both at the same time. Every page that consumed these endpoints felt sluggish, and the downstream LCP cost was showing up in Search Console.

Throwing more hardware at it would buy us a few months at best. We needed to find the actual bottlenecks — across the database, the cache, and the request pipeline — and fix them at the root.

The Solution

A three-pronged audit, in this order:

Query-level. Profiled every slow endpoint, killed N+1s, added composite indexes, and rewrote three offending JOINs. Most of the easy wins live here, but only if you have APM data telling you which queries actually run hot.
Caching. Redesigned the cache strategy around Redis with proper key hierarchies and TTLs. Pushed read-heavy endpoints to a write-through pattern so cache invalidation became a property of the write path, not a cron job.
API layer. Refactored the request pipeline, removed synchronous logging, and moved non-critical work — analytics events, audit writes, downstream notifications — to RabbitMQ and Gearman async queues.

Each layer compounded the others. The query fixes shrank cache miss penalties. Better caching reduced load on MySQL. Async offload kept the request thread free for the work that actually mattered to the response.

Tech Stack

Node.js
MySQL
Redis
RabbitMQ
Gearman
APM tooling

My Specific Contribution

Profiling and diagnosis. Built the latency baseline; traced the worst-offender endpoints through APM.
Query and index work. Rewrote three slow JOINs; added composite indexes; eliminated N+1 patterns.
Cache redesign. Key hierarchy, TTL policy, write-through pattern for read-heavy endpoints.
Async offload. Moved non-critical work onto RabbitMQ / Gearman with idempotent consumers and DLQs.
Verification. Before/after p50/p95/p99 reports; load testing to validate the throughput claim.

Impact

75% ↓

Average API response: 800ms → <200ms across core endpoints.

+40%

Platform throughput uplift on the same hardware — supporting 3× concurrent users.

50 → 90+

PageSpeed desktop, knock-on from faster APIs (40 → 80+ on mobile).

Visuals

Flame graphs of the same endpoint before and after — synchronous logging and JOIN time dominate the "before" frame; "after" is mostly network and serialisation.

Architecture: API request returns immediately after the critical write; non-critical work enqueues to RabbitMQ / Gearman for downstream consumers.

Lessons

The trap with latency work is treating it as a single problem. It isn't — it's three problems (database, cache, app) wearing a trench coat. You won't beat it with one big rewrite; you beat it by looking at p95 and p99 weekly, fixing the worst offender, and accepting that the next week's worst offender will be in a different layer.