Security · API scanning
API Scan Service
API security scanning pulled out of a Django monolith and onto its own machines: ZAP, Nuclei and Schemathesis in parallel, with the findings posted back.
Status
Live
Role
Senior full-stack engineer
Timeline
2026
Why it had to move out
A scan needs Docker, takes every CPU core it can reach and peaks at about 1.6 GB of memory. Run beside the web application, a handful of them put the whole platform at risk.
So the scanners run on their own server and the platform sends them a URL. The service deliberately holds nothing: no database, no object storage, no merchant data. Compromising it gets an attacker a queue of URLs.
Backpressure instead of collapse
The first version queued scans as background tasks and fell over at around forty of them, with the thread pool exhausted and no signal that anything was wrong.
A bounded worker queue replaced it, and a full queue now answers 429 rather than accepting work it cannot do. Refusing a request is a better failure than accepting it and losing it.
Callbacks that cannot duplicate
Results are delivered by callback, and PDF rendering on the receiving end is slow enough that the caller sometimes retried. Every retry produced another report.
Delivery now claims the result with an atomic compare-and-clear, so the second callback finds nothing to send. Idempotency here is not a nicety: a duplicate compliance report is a question somebody has to answer.
Results
- Scanner image cut from 411MB to 217MB, and the results payload by roughly 170×.
- Thread-pool exhaustion at around 40 queued scans replaced with a bounded queue and HTTP 429 backpressure.
- Holds no database, no object storage and no merchant data: it receives a URL and a callback address.
Got something like this to build?
Start a project