When the Swadeshotsav project landed on my desk, the brief sounded simple: build a certificate generation portal for a government event. What the brief didn't mention was the scale — 100,000 certificates needed to be generated, personalised, and emailed within a 48-hour window.
The first decision was to decouple generation from the request cycle. A naive implementation would generate the PDF synchronously on each HTTP request and immediately email it. At 100K requests, that collapses within minutes. Instead, I reached for BullMQ, a Redis-backed job queue for Node.js. Every certificate request is enqueued immediately — the user gets an acknowledgement in under 100ms — while a pool of worker processes handles the actual generation.
For PDF generation I chose Puppeteer running inside a headless Chromium instance. Each worker renders an HTML certificate template with the user's details, captures it as a high-resolution PDF, and uploads it to an S3-compatible bucket. The download link is then delivered via AWS SES with a 99.2% delivery rate across all emails sent.
Real-time feedback was equally important. Nobody wants to submit a form and stare at a spinner not knowing if anything is happening. I integrated Socket.io WebSocket channels so users receive live progress updates: "In queue → Generating → Sent". This alone cut support queries by more than half.
The biggest lesson? Infrastructure is a product feature. The queuing, retry logic, dead-letter handling, and monitoring we built were just as important as the certificate design itself. If you're building for government scale, plan for failure at every layer before you write the first business-logic line.