Your AI app times out at 30 seconds.
The background worker, explained for founders.

I'm a senior backend tech lead and I run Belmo, a small European PaaS. This article mentions Belmo once near the end, but the concept it explains is platform-independent and worth understanding no matter where you deploy.

A founder messaged me last month with a bug that had haunted her for two weeks. Her app generated a PDF report for each new user, and it worked perfectly on her laptop. In production, about one in five signups got a spinning wheel that turned into a blank error page. The PDFs were fine. The signups were fine. Something in between was silently breaking, and her AI coding assistant kept suggesting fixes that did nothing.

The problem was not the code. She was doing slow work in the wrong place. Nobody had told her that production web servers have a hard limit on how long they will wait, and that the fix is a concept most tutorials skip: the background worker.

Why your web server has a stopwatch running

Here is the thing nobody explains when you ship your first app. A web server is built to answer requests quickly and then move on to the next one. When a browser asks for a page, the server is expected to respond in a fraction of a second. Everything in the stack between your user and your code is built around that assumption.

Sitting in front of your app is almost always a piece of infrastructure called a reverse proxy or load balancer. You did not configure it. Your platform did, automatically. And it has a stopwatch. If your code takes longer than roughly 30 seconds to respond, the proxy gives up, kills the connection, and hands the user a 502 or 504 error. Your code might still be running happily in the background, finishing the PDF, but the user already saw a broken page. The connection they were holding is gone.

This is why "it works on my laptop" is so common here. On your laptop there is no proxy and no stopwatch. You wait 45 seconds for the PDF, you see it appear, you assume everything is fine. In production the timeout fires at 30 and your user never finds out the work succeeded.

The list of things that quietly blow past 30 seconds is longer than you would expect. An LLM call summarizing a long document. A third-party API having a slow afternoon. Sending a batch of welcome emails one at a time. Resizing an uploaded video. A report that runs ten database queries. None of these feel slow when you write them. They feel slow at 9pm on a Tuesday when a real user hits them with real data.

The fix is to stop making the user wait

The mental model is this. When a request needs slow work, your web server should not do that work itself. It should write down "this user needs a PDF" somewhere durable, immediately tell the browser "got it, working on it," and let a separate process do the job without anyone watching a spinner.

That separate process is the background worker. Same code, same dependencies, same database, running as a second program whose only job is to chew through slow tasks. Your web server stays fast. The worker takes as long as it needs, because no proxy is timing it.

The "somewhere durable" in the middle is a queue. In the simplest version, the queue is just a table in your database: a row that says "job: generate_pdf, user_id: 8842, status: pending." The web server inserts the row. The worker reads rows where status is pending, does the work, and flips the status to done. That is the entire idea. People wrap it in libraries and add Redis and talk about brokers and consumers, but underneath the vocabulary it is a to-do list that one program writes to and another reads from.

Here is the before, the version that times out:

@app.post("/signup")
def signup(email: str):
    user = create_user(email)
    pdf = generate_welcome_pdf(user)   # 40 seconds, sometimes more
    send_email(user, attachment=pdf)   # another 5 seconds
    return {"status": "ok"}            # user has long since seen a 502

And the after, where the request returns instantly and the slow part happens elsewhere:

@app.post("/signup")
def signup(email: str):
    user = create_user(email)
    enqueue("generate_welcome_pdf", user_id=user.id)  # writes one row, returns in 5ms
    return {"status": "ok", "message": "Check your inbox shortly."}

The user gets an answer in milliseconds. The PDF arrives by email a minute later. Nobody watches a spinner, and the 30-second stopwatch never gets a chance to fire.

What the worker side actually looks like

The worker is not exotic. If your queue is a database table, a worker is a loop that wakes up, looks for work, does it, and goes back to sleep. Here is a deliberately bare version so you can see there is no magic:

import time

def run_worker():
    while True:
        job = claim_next_pending_job()   # SELECT ... FOR UPDATE SKIP LOCKED
        if job is None:
            time.sleep(2)                # nothing to do, wait and retry
            continue
        try:
            handle(job)                  # the slow work lives here
            mark_done(job)
        except Exception as e:
            mark_failed(job, error=str(e))
            # leave it for retry, or alert, or both

In a real app you would reach for a library that handles the fiddly parts. In Python that is Celery, RQ, or Dramatiq. In Node it is BullMQ or the built-in worker support in frameworks like NestJS. In Ruby it is Sidekiq or the newer Solid Queue that ships with Rails 8. They all do the same three things: accept a job, run it outside the request, and retry it if it fails. The library is a convenience. The concept is what matters, and the concept is "fast process writes jobs, slow process runs them."

One detail that trips people up: the worker is a separate process, so you are now running two things instead of one. Your web server (gunicorn app:app) and your worker (celery -A app worker). They share the same code and database, but they start separately and scale separately. If your queue backs up, you add more workers without touching the web side. If traffic spikes, you add web capacity without touching the workers. That separation is the whole point.

The trap: jobs that run twice, or never

There are two failure modes worth knowing before you ship this, because they are the ones that generate confusing support tickets.

The first is the job that runs twice. Say your worker picks up "send invoice email," sends it, then crashes before it can mark the job done. When it restarts, the job still looks pending, so it sends the invoice again. Now your customer has two invoices. The defense is to make jobs idempotent, a fancy word for "safe to run more than once." Before sending the invoice, check whether it was already sent. Before charging a card, check whether the charge already went through using an idempotency key, which Stripe and most payment APIs support for exactly this reason. Assume every job can run twice and design so the second run is harmless.

The second is the job that never runs. The worker crashes, or you forgot to start it, or it has been silently dead for three days while the web server keeps accepting requests and writing jobs to a queue that nothing is draining. This is the dangerous one, because everything looks fine from the outside. Users sign up, get a cheerful "check your inbox," and then no email ever comes. You find out when someone complains, a week later.

I have seen this at scale. On a client project with a large async pipeline, a worker deploy went out with a bad environment variable, the workers crash-looped on startup, and the web tier kept queuing jobs the whole time. The queue grew to forty thousand pending rows before anyone looked. The fix took five minutes. Noticing took fourteen hours. The lesson stuck: a background worker you do not monitor is a worker you do not have.

Monitor the queue, not just the worker

The metric that tells you the truth about a background system is queue depth: how many jobs are pending, and how long the oldest one has waited. A healthy queue drains as fast as it fills, so depth stays near zero and the oldest job is seconds old. A sick queue grows without bound and the oldest job ages into minutes, then hours.

You do not need a dashboard for this on day one. You need an alert: a tiny scheduled check, running every few minutes, that counts pending jobs older than five minutes and pings you if the number is not zero:

SELECT count(*) AS stuck
FROM jobs
WHERE status = 'pending'
  AND created_at < now() - interval '5 minutes';

If stuck is ever above zero, something is wrong: either your workers are down, or they are too slow for the incoming volume, or a specific job type is jamming the line. All three are things you want to hear about from a notification, not from a customer.

What I built, and why

I kept solving this same problem by hand for clients and for my own projects. Stand up a worker process, wire it to the queue, then build a second little system just to watch whether the first one is still breathing, because the failure mode is silence. After doing it enough times I folded it into Belmo, the platform I run. On the Pro tier you can add a background worker to a service without configuring proxies or process managers, and the log monitoring reads the worker's logs for the patterns that mean it is in trouble: a job type that starts failing, a process that crash-loops on deploy. When it sees one, it sends a Telegram alert instead of letting the queue quietly fill.

A clear boundary, though. Belmo gives you the worker process and the monitoring around it. It does not pick your queue library or write your idempotency logic, and it should not, because those choices depend on your app. The platform makes the worker easy to run and hard to silently lose. The correctness of the jobs is still yours. If you would rather run all of this on a VPS you manage by hand, the concepts here transfer one for one. The only thing that changes is who gets paged when the worker dies.

What to do tonight, regardless of which platform you use

1. Find the slow paths. Any request that calls an LLM, sends email, generates a file, hits a third-party API, or loops over a lot of rows. Those are your timeout candidates. Write them down.
2. Move the slowest one off the request. Pick the worst offender, the one most likely to pass 30 seconds, and convert it to enqueue-and-return. You do not have to migrate everything at once. One job teaches you the whole pattern.
3. Run the worker as its own process. Not a thread inside your web server, not a task that dies when the request ends. A separate program that starts on its own and shares your code and database.
4. Make that first job idempotent. Add a check at the top: has this already been done? If yes, return early. Assume it will run twice and make the second run a no-op.
5. Add the queue-depth alert. The five-minute SQL check above, on a schedule, pointed at whatever channel you actually read. This is the highest-leverage item here, because it converts a silent failure into a loud one.
6. Kill your worker on purpose and watch what happens. Confirm the alert fires and the web side keeps serving. A monitor you have never seen trigger is a monitor you do not trust yet.
7. Write down which jobs exist and what each does. One short file. When something jams at 2am, the difference between a five-minute fix and a two-hour one is whether you remember what process_batch_v2 does.

The closing

Background workers sit in a blind spot. They are too foundational for beginner tutorials, which stop at "deploy your app," and too obvious for senior engineers, who forget they ever had to learn it. So a whole generation of founders shipping AI apps with coding assistants hit the 30-second wall and have no name for what they are looking at. The wall is real, it is not your code, and the fix is a pattern you can learn in an evening.

Where I think there is real disagreement is how far to take it on day one. Some people will tell you to add a proper queue and worker before your first user, because retrofitting it under load is miserable. Others, me included on small projects, will tell you a single web process is fine until something actually times out, and that adding a worker prematurely is just two things to monitor instead of one. Where do you land? And has a silent dead worker ever cost you something you only found out about days later? I read every comment.

Related reads on this blog: How I built a Discord 'ship-tracker' bot in a weekend (and the 3-process architecture that keeps it alive 24/7). · Your crontab is silently failing. The 5 silent killers of VPS-based cron jobs (and the modern setup that fixes them in an afternoon). · Telegram alerts for any production app: a 5-minute setup (no SaaS, no signup, just curl).