I’ve been thinking a lot about how modern applications handle heavy workloads without crashing. When building systems that send emails, process media, or crunch data, we can’t afford to block users while these tasks run. That’s what brought me to distributed task queues - they let us offload work to background processes. Today, I’ll show you how to build one using BullMQ, Redis, and TypeScript. Stick around - this could change how you design your next project.

First, why use a queue? Imagine 10,000 users requesting image processing simultaneously. Without a queue, your server would drown. With BullMQ and Redis, we can manage this elegantly. Redis acts as the backbone, storing jobs and coordinating workers. BullMQ provides the tools to define, process, and monitor these jobs. TypeScript ensures we catch errors early with type safety.

Setting up is straightforward. Create a new project and install dependencies:

npm install bullmq redis @types/node tsx
npm install -D typescript @types/redis nodemon

Our tsconfig.json ensures strict type checking. We organize code into logical directories: queues for job definitions, workers for processing logic, and jobs for shared types. Here’s how we establish the Redis connection:

// src/config/redis.ts
import { Redis } from 'ioredis';

export const redisConnection = new Redis({
  host: process.env.REDIS_HOST || 'localhost',
  port: parseInt(process.env.REDIS_PORT || '6379'),
  maxRetriesPerRequest: 3
});

redisConnection.on('error', (err) => {
  console.error('Redis error:', err);
});

Now, what makes a robust queue system? Let’s define our job types first. TypeScript interfaces prevent mismatched data:

// src/jobs/types.ts
export interface EmailJobData {
  to: string;
  subject: string;
  body: string;
}

export interface ImageJobData {
  url: string;
  width: number;
  height: number;
}

Creating a queue becomes simple with BullMQ. Notice how we attach event listeners for monitoring:

// src/queues/email-queue.ts
import { Queue } from 'bullmq';
import { redisConnection } from '../config/redis';
import { EmailJobData } from '../jobs/types';

export const emailQueue = new Queue<EmailJobData>('email', {
  connection: redisConnection,
  defaultJobOptions: {
    attempts: 3,
    backoff: { type: 'exponential', delay: 2000 }
  }
});

emailQueue.on('completed', job => {
  console.log(`Email sent to ${job.data.to}`);
});

Workers process jobs independently. Here’s an email worker with error handling:

// src/workers/email-worker.ts
import { Worker } from 'bullmq';
import { redisConnection } from '../config/redis';
import { EmailJobData } from '../jobs/types';

const worker = new Worker<EmailJobData>('email', async job => {
  const { to, subject, body } = job.data;
  // Simulate email sending
  if (!to.includes('@')) throw new Error('Invalid email');
  console.log(`Sending email to ${to}`);
}, { connection: redisConnection });

worker.on('failed', (job, err) => {
  console.error(`Email to ${job?.data.to} failed:`, err);
});

What happens when jobs fail? BullMQ’s retry system saves us. The exponential backoff means failed jobs wait longer before retrying - perfect for temporary outages. For permanent failures, we log them for investigation.

Scaling is where this shines. Spin up multiple workers across servers:

# Worker instance 1
tsx src/workers/email-worker.ts

# Worker instance 2
tsx src/workers/email-worker.ts

Redis coordinates everything. Workers compete for jobs, ensuring parallel processing. BullMQ’s dashboard gives real-time insights:

// src/monitoring/dashboard.ts
import { createBullBoard } from '@bull-board/api';
import { BullMQAdapter } from '@bull-board/api/bullMQAdapter';
import { ExpressAdapter } from '@bull-board/express';
import { emailQueue } from '../queues/email-queue';

const serverAdapter = new ExpressAdapter();

createBullBoard({
  queues: [new BullMQAdapter(emailQueue)],
  serverAdapter
});

serverAdapter.setBasePath('/admin/queues');
export default serverAdapter.getRouter();

Advanced features like scheduling come built-in. Delay critical emails during off-peak hours:

await emailQueue.add('low-priority-email', {
  to: '[email protected]',
  subject: 'Weekly digest',
  body: '...'
}, { delay: 86_400_000 }); // 24 hours later

Prioritization ensures urgent tasks jump ahead. In healthcare apps, patient alerts might override marketing emails:

await emailQueue.add('high-priority', {
  to: '[email protected]',
  subject: 'URGENT: Patient update'
}, { priority: 1 }); // Highest priority

What about rate limits? BullMQ handles that too. Limit third-party API calls to avoid bans:

const apiQueue = new Queue('external-api', {
  limiter: { max: 100, duration: 60_000 } // 100/minute
});

In production, separate Redis instances for queues and cache. Use connection pooling and monitor memory usage. Always set TTLs on jobs to prevent accumulation. Test failure scenarios - what happens when Redis disconnects? How do workers recover?

I’ve seen teams transform brittle systems into resilient architectures using these patterns. The separation of concerns lets frontend remain responsive while backend workers crunch data. Have you considered how queues could simplify your current project?

Implementing this took our application from handling hundreds to millions of tasks daily. The type safety caught numerous bugs during development, and Redis’s performance surprised even our skeptical DevOps team. Give it a try - start with a simple queue for your next batch job.

Found this useful? Share it with your team and leave a comment about your queue experiences! What challenges have you faced with background jobs? Let’s discuss below.