I’ve been thinking a lot about how modern applications handle complexity. Why do some systems bend under pressure while others adapt? This question led me to explore event-driven microservices. Today, I’ll show you how to build a resilient architecture using Node.js, RabbitMQ, and MongoDB. Stick with me—you’ll gain practical skills for creating systems that scale gracefully under real-world demands.

Let’s start with our foundation. We’re building an e-commerce system with three core services: orders, payments, and inventory. Each service owns its data and communicates through events. Notice how each service has its own MongoDB database? This isolation prevents one service’s failures from cascading to others.

// Shared RabbitMQ connection setup
const amqp = require('amqplib');

class MessageBroker {
  constructor() {
    this.connection = null;
    this.channel = null;
  }

  async connect() {
    this.connection = await amqp.connect(process.env.RABBITMQ_URL);
    this.channel = await this.connection.createChannel();
    
    // Set up core exchanges
    const exchanges = ['orders', 'payments', 'inventory'];
    for (const exchange of exchanges) {
      await this.channel.assertExchange(exchange, 'topic', { durable: true });
    }
  }

  async publish(exchange, key, payload) {
    this.channel.publish(
      exchange, 
      key, 
      Buffer.from(JSON.stringify(payload)),
      { persistent: true }
    );
  }
}

What happens when a customer places an order? The orders service creates a pending order and publishes an order.created event. This triggers a chain reaction—the payments service reserves funds, while inventory checks stock. But how do we keep everything consistent if payment fails midway? That’s where the Saga pattern shines.

// Order creation with Saga
class OrderService {
  async createOrder(data) {
    const sagaId = generateId(); // Unique transaction ID
    
    // Create order in PENDING state
    const order = await db.orders.insert({
      ...data,
      status: 'PENDING',
      sagaId
    });

    // Publish event to start payment process
    await broker.publish('orders', 'order.created', {
      orderId: order.id,
      amount: order.total,
      sagaId
    });
    
    return order;
  }
}

Now consider failure scenarios. What if inventory runs out after payment processes? We need compensation logic. When payment succeeds but inventory reservation fails, we trigger a payment.refund event. This rollback capability makes our system transactional without tight coupling.

// Handling payment success but inventory failure
class PaymentService {
  constructor() {
    broker.subscribe('payments', 'payment.success', this.handleSuccess);
  }

  async handleSuccess(event) {
    try {
      await reserveInventory(event.orderId); 
    } catch (error) {
      // Critical: Trigger refund if inventory fails
      await broker.publish('payments', 'payment.refund', {
        transactionId: event.paymentId,
        sagaId: event.sagaId
      });
    }
  }
}

For reliability, we implement dead letter queues. When message processing fails repeatedly, RabbitMQ moves it to a special queue. We monitor these for troubleshooting. How often do you check your dead letter queues? I set up hourly alerts—it’s saved me from midnight outages multiple times.

// Dead letter setup in RabbitMQ
await channel.assertQueue('orders.process', {
  durable: true,
  deadLetterExchange: 'dlx', // Dead letter exchange
  deadLetterRoutingKey: 'orders.failed'
});

await channel.bindQueue('dlx.orders', 'dlx', 'orders.failed');

Monitoring is crucial. I use Prometheus with RabbitMQ’s built-in metrics exporter. Track message rates, consumer counts, and unacknowledged messages. Spot consumer lag early—it often signals bigger issues. For debugging, include sagaId in every event. This lets you trace transactions across services.

When deploying, run multiple identical consumers. RabbitMQ automatically load balances between them. During peak sales, I scale payment processors independently from inventory checks. That flexibility is why I love this architecture.

What improvements could you make? Maybe add product recommendations that react to order events. Or connect loyalty points to payment events. The pattern extends beautifully.

I’ve shared my hard-won lessons building these systems. If this helped you, pay it forward—share with a colleague who’s wrestling with distributed systems. Have questions or war stories? Drop them in the comments. Let’s keep learning together.