Build a Type-Safe File Upload Pipeline with Node.js, Multer, Sharp, and AWS S3

js

Build a Type-Safe File Upload Pipeline with Node.js, Multer, Sharp, and AWS S3

Learn to build a secure, type-safe file upload pipeline with Node.js, Multer, Sharp, and AWS S3. Validate, process, store, and scale smarter.

Apr 30, 2026

Build a Type-Safe File Upload Pipeline with Node.js, Multer, Sharp, and AWS S3

I remember the first time I had to handle file uploads in a production application. It was a mess. Users could upload anything—gigabyte videos, malicious scripts disguised as images, files with names longer than the alphabet. The server would choke, the database would bloat, and security audits turned into nightmares. That experience taught me one thing: an upload pipeline is not just about moving bytes from point A to point B. It is about validation, transformation, security, and scalability. It is about doing it in a way that your type system can enforce. Today, I want to walk you through how I built a type-safe, end-to-end file upload system using Node.js, Multer, Sharp, and AWS S3. No magic, no overwrought metaphors—just clean code and solid engineering.

Why did this topic come to my mind so strongly? Because last week a colleague asked me, “Why do we need Sharp when Multer already gives us the file?” We had a long conversation about image metadata, EXIF data, format conversion, and how a plain upload can leak user location or fail on different browsers. I realized many developers skip the middle step—the processing layer—and end up paying for storage they don’t need or serving images that break. So here is what I do now, and I think you should too.

At its core, the pipeline has four stages: receiving the file, validating and processing it, storing it in the cloud, and generating a secure URL for the client. Let me break each one down with real code.

Stage One: Receiving with Multer

Multer is a Node.js middleware for handling multipart/form-data, which is how browsers send files. I configure it with memoryStorage to keep the file in RAM as a Buffer. Why not disk? Because I want to process that buffer immediately with Sharp, and writing to disk then reading back is an unnecessary I/O hit. Here is the setup:

import multer from 'multer';

const upload = multer({
  storage: multer.memoryStorage(),
  limits: { fileSize: 10 * 1024 * 1024 }, // 10 MB
  fileFilter: (req, file, cb) => {
    const allowed = ['image/jpeg', 'image/png', 'image/webp', 'image/avif'];
    if (allowed.includes(file.mimetype)) {
      cb(null, true);
    } else {
      cb(new Error('Only JPEG, PNG, WebP, and AVIF are allowed.'));
    }
  }
});

I also enforce a file size limit and check MIME types client side? No, never trust the client. I check on the server because anyone can craft a request. A question for you: have you ever seen a file that claims to be a JPEG but is actually an executable? That is why I verify the real MIME type later with Sharp. But this filter is the first gate.

Stage Two: Processing with Sharp

Once Multer gives me a buffer, I pass it to Sharp. This is where I resize, convert format, strip EXIF metadata, and compress. Why strip EXIF? Because that little data block can contain GPS coordinates, camera serial numbers, and timestamps. If you are building a social app, do you really want to leak where every photo was taken? I don’t.

Here is a typical processing function:

import sharp from 'sharp';

async function processImage(buffer: Buffer, options: {
  width?: number;
  height?: number;
  format?: 'jpeg' | 'webp' | 'avif';
  quality?: number;
  stripMetadata?: boolean;
}) {
  let pipeline = sharp(buffer);

  if (options.width || options.height) {
    pipeline = pipeline.resize(options.width, options.height, {
      fit: 'inside',
      withoutEnlargement: true
    });
  }

  if (options.stripMetadata !== false) {
    pipeline = pipeline.withMetadata(); // actually to keep minimal? I strip by default
  }

  // Convert format
  switch (options.format || 'jpeg') {
    case 'jpeg':
      pipeline = pipeline.jpeg({ quality: options.quality || 80 });
      break;
    case 'webp':
      pipeline = pipeline.webp({ quality: options.quality || 75 });
      break;
    case 'avif':
      pipeline = pipeline.avif({ quality: options.quality || 50 });
      break;
  }

  const { data, info } = await pipeline.toBuffer({ resolveWithObject: true });
  return { buffer: data, mimeType: `image/${options.format || 'jpeg'}`, ...info };
}

Notice I used withMetadata()? Actually, Sharp by default strips metadata unless you call withMetadata(). So if I want to strip, I just don’t call it. The result is a clean, tiny image buffer ready for storage.

Stage Three: Storing in AWS S3

Now I upload that buffer to S3 using AWS SDK v3. I use a key like uploads/{uuid}.{extension} to avoid collisions. I also set the ContentType so browsers render it correctly, and CacheControl for performance. Here is the upload function:

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { v4 as uuid } from 'uuid';

const s3 = new S3Client({
  region: process.env.AWS_REGION,
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
  }
});

async function uploadToS3(buffer: Buffer, mimeType: string, folder = 'uploads') {
  const key = `${folder}/${uuid()}.${mimeType.split('/')[1]}`;
  const command = new PutObjectCommand({
    Bucket: process.env.AWS_S3_BUCKET,
    Key: key,
    Body: buffer,
    ContentType: mimeType,
    CacheControl: 'public, max-age=31536000'
  });

  await s3.send(command);
  return key;
}

Wait—what about access control? I set the bucket to private and generate signed URLs for each file. That way nobody can guess the URL and download random uploads. Which brings us to the final stage.

Stage Four: Signed URLs with a Time Limit

A signed URL is a temporary link that grants access to a private S3 object. I use getSignedUrl from @aws-sdk/s3-request-presigner. I set an expiry of 15 minutes (900 seconds). For most apps, that is enough for the user to view or download the file. After that, the link expires and the file remains private.

import { GetObjectCommand } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-s3/s3-request-presigner';

async function generateSignedUrl(key: string, expiresIn = 900): Promise<string> {
  const command = new GetObjectCommand({
    Bucket: process.env.AWS_S3_BUCKET,
    Key: key
  });
  return getSignedUrl(s3, command, { expiresIn });
}

Now my API endpoint returns the signed URL along with metadata like width, height, and format. The client can use that URL directly in an <img> tag or for download. No need to proxy through my server for every request—the client talks to S3.

Putting It All Together

The Express route looks like this:

import { Request, Response } from 'express';

app.post('/upload', upload.single('file'), async (req: Request, res: Response) => {
  try {
    const file = req.file!;
    const { buffer, mimeType, width, height } = await processImage(file.buffer, {
      format: 'webp',
      width: 1200,
      stripMetadata: true
    });

    const key = await uploadToS3(buffer, mimeType);
    const signedUrl = await generateSignedUrl(key);

    res.json({
      key,
      signedUrl,
      expiresAt: new Date(Date.now() + 900 * 1000).toISOString(),
      metadata: { width, height, format: 'webp', sizeBytes: buffer.length }
    });
  } catch (error: any) {
    res.status(500).json({ error: error.message });
  }
});

You might notice I used upload.single('file'). The field name file is what the client must use. If the client sends a different field, Multer will reject it. Small details like this prevent a class of bugs.

Type Safety Across the Pipeline

I use TypeScript not just for the code but to define the contract between stages. I created interfaces for UploadOptions, ProcessedImage, and UploadResult. When I call processImage, the return type tells me exactly what properties exist. When I generate a signed URL, the function signature declares that I need a string key. This catches mistakes at compile time rather than runtime. For example, if I accidentally pass a number instead of a string, TypeScript screams. That is the “type-safe” part.

One more question: what happens if the S3 upload fails after the image is processed? You have already consumed resources but the client gets an error. I handle this with a try-catch and optionally clean up the processed buffer. But more robustly, you could implement a dead-letter queue or retry logic. For most cases, simply returning the error and logging is enough.

Testing the Pipeline

I write tests using Jest and Supertest. I mock S3 and Sharp to avoid hitting real services. For the Sharp mock, I return a fake buffer of known size. Here is a snippet:

import request from 'supertest';
import app from '../src/app';

jest.mock('sharp', () => () => ({
  resize: jest.fn().mockReturnThis(),
  webp: jest.fn().mockReturnThis(),
  toBuffer: jest.fn().mockResolvedValue({ data: Buffer.from('fake'), info: { width: 100, height: 100, format: 'webp' } })
}));

it('returns a signed URL on successful upload', async () => {
  const response = await request(app)
    .post('/upload')
    .attach('file', Buffer.from('fake-image'), 'test.jpg');

  expect(response.status).toBe(200);
  expect(response.body).toHaveProperty('signedUrl');
});

Testing ensures your pipeline doesn’t break when you update dependencies. It also documents expected behavior.

Security Considerations

I stripped EXIF, I limited file size, I restricted MIME types. I also make sure the bucket policy prevents public reads. Signed URLs are time-limited and scoped to a specific object. No one can list objects. The upload endpoint is also rate-limited. And never, ever store the original file without processing. Even if you think you need it, you probably don’t. The processed version is smaller, safer, and consistent.

Personal Touch

Every time I set up a new project, I copy this pipeline. It saved me countless hours of debugging and several security incidents. One time, a user uploaded a photo with embedded malware in the EXIF data—Sharp’s stripping removed it. Another time, a client sent a file named ../../../etc/passwd as a “filename”. Multer handled it, but Sharp would have processed the content, not the name. Point is: do not trust input.

Encouragement

If you are building a web app today, do not skip this. Start with Multer, add Sharp, and push to S3 with signed URLs. It is not as hard as it sounds. And the result is a pipeline that scales, secures, and performs. If you found this helpful, please like, share, and comment below. I would love to hear about your own upload nightmares—or victories. Let’s learn from each other.

As a best-selling author, I invite you to explore my books on Amazon. Don’t forget to follow me on Medium and show your support. Thank you! Your support means the world!

101 Books

101 Books is an AI-driven publishing company co-founded by author Aarav Joshi. By leveraging advanced AI technology, we keep our publishing costs incredibly low—some books are priced as low as $4—making quality knowledge accessible to everyone.

Check out our book Golang Clean Code available on Amazon.

Stay tuned for updates and exciting news. When shopping for books, search for Aarav Joshi to find more of our titles. Use the provided link to enjoy special discounts!

📘 Checkout my latest ebook for free on my channel!
Be sure to like, share, comment, and subscribe to the channel!