I’ve been thinking a lot about file uploads lately. Every modern application needs them, yet many implementations struggle with performance, scalability, and security. What if we could build an upload service that handles large files efficiently without crashing servers? That’s exactly what we’ll create today using Fastify, multipart streams, and AWS S3. Stick with me - you’ll learn how to build a production-ready solution that scales.

First, let’s set up our project. Create a new directory and initialize it with npm. We’ll need several key dependencies:

npm install fastify @fastify/multipart @aws-sdk/client-s3
npm install dotenv joi mime-types

These packages give us our web framework, file handling capabilities, AWS integration, and validation tools. Notice we’re installing specific AWS SDK v3 packages - they’re more modular and efficient than the old monolithic SDK.

Now for TypeScript configuration. Create a tsconfig.json with strict type checking enabled. This catches errors early and improves code quality. Why risk runtime errors when TypeScript can prevent them during development?

Configuring Fastify requires special attention to multipart handling. We want streaming capabilities, not buffering entire files in memory. Here’s how I set it up:

await server.register(multipart, {
  limits: {
    fileSize: config.MAX_FILE_SIZE,
    files: config.MAX_FILES_PER_REQUEST
  },
  attachFieldsToBody: false
});

By setting attachFieldsToBody: false, we take manual control of file streams. This prevents Fastify from buffering files and allows direct piping to S3. How much memory could we save by avoiding buffering? For 100 concurrent 1GB uploads, we’d avoid 100GB of memory usage!

For AWS integration, we need robust S3 configuration. The @aws-sdk/lib-storage package provides the Upload class that handles multipart uploads automatically:

const upload = new Upload({
  client: s3Client,
  params: {
    Bucket: 'my-bucket',
    Key: filename,
    Body: fileStream
  },
  partSize: 5 * 1024 * 1024
});

await upload.done();

This code streams files directly to S3 in 5MB chunks. If the connection drops, it can resume automatically. What would happen if we didn’t use multipart uploads? Large files would fail completely on network issues.

Validation is crucial for security. We check file types and sizes before processing:

const allowedTypes = ['image/jpeg', 'image/png'];
if (!allowedTypes.includes(mimetype)) {
  throw new Error('Invalid file type');
}

For images, I often add Sharp.js for resizing during upload. It processes images as streams without saving intermediates. Could we prevent storage attacks by validating before any processing? Absolutely - we reject invalid files immediately.

Users love progress indicators. We implement server-sent events for real-time updates:

res.write(`data: ${JSON.stringify(progress)}\n\n`);

This sends progress percentages to the client. How responsive would your UI feel with live progress bars?

Error handling must be comprehensive. We wrap uploads in try-catch blocks and implement retry logic:

try {
  await uploadStream();
} catch (err) {
  if (err instanceof TimeoutError) {
    await retryUpload();
  }
}

For production, we add rate limiting with @fastify/rate-limit. It prevents abuse by limiting requests per IP. What could happen without rate limiting? One malicious user could overload your system.

Testing is straightforward with tools like Postman. I simulate slow connections to verify progress tracking. Always test failure scenarios - what happens when S3 credentials expire? Our error handler returns clear messages without exposing internals.

In production, consider these additions:

• Use AWS Instance Roles instead of access keys
• Enable S3 server access logging
• Set lifecycle policies to transition files to Glacier
• Implement CloudFront for faster downloads

We’ve built a high-performance upload service that streams files directly to S3. It handles large files efficiently, validates securely, and provides user feedback. This approach saves server resources while maintaining reliability. What file handling challenges could this solve for your projects?

If you found this useful, share it with your network. Have questions or improvements? Let me know in the comments - I read every response and appreciate your insights.