Scroll Top

Deploy Next.js Application into Vercel from AWS S3

Feature Image

Introduction to Vercel:

Vercel is a cloud-based infrastructure that facilitates the expansion and deployment of front-end web applications by its developers. As a serverless platform, Vercel manages the underlying infrastructure, freeing up developers to concentrate on building applications. It is a Jamstack platform designed especially for web apps and static web pages. 

 

Vercel provides several features that make it prominent among developers, which include:  

  • Instant deployment: You can deploy your application instantly, allowing you to see the changes immediately.
  • Global CDN: Vercel ensures minimal latency by distributing your applications to users worldwide via a global Content Delivery Network (CDN).
  • Automatic scaling: It dynamically adjusts the capacity of your applications to manage sudden increases in traffic.
  • Security: With features like SSL certificates and rate limitations, Vercel offers a secure environment for your apps.
  • Ease of use: Vercel has a user-friendly interface and an extensive collection of pre-configured templates.

 

By default, we can only link our git project repositories through the Vercel dashboard to perform the Vercel deployment. 

 


 

The image above displays the Vercel dashboard. Only with the help of git repositories, you can import the project into Vercel.   

 

Use case: 

You keep multiple Next.js projects in separate folders within an AWS S3 bucket. After that, upload the project to a folder within an AWS S3 bucket and deploy it to Vercel as a static site. Then, store the URL of the preview static site in a database. 

 

Architecture: 
AWS S3 to Vercel Architecture

 

As stated in the use case, we are deploying the next.js project, which we have uploaded to a Vercel folder inside an AWS S3 bucket. To upload the next.js project into AWS S3, we will utilize AWS Lambda to automate Vercel deployment and with the help of AWS Lambda Triggers for the AWS S3 event notifications.  

 

How to trigger the AWS lambda to create and deploy the next.js application from the AWS S3 folder into Vercel: 

The event notifications are only available for creating objects in AWS S3, not for creating AWS S3 folders. 

 We can make use of the sentinel file here. Once all the project’s actual contents have been uploaded, we can utilize it to trigger events from AWS S3 for folder creation by uploading this file in a required format, such as .sentinel, into the folder. You can find anything from organisation data to project-related information in the sentinel file.

 

What is a Sentinel file? 

The term “sentinel file” is not commonly used with AWS S3 or cloud computing in general. Nevertheless, the phrase “Sentinel file” finds application in various contexts beyond AWS.

A sentinel file is a unique marker in a directory or folder to signify a specific condition or status. Users commonly use it in general computing and software development to control or trigger specific processes or actions within a system. 

 

Attach the Sentinel file AWS S3 upload event as a trigger to the AWS Lambda: 

Use the sentinel file name extension as the suffix path to create an AWS S3 event notification for the uploaded sentinel file. Assign Lambda as the destination, which will execute the Vercel project creation and deployment process. 

 

AWS S3 Event Notification

 

As shown in the image above, I have created an event notification with the suffix path _.txt, which specifies that the sentinel file for my use case must be in the format of {sentinel_file_name}_.txt to trigger the event. 

 

What does Lambda do? 

 Using the authorization token, Lambda establishes a connection with Vercel, creates a project, and deploys the next.js application. If the project is already established, it will handle the deployment exclusively.

 

AWS Lambda to deploy the next.js app from AWS S3 into Vercel: 

  • If serverless isn’t installed, install it.

 

npm install -g serverless

 

  • Set the SLS Config to map your AWS account to deploy the application

 

serverless config credentials --provider aws --key AWS_ACCESS_KEY --secret AWS_SECRET_KEY

 

  • Create a serverless application with node.js as a runtime

 

sls create --template aws-nodejs --path deploy-nextjs-app-into-vercel

 

We will apply the below folder structure to the deploy-nextjs-app-into-vercel folder when we create the Node.js application.

 

 

Create a package.json file using the npm init command, which must be executed within the project folder (deploy-nextjs-app-into-vercel). 

 

Once the project files have been uploaded, the lambda function will initiate upon uploading the sentinel file to the project folder. The event data will contain the project folder’s name under the S3 bucket. Subsequently, we must generate a pre-signed URL for each file in the project and download it to the AWS lambda runtime folder (tmp/). The AWS lambda comes with 512 MB of default storage. However, it also supports up to 10 GB of ephemeral storage. If you want to increase the storage, you can modify it during the configuration.  

 

 

Ephemeral storage of Lambda

 

The lambda will determine if Vercel has already created the project. If so, it will use the Vercel rest API to retrieve the Vercel project id and proceed with the deployment using this id. Then, download the next.js application in the Lambda ephemeral storage location (/temp path) using the @vercel/client npm package. If the project is not already in Vercel, the lambda will create a new one using the Vercel REST API and deploy the application using the @vercel/client npm package.  

Lambda Code:

  1. package.json
{
  "name": "deploy-nextjs-app-into-vercel",
  "version": "1.0.0",
  "description": "",
  "main": "handler.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@aws-sdk/client-s3": "^3.370.0",
    "@aws-sdk/s3-request-presigner": "^3.370.0",
    "@aws-sdk/util-create-request": "^3.370.0",
    "@types/node": "^18.0.6",
    "@vercel/client": "^12.6.5",
    "axios": "^1.4.0",
    "follow-redirects": "^1.15.2",
    "next": "^13.4.10",
    "node-fetch": "^3.2.6",
    "os": "^0.1.2"
  }
}
2. handler.js
'use strict';
const { S3Client, GetObjectCommand, ListObjectsCommand } = require('@aws-sdk/client-s3');
const { getSignedUrl } = require("@aws-sdk/s3-request-presigner");
const https = require("https");
const { createDeployment } = require('@vercel/client');
const fs = require('fs');
const path = require('path');
const axios = require('axios');
const os = require('os');
const { parse } = require("url");

const s3Client = new S3Client(
  {
    region: YOUR_AWS_REGION,
    credentials: {
      accessKeyId: YOUR_AWS_ACCESS_KEY,
      secretAccessKey: YOUR_AWS_SECRET_KEY,
    },
});

module.exports.deployNextJsAppToVercel  = async (event, context) => {
  const token = YOUR_VERCEL_TOKEN;
  const projectName = event.Records[0].s3.object.key.split("/")[0];
  // For local testing
  // const projectName = YOUR_PROJECT_NAME;
  console.log("projectName", projectName);
  const s3BucketName = YOUR_S3_BUCKET_NAME;
  const folderName = projectName;

  let projectID = await createVercelProject(token, projectName);
  console.log(projectID);
  const vercelProject = await setNodeVersion(projectID, token);
  console.log("vercelProject:", vercelProject);
  if (!projectID) {
    try {
      const projects = await getVercelProjects(token);
      const project = projects.projects.find((proj) => proj.name === projectName);
      if (project) {
        const projectID = project.id;
        console.log('Vercel Project ID:', projectID);
      } else {
        throw new Error(`Project '${projectName}' not found.`);
      }
    } catch (error) {
      console.error('Error:', error.message);
      return {
        statusCode: 500,
        body: `Error: ${error.message}`,
      };
    }
  }
  const objectKeys = await getObjectKeys(s3BucketName, folderName);
  const filteredObjectKeys = objectKeys.filter(objectKey => !objectKey.includes('.git') && !objectKey.includes('README.md'));
  const signedUrls = await Promise.all(
    filteredObjectKeys.map((key) => getSignedUrlData(s3BucketName, key))
  );
  console.log("urls", signedUrls);
  await downloadFiles(signedUrls);
  // Step 2: Trigger a deployment in Vercel for the signed URLs
  const deployment = await triggerVercelDeployment(token, projectID, signedUrls, folderName);
  await deleteFolder(folderName)
  return {
    statusCode: 200,
    body: `Deployment got successful, the website url is ${deployment?.url}`,
  };
};

const deleteFolder = async (folderName) => {
  const temporaryDirectory = os.tmpdir();
  console.log("temporaryDirectory", temporaryDirectory);
  if (!fs.existsSync(temporaryDirectory+folderName)) {
    return;
  }

  await fs.rmdir(temporaryDirectory+folderName);
};
/**
 * getObjectKeys - get the nextjs application files as S3 objects
 * @param {*} bucketName s3 bucket name which contains the list of Next Js application
 * @param {*} folderName s3 folder contains the Next Js we want to deploy into Vercel
 * @returns 
 */
async function getObjectKeys(bucketName, folderName) {
  const listObjectsCommand = new ListObjectsCommand({
    Bucket: bucketName,
    Prefix: `${folderName}/`,
  });

  const response = await s3Client.send(listObjectsCommand);
  const objects = response.Contents;
  const objectKeys = objects.map((object) => object.Key);

  return objectKeys;
}

/**
 * getSignedUrlData - get the presigned URL for all the s3 object keys(i.e for all the project files)
 * @param {*} bucketName s3 bucket name which contains the list of Next Js application
 * @param {*} objectKey s3 object key for the Next Js application files
 * @returns 
 */
async function getSignedUrlData(bucketName, objectKey) {
  const params = {
    Bucket: bucketName,
    Key: objectKey,
  };

  const command = new GetObjectCommand(params);
  const signedUrl = await getSignedUrl(s3Client, command, { expiresIn: 360000 });
  // return signedUrl;
  const parsedUrl = parse(signedUrl);
  parsedUrl.protocol = "http:";

  // Convert the modified URL object back to a string
  const httpPresignedUrl = parsedUrl.format();

  return httpPresignedUrl;
}

/**
 * triggerVercelDeployment - triggers the Vercel Deployment inside the project created for the Next Js application.
 * @param {*} authToken authentication token for the vercel account
 * @param {*} projectID vercel project id
 * @param {*} signedUrls aws s3 nextjs project file presigned URL's
 * @param {*} folderName aws s3 nextjs project folder inside the AWS S3 bucket
 * @returns vercel deployment data
 */
async function triggerVercelDeployment(authToken, projectID, signedUrls, folderName) {
  let deployment = undefined;
  const temporaryDirectory = os.tmpdir();
  console.log("temporaryDirectory", temporaryDirectory);
  console.log('createDeployment ', createDeployment);
  for await (const event of createDeployment({
    token: authToken,
    teamId: "team_Y6iorVTQiUAff3HsrpWHZ6am",
    path: path.resolve(`/tmp/${folderName}`),
    // path: path.resolve(`./${folderName}`),
    project: {
      id: projectID
    },
    projectSettings: {
      buildCommand: null,
      devCommand: null,
      framework: 'nextjs',
      commandForIgnoringBuildStep: '',
      installCommand: null,
      outputDirectory: null,
    },
    files: signedUrls,
  })) {
    console.log('event.type', event.type);
    if (event.type === 'ready') {
      console.log('Breaking ', event);
      deployment = event.payload;
      break;
    }
    if (event.type === 'error') {
      console.log('Breaking ', event);
      break;
    }
  }
  fs.rm(`${temporaryDirectory}/${folderName}`, { recursive: true, force: true }, (error) => {
    if (error) {
      console.error(error);
    } else {
      console.log(`Directory '${folderName}' is deleted.`);
    }
  });
  return deployment;
}

/**
 * getVercelProjects - get the list of vercel projects under the account.
 * @param {*} token authentication token for the vercel account
 * @returns list of vercel projects
 */
async function getVercelProjects(token) {
  const options = {
    hostname: 'api.vercel.com',
    path: '/v8/projects/?teamId=team_Y6iorVTQiUAff3HsrpWHZ6am',
    method: 'GET',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${token}`,
    },
  };

  return new Promise((resolve, reject) => {
    const req = https.request(options, (res) => {
      let data = '';

      res.on('data', (chunk) => {
        data += chunk;
      });

      res.on('end', () => {
        if (res.statusCode === 200) {
          const projects = JSON.parse(data);
          resolve(projects);
        } else {
          reject(new Error(`Failed to fetch Vercel projects: ${res.statusCode} ${res.statusMessage}`));
        }
      });
    });

    req.on('error', (err) => {
      reject(err);
    });

    req.end();
  });
}

/**
 * downloadFile - download the file
 * @param {*} url aWS s3 objects presigned URL
 * @param {*} destination local path location
 */
async function downloadFile(url, destination) {
  const response = await axios.get(url, { responseType: 'arraybuffer' });
  fs.writeFileSync(destination, Buffer.from(response.data, 'binary'));
}

/**
 * downloadFiles - download the files
 * @param {*} urls aws s3 objects presigned URL
 */
async function downloadFiles(urls) {
  for (const url of urls) {
    console.log(url);
    let fileName = path.basename(url.split('?')[0]);
    const temporaryDirectory = os.tmpdir();
    console.log("temporaryDirectory", temporaryDirectory);
    let dirName = `${temporaryDirectory}/${url.split('?')[0].split("amazonaws.com/")[url.split('?')[0].split("amazonaws.com/").length-1]}`;
    dirName = dirName.replace(`/${fileName}`, '');
    path.resolve(dirName);
    try {
      fs.mkdirSync(dirName, { recursive: true });
    } catch(error) {
      console.log(error)
    }
    // if (!fs.existsSync(dirName)) {
    //   fs.mkdirSync(dirName, { recursive: true });
    // }
    console.log(fileName);
    const filePath = path.join(dirName, fileName);
    await downloadFile(url, filePath);
    console.log(`Downloaded: ${filePath}`);
  }
}

/**
 * createVercelProject - creates the vercel nextjs framework project
 * @param {*} token authentication token for the vercel account
 * @param {*} projectName vercel project name
 * @returns 
 */
async function createVercelProject(token, projectName) {
  const options = {
    hostname: 'api.vercel.com',
    path: '/v1/projects/?teamId=team_Y6iorVTQiUAff3HsrpWHZ6am',
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${token}`,
    },
  };

  return new Promise((resolve, reject) => {
    const req = https.request(options, (res) => {
      let data = '';

      res.on('data', (chunk) => {
        data += chunk;
      });

      res.on('end', () => {
        console.log(res.statusCode)
        console.log(data)
        if (res.statusCode === 200) {
          const { id } = JSON.parse(data);
          resolve(id);
        } else {
          resolve(undefined);
        }
      });
    });

    req.on('error', (err) => {
      reject(err);
    });

    req.write(JSON.stringify({ 
      name: projectName, 
      framework: "nextjs"
    }));

    req.end();
  });
}

// Function to set the Node.js version using Vercel API's set-build-env endpoint
function setNodeVersion(projectId, token) {
  const options = {
    hostname: 'api.vercel.com',
    path: `/v1/projects/${projectId}/?teamId=YOUR_VERCEL_TEAM_ID`,
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${token}`,
    },
  };

  const envData = {
    key: 'NODE_VERSION',
    value: '16',
    teamId: YOUR_VERCEL_TEAM_ID,
    scope: 'team',
  };

  const req = https.request(options, (res) => {
    let data = '';

    res.on('data', (chunk) => {
      data += chunk;
    });

    res.on('end', () => {
      console.log(data)
      console.log('Node.js version set successfully:', JSON.parse(data));
    });
  });

  req.on('error', (err) => {
    console.log(err)
    console.error('Error setting Node.js version:', err.message);
  });

  req.write(JSON.stringify(envData)); // Send the environment variable data in the request body
  req.end();
}

 

Serverless.yaml
service: deploy-nextjs-app-vercel
frameworkVersion: "3"

provider:
  name: aws
  runtime: nodejs18.x
  region: YOUR_AWS_REGION

functions:
  deployNextJsAppToVercel:
    handler: handler.deployNextJsAppToVercel
    timeout: 900

Conclusion:

This blog showcases an efficient solution for deploying Next.js applications from AWS S3 to Vercel, utilizing AWS Lambda for automation. By leveraging serverless computing and a clever use of a sentinel file for deployment triggers, this architecture simplifies the deployment process, allowing developers to focus on development rather than operational complexities. The comprehensive steps, which include setup and script examples, showcase the seamless integration between AWS S3 and Vercel, providing a streamlined pipeline that boosts productivity, minimizes errors, and guarantees the swift and reliable deployment of applications. This approach not only optimizes web application deployment workflows but also underscores the transformative potential of serverless computing in modern web development.

 

Github URL: https://github.com/GowrisankarK/deploy-nextjs-apps-into-vercel-from-aws-s3

Gowri Sankar, Krishnamoorthy

+ posts

Anu Ranjana M

+ posts