Deploy Next.js Application into Vercel from AWS S3

By Gowri Sankar, Krishnamoorthy , Anu Ranjana M Technical Blog March 26, 2024

Introduction to Vercel:

Vercel is a cloud-based infrastructure that facilitates the expansion and deployment of front-end web applications by its developers. As a serverless platform, Vercel manages the underlying infrastructure, freeing up developers to concentrate on building applications. It is a Jamstack platform designed especially for web apps and static web pages.

Vercel provides several features that make it prominent among developers, which include: 

Instant deployment: You can deploy your application instantly, allowing you to see the changes immediately.

Global CDN: Vercel ensures minimal latency by distributing your applications to users worldwide via a global Content Delivery Network (CDN).

Automatic scaling: It dynamically adjusts the capacity of your applications to manage sudden increases in traffic.

Security: With features like SSL certificates and rate limitations, Vercel offers a secure environment for your apps.

Ease of use: Vercel has a user-friendly interface and an extensive collection of pre-configured templates.

By default, we can only link our git project repositories through the Vercel dashboard to perform the Vercel deployment.

The image above displays the Vercel dashboard. Only with the help of git repositories, you can import the project into Vercel.  

Use case:

You keep multiple Next.js projects in separate folders within an AWS S3 bucket. After that, upload the project to a folder within an AWS S3 bucket and deploy it to Vercel as a static site. Then, store the URL of the preview static site in a database.

Architecture:

As stated in the use case, we are deploying the next.js project, which we have uploaded to a Vercel folder inside an AWS S3 bucket. To upload the next.js project into AWS S3, we will utilize AWS Lambda to automate Vercel deployment and with the help of AWS Lambda Triggers for the AWS S3 event notifications. 

How to trigger the AWS lambda to create and deploy the next.js application from the AWS S3 folder into Vercel:

The event notifications are only available for creating objects in AWS S3, not for creating AWS S3 folders.

We can make use of the sentinel file here. Once all the project’s actual contents have been uploaded, we can utilize it to trigger events from AWS S3 for folder creation by uploading this file in a required format, such as .sentinel, into the folder. You can find anything from organisation data to project-related information in the sentinel file.

What is a Sentinel file?

The term “sentinel file” is not commonly used with AWS S3 or cloud computing in general. Nevertheless, the phrase “Sentinel file” finds application in various contexts beyond AWS.

A sentinel file is a unique marker in a directory or folder to signify a specific condition or status. Users commonly use it in general computing and software development to control or trigger specific processes or actions within a system.

Attach the Sentinel file AWS S3 upload event as a trigger to the AWS Lambda:

Use the sentinel file name extension as the suffix path to create an AWS S3 event notification for the uploaded sentinel file. Assign Lambda as the destination, which will execute the Vercel project creation and deployment process.

As shown in the image above, I have created an event notification with the suffix path _.txt, which specifies that the sentinel file for my use case must be in the format of {sentinel_file_name}_.txt to trigger the event.

What does Lambda do?

Using the authorization token, Lambda establishes a connection with Vercel, creates a project, and deploys the next.js application. If the project is already established, it will handle the deployment exclusively.

AWS Lambda to deploy the next.js app from AWS S3 into Vercel:

If serverless isn’t installed, install it.

npm install -g serverless

Set the SLS Config to map your AWS account to deploy the application

serverless config credentials --provider aws --key AWS_ACCESS_KEY --secret AWS_SECRET_KEY

Create a serverless application with node.js as a runtime

sls create --template aws-nodejs --path deploy-nextjs-app-into-vercel

We will apply the below folder structure to the deploy-nextjs-app-into-vercel folder when we create the Node.js application.

Create a package.json file using the npm init command, which must be executed within the project folder (deploy-nextjs-app-into-vercel).

Once the project files have been uploaded, the lambda function will initiate upon uploading the sentinel file to the project folder. The event data will contain the project folder’s name under the S3 bucket. Subsequently, we must generate a pre-signed URL for each file in the project and download it to the AWS lambda runtime folder (tmp/). The AWS lambda comes with 512 MB of default storage. However, it also supports up to 10 GB of ephemeral storage. If you want to increase the storage, you can modify it during the configuration. 

The lambda will determine if Vercel has already created the project. If so, it will use the Vercel rest API to retrieve the Vercel project id and proceed with the deployment using this id. Then, download the next.js application in the Lambda ephemeral storage location (/temp path) using the @vercel/client npm package. If the project is not already in Vercel, the lambda will create a new one using the Vercel REST API and deploy the application using the @vercel/client npm package. 

Lambda Code:

package.json

{
  "name": "deploy-nextjs-app-into-vercel",
  "version": "1.0.0",
  "description": "",
  "main": "handler.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@aws-sdk/client-s3": "^3.370.0",
    "@aws-sdk/s3-request-presigner": "^3.370.0",
    "@aws-sdk/util-create-request": "^3.370.0",
    "@types/node": "^18.0.6",
    "@vercel/client": "^12.6.5",
    "axios": "^1.4.0",
    "follow-redirects": "^1.15.2",
    "next": "^13.4.10",
    "node-fetch": "^3.2.6",
    "os": "^0.1.2"
  }
}

2. handler.js

'use strict';
const { S3Client, GetObjectCommand, ListObjectsCommand } = require('@aws-sdk/client-s3');
const { getSignedUrl } = require("@aws-sdk/s3-request-presigner");
const https = require("https");
const { createDeployment } = require('@vercel/client');
const fs = require('fs');
const path = require('path');
const axios = require('axios');
const os = require('os');
const { parse } = require("url");

const s3Client = new S3Client(
  {
    region: YOUR_AWS_REGION,
    credentials: {
      accessKeyId: YOUR_AWS_ACCESS_KEY,
      secretAccessKey: YOUR_AWS_SECRET_KEY,
    },
});

module.exports.deployNextJsAppToVercel  = async (event, context) => {
  const token = YOUR_VERCEL_TOKEN;
  const projectName = event.Records[0].s3.object.key.split("/")[0];
  // For local testing
  // const projectName = YOUR_PROJECT_NAME;
  console.log("projectName", projectName);
  const s3BucketName = YOUR_S3_BUCKET_NAME;
  const folderName = projectName;

  let projectID = await createVercelProject(token, projectName);
  console.log(projectID);
  const vercelProject = await setNodeVersion(projectID, token);
  console.log("vercelProject:", vercelProject);
  if (!projectID) {
    try {
      const projects = await getVercelProjects(token);
      const project = projects.projects.find((proj) => proj.name === projectName);
      if (project) {
        const projectID = project.id;
        console.log('Vercel Project ID:', projectID);
      } else {
        throw new Error(`Project '${projectName}' not found.`);
      }
    } catch (error) {
      console.error('Error:', error.message);
      return {
        statusCode: 500,
        body: `Error: ${error.message}`,
      };
    }
  }
  const objectKeys = await getObjectKeys(s3BucketName, folderName);
  const filteredObjectKeys = objectKeys.filter(objectKey => !objectKey.includes('.git') && !objectKey.includes('README.md'));
  const signedUrls = await Promise.all(
    filteredObjectKeys.map((key) => getSignedUrlData(s3BucketName, key))
  );
  console.log("urls", signedUrls);
  await downloadFiles(signedUrls);
  // Step 2: Trigger a deployment in Vercel for the signed URLs
  const deployment = await triggerVercelDeployment(token, projectID, signedUrls, folderName);
  await deleteFolder(folderName)
  return {
    statusCode: 200,
    body: `Deployment got successful, the website url is ${deployment?.url}`,
  };
};

const deleteFolder = async (folderName) => {
  const temporaryDirectory = os.tmpdir();
  console.log("temporaryDirectory", temporaryDirectory);
  if (!fs.existsSync(temporaryDirectory+folderName)) {
    return;
  }

  await fs.rmdir(temporaryDirectory+folderName);
};
/**
 * getObjectKeys - get the nextjs application files as S3 objects
 * @param {*} bucketName s3 bucket name which contains the list of Next Js application
 * @param {*} folderName s3 folder contains the Next Js we want to deploy into Vercel
 * @returns 
 */
async function getObjectKeys(bucketName, folderName) {
  const listObjectsCommand = new ListObjectsCommand({
    Bucket: bucketName,
    Prefix: `${folderName}/`,
  });

  const response = await s3Client.send(listObjectsCommand);
  const objects = response.Contents;
  const objectKeys = objects.map((object) => object.Key);

  return objectKeys;
}

/**
 * getSignedUrlData - get the presigned URL for all the s3 object keys(i.e for all the project files)
 * @param {*} bucketName s3 bucket name which contains the list of Next Js application
 * @param {*} objectKey s3 object key for the Next Js application files
 * @returns 
 */
async function getSignedUrlData(bucketName, objectKey) {
  const params = {
    Bucket: bucketName,
    Key: objectKey,
  };

  const command = new GetObjectCommand(params);
  const signedUrl = await getSignedUrl(s3Client, command, { expiresIn: 360000 });
  // return signedUrl;
  const parsedUrl = parse(signedUrl);
  parsedUrl.protocol = "http:";

  // Convert the modified URL object back to a string
  const httpPresignedUrl = parsedUrl.format();

  return httpPresignedUrl;
}

/**
 * triggerVercelDeployment - triggers the Vercel Deployment inside the project created for the Next Js application.
 * @param {*} authToken authentication token for the vercel account
 * @param {*} projectID vercel project id
 * @param {*} signedUrls aws s3 nextjs project file presigned URL's
 * @param {*} folderName aws s3 nextjs project folder inside the AWS S3 bucket
 * @returns vercel deployment data
 */
async function triggerVercelDeployment(authToken, projectID, signedUrls, folderName) {
  let deployment = undefined;
  const temporaryDirectory = os.tmpdir();
  console.log("temporaryDirectory", temporaryDirectory);
  console.log('createDeployment ', createDeployment);
  for await (const event of createDeployment({
    token: authToken,
    teamId: "team_Y6iorVTQiUAff3HsrpWHZ6am",
    path: path.resolve(`/tmp/${folderName}`),
    // path: path.resolve(`./${folderName}`),
    project: {
      id: projectID
    },
    projectSettings: {
      buildCommand: null,
      devCommand: null,
      framework: 'nextjs',
      commandForIgnoringBuildStep: '',
      installCommand: null,
      outputDirectory: null,
    },
    files: signedUrls,
  })) {
    console.log('event.type', event.type);
    if (event.type === 'ready') {
      console.log('Breaking ', event);
      deployment = event.payload;
      break;
    }
    if (event.type === 'error') {
      console.log('Breaking ', event);
      break;
    }
  }
  fs.rm(`${temporaryDirectory}/${folderName}`, { recursive: true, force: true }, (error) => {
    if (error) {
      console.error(error);
    } else {
      console.log(`Directory '${folderName}' is deleted.`);
    }
  });
  return deployment;
}

/**
 * getVercelProjects - get the list of vercel projects under the account.
 * @param {*} token authentication token for the vercel account
 * @returns list of vercel projects
 */
async function getVercelProjects(token) {
  const options = {
    hostname: 'api.vercel.com',
    path: '/v8/projects/?teamId=team_Y6iorVTQiUAff3HsrpWHZ6am',
    method: 'GET',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${token}`,
    },
  };

  return new Promise((resolve, reject) => {
    const req = https.request(options, (res) => {
      let data = '';

      res.on('data', (chunk) => {
        data += chunk;
      });

      res.on('end', () => {
        if (res.statusCode === 200) {
          const projects = JSON.parse(data);
          resolve(projects);
        } else {
          reject(new Error(`Failed to fetch Vercel projects: ${res.statusCode} ${res.statusMessage}`));
        }
      });
    });

    req.on('error', (err) => {
      reject(err);
    });

    req.end();
  });
}

/**
 * downloadFile - download the file
 * @param {*} url aWS s3 objects presigned URL
 * @param {*} destination local path location
 */
async function downloadFile(url, destination) {
  const response = await axios.get(url, { responseType: 'arraybuffer' });
  fs.writeFileSync(destination, Buffer.from(response.data, 'binary'));
}

/**
 * downloadFiles - download the files
 * @param {*} urls aws s3 objects presigned URL
 */
async function downloadFiles(urls) {
  for (const url of urls) {
    console.log(url);
    let fileName = path.basename(url.split('?')[0]);
    const temporaryDirectory = os.tmpdir();
    console.log("temporaryDirectory", temporaryDirectory);
    let dirName = `${temporaryDirectory}/${url.split('?')[0].split("amazonaws.com/")[url.split('?')[0].split("amazonaws.com/").length-1]}`;
    dirName = dirName.replace(`/${fileName}`, '');
    path.resolve(dirName);
    try {
      fs.mkdirSync(dirName, { recursive: true });
    } catch(error) {
      console.log(error)
    }
    // if (!fs.existsSync(dirName)) {
    //   fs.mkdirSync(dirName, { recursive: true });
    // }
    console.log(fileName);
    const filePath = path.join(dirName, fileName);
    await downloadFile(url, filePath);
    console.log(`Downloaded: ${filePath}`);
  }
}

/**
 * createVercelProject - creates the vercel nextjs framework project
 * @param {*} token authentication token for the vercel account
 * @param {*} projectName vercel project name
 * @returns 
 */
async function createVercelProject(token, projectName) {
  const options = {
    hostname: 'api.vercel.com',
    path: '/v1/projects/?teamId=team_Y6iorVTQiUAff3HsrpWHZ6am',
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${token}`,
    },
  };

  return new Promise((resolve, reject) => {
    const req = https.request(options, (res) => {
      let data = '';

      res.on('data', (chunk) => {
        data += chunk;
      });

      res.on('end', () => {
        console.log(res.statusCode)
        console.log(data)
        if (res.statusCode === 200) {
          const { id } = JSON.parse(data);
          resolve(id);
        } else {
          resolve(undefined);
        }
      });
    });

    req.on('error', (err) => {
      reject(err);
    });

    req.write(JSON.stringify({ 
      name: projectName, 
      framework: "nextjs"
    }));

    req.end();
  });
}

// Function to set the Node.js version using Vercel API's set-build-env endpoint
function setNodeVersion(projectId, token) {
  const options = {
    hostname: 'api.vercel.com',
    path: `/v1/projects/${projectId}/?teamId=YOUR_VERCEL_TEAM_ID`,
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${token}`,
    },
  };

  const envData = {
    key: 'NODE_VERSION',
    value: '16',
    teamId: YOUR_VERCEL_TEAM_ID,
    scope: 'team',
  };

  const req = https.request(options, (res) => {
    let data = '';

    res.on('data', (chunk) => {
      data += chunk;
    });

    res.on('end', () => {
      console.log(data)
      console.log('Node.js version set successfully:', JSON.parse(data));
    });
  });

  req.on('error', (err) => {
    console.log(err)
    console.error('Error setting Node.js version:', err.message);
  });

  req.write(JSON.stringify(envData)); // Send the environment variable data in the request body
  req.end();
}

Serverless.yaml

service: deploy-nextjs-app-vercel
frameworkVersion: "3"

provider:
  name: aws
  runtime: nodejs18.x
  region: YOUR_AWS_REGION

functions:
  deployNextJsAppToVercel:
    handler: handler.deployNextJsAppToVercel
    timeout: 900

Conclusion:

This blog showcases an efficient solution for deploying Next.js applications from AWS S3 to Vercel, utilizing AWS Lambda for automation. By leveraging serverless computing and a clever use of a sentinel file for deployment triggers, this architecture simplifies the deployment process, allowing developers to focus on development rather than operational complexities. The comprehensive steps, which include setup and script examples, showcase the seamless integration between AWS S3 and Vercel, providing a streamlined pipeline that boosts productivity, minimizes errors, and guarantees the swift and reliable deployment of applications. This approach not only optimizes web application deployment workflows but also underscores the transformative potential of serverless computing in modern web development.

Github URL: https://github.com/GowrisankarK/deploy-nextjs-apps-into-vercel-from-aws-s3

Gowri Sankar, Krishnamoorthy

+ posts

Anu Ranjana M

+ posts

Cookie	Duration	Description
_GRECAPTCHA	5 months 27 days	This cookie is set by Google. In addition to certain standard Google cookies, reCAPTCHA sets a necessary cookie (_GRECAPTCHA) when executed for the purpose of providing its risk analysis.
cookielawinfo-checbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other".
cookielawinfo-checkbox-advertisement	1 year	Set by the GDPR Cookie Consent plugin, this cookie is used to record the user consent for the cookies in the "Advertisement" category.
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.

Cookie	Duration	Description
bcookie	2 years	LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser ID.
lang	session	This cookie is used to store the language preferences of a user to serve up content in that stored language the next time user visit the website.
lidc	1 day	LinkedIn sets the lidc cookie to facilitate data center selection.
Zoominfo	session	Zoominfo uses technologies to collect and store information when you interact with services it offer to their partners, such as advertising services or analytics. All of those processes are meant to improve your user experience and the overall quality of our services.

Cookie	Duration	Description
_ga	2 years	The _ga cookie, installed by Google Analytics, calculates visitor, session and campaign data and also keeps track of site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognize unique visitors.
_gat_gtag_UA_111355416_1	1 minute	Set by Google to distinguish users.
_gid	1 day	Installed by Google Analytics, _gid cookie stores information on how visitors use a website, while also creating an analytics report of the website's performance. Some of the data that are collected include the number of visitors, their source, and the pages they visit anonymously.
_hjAbsoluteSessionInProgress	30 minutes	This cookie is used to detect the first pageview session of a user. This is a True/False flag set by the cookie.
_hjFirstSeen	30 minutes	This is set by Hotjar to identify a new user’s first session. It stores a true/false value, indicating whether this was the first time Hotjar saw this user. It is used by Recording filters to identify new user sessions.
_hjid	1 year	This is a Hotjar cookie that is set when the customer first lands on a page using the Hotjar script.
_hjIncludedInPageviewSample	2 minutes	This cookie is set to let Hotjar know whether the user is included in the data sampling defined by site's pageview limit.
_hjIncludedInSessionSample	2 minutes	This cookie is set to let Hotjar know whether the user is included in the data sampling defined by site's daily session limit.
_hjTLDTest	session	Hotjar test cookie to check the most generic cookie path it should use, instead of the page hostname. This is done so that cookies can be shared across subdomains (where applicable). To determine this, we store the _hjTLDTest cookie for different URL substring alternatives until it fails. After this check, the cookie is removed.
oktgid	1 year	This cookie is used for storing the visitor ID of the user who clicked on an okt.to link.
oktsid	session	This cookie is used for storing the session ID of the user who clicked on an okt.to link.

Cookie	Duration	Description
bscookie	2 years	This cookie is a browser ID cookie set by Linked share Buttons and ad tags.
personalization_id	2 years	Twitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.
VISITOR_INFO1_LIVE	5 months 27 days	A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.
YSC	session	YSC cookie is set by YouTube and is used to track the views of embedded videos on YouTube pages.

Cookie	Duration	Description
__gwtCookieCheck	session	This cookie is used to check if the visitors' browser supports cookies.
AnalyticsSyncHistory	1 month	These cookies are used to deliver advertisements more relevant to you and your interests. They are also used to limit the number of times you see an advertisement as well as help measure the effectiveness of the advertising campaign. They remember that you have visited a website and this information is shared with other organizations such as advertisers.
li_gc	2 years	These cookies are used to deliver advertisements more relevant to you and your interests. They are also used to limit the number of times you see an advertisement as well as help measure the effectiveness of the advertising campaign. They remember that you have visited a website and this information is shared with other organizations such as advertisers.
UserMatchHistory	1 month	LinkedIn - Used to track visitors on multiple websites, in order to present relevant advertisement based on the visitor's preferences.