Migration to Serverless

Recently I made the decision to migrate Runthecomp to use serverless technology. Runthecomp is a platform to streamline the organisation of media competitions - including application submission, jury management and winner selection. The migration project was driven by a few factors:

• Reduce costs
• Ease server management
• Increase robustness of platform
• Simplify management of project

The architecture of the project was a number of Node.JS applications - 2 React application served via Express & 2 APIs built using Express and backed by MongoDB. Each application was a separate github repository containing all the code, tests and the details of deployment to an EC2 server instance respectively. The MongoDB was installed and configured manually on a separate EC2 instance.

The first step was to identity a target architecture. I quickly settled on using Mongo Atlas as a replacement for the manually EC2 install. Mongo atlas comes with a free tier which is suitably scaled for my needs. Next up I decided to move the Express served React applications should be served via AWS S3 (this blog is served via S3 so I has some experience). For the backend Express based REST api I choose the popular Serverless framework as it has excellent offline support - this was particularly important as I had a lot of integration tests. Lastly the project was using AWS Cognito for Authentication, however the various pages for user interactions where self hosted within the application, I wanted to migrate to the Cognito hosted authentication forms.

In summary the target architecture was:

Mongo Atlas for Database
AWS S3 for Front End React Application hosting
Serverless framework for backend API
AWS Cognito for user authentication

Multiple repos vs a Monorepo

Working with a recent client I have become a fan of using Monorepos as a way to manage an application that is made up of multiple components. Traditionally the source code for each component is held in a separate repository and separate build pipelines are setup to test and deploy each component separately. The difficulties of this became apparent with my client as management of the separate components became difficult as there was no clear boundaries between the components. Moving to a monorepo allowed us to simplify the setup, development, build and release of the product so I was keen to take these new learnings into this migration project.

Database

As I was already using Mongo this set was trivial and was simply signing up to Mongo Atlas and changing the configuration for my application to point to the new hosted version on Mongo atlas.

Front End Hosting

My domain as purchased with AWS Route53, to connect an S3 bucket to your Route53 domain then please follow this article, this article also includes details on how to turn your bucket into a static file host.

Once the bucket is up and running I used aws cli to push files to the bucket,

1

aws s3 cp build s3://<bucket-name> --recursive

API

To simplify the migration I chose to create one function which was a proxy function to the Express application. This meant the Express applications did not need to be modified to fit into the new serverless architecture. To set up a proxy function I used the following in the serverless.yml

1
2
3
4
5
6
7
8

functions:
  api:
    handler: appServerless.handler
    events:
      - http: ANY /
        cors: true
      - http: 'ANY {proxy+}'
        cors: true

To convert the Express application to respond to AWS events I used the very handy serverless-http module. With just a few lines of code my Express application was converted to something that could be run as a Lambda responding to API gateway events - very cool! I was able to test this all on my Laptop using the serverless-offline plugin.

Outcome

The cost savings moving to this architecture have been dramatic, it used to cost me £40 per month to run which is now down to 50p. The stability of the system is also much higher, I am no longer receiving calls that the system is unavailable. I believe this is in part due to the nature of lambdas being spun up and down on demand as opposed to the long running stateful EC2 servers. Lastly the monorepo approach has allowed me to have a clearer picture of the full application as well as manage deployments from a simple script. This script will deploy all components at the same time allowing me to easily return to the project should I need to fix bugs or add new features.

I plan on publishing a template on github which showcases the set up and will update this article when it is complete.