Development update #9
A lot of work has been put into integrating Lambda in the last couple of weeks. Our developer Jorn Fiks will share why Lambda is important for us, how it's implemented and some key takeaways:
AWS & EC2…
As been highlighted in previous development updates, our platform is currently being hosted on AWS. In specific, we are using Amazon EC2 (Elastic Compute Cloud). Which means, you are paying for pre-purchased units of capacity, which makes this kind of architecture fairly expensive and not extremely scalable.
Why is serverless good?
One of the latest development in the cloud computing industry is serverless technology. An important factor of that is infinite scaling. Once an event is being triggered (for example when a user tries to log in), a container will be initialized, which executes some code and after that task is completed, the container will be closed. Hence the reason it’s called serverless, as you don’t need the traditional always-on server. Another very important feature of the serverless architecture is that the allocation of machine resources are being managed dynamically. You only pay for the time that the application is actively consuming resources.
Lambda
With this knowledge, we wanted to change from our current architecture using EC2 to the serverless feature of AWS called Lambda. For the past few weeks we have been working on setting up the migration to Lambda. Since the serverless technology is fairly new, there isn’t a lot of documentation yet. Setting up everything for the migration has proven to be quite challenging, but very interesting.
Frameworks
One of the major decisions we had to make was the framework that we were going to use. The decision was between using AWS SAM (Serverless Application Model) or Serverless.
We’ve tested both frameworks, and both obviously have their pros and cons. An example of this is that SAM has a better packaging management system, but the local development features of the Serverless framework are better. A few of the other services we have tested with both frameworks are SQS (Simple Queue Service, used by NASA amongst others) and SES (Simple Email Service, used by HBO amongst others). For now we have chosen to use the Serverless Framework. We will track the development process closely, and switch to AWS SAM if needed.
Configuration of the framework
In our configuration, using the serverless framework, we are using a few plugins:
- Serverless-offline
- Serverless-offline-sns
- Serverless-websockets-plugin
The serverless-offline plugins are used for local development and the web sockets plugin is used to replace the current web sockets that we are using. We have also setup a SQS queue, so that we are able to write and read messages from a remote queue, an SES instance for sending emails and an SNS (Simple Notification Service, again used by NASA amongst others) instance which will handle all of our notifications.
We are also using Lambda layers to include specific dependencies needed for specific Lambda functions.
Conclusion: we are very happy that we have started with the migration to Lambda. The more we get to know the serverless architecture, the more convinced we are that this is the future of cloud computing. A truly scalable platform for lower costs will not only benefit us, it will benefit all users and the whole community in the end.
Bug Bounty
In the last few weeks we have received excellent new feedback! Great bugs have been added. Some of the new bugs were added because of new users, but a lot were added as a "by product" from rigorous testing from our Usability Group members. Many thanks again for your input guys!
Usability Group
Our Usability Group had a first test bite to get a sense of how to work together in order to improve the platform.
The current output of that session is not yet related to a single feature, but was just generally about the whole feel of the platform. To get a sense of what has been proposed so far by the Usability Group members:
Keep in mind, focussed tests have not taken place yet. More about that in a future update…
