TL;DR Links
If anyone else is interested, I’ve written an alternative to this stack in CloudFormation which is deployed via AWS ECS (through either SPOT and ON-DEMAND Fargate containers) and supports all relevant micro-services.
It has been tested alongside Performance Monitoring on a platform with 5 different environments which generates on average about 5k events per hour using just t2.* instance classes for RDS/Redis/Kafka.
Why use it ?
Because the current “working” alternatives are:
- docker-compose
- HELM chart
Both of these solutions though have some downsides, specifically:
- Scaling ingestion of events is a bit hard due to the hard capacity limits of both solutions
- It is a well known fact that database systems perform better on NON-docker infrastructure points
- Keeping up with the different changes in versions is usually a hassle
- Customizing the different bits and pieces such as integrations require a lot of man hours
Description of deployed infrastructure
For those of you who use Amazon Web Services as their preferred cloud provider, we came in your help with a fully scalable, easy to maintain and secure infrastructure based on the following AWS services:
- AWS ECS Fargate
- AWS RDS
- AWS ElastiCache
- AWS MSK (Kafka)
- AWS OpsWorks
- AWS VPC
- AWS CloudWatch
Here’s how a diagram of the deployed infrastructure looks like:
You can deploy it by following these simple steps:
- Go to AWS CloudFormation
- Hit “New stack”
- Upload the template (check the GitHub repo)
- Modify parameters to fit your needs (if needed)
- Hit “Create stack”