Introduction

A collaborative social media platform with a large user base faced performance and cost issues due to its complex Infrastructure. The company deployed the application on Amazon ECS as containers, while the database was hosted on RDS in two separate AWS regions for staging and production. However, this resulted in latency issues and distrust in the deployment process. Additionally, maintaining the infrastructure in both regions was expensive, and the company needed to reduce costs.

Challenge

Consolidating infrastructure and simplifying deployment can be challenging, requiring careful planning to avoid data loss and downtime during migration. Managing a self-managed Kubernetes cluster can also be complex,  and incurring additional costs. Consolidating environments into one region can lead to a single point of failure, highlighting the importance of disaster recovery and business continuity plans. Despite these challenges, a well-planned strategy can result in significant cost reductions and other benefits.

Solution

We approached this problem by consolidating its infrastructure and automating its deployment process. We suggested and adopted the following steps to achieve its objective:

Migration to Kubernetes: We migrated its application to a single self-managed Kubernetes cluster in one AWS region, resulting in easier automation, simplified infrastructure, and reduced ongoing maintenance.

Consolidation of Environments : To address their cost concerns, We have implemented two distinct infrastructure changes. First, consolidated their two Amazon Elastic File System (EFS) instances into a single region. Second, consolidated their two Amazon Relational Database Service (RDS) servers into one. Additionally, they moved all of their applications to a single self-managed Kubernetes cluster within the same region.

Implementation of CI/CD: The company had an existing CI pipeline in Jenkins that was causing a lot of problems, including being injected with crypto malware and requiring new user additions every time the dev team changed. To solve these issues, We have moved the pipeline to Drone CI, which provided container-based pipelines and Github-based login access. This move improved reliability and consistency, resulting in reduced downtime and maintenance costs. CI/CD alerting was also added to Slack, providing developers with immediate notifications of any pipeline issues. To further enhance the infrastructure, namespaces were utilized to separate the staging and production environments within a single Kubernetes cluster.

Technologies

Terraform
Kubernetes
Docker
Drone Ci
Amazon EC2
Amazon Elastic File System
Amazon RDS

Results

The implementation of the above steps resulted in the following benefits for Company:

Reduced Monthly Recurring Cost : As a result of consolidating infrastructure and simplifying deployment, a company was able to reduce their monthly recurring infrastructure cost by approximately 70%.

Zero down time in migration : During the migration process, one of the challenges that Company faced was ensuring zero downtime as they had a large user base. However, we were able to achieve this through careful planning and execution during the database replication phase.

Streamlined Infrastructure through Automation : By consolidating their infrastructure and migrating to Kubernetes, the company was able to simplify its processes and focus on its core business. This automation and simplification led to a significant reduction in ongoing maintenance costs and improved reliability of their deployment process.