About the Client
The client is an American global music corporation, the world’s leading music company. It owns and operates a broad array of businesses engaged in recorded music, music publishing, merchandising, and audiovisual content in more than 60 countries.
- Several third-party applications across the enterprise were used across the security layer
- Lack of a single application that can exactly match the customers' security requirement across the enterprise level
- Multiple 3rd party applications lead to underutilization and higher TCO
- Need for cost-effective and long-term application that can fulfill the enterprise-level security requirements
An application was developed featuring Auth 2.0/ JAVA API/ AWS DynamoDB was developed
DynamoDB was preferred as the Database as there is a need for a higher level of Data durability and linear scalability nature
Leveraged DAX in-memory acceleration to lower the required DynamoDB throughput capacity for a given application that is read-heavy
- Auth 2.0/ JAVA API/ AWS DynamoDB based application
- DynamoDB was used as the database
- DAX in-memory acceleration is used to lower the required DynamoDB throughput capacity for a given application that is read-heavy
- CI/ CD pipeline is set up by the DevOps team, and this solution uses GitHub for source code repository
- Jenkins dockerized application is responsible for building and deploying (Spinnaker used to deploy) in the dev environment
- Development activities and QA (Functional testing, Regression testing, Locust load testing, etc.) are performed in the dev environment
- QA Certified dev code is promoted to UAT testing
- Spinnaker pipeline application acts as a promoting tool and is used to Deploy in UAT and Prod (once after UAT is signed off)
To fulfill SocialHi’5 need for a client self-service portal that was also easy to maintain, Agilisium’s 5-member expert team built a custom web application with a heavy focus on the visualization of campaign outcomes. They also developed in parallel a DevOps process to maintain, scale and operate this portal.
Web Application Architecture
A variety of AWS services and some open source technologies were used to build and run the web application. The web layer used the PHP framework, included a login and authentication system, and used AWS QuickSight to render its outcome dashboards.
The app layer was built on Python, and the backend services were run on Elastic Container Service (ECS) dockers with Auto Scaling and Auto Load Balancing (ALB) to ensure high availability of the portal. The database was run in a private subnet and used RDS MySQL as the database service.
As mentioned earlier, SocialHi5 necessitated that the solution offered was easy to maintain, scale, and operate. To that end, Agilisium’s DevOps engineers developed a 2-part DevOps process focusing on
- CI/CD for web application development
- Infrastructure Provisioning for maintenance.
Continuous Integration/Continuous Deployment (CI/CD Process)
All application (Web & App Tier) maintenance was articulated via AWS’s Code Pipeline. AWS’s Code Commit, Code Deploy, and Code Build services were invoked to automate the enhancement and maintenance of the self-service portal.
CI/CD Process Flow: Web Tier
CI/CD Process Flow: Web Tier
All infrastructure was hosted on an exclusive SocialHi5 Virtual Private Cloud (VPC), to add an extra layer of confidentiality. AWS CloudFormation templates were used to spin up and maintain a host of AWS services utilized for the self-service portal.
Serverless Web application hosting: EC2, ECS, RDS, S3, SSM, VPC, NAT Gateway, ALB with Autoscaling Group, LAMBDA, Certificate Manager, Route53 were some of the services used to get the portal live.
Security: Web Application Firewall (WAF) was used with Cross-site scripting, Geo match, and SQL injection rules to protect from common cyber threats in conjunction with the AWS inspector service.
Monitoring and Logging: CloudWatch, OpsWorks, Config & Inspector services were also invoked to cover configuration management, logging, and monitoring of the application and infrastructure.
- Amazon CloudWatch metrics are enabled to determine the health of each component of the workload. All the metrics are monitored by the Grafana tool
- Prometheus - used to collect the metrics from POD, node (backend)
- Grafana - used to display the content (frontend)
- Kiali - used to live network monitoring
- Splunk / Loki - log monitoring
- IAM best practices and principles are followed
- Alert mechanisms are set so that anything that is not in compliance is immediately notified
- SSL certificate is installed on the Classic Load Balancer
- All Data stores are in private subnets
AWS services used:
- Amazon DynamoDB
- AWS Identity & Access Management
- Amazon OpenSearch Service
- Amazon Simple Queue Service
- AWS CloudFormation
- AWS CloudTrail
- Amazon CloudWatch
- Amazon Kubernetes
- Elastic Load Balancing (ELB)
- AWS Lambda
- The solution developed has a long-term road map, where the third-party security applications will be sunset
- One application has gone live with this security platform, and 10+ applications will be migrated into this platform in 2021
- DynamoDB with DAX use case provided extremely high performance with a response time measured in microseconds
- DynamoDB with DAX reduced the latency by 80% i.e. from 10s to 1-2 s for an endpoint