Implementing long running ETL deployments with AWS CloudFormation

Enabling Devops in RSAC


15 minutes
of Infrastructure deployment process reduced
of deployment time potentially reduced
Faster ETL job deploy and execution process
Significant time saved for IT team due to automation
Allows for better planning and execution

Client Profile

Reliance Steel & Aluminum Co. (RSAC) is one of the largest metal services provider in North America. The company provides value added metals processing services and distributes a full line of over 100,000 metal products to more than 125,000 customers across various industries. RSAC is a Fortune 500

Business challenges

Agilisium Solution

  • Automated Infrastructure deployment using AWS Cloud Formation Templates.​
  • Automated High Availability to failover onto a different AWS region using Cloud Formation Template.​
  • Automated Application deployment using AWS AirFlow scheduler. Spark based ETL jobs triggered by AirFlow as and when needed.​
  • EMR Spark based framework developed to run a distributed application environment.​
  • All Source code repositories managed using Gitlab.​
  • AWS Autoscaling features implemented to scale out the application using CloudWatch metrics.​
  • Ansible playbook to manage and ease the deployment process.​

Tech-stacks Used

AWS – S3
AWS Cloudwatch

Business outcome

  • Infrastructure deployment process reduced to 15 minutes, potentially reduction of 98% time when compared to manual deployment.​
  • Faster ETL job deploy and execution process. ​
  • DevOps culture encourages collaboration & eliminates silos.​
  • Significant time saved for IT team due to automation.​
  • Allows for better planning and execution.​
  • Lesser resources required. Developers can now focus on core activities.
Get the Most Out of Embedded Analytics with Amazon Opensearch 
From data rich visual analytics to interactive dashboards that make data fetching a piece of cake, Team Agilisium is with you, every step of the way. 
Talk to Us
Got a question? Don’t hesitate to give us a call today or shoot us an email. 
Please enter a business email
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.