The subscription first growth strategy, aimed at converting casual listeners to subscribers, has its own challenges. A key challenge is how to acquire Gen X, world’s first true digital natives, and turn them into fans with shrinking marketing budgets. The fact that avid fans who typically represent 10% to 20% of a franchise’s user base, drive above 80% of the business value makes marketing intelligence the key business driver.
Universal Music Group (UMG) faced challenges in gaining timely insights from mountains of data shared by its distribution partners. The existing system woudn’t scale up and process data faster even with additional investments. UMG was looking out for cost-efficient new-age solution that would process data faster, provide insights at speed of thought, and enhance business agility.
- Existing system couldn’t scale up to the exponential increase in data volume from streaming partners (Spotify, Apple, YouTube etc.). This had a proportionate impact on all downstream processes and delayed key tactical business decisions.
- Deep dive analyses such as Sales as of LYSD (Last year same day) was impossible; due to the sheer data volume, longer data retrieval and processing time of the existing system.
- Potential licensing cost of servers/tools shot up as data from streaming partners skyrocketed. UMG was looking out for cost efficient, scalable solution that does not undermine speed and business agility.
Agilisium devised a cloud based, elastically scalable architecture that offers faster analytics and business agility in a cost-efficient manner.
- Custom Java, python scripts were used to retrieve data from FTP servers, where streaming partners share their data.
- AWS Elastic MapReduce (EMR) was used to scale out data processing across nodes, and store processed data in AWS S3 storage.
- A data workflow was orchestrated to automatically move data from S3 into AWS Redshift using Data pipeline.
- Around existing 300 MicroStrategy reports were integrated to use AWS Redshift
- Redshift enabled the UMG Reporting & Analytics users to access processed data for reporting & analytics need on large volume of Data.
- Qubole enabled UMG analysts to query the raw data as needed for deeper Analytics, leveraging Data Lake built by Agilisium in AWS environment.
Technologies used – Java, Python, AWS S3, AWS EMR, Qubole, Redshift
Team size – 2 SMEs, 3 Architects, 5 Senior developers, 5 Developers
Redshift Cluster – 6 Node Redshift Cluster
Data size – Total 96 TB / 250+ million records/day
Project duration – 8 months
Project Governance – Agile delivery governed by Joint Steering Committee, Daily Scrum, Weekly Status Reports (WSRs), and Weekly Informative Dashboards.
Delivery model – Hybrid
- 4.5x reduction in data processing time enabled Universal Music Group understand consumption pattern & affinity to decide where to focus & invest ad dollars.
- 5x reduction in data cleansing, if erroneous data from source system was loaded, enhanced the business agility.
- Deep dives using historical data (not possible in older system) made possible in a faster and cost-efficient manner in AWS.
- The solution helped UMG gain insights at the speed of thought.