The subscription first growth strategy, aimed at converting casual listeners to subscribers, has its own challenges. A key challenge is how to acquire Gen X, world’s first true digital natives, and turn them into fans with shrinking marketing budgets. The fact that avid fans who typically represent 10% to 20% of a franchise’s user base, drive above 80% of the business value2 makes marketing intelligence the key business driver.
Universal Music Group (UMG) faced challenges in gaining timely insights from mountains of data shared by its distribution partners. The existing system wouldn’t scale up and process data faster even with additional investments. UMG was looking out for cost-efficient new-age solution that would process data faster, provide insights at speed of thought, and enhance business agility.
- The Existing system on traditional stack couldn’t scale up to the exponential increase in data volume from streaming partners (Spotify, Apple, YouTube etc.). This had a proportionate impact on all downstream processes and delayed key tactical business decisions.
- Deep dive analyses such as Sales as of LYSD (Last year same day) was impossible; due to the sheer data volume, longer data retrieval and processing time of the existing system.
- Potential licensing cost of servers/tools shot up as data from streaming partners skyrocketed. UMG was looking out for cost efficient, scalable solution that does not undermine speed and business agility.
Agilisium devised a cloud based, elastically scalable architecture that offers faster analytics and business agility in a cost-efficient manner.
- Custom Java, python scripts were used to retrieve data from FTP servers, where streaming partners share their data.
- AWS Elastic MapReduce (EMR) was used to scale out data processing across nodes, and store processed data in AWS S3 storage.
- A data workflow was orchestrated to automatically move data from S3 into AWS Redshift using Data pipeline.
- Around existing 300 MicroStrategy reports were integrated to use AWS Redshift
- Redshift enabled the UMG Reporting & Analytics users to access processed data for reporting & analytics need on large volume of Data.
- Qubole enabled UMG analysts to query the raw data as needed for deeper Analytics, leveraging Data Lake built by Agilisium in AWS environment.
- Furthermore, the integration of sales and financial data of physical music format were accomplished by Snaplogic, with Redshift as the storage layer. This way the customer achieved complete digital transformation of their technology landscape with cloud-based scalable architecture.
Technologies Used: Java, Python, AWS EMR, Snaplogic, AWS S3 Redshift, Qubole, Microstrategy
Team Size: 2 SMEs, 3 Architects, 5 Senior developers, 5 Developers
Delivery model: Hybrid
Project Duration: 8 months
Project Governance: Agile delivery governed by Joint Steering Committee, Daily Scrum, Weekly Status Reports (WSRs), and Weekly Informative Dashboards.
Redshift Cluster details:
6 Node Redshift Cluster
96 TB Storage
250+ million records/day
- 5x reduction in data processing time enabled Universal Music Group understand consumption pattern & affinity to decide where to focus & invest ad dollars.
- 5x reduction in data cleansing, if erroneous data from source system was loaded, enhanced the business agility.
- Deep dives using historical data (not possible in older system) made possible