Case Study
Enabling quicker Marketing avenue spending decisions with Agilisium’s AWS Big Data solution for Universal Music Group
Digital is now responsible for exactly 50% of all music “sales”, a new high for the global music industry. Within that sector, streaming is now responsible for 59% of all digital revenues, and that figure will continue to rise. – IFPI Global Music Report 2017
The shift from physical to digital, downloads to streaming, ownership to access in the global music industry not only changed consumption and purchase habits but also brought in a host of new challenges and opportunities. To stay relevant, the global music industry shifted towards a subscription-based, rental services model, which helped it rebound after two decades of decline.

The subscription first growth strategy, aimed at converting casual listeners to subscribers, has its own challenges. A key challenge is how to acquire Gen X, world’s first true digital natives, and turn them into fans with shrinking marketing budgets. The fact that avid fans who typically represent 10% to 20% of a franchise’s user base, drive above 80% of the business value makes marketing intelligence the key business driver.

Universal Music Group (UMG) faced challenges in gaining timely insights from mountains of data shared by its distribution partners. The existing system woudn’t scale up and process data faster even with additional investments. UMG was looking out for cost-efficient new-age solution that would process data faster, provide insights at speed of thought, and enhance business agility.

The Challenge

  • Existing system couldn’t scale up to the exponential increase in data volume from streaming partners (Spotify, Apple, YouTube etc.). This had a proportionate impact on all downstream processes and delayed key tactical business decisions.
  • Deep dive analyses such as Sales as of LYSD (Last year same day) was impossible; due to the sheer data volume, longer data retrieval and processing time of the existing system.
  • Potential licensing cost of servers/tools shot up as data from streaming partners skyrocketed. UMG was looking out for cost efficient, scalable solution that does not undermine speed and business agility.

Our Solution

Agilisium devised a cloud based, elastically scalable architecture that offers faster analytics and business agility in a cost-efficient manner.

  • Custom Java, python scripts were used to retrieve data from FTP servers, where streaming partners share their data.
  • AWS Elastic MapReduce (EMR) was used to scale out data processing across nodes, and store processed data in AWS S3 storage.
  • A data workflow was orchestrated to automatically move data from S3 into AWS Redshift using Data pipeline.
  • Around existing 300 MicroStrategy reports were integrated to use AWS Redshift
  • Redshift enabled the UMG Reporting & Analytics users to access processed data for reporting & analytics need on large volume of Data.
  • Qubole enabled UMG analysts to query the raw data as needed for deeper Analytics, leveraging Data Lake built by Agilisium in AWS environment.

Key Highlights

Technologies used – Java, Python, AWS S3, AWS EMR, Qubole, Redshift

Team size – 2 SMEs, 3 Architects, 5 Senior developers, 5 Developers

Redshift Cluster – 6 Node Redshift Cluster

Data size – Total 96 TB / 250+ million records/day

Project duration – 8 months

Project Governance – Agile delivery governed by Joint Steering Committee, Daily Scrum, Weekly Status Reports (WSRs), and Weekly Informative Dashboards.

Delivery model – Hybrid

Results and Benefits
  • 4.5x reduction in data processing time enabled Universal Music Group understand consumption pattern & affinity to decide where to focus & invest ad dollars.
  • 5x reduction in data cleansing, if erroneous data from source system was loaded, enhanced the business agility.
  • Deep dives using historical data (not possible in older system) made possible in a faster and cost-efficient manner in AWS.
  • The solution helped UMG gain insights at the speed of thought.