
Overview
An American video game developer based in Los Angeles, California, faced streaming data challenges for their multiplayer online battle arena game. This hugely popular game serves millions of players month on month. The existing data ecosystem failed to keep up with the growing data storage and data analytics demands of the game. Due to its expertise in Data and Analytics space with niche solutioning capabilities in AI&ML, the client approached Agilisium to eliminate data processing bottlenecks and achieve faster streaming data analytics.
The Challenge
The existing data ecosystem was stored in Vertica (an SQL database) and analyzed for insights via Databricks. However, this system not actively maintained, which translated into data loss and multi-day data delays. Leading to high levels of complexity in the Analytics team’s ability to assess patch outcomes. Ability to maintain, monitor or derive the quality of player knowledge, was impacted.
Our Solution
- A Gaming session data was pushed to Kinesis Data Streams through 7 streams via Kinesis Agent. The data was converted into JSON and loaded into S3, for immediate raw data availability.
- The S3 gaming session data is branched out for batch and real-time analytics.
- Batch: S3 gaming session data was cataloged using Glue crawlers and stored in Athena. This data was transformed and stored in Redshift for downstream BI reports using Lambda.
- Real-time: S3 gaming session data was processed by Kinesis Analytics, in real-time, and pushed downstream via Kinesis Firehose into Redshift. This processing was repeated at 5-minute intervals for near real-time analytics.
- Near real-time availability of Raw data for the purpose-built Spark recommendation engine
- Data produced in multiple regions was seamlessly replicated to a central region.