Tracking consumer sentiment across local APAC languages

Context

Riot Games publishes its titles across eight markets in Asia Pacific. These are Vietnam, Taiwan, Indonesia, Malaysia, Singapore, Japan, India and South Asia, and Australia and New Zealand. Players in these markets discuss the games on local forums and social platforms. Commercial sentiment tracking tools did not index most of these sources and did not read the local languages accurately.

The publishing teams therefore had no dependable measure of how players felt. Information reached leadership second hand. A regional manager might notice a complaint thread and raise it in a meeting, usually once the issue had already affected the business.

Mandate

Build a sentiment measure that the Asia Pacific publishing organisation could use in its regular planning. The measure had to cover the forums players actually used and read the local languages accurately.

It also had to arrive on a fixed weekly schedule, so that teams could review it as a matter of routine rather than only when something went wrong.

Approach

I wrote a scraper in Python to collect posts from the forums and social platforms that players used in each market. Local-language sources were weighted more heavily than English ones. What the scraper collected and retained was limited to what copyright and privacy rules allowed.

Each post was then scored by a multilingual transformer model covering Traditional Chinese, Tagalog, Bahasa Indonesia and the other languages in scope. The standard version of the model handled regional slang, local symbols and sarcasm poorly. I worked with the publishing teams in each market to test its accuracy against native speakers and fine-tune it on the terms that were actually in use.

The scored posts were grouped into recurring themes such as patch reception, character and skin releases, matchmaking and monetisation. These were aggregated into a weekly report that a general manager could read in a few minutes.