Knowledge AI for Oracle NPC in MMORPG

Summary

• The client is developing a massively multiplayer online role-playing game (MMORPG) set in a vast, immersive fantasy world. This world is rich in lore, characters, quests, and items, offering players a deep and engaging gameplay experience. The game boasts a dynamic environment where players can interact, form alliances, and embark on epic adventures.

• The client envisioned an innovative feature for their MMORPG: an Oracle Non-Player Character (NPC) that players could approach and consult. This Oracle would serve as an in-game knowledge base, answering players' questions about the game's lore, mechanics, or personal progress. Notably, the Oracle's knowledge would be gated, revealing information only in line with a player's in-game achievements and progress.

• A sophisticated knowledge system was implemented to bring the Oracle NPC to life. At its core, the method utilized the LLaMa 2 as the primary LLM. Milvus, a robust vector database, managed and retrieved vast amounts of in-game data. The embedding of this data for efficient querying was achieved using the SetFit algorithm, ensuring that players received accurate and contextually relevant answers from the Oracle.

Tech Challenge

• Scale: MMORPGs, by nature, cater to many players simultaneously. The Oracle NPC, being a unique feature, was expected to be frequently consulted by thousands of players. This meant the system had to handle a massive volume of queries in real-time, ensuring each player received accurate and timely responses without any noticeable lag.

• Latency: Even a slight delay can disrupt the immersive experience in gaming. The challenge was not just to provide accurate answers but to do so with minimal latency. Integrating an LLM like LLaMa 2 with the game's infrastructure while ensuring that players' response time remained imperceptible was a significant hurdle.

• Personalized Knowledge: The Oracle's knowledge had to be adaptive, revealing information based on a player's progress. This meant the system couldn't simply provide generic answers. It had to recognize each player's achievements, unlocked content, and current game status, tailoring its responses accordingly.

• Privacy: While cloud-based solutions are prevalent, the client wanted to host their LLMs to ensure better control, customization, and data security. This introduced challenges related to infrastructure, maintenance, and seamless integration with the game's existing systems.

• Patches: MMORPGs are dynamic, with new content, quests, and lore frequently added. The Oracle NPC had to be designed to quickly assimilate this further information, ensuring its knowledge base remained up-to-date and relevant without requiring extensive manual intervention.

Solution

• The system was primarily built using Rust to handle the immense scale and volume of simultaneous player queries. This programming language ensured speed, scale, and thread safety, offering a significantly more efficient performance than traditional Python-based solutions. Rust's memory safety features ensured the system remained stable despite heavy loads.

• The latency challenge was met head-on by leveraging GGML for efficient LLM inference and quantization. This ensured Oracle's responses were accurate and delivered with imperceptible delay, maintaining the game's immersive experience.

• To make Oracle's knowledge adaptive and tailored to each player's progress and to have up-to-date info on new patches, we've built several data pipelines to transform and feed the data into Milvus and FAISS. These technologies facilitated efficient knowledge storage and retrieval, allowing the system to quickly access relevant information based on a player's achievements and game status. Also, additional metadata-based tweaks were done to efficiently answer SQL-like questions (e.g., counts, time period-based info, comparisons, etc.).

• The decision to host their LLMs introduced infrastructure challenges. However, we've achieved dynamic load balancing and scaling by utilizing Kubernetes (EKS) with AWS. This ensured the system could handle varying loads, from quiet times to peak player activity, without any hitches.

• Fine-tuning LLaMa 2 with Python & PyTorch: The Oracle's unique voice and integration of game lore were achieved by fine-tuning LLaMa 2 using the HuggingFace toolset powered by PyTorch. The instruct dataset gave the Oracle a specific style, seamlessly weaving the game's lore into the LLaMa 2 weights. Additionally, SetFit was trained to optimize the embedding space, ensuring Oracle's knowledge remained relevant and contextually accurate.

Impact

• The introduction of the Oracle NPC transformed the MMORPG's gameplay dynamics. Players now had a unique, interactive knowledge base, deepening their engagement and immersion in the game's expansive world.

• With Oracle's ability to provide tailored information based on individual player progress, gamers experienced a personalized journey. This dynamic adaptation enriched the gameplay and reduced the need for external game guides or forums, keeping players within the game environment.

• The robust technological backbone ensured that Oracle NPC could cater to many players simultaneously without latency issues. Even during peak times, this seamless performance solidified MMORPG's reputation for innovation and technical excellence, attracting even more players.