#003 AI NPC (part 2)
(Part 2 of 3)
INTRO
In Part 1 of this series, we explored the fascinating evolution of AI NPCs in video games. We compared traditional branching-dialogue (state machine) solutions to the newer LLM-driven characters beginning to emerge today.
Here we delve deeper into advancements driving adaption, examine the potential impact of AI NPCs on gaming, showcase some of the games that are already pushing the boundaries and look at some of the challenges.
Conceptual Framework - The Motivation Axis
The full potential of AI NPCs is still being explored and likely will be for a long time. The human brain is the most complex structure known to exist. We know almost nothing about human consciousness and AGI (depending on who you ask) is still a ways away. I personally don't think LLMs (even with strong reasoning) will get us there and for more on LLM limitations and their insufficiency for AGI, I suggest checking out Yann Lecun on Lex Friedman HERE. But we dont need AGI to make great games.
Our games just need to be fun! They need to be compelling, engaging and delightful. When considering LLM-based NPC design, I find it can be useful to simplify all this into a conceptual framework I call the 'The Motivation Axis'.
‘‘The Motivation Axis' serves as a high-level design abstraction for each NPC 'type'. It’s a simple model built around two fundamental components: Gameplay and Freeform. This concept aligns with models like the Fogg Behavior Model, which emphasizes that behavior arises from a convergence of motivations.
By understanding our design intention and how the player can influence those motivations, we can construct a template for authoring these NPCs that is platform/technology/tool independent.
Gameplay - We can think of this as the 'purpose' of the NPC. Why do they exist from a game design perspective? As a game designer, what do you need this NPC FOR? It should have a reason to be in the game (even if it’s just a background crowd character to add ‘color’ to your world). A ‘quest giver’ is the penultimate ‘NPC with a purpose’ but we can go much further.
Freeform - A natural, open-ended and entirely creative dialogue. Linguists refer to the 'generative' function of language as "the capacity to produce an infinite number of sentences from a finite set of elements". This property distinguishes human language from animal communication systems, which typically lack such open-ended productivity. NOTE: In games, Freeform is usually highly constrained by many factors (memory, local knowledge, emotion etc.) but it is fundamentally generative. This NPC output is NOT goal driven (conceptually), but highly responsive to current stimuli (often topics introduced by the player).
Notice these two systems are in tension with each other as one is goal-driven and one is open-ended. If you consider your own conversations with other humans, they usually contain some aspects of each (and in real life they often overlap without rigid delineation). Any given AI NPC implementation will likely fall somewhere on the spectrum between these two extremes.
MAIN
The recent surge in AI NPC development can be attributed to several key factors:
LLM/NLP (duh!): Recent advancements in Natural Language Processing (NLP) and Large Language Models (LLMs) have significantly enhanced AI-driven Non-Player Characters (NPCs) in video games. As the cost/token drops, we as game developers will be even more free to utilize these technologies.
Emergence of Character Engines: These ‘platforms’ provide specialized tools and engines for creating AI-driven characters with unique personalities, emotions, and long-term memory. All of these platforms allow developers to create "character brains" with customizable personalities, emotional ranges, and autonomous decision-making capabilities. These characters can then be integrated into various game engines and worlds.
Inworld AI: Offers a production-ready framework for building AI agents with client-side logic and local model inference, optimized for real-time data ingestion, low latency, and massive scale.
Convai: Provides a platform for creating AI-driven conversational characters that can be seamlessly integrated into games, enhancing player immersion through natural language interactions.
Artificial Agency: Developed the "AI Engine" that enables NPCs to exhibit improvisational behaviors, allowing for dynamic responses to player actions and creating more lifelike and unpredictable game scenarios.
NVIDIA: Through its Avatar Cloud Engine (ACE), NVIDIA delivers cloud-native AI models and services to create realistic NPCs with advanced conversational abilities, leveraging its powerful GPU technology for real-time performance.
Charisma.ai: Specializes in crafting interactive narratives by enabling digital characters to engage in complex storytelling, utilizing emotion-driven dialogue systems to create deep and engaging player experiences.
Soul Machines: Pioneers in creating autonomously animated digital humans with hyper-realistic appearances and emotionally responsive behaviors, aiming to humanize NPC interactions in virtual environments.
Alethea AI: Introduces intelligent NFTs (iNFTs) that combine blockchain technology with AI, allowing for the creation of interactive and evolving NPCs that can own and trade assets within gaming ecosystems.
Replika: Offers AI companions designed for meaningful conversations, with a focus on personal growth and emotional support, which can be adapted to serve as empathetic NPCs in narrative-driven games.
GoodAI: Developed "AI People," a sandbox environment showcasing NPCs with advanced cognitive architectures, capable of learning and adapting to complex scenarios, pushing the boundaries of NPC intelligence.
Master of Code Global: Provides custom AI-powered conversational solutions, including chatbots and virtual assistants, tailored for seamless integration into gaming platforms to enhance player engagement through responsive dialogues.
Increased Computing Power: The availability of more powerful hardware, including GPUs and cloud computing, has enabled the development and deployment of more complex AI models for NPCs. There are exciting advances around small purpose-built hardware like Nvidia's Project DIGITS and even more optimized 'on device AI’ happening now.
Computer Vision: Computer vision (CV) algorithms allow NPCs to perceive and interpret their surroundings, leading to world-state awareness that mimics actual beings with visual senses. Computer vision relies on deep learning algorithms, particularly Convolutional Neural Networks (CNNs), to analyze and understand images or video. With this approach, NPCs are able to recognize objects, track movement, and react to visual cues, potentially improving their navigation and interaction with the game world. These systems are still relatively expensive and not widely used in games yet. ( Shameless plug, I would be remiss If I failed to mention my brother Adrian Kaehler wrote the seminal O’Reilly book on CV, grab a copy HERE. )
Games Utilizing AI NPCs
“AI” in a range of permutations has been around for a while. Lets look at a few games showcasing AI NPCs in interesting ways:
Current Games with AI NPCs
Red Dead Redemption 2: Launched in 2018, this game is renowned for its immersive open world where NPCs exhibit dynamic and believable behavior. One of my personal favorites. This AI doesn't use LLMs but is still a brilliant high-watermark IMHO.
Alien Isolation: The Xenomorph in this game is powered by AI, learning from its encounters with the player and adapting its behavior, creating a unique and terrifying experience. Uses machine-learning without LLMs.
Covert Protocol: This demo, co-released by Inworld and NVIDIA, showcases the potential of AI NPCs in interactive storytelling. Players interact with NPCs powered by Inworld AI, each with unique personalities and motivations, to solve a mystery.
Wol: This augmented reality experience from Niantic features an AI-powered owl guide named Wol who interacts with players and provides information about their real-world surroundings.
Vaudeville: An experimental whodunit game that utilizes advanced AI technologies to generate dialogues in real-time, offering a unique narrative-driven detective experience.
Herika - The ChatGPT Companion: A mod for The Elder Scrolls V: Skyrim introducing an AI-powered companion capable of engaging in open-ended conversations, enhancing player immersion.
Dr. Armitage Quests: my own free ChatGPT text adventure game that is undergoing constant revision. If you have paid OpenAI plan, check it out!
The HOWL Protocol: another game I built with the Cosmic Foundry team. Signup and checkout the super-cool prototype!
Upcoming Games with AI NPCs
Several upcoming games are also generating excitement for their use of AI NPCs:
NARAKA: BLADEPOINT: The PC version is set to introduce AI companions in 2025, allowing players to team up with AI-driven teammates that perceive, plan, and act like human players.
Roblox: It’s no secret the massively popular UGC platform is going ‘all-in’ on developer tools that leverage genAI, including a variety of LLM-based NPC implementations.
Mecha Break: A multiplayer mech battle game scheduled for release in 2025, featuring AI NPCs powered by advanced language models, enabling natural language interactions.
inZOI: An upcoming life simulation game by Krafton, aiming to incorporate AI-driven NPCs for more lifelike interactions.
Slothtopia: This social simulator utilizes Inworld AI to power its NPCs, creating a more dynamic and interactive world for players to explore.
Youmio: A web3 project centered around player-created AI NPCs.
While the future of AI NPCs is bright, there are both challenges and opportunities that game teams will need to address. As we often say here at Shogun, any given AI technology may or may not be appropriate for your game. Do not fall into the “oooh SHINY!” trap. A careful evaluation period is necessary before deciding to embark on implementation and we recommend a full review of the following topics (at minimum):
Challenges
Technical Complexity: Implementing sophisticated AI models for NPCs can be technically challenging, requiring expertise in areas like machine learning, NLP, and computer vision. Developers need to ensure that the AI models can integrate seamlessly with existing game engines and perform efficiently in real-time. Using some of the above-mentioned platforms that have engine integration can be huge boost here.
Control: When developers begin to work with LLMs, they can quickly become frustrated with the 'black box' nature of the technology. They sometimes seem to have a 'mind of their own' which is a double-edged sword. There are significant design challenges between leveraging the open nature of the tools with the desire to script/control/direct these experiences. This challenge is not unlike writing for interactive narrative experiences broadly but localized to a single NPC. My own experience on HOWL found that controlling the LLM (in particular, time-boxing it’s knowledge to 1919) was one of the biggest technical challenges.
Computational Costs: Running complex AI models can be computationally expensive, potentially impacting game performance and requiring optimization strategies. This can be particularly challenging for games designed to run on a wide range of hardware, from high-end PCs to mobile devices. Using different LLMs for different things (see Part 1) is a major tool for controlling this.
Ethical Considerations: Ensuring that AI NPCs behave in a fair and unbiased manner is crucial, as is addressing potential concerns about player manipulation and privacy. Developers need to be mindful of how AI NPCs might influence player choices, collect player data and mitigate harm. More on this in Part 3.
Opportunities
Enhanced Player Experience: AI NPCs have the potential to significantly enhance the player experience by creating more immersive, engaging, and personalized gameplay. Imagine an “unreliable narrator” that changes his/her guidance based on how they feel about the Player? Or stories that evolve based on gossip and rumor amongst NPCs and information warfare is the primary mechanic. There are endless possibilities.
Deeper Storytelling: AI-driven characters can contribute to richer and more dynamic narratives, where player choices have a meaningful impact on the story. This could lead to emergent narratives that unfold organically based on player actions and interactions with the world. We can see narrative authors sketching broad parameters (perhaps just major story beats or events) but the specifics allow for a more robust “possibility space” and customized telling of the broader story.
Increased Replayability: The ability of AI NPCs to adapt and evolve can lead to unique and unpredictable gameplay experiences, encouraging players to revisit games multiple times. For example, in "Alien Isolation," the AI-powered Xenomorph learns from its encounters with the player, making each playthrough a different experience.
New Game Mechanics: AI NPCs can pave the way for innovative game mechanics and genres, blurring the lines between traditional game categories. For example, AI could enable games where players form dynamic relationships with NPCs, with these relationships impacting the gameplay and even ‘out-of-game’ meta in exciting ways.
We'll deep dive into these and more use-cases in Part 3.
Potential Impact on the Gaming Industry
AI NPCs are likely a profound impact on the gaming industry in specific genres where they make sense. Nobody really knows where all this is going but a current survey of the landscape makes a few trends apparent. When we zoom out and consider the business side of game development we can see a few potential areas of impact:
Increased Development Costs: The development of sophisticated AI NPCs may require significant investment in research and development, potentially increasing game development costs. This could lead to a greater emphasis on high-budget AAA games while potentially making it more challenging for smaller indie developers to compete. Recent data from OpenAI (and the Deepseek R1 controversy) may indicate these costs will drop to near-zero within a few years.
New Revenue Streams: AI NPCs could open up new revenue streams, such as personalized in-game experiences or AI-powered companions that players can purchase. This could change the way games are monetized, with a greater focus on in-game purchases and microtransactions. Platforms like Character.ai are already seeing huge success here.
Changes in Game Design: The use of AI NPCs may lead to changes in game design philosophies, with a greater emphasis on dynamic storytelling and emergent gameplay. This could lead to games that are less reliant on pre-scripted events and more focused on player agency and interaction with the world and compelling game writers to become familiar with this approach.
Expansion of Gaming Audiences: AI NPCs could make games more accessible to new audiences, such as those who prefer more narrative-driven or social experiences. This could broaden the appeal of gaming and attract players who may not have been interested in traditional game genres. We can envision a kind of ‘gaming companion’ who plays with you when your friends are not available for example, perhaps even inviting you through SMS to play when they are ‘lonely’.
Conclusion
The birth of these AI NPCs can be both exciting and a bit daunting for developers. From a cynical perspective they have been discussed as a bit of a ‘solution looking for a problem’. As with all new technologies, the innovation and creativity of our industry will find ways to utilize the tech to create something new for ever-more-demanding audiences, but this takes time. In Part 3, we delve deeper into advanced topics, exploring unconventional applications and examine a few ethical considerations.
Works cited
1. Integrating AI in Game Development: The Role of Evolved NPCs - Infiniticube, accessed February 11, 2025, https://infiniticube.com/blog/integrating-ai-in-game-development-the-role-of-evolved-npcs/
2. The Role of Artificial Intelligence in Video Game Design: Creating realistic NPCs | MoldStud, accessed February 11, 2025, https://moldstud.com/articles/p-the-role-of-artificial-intelligence-in-video-game-design-creating-realistic-npcs
3. What's the Difference: NLP, Generative AI, and Computer Vision - CEI America, accessed February 11, 2025, https://www.ceiamerica.com/blog/an-ai-overview-what-is-the-difference-between-nlp-generative-ai-and-computer-vision/
4. Natural Language Processing (NLP) and Computer Vision - Kili Technology, accessed February 11, 2025, https://kili-technology.com/data-labeling/computer-vision/natural-language-processing-and-computer-vision
5. AI NPCs: The Future of Game Characters - Naavik, accessed February 11, 2025, https://naavik.co/digest/ai-npcs-the-future-of-game-characters/
6. AI Agents: The Next Frontier for the Gaming Industry | FXMedia: Solutions for Metaverse, accessed February 11, 2025, https://www.fxmweb.com/insights/ai-agents-the-next-frontier-for-the-gaming-industry.html
7. How artificial intelligence is changing computer games: from smart NPCs to procedural world generation - The CEO Views, accessed February 11, 2025, https://theceoviews.com/how-artificial-intelligence-is-changing-computer-games-from-smart-npcs-to-procedural-world-generation/
8. Adaptive Worlds: Generative AI in Game Design and Future of Gaming, and Interactive Media - Digital Commons@Lindenwood University, accessed February 11, 2025, https://digitalcommons.lindenwood.edu/cgi/viewcontent.cgi?article=1693&context=faculty-research-papers
9. AI NPCs and the future of gaming - Inworld AI, accessed February 11, 2025, https://inworld.ai/blog/ai-npcs-and-the-future-of-video-games
10. Best AI games 2024: Released and upcoming - Inworld AI, accessed February 11, 2025, https://inworld.ai/blog/best-ai-games-2023
11. (PDF) A review of AI-based game NPCs research - ResearchGate, accessed February 11, 2025, https://www.researchgate.net/publication/382389385_A_review_of_AI-based_game_NPCs_research
12. How AI disrupts the Video Game Industry in 2024, accessed February 11, 2025, https://whimsygames.co/blog/how-ai-disrupts-the-video-game-industry-in-2024/
13. Dynamic Adaptation: Leveraging AI In Videogame Creation And Play - Forbes, accessed February 11, 2025, https://www.forbes.com/councils/forbesbusinesscouncil/2025/02/10/dynamic-adaptation-leveraging-ai-in-videogame-creation-and-play/
14. Fogg Behavior Design behaviordesign.stanford.edu