Opportunities and Challenges for Runtime Behavior Generation in Games

Exploring the potential of runtime behavior generation using large language models in game development.
Adventure Game: Meteor Shower
Adventure Game Demo: Generate combined new spells at runtime.

Procedural content generation (PCG) has been a staple in game development for decades, enabling the creation of various game components algorithmically. Recent advances in large language models (LLMs) have opened new possibilities for generating game behaviors at runtime based on player input. This capability introduces novel gameplay interactions and challenges traditional game development workflows.

The system, GROMIT, leverages LLMs to generate relevant behaviors when triggered by player actions, compiles the generated code, and integrates it into the game without developer intervention. This approach allows for new forms of gameplay and player interactions that were not pre-defined by the developers.

GROMIT System Diagram
System diagram of GROMIT, highlighting input processing, scene understanding, LLM, and output processing.

GROMIT was demonstrated in three scenarios to explore its potential applications:

  • Sandbox: User manipulate the environment with explicit commands.
  • Escape Room: Implicit interactions between objects are generated, providing new ways to solve puzzles.
  • Adventure Game: Combines spells at runtime, showcasing generative mechanics within a game.

Quantitative evaluations show that GROMIT can achieve a success rate of up to 85% in generating behaviors when the game systems are designed with GROMIT in mind. Developer interviews reveal positive interest in using runtime behavior generation but highlight concerns regarding quality, community expectations, and integration with existing workflows.

Sandbox: Meteor Shower
Sandbox Demo: Players manipulate the environment with explicit commands

Key contributions of this work include:

  • A system design for integrating runtime behavior generation in a modern game engine.
  • Evaluations characterizing the performance of runtime behavior generation.
  • A thematic analysis of developer interviews regarding implementation in workflows.

Escape-1
Escape Room Demo: generate interactions between objects.
Escape-2

Runtime behavior generation offers exciting opportunities for creating personalized and dynamic gameplay experiences. However, addressing concerns about the quality and control of generated behaviors is crucial for wider adoption. Future work will focus on developing guardrail systems to manage the quality and impact of generated behaviors effectively.

← Back