ProPlay: Procedural World Models for Self-Evolving LLM Agents

ProPlay introduces a procedural world model designed for self-evolving LLM agents, enabling them to rehearse future actions based on learned knowledge and refine their understanding of environment dynamics through interaction. By utilizing a procedure graph that captures causal transitions and reliability records for task-specific contributions, ProPlay enhances agents' ability to explore and learn in partially observable environments. Experimental results demonstrate that ProPlay outperforms existing methods in improving environment understanding and self-evolution capabilities, making it a significant advancement for practitioners developing adaptive AI systems.