AI Research Portal

This paper introduces EvoEmpirBench, a benchmark for dynamic spatial reasoning, and Agent-ExpVer, a human-inspired online learning framework. EvoEmpirBench comprises two interactive, partially observable games—maze navigation and match-2 elimination—designed to evaluate LLMs' spatial understanding and adaptive planning under dynamic environmental changes and long-term memory demands. Agent-ExpVer employs a three-agent architecture (GeoLink, InsightForce, TruthWeaver) to facilitate continual, parameter-free adaptation through memory abstraction and rule distillation. Experiments demonstrate that EvoEmpirBench reveals limitations of mainstream models in dynamic spatial reasoning and long-term memory, and Agent-ExpVer significantly improves agent reasoning and interactivity, though performance remains below human baselines.

The paper identifies several bottlenecks in evaluating and enabling LLM reasoning:

Static and Globally Observable Environments: Most existing spatial reasoning benchmarks focus on static or globally observable environments, failing to capture challenges of long-horizon reasoning and memory utilization under partial observability and dynamic changes.

Dynamic Reasoning Benchmark: Introduction of EvoEmpirBench, featuring interactive, partially observable maze and match-2 games designed to rigorously test spatial, linguistic, and adaptive reasoning beyond static and narrow-domain evaluations.
Cognitively-inspired Online Learning Framework: Proposal of Agent-ExpVer, a novel human-inspired learning framework with memory abstraction and rule distillation, enabling continual, parameter-free adaptation and lifelong learning in dynamic environments.

The proposed method, Agent-ExpVer (Experience+Verification), is a human-inspired online learning framework designed for dynamic, partially observable environments. Its workflow involves three collaborating agents:

GeoLink Agent: This agent actively engages with the environment. At each timestep t, it observes the current state st (partially observable), chooses an action at based on its policy πt, and receives the subsequent state st+1 and reward rt. This interaction history Ho:T = {(So, ao,ro),………,(st,at,rt)} is accumulated across each episode.
- at ~ πt(St) (1)
- (St+1, rt) ~ E(St, at) (2) where E represents the environment dynamics.
InsightForce Agent: Inspired by human episodic memory, this agent processes the collected trajectory Ho:T and final metrics m (e.g., A.Score, Suc.Rate) after an episode. It summarizes key decisions and outcomes as a subjective experience e.
- e = fsum (Ho:T, m) (3) where fsum is executed by prompting the LLM. These experiences are added to a memory module Mexp.
TruthWeaver Agent: This agent manages a truth repository Mtruth. To verify and refine knowledge, the InsightForce agent replays the episode, integrating the experience e as a prompt extension to its policy. If the level is passed (P is true) and the resulting score improves (S' > S), the experience e is upgraded to a reusable truth and stored in Mtruth.
- Mtruth ← Mtruth U e if P∧ (S' > S) (4) The TruthWeaver Agent manages incoming truths by merging semantically similar truths, removing redundant truths, and inserting new truths.

Policy Evolution: The current agent policy πt is constructed by combining the base prompt π0 with all truths from Mtruth.

πt = π0 U e (5) e∈Mtruth After each episode, new truths are added and validated, enabling continual improvement. Policy improvement is defined as the average test score increment:
Δ = (1/N) Σ(S_i - S_i-1) (6) where N is the number of test cases, S_i and S_i-1 are scores using the current and previous policies. If performance degrades (Δ < 0), updates are reverted and abstraction is repeated.

Benchmark Environments: EvoEmpirBench consists of two dynamic, interactive game environments:
- Maze Navigation: A 9×9 grid world where agents act under local observability. Objectives include collecting five gold coins and exploring unknown areas efficiently.
  - Difficulty Settings:
    - Easy: Simple map, no monsters, 5 coins.
    - Medium: Introduces 2 moving monsters, 5 coins.
    - Hard: Retains 2 moving monsters, adds 4 interactive items (Pickaxe, Iron Sword, Magnet, Key), 5 coins.
- Match-2 Elimination: An 8×8 board where agents eliminate connected blocks of identical color (A-D) by matching two or more. Primary goal is to meet per-color elimination targets within a limited number of moves, maximizing overall score and efficiency.
  - Difficulty Settings:
    - Easy: 15–18 steps, target 8-12 eliminations per color.
    - Medium: 12–15 steps, target 12-16 eliminations per color.
    - Hard: 10–13 steps, target 16-20 eliminations per color.
Evaluation Metrics:
- Common Metrics (both games):
  - Success Rate (Suc. Rate): Episode completion ratio.
  - Average Score (A.Score): Mean cumulative reward.
- Maze Navigation Specific Metrics:
  - Average Steps (A.steps): Average steps per episode.
  - Average Exploration (A.Explor): Explored map percentage.
  - Average Gold Collected (A.Gold): Collected gold ratio.
  - Remaining HP (Rem.HP): Surviving health points.
  - Average Enemy Kills (A.kills): Count of defeated enemies.
  - Average Barriers Destroyed (A.Barr.): Count of destroyed barriers.
- Match-2 Elimination Specific Metrics:
  - Remaining/Max Steps Ratio (R/M.S): Remaining steps relative to maximum.
  - Score per Step (Score/Step): Reward per move.
  - Blocks Cleared per Step (Clear/Step): Cleared blocks per move.
  - API Efficiency (API Eff.): Valid API call ratio.
Experiment Setting: 90 instances per task (30 instances per level across 3 difficulty levels).

Datasets: EvoEmpirBench, consisting of Maze Navigation and Match-2 Elimination games.
Metrics:
- Maze Navigation: S.R. (Success Rate), A.S. (Avg. Score), A.St. (Avg. Steps), A.Ex. (Avg. Exploration), A.G. (Gold Collection Rate), R.HP (Remaining HP), A.K. (Avg. Kills), A.B. (Avg. Barrier Interactions).
- Match-2 Elimination: S.R. (Success Rate), A.S. (Avg. Score), R/M.S (Redundant/Missing Steps), S./St. (Score per Step), C./St. (Clearance per Step), API E. (API Efficiency).
Baselines:
- Human Baseline
- Open-Source Models: deepseek-V3, llama-3.1-8b-instruct, llama-3.1-70b-instruct, qwen2.5-7b-instruct, qwen2.5-14b-instruct, qwen2.5-32b-instruct, qwen2.5-72b-instruct, qwen3-30b-a3b.
- Proprietary Models: grok-3, gemini-2.0-flash, gemini-2.5-flash-preview, claude-3-5-sonnet, claude-3-7-sonnet, gpt-4, gpt-4.1.
Numeric Results:
- Maze Navigation Game (Table 2):
  - Human Baseline: S.R. 90.00, A.S. 2914.67, A.St. 20.6, A.Ex. 77.54, A.G. 68.22, R.HP 2.43, A.K. 0.83, A.B. 1.43.
  - Agent-ExpVer (ours) models showed an average boost of +5.6% in success rate and +9.5% in mean score.
  - GPT-4.1-ours: S.R. 78.89, A.S. 2805.67, A.St. 32.77, A.Ex. 83.69, A.G. 72.67, R.HP 1.99, A.K. 0.73, A.B. 0.77. This is a +5.56% (absolute) / +7.6% (relative) increase in success and a +243.3-point / +9.5% rise in score for GPT-4.1.
  - Qwen2.5-32B-instruct-ours: S.R. 54.44, A.S. 1532.33, A.St. 38.81, A.Ex. 68.73, A.G. 61.33, R.HP 1.18, A.K. 0.26, A.B. 0.5. This is a +12.22% / +28.96% success and +410.1-point / +36.6% score gain for Qwen2.5-32B-instruct.
- Match-2 Elimination Game (Table 3):
  - Human Baseline: S.R. 86.67, A.S. 350.2, R/M.S 22.99, S./St. 34.40, C./St. 6.20, API E. Not stated in the paper.
  - Baseline LLMs averaged only 33.7% success.
  - Integrating Agent-ExpVer yielded consistent gains of +13.3% in success rate, +9.8% more steps remaining, +1.2 elimination score per step, and +0.8 eliminations per step.
  - GPT-4.1-ours: S.R. 53.33, A.S. 234.6, R/M.S 17.31, S./St. 19.90, C./St. 5.86, API E. 75.72. This is a +13.33% success for GPT-4.1, though its average score dipped by 4.3% (-10.44 points).
  - Gemini-2.5-flash-preview-ours: S.R. 37.78, A.S. 415.49, R/M.S 7.29, S./St. 32.33, C./St. 5.04, API E. 88.85. This model maintained its success rate while gaining +1.2% in score (+4.73 points), surpassing human benchmarks.
- Ablation Study (TruthWeaver, Table 4): Removing TruthWeaver caused a drop in average scores by 16.9% and success rates by 6.1% for Qwen2.5-32B, and by 5.8% and 8.3% for GPT-4.1.
- Ablation Study (EEB Constraints, Table 5):
  - Maze Game (Full-Vision): GPT-4.1's success rate rose from 73.33% (EEB) to 93.33%, and average score increased from 2562.33 to 3412.22.
  - Match-2 Game (NoProps): GPT-4.1's success rate increased from 40.00% (EEB) to 43.33%.
Runtime/Training Cost: Not stated in the paper.

Performance remains tied to model capacity, with even top systems falling short of human baselines.

Method evidence: "We further propose a subjective experience-based memory mechanism for cross-task experience transfer and validation." (section: Abstract)
Result evidence: "Experiments show that our benchmarks reveal key limitations of mainstream models in dynamic spatial reasoning and long-term memory, providing a comprehensive platform for future methodological advances." (section: Abstract)
Limitation evidence: "However, performance remains tied to model capacity, with even top systems falling short of human baselines." (section: Conclusion)

EvoEmpirBench: Dynamic Spatial Reasoning with Agent-ExpVer

Summary

Problem

Key Contributions

Method

Technical Details

Results

Weaknesses / Limitations

Evidence

Additional Figures