This paper introduces History-Aware Reasoning (HAR), a framework designed to enhance the reasoning capabilities of Graphical User Interface (GUI) agents, particularly by improving their short-term memory in long-horizon tasks. The core idea is an error-centric learning approach that encourages agents to reflect on their mistakes and acquire episodic reasoning knowledge through a tailored reflection scenario, correction guidelines, and a hybrid Reinforcement Learning (RL) reward function. The proposed end-to-end model, HAR-GUI-3B, shifts the agent's reasoning mode from history-agnostic to history-aware, demonstrating improved performance and generalization across various GUI-related benchmarks.
Problem
Existing GUI agents face several bottlenecks:
Weak Short-Term Memory: Current native GUI agents exhibit weak short-term memory during episodic reasoning, interpreting chained interactions as discrete screen understandings, making them unaware of historical interactions within an episode.
: The inherent Chain-of-Thought (CoT) reasoning mode of foundation Multimodal Large Language Models (MLLMs is often history-agnostic, which undermines performance in episodic reasoning that requires leveraging previous execution clues.
History-Agnostic Reasoning
Challenges in Long-Horizon Tasks: For long-horizon GUI tasks, effectively leveraging historical interaction information, which connects each screen to the overall goal-oriented episode chain, is crucial for current decision-making but is poorly handled by existing methods.
Limited Domain-Specific Optimization: Early methods relying on generic MLLMs, function calls, and context engineering require meticulous prompting and face performance bottlenecks due to model sensitivity, limiting scalability and domain-specific optimization.
Key Contributions
History-Aware Reasoning (HAR) Framework: Proposal of an error-centric learning approach that encourages GUI agents to reflect on their errors and acquire episodic reasoning knowledge to enhance short-term memory in long-horizon interactions.
Reflective Learning Scenario: Construction of a reflective learning scenario for cognitive correction, introducing external GUI reasoning knowledge tailored for each incorrect instance.
Tailored Correction Guidelines Synthesis: Development of a method to synthesize tailored correction guidelines from hard samples to facilitate clue-oriented reasoning enhancement.
Hybrid RL Reward Function: Design of a hybrid RL reward function that incorporates format, action, and memory-augmented components to guide the agent's self-evolution.
HAR-GUI-3B Model: Development of a native end-to-end model that alters the inherent reasoning mode from history-agnostic to history-aware, equipping the GUI agent with stable short-term memory and reliable perception of screen details.
Method
The History-Aware Reasoning (HAR) framework is an error-centric learning approach with two main training stages:
1. GUI Scenario Warm-up Stage
This stage injects comprehensive domain-specific knowledge into the agent via supervised fine-tuning (SFT) using GUI-related data.
GUI Understanding Enhancement: Collects domain-specific tasks like graphic caption generation, UI-element clickability prediction, screen question answering, screen summary, and screen grounding from public sources to enhance the foundation model's screen perception.
Action Summarization Integration: Introduces an action summarization task to improve the agent's action semantic understanding at the episode level. A teacher model synthesizes Action-to-Summary (Act2Sum) data using instruction templates:
OAct2Sum = Mteacher(It, Psum(G, At), θ)
where G is the goal, It is the current screen image, At is the action, Psum is the instruction template, and Mteacher is the teacher model.
System-2 Reasoning Data Distillation: Utilizes an advanced teacher model to synthesize System-2 Chain-of-Thought (CoT) for episodes, filtering positive samples to create training data for enhancing the base model's reasoning capability.
2. Learning From Failure Stage
This stage enhances the agent's short-term memory through error-aware cognitive corrections within a reflection scenario using Reinforcement Learning (RL).
Guidance Synthesis for Error Correction: Identifies historical incorrect samples (Dhis) by performing inference with the warm-up agent (Mwarm-up). A teacher model (Mteacher) generates up to three error correction guidelines (G) for each incorrect instance.
Reflection Scenario Construction: Constructs a reflection scenario where the agent (Mwarm-up) is prompted with the guidelines (G, Gt), historical context (I<t), incorrect action (Aerror), and corresponding CoT (Cerror) to perform self-evolution.
Error-Aware Cognitive Correction: Employs the GRPO algorithm for self-exploration. Each candidate response (Oi) yields a reward (ri), which is normalized to compute the relative advantage Ei:
Ei = (ri - mean({r1,...,rn})) / std({r1,...,rn})
Hybrid Reward Function: The reward ri for each sample consists of three parts:
Format Reward (rformat): Rule-based, 1 if the output conforms to the required format, else 0.
Action Reward (raction): 1 if the predicted action matches the label, else 0. For actions involving screen coordinates, a higher reward is assigned based on Euclidean distance.
Fdist(P1, P2) = √(x1 − x2)2 + (Y1 − Y2)2
Fabs(P1, P2, T) = {1 - Fdist (P1, P2)/T, if Fdist < T; 0, else}
Pnorm = (Px/W,py/h), Pnorm* = (px/W,py/h)
Dnorm = Fdist(Pnorm, Pnorm*)
raction = {1+ Fabs(P, P*, Tabs), if Dnorm ≤ Tnorm; |Fabs(P, P*, Tabs)|, else}
Memory-Augmented Reward (rmemory): Model-based function (FMAR) determines if the explicit CoT includes the agent's logical analysis of previous interactions.
rmemory = FMAR(PMemory (Cti, Tet))
The final hybrid reward is:
ri = rformat × (raction + γ × rmemory)
where γ is a hyperparameter.
Round-1 RL: Enhances short-term memory and shifts reasoning from history-agnostic to history-aware.
Round-2 RL: Aligns the execution format and employs a Task Mixing Training Strategy (TMTS), mixing grounding and GUI episodic reasoning tasks to enhance screen detail perception and knowledge integration.
Technical Details
Base Model: HAR-GUI-3B is built upon Qwen2.5-VL-3B-Instruct.
Teacher Model: Qwen2.5-VL-72B-Instruct serves as the teacher model for all data synthesis.
Data: 15k grounding data (OS-Atlas), 15k hard samples.
Post-training (SFT): After HAR framework training, HAR-GUI-3B is further fine-tuned on GUI understanding and GUI episodic reasoning benchmarks individually.
Learning Rate: 2e-5
Batch Size: 256
Epochs: 4 per task.
Hyperparameters:
Tnorm = 0.1
Tabs = 40
Tabs T2 = 200
γ = 0.2
Optimization:
Optimizer: AdamW
Fine-tuning Method: LoRA with rank 64 and alpha 128 applied to the LLM and Vision Backbone.
Hardware: All experiments performed on 8 NVIDIA A100 80GB GPUs.
GUI Understanding (Overall): HAR-GUI-3B achieves 82.1, outperforming MP-GUI (84.3) and UI-TARS-1.5-7B (82.1) in some metrics, while having fewer parameters. It shows an overall gain of 3.07 points compared to Qwen2.5-VL-3B.
Runtime/Training Cost: Not stated in the paper.
Weaknesses / Limitations
Training solely with episodic reasoning weakens the agent's grounding capability.
A prompt-constrained focus on historical context can lead to performance degradation.
The agent's reasoning mode should not be constrained, but autonomously shaped and adapted during self-evolution.
Evidence
Method evidence: "To alleviate this weakness, we propose a History-Aware Reasoning (HAR) framework, which encourages an agent to reflect on its own errors and acquire episodic reasoning knowledge from them via tailored strategies that enhance short-term memory in long-horizon interaction. The framework mainly comprises constructing a reflective learning scenario, synthesizing tailored correction guidelines, and designing a hybrid RL reward function." (section: Abstract)
Result evidence: "As shown in Tab.1, Tab.2, and Tab.3, HAR-GUI-3B consistently surpasses the current advanced methods, even those with far more parameters, e.g. MP-GUI (8B), SeeClick (9.8B), and Qwen2.5-VL-7B." (section: 4.2 Main Results)
Limitation evidence: "training solely with episodic reasoning weakens the agent's grounding capability, whereas our TMTS mitigates this degradation. (ii) Comparing MGRPO and MGRPO, it is evident that a prompt-constrained focus on historical context can lead to performance degradation. We argue that the agent's reasoning mode should not be constrained, but autonomously shaped and adapted during self-evolution." (section: 4.3 Ablation Study)
Additional Figures
Figure 1: Overview of the Histoty-Aware Reasoning (HAR) framework. HAR framework is an error-centric learning approach designed to enhance the reasoning capability of the GUI agent by performing error-aware cognitive correction within a tailored reflection scenario. The framework consists of two critical training stages: (i) GUI Scenario Warm-up Stage. During this phase, comprehensive domain-specific knowledge is injected into the agent via GUI-related data collection and synthesis for knowledge distillation. (ii) Learning From Failure Stage. In this stage, the agent’s short-term memory is enhanced. It involves
Figure 2: Short-term memory emergence and reasoning en- hancement through HAR framework. Benchmarks. We extensively evaluate the proposed HAR-
Figure 3: Effectiveness of HAR framework. M Q is Qwen2.5-VL-3B-Instruct zero-shot results. M GRPO refers to the method keeping the same settings as M HAR-GUI , but excluding the reflection scenario (using the inference-