The Co-EPG framework is a self-iterative training loop for Co-Evolution of Planning and Grounding, built upon a P-G dual-model architecture.
P-G Dual-Model
The architecture decouples planning and grounding into two cooperative subtasks.
- The planning model (π) acts as a high-level strategist. Given the current observation o_t, task description Q, and interaction history h_t, it generates a multi-part action decision:
p_t, a_t^type, a_t^value = π(Q, o_t, h_t). (1)
- The grounding model (φ) utilizes the specific plan p_t with the visual input from the vision observation (e.g., a screenshot) o_t^vision to predict the exact coordinates of the target element:
a_t^coor = φ(o_t^vision, p_t). (2)
The outputs are combined to form the action a_t = (a_t^coor, a_t^type, a_t^value) to be executed.
Co-Evolving Optimization Loop
The loop includes two core steps: Iterative Training and Data Enhancement.
- Iterative Training: In each iteration k, the models π_k and φ_k are fine-tuned on dataset D_k-1. The planning model π_k is then refined into π'_k through collaborative GRPO training, guided by the C-DREM mechanism. C-DREM adaptively aggregates rewards from an ensemble of grounding models (φ_k and VLMs like Qwen2.5-VL-72B-Instruct and Qwen2.5-VL-32B-Instruct). The improved grounding model φ_k is strengthened by fine-tuning on high-quality data distilled from the previous stage.
- Data Enhancement: A self-enhancing data evolution mechanism is developed. Initially (k=0), two specialized pools of open-source VLMs, Planner II and Verifier Φ, are formed. Planner II generates plans, and Verifier Φ validates them to construct the initial dataset D_0. In subsequent iterations (k > 1), the updated planning model π'_k and grounding model φ_k participate in data production, with Planner II enhancing planning diversity and Verifier Φ improving discrimination reliability.
C-DREM (Confidence-based Dynamic Reward Ensemble Mechanism)
C-DREM aggregates collective intelligence from diverse grounding models.
- Plan Reward: The grounding model's prediction accuracy is used as a reward for the planning model.
Acc^plan = {1, if a^coor ∈ bbox, 0, otherwise. (3)
The plan reward r^plan is a weighted sum of rewards from N grounding models:
r^plan = Σ_{j=1}^N w_j ⋅ Acc_j^plan (4)
The weight w_j is determined by a static prior σ_j and a dynamic confidence score c_j:
w_j = exp(σ_j ⋅ c_j) / Σ_{n=1}^N exp(σ_n ⋅ c_n) (5)
The dynamic confidence score c_j is the sum of log-likelihoods of the predicted coordinate token τ_l of a^coor, normalized by its length L:
c_j = (1/L) Σ_{l=1}^L log P(τ_l | o, h, p). (6)
- Action Type Reward: Depends on whether the predicted action type a^type matches the ground truth g_t^type:
r^type = {1, if a^type = g_t^type, 0, otherwise. (7)
- Action Value Reward: Calculated based on the F1 score between the predicted value a^value and the ground truth g_t^value:
r^value = {1, if F1(a^value, g_t^value) > 0.5, 0, otherwise. (8)
- Final Reward: The final reward r_i of the i-th generation for GUI tasks is calculated by r^plan, r^type, and r^value:
r_i = {0, if r^type = 0 or r^value = 0, r_i^plan, otherwise. (9)
- Group Computation: Rewards are normalized across G generated Rollouts to compute the advantage A_i, which serves as the objective for GRPO policy optimization:
A_i = (r_i - mean({r_1, r_2, ..., r_G})) / std({r_1, r_2, ..., r_G}) (10)