AI Research Portal

This paper addresses the challenge of robust causal discovery from observational data when prior knowledge contains imperfections. The core idea is to harmonize knowledge and data through a framework called Robust Causal Discovery under Imperfect Structural Constraints (RoaDs). RoaDs employs Prior Alignment using a surrogate model to dynamically modulate constraint weights based on observational data, and Conflict Resolution via a multi-task learning (MTL) framework optimized by Multi Gradient Descent Algorithm (MGDA) to balance data-driven and knowledge-driven objectives. The main empirical claim is that RoaDs demonstrates superior robustness and effectiveness compared to state-of-the-art methods across diverse noise conditions and structural equation model types, even under imperfect structural constraints.

The paper identifies several bottlenecks in existing causal discovery methods when dealing with imperfect structural constraints:

Presupposition of perfect priors: Existing methods typically assume perfect prior knowledge or can only handle specific, pre-identified error types.

Introduction of a consistent constraint assumption and a surrogate model to learn continuous weights for priors.
Design of a knowledge-based optimization goal based on consistent constraints, with a theoretical proof of its asymptotic consistency.
Employment of Multi Gradient Descent Algorithm (MGDA), enhanced with gradient normalization, to efficiently find a balanced Pareto stationary point for the multi-task learning problem.
Demonstration of superior robustness and effectiveness of RoaDs against state-of-the-art methods across diverse and challenging settings through experimental evaluation.

The RoaDs framework tackles imperfect structural constraints through two main components: Prior Alignment and Conflict Resolution.

Prior Alignment This component refines imperfect constraints by aligning them with observational data.

Surrogate Model: A surrogate model M = {M(1),...,M(nv)} is used to test the credibility of different constraints under the dataset X. For an arbitrary Xj, M(j) solves E[Xj|Π] to find the weights of edges pointing to Xj, defined as W:,j = M(j)([B:,j]≠0).
Asymptotic Consistency: Under the assumption of consistent constraints (Assumption 1), it is theoretically proven that as the number of samples nd → ∞, the ground-truth DAG satisfies W*:,j = M(j)([B*:,j]≠0) = M(j)(I:,j).
Knowledge-driven Optimization Objective: After prior alignment, W^c = M_X([B^c]≠0) is used to model the knowledge-driven objective. This objective aims to promote the non-parametric weighted adjacency matrix W(θ) towards the refined DAG encoded in W^c. It is initially defined as minimizing the l1 norm of the difference between binarized structures: min ||[I(W(θ) - s > 0) – I(W^c – τ > 0)] ο B^c||1 θ
Smooth Approximation: To facilitate tractable optimization, the Heaviside step function I(·) is substituted with a continuous sigmoid function σ(·), yielding the objective: min || [σ(W(θ) – s) – σ(W^c – τ)] ο B^c||1 θ This is denoted as C(W(θ), W^c).

Conflict Resolution This component leverages multi-task learning (MTL) to explicitly manage the trade-off between data-driven and knowledge-driven objectives.

Multi-task Learning Formulation: The problem is formulated as a multi-objective optimization problem with two tasks: min F_X(θ) + λ2H(W(θ)) min C(W(θ), W^c) θ where F_X(θ) is the data-driven objective, H(W(θ)) is the acyclicity constraint, and C(W(θ), W^c) is the knowledge-driven objective.
Optimization: The Multi Gradient Descent Algorithm (MGDA) is employed to solve this multi-objective problem. The parameters θ are updated according to: θt+1 = θt + ηdt where η is the learning rate, and dt is defined from: (dt, κt) = argminκ + ||d||2 d,k s.t. Φα(θt, X) = ∇[FX(θt) + λ2H(W(θt))]Td(1) ≤ κ Φβ(θt, W^c) = ∇C(W(θt), W^c)Td(2) ≤ κ
Gradient Normalization: To ensure both objectives contribute meaningfully despite disparate scales, gradients are normalized: Φα = Φα· [(FX(θt) + λ2H(W(θt))) · ||Φα||2]−1 Φβ = Φβ· [C(W(θt), W^c)· ||Φβ||2]−1

Overall Algorithm (Algorithm 1) RoaDs performs a warm-up stage using only the data-driven objective for ts iterations. The main loop then uses the Adam optimizer and MGDA to adjust parameters and resolve conflicts.

Architecture: Each function f_j in the structural equation model (SEM) is parameterized using a Multilayer Perceptron (MLP): f_j(X) = MLP(X; θ_j). The weighted adjacency matrix W(θ) is approximately expressed as W(θ) ∈ R^(nv x nv), where [W(θ)]_ij = ||[A^(1)]:,i||2.
Key Equations:
- Reformulated continuous optimization problem for causal discovery: min Σj=1^nv ||xj – MLP(X; θj)||2 + λ1||W(θ)||1 θ s.t. h(W(θ)) = tr(e^(W(θ)∘W(θ))) – nv = 0.
- Knowledge-driven optimization objective (smooth approximation): min || [σ(W(θ) – s) – σ(W^c – τ)] ο B^c||1 θ
- Multi-task learning objectives: { min F_X(θ) + λ2H(W(θ)) { min C(W(θ), W^c) θ
- MGDA update rule: θt+1 = θt + ηdt where (dt, κt) = argminκ + ||d||2 d,k s.t. Φα(θt, X) = ∇[FX(θt) + λ2H(W(θt))]Td(1) ≤ κ Φβ(θt, W^c) = ∇C(W(θt), W^c)Td(2) ≤ κ
Loss/Objective: The data-driven objective F_X(θ) is a least squares loss or negative log-likelihood loss. The acyclicity constraint H(W(θ)) is defined as h(W(θ)) = tr(e^(W(θ)∘W(θ))) – nv = 0. The knowledge-driven objective is C(W(θ), W^c).
Optimization: The constrained optimization problem is transformed into an unconstrained form using the Augmented Lagrangian Method (ALM). The Multi Gradient Descent Algorithm (MGDA) is used for multi-objective optimization. The Adam optimizer is used in the main loop.
Data Handling: Synthetic graphs are generated using Erdős-Rényi (ER) and Scale-Free (SF) models. Data is generated based on Structural Equation Models (SEM). Exogenous noise variables are drawn from Gaussian, Exponential, Gumbel, and Uniform distributions, with equal variance (EV) and non-equal variance (NV) settings. For nonlinear settings, data is generated using MLP or Gaussian Processes (GP). Imperfect constraints are simulated by sampling true edges as positive constraints and non-existent edges as negative constraints, then randomly flipping a fraction pt of sampled edges.
Implementation Details: The parameters s = 0.3 and τ = 0.01 are set in Eq. (8). Other parameters are default in GOLEM and NOTEARS-MLP. Each experiment is repeated ten times. A warm-up stage runs for ts iterations.

Datasets:
- Synthetic graphs: Erdős-Rényi (ER) and Scale-Free (SF) with nv = 20 nodes and knv edges.
- Sachs dataset: 853 sample observational data (11 variables).
Metrics: F1-score (↑: higher is better) and Structural Hamming Distance (SHD) (↓: lower is better).
Baselines: PC-stable, LINGAM, NTS-B, ECA, GOLEM-EV, GOLEM-NV, NOTEARS-MLP.
Numeric Results:
- Linear SEM (EV): RoaDs achieves an average F1-score improvement of approximately 4.4% and 17.0% decrease in SHD compared to GOLEM-EV (e.g., for Gauss (ER), RoaDs F1(↑) 0.821 vs GOLEM-EV F1(↑) 0.807; RoaDs SHD(↓) 11.4 vs GOLEM-EV SHD(↓) 12.1).
- Linear SEM (NV): RoaDs maintains competitive performance with PC, demonstrating superior resilience in challenging scenarios (e.g., for Gauss (ER), RoaDs F1(↑) 0.384 vs PC-stable F1(↑) 0.397; RoaDs SHD(↓) 32.7 vs PC-stable SHD(↓) 29.5).
- Nonlinear SEM: RoaDs surpasses ECA by an average of 14.5% and NTS-B by 15.4% in F1-score. For GP noise, NOTEARS-MLP achieves an F1-score below 0.1.
- Sachs dataset (threshold = 0.1): RoaDs F1-score 0.480, SHD 12.0, Precision 0.750, Recall 0.353, outperforming all competing approaches.
Runtime/Training Cost: Not stated in the paper.

The current work uses MGDA to randomly identify the solution on the Pareto front, which may not align with a decision-maker's specific preferences.

Method evidence: "Our approach achieves this through two components: Prior Alignment, which employs a surrogate model dynamically modulating the weights of imperfect constraints based on the observational data; Conflict Resolution, which leverages multi-task learning (MTL) to explicitly manage the trade-off between the data-driven and knowledge-driven objectives." (section: Abstract)
Result evidence: "RoaDs can harness the benefits of correct priors while resisting misleading ones via prior alignment, resulting in an average F1-score improvement of approximately 4.4% and 17.0% decrease in SHD compared to GOLEM-EV." (section: Results and analysis)
Limitation evidence: "However, this work use MGDA to randomly identify the solution on Pareto front, which may not align with decision-maker's specific preferences." (section: Conclusion)

Robust Causal Discovery Under Imperfect Structural Constraints

Summary

Problem

Key Contributions

Method

Technical Details

Results

Weaknesses / Limitations

Evidence

Additional Figures