The Coarse-to-Fine Classification (CFC) framework progressively achieves advanced OOD classification by tackling the approximation of OOD space without labeled information and the derivation of meaningful outlier class labels.
Coarse Classifier with LLMs:
The first step involves designing LLM-based prompts for coarse-grained graph node OOD identification. LLMs leverage their expert knowledge and reasoning capabilities to detect OOD samples on the test set without prior OOD information and generate a candidate OOD label space.
- Easy-Reject: Used for small coverage and far-OOD cases. LLMs determine if a test node's label belongs to the provided ID label space using confidence-aware prompts. If identified as OOD, the LLM provides an outlier class label beyond the ID label space. A confidence threshold of 0.7 is applied to filter out noisy labels.
- Hard-Reject: Used for large coverage and near-OOD cases. LLMs first summarize ID classes into major categories, then generate possible outlier class labels within these major categories but not included in the ID classes, creating a candidate OOD label space. This candidate space is then used by LLMs to determine the label of the input test node, generate a confidence score, and provide a predicted category.
GNN-based Fine-Classification:
After obtaining a coarse-grained OOD set (Vood) through LLM-based OOD detection, a GNN-based fine-grained classifier is constructed for ID classification and precise OOD detection.
- Denoising: A label propagation method is used to correct falsely identified OOD samples from the LLM-based detection. The kth iteration of label propagation is formulated as Yl(k) = D⁻¹AYl(k-1). ID training samples are reset to their initial labels (Yi(k) = Yi(0) for Vi ∈ Vtrain) to maintain label information. After K-order propagation, candidate OOD samples' labels are updated using maximum probability, and OOD samples predicted as ID are discarded, forming a new OOD set V'ood.
- OOD Data Augmentation: To address the issue of a small number of semantic OOD samples, an improved manifold mixup method is extended to augment OOD data. K nodes with low classification confidence in the training set are collected. Manifold mixup is applied to these near-boundary nodes and the center of the OOD samples in V'ood. The generated OOD samples are formulated as xi = αh + (1 − α)hc, i < K, Yi = C + 1, where hk = GNN (A,hk-1) is the hidden embedding with a GNN encoder, hc is the center embedding of OOD samples, and α > 0 is a hyperparameter. This creates an augmented OOD set Vood = {V'ood, Va}.
- ID Classification and OOD Detection: A GNN-based classifier fc+1 is trained on Vtrain and the augmented Vood. For a two-layer GCN model, the formulation is Z = softmax (Â ReLU(AXW⁽⁰⁾W⁽¹⁾)), where Z is the GCN's output predictions, Â = D⁻½ (A + In) D⁻½ is the normalized A + In matrix, and W = (W⁽⁰⁾, W⁽¹⁾) are the weights. The objective function L is L = (1 / |Vtrain ∪ Vood|) Σvi∈Vtrain∪Vood yi log(zi), where y and zi are the label and prediction of node vi.
OOD Classification:
After detecting OOD samples, the method proceeds with OOD classification for these nodes by leveraging the potential OOD label space. Similarity measures (e.g., word-level or semantic-level comparisons like TF-IDF) are used to merge similar categories and filter out categories with too few samples, forming the post-OOD label space. LLMs then generate annotations for the OOD samples based on this post-OOD label space.