HyperLoad is a cross-modality enhanced framework that leverages pre-trained large language models (LLMs) for green data center cooling load prediction. It addresses challenges like data scarcity, load distortion, and multi-source data fragmentation by integrating textual prior knowledge with time-series data. The core idea involves a Cross-Modality Knowledge Alignment phase to map different modalities into a shared latent space, followed by a Multi-Scale Feature Modeling phase that injects domain-aligned priors and captures cross-device temporal dependencies. HyperLoad consistently outperforms state-of-the-art baselines in both data-sufficient and data-scarce settings, demonstrating high accuracy and stability for sustainable green data center management.
Problem
The paper identifies several bottlenecks in existing data center cooling load prediction methods:
Data Scarcity: Existing methods struggle with small-sample scenarios caused by cold start, load distortion, multi-source data fragmentation, and distribution shifts in green data centers, leading to overfitting and poor generalization. Emerging facilities face severe data scarcity due to limited operational history.
Inadequate Modeling of Complex Dynamics: Traditional models rely on empirical rules or thermodynamic physical models which are costly and lack quantitative rigor. Deep learning models like RNNs suffer from information decay over long sequences, CNNs cannot preserve temporal order, and Transformer-based models struggle with multiple heterogeneous factors influencing load variations, making single-modality modeling inadequate.
Lack of Contextual Reasoning: While some LLM-based approaches enhance time-series feature representation, they often have limited contextual reasoning, reducing prediction accuracy. Directly processing raw text and temporal data ignores distributional and semantic disparities, weakening the value of textual priors.
Failure to Capture Inter-device Dependencies: Green data centers exhibit nonlinear, dynamically evolving feature interactions among devices. Most existing methods fail to model these couplings or adaptively adjust feature weights, limiting their effectiveness in capturing intricate dependencies within complex operational datasets.
Key Contributions
Construction of the DCData dataset for data center cooling load forecasting, providing a standardized foundation for model development and evaluation.
First-time application of LLMs' trend-understanding and reasoning capabilities, learned from diverse distributions, to green data center load forecasting, aiming to improve prediction accuracy and reduce the demand for large amounts of data.
Introduction of a KARI strategy to reshape the text encoder feature space, narrowing distributional gaps between text priors and time-series data, thereby enhancing prior knowledge utilization.
Proposal of an Adaptive Domain-specific Prefix Tuning (ADPT) strategy that encodes critical green data center background as learnable prefix vectors, enabling rapid task adaptation without extensive retraining.
Design of an Enhanced Global Interaction Attention (EGIA) mechanism to efficiently capture coupling relationships and global patterns across variables, enhancing the model's ability to model key dependencies among devices.
Empirical demonstration that HyperLoad outperforms SOTA baselines and achieves high accuracy and stability even under data scarcity through experiments on DCData.
Method
HyperLoad operates in two main phases: Cross-Modality Knowledge Alignment and Multi-Scale Feature Modeling.
Cross-Modality Knowledge Alignment Phase
This phase employs the Knowledge-Aligned Representation Integration (KARI) strategy to project textual prior knowledge and load data features into a shared embedding space, enhancing cross-modal consistency.
Time-Series Modality Input Construction: Reversible instance normalization is applied to each feature column of the historical observation data X ∈ RTXM to standardize them to zero mean and unit variance. The normalized value for the n-th observation x(j) in the i-th feature column is obtained as:
x(j) = (x(j) - μ(j)) / σ(j)
Text Modality Input Construction: A Context-Aware Temporal Synthesis Template (T_CATS) is constructed, comprising a domain knowledge base (Background, Instruction, Trend, Statistics) and a textual representation of the time-series data.
T_CATS(i) = [T_b(1), T_s(i)]
Feature Encoding: The time series encoder (F_TS(·)) processes P(i) to derive the global time-series feature V_S(i) ∈ Rd. The text encoder processes T_CATS(i) to yield the text feature V_E(i) ∈ Rd.
KARI Loss: The KARI strategy minimizes the distance between each sequence feature and its corresponding text feature using a contrastive loss, projecting them into a shared feature space.
L_KARI = - log (exp(C(V_S(i), V_E(i))/τ) / Σ_B exp(C(V_S(i), V_E(k))/τ))
where C(,) is cosine similarity, τ is temperature (fixed to 0.05), and B is batch size.
Multi-Scale Feature Modeling Phase
This phase injects domain prior knowledge and captures cross-variable dependencies to enable efficient prediction.
Adaptive Domain-specific Prefix Tuning (ADPT): The text encoder, trained in the first phase, encodes the Context-Aware Temporal Synthesis Template as a prefix. This template encoding V_T(j) is concatenated with the corresponding time-series encoding to form the input V_in(j) = [V_T(j), V_E(j)], which is then fed into the LLM.
Enhanced Global Interaction Attention (EGIA) Mechanism: This mechanism models multivariate sequence representations and their adaptive correlations. Time series of each device-level variable are embedded as independent tokens S(i) = Embedding(P(i)), where S(j) ∈ RMxd. These embedded features are linearly projected into query (Q(i)), key (K(i)), and value (H(i)) representations. Correlations among variables are modeled, giving greater weights to highly correlated variables in their subsequent interaction with the value representation:
V_E(i) = Attn(Q(i), K_C(i), H(i)) = Softmax(Q(i)K_C(i)T / √d) H(i)
Pre-trained LLM-based Backbone: LLaMA-7B is adopted as the backbone. All parameters except the output prediction head are frozen. The combined multimodal features V_in(j) are fed into the LLM (O_LLM) to obtain an encoded representation, which is then passed through a trainable linear projection layer (F_proj(·)) to get the output V_out = F_proj(O_LLM(V_in(j))).
Prediction and Loss: The output is inverse-normalized to obtain the final prediction ŷ(i). The Mean Squared Error (MSE) is used as the training objective:
L_MSE = (1/K) Σ_k=1^K (ŷ_k(j) - y_k(j))²
Technical Details
Architecture: LLaMA-7B is used as the backbone model. It employs an 8-layer LLaMA backbone.
Input/Output: Input sequence length of 96. Forecasting lengths (PL) of 12, 24, 48, and 96 steps.
Mean Squared Error (MSE) for prediction: L_MSE = (1/K) Σ_k=1^K (ŷ_k(j) - y_k(j))²
Optimization: Adam optimizer.
Training Settings:
Initial learning rate: 7 × 10-4
Batch size: B = 64
Temperature (τ) for KARI loss: 0.05
Training performed on a single NVIDIA A100 GPU.
Data Handling:
Dataset: DCData, collected from a data center in Dongguan, spanning October to December 2024. Contains 5-minute interval data for 41 variables (outdoor temperature/humidity, inlet/outlet temperatures of cooling water/chilled water, cooling load).
Preprocessing: Outlier detection, missing value imputation, data normalization (reversible instance normalization to zero mean and unit variance), and data splitting.
Data Partition: Chronologically partitioned into training, validation, and test sets in a 7:1:2 ratio.
Data Scarcity Settings: Models trained using 50% and 25% of the training set (partitioned in chronological order) to simulate limited-data conditions.
Results
Datasets: DCData (13,438 records).
Metrics: Mean Squared Error (MSE) and Mean Absolute Error (MAE).
Corresponding MSE reductions are 70.7%, 28.9%, 5.4%, 53.2%, 20.0%, 34.5%, 11.9%, 29.1%, 87.2%, and 37.7%, respectively.
Data Scarce (50% Training Data):
HyperLoad: MSE 0.0128, MAE 0.0757
Best Baseline (PatchTST): MSE 0.0227, MAE 0.1046
HyperLoad reduces MSE by 6.5%, 3.3%, 4.5%, and 3.9% and MAE by 3.7%, 4.6%, 4.9%, and 4.1% compared to other SOTA baselines.
Data Scarce (25% Training Data):
HyperLoad: MSE 0.0294, MAE 0.1170
Best Baseline (PAttn): MSE 0.0360, MAE 0.1358
HyperLoad reduces MSE by 16.7%, 13.2%, 1.7%, and 2.6%, while MAE decreases by 9.5%, 10.5%, 4.1%, and 4.3% compared to other SOTA baselines.
Ablation Study (PL=96, full training dataset):
HyperLoad: MSE 0.0192, MAE 0.0931
w/o ADPT: MSE 0.0220, MAE 0.1006 (12.7% MSE, 7.5% MAE reduction with ADPT)
w/o EGIA: MSE 0.0211, MAE 0.0981 (9.0% MSE, 5.1% MAE reduction with EGIA)
w/o KARI: MSE 0.0194, MAE 0.0941 (1.0% MSE, 1.1% MAE reduction with KARI)
Backbone Comparison:
LLaMA: Time (s / iter) 0.3928, MSE 0.0192, MAE 0.0931
BERT: Time (s / iter) 0.4289, MSE 0.0217, MAE 0.1019
GPT-2: Time (s / iter) 0.1943, MSE 0.0224, MAE 0.1012
Weaknesses / Limitations
No explicit paper-reported limitations were found.
Evidence
Method evidence: "We introduce HyperLoad, a cross-modality framework that exploits pre-trained large language models (LLMs) to overcome data scarcity. In the Cross-Modality Knowledge Alignment phase, textual priors and time-series data are mapped to a common latent space, maximizing the utility of prior knowledge. In the Multi-Scale Feature Modeling phase, domain-aligned priors are injected through adaptive prefix-tuning, enabling rapid scenario adaptation, while an Enhanced Global Interaction Attention mechanism captures cross-device temporal dependencies." (section: Abstract)
Result evidence: "Under both data sufficient and data scarce settings, HyperLoad consistently surpasses state-of-the-art (SOTA) baselines, demonstrating its practicality for sustainable green data center management." (section: Abstract)
Limitation evidence: Not stated in the paper.
Additional Figures
Figure 1: Schematic diagram of HyperLoad performing green data center cooling load prediction tasks. data centers have become vital to reducing the ICT sector’s carbon footprint. Green data centers should serve as flexible
Figure 2: Limitations of time-series forecasting algorithms. The methodology unfolds in two phases: 1. Cross-Modality Knowledge Alignment phase. The Knowledge-Aligned Representation Integration
Figure 3: (a) Data collection and training process. (b) Framework of HyperLoad. (c) Schematic illustration of the KARI loss. ( represents text modality, ⃝ represents time-series modality.) (d) Schematic illustration of the EGIA mechanism. strategy that encodes critical green data center back- ground as learnable prefix vectors, enabling rapid task
Figure 4: Prediction results for different horizons. (Blue lines: ground truth; orange lines: predictions.)
Figure 5: Model performance in the data scarce setting.