Validation Results

To evaluate the accuracy of our simulator, we performed a comprehensive regression analysis comparing SCALE-Sim's predicted GEMM execution cycles against measured fusion-kernel runtimes on Google TPU v4 and TPU v6e across a wide range of matrix dimensions (<128, 128–1024, and 1024–4096).

The TPU fusion time represents on-chip execution only—it does not include HBM-to-core data transfer—therefore isolating the core compute behavior of the TPU's systolic array and fusion engine.

In each regression figure, the four sub-plots provide a complete statistical view of the simulator's prediction quality:

1. Scale-Sim Cycles vs. Fusion Average (top-left)
   • X-axis: SCALE-Sim predicted total cycles for the GEMM under the chosen dataflow.
   • Y-axis: Measured TPU fusion-kernel time (µs).
   • This plot presents the direct relationship between simulator predictions and TPU runtime, along with the fitted linear regression line.
2. Residual Plot (top-right)
   • X-axis: Predicted TPU fusion time (µs) from the regression model.
   • Y-axis: Residuals = measured fusion time − predicted fusion time (µs).
   • This view shows how prediction errors are distributed across the range of predicted runtimes and whether any systematic bias exists.
3. Actual vs. Predicted Values (bottom-left)
   • X-axis: Actual measured TPU fusion time (µs).
   • Y-axis: Predicted fusion time (µs) from the regression model.
   • The dashed diagonal line represents perfect prediction (y = x), allowing visual inspection of the fit tightness.
4. Distribution of Residuals (bottom-right)
   • X-axis: Residuals (µs).
   • Y-axis: Frequency (number of GEMM samples).
   • This histogram summarizes overall prediction error, verifying that most residuals cluster tightly around zero.

Across all six TPU and dimension configurations, SCALE-Sim demonstrates a strong linear correlation with real TPU compute behavior. The regression models consistently achieve high R² values (up to 0.99), low RMSE, and stable residual distributions. These results show that the simulator faithfully captures both compute-bound and memory-influenced GEMM performance trends on modern systolic-array accelerators.

The close alignment between predicted cycles and measured fusion times—combined with regression slopes near 1 and small intercepts—confirms SCALE-Sim's ability to model execution latency with high fidelity. The residual and histogram plots further reinforce that prediction errors remain centered around zero with small variance, demonstrating robustness across GPU generations and matrix sizes.

Overall, this validation confirms that SCALE-Sim provides reliable, cycle-level performance estimates for GEMM workloads on TPU-class architectures, enabling researchers and practitioners to explore dataflows, evaluate architectural designs, and develop cross-stack performance analyses with confidence.

SCALE-Sim supports both Weight-Stationary (WS) and Input-Stationary (IS) dataflow modes, allowing users to manually choose which stationary pattern to simulate. The simulator does not attempt to infer the TPU's actual dataflow; instead, users evaluate both modes depending on the workload. To validate that SCALE-Sim's dataflow models are correct, we compare simulator predictions under WS, IS, and the best-of-two selection against measured TPU fusion-kernel times.

The TPU time used in this evaluation is the fusion kernel runtime, representing only on-chip execution within the systolic array and fusion engine. It does not include HBM-to-core data transfers, allowing us to isolate and evaluate pure compute behavior.

Each scatter plot uses:

• X-axis: SCALE-Sim predicted total GEMM cycles under a specific dataflow configuration (WS, IS, or Best-of-WS/IS).
• Y-axis: Measured TPU fusion-kernel time (µs), i.e., the on-chip compute latency observed during execution.

When we force SCALE-Sim to use a single incorrect dataflow (e.g., simulating everything in WS even when TPU benefits more from IS), the correlation to TPU runtimes is noticeably weaker (Correlation = 0.499). In contrast, when we simulate both WS and IS and take the smaller of the two cycle counts for each GEMM, the correlation improves significantly (Correlation = 0.751).

This result demonstrates two important points:

1. SCALE-Sim's WS and IS execution models are both accurate, since the real TPU runtime consistently aligns with the better of the two predictions.
2. TPU hardware behavior frequently corresponds to whichever dataflow offers better reuse for the given GEMM—mirrored by the simulator's "best-of-two" prediction strategy.

Overall, this validation confirms that SCALE-Sim provides faithful cycle-level modeling across multiple dataflow strategies, and that its user-selectable WS/IS simulation modes accurately reflect TPU-class systolic-array behavior.