The Illusion of Reasoning: Exposing Evasive Data Contamination in LLMs via Zero-CoT Truncation

Researchers introduced Zero-CoT Probe, a black-box method to detect data contamination in large language models.

The method truncates chain-of-thought outputs and compares performance on original and perturbed benchmarks.

This helps uncover both direct memorization and stealthy, paraphrased contamination that can hide behind generated reasoning.

The approach could improve trust in benchmark results, reasoning evaluations, and leaderboard claims.