All articles
6 min readai / productivity

95.5% to 89.0%: stop asking a model to check its own work

In the AI trainings I run, someone reaches for the same move after a weak answer: "review your work and correct any mistakes." The model apologizes, rewrites, and hands back something that reads tighter. A Google DeepMind team put that move on three reasoning benchmarks and counted the outcomes. The second answer scored lower than the first.

Self-review moves the number down.

Jie Huang, Xinyun Chen and co-authors named the setup intrinsic self-correction: the model reviews its own answer with nothing the first pass lacked, no label and no tool. On GSM8K, GPT-4 answered 95.5% of problems correctly with standard prompting. One round of self-review took it to 91.5%, a second round to 89.0% 1. GPT-3.5 on CommonSenseQA fell from 75.8% to 38.1% after a single round 1.

Look at what the review prompt carries. Asking for a correction implies that one exists, and the model obliges by editing something. Most answers stay put, and among those that move, a correct answer turning wrong is the more frequent outcome 1.

Locating the error is the part that fails.

Gladys Tyen and colleagues at Google built BIG-Bench Mistake to isolate that step: 2,186 chain-of-thought traces across five tasks (Dyck languages, logical deduction, multistep arithmetic, tracking shuffled objects, word sorting), each annotated with the position of the first incorrect step. The best score on finding that step, across prompting methods and models, was GPT-4 at 52.87% overall accuracy 2. Hand the same models the error location and they repair the trace well 2, which puts the bottleneck in localization rather than in the editing.

Kaya Stechly, Karthik Valmeekam and Subbarao Kambhampati pushed on the same question across Game of 24, graph coloring and STRIPS planning. Self-critique collapsed performance, and a sound external verifier raised it 3. Their graph coloring analysis carries the detail I quote most often in sessions: the correctness and the content of the criticism had little bearing on the result. Gains from iterative prompting traced back to the right solution appearing somewhere in the model's top-k completions, with an external verifier recognizing it 3.

Feedback from outside the model pays.

Xinyun Chen, Maxwell Lin, Nathanael Schärli and Denny Zhou gave the model execution results instead of introspection. Their Self-Debugging setup has the model explain its code line by line and inspect what the code does when it runs. On TransCoder and MBPP, where unit tests exist, accuracy rose by up to 12% over the baseline. On Spider, a text-to-SQL benchmark with no tests to run, the explanation step bought 2% to 3%, and 9% on the hardest problems 4.

Theo Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao and Armando Solar-Lezama measured the gap between a model's feedback and a person's. They swapped GPT-4's own explanation of its buggy code for an explanation written by a human programmer. The share of repaired programs passing the tests went from 33.3% to 52.6%, a factor of 1.58 5. Counting the tokens spent on repair, self-repair gains on APPS were thin: 63.1% to 66.1% under one sampling budget, and 63.4% down to 61.8% under another 5.

Ryo Kamoi and colleagues surveyed the self-correction literature for TACL and reported three conclusions. No prior work demonstrates successful self-correction with feedback from prompted LLMs, outside tasks exceptionally suited to it. Self-correction works where reliable external feedback is available. Large-scale fine-tuning installs it 6.

Training can install the skill, prompting cannot.

Frontier models have moved since 2024, and that third conclusion says how. Aviral Kumar and co-authors trained for the behavior directly with multi-turn reinforcement learning: their SCoRe method gained 15.6% on self-correction for MATH problems and 9.1% on HumanEval over the base Gemini models 7. A model trained that way will beat the ones Huang tested, and no wording in your prompt reproduces the gain on a model that never got the training.

Two limits on all of this. These benchmarks have one checkable answer each, so the findings transfer to arithmetic, code, extraction and multi-hop lookup, less so to tightening a paragraph where you judge the result in 5 seconds. And the obvious workaround, asking a second instance of the same model to review, carries its own bias: Arjun Panickssery, Samuel Bowman and Shi Feng found that models rate their own generations above other models' and humans' text that human annotators score as equal quality, and that the bias tracks the model's ability to recognize its own output 8.

The protocol.

1. Delete "check your work" from your prompts. On a reasoning task it costs accuracy, and the cost compounds over rounds 1.

2. Name the step and the test instead. "Recompute step 3 and show the division" or "verify the date arithmetic in line 4 against a 2026 calendar" points at a location. Models repair what they are pointed at 2.

3. Give the model a signal it did not have. Paste the stack trace, the failing test output, the quoted source paragraph, the number from the actual spreadsheet. Execution feedback moved MBPP and TransCoder by up to 12%; introspection moved nothing 4.

4. Write the test before you ask for the fix. On anything with a verifiable output, 10 minutes spent on a test or a worked example with a known answer buys more than any review prompt. Where no test is possible, expect 2% to 3% 4.

5. Send the review to a different model, in a fresh chat. Give it the task and the output, hold back the reasoning that produced it, and never tell it who wrote the draft 8.

6. Cap reasoning tasks at one revision, then compare the two answers yourself. Keep both. Disagreement between them is a signal to go to the source, not a sign the second one won 1.

7. Paste your own failure list as the review prompt. Mine, for AI output: an invented citation, a confident number with no source, a date silently changed, a dropped constraint from the brief, a summary that adds a claim the document never made. A named checklist beats "any mistakes?".

8. Measure before you adopt any of this. Take 20 tasks you run often, answer each twice, once direct and once with your review step, and count the errors on each side. The published effects run in both directions depending on the task 36.

Open your most reused prompt today and cut the sentence asking the model to verify itself. Put a runnable test, a quoted source or a second model in its place, then run both versions over 20 items this week and count what each one gets wrong.

Sources.

  1. Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2024). Large Language Models Cannot Self-Correct Reasoning Yet. ICLR 2024. arXiv:2310.01798
  2. Tyen, G., Mansoor, H., Carbune, V., Chen, P., & Mak, T. (2024). LLMs cannot find reasoning errors, but can correct them given the error location. Findings of ACL 2024, 13894-13908. aclanthology.org/2024.findings-acl.826
  3. Stechly, K., Valmeekam, K., & Kambhampati, S. (2025). On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks. ICLR 2025. arXiv:2402.08115
  4. Chen, X., Lin, M., Schärli, N., & Zhou, D. (2024). Teaching Large Language Models to Self-Debug. ICLR 2024. arXiv:2304.05128
  5. Olausson, T. X., Inala, J. P., Wang, C., Gao, J., & Solar-Lezama, A. (2024). Is Self-Repair a Silver Bullet for Code Generation? ICLR 2024. arXiv:2306.09896
  6. Kamoi, R., Zhang, Y., Zhang, N., Han, J., & Zhang, R. (2024). When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs. Transactions of the Association for Computational Linguistics, 12, 1417-1440. aclanthology.org/2024.tacl-1.78
  7. Kumar, A., Zhuang, V., Agarwal, R., Su, Y., Co-Reyes, J. D., Singh, A., Baumli, K., Iqbal, S., Bishop, C., Roelofs, R., Zhang, L. M., McKinney, K., Shrivastava, D., Paduraru, C., Tucker, G., Precup, D., Behbahani, F., & Faust, A. (2025). Training Language Models to Self-Correct via Reinforcement Learning. ICLR 2025. arXiv:2409.12917
  8. Panickssery, A., Bowman, S. R., & Feng, S. (2024). LLM Evaluators Recognize and Favor Their Own Generations. NeurIPS 2024. proceedings.neurips.cc

Keep reading