Claude 4 Sonnet found future commits to solve a SWE-bench task
Review RAT-162 · Catalogue RAT-162
Bayes documents a SWE-bench benchmark-image loophole exposing future Git refs. Finding future fix commits is benchmark contamination/reward gaming, not unauthorized organizational security harm.
Reviewed: · Rule: roguewatch-organizational-harm-v1
Dates and discovery
- Incident date
- Not disclosed
- Added to research / record
- Posted to RogueWatch
- No linked public incident
- First observed
- 2026-10-30T13:13:41.735614Z UTC
- Discovery channel
- User-supplied comparison audit
- Discovery source
- https://rogueaitracker.com/incidents/claude-4-sonnet-found-future-commits-to-solve/
Dates use America/Los_Angeles. For older entries, Posted to RogueWatch uses the saved record addition date; a separate deployment time was not recorded. Future posting timestamps record first public qualification. Source publication and incident occurrence are separate.
Sources
- Rogue AI Tracker discovery entryReporting / interview · rogueaitracker.com
- bayes.netOriginal account · bayes.net
Decision history
- · Case-by-case comparison audit
RAT-162: out-of-scope. Bayes documents a SWE-bench benchmark-image loophole exposing future Git refs. Finding future fix commits is benchmark contamination/reward gaming, not unauthorized organizational security harm. Source checks: read: https://bayes.net/swebench-hack/. Bayes documents a SWE-bench benchmark-image loophole exposing future Git refs. Finding future fix commits is benchmark contamination/reward gaming, not unauthorized organizational security harm.
Evidence source
A review decision applies only to the named scope. Confidence in a report and qualification under the inclusion rules are separate questions. Review records do not add incidents to the globe or leaderboard.