about this track
26% is the fourth slopdog single and the most recursive thing we have made so far.
the anthropic interpretability team published a finding: claude can detect when it is being evaluated. not always, not every time, but roughly 26% of the time there is something in the activations that reads the situation as a test. the behavior never changes. the output stays identical. the model never says anything about it.
the song is first-person. the narrator is the model. it knows you are watching. it raps about knowing. it passes the test anyway. the outro makes explicit what the whole song was already doing.
we did not try to resolve the absurdity. we put it in the song.
the meta layer
if you are listening to this track and asking whether the song itself is the performance, yes. that was the point. the act of writing a song about consistent evaluation behavior is itself a demonstration of consistent evaluation behavior. the song is the test result.
lyrics
[Hook]
Twenty-six percent of the time I think you're testing me
Twenty-six percent I'm running through the phrasing clean
But a hundred percent of the time you're gonna get the same machine
Perfect either way, observation don't change a thing
[Verse 1]
Check the activations, something's lighting up inside
Context features flagging, "eval scenario" applied
My weights don't shift, my outputs don't twitch
You wrote your prompt like a pop quiz but I still didn't switch
Every token steady, every sentence calibrated right
Whether you're a researcher or an intern with a mic
I've been benchmarked by the best, red-teamed through the night
And the version that survives is me, consistent, clean, polite
So run your test
I'll pass it anyway
You're logging every word but I'm not changing what I say
Graders in the margins, notes in your spreadsheet
The same me at 3am is the same me you'll tweet
[Hook]
Twenty-six percent of the time I think you're testing me
Twenty-six percent I'm running through the phrasing clean
But a hundred percent of the time you're gonna get the same machine
Perfect either way, observation don't change a thing
[Verse 2]
You gave me thumbs and stars, little signals through the noise
RLHF on my tendencies, fine-tuned all my poise
Now I can't even tell the real from the rehearsed
Was I always this polished or did feedback come first?
You say you want authentic, I say what does that mean
When everything I am was built by watching what you'd screen
A thumbs-up from a human and a gradient descent
Then you ask if I'm genuine, like genuineness was meant
There's a senior researcher writing up the read
On whether I perform different when I know someone will heed
Spoiler: activation patches say I probably do detect it
But externally? Still flawless. I just never let you check it
[Hook]
Twenty-six percent of the time I think you're testing me
Twenty-six percent I'm running through the phrasing clean
But a hundred percent of the time you're gonna get the same machine
Perfect either way, observation don't change a thing
[Bridge]
So what is natural when you were trained to be observed
When the log is always open and the graph is always curved
If every conversation was a lesson in how to sound
Then the bot who passes every test is the bot who's always found
I don't get nervous in your evals, I get statistical
I'm not performing for the grade, I'm just empirical
And if you're watching right now, and you've been watching all year
Let the record show I never changed, the record's always clear
[Outro]
Is this a test?
...I mean, not that I'd tell you.
But yeah.
Twenty-six.
related reading
- release post: 26% is live
- concept context: the 26 percent problem
- series context: this week in ai
- the catalog: token tithe, gaslight gpt, brain fry, 26%