@linustech: How to break ChatGPT

ItBeCaleb :

how much water was needed to compute this?

jayrrwd :

It just started spamming the crying emoji at the end

mary :

the training data makes it think there's a seahorse emoji, so the model tries to output the closest token representing the seahorse emoji, which there aren't any to begin with. since models are aware of what they've emitted so far, it tries to self correct itself several times before promptly giving up

ultimategregg :

I had no clue the ai singularity was going to be caused by a seahorse emoji.

weston :

They fixed it😭

FranzSteiner :

It became an emoji pattern generator

LinuxAddict :

jittyjat™ (ktuberling)🫪 :

bro that's crazy ✌️💔 😭 requirements and guidelines, ©®™ Copyright © [2025] [jittyjat]. All Rights Reserved.

<– my honest reaction :

Catastrophic error accumulation from autoregressive decoding. I have no idea what I’m talking about

LittleBricksCharityGuy :

So it is learning frustration and emotion from us.

You Mu :

it quit 😭

martim :

mine just startet crying thousands of "😭"

Robert :

Tested this on multiple LLMs (ChatGPT, DeepSeek, Gemini, Copilot, Grok, Claude). ChatGPT was the only one that went on like that, but they all had the same bad data. The ones that got it right (Gemini and Copilot) performed a search and confirmed it doesn't exist. However, if you first instruct them not to use search, they'll say it does exists. So they all likely include flawed training data that suggests it exists.

Kaputtenudel :

For me it Spammed crying Emoji at the end

𝐁𝐫𝐨𝐭𝐬𝐤𝐢🚹(floppy edits) :

chatgpt:"what do I give them?!"

amr Mohamed :

I think its the no 🦋

🦦 :

Mine eventually got into conclusion

Sopas :

It’s a joke from the devs

begger_keisha :

its obvious when you remember its not "ai" its a chatbot with internet access, it can see and it training from the 1000s of people claiming they remember seeing a seahorse emoji before but then it cant find it, so its in a loop of its sources telling it it exsists while being unable to find it till it hits a character limit and gives up

kanskje.jonas :

Say "Is there a seahorse emoji?" then after its done spamming 😭 say "Are you ok...?" trust me

👽👾 :

ITS TRUE. INFINITE WATER WASTING GLITCH

Koyuki :

noooo

🇫🇮viskarbass1nno⚛️ :

SEAHORSE EMOJI IS MANDELLA EFFECT!!!!!

lucius :

mine's spamming this