@lthlnkso: Evaluating the accuracy of pangram labs at detecting AI. #AIDetector #LLM #ArtificialIntelligence.

Quick Thoughts
Quick Thoughts
Open In TikTok:
Region: US
Wednesday 01 April 2026 19:23:36 GMT
61775
4178
95
129

Music

Download

Comments

br1sg6
B :
Actually super impressed that there were 0 false positives in ~1000 documents. I understand that not everyone is looking for an “ethical” AI detector, but not producing false positives might be the most important selling point to me.
2026-04-01 19:32:44
847
eclipse1233
Eclipse :
wouldn't the fact that the documents you chose were all publicly available texts likely in the different llms data set. is it accurate detection or just a bot that knew which ones were fully human written
2026-04-01 22:59:54
147
fafobacon
FAF0 BACON :
As someone who is building a unique Ai-first, cognitive dissonance fact check platform, what an honor it would be to have QT fact check the fact checker! Nice work.
2026-04-02 11:38:54
4
hildebertoid
hilderbertoid :
1% false positives is way too high to use to detect AI written text in a school setting considering the consequences of such an accusation and the number of essays, papers etc. written by each student every year.
2026-04-02 14:35:50
28
vpetewalker
Peter Walker :
Maybe this is a stupid question but since these are publicly available documents that you’re using to test the AI on, what’s to say it wasn’t already trained on those and just knows out of the box that those are not AI generated documents? Tyvm
2026-05-01 17:16:40
4
finitebanshee
Finite :
I feel that detecting AI-written text has actually gotten easier with newer models, as they are more heavily fine-tuned towards a particular style. Older models looked more like raw internet text.
2026-04-01 19:48:20
27
khadembadiyan
khadem :
Are all of the samples you used for AI generated text one shot? I’m curious of the performance of one shot vs iterative generation where it’s also completely “written” by an LLM but a human was trying to get the LLM to not write in an identifiable LLM style
2026-04-07 18:37:04
6
tubegobz24
Susan Sutton :
The issue is the ai text you used needs to be muddied. You need to feed samples to Ai and say something like “using the tone and style from these samples write me …” that way the paragraph of Ai text is not just generic tone. That will usually make any dector fail 70%+ of the time
2026-04-22 22:08:34
3
miggymf
Miguel Fuentes :
shout out to the homie Katherine, their founding research scientist
2026-04-01 21:33:02
16
lthlnkso
Quick Thoughts :
Should add: I think it's less accurate with short text. I was using 5 paragraphs. With a short text, I expect closer to 1% FPR.
2026-04-01 21:08:39
16
imaginaryman7
imaginaryman :
Every student should memorize this web site and save your thoughts. I would submit all writings to check and fix any that hit. Anytime a claim is made against them, they should have the professor use this web site, as an example of a better tool than a crappy tool, and check and confirm AI or not, and if they don’t relent, raise the issue with the department and get the professor to stop using garbage tools. And, when the next tool comes along that is better, everyone should switch to it. Thank you for your thoughts. May this save a non AI using person from a false claim of AI. 🥰
2026-04-01 19:50:36
8
coolseandotcom
Sean Holloway :
As a former radar engineer it feels surreal to encounter an ROC curve out in real life
2026-04-02 05:30:04
7
reid8208
reido :
I’m happy to hear they are prioritizing specificity, rather than sensitivity. That’s the way to go, if they want to avoid lawsuits from people whose lives are ruined from false positives.
2026-04-01 19:46:30
70
rocky123469
Rocky1234 :
Heyyy I was the one who joined your live and told you that AI detectors are accurate!!! I’m so glad you took what I said and researched and made a video about it.
2026-04-01 21:10:28
14
cumberlandcarnivores
Cumberland Carnivores 🇨🇦 :
Try running it through CopyCleanse first next time. You'd be surprised at how much of that detection is just picking up AI watermarks and nothing more.
2026-04-02 01:15:29
5
scott_bucher
Scott Bucher :
Any concerns that it is detecting that something isn’t AI by the fact that it was a piece of text that predates LLMs (at least predates when they started fully being used)? Essentially, could their api be checking against a certain set of data that they know isn’t AI because of its publishing date and thus perform better on your tests?
2026-04-01 19:50:38
8
digitalosmosis4
digitalosmosis :
The confidence you need to have in your own product to give free credits to Quick Thoughts is the most impressive part of this video.
2026-04-01 22:10:12
6
ericjaakkola
ericjaakkola :
Take a human document, have ai make suggested changes and see if it's detected as Ai written
2026-04-01 22:16:58
0
jcass358
JCass358 :
Nice
2026-04-03 01:08:00
3
dataenthusiast
. :
You’ll have to retest with new model releases. Models will get better at writing over time
2026-04-02 07:10:51
3
coogalier
Coogalier :
Love this experiment
2026-04-01 22:19:13
3
m.vale.advocate
Maggie :
I tried this with a paper I wrote in 2016 for school (that was never shared online) and it flagged it as AI assisted.
2026-04-02 05:05:05
4
iamjustinkingsbury
Justin :
Dude nice job 😂😂😂 where do you find the time
2026-04-02 04:00:11
3
kelnishi
KelNishi :
Would love to see how video transcripts and comment sections faire.
2026-04-03 08:18:25
2
To see more videos from user @lthlnkso, please go to the Tikwm homepage.

Other Videos


About