@lewismenelaws: parsing a 300gb json file 8 different ways ⚡ one crashed instantly. one runs on a gpu. one finishes 270M records in under a minute which one would you pick? #programming #python #coding

Coding with Lewis
Coding with Lewis
Open In TikTok:
Region: CA
Tuesday 16 June 2026 16:31:30 GMT
102293
4516
100
142

Music

Download

Comments

zyeri78
zyeri :
"just pick the right programming language", bruh if you have a 300gb JSON file you're already doing something wrong to begin with
2026-06-16 22:48:08
608
joshlove007
Joshua Lovelace :
300 gb json file? What's in it, your entire file system?
2026-06-16 22:34:23
116
mzati.exe
PlasticFoods :
me when I heard 300gig json
2026-06-16 20:10:33
318
alotofnumbersreal
pipis :
How it feels comparing parsers and languages without knowing CPU-bound from IO-bound tasks.
2026-08-15 21:28:33
1
__nobody__500
َ :
Can i ask what are you using for your terminal so it look like that
2026-06-16 17:24:59
12
mmbvcxzasdfghjklqwertyui
user3739672701606 :
Apache Spark?
2026-06-16 17:46:31
0
theeternalbruh
TheEternalBruh :
Why was the simdjson parallel 10x slower than plain simdjson? 48s v 4s, doesn’t make sense
2026-06-18 15:45:55
1
melchizedeknative
Melchizedek :
I wonder if A.I. can make a better parser.
2026-06-16 19:02:12
1
dgubz
dgubz :
Me personally I can do it in a blink of an
2026-07-12 22:50:04
0
dr_zeeky
DrZeeky :
What the hell could you possibly have in a json file for it to be that big 😳😳
2026-06-16 22:36:41
4
hnngwnwmypxgvuod
hnngwnwmypxgvuod :
idk man jq just gonna do some magic for me
2026-06-19 00:10:45
2
mrjay42__
Jay :
Possible stupid question: if there's 300GB of data, ultimately your bottleneck is going to be IO? Typically on the multi thread approach, there is a number of threads over which the speed of your SSD/HDD is going to be the hard limit, isn't it?
2026-07-12 01:21:14
1
djjayroberts
Jay Roberts | DJ & Stuff :
This seemed like a solution wandering around aimlessly looking for a problem. If you have a JSON file that big you’re probably transferring data system to system so real time processing isn’t the goal. Yes you don’t want to wait hours if importing to a database but 30 seconds to import a file once wouldn’t be outside mission parameters.
2026-06-18 12:21:33
4
erioooi
ɔiɿɘᎸꙅoƚɒɿƚꙄ ɔiɿƎ :
Flat file database. Wth
2026-06-17 02:48:43
4
mrkewlschool
Mr. Kewl School :
I hash that JSON because JSON is for jr. devs and I use corporate resources to keep said hash in memory 💪
2026-06-18 03:13:11
0
lowflex101
️ :
How can i get that 300gb json file? I’m building a fast cli tool that process very large log files. I want to test it with my tool.
2026-06-16 19:03:53
0
_niiboye_
Nii Boye 🇬🇭🎮 :
try it with Zig lang and V lang and let me know the outcome
2026-06-17 19:00:51
0
kubernetass
uwuosas :
Who is Json? And why is 300gig?
2026-07-05 19:44:43
0
superdupercompooper
SuperDuperCompooper/user42069 :
Is there a reason on your video I can't bloody change the f****** cursor position and rewind a little bit What the f*** is going on with TikTok
2026-06-17 02:20:10
4
zylex47
Zylex :
What is parse what is a json file
2026-06-16 16:37:55
3
todda245848
toda :
Imagine 10 years as a developer never heard of SIMDJSON. I must be the dumbest developer on earth😭
2026-06-17 04:13:44
2
jeffsheldon3
Jeff Sheldon :
For everyday use, I use jq. For those asking why we have big json files, it happens. Today I was parsing an 80mb figma api response for a medium sized ui design, to extract style tokens/variables.
2026-06-17 22:35:05
1
rangsk_yt
Rangsk :
Parse it once into a better format and then use that better format for real time operations and queries. May I suggest a database?
2026-06-17 04:11:10
4
josephtole1
Joe :
What prompt are you using for your terminal? I use p10k but yours looks better.
2026-06-16 18:03:37
2
gargamellaughs
Gargamel :
What happened with the Anthropic gig?
2026-06-16 21:48:58
1
To see more videos from user @lewismenelaws, please go to the Tikwm homepage.

Other Videos


About