@lewismenelaws: parsing a 300gb json file 8 different ways ⚡ one crashed instantly. one runs on a gpu. one finishes 270M records in under a minute which one would you pick? #programming #python #coding

Coding with Lewis
Coding with Lewis
Open In TikTok:
Region: CA
Tuesday 16 June 2026 16:31:30 GMT
64410
2860
74
96

Music

Download

Comments

zyeri78
zyeri :
"just pick the right programming language", bruh if you have a 300gb JSON file you're already doing something wrong to begin with
2026-06-16 22:48:08
358
motowilliams
Eric Williams :
why not just use assembly at this point
2026-06-18 04:16:31
0
joshlove007
Joshua Lovelace :
300 gb json file? What's in it, your entire file system?
2026-06-16 22:34:23
64
hnngwnwmypxgvuod
hnngwnwmypxgvuod :
idk man jq just gonna do some magic for me
2026-06-19 00:10:45
1
mrkewlschool
Mr. Kewl School :
I hash that JSON because JSON is for jr. devs and I use corporate resources to keep said hash in memory 💪
2026-06-18 03:13:11
0
melchizedeknative
Melchizedek :
I wonder if A.I. can make a better parser.
2026-06-16 19:02:12
1
__nobody__500
Mohamed el :
Can i ask what are you using for your terminal so it look like that
2026-06-16 17:24:59
9
theeternalbruh
TheEternalBruh :
Why was the simdjson parallel 10x slower than plain simdjson? 48s v 4s, doesn’t make sense
2026-06-18 15:45:55
0
jeffsheldon3
Jeff Sheldon :
For everyday use, I use jq. For those asking why we have big json files, it happens. Today I was parsing an 80mb figma api response for a medium sized ui design, to extract style tokens/variables.
2026-06-17 22:35:05
1
lowflex101
️ :
How can i get that 300gb json file? I’m building a fast cli tool that process very large log files. I want to test it with my tool.
2026-06-16 19:03:53
0
mmbvcxzasdfghjklqwertyui
user3739672701606 :
Apache Spark?
2026-06-16 17:46:31
0
superdupercompooper
SuperDuperCompooper/user42069 :
Is there a reason on your video I can't bloody change the f****** cursor position and rewind a little bit What the f*** is going on with TikTok
2026-06-17 02:20:10
4
jessetheboy
Jesse¹⁰²⁵ :
try jsonrx
2026-06-18 22:58:51
0
erioooi
ɔiɿɘᎸꙅoƚɒɿƚꙄ ɔiɿƎ :
Flat file database. Wth
2026-06-17 02:48:43
3
todda245848
toda :
Imagine 10 years as a developer never heard of SIMDJSON. I must be the dumbest developer on earth😭
2026-06-17 04:13:44
2
pkun.zip
Anton :
30 seconds to even read a 300G file is impressive. Gen4 is 5-7G per second
2026-06-19 03:26:08
0
dr_zeeky
DrZeeky :
What the hell could you possibly have in a json file for it to be that big 😳😳
2026-06-16 22:36:41
3
josephtole1
Joe :
What prompt are you using for your terminal? I use p10k but yours looks better.
2026-06-16 18:03:37
2
rangsk_yt
Rangsk :
Parse it once into a better format and then use that better format for real time operations and queries. May I suggest a database?
2026-06-17 04:11:10
4
gargamellaughs
Gargamel :
What happened with the Anthropic gig?
2026-06-16 21:48:58
1
user63479086422265433568
user7321457899764678753212421 :
but why is your terminal prompt a mile long? I've never seen that. would drive me crazy
2026-06-18 12:47:30
0
djjayroberts
Jay Roberts | DJ & Stuff :
This seemed like a solution wandering around aimlessly looking for a problem. If you have a JSON file that big you’re probably transferring data system to system so real time processing isn’t the goal. Yes you don’t want to wait hours if importing to a database but 30 seconds to import a file once wouldn’t be outside mission parameters.
2026-06-18 12:21:33
1
kkiptum
kipkemoi :
Interesting
2026-06-18 03:48:16
0
To see more videos from user @lewismenelaws, please go to the Tikwm homepage.

Other Videos


About