I Can't Believe Rust is Replacing Java

So XAI just open sourced the X

recommendation algorithm and they

massacred my boy. Scrolling down, I

expected to see language stats like that

of the 2023 repo. But no, they were

gone. The only languages listed, Rust

and Python. So this wasn't just a simple

refactor of the algorithm. It was a

complete rewrite. One that had me

thinking, wait a second, I thought Rust

was supposed to replace C and C++, not

Java and Scala. So I wanted to take a

closer look and figure out why replace

Java with Rust. And luckily for me, we

have access to both code bases. So we

can take a look at exactly what they

replaced like Twitter's custom scholar

framework product mixer which they use

to build services like homemixer what

serves the for you recommendation feed

and systems that produced handgineered

systems like sim clusters tweep cred

real graph early bird and dozens more

interconnected microservices that made

up this algorithm. So the engineers

would literally sit down and say, "hm,

let's add a feature for whether the

author of a post replied to the comments

and weight it at 75." That's

handineering.

Yeah, that all got completely ripped out

of the codebase. No more handgineered

features or in other words, no more Java

and Scala because these are the

languages that were used to build those.

So those dozens of interconnected

microservices went away and after it was

all rewritten which we'll dive into we

were left with this four directories

four components that is the entire

algorithm kind of we'll get into that

rewrite here in just a second but we

need some context first you see

homemixer which was our last little bit

of scaline java left was entirely

rewritten in rust it still serves the

for you feed but with a completely

different approach that I'll explain.

Thunder, also written in Rust, is an

in-memory post store. Candidate

pipeline, Rust again, is just a generic

framework for building recommendation

pipelines. Here's what those pipeline

traits look like. And Phoenix written in

Python, which is the heart of it all. It

has retrieval for out of network, but

also a Grockbased transformer, which

does the actual ranking of your post and

what you should and shouldn't see based

on your interactions and your behavior.

That's the new system. But in the old

system, pipeline components were in

Scala classes, extending product mixer

traits. Here's one of their scores. It's

Scala's scorer trait. Now, here's the

Rust equivalent. It's the same pattern,

traitbased pipeline composition, but

Rust gives you, as much as I hate to

admit it, uh quite a few perks like

async native code, explicit thread

safety with send and sync bounds, and

zero garbage collection overhead. Not

only that, but with Rust, as we know,

the compiler enforces correctness.

Whereas with Scala, a lot of it is left

to runtime. The old algorithm also had

explicit hand-tuned weights. So like how

I said you reply to comments plus 75 or

if you report a post minus 369. All of

these numbers were handpicked by humans

presumably throughout numerous meetings

with lively debates and fine-tuned over

the years. And most of these

heruristics, just like the handgineered

features, ripped right out of the

codebase, because Phoenix, the Gro's

transformer, doesn't need them, at least

how they were. It learns engagement

patterns directly from your behavior.

Which is why when you click on a single

political post, your entire feed is just

filled with politics, hoping you get

caught up in it and stay on the platform

forever, arguing with what are probably

foreignrun bots. Now, back to HomeMixer

real quick. And the old algorithm

homemixer orchestrated the dozens of

services calling sim clusters for

community data, tweet cred for

reputation scores, real graph for

relationship strength, then combining

all of those signals with the hand-tuned

weights. That was the algorithm. Now

homemixer just fetches candidates from

thunder in network and Phoenix retrieval

out of network, then feeds it to the gro

transformer in Phoenix to score. It all

got simpler. The codebase shrank because

the intelligence moved into the model.

But there is an issue with this codebase

and that is well it's not reproducible

for us. We have no model weights. There

are no cargo.l files. So we can't even

compile it if we wanted to. The proto

definitions are external and all of that

logic from the old algorithm is now

inside the model which we can't see. We

went from a 48 million parameter mask

net running inference to a gra

transformer that absorbed what all of

those microservices used to do which I

think is a clear reason why XAI replaced

Java and Scala with Rust in Python. Now

there's no official engineering blog yet

but we can connect the dots. It's not

that all of these microservices and the

codebase was rewritten in Rust and

Python, but that it was all absorbed by

the model and they wrote the Rust and

Python code to really just connect and

orchestrate everything. And for good

reason, I think. Thunder, the in-memory

post store, well, it needs

submillisecond latency for real-time

serving. And on the JVM,

garbage collection pauses are

unpredictable. This is JVM's kryptonite.

At least in my experience, it's always

been my biggest complaint about it. And

while you'd be fine 99% of the time,

you'll randomly get a 200 millisecond

pause right when you're serving a feed

request, which I would say is

unacceptable. And that's why X decided

to use Rust. But this seems to happen

more on X than ever before. So, I don't

know if they're just vibe coding their

Rust code or or what they're doing over

there because it ain't working right.

And also, all of Elon's companies use

Rust in one way or another from Tesla

and SpaceX. It seems that XAI was built

from day one with this exact stack in

mind. We are looking at a genuine

architectural shift in the industry.

Discord rewriting a lot of Go in Rust.

Cloudflare building a whole EngineX

replacement in Rust Linux accepting Rust

into the kernel. And now this a decade

of Scola microservices being replaced by

four Rust components. one is technically

Python and a transformer that learned to

do what a thousand lines of hand-tuned

code used to do. The question isn't even

whether Rust is replacing Java or Scala.

It's whether AI is AI is replacing

algorithms that used to be this vast

codebase. And Rust, well, it just

happens to be the language that people

reach for to glue all of the pieces

together. And if you want to do that,

you need to understand how these systems

actually work. That's why I've been

using Brilliant because their how AI

works course actually walks you through

transformer architecture and attention

mechanisms, the stuff powering the

systems like Phoenix and Grock. But

what's nice about it is that you're not

watching lectures. You're solving

problems until the concepts actually

click. Because if you've ever thought,

I'm not a math person, it it is likely

due to how you were taught, not because

you're not a math person. And Brilliant

accounts for this in all of their

programs from math and physics to

computer science and programming by

personalizing everything. They start you

at the right level and they keep pushing

you forward as you get better. It's also

a very good way to just kill some

downtime because you're able to do a lot

of these problems on your phone. So if

you're sitting there waiting for your

doctor or waiting for whatever, you can

just knock out a couple of these instead

of scrolling. Do something that actually

helps you. So, if you want to actually

understand what's happening when a model

learns engagement patterns instead of

relying on handtune weights, go to

brilliant.org/forest

night free for 30 days and my viewers

get 20% off annual. Just click the link

in the description. If you want me to go

deeper on any of this, the Grock

architecture, the pipeline patterns, or

how this compares to Meta or Tik Tok,

just let me know in the comments.