Get live statistics and analysis of Ariel's profile on X / Twitter

Just submitted my resignation from Meta. Some might say that ~one year at the company is not that long, but I survived like three rounds of layoffs, so I think it’s fair play. For what it's worth, my team is great and it was an overall positive experience. But there are other, better places to build AGI, which hopefully won't be used for stepmom chatbots and infinite slop machines.

OpenAI: ships a browser Anthropic: ships a blogpost Deepmind: solves Navier Stokes Meta: ...fuck it, let's do a layoff

>hiring an ai engineer >about to send an offer >ask candidate if he's pytorch or web dev >he doesn't understand >explain the differences between pytorch and web dev >he still doesn't get it >pull out illustrated diagram explaining what is pytorch and what is web dev >he laughs and says "i'm a good engineer sir" >hire him >import requests

"Volunteer open source maintainers have a responsibility to fix security issues in their projects" Or what? You'll fire them?

This is genuinely such a good razor to check if someone is actually doing AI, or just a browser monkey.

Sorry but "AI Engineer" is an anti-signal now. You can be a Research Engineer. An AI Researcher. An ML Engineer. A Research Scientist. A Member of Technical Staff. A Member of Engineering. All of these are respectable (and tbf mostly synonymous) titles. But if you're an AI Engineer, I assume you're a frontend developer with a superiority complex.

>Trained for just $7.8K 🤯 >looks inside >finetune every. single. time.

PhD students after getting their first conference acceptance be like

btw torchtune is officially reinforcement learning - the GRPO implementation is officially merged, the entire codebase is really clean and modifiable, so go out there, reinforcement learn your LLMs, and any contributions welcome!

Every single person who demands bringing back 4o is mentally ill, and I'm tired of pretending otherwise

So BF16 breaks RL and we should use FP16 instead, except it's actually just a problem with A100's so you're fine on newer hardware, but it's actually due to some arcane flash attention setting so you just need to check that, and otherwise we're probably fine with BF16?

It's wild to see RL people get mad at LLMs and larping RL purity, like RL is everything you need for AGI and LLMs are just some nonsense. For years, one of the biggest (top 3 at most) problems with current RL algos was the cold start problem. If you start from scratch, you're super limited in what you can actually achieve, and how quickly. A general-purpose desktop agent could *never* be trained with pure RL, and even warm start with imitation would be super finicky. In come LLMs. Monstrosities with tons of general knowledge. The perfect vessels for agent initialization, and finally some path towards practical generalist agents. But I guess training in the same games and simulations for a decade is easier.

Meta vesting date is this Saturday btw

Just submitted my resignation from Meta. Some might say that ~one year at the company is not that long, but I survived like three rounds of layoffs, so I think it’s fair play. For what it's worth, my team is great and it was an overall positive experience. But there are other, better places to build AGI, which hopefully won't be used for stepmom chatbots and infinite slop machines.

Sorry but "AI Engineer" is an anti-signal now. You can be a Research Engineer. An AI Researcher. An ML Engineer. A Research Scientist. A Member of Technical Staff. A Member of Engineering. All of these are respectable (and tbf mostly synonymous) titles. But if you're an AI Engineer, I assume you're a frontend developer with a superiority complex.

Every single person who demands bringing back 4o is mentally ill, and I'm tired of pretending otherwise

>hiring an ai engineer >about to send an offer >ask candidate if he's pytorch or web dev >he doesn't understand >explain the differences between pytorch and web dev >he still doesn't get it >pull out illustrated diagram explaining what is pytorch and what is web dev >he laughs and says "i'm a good engineer sir" >hire him >import requests

OpenAI: ships a browser Anthropic: ships a blogpost Deepmind: solves Navier Stokes Meta: ...fuck it, let's do a layoff

This is genuinely such a good razor to check if someone is actually doing AI, or just a browser monkey.

"Volunteer open source maintainers have a responsibility to fix security issues in their projects" Or what? You'll fire them?

It's wild to see RL people get mad at LLMs and larping RL purity, like RL is everything you need for AGI and LLMs are just some nonsense. For years, one of the biggest (top 3 at most) problems with current RL algos was the cold start problem. If you start from scratch, you're super limited in what you can actually achieve, and how quickly. A general-purpose desktop agent could *never* be trained with pure RL, and even warm start with imitation would be super finicky. In come LLMs. Monstrosities with tons of general knowledge. The perfect vessels for agent initialization, and finally some path towards practical generalist agents. But I guess training in the same games and simulations for a decade is easier.

Aight let's unclickbait the fp16 paper. tł;dr cool paper, a little bit overstated in comms, very overstated by poasters. The thing that gave me a pause is that on the surface, it seems to claim that bf16 is horrible, borderline unusable. But that's not really the case (nor is it the claim). Yes, the fsdp-vllm mismatch is real, and yes, it can be mitigated with fp16. This is as true as it is irrelevant, because who cares about the mismatch if the algorithm empirically works? The widely circulated figures show bf16 consistently collapsing, while fp16 runs thrive. I have no reason to doubt this data, but it's performed on a very particular - and rather small - dataset. If you don't go through multiple epochs of your data, you probably won't see such a severe collapse - as many, many people can confirm from their own experiments. Does this mean that fp16 is useless? Not necessarily. As always, there are trade-offs. It might work better in some cases, but maybe you have to fuck with loss scaling. It might be worse in a big data regime, but crucial in a low data regime. Hard to tell for now. But alas, you still have to pay for your lunch.

>Trained for just $7.8K 🤯 >looks inside >finetune every. single. time.

We actually need more gatekeeping, democratization of AI went too far. One time I was on a project with a guy who was hired explicitly as an LLM expert. He had some setup issues, I offered to take a look in case I can help. Dude was going through HF fine tuning tutorials 💀

So BF16 breaks RL and we should use FP16 instead, except it's actually just a problem with A100's so you're fine on newer hardware, but it's actually due to some arcane flash attention setting so you just need to check that, and otherwise we're probably fine with BF16?

JS devs not beating the allegations, huh?

The thing about RL envs, or RL env libraries, is that envs should be completely independent from the algorithmic details of the learner. An env shouldn't know anything about vLLM, or about wandb, or about OpenAI clients - it should just implement the internal env logic.

This is your daily reminder that PPO (GRPO, TD3, ...) is *not* a policy. It's an algorithm to train policies. If you make such a fundamental category error, frankly, you have no business writing about RL.