Get live statistics and analysis of Maxime Rivest 🧙‍♂️🦙🐧's profile on X / Twitter

Honey, if we buy 2 gpus (for 35 000$) we could probably save 200$/month in AI subscriptions, what do you think?

This reminds me how Anthropic was the first to introduce: Projects Canvas Computer Use Coding CLI MCP and now Skills. We can say many things about Anthropic, but we cannot say that they don't actively innovate and share regarding LLM usage and ergonomics.

This morning, I thought I'd push Claude Code a little. I asked it to get speech-to-text running on my laptop using NVIDIA's 0.6b Parakeet model. Less than two hours later, I had speech-to-text better than OpenAI's Whisper running locally on my laptop CPU, transcribing as a stream while I spoke. And I never opened anything other than Claude Code for that project—it all happened in one context window, including dependency installs and everything! We'll have to be even more ambitious next time.

Today, I pruned 87.24% of Qwen 30B for a sentiment classification task while keeping 100% of its accuracy. This means we get to use big models on gpus with not that much RAM (potentially running models that would normally require an H100 on 3090 type gpus)! Imagine pruning Kimi-k2, GPT-OSS, or Qwen Coder. MoE models can be pruned for task-specific generation, which is particularly relevant for DSPy programs, as each signature is by definition task-specific. Why is nobody talking about that? Why is there no tools for pruning like thre are for inference? What we need to figure out are questions like: Does Qwen Coder use only 15% of its weights when coding in, say, R statistics? Or even more specifically, when creating ggplot visualizations in R? If so, you could feasibly run Qwen Coder's relevant experts on consumer GPUs to write code for your data visualizations.

A simple introduction to DSPy Whole script at the end. DSPy is simple and powerful. It is the best way to build LLM software right now. Despite that, lots of people keep putting off learning it. I know I did—for a whole year! I

I started this morning, and it's now ready for you to try: > uv pip install maivi > maivi That will get you a free and open-source voice-to-text model running locally on your CPU/laptop—you don't need a GPU! The model is (according to benchmarks) better than Whisper, and our implementation is faster to full results than Groq because it transcribes faster than realtime! The transcription will land in your clipboard, you can make it auto-paste too. P.S. It's supposed to be working on Mac and Windows, but I've only tried it on Linux so far. Enjoy! Link to repo in comments.

Beautiful! You can now do: > uv pip install mcp2py dspy And in just 6 lines of Python code, you have an AI agent that can retrieve information through Google Chrome MCP DevTools. With the added bonus that you're only 1 or 2 steps away from doing prompt optimization for that agent with Gepa.

It's hard to not like dspy with such results. Using dspy it seems that we can prompt gemma 1b to 'act' like sonnet 4 in a classification task. Until now we could only get that type of results with half-a-day of prompt engineering on 8b to 24b models depending on the subject being classified. Going from 8b to 1b leads to huge costs saving!

A Guide to Hacking DSPy into doing Automatic System Prompt Optimization In this tutorial, I’ll show you how I’ve modified and customized DSPy to make it handle system prompt optimization. Usually DSPy is doing program optimization. DSPy is very much batteries included,

For $10/hour, you can self-host Kimi and generate stories (or anything else) at 440 tokens/sec on 8 H200 GPUs with Prime Intellect. The key is not to worry about latency. For example, if you want 26,400 tokens from 24 questions, you can self-host Kimi and get that in about a minute (once everything is up and running 😉). That works out to about $6 per million output tokens. Pretty cool! On Groq, though, Kimi K2-0905 (256k) is $3.00 per million output tokens. So I can’t even self-host cheaper than Groq—at least not with the load and settings I tried. But it’s still about 2.5× cheaper than Sonnet 4.0, which this model (according to benchmarks) is roughly equivalent to. I think self-hosting such a powerful model is incredibly empowering. Either Groq is heavily subsidizing Kimi tokens, or they really do have amazing technology. 🙂 ps: for short similar questions, answered by only a few tokens, we can get about 573 token/second output leading to about 150 000 prompts per hours. So 150K simple classifications per hour for kimi k2 on 8xh200

Yesterday I realized that we could turn MCP on its head. Since it's a self-documenting protocol of tools (functions) with structured, typed outputs, any MCP server can be easily mapped into a module/library in Python (or most other languages). That means you can make a library/tool, open it to the world as an MCP server (for LLMs), but also for any other REPL and CLI out there. I'm extremely excited about this library. The interface design is done. We've got a name. We've got a repository. Hopefully by the end of the day it's ready to use. mcp2py will make your Python REPL into an MCP client in a very Pythonic, simple, yet powerful way.

Attachments with DSPy is out! Attachments is an ambitious MIT licensed open-source python library which aims to be *the* community funnel to go from file/str → text + base64 → LLMs. I won't do it alone! Get in touch and start creating pipelines with me. Let me know if you find bugs (it is still early :D ) and come contribute to support more inputs and outputs!!

My tutorial on: How to build an Automatically Branching Chat with DSPy is out! With DSPy, I am able to easily rein in the chaos of LLM outputs and actually use LLM generation in my coding logic. No manual parsing or fiddling with strings required. If you follow along and run all the code, by the end of the tutorial you will get to have an automatically branching chat with Kimi-k2 running at amazing speed thanks to Groq! (Or any other model; you just need to change one string.)

Honey, if we buy 2 gpus (for 35 000$) we could probably save 200$/month in AI subscriptions, what do you think?

This reminds me how Anthropic was the first to introduce: Projects Canvas Computer Use Coding CLI MCP and now Skills. We can say many things about Anthropic, but we cannot say that they don't actively innovate and share regarding LLM usage and ergonomics.

Today, I pruned 87.24% of Qwen 30B for a sentiment classification task while keeping 100% of its accuracy. This means we get to use big models on gpus with not that much RAM (potentially running models that would normally require an H100 on 3090 type gpus)! Imagine pruning Kimi-k2, GPT-OSS, or Qwen Coder. MoE models can be pruned for task-specific generation, which is particularly relevant for DSPy programs, as each signature is by definition task-specific. Why is nobody talking about that? Why is there no tools for pruning like thre are for inference? What we need to figure out are questions like: Does Qwen Coder use only 15% of its weights when coding in, say, R statistics? Or even more specifically, when creating ggplot visualizations in R? If so, you could feasibly run Qwen Coder's relevant experts on consumer GPUs to write code for your data visualizations.

This morning, I thought I'd push Claude Code a little. I asked it to get speech-to-text running on my laptop using NVIDIA's 0.6b Parakeet model. Less than two hours later, I had speech-to-text better than OpenAI's Whisper running locally on my laptop CPU, transcribing as a stream while I spoke. And I never opened anything other than Claude Code for that project—it all happened in one context window, including dependency installs and everything! We'll have to be even more ambitious next time.

I started this morning, and it's now ready for you to try: > uv pip install maivi > maivi That will get you a free and open-source voice-to-text model running locally on your CPU/laptop—you don't need a GPU! The model is (according to benchmarks) better than Whisper, and our implementation is faster to full results than Groq because it transcribes faster than realtime! The transcription will land in your clipboard, you can make it auto-paste too. P.S. It's supposed to be working on Mac and Windows, but I've only tried it on Linux so far. Enjoy! Link to repo in comments.

The developer of Ghostty (a trendy new terminal) just made a post showing how he used AI (as a professional developer) to implement a new feature in Ghostty (link in first reply). In that post, he comments on how AI enables him to cook while cleaning up his codebase for the new feature. I very much agree, but it also demonstrates that we are urgently ready for some hands-free interaction with the computer. We need: > Live-ish voice transcription > Live-ish text-to-speech and, very importantly: > Very robust voice commands, themselves developable in voice. A sort of voice macro system, if you will.

I used to say: MCPs are unnecessary; you can just use/make APIs. I was wrong. MCPs have the special property (protocol) of providing docstrings and parameter details that make the MCP usable by LLMs, but they also make that API usable by humans. Using APIs tends to be very difficult—you learn a few incantations for any API web service you need and you later forget how it works. The MCP makes it feel like good, well-documented functions. It's actually amazing we needed LLMs until we made a nice documentation convention for that!

A simple introduction to DSPy Whole script at the end. DSPy is simple and powerful. It is the best way to build LLM software right now. Despite that, lots of people keep putting off learning it. I know I did—for a whole year! I

Beautiful! You can now do: > uv pip install mcp2py dspy And in just 6 lines of Python code, you have an AI agent that can retrieve information through Google Chrome MCP DevTools. With the added bonus that you're only 1 or 2 steps away from doing prompt optimization for that agent with Gepa.

Since Sonnet 4.5 came out, I switched back to Claude Code. I left Codex, I left GPT-5 Pro, Gemini Ultra DeepThink, Grok Heavy, and went on living in the terminal. All in all, I've found it to be more productive. There's a je ne sais quoi about Sonnet 4.5 in the Claude Code harness. But this makes me a bit sad. It makes me sad because I love programming—I love the state of flow it creates, and I love that I could be paid for it. I'll certainly continue to code for fun, but it's hard to justify doing anything I do without Claude Code, which unfortunately doesn't give me the same flow state...

I am trying 'all' the ~$200/month AI subscriptions right now: OpenAI Pro, Claude Max, Gemini Ultra, and SuperGrok Heavy for a month. Here is my review so far: ** SuperGrok Heavy I will cancel Grok... it's the slowest and not often the smartest. My girlfriend loves its scientific writing. I find its coding instruction interpretation often quite off. It is ALWAYS the last one to finish... The voice mode seems to be actually intelligent-ish. This is not the case for OpenAI voice mode. OpenAI voice mode is a very empty shell. ** Claude Max Opus and Sonnet chat responses and coding style are 'too much' for me strangely. It does not transfer that much to Claude CLI. Opus finishes the whole response and starts generating before all 3 others, so I usually go on and start reading it, but it's meh... This reduced latency is nice. It keeps me more in the flow. When I wait too long, I can get distracted. Claude Code is very nice, and it's currently my favorite way to do things like: install stuff on my Linux machine, create repos, manage Git, push to PyPI, implement a plan into a working, well-packaged repository, convert video and audio files, etc. ** Gemini Ultra I only just got that one for deep thinking. I was super disappointed to be rate limited for the whole day after about 5 prompts. It seems very good, but it is getting into a little long side of interactions. Very good for a well-planned prompt and clear 'deliverable'. Ironically, Gemini Pro was holding itself very strongly against the other subscriptions. I was often ending up pursuing a chat with Gemini-2.5-Pro instead of the other ones because it was better. Long context is nice; it's at average speed. Images are looking good, but I don't use them much. Jules and gemini-cli are nice, but they don't require nor pair with the Ultra subscription... ** OpenAI Pro I am not a fan of OpenAI (the company). Like many, I feel betrayed by them. I was a huge fan before they turned into a product-making, hyping organization. I don't like that Codex is only interacting with your git repo. I don't like that my subscription doesn't carry over to codex-cli. They have a nice variety of models, and I love most of them. o3 is very very good for initial scripts for code. gpt-4.5 is very nice to co-design APIs and crazier ideas with (and it generates right away), I love 4.5. I like 4.1, it's very quick, and I often go to it to work on formatting types of things. o3-pro is quite slow, but usually it 'gets' the task very well, and the output goes very far. So which would I keep? ------------------------- To my surprise, if OpenAI included codex-cli in the subscription with generous rate limits, I would probably keep only that subscription because it would have 'very' of everything. If they don't, I'll probably keep them and Claude Code. Currently, I would cancel Gemini Ultra and SuperGrok Heavy. Maybe I could cancel Claude Max too and use Qwen Code CLI and pay per token. I would need to evaluate that one.

For $10/hour, you can self-host Kimi and generate stories (or anything else) at 440 tokens/sec on 8 H200 GPUs with Prime Intellect. The key is not to worry about latency. For example, if you want 26,400 tokens from 24 questions, you can self-host Kimi and get that in about a minute (once everything is up and running 😉). That works out to about $6 per million output tokens. Pretty cool! On Groq, though, Kimi K2-0905 (256k) is $3.00 per million output tokens. So I can’t even self-host cheaper than Groq—at least not with the load and settings I tried. But it’s still about 2.5× cheaper than Sonnet 4.0, which this model (according to benchmarks) is roughly equivalent to. I think self-hosting such a powerful model is incredibly empowering. Either Groq is heavily subsidizing Kimi tokens, or they really do have amazing technology. 🙂 ps: for short similar questions, answered by only a few tokens, we can get about 573 token/second output leading to about 150 000 prompts per hours. So 150K simple classifications per hour for kimi k2 on 8xh200

It's hard to not like dspy with such results. Using dspy it seems that we can prompt gemma 1b to 'act' like sonnet 4 in a classification task. Until now we could only get that type of results with half-a-day of prompt engineering on 8b to 24b models depending on the subject being classified. Going from 8b to 1b leads to huge costs saving!

Yesterday I realized that we could turn MCP on its head. Since it's a self-documenting protocol of tools (functions) with structured, typed outputs, any MCP server can be easily mapped into a module/library in Python (or most other languages). That means you can make a library/tool, open it to the world as an MCP server (for LLMs), but also for any other REPL and CLI out there. I'm extremely excited about this library. The interface design is done. We've got a name. We've got a repository. Hopefully by the end of the day it's ready to use. mcp2py will make your Python REPL into an MCP client in a very Pythonic, simple, yet powerful way.