Get live statistics and analysis of Charles Packer's profile on X / Twitter

Introducing MemGPT 📚🦙 a method for extending LLM context windows. Inspired by OS mem management, it provides an infinite virtualized context for fixed-context LLMs. Enables perpetual chatbots & large doc QA. 🧵1/n Paper: arxiv.org/abs/2310.08560 GitHub: github.com/cpacker/memgpt

Prior to GPT-5, Sonnet & Opus were the undisputed kings of AI coding. It turns out the GPT-5 is significantly better than Sonnet in one key way: the ability to recover from mistakes. Today we're excited to release our latest research at @Letta_AI on Recovery-Bench, a new benchmark for measuring how well model can recover from errors and corrupted states. Coding agents often get confused by past mistakes, and mistakes that accumulate over time can quickly poison the context window. In practice, it can often be better to "nuke" your agent's context window and start fresh once your agent has accumulated enough mistakes in its message history. The inability of current models to course-correct from prior mistakes is a major barrier towards continual learning. Recovery-Bench builds on ideas from Terminal-Bench to create challenging environments where an agent needs to recover from a prior failed trajectory. A surprising finding is that the best performing models overall are clearly not the best performing "recovery" models. Claude Sonnet 4 leads the pack in overall coding ability (on Terminal-Bench), but GPT-5 is a clear #1 on Recovery-Bench. Recovering from failed states is a challenging unsolved task on the road towards self-improving perpetual agents. We're excited to contribute our research and benchmarking code to the open source community to push the frontier of continual learning & open AI.

Really great reading list from @swyx - amazing to see MemGPT in the top 5 agent papers, side-by-side with one of my favorite LLM papers: ReAct IMO ReAct (@ShunyuYao12 et al) is *the* most influential paper in the current wave of real-world LLM agents (LLMs being presented observations, reasoning then taking actions in a loop). Pick any LLM agents framework off of github today - chances are the core agentic loop they're using is basically ReAct. MemGPT was our vision at Berkeley (@sarahwooders , @nlpkevinl , @profjoeyg , et al, now at @Letta_AI) for the next big thing in LLM agents *after* ReAct. LLM agents break down into two components - (1) the LLM under the hood which goes from tokens to tokens, and (2) the closed system around that LLM that prepares the input tokens and parses the output tokens. The most important question in an LLM agent is *how* do you place tokens in the context window of the LLM? This determines what your agent knows and how it behaves. The reason LLM agents today "suck" is because this problem (assembling the context window) is an incredibly difficult open research question. MemGPT predicts a future where the context window of an LLM agent is assembled dynamically by an intelligent process (you could call this another agent, or the "LLM OS"). Today, the work of context compilation is largely done by hand. Tomorrow, it'll be done by LLMs.

Excited to finally announce @Letta_AI ! The next frontier in AI is in the stateful layer above the base models - the "memory layer", or "LLM OS". Letta's mission is to build this layer in the open (say "no" 🙅 to privatized chain of thought).

💤 sleep-time compute: make your machines think while they sleep -> arxiv.org/abs/2504.13171 over the past several months we (at @Letta_AI) have been exploring how effectively utilize "sleep time" to scale compute. the concept of "sleep-time compute" is deeply tied to memory - applying compute at sleep-time is only possible if your agent has persistent state which can continuously re-written with additional compute cycles. in fact, the concept of "heartbeats" in MemGPT was actually inspired by the idea of an AI receiving heartbeats to "awake" it during sleep. prior to MemGPT's release, I actually spent many weeks trying to perfect heartbeats to try and get the agent to learn something useful during its downtime - but never quite got it to work quite right 😅 our new sleep-time agent design in @Letta_AI takes the heartbeats idea to next level and allows you to arbitrarily scale the amount of compute you want to apply at sleep-time with multiple agents with memory wired together. while one agent sleeps, a fleet of agents can work to re-assemble its memory asynchronously in the background

awesome to see another example of sleep-time compute deployed in production, especially on such a slick consumer app "setting up the memory can take up to 6 hours" congrats on the launch @nycintelligence 👏 letta.com/blog/sleep-tim…

How do we get from LLMs-as-chatbots to LLMs-as-agents? Programatic memory (context) management through an LLM OS. The biggest problem today with LLM agents is memory. Not just “memory” in a semantic sense (how can we get LLM agents to remember facts+preferences over time, similar to how a human does?), but also memory management - what tokens do we put into the context window at each LLM inference step, and why? The memory management (or “context management”) problem is the fundamental problem in programming LLMs to turn them from autocomplete engines into compound agentic systems that can interact with the world and learn from experience. JSON mode, tool use / function calling, RAG, chain-of-thought: these are all early forays into building the “LLM OS” for context management. We’re still extremely early in the history of LLM OS development. In our (free!!) @DeepLearningAI course in collaboration with @AndrewYNg , we distill the main ideas behind what it even means to do “LLM memory management” into clean and concise example and guide you through building a version of MemGPT (one of the early examples of an LLM OS) yourself, entirely from scratch.

Super excited about this release: Letta Evals is the first evals platform *purpose-built* for stateful agents. What does that actually mean? When you eval agents w/ Letta Evals, you can literally pull an agent out of production (by cloning a replica of its active state), evaluate it, then pushes live changes back into prod (if desired). We eval humans all the time: standardized tests (SAT/ACT), job interviews, driving tests. But when a human takes a test, they actually get the ability to *learn* from that test. If you fail your driving test, that lived experience should help you better prepare for the next "eval". With Letta Evals, you can now run evaluations of truly stateful agents (not just prompt + tool configurations). This is all made possible due to the existence of AgentFile (.af), our open source portable file format that allows you to serialize any agent's state, which powers the efficient agent replication needed to run Letta Evals at scale. Will be exciting to see what people build with this. Letta Evals is already in production at companies like @BiltRewards , and is used by our own research time to run comprehensive (agentic) evaluations of new frontier models like GPT-5, Sonnet 4.5 and GLM-4.6.

walked into a random coffee shop in SF and found someone building a PR review bot on @Letta_AI i think we’re gmi 😭

Excited to finally announce @Letta_AI ! The next frontier in AI is in the stateful layer above the base models - the "memory layer", or "LLM OS". Letta's mission is to build this layer in the open (say "no" 🙅 to privatized chain of thought).

Prior to GPT-5, Sonnet & Opus were the undisputed kings of AI coding. It turns out the GPT-5 is significantly better than Sonnet in one key way: the ability to recover from mistakes. Today we're excited to release our latest research at @Letta_AI on Recovery-Bench, a new benchmark for measuring how well model can recover from errors and corrupted states. Coding agents often get confused by past mistakes, and mistakes that accumulate over time can quickly poison the context window. In practice, it can often be better to "nuke" your agent's context window and start fresh once your agent has accumulated enough mistakes in its message history. The inability of current models to course-correct from prior mistakes is a major barrier towards continual learning. Recovery-Bench builds on ideas from Terminal-Bench to create challenging environments where an agent needs to recover from a prior failed trajectory. A surprising finding is that the best performing models overall are clearly not the best performing "recovery" models. Claude Sonnet 4 leads the pack in overall coding ability (on Terminal-Bench), but GPT-5 is a clear #1 on Recovery-Bench. Recovering from failed states is a challenging unsolved task on the road towards self-improving perpetual agents. We're excited to contribute our research and benchmarking code to the open source community to push the frontier of continual learning & open AI.

Introducing MemGPT 📚🦙 a method for extending LLM context windows. Inspired by OS mem management, it provides an infinite virtualized context for fixed-context LLMs. Enables perpetual chatbots & large doc QA. 🧵1/n Paper: arxiv.org/abs/2310.08560 GitHub: github.com/cpacker/memgpt

Really great reading list from @swyx - amazing to see MemGPT in the top 5 agent papers, side-by-side with one of my favorite LLM papers: ReAct IMO ReAct (@ShunyuYao12 et al) is *the* most influential paper in the current wave of real-world LLM agents (LLMs being presented observations, reasoning then taking actions in a loop). Pick any LLM agents framework off of github today - chances are the core agentic loop they're using is basically ReAct. MemGPT was our vision at Berkeley (@sarahwooders , @nlpkevinl , @profjoeyg , et al, now at @Letta_AI) for the next big thing in LLM agents *after* ReAct. LLM agents break down into two components - (1) the LLM under the hood which goes from tokens to tokens, and (2) the closed system around that LLM that prepares the input tokens and parses the output tokens. The most important question in an LLM agent is *how* do you place tokens in the context window of the LLM? This determines what your agent knows and how it behaves. The reason LLM agents today "suck" is because this problem (assembling the context window) is an incredibly difficult open research question. MemGPT predicts a future where the context window of an LLM agent is assembled dynamically by an intelligent process (you could call this another agent, or the "LLM OS"). Today, the work of context compilation is largely done by hand. Tomorrow, it'll be done by LLMs.

How do we get from LLMs-as-chatbots to LLMs-as-agents? Programatic memory (context) management through an LLM OS. The biggest problem today with LLM agents is memory. Not just “memory” in a semantic sense (how can we get LLM agents to remember facts+preferences over time, similar to how a human does?), but also memory management - what tokens do we put into the context window at each LLM inference step, and why? The memory management (or “context management”) problem is the fundamental problem in programming LLMs to turn them from autocomplete engines into compound agentic systems that can interact with the world and learn from experience. JSON mode, tool use / function calling, RAG, chain-of-thought: these are all early forays into building the “LLM OS” for context management. We’re still extremely early in the history of LLM OS development. In our (free!!) @DeepLearningAI course in collaboration with @AndrewYNg , we distill the main ideas behind what it even means to do “LLM memory management” into clean and concise example and guide you through building a version of MemGPT (one of the early examples of an LLM OS) yourself, entirely from scratch.

it's interesting to see openai's (unofficial) multi-agent framework implement multi-agent via message passing instead of via shared context / "groupchat" (ala autogen) imo message passing is the way to go (hence why we implement multi-agent in the same way in @Letta_AI), though it raises a lot of interesting questions about shared context (how can you get one agent to share memories or data streams with another agent) that are unanswered / out of scope in the groupchat model of multi-agent

"i will never touch python asyncio again in my life" - @sarahwooders