Claude Mythos Changes Everything. Your AI Stack Isn't Ready.

Brief Summary

This video discusses the implications of the leaked Claude Mythos model, highlighting its advanced capabilities and the need for significant adjustments in AI development and usage. It emphasizes simplifying systems, focusing on outcomes over processes, and preparing for a new era of AI-driven automation. The key takeaways include:

  • Claude Mythos is a step change in AI capabilities, requiring a shift in how we build and use AI systems.
  • Simplicity is key; larger models demand less human-specified processes and more outcome-focused approaches.
  • Focus on clear outcome specifications, constraints, and effective tool definitions to leverage the full potential of advanced AI models.

Claude Mythos Leaked and Everything Changed

The leak of Claude Mythos, a new AI model trained on Nvidia's GB chips, marks a significant moment in AI development. Anthropic has confirmed its existence, naming it Capy Bara. Security researchers are impressed by its ability to find vulnerabilities in IT infrastructure, even identifying zero-day exploits in well-established projects like Ghost. This model's capabilities necessitate immediate battle testing against existing systems to identify and address potential weaknesses.

Security Researchers Say It's Terrifyingly Good

Security researchers have found Claude Mythos to be exceptionally effective at identifying vulnerabilities, surpassing human capabilities. Its ability to quickly uncover zero-day vulnerabilities in widely used software underscores the need for proactive security measures. Anthropic is allowing security researchers to test Mythos against popular utilities to strengthen defenses before its public release, highlighting the model's potential impact on IT security.

The Bitter Lesson of Building with LLMs

The arrival of more powerful models like Claude Mythos necessitates a shift towards simplicity in AI systems. The "bitter lesson" is that complex human-designed scaffolding often hinders rather than helps these models. As models become more capable, the focus should be on defining clear outcomes and allowing the AI to determine the most efficient process, which requires letting go of preconceived notions about how tasks should be executed.

Question 1: Check Your Prompt Scaffolding

Prompt scaffolding, the way prompts are structured to drive results, needs reevaluation. It's important to differentiate between instructions that the model truly needs and those added because of the user's perception of the model's requirements. Complexity should only be added when it demonstrably improves outcomes. The focus should be on communicating "what" and "why" to the model, rather than specifying "how," to leverage the model's increased intelligence.

Specify What and Why, Not How

When using more intelligent models, it's crucial to shift from procedural instructions to specifying the desired outcome and its purpose. For example, instead of detailing the steps for a customer support agent, simply state the goal of resolving the customer's issue using available resources and policies. This approach allows the model to leverage its intelligence to determine the most efficient path to the desired result.

Question 2: Retrieval Architecture and Memory

The approach to retrieval architecture and memory should evolve with the model's capabilities. Instead of predetermining retrieval logic, allow the model to handle retrieval based on its understanding of the situation. Present a well-organized, searchable repository of information and trust the model to find what it needs. This involves letting go of control and allowing the model to leverage its intelligence to efficiently fill its context window.

Let the Model Fill Its Own Context Window

As models improve, they become better at using their context window effectively. The focus should be on specifying the goal and providing access to necessary resources, allowing the model to decide what to include in its context window. This approach requires trusting the model's ability to determine the most relevant information for achieving the desired outcome.

Question 3: Hard-Coded Domain Knowledge

Evaluate the amount of hard-coded domain knowledge in AI systems and determine what the model can infer from context. Business rules and specific instructions should be re-examined to see if they are still necessary with more intelligent models. The art of prompting is evolving to focus on what to leave out, allowing the model to leverage its intelligence and infer information from the given context.

The Art of Prompting Is What You Leave Out

The skill of prompting is shifting from specifying what to include to understanding what to omit. Overly detailed prompts can constrain the model and prevent it from leveraging its intelligence effectively. By focusing on the essential information and allowing the model to infer the rest, better results can be achieved.

Question 4: Verification and Eval Gates

Verification and evaluation processes need to adapt to the increasing accuracy of AI models. For non-technical work, maintaining high standards is crucial. For software development, a single, comprehensive evaluation gate at the end of the process is recommended to check all requirements. This approach simplifies the pipeline and ensures that the model meets the required standards.

Why Mythos Will Only Be on Max Plans

Due to the high cost of running advanced models like Mythos, they are likely to be initially available only on premium plans. Users need to assess whether the investment in these plans is worthwhile, considering the potential productivity gains and competitive advantages. The ability to leverage these models effectively can provide a significant edge.

What a Mythos-Ready System Looks Like

A Mythos-ready system is characterized by clear outcome specifications, well-defined constraints and guardrails, and an excellent set of tools. Outcome specifications should focus on the desired result, while constraints and guardrails ensure compliance with business rules. The model should have access to effective tools that it can use to achieve the desired outcome.

Simplify Before the Train Leaves the Station

The key to preparing for advanced AI models like Claude Mythos is to simplify systems and processes. This involves re-evaluating existing workflows, removing unnecessary complexity, and focusing on clear outcome specifications. By simplifying, organizations and individuals can create an environment where AI can be most effective, leading to increased productivity and innovation.

Summarize any YouTube video in seconds

Get summaries like this one for any video, in your language, with the free Summ app.

Download on the App Store
Get it on Google Play

Transcript

Full transcript of “Claude Mythos Changes Everything. Your AI Stack Isn't Ready.”. Click a timestamp to open the video at that moment. Need another video? Use the YouTube transcript generator.

0:00 There are moments in AI when everything changes and we just had one of those the last few days. Claude Mythos leaked. Now Claude Mythos is the first model as far as we know that has been trained on Nvidia's new GB chips. It is a massive model. It is a step forward. Anthropic has confirmed its existence and they've given it a new lineage name. So it won't be called Sonnet. It won't be called Opus. It appears to be called Capy Bara. I don't know why we've switched to furry animals, but here we are. This is the biggest model in the world. It is by most measures according to Claude and it is going to be the most powerful model in the world. But don't just take my word for it. Don't just take anthropics word for how powerful it is. Look at what security researchers themselves are saying. Security researchers themselves

0:45 are saying that cloud mythos is terrifyingly good at finding vulnerabilities in your own infrastructure better than a human. In fact, one of the most experienced security researchers in the world stood up at a conference in San Francisco in the past few days and said that Claude Mythos immediately found zeroday vulnerabilities in Ghost, which is a 50,000 star GitHub repo that has never had major issues before. And as soon as Mythos was let loose on it, it found a bunch of issues that even the world's best security researchers haven't found. And this is why Anthropic is taking the unusual step of allowing security researchers to dig into Mythos to battle test Mythos against some of the most popular utilities on the internet and to harden up their defenses ahead of time

1:31 because as soon as Mythos is released, Mythos is going to be able to act as a threat to any IT repo out there and identify vulnerabilities that even good security researchers haven't been able to find, which immediately suggests that the first thing you should do if you are in IT or security as soon as mythos comes out is to say let's battle test it against our own systems and let's see what vulnerabilities it finds. That is that is job number one day zero you got to do that. But let's go beyond that. Let's look at what Claude mythos means. I want to be really honest with you. This model is one of the inflection points in 2026 that we all need to pay attention to. This is a chance to catch up before the train leaves the station.

2:18 And I know that you may hear from me, things are getting faster. Things are getting faster a lot. But guess what? That's what it's like when you're on an exponential curve. Things keep getting faster. And I'm here to tell you, this moment before Mythos release, and it may release as soon as next month or the month after, this moment is your chance to get things figured out before it arrives and upends everything. And I don't just mean about security. I mean how you build stuff, how you prepare etc. Why is that? Well, when models get bigger, they force you to simplify. They force you to think what can I delete about my systems and my practices because the model can do so much more now that that it couldn't do before. And that is what we have to understand. That

3:03 is what we would call the bitter lesson of building with LLMs. We as humans think we have a lot of value to add to these models. We can add our judgment. we can complexify. We can add a lot of scaffolding and systems around these models and it will make them better. And as they get more powerful, the bitter lesson is that simpler works best. And so I'm going to go through and in this video I'm going to lay out some of the things you should be checking ahead of time. But there's a larger thing that you should take away, which is simply be sure that you are taking the idea that the LLM can do a lot more seriously. And we'll get into what that is. We'll cover mythos when it comes out. We'll talk about all of that. But for now, as we go through the next few questions, as you audit yourself, as you think about whether you're ready for mythos, that's

3:50 the larger lesson. But anyone can tell you that. What are the what are the specific questions that help you know if you're ready for a big model change? If you're ready for effectively a step change. And by the way, I know that models come out all the time, but step changes are more rare. And what we get when we do a pre-training run on GB300s and it's much larger and more powerful underneath is we get the scaling law in effect. Right? So what I'm trying to tell you is yes there'll be another model along from somebody in the next week or so but until mythos comes out and there will be other models that are in that lineage that come out from other model makers that are also big that are also trained on GB300 those are the step change that we're going to see in the first half of the year. And so you need to learn to differentiate between the models that are 5 10 15% better and the models that are significantly better, a

4:37 step change better. And that's why this is a big deal because we are going to have another lurch up where the models are going to get significantly stronger in the next few months. And and you need to get ready for that now. Okay, here's your questions. I'm going to give you four specific things to look at that are going to likely break when Claude Mythos comes out and you need to take them seriously. If you are building in AI at all, this is going to come up. Number one is your prompt scaffolding. How you think about prompting to drive results. Ask yourself, and this is not a per prompt document question. This is a per line question. Really dig in. Is this instruction here because the model needs it or is it here because I needed the model to need it? This is the question that I want to really dig into. Anthropic's recommendation is very

5:22 unambiguous here. Consider adding complexity only when it demonstrabably improves outcomes. And this is true not just for anthropic models. Open AI's codeex guide tells us something similar when it says just tell it what you need without writing long instructions. This is going to be more and more the case, right? We are going to more and more be asked to communicate with the model what and why we need something in the end and less and less about how. So for example, let's say you have a customer support agent with a 3000 token system prompt and half of it is procedural, right? Like first classify the intent, then check your response for hallucinated URLs, then do X, then do Y. That sequence was written down because the model would skip knowledge checks enough of the time that it was needed. Think

6:08 about whether you're really going to need that. When a model gets two or three times smarter, you may be able to delete that like 30 40 50% of that 3,000 token prompt because so much of the procedural is just not needed when the model is more intelligent. Let's say you're not building agents. What's the implication for you if you're just chatting? If you're using co-work and you're wanting to get work done and you're less technical, it's pretty simple. Ask for what you want in the end and explain why in plain language. And you don't need to elaborate on how to get there. As long as the model has access to the inputs it needs, the data it needs, it's going to get there on its own. Increasingly across 2026, this is the bitter lesson we have to learn. All of the way we have described process,

6:54 the things that are precious to us are things that are associated with our ability to execute work in a certain series of steps and somehow we've decided that's an important reflection of our work identity. What cloud mythos and similar models are going to teach us is that that doesn't matter anymore and what matters is the outcome and our ability to name the outcome and let go of the process. You got to let go of the process with these models. And that, by the way, also goes for when you're auditing. Let's say you have a business process that you thought you couldn't make better. I would bet you if you give it to Mythos or maybe you give it to Andre Carpathy's Auto Research, you're going to find ways to make that better that a human can't figure out. The LLMs are now better than us at finding efficient ways through process and we

7:39 should stop specifying. So that's point number one. Check your prompts. Stop oversp specifying. Lesson number two is about retrieval architecture and memory. You're basically asking how is the model changing the way it relates to memory based on the model's gain and capabilities. And so another way of looking at it is to say in the past we had to carry a lot of the logic for retrieval on our side so the model would retrieve correctly. But really how much of my retrieval logic should belong to the model if the model's smarter. This is a much more nuanced take than takes like that say sort of rag is dead. I've heard I've heard people like beating on retrieval augmented generation a lot. Think more broadly. If you're in a large context window situation, if you have like a million tokens, 10 million tokens, 100 million tokens, increasingly you should start to think about how the

8:26 model wants to handle retrieval for that situation and how the model thinks it can handle it efficiently. And you should think less about predetermining how all of that works. Now, there are going to be pieces you're going to have to decide from the get-go. You'll have to decide what goes into the initial context window. you're going to have to decide whether the model is going to be invited to look at particular repos etc. But once you've made those initial decisions, a lot of the rest of it with these very powerful models essentially relies on you being able to present a really well organized searchable repo of some sort. Maybe it's documents, maybe it's code, etc. Maybe it's the file system on your computer if you're non-technical. And then you need to say

9:11 you go ahead and have a look. You look for what you want. And you need to trust the model to find what it needs to find. And there again, you see the theme. The theme is we have to let go so the model can do more. If we want to take advantage of the power and that is going to be one of the one of the job skills that is hard to measure in 2026 is the ability of someone to see a new model coming like mythos and to say I can see in advance. I understand how to model the improvements that an new intelligence is going to bring to the table and I'm going to change my workflow. I'm going to be ready so that I cut down the prompt where I need to. I'm going to be ready so I adjust my retrieval strategy and I'm not oversp specifying retrieval. I'm going to let the model pick what it wants to pick out of this file system, out of this repo, etc. Because that is where we need to

9:57 go. We need to assume that the model is increasingly going to be better than we are at deciding what to put in its context window if we can specify where it should go and we can have a directed prompt and we can understand with the model what we want to accomplish. And so our goal increasingly is to say here is the goal, go get it done and then to measure success. And that's it. That is the goal. And the smarter the model gets the more our work resolves down to that. and we need to get out of the way. And that in my observation is really hard. That's why it's called the bitter lesson because we humans like to think that our contributions here are special. That we have a way of doing scaffolding. That we have a way of doing rag. That our special system prompt. No, let it go.

10:44 That was for that moment. That was for that model. And we need to start to think increasingly about how we put these models in the heart of our workflows in ways that allow them to do their most powerful work in a direction we set in alignment with where we want to go. And our job is to point them in that direction very very clear. So retrieval architecture is another example of that. You just have to be very clear. Here are the resources you can access. Here is the goal and increasingly the model is going to figure that out. Now, do I have specific advice for you because Mythos is already out in the world and I've tested against 20 different things. I do not. We will get to that when it comes out. But I do have very very high confidence that Mythos is going to be much better at this, much better at intelligently

11:31 filling its context window than previous models because that is exactly what you get with the scaling law. When you get more intelligence, we we have seen over and over again, you get better at using your context window. This is going to be another jump in that direction. And it is a good chance to remember one of the big lessons that we've should have learned over and over again over the last two years. As models improve, this stuff gets simpler. Okay. On the simplification train, number three, look at how much you have to hardcode domain knowledge versus how much the model can infer. In other words, when you're thinking about these things that you want to emphasize over and over again to a model, ask yourself, which of these business rules did I write down? Because the model could not infer this from context, and which of these can I

12:17 actually let go of? And this is true for us non-technical folks when we look at how we work with co-work as well, because how often are you writing down this is what I do, this is my role, and maybe you have a saved prompt for that, or maybe you have it in memory somewhere. you're increasingly not going to need to do that because the model is going to infer it from the context you give so reliably that you'll be fine. Another example is uh let's say you have a house style for writing your client reports. The model can just infer that at a very high fidelity from a given example report. That's the whole point of the scaling law is that intelligence gets better at reliably answering your query. In this case, a query about constructing a report will be more reliably answered in the voice of that. And so you should count your rules. You should count the things you have to remind the model of. Ask yourself, do

13:05 you really need to fill the token window for better models with this? Be prepared to let go of some of that. And look, this is also hard for me. This is not something that I'm saying isn't hard for all of us. I had an example recently where there was a a prompt that I was using around how I do research. And I'd been using it for a couple of model generations. And one day I forgot to put the full 10-line prompt in and I just put a oneliner and said go research and I got a better result back because the 10line prompt and this is a micro example, right? The 10line prompt was more detailed about methodology than it needed to be and was over constraining my model. It had hard-coded my domain knowledge about what resources to look

13:50 up, etc. And that had been really good two model generations ago. And now I needed to let it go. Now I needed to let it go. I needed to just let the model go find the right resources and come back with a research report. And so as much as the art of prompting for the first couple years of LLM was about what you put in, increasingly the art of prompting is about what you leave out. And it's still an art. It's still hard. You still have to prompt. And the skill is valuable. It's just that the skill is evolving because the models are getting better. Okay. Question number four, this one's all about, did the model do what we asked it to do? Can we sniff check it and show that it worked? Now, if you're doing non-technical work, the answer is going to be increasingly clear. You're

14:36 going to look at it, say, "Wow, that looks really good." And move on. And a lot of people are going to do that. And the art of checking non-technical work is increasingly going to involve you having a very high standard. One of the challenges of working with good models is learning to make sure our bar is really high. Don't be afraid just because this model is really good to look at something it produces like a PowerPoint deck or an Excel and say this isn't quite right. You got 99% of this right, but this is the 1% I want fixed. Apply your high standard. That's how we get good work. That's how we don't pass slop on. But if you're building software, this is one of those things you have to apply a lot of judgment to because 99% and verification checks in software is a very different game from 85% in verification checks. And we are

15:22 in a world now where we are closer to 99% right more of the time than we are to 85%. And mythos is just going to push us more in that direction. Which suggests to me that we need to be really smart about writing our emails. I would say we are moving toward a point where we want one eval gate at the end of the software process and it needs to check absolutely everything and send things back when it doesn't work because if we do intermediate evals along the way net there's enough right about what these systems build it's just not worth it. You don't want to mess around with a more complicated pipeline. Simplify, simplify. Just write the eval at the end. And when you write the eval at the end, make sure that it tests absolutely everything. Your functional requirements, your non-functional

16:08 requirements. And then make sure when it comes back that you are sure if you read that eval script and all the tests pass that you're confident it's good. Because increasingly we need to be at a point where we're not the ones who are the bottleneck for telling whether a particular piece of software reliably calls dependencies and whether the code is clean and has appropriate exception handling, whether edge cases are handled. Like that all needs to go into automated evals or else we're going to be overwhelmed by the number of pieces of software we can produce and the amount of checks we have to do. Like there are already conversations being had in San Francisco around the fact that humans cannot review all the code and we have to find ways to start to scale out of ourselves. We are the bottleneck. Mythos is going to make that worse. If you are depending on humans

16:56 and human handoffs as a key part of your agentic software development pipeline, you're in trouble. And there is sort of an analogy there for non- tech folks. If you are depending on humans as a handoff between PowerPoint and Excel and you're doing AI for both of those pieces, you should be looking seriously at whether you can automate those non- tech artifact handoffs because Mythos can help you do that. And I'm not talking about Mythos as if this is the only model that will ever be able to help you with this. I'm saying it is the first leaked model of a new class. And we're going to see similar models come out from other hyperscalers. Google will have one. Open AAI will have one. They will also probably drop them in the next couple of months. We will all together regardless of the underlying model we use be in this new world and we need to

17:42 think about that. Do you know another reason why we need to think about simplicity? It's very it's very obvious these are not cheap models. Enthropic basically confirmed that these are going to be very expensive models to run. You want to be very efficient with them. You want to make sure they're using the tokens as efficiently as possible. You don't want to clutter them up with a bunch of human described process. You want to use them as effectively as you can. I am willing to bet you that Mythos when it launches is only going to initially be available for max plan users for CLA because it's so expensive to serve. And we are headed toward a world where increasingly the first and best models are only going to be available on those premium plans because of how expensive they are to build and

18:28 train and run and serve. And so we need to think to ourselves like if you're thinking about your future, about your career, you need to ask yourself basically, am I in a position where I can invest in intelligence in one of these plans in order to get access to this and then I will leverage it to the hilt and I will make it worth it. I will tell Claude Mythos, save me my 200 bucks. And you can do this now. You don't have to wait for Mythos. Just tell your AI, look at my recurring subscriptions for my household and find me $200 in savings. Most households in America can find $200 a month in savings somewhere from some of their subscriptions just because the subscription economy latches onto our credit cards like baracles and we just need to find some space and LLM's right now without waiting for claude mythos are already very good at that. And so

19:14 this is one of those things where think about the kind of intelligence you want to purchase and the timeline you want to buy it on because the people who are able to get Claude mythos now if you have a plan to leverage it effectively are going to be ahead. There's just no other way to put it. they have access to a better brain. Now, that doesn't mean that Mythos will always be expensive. We have the Vera Rubin chipset coming up behind the GB300. It's a whole new generation of chip from Nvidia. And as that starts to come online and even better models start to come out and it becomes cheaper and cheaper to serve Mythos. We are going to be in a position in a few months, 6 months maybe less, where you start to see the cost come down and you start to instead see more expensive models come out that are even better than Mythos. And so really when you start to think about what you're investing in, if you think about the

19:59 plans you're purchasing and what you want to invest in, this goes for corporations buying for for their employees, it goes for individuals, think about it as what trajectory, what curve do I want to be on? Do I want to pay to be on the cutting edge curve and I'm going to use it to the hilt and I'm going to build really cool things and I'm going to learn and I'm going to leverage it to 10x myself, etc. or do I want to be a step behind on like a pro plan or whatever the 20 buck a month plan is? And then in that case, I'm willing to wait and I'm not going to be the first. I'm willing to take the hit on my career for that. I'm willing to take the hit for my employees if I'm a company. And in that situation, I should not expect productivity that will resemble what people who are on those cutting edge plans get. And that's a very serious thing. You might think, oh, but talent will make up for it. No,

20:45 human talent will not. Like increasingly the whole point of human talent is to simplify and get out of the way so that AI can do its thing. And the bigger the models get, the more that becomes obvious. And Mythos is one of those moments when I think it's going to be very very obvious that we are in a different world and that the people who have that $200 a month plan are going to effectively have superpowers. And so think about whether you want those superpowers or not. I'm not saying there's an easy answer. I know $200 a month is a lot for a lot of us. And I know it's a lot for companies like if you're paying for it by token, it's also not going to be cheap. Think about whether it's worth it. Think about whether you can leverage it. Think about your multimodel strategy if you're a company. What are you using for mythosshaped problems? Do you have a

21:31 reliable way to see what is a complicated problem that is worth putting a cutting edge model on versus not? And by the way, if you're thinking I'm kidding about the gain in capability, we had another one of those stock market spook events where like security stocks for cyber security firms dropped between five and 9% as soon as Claude Mythos was even leaked. The entire world is starting to take AI models seriously because they have seen enough evidence to show what these models are capable of doing. And so yes, I fully expect that the leaked blog post that came out on the anthropic servers is broadly correct. It is going to be a big jump in coding. It is going to be a big jump in the ability to produce excellent artifacts like Excel and PowerPoint. Is going to be a big jump in the ability to reason. It is going to be a massive jump in cyber security and

22:16 more. We should take that seriously. When you look at what mythos is going to mean, you should assume it means a step change that's coming in the next month or two. I've been telling you 2026 isn't slowing down. This is an inflection point. Pay attention. So, what does a simpler system look like? What does a mythos ready system look like? I think a lot of the time like the larger public conversation has been around the security thing. It's been around, oh, this is a big model and those are fair to call out, but I want to go past that. I want to look at what it means to have a well architected system that lets you sleep at night before mythos comes. Number one, make sure you have very clear outcome specifications. Specify your intent in a way that makes sense for a smarter model. Let me give you an example. Back in the customer service

23:02 world, let's say what you want to specify is resolve this customer's issue using our knowledge base, our policies, and our account history. And you make sure the model has access to that. The customer should leave satisfied, and the resolution should comply with our return policy. That is actually a decent outcome specification. Compare that to what most production systems I see look like. It's essentially a process, right? Well, first classify the intent into one of 14 categories. Then route to the appropriate handler. Then retrieve the top five knowledgebased articles using hybrid search with alpha equals.7. Then generate a response using only the retrieved context. That is what most of the prompts that I look at today look like. You see the difference between that and the outcome one. The outcome one just says I need to resolve the issue. It needs to be in line with this policy. Here's where the policies are.

23:47 You may not be able to do that today with today's models. But be ready to do that. Get ready now because it's going to take time to rearchitect your systems. Start to think about how you need to prioritize your work to get set. Number two, think about your constraints and guard rails. These are things that must be true regardless of how the model achieves the outcome. Because when you give the model more flexibility, you need to be more clear about your constraints and guardrails. For example, never disclose customer financial data. That's a pretty good customer guard rail. Always verify refund eligibility against our policy. There's another one. These should survive model upgrades because they represent ongoing business rules that you want any model to follow regardless of how smart it gets. Get good at those. Number three, think about the tools your model can use. You want to make sure you have an excellent set

24:33 of tools that have the right capabilities where a model can look at the tool set and say, I know what this does. I can use it and it's going to be effective at what it does. So, right, maybe it's search your knowledge base. Maybe it's look up an account. Maybe it's process a refund. The model is going to decide increasingly what to call and in what order and it's going to be up to you to define what the tools do and to be effective about that and present those tools effectively. So put some work into your tool definitions. And then number four, and then last but not least, if you're thinking about larger aentic software, we're going to go back to multi-agent coordination patterns, which I've talked about before. I've talked about cursor's idea that you have like a two agent hierarchy and that's much more effective than an agent swarm. You now need to be thinking about long-term planning with agents and

25:20 what it means to let an agent like Mythos spin up agents from a variety of capability sets to get a task done. Like you should be in a position where you understand how to present an outcome spec that mythos can take. How you then present a overall set of evals Mythos has to work against and how you give Mythos a tool suite that allows it to spin up a lot of instantiated agents to execute against that plan. How it has places to track progress and how it can measure itself with eval. You need to take the idea that Mythos is the planner of your software more seriously. And again, I don't just mean Mythos. Any of these next generation models is going to do this. Codeex is going to drop a new model not too long from now. And it's also going to be more and more able to do this. And all we're doing is

26:07 extending the current pattern we see at factory.ai, at Cursor, etc. And we're saying these models are more capable. You're going to give them more to work with. You're going to be more confident that they will get it done well. And you're going to make sure the eval allow them to assess themselves in a way that is reliable so you can trust the results. And by the way, you can also have a different instantiated Mythos model assess the work of that other model so that you have a second pair of eyes, right? You don't need to trust the model that did the work to do the eval as well. But really increasingly once you can specify the outputs and the guardrails and the tools and you just give the model a sense of the harness and architecture to get to the outcome, you're going to step out of the way. And that's increasingly going to be the case not just for technical work but also for knowledge work. Like we're going to

26:53 start to architect in a sense technical flows for non-technical work. And we're just at the beginning of this, but I think Mythos is going to invite us to start thinking about that. like our under the desk software, our software we build as nontechnical people is going to get increasingly sophisticated because we're going to have access to models that can do increasingly sophisticated things just by communicating an intent in plain chat. And so we should be looking as IT administrators, we should be looking as leaders at the idea that this under the desk software category that was just for personal use may increasingly be developing really useful team applications that never end up touching the engineering team. How do you maintain that? How do you think about that? How do you teach the idea that you want these concepts, guardrails, outcomes, specifications for non- tech folks, for folks who are like,

27:39 "What is a spec?" Well, a spec is just defining what you want the model to do, right? There's ways we can teach this that is really not going to be intimidating and that doesn't require using the terminal or the command line. And it will allow you to build useful working software to solve non-technical problems. And this goes for families, too. Let's say you want to build a family calendar. like it's increasingly going to be possible to not just build one for your personal use that you start up personally on your laptop but like that you just build and you can deploy and maintain for the family and it's not even going to be something you think about as a piece of software because you never touched the code it just got built for you because you specified it and you said please hook into my Google now do we need better agentic primitives do we need better ways to hook into things like a Google calendar so it's easier absolutely that is one of the big pain points right now but that is the

28:24 direction we're headed And so if you're looking at this from a role perspective, from a team perspective, you should be stepping back and asking yourself, how much of my role is compensating for a model's limitations today versus how much of my role is really thinking about how to architect and correctly aim artificial intelligence so it accomplishes a lot of work. That's the part you want to be on, right? That's where you want to lean your role. And everybody has to some degree a chance to lean there, right? who can lean more into I want to guide the model toward work that it needs to do and less into my job is to compensate for model limitations because model limitations are going to keep shrinking. We're about to have another big step change there with mythos. All right. So what does this mean for you? I want to step back and simplify this in the spirit of

29:11 mythos. You need to take the idea that we are going to keep seeing model generations getting better very very seriously. We are not hitting a wall. That means you need to think across your role and across your technical systems about how you can dramatically simplify. How can you simplify so that the model has room to be intelligent? If you take one thing away, that is what I want you to take away. And yes, you absolutely will need to look in detail to get there because most of us have added a lot of guardrails and a lot of croft around our work so that we feel like we add value and or so that the weaker models can do their jobs. That's fine. The challenge is to grow by being able to simplify

29:56 what you ask the model to do against a larger outcome set so that we get out of the way and let the model do its job. And we focus on making sure that we are aiming the model in the direction of a big cool goal. And we focus on making sure we're building the pipeline and the support and the and the tool availability and the availability of data so the model can do its job. And we also have a good sense of what good looks like. I've I've talked about that a lot. That's going to matter for technical and nontechnical work. Is the code clean? Is the Excel absolutely right? That's not going to stop being important. We just need to make sure that we give the model what it needs so that it can accomplish those goals successfully and when we measure it and check it, we have confidence that the model got it done. So, Claude Mythos is coming. The inflection point is here.

30:41 This is another one of those moments when you need to be able to catch the train before it leaves the station. I hope this gives you a set of actionable recommendations to go with today to actually get ready for this because this is one of those things. If you look at the checklist I gave you here in this video, you're going to need to do some work to get ready for this. If you want to be ready day one, your systems need to be ready to simplify. So take the time, whether it's your own personal checklist and you're saying how what what canned prompts do I use? Can I simplify those? Or whether it's a work checklist and you're looking at like aic systems and how you prepare for them and guardrails. Get ready. It's coming. It's not going to be that long. It's a matter of weeks now.

Share

More video summaries