Keep on to blur preview images; turn off to show them clearly
Minimal Building Blocks for Just-In-Time Agents big fan of Skills to distribute capabilities and Subagents to focus the use of context + do specialization there’s pattern of orchestrator agent doing work and using specialized subagents is a powerful pattern that allows the orchestrator to run for a long time while preserving context once we decide what an subagent contains, bootstrapping easily. Minimally focusing on composing skills and detailed instructions is a good primitive for defining and spawning a subagent basically move towards a world of dynamically generated subagents by an orchestrator by attaching skills Just-In-Time. We have skills repos, tell the subagent which skills it needs to use an write a spec on the Task for the subagent to execute
agents, harnesses, and evals @LangChainAI, prev @awscloud, phd cs @ temple
two exciting patterns (among many) in agent engineering 1. Specialized capabilities distributed via Skills and SubAgents allow companies to pick one problem in agent building and go HAM on it. Agentic local search and web search are early leaders here 2. Open harnesses, let you inspect, edit, and generally fully customize the operating env for your agent. This makes it easy to plug in 1) to make your agent better.
agents, harnesses, and evals @LangChainAI, prev @awscloud, phd cs @ temple
i think the debate in this thread on “is that an agent” comes from the change in the last year or so of “agenticness” natively post-trained into models and integrated into APIs so that “agent” behavior is blurry, maybe hard to appreciate the step up from the pure chat completions UX the default chat experience for tons of ppl today is multi-step, interleaved thinking, many tool calls per user message —> that’s pretty agentic to how things used to be there’s no hard cutoff for agenticness but more steps/autonomy per user message is a good proxy and a lot of interactions are like that today! and it helps ppl do a lot more useful and also just fun stuff
agents, harnesses, and evals @LangChainAI, prev @awscloud, phd cs @ temple
software is on a path towards being less “human friendly” because the patterns of human software consumption are just different from agents we build software for the entity that uses the software, and every month that tilts to towards agents —> though look outside tech, we’re not even close to full adoption but there’ll also be a whole fun, beautiful explosion of “human centric software” that makes us feel something like games do moving towards the era of “artisanal software” —> we buy handmade because of beauty and craft, software will be the same
agents, harnesses, and evals @LangChainAI, prev @awscloud, phd cs @ temple
everyone’s got a fave slide from a State of ___ report, this one’s mine! Observability —> Evals —> Improvement basically: 1. Observability: we wanna make our agents better…so we gotta know what they’re actually doing. Log everything is a good default, if it’s not useful delete it later but at least you CAN inspect it 2. Evals: we have Tasks we care about, we should test our agent on them and extend them as we get more feedback 3. Improvement: We have these magic machines that can process tons of data and pattern match what’s happening, we should use them on our data to figure out how to improve our prompts, skills, subagents… my mission (should i choose to accept it which i already have): get that 29.4% to 0 i used to be (and still am to some extent) heavily “vibes based” for coding and making agents better. like just use it and see the vibes and try stuff but the pattern of analyzing agent traces with an agent to improve an agent is also good…it’s actually great “look at your data” - @HamelHusain “don’t forget to store your data so you can look at it” - me also lots of fun nuggets in the rest of this report like what ppl actually use agents for
agents, harnesses, and evals @LangChainAI, prev @awscloud, phd cs @ temple
skills are a fantastic distribution mechanism for capabilities we “pip install” libraries to get capabilities in software skills install capabilities into agents, while preserving our most precious resource…context there’s still work to do in making several skills work well with each other in a single harness —> every thing you add to your agent harness affects everything else but that’s the fun of good agent engineering
agents, harnesses, and evals @LangChainAI, prev @awscloud, phd cs @ temple