this concept of engineering harnesses to “shape jagged intelligence” is one my favorite mental models for extracting intelligence from today’s systems
every phase from pre-training to RL to harness design is an exercise in shaping the behavior of the shoggoth
of these, the most accessible shaping behavior is in agent/harness design - things like prompt design, context engineering, tool design are levers that amplify certain dimensions from the training settings (ex: match the formatting or problem formulation seen in post-training) or shackle the model to so it operates in high intelligence where its intelligence (ex: low context use)
we’re trying to set up this incredibly intelligent system for success by designing its world around it, interesting how we (try to)do the same to help people succeed in the real world
leading deepagents and building evals @LangChainAI, prev @awscloud, phd cs @ temple