We finally did an LLMOps episode! (and its live on HN) @Humanloop recently launched their Evals product, and so I thought it was the perfect place to ask: dId gPt4 GeT DuMbEr? Had so much fun chatting with @RazRazcle (and being our own sound engineers lol) in London, and am excited to share our deep dives into: - Humanloop Origins - What is HumanLoop? - Who is the Buyer of PromptOps? - HumanLoop Features - The Three Stages of Prompt Evals - The Three Types of Human Feedback - UI vs BI for AI - LangSmith vs HumanLoop comparisons - The TAM of PromptOps - How to Be Early - 6 Orders of Magnitude - Becoming an Enterprise Ready AI Infra Startup - Killer Usecases of AI - HumanLoop's new Free Tier and Pricing - Addressing Graduation Risk - On Company Building - On Opinionatedness - HumanLoop Hiring - How HumanLoop thinks about PMF - Market: LMOps vs MLOps - Impact of Multimodal Models - Prompt Engineering vs AI Engineering - LLM Cascades and Probabilistic AI Languages - Prompt Injection and Prompt Security - Finetuning vs HumanLoop - Open Standards in LLM Tooling - Did GPT4 Get Dumber? - Europe's AI Scene - Why you should just move to SF (in The Arena) - Continual Learning - Motivations from Academia to Startup We hop you enjoyed this as much as we did recording it!
@humanloop @RazRazcle lmao HN hated it haha news.ycombinator.com/item?id=377088…
@humanloop @RazRazcle putting on record (bc i had trouble finding for @brittwalker_ ) my Impossible Triangle Theory of Devtools. another one that emerged recently is the agent web infra triangle: - browser automation (@browserbasehq) - web scraping (@expandai_ ) - llm web search (@ExaAILabs )
@humanloop @RazRazcle @brittwalker_ @expandai_ @ExaAILabs giving a talk on this today for LLM Infra at @mastra's conf swyx.io/impossible-tri…
