Keep on to blur preview images; turn off to show them clearly
Wow, first good reason to go to the moon.
AI @amazon. All views personal!
"I think that since our species lives on information, information is the ultimate trade good because it is a nonrivaled good. You can share it with someone else without being deprived of it yourself. It can be multiplied, and that makes it the ideal medium of reciprocity, conferring a large benefit to someone else at a small cost to oneself." Beautiful words @sapinker! And thank you, @tylercowen for hosting him and sharing the transcript.
https://t.co/k8dEZBW68s
Me too: in 2016 in 2017 in 2018 Keep showing up.
🧑💻 https://t.co/Y30jsaHwz9 $20K/m ⚡️ https://t.co/vatLDmi9UG $17K/m 📈 https://t.co/3EDxln5mdi $16K/m ⭐️ https://t.co/MZc8tG9xWi $8K/m 🧬 https://t.co/SfrVXVtmdA $.5K/m 🍜 https://t.co/r07EpGSYJ2 $0K/m 🧾 https://t.co/7olaOzV8Xd $0/m +18 https://t.co/4zCWHGJp1S
Nicole just casually twisting the knife
📈 Leading Growth @GroqInc 📌 Prev @a16z @HubSpot @TheHustle 💻 Chronically online: https://t.co/AkbwhoTr0K 📘 Wrote https://t.co/w1DBDrOZdI 🎙 Podcast at @sydlis 👇
RT @proxy_vector: @shl lol we also replaced "zeitgeist" with "vibe shift" and somehow it carries more weight now language evolves by getti…
Founder/CEO @Gumroad
The Devil is in the Chat Template ========================= Potatoes, potahtoes are not the same thing, contrary to how Phoebe Buffay would have you believe. This is a must read blog if you are losing your sleep over AGI, or if you simply work on implementing open source AI models. If you are a public intellectual on "AI security" and if you don't understand this blog, you are not qualified to comment on the subject matter. Read a book, as they say. Now that I am done with my ranting, I want to tell you this blog will tell you what all things can go wrong that may end up making your frontier model "DUMB". LLM inference is very fragile. The inference engine has to present the input in a strict format (Chat template) to the LLM. If you deviate a bit, the outputs are not great. If not anything, it will reduce your AGI anxiety - technology is not going to be the skynet. Thanks @vllm_project and Lilian Weng. Here they narrate how they worked on feedback from the Kimi team to improve tool call success rate for Kimi k2 model being run on vLLM to near 100%. They did it very fast after getting the feedback. Kudos. Your community service is greatly appreciate 🧡💕 Key Lesson (quoting) The Devil is in the Chat Template: The chat_template is the critical handshake between a model and its serving framework. When integrating a new model, meticulously validate every piece of its template logic against the framework’s specific behaviors and assumptions. Peel Back the Abstraction Layer: High-level APIs like /chat/completions are convenient but can obscure root causes. When debugging, don’t hesitate to drop down to lower-level endpoints like /completions. Manually building the input is a powerful technique to isolate the problem. A Pro-Tip: Token IDs are the Ultimate Ground Truth: For the most subtle issues, inspecting the final sequence of token IDs sent to the model is the only way to be certain. While I didn’t need to resort to this for the issues above, it’s a critical tool in the toolbox. Techniques like using the OpenAI-compatible API to return token IDs can be a lifesaver. For those interested, we also highlighted this in our Agent Lightning post. Understand Framework Design Philosophy: vLLM’s strict handling of **kwargs is not a bug, but a deliberate security choice. Understanding these design decisions helps in quickly identifying the root cause rather than getting stuck on unexpected behavior. The Open Ecosystem Challenge: Advanced features like a tool-call “Enforcer” are hallmarks of polished, proprietary services. Implementing these capabilities robustly and elegantly in open-source projects like vLLM is a vital challenge for the community to address.
AI @amazon. All views personal!