LogoThread Easy
  • 탐색
  • 스레드 작성
LogoThread Easy

트위터 스레드의 올인원 파트너

© 2025 Thread Easy All Rights Reserved.

탐색

Newest first — browse tweet threads

Keep on to blur preview images; turn off to show them clearly

Wow, first good reason to go to the moon.

Wow, first good reason to go to the moon.

AI @amazon. All views personal!

avatar for GDP
GDP
Sun Nov 02 16:14:53
"I think that since our species lives on information, information is the ultimate trade good because it is a nonrivaled good. You can share it with someone else without being deprived of it yourself. It can be multiplied, and that makes it the ideal medium of reciprocity, conferring a large benefit to someone else at a small cost to oneself."

Beautiful words @sapinker! And thank you, @tylercowen for hosting him and sharing the transcript.

"I think that since our species lives on information, information is the ultimate trade good because it is a nonrivaled good. You can share it with someone else without being deprived of it yourself. It can be multiplied, and that makes it the ideal medium of reciprocity, conferring a large benefit to someone else at a small cost to oneself." Beautiful words @sapinker! And thank you, @tylercowen for hosting him and sharing the transcript.

https://t.co/k8dEZBW68s

avatar for Michael Frank Martin
Michael Frank Martin
Sun Nov 02 16:14:40
Me too:

in 2016
in 2017
in 2018

Keep showing up.

Me too: in 2016 in 2017 in 2018 Keep showing up.

🧑‍💻 https://t.co/Y30jsaHwz9 $20K/m ⚡️ https://t.co/vatLDmi9UG $17K/m 📈 https://t.co/3EDxln5mdi $16K/m ⭐️ https://t.co/MZc8tG9xWi $8K/m 🧬 https://t.co/SfrVXVtmdA $.5K/m 🍜 https://t.co/r07EpGSYJ2 $0K/m 🧾 https://t.co/7olaOzV8Xd $0/m +18 https://t.co/4zCWHGJp1S

avatar for Marc Lou
Marc Lou
Sun Nov 02 16:12:56
Nicole just casually twisting the knife

Nicole just casually twisting the knife

📈 Leading Growth @GroqInc 📌 Prev @a16z @HubSpot @TheHustle 💻 Chronically online: https://t.co/AkbwhoTr0K 📘 Wrote https://t.co/w1DBDrOZdI 🎙 Podcast at @sydlis 👇

avatar for Steph Smith
Steph Smith
Sun Nov 02 16:11:42
RT @proxy_vector: @shl lol we also replaced "zeitgeist" with "vibe shift" and somehow it carries more weight now

language evolves by getti…

RT @proxy_vector: @shl lol we also replaced "zeitgeist" with "vibe shift" and somehow it carries more weight now language evolves by getti…

Founder/CEO @Gumroad

avatar for Sahil Lavingia
Sahil Lavingia
Sun Nov 02 16:09:34
The Devil is in the Chat Template
=========================

Potatoes, potahtoes are not the same thing, contrary to how Phoebe Buffay would have you believe.

This is a must read blog if you are losing your sleep over AGI, or if you simply work on implementing open source AI models.

If you are a public intellectual on "AI security" and if you don't understand this blog, you are not qualified to comment on the subject matter. Read a book, as they say.

Now that I am done with my ranting, I want to tell you this blog will tell you what all things can go wrong that may end up making your frontier model "DUMB". 

LLM inference is very fragile. The inference engine has to present the input in a strict format (Chat template) to the LLM. If you deviate a bit, the outputs are not great. If not anything, it will reduce your AGI anxiety - technology is not going to be the skynet.

Thanks @vllm_project  and Lilian Weng. Here they narrate how  they worked on feedback from the Kimi team to improve tool call success rate for Kimi k2 model being run on vLLM to near 100%.

They did it very fast after getting the feedback. Kudos. Your community service is greatly appreciate 🧡💕

Key Lesson (quoting)
The Devil is in the Chat Template: The chat_template is the critical handshake between a model and its serving framework. When integrating a new model, meticulously validate every piece of its template logic against the framework’s specific behaviors and assumptions.

Peel Back the Abstraction Layer: High-level APIs like /chat/completions are convenient but can obscure root causes. When debugging, don’t hesitate to drop down to lower-level endpoints like /completions. Manually building the input is a powerful technique to isolate the problem.

A Pro-Tip: Token IDs are the Ultimate Ground Truth: For the most subtle issues, inspecting the final sequence of token IDs sent to the model is the only way to be certain. While I didn’t need to resort to this for the issues above, it’s a critical tool in the toolbox. Techniques like using the OpenAI-compatible API to return token IDs can be a lifesaver. For those interested, we also highlighted this in our Agent Lightning post.

Understand Framework Design Philosophy: vLLM’s strict handling of **kwargs is not a bug, but a deliberate security choice. Understanding these design decisions helps in quickly identifying the root cause rather than getting stuck on unexpected behavior.

The Open Ecosystem Challenge: Advanced features like a tool-call “Enforcer” are hallmarks of polished, proprietary services. Implementing these capabilities robustly and elegantly in open-source projects like vLLM is a vital challenge for the community to address.

The Devil is in the Chat Template ========================= Potatoes, potahtoes are not the same thing, contrary to how Phoebe Buffay would have you believe. This is a must read blog if you are losing your sleep over AGI, or if you simply work on implementing open source AI models. If you are a public intellectual on "AI security" and if you don't understand this blog, you are not qualified to comment on the subject matter. Read a book, as they say. Now that I am done with my ranting, I want to tell you this blog will tell you what all things can go wrong that may end up making your frontier model "DUMB". LLM inference is very fragile. The inference engine has to present the input in a strict format (Chat template) to the LLM. If you deviate a bit, the outputs are not great. If not anything, it will reduce your AGI anxiety - technology is not going to be the skynet. Thanks @vllm_project and Lilian Weng. Here they narrate how they worked on feedback from the Kimi team to improve tool call success rate for Kimi k2 model being run on vLLM to near 100%. They did it very fast after getting the feedback. Kudos. Your community service is greatly appreciate 🧡💕 Key Lesson (quoting) The Devil is in the Chat Template: The chat_template is the critical handshake between a model and its serving framework. When integrating a new model, meticulously validate every piece of its template logic against the framework’s specific behaviors and assumptions. Peel Back the Abstraction Layer: High-level APIs like /chat/completions are convenient but can obscure root causes. When debugging, don’t hesitate to drop down to lower-level endpoints like /completions. Manually building the input is a powerful technique to isolate the problem. A Pro-Tip: Token IDs are the Ultimate Ground Truth: For the most subtle issues, inspecting the final sequence of token IDs sent to the model is the only way to be certain. While I didn’t need to resort to this for the issues above, it’s a critical tool in the toolbox. Techniques like using the OpenAI-compatible API to return token IDs can be a lifesaver. For those interested, we also highlighted this in our Agent Lightning post. Understand Framework Design Philosophy: vLLM’s strict handling of **kwargs is not a bug, but a deliberate security choice. Understanding these design decisions helps in quickly identifying the root cause rather than getting stuck on unexpected behavior. The Open Ecosystem Challenge: Advanced features like a tool-call “Enforcer” are hallmarks of polished, proprietary services. Implementing these capabilities robustly and elegantly in open-source projects like vLLM is a vital challenge for the community to address.

AI @amazon. All views personal!

avatar for GDP
GDP
Sun Nov 02 16:09:28
  • Previous
  • 1
  • More pages
  • 1305
  • 1306
  • 1307
  • More pages
  • 2111
  • Next