Keep on to blur preview images; turn off to show them clearly
Kimi AMA on K2 Thinking: 1. $4.6M training cost is not an official number 2. Trained on H800s (nerfed H100s) 3. KDA (Kimi Delta Attention) hybrids with NoPE MLA perform better than full MLA with RoPE 4. Muon scales well to 1T parameters. “there are tens of optimizers and architectures that do not survive the grill.” 5. Kimi K2 will have vision 6. K2 Thinking is natively INT4 to be friendlier to non-Blackwell GPUs while leveraging the existing int4 inference marlin kernels.
- “wen K3?” - “before sam's trillion-dollar data center is built” 😂 AMA link: https://t.co/6yZSsjQXvM
Assuming the model companies can touch every market and, more importantly, *do it well*, and that the advantage to being multi model is limited feels a lot like sitting out the last 20 years because Google/Amazon/Facebook could be the only winners
partner @a16z // saas + b2b fintech // strong opinions on 🍕
RT @jamisonfox: Big milestone for @GammaApp today: we’ve raised a $68M Series B led by Sarah Wang at @a16z. It’s been humbling to see this…
Growth investing @a16z
RT @thatsjonsense: 5 years ago, we started a company with a mission that sounded a little nuts: reinvent the slide deck. Did I think we'd…
Growth investing @a16z
Since launching my newsletter 2 weeks ago, I have gained at least 1 new subscriber per day, every single day... except today. 🥲 Can we keep the streak going? 😬 I promise to make it worthwhile for you! https://t.co/b9bS78DQk3
I build stuff. On my way to making $1M 💰 My projects 👇
RT @a16z: We’re proud to lead Gamma’s Series B at a $2.1B valuation and back the team building the anti-PowerPoint. In a post-AI world, th…
Growth investing @a16z