探索 | Thread Easy - 展开 Twitter 线程｜阅读、总结与创作

I don’t usually post whenever someone follows me, but… Been wanting this one for years 🥹

I build stuff. On my way to making $1M 💰 My projects 👇

Florin Pop 👨🏻‍💻

Sun Dec 07 08:48:40

RT @iamgrigorev: nice, very thoughtful write about training MoE's 1. moe training dynamics is decoupled (only part of FFN is activated), t…

We're in a race. It's not USA vs China but humans and AGIs vs ape power centralization. @deepseek_ai stan #1, 2023–Deep Time «C’est la guerre.» ®1

Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)

Sun Dec 07 08:45:38

I'm starting to feel bad about this. Indians are self-bullying over losing face with the crappy red carpet Putin got. I think Putin understands very well what India is, and he gladly presides over similar (just chilled) squalor in much of Russia. would be funny if he tripped

We're in a race. It's not USA vs China but humans and AGIs vs ape power centralization. @deepseek_ai stan #1, 2023–Deep Time «C’est la guerre.» ®1

Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)

Sun Dec 07 08:43:55

🇨🇳 China can be very very cheap, I'd say hotels and accomodation can be cheaper than Thailand We had a 63rd floor Chongqing hotel suite with 2 bathrooms, 1 living room, 1 bedroom for just $300/night Normally suites in other places easily go for $1500 and up And I've seen other hotels with regular good rooms that go for $20 to $30/night Food can also be very cheap, but not a lot of Western food to be fair, so you gotta be open to new flavors 😋

🇪🇺 https://t.co/NdorAWrhrB @euacc 📸 https://t.co/lAyoqmT9Hv $115K/m 🏡 https://t.co/1oqUgfDEsx $36K/m 🛰 https://t.co/ZHSvI2wRou $42K/m 🌍 https://t.co/UXK5AFra0o $14K/m 👙 https://t.co/RyXpqGvdBB $14K/m 💾 https://t.co/M1hEUBB6da $6K

@levelsio

Sun Dec 07 08:43:52

There is lot more to say on these beautiful visualizations. Please let me know of any errors or other cool things we should know. More at my blog: How to Get and Interpret GPU Memory Profiling https://t.co/3uPt0S6RIp

Mostly research and code. If universe is an optimizer, what is its loss function? All opinions are my own.

Shital Shah

Sun Dec 07 08:39:58

In above picture, the green stripe is storage needed to hold sqrt of grad^2 for Adam. Fused Adam computes updates to apply layer by layer causing stripes. At the top it applies those updates in one shot. The spike is temp buffer needed for division before applying update.

There is lot more to say on these beautiful visualizations. Please let me know of any errors or other cool things we should know. More at my blog: How to Get and Interpret GPU Memory Profiling https://t.co/3uPt0S6RIp

Shital Shah

Sun Dec 07 08:39:58

探索

最新在前，按卡片方式浏览线程

探索

最新在前，按卡片方式浏览线程

I don’t usually post whenever someone follows me, but… Been wanting this one for years 🥹

RT @iamgrigorev: nice, very thoughtful write about training MoE's 1. moe training dynamics is decoupled (only part of FFN is activated), t…

I'm starting to feel bad about this. Indians are self-bullying over losing face with the crappy red carpet Putin got. I think Putin understands very well what India is, and he gladly presides over similar (just chilled) squalor in much of Russia. would be funny if he tripped

There is lot more to say on these beautiful visualizations. Please let me know of any errors or other cool things we should know. More at my blog: How to Get and Interpret GPU Memory Profiling https://t.co/3uPt0S6RIp

In above picture, the green stripe is storage needed to hold sqrt of grad^2 for Adam. Fused Adam computes updates to apply layer by layer causing stripes. At the top it applies those updates in one shot. The spike is temp buffer needed for division before applying update.