Keep on to blur preview images; turn off to show them clearly
i am just playing guys, you know i love you.
RL and efficient distributed pretraining • eXperiments lab • memes and training lores
the plan? recognize the ascent of web-scale pretraining and its importance for future economic growth around the year 2021. understand a significant percentage of the data powering modern models comes from reddit or wikipedia, two massive community-organized knowledgebases. slowly buy and bribe access to the accounts of influential moderators and embed yourself deeper into the volunteer cabals that control the modern internet. and then, wait. wait for years until the data that you control becomes an indispensable part of the modern frontier model pipeline. and then, act. start with something small: a fake fact, spread across several articles and subreddits, goes undetected. months later it's embedded into the parameters of Gemini, chatGPT, and Claude. the models faithfully recount said fact as true with no thought as to how it might be misinformation that goes all the way back to the training data. good. your hard work has paid off, and the approach works beautifully. you control the fundamental information sources of the digital economy. you control the models. the internet belongs to you. the models belong to you. Claude belongs to you.
phd research @cornell // language models, information theory, science of AI
the plan? recognize the ascent of web-scale pretraining and its importance for future economic growth around the year 2021. understand a significant percentage of the data powering modern models comes from reddit or wikipedia, two massive community-organized knowledgebases. slowly buy and bribe access to the accounts of influential moderators and embed yourself deeper into the volunteer cabals that control the modern internet. and then, wait. wait for years until the data that you control becomes an indispensable part of the modern frontier model pipeline. and then, act. start with something small: a fake fact, spread across several articles and subreddits, goes undetected. months later it's embedded into the parameters of Gemini, chatGPT, and Claude. the models faithfully recount said fact as true with no thought as to how it might be misinformation that goes all the way back to the training data. good. your hard work has paid off, and the approach works beautifully. you control the fundamental information sources of the digital economy. you control the models. the internet belongs to you. the models belong to you. Claude belongs to you.
phd research @cornell // language models, information theory, science of AI
researchers when asked to switch from bf16 to fp16 and do loss scaling because it is way better for RL
i am just playing guys, you know i love you.
老实说，我感觉果U跑本地大模型的时代真要来了。要是M5、M6的AI性能再整个大活，咱直接一步到位 Pro 版，告别万年丐。毕竟一张 RTX 3080 就两千多大洋了。有老哥测试 Gemma 3n 4B，M4 Pro 速度已经能跟 3080打个来回，功耗才35瓦，对面可是350瓦！
🧠在家居士 | 🥦素食者 | 🏃🏻马拉松爱好者 | 💰省钱小能手 | 搭🪜技术资深学者 | 👨💻科技宅 | 🆕更新狂 | 🆅 六边型战五渣
AMD is expanding its processor portfolio beyond the x86 architecture with its first ARM-based APU, internally known as “Sound Wave.” https://t.co/2gDR2FA7pW
Wonderer. Amor fati. Scaling trust.