Keep on to blur preview images; turn off to show them clearly
If you want the full deep dive (with visuals + demo + and fine-tuning instructions), watch my new video: https://t.co/TDdhedJiDn
Data Science Director at Agoda | Ex-TOP-20 on Kaggle | Playing with robots for fun
Another problem during the inference is non-smooth motion because of the chunking. Model predicts the next chunk, executes it, and then pauses to predict the next one (video below, x3 speed). If you try to predict a chunk before the previous one is executed, it can lead to fatal mistakes if the model jumps to a new action mode while executing a very different one. The solution is inpainting - that is often used in image generation. We can predict the next chunk while the old one is being executed, but we force this new prediction to match the end of the previous chunk exactly. The result is a much smoother motion without jumps and pauses, and higher performance and throughput of the model.
If you want the full deep dive (with visuals + demo + and fine-tuning instructions), watch my new video: https://t.co/TDdhedJiDn
Pi0.5 by @physical_int is one of the best open-source end-to-end robotics policies right now 🤖 It’s an upgraded version of Pi0. And the most recent PI advancements are built on top of it. Let's discuss how it works:
What changed vs Pi0? - FAST tokenization - there was an optional FAST version on Pi0, but for Pi0.5, it is an essential part of the training - System 2 - following the Hi Robot paper, Pi 0.5 is using its VLM part as a reasoning high-level system 2 to reason and plan complex tasks - Better training recipe and a couple of smaller tweaks.
Even though we won first place, we believe there is still huge room for improvement. We achieved a 26% q score and an 11–12% binary success rate. The main reasons the policy still fails are: - Dexterity issues (grasps, releases) - Progress errors in long sequences - Getting confused after entering out-of-distribution states
We open-sourced everything in our solution: the code, model weights, and a detailed tech report. Code: https://t.co/LLSd6VtbaE Weights: https://t.co/f3ZUF175rV Tech report: https://t.co/TeFiiTha0d I will also record a video walkthrough with more details later. Stay tuned 🎥
We open-sourced everything in our solution: the code, model weights, and a detailed tech report. Code: https://t.co/LLSd6VtbaE Weights: https://t.co/f3ZUF175rV Tech report: https://t.co/TeFiiTha0d I will also record a video walkthrough with more details later. Stay tuned 🎥
Data Science Director at Agoda | Ex-TOP-20 on Kaggle | Playing with robots for fun
A couple of days ago, I presented our 1st place solution for the 2025 BEHAVIOR Challenge at @NeurIPSConf . Now, we've open-sourced our solution: code, model weights, and a detailed tech report. Let me unpack what we did 👇
What is the BEHAVIOR Challenge? In this competition, we had to train a policy that could complete 50 robotic household tasks in a high-quality simulation. The policy controls a bimanual humanoid robot with a mobile base, and the tasks range from 1 to 14 minutes. Read more details in @drfeifei post: https://t.co/jDviv5d6pB