探索

最新在前,按卡片方式浏览线程

开启时会模糊预览图,关闭后正常显示

Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understanding

Contributions:
• We propose Motion4D, a model that integrates 2D priors from foundation models into a dynamic 3D Gaussian Splatting representation. This achieves consistent motion and semantic modeling from monocular videos.

• We design a two-part iterative optimization framework comprising:
  - Sequential optimization, which updates motion and semantic fields in consecutive stages to maintain local consistency.
  - Global optimization, which jointly refines all attributes to ensure long-term coherence.

• We introduce iterative motion refinement using 3D confidence maps and adaptive resampling to enhance dynamic scene reconstruction, alongside semantic refinement to correct 2D semantic inconsistencies through iterative updates with SAM2.

• Our Motion4D significantly outperforms both 2D foundation models and existing 3D methods in tasks including video object segmentation, point-based tracking, and novel view synthesis.

Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understanding Contributions: • We propose Motion4D, a model that integrates 2D priors from foundation models into a dynamic 3D Gaussian Splatting representation. This achieves consistent motion and semantic modeling from monocular videos. • We design a two-part iterative optimization framework comprising: - Sequential optimization, which updates motion and semantic fields in consecutive stages to maintain local consistency. - Global optimization, which jointly refines all attributes to ensure long-term coherence. • We introduce iterative motion refinement using 3D confidence maps and adaptive resampling to enhance dynamic scene reconstruction, alongside semantic refinement to correct 2D semantic inconsistencies through iterative updates with SAM2. • Our Motion4D significantly outperforms both 2D foundation models and existing 3D methods in tasks including video object segmentation, point-based tracking, and novel view synthesis.

Paper: https://t.co/WFqbtsThCk Project: https://t.co/SEmM7IIDB5

avatar for MrNeRF
MrNeRF
Whenever I see programmer salary numbers from Europe, I always have to do a double take. It's hard to fathom that we work in the same industry. With bonuses this year, many of our folks are at $400,000+, and we're not even in any AI-hype sector.

Whenever I see programmer salary numbers from Europe, I always have to do a double take. It's hard to fathom that we work in the same industry. With bonuses this year, many of our folks are at $400,000+, and we're not even in any AI-hype sector.

Father of three, Creator of Ruby on Rails + Omarchy, Co-owner & CTO of 37signals, Shopify director, NYT best-selling author, and Le Mans 24h class-winner.

avatar for DHH
DHH
RT @TechFundies: Databricks held a webinar on agentic AI with 3 large enterprise CIOs / AI specialists.

My punchline takeaway is that agen…

RT @TechFundies: Databricks held a webinar on agentic AI with 3 large enterprise CIOs / AI specialists. My punchline takeaway is that agen…

Professor at NYU. Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.

avatar for Yann LeCun
Yann LeCun
in retrospect, something i like with our synth announcement: we never mention france or the eu. for all mankind.

in retrospect, something i like with our synth announcement: we never mention france or the eu. for all mankind.

Artisanal baker of reasoning models @pleiasfr

avatar for Alexander Doria
Alexander Doria
in retrospect, something i like with our synth announcement: we never mention france or the eu. for all mankind.

in retrospect, something i like with our synth announcement: we never mention france or the eu. for all mankind.

Artisanal baker of reasoning models @pleiasfr

avatar for Alexander Doria
Alexander Doria
in retrospect, something i like with our synth announcement: we never mention france or the eu. for all mankind.

in retrospect, something i like with our synth announcement: we never mention france or the eu. for all mankind.

Artisanal baker of reasoning models @pleiasfr

avatar for Alexander Doria
Alexander Doria