Tsun-Yi Yang

I am a Staff Research Scientist at Graphcore, focusing on world models, dynamic scenes, and novel view synthesis.

Previously I was founder and CEO of MIMI AI, a senior research scientist at Meta Reality Labs, a senior LLM research engineer at Robin AI, and an applied scientist at Amazon. Before that I worked at Scape Technologies and Academia Sinica.

I received my Ph.D. in computer science from National Taiwan University. My research focuses on world models, dynamic scenes, novel view synthesis, spatial intelligence, 3D scene understanding, and applied multimodal systems.

Highlights

VibePhysics: Realistic Walkthrough with Blender Animation

A demo of turning real-world capture into a navigable 3D scene — training-free 4d recon+indexing system in smooth blender navigation system. [code]

Publications

VibePhysics simulation teaser

VibePhysics: CPU-Runnable Physical Simulation for Robotics

A lightweight framework for turning real-world videos and images into 3D maps, Blender scenes, and physical simulations — bridging feedforward reconstruction, sparse mapping, robotics, and physics in one CPU-friendly workflow.

Tsun-Yi Yang

MIMI AI, 2025 [code]

SceneScript synthetic scenes

SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model

Armen Avetisyan, Christopher Xie, Henry Howard-Jenkins, Tsun-Yi Yang, Samir Aroudj, Suvam Patra, Fuyang Zhang, Duncan Frost, Luke Holland, Campbell Orme, Jakob Engel, Edward Miller, Richard Newcombe, Vasileios Balntas

ECCV 2024 [project] [blog] [paper]

Aria Synthetic Environments scenes

Aria Synthetic Environments Dataset

100k indoor data from scratch. Including layout, material, trajectories, along with 1000+ GPU-hours rendering results. Enabling pointcloud based VLM pre-training and post-training.

Suvam Patra, Samir Aroudj, Tsun-Yi Yang

Project Aria, 2024 [dataset]

Robin AI legal AI model article

Regulated-Domain LLM Post-Training for Legal AI

Tsun-Yi led cross domain teams including legal, product, and engineering along with collaboration with Anthropic and AWS team. Post-training outperforming OpenAI o1-preview in legal domain with extensive analysis. Efforts including RLHF, synthetic data, fine-tuning.

Tsun-Yi Yang, Jack Savage, Kitty Boxall, Will Barnsley, Karolina Lukoszova, Carina Negreanu

Robin AI, 2024 [news] [pdf]

OrienterNet teaser

OrienterNet: Visual Localization in 2D Public Maps with Neural Matching

Paul-Edouard Sarlin, Daniel DeTone, Tsun-Yi Yang, Armen Avetisyan, Julian Straub, Tomasz Malisiewicz, Samuel Rota Bulo, Richard Newcombe, Peter Kontschieder, Vasileios Balntas

CVPR 2023 [project] [paper]

Nerfels renderable neural codes

Nerfels: Renderable Neural Codes for Improved Camera Pose Estimation

Gil Avraham, Julian Straub, Tianwei Shen, Tsun-Yi Yang, Hugo Germain, Chris Sweeney, Vasileios Balntas, David Novotny, Daniel DeTone, Richard Newcombe

CVPR Workshops 2022 [paper]

NinjaDesc content-concealing descriptors

NinjaDesc: Content-Concealing Visual Descriptors via Adversarial Learning

Tony Ng, Hyo Jin Kim, Vincent Lee, Daniel DeTone, Tsun-Yi Yang, Tianwei Shen, Eddy Ilg, Vassileios Balntas, Krystian Mikolajczyk, Chris Sweeney

CVPR 2022 [paper]

Implicit map augmentation for relocalization

Implicit Map Augmentation for Relocalization

Yuxin Hou, Tianwei Shen, Tsun-Yi Yang, Daniel DeTone, Hyo Jin Kim, Chris Sweeney, Richard Newcombe

ECCV Workshops 2022 [paper]

UR2KiD unified retrieval and keypoints

UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description

Tsun-Yi Yang, Duy-Kien Nguyen, Huub Heijnen, Vassileios Balntas

arXiv 2020 [paper]

FSA-Net head pose estimation

FSA-Net: Learning Fine-Grained Structure Aggregation for Head Pose Estimation From a Single Image

Tsun-Yi Yang, Yi-Ting Chen, Yen-Yu Lin, Yung-Yu Chuang

CVPR 2019 [paper]

DAME WEB landmark retrieval

DAME WEB: DynAmic MEan with Whitening Ensemble Binarization for Landmark Retrieval without Human Annotation

Tsun-Yi Yang, Duy-Kien Nguyen, Huub Heijnen, Vassileios Balntas

ICCV Workshops 2019 [paper]

SSR-Net age estimation

SSR-Net: A Compact Soft Stagewise Regression Network for Age Estimation

Tsun-Yi Yang, Yi-Hsuan Huang, Yen-Yu Lin, Pi-Cheng Hsiu, Yung-Yu Chuang

IJCAI 2018 [paper]

DeepCD complementary descriptors

DeepCD: Learning Deep Complementary Descriptors for Patch Representations

Tsun-Yi Yang, Jo-Han Hsu, Yen-Yu Lin, Yung-Yu Chuang

ICCV 2017 [paper]

Accumulated stability voting descriptor

Accumulated Stability Voting: A Robust Descriptor From Descriptors of Multiple Scales

Tsun-Yi Yang, Yen-Yu Lin, Yung-Yu Chuang

CVPR 2016 [paper]

For the full publication list and citation counts, see Google Scholar.

Awards, Services, Grants

  • 2026, Neural Information Processing Systems (NeurIPS), Reviewer
  • 2026, IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), Reviewer
  • 2023, IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), Reviewer
  • 2023, International Conference on Learning Representations (ICLR), Reviewer
  • 2022, IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), Reviewer
  • 2022, IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), Image Matching Workshop, Reviewer
  • 2021, IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), Outstanding Reviewer
  • 2021, IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), Image Matching Workshop, Reviewer
  • 2021, IEEE / CVF International Conference on Computer Vision (ICCV), Reviewer
  • 2020, Association for the Advancement of Artificial Intelligence (AAAI), Program Committee (PC)
  • 2020, Neural Information Processing Systems (NeurIPS), Reviewer
  • 2019, IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), Outstanding Reviewer
  • 2019, IEEE / CVF International Conference on Computer Vision (ICCV), Reviewer
  • 2019, Association for the Advancement of Artificial Intelligence (AAAI), Reviewer
  • 2018, Scholarship, Pan Wen Yuan Foundation

Education

  • PhD in Computer Science, National Taiwan University, 2019
  • MS in Applied Physics, National Taiwan University, 2013
  • BS in Physics, National Taiwan University, 2011