He Wang
Prof.He Wang

Tenure-track Assistant Professor at Peking University

Director of Embodied Perception and InteraCtion (EPIC) Lab

Director of PKU-Galbot Joint Lab of Embodied AI

I am a tenure-track assistant professor in the Center on Frontiers of Computing Studies (CFCS) at Peking University. I founded and lead the Embodied Perception and InteraCtion (EPIC) Lab with the mission of developing generalizable skills and embodied multimodal large model for robots to facilitate embodied AGI.

I am also the director of the PKU-Galbot joint lab of Embodied AI and the BAAI center of Embodied AI. I have published more than 50 papers in top conferences and journals of computer vision, robotics, and learning, including CVPR/ICCV/ECCV/TRO/ICRA/IROS/NeurIPS/ICLR/AAAI. My pioneering work on category-level 6D pose estimation, NOCS, received the 2022 World Artificial Intelligence Conference Youth Outstanding Paper (WAICYOP) Award, and my work also received ICCV 2023 best paper finalist, ICRA 2023 outstanding manipulation paper award finalist and Eurographics 2019 best paper honorable mention.

I serve as an associate editor of Image and Vision Computing and serve as an area chair in CVPR 2022 and WACV 2022. Prior to joining Peking University, I received my Ph.D. degree from Stanford University in 2021 under the advisory of Prof. Leonidas J.Guibas and my Bachelor's degree from Tsinghua University in 2014.

NEWS
SELECTED PUBLICATIONS
Task-Oriented Dexterous Grasp Synthesis via Differentiable Grasp Wrench Boundary Estimator

Jiayi Chen,Yuxing Chen,Jialiang Zhang, He Wang

IROS 2024

Open6DOR: Benchmarking Open-instruction 6-DoF Object Rearrangement and A VLM-based Approach

Yufei Ding,Haoran Geng , Chaoyi Xu ,Xiaomeng Fang,Jiazhao Zhang,Songlin Wei, Qiyu Dai, Zhizheng Zhang, He Wang

IROS 2024 (Oral Presentation)

NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation

Jiazhao Zhang, Kunyu Wang ,Rongtao Xu* ,Gengze Zhou ,Yicong Hong ,Xiaomeng Fang ,Qi Wu ,Zhizheng Zhang , He Wang

RSS 2024

SAGE: Bridging Semantic and Actionable Parts for Generalizable Manipulation of Articulated Objects

Haoran Geng*, Songlin Wei*, Congyue Deng, Bokui Shen, He Wang, Leonidas Guibas

RSS 2024

            
          
Enhancing Generalizable 6D Pose Tracking of an In-Hand Object with Tactile Sensing

Yun Liu*, Xiaomeng Xu*, Weihang Chen, Haocheng Yuan, He Wang, Jing Xu, Rui Chen, Li Yi

RA-L

MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentation

Mi Yan, Jiazhao Zhang, Yan Zhu, He Wang

CVPR 2024

Category-Level Multi-Part Multi-Joint 3D Shape Assembly

Yichen Li ,Kaichun Mo, Yueqi Duan , He Wang, Jiequan Zhang, Lin Shao, Wojciech Matusik, Leonidas Guibas

CVPR 2024

            
          
STOPNet: Multiview-based 6-DoF Suction Detection for Transparent Objects on Production Lines

Yuxuan Kuang*, Qin Han*, Danshi Li, Qiyu Dai, Lian Ding, Dong Sun, Hanlin Zhao, He Wang

ICRA 2024

GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusion

Jiazhao Zhang*, Nandiraju Gireesh*, Jilong Wang, Xiaomeng Fang, Chaoyi Xu, Weiguang Chen, Liu Dai, He Wang

ICRA 2024

            
          
ASGrasp: Generalizable Transparent Object Reconstruction and 6-DoF Grasp Detection from RGB-D Active Stereo Camera

Jun Shi, Yong A, Yixiang Jin, Dingzhe Li, Haoyu Niu, Zhezhu Jin, He Wang

ICRA 2024

UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning

Weikang Wan*, Haoran Geng*, Yun Liu, Zikang Shan, Yaodong Yang, Li Yi, He Wang

ICCV 2023 (Oral & Best Paper Finalist, final reviews of all strong accepts)

            
          
GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts

Haoran Geng*, Helin Xu*, Chengyang Zhao*, Chao Xu, Li Yi, Siyuan Huang, He Wang

CVPR 2023 (Highlight, , final reviews of all accepts)

            
          
3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification

Jiazhao Zhang*, Liu Dai*, Fanpeng Meng, Qingnan Fan, Xuelin Chen, Kai Xu, He Wang

CVPR 2023

            
          
UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy

Yinzhen Xu*, Weikang Wan*, Jialiang Zhang*, Haoran Liu*, Zikang Shan, Hao Shen, Ruicheng Wang, Haoran Geng, Yijia Weng, Jiayi Chen, Tengyu Liu, Li Yi, He Wang

CVPR 2023

            
          
PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observations

Haoran Geng*, Ziming Li*, Yiran Geng, Jiayi Chen, Hao Dong, He Wang

CVPR 2023

            
          
Delving into Discrete Normalizing Flows on SO(3) Manifold for Probabilistic Rotation Modeling

Yulin Liu*, Haoran Liu*, Yingda Yin*, Yang Wang, Baoquan Chen, He Wang

CVPR 2023

            
          
DiGA: Distil to Generalize and then Adapt for Domain Adaptive Semantic Segmentation

Fengyi Shen, Akhil Gurram, Ziyuan Liu, He Wang, Alois Knoll

CVPR 2023

            
          
Adaptive Zone-aware Hierarchical Planner for Vision-Language Navigation

Chen Gao, Xingyu Peng, Mi Yan, He Wang, Lirong Yang, Haibing Ren, Hongsheng Li, Si Liu

CVPR 2023

DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation

Ruicheng Wang*, Jialiang Zhang*, Jiayi Chen, Yinzhen Xu, Puhao Li, Tengyu Liu, He Wang

ICRA 2023 (Outstanding Manipulation Paper Award Finalist)

            
          
GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF

Qiyu Dai*, Yan Zhu*, Yiran Geng, Ciyu Ruan, Jiazhao Zhang, He Wang

ICRA 2023

            
          
A Laplace-inspired Distribution on SO(3) for Probabilistic Rotation Estimation

Yingda Yin, Yang Wang, He Wang, Baoquan Chen

ICLR 2023 (notable top 25%)

            
          
Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance

Xueyi Liu, Ji Zhang, Ruizhen Hu, Haibin Huang, He Wang, Li Yi

ICLR 2023

            
          
Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild

Jiayi Chen*, Mi Yan*, Jiazhao Zhang, Yenzhen Xu, Xiaolong Li, Yijia Weng, Li Yi, Shuran Song, He Wang

AAAI 2023 (Oral Presentation)

            
          
ASRO-DIO: Active Subspace Random Optimization Based Depth Inertial Odometry

Jiazhao Zhang, Yijie Tang, He Wang, Kai Xu

IEEE Transactions on Robotics (T-RO)

Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects

Qiyu Dai*, Jiyao Zhang*, Qiwei Li, Tianhao Wu, Hao Dong, Ziyuan Liu, Ping Tan, He Wang

ECCV 2022

            
          
Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations

Hao Shen*, Weikang Wan*, He Wang

Robotics and Automation Letters (RA-L) and IROS 2022

1st prize winner of SAPIEN ManiSkill Challenge 2021 (no external annotation track)

            
          
FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filtering

Yingda Yin, Yingcheng Cai, He Wang, Baoquan Chen

CVPR 2022 (Oral Presentation)

            
          
Projective Manifold Gradient Layer for Deep Rotation Regression

Jiayi Chen, Yingda Yin, Tolga Birdal, Baoquan Chen, Leonidas Guibas, He Wang

CVPR 2022

            
          
ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation

Jiayi Chen, Yingda Yin, Tolga Birdal, Baoquan Chen, Leonidas Guibas, He Wang

CVPR 2022 (Oral Presentation)

            
          
HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction

Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, Li Yi

CVPR 2022

            
          
Multi-Robot Active Mapping via Neural Bipartite Graph Matching

Kai Ye*, Siyan Dong*, Qingnan Fan, He Wang, Li Yi, Fei Xia, Jue Wang, Baoquan Chen

CVPR 2022

            
          
CodedVTR: Codebook-based Sparse Voxel Transformer with Geometric Guidance

Tianchen Zhao, Niansong Zhang, Xuefei Ning, He Wang, Li Yi, Yu Wang

CVPR 2022

            
          
Domain Adaptation on Point Clouds via Geometry-Aware Implicits

Yuefan Shen*, Yanchao Yang*, Mi Yan, He Wang, Youyi Zheng, Leonidas J. Guibas

CVPR 2022

            
          
Leveraging SE(3) Equivariance for Self-supervised Category-Level Object Pose Estimation from Point Clouds

Xiaolong Li, Yijia Weng, Li Yi, Leonidas J. Guibas, A. Lynn Abbott, Shuran Song, He Wang

NeurIPS 21

            
          
CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds

Yijia Weng*, He Wang*†, Qiang Zhou, Yuzhe Qin, Yueqi Duan, Qingnan Fan, Baoquan Chen, Hao Su, Leonidas J. Guibas

ICCV 2021 (Oral Presentation)

            
          
Single Image 3D Shape Retrieval via Cross-Modal Instance and Category Contrastive Learning

Mingxian Lin, Jie Yang, He Wang, Yu-Kun Lai, Rongfei Jia, Binqiang Zhao, Lin Gao

ICCV 2021

            
          
3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection

He Wang*, Yezhen Cong*, Or Litany, Yue Gao and Leonidas J. Guibas

ICCV 2021

            
          
MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization

Jiahui Huang, He Wang, Tolga Birdal, Minkyuk Sung, Federica Arrigoni, Shi-Min Hu, and Leonidas J. Guibas

CVPR 2021 (Oral Presentation)

            
          
Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments

Siyan Dong*, Qingnan Fan*, He Wang, Ji Shi, Li Yi, Thomas Funkhouser, Baoquan Chen, Leonidas J. Guibas

CVPR 2021 (Oral Presentation)

            
          
Rethinking Sampling in 3D Point Cloud Generative Adversarial Networks

He Wang*, Zetian Jiang*, Li Yi, Kaichun Mo, Hao Su, Leonidas J. Guibas

CVPR 2021 Workshop on Learning to Generate 3D Shapes and Scenes

            
          
PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

Kaichun Mo, He Wang, Li Yi, Xinchen Yan and Leonidas J.Guibas

ECCV 2020

            
          
Curriculum DeepSDF

Yueqi Duan*, Haidong Zhu*, He Wang, Li Yi, Ram Nevatia, Leonidas J. Guibas

ECCV 2020

            
          
Category-level Articulated Object Pose Estimation

He Wang*, Xiaolong Li*, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song

CVPR 2020 (Oral Presentation)

            
          
SAPIEN: A SimulAted Part-based Interactive ENvironment

Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X.Chang, Leonidas J. Guibas and Hao Su

CVPR 2020 (Oral Presentation)

            
          
Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation

He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, Leonidas J. Guibas

CVPR 2019 (Oral Presentation), WAICYOP Award 2022

            
          
GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud

Li Yi, Wang Zhao, He Wang, Minhyuk Sung, Leonidas Guibas

CVPR 2019

            
          
Learning a Generative Model for Multi-Step Human-Object Interactions from Videos

He Wang*, Soeren Pirk*, Ersin Yumer, Vladimir Kim, Ozan Sener, Srinath Sridhar, Leonidas J. Guibas

Eurographics 2019 (Best Paper Honorable Mention)

AWARDS
  • ICCV 2023 Best Paper Finalist
  • ICRA 2023 Outstanding Manipulation Paper Finalist
  • 2022 World Artificial Intelligence Conference Youth Outstanding Paper Award
  • Eurographics 2019 Best Paper Honorable Mention
  • 1st prize winner of SAPIEN ManiSkill Challenge 2021 (no external annotation track)
TEACHING
PROFESSIONAL SERVICE
  • Associate Editor: Image and Vision Computing
  • Area chair (AC):
    Conferences: CVPR 2022, WACV 2022
    Seminars: VALSE
  • Program committee/reviewer:
    Conferences: CVPR, ICCV, ECCV, NeurIPS, ICLR, AAAI, SIGGRAPH, RSS, IROS, ICRA
    Journals: IEEE TPAMI, IEEE RAL
OPPORTUNITIES

We are actively looking for interns,full-time employees, master/PhD students, and postdocs. Feel free to contact me if you are interested in my research or potential collaborations.

  • For graduate school applicants, we have two openings for PhD students (in addition to master/PhD openings for foreigners) each year, and please contact me at least one year prior to the application deadline.
  • For visiting students or research interns, we welcome undergradute and graduate students from top universities all world wide to apply for >6 months research internship. Our interns have published many top-tier conference/journal papers and have been admitted to PhD/MS programs in Stanford, CMU, UCLA, UCSD, etc.
  • In Embodied AI center of BAAI and our Embodied AI startup, Galbot, we also actively hire full-time research scientists, engineers and interns. Email me if you are interested.