Eric-Tuan Le

I am a Research Scientist at Meta AI working on advancing vision capabilities in the Llama family of models. Previously, I completed my PhD at University College London's Smart Geometry Processing Group under Iasonas Kokkinos and Niloy J. Mitra, specializing in Deep Learning, Computer Vision, and Computer Graphics with emphasis on 3D representation and reconstruction.

I hold two Master's degrees: one in Applied Mathematics from CentraleSupelec, where I graduated with the highest honors, and another in Machine Learning and Computer Vision (Master MVA) from Ecole Normale Supérieure Paris-Saclay (formerly known as ENS Cachan).

Prior to joining Meta full-time, I gained diverse industry experience through internships at Adobe Research (San Jose) working on primitive fitting for large 3D point clouds, Snap Inc. (London) focusing on 3D body mesh reconstruction from 2D images, and Meta AI (London) where I contributed to 3D generation research.

Email  /  CV  /  Google Scholar  /  Twitter  /  Github

We introduce Llama 3, a new family of foundation models supporting multilinguality, coding, reasoning, and tool use. The largest model is a 405B-parameter Transformer with a 128K-token context window. Llama 3 matches GPT-4 in quality across various tasks. We publicly release pre-trained and post-trained versions, along with Llama Guard 3 for safety. Additionally, we explore integrating image, video, and speech capabilities, achieving competitive performance.

We introduce Meta 3D Gen (3DGen), a fast, state-of-the-art pipeline for text-to-3D asset generation. 3DGen creates high-quality, PBR-compatible 3D assets in under a minute and supports generative retexturing. By integrating Meta 3D AssetGen and TextureGen, it combines view, volumetric, and UV space representations, achieving a 68% win rate over single-stage models. 3DGen outperforms industry baselines in prompt fidelity, visual quality, and speed.

We introduce MeshPose, a method that unifies DensePose and Human Mesh Reconstruction (HMR). By leveraging new losses, we use weak DensePose supervision to localize a subset of mesh vertices in 2D (‘VertexPose’) and lift them to 3D, creating a low-poly body mesh (‘MeshPose’). Our end-to-end system achieves competitive DensePose accuracy while remaining lightweight and efficient, making it suitable for real-time AR applications.

We introduce StyleMorph, a 3D-aware generative model that disentangles shape, pose, object appearance, and background for high-quality image synthesis. It learns a 3D morphable model in an unsupervised manner by morphing a canonical template and leveraging implicit surface rendering of "Template Object Coordinates" (TOCS), an unsupervised alternative to UV maps. A StyleGAN-based deferred neural rendering network conditions synthesis on 2D TOCS maps and independent appearance codes. StyleMorph achieves competitive results on multiple datasets while enabling joint disentanglement of shape, pose, and texture.

We introduce SoftMesh, a fully differentiable pipeline that converts 3D point clouds into probabilistic mesh representations for direct 2D image rendering. By learning point connectivity with only 2D rendering supervision, SoftMesh reduces the need for full mesh supervision. We evaluate its performance on silhouette, normal, and depth rendering for both rigid and non-rigid objects, exploring transfer learning and cross-category learning. SoftMesh achieves competitive results, rivaling methods trained with full mesh supervision.

We introduce Cascaded Primitive Fitting Networks (CPFN), which detect both large and fine-scale primitives in high-resolution point cloud scans. CPFN utilizes an adaptive patch sampling network to combine global and local primitive detection results. A dynamic merging formulation aggregates primitives across scales. Our evaluation shows that CPFN improves state-of-the-art SPFN performance by 13-14% and enhances fine-scale primitive detection by 20-22% on high-resolution datasets.

We introduce Lean Point Networks (LPNs), which enhance point processing networks by improving memory usage, inference time, and accuracy. LPNs feature three novel blocks: a memory-efficient convolution block for point sets, a crosslink block for efficient information sharing across resolution levels, and a multi-resolution processing block for faster information diffusion. These blocks enable the design of deeper, wider point-based architectures. Our approach achieves significant improvements in accuracy and memory consumption across multiple segmentation tasks, using LPN modules as drop-in replacements in existing architectures like PointNet++, DGCNN, SpiderNet, and PointCNN.

Source code from Jon Barron's website