Orbbec tells LDS how it captures robot training data from people instead of robots

The global shutter matters more than the megapixel count for this application, because rolling-shutter distortion during fast hand motion corrupts exactly the frames a manipulation policy needs. Orbbec said the 6-axis IMU, combining accelerometer and gyroscope, runs at high sampling rates "to track precise physical motion and orientation," and that the system "ensures a synchronization accuracy of 1ms or better to guarantee precise spatial and temporal alignment across all visual and inertial data streams."

Orbbec described each capture framework as answering a distinct question about a task.

EGO records "human first-person visual perception, task processes, and real-world context during task execution." Its purpose is to help embodied AI systems understand how humans perceive environments and interact with the physical world, rather than just what the hands did.

UMI handles handheld manipulation, "recording human demonstration processes such as grasping, moving, placing, assembling, and organizing objects." This is the closest analogue to what a robot arm must eventually reproduce.

WristCam exists because of a specific failure mode. It captures "localized visual details that may be occluded from egocentric or external viewpoints," which is the moment of contact that a head-mounted camera loses sight of precisely when it matters.

Taken together, Orbbec argues, the three provide "the data foundation for learning the relationships among environment observation, task understanding, action execution, and interaction outcomes."

Multi-viewpoint capture only helps if the streams line up in space and time. Orbbec detailed two preprocessing stages that do this work.

For spatial consistency, the pipeline runs "joint calibration processes for fisheye lenses, high-distortion lenses, and heterogeneous sensors," computing intrinsic parameters such as focal length and distortion coefficients alongside extrinsic spatial pose, to achieve 3D alignment across viewpoints.

For temporal consistency the answer is hardware rather than software. Multi-device setups spanning EGO, WristCam and UMI rely on "multi-camera hardware synchronization, external trigger mechanisms, and master-slave timing design," which Orbbec said enables "microsecond-level clock synchronization and frame-level alignment, minimizing temporal offset errors among different data sources."

That distinction is the useful engineering detail. Software timestamp alignment after the fact is where home-built capture rigs typically lose fidelity, and it is what a packaged system is actually selling.

The platform's RGB-D EGO device pairs Orbbec's Gemini 330 stereo camera and MX6800 depth engine with a customized version of LingBot-Depth 2.0 from Robbyant, Ant Group's embodied-AI company. The camera captures RGB and depth synchronously, while the model is designed to fill missing depth and sharpen boundaries around transparent, reflective or occluded objects, which are the surfaces conventional depth sensing handles worst.

Orbbec reports that LingBot-Depth 2.0 was trained on 150 million samples and that it improves depth completion over the previous version. Those figures come from company testing and should not be read as independent validation of the collection platform as a whole. Economic Observer separately reported the July 7 launch and the Orbbec and Robbyant integration.

The platform is most relevant to robotics teams currently building their own capture rigs and synchronization pipelines, where a packaged system removes real integration work.

Procurement still needs task-level evidence that the published sources do not provide. Neither pricing, customer deployments, nor comparative measurements of collection throughput and dataset quality have been released. Teams should test calibration drift over sessions, clock synchronization under load, occlusion handling, export formats, annotation workflows, and ultimately whether demonstrations collected this way improve downstream policy performance. The launch makes the hardware stack more accessible. It does not yet establish how much time or cost it saves in a production data program.