Case Study: Embodied AI Data Collection Project

1. Project Background

This project was initiated to fulfill the client's requirement for on-site data collection, requires us to deploy personnel on-site. These on-site personnel were primarily responsible for operating robot dogs, and execute movement trajectories according to corresponding instructions. Notably, the robot dogs were equipped with a first-person perspective camera, served as the primary tool for data capture throughout the project.

2. Project Introduction

2.1 Team Structure

2-person collection teams serve as the fundamental operational unit, with each team responsible for coordinating one robot dog throughout data collection.

 

2.2 Collection Preparation

2.2.1 Collection Equipment

2.2.1.1 Robot Dog (Remote Control with Handle) serve as the physical data collection platform, executing mobile data capture in target environments.

2.2.1.2 Laptop or Mobile Device function as the data collection platform interface, establishing stable connections with the robot dog's camera system for real-time monitoring.

2.2.1.3 Scene Configuration

1. Indoor Scene

Scene Range: maintain controlled spatial parameters of 30-50 meters;

Scene Props: Static Interference Objects, Dynamic Interference Objects;

2. Outdoor Scene

Current Scene: Open Space, Later May Involve Crowded Spaces Such as Cafes, Parks, and Supermarkets;

Scene Props: Static Interference Objects, Dynamic Interference Objects;

2.2.2 Custom Instructions

Before the collection begins, the client will provide a plan and instructions for the day's collection, including training objectives.

 

2.3 Operation Procedures

2.3.1 Data Collection Task

According to the programmed instructions, the robot dog is controlled to collect data along a specific route in a specific scenario, generating a first-person perspective video of the robot dog's path.

Working in pairs, one person uses a mobile phone or computer to confirm the robot dog's activation on the collection platform, while the other person operates the robot dog using a controller to collect data according to the prepared instruction path. Once data is collected, it is uploaded directly to the platform, where a collection ID is generated. The collector must register the ID and the captured scene image with the corresponding scene in the collection form and enter the name of the collector operating the robot dog and the date of collection.

 

2.3.2 Inspection Task

On the inspection platform, the corresponding captured video is reviewed based on the ID. The video content must be verified to ensure that the captured data follows the instruction path. If not, any discrepancies between the instruction and the video content must be recorded. The first-person perspective video captured by the robot dog during the collection process is also checked to ensure that it meets the requirements. If there is any lag, excessive camera movement, the target object is not visible at the starting point, or the image is inconsistent with the instruction, the video will be deemed unqualified. 

 

2.4 Project Challenges

2.4.1 Equipment Safety

The customer's data collection equipment is very expensive, but data collection may be subject to operational damage during operation, and may even require operation in crowded outdoor locations such as cafes, parks, and supermarkets.

Preventive Measures
1. Daily joint inspections: Post-operation equipment verification conducted by client representatives and on-site managers, documented in shared digital records

2. Insurance coverage: Client-provided equipment insurance complemented by project-managed personnel accident coverage

 

2.4.2 Equipment Stability

The customer's data collection equipment (robot dog) frequently disconnects and is unable to collect data. Bugs in the platform system prevent the robot dog from connecting properly, impacting the day's data collection output.

Preventive Measures

1.When equipment issues occur, the on-site manager will promptly report the equipment status to the customer and move the equipment to the customer's maintenance area for inspection and repair by the customer's technical team.

2. Record maintenance issues and time lost due to equipment failures, and inform customers of the daily output level through daily reports.

 

3.Lessons Learned & Future Challenges

3.1 Lessons Learned

3.1.1. Standardized incentive systems linking performance metrics to tangible rewards/punishments have proven essential for maintaining operational consistency across distributed teams.

3.1.2. Daily reporting mechanisms synchronizing output metrics and on-site conditions have enhanced client alignment and enabled proactive issue resolution.

3.1.3. When outsourcing valuable equipment and personnel, the safety of both personnel and equipment must be considered.

3.2 Future Challenges

3.2.1. Current embodied data collection expertise requires systematic codification.

3.2.2. Current reliance on client technical personnel for issue resolution necessitates development of dedicated on-site technical capabilities.

 

With the rapid development of artificial intelligence, the importance of datasets has become prominent. By accurate data annotation and scientific data collection, we can improve the performance of AI model, which enable them to cope with real application challenges. In the future, all fields of data-driven innovation will continue to drive intelligence and achieve business results in high-value. Welcome to contact Nexdata for similar data service at [email protected].

 

To learn more about solutions for Embodied AI, visit our Embodied AI Data Solution Page.