Physical AI models must learn how environments change and how robots should act within them. Achieving this requires high-quality, action-grounded robot demonstrations, realistic simulation environments, reliable policy evaluation, and safe deployment workflows. World Foundation Models (WFMs) can model and predict physical-world dynamics, while World-Action Models (WAMs) connect this understanding to executable robot actions.
This exploratory project is developing an end-to-end, reusable physical-AI pipeline covering robot teleoperation, omni-modal data preparation, synthetic-data generation, simulation-based augmentation, robot-policy fine-tuning, simulation evaluation, and staged deployment. A single-arm robot is the starting point for validating the pipeline, with plans to extend the work to additional robot platforms and embodiments.
The intern will focus primarily on robot data collection, dataset preparation, and simulation development, while also contributing to policy training, evaluation, and deployment. Likely responsibilities include:
- Supporting teleoperation experiments and collecting demonstrations containing camera observations, robot states, action trajectories, and task instructions.
- Developing Python and shell tools to annotate, convert, organise, and validate robot-learning datasets.
- Implementing data-quality checks for camera synchronisation, action schemas, annotation boundaries, frame rates, and dataset readiness.
- Reconstructing real environments using 3D Gaussian Splatting (3DGS), preparing simulation scenes and tasks, implementing domain randomisation, and defining measurable success criteria in NVIDIA Isaac Sim and Isaac Lab.
- Aligning visual scene representations with robot models and collision geometry required for physical interaction in simulation.
- Generating and quality-checking simulated demonstrations for downstream robot-policy training.
- Fine-tuning and comparing robot policies trained with real, synthetic, and simulated data.
- Evaluating policies using held-out data and repeatable simulation tests before physical deployment.
- Improving client-server inference and robot-control workflows, including observation and action interfaces, control frequency, latency measurement, and safety checks.
- Conducting staged robot trials, progressing from replay and open-loop validation to supervised closed-loop execution.
- Producing reusable code, configurations, experiment reports, benchmarks, and operational documentation.
Any work involving physical robot hardware will follow documented calibration, supervision, and safety procedures.
