Running high-fidelity O3DE simulations in AWS RoboMaker

Running high-fidelity O3DE simulations in AWS RoboMaker

Developing autonomous robots is hard. Because it requires both hardware and software to be developed, integrated and tested, doing so in a physical environment is difficult and costly. Using simulations, robot developers can test their code quickly, iterating many times before finally deploying software to the hardware for testing in the real world. Having a high-fidelity simulator allows developers to test vision, perception, and recognition systems for their robots with high confidence. Open 3D Engine (O3DE) is a community driven, open source simulator that provides that level of high-fidelity realistic rendering needed for robotic simulations.

An Introduction to O3DE

O3DE is a 3D engine capable of creating and running realistic 3D worlds for gaming. To achieve photo-realistic rendering, it uses a configurable, multi-threaded rendering engine known as Atom, which supports features such as physically based rendering (PBR), ray-tracing, post-processing, and forward + rendering. O3DE also is capable of simulating real-world physics through its Nvidia PhysX based physics engine.

O3DE is based on a modular system architecture known as the Gems system. This allows you to select and configure any number of packages (Gems) for a project depending on its needs. Gems can come from the O3DE project itself, third-party providers, or custom made. With the recent addition of the ROS 2 gem, this enables O3DE to work with robotics simulations that use the Robotic Operating System (ROS) to develop robots. By providing scene rendering and physics modeling, robotics developers can test and train perception models that closely resemble the real world in a pure simulation environment.

In order to speed up the development cycle, developers can scale out their simulations by running them in parallel in the cloud. Being able to run simulations in the cloud adds performance, cost effectiveness, and scalability to the robot development process by providing access to high performance GPU instances on demand without the need for costly investment in physical hardware or on-site infrastructure. AWS RoboMaker supports GPU-based simulation jobs by running any simulator in an OCI compliant container such as Docker.

This tutorial will go over creating a Docker image from an O3DE sample application (Robot Vacuum Sample) and deploying it as a simulation job in AWS RoboMaker.

Prerequisites

This tutorial requires an AWS account login with Amazon ECR and AWS RoboMaker access IAM permissions. It also requires a development system, or an AWS Cloud9 development environment with the following requirements:

• Linux Ubuntu 20.04 or 22.04
• At least 60 GB of free disk space
• AWS CLI installed
• Docker engine installed.

Walkthrough

Build the container images

The Robot Vacuum Sample application comes with a Dockerfile that is used to build container images for both the simulation as well as the navigation stack.

*Note: All of the command line examples in this walkthrough will be based on a workspace folder $WORKSPACE, which can exist anywhere on the host system that is accessible to the currently logged in user. It is recommended to set the environment variable $WORKSPACE to $HOME/o3de-sample. *

• Set the environment variable for $WORKSPACE

to use for this walkthrough.export WORKSPACE=$HOME/o3de-sample

• Clone the Robot Vacuum Sample application repository. mkdir -p $WORKSPACE git clone https://github.com/o3de/RobotVacuumSample $WORKSPACE/RobotVacuumSample

• Build a simulation docker image named o3de_robot_vacuum_simulation

.cd $WORKSPACE/RobotVacuumSample/Docker docker build --build-arg IMAGE_TYPE=simulation -t o3de_robot_vacuum_simulation .

• Build the navigation stack docker image named o3de_robot_vacuum_navstack

.cd $WORKSPACE/RobotVacuumSample/Docker docker build --build-arg IMAGE_TYPE=navstack -t o3de_robot_vacuum_navstack .

• If your environment meets the minimum O3DE hardware requirements, you can run these docker containers by following the docker readme.

Build and push the AWS RoboMaker container images to Amazon ECR

The container images created for the demo are designed to run locally on a Linux host machine. In order to run on AWS RoboMaker, the container images need to have the NICE DCV plugins installed. NICE DCV is a high-performance remote display protocol that will be used to stream the simulation from AWS RoboMaker.

• Extend the simulation docker image above to enable NICE DCV by creating a file $WORKSPACE/Dockerfile.simulation.robomaker

and copying the following content into it.FROM o3de_robot_vacuum_simulation:latest ENV WORKSPACE=/data/workspace WORKDIR $WORKSPACE ENV DEBIAN_FRONTEND=noninteractive ENV LAUNCH_FULLSCREEN_OPT=1 # Install NICE DCV (from https://download.nice-dcv.com) RUN wget https://d1uj6qtbmh3dt5.cloudfront.net/NICE-GPG-KEY RUN gpg --import NICE-GPG-KEY RUN wget https://d1uj6qtbmh3dt5.cloudfront.net/2023.0/Servers/nice-dcv-2023.0-15487-ubuntu2204-x86_64.tgz RUN tar -xvzf nice-dcv-2023.0-15487-ubuntu2204-x86_64.tgz \ && mkdir -p /etc/modprobe.d \ && apt-get install -y /data/workspace/nice-dcv-2023.0-15487-ubuntu2204-x86_64/nice-dcv-server_2023.0.15487-1_amd64.ubuntu2204.deb \ /data/workspace/nice-dcv-2023.0-15487-ubuntu2204-x86_64/nice-dcv-gl_2023.0.1039-1_amd64.ubuntu2204.deb \ /data/workspace/nice-dcv-2023.0-15487-ubuntu2204-x86_64/nice-dcv-gltest_2023.0.318-1_amd64.ubuntu2204.deb # Install xterm for direct console access RUN apt update && apt-get install -y xterm

• Build a new docker image named o3de_robot_vacuum_simulation_robomaker

.docker build -t o3de_robot_vacuum_simulation_robomaker -f $WORKSPACE/Dockerfile.simulation.robomaker $WORKSPACE

• Create a repository in the Amazon ECR for the new simulation docker image. aws ecr create-repository --repository-name o3de-robot-vacuum-simulation

The command will return a JSON result for the new repository:

{ "repository": { "repositoryArn": "arn:aws:ecr:us-west-2:000000000000:repository/o3de-robot-vacuum-simulation", "registryId": "000000000000", "repositoryName": "o3de-robot-vacuum-simulation", "repositoryUri": "000000000000.dkr.ecr.us-west-2.amazonaws.com/o3de-robot-vacuum-simulation", "createdAt": 1674082385.0, "imageTagMutability": "MUTABLE", "imageScanningConfiguration": { "scanOnPush": false }, "encryptionConfiguration": { "encryptionType": "AES256" } } }

• Set an environment variable for the simulation Amazon ECR repository with the repositoryUri

results from the JSON output.export ECR_SIMULATION_URI=000000000000.dkr.ecr.us-west-2.amazonaws.com/o3de-robot-vacuum-simulation

• Log into Amazon ECR from your local Docker runtime. aws ecr get-login-password | docker login --username AWS --password-stdin $ECR_SIMULATION_URI

• Tag the docker image o3de_robot_vacuum_simulation_robomaker

with the same repository URI.docker tag o3de_robot_vacuum_simulation_robomaker $ECR_SIMULATION_URI

• Push the new simulation docker image to Amazon ECR. docker push $ECR_SIMULATION_URI

Note: Due to the size of the container image (~20 GB) it may take some time to complete and upload depending on your internet connectivity. - Extend the navigation docker image above to enable NICE DCV by creating a file $WORKSPACE/Dockerfile.navstack.robomaker

and copying the following content into it.FROM o3de_robot_vacuum_navstack:latest ENV WORKSPACE=/data/workspace WORKDIR $WORKSPACE ENV DEBIAN_FRONTEND=noninteractive # Install NICE DCV (from https://download.nice-dcv.com) RUN wget https://d1uj6qtbmh3dt5.cloudfront.net/NICE-GPG-KEY RUN gpg --import NICE-GPG-KEY RUN wget https://d1uj6qtbmh3dt5.cloudfront.net/2023.0/Servers/nice-dcv-2023.0-15487-ubuntu2204-x86_64.tgz RUN tar -xvzf nice-dcv-2023.0-15487-ubuntu2204-x86_64.tgz \ && mkdir -p /etc/modprobe.d \ && apt-get install -y /data/workspace/nice-dcv-2023.0-15487-ubuntu2204-x86_64/nice-dcv-server_2023.0.15487-1_amd64.ubuntu2204.deb \ /da