Build headless robotic simulations with AWS Batch

Build headless robotic simulations with AWS Batch

Introduction

Headless robotic simulations with AWS Batch allow robot developers to increase their velocity by running their code through thousands of scenarios and iterating before moving onto physical device testing. The real-world environments and situations a robot can find itself in are nearly endless. What’s worse, it is time consuming and costly to deploy and test every scenario on physical robots. AWS Batch is a service that gives robot developers an easy way to run batch robotics simulation at massive scale with custom control of what compute types to use.

Note that AWS Batch is best used for running headless batch simulations at scale. If you are looking for interactive simulations with a GUI, we recommend AWS RoboMaker simulation.

Overview

In this blog, you will create an AWS Batch compute environment and job to run your containerized robot and simulation applications. This blog takes the Amazon CloudWatch robot monitoring sample originally built with AWS RoboMaker and updates it to show how you can accomplish the task with AWS Batch instead. The sample runs a robot navigation test and sends data to Amazon CloudWatch to monitor the robot’s position and speed.

We will go through the following steps:

• Prepare robot and simulation containers.
• Create a Dockerfile to install docker-compose and the AWS Command Line Interface (AWS CLI).
• Build and push the container image to Amazon Elastic Container Registry (Amazon ECR).
• Create a docker-compose.yaml file and upload it to Amazon Simple Storage Service (Amazon S3).
• Set up permissions for your AWS Batch jobs and robot and simulation applications.
• Create an AWS Batch compute environment, job queue, job definition, and job using the AWS Batch Wizard.
• View the logs in Amazon CloudWatch.

Prerequisites:

The following are requirements to follow along with this blog:

• An AWS account with permissions to AWS Identity and Access Management (AWS IAM), Amazon S3, Amazon ECR, Amazon CloudWatch, and AWS Batch.
• Basic understanding of Docker containers.
• Docker, the AWS CLI, and the VCS Tool installed on your machine.
• How to install the AWS CLI: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
• How to install Docker: https://docs.docker.com/get-docker/
• How to install the VCS Import Tool: sudo pip3 install vcstool

Prepare your robot and simulation containers

This blog takes the Amazon CloudWatch robot monitoring sample originally built with AWS RoboMaker and updates it to show how you can accomplish the task with AWS Batch instead. This section refers to content in a previous blog and documents the required changes so the containers will work with AWS Batch.

• Clone the Amazon CloudWatch robot monitoring sample repository

Note*: In the AWS Robotics sample applications, the code is already structured with ROS workspace directories. Therefore, you don’t need to create a workspace and source code directory. However, for most open-source ROS packages and likely for your code, first create your workspace directory and clone the source code into*<workspace>/src

.git clone https://github.com/aws-robotics/aws-robomaker-sample-application-cloudwatch.git cloudwatchsample && cd cloudwatchsample

• Containerize the robot and simulation applications by following the steps described in Preparing ROS application and simulation containers for AWS RoboMaker with some minimal name changes. Start from Build a docker image from a ROS workspace for AWS RoboMakerstep 2.- In step 3, place the Dockerfile

in thecloudwatchsample

directory. - In step 6 and 7 you may choose to rename your applications to cloudwatch-robot-app

andcloudwatch-sim-app

You may also choose to rename your application tags asbatch-cloudwatch-robot-app

andbatch-cloudwatch-sim-app

respectively. - Under step 1 of Publish docker images to Amazon ECR, ensure you use the samerobotappandsimappvariables as your application tag names from steps 6 and 7.

• In step 3, place the
• Stop when you have pushed your ROS-based robot and simulation docker images to Amazon ECR (just before you get to the section titled: Create and run robot and simulation applications with containers in AWS RoboMaker).

Create a Dockerfile to install docker-compose and the AWS CLI

We will launch both the robot and simulation containers in AWS Batch. This allows you to run both containers at the same time and have them communicate with each other. In order to have this process run at scale, we will have a special Docker container that AWS Batch uses to run the docker-compose file. The Docker container must have the AWS CLI installed to download the docker-compose file from Amazon S3, and also have Docker Compose installed in order to run the docker-compose up

command for creating and starting the robot and simulation containers.

• Inside the cloudwatchsample

folder, create a new folder for storing our AWS Batch Docker files.mkdir batch-docker && cd batch-docker

• Create a new file named Dockerfile

and copy the contents below into the file. This Dockerfile installs Docker Compose and the AWS CLI. This will then allow you to run a command so AWS Batch can copy yourdocker-compose.yaml

object from Amazon S3.

Dockerfile:FROM ubuntu:focal ARG DEBIAN_FRONTEND=noninteractive # Install prerequisites RUN apt-get update && apt-get install -y curl && apt-get install wget RUN apt-get update && apt-get -y install awscli RUN apt -y install amazon-ecr-credential-helper RUN apt-get -y install jq RUN mkdir ~/.docker && echo "{\"credsStore\": \"ecr-login\"}" | jq > ~/.docker/config.json RUN curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose RUN chmod +x /usr/local/bin/docker-compose RUN ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

Build and push the container image to Amazon ECR.

We are now going to use the console to create an Amazon ECR repository and then build and push the Docker image to the repository.

• Create a repository in Amazon ECR. In the Amazon ECR console, from the navigation menu, choose Repositories, Create repository. Keep your repository private and give it a name such asbatch-dockercompose-awscli

. Choose,Create repository. - Select your new repository and choose View push commands. This opens a window that provides you the AWS CLI commands to execute to build and push your container image to Amazon ECR.

Note: If you are running the push commands from a non-linux platform, you will need to update thedocker buildcommand to include*--platform linux/amd64**at the end.*

The full command will be:docker build -t batch-dockercompose-awscli . --platform linux/amd64

Create a docker-compose.yaml file and upload it to Amazon S3

• Create a docker-compose.yaml

file and copy the contents below into the file. The file will run both the robot and simulation application and pass through the proper permissions.version: "3" services: robot: image: <account-number>.dkr.ecr.<region>.amazonaws.com/batch-cloudwatch-robot-app network_mode: host command: bash -c "sudo apt-get -y upgrade && roslaunch cloudwatch_robot rotate.launch" environment: - AWS_CONTAINER_CREDENTIALS_RELATIVE_URI - TURTLEBOT3_MODEL sim: image: <account-number>.dkr.ecr.<region>.amazonaws.com/batch-cloudwatch-sim-app network_mode: host command: roslaunch cloudwatch_simulation bookstore_turtlebot_navigation.launch environment: - AWS_CONTAINER_CREDENTIALS_RELATIVE_URI - TURTLEBOT3_MODEL

• Update the image entries in the docker-compose.yaml

with the appropriate image URI for both the robot and simulation applications from Amazon ECR that you created earlier. You can find the image URIs in the Amazon ECR console next to your repository names. Thedocker-compose.yaml

file setsnetwork_mode

tohost

for both