Create your First App

Overview

This section describes how to create a simple Ra-Ya app.

We are going to create an application that will scan the area and look for a cup. Once he finds the cup, he will navigate and pick the cup.

Sounds exciting? Let's start!

1. Setting up the environment

1.1  Requirements

Python

You need an installed version of Python 3.8.10.

We recommend installing the SDK in a virtual environment with some environment management systems like conda.

Docker

Install Docker and verify that you can execute commands without Sudo:

• Ubuntu 18.04 (steps 1 + 2)
• Ubuntu 20.04 (steps 1 + 2)
• Other distros
• Windows

Windows X Server (relevant only for Windows)

You will need this package in order to visualize data, for example using matplotlib to visualize Lidar radar information or using our images tool in order to display frames from robot cameras.

The first step is to install and configure VcXsrv Windows X Server (Xming also could be used). For that, you will need a package manager tool. In this tutorial we chosechoco.

choco install vcxsrv

Please enter this command as an administrator in PowerShell or Windows Command Prompt. After that, restart your machine.

Then, Open Xlaunch from the start menu and follow the configuration steps:

Save the config file in your Startup folder (which you can find by hitting Windows key + r and running shell:startup).

1.2 RaYa SDK installation

We recommend installing the SDK in a virtual environment with some environment management systems like conda. In that case, you can skip the first command.

pip3 install rayasdk

2. Creating the App

In the terminal, create an empty directory (project directory from now on) and enter it:

mkdir -p ~/raya_projects/helloworld
cd ~/raya_projects/helloworld

If the project directory already exists, remove it or create it with a different name. The project folder must be empty.

2.1. Create a new Ra-Ya project:

Use the Ra-Ya SDK to populate the project folder:

rayasdk init --app-id "helloworld" --app-name "Hello World"

The following directory structure should be created (more information about the project structure here):

.
├── dat
├── doc
├── log
├── res
└── src
│  ├── app.py
├── __main__.py
└── exec_settings.json
└── manifest.json

2.2. Adding "Hello World" code

Open the project folder with your favorite editor, and replace the content of src/app.py  with the following code:

from raya.application_base import RayaApplicationBase
from raya.controllers.navigation_controller import ANG_UNIT, POS_UNIT

CAMERA = 'head_front'
TARGET_OBJECT = 'cup'
MAP = 'unity_apartment'
MODEL = 'apartment_objects'
ARM_NAME = 'right_arm'

class RayaApplication(RayaApplicationBase):

    async def setup(self):
        self.motion = await self.enable_controller('motion')

        self.nav = await self.enable_controller('navigation')
        if not await self.nav.set_map(MAP, wait_localization=True, timeout=3.0):
            self.finish_app()

        self.cameras = await self.enable_controller('cameras')
        await self.cameras.enable_color_camera(CAMERA)

        self.current_detections = []
        self.cv = await self.enable_controller('cv')
        self.detector = await self.cv.enable_model(model='detectors',type='object',
                                                    name=MODEL,
                                                    source=CAMERA,
                                                    model_params={}
                                                    )

        self.grasp = await self.enable_controller('grasping')
        self.arms = await self.enable_controller('arms')

    async def loop(self):
        await self.motion.rotate(angle=360.0, angular_velocity=10.0, ang_unit=ANG_UNIT.DEG, wait=False)
        
        resp = await self.detector.find_objects([TARGET_OBJECT], wait=True, timeout=40.0)
        if resp and self.motion.is_moving():            
            await self.motion.cancel_motion()
        else:
            self.finish_app()

        obj_x = resp[0]['center_point_map'][0]
        obj_y = resp[0]['center_point_map'][1]

        await self.sleep(1.0)

        await self.nav.navigate_close_to_position(x=obj_x, y=obj_y,
                                                pos_unit=POS_UNIT.METERS,
                                                wait=True)

        await self.sleep(1.0)

        await self.grasp.pick_object(detector_model=MODEL, 
                source=CAMERA, object_name=TARGET_OBJECT,
                arms=[ARM_NAME], wait=True)
        
        self.finish_app()

    async def finish(self):
        if self.motion.is_moving():
            await self.motion.cancel_motion()

For deeper information about this example, see the Hello World App page.

3. Run your app

You just wrote your first Ra-Ya app. It's time to run it!

In order to be able to run the application, you need to follow a few steps:

3.1.Open & activate the simulator

rayasdk simulator

After running the command, the simulator should be open and you will be asked to signup. Don't worry, it's a very short and simple process.

Please select the apartment scene.

In addition, you should see an output like this in your terminal:

[bringup-1] Scanning DDS domain 1...
[bringup-1] Serving on http://localhost:8789
[bringup-1] Serving on http://0.0.0.0:8789
[bringup-1] Serving on http://10.0.0.17:8789
[bringup-1] Serving on http://172.17.0.1:8789
[bringup-1] Acquired DDS domain 1

This output means that the simulator environment is ready and a "virtual" robot was created and can be detected.

3.2.Scan + Connect to the device (robot/simulator)

If you are using Mac/Windows, please verify that Docker is running. You just need to search for the application and open it.

The current terminal is being used by the simulator, so please open a new terminal.

In order to be able to run the application, we need first to connect it to the target device (even if it's a real robot or the simulator in your local machine).

• Scan for devices. In order to be able to connect to a device, we first need to scan and find it in your network.
  Shell 
  Shell

  rayasdk scan

  

•  Copy the robot_id from the terminal, and run:
  Shell
  Shell

  rayasdk connect --robot-id={robot_id}

3.3.Run 

Finally, you can run your app!

rayasdk run

At this moment, you should see the robot moving in the simulator window.

We extremely encourage you to visit our Platform Fundamentals section. It covers some fundamental concepts that will help you to understand the Ra-Ya ecosystem in a better way.