Software Development, Embedded, Robotics, Automotive, Arduino

How to set up a robocar platform with a remote control unit

At itemis, we are involved in automotive software projects in terms of modeling (domain specific languages for architecture and behavior), tooling (architecture, feature models, implementation, Machine Learning) and concepts/standards (AUTOSAR, Genivi, openADX). At our office in Stuttgart, we are setting up a tangible demonstrator – a robocar platform as a flexible base with an initial showcase of machine learning.

The first version of the car was based on the Sunfounder platform. Our good experiences quickly lead to further ideas and plans and we ran into the limits of the Sunfounder platform. So we decided to set up a new demonstrator platform based on a Traxxas Slash 4x4 Platinum.

The current milestone

The current goal of the project is to get control of the RC Car with a remote control unit (XBOX360 controller) and to capture video frames from an racing track via the SainSmart 5MP camera module with 175 degree panoramic wide angle. The captured images will run through image processing and will serve as an input to an A.I. algorithm of behavioral cloning. The A.I. also receives the steering angle of the RC Car and it will be trained to understand the behaviour of steering on a predefined track. So the RC Car should be able to steer autonomously through the whole track.

The RC Car Traxxas Slash 4x4 Platinum Edition (Model 6804R)

The project is based on the Traxxas Slash 4x4 Platinum Edition (Model 6804R) RC racing truck with the original Traxxas NiMH Power Cells with a capacity of 3000 mAh. Its drive system constists of the Traxxas 3355R Velineon VXL-3s waterproof Electronic Speed Control (ESC) which is used to control the connected Traxxas Velineon brushless DC motor.

Not included was the radio system (transmitter and receiver), but we bought it additionally for the initialization sequence of the ESC and to analyze the necessary PWM duty cycles. It also came with a digital and waterproof Traxxas 2075 servo, which has a maximum rotation angle of 180° and is intended to use for the steering of the RC Car. The following picture shows the out of the box state of the car with the above mentioned important basic components.

RC-Car-Traxxas-Slash-4x4-Platinum-Edition-Model-6804R

Additional components needed to run the RC Car

We also needed some additional components to make the RC Car running for our needs:

  • The control unit is the linux based single board computer Raspberry Pi 3 with an Raspbian Stretch Image flashed on its SD card.

    Raspberry-Pi-3
    [Source: https://www.raspberrypi.org/products/raspberry-pi-3-model-b-plus/]

  • We used two PCA9685 16-Channel 12-Bit PWM Servo Driver from Sunfounder to get control of the ESC and the steering servo. Later on, we explain why we used two of these modules.

    PCA9685-16-Channel-12-Bit-PWM-Servo-Driver
    [Source: https://www.sunfounder.com/pca9685-16-channel-12-bit-pwm-servo-driver.html]

  • A DC-DC module, which converts an DC power input of 6.5 to 40V to an output of 5V and 2A, is also used. It is needed for the input voltage of the PCA9685 modules and is also used to provide 2x5V input for each PCA9685 module.

    Step-Down-DC-DC-Converter-Module
    [Source: https://www.sunfounder.com/step-down-dc-dc-converter-module-for-raspberry-pi.html]

  • Furthermore, we use an original XBOX360 controller for PC purpose with a bluetooth to USB dongle for the manually steering and throttle input.

  • The SainSmart 5MP camera module with 175 degree panoramic wide angle for Raspberry Pi 3 is intended to capture the video frames for the image processing and provides the input for the A.I. algorithm.

    SainSmart-5MP-Camera-Module
    [Source: https://www.sainsmart.com/products/noir-wide-angle-fov160-5-megapixel-camera-module]

  • It was also necessary to have an portable power source, so the car can drive freely and because of that we bought an Anker Power Core with a capacity of 13000 mAh.

  • It is also recommended to have wireless access to the Raspberry Pi 3. For example, we use a separate router from TP-Link and we are able to get remote access to the Raspberry Pi 3 via VNC Viewer and Server application. It is easy to install and there are several online tutorials. This is also very helpful when going to other locations for a Robocar race: We do not have to rely / configure any other Wifi connections and we can rely on the router giving out static addresses for our devices.

Assembling the components

We assembled and connected all of the components like it is shown in the following picture:

Caution:

Plug in the Anker Power Core and the Traxxas Power Cell after assembling the components. Assembling while there is a power source connected can cause damage to the components or maybe do harm to a person.


Assembling-the-Components

 

At this point we could power up the whole system. First power up the Raspberry Pi 3 with the Anker Power Core and let the system boot up. Afterwards, you can plug in the Traxxas Power Cell and press the EZ Button on the ESC until the LED is blinking green. If you have not already done the initialization process, you first need to do the following step.

Initialization Process

There are two ways to initialize the ESC to get it controlled with a PWM signal. The initialization sequence needs to register the throttle signal range for driving forwards and backwards. For the initialization sequence it is easier to use the original radio system (receiver and transmitter) which is compatible to the ESC and follow the instruction from the delivered Quick start guide.

  1. Press and hold the Set Button on the transmitter while it is not powered on
  2. Turn on the transmitter until the red LED blinks
  3. Press and hold the Link Button on the Receiver before pressing the EZ-Button on the ESC until both LED, on the transmitter and receiver, flashes green.
  4. Calibration mode is now started: If ESC LED blinks red once, then you need to pull the throttle to full range und hold it until the LED blinks red twice, then you need to push the throttle to full range an hold it until it blinks red again.
  5. If calibration process was successful, the ESC LED blinks green.
  6. Now the Initialization process is finished successfully.

This process is quite difficult to manage without the appropriate equipment. But fortunately, you only need to do this process once, unless you want to change the operation mode of the car.
We are using the training mode with the 50:50 range of the throttle to lower the velocity.

Analysis of the PWM duty cycles with an Arduino Uno

After the initialization process is done, we need to figure out the duty cycle in milliseconds of the PWM signals, which are used to steer the servo and to control the throttle.

You can either choose to use an oscilloscope to observe the signals or use an Arduino Uno and the following sketch for the Arduino IDE. We chose the last option because we had an Arduino Uno in stock.

byte PIN = 3;
int pwm_value;

void setup() {
  pinMode(PIN, INPUT);
  Serial.begin(9600);
}

void loop() {
  pwm_value = pulseIn(PIN, HIGH);
  Serial.println(pwm_value);
}

 

In this case you also need the radio system (receiver and transmitter) to observe the previously sent signals from the transmitter to the receiver.

Here is explained how to connect the Arduino Uno to the transmitter and how to read the signals correctly.

We had the same results for the duty cycles for the transmitter shown in the tutorial on the Url. But later on, in our software, we observed a small delay caused from the used functions of the PCA9685 library. A small inaccuracy can have a remarkable impact on the throttle control. Therefore, it is also recommended to observe the generated PWM duty cycles from the used software. You can use the following modified Arduino sketch to figure out the duty cycles of the PCA9685 module for steer and throttle.

You can also compare the transmitter signal with the generated PWM signal from the PCA9685 module. Just additionally connect the PCA9685 module PWM pin with PWM IN Pin 5 of the Arduino Uno and make a ground connection between PCA9685 module and Arduino Uno.

byte PIN = 3;
byte PIN_Rpi = 5;

int pwm_value, pwm_raspi_value;

void setup() {
  pinMode(PIN, INPUT);
  pinMode(PIN_Rpi, INPUT);
  Serial.begin(9600);
}
void loop() {
  pwm_value = pulseIn(PIN, HIGH);
  Serial.println((String)"PWM signal from Traxxas Transmitter\t" + pwm_value);
  pwm_raspi_value = pulseIn(PIN_Rpi, HIGH);
  Serial.println((String)"PWM signal from pca9685 module\t" + pwm_raspi_value);
}

Setup the Raspbian Image

We are using the Raspbian Stretch image version which is shown in the following terminal output.

Raspbian-Stretch-Image-Version


It has also some additional software installed:

  1. Activated Wifi (several tutorials can be found online)
  2. wiringPi (it was already installed, but it is recommended to check if it is already installed)
  3. wiringPiPca9685 (library for the pca9685 module based on wiringPi)
  4. Joystick API (used for Gaming Device Input)
  5. OpenCV Version 3.2.0 (intended for camera usage and image processing)
  6. Eclipse 3.8 Juno with CDT plugin (IDE for C/C++ code)
  7. VNC Server on Rpi 3 and VNC Viewer on PC (intended for remote access to Rpi3)

After these installations, which can take some time, the image is ready to use. Otherwise you can use the attached image and flash it on your SD card. Make sure you have at least a 16GB SD card.

Import the project to the Eclipse IDE

The project can be imported to the Eclipse IDE by the following steps:

  1. Open the Eclipse IDE and press → New → Makefile Project with Existing Code.

    Import-Project-to-Eclipse-IDE-Step-1

  2. Type in the project name and search for the path with the source code. Then press finish.

    Import-Project-to-Eclipse-IDE-Step-2

Explanation of the project content

Inside the project folder there are several C header and source files. There is also a CMakeLists.txt file which gives the instructions to build the project, to generate the executable and object files and to link the libraries. The project file contains project specific instructions which are read when importing the source code to Eclipse IDE. The images folder will contain the captured images for image processing and as an input path for the A.I. algorithm.

Explanation of the general function of the sourcefiles:
(For more specific explanations look up the source code and the comments)

Project-Folder

  1. CalculateAngle: Contains two functions which are used to calculate and map angle values.

    int calculateSteeringAngle(float mapped_axis_value);

    This function maps a duty cycle value to a servo angle and maps the servo angle to the real steering angle of the Traxxas RC Car.

    float convertReceivedSteeringAngle(int steeringAngle);

    This function converts a received angle (int) from the UDP socket to a duty cycle between [0 .. 100%].
  2. CaptureVideo: Contains two functions which are used to open the camera and to capture and save images from the SainSmart Camera module.

    int initVideo();

    Opens the Camera Module plugged to the Camera Serial Interface via the 4l2 camera driver.

    void *captureVideo();

    This function captures a video stream for further image processing. It is implemented as a pthread function. So the video capturing can be processed parallel to the other tasks, because it takes some time to capture and save images and this would have an impact on the performance of the program.
  3. Joystick: Contains three functions which opens, closes and reads the XBOX360 input events via the Joystick API

    void Joystick_Init();

    Opens the Joystick device from specified path in filesystem and provides read access in non-blocking mode.

    int read_joystick_event(struct js_event *jse);

    This function reads in a joystick event and checks its length. An event has to consist of 8 bytes. The content of an event is described by the js_event struct and is defined in the joystick.h file. The input event is saved to a struct js_event which is passed as an argument to this function. The function returns the number of bytes which are read from joystick input path /dev/input/js0.

    void close_joystick();

    Closes the joystick input device on exit of the program.
  4. Motor: This file contains seven functions, which are responsible for the value mapping for the servo and throttle and the conversion from the value mapping to a pwm duty cycle.

    float map_axis_servo(int axis_value);

    This function maps the left stick of the xbox 360 controller to a value between [-0.5 .. 0.5]. This represents the duty cycle for left and right steering range in percent considering the value of 0.5 being the neutral position. So either there is an addition or difference of [0 .. 0.5] to the neutral value. Furthermore, it is possible to change the first axis value on which the axis should react with steering and you can also increase and decrease the possible maximum steering angle with the macros described in motor.h file.

    void steerVehicle(int fd, short value);

    This function receives the mapped steering axis value from the previous function as a float value and maps this value to a duty cycle value in ms. After calculating the amount of necessary ticks for a 12-bit channel resolution, the tick value will be written on the I2C1 bus with the PCA9685 module (slave address 0x41) connected and the predefined channel base 0 for the servo motor.

    It also uses the int calculateSteeringAngle(float mapped_axis_value) function to calculate the current steering angle for the A.I. process. It is necessary to make a read/write protection for this global resource, because one or more threads need to have an access to this resource.

    void steerVehicleAutonomous(int fd);

    This function is quite similar to the previous function, but is only used in the state of autonomous driving. It is also responsible for steering, but it does not receive its input from the joystick device but from an UDP Socket which is connected to the A.I. process. There is an additional global resource which contains the steering angle received by the UDP Socket and therefore it also needs to be protected because there is an access to this resource from one or more other processes.

    float map_axis_throttle_forwards(int axis_value);

    This function is used to map the input from the RT axis of the xbox360 controller to a float value between [0.57 .. 1.00]. The lower value is the neutral position of the throttle and this state remains until the axis value reaches the value 0, which is reached when the button is half-pressed. The maximum upper range is defined in the motor.h file by the macro THROTTLE_RANGE_FORWARDS, which divides the 100% of the forward throttle value by the factor of the macro value. So a macro value of 20 results in a maximum upper range of 5% added to the neutral_offset, which in this case is the lowest possible value for the car to move forwards. But it is still around 6mph.

    void driveVehicle_forwards(int fd, short value);

    This function receives the mapped throttle axis value from the previous function as a float value and maps this value to a duty cycle value in ms. After calculating the amount of necessary ticks for a 12-bit channel resolution, the tick value will be written on the I2C1 bus with the PCA9685 module (slave address 0x40) connected and the predefined channel base 4 for the ESC.

    float map_axis_throttle_backwards(int axis_value);

    This function is quite similar to the float map_axis_throttle_forwards(int axis_value) function. The differences are, that it uses the LT axis of the xbox360 controller and a float value between [0.00 .. 0.56]. The upper value is the neutral position of the throttle and this state remains until the axis value reaches the value 0, which is reached when the button is half-pressed. The minimum range is also defined in the motor.h file by the macro THROTTLE_RANGE_BACKWARDS, which has the same function like the macro in the forwards mapping function. In this case the value will be subtracted from the neutral_offset. So subtracting 5% from the neutral_offset will result in the lowest possible value for the car to move backwards.

    void driveVehicle_backwards(int fd, short value);

    Similar to the void driveVehicle_forwards(int fd, short value) function, but it receives the mapped axis value from the float map_axis_throttle_backwards(int axis_value) function.
  5. mySocket: This file contains two functions. Firstly to initialize the socket and secondly to listen on the local loopback address and a port for UDP Datagrams.

    void init_socket();

    This function opens a socket for UDP/IP communication with UDP Datagrams.

    void *receiveAngle();

    This function calls void init_socket() to open the socket. It also configures and binds the previously opened socket. It is implemented as a thread, so it can do the task of listening to the UDP Port without interrupting the whole process. This thread will only work, when the autonomous driving state is active. Otherwise the thread will be cancelled. It is also necessary to protect two global resources for read/write access. On the one hand it is the button state which can change between autonomous driving state and manual state. On the other hand it protects the received angle from the UDP Socket for the steering function used in the autonomous state.
  6. PWM: This file contains three functions, which are used to initialize the two PCA9685 modules and to generate an appropriate PWM signal.

    int calcTicks(float impulseMs, int hertz, int on_off);

    This function transforms a duty cycle in ms to an aquivalent tick value of 12-bit resolution with a given frequency for the PCA9685 module output to generate a PWM signal.

    float map(float input, float min, float max);

    This mapping function maps the input [0 .. 1] to a maximum and minimum value which are given as an argument and returns a resulting float value, which is used as the first argument for the previously mentioned function.

    int PCA9685_Init(const int i2cAdress, const int pinBase, const int freq);

    This function initializes each PCA9685 module with its own I2C adress, pin base and frequency.
  7. timeMeasurement: These functions are only used to analyze time consumption for processes. There is no functional task for the program.
  8. xbox360: This file contains two functions. The first function is a get helper function to have access in a C++ file to a static C variable. The second function is a pthread function and is intended to work parallel with the other thread based processes mentioned in previous files. It also represents a state machine for xbox360 controller input events and a differentiation between manual and autonomous state.

    int getControlState();

    Simple get function similar to the ones used in object oriented programming languages. Returns the value of a global static variable which needs to be shared between two threads.

    void *read_Xbox360_Event();

    This function calls the PCA9685_Init( ) function twice to initialize the two PCA9685 modules. Afterwards, it reads in all events from the joystick event buffer and counts the amount of events. These events will be processed after checking the state of the process (either manual or autonomous). In manual mode, there is an differentiation between the input events of steering and throttle forwards and throttle backwards. By entering the autonomous mode for the first time, a thread is started which opens a UDP socket to receive input instructions for steering from an A.I. algorithm. The throttle values are still coming from the xbox360 controller. On exitting the autonomous state, the UDP_receive thread will be cancelled to lower cpu consumption.

Functional project overview

In my opinion it is also a good thing to have an overview of dependencies and the workflow of a project. Therefore I made a process overview and I will add some explanations to the following state charts.

Main-Thread

Statechart-Main-Thread

The main thread initializes the joystick device, the thread resources for synchronization and starts two threads, which will do there predefined tasks parallel. It also waits for the exit of the two threads or an exit signal from a terminal. In this case On Program Exit will be executed.

Task: Thread Video_Capture

This thread is responsible to capture images for a video stream and also reads a steering angle from a global variable and writes its value to the filename of the current image of the video stream. This read process needs to be read/write protected via mutex to the access of the Thread Xbox_Control. Afterwards it saves the image to the predefined path and shows the current image on a screen.

With waitkey(15) it is almost a resolution of 60fps because it captures and holds an image for 15ms, which results in more than 60 frames per second. But in reality, it has not the 60 frames per second, because saving and displaying the image needs time for processing with around 100ms. So an idea is to separate the capturing from the saving and displaying task and implement it into an additional thread.

Statechart-Task-Thread-Video_Capture

Task: Thread Xbox_Control

This thread reads in joystick events from the XBOX360 controller and counts the number of events. Afterwards it processes the events and checks for the state of the process, if its either manual or autonomous state. If there is an Button A press, it toggles the controlState variable.

This write process needs to be read/write protected via mutex to the access of the Thread UDP_receive. A condition state checks the current process state and then continues processing the state machine of the current process state. In the end it sleeps 10ms to reduce cpu consumption.

Statechart-Task-Thread-Xbox_Control

State Machine Manual Mode

In manual mode the XBOX360 controller input is used to control the RC Car. It checks for input events concerning the steering or throttle of the RC Car. It also differentiates between axis or non-axis event type, because the event numbers are assigned multiple times to either a button or an axis event.

In our case, there are only axis events used to control the RC car. Therefore, the event number differentiates between each event and executes the associated instruction. In the case of steering, a write process on a global variable occurs and needs to be read/write protected via mutex to the access of the Thread Video_Capture.

Statechart-State-Machine-Manual-Mode

State Machine Autonomous Mode

The autonomous mode reacts on a change of the process state through a toggle flag. This flag is used to instantiate the Thread UDP_receive only once at the beginning, which is responsible for the autonomous steering through the A.I. process. This thread initializes the UDP socket and afterwards it starts with its predefined task. This task will be discussed in the following section.

Apart from the new Thread started at the beginning, the autonomous state machine is quite similar. The throttle control is identical to the manual mode. Only the steering will be processed by the Thread UDP_receive.

Statechart-State-Machine-Autonomous-Mode

Task: Thread UDP_receive

This Thread needs to check the process state via the getControlState( ) function. This read process needs to be read/write protected via mutex to the access of the Thread Xbox_Control.

If the state is on autonomous mode, the thread waits for an UDP datagram on the previously opened socket. Afterwards, it checks whether a correct UDP datagram has been received and if so, it converts this angle to an integer value and writes it on a local variable. This read process from the UDP buffer recvbuff needs to be read/write protected via mutex to the access of the A.I. process.

But at the moment it is only implemented as a dummy variable and has no synchronization effect, because the A.I. process is not finished. The same applies for the steerVehicleAutonomous(fd) and the synchronization with the A.I. process. But the global variable shared with the Video_Capture Thread needs to be read/write protected via mutex. Afterwards it sleeps for 1ms to reduce cpu consumption.

Statechart-Task-Thread-UDP_receive

On Program Exit

If there is an exit command from a terminal, the program stopped by the eclipse GUI or there are errors which calls the exit( ) function, the main thread is cancelled and does the following instruction to free memory areas used for thread purpose. It also closes the joystick input device.

Statechart-On-Program-Exit

Final assembly of the RC Car

Finally, after explaining the software, we can look at the current state of the assembled car.

Final-Assembly-of-the-RC-Car

We hope this blog post gives a good overview on the required steps to set up a car like ours. If you discovered a problem or inconsistencies, please do not hesitate to leave a comment.

Troubleshooting

Previously I mentioned that I will explain, why we used two PCA9685 modules. When we were using only one module and only gave steering inputs from the XBOX360 controller, everything seemed to work. But after we gave the first throttle input to the ESC, a following steering input leaded to an unpredictable behaviour of the throttle.

After some time analyzing this problem, we were still clueless what the cause of this problem was. We also were looking forward to an event and had a short time window, so we decided to solve the problem by separating the servo and the ESC control.

We also decided to deal with this issue again at a later point, because the priority to solve this issue was not high. But if someone has an explanation based on our source code, please feel free to share with us.

Further plans

We already have a number of ideas for further improvements and extensions to the car:

  • Add a LIDAR sensor
  • Adding an IMU
  • Replacing the Raspberry with a Nvidia Xavier
  • Implementing some of the functionality according to AUTOSAR (based on the Minnowboard)
    
About Andrè Stolpp

Andre Stolpp is a working student at itemis AG in Stuttgart. His area of study is Master of Science as a mechatronics engineer. He is responsible for the assembly of the mechanics and electronics related parts of the Robocar.