Embedded testing and debugging with state machines

Information: YAKINDU Statechart Tools Is Now itemis CREATE

The challenge of embedded testing: When developing embedded systems, the source of failures is often difficult to find. Embedded systems seldom have an easily accessible interface to the outside world. This makes debugging the software difficult, and failures might be related to the hardware as well as to the software. To simplify searching for the error, the YAKINDU Statechart Tools (is now itemis CREATE) plugin YET has been developed – Execution Trace. This tool greatly improves the debugging of generated code by displaying its inner workings in real time in the graphical statechart editor.

When developing embedded systems, hardware and software for one specific product are often created at the same time by different teams. Due to this circumstance, these components also have to be tested in parallel, and the software sometimes cannot be tested from the beginning - due to the missing hardware. Using models and model-driven development to describe the system’s behavior can help to close this gap. Models introduce the needed level of abstraction especially in projects where the coupling to the hardware is necessarily high.

The additional layer of abstraction forces you to separate the logic (“what to do”) from the implementation details on your respective hardware platform (“how to do it”).
Embedded systems are often reactive and stateful by design. Therefore, state machines are a good pattern to model these systems – they describe systems by their different states and the events they can react on.

Of course, no paradigm is perfect, and they cannot guarantee bugfree software on the first try. Testing is still vital to assert an optimal grade of software quality. However, testing is a tedious and time-intensive process, especially testing of embedded systems.

The tedious process of embedded testing

Imagine you are developing an embedded system, switch it on for the first time, and nothing works – you press a button, expecting some reaction, but it doesn’t do anything. An LED should come on, but it does not. Now you start to wonder: Is it a hardware fault? Or is the software buggy? Is the button wired incorrectly or is it just the code that doesn’t do what it’s supposed to do?

Signal paths in an embedded system can often be quite complicated, and finding the reason why they don’t work can be cumbersome and time-consuming. Due to the coupling of hardware and software, you often have to find out first whether you are doing the wrong thing – or doing the right thing wrong.

Embedded testing often requires external measuring equipment to analyze errors and find out whether the hardware is faulty. So you go and get your multimeter, measure the voltage of the button’s pins, then the LED’s … and so forth. How nice would it be if there were a simpler way!

Right now, YAKINDU Statechart Tools (is now itemis CREATE) already covers a big chunk of the embedded software development process. It allows you to do the following:

• Modeling reactive and stateful systems as state machines
• Drawing state diagrams and validating them on the fly
• Simulating state machines interactively and checking whether they are doing the right thing
• Writing tests for state machines
• Generating source code in your target language
• Generating test code in the same language to test the generated code

Wouldn’t it be great if we could reuse the graphical statechart model in the integration testing phase, staying on its higher abstraction level? And if we have encountered any bugs – what about debugging support?

These tasks, even though they are clearly separate, do have some similarities. In both cases, you do two things:

• You manipulate the system in some way.
• You observe the system’s reaction.

While testing is a more formal process with clearly defined actions and expected reactions, debugging is basically the same thing, just slower and interactive.

Right now, YAKINDU Statechart Tools (is now itemis CREATE) offers both these methods only on the model itself – you can test the statechart with SCTUnit, and you can “debug” it by interactively simulating an interpreted instance of the statechart.

However, once you have generated your source code, it is outside of what you can control with YAKINDU Statechart Tools (is now itemis CREATE). And especially after you compiled the code and flashed it onto your hardware, it is completely disconnected from your model.

YET another testing tool...

This is the gap that YET (Execution Trace) closes and which makes YAKINDU Statechart Tools (is now itemis CREATE) your best friend in development and testing of embedded systems. YET traces the execution of your software automatically, allowing you to take a look at the inner workings.

This is done by transmitting these traces (small snippets of information about the current state of the software) in a defined format from the generated code to a running instance of YAKINDU Statechart Tools (is now itemis CREATE), which interprets the traces in their proper order, calculates the current state of the state machine and displays it in the usual graphical editor. The transmission of traces can either be live via a network connection or post-mortem via a file. This way you can replay software executions that went wrong after they happened. So you can either:

• Interact with a currently running system.
• Replay software executions that have happened in the past.

To visualize the current state of the machine, we reused the simulation interface of YAKINDU Statechart Tools (is now itemis CREATE). In the traditional simulation mode, the simulation interface interacts with an interpreter that interprets the statechart. For implementing YET, we just unplugged the interpreter and replaced it by the tracing output of your system. The UI remains exactly the same, so there is no learning curve. The same is possible with SCTUnit, you can also run your test script against a real hardware target instead of an interpreted statechart.

Additionally, when using a live network connection, YET even offers bi-directionality. This allows you not only to observe the system but also to manipulate it! Using the same interface you are already familiar with from the basic statechart simulation, you can change variable values, raise events and observe the state of the machine directly on the target platform.

There is virtually no setup cost – except you have to use an extended code generator that puts tracing output into the generated code. Due to the huge variety of embedded systems and operating systems, you just have to provide an implementation of the actual output, for example via UART. The trace protocol is really simple and human-readable. A single trace could look like this:

#22848,blinky.stop,true

What this means is very simple - at a timestamp of 22848 milliseconds, the value stop of the state machine named blinky changed its value to true.
Should something go wrong in your system, you can read the traces and find out what happened, or let YAKINDU Statechart Tools (is now itemis CREATE) re-play the traces and display their effects to you.

A real-world example

See the video below to see the process in action:

Information: YAKINDU Statechart Tools Is Now itemis CREATE

At first, the statechart simulation is started. It has been configured beforehand to listen to a certain UDP port for incoming messages. Until the initial message appears, the simulation is basically in “stand-by mode” – no state is active (just like before the enter method is called in generated code). The generated code has been flashed onto an ESP8266 beforehand, shown in the video.

When it is started, it sends the awaited initial message, activating the first state (Start). The state machine then waits for the event “start”, which is raised via the simulation interface in the video. The connection via UDP is bi-directional and sends this event back to the running code on the ESP8266, which in turn activates the Blink state.

You can observe how the running code continuously transmits its current active state back to YAKINDU Statechart Tools (is now itemis CREATE), where it is displayed in the diagram. In the actual system, the microcontroller always switches the LED on when it is in state On and off when in Off. Also, the variable count is updated by the software, which you can observe in YAKINDU Statechart Tools (is now itemis CREATE).

Additionally to raising events, it is also possible to change variable values on the remote system – in the video, the cycle time is changed to 500 ms from the simulation interface and later on to 2000 ms. You can observe the actual embedded system reacting to these changes live and instantly and blinking the led faster or slower depending on the set cycle time.

In contrast to a usual simulation, this is not executed in the interpreter, but instead directly in the generated code that has been compiled and is running on the actual hardware.

Not only does this new feature simplify your embedded testing and debugging process, it also opens the door to a huge variety of new and exciting possibilities. It would be possible to integrate YAKINDU Statechart Tools (is now itemis CREATE) with trace analysis tools like, for example, Impulse or EB Solys or deploy automated trace analysis detecting any illegal states in your software.

Don’t miss out on these features and get YAKINDU Statechart Tools (is now itemis CREATE) now!

Information: YAKINDU Statechart Tools Is Now itemis CREATE