Error detection

The testing and validation of embedded software systems is a heavy burden for software companies. Countless hours and large amounts of money are used on carrying out one test after another – with no guarantee that this will ensure a perfect product. An alliance between CISS and IAR Systems aims to rectify this.

On the screen there is complex arrow diagram showing a number of boxes connected with arrows in an ingenious system which a lay person would have difficulty in making head or tail of. But for the trained eye it is a simple and easy to understand overview of a piece of software. The program can even translate the diagram to program code making the software designer’s day a good deal easier. In reality, however, it is not until the program is finished that the truly difficult work can begin.

Mathematically impossible

”With this model-based system it has become much easier to design software, but when the program is finished we must begin testing to ensure that it is working as it should – and that it is robust and reliable. It is here where things become very difficult and demanding on resources,” explains Ulrik Larsen, PhD Student who, for almost two years, has been working on making this test procedure easier.

It is true that the programs can be tested on the model, but it still has to be done manually and when the program is embedded in a chip, the whole testing procedure starts from the beginning again.

“A myriad of new error-possibilities appear when a piece of software is laid down in a chip. A number of other programs are actually also present on the same chip and the interaction between these explodes the number of possible states which the system is capable of taking on. It is therefore mathematically impossible in many cases to calculate and test all of the system’s states,” explains Ulrik Larsen.

This is the basic reason why he, together with IAR Systems, is working on automating and simplifying these test procedures.

CISS understands the real world 

At IAR Systems in Århus, Denmark, they know that there is a very big market waiting out there – if they can succeed in making the test work less complicated.

“Our customers are very carefully watching the development work we are doing. If we can make testing just 15% easier there will be a lot of money saved,” says Henrik Leerberg, Director, IAR Systems. He has worked together with Aalborg University since 1996 and knows many of the people who are at CISS today. It is for this reason that he had no qualms about entering into a collaboration.

“Aalborg University and now CISS have shown, for many years, that they are very good at tailoring their research work to industry. We are able to make use of their efforts in our daily work and our customers are also involved in the process so that there is a constant outlet open to the real world,” says Henrik Leerberg.

Attack from the rear

The starting-off point for the project is IAR Systems' software design program – VisualSTATE, and the aim is to create automated test methods in this program.

“Our goal is to automate the testing of the model and to make it easier to test the end product. We can do this partly by using a method called Compositional Backwards Reachability. In ordinary language this means that we go backwards through the system. Instead of starting at the beginning and slavishly going through all the states of the program, we begin at the end. From the desired result we work backwards to find out what is necessary to reach this result. We can cut everything else away. Using this method we can limit the number of states and generate a number of test cases which we can also carry out on the final product,” explains Ulrik Larsen.

The art of limiting

“Even this program can have its problems, however. Sometimes the path ramifies to such a degree that it not possible to test all the possibilities. This kind of calculation capacity simply does not exist anywhere in the world – yet. Therefore IAR Systems have to find new methods.

“Even though it certainly doesn’t suit the software designers, we are forced to limit the number of ways in which they can express themselves. The more ways in which they can screw a program together, the more difficult it will be to test afterwards. Therefore a lot of our work concerns finding standards. If the designers are prepared to stick to these standards we will, on the other hand, be able to guarantee that the system can be tested automatically. This is the actual object of the whole exercise,” says Henrik Leerberg.