Executable HMI Specifications with Flash and Yakindu Statecharts

Human machine interfaces (HMI) are inherently interactive and dynamical. An important goal of HMI engineering is to optimize the interaction between the human being and the machine. The ergonomics and efficiency of an HMI is mainly driven by the quality of its design. This makes the HMI design a central part of HMI specifications. Due to the HMIs dynamical nature the design must cover the interfaces functional behavior as well as its form and appearance. From this point of view executable specifications would be desirable.

Statecharts are an executable formalism that can be used to specify the reactive behavior of HMIs at an adequate level of abstraction. Using statechart the function can be specified very well, but statecharts do not cover the design aspect of form at all. So they can't provide any concrete visual experience. This part could be covered by different approaches. A prominent one is to use Adobe's Flash technology for prototyping visual designs. Flash allows rich visual effects including animations etc. and is very popular in the 'design oriented' disciplines. So putting both together seems to be a promising approach - and that is what we did. We integrated our Yakindu Statechart Tools with Flash applications and set up two showcases. The first shows a electric window lifter and the second an interactive driver information panel.

Check out the the screencast to get an impression how the solution works.

Each showcase consists of a Flash application, a statechart, and a simple textual mapping model that describes how Flash and statechart events and data have to be mapped towards each other.Technically the mapping model is used to generate a flash component that has to be embedded into the  Flash application. This component registers the appropriate event handlers and handles the communication issues with the Yakindu statechart simulator that executes the statechart. When working with the Flash based design active states, transitions, and value changes are also visualized on the model level.

The whole tooling is based on the Eclipse platform. The Adobe Flash Builder is based on Eclipse as well as the Yakindu Statechart Tools and the mapping model including the model parser and the rich editor including content assist for references to Flash components is based on Eclipse Xtext technology.

This approach brings up some interesting questions. First of all - how can the formal statechart model help when it comes to the concrete implementation of the HMI? Can it be used to generate code? What are the preconditions and must the statechart be refined for that purpose? I think i will cover some of these questions in following posts.

Another interesting point is of course that this approach also can be useful for other use cases than HMI specifications. It can be used whenever a visual rich animated frontend will be required in simulation contexts. And of course Flash applications may be integral part of  concrete System implementations...