Graphical User Interfaces & User-centric Interaction

Graphical User Interfaces
& User-centric Interaction

Introduction

Flexibility, Modularity and Intuition

Our thoughts on future user interfaces are mostly shaped by media and films. Whether you have Star Trek, Minority Report or another motion picture in mind, you can find media-shaped influences in the design of modern user interfaces. Even though interaction via speech and spatial manipulation is getting more and more popular, traditional Graphical User Interfaces (GUIs) are still the way to go and will probably prevail for the time being. What looks good on the silver screen, like Tom Cruise elegantly skimming through crime-databases in a ballet-like fashion, will understandably never be feasible for anyone who ever tried to hold their arms up in front for more than mere minutes.

With technological advances and ever increasing computational power behind the scenes, GUIs rely on established systems like "What you see is what you get" (WYSIWYG) and "Windows, Icons, Menus and Pointer " (WIMP). The mimicry of real world operations is probably the most well-known concept amongst consumers finding virtual representations on their devices’ screens for existing experience of interaction: swiping like flicking pages of a book and two-finger-rotate to turn something, but also introducing new paradigms like the infamous pinch-to-zoom or even more finicky gestures using multiple fingers at once.

In robotics, best practice user interfaces serve to ease the interaction, add intuitiveness and comprehensibility of complex operations. Touch-displays even found their way into manufacturing environments and one can find tablet-PCs and smartphones with different IP-protection classes. We at GESTALT are experts in designing and developing graphical user interfaces for robotics. We target user-centric design, carry out user studies and provide interfaces for connecting external services as well as industrial robots from arbitrary manufacturers. Why not send us an email at info@gestalt-robotics.com or give us a call at +49 30 616 515 60?

Dashboard for a multi-robot control scenario
Dashboard for a multi-robot control scenario, Design: Telekom Design Gallery, Implementation and Integration: GESTALT Robotics

Tech Outline

UI or UX? GUI or NUI? Unimodal or multimodal? User-centric!

The role model for the design of modern human-machine-communication is the natural human-human-communication, which for the most part is multimodal via speech (auditory) and via gestures (visual or haptic). The objective of using natural communications is to increase the acceptance of the technical system by the user as well as a to reduce training durations and operating cycles. Based on the paradigm of direct interaction, principles of action and cognitive processes are addressed, which transfer the interaction with real objects as closely as possible to the interaction with virtual objects within the user interfaces. In general, one seeks to get rid of technical aids, e.g. a special glove or markers for gestural interaction, in order to avoid negative effects on the guiding criteria of effectiveness, efficiency and user satisfaction. In recent years, the term Natural User Interface (NUI) was established in contrast to the classic term GUI.

Whereas classic GUIs rely on input devices such as keyboard and mouse, NUIs also consider novel types and methods of interaction such as voice input, touch and multi-touch operations. On a closer look at voice control, it is quite useful for service robotics in domestic environments but suffers from a lack of robustness within noisy industrial environments. Due to this issue, in the field of industrial robotics, voice control is currently only used for teleoperation purposes and for the close collaboration of humans and robots in laboratory environments.

Multimodality is the usage of multiple communication channels simultaneously or consecutively. This principle plays an important role in the design of modern control systems. Numerous scientific publications have shown that the multimodal design of industrial control and programming systems increases user productivity and acceptance. The prerequisite is an application and user-specific design of the systems. Multimodal control systems combine, for example, gestures (finger, hand and touch gestures), speech and various forms of output, such as augmented reality.

This very basic test for gesture-based music control in conjunction with simple visual feedback showcases simplicity and intuitiveness. Whereas some actions could be performed right there on the screen, this example was set to be in the kitchen, to be in control even when the hands would be covered in dough or grime from cooking.

What to choose unimodal or multimodal communication, GUIs or NUIs?

Actually, there is no general response. Instead, it is more about the user-centric design referring to user-oriented interfaces achieving high usability and user experience. Rather, the central feature of the user-centric design process is a goal-oriented, iterative interchange between analysis, design and evaluation processes and the further iterative incorporation of the respective results.


Applications & Use Cases

Human-Machine-Interfaces for Industrial Control

Optimized man-machine interfaces in (partially) automated technical systems support cognitive activities of the user such as planning, programming, diagnostics, monitoring, maintenance and service. Due to their abilities of abstraction, interpolation and adaptation, humans will still be required in most industrial processes within the next decades.

Modern industrial control systems are usually characterized by a high range of functions and complex operating, monitoring and coordination tasks. In contrast, the consumer products are primarily tailored to the needs and capabilities of users and consequently provide only a limited, easy-to-use scope of control functions. However, due to the complexity of industrial control, the user-centric adaptation of the complex operation functions is the main challenge.

The central objective of interaction design for industrial human-machine-systems is to realize an effective and efficient process management of the information exchange between human and technical system. Optimized interaction concepts increase efficiency but also increase the acceptance of the technical system and provide motivating and appealing work activities.


Conclusion

It's all about the context and the user

The possibilities in combining modalities and interaction methods within GUIs seem endless. What makes the difference and should be leading factors are the individual user (group) and the context. A user interface for a fair exhibit or a company's show room will definitely be different from a user interface for robots within the steel industry.

We develop tailored graphical user-interfaces, and evaluation through user studies for all kinds of robotic applications. Get in touch with us to discuss your use cases. You can contact us under info@gestalt-robotics.com or give us a call at +49 30 616 515 60 – we would love to hear from you.