UI development / implementation in unreal engine

the Auqua project

The AuQuA project, a software developed by the University of RWTH Aachen in collaboration with weltfern. This allows unqualified and more qualified workers to assemble objects correctly using AR projections. A distinction is made between the assembler and the director.

 Share innovation

AI & AR ENHANCED TRAINING

The development of an intelligent assembly support system that automatically recognizes assembly steps based on artificial intelligence and generates corresponding assembly instructions. In addition, instructions can be projected onto workpieces or work surfaces. However, human interaction is indispensable for the system to function.

The instructions are directed via a projection with the help of a beamer. A real-time display from the work surface allows flexible interventions by the director.

An AI-driven 'object occlusion' makes it possible to mask hands as well as components. Control via a tablet makes it possible to start applications and select scenarios. These scenarios can be guided by text explanations and / or picture explanations.

PROBLEM & SOLUTION

ENHANCED BY AR & AI

To assemble an object correctly as an assembler, the object to be assembled is placed on the work surface. Then the assembler follows the instructions on the work surface to attach the object according to the instructions. The object is completely assembled when the corresponding instruction appears on the work surface

TRANSFER OF KNOWLEDGE

IDEALTIME HAPPY & BADFLOW

As a director, one creates the instructions using the AuQuA software and gives instructions to the assembler on how to properly assemble the selected object. In addition, it is possible to check how well the assembler works with the program and how precisely the instructions are followed. In the event of incorrect work steps, the director can intervene directly and, for example, change the instructions or provide assistance. The time for the respective work step can be tracked exactly Happy and Badflow can be determined perfectly - which gives the project another research purpose.

The Problem

TIME FACTOR IN ASSEMBLY

Assembly always follows the same principle - machines or complex devices are assembled from individual parts in assembly steps. Important, though, on a time axis is the order in which each step is placed. Time management can be controlled via the interface using the time control buttons. If the specified assembly time is exceeded, visual feedback is given - this signals the Badflow.

modular interface for assembly processes

CREATE ASSEMBLIES

The modular interface developed by weltfern offers the engineer the possibility to realize and instruct individual assemblies. Either directly on the component step by step with the auqua interface and direct projection or on the basis of assembly documents. The application thus offers a completely configurable assembly support system.

direct instructions on components

REALTIME PROJECTION

Real-time projection using a standard consumer projector directly onto the work surface enables installation even for untrained personnel. Individual work parts can be highlighted and correct assemblies rewarded. The instructor can directly influence the projected content via the interface and intervene in case of doubt. Error notifications, in case of selection of wrong parts, audiovisual signals, etc. allow an optimal exchange.

Image source: WZL - RWTH Aachen

effectiveness research

Saving time with less qualified workers?
Research for future improvement!

Image source: WZL - RWTH Aachen

direct instructions on components

CONTINUOUS IMPROVEMENT

An intelligent assembly support system should be subjected to close effectiveness testing despite artificial intelligence. This is the only way to further improve this system. This means that all assembly instructions are put to the test. Depending on the component, the time and complexity of the assembly steps should be varied precisely. Successes and failures should be closely observed and efforts minimized.

location-independent assembly of components

INTERNATIONAL DEPLOYMENT

Thanks to more than 20 research partners, the AuQuA project is based on the idea of international use. In addition to the complex assemblies, there are now cultural and origin-related differences that must be taken into account for successful assembly. Thanks to the modular structure of the interface, not only languages but also signal routing can be changed and adapted. This is the only way to make the project suitable for industry and expand assembly abroad.

AI ENHANCEMENT

Image source: WZL - RWTH Aachen

Work in progress is shown here the AI development of object-oriented recognition and hand tracking from the RWTH. Through object occlusion a separation and masking of the hands is possible!