Publications

Abstract

In this paper, we present our framework for Augmented and Mixed Reality touristic applications, which we are currently developing with the city of Saarlouis (Germany). It aims to provide tourists and visitors with a new augmented experience to discover the unique history of the city. Whereas most Augmented and Mixed Reality touristic applications are designed for a single user without any human city guide, our app, called SaAR-Louis, offers a more immersive touristic experience as it displays holographic information via Aug-mented and Mixed Reality on HoloLens/iPad in addition to the explanations of the tour guide. Moreover, thanks to geolocation and 3D scanning of the sur-roundings, our application provides tourists with the unique experience of dis-covering 3D reconstructed ancient/old buildings that no longer exist and offers multimodal interaction with a 3D Avatar that gives insights about the historic buildings. In the following, we will explain the app architecture, the pipeline of creating the 3D virtual revival of the city of Saarlouis, and the challenges be-hind designing, conceiving, implementing, and deploying the SaAR-Louis app.

Abstract

In Germany, today’s transmission grid topology often still corresponds to the technical state of the art of one-way energy flow from central power plants to consumers. The energy transition with the phase-out of nuclear power plants by 2022 and the expansion of renewable energies on the distribution grid level confronts low-voltage grids with challenges such as unmonitored overloads or voltage range violations. In addition, the volatility of loads and renewables makes it challenging to predict future grid conditions, plan preventive measures, and apply them when needed. This chapter presents the integration of renewable energies using photovoltaic systems as an example. The integration steps include the necessary information technology steps, such as collecting data from the photovoltaic system, their forwarding, preprocessing, and storage in a database, and their further processing by other services. As services, we will showcase (a) AI-based prediction methods that predict the power generation of a photovoltaic system and (b) downstream services that use these predictions for other purposes. Downstream services could be, e.g., a power grid calculation to predict future grid conditions or to provide price information for the electricity at a charging station.

Abstract

The municipal utility supervises all aspects of the city's public utili-ties, e.g., the power grid. For example, power grid maintenance requires the technician to know where underground cables are installed while having their hands free for maintenance work. Using an immersive headset such as Ho-loLens provides a technician with these capabilities. This contribution presents DENKI (Data, Energy, Network, Knowledge, Interaction). This HoloLens ap-plication supports technicians of the municipal utility Stadtwerke Saarlouis GmbH with a detailed overview of the equipment, its utilization, and possibili-ties to investigate malfunctions. We will exemplify some app functions.

Abstract

The German energy transition confronts the operators of low-voltage grids with new challenges. Local energy producers or large consumers, like, e.g., solar panels, heat pumps, and e-mobility lead to unexpected grid behavior. Be-cause current grids are only sparsely monitored, local unmonitored overloads or violations of the voltage range are possible. To overcome these difficulties a smart monitoring and prediction system is needed. The system must handle dif-ferent data sources fast and efficiently, so the operators can react to local grid problems. This is solved by using a streaming service to aggregate the data effi-ciently. Then, the implemented data pipeline is used to train AI-based models to interpolate the unmeasured parts of the grid. These models consider both meas-ured data and predictions, like load profiles and photovoltaic forecasts. Since the grid is not fully observed, a data generator that physically simulates detailed grid scenarios is used to generate large sets of training data. Finally, an interactive GUI is implemented to visualize the data monitoring and predictions in the con-text of the grid and thus strengthen the user’s trust in the system. The presented assistance system is developed in close cooperation with energy experts and grid operators.

Abstract

The dynamic production of green energy requires capable mediate storage solutions, often in the form of large batteries. Here, a Vanadium Redox Flow Battery is used to collect solar panels’ energy and, later, power electric vehicles. Since the balance of the chemical liquids inside the battery can change over multiple loading cycles, a system for predictive monitoring is needed. This paper presents the project “hILDe - Novel, cost-effective and highly accurate indication of imbalance and state of charge of vanadium redox flow batteries using AI-assisted detection of specific colors, which features an absorbance sensor for chemical liquids and an AI-empowered monitoring system to inter-pret and predict sensory data. The current progress in our lab scenario suggests that the deployment of our sensor and monitoring system enables an accurate and cost-efficient imbalance and state of charge sensor for Vanadium Redox Flow Batteries. In the future, the system will be tested in full-sized batteries to verify its scalability and commercial potential.

Abstract

Purpose : Cornea guttata (CG) prevalence post keratoplasty varies from 15 to 18%, with 1 to 2% of the cases presenting with significant negative outcomes. The purpose of this research project is to create a program based on artificial intelligence (AI) that helps with the detection of CG in the donor corneas (DC) in the eye bank.

Abstract

In recent years, more and more AI models and algorithms get used in previously uncharted domains. The medical domain is one of them and already shows a significant usage of AI methods, for example computer vision algorithms for the analysis of medical imagery. The field of ophthalmology studies medical conditions relating to the eye. One of those conditions, Cornea guttata, can be identified by analysing post mortem microscope images of the donor’s cornea endothelium, which needs to be done manually by a skilled professional. To help facilitate this analysis, this paper proposes a hybrid Decision Support System that combines computer vision methods and AI classifiers to guide the decision of the clinicians. By conducting a UX-driven study with professionals from an eye bank, we show that our Decision Support System is able to help users with the classification of Cornea guttata in microscope images.

Abstract

Eine Kernaufgabe von Verteilnetzbetreiber ist die Gewährleistung einer sicheren Stromversorgung. Die zunehmende Volatilität der Stromeinspeisungen (Solar, Wind) stellen Netzbretreiber vor neuen Herausforderungen. Die Autoren zeigen erste Lösungen einer KI-basierten Netzsimulation und eines KI-unterstützten Netzbetriebs.

Abstract

In Deutschland entspricht die heutige Übertragungsnetztopologie häufig noch dem technischen Stand des einseitigen Energieflusses von zentralen Kraftwerken zu den Verbrauchern. Die Energiewende mit dem Ausstieg aus den Kernkraftwerken bis 2022 und dem Ausbau der erneuerbaren Energien auf Verteilnetzebene konfrontiert Niederspannungsnetze mit Herausforderungen wie z. B. unüberwachten Überlastungen oder Verletzungen des Spannungsbereichs.

Abstract

Cornea guttata may not be recognized in the eye bank and recent studies have displayed that guttae are transplanted in about 15% of cases in varying severities. The purpose of this study was to establish semiquantitative criteria for the detection of cornea guttata in donor corneas in the eye bank. In this retrospective cohort study, preoperative endothelial pictures of donor corneas were collected and classified according to the post-penetrating keratoplasty cornea guttata grade into three distinct groups: group 1 consists of healthy corneas with no guttae (guttata grade 0); group 2 constitutes corneas with mild asymptomatic cornea guttata (guttata grade +); and group 3 comprises corneas with advanced widespread cornea guttata (guttata grade ++/+++/++++). The preoperative pictures of each group were then individually analyzed using the following five semiquantitative criteria: The number and the area of the cell-depleted surfaces, the presence of less than 50% of the cells having a hexagonal or a circular shape, the presence of cell membrane defects and interruptions, the presence of blebs in the cell membrane, and the presence of groups of cells with a distinct whitish color.

Abstract

In the context of smart grids, the need for forecasts of the power output of small-scale photovoltaic (PV) arrays increases as control processes such as the management of flexibilities in the distribution grid gain importance. However, there is often only very little knowledge about the PV systems installed: even fundamental system parameters such as panel orientation, the number of panels and their type, or time series data of past PV system performance are usually unknown to the grid operator. In the past, only forecasting models that attempted to account for cause-and-effect chains existed; nowadays, also data-driven methods that attempt to recognize patterns in past behavior are available. Choosing between physics-based or data-driven forecast methods requires knowledge about the typical forecast quality as well as the requirements that each approach entails. In this contribution, the achieved forecast quality for a typical scenario (day-ahead, based on numerical weather predictions [NWP]) is evaluated for one physics-based as well as five different data-driven forecast methods for a year at the same site in south-western Germany. Namely, feed-forward neural networks (FFNN), long short-term memory (LSTM) networks, random forest, bagging and boosting are investigated. Additionally, the forecast quality of the weather forecast is analyzed for key quantities. All evaluated PV forecast methods showed comparable performance; based on concise descriptions of the forecast approaches, advantages and disadvantages of each are discussed. The approaches are viable even though the forecasts regularly differ significantly from the observed behavior; the residual analysis performed offers a qualitative insight into the achievable forecast quality in a typical real-world scenario.

Abstract

This article first examines the role of a Smart Meter as a supplier of measurement data for a Smart Grid. Based on this, future scenarios in which different granularities of smart meter data are available are described. In this context, the necessity for scenario-specific concepts for the integration of the different granularity levels of smart meter data is emphasized. Target-oriented concepts for the use of scenario-specific smart meter data are then presented together with simulative validation. It will be shown how and in what form smart meters offer added value for decentralized distribution grid automation systems. Finally, the limits and advantages of smart meter data will be discussed and summarized. With the aim of conceptualizing a blueprint for the entire European Union, the experiences of the project partners of the SINTEG showcase project DESIGNETZ flow together in this article to provide a cross-competence aggregation of the existing know-how.

Abstract

Fernsehen ist die beliebteste Aktivität der Deutschen, immer noch vor der Nutzung des Radios und Internets. Doch die Art und Weise fernzusehen, hat sich in den letzten Jahren stark verändert. Dieses Buch befasst sich mit der spannenden Thematik der parallelen Benutzungder Medien Fernsehen und Internet bzw. des semantischen Web, mit dem Fokus auf einer Suche und benutzerzentrierten Interaktionen, die intuitiv vorm Fernsehgerät durchgeführt werden können. In dieser überarbeiten Version seiner Dissertation führt der Autor zuerst eine detaillierte Analyse der heutigen technischen Ansätze im Bereich Smart-TV durch. Dabei werden ihre Möglichkeiten in puncto Interaktion, Design und funktionaler Erweiterbarkeiten präsentiert. Zusätzlich wird das System namens Swoozy, welches die Grundlagen des neuen interaktiven semantischen Fernsehens (Semantic TV) setzt, detailliert vorgestellt. Durch die technische Realisierung von Swoozy wurde eine nahtlose Verbindung zwischen (nicht- bzw. linearen) Videos, den Cloud-basierten Diensten und der semantischen Verarbeitung geschaffen. Somit erhalten Zuschauer durch das semantische Fernsehen eine neue Möglichkeit mit Videoinhalten zu interagieren und effizient nach Hintergrundwissen zu suchen.

Abstract

Diese Doktorarbeit befasst sich mit der Thematik der parallelen Benutzung der Medien Fernsehen und Internet bzw. des semantischen Web mit dem Fokus auf Suche und benutzerzentrierter Interaktion, die intuitiv vorm Fernsehgerät durchgeführt werden können. Ziel ist es, Zuschauern eine neue Möglichkeit zu geben, mit Videoinhalten zu interagieren und effizient nach Hintergrundwissen zu suchen. Hierbei wird der Fokus auf die Extraktion und Gewinnung von Semantik aus Videos bzw. Live-TV-Programmen durch automatische Annotationsverfahren und kombinierte Extraktionsverfahren aus den Bereichen Text- und Videoanalyse gesetzt. Daraus entsteht das Konzept des neuen interaktiven semantischen Fernsehens (Semantic TV). In der vorliegenden Arbeit liegen die Schwerpunkte auf der Konzeption dieses Systems, der technischen Realisierung einer Verbindung zwischen (nicht- bzw. linearen) Videos, Cloud-basierten Diensten und semantischer Verarbeitung.

Abstract

We propose an XML-based user-interface markup language for prototyping and designing graphical user interfaces (GUI) in the automotive domain, called autoUI-ML. This markup language will help to accelerate and optimize the development workflow for both, designers and HCI developers. Furthermore the language implicitly constitutes an interface to the semantic based multimodal dialog system SiAM-dp. The autoUI-ML provides a high variability and offers the possibility to integrate an own corporate design. Moreover, the seamless and flexible connection of autoUI-ML to state-of-the-art rendering solutions allows the generation of professional and modern graphical user interfaces.

Abstract

The increasing complexity and flexibility in future production environments leads to huge amounts of data and the difficulty to analyze them in order to react to short-termed events. To tackle this challenge, an electronic performance board and editor was developed that allows users to create dashboards with proper visualizations of production data. This information can be shared among stakeholders and might be displayed on various devices, from personal tablets up to large public screens in the factory hall.

Abstract

The innovations in information and communication technologies change our daily life and the way how to interact with intelligent systems. Powerful computers are becoming smaller and are integrated almost anywhere, even in televisions. Today’s connected television systems are offering a lot of technical functionalities including these, which are currently integrated in smartphones. In this article, we describe an innovative approach in form of an intelligent television system named Swoozy, which enables viewers to discover extended information, such as facts, images, shopping recommendations or video clips about the currently broadcast TV program by using the power of technologies of the Internet and the Semantic Web. Via a gesturebased user interface viewers will get answers to questions they may ask themselves during a movie or TV report directly on their television. These questions are very often related to the name and vita of the featured actor, the place where a scene was filmed, or purchasable books and items about the topic of the report the viewer is watching. Furthermore, a new interaction concept for TVs is proposed using semantic annotations called Grabbables that are displayed on top of the videos and that provide a semantic referencing between the videos’ content and an ontological representation to access Semantic Web Services.

Abstract

🏆Best Paper Award - In this article, we describe an innovative approach to an intelligent television system named Swoozy that enables viewers to discover extended information such as facts, images, shopping recommendations or video clips about the currently broadcasted TV program by using the power of technologies of the Semantic Web (Web 3.0). Via a gesture-based user interface viewers will get answers to questions they may ask themselves during a movie or TV report directly on their television. In most cases, these questions are related to the name and vita of the featured actor, the place where a scene was filmed, or purchasable books and items about the topic of the report the viewer is watching. Furthermore, a new interaction concept for TVs is proposed using semantic annotations called “Grabbables” that are displayed on top of the videos and that provide a semantic referencing between the videos’ content and an ontological representation to access Semantic Web Services.

Abstract

We will show how to build innovative multimodal dialog user interfaces that integrate multiple heterogeneous web services as data sources on the basis of the Ontology-based Dialog Platform (ODP). More specifically, we will describe how to exploit ODP’s well-defined extension points and how generic ODP processing modules can be adopted, in order to support a rapid dialog system engineering process. By means of the latest ODP-based educational information system CIRIUS and the ODP workbench, a set of Eclipse-based editors and tools, we demonstrate step-by-step along the generic multimodal dialog processing chain what has to be done for developing a new multimodal dialog user interface for a specific application domain.

Abstract

This article gives an overview of the basic dialog shell for multimodal interaction that was created as the foundation for the software components developed in the Core Technology Cluster of the THESEUS research program. It explains the basic setup of systems based on the dialog shell for specific use cases. The general processing paradigm for interaction in our framework using semantic interface elements as well as several dialog phenomena handled by the dialog shell are described. Finally, a selection of different demonstrator systems that were built for a range of applications is presented briefly. They offer access to quite varied use case scenarios featuring multimodal input and output, including gestures, multi-touch, spoken input and output, and interactions across mobile and stationary devices.

Abstract

In this work we suggest a new device to instrument everyday objects in the end-user life cycle phase that makes everyday life easier for the elderly. The instrumentation of products, such as food, transforms them into smarter and more intelligent products. The equipped product notifies the user when food is spoiled and—interconnected with other smart products—advises against side effects that occur when food is consumed while a certain medication is ingested. We describe the considerations that were made towards a first prototype from a technical perspective. An interaction model was developed, a requirements analysis performed and several design and development considerations made. An architecture shows the information flow between sensors, actuators, and the Internet of Things. In the end, the prototype of a product sleeve is presented that is easy to handle and intuitive to operate. A feedback study is planned in the near future.

Abstract

Using label technology, a physical object may be employed to build a continuously growing data collection, which can, for instance, be exploited for product quality monitoring and supply chain management. Along the object's life-cycle, queries to such a collection may stay quite similar, e.g., 'get unusual observations'. However, expectations to a 'good' answer may change, as with time different entities will come into contact with the object. This article reports on work in progress concerning a framework for collecting data about things, which aims at decoupling logic employed for interpreting such a collection from processing hardware and using the collection itself for transporting such logic. Main contributions include an approach to hardware-abstraction of processing logic at the object or remote, an app store for retrieving interpretation and presentation logic, and interaction forms with such memories.

Abstract

Using label technology, a physical object may be employed to build a continuously growing data collection, which can, for instance, be exploited for product quality monitoring and supply chain management. Along the object's life-cycle, queries to such a collection may stay quite similar, e.g., 'get unusual observations'. However, expectations to a 'good' answer may change, as with time different entities will come into contact with the object. This article reports on work in progress concerning a framework for collecting data about things, which aims at decoupling logic employed for interpreting such a collection from processing hardware and using the collection itself for transporting such logic. Main contributions include an approach to hardware-abstraction of processing logic at the object or remote, an app store for retrieving interpretation and presentation logic, and interaction forms with such memories.

Abstract

In this paper we present a new way of emotional interaction with products. Based on the rapid prototyping Microsoft Gadgeteer platform, we concretized our vision of an on-product app by implementing an anthropomorphic intelligent milk carton. The purpose of this realization is to give customers a better view of a product's life-cycle. This realization also demonstrates that the frontier between pure mobile applications development and the creation of tangible objects is very thin and opens new way to integrate the Internet of Things over an anthropomorphic user interface, thus leading to a new product to human interaction form.

Abstract

The digital workflow of the conception of a newspaper or a magazine involves several tasks. While professional solutions like QuarkXpress or Adobe InDesign combined with InCopy only offer the possibility for one person to modify a document at the same time, we have developed a concept in form of a prototype that will help journalists and the editorial staff to work collaboratively together using a touch-based table with a new approach of pagination and text editing through multitouch gestures.

Abstract

We present Calisto, a service-oriented information kiosk system for public places, like museums or hotel lobbies. Calisto supports collaboration between multiple users. They can connect their mobile devices to the large public terminal and share interesting facts and media contents via intuitive multimodal interaction. The novel contribution of our work is a seamless combination of a touch-based kiosk system and mobile devices for accessing heterogeneous information services.

Abstract

We describe a Web 3.0 interaction system where the mobile user scenario is combined with a touchscreen-based collaborative terminal. Multiple users should be able to easily organize their information/knowledge space (which is ontology-based) and share information with others. We implemented a MP3 and video player interface for the physical iPod Touch (or iPhonej and the corresponding virtual touchscreen workbench. The Web 3.0 access allows us to organize and retrieve multimedia material from online repositories such as YouTube and LastFM.

Abstract

In the framework of a scientific cooperation between the Centre de Recherche sur les Médiations (CREM), University Paul Verlaine-Metz, et le DFKI (German Research Center for Artificial Intelligence), we have developed a collaborative environment with a multi-touch interface (Collaborative Intelligent Newspaper Editor : « Colline »), to assist an editorial newspaper staff in the conceptual phase. We started with a research action, which lasted for 2 years within editorial staffs of local, regional and international newspapers in France, U.S.A. and Germany. The prototype is being validated on an international level.

Abstract

Adaptive information systems typically exploit knowledge about the user’s interests, preferences, goals etc. to determine what should be presented to the user and how this presentation should take place. When dealing with mobile users, however, information about their motions—the places visited, the duration of stays, average velocity etc.—can be additionally exploited to enrich the user model and better adapt the system behavior to the user’s needs. This paper discusses the use of positioning data and background knowledge to achieve such a motion-based adaptation of information provision