Projects

During my bachelor and master courses in Human-Computer Interaction, I realized several projects, which I want to show here briefly.

Multimodal Interaction [2017]

The Goal of this project was to implement a multimodal interface using pointing gestures and speech recognition. The project was realized using the SimulatorX framework, implemented in Scala with a board game theme where the central module was implemented with an Augmented Transition Network (ATN) integrating all human input into a system state.

Machine Learning [2017]

In this project, we implemented a multilayer perceptron neural network from the scratch instead of using a predefined framework, using gradient descent to minimize the loss function to recognize simple gestures that let users control a slideshow application. It was a great opportunity to understand how and why machine learning with neural networks work, like backpropagation function or parameters like the regularization parameter lambda.

Following this project I made a course about deep learning, stepping deeper into history beginning with AlexNet and followed by various architectures like CNN, LSTM, GAN, RNN, and hybrids.

3D User Interfaces [2017]

In this Unreal Engine based project, we implemented the best interaction and traveling technique for Virtual Reality (VR) based on the latest research publications and user evaluations using an oculus rift with controllers. The goal was to have an intuitive user interface creating and manipulation 3D primitives and navigation through a virtual world.

Dataphysicalization [2018]

Dataphysicalization is the idea to physicalize data and use all the advantages of physical artifacts like haptics, 3d shape or material . See dataphys.org for additional information and see a list of physicalizations.

We wanted to confront peoples ecological behavior physicalizing the personal carbon footprint on the weekly use of transportation, food behavior, and energy consumption. See the publication for more details.

Multimodal-Multiuser VR [2016]

Being given a lab full of cool equipment, we created a unity based client-host application integrating multiple users with different modalities (VR Headsets and Touch display) into a shared space of the historic scene of the HammaburgSee the publication for more details.