Vibe-Coding HoloLens

This semester we will experiment with HoloLens Extended Reality waveguide displays. Using Unity, we will explore rapid “vibe-coding” approaches to prototype immersive experiences that merge perception, joint attention and gaze&point interaction. Participants will learn by building, improvising, and iterating on their own ideas. The seminar is open to all, regardless of prior experience, welcoming anyone curious about XR, creativity, and experimental interfaces.

What is HoloLens 2 ?

The HoloLens 2 lens is a transparent glass waveguide with nanoscale diffractive gratings. A tiny laser-based light engine generates the image, the waveguide traps and steers that light through the lens by internal reflection, and then other grating regions send it into the eye. The practical per-eye render size is 1440 by 936, and the system is designed to run at 60 hertz.

HW specs

HoloLens 2: Qualcomm Snapdragon 850 System-on-Chip; 64-bit octa-core Kryo 385 CPU (ARM8 architecture); 8 cores total; up to 2.96 GHz; 4 performance cores at up to 2.96 GHz and 4 efficiency cores at about 1.8 GHz; second-generation custom Holographic Processing Unit (HPU); 4 GB LPDDR4x RAM; 64 GB UFS 2.1 storage; 4 visible-light head-tracking cameras; 2 IR eye-tracking cameras; 1 MP time-of-flight depth sensor; accelerometer, gyroscope, magnetometer; 8 MP photo / 1080p 30 fps video camera; 5-channel microphone array; built-in spatial sound speakers; Wi-Fi 802.11ac 2x2; Bluetooth 5.0; USB Type-C DRP; 2K 3:2 light engines; >2.5k radiants; 566 g; battery life 2–3 h active use.

SW stack

Windows Holographic / Windows Mixed Reality (WMR) on-device, Microsoft Store and Intune for distribution, and Unity + OpenXR + MRTK + Visual Studio for development

How to use it

There are multiple means of interaction: pointing, gestures, moving around, gazing, voice commands.

Plot twist

HoloLens 2 comes with an important strategic caveat: being a closed Microsoft product, its viability is in great extent dependent on Microsoft’s ecosystem and support lifecycle. Since Microsoft stopped producing HoloLens 2 in October 2024 and committed support only through December 31, 2027, HoloLens 2 should be best understood as a mature but discontinued residue of one highly specialized branch of technological evolution.

Vibe Coding

In computer programming, vibe coding is a software development practice assisted by artificial intelligence (AI) such as by chatbots (such as Codex or Claude Code) or AI agents (e.g. OpenClaw). The software developer describes a project or task in a prompt to a large language model (LLM), which generates source code automatically. Vibe coding may involve accepting AI-generated code without reviewing the output thoroughly, instead relying on results and follow-up prompts to guide changes.

Question: Do You vibe code ? If yes, how &with what model/tool ?

(Y)our projects

Which dream would You like to realize ?

Which project would You like to build ?

HoloSky

0th Bildunginstrument from the Extended Educational Environment suite.

Core tech: Hololens2 (will work also without); Unity + OpenXR Plugin; Universal Rendering Pipeline (URP); Universal Windows Platform (UWP); good old Websockets

Core development methodology: Vibe coding / AI💧

Eye-2-Eye

What about using HoloLens to solve the Eye2Eye problem, finally ?

Dev environment

In order to develop HoloLens appz, one needs:

Windows ;(

Unity 2022.3 LTS (e.g. 2022.3.62f3) + Universal Windows Platform Build Support + Windows Build Support (IL2CPP)

Visual Studio + Visual Studio Installer + bunch of dependencies

OpenXR plugins