Freemixr

Medium: 

VR Experience

Role: 

VR UX Co-ideator/Designer/Engineer

Tools: 

Unity, C#, Oculus Integration SDK

For: 

MIT Reality Hack

Year: 

2023

Collaborator(s): 

Rui Jie Wang, Paige Sun, Ramanpreet Nara, Junah Park

Process

{{sect1}}

Overview

Collaborating with a brilliant team of professional XR designers and software engineers at the MIT Reality Hack in January 2023, I helped design and prototype a VR VJing and DJing tool.

{{sect2}}

Inspiration

We were collectively curious about how to make DJing and VJing more accessible to beginners and dreamed up a spatial interface for adding effects to music tracks with intuitive gestures and freeform performance. We were inspired by the general feel of 2D digital audio workstation interfaces like Ableton Live's and translated that to 3D space with a forward-facing timeline in the playful and entertaining style of rhythm games like Rockband and Dance Dance Revolution.

{{sect3}}

Approach

The tool:

Freemixr is easy to use, with a simple, intuitive, and playful interface built for the Quest 2 & Pro headsets. Our software also allows you to trigger visual effects, layer audio filters, and manipulate your performance timeline in 3D space.

Key features:

  • Playful clap interactions for spawning visual effects
  • Depth Controller for visualizing audio & visual effects
  • Preview and Edit modes
  • DJ/VJ Controller UI

initial brainstorming whiteboard

paper prototype

figma ui design

blender ui design

The team:

We continuously kept our time limit in mind even when great ideas were encountered. We monitored the project tasks split between members and balanced where to pool brainpower and when to merge our changes. Merging changes we found took considerable time so we determined the best milestones to accomplish before committing to changes. Pair programming allowed multiple people to quickly get through and understand the project tasks, and provided continuous feedback while working. Bouncing ideas and communicating regularly prevent divergent complications.

{{sect4}}

Outcome

Introducing Freemixr - the ultimate spatial songverse editor for the next generation of immersive creators. Freemixr lets you create stunning visual experiences in real-time, synced perfectly to your music with a clap of your hands!

Imagine manipulating visual and audio effects on the fly, to create a truly immersive and interactive experience for your audience.

Github repo

{{sect5}}

Demo

{{sect6}}

Challenges

The base idea for this project could have gone in many different directions. We could imagine all kinds of features all day, however we could not implement everything in the time given. We needed to select features that were core to our goal and provided the most delightful experience. Specific challenges included operating on objects on the timeline that represented a visual effect in space, designing a VR UI that best represented the actions available.

{{sect7}}

Next Steps
  • Extending more effects that are user generated
  • Keyframe features
  • More songs, backgrounds
  • Support lyrics
  • More polish on interesting hand interactions to control audio visualizations
  • Multiplayer

Process

{{sect1}}

Overview

Collaborating with a brilliant team of professional XR designers and software engineers at the MIT Reality Hack in January 2023, I helped design and prototype a VR VJing and DJing tool.

{{sect2}}

Inspiration

We were collectively curious about how to make DJing and VJing more accessible to beginners and dreamed up a spatial interface for adding effects to music tracks with intuitive gestures and freeform performance. We were inspired by the general feel of 2D digital audio workstation interfaces like Ableton Live's and translated that to 3D space with a forward-facing timeline in the playful and entertaining style of rhythm games like Rockband and Dance Dance Revolution.

{{sect3}}

Approach

The tool:

Freemixr is easy to use, with a simple, intuitive, and playful interface built for the Quest 2 & Pro headsets. Our software also allows you to trigger visual effects, layer audio filters, and manipulate your performance timeline in 3D space.

Key features:

  • Playful clap interactions for spawning visual effects
  • Depth Controller for visualizing audio & visual effects
  • Preview and Edit modes
  • DJ/VJ Controller UI

initial brainstorming whiteboard

paper prototype

figma ui design

blender ui design

The team:

We continuously kept our time limit in mind even when great ideas were encountered. We monitored the project tasks split between members and balanced where to pool brainpower and when to merge our changes. Merging changes we found took considerable time so we determined the best milestones to accomplish before committing to changes. Pair programming allowed multiple people to quickly get through and understand the project tasks, and provided continuous feedback while working. Bouncing ideas and communicating regularly prevent divergent complications.

{{sect4}}

Outcome

Introducing Freemixr - the ultimate spatial songverse editor for the next generation of immersive creators. Freemixr lets you create stunning visual experiences in real-time, synced perfectly to your music with a clap of your hands!

Imagine manipulating visual and audio effects on the fly, to create a truly immersive and interactive experience for your audience.

Github repo

{{sect5}}

Demo

{{sect6}}

Challenges

The base idea for this project could have gone in many different directions. We could imagine all kinds of features all day, however we could not implement everything in the time given. We needed to select features that were core to our goal and provided the most delightful experience. Specific challenges included operating on objects on the timeline that represented a visual effect in space, designing a VR UI that best represented the actions available.

{{sect7}}

Next Steps
  • Extending more effects that are user generated
  • Keyframe features
  • More songs, backgrounds
  • Support lyrics
  • More polish on interesting hand interactions to control audio visualizations
  • Multiplayer

Outcome

Other work

Want to create something awesome? Drop me an email.

→ Hi@email.com