Enriching the Narrative
Experience in Place

@ Design Freshman Studio

This is an environment project I did as a sophomore. Assuming that over the course of 10 years, augmented reality gets commonly used and is ubiquitous. I am trying to design an augmented reality experience using Hololens for incoming CMU design department visitors to learn about one of the freshman projects.


Three Potential Audiences

Before jumping into designing the details of my interactions, it is critical for me to start with simulating the visitor personas. What is their prior knowledge before this AR experience and what do they want to gain from this tour should be considered while designing my experience.

More from Yufei



The First One

I chose the light engine project as my first approach. This design mainly focuses on having the overall experience rather than focusing just on the freshman project itself. Some features of this design include:

  1. Self-guided tour. The Hololens itself can direct visitors to places that they can explore. 

  2. Recognizing professors (face recognition) as professors walk past the visitor. 

  3. Special gestures that can let the Hololens know that the visitor wants to go to the restroom.

  4. Supports my light engine project interactions. 

Sketching out Idea in VR

The First One

After making the storyboard on paper, I moved from 2D storyboards to 3D storyboards. At this time we are provided with VR headsets and now we are able to sketch out our interaction ideas in the 3D space in VR.

More Storyboard

Change my Focus to Isometric System Design

After drawing my storyboard for the light engine project, I realized that my project focused too much on the virtual reality part but did not take advantage of the augmented reality that visitors can interact with both physical space and the virtual space. My previous design is lacking the physical interactions as people do not really interact with anything that is physically there. At this time I chose to try another approach and switched my focus to the isometric system design

storyboard 3.jpg

Design Process

The vague direction of my interaction is fixed, now I am trying to design the details of my experience and starting to design my sketch video scenes so that I can show my ideas to my audience.

Interaction Scripts

When I had trouble with imaging what information I should put into my audio experience and what information should I put into the visual experience, I decided to write my scripts down so that I know how much information I am pushing my audiences to absorb at a time. This helps me to know if I should cut down some information at different stages of the interactions.

Project Title

Isometric System Project


intro to project


This is a project which requires 5-6 freshmen to work collaboratively to design a system of their choice and visualize it to the audience. Each student is required to design at least one part of their system and fit that section into one cubed space.


 [detect glaze from start to end of paragraph to (flip page) or just use hand gestures to scroll down]


Intro to interaction


"What is on the table is the rooms that one group of the freshman students drew for a space exhibition system."

"If you would like to, rearrange the rooms so that they are all connected to each other and forms a complete system. Pick one cubed space up to see detailed information about that room. When you are finished, do certain gesture to confirm that you are done."

interacting with objects on table

audio + visual


"This is the Mars exploration room. In the system, it serves as the room for families to explore what it is like on the planet Mars after the spaceship lands on the planet. The ice pool in the corner … to simulate … on Mars. The xxx machine had a selfie camera in which people can take group photos together and get a photo as a souvenir as they exit the exhibition."


  • Section: Mars Exploration Room

  •  Function: the room for families to explore what it is like on the planet Mars

  •  Designer: Laurel


use “zoom-in” gesture to enter 1st person perspective, “zoom-out” gesture to quit 1st person perspective. 



After people are done with rearranging the physical pieces, make visitors save their designs by pinching a window that confirms the completion.

Analyze the System

audio + visual


explain the whole system in detail + what each section serves for the overall system


  • project the visitor designed system into AR

  • bullet points to explain the overall system


Interact with the system in AR

  • swap to see the colored/toned/outlined version

  • tap on one of the room to see the detailed information


visual + audio


  • the layout is the same as the previous interaction

  • a student avatar shows up in the visual field of the visitor as he/she is ready to leave

  • the bullet points about the overall system disappear from the panel and now it shows the bullet points of the takeaways of this project (that is told by the student avatar)


The audio is the record of student's voice and they are telling the visitors what he/she have learned from this project. As the visitor walks away from the interactive site, the student appreciates the visitor's time and directs the visitor to the next stop of the tour. 

The Sketch Video

The Final Sketch Video

This is a sketch video that simulates the interaction I want to archive in my design. It is a mere explanation of what I want to show to the visitors and was done in a short period of time so it is not a finished piece yet (need to stop the video at some places to read the explanations).

UI/UX & product designer

Four Words to describe me: 




And here is my portfolio ;p
Enjoy my work!

  • LinkedIn
  • Facebook
  • Instagram