Experiential Design / Assignment 04

1. ASSIGNMENT BRIEF

Week 10 – Week 15

• For the final task, we were required to develop a fully functional Augmented Reality (AR) application based on the ideas and features proposed in Task 1 and Task 2. The goal was to bring together all previous planning, design, and prototyping efforts into a complete and interactive AR experience.

2. PROCESS WRITTING 

2.1 Image Scanning & Model Interaction (Move, Rotate, Zoom) - Exterior

Figure 2.1.1 Exterior Modeling 

This was the very first feature I implemented. Using Vuforia’s Image Target system, I made it possible for a 3D model of the building to appear on top of a  image through the phone and webcam.

Figure 2.2.2 Image Target In Unity

After the model appeared, I needed to let users interact with it. So I wrote scripts allowing the user to:

• Move the model using drag gestures or mouse input.
• Rotate the model left or right to view it from all angles.
• Zoom in and out using pinch gestures on mobile or the scroll wheel on desktop.

Figure 2.2.3 Ui Design For Exterior 

At first, every time the user pressed the Reset or Cancel button, a new instance of the building model would spawn. If pressed repeatedly, it would cause multiple overlapping models, which led to serious performance issues and eventually caused Unity to crash.

**Forget to take image for it **

To resolve this, I modified the spawning logic in the script. I changed the code to check if a model already exists, and if it does, the system will not spawn a new one. This ensured that only one instance of the model appears on screen at any time, improving memory efficiency and avoiding crashes.

2.2.1 Entering the Building Interior & Model Freezing

The next function was allowing users to enter the interior of the building to explore different rooms.

First Attempt

Initially, I split the model into two separate layers - Level 1 Building and Level 2 Building

But after i tried this i realize that using one model will be better So I switched to using a single complete building model, where the interior view is just a new camera perspective from inside the model.

Figure 2.2.1.1 Interior Building - Model

2.2.2Camera Navigation

When a user taps "Enter", the camera moves to a preset entry point inside the building, usually starting at the main entrance or lobby.

However, I found that once inside, the AR camera couldn’t rotate, which made the experience very limited. So I wrote a script allowing the AR camera to rotate 360 degrees, giving users the ability to look around the entire interior freely, similar to a virtual tour.

Figure 2.2.2.1 AR Camera Inspector Look

2.2.3 Stabilizing the Model

A major problem appeared at this stage: when users moved the camera or when lighting or background objects in real life changed, the entire building model would glitch, jitter, or move out of place.

To fix this, I updated the "Enter" script to freeze the building model’s position and rotation using Unity’s RigidbodyConstraints or by simply disabling updates to its transform. This ensured that once the user is inside, the building stays completely stable, even if the environment changes.

2.3 Torchlight & Day/Night Lighting System

Attempts That Didn’t Work

I tried two approaches at first:

• Camera Masking: I added a light effect texture to the canvas and tried masking around it to create a flashlight view. But this did not blend well with the environment.

• Black Overlay + Glow: I created a black canvas with a transparent circle in the center and added a glow effect. It gave the feeling of a torchlight, but the performance was poor and the light didn’t affect 3D objects realistically.

Final Working Method

Eventually, I attached a Spotlight to the AR camera:

• This spotlight points in the camera's forward direction, behaving like a torchlight.
• It turns on when entering night mode and off in day mode.

I also scripted the Directional Light to:

• Turn off when entering night mode.
• Turn on when switching back to day mode.

To make the environment more atmospheric, I added an Ambient Light for night mode with a slight blue-green tint, enhancing the spooky or mysterious feel.

During the day, I added extra Point Lights to brighten the scene further and create a visible contrast between the torchlight and normal lighting conditions.

Figure 2.3.1 Camera Making method Outcome (testing only not use at last) 

Figure 2.3.2 After Adding Day Ambient Light

Figure 2.3.3 After Adding Night Ambient Light

Figure 2.3.4 Adding Spot Light and attach under AR Camera to act as torchlight

2.4 Room Lighting Logic

While the torchlight worked well, I noticed that the rooms were not dark enough, which reduced the contrast between the day and night modes.

I tried:

• Adding multiple point lights during the day to illuminate the environment.
• Introducing a soft ambient blue-green light at night to increase creepiness.

But after testing, I felt that even with ambient light, the night environment didn’t look dark or moody enough. The lights made everything too visible and took away from the torchlight effect.

So I removed most ambient lights during night mode and relied solely on the spotlight torch to control visibility, which created a better sense of immersion and mood.

Figure 2.4.1 Attach it under lighting manager script to turn off and on

2.5 Interactive Map for Interior Navigation

The map feature allows users to teleport between rooms in the building.

Methods I Tested:

• Moving the building to the camera – this was unstable and caused orientation issues.
• Moving the camera to room positions – this method worked perfectly and was smooth.

Implementation Steps:

• Created empty GameObjects at each room location (called "Room Points").
• Stored references to these points in a RoomManager script.
• Designed a map UI with buttons representing each room.
• Assigned OnClick() functions to each button, calling the script to move the camera to the selected room’s position.
• Added hover color changes and click animations for visual feedback.

Figure 2.5.1 Adding Button For All Room

Figure 2.5.2 Assigning On Click() Event

2.6 Room Points

Room Points are invisible empty objects placed inside the building. Each represents a camera target position.

When a user clicks a map button, the script tells the AR camera to teleport to the corresponding Room Point, creating a smooth and responsive navigation experience inside the model.

Figure 2.6.1 Room Point List For All Room

Figure 2.6.2 Assigning Room Point 

2.7 Button Functions & Interaction Design

Exterior View Buttons:

• Reset: Re-centers the model to its default position.
• Enter: Moves the user into the interior (main hall).
• Cancel: Deletes the model and lets users rescan.

Interior View Buttons:

• Open Map: Shows the map UI for navigation.
• Map Navigation: Buttons for moving between rooms.
• Close Map: Hides the map and returns to the interior view.
• Light On/Off: Switches between Day and Night modes.
• Exit: Returns to the exterior view.

All buttons are linked using Unity's Inspector with OnClick() methods pointing to specific functions (e.g., EnterButton.cs, RoomManager.cs, etc.).

Challenges Faced

• Button Issues

At first, I tried to control button logic entirely through script without assigning anything in the Unity Inspector. This caused bugs and made debugging harder.

Eventually, I learned to use Unity’s OnClick events, which made linking UI and scripts much smoother and less prone to error.

Figure 2.7.1 On Click() Event For Cancel Button

Figure 2.7.2 On Click() Event For Reset Button

Figure 2.7.3 On Click() Event For Enter Button

2.8 Modeling Transfer 

In the early stages of development, I used an incomplete version of the building model for testing and functionality setup. This allowed me to implement interaction scripts and user interface elements without waiting for the finalized model. Crystal was responsible for applying the materials to the final version of the building, which was created in Blender.

We encountered several challenges during the file transfer and integration process. Initially, we attempted to export the final model from Blender as an FBX file. However, when importing the FBX into Unity, the materials did not display correctly. After researching online, we discovered that Blender’s material assignments do not always carry over properly into Unity unless applied in a very specific way.

To solve this, Crystal applied all materials directly within Blender, one by one. After completing the material work, she exported the updated model and shared the Unity scene with me by exporting the entire unit as a Unity package. I then imported this package into my project, merging it with the current working files.

Even after the transfer, some materials were still missing or incorrectly applied. I manually reassigned these materials within Unity to ensure visual consistency. Additionally, some parts of the model were misaligned or disconnected, resulting in awkward geometry. I addressed this by adjusting the model’s scale, position, and orientation within Unity, replacing the placeholder model used during earlier testing.

Figure 2.8.1 Exterior ( The Scene We did our work ) , Sample Scene ( The imported Model Scene)

Figure 2.8.2 Reassigning material to model

2.9 UI/UX Visual Redesign

Midway through the project, We  decided to redesign the visual style of the UI panels and buttons. Because in the earlier designs were too cartoonish with thick black outlines and didn’t match the mood of the experience.

We Recreated all icons and panels in Adobe Illustrator. that Focused on a cleaner, more modern style.

Figure 2.9.1 / 2.9.2 Pop Up Panels Frame Design / Button Design / Icon Design  (1st version → final version)

These changes helped create a more immersive and polished visual experience.

2.10 Instruction Panel & User Onboarding

This was one of the final features I implemented to guide users through the app.

Setup:

• First, I added an Enter Page that appears after scanning.
• Then I created instruction panels that guide users step-by-step.

Issues Faced:

• Initially, panels didn’t appear correctly. I was hiding them using alpha (opacity) and disabling elements, but this caused bugs.
• I changed the method to completely disable and enable GameObjects one by one through code.
• Buttons and other UI were hidden during instructions, and only shown after the instruction finished playing.

I originally wanted to freeze all controls (camera and inputs) during the instruction, but this conflicted with other features. After multiple failed attempts, I allowed users to move the camera during instruction playback to maintain usability.

There are two sets of instructions:

• One for the exterior view
• One for the interior view

In some cases, instructions didn’t appear because I forgot to assign references in the Unity Inspector. Once assigned correctly, it worked as expected.

Figure 2.10.1 When enter Page Set Up

Figure 2.10.2 Enter Instruction Set Up

Figure 2.10.3 Interior Before Enter Instruction Set Up

2.11 Collaborating & Merging Files with Teammate

I worked with Crystal on this project, and we faced several issues when trying to share and merge files.

Initial Attempt – GitHub

We first tried GitHub, but:

• Crystal could upload using Git LFS.
• I couldn’t download properly because of LFS issues that showed constant errors.
• This method failed after many attempts.

Second Attempt – File Export

Next, I finished my part and exported the Unity project to Crystal. But:

• Her Unity couldn’t open the project.
• It showed Vuforia-related errors and missing libraries.

Final Working Method – Unity Cloud

After asking a friend, I learned about Unity Cloud Collaboration.

• I uploaded the project to Unity Cloud and invited Crystal as a collaborator.
• She still couldn’t open it from her end, as files were missing in Unity Hub.

Eventually, we asked our lecturer for help. He:

• Uploaded the file on his Unity Cloud account.
• Added both of us as collaborators. but after editing the file we could not check in our things in unity (showing we have no permission 

Figure 2.11.1 Unity Cloud Set Up

Figure 2.11.2 Unity Version Control In Unity

2.12 Scrollable Instruction Text (Manual Scroll for PC & Mobile)

While doing the final steps, Crystal tried to implement scrolling for the history panel, but it wasn’t working. So i helped on it and i thought it would be easy, but it turned out to be more difficult than expected. I tried many ways—some allowed scrolling but couldn’t be controlled properly, and others didn’t scroll at all.

What I Tried:

• Scrollbar Only: Worked, but not user-friendly.
• Touch Input Script: Caused issues with Scroll Rect behavior.
• Mouse Scroll Script: Worked on PC, not on mobile.
• Rebuilt Scroll View: Created a new one with Scroll Rect, Content Size Fitter, and Vertical Layout Group, but still had layout issues.

Final Working Method:

After many attempts, I got it working using Unity’s default Scroll View setup:

• Linked the correct Content object
• Used RectMask2D for the Viewport
• Set Content Size Fitter to Vertical: Preferred Size
• Made sure the Canvas scaled with screen size
• Confirmed Event System and UI hierarchy were correct

Now it scrolls smoothly on both PC (mouse wheel) and mobile (touch), even though it took much longer than expected to get right.

3. FINAL OUTCOME

Note: When using the BuildSight app, it is recommended to have stable and even lighting. Poor or uneven lighting conditions may cause the AR model to become unstable or lose tracking.

CLICK HERE - Google Drive With Unity Zip File / App Build File / Image For Image Target

3.1 presentation video 

 

3.2 Walkthrough Video

3.3 presentation slide 

Final Presentation by moorenee817

3.4 Group Member Blog Link - CRYSTAL KANG WEI KEE

3.5 ALL Item Submission Link - CLICK HERE

4. REFLECTION WRITTING 

Working on this assignment to create a functional AR prototype has been both a challenging and rewarding experience. Some features, such as the light on/off function and the manually scrollable text, were particularly difficult to implement. I encountered multiple issues while trying to make them work, but after many attempts and troubleshooting, I was finally able to overcome those obstacles.

Throughout this process, I gained a deeper understanding of how AR applications work and what experiential design truly means. It was fascinating to learn how the apps we use daily are built using Unity. Despite the challenges, I found the process quite enjoyable—especially watching everything come together in the end.

One recurring issue was the UI elements occasionally shifting position, which required constant adjustments and testing. Additionally, the process of sharing project files was sometimes frustrating, but those problems were eventually resolved.

This project also helped improve my UX design skills, as I had to think carefully about user interaction and experience while developing the prototype. Overall, I’ve gained valuable hands-on experience, learned new technical and design skills, and truly enjoyed the satisfaction of seeing my work come to life.