← Back to portfolio
Academic · VR · AI · Accessibility · 2024
Enabling
VR Accessibility
AI-driven 3D object descriptions for inclusive virtual environments
Objective
Making VR more inclusive.
To enhance accessibility within virtual reality environments by using AI-powered image recognition to generate real-time descriptions of 3D objects — making immersive spaces more inclusive and understandable for all users.
This project involved designing an immersive VR experience aimed at enhancing user interaction within a simulated environment. The objective was to explore how VR can be utilised to create engaging and intuitive user interfaces that respond to natural movements and gestures.
Designing for accessibility in emerging tech isn’t optional — it’s essential.
Research & Technical Exploration
Testing four AI engines.
Before building anything, I tested four AI platforms to evaluate their ability to recognise and describe 3D objects within a VR context:
- Google Cloud Vision — image recognition
- Clarifai — visual AI platform
- Vize.ai — custom image classification
- ChatGPT — natural language description
Evaluation Criteria
How the engines were assessed.
- Recognition accuracy — could it identify the object correctly?
- Response clarity — was the description usable as voice output?
- Relevance to 3D object environments — not just photos
- API integration capability with VR platforms
Chosen Solution
Google Cloud Vision won.
After multiple trials, Google Cloud Vision API was selected for its high recognition accuracy, contextually relevant object descriptions, and compatibility with prototype tools like Shapes XR and PopAi.
Prototype Development
Built in Shapes XR.
Developed a working prototype in Shapes XR to simulate AI-assisted object description in a VR environment. Two key features:
01
AI Description Icon
Tap to receive instant AI-generated text descriptions of objects in the scene.
02
Voice Playback
Listen to the object description via voice for improved accessibility and hands-free interaction.
AI Integration
How it all connected.
The technical challenge was keeping the AI layer invisible to the user — the experience needed to feel like the VR environment was simply explaining itself, not like a separate tool had been bolted on.
- Used PopAi to generate and format natural-language object descriptions
- Integrated descriptions into Shapes XR with voice functionality, enhancing user engagement and comprehension
- Ensured a smooth, non-intrusive experience that blends AI-driven assistance with immersive VR design
Conclusion
Designing for inclusion in emerging tech.
This project explored the intersection of AI, XR, and accessibility. Through research, testing, and hands-on prototyping, it demonstrated how smart technology can make virtual environments more inclusive for everyone.
With the right tools and intent, we can make immersive spaces more human, one feature at a time.