Conversational agents in Virtual Reality

Wouldn’t it be cool to be able to talk to agents in a VR environment? This project was aimed at enhancing the learning experiences of users through VR immersive settings in a manner that does not let the user become entirely dependent on the technology. It can be further developed for various applications such as simulating a “teacher” in VR, encouraging social experiences, or language learning (to help users better speak a language without the pressure and anxiety to be perfect).

In this project, I created such interactable agents with realistic conversations using NLP APIs (DialogFlow, OpenAI) in Unity and implemented three different behaviors (angry, happy, sad) for characters using asset scripts in Unity! I built an application with the three agents in Unity and deployed it on Oculus Quest 2. Check out a demo video and our github repository for more details!

<!--
  See https://www.debugbear.com/blog/responsive-images#w-descriptors-and-the-sizes-attribute and
  https://developer.mozilla.org/en-US/docs/Learn/HTML/Multimedia_and_embedding/Responsive_images for info on defining 'sizes' for responsive images
-->

  <source
    class="responsive-img-srcset"
    
      srcset="/assets/img/agents-background-480.webp 480w,/assets/img/agents-background-800.webp 800w,/assets/img/agents-background-1400.webp 1400w,"
      type="image/webp"
    
    
      sizes="95vw"
    
  >

<img
  src="/assets/img/agents-background.png"
  
    class="img-fluid rounded z-depth-1"
  
  
    width="100%"
  
  
    height="auto"
  
  
  
  
    title="example image"
  
  
  
    loading="lazy"
  
  onerror="this.onerror=null; $('.responsive-img-srcset').remove();"
>

</picture>

</figure>

</div>

</div>

–>