skip to primary navigationskip to content

Department of Computer Science and Technology

Part II CST

 

Course pages 2022–23

Extended Reality

Principal lecturer: Dr Cengiz Oztireli
Taken by: Part II CST
Code: ER
Hours: 16 (12 hrs lectures, 4hrs practicals)
Format: In-person lectures
Class limit: max. 30 students
Prerequisites: Further Graphics, Introduction to Graphics

Syllabus & Schedule

  • Week 1
    • Introduction
      • Course schedule and concept
      • The history of AR/VR/XR
      • Applications of AR/VR/XR
    • Overview of the XR pipeline – 3 main components of XR
      • Display technologies and rendering
      • Machine perception and input interfaces
      • Content generation – geometry, appearance, motion
    • Overview of the AR/VR/XR frameworks
    • Practical 1
      • Project structure
      • Project themes
      • Introduction to the XR programming platform
  • Week 2
    • 3D scene representations and rendering
      • Object representations with polygonal meshes
      • Efficient rendering via rasterization
      • Practical XR rendering of textured geometries
    • Tracking and pose estimation for XR
      • Rigid transforms
      • Camera pose estimation
      • Body/ hand tracking
    • Practical 2
  • Project draft proposal due
  • Project feedback
  • Rendering and displaying a first object
  • Week 3
    • 3D scene capture and understanding
      • Depth estimation
      • 3D geometry capture
      • Appearance acquisition
    • Lighting estimation and relighting
      • Lighting models for 3D scenes
      • Estimating lighting in a scene
      • Relighting rendered models
    • Practical 3
      • Acquiring and utilizing depth from sensors
      • Relighting a displayed object
  • Week 4
    • Physically-based simulation of rigid objects
      • Rigid-body dynamics
      • Collision detection
      • Time integration
    • Physically-based simulation of non-rigid volumetric phenomena
      • Particle systems
      • Basic fluid dynamics
    • Practical 4
  • Practical due
  • Project prototype due
  • Project check-in
  • Simulating and rendering a simple object
  • Week 5
    • Advanced topics in XR I: AR/VR displays technologies
      • Micro LED-based displays
      • Waveguide-based displays
      • Next generation displays
  • Week 6
    • Advanced topics in XR II – Mobile AR
      • Computer vision on limited hardware
      • UX/UI challenges
      • Content challenges
  • Project due

 

Assessment

  • A practical exercise, worth 20% of the mark. This will cover the basics of AR/VR development and some of the practical techniques the students will use in their projects. This is individual work. No mobile device or GPU hours are needed.
  • An AR/VR project worth 80% of the marks:
    • Course projects will be designed by the students following the possible project themes proposed by the lecturer and will be checked by the lecturer for appropriateness.
    • The students will form groups of 2-3 to design, implement, report, and present the course project.
    • The final mark will be an implementation mark (40%), a report mark (20%), and a presentation/demo mark (20%). Each team member will be evaluated based on their contribution.
    • Each project will have extensions to be completed only by the ACS students. Each student will write a different part of the report, whose author will be clearly marked. Each student will further summarise her/his contributions to the project in the same report.