Extended Reality
Lecturer: Dr Fangcheng ZhongPrincipal lecturer: Dr Cengiz Oztireli
Taken by: Part II CST
Code: ER
Term: Lent
Hours: 16 (12 hrs lectures, 4hrs practicals)
Format: In-person lectures
Class limit: max. 30 students
Prerequisites: Further Graphics, Introduction to Graphics
Moodle, timetable
Syllabus & Schedule
- Week 1
- Introduction
- Course schedule and concept
- The history of AR/VR/XR
- Applications of AR/VR/XR
- Overview of the XR pipeline – 3 main components of XR
- Display technologies and rendering
- Machine perception and input interfaces
- Content generation – geometry, appearance, motion
- Overview of the AR/VR/XR frameworks
- Practical 1
- Project structure
- Project themes
- Introduction to the XR programming platform
- Introduction
- Week 2
- 3D scene representations and rendering
- Object representations with polygonal meshes
- Efficient rendering via rasterization
- Practical XR rendering of textured geometries
- Tracking and pose estimation for XR
- Rigid transforms
- Camera pose estimation
- Body/ hand tracking
- Practical 2
- 3D scene representations and rendering
- Project draft proposal due
- Project feedback
- Rendering and displaying a first object
- Week 3
- 3D scene capture and understanding
- Depth estimation
- 3D geometry capture
- Appearance acquisition
- Lighting estimation and relighting
- Lighting models for 3D scenes
- Estimating lighting in a scene
- Relighting rendered models
- Practical 3
- Acquiring and utilizing depth from sensors
- Relighting a displayed object
- 3D scene capture and understanding
- Week 4
- Physically-based simulation of rigid objects
- Rigid-body dynamics
- Collision detection
- Time integration
- Physically-based simulation of non-rigid volumetric
phenomena
- Particle systems
- Basic fluid dynamics
- Practical 4
- Physically-based simulation of rigid objects
- Practical due
- Project prototype due
- Project check-in
- Simulating and rendering a simple object
- Week 5
- Advanced topics in XR I: AR/VR displays technologies
- Micro LED-based displays
- Waveguide-based displays
- Next generation displays
- Advanced topics in XR I: AR/VR displays technologies
- Week 6
- Advanced topics in XR II – Mobile AR
- Computer vision on limited hardware
- UX/UI challenges
- Content challenges
- Advanced topics in XR II – Mobile AR
- Project due
Assessment
- A practical exercise, worth 20% of the mark. This will cover the basics of AR/VR development and some of the practical techniques the students will use in their projects. This is individual work. No mobile device or GPU hours are needed.
- An AR/VR project worth 80% of the marks:
- Course projects will be designed by the students following the possible project themes proposed by the lecturer and will be checked by the lecturer for appropriateness.
- The students will form groups of 2-3 to design, implement, report, and present the course project.
- The final mark will be an implementation mark (40%), a report mark (20%), and a presentation/demo mark (20%). Each team member will be evaluated based on their contribution.
- Each project will have extensions to be completed only by the ACS students. Each student will write a different part of the report, whose author will be clearly marked. Each student will further summarise her/his contributions to the project in the same report.
- Note: Submission is by one submission per group but work by each individual must be clearly labelled.