Mar 31, 2007 - manipulated/active objects. â· system control. â· Slow client will render: â· rest of the scene. Results combined into multi-frame rate display.
MFR Techniques User Study Discussion Conclusion
Multi-Frame Rate Rendering and Display Jan P. Springer1
Stephan Beck1
Dirk Reiners2 1
Bauhaus-Universität Weimar
IEEE VR 2007, 2007/03/14
Felix Weiszig1
Bernd Froehlich1 2
University of Louisiana at Lafayette
1 / 23
MFR Techniques User Study Discussion Conclusion
Motivation
Observations for Complex Applications
High frame rates:
Low(er) frame rates:
I
Object manipulation
I
Head tracking
I
System control
I
Navigation
IEEE VR 2007, 2007/03/14
2 / 23
MFR Techniques User Study Discussion Conclusion
Motivation
Multi-Frame Rate Rendering and Display Asynchronous rendering I I
Distribute scene to two clients (graphics cards / computers) Fast client will render: I I
I
manipulated/active objects system control
Slow client will render: I
rest of the scene
Results combined into multi-frame rate display I I
Optical superposition Digital composition
IEEE VR 2007, 2007/03/14
3 / 23
MFR Techniques User Study Discussion Conclusion
Optical Superposition Digital Composition Client
Optical Superposition
Master Client
FC Master SC
Inspired by Majumder and Welch, Compter Graphics Optique: Optical Superposition of Projected Computer Graphics, IPT - EGVE 2001
IEEE VR 2007, 2007/03/14
4 / 23
MFR Techniques User Study Discussion Conclusion
Optical Superposition Digital Composition Client
Optical Superposition
Master Client
FC Master SC
Inspired by Majumder and Welch, Compter Graphics Optique: Optical Superposition of Projected Computer Graphics, IPT - EGVE 2001
IEEE VR 2007, 2007/03/14
4 / 23
MFR Techniques User Study Discussion Conclusion
Optical Superposition Digital Composition Client
Optical Superposition
Master Client
Slow client (SC)
Fast client (FC)
Optically combined image on display IEEE VR 2007, 2007/03/14
5 / 23
MFR Techniques User Study Discussion Conclusion
Optical Superposition Digital Composition Client
Optical Superposition
IEEE VR 2007, 2007/03/14
Master Client
6 / 23
MFR Techniques User Study Discussion Conclusion
Optical Superposition Digital Composition Client
Optical Superposition
Master Client
Properties and issues + Easy to implement + Fast interaction and object manipulation − No occlusion between objects on fast and slow client − Half transparency for overlapping objects from FC and SC − Popping artifacts during selection and deselection of objects − Requires 2 × number of projectors
IEEE VR 2007, 2007/03/14
7 / 23
MFR Techniques User Study Discussion Conclusion
Optical Superposition Digital Composition Client
Digital Composition
Master Client
FC Master SC
Inspired by Sort-Last parallel graphics
IEEE VR 2007, 2007/03/14
8 / 23
MFR Techniques User Study Discussion Conclusion
Optical Superposition Digital Composition Client
Digital Composition
Master Client
FC Master SC
Inspired by Sort-Last parallel graphics
IEEE VR 2007, 2007/03/14
8 / 23
MFR Techniques User Study Discussion Conclusion
Optical Superposition Digital Composition Client
Digital Composition
Master Client
Slow client
Fast client
Digitally composited image on display IEEE VR 2007, 2007/03/14
9 / 23
MFR Techniques User Study Discussion Conclusion
Optical Superposition Digital Composition Client
Digital Composition
IEEE VR 2007, 2007/03/14
Master Client
10 / 23
MFR Techniques User Study Discussion Conclusion
Optical Superposition Digital Composition Client
Digital Composition
Master Client
Properties and issues + Fast interaction and object manipulation + Perfect occlusion between objects on fast and slow client − Implementation more difficult I I
Transfer of depth/color buffer from SC to FC Transfer of view transform from SC to FC
− Popping artifacts during selection and deselection of objects − Increased latency for images generated by SC − Network limits update rates of SC
IEEE VR 2007, 2007/03/14
11 / 23
MFR Techniques User Study Discussion Conclusion
Experiment Setup Results
User Study Hypothesis Digital or optical multi-frame rate method improves interaction performance with respect to single-frame rate method at low frame rates
Experiment Setup I I
Basic 3 DOF docking task, head tracked, 16 participants Render methods: I I I I
single-frame rate @ 10 Hz (SF10 ) multi-frame rate w/ optical superposition @ 10/30 Hz (MFopt ) multi-frame rate w/ digital composition @ 10/30 Hz (MFdig ) single-frame rate @ 30 Hz (SF30 )
I
Measure task completion times (TCT)
I
Determine user preference (scale from 1 to 5)
IEEE VR 2007, 2007/03/14
12 / 23
MFR Techniques User Study Discussion Conclusion
Experiment Setup Results
Experiment Setup
IEEE VR 2007, 2007/03/14
13 / 23
MFR Techniques User Study Discussion Conclusion
Experiment Setup Results
Results
10,000 9,000 8,000 7,000
TCT (in msec)
6,000 5,000 4,000 3,000 2,000 1,000
SF10
MFOpt MFDig
IEEE VR 2007, 2007/03/14
SF30
14 / 23
MFR Techniques User Study Discussion Conclusion
Experiment Setup Results
Results
10,000 9,000 8,000
5
7,000
User Preference
TCT (in msec)
6,000 5,000 4,000 3,000 2,000
4
3
2
1,000
1
SF10
MFOpt MFDig
IEEE VR 2007, 2007/03/14
SF30
SF10
14 / 23
MFOpt MFDig
SF30
MFR Techniques User Study Discussion Conclusion
Artifacts Latency
Discussion: Artifacts
I I I I
Popping at selection Popping at deselection Transparency Manipulation effecting the whole scene (e. g. moving a light source)
IEEE VR 2007, 2007/03/14
15 / 23
MFR Techniques User Study Discussion Conclusion
Artifacts Latency
Discussion: End-to-End Latency
I
Optical superposition No additional latency
I
Digital composition TRead Color/Depth on SC + TSend over Network + TDraw Color/Depth on FC
IEEE VR 2007, 2007/03/14
16 / 23
MFR Techniques User Study Discussion Conclusion
Artifacts Latency
Draw Read
End-to-End Latency: Graphics
I I
1280 × 1024
1600 × 1200
MB/s
ms
MB/s
ms
BGRA_EXT DEPTH
997 565
5.3 9.3
939 733
8.4 10.8
BGRA_EXT DEPTH
2081 1213
2.5 4.3
2166 1372
3.6 5.7
nVidia GeForce 8800 GTX, driver rev. 97.46 Note: selecting the “right” color format makes a difference
IEEE VR 2007, 2007/03/14
17 / 23
MFR Techniques User Study Discussion Conclusion
Artifacts Latency
End-to-End Latency: Network
I
Resolution 64 Bit Color/Depth
Packet Size in MBytes
Transfer Time in ms
1024 × 768 @ 15 Hz 1280 × 1024 @ 9 Hz 1600 × 1200 @ 6 Hz
6 10 15
66 111 166
Assuming 90 Mbytes/s bandwidth
IEEE VR 2007, 2007/03/14
18 / 23
MFR Techniques User Study Discussion Conclusion
Artifacts Latency
End-to-End Latency: Example
Sensors FC Update ~20ms
Master
Frame Rate: 30Hz TFrame = TDraw + TRender 33ms = 10ms + 23ms Send ~111ms
Update ~20ms
SC
Frame Rate: 10Hz TFrame = Trender + TRead 100ms = 80ms + 20ms
I
Resolution: 1280 × 1024
I
Compression may not decrease network latency see Roth and Reiners, Sorted Pipeline Image Composition, EGPGV06
IEEE VR 2007, 2007/03/14
19 / 23
MFR Techniques User Study Discussion Conclusion
Summary Future Work
Summary I
Multi-frame rate rendering I I I
I
Client
Optical superposition I I I
I
Improves object manipulation and system control Does not improve navigation Works with stereo and head tracking
Master Client
Composition artifacts (half-transparency, popping) Precise manipulation very difficult Useful for system control and foreground elements Client
Digital composition I I I I
Master Client
Few artifacts — most can be fixed Artifacts may not be noticed Requires very fast network (e. g. 10 GBit) User study confirms performance almost as good as rendering everything fast
IEEE VR 2007, 2007/03/14
20 / 23
MFR Techniques User Study Discussion Conclusion
Summary Future Work
Summary I
Multi-frame rate rendering I I I
I
Client
Optical superposition I I I
I
Improves object manipulation and system control Does not improve navigation Works with stereo and head tracking
Master Client
Composition artifacts (half-transparency, popping) Precise manipulation very difficult Useful for system control and foreground elements Client
Digital composition I I I I
Master Client
Few artifacts — most can be fixed Artifacts may not be noticed Requires very fast network (e. g. 10 GBit) User study confirms performance almost as good as rendering everything fast
IEEE VR 2007, 2007/03/14
20 / 23
MFR Techniques User Study Discussion Conclusion
Summary Future Work
Summary I
Multi-frame rate rendering I I I
I
Client
Optical superposition I I I
I
Improves object manipulation and system control Does not improve navigation Works with stereo and head tracking
Master Client
Composition artifacts (half-transparency, popping) Precise manipulation very difficult Useful for system control and foreground elements Client
Digital composition I I I I
Master Client
Few artifacts — most can be fixed Artifacts may not be noticed Requires very fast network (e. g. 10 GBit) User study confirms performance almost as good as rendering everything fast
IEEE VR 2007, 2007/03/14
20 / 23
MFR Techniques User Study Discussion Conclusion
Summary Future Work
Future Work I
Refine digital composition approach I I I
I
I
Further user studies I I
I
Transfer screen rectangle of selected object for non-tracked scenarios Implement popping artifact fixes Transparency artifact solution for special cases (e. g. volume rendering) Evaluate on multi-GPU system
Lowest limit for head tracking update rates? Which frame rate ratios for SC and FC work well?
Combine with other parallel rendering strategies I
Resource (re-)allocation/balancing
IEEE VR 2007, 2007/03/14
21 / 23
MFR Techniques User Study Discussion Conclusion
Thank you for your attention.
IEEE VR 2007, 2007/03/14
22 / 23
MFR Techniques User Study Discussion Conclusion
IPT - EGVE 2007 Weimar, Germany
I
I
Submission deadline: March 31, 2007 Conference: July 15 – 18, 2007
http://www.uni-weimar.de/medien/vr/ipt-egve IEEE VR 2007, 2007/03/14
23 / 23
