Designing User Interactions for Advanced ...

Usability Design for Immersive Virtual Reality & Other Advanced Technologies VR Meetup June 27, 2017 Agile Interactions, Inc. Martha J. Lindeman, Ph.D.

Agenda  Overview  Details  Resources

Usability Design for Immersive Virtual Reality & Other Advanced Technologies

OVERVIEW Agile Interactions, Inc. Martha J. Lindeman, Ph.D.

You Need a Treasure Map,

NOT technical details that may change tomorrow or next month. © 2006-2017 Agile Interactions, Inc. All rights reserved.

My specialty: How People Think

• Cognitive psychology • Psychobiology • Word meanings • AI & computers Knowing “what” they think about requires a domain analysis! Then excellent design makes brains’ gears run smoothly.

Use Results-centered Design

TASKS

Results (Success)

TOOLS

THINGS

© 2006-2017 Agile Interactions, Inc. All rights reserved.

NOT just Input/Output to users!  How do you choose and place user-interface ‘things’?  Tactics: Load fast, bug free, consistent, never freezes platform compatible, design for usefulness/engagement  How do you plan to benefit from your app?  Angry birds app made more $$ from ads than app sales  Design to generate in-app purchases  Increase targeted audience  Tablet apps generate 70% more revenue than phone apps  How about a subscription model

 How do you optimize other people’s motivations?  To use the app more often  To share/recommend app to others

It is all about communication!!

The GAINS Bridge to Success

© 2006-2017 Agile Interactions, Inc. All rights reserved.

Usability Design for Immersive Virtual Reality & Other Advanced Technologies

DETAILS Agile Interactions, Inc. Martha J. Lindeman, Ph.D.

Why am I the speaker? • 25+ years consulting in interaction design & training designers

› Specialty: Design user-interactions of all types at all levels › Large companies, government agencies, and startups › Very large variety of industries and domains • Ph.D. in psychology (NSF Graduate Fellow; Harvard, 1985) › General exam: cognitive psychology with artificial intelligence › Minor exams: psychobiology and creation of word meanings › Dissertation: machine models of learning natural categories • Computer science › Programmed in six languages, including binary & assembly › Serial-interface device designed and built by wire-wrapping • Academia (adjunct) › Created & taught online User-eXperience (UX) design course › Instructor for several MBA courses, including quantitative analysis and management of digital organizations © 2006-2009 Agile Interactions, Inc. All rights reserved.

Details Discussed in This Talk ●

Where to start?

●

What is an “Advanced Technology”?

●

Which usability design strategy to use?

●

What’s the path to “Success”?

●

How to build your bridge to success?

●

Suggested resources (sites, books, videos, etc.)?

Where to start?  How Apple made their money by using usability  Focus on the user/customers’ experience  Start with benefit(s) to the user/customer

 Steve Jobs Q&A – WWDC (1997) on YouTube “One of the things I’ve always found is that you’ve got to start with the customer experience and work backwards to the technology. You can’t start with the technology and try to figure out where you’re going to try to sell it. And I’ve made this mistake probably more than anybody else in this room and I’ve got the scar tissue to prove it. I know that it’s the case! And as we have tried to come up with a strategy and a vision for Apple, it started with ‘What incredible benefits can we give to the customer? Where can we take the customer?’ Not starting with ‘Let’s sit down with the engineers and figure out what awesome technology we have and then how we’re going to market that. And I think that is the right path to take.”

What is customer’s experience? & Who is the customer? (Flight attendant’s interaction with passenger)

 What if passenger had no knowledge of VR?  Hololens hinders eye-contact communication  Assesses passenger’s emotions by facial info

Start with a customer benefit!  Some people have made a LOT of money from VR  Made a customer benefit into a separate industry  1929: First immersive VR system Flight training is now a multi-billion dollar industry! No longer called a “VR system”.

Details Discussed in This Talk ●

Where to start?

●

What is an “Advanced Technology”?

●

Which usability design strategy to use?

●

What’s the path to “Success”?

●

How to build your bridge to success?

●

Suggested resources (sites, books, videos, etc.)?

What is an Advanced Technology?

User/Customer defines “Advanced”

“Hype cycle for emerging technologies” Gartner, 2016

Wearable with Gesture Input  SixthSense Technology (TED talk, 2009)

Wearable with Gesture Input  SixthSense Technology (TED talk, 2009)  MIT media lab student project

 He is now at Samsung Research America  Director of Research  Head of Think Tank Team

 Code is available as open source  GPL 3.0 license and contact info for commercial  https://code.google.com/archive/p/sixthsense/

Other Advanced Technologies ● Projective computers ● Shared spacetime ● Wearables

● Internet of Things ● Manufacturing automation ● Robotics ● Humanoid robotics ● Synthetic biology ● Self-driving vehicles ● Nanotechnologies ● Others to be invented

2014 © 2006-2017 Agile Interactions, Inc. All rights reserved.

Details Discussed in This Talk ●

Where to start?

●

What is an “Advanced Technology”?

●

Which usability design strategy to use?

●

What’s the path to “Success”?

●

How to build your bridge to success?

●

Suggested resources (sites, books, videos, etc.)?

Which UX design strategy to use? The four dangerous strategies

The designer

Experts do NOT think like novices!

Try to mentally put yourself into the customer’s experience! © 2006-2017 Agile Interactions, Inc. All rights reserved.

Use Results-centered Design

TASKS

Results (Success)

TOOLS

THINGS

© 2006-2017 Agile Interactions, Inc. All rights reserved.

Remember!!

“Experience” is in the User’s Head! To understand users’ experiences OBSERVE THEM & ASK THEM! This is called “usability testing”. It is done before and after release.

Details Discussed in This Talk ●

Where to start?

●

What is an “Advanced Technology”?

●

Which usability design strategy to use?

●

What’s the path to “Success”?

●

How to build your bridge to success?

●

Suggested resources (sites, books, videos, etc.)?

Definitions and a Question  An interface is the place where two systems come together and interact (have an effect on each other).  The user interface is the place where people interact with the objects they are using.  Types of interaction range from passively receiving input (e.g., viewing) to complete bodily input & output (e.g., virtual reality ‘presence’).  How do we train people to design great user “experiences” when anything anywhere at any time can be the user interface? © 2006-2017 Agile Interactions, Inc. All rights reserved.

Old Principles, BUT New Details  Keep it “glanceable” (only what need to see now!)  Use most effective sensory modalities  Use background intelligence to filter information

 Consider the users’ environments (e.g., privacy, display vs audio, if they access autoplaying audios in websites)  Design for off-line (what happens when not connected to Internet or something goes wrong) (https://www.wired.com/2015/08/5-principles-designing-wearables/)

Interactions create the experience The User Experience

User’s mental model

☺ or

☹

❖

Triggered by the user-interface interactions

❖

Controls all user actions unless consciously overridden

❖

Active mental model determines task difficulty

❖

Intuitive = can achieve goal (do tasks) without learning

❖

Unlearning (a very high cost) may need to happen first ➢ Novices do not have to unlearn. ➢

Mental models wrap information into chunks.

➢ Experts unlearn ONLY by making mistakes! © 2006-2010 Agile Interactions, Inc. All rights reserved.

Key: Mental Models & Attention

© 2006-2017 Agile Interactions, Inc. All rights reserved.

Design for body and mind! Subconscious mental models control automated behaviors.

Conscious mind uses mental models to make decisions.

Details Discussed in This Talk ●

Where to start?

●

What is an “Advanced Technology”?

●

Which usability design strategy to use?

●

What’s the path to “Success”?

●

How to build your bridge to success?

●

Suggested resources (sites, books, videos, etc.)?

Current State of VR Design  2016: What Google said about designing UI for VR “Put bluntly, we here at Google---we really know that we don’t know very much about VR yet. So we are looking at highly experimental ideas to push the boundary of VR, to push us to reconsider what’s possible, and we are exploring this vast new territory right alongside you.”  50 Lessons from 80+ VR & AR Prototypes  Rob Jagnow, Software Engineer  Google Daydream Labs

New Challenges in VR Design  From “VR Design Process - Google I/O 2016”  New challenges in VR design  New UI standards and design patterns  Head-mounted displays create unique concerns  New input methods and tools

 New skill sets and design tools  New team foundations

 Need more time because so many unknowns

Example: VR Design for Education VR in the Classroom: Early lessons learned from Google Expeditions 5/20/2016  Studied with 1,000,000+ students in 11 countries  Example ‘lessons learned’ during the first year  Teachers need very specific tools to do their job  Locate each student’s focus of attention (tiny circles)  Direct entire class at once (e.g., teacher has a pause button to control system)

 Tagging an app for grade & subject limited its use

Example VR-Design Fundamentals Designing Screen Interfaces for VR 5/19/2017  Field-of-view issues  New unit of measurement for visual design in VR  Text and Target sizes

 Plus much too much to include in a single talk!

 The research information will continue to pour out!  Some will be general and some device specific  You need to focus on people, not the technology

Pull users from Goals to Success

The AIN cycle

© 2006-2017 Agile Interactions, Inc. All rights reserved.

“AIN” cycle pulls people through Attraction Concepts I understand

Information Control

Attraction

I decide

Next action

Closure I do it © 2006-2017 Agile Interactions, Inc. All rights reserved.

Finally, an interaction may seem trivial, but enough bad interactions become torture.

© 2006-2017 Agile Interactions, Inc. All rights reserved.

Evaluate with SMART Objectives ●

Significance (Is this interaction important and why?)

●

Motivation (What will motivate the person to do/like this?)

●

Achievable (Is this realistic for the person & situation?)

●

Results (How will achievement or penalty be measured?)

●

Time-bound (Are there clearly defined time frames?)

There are no “right” answers! Choose the answers for a specific user in a specific situation (e.g., no time frames and no penalties for failure). http://en.wikipedia.org/wiki/SMART_(project_management) on March 2, 2009. © 2006-2017 Agile Interactions, Inc. All rights reserved.

Usability Design for Immersive Virtual Reality & Other Advanced Technologies

RESOURCES Agile Interactions, Inc. Martha J. Lindeman, Ph.D.

Details Discussed in This Talk ●

Where to start?

●

What is an “Advanced Technology”?

●

Which usability design strategy to use?

●

What’s the path to “Success”?

●

How to build your bridge to success?

●

Suggested resources (sites, books, videos, etc.)?

Book & app are excellent design aids Free App

Book

© 2006-2017 Agile Interactions, Inc. All rights reserved.

Providers have “Best Practices”

Two Books for Exploring 2017

2016

2015 Review of VR-Interaction Types  “reviewing the interaction design aspects …and the key elements needed to improve” them.  Six key elements: medium (the virtual world), immersion, feedback, interactivity, and participants  For his word “CAVE”, read “3D cubical surround”  Source article  Virtual reality and the CAVE: Taxonomy, interaction challenges and research directions  Muhanna A. Muhanna  Journal of King Saud University - Computer and Information Sciences; Volume 27, Issue 3, July 2015, Pages 344–361  https://doi.org/10.1016/j.jksuci.2014.03.023

Curated List of Sources

uxofvr.com © 2006-2017 Agile Interactions, Inc. All rights reserved.

Example set of guidelines

http://www.gamasutra.com/view/feature/13 2341/the_13_basic_principles_of_.php?pri nt=1

© 2006-2017 Agile Interactions, Inc. All rights reserved.

Sampling of videos (UX design) ●

https://www.youtube.com/watch?v=XjnHr_6WSqo

●

https://www.youtube.com/watch?v=hxV-1pJ2hjk

●

https://www.youtube.com/watch?v=-mcXAMDch7s

●

https://www.youtube.com/watch?v=Sq9NOukgxQc

●

https://www.youtube.com/watch?v=lGUmTQgbiAY

●

https://www.youtube.com/watch?v=UuceLtGjDWY

●

https://www.fastcodesign.com/90109725/vr-is-stillnowhere-near-its-youtube-moment

© 2006-2017 Agile Interactions, Inc. All rights reserved.

Other Books for Exploration

© 2006-2017 Agile Interactions, Inc. All rights reserved.

More on Augmented Reality

© 2006-2017 Agile Interactions, Inc. All rights reserved.

Questions?

THANK YOU!

Agile Interactions, Inc. Martha J. Lindeman, Ph.D.