SE365 · Lecture 12 · Annotated SlidesDECIDE, Usability Testing and Experiments
Original
Overview

DECIDE, Usability Testing and Experiments

The DECIDE framework, usability testing (labs, portable and remote testing, eye tracking, data types, how many participants), experiments and experimental design (different, same or matched participants), and field studies.

How to read this page

Every original slide is shown unchanged. Yellow marks on a slide point to a numbered note: hover or tap a mark to jump to its note, and hover a note to see its marks. Questions tagged Past exam appeared on a real paper; Prediction questions are likely angles, not confirmed. Pick an answer to each question: a wrong pick tells you why it is wrong, so you can try again.

All 28 slides
  1. 1 Slide 1
  2. 2 Overview
  3. 3 Learning Outcomes
  4. 4 1) DECIDE: a framework to guide evaluation
  5. 5 1.1) Determine the goals
  6. 6 1.2) Explore the questions
  7. 7 1.3) Choose the evaluation approach methods
  8. 8 1.4) Identify practical issues
  9. 9 1.5) Decide about ethical issues
  10. 10 1.6) Evaluate, interpret & present data
  11. 11 Key points
  12. 12 2. Usability testing
  13. 13 2. Usability testing (cont)
  14. 14 2.1) Experiment & Usability testing
  15. 15 2.2) Usability testing & Research
  16. 16 2.3) Usability lab with observers watching a user & assistant
  17. 17 2.4) Portable equipment for use in the field
  18. 18 Tobii Glasses Mobile Eye- Tracking System
  19. 19 2.5) Remote Usability Testing
  20. 20 2.5) Testing conditions
  21. 21 2.6) Some type of data
  22. 22 2.7) How many participants is enough for user testing?
  23. 23 3. Experiments
  24. 24 3.1 Experimental design
  25. 25 3.2 Different, same, matched participant design
  26. 26 4. Field Studies
  27. 27 Key points
  28. 28 Key points
Slide 1 / 28

Slide 1

Slide 1: Slide 1
Slide 2 / 28

Overview

Slide 2: Overview
Slide 3 / 28

Learning Outcomes

Slide 3: Learning Outcomes
Slide 4 / 28

1) DECIDE: a framework to guide evaluation

Slide 4: 1) DECIDE: a framework to guide evaluation

Notes

  1. Tip DECIDE: Determine the goals, Explore the questions, Choose the approach and methods, Identify practical issues, Decide about ethical issues, Evaluate, interpret and present the data.

Exam angle

PredictionQ1

In DECIDE, the "I" stands for:

Slide 5 / 28

1.1) Determine the goals

Slide 5: 1.1) Determine the goals
Slide 6 / 28

1.2) Explore the questions

Slide 6: 1.2) Explore the questions
Slide 7 / 28

1.3) Choose the evaluation approach methods

Slide 7: 1.3) Choose the evaluation approach methods
Slide 8 / 28

1.4) Identify practical issues

Slide 8: 1.4) Identify practical issues
Slide 9 / 28

1.5) Decide about ethical issues

Slide 9: 1.5) Decide about ethical issues
Slide 10 / 28

1.6) Evaluate, interpret & present data

Slide 10: 1.6) Evaluate, interpret & present data
Slide 11 / 28

Key points

Slide 11: Key points
Slide 12 / 28

2. Usability testing

Slide 12: 2. Usability testing

Notes

  1. Usability testing: measure users’ performance on typical tasks in controlled conditions (time, errors, satisfaction, think-aloud).

Slide 13 / 28

2. Usability testing (cont)

Slide 13: 2. Usability testing (cont)
Slide 14 / 28

2.1) Experiment & Usability testing

Slide 14: 2.1) Experiment & Usability testing
Slide 15 / 28

2.2) Usability testing & Research

Slide 15: 2.2) Usability testing & Research
Slide 16 / 28

2.3) Usability lab with observers watching a user & assistant

Slide 16: 2.3) Usability lab with observers watching a user & assistant
Slide 17 / 28

2.4) Portable equipment for use in the field

Slide 17: 2.4) Portable equipment for use in the field
Slide 18 / 28

Tobii Glasses Mobile Eye- Tracking System

Slide 18: Tobii Glasses Mobile Eye- Tracking System

Notes

  1. Eye tracking shows where users look (heat maps, gaze plots).

Slide 19 / 28

2.5) Remote Usability Testing

Slide 19: 2.5) Remote Usability Testing

Notes

  1. Remote usability testing: users test from their own location; recorded online.

Slide 20 / 28

2.5) Testing conditions

Slide 20: 2.5) Testing conditions
Slide 21 / 28

2.6) Some type of data

Slide 21: 2.6) Some type of data
Slide 22 / 28

2.7) How many participants is enough for user testing?

Slide 22: 2.7) How many participants is enough for user testing?

Notes

  1. About 5–12 participants are often enough for a usability test (Nielsen: 5 users find most problems).

Exam angle

PredictionQ1

According to Nielsen, about how many users find most usability problems in a test?

Slide 23 / 28

3. Experiments

Slide 23: 3. Experiments

Notes

  1. Experiments test a hypothesis: independent variable (what you change) → dependent variable (what you measure).

Exam angle

PredictionQ1

In an experiment comparing two menu designs on task time, the menu design is the:

Slide 24 / 28

3.1 Experimental design

Slide 24: 3.1 Experimental design
Slide 25 / 28

3.2 Different, same, matched participant design

Slide 25: 3.2 Different, same, matched participant design

Notes

  1. Designs: different participants (between-subjects: no order effects, needs more people), same participants (within-subjects: fewer people, order effects → counterbalance), matched participants.

  2. Trap Within-subjects (same participants) designs suffer from ORDER effects; fix with counterbalancing.

Exam angle

PredictionQ1

Each participant tries both designs. This is a:

Slide 26 / 28

4. Field Studies

Slide 26: 4. Field Studies

Notes

  1. Field studies: observe real use in natural settings; less control, more realism.

Slide 27 / 28

Key points

Slide 27: Key points
Slide 28 / 28

Key points

Slide 28: Key points