SE365 · Lecture 11 · Annotated SlidesEvaluation Foundations
Original
Overview

Evaluation Foundations

Why, what, where and when to evaluate, the types of evaluation (controlled settings with users, natural settings, settings without users), usability and living labs, an airline-booking usability case study, evaluation methods, participants’ rights and interpreting data.

How to read this page

Every original slide is shown unchanged. Yellow marks on a slide point to a numbered note: hover or tap a mark to jump to its note, and hover a note to see its marks. Questions tagged Past exam appeared on a real paper; Prediction questions are likely angles, not confirmed. Pick an answer to each question: a wrong pick tells you why it is wrong, so you can try again.

All 27 slides
  1. 1 Slide 1
  2. 2 Overview
  3. 3 Learning Outcomes
  4. 4 1) Introduction
  5. 5 1) Introduction (cont)
  6. 6 2. Why, what, where & when to evaluate
  7. 7 3. Types of evaluation
  8. 8 Usability Lab
  9. 9 Living Lab
  10. 10 Evaluation Case Studies
  11. 11 3.1 Usability testing & field studies can compliment
  12. 12 4. Case study: Airline booking system
  13. 13 4.1 Task Instruction
  14. 14 4.1 Task Instruction (cont)
  15. 15 4.2 Testing with users
  16. 16 4.2 Testing with users (cont)
  17. 17 4.2 Testing with users (cont)
  18. 18 4.2 Testing with users (cont)
  19. 19 4.2 Testing with users (cont)
  20. 20 5. Research & Analysis
  21. 21 5. Research & Analysis (cont)
  22. 22 Evaluation methods
  23. 23 Participants’ rights and getting their consent
  24. 24 Things to consider when interpreting data
  25. 25 Key points
  26. 26 Summary
  27. 27 Link
Slide 1 / 27

Slide 1

Slide 1: Slide 1
Slide 2 / 27

Overview

Slide 2: Overview
Slide 3 / 27

Learning Outcomes

Slide 3: Learning Outcomes
Slide 4 / 27

1) Introduction

Slide 4: 1) Introduction
Slide 5 / 27

1) Introduction (cont)

Slide 5: 1) Introduction (cont)
Slide 6 / 27

2. Why, what, where & when to evaluate

Slide 6: 2. Why, what, where & when to evaluate

Notes

  1. Why: check users can use the product and like it. What: prototypes to full systems. Where: lab, field, remotely. When: formative (during design) and summative (to assess a finished product).

Exam angle

PredictionQ1

Evaluation during design to improve the product is:

Slide 7 / 27

3. Types of evaluation

Slide 7: 3. Types of evaluation

Notes

  1. Types: controlled settings with users (usability testing, experiments), natural settings with users (field studies), any setting without users (inspections, heuristics, walkthroughs, models, analytics).

Exam angle

PredictionQ1

Heuristic evaluation is done:

Slide 8 / 27

Usability Lab

Slide 8: Usability Lab

Notes

  1. Usability lab: controlled environment with recording equipment; living lab: real everyday environment instrumented for study.

Slide 9 / 27

Living Lab

Slide 9: Living Lab
Slide 10 / 27

Evaluation Case Studies

Slide 10: Evaluation Case Studies
Slide 11 / 27

3.1 Usability testing & field studies can compliment

Slide 11: 3.1 Usability testing & field studies can compliment

Notes

  1. Usability testing: users do typical tasks; measure time, errors, satisfaction. Field studies: real use in context.

Slide 12 / 27

4. Case study: Airline booking system

Slide 12: 4. Case study: Airline booking system
Slide 13 / 27

4.1 Task Instruction

Slide 13: 4.1 Task Instruction
Slide 14 / 27

4.1 Task Instruction (cont)

Slide 14: 4.1 Task Instruction (cont)
Slide 15 / 27

4.2 Testing with users

Slide 15: 4.2 Testing with users
Slide 16 / 27

4.2 Testing with users (cont)

Slide 16: 4.2 Testing with users (cont)
Slide 17 / 27

4.2 Testing with users (cont)

Slide 17: 4.2 Testing with users (cont)
Slide 18 / 27

4.2 Testing with users (cont)

Slide 18: 4.2 Testing with users (cont)
Slide 19 / 27

4.2 Testing with users (cont)

Slide 19: 4.2 Testing with users (cont)
Slide 20 / 27

5. Research & Analysis

Slide 20: 5. Research & Analysis
Slide 21 / 27

5. Research & Analysis (cont)

Slide 21: 5. Research & Analysis (cont)
Slide 22 / 27

Evaluation methods

Slide 22: Evaluation methods
Slide 23 / 27

Participants’ rights and getting their consent

Slide 23: Participants’ rights and getting their consent

Notes

  1. Participants’ rights: informed consent, privacy, the right to stop at any time.

Slide 24 / 27

Things to consider when interpreting data

Slide 24: Things to consider when interpreting data

Exam angle

PredictionQ1

A lab study’s results may not reflect real-world use. This concerns:

Slide 25 / 27

Key points

Slide 25: Key points
Slide 26 / 27

Summary

Slide 26: Summary
Slide 27 / 27

Link

Slide 27: Link