Content Details - evaluation techniques - Module 3 - Human Computer Interaction (HCI) - Nebula SkillHub

1. Goals of Evaluation

Evaluation is a mandatory part of the iterative design cycle. Its primary goals are to:

**Assess Functionality:** Ensure the system does what it is supposed to do (meets functional requirements).
**Measure Usability:** Test the effectiveness, efficiency, and satisfaction against usability goals.
**Identify Problems:** Pinpoint specific design flaws (usability bugs) that hinder user performance.
**Monitor User Experience (UX):** Gauge subjective qualities like delight, trust, and aesthetics.

2. Taxonomy of Evaluation Methods

Evaluation techniques are broadly categorized based on who is performing the assessment: actual users or usability experts.

User-Based Methods (Empirical)

Gathering primary data from interaction with real end-users.

**Examples:** Usability Testing, Field Studies, Surveys, A/B Testing.
**Strength:** Provides the most accurate data on actual usage and context.

Expert-Based Methods (Analytical)

Experts review the interface using established principles and guidelines.

**Examples:** Heuristic Evaluation, Walkthroughs, Predictive Modeling.
**Strength:** Fast and inexpensive ("discount usability").

3. Expert Evaluation Techniques

These are typically used early in the design phase when quick, frequent feedback is needed on prototypes.

A. Heuristic Evaluation (HE)

3-5 usability experts independently review an interface against a set of published usability principles (heuristics), such as Nielsen's 10 Heuristics.

**Process:** Experts walk through the interface, list violations of the heuristics, and rate the severity of each problem.

B. Cognitive Walkthrough (CW)

A highly structured process where evaluators simulate a user's problem-solving process step-by-step for a specific task.

**Focus:** Evaluating the **learnability** of a system, particularly for first-time users (e.g., will the user know what to do at each step?).

C. Predictive Modeling (e.g., GOMS)

Using mathematical models to predict human performance time for a specific task based on fundamental cognitive and motor actions.

**GOMS:** **G**oals, **O**perators, **M**ethods, and **S**election rules. Used to compare efficiency of different interface designs.

4. Usability Testing (Empirical)

The gold standard for evaluation, involving real users performing representative tasks under controlled or observed conditions.

A. Core Activities in Usability Testing

**Test Plan:** Defining goals, selecting representative users, and creating realistic tasks.
**Execution:** Users perform tasks while being observed, recorded, and timed.
**Data Collection:** Collecting both **quantitative data** (time, errors, completion rate) and **qualitative data** (user comments, facial expressions).

B. Think-Aloud Protocol

A critical technique where the user is asked to continuously vocalize their thoughts, intentions, and feelings while performing the task.

**Benefit:** Reveals the user's **mental model** and cognitive struggles, diagnosing the *reason* for an error, not just the error itself.

5. Advanced & Continuous Evaluation

Field Studies (Contextual Inquiry)

Observing users in their natural environment (office, home, factory floor) to understand the *actual* context of use.

**Benefit:** High ecological validity; essential for understanding cultural or environmental impacts on design.

A/B Testing (Controlled Experiment)

Presenting two different versions of an interface element (A and B) to different segments of live users to see which performs better on a chosen metric (e.g., click-through rate).

**Benefit:** Provides statistically significant data on live product performance.

evaluation techniques

Study Notes

HCI Evaluation Techniques