Selecting Participants

Participants are limited to Humana employees who do not traditionally use physician finder as part of their job function. This should represent the average user. Three participants represent the average, non-employed user. One participant is a slightly above average user. Two users visit PFP daily, one user visits less than once per month, and one user had never visited PFP before. Criteria concerning sex, age, or general internet experience was not a factor in determining participants.

Think Aloud Approach

Participants were encouraged to talk through the cognitive process. This allows for capturing the thoughts and feelings contributing to the collection of qualitative data.

  • Pre-Evaluation: User fills out demographic survey
  • Evaluation: User completes tasks
  • Post-Evaluation: User fills out survey indicating the quality of the user’s experience

Pre-Evaluation Phase (10 minutes)

As participants arrived, an attempt was made to make them feel at ease and to assure them that they are not the ones being tested; there were no right or wrong answers. Only the website was being reviewed and that their input would be viewed as an aggregate answer.

Users are asked to fill out the demographic survey and sign the consent form that informs them the session is videotaped for educational purposes and the evaluation is on the functionality of the website, not an evaluation of the user’s intelligence. Every effort is made to make the user feel relaxed and that their opinion is valued.

Evaluation Phase (30 minutes)

One-on-one evaluation was conducted by Sandy Nichols, Humana Employee of 10 years, currently an art student at the Art Institute of Atlanta as a web design student. Testing was conducted one-on-one in an office with two cameras used. The facilitator gave a brief overview of the site and warm up the user with some general questions about the site:

  • What is your initial impression?
  • Is it clear where to begin looking for a physician?

This encourages the user to start talking, and the facilitator encourages the user to think aloud while performing tasks.

Two cameras were used. One captured the expressions of the users and was used to produce the qualitative data. The second camera captured the users screen, and was used to calculate quantitative data like # of clicks and time-on-task.

Post-Evaluation Phase (10 minutes)

After the tasks were completed, a series of questions were asked to find out the overall experience. Questions included:

  • What were your difficulties?
  • Did you accomplish the tasks like you expected?
  • Did the terms and labels clearly represent what was on the page?

Besides the post-evaluation survey, the user was given a chance to further clarify the reasoning for certain behavior. This could be valuable insight about the user’s experience, and is recorded as qualitative data.

User is thanked for his/her time and input.

Upon completion of the usability testing, each participant received a $10 gift card to Panera Bread Company.