GOALS

The company's web presence and data warehousing has evolved into a complex, highly sophisticated system. Through this sophistication comes expanded customer service via the numerous portals available on Humana's website. After hearing a complaint about Physician Finder, I needed an accurate measurement of how successful the Physician Finder Plus (PFP) really is.

The goal of this evaluation was to validate recent comments about the complexity of PFP. A variety of tasks were chosen to fully explore different sections of the PFP website. A usability study was performed during May 18 - 24, 2006, with participants that were Humana employees; three out of four employees participating do not use PFP as part of their job function.

Goals equate to real life task scenarios. These scenarios will be performed by the participants while being observed. Observations will be tallied into qualitative and quantitative data that will be compared to the needs of the user and draw conclusions on how helpful the Humana Physician Finder website is to users. The following usability goals will be examined:

  • Site structure and labeling are clear and intuitive
  • Conduct one-on-one usability tests to determine if the site is easy enough to locate specific physicians in their designated health plan
  • Evaluate user behavior with the hospital comparison tool

Concerns


Value Proposition
  1. Initial impression
  2. Meets customer needs?
  3. Perceived benefits?
  1. Rank on a scale, should be at least “agree”
  2. Tasks should take less than 3 minutes (Quantitative)
  3. By questioning user, identify benefits of site (Qualitative)

Navigation (numbered user tasks)
  1. Able to locate provider by name in the correct health plan?
  2. Able narrow a search?
  3. Encouraged to choose Urgent care instead of hospital?
  4. Locate nearby hospital?
  5. Locate the hospital comparison tool?
  6. Locate mammogram center?
  • Tasks should take less than 3 minutes (Quantitative)
  • Ranks the websites ease of use (Qualitative)
Hospital Tool (feature located on secure site only) Can find and use the hospital tool?
  • Tasks should take less than 3 minutes (Quantitative).
  • Ranks the websites ease of use (Qualitative)

Metrics

Testing is designed to gather both quantitative data (statistics) and qualitative data (comments, reactions). The following were used in this test:

  1. User will fill out demographic survey
  2. Observe users performing tasks (quantitative) and document any relevant information including emotions and comments (qualitative). Measurements will include:
    • Observations of the tasks with video tape (quantitative)
    • Number of mouse clicks (quantitative)
    • Time spent on task (quantitative)
    • Open-ended questions (qualitative)
    • User Demographics (qualitative)
  3. Post questionnaire (qualitative), ask users open ended questions about the experience (qualitative) in a post questionnaire