Goals
Ease finding sought out information on Rutgers University websites
Evaluate the effect of menu orientation on search: Is there a a search advantage to vertical or horizontal drop-down menus?
Compare (behavioral & attitudinal) engagement metrics:
Does user looking behavior predict search speed or accuracy?
Do behavioral metrics align with self-reports of usability, recall or user confidence?
Methods
Search task:
40 students majoring in Psychology or Sociology search on Rutgers University websites for 3 items: 1) Academic department, 2) Course list, & 3) Research opportunities.
Self-report - usability, recall & confidence:
After finding the target, participants were asked:
1) How easy was the search task?
2) Do you recall the gender of student shown in the web sites,
3) How confident are you with your search & recall performance?
Metrics
Search time
Eye-movements, Fixation frequency & duration, Pupil size
Accuracy on 1st click test & reaching final test page
Self-report questions:
Perceived usability & Confidence
Recall of irrelevant information (i.e., photos of male & female students)
Findings
Sample trial where student searches for undergraduate courses in the top-horizontal menu condition. Eye-movements are shown in blue.Search was faster for vertical than horizontal menus (Mean diff = 1/2s).
Accuracy was similar for horizontal & vertical menu conditions
More eye-movements & fixations were used when searching horizontal menus.
Perceived usability & user confidence did not predict search performance
Impact of distractor images:
Recall of (irrelevant) student images did not influence search speed or accuracy
User-gender predicted recall of student images:
Recall was better when gender was opposite to user gender.
Pupil size (a known measure of interest level) also was affected by gender with pupil size tending to be larger when viewing opposite gender.
Questions & Future Research
Is search faster for vertical menus due to: 1) eye-movement patterns: fewer saccades of initial scan of left vertical column (i.e., deviation from F shape eye-movement pattern)? 2) crowding from pop-up horizontal menu’s close proximity to main horizontal menu
Why do recalled distractor images have no effect on search?
What is the effect of familiarity with web pages on search performance? (See study 2)
Faglio, S. & [Advisor:] Aks, D.J. (2010). Studying the effectiveness of web design with eye-tracking: An analysis of Rutgers University’s Web pages. Rutgers University Undergraduate Symposiums
Comments