Home
Usability
Usability Testing for Community Data Sites
Usability
Testing of Community Data and Mapping Systems
Usability testing is the only way to ensure that a web site designed
for the public is truly usable. Devoting resources to such testing pays
off by making a web site more efficient, effective, and credible, and
leaves a trail of satisfied users who come back to the site and recommend
it to their colleagues. This document summarizes research behind why usability
testing is important, especially in web-based GIS, and concludes with
a basic protocol for applying usability testing to community data and
mapping systems. Read this paper >
Presentation (1.5 MB) at PPGIS Conference, July 2003 (Note: This PPT presentation provides an overview, the paper above provides more detail and rationale.)
| For further info: Denice Warren at [email protected] |
Tools and templates for testing
Sample
Test Plan
This is the big usability testing planning document we used. It includes
all of our research questions, with references to which user tasks corresponded
to each question. It also covers criteria we had for participants, the
plan for recruiting, techniques to be used during testing, a checklist
for items to bring to the test, and an overview of the testing protocol.
Phone
Screen Sheet
We used this sheet as a guide for the initial phone screening of testers.
Screening is used to get testers within the target audience, and then,
in the second round of testing, to roughly match testers with the first
cohort.
Script
for beginning Usability Testing
This is the script that our primary interviewer used to begin the testing
session once we arrived on site.
Release
Form
A release form to be signed by the usability tester. It gives us permission
to use use data from the testing session, but assures the tester that
we won't use the data for other purposes. Also included is a clause that
allows us to temporarily track pages visited from their computer, and
something about compensation. (Note: You'd want to print this on
letterhead.)
User
Questionnaire
These questions expand on information gathered during the initial screening.
Questions were chosen to help us understand the test results better. For
example, if someone lives in a particular neighborhood, then they might
be able to navigate the maps in that area better than someone from another
neighborhood.
System
Test
While the interviewer is going through the Script, Release Form, and Questionnaire,
the notetaker conducts a series of calisthenics on the user's computer
to see how well the web site displays and prints on the user's system,
as well as getting basic information about the system specs. At this point,
the computer is also logged in to the cookie-based tracking system so
that clicks through the web site can be analyzed later.(The tracking system
is described in this document, as well.)
Sample
Usability Testing Tasks
Both the interviewer and notetaker have copies of this document on a clipboard.
Each user task is printed (one per page) with notes for the interviewer
about possible prompts and space for the notetaker (and interviewer) to
take notes about user behavior on each task. The interviewer arrives at
the test with a set of index cards, with each written task cut and taped
onto individual note cards.
Final
Report
This document restates the research questions in the Sample Test Plan
and then answers each question, referring to the quantitative click-tracking
data gathered as well as the notes taken as users were thinking aloud.
The report spans two cohorts of usability testing, and describes the site
changes that took place after the first cohort of testers. The second
round of testing was identical to the first, and confirmed the effectiveness
of the initial design changes. Plus, it resulted in new design improvements
and research questions. (Note: This report was written for in-house
use, so contains a lot of organizational jargon. A report to be given
to an outside web designer, or a board, would need to be much more concrete
and complete. Writing the report can be the most time-consuming part of
testing.)
Some good, relevant usability articles on the web
Keeping
Office Politics out of Design
A set of practical tips that includes using results from usability testing
to de-politicize the web design process.
Don't
Test Users, Test Hypotheses
"When testing websites or applications, I've found that generating
hypotheses about user behavior helps inform the observation process, structure
data collection and analysis, and organize findings. It also keeps you
honest by being explicit about what you are looking for."
Myth of the Stupid User articles
The
Myth of the Stupid User
"The "myth
of the stupid users" is common within large sections of the software
and web development communities. This might seem a relatively harmless
belief, but it has a significant effect on the quality of Internet sites,
on the extent to which sites can be used by their intended audience, and
on the extent to which sites meet business and marketing requirements."
Can
We Learn From Stories About Stupid Users?
"Is the user
really a stupid person who is completely ignorant of the things in the
world? No, not really, but at least he or she is seemingly regarded as
such. The stupid user is a very popular myth, an illusion which is often
used to defend stupid programmers or system designers. It is very handy
to be able to put the blame on a person who can be pointed to as acting
funny, knowing nothing, and who cannot even complain on problems in an
appropriate language."
"This
is broken" web site
A new project to
make businesses more aware of their customer experience, and how to fix
it.
General usability web sites on the web
Usability.gov
Sponsored by the National Cancer Institute this website explains why usability
is important, reviews different usability methods, provides guidelines
and checklists, and catalogues other online resources. It also covers
accessibility issues and provides Internet and search engine research
statistics.
Usability
News
This website publishes
recent articles in usability and tracks conferences and events in the
field.
The
Usability Library
A collection of book descriptions on a range of topics from accessibility
to return on investment (ROI) in the field of usability, organized by
category.
Usability
Resources
Usability Resources contains bibliographies, web references, files to
download and other material relating to usability activities.
SURL:
Software Usability Research Laboratory's Newsletter
The SURL team specializes in software/website user interface design research,
human-computer interaction research, and usability testing and research.
They publish the results through their online newsletter.
SURL:
Optimal Web Design
This section of SURL focuses on specific articles in designing and building
useful websites.
References from "Usability Testing of Community Data and Mapping Systems" paper
Bernard, Michael L, 2001. "Criteria for Optimal Web Design: How can I reduce the major user annoyances on my site?" Software Usability Research Laboratory, Wichita State University. Retrieved 9 June 2003: http://psychology.wichita.edu/optimalweb/annoyances.htm
Cartwright, W., Crampton, J., Gartner, G., Miller, S., Mitchell, K., Siekierska, E., and Wood, J., 2001. "Geospatial Information Visualization User Interface Issues." Cartography and Geographic Information Society, Vol. 28, No. 1, January 2001. Retrieved 9 June 2003: http://www.geovista.psu.edu/sites/icavis/agenda/PDF/Cartwright.pdf
Fogg, B.J., Marshall, J., Laraki, O., Osipovich, A., Varma, C., Fang, N., Paul, J., Rangnekar, A., Shon, J., Swani, and P., Treinen, M., 2001. "What Makes Web Sites Credible? A Report on a Large Quantitative Study." Persuasive Technology Lab. Stanford University. Available at www.webcredibility.org
Fogg, B.J., Kameda, T., Boyd, J., Marshall, J., Sethi, R., Sockol, M., and Trowbridge, T. (2002). "Stanford-Makovsky Web Credibility Study 2002: Investigating what makes Web sites credible today." A Research Report by the Stanford Persuasive Technology Lab & Makovsky & Company. Stanford University. Available at www.webcredibility.org
Haklay, M., and Tobon, C., 2002, Usability Engineering and PPGIS: Towards a Learning-improving Cycle, presented at the 1st Annual Public Participation GIS Conference, Rutgers University, New Brunswick, New Jersey, 21st-23rd July.Available at: http://www.casa.ucl.ac.uk/muki/pdf/Haklay-Tobon-URISA-PPGIS.pdf
IBM. "User rights: The customer is always right." IBM Corporation. Retrieved 9 June 2003: http://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/12
Lee, Alfred T, 1999. "Web Usability: A Review of the Research," in The SIGCHI Bulletin. Vol. 31, No.1, January 1999. Edited by Ayman Mukerji. Minneapolis, MN. Pp 38-40. Retrieved 9 June 2003: http://www.acm.org/sigchi/bulletin/1999.1/lee.pdf
Microsoft Corporation, 2000. "UI Guidelines vs. Usability Testing." MSDN Library. Retrieved 9 June 2003: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwui/html/uiguide.asp
Nielsen, Jacob, 1997. "The Use and Misuse of Focus Groups." Alertbox. Retrieved 9 June 2003: http://www.useit.com/papers/focusgroups.html
Nielsen, Jacob, 2003. "Employee Directory Search: Resolving Conflicting Usability Guidelines." Alertbox. Retrieved 9 June 2003: http://www.useit.com/alertbox/20030224.html
Selvidge, Paula, 1999. "How Long is Too Long to Wait for a Website to Load?" Usability News Vol. 1, Issue 2. Retrived 9 June 2003: http://psychology.wichita.edu/surl/usabilitynews/1s/time_delay.htm
Selvidge, Paula, 2003. "Examining Tolerance for Online Delays." Usability News Vol.5, Issue 1. Retrived 9 June 2003: http://psychology.wichita.edu/surl/usabilitynews/51/delaytime.htm
Slocum, Terry A., Blok, C., Jiang, B., Koussoulakou, A., Montello, D.R., Fuhrmann, S., and Hedley, N.R., 2001. "Cognitive and Usability Issues in Geovisualization." Cartography and Geographic Information Society, Vol. 28, No. 1, January 2001. Retrieved 9 June 2003: http://www.geovista.psu.edu/sites/icavis/agenda/PDF/SlocumLong.pdf
Usability.gov. "Methods for Designing Usable Web Sites." Retrieved 9 June 2003: http://www.usability.gov/methods/data_collection.html
Home
Usability
 |
Greater New Orleans Community Data Center
Last modified: April 3, 2006 |
