Criteria for Evaluation of Internet Information Resources

This is a "toolbox" of criteria that enable Internet information sources to be evaluated for use in libraries, e.g. for inclusion in resource guides, and helping users evaluate information found. Comments are welcomed by Alastair Smith.
Other resources providing criteria are listed in the Evaluation of Information Sources section of the Information Quality WWW Virtual Library
Another version of these criteria are at:
Smith, Alastair G. (1997) Testing the Surf: Criteria for Evaluating Internet Information Resources. The Public-Access Computer Systems Review 8, no. 3 http://info.lib.uh.edu/pr/v8/n3/smit8n3.html
________________________________________
Outline
• Scope o Breadth o Depth o Time o Format
• Content o Accuracy o Authority o Currency o Uniqueness o Links made to other resources o Quality of writing
• Graphic and multimedia design
• Purpose o Audience
• Reviews
• Workability o User friendliness o Required computing environment o Searching o Browsability and organisation o Interactivity o Connectivity
• Cost
________________________________________
Scope
What items are included in the resource? What subject area, time period, formats or types of material are covered? Is the scope stated, e.g through meta information such as an introduction, or only implied? Does the actual scope of the resource match expectations? Aspects of the scope include:

Breadth
Are all aspects of the subject covered?

Depth
To what level of detail in the subject does the resource go?

Time
Is the information in the resource limited to certain time periods?

Format
Are certain kinds of Internet resources (for example telnet, Gopher, FTP) excluded?

Content
Is the information factual, or opinion? Does the site contain original information, or simply links? Sites can be useful both as information resources in themselves, and as links to other information. However users can be frustrated by lists of resources which look promising, but turn out to simply contain more links.
Is the resource an integral resource, or has it been abstracted from another source, perhaps losing meaning or links in the process?
Specific aspects related to the content include the accuracy, authority, currency and uniqueness of a resource.

Accuracy
Is the information in the resource accurate? You may wish to check this against other resources, or by checking some information about which you have special knowledge.
Are there political or ideological biases? The Internet has become a prime marketing and advertising tool, and it is advisable to ask "what motivation does the author have for placing this information on the Net". Frequently the answer is that the information is placed to advertise, or support a particular point of view.

Authority
Does the resource have some reputable organisation or expert behind it? Does the author have standing in the field? Are sources of information stated? Is the information verifiable? Can the author be contacted for clarification or to be informed of new information?
Examing the URL can give clues to the authority of a source. For instance a tilde "~" usually indicates a personal web directory, rather than part of the organisation's official web site.

Currency
How frequently is the resource updated, or is it a static resource? Are dates of update stated, and do these correspond to the information in the resource? Does the organisation or person hosting the resource appear to have a commitment to ongoing maintenance and stability of the resource?
Browsers may allow you to view the date of creation and modification of a file (in Netscape View|Document Info). Remember that this may not be the date that the actual information was created or reviewed.

Uniqueness
Is the information in this resource available in other forms (for example other sites, Gopher, WWW, print, CD-ROM)? What advantages does this particular resource have? If the resource is derived from another format, for example print, does it have all the features of the original? Have extra features been added? Does it complement another resource, for instance by providing updates to a print source?
On the Internet, redundancy may be valuable - a particular site may not be available when required, and an alternative or mirror site may have to be used. Also, some users may not be able to access certain types of resource, for example telnet or image-based web sites.

Links made to other resources
If the value of the site lies in its links to other resources, are the links kept up to date, and made to appropriate resources? Are the links made in such a way that it is clear that an external site is being referred to. There are potential copyright issues with sites that, for instance, enclose an external link in frames so that the source of the information is unclear.

Quality of writing
Is the text well written? While hypertext linking and multimedia are important elements of the Web, the bulk of the information content on the Web still lies in text, and quality of writing is important for the content to be communicated clearly.

Graphic and multimedia design
Is the resource interesting to look at? Do the visual effects enhance the resource, distract from the content, or substitute for content? If audio, video, virtual reality modeling, etc are used, are they appropriate to the purpose of the source?
A related criteria to graphic design is navigational design, mentioned below in the context of browsability and organisation.
Purpose
What is the purpose of the resource? Is this clearly stated? Does the resource fulfill the stated purpose?

Audience
Who are the intended users of this resource? At what level is the resource pitched: a subject expert, a layperson, or a school student? Will the resource satisfy the needs of the intended users? Does the user group have the connectivity to access the resource? Does your user group correspond to the intended audience?

Reviews
What do other reviewing services say about the site? The use of reviewing journals has been a mainstay of collection development in print collections; librarians in the Internet environment will need to become familiar with the strengths and weaknesses of the range of sources reviewing Internet resources.

Workability
Is the resource convenient and effective to use? This is the area where criteria for Internet resources differ most from print sources.
An issue in providing access to electronic documents is whether a library should just provide links to the originating site, or "acquire" the publication for local access. Poor workability may indicate that the library should store the data locally, if intellectual property considerations allow this.
Aspects of workability include:

User friendliness
Are any special commands clear? Is help information available? Have user interface issues been addressed, such as menu design, readability of screens, etc.
Required computing environment
Can the resource be accessed with standard equipment and software, or are there special software, password, or network requirements?
Has the resource been designed to work well with one software and user interface (for example the latest Netscape release on a T1 connection) but be difficult to use with others (for example Lynx at 2400bps)? It is useful to test resources with a variety of browsers and connections. Telnet resources may pose problems to users who have not installed a telnet client. Images and other multimedia may create problems if users have not installed the correct viewer.
While the extent to which older browsers are still used is a source of argument, there are still Lynx only users, frames challenged users, visually impaired users out there, and sites should attempt to cater for them. This criteria is less important where users are in a defined computing environment, such as a library's inhouse terminals.

Searching
How effectively can information be retrieved from the resource? Is the resource organised in a logical manner to facilitate the location of resources? Is the organisational scheme appropriate, for example chronological for an historical source, or geographical for a regional resource? Is a useful search engine provided? What operators and ranking features are available? Is the search engine interface intuitive? Does the search engine index the whole resource?

Browsability and organisation
Is the resource organised in a logical manner to facilitate the location of resources? Is the organisational scheme appropriate, for example chronological for an historical source, or geographical for a regional resource?

Interactivity
Where interactive features such as forms, cgi scripts etc are provided, do these work? Do they add value to the site?

Connectivity
Can the resource be accessed with standard equipment and software, or are there special software, password, or network requirements? Can the resource be accessed reliably, or is it frequently overloaded or offline? Is a local mirror site available, or do international traffic charges have to be incurred?

Cost
Currently Internet information resources are perceived as being "free". However costs do exist, and are likely to become more important. Costs can be divided into (a) costs of connecting to the resource and (b) costs associated with the use of the intellectual property contained in the resource.
In terms of (a), users paying traffic charges are already having to consider the costs of connection, and may want include this in criteria for selection, for instance to favour text based rather than image intensive sites, if the image content is the same.
Increasingly we will see sites where (b) is a consideration, and a charge is made for the intellectual content of the site. Of course, libraries have been dealing with charged online services such as Dialog for many years, but the Internet has created an expectation and an opportunity to make charged services available to end users. Libraries have a role in negotiating subscriptions and site licenses for organisational access to charged services. If online transactions are used to pay for information, the security of these transactions at a site may become important. Charged services may be available with limited functionality, or for trial periods, for free; librarians will need to decide whether to provide the enhanced or the limited version.