Authority Levels

At the Journal of the Association for History and Computing (JAHC), we have long searched for a means to indicate the level of authority of the electronic materials we post. One of the greatest problems facing the wider use of the Internet is the difficulty of assessing the value of electronic documents. This is sometimes referred to as an issue of "trust" or "authority," the term we prefer. This issue is part of a larger problem affecting the Internet: the increasing difficulty of avoiding a wide range of evils, including Spam, Phishing (phoney web sites intended to elicit financial data, passwords, etc.), and various sorts of financial scams. At the Berglund Center for Internet Studies at Pacific University, where the JAHC is housed, we intend to work at developing standard practices that might assist users of the Internet in making better use of it. We begin with the issue of how best to assess and signal the relative authority of the documents that we ourselves publish.

In our offices we house the servers of several well-established electronic centers which together have well over five million hits per year. We publish several journals of significance and influence, such as the JAHC, Interface on the Internet, and E-AsPac, the electronic journal of the scholarly association, Asian Studies on the Pacific. These materials are usually produced by professionals in the relevant fields, and are frequently peer-reviewed. As such, they have the highest possible level of trust or authority. However, we also publish many projects done by graduate and undergraduate students. Between these two poles we publish conference papers and occasional papers done by members of our audiences. These materials, then, are created for different purposes and to different review standards.

We know from traffic analysis, however, that our audience often selects these materials indiscriminately, based more on their immediate research needs than on the reliability or authority of the materials. In an effort to further educate our audiences, and out of our desire to help establish standards and practices for electronic materials, we are considering "stamping" our materials posted from fall of 2004 with seals indicating their Authority Level.

By Authority Level we do not mean to indicate "good" and "bad" but rather authoritative and non-authoritative, or those with higher levels of trust and those with lower levels. This distinction is an important one. Many influential works have been created by writers who did not undergo peer-review, did not possess a terminal degree in the field, and who wrote badly with scant attention to scholarly niceties. However, such materials should best be judged by other professionals, or by the test of proving useful or provocative over time. They cannot be completely trusted at the first reading; they do not speak with great authority.

By applying a seal attesting to Authority Level, we would be stating that insofar as we can determine, the reader can "trust" the conclusions of the pieces posted with the indicated level of confidence. In short, pieces at lower levels may be excellent, but we can't determine that they are, and the reader should have a lower initial level of confidence in them and approach them with more critical attention.

These levels are explained below, but it should be noted that invariably some materials are in grey areas between categories.

Authority Level 5

Level 5 materials have to pass a number of tests and represent the highest possible level of trust or authority. They are equivalent to materials published in paper form by professional journals or presses. They could be placed in a library and should prove useful for some length of time.

They have the following characteristics, but no piece, of course, will necessarily have all of them:

• Materials certified as Authority Level 5 have been reviewed by scholars in the appropriate field following practices long utilized in professional journals and other refereed publications. That is, they have been read closely by one or more established professionals comfortable with the topic area of the piece. The review process was "double-blind." That is, neither the authors nor the reviewers were aware of each other's identities.
• In the judgments of these readers, the piece makes a contribution to the topic being discussed; that is, they are not merely a restatement of existing opinions.
• The materials usually have citations and bibliographies sufficient to permit readers to retrace the author's research steps so as to form their own opinion as to the strength and weaknesses of the pieces when measured against the set of evidence on which they were constructed.
• The author of the piece is known, can be contacted to discuss or defend his or her positions, and themselves have some specialized education, training, or experience relative to their topic.
• The piece is well written and organized and has a minimum of spelling, grammatical or formatting errors, showing the qualities of mind necessary to good research, thought, and writing.
• Limitations of Level 5 materials: Despite these strict standards, even the best research and writing has areas of relative weakness, and professionals in the field will often disagree about each others' positions. The more "cutting edge" a piece, the more likely it is to be controversial. For the average reader, however, these controversies are likely to be unimportant ones.

Authority Level 4

Materials certified as Level 4 are not different so much in quality as in degree from Level 5 materials. They too are highly trustworthy, but they have followed less complex paths to publication. They are analagous to materials published in important highly edited popular sources such as The New York Times. They might have the following characteristics:

• The level of authority of the writer may or may not be the highest possible in the field of study, but the process of review is usually less thorough than those found at Level 5.
• The pieces have been read by somebody with some expertise in the field and in effect are certified as free from egregious errors of fact or important omissions.
• The authors are known, can be contacted, and do have some expertise in the field about which they are writing.
• The materials used in the research are obvious if not always strictly cited. A reader should be able, then, to roughly reconstruct the author's research paths.
• Limitations of Level 4 materials: These materials can be used with confidence, but it may well be that there are more authoritative materials to be found on these same topics.

Authority Level 3

Level 3 materials are often the result of research projects done by inexperienced authors, but guided by experienced ones. These might include extensive undergraduate research projects such as senior theses. They have the following characteristics:

• Level 3 materials should be free from errors of fact. If some of their conclusions are controversial ones, these should be held by at least some authorities in the appropriate field.
• Level 3 materials should depend to a considerable extent upon published and easily available materials so that readers can themselves explore the topic further with some confidence.
• The basis of conclusions drawn in level 3 materials should be clear and rooted in research, not in mere opinion or prejudice.
• Level 3 authors should be known, and can be contacted to discuss their works.
• The works are dated as to time of origin and any updates that occurred,
• Limitations of level 3 materials: Level 3 materials may provide an adequate beginning for serious research, but should not be the sole foundation of such research. They might well provide a good orientation to the topic and suggest additional research paths, but this is their major strength.

Authority Level 2

Authority Level 2 materials have serious limitations and it is our intention not to publish such materials. For our purposes, however, they might be known by the following characteristics:

• Writing is poorly organized, there are spelling and grammatical errors, all suggestive of superficial thinking and research.
• Materials are inadequately cited and there is no useful bibliography so that it is impossible to retrace the author's steps.
• The nature of the evidence relied upon is not clear.
• The author in no way relies upon any obvious authoritative sources or materials.
• The author is not known or cannot be contacted.
• The materials are not dated.
• Utility of Level 2 materials: It may be that serious scholars or researchers could utilize such materials, but most often as the raw material of their own studies. For example, scholars studying popular culture might find blog entries on current films or music or politics useful but would be unlikely to cite them to support their own positions.

Authority Level 1

Authority Level 1 materials: Again, we will never post such materials, but for analytical purposes we would define them as having the following characteristics:

• A hypothetical "reasonable reader" would find them indicative of non-logical thought processes.
• Writing would be confusing or ambiguous.
• Organization would be so poor as to leave the reader lost and confused; we do not know why paragraphs are arranged as they are.
• There would be no indication of author and no attempt to establish authority.
• Utility of Level 1 materials: Like Level 2 materials, they might be useful for some scholarly projects, but not to convey information or analysis.

Please note that the standards for Level of Trust for posted pieces vary from those appropriate to electronic sites or clusters per se. For a discussion of the qualities of "good" sites see: http://mcel.pacificu.edu/JAHC/FEATURES/EPEEF.HTML

We will not always find the use of seals designating levels of authority to be appropriate. Many of our pieces, such as editorials like this one, as well as book and software reviews, are primarily opinion-based and we will not designate these as possessing any particular level of authority. Their authority ultimately rests upon your judgements as the reader and the reception of the materials and their authors over time.

As stated above, we are considering adopting these practices. We would be happy to hear from readers with other ideas or those suggesting alterations in these standards. Please contact Jeffrey Barlow at [email protected]. In the subject line, please place "JAHC authority levels" so that the message will receive prompt attention.