The Evaluation of College Teaching

There are few tasks at a university more important than the evaluation of teaching. Without it, professors themselves are unable to determine the direction of needed improvement and thereby become vulnerable to the process of stagnation. Without it, academic units are unable to identify and encourage professors who truly are effective in the classroom with their students.

Yet, despite the importance of this activity, academic organizations find themselves still struggling to find a satisfactory approach to this problem. This concern led the Faculty Senate at the University of Oklahoma to constitute a committee in February 1983, with the charge of reviewing the methods currently used to evaluate teaching on this campus and, if necessary, to propose an alternative system of collecting information that would provide a better base for personnel decisions and for the improvement of teaching.

Several years ago, as directed by the state regents, the University of Oklahoma mandated that all courses would be evaluated by students. Although it was not mandated how these evaluations would be used, academic units have come over the years to rely heavily on these student evaluations when they evaluate the teaching of the faculty. After studying the problem at length, the Faculty Senate committee eventually came to the conclusion that a better system of evaluation would require two fundamental adjustments. The first is the need to examine multiple dimensions of teaching, something more than just what the teacher does in the classroom. The second is the need for multiple sources of information, something more than an exclusive reliance on student evaluations of their teachers.

The following report describes the reasons for believing that these two principles are fundamental to effective evaluation; it also presents some guidelines for academic units that wish to establish evaluation procedures that incorporate multiple dimensions of teaching and multiple sources of information.

The Nature of Teaching. Before evaluating teaching, one must develop a clear concept of that which is to be evaluated. For purposes of evaluation, teaching can be defined as:

1. “…helping someone else learn something.”

   • To advance this one step further, good teaching can be defined as:

2. “...being effective in the process of helping someone else learn something significant.”

The two added elements of effectiveness and significance both seem necessary to warrant the label of “good teaching.”

The act of teaching can also be viewed as an interactive process that involves a teacher and students. This interaction occurs within a context or environment that can influence the success of that interaction. This is illustrated diagrammatically in Figure 1 below.

Figure 1 An Interactive Model of Teaching and Learning

This definition of good teaching and the interactive character of teaching have a number of implications for evaluation.

1. For purposes of evaluation, the primary purpose of teaching is to generate as much significant learning as possible. Students and teachers may bring additional purposes to the classroom but, for purposes of evaluation, the main concern is the amount of significant learning generated.

2. The teacher is an important but indirect factor in the process of learning. This is simply a recognition of the fact that it is the student who does the learning; the teacher’s role is to help the student in whatever ways possible.

3. In higher education, the teacher has primary responsibility for key decisions about a course. These decisions include such things as determining the scope of a course, identifying the educational goals, selecting reading materials, constructing tests, and assigning grades.

4. The quality of the teacher’s classroom behavior also has a major effect on the students’ reaction to the course on a day-to-day basis. This refers to characteristics such as the clarity of their explanations, the enthusiasm they show for the subject, the rapport they develop with students, and the degree to which they are organized and prepared for class on a regular basis.

5. Teaching takes place within several kinds of contexts, all of which can have a significant influence on the quality of the teaching and learning. Examples include the following:

   • a) Physical - the characteristics of a classroom and the time at which a course is scheduled.

   • b) Social - the relationship between the teacher and the students is an interactive one; students can inspire or discourage the teacher (and vice versa).

   • c) Institutional - the attitude and actions of the department and the larger institution; do their attitudes and actions encourage or discourage good teaching?

   • d) Personal- the situation of the teacher’s non-professional life; has there been an illness, divorce, or financial problems? In summary, teaching can be viewed as an interactive process that takes place within several types of contexts for the purpose of generating as much significant learning as possible.

The Nature of Evaluation. The type of evaluation appropriate for use in higher education is four dimensional. It calls for an examination of the input, the process, the product, and the context of an event or action. When this general framework is applied to the specific situation of college courses, it results in the five items identified in Figure 2 shown below.

The first dimension of teaching evaluation is the teacher’s input, his or her knowledge of the subject matter. A teacher who is up-to-date in his or her field and has undertaken the required research and preparation for a class provides the input necessary for significant learning. It follows that the breakdown of this input component diminishes the learning process.

Figure 2 Five Components of Teaching

The second dimension of evaluation, the “process,” involves two separate activities in college teaching: course decisions and classroom behavior. When a professor teaches a course, he or she makes decisions about the scope of the subject matter to be covered, the teaching strategy to be used, the grading system, course policies, etc. In this activity, teachers need to give consideration to a variety of factors (the nature of the curriculum, the characteristics of the students, etc.) and design the course accordingly. Also part of the “process” of teaching, but quite different in nature, is what a professor does in the classroom. Once the basic course decisions have been made and the professor steps into the classroom, he or she must engage whatever communication and interaction skills they have to deliver lectures, lead discussions, ask questions, motivate students, and generate interest.

The third dimension is concerned with the “product.” In college teaching, this is the amount and type of learning that occurs in a given course.

The fourth dimension is context. In college teaching, there are several types of contexts that affect the quality of a given case of teaching: physical, (e.g., the characteristics of the classroom), social (e.g., the nature of the students), institutional (e.g., the support given to teaching), and personal (e.g., other events in the life of the teacher).

What then are the questions that have to be answered in order to make confident and valid judgments about the quality of teaching? The five general questions and related sub-points shown below seem applicable to all classroom teaching in a university setting. The manner in which answers are found to these questions will vary from department to department and from college to college, but the questions themselves are inherent in the nature of teaching and in the nature of evaluation.

• I. Does the teacher have adequate and up-to-date knowledge of the subject matter?

  • Academic and/or practical experience

  • Efforts to improve

• II. How good were the teacher’s decisions about the course?

  • Goals

  • Teaching strategy

  • Reading/laboratory/homework assignments

  • Testing

  • Course grading

• III. How well did the teacher’s classroom behavior promote good learning?

  • Organization and clarity

  • Enthusiasm

  • Interaction with the class as a whole

  • Relationships with individual students

  • Methods and techniques (implementation skill)

• IV. How good were educational results of the course?

  • Amount of learning

  • Significance of what was learned

  • Attitude towards learning more about the subject

• V. How much was the quality of the teaching and learning influenced by contextual factors?

  • Physical context

  • Social context

  • Institutional context

  • Personal context

Evaluating the Quality of Teaching. The quality of teaching, therefore, can be conceptualized as consisting of five components: the teacher’s knowledge of the course subject matter, the teacher’s course decisions, the teacher’s classroom behavior, the amount of significant learning, and the influence of contextual factors. In order to effectively evaluate any particular instance of teaching, one must engage in the task of collecting and analyzing information about each one of these components.

No single source of information is adequate for assessing all five components of teaching. This means that multiple sources of information are not only advisable but are in fact necessary. Therefore, different information sources need to be assessed to determine their relative value for answering questions about each of the five components.

To this end, the two-dimensional table shown in Figure 3 can be useful. This table identifies six basic sources of information: course materials, three types of students, the teacher, the teacher’s peers, administrators, and instructional consultants. The table can then be used to decide which source or combination of sources would be best for examining each component of teaching.

However a separate chart should be used for each of the three evaluation situations common in academic settings: annual personnel decisions, periodic personnel decisions (e.g., promotion and tenure), and faculty self-improvement. These three situations have some degree of similarity, but the differences are sufficient to warrant separate consideration. Suggested sources of information for each of these evaluation situations are discussed on the following pages.

Annual Decisions. All academic units in a college or university evaluate the faculty members in that unit annually. The “evaluator,” usually the chairperson or some kind of executive committee, must discern how well each faculty member taught that year compared with others in the unit. For annual decisions about the quality of that faculty member’s teaching, the use of five of the eight possible sources of information is suggested, including course materials, present (currently enrolled) students, the individual teacher, peers, and applicable administrators.

The use of senior students (e.g., exit surveys) and alumni sources is excluded on practical grounds. It would be difficult, if not impossible, for an executive committee to collect information annually from these students or former students and to use that information in evaluating every faculty member every year. We additionally recommend not using instructional consultants for personnel decisions, annual or periodic. Most consultants believe that involvement in personnel decisions would interfere with faculty readiness to contact them for diagnostic evaluation intended for self-improvement.

Special note should be taken of the possibility of obtaining information from faculty members about their own courses. Although professors—like students—are present at essentially all of their own classes, academic units do not routinely ask them for information about their own courses. This could be done by using a simple, one-page questionnaire such as that shown in Figure 4. By filling out one of these for each course, professors could comment on such things as the quality of the students, the effect of the classroom or the scheduled hour of the course, etc. This would be very useful information for anyone trying to assess the quality of a particular professor’s teaching that year.(*)

The following list (also illustrated in Fig. 3)* summarizes the recommended sources of information for the evaluation of teaching in annual personnel decisions:

1. Teacher’s Knowledge: Teacher’s comments (personal experience, formal training, professional reading, research, conferences, continuing education), peers and administrators, and course materials will be used.

2. Course Decisions: Course materials (course syllabi, textbooks or textbook list, handouts, exams) will be the primary source. These may be supplemented with student evaluations, teacher’s comments, and peer and administrator comment.

3. Classroom Behavior: Student evaluations will be the primary source. These may be supplemented with teacher’s comments, and peer and administrator comments (classroom visits, informal comments, etc.).

4. Learning: Selected course materials (graded exams, papers) will be the primary source. Student evaluations might also be used with questions such as: Did the student learn the subject matter? Did the student achieve the stated goals of the class? Additional sources could include teacher’s comments, and comments of peers and administrators.

5. Contextual Factors: Present students, the individual teacher, and comments from peer and administrators will be used.

Periodic Decisions. All University faculty and administrators realize the importance of periodic personnel decisions. In such cases, evaluators must decide whether a faculty member’s teaching during the applicable period was sufficient to warrant academic tenure, a promotion, or a teaching award. When academic units assess teaching as part of a tenure or promotion decision, they must call upon all possible sources for credible information. In addition to the sources used for annual personnel decisions, senior students and course alumni can be contacted. The use of instructional consultants is excluded for the same reason as noted earlier: most consultants prefer to restrict their role to the evaluation of teaching for self-improvement.

The following list summarizes the recommended sources of information for evaluation of teaching in periodic personnel decisions.

Figure 3 Evaluation of Teaching Annual Personnel Decisions

Figure 4*

1. Teacher’s Knowledge: Teacher’s comments (personal experience, formal training, professional reading, research, conferences, continuing education), peers and administrators, and course materials will be used as primary sources. They may be supplemented with comments from alumni students and outside peer evaluators.

2. Course Decisions: Course materials (course syllabi, textbooks or textbook lists, handouts, exams) will be used as the primary source. They may be supplemented with student evaluations, exit/alumni surveys, teacher’s comments, and peer and administrator comments.

3. Classroom Behavior: A summary of student evaluations during the pertinent period of time will be the primary source, supplemented with exit/alumni surveys, teacher’s comments, and peer and administrator comments.

4. Learning: Selected course materials (graded exams and papers) will be the primary source. Student evaluations (with questions such as: Did the student learn the subject matter?/Did the student achieve the stated goals of the class?), exit/alumni surveys (effective learning, did course help professionally, etc.), teacher’s comments, and comments of peers and administrators will also be used.

5. Contextual Factors: Present student, teacher’s comments, and comments from peers and administrators will be used.

Self-Improvement. The university and each academic unit bears the responsibility of helping each faculty member develop his or her professional skills. All teaching faculty should be interested in what they can do to improve their teaching. Both the university and the academic unit can provide resources and information for faculty seeking to improve their teaching effectiveness. The university can probably contribute most by supporting an instructional development program. The academic units, through the office of the chairperson, need to inform their faculty of the availability of support services, and to encourage their use. Faculty members themselves need to use whatever resources are available to better understand and improve their teaching. Possible resources include present students, peers, administrators, and instructional consultants. Of these, the consultant—if available—can be a very important resource by providing informed, personalized feedback as well as general information about teaching and learning.

Conclusions. The aim of this report has been to develop a system for the evaluation of teaching that goes beyond the current emphasis on student evaluations of teaching, and to recommend specific procedures for use by academic units and by faculty members themselves. The specific recommendations in this report are offered as a basis for discussion, not as a package that must be adopted or rejected in toto. However, the underlying principles of multiple dimensions of teaching and multiple sources of information are seen as applicable to all evaluation situations. The question then becomes one of how these principles can best be applied to the teaching situations in a particular department or college. If this can be done, the evaluation of teaching is likely to be significantly improved.