Educational stakeholders are increasingly engaged in discussions about the effective design, distribution, and evaluation of e-learning. We invite educators to build on already existing scholarship as they make future e-learning decisions. Specifically, we combine four categories of academic scholarship from Boyer (1990) with six assessment criteria from Glassick, Huber, and Maeroff (1997) to construct a matrix that may be applied to any post-secondary learning or teaching context. We argue that while each medium in which faculty might find themselves teaching differs from others, the teaching itself, and effective teaching in general, is definable and, therefore, can be evaluated using the matrix.

INTRODUCTION

A range of educational stakeholders (faculty, programs, departments, institutions, students, and the tax-paying public who support higher education) are increasingly engaged in important discussions about the effective design, distribution, and evaluation of e-learning (e.g., Graham, Cagiltay, Lim, Craner, & Duffy, 2001; Young & Young, 1999). With these discussions in mind, we invite educators to build on already existing scholarship in the field of teaching and learning to guide them as they make e-learning decisions for the future. We feel that while it is important to understand that each medium in which we might find ourselves teaching differs from others (e.g., teaching web-based or computer-mediated classes is very different from teaching classes with interactive video) the teaching itself, and effective teaching in general, is definable and, therefore, can be evaluated.

In our discussion here, for example, we revisit Ernest Boyer’s (1990) four categories of academic scholarship in Scholarship Reconsidered. We then combine Boyer’s categories with the six criteria from Glassick, Huber, and Maeroff’s (1997) Scholarship Assessed to construct a matrix that may be applied to a wide range of postsecondary learning or teaching contexts. Using Laura’s undergraduate teaching experience and her work in faculty development, we then explain and illustrate how the matrix can be used to assist a scholar-teacher as she evaluates and shapes e-learning and teaching for the benefit of diverse stakeholders.

ERNEST BOYER: SCHOLARSHIP RECONSIDERED

Rather than view scholarship as something distinct from service, teaching, and administration, Ernest Boyer (1990) argues we must, for the health of higher education, define scholarship broadly enough to embrace the full range of academic work—work that serves a wide range of stakeholders inside and outside the academy. In Boyer’s (1990) scheme, the scholarly work of the professoriate has four interrelated functions: discovery, integration, application, and teaching. The scholarship of discovery is essentially what academics traditionally consider research to be—investigation for its own sake. The scholarship of integration consists of “making connections across the disciplines, placing the specialties in larger context, illuminating data in a revealing way, often educating nonspecialists, too” (Boyer, 1990, p.18). Boyer’s (1990) third category, the scholarship of application, implies questions such as, “How can knowledge be responsibly applied to consequential problems?” and “Can social problems themselves (Boyer’s emphasis) define an agenda for scholarly investigation?” (p. 21). What’s exciting about Boyer’s questions is that “[n]ew intellectual understandings can arise out of the very act of application. . . . In such activities as these, theory and practice vitally interact, and one renews the other” (p. 23). Boyer’s (1990) fourth category, the scholarship of teaching, describes our crucial work with students and our responsibility to remain lifelong learners. Boyer (1990) reminds us that “good teaching means that faculty, as scholars, are also learners” (p. 24). Among other things, effective teaching—with or without technology—requires that “[p]edagogical procedures must be carefully planned, continuously examined, and relate directly to the subject taught” (Boyer, 1990, pp. 23-24).

GLASSICK, HUBER, AND MAEROFF: SCHOLARSHIP ASSESSED

Extending Boyer’s reconsideration of what it means to be engaged in scholarly work, Charles Glassick, Mary Huber, and Gene Maeroff explicate six criteria for assessing scholarship in their 1997 publication, Scholarship Assessed. After surveying nearly 900 chief academic officers of United States postsecondary institutions, Glassick, Huber, and Maeroff (1997) inductively constructed the six criteria: clear goals, adequate preparation, appropriate methods, significant results, effective presentation, and reflective critique. We specifically apply these criteria to the work of faculty in e-learning environments. To do that, we show how Boyer’s (1990) four categories of scholarship and Glassick, Huber, and Maeroff’s (1997) six criteria for assessing scholarship intersect to form a matrix that faculty can use to reconsider, reassess, and ultimately, shape or reshape e-learning pedagogies (Figure 18.1).

Figure 18.1 Matrix Combining Scholarship Reconsidered and Scholarship AssessedFigure 18.1 Matrix Combining Scholarship Reconsidered and Scholarship AssessedSix Criteria for Assessing Scholarship, Glasick, Huber, Maeroff, 1997.Note: Faculty may use this matrix to guide them as they assess the full range of their scholarly work.

Before looking at the matrix, itself, we feel it is important to recognize the oppositional pull of the two axes. Boyer’s (1990) expansion of the definition of scholarship tends to be open and heuristic in nature. Glassick, Huber, and Maeroff’s (1997) assessment criteria tend to be focused and strategic. By creating a matrix with axes pulling in different directions, we hope faculty and administrators will be able to understand that while e-learning opportunities and pedagogies might open many new doors, we must still respond to the real, concrete need to assess what we do.

Clear Goals

Glassick, Huber, and Maeroff (1997) pose three questions to determine whether the scholar has established clear goals: “Does the scholar state the basic purposes of his or her work clearly? Does the scholar define objectives that are realistic and achievable? Does the scholar identify important questions in the field?” (p. 25). We have recast those questions to account for the decisions that faculty need to make when considering teaching with technology: How can I state the basic purposes of my teaching with technology? What are some realistic and achievable objectives for my teaching with technology? What are some important questions for students to explore with technology? And perhaps most important, is technology even an issue?

In Laura’s business communication course, for example, her main objective is to expand and improve business students’ reading, writing, speaking (or presentational), and computer literacies. In the course syllabus, Laura outlines the specific information and competencies students should expect to acquire throughout the semester by completing learning activities that depend on students’ effective use of technologies such as Blackboard course management software, discussion forums, email messages, Microsoft Word, PowerPoint, and the Internet. In brief, students should be able to 1) identify three basic purposes of business communication, 2) organize information to fit their audiences, purposes, and the situation, 3) construct documents that are clear, complete, correct, visually inviting, save the reader’s time, and build good will by creating a friendly, businesslike, positive style, 4) work effectively in groups by understanding strategies for responding to individual’s differing styles, overcoming barriers to success, resolving conflict, and completing assignments on time, 5) understand the effect of verbal and nonverbal communication in business settings, and 6) make short oral presentations that are clear, well organized, stay within time limits, and demonstrate good audience awareness.

Laura’s six outcomes are fairly standard for a course in business communication. What is different is that she includes computer literacies for students as an area that needs to be expanded and improved upon. What Laura has done is move the technological aspects of the classroom into the mainstream. Being able to effectively communicate with email, research on the web, use PowerPoint as a presentation tool, and the like are skills that need to be refined in the same way as other literacy skills have traditionally been refined. Throughout Laura’s course, she has not included technology as something special or as a mere add-on to course requirements and assignments, but rather, she has integrated technology so that it becomes the norm.

Adequate Preparation

As a result of their survey, Glassick, Huber, and Maeroff (1997) offer the following questions to assess the scholar’s preparation: “Does the scholar show an understanding of existing scholarship in the field? Does the scholar bring the necessary skills to his or her work? Does the scholar bring together the resources necessary to move the project forward?” (p. 27). Again, we have recast these questions: What is my understanding of existing scholarship on teaching with technology? What skills do I bring my teaching with technology? What resources can I use to promote learning, or, more specifically, to promote learning with the aid of technology?

Whether a teacher is a compassionate pioneer of the past or a wary or resistant adopter of the present, all scholar-teachers who choose to enhance their course with some form of technology should at least become familiar with the growing body of research on e-learning. Best practices, guidelines, and research are increasingly available on the Internet and in online, peer-reviewed journals such as The Technology Source, The National Teaching and Learning Forum, The journal of Scholarship of Teaching and Learning, journal on Excellence in College Teaching, and Educational Researcher Online. In addition, teachers can participate in national movements to support excellence in teaching with technology as well as in assisting faculty as they learn to adapt new technologies to promote learning. Such movements and organizations include, for example, the Teaching, Learning, and Technology Group (TLT), Multimedia Educational Resource for Learning and Online Teaching (MERLOT), the Flashlight Project for evaluating teaching with technology, and various grant-funded projects such as Preparing Teachers to Use Technology (PT 3 Grants) to support faculty integration of technology into K–12 and post-secondary programs and courses. Satellite broadcasts on key subjects such as intellectual property, copyright, and effective instructional design using technology are also available to faculty wishing to understand the issues, problems, and solutions to e-learning challenges. At many colleges and universities, inhouse workshops and small-group or one-to-one consultations are also available to faculty who seek support, teaching tips, or guidelines for the effective integration of technology into their courses.

At our institution, for example, three campus units—the center for learning and teaching excellence, distance learning and technology, and instruction technology instruction support—have co-sponsored and designed week-long summer institute courses to assist faculty as they retool and redesign their own courses for fully online or technology-enhanced instruction. Summer institute courses available to all university faculty and other college teachers include, for example, “Enhancing Instruction Through Technology,” “Teaching, Learning, and Assessment in Distance Education,” ‘‘Active Learning with Technology,” “Introduction to Marcromedia Flash,” “Digital Video for Instruction and Research,” and “Teaching, Learning, and Assessment in Distance Education.” Faculty participation in such workshops gives evidence of postsecondary teachers’ growing awareness and desire to prepare adequately for teaching in technology-enhanced environments. Glassick, Huber, and Maeroff’s (1997) particular question about adequate preparation also shows us how important it is to embrace Boyer’s (1990) taxonomy. In other words, once we accept that a scholarship of teaching exists, reviewing the literature, keeping up with current scholarship, and contributing to the scholarly conversation become a norm.

Appropriate Methods

Here, Glassick, Huber, and Maeroff’s (1997) defining questions are straightforward: “Does the scholar use methods appropriate to the goals? Does the scholar apply effectively the methods selected? Does the scholar modify procedures in response to changing circumstances?” (p. 28). Here are our revisions: What methods of teaching with technology can I use to achieve the learning goals for the course? How can I effectively apply the selected e-learning teaching methods? How can I modify my use of technology in response to changing circumstances? These particular questions must be viewed as central to teaching with technology. Too often, newcomers to e-learning tend to see appropriate methods as one whole. They, wrongfully, assume that one pedagogy works for every e-learning situation. To make such an erroneous assumption is equivalent to assuming that instructors who are accustomed to lecturing to 250 students will use that same pedagogy when they teach a seminar to 15 students.

In the undergraduate classes and faculty development workshops that Laura conducts, she works to promote active learning by structuring lessons aided by various technologies. For example, Laura often uses short PowerPoint presentations to facilitate interactive discussions that encourage student/participants’ active involvement using questions, hyperlinks to interactive Internet resources, time for reading and reflection, and engagement or exchange with peers. She also facilitates active/cooperative learning techniques using a “cooperative learning randomizer,” a simple Excel spreadsheet designed to ensure individual accountability and encourage positive interdependence—two important features of effective cooperative learning. The randomizer is designed to be easily modified for both large lecture settings or small classes, depending on the number of assigned groups/teams and the number of student members in those groups/teams. In other words, a teacher can readily and appropriately modify the technology for varied teaching situations. For example, when Laura presented on teaching with technology to a large group of 75 faculty members at the University of Texas, San Antonio, she modified the randomizer so that teams of five faculty members would be randomly called upon as they were seated at 15 round tables. In contrast, when she teaches undergraduate students in computer-mediated classrooms, Laura modifies the randomizer to call individually upon 20 to 25 students working together in a corresponding number of teams, usually five to seven. In a computer-mediated classroom setting, she also prefers to use the cooperative learning technique of write-pair-square by partnering students sitting next to one another at computers and then creating teams composed of four by joining paired partners who are, ideally, located near one another throughout the room.

Significant Results

Pragmatic considerations are the focus of Glassick, Huber, and Maeroff (1997) in posing questions that assess the results of the scholar’s work: “Does the scholar achieve the goals? Does the scholar’s work add consequentially to the field? Does the scholar’s work open additional areas for further exploration?” (p. 29). Our revisions: When using technology, how am I achieving the intended learning goals for students? How is my work with technology adding consequentially to students’ learning and to the field? How is my work with technology opening additional areas for further exploration for students?

An extension of Glassick, Huber, and Maeroff’s (1997) pragmatism here may lead us to the realization that though it may be difficult to assess that any particular technology specifically leads to an increase in student learning (Joy, II & Garcia, 2000; Russell, 1999; Twigg, 2001), to not incorporate certain technologies will unequivocally leave students disadvantaged. When word processing was a new technology, for instance, researchers asked whether word processing made one a better writer. That question became irrelevant once word processing became the “normal” way to write. As other technologies such as email, using the web for reference, and presentation software become the “normal” way to do business, we would be failing our students if we did not incorporate pedagogies using these tools into our curriculum.

Effective Presentation

Glassick, Huber, and Maeroff (1997) also don their rhetorical hats to examine the effectiveness of the scholar’s work: “Does the scholar use a suitable style and effective organization to present his or her work? Does the scholar use appropriate forums for communicating work to its intended audiences? Does the scholar present his or her message with clarity and integrity?” (p. 32). Here are our revisions: How are my style and organization for teaching with technology affecting students’ learning? What e-learning forums are most appropriate for communicating with students and others? With which technologies can I enhance the clarity and integrity of my communication with students and others?

Laura works to use technology to facilitate interactive, rather than passive, learning and to take advantage of its ability to appeal to diverse learning styles. She increases interaction and aids students who learn in a variety of ways through I) well-designed PowerPoint presentations that encourage interaction between herself as a presenter and her students; such interactive presentations presumably appeal to students who learn best through both aural and visual means, 2) online discussions designed to stimulate deeper level thinking and broader participation by a majority of students, and where some students who may have traditionally felt reticent in class or some how silenced by the larger group, now feel comfortable to be active participants, and 3) interactive web sites or simulations that enable students to more readily engage with course content or ideas. Although little research has been done to prove it, the web has the potential to offer students the rare opportunity of appealing to all three learning styles. The use of text in a web site, for example, likely appeals to aural learners, its use of graphics to visual learners, and its necessary use of the mouse and “point and click” interface engages kinesthetic learners. Furthermore, while significant barriers to web accessibility are ongoing and legitimate concerns for users with a variety of physical or economic challenges, the Internet has also opened up possibilities for their access to higher education previously unimagined or unavailable (Schmetzke, 2001; U.S. Department of Commerce, 2000).

Reflective Critique

The last criterion in Glassick, Huber, and Maeroff’s (1997) list implies the following questions: “Does the scholar critically evaluate his or her own work? Does the scholar bring an appropriate breadth of evidence to his or her critique? Does the scholar use evaluation to improve the quality of future work?” (p. 34). Our questions are: How can I critically evaluate my teaching with (or without) technology? What breadth of evidence do I need to bring to my critique? How can I best use evaluation to improve the quality of future teaching with (or without) technology?

Teachers can initiate formative and summative evaluations of their teaching with technology in a variety of ways. As in any course, they may begin by gathering information from students’ perspectives using student evaluations or surveys that ask specific questions about students’ experience in a web-enhanced course or program. Classes supported by course management software such as Blackboard conveniently allow Laura to construct online surveys that invite students to offer feedback about their learning experience. Blackboard’s online survey capability also provides Laura the ability to survey faculty members about their technology skills, level of comfort, and opinions with regard to technology-enhanced learning before and after they participate in summer institute courses. This information helps shape and reshape the ways Laura uses technology to enhance learning in faculty development workshops. Following a workshop or activity, she also uses more conventional assessments like the Plus/Delta Classroom Assessment Technique (Angelo & Cross, 1993) to discover what participants thought worked well and what they would suggest she modify about the workshop or activity for future participants. In addition to gaining information from students or faculty participants’ perspectives, Laura relies on her own experience during the lesson, writing notes to herself on her outlined lesson plans about what worked, what did not, and what she will revise for the future. As a scholar-teacher, she also works to discuss and write about her methods for teaching with technology in order to share what she has learned and receive feedback about her methods. This chapter itself is an example of three scholar-teachers making their work public and available for critique. By publishing and distributing the matrix that combines the work of Boyer (1990) and Glassick, Huber, and Maeroff (1997), we invite other scholar-teachers to investigate, replicate, and/or adapt our work in teaching, and, more specifically, in teaching with technology.

We close by returning to the matrix that we have now modified into three additional matrices that focus faculty attention on only one of Boyer’s (1990) four scholarly categories: teaching (with technology). The first of three modified versions of the matrix includes a blank matrix that faculty may fill out and use to assist them as they make the transition from teaching in nondigital to digital environments (Figure 18.2). The second is a blank matrix that faculty may fill out and use to guide them as they work to ensure quality learning and teaching with or without technology (Figure 18.3). The third is an example matrix that Laura filled out to guide her as she developed and conducted a day-long workshop on teaching with technology for 75 cross-disciplinary faculty members at the University of Texas, San Antonio, on May 15, 2001 (Figure 18.4).

Figure 18.2 Transitioning MatrixFigure 18.2 Transitioning MatrixNote: Faculty may use this matrix to help them make the transition from teaching in nondigital to digital environments.
Figure 18.3 Blank Matrix Focused on the Scholarship of TeachingFigure 18.3 Blank Matrix Focused on the Scholarship of TeachingSix Criteria for Assessing Scholarship, Glassick, Hueber, Maeroff, 1997.Note: Faculty may use this matrix to assess their teaching with or without technology.
Figure 18.4 Completed Matrix Faculty Development Workshop, May 15, 2001Figure 18.4 Completed Matrix Faculty Development Workshop, May 15, 2001Six Criteria for Assessing Scholarship, Glassick, Huber, Maeroff, 1997.

CONCLUSION

The effective design and implementation of technology-enhanced courses, curricula, and programs are two of the most compelling and pervasive forces shaping teaching and learning in the 21st century. We suggest that faculty would do well to consult already existing scholarship on teaching and learning—such as that articulated by Boyer (1990) and Glassick, Huber, and Maeroff (1997)—to assist them as they make pedagogical decisions for the future of e-learning.

REFERENCES

  • Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd ed.). San Francisco, CA: Jossey-Bass.
  • Boyer, E. (1990). Scholarship reconsidered: Priorities of the professoriate. Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching.
  • Glassick, C., Huber, M., & Maeroff, G. (1997). Scholarship assessed: Evaluation of the professoriate. San Francisco, CA: Jossey-Bass.
  • Graham, C., Cagiltay, K., Lim, B., Craner, J., & Duffy, T. M. (2001). Seven principles of effective teaching: A practical lens for evaluating online courses. The Technology Source. Retrieved December 10, 2001, from http://horizon.unc.edu/TS/dcfault.asp?show=article&id=839
  • Joy II, E. H., & Garcia, F. E. (2000). Measuring learning effectiveness: A new look at no-significant-difference findings. Journal of Asynchronous Learning Networks, 4 (I). Retrieved December 11, 2001, from http://www.aln.org/alnweb/journal/jaln-vol4issuel.htm
  • Russell, T. L. (1999). No significant difference phenomenon. Raleigh, NC: North Carolina State University Press.
  • Schmetzke, A. (2001). Online distance education: “Anytime, anywhere” but not for everyone. Information Technology and Disabilities, 7 (2). Retrieved December 10, 2001, from http://www.rit.edu/~easi/itd/itdv07n2/axel.htm
  • Twigg, C. (2001). Innovations in online learning: Moving beyond no significant difference. Troy, NY: Center for Academic Transformation Rensselaer Polytechnic Institute. Retrieved December 11, 2001, from http://www.center.rpi.edu/PewSym/mono4.html
  • U.S. Department of Commerce. (2000). Falling through the net: Toward digital inclusion. Washington, DC: U.S. Government Printing Office. Retrieved December 10, 2001, http://www.ntia.doc.gov/ntiahome/fttn00/contents00.html
  • Young, C., & Young, L. (1999). Assessing learning in interactive courses. Journal on Excellence in College Teaching, 10 (1), 63-76. Retrieved December 10, 2001, from http://ject.lib.muohio.edu/contents/arcicle.php?article=186

Contact:

  • Laura Bush

  • Center for Learning and Teaching Excellence

  • Arizona State University

  • PO Box 870101

  • Tempe, AZ 85287-0101

  • Voice (480) 965-5620

  • Fax (480) 727-7068

  • Email [email protected]

  • Barry Maid

  • East College

  • Arizona State University East

  • 7001 E. Williams Field Road

  • Mesa, AZ 85212

  • Voice (480) 727-1190

  • Fax (480) 727-1777

  • Email [email protected]

  • Duane Roen

  • Center for Learning and Teaching Excellence

  • Arizona State University

  • Tempe, AZ 85287-0101

  • Voice (480) 965-3555

  • Fax (480) 727-7068

  • Email [email protected]

Laura Bush is an Instructional Professional at Arizona State University’s Center for Learning and Teaching Excellence (CLTE). In addition to teaching the undergraduate course Business Communications, she regularly facilitates faculty workshops in active learning and the use of computers and the Internet to aid student learning. She also designed the CLTE and Wakonse web sites and now oversees their maintenance and further development. Before earning her PhD in English with an emphasis in American literature, autobiography theory, and computer-mediated writing classrooms, Laura taught composition and literature for five years as a faculty member in the Department of English at Ricks College in Rexburg, Idaho.

Barry Maid is Professor and Head Faculty of Technical Communication at Arizona State University East (ASU) where he led the development of a new program in Multimedia Writing and Technical Communication. Before coming to ASU, he taught at the University of Arkansas, Little Rock, where, among other duties, he directed the Writing Center and the First Year Composition Program, chaired the Department of English, and helped in the creation of the Department of Rhetoric and Writing. Though most of his time is now spent building his second independent writing department, he tries to keep in touch with his professional interests of computers and writing, writing program administration, and academic/industry partnerships.

Duane Roen, Professor of English, currently directs the Center for Learning and Teaching Excellence at Arizona State University. Previously, he directed the Composition Program there, as well as the Writing Program at Syracuse University. Serving as Coordinator of Graduate Studies in English at the University of Arizona from 1990 to 1992, he also worked as Director of Rhetoric, Composition, and the Teaching of English from 1988 to 1992. He has published six books and has written more than 130 articles, chapters, and conference papers—mostly on various aspects of teaching and learning.