9 Moving From the Scholarship of Teaching and Learning to Educational Research: An Example From Engineering

The late Ernest Boyer (1990) introduced the scholarship of teaching as one of four interdependent dimensions of scholarship, with the scholarships of discovery, integration, and application rounding out the quartet. In the years since this work was published, the scholarship of teaching (now more commonly called the scholarship of teaching and learning, or SoTL) has taken hold, and the promotion of SoTL is often a major activity of faculty development centers (Sorcinelli, Austin, Eddy, & Beach, 2006).

Involvement in SoTL usually begins with faculty’s interest in how students in their own classrooms are learning (Huber & Hutchings, 2005), and the purpose of SoTL is to improve learning by improving teaching (Boyer, 1990). Thus SoTL tends to be very personal and situated in one person’s classroom. The very personal nature of SoTL might lead to context-specific results that could be difficult to generalize and apply to broader settings. In some disciplines, this may diminish the perceived impact or significance of results.

Recently, there have been calls for increasing the impact of SoTL results. Faculty have been urged to “go meta” with their studies and to look at broader questions of how students learn that go beyond the specifics of their individual classrooms (Hutchings & Shulman, 1999; Schroeder, 2005). But what does “going meta” really mean? And what models can we provide to faculty to help them do this?

In engineering education, as in most disciplines, the majority of studies to this point have been classroom and curriculum focused. Several factors now point to the readiness of the engineering discipline to move from SoTL into the realm of engineering education research (Gabriele, 2005). Colleges of engineering have recently created new engineering education departments (Haghighi, 2005), and the premiere American journal in this field, the Journal of Engineering Education, has developed more stringent publication criteria (Felder, Sheppard, & Smith, 2005). In order to support more rigorous studies in engineering education, the Center for the Advancement of Scholarship in Engineering Education was founded by the National Academy of Engineering, and the American Society for Engineering Education will sponsor a year of dialogue about scholarship in engineering education in 2006.

This chapter describes a program in engineering education that may be useful both as a mechanism to further the discussion of SoTL, and as a model that could be applicable to disciplines other than engineering.

Conducting Rigorous Research in Engineering Education

Assessment of the 2004 and 2005 Workshops

The RREE grant was assessed every year using multiple strategies. The first strategy was a survey of the participations’ satisfaction with various aspects of the five-day summer workshop. Table 9.2 displays the questions as well as the results from the 2004 and 2005 cohorts. While the average scores are quite similar overall, they differ appreciably on the goal attainment items that reflect the different foci of the two years. The 2004 engineers appraised their mastery of learning principles more highly than did the 2005 group, and this latter cohort considered their understanding of educational research methods stronger than did the former group.

The second assessment strategy was a pre-participation and post-participation survey of perceived knowledge gains. Table 9.3 shows the pre-and post-differences for the 2004 and 2005 cohorts, and they too reflect the change in emphasis from year to year. On most of the content familiarity items, the 2005 cohort reported greater gains, but not on most of the “more specific content knowledge” items, which the 2004 group learned just from acquiring familiarity with the literature. The results on the final question, which ask about one’s comfort level designing education research, predictably favored the 2005 group.

An analysis of participant research journals, the third assessment strategy, revealed a shift from teaching to research issues from 2004 to 2005, and about a quarter of the 2005 entries addressed topics at the more rigorous, research-oriented end of the continuum shown earlier in Table 9.1.

The richest data, however, emerged from the fourth assessment strategy: evaluator observations of the 2005 workshop group discussions. How participants understood the generalizability of research studies reflected their appreciation of the distinction between SoTL and rigorous educational research. On the first day of the workshop they had disagreements, with some engineers contending that if research couldn’t be generalized, it couldn’t be good. Reflecting her discipline’s approach to experimentation, one participant asked her discussion group, “If you do something in your classroom, isn’t it automatically generalizable?”

The following day, several individuals indicated understanding and acceptance of generalizability as an important goal of rigorous educational research. When the groups were asked to identify the characteristics of a good research question, they listed attributes including generalizability, “universal significance,” contribution to society, and the ability to generate more questions. The facilitator then asked the groups to clarify the meaning of significance. They volunteered interpretations such as personal significance, passion, publication, and relevance to “something bigger.”

By the end of the workshop, participants were still considering generalizability. When the facilitator introduced qualitative research methods, one participant asked how the focus on understanding a specific setting related to the need for generalizability stressed earlier in the week. During the final poster presentations, another participant explained his motivation for involving participants from three universities in his research project. He explained that, in his case, the small class sizes at each of the institutions limited the potential generalizability of his research, but combining studies would make his results more applicable to a variety of settings.

The participants also displayed resistance to the idea of choosing a theoretical framework and measurements—again, issues that engineers do not have to grapple with in their disciplinary research.

Conclusion

It may be useful to think of faculty participation in the teaching and learning process as a continuum with excellent teaching at one end and rigorous educational research on the other. Additionally, faculty developers may want to think about how to prepare interested faculty to venture into the realm of educational research, perhaps to “go meta” with their modest SoTL research.

When developing programs to help faculty move toward educational research, we should keep in mind the paradigm differences between disciplinary research and educational research. Facilitating paradigm shifts may be as (or more) important to making the transition to educational researcher as is obtaining the requisite knowledge and skills. Our experience with the RREE project has shown that paradigm shifts are difficult for faculty to make, and it takes time to understand the design and decision-making steps involved in educational research that may be unnecessary in other disciplines. Therefore, programs that prepare faculty to make this transition need to be long term. A few hours or a few days is too short a time period for faculty to assimilate these changes.

Lastly, we hope to spur discussions about new directions for SoTL. While respecting the value of the personal studies usually conducted in SoTL, we suggest that some faculty may be interested in conducting research that can yield findings useful to educational or learning theory. This kind of work can involve truly interdisciplinary collaborations, with disciplinary experts informing the work of learning scientists, and learning scientists informing the work of disciplinary experts.

Note

We wish to thank the National Science Foundation for supporting this work through grant number DUE–0341127, which funds Conducting Rigorous Research in Engineering Education: Creating a Community of Practice, also called the RREE project. We also thank Dr. Norman Fortenberry, of the National Academy of Engineering’s Center for the Advancement of Scholarship in Engineering Education, for his partnership in this project. Special thanks goes to the RREE Executive Committee who jointly developed the “SoTL to educational research” model at the June 11, 2005, executive committee meeting. In addition to authors Streveler and Smith, the RREE Executive Committee members who were part of this discussion are Robin Adams, Nancy Chism, George B. Forsythe, Frank Huband, Marcia Mentkowksi, Ron Miller, and Marilla Svinicki. Appreciation goes to Debra Fowler, who assisted with evaluation of the RREE in 2004, and Maura Borrego, who contributed to the 2005 evaluation and who also helped to create the SoTL to educational research model. Our thanks also go to Mary Deane Sorcinelli, who is a member of the RREE Executive Committee, but who was not able to attend the June 11, 2005, meeting.

References

• Boyer, E. L. (1990). Scholarship reconsidered: Priorities of the professoriate. Princeton, NJ: Carnegie Foundation for the Advancement of Teaching.
• Brown, J. S., Collins, A., & Duguid, P. (1989, January/February).Situated cognition and the culture of learning. Educational Researcher, 18(l), 32–42.
• Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching and crafts of reading, writing, and mathematics. In L.B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 453–494). Hillsdale, NJ: Lawrence Erlbaum.
• Felder, R. M., Sheppard, S. D., & Smith, K. A. (2005, January). Guest editor’s foreword: A new journal for a field in transition. Journal of Engineering Education, 94(1), 7–10.
• Gabriele, G. A. (2005, July). Guest editorial: Advancing engineering education in a flattened world. Journal of Engineering Education, 94(3), 285–286.
• Haghighi, K. (2005, October). Guest editorial: Quiet no longer: Birth of a new discipline. Journal of Engineering Education, 94(4), 351–353.
• Huber, M. T., & Hutchings, P. (2005). The advancement of learning: Building the teaching commons. San Francisco, CA: Jossey-Bass.
• Hutchings, P., & Shulman, L. S. ( 1999, September/October). The scholarship of teaching: New elaborations, new developments. Change, 31(5), l0–15.
• Paulsen, M. P. (2001 ). The relationship between research and the scholarship of teaching. In C. Kreber (Ed.), New directions for teaching and learning: No. 86. Scholarship revisited: Perspectives on the scholarship of teaching (pp. 19–29). San Francisco, CA: Jossey-Bass.
• Schroeder, C. M. (2005, October). Going “meta” with SoTL: Research-based frameworks as the missing link. Paper presented at the 30th annual conference of the Professional and Organizational Development Network in Higher Education, Milwaukee, WI.
• Sorcinelli, M. D., Austin, A. E., Eddy, P. L., & Beach, A. L. (2006). Creating the future of faculty development: Learning from the past, understanding the present. Bolton, MA: Anker.