Workshopping a Workshop: Collaborative Design in Educational Development
Skip other details (including permanent urls, DOI, citation information)
: This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Please contact [email protected] to use this work in a way not covered by the license.
For more information, read Michigan Publishing's access and usage policy.
Abstract
Working remotely and collaboratively, our interdisciplinary team created an educational development workshop, Thinking Skills for the 21st Century: Teaching for Transfer, in which participants not only experience, apply, and reflect on teaching across educational settings but also connect this work to principles that have been demonstrated by learning science to support the transfer of knowledge. We used backward design to develop the workshop and evidence-based pedagogies in its implementation. We facilitated the workshop at two different national meetings for distinct audiences and also as part of an on-campus faculty development program. Here, we report on the workshop development and revision, assessment of participant experience, and the evolution of our collaborative process.
Keywords: case studies, collaboration, faculty development, pedagogy, teaching and learning
Educational development workshop quality can be hit or miss. We each have had the experience of attending a workshop with a promising description only to be disappointed by the outcome. While those trained as educational developers typically utilize the “do as I do” philosophy to deliver content and also model best practices, sometimes individuals who specialize in discipline based education research or scholarship of teaching and learning use a “do as I say” approach with an emphasis on sharing experiences rather than constructing knowledge (Mulnix, 2013). We each have also been disappointed when an extended workshop, which from the description looks to be a unified whole, instead consists of multiple short presentations related to each other only by a common title. In both of these cases we have left the event with some level of frustration over a lost opportunity.
Even when workshops are learner centered and integrated across presenters, content can focus on the “what” and “how” at the expense of “why.” A session on clicker type interventions might cover available technologies and guide question development, both critical for faculty to understand, but it may not explore the cognitive reasons behind the success such tools have in enhancing learning. Mulnix (2016) has argued that understanding principles of learning is key to empowering faculty to adapt evidence based pedagogical practices to their own context.
Given our own experiences, we were especially alert to design as we created a half day workshop entitled Thinking Skills for the 21st Century: Teaching for Transfer for an annual meeting of a national science focused organization (American Association for the Advancement of Science [AAAS], venue #1). We were intentional not only in creating a workshop in which faculty were students learning about learning, but also in coordinating the delivery across four speakers so as to progressively build on and reiterate content. Our goals were to cover learning science principles, model use of those principles in multiple contexts, and provide abundant opportunities for participants to practice with the content. Here we report on the process of creating this learner centered, multi presenter, extended workshop using best practices in educational development and then adapting it further for two different audiences: a national meeting with a higher education focus (Association of American Colleges and Universities [AAC&U], venue #2) and as part of a faculty development day on a university campus (University of South Alabama, venue #3).
In addition, we reflect on the experience of working collaboratively as a virtual team. Team projects can be fraught with pitfalls from personality conflicts to disruptive communication behaviors to lack of attention to deadlines, all of which can be exacerbated when team members work virtually. We consider reasons why our geographically distributed group did not experience these types of group disruptions and was highly productive, providing insights that can facilitate the success of other teams.
Workshop Design Process
Our identities as scientists and educational developers made the venue #1 meeting a natural choice for presenting a workshop, and our interests coincided with the association’s education initiatives. The feedback on the first workshop was sufficiently positive that we chose venue #2 as a way to more directly impact the conversation of inclusive excellence for all undergraduates, although this subsequent opportunity also meant an audience more diverse in their professional roles (e.g., administrators as well as faculty), requiring us to tailor the workshop’s content accordingly. The third offering was to a faculty audience from diverse fields such as nursing, anthropology, English, and mathematics. In each of the iterations, we built on the lessons learned from the prior workshops and fine tuned the workshop for a new audience.
The germ of our idea for venue #1 came from Mulnix (2013) which describes an all day workshop focused on learning about and using the principles underlying the transfer of knowledge and skills as outlined in Chapter 6 of Education for Life and Work (Pellegrino & Hilton, 2012). This chapter became our unifying content, and we focused on the six instructional design elements shown to support the ability to transfer knowledge to novel situations:
Providing multiple and varied representations of material.
Engaging learners actively with material through elaboration with self and others.
Relating content to real life through the use of examples and cases.
Guiding learners through a challenge with prompts for reflection and metacognition.
Motivating learners for learning.
Providing opportunities for practice with specific and timely feedback.
As faculty members who were from four different institutions with varied disciplinary backgrounds and areas of expertise within educational development, we recognized the workshop as an opportunity to include diverse perspectives and highlight multiple pedagogical approaches; thus, each of us took primary responsibility for one section. We identified instances of knowledge transfer in our own work that could be used as content for the workshop. Our practical experiences then formed the foundation for integrating the theory relating to knowledge transfer into the workshop.
Over the nine month period between acceptance of our abstract and the venue #1 meeting, we gathered at least monthly via synchronous virtual collaboration technologies to share and critique ideas. Because we were committed to modeling best pedagogical practices, we used backward design to plan the workshop (Wiggins & McTighe, 2005), with each of us articulating learning goals and identifying measurable outcomes for our section. We then met to generate a workshop outline that explicitly integrated content (the six design elements that promote knowledge transfer), activities (e.g., engaging in a case study), assessments (e.g., answers to clicker questions, products of group work), and opportunities for reflection and metacognition throughout the entire workshop. Across our conversations, we each responded to suggestions provided by the others and modified our sections as appropriate to meet the group’s goals. For example, we varied participant interactions across the entire workshop time frame, deciding when to ask for written reflection, pair and share conversation, and group work. We adapted segments to illustrate different ways in which reporting out of a group conversations could occur (e.g., one segment included a gallery walk while another had small groups report to everyone). We were also intentional about coordinating our sections so as to produce smooth transitions. We coordinated the learning outcomes for each segment to create a learning arc for participants with each of us referring to material presented earlier and alluding to upcoming content.
The subject matter itself—what features of learning environments promote knowledge transfer—allowed us to embody the concepts being presented. We designed the workshop structure and activities to: (a) provide multiple and varied representations of material; (b) motivate participants to learn; (c) actively engage participants; (d) relate content to the participants’ own experiences; (e) provide prompts for reflection and metacognition; and (f) incorporate numerous opportunities for practice with feedback. In addition to explicating six features that support knowledge transfer, we sought to directly connect to experiences of participants as researchers and instructors as a means of teaching about knowledge transfer.
The result of our efforts was a coherent half day workshop (Appendix A) with four scaffolded 35 minute sections built to increase participants’ knowledge via multiple encounters with the six principles and numerous opportunities to practice transfer in different contexts. Formative and summative assessments in the form of individual reflection and clicker questions were included at the end of each section to provide participants and us with real time feedback. As the workshop developed across iterations, we revisited these assessments to ensure they continued to align with our workshop goals and activities. This was especially true as we adapted the workshop to serve different audiences at each different venue.
The Workshop
The workshop was divided into four scaffolded sections, each facilitated by one of us.
Introduction: Workshop Goals and Six Features of Transfer
Section 1: Using a Real Life Example for Elaboration
Section 2: Using Guidance and Metacognition to Support Challenging Tasks
Section 3: Using Technology to Provide Practice with Feedback
Section 4: Using Case Studies to Provide Multiple and Varied Representations of Material
Conclusion: Synthesis and Final Reflection
In the descriptions below, we define participant learning outcomes, describe activities (which often doubled as formative assessments), provide a summary of the experience, and discuss any modifications we made in the iterations of the workshops. During the workshop, those facilitators not leading a given section acted as teaching assistants. Each section ended with clicker questions that asked participants to reflect on their experience and provided opportunity for participant metacognition (see Appendix B for a complete list of questions used). For example, Section 1 questions and responses:
Which of the following ideas from this session are you most interested to apply to your classroom?
a. Use class time to read an example (multiple representations of material).
b. Have students discuss the example in small groups (elaboration and feedback).
c. Provide opportunities to practice.
d. Provide feedback by giving sample answers (practice with feedback).
What did you learn about your own learning?
a. The social component (elaboration) to learning is important.
b. Novelty of content grabbed my attention.
c. Understanding the concepts was easier after I had seen examples in the story.
d. Having a partner to check my thinking with was valuable.
e. All of the above.
At the venue #1 and #3 workshops, questions focused specifically on ideas that could be implemented in the classroom. For the venue #2 workshop, where we had a mixed audience, we adjusted questions to include both teaching and administrative participants. At all three meetings we included a question about the participants’ learning gains (see Appendix B).
Introduction: Workshop Goals and Six Features of Transfer
In the opening section of the workshop, participants were provided with a brief definition of knowledge transfer: The ability to use what one has learned in a novel context. Bloom’s taxonomic levels, both the original and modified versions, were used to explain that knowledge transfer consisted of the activities at higher cognitive levels than remembering and understanding, including application, synthesis, evaluation, and creation (Anderson & Krathwohl, 2001; Bloom, 1956). We noted in particular that use of knowledge at higher order levels required deeper learning than learning characterized by the lower levels of the taxonomical pyramid. Next, we presented a mini lecture of the six features demonstrated by empirical evidence (reviewed in Pellegrino & Hilton, 2012) to support knowledge transfer using a series of simple slides including brief text and an image illustrating each feature.
We varied the presentation of introductory material for the venue #2 meeting by assigning it as homework. Participants were asked to listen to a seven minute screen cast that covered transfer and features that promote it prior to their arrival at the workshop. A major factor in making this change was to create sufficient time for the last presenter who was rushed in the original offering of the workshop at venue #1. In anticipation that some people would not have done the homework, we included a three minute review of transfer and Bloom’s taxonomy and provided a handout on the material for participants.
We observed that the “flipped” classroom was less successful than the on site instruction used at venue #1 for several reasons. Not surprisingly, upon being asked, many participants admitted to not having done the homework. They indicated in their evaluations that the shortened time for the introductory material was insufficient. This lack of preparation created an uneven start to the workshop, where—just like our students—some participants were ready to jump into learning the next information while others were completely unfamiliar with the topic and had a difficult time diving into activities. Additionally, because groups did not have the shared experience of learning about transfer, the group conversations were not as robust. To create a shared learning environment, we returned to the mini lecture format at the venue #3 workshop, and each of us committed to streamlining our individual sections. This experience reminded us that instructing faculty and administrators can be very much like teaching students.
Section 1: Using a Real Life Example for Elaboration
For the first section we provided an opportunity for participants to apply their new knowledge about transfer through identifying examples of knowledge transfer in a novel context.
Learning outcomes
Participants will be able to:
Identify specific examples of transfer and name features of the environment that promoted the transfer;
Use elaboration and reflection to solidify understanding and identify personally relevant examples of transfer; and
Apply new understanding to their own situations.
Activities included in this section
Activities are reading, individual writing, paired discussion, small group discussion, reflection, and clicker questions with metacognitive prompts.
Section description
Participants read a shortened version of Mulnix (2016) describing the insights gained about the process of learning while training a cadaver dog to detect human remains. The story provided multiple examples of knowledge transfer including taking book based knowledge and recognizing it in the real world and seeing the application of a real life experience from a nonacademic environment to a classroom setting. In addition, the story intentionally illustrated how multiple representations of ideas and practice with feedback were essential for the author to gain a deep understanding. The topic was chosen in part because it could be comprehended with relatively little previous knowledge and because its novel, real life content would be sufficiently interesting to motivate readers.
The reading questions that accompanied the story asked participants to identify examples of both knowledge transfer and the six features of the learning environment known to promote the ability to use knowledge across contexts. Participants first worked individually. Small group discussion provided additional opportunities for elaboration of ideas and also for informal feedback from peers. Participants then engaged in reflection and metacognition by identifying how the structure and process of the section illustrated elements of a learning environment that promoted transfer. The ways in which this section illustrated our content are summarized in Table 1. In the last half of the section, participants were asked to work in pairs to apply a pedagogical strategy that embraced one of the features that promotes transfer to their own work. We ended the section with two clicker questions to prompt reflection and metacognition (Appendix B).
| Within Story | Structure of Section | |
|---|---|---|
| Multiple representations | Text and pictures | Used individual and group work |
| Elaboration | Explanation within the story | Individual and small group work to identify examples |
| Real life examples and cases | Story itself was a real life example | Participants applied their new knowledge about transfer to the story |
| Reflection and metacognition | Story included moments in which the author exhibited reflection and metacognition | Participants were asked questions at the end of the section about their learning process |
| Motivating | Novelty of content of story (training a cadaver dog) | Instructors provided enthusiasm; participants engaged in a discovery process |
| Practice with feedback | Several components of the story showed persons in the process of practicing and receiving feedback | Participants received feedback from peers in small group discussions and from reporting out during a summarizing exercise |
Summary
This first section introduced the content of transfer, and it also communicated how the content of transfer was embedded in our process. Our essential design of the overall workshop required that as participants learned about transfer, they actively engaged in design features and participated in activities that promoted the deep learning necessary for transfer.
We tailored the first section for different workshop audiences by changing instructions and prompts. Faculty were asked to consider students as the learners, and administrators were asked to consider faculty as learners. This perspective difference was most obvious in the second half of the section in which participants were asked to transfer their new knowledge to their own situations as teachers, deans, department chairs, and educational developers.
Section 2: Using Guidance and Metacognition to Support Challenging Tasks
The second section, gave participants an opportunity to experience intentional guidance and metacognition as learners engaged in the challenging task of development of learning outcomes.
Learning outcomes
Participants will be able to:
Apply principles of guidance and metacognition to their teaching environments; and
Develop well written and audience appropriate learning outcomes.
Activities included in this section
Activities are quick write, group brainstorm, think pair share, examples, scaffolding, elaboration, small group discussion, whole group discussion, reflection, and clicker questions with metacognitive prompts.
Section description
This section used the process of writing learning outcomes to demonstrate supporting scaffolding of knowledge for learners who are approaching difficult topics. First, the facilitator asked participants to think and write about who their learners are (in a mixed audience this included students in courses, faculty in educational development activities, or faculty in a department or across a campus). As a whole group we brainstormed answers to two questions: (a) how do we know that learning has occurred? and (b) how do students know what is expected of them?
Next, the facilitator guided participants to write one concept or skill they want learners to know or be able to do as a demonstration that learning has occurred. After individual reflection, participants shared the knowledge or skill with a neighbor and then reported some ideas to the wider audience (think pair share). Answers ranged from class level concepts to whole institution ideas about general education requirements. For the next step, the facilitator asked participants to identify ways they know if a learner has achieved the knowledge or skill. Participants considered what behaviors learners demonstrate when they have acquired or mastered the pertinent content knowledge or skill (e.g., ability to critically read a text or write a research paper). Participants focused both on method of assessment (e.g., exam, presentation, essay) and types of assessment questions that would demonstrate student proficiency with the knowledge or skill. The facilitator next presented a three minute mini lecture with examples of learning outcomes along a spectrum from vague outcomes at lower levels of Bloom’s taxonomy (Anderson & Krathwohl, 2001; Bloom, 1956) to more specific outcomes at higher levels of Bloom’s taxonomy (Table 2).
| Vague | Less Vague at Lower Levels of Bloom’s Taxonomy | More Specific at Middle Levels of Bloom’s Taxonomy | Examples at Higher Levels of Bloom’s Taxonomy | |
|---|---|---|---|---|
| Introductory Cell Biology Course | Students will understand the cell cycle. | Students will be able to describe the steps of the cell cycle. | Students will be able to diagram the chromosomes, nuclear envelop, and spindle apparatus throughout the cell cycle in an animal cell. | Students will compare, contrast, and identify animal cells at different stages of the cell cycle. |
| Introductory Physics Course | Students will understand simple d.c. circuits. | Students will describe the role of a battery, resistors, switches, and wires in constructing a d.c. circuit. | Students will be able to analyze diagrams of simple d.c. circuits involving a battery, resistors, and switches. | Students will be able to construct accurately a functioning physical circuit based on a circuit diagram. |
| Introductory Calculus Example2 | A successful student in this course should learn how to sketch graphs of functions. | Students will be able to sketch graphs when given a function. | Students will be able to graph functions with approximate axes and identify the local and global extremes. | Students will graph and interpret functions and use derivatives to calculate the slope of a graph at a point. |
| Faculty Educational Development Workshop1 | Faculty will understand the principles of backward design. | Faculty will be able to describe the steps of backward design and how to apply them to teaching. | Faculty will be able to assess how the components of backward design support student learning. | Faculty will create a student learning activity using backward design principles. |
| General Education Program Level1 | Students will understand the differences between primary and secondary literature. | Students will describe how to search for primary literature in the library database. | Students will be able to examine sources found in the library database and recognize key characteristics of high quality research materials. | Students will be able to evaluate the quality of resources and critique the claims from a variety of source types. |
Following the mini lecture of examples, each table group had time to reflect and identify features that changed in each of the sample learning outcomes as they were transformed from vague and lower level to specific and higher level. Participants returned to the outcome they had written and refined it to make it specific, measurable, attainable, realistic, and time bound (Ambrose, Bridges, DePietro, Lovett, & Normal, 2010). Participants were also provided with a list of action verbs that can be mapped to different cognitive levels of Bloom’s taxonomy (Marquette University, 2018). As a group, participants shared revised learning outcomes, received additional feedback, and discussed how they could apply these ideas to their own professional work. Finally, the facilitator asked participants to discuss their experience as learners. The section ended with two clicker questions (Appendix B).
Summary
In the discussion that concluded this session, we highlighted how we started with a problem and deconstructed it to provide scaffolding before building a new solution (Pellegrino & Hilton, 2012). The facilitator provided guidance through scaffolding small pieces of the task—the entire process for writing learning outcomes was not presented initially, but small pieces were added to increase task complexity with opportunities to check in with the facilitator and learn from other participants. Additionally, we did not start with a specific, complex learning outcome, but built up to it with time allotted for individual thinking, feedback from the group, and opportunities to revise. The section included time for individual reflection where participants wrote and shared responses to the question, “How can you apply transfer principles of guidance and metacognition in your teaching?” This allowed participants an opportunity to think about ways to apply information to their own work and provided practice developing their own metacognitive skills.
We made only minimal changes to this section between venue #1, venue #2, and venue #3 workshops because the process of guidance with challenging tasks and metacognition applied equally to each audience. However, in anticipation of different audiences at venue #2 and venue #3, we tailored the examples of learning outcomes to the participants (Table 2).
Section 3: Using Technology to Provide Practice with Feedback
For the third section, participants experienced how technology can support learner knowledge transfer via hands on activities with clickers.
Learning outcomes
Participants will be able to:
Recognize the value of formative assessment in providing feedback;
Implement peer instruction with interactive classroom response systems (clickers);
Assess the role of different types of clicker questions in providing feedback;
Practice providing feedback on the outcomes of clicker questions; and
Design Ranking Tasks (O’Kuma, Maloney, & Hieggelke, 2000) that act as formative assessment activities (venue #1 only).
Activities included in this section
Activities are multiple choice clicker questions, Peer Instruction (Crouch & Mazur, 2001), individual writing, alphanumeric clicker questions, Ranking Tasks (O’Kuma et al., 2000), small group discussion on agile lecturing, reflection, and clicker questions with metacognitive prompts.
Section description
In this section, participants played the role of students in a classroom each having their own clicker (iClicker2 handheld devices, which have an LCD display and allow for both multiple choice and alphanumeric responses). The facilitator led them through a series of challenging clicker questions on different topics (some science related, some from other fields) to have participants experience the feeling of being a novice in the classroom. We used Peer Instruction with many of the clicker questions, and the facilitator modeled best practices in the use of clickers (Crouch & Mazur, 2001). The types of questions required knowledge, comprehension, application, and analysis skills (Crouch & Mazur, 2001; Deslauriers, Schelew, & Wieman, 2011; Smith et al., 2009).
We also used the clickers to introduce another example of a Classroom Assessment Technique (CAT; Angelo & Cross, 1993), the Ranking Task. Ranking Tasks are exercises principally conceptual in nature that challenge students to make comparative judgments about a set of variations on a particular situation. Effective Ranking Task exercises ask students to compare, contrast, and rank order between six and eight situations. For example, in the venue #1 workshop, to best connect with scientists in the audience, the classic ideal gas equation PV = nRT was the basis for the Ranking Task analyzed by participants. Participants read from a graph the values of pressure and volume experienced by one mole of an ideal gas under different conditions and rank ordered the six different readings on the basis of highest to lowest temperature of the gas.
Finally, participants practiced agile lecturing where they had to discuss at their tables and respond to hypothetical clicker data distributions, which might require particular teaching decisions and creativity on the part of the instructor. Two of the data distributions are shown in Figure 1. In each case, participants discussed the data at their tables, created scenarios within which the data may have been produced, and then came to consensus on how best to respond to such results in class. The table discussions were then shared with the larger group so that each data scenario was addressed. Participants completed the end of section reflection and clicker questions (see Appendix B).
Figure 1. Examples of Agile Lecturing Used to Discuss Ways to Address Student Misconceptions or Understandings as Discovered from Clicker DataSummary
This section of the workshop specifically explored practice with feedback, although the facilitator briefly reviewed all the features of transfer we used. We did not include a ranking task question at the venue #2 and venue #3 workshops to allow a greater amount of time for agile lecturing, an activity better suited to a diverse audience of administrators, faculty, and educational developers. As a result of listening to participants conversations and group discussions during the venue #1 workshop that indicated challenges in their metacognitive abilities, we added a slide called “making thinking visible” to get participants thinking about how we know what we know (Ritchhart, Church, & Morrison, 2011). We realized that this was an important moment—the moment when a group of learners is looking at data—where we could help faculty understand how to model the expert thinking process and make their thinking transparent to students.
Section 4: Using Case Studies to Provide Multiple and Varied Representations of Material
The final section utilized samples of case studies to allow participants to experience Problem Based Learning (PBL) (Roberts, 2016) and to focus participant attention on types of interdisciplinary cases they might develop for their own teaching.
Learning outcomes
Participants will be able to:
Generate topics and assessments for problem based group work for their own courses; and
Create assessments based on learning outcomes for problem based activities.
Activities included in this section
Activities are case study reading, individual, writing, jig saw, group discussion and report out, application and elaboration, and clicker questions with metacognitive prompts.
Section description
For the venue #1 presentation, a variation on a published case was used to illustrate development and implementation of a case study in a course. The case study, “Dan Tries Problem Based Learning: A Case Study” published by (White, 1996) was modified for detail and length. The case was presented in sections on (a) getting started, (b) planning of exams around group/case learning, (c) grading of exams and giving feedback; and (d) course and faculty evaluations. Participants were divided into groups who read the whole case, but discussion of each section was jig sawed (Angelo & Cross, 1993). Each group addressed discussion questions on one of four sections and reported out to the group.
Discussion questions pointed groups to several key issues raised by the case, including potential pitfalls when introducing unfamiliar and “risky” pedagogical techniques into courses, particularly for those undergoing evaluation for promotion and tenure. How does one balance active learning and content coverage? How does one explain the changes to colleagues or administrators? How does one deal with potentially negative student evaluations as he or she adjusts to new styles of teaching?
We made several significant changes between the venue #1 workshop and the venue #2 and #3 workshops as described below. The primary change was the use of a new case study focused more on student learning and teaching for knowledge transfer. We used a case from a colleague at a Midwestern liberal arts university who developed and implemented a case study into her course, “Dr. Anderson tries Problem Based Learning (PBL)” (Anderson, Bunnell, & Yates, 2017). In addition to the benefit of having specific content and specific assessments to discuss, the new case allowed for opportunities to engage an audience with more diverse campus roles. The case was presented in three segments: (a) mini lecture on background and development of PBL as a learning activity; (b) discussion of incorporation of interdisciplinarity; and (c) assessments used to evaluate student learning.
Segment 1: Background and development
In segment one at venue #2 and venue #3, the facilitator presented the challenges faced by a colleague in teaching an Ecology course that motivated her to develop a PBL project focused on Lake Erie water resources. Her large lecture course met both major and distribution requirements, which meant it was populated with students with different expectations. In addition, the complexity of material, paired with a traditional “sage on the stage” teaching format, meant that students seldom truly engaged with course content. In developing the PBL component, the faculty member considered ways to engaging a diverse student audience, incorporate active learning, improve student learning of a complex ecological concept, and use the interdisciplinary nature of ecology.
Following the presentation of the case, faculty workshop participants were asked to consider: (a) how their students were similar to or different from those portrayed in the case; (b) whether they, as faculty, experienced similar teaching challenges; and (c) how they could envision responding to such challenges in their own courses. Questions for administrative participants at the venue #2 workshop focused on how they support faculty at their own institutions in developing similar types of activities.
Segment 2: Interdisciplinarity
Participants next read the second part of the case that was focused on the efforts to include interdisciplinary perspectives within the PBL module. The case described a jig saw activity in which students were required to examine an ecological case from a particular disciplinary perspective (often different from their own major). Following presentations by individual students from each assigned perspective, each small group (which included one of each “expert”) discussed the value of the interdisciplinarity, and they used those perspectives to develop a management plan incorporating all of the various stakeholder perspectives. As summative assessment, students were required to look back at the problem from their own major/disciplinary perspective.
Workshop discussion questions for segment 2 included: (a) focus on the potential for development of cases in their own fields that might be amenable to an interdisciplinary exercise; (b) what resources they might use in a similar activity; and (c) how this type of exercise meets the teaching for transfer strategy of using multiple and varied representations. At venue #2, administrative participants were asked whether existing programs on their own campuses facilitated this kind of interdisciplinary project, and if not, where programs could be implemented.
Segment 3: Opportunities for assessment of student learning
The final segment involved discussion of assessment of student learning in the PBL case. Participants reviewed assessments in the Ecology PBL activity, including pre and post tests on interdisciplinarity in solving real world problems, an essay from an “expert’s” perspective, student presentations, discussion participation, and an essay written from students’ major/discipline perspectives.
Workshop participants discussed effectiveness of assessments described in the case and then developed learning outcomes and assessments to meet the teaching for transfer goal of giving students practice with feedback in their own case ideas from segment two. At venue #2, administrative participants discussed the understanding of and attitude toward assessment on their own campuses and how they might change a culture which views assessment negatively. At the end of the section, clicker questions aimed at the various audiences were presented (see Appendix B).
Summary
We deliberately placed this PBL section at the end of workshop because the case study—particularly the revised case used at venue #2 and #3—put into practice all of the features of a transfer encouraging learning environment. Participants learned how a well constructed PBL activity can give multiple and varied representations of material and how a case that is related to both course content and real life can include assessments used to engage material through elaboration with self and others. By introducing workshop participants to this content as a case study, we were able to model several of these aspects as well.
The new case study for venue #2 and #3 with the particular focus on interdisciplinarity was a topic more appropriate for wider ranging perspectives at these workshops. Discussion questions aimed at different constituencies were added to each segment to engage all audience members in thoughtful reflection of problem based/case study learning as a mechanism to increase students’ transfer of knowledge and skills across contexts.
Conclusion: Synthesis and Final Reflection
To conclude the workshop, we circled back to our learning outcomes and asked participants to write down as many of the six features of knowledge transfer they could recall. They shared examples in their groups of where and how each was experienced during the workshop. Participants reflected on how the ideas of transfer helped them meet their individual workshop goals and practiced elaboration by generating three ideas to implement in their professional work during the academic year.
What Did We Learn?
Workshop Design and Modification
Master teachers reflect on their practice in ways that allow them to improve what they do and deepen their understanding of the processes of teaching and learning (Bain, 2004; Handelsman, Miller, & Pfund, 2007). Our immediate observations after the venue #1workshop—including enthusiastic participation and responses from attendees—signaled that the workshop had been sufficiently successful to offer it again. In the revision process of the content and execution of subsequent offerings, we had four sources of feedback on each workshop: our self assessments, peer evaluations from coleaders, clicker data from participants, and a typical postworkshop evaluation.
We learned that repeating the workshop in different venues paralleled our experiences of teaching a particular content (e.g., cell biology) across different academic years and for different audiences of students (e.g., first year, upper level, or nonmajor courses). Still, because we were teaching in a team, our collective goals of adapting the workshop allowed us to solve these challenges collaboratively. For instance, with no next class period into which to spill, we engaged in an iterative process to solve the problem of fitting our evolving content into a half day workshop. As with teaching a course multiple times, we did better at pacing at venue #3 in part because we were more attentive to the effect of how the teaching choices each of us made individually in the moment (e.g., how elaborately to answer a question) impacted the overall flow of the workshop, and we collectively adjusted accordingly. We also noted that there were variables we had not formally recognized: the few minutes it takes an audience to settle after a break and dealing with inevitable technology transitions.
We should have anticipated, given the variety of our teaching experiences, that modification for different audiences would be more than a trivial undertaking, particularly the transition from an audience of science educators (akin to an upper level course) to one largely consisting of administrators (akin to an introductory level) at venue #2. Again, we undertook collaborative solutions to: (a) tailor our content, especially the reflective questions at the end of each section and (b) encompass the perspectives of those participants who work with faculty as well as those who work with students.
We initially struggled to stretch the concepts of knowledge transfer to the roles of university administration. How would we help associate deans and provosts engage their faculty as learners to apply new information to the context of their teaching? Our focus on teaching for knowledge transfer directly provided us with the method; we used real life examples, scaffolding, elaboration, practice, and metacognitive prompts. For example, we exchanged some of the science content in Section 3 on use of clickers for a more generalizable example involving a Tabla drum. This design choice allowed participants to struggle with a deceptively complex question outside of their expertise; created excellent group discussion and debate; and provided an opportunity for administrators to observe themselves as learners and thus transfer the content of the workshop to their own contexts.
Finally, we discovered the long term value of collecting feedback from participants at multiple points during the workshop. We included formative questions at the end of each section to prompt reflection on the part of the participants, both to help them gauge their understanding and to become more aware of themselves as learners (e.g., what aspect of a section resonated with them as learners?). As leaders, this gave us real time feedback about participants’ experience through their answers to the clicker questions or from their conversations at tables. An unanticipated benefit of using clickers was that following the workshop we were able to review the time stamped feedback, which provided us with glimpses into the progression of participant learning. Specifically, answers following each of the first three sections indicated that participants were interested in incorporating more group work into their own courses. This result provided us with evidence that participants were meeting one of our key workshop goals: they were learning content, gaining personal experience of the value of collaborative learning, and more motivated and likely to provide opportunities for collaborative work in their own courses.
Additionally, in the fourth section of the workshop, the content specifically focused on group work in an interdisciplinary context. Participants’ metacognitive clicker responses during this section not only indicated a continued interest in incorporating group work into their teaching, but also for the first time showed an interest in incorporating interdisciplinary perspectives into courses. This shift was significant to us because it paralleled our scaffolded design and meant our pedagogy was effective even at the end of a half day workshop. By doing formative assessments throughout the workshop (rather than a typical final evaluation) and purposefully aligning these with our goals and learning outcomes, we were able to “see” our participants learning.
Collaboration
Our group developed the entirety of the workshop through online meetings (the first time we met in person was at the venue #1 workshop), and this process generated insights into the nature of collaboration, particularly virtual collaboration. While teams are a norm in today’s academic environment, few groups develop a depth of interdependence, longevity, and productivity. Ours has. We have not only led the workshop multiple times, but also produced numerous joint publications (Chaudhury, Mandeltort, Mulnix, Vandegrift, & Yates, 2015; Mulnix & Vandegrift, 2014; Mulnix, Vandegrift, & Chaudhury, 2016).
While we were intentional about our approach to developing the workshop, we were not consciously attentive to developing ourselves as a team. What then allowed this robust collaboration to develop and to do so in a virtual environment? We suggest several aspects that contributed to our success: (a) a distributed leadership model (Bell & Kozlowski, 2002; Hill & Bartol, 2016); (b) attention to mutual mentoring (Berg & Seeber, 2016); and (c) personal attributes of those successful in educational development.
Hill and Bartol (2016), building on the model proposed by Bell and Kozlowski (2002), argue that effective collaboration of geographically distributed teams is characterized by an empowering leadership model. Under this type of leadership, members of a team share power and create a facilitative and supportive environment in which each member “appl[ies] relevant knowledge and judgment in order to successfully collaborate virtually with other team members” (p. 160).
This model certainly fits our experience. When we initially came together, one member had organized the venue #1 submission and took a primary leadership role in recruiting the other team members, but as the first workshop developed and as we have undertaken subsequent projects, we adopted a highly distributed form of leadership. We often shift roles within a project: sharing leadership, recording ideas, setting priorities, establishing action plans, holding each other accountable, and prompting reflection. We have fluidity of primary responsibility within as well as across our projects, and each of us has taken organizing, facilitating, and decision making roles.
We further shared power through respectful peer review. To counter this often overlooked aspect of collaboration when individual segments are knitted together rather than collaboratively integrated, we exchanged materials as near final products expecting feedback from each other and remaining open to improving our own work. For instance, we collectively edited the cadaver dog story to a much shortened length that still communicated the essential points. We also sought and provided feedback for each other about presentation style and mechanics.
Such practices demonstrate mutual mentoring (Sorcinelli & Yun, 2009), a model of partnership in which there is reciprocity of benefits rather than a strict mentor–protégé hierarchy. We sought to learn from each other as we improved the workshop. Some of us became more adept at backward design as a result of another’s expertise, for instance, while others became more technically savvy under the tutelage of another member.
Moreover, mutual mentoring seeks to build relationships that contribute broadly to the success of individuals as well as the group. Once we established effective collaboration for the workshop, our interactions expanded to include problem solving unrelated to this project. For instance, when one of us was having significant grant administration issues, the others offered suggestions for next steps. When another had a difficult series of interactions with a colleague, others offered listening ears and possible interventions. Importantly, we also all cheered others’ successes. These encounters allowed trust to become implicit in our relationships, allowing us to move from collaborators to colleagues. Our geographic distribution may have been especially helpful in this regard (Berg & Seeber, 2016). Because none of us knew the specific players or politics at the other campuses, we could ask probing questions that clarified circumstances and opened up new avenues of response. Our interactions not directly related to the development and presentation of the workshop, have strengthened our ability to work as a team as we have become more comfortable with each other as colleagues, collaborators, and friends.
Collectively, distributed leadership, peer review, and mutual mentoring allowed us enough familiarity with each other and with our collective content to model the kind of agile teaching necessary for a learner centered workshop. Those of us not presenting could be effective as we circulated among the tables because we knew what the presenter wanted participants to gain from the session. Our extensive discussions of the overall content meant each of us could provide alternative examples of the concepts when working with individuals. This mutual development of the workshop, even though we each lead a section, gave us shared intellectual ownership of the final product. Consequently, we have each been able to take various components of the workshop and independently share them with other educational development audiences.
Hill and Bartol (2016) argue that the ability of team members to regulate their own behaviors and performances is especially important in virtual collaborations. All of us demonstrated trust, responsibility, and empathy. We took deadlines and commitments seriously, yet recognized and accommodated circumstances when one of us was delayed in completing assigned tasks. We drew on emotional intelligence about the verbal cues indicating rising emotion or waning energy and adjusted our virtual responses accordingly (Bradberry & Greaves, 2009). We began each virtual meeting checking in with each other about nonworkshop based daily experiences, which allowed us to build our relationships.
Likewise, we shared theoretical and philosophical dispositions as educational developers, including the values and practices that embrace growth mindsets (Dweck, 2006), which in turn allow for risk taking and openness to learning. Each of us understands knowledge is constructed and that working through intellectual difficulties is a necessary component of learning. This promotes patience and active listening. In aggregate, this epistemological grounding allowed each of us to be vulnerable and seek out critique and to make honest yet supportive contributions to improving the workshop. In essence, we were doing professional interventions with and for each other, weaving our professional knowledge directly into the workshop and into our working relationships.
Summary
Leading educational development workshops creates opportunities for transmission of new content to faculty and administrators as well as a chance to model effective pedagogies. Learning science theory tells us that engagement in learning is necessary for transfer of knowledge to new situations (Ambrose et al., 2010; Bransford, Brown, & Cocking, 2000; Pellegrino & Hilton, 2012). Thus, modeling learner centered teaching strategies not only deepens learning the content of the workshop but also provides examples for how faculty can use the pedagogies in their own classrooms. As facilitators, our collaborative process enriched our own understanding of teaching for transfer. Our collective experiences across multiple workshops reinforced the importance of intentional and iterative design process in educational development workshops.
Team development and delivery of workshops also provides professional growth opportunities for those leading the workshop. When the knowledge and skills educational developers use to lead experiences for faculty are brought to bear in the collaboration itself, the process is mutually beneficial. Members of the team have a chance to transfer knowledge to new contexts. Our reflections also suggest to us that directed attention to tailoring the workshop for specific audiences pushed each of us to consider new perspectives and expand our own knowledge bases. This focus on learners rather than on our individual contributions also advanced our collaborative spirit by reducing the focus on what any one of us had created. By keeping the needs of the audience at the center of our work, we were each more willing to consider alternatives to those pet ideas in which our egos were highly invested.
We encourage those that are developing and leading workshops as part of a team to recognize the process as an opportunity for their own development and to bring their expertise to bear on that process. While the time necessary to create a truly collaborative product is greater than the time required to knit together individual sections, we found our own intellectual growth to be worth the additional investment. The very knowledge and skills that make us excellent at supporting the growth of teachers and scholars can reap benefits for ourselves when used intentionally in our collaborations.
References
- Ambrose, S. A., Bridges, M. W., DePietro, M., Lovett, M. C., & Normal, M. K. (2010). How learning works: Seven research based principles for smart teaching. San Francisco, CA: Jossey Bass.
- Anderson, L.A., Bunnell, S.L., Yates, J.R. (2017). Teaching Ecology and Interdisciplinary Thinking in a Case Study. Case Studies in the Environment. 1(1):1–11.
- Anderson, L. W., & Krathwohl, D. R. (Eds.) (2001). A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educational objectives. New York, NY: Pearson.
- Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco, CA: Jossey Bass.
- Bain, K. (2004). What the best college teachers do. Cambridge, MA: Harvard University Press.
- Bell, B. S., & Kozlowski, S. W. J. (2002). A typology of virtual teams: Implications for effective leadership. Retrieved June 13, 2018, from Cornell University, School of Industrial and Labor Relations site: http://digitalcommons.ilr.cornell.edu/hrpubs/8/.
- Berg, M., & Seeber, B. (2016). Slow professor: Challenging the culture of speed in the academy. Toronto: University of Toronto Press.
- Bloom, B. S. (1956). Taxonomy of educational objectives. Vol. 1: Cognitive domain. New York, NY: McKay.
- Bradberry, T., & Greaves, J. (2009). Emotional intelligence 2.0. San Diego, Calif.: TalentSmart.
- Bransford, J., Brown, A., & Cocking, R. (Eds.) (2000). How people learn: Brain, mind, experience and school. Washington, DC: National Academies Press.
- Chaudhury, S. R., Mandeltort, L., Mulnix, A. B., Vandegrift, E. V. H., & Yates, J. B. (2015). Using scientific visualization to enhance the teaching and learning of core concepts. In D. M. Baylen, & A. D’Alba (Eds.), Essentials of using and integrating visual and media literacy—Visualizing learning. Cham: Springer.
- Crouch, C. H., & Mazur, E. (2001). Peer instruction: Ten years of experience and results. American Journal of Physics, 69, 970–977.
- Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large enrollment physics class. Science, 332(6031), 862–864.
- Dweck, C. S. (2006). Mindset: The new psychology of success. New York: Random House Incorporated.
- Handelsman, J., Miller, S., & Pfund, C. (2007). Scientific teaching. New York, NY: W.H. Freeman.
- Hill, N. S. and Bartol, K. M. (2016). Empowering Leadership and Effective Collaboration in Geographically Dispersed Teams. Personnel Psychology, 69: 159–198. doi:10.1111/peps.12108
- Marquette University. (2018). Blooms taxonomy action verbs. Retrieved from http://marquette.edu/assessment/includes/documents/BloomsTaxonomyActionVerbs.pdf
- Mulnix, A. B. (2013). Communicating a new model: Learner centered strategies in faculty development. Journal on Centers for Teaching & Learning, 5, 23–47.
- Mulnix, A. B. (2016). What my cadaver dog taught me about teaching and learning. Journal on Excellence in College Teaching, 27(1), 5–21.
- Mulnix, A. B., & Vandegrift, E. V. H. (2014). A tipping point in STEM education reform. Journal of College Science Teaching, 43(3), 12–14.
- Mulnix, A. B., Vandegrift, E. V. H., & Raj Chaudhury, S. (2016). How important is achieving equity in undergraduate STEM education to you? Journal of College Science Teaching, 45(4), 8–11.
- O’Kuma, T. L., Maloney, D., & Hieggelke, C. J. (2000). Ranking task exercises in physics. Boston, MA: Addison Wesley.
- Pellegrino, J. W., & Hilton, M. L. (Eds.) (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Washington, DC: National Academies Press.
- Ritchhart, R., Church, M., & Morrison, K. (2011). Making thinking visible: How to promote engagement, understanding, and independence for all learners. Hoboken, NJ: John Wiley & Sons.
- Roberts, J. W. (2016). Experiential education in the college context: What it is, how it works, and why it matters. New York, NY: Routledge.
- Smith, M. K., Wood, W. B., Adams, W. K., Wieman, C., Knight, J. K., Guild, N., & Su, T. T. (2009). Why peer discussion improves student performance on in class concept questions. Science, 323(5910), 122–124.
- Sorcinelli, M. D., & Yun, J. H. (2009). When mentoring is the medium: Lessons learned from a faculty development initiative. To Improve the Academy, 27, 365–384.
- White, H. B. (1996). Dan tries problem based learning: a case study. In L. Richlin (Ed.), To improve the academy (Vol. 15, pp. 75–91). Stillwater, OK: New Forums Press and the Professional and Organizational Network in Higher Education.
- Wiggins, G. P., & McTighe, J. (2005). Understanding by design. Boston, MA: Pearson.
Appendix A: Example Timeline for the Three Hour Workshop
| Time | Section |
|---|---|
| 12 minutes | Introduction: Workshop goals and six features of transfer |
| 35 minutes | Using a real life example for elaboration |
| 35 minutes | Using guidance and metacognition to support challenging tasks |
| Break | |
| 35 minutes | Using technology to provide practice with feedback |
| 35 minutes | Using case studies to provide multiple and varied representations of material |
| 5 minutes | Conclusion: Synthesis and final reflection |
Detailed Timeline for Section 2: Using guidance and metacognition to support challenging tasks
| Time | Activity |
|---|---|
| 1 minute | Section goals and learning outcomes |
| 1 minute | Quick write: Who are your learners? |
| 2 minutes | Brainstorm and facilitated discussion: How do you know learning has occurred? |
| 2 minutes | Brainstorm and facilitated discussion: How do learners know what is expected of them? |
| 4 minutes | Write Pair Share: Write one item you want learners to know or be able to do. |
| 2 minutes | Write Pair Share: How do you know if learners have achieved the knowledge or skill? |
| 4 minutes | Mini lecture: Learning outcomes examples student centered, specific, active, and measureable. |
| 5 minutes | Individual work: Revise learning outcomes Small group discussion |
| 3 minutes | Facilitated discussion: What did we do in this session, and how does session activities relate to transfer? |
| 2 minutes | Small group reflection discussion: How can you apply transfer principles of guidance, scaffolding, and metacognition to your work? |
| 2 minutes | Formative clicker questions |
Appendix B Metacognitive and Reflective Clicker Questions
| Question | Section | Possible clicker responses |
|---|---|---|
| Which of the following ideas from this section are you most interested to apply to your classroom/work? (Section 31) | Section 1 | A. Use class time to read an example (multiple representations of material). |
| B. Have students discuss the example in small groups (elaboration and feedback). | ||
| C. Provide opportunities to practice. | ||
| D. Provide feedback by giving sample answers (practice with feedback). | ||
| E. Connect content to real life examples.2 | ||
| Section 2 | A. I will write learning outcomes. | |
| B. I will help my students develop their metacognitive abilities. | ||
| C. I will try a variety of class activities. | ||
| D. I will provide guidance to my students as they explore challenging topics. | ||
| E. I will scaffold learning.2 | ||
| Which of the following ideas from this section are most applicable to your classrooms? (Section 32) | Section 3 | A. I will use Peer Instruction to engage students with clickers. |
| B. I will design clicker questions beyond simple multiple choice. | ||
| C. I will use Ranking Task exercises that I find.1 | ||
| D. I will design Ranking Task exercises.1 | ||
| E. Promoting the value of Agile Lecturing.2 | ||
| Section 4 | A. I will find opportunities to use group work in my course. | |
| B. I will include metacognitive questions on exams. | ||
| C. I will find case studies that focus on my course content. | ||
| D. I will design/write cases for my course. | ||
| Which of the following ideas from this section are you interested to apply to your administrative role? | Section 1 | A. Mapping one of these features across a curriculum. |
| B. Embedding a feature into holistic faculty evaluation. | ||
| C. Examining the intersection of equity and these features. | ||
| D. Enhancing resources for professional development to support deep learning. | ||
| Section 4 | A. I will encourage incorporation of group work on campus. | |
| B. I will include metacognitive questions in campus wide assessment. | ||
| C. I will promote interdisciplinary teaching and learning. | ||
| D. I will support opportunities for inclusion of case studies in teaching and learning. | ||
| What did you learn about your own learning? | Section 1 | A. The social component (elaboration) to learning is important. |
| B. Novelty of content grabbed my attention. | ||
| C. Understanding the concepts was easier after I had seen examples in the story. | ||
| D. Having a partner to check my thinking with was valuable. | ||
| E. All of the above. | ||
| Section 2 | A. Group work facilitated interesting conversations. | |
| B. Having guidance through a series of steps helped me stay focused. | ||
| C. It was challenging to figure out parts of the task on my own. | ||
| D. Putting the section into the context of my own discipline helped me understand the relevance. | ||
| E. Learning outcomes for the session provided scaffolding. | ||
| Section 3 | A. Participants had to work together. (41) | |
| B. Participants were guided through a series of steps. (41) | ||
| C. Participants were given a challenging task and had to figure it out on their own. (41) | ||
| D. Participants focused on an idea specific to their own teaching. (41) | ||
| Section 4 | A. I benefited from group work.2 | |
| B. Scaffolding of the case contributed to my understanding.2 | ||
| C. A challenging real life task was motivating.2 | ||
| D. Multiple and varied representations encouraged interdisciplinary thinking.2 |
