“Let’s start a journal!”: The Multidisciplinary Graduate Student Journal as Educational Opportunity
Skip other details (including permanent urls, DOI, citation information)
This work is licensed under a Creative Commons Attribution 4.0 International License. Please contact firstname.lastname@example.org to use this work in a way not covered by the license. :
For more information, read Michigan Publishing's access and usage policy.
The University of Lethbridge is a medium-sized, primarily undergraduate, comprehensive research university on the Canadian Prairies in Alberta. It has a small but growing graduate school, within which most students are studying at the masters level. For many years, the graduate student elected representative body, the Graduate Students Association (GSA), has sponsored an annual refereed conference, Meeting of the Minds. In 2015 the GSA decided to supplement this conference with an accompanying journal, also called Meeting of the Minds. This article discusses the lessons learned in establishing this journal and overseeing its first two years of operations (and first year of publication). The article concentrates on two sets of problems: 1) philosophical, economic, and sociological issues that arose at the conceptual level while establishing a multidisciplinary, institution-focused graduate journal; and 2) technical, bibliographic, organizational, and economic issues encountered in attempting to address these conceptual concerns and ensure the long-term viability of the research accepted and published. Although the journal was not able to solve all the problems that arose during the first two years of operation, several solutions on the organizational, technological, economic, and bibliographic levels were developed that might be used by others establishing similar scholar- or student-led journals.
The journal and its institutional context
Meeting of the Minds (ISSN 2560-6670) is a graduate student–run scholarly journal at the University of Lethbridge in Alberta, Canada (for two access points to the journal, see Graduate Student Association [University of Lethbridge] 2017a, b).
The University of Lethbridge (U of L) is a medium-sized (approximately 8,500 students), primarily undergraduate university with a strong research mandate (for statistical information about the University, see University of Lethbridge 2017b). Despite its size and undergraduate focus, the U of L is one of four Comprehensive Academic and Research Institutions (CARIs) established by the Province of Alberta to “conduct pure and applied research in a wide range of disciplines” and serve “the needs of learners interested in a comprehensive, research intensive environment” (Government of Alberta 2007). The other CARIs are the University of Alberta in Edmonton, the University of Calgary, and Athabasca University. CARI institutions are distinguished in Alberta from more teaching-focused institutions such as baccalaureate and applied studies institutions (e.g., Mount Royal University, Grant MacEwan University), polytechnical institutions (e.g., Northern Alberta Institute of Technology, Southern Alberta Institute of Technology), and community colleges (e.g., Red Deer College) (see Government of Alberta 2007).
In keeping with its CARI status, the U of L has a small but growing graduate school of approximately 450 students who are pursuing their Masters mostly in the social sciences, humanities, and professional disciplines. Graduate instruction at the U of L is largely carried out on a one-on-one, apprentice-style basis: students are encouraged to identify a supervisor during the admissions process who then closely oversees their subsequent program, including signing off on coursework and supervising the development of their thesis prospectus. In many departments, there are few or no standing graduate courses; students complete their coursework through independent studies courses taught by the graduate faculty to individual students or small groups following related programs.
An emphasis on scholarly communication in the School of Graduate Studies
This type of a highly personal Master’s program across a range of research disciplines in a small but comprehensive research university brings with it advantages and disadvantages. On the one hand, the close relationship between entering graduate students and graduate faculty provides students with an intense apprenticeship experience more commonly associated at other universities with the PhD rather than the MA or MSc. On the other, this same intensity and focus on the individual can make it difficult for students to interact with others at a similar career stage in their discipline. Particularly in the social sciences and humanities, which enroll fewer students than the sciences or professional disciplines, it is not unusual in some departments for a “cohort” to consist of no more than one or two students (if that) in any given year.
As a means of addressing this potential for isolation, the U of L School of Graduate Studies (SGS) and Graduate Student Association (GSA) devote considerable attention to developing opportunities for in-program students to cultivate their skills in scholarly communication. In collaboration with Mitacs, a non-profit organization that manages and funds training programs in the post-secondary sector, the SGS offers graduate students workshops on a variety of professional skills, including writing and presenting to academic, commercial, and public/non-specialist audiences (Mitacs 2014; University of Lethbridge 2017c). The GSA and SGS also support participation in nationally and internationally recognized communication competitions and programs such as the “Three Minute Thesis” and “Images of Research” (Canadian Association of Graduate Students 2017; University of Lethbridge 2017a, e). Moreover, since 2006 the GSA has sponsored an annual refereed interdisciplinary graduate conference, Meeting of the Minds (Graduate Student Association [University of Lethbridge] 2017c).
In addition to providing students with opportunities to share their work with others and training in the communication of research to peers and the wider public, the U of L has also devoted considerable attention over the last decade to introducing students to the latest developments in scholarly communication as a discipline. It has offered seminars and colloquia to students on current trends in digital scholarly communication and reputation management. The Lethbridge Journal Incubator, a training initiative of the U of L library and SGS to address scholarly communication in an open-access, digital environment, has trained several generations of students in journal management, editorial practice, and scholarly publishing (Cowan 2013; O’Donnell et al. 2015). In 2015 the Incubator was included in a newly designated Centre for the Study of Scholarly Communication (CSSC), placing this work in the context of a broader, institutional focus on promoting best practice in contemporary scholarly communication (Centre for the Study of Scholarly Communication 2017).
The Meeting of the Minds graduate journal
The Meeting of the Minds graduate student journal was conceived of as an element of this approach. Proposed and funded by members of the GSA executive and supported by the SGS, the goal of this journal was to develop a supportive forum for the publication of graduate student work, particularly work associated with the annual Meeting of the Minds conference. The journal also sought to provide students with training and experience in writing, editing, and refereeing scholarly and scientific research, drawing on already existing programs within the SGS, CSSC, and Lethbridge Journal Incubator.
Beyond these two goals, the members of the original editorial board also established a vision for the kind of journal they wanted to produce. Like the conference from which it was derived, the Meeting of the Minds journal would be multidisciplinary and consider work from all subjects taught at the U of L. As is also true of the conference, the refereeing and editorial process would be supportive, helping students get work ready for publication rather than identifying and rejecting work that was not yet at this standard (i.e., gatekeeping). In addition to regular contributions such as reviews and research articles, the journal would also consider work that might otherwise prove difficult to publish in mainstream journals: extended abstracts, blog posts, teaching materials, research data sets, and so on (for a justification of this approach, see Public Library of Science n.d.). The referees (or, better said, reviewers) were to be drawn from the graduate faculty and students (for a discussion of the importance of training in understanding peer review, see APA Science Student Council 2007; Strickland 2016), and the editorial work was to be carried out entirely by students under the guidance of faculty from the CSSC.
The rest of this article discusses the lessons learned in establishing this journal and overseeing its first two years of operation. Despite beginning with what was considered a clear vision, a solid precedent in the annual conference, and an obvious need for a publication forum for student work, the journal failed to catch on in its first year with authors. A number of philosophical, economic, and sociological reasons inherent to contemporary academia were at play—from the fundamental conservatism of what remains very much a prestige economy (Blackmore and Kandiko 2011; English 2009; Fyfe et al. 2017; Suber 2010) to what seemed to be a lack of agency on the part of students in some disciplines in which collaborative research is the norm. The journal received a strong expression of interest from potential editors and reviewers but only one submission.
In the second year, the journal addressed these concerns by rethinking the mission and reconsidering the approach to core aspects of scholarly communication, from the role of reviewers and editors to the question of what scholarship means for students who are beginning their professional lives. The Call for Papers was redesigned to focus on the kind of work that cannot be published in a disciplinary journal, and the review process was changed to focus even more explicitly on mentorship rather than gatekeeping.
Together, these changes resulted in balanced and greater success: the journal maintained strong editorial interest and attracted the work of an equal number of authors. In doing so, some unforeseen problems in how graduate students and their supervisors understand the publication process were discovered, such as problems interpreting peer review and a Call for Papers that worked extremely well in some disciplines but not others. The solutions arrived at for these problems are in most cases easily generalized to similar publishing problems.
In its third year, the journal began to struggle again, this time editorially rather than authorially as the initial editorial board at the primarily masters-level completed their programs and moved on to other endeavors. Although author interest remains high, the journal was forced by a lack of editors to go back into hibernation at the end of the 2017–2018 academic year. The journal was designed specifically to allow for this possibility (O’Donnell et al. 2018).
In the end, the lessons learned, particularly about editorial workflow, establishing a supportive review system, and using the journal to educate students in scholarly communication, remain useful. After an initial year in which almost no authors contributed to the journal, considerably more authorial interest was captured in the second and third years by focusing on the areas in which a graduate journal can deliver experience and training not found in any other context.
First year: Too many editors, not enough authors
As mentioned above, in its first iteration, the Meeting of the Minds journal was conceived primarily as a support forum for the annual graduate conference of the same name. Although the editors and faculty advisors considered publishing other forms of research, they assumed that most submissions would be conference participants’ abstracts or papers. Because these forms of scholarly communications can be difficult to publish in mainstream disciplinary journals, they represented a ready potential source of content for a graduate-run local journal.
In early 2016 concurrent calls were issued for submissions to the journal and volunteers to staff the journal’s editorial board. Perhaps surprisingly, while the Call for Papers attracted only one submission, the call for editorial assistance resulted in a strong response, as sixteen graduate students stepped forward from departments across the social sciences, humanities, natural sciences, and professional faculties. The disparity in the volume of responses received from potential authors and editors seemed to indicate a need for additional consideration of the scope and purpose of the journal. A decision was made to suspend publication until the following academic year and use the intervening period to explore what had gone wrong.
Reviewing the journal’s previous experience, the project leaders and faculty advisors concluded that the lack of submissions could have resulted from any number of factors, including:
- a general hesitation to submit articles to an unproven publication;
- disciplinary publication practices (particularly in the sciences, where most U of L graduate students work) that discouraged single-author or student-only publications in favor of collaborative work with supervisors and other faculty members;
- a reluctance to publish conference abstracts or papers (either due to unfamiliarity with the form or fear of preempting subsequent more impactful publications in professional disciplinary journals);
- a lack of student autonomy or ownership of the results of research collaborations, preventing them from determining where to publish results they have helped to produce;
- and pressure from supervisors (largely but not exclusively in the sciences) who consider publication in a graduate journal to be a distraction rather than an apprenticeship/training opportunity.
It was clear from the interest shown by potential editors that there was a demand for the kind of training the graduate journal would provide. It was also clear from the lack of submissions that the journal was far less attractive than had hoped as a forum for actual publication by students, at least under the original mission.
Rethinking the mission
With the beginning of the 2016–2017 academic year, a new editorial board met to revamp the journal’s approach and mission. The board’s working assumption was that the most important factors restricting the number of submissions were the journal’s lack of disciplinary standing and prestige and, as a result, the sense either on the part of the students or through pressure from supervisors that it was a less than ideal forum for students’ best work (i.e., factors 1 and 5 above). Or, as one faculty member put it, “if a graduate student has something important to publish, shouldn’t they publish it somewhere important?”
At the same time, however, the editorial board also continued to believe that a journal aimed at and run by graduate students remains an important and constructive initiative for the U of L graduate school. Whereas students probably should publish important work “somewhere important,” evidence suggests that current professional publication practices and expectations provide less than optimal professional development and training for graduate students. Students receive little direct training in publication practices from their supervisors (see Brown 2012 for a summary). Those who publish as junior members of large research teams often have very little control or even direct input into the publications themselves (Lei and Chuang 2009). Too narrow a focus on disciplinary practice, moreover, can ultimately work against the professional development of individual students, reducing their “academic literacy” (Bastalich, Behrend, and Bloomfield 2014).
Inherent conservatism in the traditional apprenticeship model by which students tend to be professionalized is also an issue. Scholarly communication is undergoing what is likely to be the most significant and rapid change since the publication of the first journals 350 years ago (Bourne et al. 2011). Students are trained for this disrupted world by being asked to emulate the practices of scholars who in many cases began their careers and booked their first successes before several of the most significant of these changes occurred. Few supervisors have explicitly studied best practice in contemporary scholarly communication, even within their own research domains. This means that students are being trained in their core professional activity by instructors who often rely on tacit knowledge based on how things have worked for them in the past rather than explicit knowledge of current best practice. These instructors may therefore lack core competencies in the new information literacy (see, in various contexts, Carlson et al. 2011; Porter 2013; Genoni, Merrick, and Willson 2006).
The editorial board concluded that these competing pressures led to the lopsided interested in editing, rather than contributing content to, the journal. Anecdotal evidence from publication workshops and participants in the Lethbridge Journal Incubator program suggests that graduate students believe there is more to contemporary scholarly communication than they are necessarily learning through traditional channels. The reluctance to submit research for publication, however, reflects a real and potentially legitimate concern about the relationship of a graduate journal to traditional professionalization as this is being taught and evaluated by their supervisors, colleagues, and potential employers.
A solution: “Publish the unpublishable”
As a solution to the lack of submissions, the editorial board for Meeting of the Minds decided to focus on the (traditionally) “unpublishable.” On one hand, this term could be used to cover various forms not commonly accepted by traditional professional journals but increasingly seen in contemporary practice: extended abstracts; discussion and technical papers; cross-disciplinary projects; data sets; methods and protocols; popular pieces; work on teaching; and meta-discussions of graduate study within a student’s discipline. On the other hand, it could also encourage students to experiment with entirely new forms of publication: “iconoclastic” and “experimental” genres that represent more a testing of the boundaries and conventions of contemporary practice than practice in more widely recognized but novel forms of publication.
Second round success: Eight editors and nine submissions
A new Call for Papers, explicitly addressing this question of “publishability” in a graduate context, went out under the theme “Publishing the unpublishable” in the fall of 2016. The strategy had a much greater success, resulting in a total of nine submissions: eight articles (i.e., submissions corresponding more-or-less to traditional research outputs, albeit in one case in a very contrarian fashion) and one blog (a submission that attempted to adopt what its author argued was an explicitly less formal, non-scholarly approach to a research topic).
The editors were very pleased with this response and the generally high quality of the submissions. At the same time, however, the submissions also led immediately to a second lesson about running a multidisciplinary graduate journal. While the new theme and approach had increased the number of submissions to the journal, it had done nothing to address the range of disciplines they represented. All of the papers received came from students in the social sciences and (especially) humanities. Although the editorial board was well distributed across the disciplines, the new theme had clearly failed to resonate with students within all disciplines of the comprehensive graduate school.
The editorial board later discovered that the lack of submissions from students outside of the social sciences and humanities was in part due to differences in disciplinary conventions. Whereas students in the social sciences and humanities seemed relatively comfortable with a call built around a provocative and partially ironic theme, students in the sciences said that they and their supervisors understood things much more literally—that is, as a nonsensical call for submission for publication of papers that their authors considered not suitable for publication. One board member reported a conversation in which she had been explaining the journal to a science student when a faculty supervisor had stepped in and told the student she would not be publishing in the graduate journal. Another member reported snide comments being made in a lab about an alleged journal that would publish trash (otherwise known as unpublishable material). The obvious lesson was that, in this section of the university at least, the theme had sparked confusion and condescension instead of experimentation and interest. The board has since discussed how to address this issue in the future, but, short of deciding to focus primarily on the social sciences and humanities, it has yet to come up with an approach that might bridge this disciplinary gap and make the journal more attractive to researchers in the natural sciences.
What if something really is unpublishable?
A third issue that arose from the call had to do with the question of standards. By explicitly encouraging authors to submit things they might not otherwise be able to publish, the editors had hoped to encourage students to push the bounds of traditional scholarly communication and use the journal for publication experiments that they might not be able (or dare) to pursue in more traditional professional contexts. This left open the question of minimum standards, however: Were we committing to publish anything? Were there no formats, types of content, or approaches that lay beyond the pale of our journal?
To the extent that the editorial board had thought of this before issuing its call, they anticipated that it would involve an issue of quality or genre. That is, the board expected to receive some very early drafts or research that reviewers considered immature (low quality) or material in recognizable genres that are not commonly seen in scholarly contexts—for example, poems, short stories, or other forms of creative work. In designing the journal, the board had, indeed, anticipated the need to accommodate this. Although the core of the journal was designed to support and provide archiving and metadata for traditional forms of scholarly publication, the journal also adopted a system to publish blogs and similar types of submissions in a suitable way (i.e., WordPress, allowing comments, etc.).
In the end, the submission that tested the journal’s limits involved an issue not anticipated: authorial intention. For seven of the eight articles received, the authors had submitted material with a clear intention to communicate evidence-based research and argument (indeed, they were, quite conservative from a generic point of view). The eighth article, conversely, was a first-person, profanity-laced discussion of local transportation systems. It had no citations and, beyond invoking “autoethnography” as a method, self-consciously and explicitly rejected scholarly method and analysis. The editor assigned to this submission, a graduate student in neuroscience, found it impossible to see any merit in a submission of this sort to a scholarly journal. The editorial board was thus faced with the first real challenge to its approach, theme, and conception of the journal. Was this the type of iconoclastic, unusual, and unpublishable piece they had encouraged with their call or was it in fact unpublishable? How do you publish something that rejects academic method in an academic journal?
The board was initially reluctant to decide and appealed to the faculty advisors for help. The advisors, for their part, felt that this was something for the students to decide—determining what to include in a journal is a core editorial decision, after all. Eventually, the editorial board decided that this piece was simply too unpublishable to publish in an academic journal and that the author’s explicit rejection of scholarly convention and method meant that there was nothing for the editors to work with. The result was a desk rejection.
The review and mentorship process
The seven remaining articles were sent out to reviewers for comment and mentoring (the blog was ultimately published to the author’s own site rather than in the journal). Each reviewing pair (one faculty member and one graduate student) was asked to fill out a Reviewer’s Questionnaire and attach an annotated version of the article for the author’s benefit (see appendices). Editors would then summarize and highlight what they considered to be the most important aspects of the reports (which allowed the editor to hone their skills in interpreting reviewer reports) and either pass on the additional comments verbatim or paraphrase them if the editor felt they were not constructively phrased.
Avoiding yes/no decisions
During the initial, unsuccessful call for papers in 2015, the editorial board decided that the journal would operate on a “peer mentorship” rather than “peer review” basis. This meant that the journal would be biased toward publication in all cases except the most seriously inappropriate work, such as the explicitly non-academic submission discussed above. In cases where external readers felt that a submission was not ready for publication, the editors and reviewers, most of whom were students and faculty at the U of L, would commit to working with the authors to bring the quality of the submissions up to standard. The editorial board realized that this decision was a model that would neither scale easily nor reflect the standard practice among academic journals. They felt, however, that scale was unlikely to become a problem for the near future and that the benefits of faculty mentoring outweighed any lack of “realism” in the review process.
Having made this early policy decision, the board devoted considerable thought to its Reviewer’s Questionnaire. Based on the experience of faculty advisors and some students who had experience with the review process through the Lethbridge Journal Incubator, the board anticipated having some difficulty in ensuring a consistent approach and philosophy toward publication and mentorship amongst reviewers and editors (see, e.g., D’Andrea and O’Dwyer 2017). In a standard journal, referees are asked to recommend rejection or acceptance and the purpose is to serve as a filter (see Biagioli 2002). The goal in such cases is to prevent work from being published unless it meets some minimum standard for professional research. Although good reviewers and editors offer constructive criticism to papers they reject (as well as to those they recommend for publication), the emphasis of the process is determining whether a paper meets a minimum standard rather than how it could be brought into alignment with such a standard.
In the case of Meeting of the Minds, the editorial board was asking reviewers to help decide how authors could improve submissions until they were ready for publication, regardless of their initial state. Amongst other things, this meant that there was no real gate for the reviewers to keep when they evaluated manuscripts. The board did not expect to reject anything, so even a negative evaluation would mean simply that more work had to happen. The board wanted to steer reviewers away from making a “yes/no” recommendation on a given paper. Instead of recommending whether a paper was publishable, the board wanted the reviewers to comment on how it could become publishable.
In designing the review form, the board deliberately did not give reviewers an opportunity to say whether they thought the submission was publishable. They were asked instead to state whether they thought the paper was “good graduate level work,” a question that regrettably ended up becoming a proxy for “accept/reject” in some reviewers’ and students’ eyes. Based on the model of the Public Library of Science, the questionnaire also asked reviewers to discuss the paper’s overall soundness and strengths and weaknesses.
Non-anonymous review and managing conflicts of interest
One additional problem that immediately arose regarding the review process was how “blind” it should be. Although double blind peer review (i.e., the name of the author is not revealed to the reviewer and the name of the reviewer is not revealed to the author) is often understood to represent best practice, the conditions of this journal made adopting such a model almost impossible. At a small school with an even smaller graduate program, there are simply not enough people to ensure anonymity. Faculty members tend to have non-overlapping areas of expertise, and students are few enough that any reviewer with the expertise to review a piece is almost certainly able to guess the author from the topic alone.
Compounding this problem of identity was the potential for formal conflicts of interest. Given the small size of the school, it was going to prove almost impossible to find reviewers who were not in some way connected to the author as classmates, instructors, employers, supervisors, or, especially in the sciences and some of the social sciences, as co-authors.
At the same time, some relationships really were too close to be managed even in a non-blind system. Most participants would be in some form of unavoidable conflict: thesis/program supervisors, for example, or, in the case of papers that had been previously submitted as coursework, the instructor responsible for grading the piece. In both cases, the potential reviewer had too great a personal stake in the outcome of the process. In the case of supervisors and their students, it could prove difficult to separate the submission review process from the larger pedagogical process by which their theses, which might be the source of the article, were being written and supervised. In the case of course instructors, if they were asked to review submissions they had previously graded, the exercise would almost inevitably require them to validate their original assessment or to adjudicate how far the paper had improved (or not) since its original appearance as coursework.
Beyond this, however, the board anticipated, and indeed discovered, that it would be impossible to avoid choosing reviewers who, in other contexts, would be considered too close to the authors. For example, the history department’s nine graduate students, which included two editorial board members, contributed five submissions to the first two issues of the journal. This meant that both student and faculty reviewers would inevitably have intellectual and personal relationships with on another that, in a professional national or international journal, would be considered too close for the reviewing process.
No double blind
Given these two interrelated, and thus unavoidable, problems, the editorial board decided that a double-blind review process was not essential to the journal’s core mission of providing an experiential learning and mentorship opportunity to student authors. Instead, they proposed a hybrid solution in which one faculty reviewer and one student reviewer would collaborate in guiding a submission by a known author toward publication. Although initially in each case blind reviewers (i.e., readers whose identities were not known to the authors) were used, the maintenance of this anonymity was not considered to be crucial. In some cases, reviewers asked to work directly with students in getting their work up to publishable standard; in other cases, it was simply too hard given disciplinary specializations to maintain anonymity.
By focusing the peer review process on mentorship rather than gatekeeping, the board removed the responsibility of evaluating the publishability of the article from the reviewer, thereby diminishing the importance of double blind reviews and avoiding low-level conflicts of interest. This hybrid process, moreover, also provided potential authors and reviewers multiple points of contact as articles moved from submission to publication. This process incentivized both student submissions by promising close collaboration with faculty reviewers and faculty participation by promising multiple points for student contact and education. Students were encouraged to record their participation in the process on their CVs, while faculty members received an official letter of thanks for inclusion in their annual performance reviews.
Interpreting peer review
The significant difference between submitting an article to a journal rather than a paper to a professor became an unforeseen difficulty of working with novice authors. Instructors, as a rule, have a longer and closer relationship with the authors of the papers they grade than do reviewers with authors of papers they evaluate. They also have a higher stake in the long-term success of the student-authors. The focus of their evaluation is therefore usually the development and certification of the student rather than serving as a gatekeeper for any single piece of work. This is true even with theses, for which supervisors and committee members devote time to mentoring work they then evaluate. The complexity of this relationship is reflected in the grades available to instructors for use in evaluation—ranging from failure or “Minimal pass” through “Satisfactory,” “Good,” and “Excellent” (i.e., the A+–F grading system; see University of Lethbridge 2017d, 65). Given all this, students submitting papers are used to receiving constructive comments on their work that tend to concentrate on its strengths and opportunities for improvement. Perhaps especially in the case of high-achieving students who pursue a graduate education, students rarely if ever see gatekeeping style comments on their papers—that is, comments focused on what is wrong with a paper or why it is unacceptable.
Peer reviewers, conversely, even in a mentorship-focused journal such as Meeting of the Minds, have a much more instrumental and impersonal relationship with authors. Their job is to evaluate the submitted manuscript, rather than develop or certify the author. The question before them traditionally, therefore, is whether a piece is acceptable or not. Even in a mentorship-oriented process, reviewers and editors are still measuring submissions against the relatively high standard of publishability—meaning presumably some combination of originality, technical competence, and interest or novelty. Instructors focus on the value of a paper as a lesson for future work: Does the paper succeed in ways that students can emulate in their next paper or show flaws that the students should learn to avoid next time? Reviewers and editors, however, are focused primarily on the positive and negative qualities of the work in hand: Does this paper prove its case and is it up to publishable standards?
Possibly because of this difference, several authors who submitted to the journal found the peer review process difficult. Although our reviewers were mostly thorough and constructive, and the reviews they provided largely positive, their reviews retained enough of the gatekeeping function to make some authors (and editors) uncomfortable. The student editors and authors both tended to interpret the reports much more harshly than did the faculty advisors on the editorial board. Several authors responded to constructive, critical review reports or detailed copyediting by asking to withdraw their papers. In one case, when a negative answer was given to the first question on the form (“Is the submission considered a good graduate level work within the particular discipline?”), the editor worried that the report, which was otherwise very constructive and quite strong, might cause the author of the piece to withdraw the submission or become demoralized.
The editorial board had not anticipated either of these problems. In the case of the students who wanted to withdraw their papers after receiving detailed criticism, one-on-one mentorship and instruction on how to interpret and profit from such reports was helpful. In the end no student withdrew a paper after submission. The editors, some of whom had never seen an evaluation of work other than their own, required some group discussion on the nature of the review context and some experience in brokering reviews to learn how to understand and present direct and frank criticism to student authors.
In retrospect, because real-world review reports at professional journals are commonly not as constructive or careful as these mentorship-oriented reports, this training in the interpretation of peer review reports and copyediting suggestions may be one of the more valuable aspects of the program.
Authors were given their review reports and editorial suggestions and asked to revise and resubmit over the Christmas holidays. The editorial managers then negotiated back and forth with the authors until each were satisfied that the submission had reached a publishable standard of scholarship. This, once again, was a curious process, because the editorial manager in charge of each submission was generally a student with little to no publication experience other than as a reader.
Articles were copyedited by the editorial managers and then passed on to production, which was managed by one student and one faculty advisor (for more detail, see O’Donnell et al. 2018). The journal published articles as PDFs, preparing them in Word using a style template (see Scholar Space 2011 for a guide). They were then published to a “Community” repository on Zenodo (see https://bit.ly/MotM-Zenodo), the CERN-based Open Science Repository funded by the European Union OpenAIRE project, which provided both Digital Object Identifiers (DOIs) for each article and permanent archival storage. Metadata for and links to each article were also published to a WordPress-hosted site at a custom domain purchased by the GSA (https://ulgsajournal.com/publications/). The articles were released as soon as the authors approved their proofs, and the entire issue was published between March 12 and 21, 2017.
The editors of the Meeting of the Minds graduate journal were under pressure from the GSA to ensure that the first issue of the journal was published to coincide with its namesake conference on March 17 and 18 of 2017. Although the journal did not make that date in all cases, it was able to publish five of seven accepted articles before the conference began. In two cases, complications with the production or proofing process led to delays in final publication. By the time the articles were released, the journal had acquired a Facebook page, a Twitter handle, and a recurring presence on monitors used to broadcast campus news across the university.
These vectors, combined with access to the graduate student email list, were enough to meet basic social media requirements for marketing the publication. Several members of the board, however, argued that the journal needed to do more to advertise itself, so a party with DJ and band was organized with sponsorship from a local car dealer.
A final surprise involved how the journal was received once it was published. On the one hand, the authors and the GSA reported to be very happy with the result, although there had been some friction with the GSA about whether a party was the appropriate way to launch an academic journal. On the other, some resistance to and criticism of the journal came from the university community, again primarily, but not exclusively, from researchers in the sciences.
The journal’s decision to use largely internal reviewers and its promise to mentor articles to publication led some students and faculty members to express concern that the peer review process was not rigorous enough to produce a truly excellent journal. Oddly, some students criticized the fact that authors were asked to provide names of potential reviewers—an approach taken by many professional disciplinary journals.
The journal also received continuing criticism about the very idea of mentorship. In a post to the Facebook page “U of L Confessions” on March 25, 2017 (just as the last of the seven articles were being published), an anonymous critic asked:
Am I the only one who thinks the graduate journal is a big running joke? What, is it too tough to get published in a standard journal, you need to create one with less restrictive requirements for publication? The peer-review process might need to be reviewed, but the solution is not to reduce the amount of rigour involved in in scrutinizing academic work.
And finally, after the first issue was published, the journal received concerned inquiries from faculty members about how it did or did not match what they considered to be the essential criteria for “academic publication”: Is there an ISSN? Does the journal use DOIs? How are articles archived? To what extent does the journal encourage students to divert articles that could be published in professional disciplinary journals to this “less professional” graduate effort?
Postscript in lieu of a conclusion
As they anticipated, the editorial board of the Meeting of the Minds journal underwent a major membership change between its first and second year of publication. The second call for papers brought in a similar number of papers, this time including papers from students from outside the university and one from a University of Lethbridge undergraduate. The churn in the editorial board and several unanticipated leaves, including of one faculty advisor, however, resulted in difficulty editing the second issue. The journal was forced to go into hibernation in the spring of 2018.
While this was regrettable, the possibility that turnover in the primarily Masters-level graduate school might prevent maintaining the journal from year to year was built into the journal’s design assumptions (see O’Donnell et al. 2018 for a discussion). Regardless of the form the journal takes in future years, however, putting the journal together was a useful professional experience and established a template for apprentice researchers in the U of L graduate school, should a new team of students want to take up the challenge.
Information sent to faculty members
Contacting Faculty Reviewer
[Introduce yourself, your affiliation] and am currently acting as an editor for the Meeting of the Minds Graduate Journal. The Graduate Journal is an interdisciplinary, open-access journal aimed at providing mentorship opportunities for students who are interested in the journal reviewing and writing process. On behalf of the editorial board, I would like to invite you to participate in the reviewing process for the Meeting of the Minds Graduate Journal.
Submissions have been divided to respective editors and we are currently looking for faculty and graduate reviewers. The role of the reviewers is to facilitate student mentorship in terms of providing feedback and comments on the student’s piece of work and communicating with the editor. Further details on this process are given below.
Currently, [author’s name] submission titled [Title of article] is under review and your help with this process will be greatly appreciated! [author’s name] is a [affiliation with the University and specific department]. Please let me know if you may be available to help with the review process or if you can direct us to someone within your department that may be interested in mentoring graduate students through the publication process.
Please suggest a graduate student who you feel would be an appropriate peer reviewer on this submission. We ask that you work with said graduate student to help them understand the process of peer reviewing.
Thank you for your time!
The Meeting of the Minds Graduate Journal uses a mentorship process to evaluate submissions rather than a strict review process. Put simply, we are trying to improve submissions to a publishable level rather than trying to publish only the best of the best. Please keep this in mind as you review your submission.
As a reviewer, you are asked to fill out the attached Reviewer’s Questionnaire, as well as provide a copy of the submission with relevant annotations (using Word’s ‘Track Changes” function, a PDF annotator, or equivalent application).
(Attach article and review questionnaire)
Original reviewer’s questionnaire
**This document includes questions reviewers will use to determine the level of work that has been submitted. Managing editors will receive these documents from reviewers and/or faculty, from which comments are interpreted, and the ME will make the final decision of “accept” or “requires further revisions”. **
- Is the submission considered a good graduate level work within the particular discipline?
- What is being done correctly by the graduate student (if author is a graduate)?
- Describe strengths and weaknesses of the submission.
- Please suggest improvements to the piece.
- Is there literature that should be included in the submission that can help improve the piece?
- If you have any comments not intended for the author, please share them here:
Revised reviewer’s questionnaire
- Describe strengths and weaknesses of the submission.
- Please suggest improvements to the piece.
- Is there literature that should be included in the submission that can help improve the piece?
- If you have any comments not intended for the author, please share them here:
The corresponding author is dpod. Authorship is alphabetical after the first three drafting authors. Author contributions, described using the CASRAI CRedIT typology (http://casrai.org/credit), are as follows:
- Conceptualization: pe,cv,ku,dpod;
- Methodology: pe,cv,ku,dpod;
- Software: dpod;
- Investigation: sc, kd, pe, sf, rg, jm, rp, cr, vs, et, cv, ku, dpod;
- Resources: ku,dpod;
- Data Curation: dpod;
- Writing – Original Draft Preparation: pe,cv,dpod;
- Writing – Review & Editing: pe,dm,cv,dpod;
- Supervision: rg,dpod;
- Project Administration: rg,dpod;
- Funding Acquisition: ku,dpod.
Paul Esau is a PhD student in History at Wilfrid Laurier University in Waterloo, Ontario.
Carey Viejou is a Master of Arts student in the History department at the University of Lethbridge. She studies women’s political activity in Alberta in the early 20th century.
Sylvia Chow is a graduate student at the University of Lethbridge.
Kimberly M. Dohms is an Environmental Assessment Coordinator at the Canadian Wildlife Service. She completed her PhD in Evolution and Behaviour at the University of Lethbridge and is active in avian research, science outreach, and social activism.
Steve Firth holds an MA in Philosophy and is a PhD candidate at the University of Helsinki in Finland. He is working on ethics —especially those relating to disability and authors a philosophical blog entitled “Hail Fellow Well Met,” which seeks to advance philosophy and knowledge through accessible discussion of everyday dilemmas and events.
Jarret McKinnon is a Master’s student in the Department of Neuroscience at the University of Lethbridge. His research is geared toward the understanding of learning, memory, and cognitive processes and how these interact and contribute to semantic knowledge.
Dorothea Morrison is an MA English student at the University of Lethbridge studying the intergenerational transmission of gender in the fiction of Margaret Atwood.
Reed Parsons is a graduate student at the University of Lethbridge.
Courtney Rieger is a graduate student at the University of Lethbridge.
Vanja Spirić graduated from the Individualized Multidisciplinary MA program with a concentration in Sociology. She participated in the Lethbridge Journal Incubator program in 2016 and served as a board member on the Meeting of the Minds Graduate Journal in 2016–2017.
Elaine Toth earned a BA with Distinction from the University of Lethbridge in April 2015. She is currently an MA Student in History. The working title of her thesis is Childhood Agency: Life Stories of Post-Second World War Dutch Immigrant Children to Alberta.
Kayla Ueland is a second year law student at the University of Calgary.
Rumi Graham is a University Copyright Advisor and Graduate Studies Librarian.
Daniel Paul O'Donnell is a Professor of English and Associate Member of the Academic Staff of the University of Lethbridge. He is the editor of Digital Studies/Le champ numérique and supervises the Lethbridge Journal Incubator.
- Bourne, Phil E., Tim Clark, Robert Dale, Anita de Waard, Ivan Herman, Eduard Hovy, and David Shotton, eds. 2011. “Improving the Future of Research Communication and e-Scholarship.” Force11 White Paper, 27 October. https://www.force11.org/about/manifesto.
- American Historical Association. 2013. “AHA Statement on Policies Regarding the Embargoing of Completed History PhD Dissertations.” AHA Today, July 22, 2013. http://blog.historians.org/2013/07/american-historical-association-statement-on-policies-regarding-the-embargoing-of-completed-history-phd-dissertations/.
- Anderson, Rick. 2013. “Dissertation Embargoes and the Rights of Scholars: AHA Smacks the Hornet’s Nest.” The Scholarly Kitchen. July 26. https://scholarlykitchen.sspnet.org/2013/07/26/dissertation-embargoes-and-the-rights-of-scholars-aha-smacks-the-hornets-nest/.
- APA Science Student Council. 2007. “A Graduate Students’ [sic] Guide to Involvement in the Peer Review Process.” APA. https://www.apa.org/research/publishing/peer-review.pdf.
- Bastalich, Wendy, Monica Behrend, and Robert Bloomfield. 2014. “Is Non-Subject Based Research Training a ‘Waste of Time’, Good Only for the Development of Professional Skills? An Academic Literacies Perspective.” Teaching in Higher Education 19 (4): 373–84. https://doi.org/10.1080/13562517.2013.860106.
- Biagioli, Mario. 2002. “From Book Censorship to Academic Peer Review.” Emergences: Journal for the Study of Media & Composite Cultures 12 (1): 11–45. https://doi.org/10.1080/1045722022000003435.
- Blackmore, Paul, and Camille B. Kandiko. 2011. “Motivation in Academic Life: A Prestige Economy.” Research in Post-Compulsory Education 16 (4): 399–411. https://doi.org/10.1080/13596748.2011.626971.
- Brown, William Christopher. 2012. “Developing Professionally through Graduate Student Publications.” In Progress . http://inprogressjournal.net/archives/current-issue/developing-professionally-through-graduate-student-publications/.
- Canadian Association of Graduate Students. 2017. “National 3-Minute Thesis Competition.” Canadian Association of Graduate Students. http://www.cags.ca/3mt.php.
- Carlson, Jacob, Michael Fosmire, C. C. Miller, and Megan Sapp Nelson. 2011. “Determining Data Information Literacy Needs: A Study of Students and Research Faculty.” Portal: Libraries and the Academy 11 (2): 629–57. https://doi.org/10.1353/pla.2011.0022.
- Centre for the Study of Scholarly Communication. 2017. “Centre for the Study of Scholarly Communication.” Centres & Institutes. http://www.uleth.ca/research/centres-institutes/centre-study-scholarly-communication.
- Cowan, Sandra. 2013. “Open Access Journal Incubator at University of Lethbridge Library.” In Library Publishing Toolkit, edited by Allison P. Brown, 179–86. Geneseo, NY: IDS Project Press, Milne Library, SUNY Geneseo. http://idspress.idsproject.org/omp/index.php/IDSProject/catalog/book/25.
- D’Andrea, Rafael, and James P. O’Dwyer. 2017. “Can Editors Save Peer Review from Peer Reviewers?” PLOS ONE 12 (10): e0186111. https://doi.org/10.1371/journal.pone.0186111.
- English, James F. 2009. The Economy of Prestige: Prizes, Awards, and the Circulation of Cultural Value. Cambridge, MA: Harvard University Press. http://books.google.ca/books?id=vY3UOFDA2sAC.
- Fister, Barbara. 2013. “The AHA Asks ‘What About the Children?’” Inside Higher Ed, July 24. https://www.insidehighered.com/blogs/library-babel-fish/aha-asks-what-about-children.
- Fyfe, Aileen, Kelly Coate, Stephen Curry, Stuart Lawson, Noah Moxham, and Camilla Mork Rostvik. 2017. “Untangling Academic Publishing: A History of the Relationship between Commercial Interests, Academic Prestige and the Circulation of Research.” University of St. Andrews. https://research-repository.st-andrews.ac.uk/bitstream/handle/10023/10884/Fyfe_etal_UntanglingAcPub_CC.pdf?sequence=1.
- Genoni, Paul, Helen Merrick, and Michele A. Willson. 2006. “Scholarly Communities, E-Research Literacy and the Academic Librarian.” The Electronic Library 24 (6): 734–46. http://www.emeraldinsight.com/doi/abs/10.1108/02640470610714189.
- Government of Alberta. 2007. “Comprehensive Academic and Research Institutions.” Alberta Innovation and Advanced Education. December 31. http://advancededucation.alberta.ca/post-secondary/institutions/public/types/cari/.
- Graduate Student Association [University of Lethbridge]. 2017a. “Meeting of the Minds Graduate Journal.” Meeting of the Minds Graduate Journal. https://ulgsajournal.com/.
- ———. 2017b. “Meeting of the Minds Graduate Student Journal at the University of Lethbridge (ISSN: 2560-6670).” Zenodo. https://zenodo.org/communities/meetingofmindsjournal-universityoflethbridgegsa/?page=1&size=20.
- ———. 2017c. “Meeting of the Minds.” Graduate Students’ Association. December 13, 2017. http://www.ulgsa.ca/mom2018/.
- Lei, Simon A., and Ning-Kuang Chuang. 2009. “Research Collaboration and Publication during Graduate Studies: Evaluating Benefits and Costs from Students’ Perspectives.” College Student Journal 43 (4): 1163–68. http://search.ebscohost.com/login.aspx?direct=true&db=s3h&AN=55492494&site=ehost-live&scope=site.
- Mitacs. 2014. “About Mitacs.” Mitacs. April 13. https://www.mitacs.ca/en/about-mitacs.
- Moore, Samuel, Cameron Neylon, Martin Paul Eve, Daniel Paul O’Donnell, and Damian Pattinson. 2017. “‘Excellence R Us’: University Research and the Fetishisation of Excellence.” Palgrave Communications 3 (January). https://doi.org/10.1057/palcomms.2016.105.
- O’Donnell, Daniel Paul, Heather Hobma, Sandra Cowan, Gillian Ayers, Jessica Bay, Marinus Swanepoel, Wendy Merkley, Kelaine Devine, Emma Derin, and Inge Genee. 2015. “Aligning Open Access Publication with the Research and Teaching Missions of the Public University: The Case of the Lethbridge Journal Incubator (If ‘If’s and ‘And’s Were Pots and Pans).” The Journal of Electronic Publishing: JEP 18 (3). https://doi.org/10.3998/3336451.0018.309.
- O’Donnell, Daniel Paul, Carey Viejou, Sylvia Chow, Kimberly Dohms, Paul Esau, Steve Firth, Rumi Graham, et al. 2018. “Zombie Journals: Designing a Technological Infrastructure for a Precarious Journal.” Zenodo. https://doi.org/10.5281/zenodo.1208720.
- Owens, Trevor. 2013. “Notes toward a Bizarro World AHA Dissertation Open Access Statement.” Trevor Owens. http://www.trevorowens.org/2013/07/notes-toward-a-bizarro-world-aha-dissertation-open-access-statement/.
- Porter, Dot. 2013. “Medievalists and the Scholarly Digital Edition.” Scholarly Editing 34:1–26.
- Public Library of Science. n.d. “PLOS ONE: Accelerating the Publication of Peer-Reviewed Science.” PLOS ONE. Accessed October 20, 2017. http://journals.plos.org/plosone/s/journal-information.
- Scholar Space. 2011. “Microsoft Word for Dissertations: Word for Dissertations Guide Content and Resources.” University of Michigan Library. http://guides.lib.umich.edu/c.php?g=283073&p=1886001.
- Strickland, Justin. 2016. “Navigating the Peer Review Process: Strategies for Successfully Responding to Reviewers’ Comments.” American Psychological Association. http://www.apa.org/science/about/psa/2016/11/peer-review.aspx.
- Suber, Peter. 2010. “Thoughts on Prestige, Quality, and Open Access.” Logos 21 (1–2): 115–28. https://doi.org/10.1163/095796510X546959.
- University of Lethbridge. 2017a. “Discover Thrive.” School of Graduate Studies. https://www.uleth.ca/graduate-studies/students-experience.
- ———. 2017b. “Fall 2017.” Institutional Analysis. http://www.uleth.ca/analysis/content/view-facts.
- ———. 2017c. “THRIVE: Professional Skills Program for Graduate Students.” School of Graduate Studies. https://www.uleth.ca/graduate-studies/thrive.
- ———. 2017d. Undergraduate Calendar and Course Catalogue 2017/2018. Lethbridge AB: University of Lethbridge. https://www.uleth.ca/ross/sites/ross/files/imported/academic-calendar/2017-18/cal_doc.pdf.
- ———. 2017e. “University of Lethbridge Images of Research.” School of Graduate Studies. December 13. https://www.uleth.ca/graduate-studies/our-students-experience/images-research.
In conceding this point about prestige and “importance,” the authors recognise that the underlying issue of “excellence” and “quality” in this context is problematic. As Peter Suber puts it, “If a journal can be excellent from birth, but not prestigious from birth, or if new journals typically achieve quality before they achieve a reputation for quality, then we have a noncynical reason to think that quality and prestige can diverge” (2010, 117; see also Moore et al. 2017).
See for example, the debate about the degree to which the “AHA Statement on Policies Regarding the Embargoing of Completed History PhD Dissertations” (2013) represents either good practical advice or a reactionary reinscription of outdated assumptions (Anderson 2013; Fister 2013; Owens 2013)