Pascarella, E. T. (1996). On Student Development in College: Evidence from the National Study of Student Learning. In L. Richlin (Ed.), To Improve the Academy, Vol. 15 (pp. 17-29). Stillwater, OK: New Forums Press and the Professional and Organizational Development Network in Higher Education. Key words: Student Development, Thinking Skills, Intellectual Development, Campus Climate.

This paper summarizes some of the major findings of the National Study of Student Learning, a longitudinal investigation of the factors influencing student intellectual development at 23 diverse colleges and universities in 16 states. Findings from the following analyses are presented: effects of perceived teacher behaviors on general cognitive skills and internal locus of attribution; influences on critical thinking; cognitive effects of two- and four-year colleges; cognitive effects of historically Black and predominantly White colleges; and cognitive effects of Greek affiliation.

In 1991 my colleague, Patrick Terenzini, and I published a book entitled How College Affects Students: Findings and Insights From 20 Years of Research. In that book we reviewed and summarized the results of about 2600 studies on the impact of college. One of our basic conclusions from that body of research is that there was still much to learn about collegiate influences on student learning and intellectual growth. The National Study of Student Learning (NSSL), a 3-year longitudinal research project begun in 1992 under the auspices of the federally-funded National Center on Postsecondary Teaching, Learning, and Assessment (NCTLA), was intended to expand knowledge of the ways in which college academic and non-academic experiences influenced student learning and cognitive growth. This paper summarizes and discusses some of the major findings to date from NSSL.

Method

The National Study of Student Learning (NSSL), conducted at the University of Illinois at Chicago, is a study of 2685 students at 23 diverse institutions in 16 different states. The students were followed from their entrance to college through the end of their third year of college, although in this paper I will focus primarily on analyses of their first-year experiences. The institutional sample includes two­year colleges, liberal arts colleges, regional universities, historically Black colleges, and public and private research universities. In aggregate, the student population of the 23 participating institutions approximates the Fall 1992 national population of undergraduates by ethnicity and gender.

The first set of data was collected in the Fall of 1992 as the students were entering college. The initial data collection lasted about three hours and included surveys of entering-student characteristics, standardized tests of academic proficiency, and scales measuring such traits as students’ openness to diversity and challenge their internal locus of control for academic success. The measure of precollege academic proficiency were the Reading, Comprehension, Mathematics, and Critical Thinking modules of Form 88 of the Collegiate Assessment of Academic Proficiency (CAAP). CAAP was developed by the American College Testing Program (ACT) to assess general intellectual skills typically acquired by students during the first 2 years of college (ACT, 1989).

The first follow-up testing of the sample took place in the Spring of 1993. This data collection required about 3.5 hours and included the same modules from Form 88B of the CAAP and an extensive series of questionnaire instruments. The latter not only measured students’ openness to diversity/challenge and internal locus of control for academic success, but also elicited coursework taken, perceptions of teaching received, and level of involvement in various aspects of the academic and, non-academic experience of college. The second follow-up was conducted in the spring of 1994 and included the Writing Skills and Science Reasoning Modules of the CAAP, as well as a set of questionnaire instruments similar to those administered in the first follow up.

Although sample response rates tended to be high, we weighted the sample to each institution’s first-year population by gender and ethnicity (White, Black, Hispanic, other). This adjusted for response bias by gender, ethnicity, and institution. In all data analyses we employed least-squares regression or analysis of covariance to statistically control for a battery of potentially confounding influences. These included, but were not limited to: a precollege (Fall 1992) measure of the outcome being considered; institutional context (i.e., the average student-body score on the precollege measure at each institution); a precollege measure of academic motivation, race, gender, age, work responsibilities, full- or part-time enrollment, and type of coursework taken. For a more extensive discussion of the methodology of the National Study of Student Learning see Pascarella, Bohr, Nora, & Terenzini (1995) or Pascarella, Edison, Nora, Hagedorn, & Terenzini (1996).

Findings & Implications

Effects of Perceived Teacher Behaviors on General Cognitive Skills

In a first analyses of the NSSL data, we examined the extent to which student perceptions of teacher organization and preparation, and teacher skill and clarity influenced the development of general cognitive skills in the first year of college. These two teacher behaviors have been identified in previous research as the most important predictors of student course achievement (e.g., Cohen, 1981; Feldman, 1994). With controls made for confounding influences, the extent to which students judged the overall instruction received during their first year of college as high in teacher organization and preparation had a significant positive association with end-of-first-year reading comprehension, mathematics, critical thinking, and a composite score based on all three CAAP modules. That is, first-year students who perceived their instructors to be organized and prepared (e.g., “presentation of material is well organized,” “class time is used effectively,” and “course goals and requirements are clearly explained”) tended to demonstrate greater cognitive gains than their peers who perceived that they had received less well-organized instruction (Pascarella, Edison, Nora, Hagedorn, & Braxton, 1995).

These findings have at least two implications for higher education practitioners and policy makers. First, they suggest that the positive link between perceived teacher organization/preparation and course achievement might extend to broad-based general cognitive proficiencies. Second, and perhaps more important from a policy standpoint, effective teacher organization and preparation skills, such as those identified by students in this study, are ones that can be taught, and learned, through teaching improvement efforts (Weimer, 1990).

Effects of Teacher Behaviors on Internal Locus of Attribution

In a second analysis we attempted to determine the impact of various teaching behaviors on the extent to which students gain a greater sense of personal control over their academic accomplishments. The NSSL team developed an internal locus of attribution for academic success scale that asked students to respond to such items as: “Good luck is more important than hard work for college success” and “Grades depend on how hard the instructor grades, not on how carefully I study.” With controls made for potentially confounding influences, three perceived teacher behaviors had significant positive effects on the extent to which students made first-year gains in internal locus of attribution for academic success. These were teacher organization and preparation, teacher skill and clarity, and teacher support. In other words, first-year students who judged that the overall instruction they received was high on each of these three teacher behavior dimensions tended to demonstrate greater gains in internal locus of attribution for academic success than their peers who rated the overall instruction they received lower on these three teacher behaviors (Pascarella, et al, 1996). This finding suggests that effective teaching practices, that are themselves learnable, may not only positively influence student learning, they may also facilitate the extent to which students come to feel in control of their academic success in college.

Influences on Students’ Openness to Diversity/Challenge

A third analysis involved an examination of the extent to which students’ development of openness to cultural/racial diversity and challenge during the first year of college was influenced by measures of the institutional environment and students’ academic and non-academic experiences. The scale measuring students’ openness to diversity/challenge was an eight-item, Likert-type measure with internal consistency reliabilities of .83 for the precollege scale and .84 for the end-of-first-year scale. Diversity/challenge scale items included: “I enjoy having discussions with people whose ideas and values are different from my own,” “Learning about people from different cultures is a very important part of my college education,” “I enjoy taking courses that challenge my beliefs and values,” “The courses I enjoy most are those that make me think about things from a different perspective,” and “Contact with individuals whose background (e.g., race, national origin, sexual orientation) is different from my own is an essential part of my college education.”

In the presence of statistical controls for confounding influences, a number of variables had significant net positive effects on end-of-first-year openness to diversity/challenge. These included: a non-discriminatory racial environment at the institution attended, on-campus residence, participation in a racial or cultural awareness workshop, and extent of involvement with diverse student peers. Greek affiliation had a significant negative effect on openness to diversity and challenge for both men and women (Pascarella, et al, 1996).

Influences on Critical Thinking

A fourth analysis of the NSSL data was conducted to determine the influence of in-class and out-of-class experiences, together and separately, on first-year students’ critical thinking. Statistical procedures controlled for the effects of students’ initial level of critical thinking, degree aspirations, age, gender, ethnicity and social origins, as well as institutional characteristics such as form of control (private or public) and type (two- or four-year), and several measures of institutional environmental emphasis (e.g., analytical, scholarly, aesthetic). Both in-class and out-of-class experiences had small, but statistically significant and unique, positive effects on changes in critical thinking during the first year in college (Terenzini, Springer, Pascarella & Nora, 1994). In fact, out-of-class-experiences were somewhat more important to development of critical thinking than in-class experiences.

The in-class/instructional variables significantly and positively associated with end-of-first-year critical thinking were the number of courses taken in the humanities and fine arts and the natural sciences and engineering, as well as the total number of credit hours completed in the first year. Significant out-of-class experiences were student involvement in clubs and organizations, attendance at a racia1/cultural awareness workshop, and student perceptions of faculty concern for student development and teaching.

The findings underscore the importance of out-of-class (as well as in-class) experiences to student intellectual growth in college (Astin, 1993; Baxter Magolda, 1992). Indeed, I believe the findings argue for rethinking the current structural and functional relationships between academic and student affairs divisions in our colleges and universities. If students develop intellectually as a consequence of an interconnected and holistic set of in-class and out-of-class influences, then our administrative structures, program planning, and implementation should be similarly interconnected and collaborative.

Cognitive Effects of Two-Year and Four-Year Colleges

A fifth analysis of the NSSL data compared the cognitive impacts of five two-year and six four-year colleges and universities, matched on student precollege ability (i.e., a composite of scores on the CAAP precollege modules). With a battery of confounding influences controlled statistically, we found no significant differences between two-year college students and their four-year college counterparts on end-of-first-year reading comprehension, mathematics, critical thinking, and a composite score based on three CAAP module scores (Pascarella, Bohr, Nora, & Terenzini, 1995). This cognitive parity between two- and four-year colleges was essentially replicated with end-of-second-year scores on the CAAP writing skills and science reasoning modules (Pascarella, et al, 1995-96).

Recent research on the occupational and economic consequences of where one begins postsecondary education suggests that two-year colleges may be a cost-effective means to obtain the first two years of college without sacrificing job market competitiveness in such areas as income or job status (Whitaker & Pascarella, 1994). The results of the first two years of the National Study of Student Learning suggest further that students who begin college at two-year institutions also might not sacrifice intellectual gains. Our results indicate that, at least on certain outcome dimensions, the cognitive impacts of two-year colleges may be indistinguishable from those of four-year institutions that enroll similar students.

Cognitive Effects of Historically Black and Predominantly White Colleges

In a sixth analysis of the NSSL data we compared the cognitive effects on Black students of attendance at two historically Black colleges and sixteen predominantly White colleges. Controlling statistically for a battery of confounding influences we found small and statistically non-significant differences in end-of-first-year CAAP scores in reading comprehension, mathematics, critical thinking, and composite achievement (Bohr, Pascarella, Nora, & Terenzini, 1995). This cognitive parity between historically Black and predominantly White colleges was replicated with end-of-second-year scores on the CAAP writing skills and science reasoning modules (Pascarella, Edison, Nora, Hagedorn, & Terenzini, in press).

An impressive body of evidence suggests that historically Black colleges create supportive campus climates that foster students’ satisfaction, sense of community, and adjustment to college, and also increase the likelihood of persistence and degree completion (e.g., Allen, 1986, 1987; Allen, Epps, & Haniff, 1991; Pascarella, Smart, Ethington & Nettles, 1987; Pascarella, Smart & Stoecker, 1989; Thomas & Gordon, 1983). The NSSL results suggest further that these colleges also create social and psychological environments supportive of their students’ intellectual development, despite possible limitations in educational resources. The findings have implications for predominantly-White institutions that are concerned about the intellectual growth and achievement of their Black students. If supportive environments foster learning at Black colleges, how might similar climates for learning be fostered at White institutions? What should such environments include, and what obstacles exist to their development? Of course, the racial composition of historically Black colleges might be integral to the learning environments they create. Black colleges can, however, provide model programs and services to enhance student learning that might be transferable to White institutions. One example is the “ladder” of general science programs at Xavier University (LA), designed to implement high expectations for intellectual achievement while providing systematic structures for social, psychological, and academic support (Andreas, 1991; Kuh, Schuh, Whitt, & Associates, 1991).

Cognitive Effects of Greek Affiliation

Finally, the NSSL also examined the cognitive effects of affiliation with a social fraternity or sorority (i.e., Greek affiliation) during the first year of college. With statistical controls for potentially confounding variables, men who were members of social fraternities had significantly lower end-of-first-year reading comprehension, mathematics, critical thinking and composite achievement than their peers who were not affiliated with Greek organizations. First-year fraternity membership had its strongest negative effect on critical thinking (Pascarella, Edison, Whitt, Nora, Hagedorn & Terenzini, in press).

Additional analyses revealed that, for men, ethnicity influenced the cognitive effects of Greek affiliation. Joining a fraternity had a strong negative effect on all four cognitive outcomes for White men, but a modest positive influence on all four cognitive outcomes for men of color (Pascarella, Edison, Whitt, Nora, Hagedorn, & Terenzini, in press). What the data cannot tell us, however, is whether the fraternities to which the NSSL students belonged were predominantly White, predominantly of color, or both. Analysis of data for women indicated that joining a sorority during the first year of college also had a negative effect on cognitive development. However, only the negative effects for reading comprehension and composite achievement were statistically significant. The overall magnitude of the negative cognitive influences of Greek membership tended to be smaller for women than for men.

The results of these analyses tend to reinforce findings of previous research. Pike and Askew (1990), for example, in their single-institution study, found statistically significant, but small, negative effects of fraternity membership on the cognitive development of college seniors. The NSSL study indicates, however, that the negative cognitive effects of Greek affiliation might be discernable as early as the end of the first year of college. This might lead to consideration of policies on timing of Greek membership, suggesting, perhaps, that Rush and new-member activities, especially for White men, be deferred to the second semester—or even the second year—of college.

Limitations

The NSSL data have several limitations that should be kept in mind when interpreting the findings. First, although the overall sample is multi institutional and consists of a broad range of two- and four-year institutions from 16 states throughout the country, the fact that the analyses were limited to 5 two-year and 18 four-year colleges means that we cannot necessarily generalize the results to all two- and four-year institutions in the United States. Similarly, although attempts were made in the initial sampling design and subsequent sample weighting to make the sample as representative as possible at each institution, the time commitment and work required of each student participant undoubtedly led to some self-selection. We cannot be sure that those who were willing to participate in the study responded in the same way as would those who were invited but declined to participate. Third, our measures of cognitive development were limited to reading comprehension, mathematics critical thinking, writing skills, and science reasoning. While these are important dimensions of cognitive development they are certainly not the only ways in which cognitive or intellectual development can be operationally defined. Alternative conceptualizations or operational definitions of the dependent measures might have yielded findings different from those yielded by the NSSL analyses. Fourth, the NSSL analyses conducted so far are limited by the fact that we have only traced the sample over the first year or, in some cases, the first two years of college. We cannot be sure that the results we report would hold for subsequent years of college. Finally, the National Study of Student Learning is based on correlational, rather than experimental data. Although we attempted to control the influence of a large number of potential confounding variables with statistical procedures, such procedures cannot provide the same level of control, or certainty about causation, as true experiments.

Despite its limitations, the National Study of Student Learning data provide an important resource for higher education professionals concerned with the wide range of factors that may influence student learning and cognitive development in college. We know of no other data base that contains such in-depth measures of student learning and cognitive development from a sample of students from such diverse two-year and four-year institutions. Future analyses will address more completely the development of students’ science reasoning and writing skills after two year of college, and will revisit the development of critical thinking and reading comprehension at the end of the third year of college.

References

  • Allen, W. (1986). Gender and campus race differences in black student academic performance, racial attitudes and college satisfaction. Research Report, Southern Education Foundation.
  • Allen, W. (1987). Black colleges vs. white colleges: The fork in the road for black students. Change, 19, 28-39.
  • Allen W., Epps, E. & Haniff, N. (Eds.). (1991). College in black and white: African-American students in predominantly white and in historically black public universities. Albany, NY: SUNY.
  • American College Testing Program (ACT). (1989). Report on the technical characteristics of CAAP: Pilot Year 1: 1988-89. Iowa City: ACT.
  • Andreas, R. (1991). Where achievement is the rule: The case of Xavier University of Louisiana. 1n G. Kuh and J. Schuh (Eds.), The role and contribution of student affairs in involving colleges. Washington, D.C.: National Association of Student Personnel Administrators.
  • Astin, A. (1993). What matters in college: Four critical years revisited. San Francisco: Jossey-Bass.
  • Baxter Magolda, M. (1992). Knowing and reasoning in college. San Francisco: Jossey-Bass.
  • Bohr, L., Pascarella, E., Nora, A., & Terenzini, P. (1995). Do Black students learn more at historically Black or predominantly White colleges? Journal of College Student Development, 36, 75-85.
  • Cohen, P. (1981). Student ratings of instruction and student achievement: A meta-analysis of multisection validity studies. Review of Educational Research, 51, 281-309.
  • Feldman, K. (1994). Identifying exemplary teaching: Evidence from course and teacher evaluations. Paper commissioned by the National Center on Postsecondary Teaching, Learning, and Assessment. Stony Brook, NY: SUNY at Stony Brook.
  • Kuh, G., Schuh, J., Whitt, E., & Associates. (1991). Involving colleges. San Francisco: Jossey-Bass.
  • Pascarella, E., Bohr, L., Nora, A., & Terenzini, P. (1995). Cognitive effects of two-year and four-year colleges: New evidence. Educational Evaluation and Policy Analysis, 17, 83-96.
  • Pascarella, E. Edison, M., Hagedorn, L., Nora, A., & Terenzini, P. (1996). Influences on students’ internal locus of attribution for academic success in the first year of college. Research in Higher Education.
  • Pascarella, E., Edison, M., Nora, A., Hagedorn, L., & Braxton, J. (1995, April). Effects of teacher organization/preparation and teacher skill/clarity on general cognitive skills in college. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
  • Pascarella, E., Edison, M., Nora, A., Hagedorn, L., & Terenzini, P. (1995-96). Cognitive effects of community colleges and four-year colleges. Community College Journal, 66, 35-39.
  • Pascarella, E., Edison, M., Nora, A., Hagedorn, L., & Terenzini, P. (1996). Influences on students’ openness to diversity and challenge in the first year of college. Journal of Higher Education.
  • Pascarella, E., Edison, M. Nora, A., Hagedorn, L. & Terenzini, P. (in press). Additional evidence on the cognitive effects of college racial composition: A research note. Journal of College Student Development.
  • Pascarella, E., Edison, M., Whitt, E., Nora, A., Hagedorn, L., & Terenzini, P. (in press). Cognitive effects of Greek affiliation during the first year of college. National Association of Student Personnel Administrators Journal.
  • Pascarella, E., Smart, J., Ethington, C. & Nettles, M. (1987). The influence of college on self-concept: A consideration of race and gender differences. American Educational Research Journal, 24, 49-77.
  • Pascarella, E., Smart, J. & Stoecker, J. (1989). College race and the early status attainment of black students. Journal of Higher Education, 60, 82-107.
  • Pascarella, E. & Terenzini, P. (1991). How college affects students: Findings and insights from twenty years of research. San Francisco: Jossey-Bass.
  • Pike, G. & Askew, J. (1990). The impact of fraternity or sorority membership on academic involvement and learning outcomes. NASPA Journal, 28, 13-19.
  • Terenzini, P., Springer, L., Pascarella, E., & Nora, A. (1994, November). The multiple influences of college on students’ critical thinking skills. Paper presented at the annual meeting of the Association for the Study of Higher Education, Tucson, AZ.
  • Thomas, G. & Gordon, S. (1983). Evaluating the payoffs of college investments for black, white and Hispanic students. Baltimore, MD: Report No. 344. Center for Social Organization of Schools, Johns Hopkins University.
  • Weimer, M. (1990). Improving college teaching. San Francisco: Jossey-Bass.
  • Whitaker, D. & Pascarella, E. (1994). Two-year college attendance and socioeconomic attainment: Some additional evidence. Journal of Higher Education, 65, 194-210.

  • Contact:

  • Ernest T. Pascarella

  • College of Education (M/C 147)

  • University of Illinois

  • 1040 West Harrison Street

  • Chicago, IL 60607-7133 USA

  • (312) 996-8181

  • UI2639@UICVM (Bitnet)

Ernest Pascarella is Panel Study Director of the National Center for Postsecondary Teaching, Learning and Assessment and Professor of Higher Education at the University of Illinois in Chicago. He is co-author with Patrick Terenzini of How College Affects Students (Jossey-Bass) and has authored over 100 journal articles on student persistence in higher education and the impact college has on students. He also has served as President of the Association for the Study of Higher Education. Pascarella has received awards form the American Educational Research Association, the Association for Institutional Research, and the Association for the Study of Higher Education.

The National Study of Student Learning is supported by Grant No.: R11G10037 from the U.S. Department of Education to the National Center on Postsecondary Teaching, Learning, and Assessment. Some sections of this paper appeared in the March 1996 issue of the Journal of College Student Development.