Few works on higher education have generated as much press and interest as Richard Arum and Josipa Roska’s Academically Adrift: Limited Learning on College Campuses (University of Chicago, 2011). Reviews, attacks, accolades and discussion have accompanied its publication and we are now seeing the ultimate measure of interest in higher education: the conference. Within months I am sure that all of us concerned about the book and its findings will have opportunities to gather in Hiltons and Marriotts, watch PowerPoint presentations and debate this latest “crisis” in higher education.
The hand-wringers despair, the realists shrug their shoulders and the policy-makers and politicians worry about American students “falling behind.” The rhetoric of national competition is inherently fascinating, but in this situation, somewhat peripheral. No nation has higher education truly sorted out. Instead, we have a relatively straightforward problem. A large number of students, drawn from a heady mix of decent institutions, did not demonstrate significant improvement on a test after two years of college. As the test purports to measure higher level skills, the institutions must be doing something wrong.
The CLA, or the the Collegiate Learning Assessment, claims to offer “an authentic approach to the improvement of teaching and learning in higher education through a continuous improvement model, and recognizes faculty as central actors in educational improvement efforts.” It is a test that sees the institution, though the aggregate score of its students, as the key unit of analysis. The CLA has three components: two writing tasks and a performance task, and its designers believe that it captures critical thinking, analytical reasoning, problem solving and writing. The writing tasks are about arguments. In one, students make an argument (45 minutes) and in the other, they critique an argument (30 minutes). The performance task asks students to give answers to open-ended questions about hypothetical but realistic situations. An overview of the scoring is found here. Each performance task has a library of documents, such as memos, reports, articles, maps, diagrams, notes, etc., and students use these in providing their answers. Students have 90 minutes to respond.
The data driving the book is a data set of CLA scores from 2,322 students at 24 institutions who took the exam at the start of their first year of college, in Fall 2005, and again at the end of their sophomore year, in Spring 2007. Results of the performance task are at the heart of Academically Adrift. In the “DynaTech” model, students are asked to craft a memo advising an employer about whether or not to purchase an airplane type that recently crashed. Another situation involved advising a campaigning politician about a proposed policy. Arum and Roska provide an overview of the assessment, as well as summarize the praise and criticism of the CLA. And by the way, there is a high correlation between an institution’s CLA scores and its aggregate SAT scores.
The book makes four broad claims, or lessons, from the authors research. First, they note that academic learning rarely receives the priority it deserves. Second, students do not seem to be making great gains; learning is limited. Third, when there is learning, the broader picture is one of increasing difference, and fourth, rates of learning within each institution have great variability. Learning is really what the instrument is trying to measure.
One of the most insightful maxims ever impressed upon me about evaluation is that one should measure what you value and value what you measure. It is simple, frightfully accurate and extremely powerful. Those that are able to develop new ways of measuring, and especially those that are able to apply these to practices that have not been paying attention, are often in a position to recast debates. This is what is behind the attention given to the CLA and Academically Adrift. It is an assumption held by many people that a college education makes you smarter, better able to think at a higher level. Colleges do not claim it – usually – but they do not disabuse the notion, either. Higher education is home to many smart people and there are often multiple benefits from an aggregation of brains. And as for the development of higher level thinking among undergraduates, it is a handy way of allowing the liberal arts, which intellectually are at the center of the institution, to claim primacy on young minds.
It is not clear to me, even as a working assumption, why two years of general education study at an institution of higher education would result in noticeably greater higher thinking skills (problem solving and analytical reasoning) unless the college or university sought this as a goal. At most institutions, the first two years of study consist mostly of general education courses. Students are introduced to the disciplines, work on writing and quantitative skills, and, if they are fortunate, have some opportunities to reach across disciplines to integrate some of their thinking. Those first 60 credits of study are often a mixture of starts, stops, and re-starts.
On the other hand, institutions of higher education that have imposed rising junior exams should be in a much better position to see gains, provided, of course, that their planned for outcomes included these higher level skills. Writing one would expect to improve, but not necessarily problem solving and more sophisticated skills.
Intentionality matters a great deal. The point is further driven home in the analysis of student time. Unsurprisingly, students today are spending less time doing academic work (reading books, writing papers, studying), than they were twenty-five years ago. Do something repeatedly, under the guidance of an expert, and performance improves. This, in essence, is the take away message from the book: academic learning requires work. It is not a new or particularly radical message, but it is one that needs to be restated again and again. The authors believe that for learning to take place, it has to be valued, prioritized and emphasized. Do not talk about persistence, they state, but about learning.
Arum and Roksa have written a thought provoking book. And while I may chafe at the conference discussions restating the obvious, it is also an helpful wake up call to those within higher education. It remains to be seen, however, whether Academically Adrift causes much thoughtful exchange outside of the academy.