Assessing Experiential Learning, Jobs and All: A Response to the Three Professors1
Robert J. Condlin*
I feel sorry for Professor Yackee. He started a conversation about legal employment and ended up in a debate about clinical education. 2 That’s a little like going to a Barry Manilow concert and having Gene Simmons walk on to the stage. In fairness, he opened the door to the larger issue on direct (perhaps inadvertently) when he acknowledged, ever so briefly, that one could “imagine . . . positive consequences of skills training,” 3 and once the door was opened Professor Findley walked through it on cross, to give the conversation a wholly new character. As I see it, there now are three questions on the table: 1) does clinical practice experience improve a law student’s chances of getting a legal job, 2) if not, would it if employers were given better information about student practice experience, and 3) if not, are there other reasons to justify a law school’s decision to fund a clinical program. The answer to question number 1, at least for many private law firms (and all of Biglaw), is almost certainly no, 4 but there is considerable room for disagreement on questions 2 and 3, and I will express my views on them shortly. First, however, a few words about the ostensible disconnect between clinical practice experience and private law firm employment.
I. Do Private Law Firms Consider Clinical Practice Experience in Making Hiring Decisions?
More than twenty years ago the MacCrate Task Force reported that private law firms did not value clinical practice experience in making decisions of whom to hire, 5 and Professor Yackee’s study suggests that not much has changed in the intervening years. 6 His attempt to put an empirical punctuation mark on this not entirely surprising conclusion 7 does not succeed completely, however, because he does not consider the full range of variables that might drive the legal employment decision and finds only a correlation (or the lack thereof) between employment outcomes and clinical practice opportunities, not a cause and effect. 8 These limitations notwithstanding, his study’s design is inventive and thoughtful, its analysis careful and thorough, and its conclusions measured and cautious. To paraphrase Daniel Webster, it’s a small study, but it makes sense. 9
Cause-and-effect is the gold standard of statistical analysis, 10 however, so it also is not surprising that Professor Kuehn would argue that “it is not possible to draw any reliable conclusion” about the relationship of clinical practice experience to legal employment from Professor Yackee’s study. 11 Professor Kuehn’s objections are interesting and forcefully argued, 12 but in the limited space I have available here I will focus on the issues raised by Professor Findley instead. If clinical practice experience is an important part of a law student’s education, then it does not matter much if the experience helps students get jobs. Jobs will happen. More interesting are the questions of whether practice experience should play a role in the hiring process, and, if not, whether it nonetheless contributes something of value to the development of a lawyer. My answers will be “no” to the first question, and “yes” to the second—that most of the time clinical practice experience will be more valuable to students than employers. I will explain both answers in detail.
II. Would Law Firms Consider Clinical Practice Experience in Making Hiring Decisions if They Had Better Information About It?
If private law firms do not take clinical practice experience into account in hiring, one might ask, as Professor Findley does, 13 why this is the case. Professor Yackee suggests that it is because the data is too “costly and difficult” to collect, 14 but there may be other reasons. For one, law firms understandably believe that conventional academic credentials (i.e., GPA, class rank, law review) do the best job of identifying the smartest students and that the smartest students will overcome experience deficiencies quickly and be more productive during their entire time at the firm than students with the best practice experience, even if this will not always be true the “first day on the job.” 15 Intelligence differences among law students often are small, of course, and one can read more into academic numbers than is warranted, but the assumption that students with the best academic records will contribute the most over time is reasonable and, as such, it is understandable that firms would act on it.
Sometimes firms are not interested in student practice experience because they prefer to do their own training. In-house programs can be tailored to a firm’s particular practice and culture, pegged at a higher level of sophistication and difficulty than the generic programs offered in law schools, 16 and taught by experts in the field (who may be members of the firm). There will be exceptions, of course. A firm might value practice experience when it has independent information about a candidate’s practice proficiency (e.g., when a student applies for a job in the office where he did his clinical practice work), when the experience has an immediate job-related benefit (e.g., when the student’s experience is with the type of work the firm does itself), 17 or when the firm does not have the resources or personnel to do its own training and having someone else do it is the overriding concern. Most of the time, however, practice experience plays second fiddle to academic achievement and law school reputation in private law firm hiring because smart money bets on talent, not training, and law firm money is smart money. 18
Professor Findley argues that firms would take practice experience into account if they were given better information about it, and he describes several ways in which this could be done. 19 In one sense, providing more information is not difficult. Law school transcripts already disclose student clinical practice coursework, and recommendations from practice supervisors could describe the students’ levels of practice proficiency. Information of this sort is more costly and time consuming to prepare and work with than academic transcripts, of course, but when firms think something is important, cost usually is not an obstacle. The difficulty with practice-based recommendations, however, is that when not perfunctory or boilerplate, they often are expressed in overstated language that makes them hard to evaluate. Rarely do they describe mistakes, bad judgments, or motor-skill errors students made during their practice work, for example, notwithstanding the overwhelming probability that such things happened, or identify personal qualities that make the students difficult to work with or ill-suited for certain types of work. Hyperbole and selective reporting in recommendations are easy to understand. In a world where most job applicants come described in “better than sliced bread” terms, the slightest negative comment could cost a student a job, and clinical practice supervisors understandably are reluctant to take that risk. Unfortunately, recommendations designed to equalize the playing field in this way often drop out of the hiring process because it is difficult to know what they mean.
There are additional difficulties with practice evaluation data. For example, it is easy to compare standardized test scores and grades across different schools and students, but there is no easy way to compare one student’s practice experience against another’s, no matter how scrupulously the experiences are described. More so than conventional coursework, the nature, scope, and quality of practice instruction vary widely from school to school and course to course. 20 Even in the same course, one student may have limited success in managing a set of difficult tasks in complicated cases, while another may breeze through a large number of routine tasks in easy cases. The practice supervisor may think that the first student is more talented than the second, but a recommendation describing the first student’s experience, if honest, will report mixed views, while a recommendation describing the second student’s experience, equally honest, will report a string of unqualified successes, and an employer will not know how to compare the two recommendations.
The “objective” factors Professor Findley lists for comparing one practice program against another—“whether clinical faculty have parity of status, governance rights, and pay with doctrinal faculty; clinical-student teacher ratios; and the like” 21—have no necessary, and often only a coincidental, connection with the quality of instruction and, thus, provide little help in making the needed comparisons. This comparative judgment problem exists with traditional law school courses as well. An A grade in one Constitutional Law course may reflect an entirely different level of understanding and ability than an A grade in another Constitutional Law course. But the assumption with conventional coursework is that over a law school career a student’s overly positive and overly negative grades will even out so that his final GPA is likely to be an accurate measure of his overall ability. Clinical practice instruction does not provide a big enough sample of work for this same assumption to be true for supervisor evaluations.
These difficulties are exacerbated if one uses clinical grades rather than supervisor recommendations to evaluate students. Grading in clinical courses varies considerably from course to course and school to school. Some clinical courses are not graded, and others are graded on a pass/fail basis, but most, at least those taught by full-time clinical professors, are graded on the A–F scale used in the traditional curriculum, 22 and clinical grades often are higher on that scale than other law school grades. 23 Practice supervisors reward effort and improvement as much as level of achievement (it would be hard not to, given the closeness of the working relationship), and most clinical students work very hard and improve quite a bit. Law students have different aptitudes for clinical practice, however, in the same way that they have different aptitudes for understanding legal doctrine. Not everyone is a natural at interviewing a client, negotiating an agreement, or arguing a motion in the same way that not everyone is a natural at analyzing a judicial decision, interpreting a text, or evaluating competing policies, and because of this, one would expect to see roughly the same grade distribution in clinical courses that one sees in the rest of the curriculum (if not always in the same order). When this does not happen flags are raised, and employers find it difficult to know what weight to give the grades.
Finally, Professor Findley suggests that incorporating evaluations of law school clinical programs into the U.S. News & World Report (“USNWR”) law school rankings would help employers compare practice experience from different schools in the way that the rankings help them make the same comparison with traditional coursework. 24 At the same time, however, he also says that it is not possible for outsiders to evaluate the quality of a law school’s clinical program, that only insiders can make such judgments. 25 But if there are no uniform standards for judging the quality of practice instruction across school lines, if each clinical program is qualitatively a tub on its own bottom, then it is not clear how different programs could be compared and ranked in USNWR fashion. Ranking requires a uniform standard of measurement, and without it there is nothing against which to rank. If only an insider can assess the quality of a clinical program, realigning the law school rankings to take comparative clinical program data into account may turn out to be more difficult than it at first appears.
For many reasons then—the overstated nature of clinical practice evaluations, the uneven quality of clinical practice experiences, the availability of more effective in-house training options, the difficulty of comparing clinical practice programs from one school to the next, and others—it is understandable that private law firms would trust conventional academic credentials more than practice experience in deciding whom to bring into the firm.
III. Are There Non–Hiring Related Reasons to Justify a Program of Clinical Instruction?
At the end of his article Professor Yackee offers up a somewhat measured paean to clinical education, stating, “Law schools might rationally and justifiably invest in skills training to achieve other worthwhile goals unrelated to JD employment outcomes.” 26 Among these other goals, he says, are providing students with “a more enjoyable time in law school,” helping students “enter their first job[s] with more confidence and less stress,” giving them the opportunity to “have a meaningful impact on the lives of the legally underserved,” and lessening the risk that they will “commit professional malpractice in their first jobs.” 27 Because of this, he concludes,
[E]xperiential learning opportunities certainly have a role to play in modern legal education, and perhaps an important one. But in deciding how much to spend on providing such opportunities, law schools might want to consider the lack of evidence that such opportunities are likely to improve their graduates’ overall prospects of obtaining a quality job as a lawyer. 28
This is a strange description of the benefits of clinical practice instruction. There is nothing in the list, for example, about understanding the role of lawyer behavior in implementing statutes and doctrines in a fair and just manner, developing a capacity for strategic “ends-means thinking,” 29 learning to live professional lives in both a skillful and humane fashion, making legal procedures and institutions intelligible and responsive to people caught up in them, or understanding the role of substantive legal knowledge and analytical skill in professional practice. But Professor Yackee does not stop there. He also argues that any “positive consequences of skills training” must be measured with “modern empirical methods” 30 (which I take to mean quantitative methods) before a school can be sure that the benefits of such training outweigh its costs. 31 Personal and professional development can be difficult to measure in quantitative terms, however, particularly in young people still forming their professional identities, and forcing a program of instruction to justify its existence in a manner that does not fit its objectives or circumstances can be a sub rosa way of scuttling the program. One gets the sense that Professor Yackee is not a fan of clinical education, his proffers of agnosticism notwithstanding, 32 though that may be just me indulging nostalgically in a lost paranoia.
Professor Findley’s response to these suggestions takes us full force into the clinical education debate. This is an old debate and not a simple one. Its opening arguments go back to the 1930s, and its subsequent remarks continue unabated to the present day. 33 Professor Findley’s article, while short, raises too many issues to be discussed in the limited space I have here, so I will restrict my comments to his most prominent points. His principal argument, based on what he describes as “effective pedagogy for adult learners,” 34 goes back to the 1970s. Adults, he argues, comprehend, retain, and recall cognitive material best when they actively engage in resolving real-life, concrete problems that ask them to change their social role and reflect on and assess their resolution of those problems as part of a process of mutual inquiry by teacher and student. 35 He describes this argument as based on “sound pedagogy and intellectual rigor,” 36 and, in what probably is his most provocative statement, says, “Clinical education meets th[e] needs [of adult learners] in ways that traditional classroom education simply cannot.” 37
This last point seems demonstrably false, even on Professor Findley’s own terms. Take the example of classroom case analysis. In analyzing cases, students identify and evaluate decision rationales, parse ambiguous and confusing language, look for reasoning errors and evidentiary gaps, identify and evaluate relevant policy considerations, and reconcile other cases on the same subject, all in an intellectually “active” and “engage[d]” manner. 38 Analyzing cases in this way is a “real-life experience,” 39 requiring students to solve “concrete problems” 40 of the type they are likely to confront in actual law practice, and one that involves them in “chang[ing] . . . [their] social roles” 41 (from student to lawyer). In discussing their interpretations with others, students also “reflect upon” 42 their resolutions as part of a process of “mutual inquiry” 43 with teachers and classmates. Case analysis differs from clinical practice instruction only in what is learned, not how it is learned, 44 pedagogically, the two processes are the same. 45 This should not be surprising since all skills learning, of whatever kind, is “experiential,” “hands-on,” and “problem solving.” 46 The problems and experiences vary, and sometimes the hands are on different things, but the process through which skills are learned is always the same. Why clinicians continue to argue for a pedagogical divide between the classroom and the clinic is one of the enduring mysteries of the clinical education debate. 47
Professor Findley makes a “Fourth Amendment” argument that illustrates another dimension of this point. 48 After asserting that clinical instruction “deepen[s] students’ understanding of substantive material by providing a context [with which] to help them understand doctrine more deeply,” 49 he adds, students “inevitably gain a fuller understanding [of Fourth Amendment doctrine] when they try to solve problems for real clients.” 50 He probably does not intend the second half of this statement to mean that clinical instruction is the only means by which to gain a full understanding of Fourth Amendment doctrine. That would suggest that Akhil Amar, Dan Capra, Tom Davies, Wayne LaFave, Arnold Loewy, Chris Slobogin, and many other well-known Fourth Amendment scholars do not have a “full understanding” 51 of the Fourth Amendment, since none of them, so far as I can tell, has represented a “real client” 52 in the sense Professor Findley seems to use the term. Whatever one thinks of the views of these scholars, there is no denying that their understanding of the Fourth Amendment is “deep” 53 and genuine. Professor Findley must mean only that representing clients can add nuance, sophistication, and depth to doctrinal understanding, not that representing clients is a prerequisite to a full understanding of doctrine in the first instance, but even that is true only some of the time, for reasons I will explain shortly.
Professor Findley continues his Fourth Amendment argument with an analogy to medical education. “Imagine,” he says, “the medical school that sought to teach anatomy by having students read about it and discuss it without allowing them to experience the human body through hands-on exploration.” 54 Medical education analogies also have been a staple in the argument for clinical legal education from the earliest days, but the differences between law and medicine have made most of them non-starters. Take Professor Findley’s anatomy analogy as a case in point. Fourth Amendment doctrine is principally a body of ideas—rules, policies, and principles—not a physical body, and a body of ideas is understood intellectually more than through “hands-on” contact. Practice instruction can teach students to manipulate Fourth Amendment doctrine (learned in the classroom) to advance client ends, but using doctrine in a self-interested manner is not the same as understanding doctrine in the first instance. Under some circumstances, in fact (e.g., when it causes one to believe that law is infinitely malleable because there are always “cases on both sides”), 55 it can make genuine understanding less likely.
In the end, the case for clinical education needs substantive arguments, not pedagogical ones, to silence the naysayers. What new skills, bodies of knowledge, values, perspectives on the world, theories of justice, and the like (subject matters, in effect) do students learn in clinical courses that they do not (or cannot) learn in conventional law school courses, and why is it important that law students learn these things? 56 Until these questions are answered, the debate over clinical education is not likely to end. People may get tired of it for periods of time, and it may go into temporary remission, but as the recent paroxysm over “practice ready graduates” demonstrates, inevitably it will flare up and rekindle old conflicts. 57 Suggestions of how to answer these questions are out there, 58 and the questions themselves seem manageable, but, for one reason or another, many (but not all) 59 of those who write about clinical education seem to prefer discussions of “Best Practices” 60 to discussions of legal and political theory, and this reinforces the view that clinical practice instruction is about little more than technique.
Professor Findley also argues that “the ABA is prodding law schools toward experiential learning because it is good for the profession—that is, it provides value independent of a competitive hiring advantage for some schools or some graduates.” 61 No doubt, Professor Findley understands ABA politics better than I, but even I realize that the Association represents the interests of a mixed bag of constituents, under the influence of a mixed bag of personalities, and for a mixed bag of reasons. Clinicians have looked to the ABA for leverage in their struggles with traditional law faculty from the earliest days. Those early efforts were couched in the rhetoric of providing legal services to the poor, but since the poor had limited political clout these efforts often met with limited success. As economic conditions in law practice worsened, however, and law firms began to recognize the value of transferring new lawyer training costs to law schools, the political clout behind the ABA’s support for clinical education strengthened appreciably. No doubt some in the Association have educational goals in mind in pressuring law schools to provide more practice instruction, but a close reading of the Association’s reports on the subject, particularly the recent Report and Recommendations from the ABA’s Task Force on the Future of Legal Education, makes it clear that economic concerns also are at the heart of the matter. 62
Finally, I have a small objection to rhetorical style. Like a lot of clinical scholarship, Professor Findley’s case for the value of practice instruction frequently draws on evocative buzzwords more than fully developed arguments for its substantive support. For example, he says that practice instruction develops the capacity for “life-long learn[ing]” 63 without describing the attributes of life-long learning or saying how it is measured and that clinical instruction produces “reflective practitioners” 64 without saying what “reflection” consists of or how it is different from the law school–wide experience of reconsidering views when they are challenged or questioned. 65 Similarly, when he uses the term “clinical education” (and its cognates) he usually pairs it with adjectives such as “sound,” 66 “deep,” 67 “important,” 68 and “uniquely powerful” 69 as if the adjectives were part of the noun. To an unsympathetic reader all of this can look like question-begging argument under the guise of description, and reinforce the suspicion that the case for clinical education is based on jargon as much as reasons and evidence. Professor Findley’s arguments would lose none of their force if he deleted this language.
It is the tradition in this country to end on a positive note, so I should say in closing that I believe clinical practice instruction is an indispensable part of a complete legal education. Like Professor Findley, I began my teaching career as a clinician and moved to the dark side only later in life. I taught practice courses for a little over ten years and simulation-based skills courses for another thirty, in cities of various sizes; for public and private law schools; out of neighborhood legal services offices, private law firms, a public interest backup center, a large university’s student legal services office, and a government law office; in large classes and small seminars; and in packages ranging from fifteen hours of credit for one semester to one hour of credit for independent study. I believe in the value of clinical education because I believe what law delivers is as important as what it promises, and clinical education, at its best, teaches students how to deliver on law’s promises. 70 On the other hand, I also believe that practice instruction should not be the dominant part of a law school curriculum or play a larger role than it does in most schools at the present time. 71 Its content is too uneven, episodic, and situation-driven to be a complete program of instruction for even the study of lawyer practice skills, and becoming a lawyer involves much more than learning lawyer practice skills. In the end, the ultimate practice skill is the ability to think critically from an informed legal, practical, and social perspective, 72 and teaching students to think this way (and transmitting the body of knowledge on which such thinking is based) is the work of all law professors, traditional and clinical alike. Most law faculties have divided that task effectively for a very long time, and I would not mess with those arrangements except in extreme circumstances. I also would leave it to others to define extreme.
∗ Professor of Law, the University of Maryland Carey School of Law. Work on this article was funded by a grant from the UM Foundation, for which I am grateful. Richard Boldt, Molly Brimmer, Christopher Condlin, Steve Ellmann, Sue McCarty, Steve Pepe, Michael Pinard, and Charlie Sullivan made many helpful improvements to an earlier draft.
**Cite as: Robert J. Condlin, Assessing Experiential Learning, Jobs and All: A Response to the Three Professors, 2015 Wis. L. Rev. Forward 65, http://wisconsinlawreview.org/assessing-experiential-learning-jobs-and-all-a-response-to-the-three-professors/