Lion Gardiner- Presentation
Thank you for having me and letting me be here with you to learn some archaeology. My doctoral research is in deep sea biology.... And, I'm very much interested in how people learn and how we can make higher education institutions more effective in facilitating that learning. I will talk about how all this luminous research on this work, so we can be more effective. What I'm going to do is share with you the results of some research. I'm going to go like lightning. I've given you one handout, you don't need to look at it yet. And, I have several handouts for you that I deliberately didn't give you, so you wouldn't look at them. But, when I'm done we'll be passing them out, so everyone will get one.
I'd like to start out thinking with you why we're doing all this. It is to create value for our society, is it not? And, for our students. To help them learn, help them develop in important ways, certainly as archaeologists, but as human beings and useful citizens, as well. I guess you're dealing with two populations, undergraduate majors and people who are not majoring in archaeology in our courses, lots of them. What we're going to say here is related to all of them. Well, what are the key competencies or abilities that society's asking us to produce? And, here's where we pull out the handout and it has many things on it. If you look at Section 1, just read down to the bullets, that's a quick summary of what studies show society's asking us for, particularly business people, employers. Now, tell me which of these competencies are inappropriate for archaeologists? Study after study, of university/college faculty members around the country; show that these are the kinds of things they value, too. We value these things, particularly higher order thinking skills. To focus on that, I'd like to use two dimensions of cognitive development just very, very briefly to illustrate what we're talking about, what's involved in developing people's higher order cognitive skills and then take a look at the research on what is necessary to do that and how are we doing now.
Critical thinking is cited by faculty members and often employers, other folks who think about these things, as one of the most important things that we can help our students develop, the capacity and the disposition of critical thinking. Critical thinking being defined as the kind of curious, unbiased, fair-minded attempt to understand the world and the kind of thinking that has some intellectual standards that go with it. A prerequisite for the capacity of critical thinking, to think critically, is what is sometimes the mature epistemology. Epistemology has to do with the assumptions we make about knowledge and where it comes from. How do we know what we know?
There's quite a bit of research on college student epistemology, epistemological development. And, I'd like to share some of that with you, quickly. This research goes back to the 1950s with William Perry at Harvard University. It's been replicated many times since. Many of our students who come to us, freshman for example, think in a dualistic mode. They think in right-wrong, good-bad, black-white, and of course my group and my way of thinking about things, my group's way of thinking about things are right, your's are not.
Here are some characteristics of dualistic thinking. I'm going to give you a handout, don't write this down, at the end with all of this on it. The idea is to get the idea right now, okay? There's some implications for instructions with dealing with dualists. Your job is to feed them the facts, the correct facts. Their job is to memorize them all and spit them back when you asked them to give it back in the same way that it was given. Now, you can't rely on universities or colleges too long before you realize that in any particular field there are authorities who are highly regarded who disagree with each other on the same stuff. This is a problem for the dualists. So, what happens is gradually they are weaned away from dualism and they go into something else called multiplicity where if the authorities, the legitimate authorities, disagree, well, one idea must be as good as another. And, it may be a step forward, but not much. Certainly not the way we think about things. Finally, perhaps the poorly named level, relativism, procedural knowledge. And, this is where we have competing opinions, hypotheses, values, points of view, we look for evidence. We realize that knowledge depends upon context. The term, the label, or the title comes from the fact that various variables are related, relative to each other. What's the best hypothesis depends on other things, depends on a variety of things. And, this is perhaps the first place where and around here that critical thinking can occur. Metacognition, thinking about your own thinking, evaluating your own thinking, becomes possible at this stage. Well, this is where we want them to be, probably. So, they can engage in critical thinking, science reasoning, and all the other things that we value so much. Where are our students in respect to epistemology? If you teach are they primarily dualists, relativists? Think about it. Now these are data I lifted from the literature. The so called "Perry Positions" are simply measures of epistemological development. This is roughly dualism down here. Multiplicity in here. Relativism up here. .03% and this is the way it goes. I have data on about 350 Rutger's students and I've never seen a relativist yet, freshman through seniors. So, that should tell you something when you talk to students about critical thinking, they may not be able to understand what we're talking about or why that would be important. Some of them may think we're incompetent because we won't tell them the "truth" or the correct facts.
Q: Have you ever measured the faculty?
Lion Gardiner: I was hoping you wouldn't ask that. I do, but a very small sample and it's very variable. My sample is very small, maybe a dozen or 20 or so. However, I have one faculty person who's in my faculty seminar on students' learning and teaching in our Teaching Effectiveness Center. And, without going into all the details because of the certain characteristics that he displayed, I made a prediction before I gave him the test, the ... And, if you want to know about the assessment, I'll tell you later on. And, I said, "He will be low." He scored very low.
Okay, how much change is there over the college experience? Here we have the cross-sectional data. First year, sophomore, how much change are we seeing? Junior. Now wait a minute, what's going on? Wouldn't you expect more drifting of those bars? Right, stay tuned. One other thing I think I should show you here. Commitment, which I skipped over before, used to be thought to be a part of epistemology when Perry published his stuff. Now it's thought to be part of identity formation, which is a different variable. But, the idea is that here we learn how to do, we've got it, we use evidence, think scientifically, for example. Here, I come to realization that all of the world is terribly complex and it's difficult to understand. I need myself to make commitments to people, principles, values, career, things of that sort. A different kind of development that feeds off of epistemological development.
Now, I'd like to take a very quick look at two components of an academic, the academic enterprise, the student's experience on campus, curriculum and instruction. We can look at other things as well. In fact, before I forget, if I can find it quickly. If you're interested in the data and you want to think about it in more detail, much of what I'm going to say comes much from out of this book called "Redesigning Higher Education." Okay, currently, the point of the curriculum is to provide for each student a balanced set of experiences that will help that person develop as far as possible in whatever the curriculum is. There are, coming out of research, two key concepts I'd like to mention now. One is that curriculum that work need to be systematically designed. We need to know what the end is, the desired end is? What are the outcomes you want? What should the students know when they're done?
We need to have a method of monitoring their development, assessing, so we know what our actual outcomes are, so we can guide students and guidance ourselves, so that we can close the performance gap on our part between intention and actuality because it keeps squeezing it over time. And, the other thing is, the other concept is what is called alignment. Intended outcomes, assessments, and what we do to help them learn and develop, all need to be aligned together based on those intended outcomes. We need to use methods to help them learn, that research shows, current research shows, makes that happen. Not doing something else that may not work. And, the assessment tools to assess need to be valid. They need to assess what we said was important.
Okay, two concepts. Let's take a look at the research on curriculum. Most of this research comes from studies of general education. Some of them huge studies, like studies at San ... and UCLA, Jim Ratcliff at Penn State and studies of that sort. And, I'm going to summarize very, very quickly. About 97% of colleges in the United States have what is called "distributional curriculum." Students have a great, great latitude in what courses they take. Therefore, if we're going to get any specific, intended results in a reliable way, we need to have a very high quality academic advising because students come from all other, their parents don't necessarily know about these things and they need good advising. Unfortunately, the research on advising, which today is conceived in the literature when you look at books on academic advising, you can see it's developmental academic advising. You start with students with these values and goals to help each student then develop a unique curriculum to help that person develop in an appropriate fashion.
Most of the literature on advising, research on advising, and what really happens describes it as a clerical process, a mechanism that helps the registrar get an idea, demand, for the next semester's courses. So, we don't have that. Therefore, maybe, the results of the studies I mentioned suggest that the amount of choice that students are given in the curriculum has very little impact on what the outcome are. The specific courses that are available have very little impact on what the outcomes are when you do correlational studies between the breadth of the curriculum and what students maybe can do after they graduated. The specific requirements, given to the students, have very little relationship to what the students know and can do afterwards. So, the general conclusion from these studies on the curricula really don't work. And, in the ... study, a huge study of approximately 25, 000 students, hundred of institutions, couple hundred variables, found that there was a correlation between what he called "true core interdisciplinary curricula." And, those curricula are in very few institutions. That's what I want to say for the moment about curricula.
How about instruction? Now, there are lots of ways in which students learn on campus: curriculum, co-curriculum, working for money, who knows? Both sessions in residence halls with other students can be very potent. In fact, the peer group has more impact on students than faculty members. Neverless, let's talk about courses. Classes, courses are the primary intellectual meeting place of the students on campus. Okay, particularly with commuting students. And, what percentage of American undergraduates are commuters? Over 80%. Think about some of these numbers. So, what is the student experience like in our courses?
I want to focus on just two variables -- intellectual demand, intellectual activity and degree of student active involvement in learning. We're being asked to raise the standards, raise the expectations of students. And, the experience is that when we raise the expectations, the students rise to that level. All the students can learn at a high level, but they're often not being asked to do that. Okay, let's take the student activity first. We know for epistemological development and then another dimension that I forgot to mention, the capacity for reasoning effectively about moral issues, ethical dilemmas, on which there is a huge amount of research. We know that for these kinds of things, lectures don't work that well. Students have to be actively involved and struggling with problems and materials, depending on what they're learning to get the cognitive change necessary. However, when we look at what happens in classes, depending upon the study, depending upon the discipline, as many as 70-90% of us use the standard lecture, traditional lecture, as our preferred teaching mode. It just doesn't work very well for these things. How about intellectual level?
How many of you have heard of the device called the "Bloom Taxonomy?" Okay, a number of people. This is a very useful tool to help people control the intellectual level, the cognitive demand, in courses, assignments, assessments. Ed's part of the workshop. He's going to help you learn about the "Bloom Taxonomy." He's got a copy for you. For fifty years, the Bloom Taxonomy has been the teacher's friend, controlling the intellectual level. It's just been revised every year. You use the Bloom Taxonomy as a research tool, we have some interesting data. We can say what the intellectual demand is in large numbers of classrooms. Just so you know, quickly, what the taxonomy is, it starts with down at the bottom with rote memory, spit back, memorized facts, no understanding. Next level is comprehension of concepts and then various levels of application and analysis, evaluation, and synthesis, problem-solving skills. It's those latter ones that most of us value most highly. Okay, so what's going on in classrooms? Overwhelmingly, you go into the classroom and you have questions that students ask teachers for example. You've got the Bloom Taxonomy and you tick them off. You do this in a hundred classes, you get a pattern. Overwhelmingly, the intellectual demand in the classroom is at low levels, one and two. Factual material and personable and low level conceptual understanding, without ever getting at those higher levels. A very small percentage of time is spent on those kinds of things.
How about assessments? You can do the same thing with the Bloom Taxonomy. In fact, anytime we make up an assessment, a cognitive assessment in contrast to playing the piano, we should have the Bloom Taxonomy or an equivalent tool there to help us distribute intellectual demand wherever we want it in the assessment. So, you can take any, you can take the taxonomy that Ed is going to give you and go back and get some of your final exams, if you're teaching for quizzes, and do an ...analysis. Write each one, take a percentage of the one's, two's, and three's and so on, and then step back and see what kind of pattern you've got. Well, this is what the researchers have done. Overwhelmingly, study after study, low levels one and two. So, there is a certain kind of alignment there, but it's not what the faculty wanted. We value higher order thinking, but they don't know how to do that. So, what the impact of this is, in class and in assessments, is to retard the development of higher order thinking skills. Number two, keep people in dualism. You have to get out of dualism and into relativism to do critical thinking, right? You have to change your assumptions about how to learn. You're not authority based, you're an independent thinker, you're constructing your own knowledge. So, those are two ?untoured effects of what's going on across the country. The third one encourages students to take what is called a surface approach to learning as opposed to a deep approach. A deep approach requires deep cognitive processing where you're trying to understand. The emphasis is on understanding, taking personal meaning and using what you're learning. The surface approach is memorization with intention to memorize isolated facts. Isolated facts are not well incorporated into our cognitive structures on the ground in the synapses, in the brain. Whereas, if we understand and we make meaning and we see relationships involved in various kinds of things we learn, that kind of learning can be incorporated with the right kind of practice and retained in long-term memory. It can be useful.
By the way, what does the research show about what conditions students to take a surface or deep approach? The number one thing is their expectations of our assessments. What is he or she going to ask me to do? So, changing our assessments can be one tool, one lever we have, in getting the students to change their behavior. I'll see where we are here. We can talk about campus climate; we can do more with academic advising. We don't have time. Alright, what can we say about the actual outcomes of all this? Now, some of you know, for the last fifteen years there's been a very active assessment movement in this country because of state governments and accrediting agencies saying, "we want to know what the students know when they graduate." There's a lot of assessment activity, but relatively few institutions really can answer the basic question, still. That's a whole other discussion. However, once again the professional literature in higher education comes to the rescue.
What I'm going to do is share with you the results of a number of studies. I'm not going to say much. I'm going to move fast. If I go too quickly, just ... me up, okay? Let's see if we see any patterns. [They're looking at something?] Now, this describes a 96 study. It just describes the study, so you know how it was set up. Seniors, right? They're ready to move out. And, the questions they ask, and I don't have any for you here, the questions are all low-level knowledge, low in the Bloom Taxonomy. These are percentages.
Lynne Sebastian: But, isn't this an explanation for why people are teaching so low in the Bloom Taxonomy? Well, if I've got kids who don't know where Canada is, you know, I kind of need to straighten that out before I can teach them to think about the major Canadian governments.
Lion Gardiner: Well, is that where you want to stay? Do you want to go through the whole semester like that? It's hard to think about things if you don't have the information. Old guy: What you're saying is that if you don't retain the factual information....[more discussion]. You have to use it. Not just listen to someone tell you what he or she knows. You have to use it in practical situations so you can figure out what it means and relate it other factual information to get patterns and principles and abstract concepts. This is what the researchers concluded.
Bill Lipe: It explains why so many of my undergraduate students think that a historigram is a unit of the metric system.
Lion Gardiner: This is masses of institutions. How about the lead ivy league and like institutions. It's better there, right? [Response: Not at all.] Okay, let's take a big study, National Adult Literacy Study at 26,000 Americans, 1992, sponsored by the National Center for Education Statistics, done by ETS.
Nancy White: Yeah, but have you seen those bus schedules?
Lion Gardiner: This is where the question about the faculty comes in. Two points, quickly, I have to get out of here. I have taken some of these same things, which I lifted out of my book, and I made a quiz that I gave personally to the honors colloquium I was doing. Sixteen students around the table, they're crème de le crème, 3.5 grade point averages. Seven simple minded questions, easy. How many of the sixteen students got them all right? Two. So, I said, "I need a bigger sample size." So, I gave it to my undergraduate class of 47. How many got them all right? One. So, honors students are better, right? But, let's bring the question closer to home. Think about your discipline, archaeology. Isn't it true that you could find a handful of concepts in that field that are really important, low level stuff that's important, but. In my field, this would be organic evolution, intellectual foundation for the life sciences. And, maybe anthropology, too, physical. How about our majors? Here's some Ohio State data, majors. Organic evolution is a highly abstract, complex, counter intuitive concept. It's not easy to get, particularly if you just hear lectures.
Glen Doran: It's not being taught, perhaps for social, religious, political...
Lion Gardner: Well, is that true in high ed?
Lots of responses: Yes.
Lion Gardiner: If I went to an Evangelical Protestant college . . . . What's the impact on our high school teachers who then send us their students? You wonder why the students do not know what they should know when they come to us and why they're unprepared. It has something to do with us. This is how they've been taught. There's some interesting data on teacher preparation in schools of education in respect to critical thinking. If you're interested, I have some transparences on that some other time. Not true?
Q: You know they're teaching twenty-five classes a week and getting students motivated?
Lion Gardiner: Not, ed professors, us. Alright, how are we doing here? Now, many of the things, not the evolution, but some of the things we're looking at are really low on the Bloom Taxonomy. Evolution is, I guess, level two, comprehension of the concept. How about thinking skills?
Here are the results from New Jersey Assessment of General Intellectual Skills. After two years of college. State of New Jersey decided this was important, so this is now available for anybody to use from ETS under the title, Tests in Critical Thinking. How about problem solving? Engineering problem solving. What's going on here? He used the Bloom Taxonomy again and he took all these things and rated every one of them on the Bloom Taxonomy. What the so called problems were is what he calls "exercises." They're at level three of the Bloom Taxonomy, simple application of concepts understood. Teach them how to run an equation, give them new values, plug in . . . ?. Not analysis, synthesis, and evaluation.
Now, let me finish up. The point here is just to lay out some general things for us to consider. If we really want to develop higher order learning, what are the conditions that facilitate that? What are the conditions on our campuses now? What needs to be done to make sure that we get the results that we want?
Okay, let's summarize what to do. What are the researchers telling us to do? Well, basically, use research as opposed to ignoring it, and use research-based practice. This is not twenty years ago. . . . . We know many, many useful things. Number one, be careful in identifying the intended outcomes. Be clear about what you want the end produce to be. What should the students be like when they walk out the door, walk out the door of the class, the course, the curriculum? All three of those levels need to be taken into consideration. Number two, alignment. Assessments need to be aligned with those intended outcomes, the processes we use to help them get to those intended outcomes need to be aligned with the intended outcomes. Everything's aligned up and down. Ed is going to give you a paper, a research paper that reports a so called four to one effect, which compares alignment with misalignment where theses things aren't not really looked at that closely. Four to one learning effect. Finally, set high expectations. We need to raise the expectations way up for the students and they will rise to it. How to do it is complex and we can discuss that. The basic principle is to set high expectations.
Let me, just quickly, give you three principles that came out of the 1998 study, national study, called Involvement in Learning. The researchers and experts on the committee, sponsored by the federal government, Department of Education, said this, three things. Three things you have to be really careful about -- setting very high education, having the students actively involved, all the time, in learning, and frequent assessment with corrected feedback so they can see how they're doing. Thank you very much. Oh, one other thing. I have other handouts to you and I will get them out to you. There are three things you'll get.