Ed Neal- Discussion
[audio turned off briefly - some questions, discussion gone]
R: What I found is that they are too general. They need to be broken down into measurable outcomes.
R: Another one of my criticisms and it stems from exactly the same elementary school stuff is that you can't measure the outcomes at one specific time and think, "ok that's the outcome and that's our measure of it" because for all you know that class, course may have outcomes for the rest of the students life or never.
Ed Neal: Well, one interesting thing and I think Lion's presentation a little earlier dealt with this is if you are interested in those kinds of outcomes for your course, if you're interested in life-long learning, would it hurt so much to say that? To lay that out and say, "This course is not just a course in archaeology. This is a course for life and you'll be learning skills in here and here are the skills that may be useful to you." It doesn't say that you can't do these things. These are simply from different course syllabi that people have given me to look at some things some people are doing and whether or not we felt they defined what they wanted sufficiently well. Yes?
R: It's interesting that you say these are from course syllabi because I think it's good for us to make quantifiable objectives for ourselves when we plan our courses, but if I listed those on my syllabi, it would look so Mickey mouse. I don't think students would really appreciate it. The other thing is that couldn't you have at least some skills like writing a beautiful. I mean not just writing a poem in iambic pentameter that uses three images and you know what I mean. I mean all of those things are important for writing a good poem, but there's something more than that in writing a good poem.
Ed Neal: Ok, it depends on what you're trying to teach. If you're simply teaching iambic pentameter, you want the student to just recognize what it is. It doesn't matter. If you're teaching creative writing and poetry then this is not an appropriate outcome. What I'm saying is that you could write an outcome, but and here's the big but. If you say you want them to be able to write a beautiful poem, what are the criteria?
R: Well, I guess some things that you teach, you teach from mastery or apprenticeship.
Ed Neal: Ok, but what I'm suggesting here is that I want you to reach. I've worked with some creative writing professors at Chapel Hill and I've asked, "How do you assess creative wring?" He said, "That's the question my students keep asking me." And, I said, "Well, let's work on that." And, they have developed criteria for assessing writing that they believe in and then the students can look at and say, "Oh, that's what you meant." I'm simply asking us all to be more honest, not to hide anything. It's too easy to say, "That's not the poem that I wanted. No that's not the poem that I wanted either."
R: I thought you ranked these more or less from what's easily measurable to most difficult to measure from 1-6, so I approached it that way. I think that if I were to use some of the rewritten measurable outcomes, I would still want to tell the students what I was doing. That I was trying to start by using a measurable example, but the goal is to get at the general by analyzing a particular historical document. So, they will be able to then critically analyses other kinds of historical documents.
Ed Neal: I think what you're both getting at here is really and there is an example of a real life, honest to god, syllabus written this way and it doesn't look Mickey mouse. When you think of what your course outcomes should be, you have to think in terms of generalized outcomes. Such as, analyzing primary documents. This is my primary goal. I'm going to go north, north-west. My specific goal is I'm going to reach the base camp at this location by noon tomorrow. That's very, very specific and you can't achieve the overall goal without achieving the specific objective that you have and you do want to do the other. So, you really have to start thinking, in terms of course design, "What are my generalized goals?" This is your contract with your students. "When you leave this course, you should be able to do the following things or know the following things. This is my promise to you." Then, when you start looking at the units, you can be very specific. You say, "I'm going to give you a primary document and I want you to do the following things with it. Notice how this relates to the big goal of our course." So, you start at the broad, general outcomes and then as you look at those and you look at what you've got to teach, "What specific kinds of things can I have them do to reach those things?" A particular outcome like crucially analyze documents, might imply a dozen very specific activities in outcomes. Don't worry about that initially. You may have those in the back of your mind but what you really need to think is, with my Western Civilization course, what should have been the general goals of that course? It should not have been to know when the French Revolution occurred in the following period and so forth and so on. The goal for that course should have been, something along the nature of, to be able to analyze historical events in the light of lets take a couple of things, in the light of the struggle between individualism and freedom. That's a big broad general theme and it could run like a thread throughout the whole course. It's a goal. That's what I'm after. But, specifically, looking at the French Revolution, how does this theme play out. So, what we have to do is when we start thinking of outcomes, this little exercise is to provoke you into thinking about these things. All of these examples could be rewritten very easily, but when most people just looking at them like "Will know how to use Microsoft Word 6.0," what does "know" mean? "Do you mean I have to know all ten thousand features of it?"
R: When we tried to collapse, we came up with change the verb and formalize it.
Ed Neal: That's a nice rule of thumb here. In the handout, do not use these words because although they are perfectly nice words it means different things to different students, so what we look for are verbs and ways to precisely define.
R: It seems to me that one of the problems of this sort of flat way of trying to do this is that for anybody when you're trying to rewrite this thing, it's decontextualized, especially for people like us. If we can momentarily recontextualize it, what we're trying to do is what kind of activity would we like for our students to do at the end of the road. To take these things, I came closely to saying that all of these could be rewritten, but without knowing what the context was, it's very difficult to do it.
Ed Neal: This is what happens when you give a True/False test to very intelligent students and we're all very intelligent students. You're looking at this test is a much more complex way and it's a good way. One of the things, to think about context, if you were teaching a secretarial course and you had this first objective here, it would be broken down into hundreds of possibilities here because a person who has to use Microsoft Word as a secretary in a department must know a lot more than a student who just has to write a paper. So, the context does matter. And, if you look at examples like "understand the human circulatory system" is this an introductory biology course or a med school course? How you rewrite it depends on what the context is.
R: But it seems to me, in general, how you rewrite it has to have in mind that the student can do some activity at the end. I could then force myself to say the student will be able to write a poem in iambic pentameter. I'm not so certain that I, as a teacher, would feel good about having encouraged my students to write … and I can't imagine that that would be a good outcome.
Ed Neal: As I say, a lot of these things are taken from syllabi.
R: Relating to what I said earlier, emphasis is on students doing active stuff and that's how they learn better. What I've seen in the past decade is an increasing passivity of the students to point that the university invests in these white boards. The professor is expected to write out the notes, print them out, and hand them out to the students. They don't even take notes anymore, nor do they want to, nor do they anyway. The increasing activity that we're suppose to get out of them, jogging with this increasing passivity of the student, might be even harder to achieve.
R: That's just glorified lecture. If you can lecture with a white board, you can lecture without one. The technology just exacerbates the problem.
R: If you give them two assignments of outside stuff to do that is very active and then I see fewer people turning them in. They don't want any active stuff. They don't even want to show up for class anymore. There's more class cutting than ten years ago.
Ed Neal: I told you I taught in Georgia. When I taught, Lester Maddox was Governor of Georgia. There was a problem in the prison system in Georgia. Prisons were in bad shape. So, Lester made a study of the prisons and he had a conclusion which he announced at a press conference that there was nothing wrong with the prison system, we just needed a better class of prisoners. I think that's the danger that we're in here. It's getting harder. The technology has muddied the waters a little bit and it's made it possible for students to demand things that are actually counterproductive to their intellectual development.
So, the question is how do we resocialize them, to use Gardiner's term. And, I think the answer to that is two things. One thing that Lion stated, you've got to hit them as a team. You can't do this by yourself, you have to enlist your colleagues. I had two clients a couple of years ago. Both of them were graduate students in History. Both of them teaching different sections of the same course. One at 8:00, one at 10:00. The one at 10:00 came to me with a funny story. She said a student, an undergraduate student, had enrolled in the class, stayed in a week, dropped, then showed up a week later. So she asked him, "What's going on?" And, he said, "Well, this class was too demanding for me. We have all these discussions. You keep pounding us with all these questions. We've got all these writing assignments. So, I figured I'd just take another section of the course. She did the exactly the same thing." So, he couldn't escape. And, he'd rather take a 10:00 class than an 8:00 class, so he went back. The other thing is, if you want to resocialize them you have to do this the first day. You have to think about it. Student walks into a 200 student classroom, instantaneously several things are going through their mind. First of all, lecture class, which means that maybe I can skip because I have a buddy in the next few rows. Maybe I can use this class to do my math homework. A bunch of assumptions suddenly are registered on that student.
There are different expectations for the different sizes of the class. They walk into a seminar room and there's only going to be ten students in this class, different assumptions. It's up to you to know that those assumptions are and smash them if they're not the ones you want or rather redirect them. How do you do this? You do this by giving them a few samples. You've impressed the hell out of me in the last days with the conversations about the things you do in class. You should start them on the first day. When they walk into a two hundred student class and they got active running on the first day, their assumptions have been thrown out the window. We have a different kind of class here. Students are very malleable. They walk into your class expecting to do active learning, they go into another class expecting a lecture, which is what they got the first day. How do you manipulate this the first day? Make it an object lesson, have the discussion, have the case study, have whatever it is, and then point out, "Hey! Look what we did here today! I asked questions of everybody in the room. Everybody responded. No one had a heart attack. Wasn't that wonderful? I will continue to do that and that has implications for you in this class. You have to come to class prepared."
One of the best practitioners for this is a fellow in the Philosophy Department, Jeff …. A young fellow about my age who teaches 250 students in the History of Philosophy course section. He does not lecture. He bounces up and down the aisle and he fires questions at this one and that one and the other one. After the first week, he has memorized their name. He is superhuman. They would figure that they can't escape, no one escapes. He gets the highest evaluations of anyone in the department and they're at the edge of their seats. Lion said, "challenge them." He challenges them, everyday from day one. They know that they better have done the readings, they better have thought about the issues because they're going to be quizzed on them. He sort of works it into a narrative. The questions he asks all sort of build to the objective of the day. I'm not suggesting that other mortals could do this type of thing, but you have to find the way that works for you.
Anne Pyburne: I just want to point out as social scientists, as Anthropologists, we have a little bit of an advantage over some of our colleagues. We can pay attention to the ethnographic profile of the people we're trying to reach. I think of reaching as a kind of ethnography. You have figure out the language of the people that you're trying to communicate with in order to get the message across. One of the most astonishing things to me about a local program called the Freshman Learning Project was the ethnographic profiles of students in this university. They were completely unlike what I thought they were like. When I found out what their backgrounds were and what their level of knowledge was and how they spend their day, I was astounded. It wasn't that it was just bad news although it was, once I knew that, I could address it. I could use it to get to them. I could bring up subjects. I could couch my information in terms of language they're using and get their attention right away. As Anthropologists, we can have an advantage in our teaching.
R: One of my anthropological colleagues…has written a book, "Coming of Age in New Jersey" and it's a study of dormitories in an institution very similar to this one. How they spend their time.
R: I agree with you at the basic premise here. I certainly have learned a lot from the active activity of putting lectures together that my students have been listening. But, there is a down side to the active approach, too. I've seen a lot of examples of it where it just simply degenerates into a sort of lazy approach. Let's put the chairs in a circle and discuss, bs one another for an hour and that's it. Students get graded on how much they talk.
Ed Neal: If you want something spontaneous to happen you want to plan it very carefully and that is something you just have to keep in mind. We were talking about small groups. Small groups are easy, but that's not group work, that's not cooperative learning. Cooperative learning requires a lot of stuff and it requires a lot of research from the teacher in order to know how to run a group properly so they don't degenerate. Not all of these techniques are going to be things that you can just plug right into your course. The way you set them up and the task you set and the requirements and the assessment you set for that discussion or whatever it is, can make it very very productive.
I was reviewing a professor several months ago, a History professor, who after half an hour in an hour course said, "Ok, I want you to get in groups and discuss something" and they had this discussion and they had some consolatory questioning afterwards before the bell rang. And, I asked him afterwards, "Well, that didn't seem to really link a heck of a whole lot to your lecture or anything. Why'd you do that?" And, he said, "Well, I noticed that they've sort of go to sleep after twenty minutes, so I thought that would just wake them up." That was his reason for using active learning. It was active, but it wasn't learning and there's a difference between the two. Active learning is whether or not the student's mind is engaged in higher level learning, syntheses, evaluation, analysis. Whether or not their mouths are moving, whether or not their bodies have shifted in the classroom is not. You have to think why am I doing this? How can I structure it in such as way that there is a product and that they are required to show me something or deliver something and that there is a legitimate and challenging question? Most of the time, I think we're gun shy. We don't really want to give them a really hard questions.
Another professor I know at Chapel Hill teaches in Religious Studies. He's sort of like the polar opposite. He teaches 250 students in his section. The way he has arranged it, because he's not a dynamic speaker, he's a very scholarly soft-spoken fellow, he has a discussion sections meet before what he calls the assembly, when they're all in one room together. And, these are essentially problem based sections. The TAs are there as facilitators if the students get stuck on the questions they've been given the TA can make some suggestions and then fade back out. When they all meet in the big class, he conducts a discussion across the groups. "Let's see what group one had as an answer to the first question today. Ah, that's interesting. Did anyone have anything different?" And, he goes from there and he conducts his discussion beautifully and they are on the edge of their seats trying to sell their ideas and their hypothesis to the questions because there are no right answers to the questions he poses. They're all philosophical, theological questions, questions of interpretation. And, so they learn very quickly in that class that their task is to do the interpretation, to do the thinking, and to figure out how best to defend a position on the basis of what they've read and what things they can induce from that. He does this. He's another teaching award winner. He did this on his own and he came to me one day and said, "Do you think this will work?" I said, "Sure, as long as you provide the appropriate kinds of questions for them and instruct the TAs to keep their mouths shut, unless there's a problem. It will work." He was one of the ones who took the time to work with his TAs very careful to instruct them of their tasks. Most of the discussion sections are what I call … sections. "Here's what the professor really meant to say when he lectured on so and so." Instead of using them as an integral part of the instruction.
Q: I just wanted to ask about the bias in these active learning techniques. Especially in oral presentations, there's a strong bias against shy students, against women students who would be outspoken by men. Of course you can overcome this with other active things, but it's a real problem when you deal with lower level undergraduates who are used to a different kind of learning.
Ed Neal: Yes. You have you think very carefully about how you introduce it and you introduce the philosophical question and that question is should one of my course goals be to help those students grow out of their shyness, their reluctance to speak. Is it ok for me to let those students out of my class and let them go through college and never learn how to speak up in a group? I don't know what the answer to that is. You have to answer that question yourself.
R: If you've got 250 students you can do this, but in a smaller class, I often select the groups myself. I don't let them group themselves. I kind of put people at similar levels of introversions and extroversions in the same group, so that I don't have an introvert sitting in a group of six extroverts and they never get a word in.
R: Even in the large classes, you may not have the knowledge of the students, but can still randomize the groups.
R: I do a lot of group work and they sit at tables and they have to rotate throughout the semester on who reports. So, at some time in the semester, the shy student has to report to get that experience. So, that everybody has as role and everybody has an active role and everyone rates each other.
Ed Neal: You're getting really close to the cooperative learning model I was talking about, doing a lot of the kind of planning and getting them to spend time in their groups and talking about how efficient we are. Is everybody contributing? There are a whole set of guidelines. K-12 we tell students, what? Do you homework. Don't talk to each other. Did you do this yourself? Then we tell them to get in groups and learn. They don't know the rules. You have to train them. The other thing that happens is, the first couple of times they're in groups in college they're bad groups, they are not organized and it reinforces "groups bah, no good." You're not only working against their K-12 experience, you're also working against whatever experience they've had in college and try to form really good groups.
R: The dualistic prejudice against learning from ignorant peers.
R: That's what I've found, especially amongst some of the students is this long, long tradition of listing to authority and treating you, the instructor, as the authority figure and waiting for your initiative and enterprise in the discussion rather than starting from themselves.
R: When you're trying to get a discussion going, you throw out a question. No one answers, they stare at you.
Anne Pyburne: The ethnographic approach I find effective is telling them, in this context, in this university in the Midwest, that the criticism of people graduating from their employers is that they don't know how to work in teams. So, in this class you're going to get some experience of working in teams to make you more employable.
R: I have also tried to use team projects in a number of different ways and I've got an adjunct teaching "Exhibit Development and Design" class and she has spent the first third of the semester team building and team projects and inventory skills and Myers Briggs and this really wonderful stuff and it's made me question how I use team projects in other course where here's the deep end. How can I facilitate their team building and not spend a third of the class that's supposed to be North American Prehistory on how to do team work?
Anne Pyburne: This is something we touched on yesterday that we really need to think about as a group, especially the course assignments. We're going to have to meet again because we are not talking about putting some courses on the web, we're talking about putting a curriculum on the web and that means we have to figure out how to divide up the labor to some extent. We don't want to all use Lynn's video in all the classes we design for the Fall or Joe's examples. We want divide up the tools a little bit, but we also need to divide up the labor. To some extent, some of the things you're worried about is that we need in the curriculum to get some of the things at the entry level so that later on you don't have to teach them how to work in teams. Teach them how to make a team in the earlier classes. In response to something you were saying, I teach a meta-class. That's what I'm teaching this semester. Alfredo is one my assistants. We are teaching people how to be students in that class. It's called "Lost Tribes in Sunken Continents: An Introduction to the Scientific Method." I pulled them in by their fascination with Atlantis and space aliens. We visit the library to learn how critique a website, to practicing forming groups. I even have lessons on how to learn from a lecture. I don't give very many lectures, but I teach a class on how to learn from a lecture because they're going to have other classes with lectures, right? In this context, we haven't been able to talk about how we're going to divide up the labor. But, we're going to need to.
Lion Gardiner: Learning to learn from a lecture, but learning to learn. We cannot assume our students have learned to do that in high school. Sadly, about 750 Rutger's students accumulated over the semesters of my courses and I asked them have you ever been taught how to study in high school or in college or in seminar or did the teacher take a little bit of extra time, anything? I think you're just going to have to assume that you're going to do it.
Anne Pyburne: The second week of class, I give them a really horrible, boring, full of facts lecture and then I stop twenty minutes before and ask them a question and ask them to answer based on the notes they've taken. It's hilarious.
R: I think one of issues with curriculum building in our field is that 1) anthropology majors are always taking their course in the context of general education students, there maybe some exceptions. And, those students interested in archaeology are only going to take two or three courses in archaeology. So, building a curriculum in archaeology is a good thing, but it's very hard to get a system approach because of those two factors.
R: Furthermore, institutions vary a lot in terms of their resources. We can have a range of courses for a certain level or something like that.
R: I think it's a good think, but I think we need to think in terms of the system and how it happens and how it gets used and what the effect is it's going to be pragmatary.
Ed Neal: Are the key outcomes of your non-majors the same as the key outcomes for majors? I think that would be a key question to ask.
R: I agree with everything that the last three speakers have said and would just suggest that one way to operationalize thinking about this as a system is to think of the courses that might be offered, but in a typical way that a student approach archaeology as an undergraduate. For the students who maybe more curious, it probably would not be hard for what a model major would be like in anthropology and how many courses an anthropology major can take, it's a very limited number in most cases. So, to ask the question, if someone is going to take some archaeology course, which kinds of things should they be and what kinds of things might they cover. To think about it in a real world department.
Anne Pyburne: I wanted to pose the question to the designers about the possibility of conceiving the course that you design in some modular way so that it may be possible to take a portion of that course and drop it into a local course with some degree of integrity. You will want to make it clear how each modular fits into your course. You might want to think about how they can be moved around.
Ed Neal: Like units?
R: That's what I thought we were supposed to be doing. I thought our goal was to not create a course and here's how you have to teach it, but here are the various things you do to test and people could say I have one that's better , I like this. I think we have to look at it that way.
Ed Neal: It will be a lot easier if you have objectives so that they'll be able to say, "Oh, that's what students are supposed to get out of this."
Anne Pyburne: Objectives are what we need to divide up.
R: Just as I have been listening the last couple of days, it seems that there are two different things going on here. One, in terms of focusing on the content and how that does and the other is the format. What we've been discussing here, there are exercises or ways of teaching that people have been coming up with that have cut across a number of the topics so much that they could be called a toolkit. I think it will be a challenge to actually decide what goes into the toolkit, what goes into the topical modules or do they cross.
R: We did put many years of effort into this toolkit. The curriculum task force has spent about ten years talking about this and establishing this list of themes we think needs to go into the curriculum. The thing is what we decided in the last week in Florida is that every single one of them needs to go into every single class. That doesn't help us much. There's actually two different structures here that you're developing simultaneously. One of them is the courses which if they turn out the right way, they'll be created in terms of objectives and ways to assess to assess those objectives. Each course will be these parts and the sum of the course will begin to address Anne's issue of what is it that we want. But, then cross-cutting that there's activities, things to do. Some of which may be more portable than others. In terms of what it is you see as a product of this process, I think early on we need to think in terms of both of these because the second set will benefit from the first set so that they won't realize that they're two separate sets.
Anne Pyburne: I'm worried that if we didn't imbed that they are two separate sets, they're not just ideas on the web.
R: I tend to think of this as a pedagogical problem. We have questions and we have real world questions that we as archaeologists are interested in solving. One issue is how do we engage students in thinking about those questions. We all have data, different kinds of data, different qualities of data. And, we all know as professionals that how you go back and forth from the information to the questions and how we navigate that, well we do it with tools. The way I try to think about this is how do we get students access to real authentic questions with real authentic data and how can we give them the tools so they can start learning how to work through that process. All courses can do it that way and I think technology can help, but you don't need computers to do that and you can do that in many ways.
Anne Pyburne: So, you're talking about besides presenting materials that people can use to teach, you're talking about materials for teaching archaeologist to teach.
R: One of the things that I'm really concerned about is in the curricula that we'll be designing through this method that the lecture road as the dominant mode of teaching will be reinforced unless it's presented in a very different fashion from what's normally presented on the web.
Anne Pyburne: That's why you're here.
R: Well, originally, I though the outcomes would be a series of syllabuses or something like that. But, as we go into this it seems to me that what we need is a series of lesson plans and the product is going to be day by day, this exercise, that exercise.
Ed Neal: I'm glad you said lesson plan before I had to.
R: I'd like to second what Liz just said. I like the concept of lesson plans it's what we do for K-12 teachers because they can take that and fit it into what they do. I'm sold on this idea of active learning but I'm also sobered by what Lion told us. It's something that's coming, but it's not the dominant way that people teach. We have to be pragmatic. We have to make up a really neat model that people won't dismiss right of the bat. And, if it's too incompatible with what people do, if it's not adaptable to what people do now they'll just ignore it. I think we have to keep that in mind. I think the key is adaptability. We provide pieces that are very adaptable that people can pick and choose from and plug into their thing and also we can provide a model if they want to adopt or not, but we don't want our product dependent on people adapting the model hole hog because they won't.