XI. Science as Communal Problem-Solving Noretta Koertge
As a first approximation a Bacon, or a logical positivist, would have viewed the ideal scientist as an inductive robot, a gatherer and systematizer of data - although even Bacon allowed a place for "prerogative instances", striking phenomena which somehow aided the human imagination in its quest for the hidden "forms" of complex processes. And the positivists' "context of discovery" admits the necessity of creative hypothesizing for scientific inquiry.
The philosophers we have studied have added a number of additional dimensions to any model of the ideal scientist (homo scientificus). She must also be a metaphysician, a methodologist, a practiced problem-solver skilled in the use of heuristics, and knowledgeable not only in the primary subject matter of her field, but also in its professional mores so that she can be successful in getting grants, getting published, and getting recognized!
Small wonder that any brief philosophical account of scientific inquiry comes across as simplistic. And when one adds in the diversity of contexts in which science is done (cf., team research requiring a big accelerator or expensive, extensive longitudinal studies with the experience of the theoretician working with pad of paper in relative isolation) and the differing cognitive structure of the projects undertaken (cf. Kinsey's pioneering demographic surveys with Levi-Strauss' search for universal abstract structures), one might well despair of finding any useful general account of science.
What we will do in this situation is follow a strategy similar to that of theoreticians in the social sciences: We will postulate a general decision-theoretic model of the scientist as problem-solver and then describe some typical dilemmas which scientists confront and analyze typical factors which are relevant to the choices which have to be made. Our approach will be normative in the following two senses:
(i) If we discover what appear to be egregious imperfections in the way scientists conduct their research (if their decisions appear to be irrational or their problem-solving techniques far from optimal), we will not succumb to naive naturalism and incorporate the "errors" into our model, no matter how widespread or systematic they may be. What we will do, however, is to look again at our own characterization of the typical choices which must be made to see whether we have indeed correctly described the "logic" of their situations, to use Popper's terminology.
(ii) We will also be alert to the possibility that contemporary scientific practice could be relatively efficient in pursuing the professional aims of present science, but nevertheless those aims could be faulted on either moral or epistemological grounds. Although we stand ready to criticize both the ends and means of scientific practice, we will not fall into the trap of many normative philosophers of proposing utopian standards which are so removed from actual science that they cannot even serve as regulative ideals. Thus we hope to avoid both the naturalist fallacy of arguing from "is" to "ought" and the utopian fallacy of forgetting that "ought" implies "can".
a. Informal Decision Theory
For the purposes of our discussion here, a rather informal rational choice model will suffice. I will not worry about infinite regress problems (how does one decide when to decide, what to decide, etc.), the issue of how to incorporate moral principles into the decision matrix (does the option of killing an innocent child merely get assigned a very large negative utility or is it ruled out a priori?), the question of how best to incorporate a measure of how uncertain we are of the various appraisals of outcomes (should we use second-order probabilities, or intervals, or?). In some cases, I don't have a preferred answer to these important formal and foundational questions and, even where I do, at this level of discussion, nothing much seems to hinge on the details of the solution taken.
I will, however, plunk for maximize-expected-utility as a decision-rule as opposed to satisficing or minimal strategies. There are various reasons for the choice of this approach, ranging from its generality to simplicity of exposition. Analyzing situations according to this model forces us to be very clear about the two distinct major elements of appraisal, viz. the desirability/value/utility of a possible outcome vs. the probability that it actually occurs. And although I agree with Simon that we often review options serially and act without further ado when we find one which is "good enough", as Simon himself says, our standards for what is satisfactory depend in part on our prior experience with similar choices. So where Simon would describe my decision to buy the first pair of socks I see as an instance of satisficing, I would describe it as a choice between "buy these" vs. "look further" with an assignment of a low probability of finding anything substantially cheaper and a high probability of spending lots of time looking, hence a low overall expected utility. Simon's satisficing standards are useful rules of thumb, but if challenged the agent's mode of justification would have to rely on (crude) estimates of expected utility. I will generally present scientific inquiry in terms of the choices made by individuals and introduce group considerations, such as professional norms or considerations of who else is pursuing similar problems, as elements within an individual's decision matrix. I will also present the various choices as if they were the result of conscious, deliberate computation. Of course, this is often not the way choices come about. Sometimes, one just "sizes" up the situation and acts "instinctively". However, one hopes that the actions thus taken are not too much at variance with what rational deliberation would dictate - and one feels open to criticism if there are major discrepancies.
b. Separating Values from Probabilities
Decision-theoretic analyses presuppose that the probability of the outcome of an action is independent of the desirability of that outcome. (The fact that I would very much like to be dealt a Queen to complete my royal flush does not make it more likely that I will be dealt one.) This is a fundamental component of rationality - one which is recognized even in folklore: "If wishes were horses, beggars would ride." However, what I call the ideological fallacy, viz., allowing the desirability of X to influence the probability assigned to X is a very common one and has long been recognized as such. Demosthenes, Third Olynthiac, 19 says: "...in such proposals the wish is father to the thought, and that is why nothing is easier than self-deceit. For what each man wishes, that he also believes to be true." Julius Caesar echoes this: "...we readily believe what we wish to be true." (Quoted in Stark, 1958, p. 47.) Perhaps less common, but equally fallacious is the pessimistic assumption that the worst is likely to happen. (Preparing for the worst is quite a different matter.)
The relationship between values and probabilities can be very complex. Let us look briefly at some potentially confusing cases.
Although one should not let probability estimates be a function of desirabilities, the converse is often quite legitimate. For example, it is certainly permissible to assign a high epistemic evaluation to measurements which are accurate (i.e., which have a high probability of being correct). On the other hand, Popper places a high value on hypotheses which are unlikely to be true. In both examples, the value assigned to a state of affairs is a function of its estimated probability. There could also be survival value in adjusting one's emotional responses in the light of probable outcomes. ("Since it's bound to happen, you might as well learn to live with it.") This tactic may also backfire, however, if it leads to an inauthentic cover-up of one's true feelings or a damping of societal indignation. (cf. Bobby Knight's unfortunate remark that if rape is inevitable, one might as well relax and enjoy it.)
I have said that desirabilities should never influence probability assignments, but there is at least one sort of case where this seems wrong-headed. Surely the fact that you like Macintoshes better than IBMs increases the probability that the machine chosen for your office will be a Mac! In any situation where the state of affairs in question is causally influenced by human action certain probabilities will be a function of certain desirabilities. Yet, if we look closely, even in this example the move from high value to high probability is also conditioned on other factors, including probabilities. I only assign a high probability to the outcome, "Mac in your office" because I believe that there is a low probability that your preference for the Macintosh will be counterbalanced by other considerations, which favor the IBM, such as lower cost, compatibility with the secretary's machine, etc.
The probability of any outcomes of scientific inquiry which are causally influenced by human choices may then also be legitimately linked to the values humans place on those outcomes, but we must always analyze those links very carefully and make sure they avoid the ideological fallacy. Thus the desirability of cold fusion may causally influence the probability that, if cold fusion exists, it will indeed be discovered; it may increase the probability that people will publish premature reports of its discovery. What the high utility value can not do, however, is raise the probability that two adsorbed deuterons at room temperature will fuse into helium. The whole trick of being a good scientist is to let one's values, especially cognitive values, guide the direction of one's research without letting them influence the interpretations of one's results. As Brecht's Galileo puts it: "...if we find anything that would suit us, that thing we will eye with particular distrust." (p. 96)
Recently critics of the idea that one can separate factual questions from value in issues science have claimed that it is utopian to expect any element of scientific thought to be untainted by the interests and ideology of the powerful scientific and societal elites. However, I will argue that with care the distinction can be made.
Let us now look in more detail at the probabilities and values which enter into typical decision situations within scientific inquiry. We will order our discussion according to the Popperian schema presented in Chapter 2.
c. Choice of Research Problem
Most of the time, scientists do not sit down and think about the general direction of their research - it often seems to be already more-or-less fixed, either by the sub-discipline they're in, the equipment available, their past personal achievements, etc. But graduate students or scientists who are for some reason switching fields often agonize over such choices and Popper suggests that science would probably be better off if all researchers were more reflective about their choice of problems.
So let us ask, what factors should influence the direction of one's research? An interview with Kinsey yields a common answer: "Gaps in our knowledge." While teaching a marriage course, Kinsey discovered that less was known about human sexual behavior than that of most other mammals. But this can only be a partial answer and smacks a little of Baconian indiscriminate fact-collecting. There are lots of gaps in our knowledge. One could have also surveyed the spitting or sneezing or scratching behavior of the American male. Scientists are generally not well advised to undertake a project simply because it hasn't been done. One also needs to ask whether it's worth doing and whether the project is feasible, or to put it in decision-theoretic terms, how valuable is the outcome of the research expected to be and how likely is it that the research can be successfully carried out?
Let's look briefly at these two components of rational decision-making in reverse order. Estimates of the probability of success become very rough when one is dealing with very innovative research. We are faced with a version of Plato's paradox of the discovery of new knowledge - if we don't know what we are looking for, how will we know when we've found it? Likewise, if we don't know what the solution is, how can we say how probable it is that we will happen upon it and recognize it as a solution to our problem! Nevertheless, sometimes powerful search methods or strong heuristics are available and then we may have reason to judge the problem to be "ripe" for solution.
We can generally be firmer about judgments of infeasibility. If research funds are not forthcoming, if crucial apparatus has not been designed yet or certain cognitive tools are missing, we may be very confident that there is a low probability of successful solution. It may nevertheless be rational to undertake the research, however, if the problem is important enough.
To assess how important or interesting or worthwhile a research project is (I will use these ascriptions of value more or less interchangeably - as do scientists), it is often helpful to distinguish two things: (a) First, how pressing is it that the problem be resolved and (b) how valuable might each of the various plausible solutions to the problem be (as far as we can tell)? Cognitive and practical/political considerations enter into both components. For example, suppose we discover a contradiction within our knowledge - this is clearly problematic, but some contradictions are more "deep" or "central" than others. (If we were to simply drop one of the inconsistent pairs from our data base, how much content would we lose?) What external circumstances hinge on resolving the inconsistency? (Does the credibility of a key witness hang on it?)
Problems also gain in importance if at least one of the possible solutions is very attractive, either intellectually because one could derive lots of theorems or solve other problems with it, or pragmatically because it could lead to useful technology or ideological victory.
There are other sources of value which influence the choice of research projects. Some problems seem to be intrinsically intriguing, not because they are central to our understanding of the world nor to our comfort, but simply because they offer an intellectual challenge. The examples which spring most readily to mind are mathematical puzzlers or logical riddles (such as those collected in Martin Gardiner's and Raymond Smullyan's books), or problems involving the construction of triangles and other geometrical figures with matches. Many (all?) cultures have puzzles such as the Tower of Hanoi or the Devil's Needle (American Indians) and undoubtedly a careful study of the structure of such folkgames would tell us a lot about human cognition.
Often many of these puzzles can be stated in very simple terms and at first glance appear to be very easy solved. It is only after exploring the most obvious solution paths and seeing how they fail that we become aware of how "deep" the puzzle is. And then we either get hooked by the challenge of "mastering" it or turn away in disdain and decide not to waste our time on "just a game".
I believe the pleasure of pure puzzle-solving is an important component of scientists' evaluations of research problems although it would be difficult to parse out the extent to which this factor is decisive because often the scientific problems which are most challenging as pure puzzles also promise to give deep insight into nature - One thinks, not only of mathematical examples in physics such as Zeno's paradoxes about motion, Olber's paradox of why the sky is dark at night, the Einstein-Rosen-Podalwski paradox in quantum mechanics, but also of Aristotle's puzzles about nutrition and growth (how does food, such as grass, take on different forms in different species, or in different organs within one animal?) or the classical problem of chemistry (in what sense are elements "contained" in compounds?). In each of these cases, scientists obviously wanted to know the answer to the problems because they were genuinely interested in understanding motion or nutrition or whatever, but I suspect they were also lured by the sheer puzzle aspect of these problems, too. Puzzle-solving may often be associated with intense feelings of competition ("if I could solve a problem which stumped Einstein or Fermat or Aristotle, wow!"), but need not be. This factor probably varies enormously amongst individuals and also from culture to culture.
When we talk about the choice of research problems, we should also include the dimension of intellectual tastes, and as in the other areas of life, it is often difficult to analyze tastes. Let me just give a few indications of what I have in mind. Some subjects, such as dinosaurs, seem to have a sort of archetypal appeal. Plant biologists and entomologists complain about "vertebrate chauvinism" and claim that to know a nice algae or milkweed bug is to love it. Children are attracted to chemistry by the "stinks and bangs", but I fancy that many an adult chemist finds real pleasure in the colors of a nice crystalline precipitate or the smells of a natural products lab. Anthropologists talk about "needing to get back into the field" and what they're describing is not just the need for more data. Other scientists may be "ham-fisted experimenters", but take delight in working up data into "pretty graphs" or looking for "nice correlations".
Here again it is sometimes difficult to parse out these sensual and aesthetic considerations from those which are more directly connected with the traditional intellectual and pragmatic goals of science. "Messy" data are not just aesthetically unpleasant -they may also indicate a poorly designed experiment. Where we need to be concerned is if scientists systematically avoid or under-investigate certain phenomena because they find them distasteful.
Even this brief survey has turned up a wide variety of factors which might be expected to influence a scientists' choice of research. Let's now summarize them, look at ways they are interrelated, and comment briefly about appropriate weightings of these components.
In ascertaining how feasible a proposed research project is, a scientists will need to think about the prospects of funding, the accessibility of technical apparatus, and necessary cognitive "tools", as well as the "ripeness" of the problem for solution (i.e., the availability of a general theoretical framework, appropriate heuristics). Feasibility is a function of financial, technical and theoretical support available. One of the reasons scientists like to work within a paradigm or research program is that it increases their confidence that certain types of problems are solvable. However, as we remarked above, estimates of the intellectual feasibility of a particular research project are generally quite fallible. An individual scientist may also project an unusual probability of success because s/he has thought of a new trick for solving it. Individuals also vary greatly in their access to material resources and support teams.
In ascertaining how valuable a research problem is, the situation becomes even more complex. The importance of the proposed research to our understanding of the world depends in part on how cognitively pressing the problem is (how big is the gap? how deep or how central is the contradiction?), but it also depends on the details of the as yet unknown solution and how fertile it will be - and this is a much more speculative evaluation.
Research directions also take on value because of their degree of relevance to social problems and interests. Social evaluations obviously influence science via funding priorities, but perhaps of equal importance are the ways in which scientists' internalized social values affect their own individual research choices.
Closely connected to the cognitive evaluation of problems, according to how much their solution would improve our understanding, is what I called their "pure puzzle" value - the formal intellectual challenge they pose to the scientist as distinct from the metaphysical or theoretical insight they give us into the world. This is most visible in sciences closely related to mathematics, but I think it exists elsewhere, although the nature of the challenge may vary. I also think we should include the evaluation which accrues from the sensual and aesthetic experience which goes along with doing various types of research. Here there are enormous differences in taste, of course, and these help insure that a wide variety of research is carried out; yet there is no reason to assume that the division of scientific labor so produced is optimal.
For convenience, I'll lump these last two components into one category and call it the "fun factor". Now we have three major contributions to the evaluation of a scientific problem - the prospective contribution of its solution to knowledge, its usefulness to society, and how fun it is for scientists to work on. It's already obvious from the discussion that although these three dimensions of evaluation may happily rank order problems in the same way, they need not run in tandem. Some of our most pressing social problems may not make fun research topics nor contribute much to fundamental science. Not all fundamental science is fun either. Yet I think it would be a mistake to denigrate the pure puzzle aspects of science or the pleasure scientists derive from working with their materials because these rewards undoubtedly help keep them going when the satisfactions
of increased cognitive understanding or doing something to help humanity are delayed or uncertain. (My conjecture, contra David Hull, is that it's the fun factor, not the get-the-son-of-a-bitch factor, which keeps scientists in the lab for eighty hours a week - but again there are probably enormous individual differences here.)
Any listing of multiple sources of value immediately invites the question, how should these disparate evaluations be aggregated? But I want to postpone commenting on current debates about science policy (e.g., should science be for the people or for the scientists?) until we have looked in more detail at the structure of other scientific decisions. Although the choice of scientific problem is temporally prior, it is influenced by the scientists' hunches about later developments in the scientific process, and we now turn to these.
d. Decisions in the Context of Pursuit
In choosing a research problem, a scientist should already have made a rough appraisal of the availability of suitable methods of investigating the topic of interest. However, as the inquiry progresses, further decisions about research strategies will have to be made. Let us analyze the structure of these choices.
Let us begin with an apparently simple decision - the choice between two research methods. For example, what factors might lead Kinsey to choose to do face-to-face interviewing of his subjects instead of administering simple questionnaires? In this example, both methods are certain to generate data (the probability of at least a modicum of success is one), so the relevant comparisons concern the relative qualities of the data generated and the relative costs of two collecting processes.
The cost of doing face-to-face interviews was enormous. It required highly trained staff (according to Pomeroy it took three months just to learn the data-entry code). Some of the interviews took ten hours. And before doing the project, it would have been plausible to assume that it might be difficult to recruit volunteers for extended personal interviews. Why then did Kinsey choose not to use a paper-and-pencil questionnaire which could be administered anonymously to dozens of people at once?
Pomeroy lists several reasons. At the beginning, Kinsey didn't know what kinds of responses would be forth-coming and the open-ended interview allowed him to follow up leads (for example, ). This seems like only an argument for using interviews in a pilot study, but Kinsey expected to find (and did find) an incredible variety and range of sexual behaviors. More importantly, he felt that by interviewing subjects he could obtain more accurate data. First of all, he could adjust his terminology to match that of the subject. Because the public discourse about sex is so restrained and restricted, Kinsey felt it was essential that sex researchers be able to use the appropriate argot in order to communicate. (Thus a questionnaire would hardly have worked with the man who proudly claimed to be the father of twelve children, but indignantly denied ever having engaged in vaginal intercourse!)
Kinsey also believed that a skilled interviewer could detect and deter both exaggerated responses and under-reporting by repeating questions, checking that dates and episodes fit together coherently, and by first veering away from issues which made subjects defensive and gently returning to them later. By all accounts, Kinsey was a very gifted interviewer and probably did get excellent data. (Whether it would be worthwhile for other researchers to choose such a costly method is another matter.)
So from Kinsey's point of view, the scientific benefits of interviewing purportedly out-weighed the costs. But I wonder if our analysis is complete without including the "fun" factor. There are many anecdotal reports (Pomeroy, video) which indicate that many people found the experience of being interviewed by Dr. Kinsey one of great personal significance. No matter how modest a man Kinsey was, this sort of reaction must have been the source of some satisfaction. It was also widely known that Kinsey's files contained information about the sex lives of highly-placed officials which if leaked could have caused them great embarrassment - or even ruined their careers. The mystique surrounding this kind of power as well as the recognition of the trust on which it was based certainly helped Kinsey professionally and again, no matter how modest he may have been, it is hard to imagine that Kinsey did not find it exciting to record the case histories of such people. (Wouldn't you like to know about the private sex life of Nancy Reagan, or Phil Donahue, or the chair of your department?) In order to be such a successful interviewer, Kinsey must have found people in the flesh, so to speak, more fascinating than check marks on a questionnaire. At the very least, I think we can conclude that Kinsey did not find interviewing aversive; otherwise, he could not have continued to collect thousands of case-histories.
We have looked at a very simple choice-situation in the context of pursuit - a relatively straightforward decision about data-collecting which did not involve probabilities. More elaborate deliberations are called for when there are uncertainties about the type of explanation it is plausible and profitable to seek and when there is no guarantee that any approach will be successful. Further complications arise if competing research teams are in the picture. (Kinsey didn't have to worry about being "scooped.")
Even a crude analysis of the differing decisions made by Crick and Watson vs. Franklin and Gosling in their pursuits of the structure of DNA will illustrate these further complexities. For purposes of this example, let us imagine that the choice is simply between two different general strategies for investigating the structure of DNA. Strategy A (that chosen by Franklin and Gosling) is to work within the crystallographic tradition of making very accurate X-ray diffraction patterns and interpreting them rigorously by classical Patterson projection methods. Strategy B (that adopted by Watson and Crick) was not so clear-cut, but relied heavily on the sort of stereochemical model-building which had earlier led Pauling to his alpha-helix model of proteins.
From a decision-theoretic viewpoint what factors should enter into such a choice, and what are we to make of the fact that different scientists made different choices? Let us begin by laying out the deliberation as simply as possible using a 2 X 2 matrix with four outcomes.
How should we compare these strategies? Well, the most obvious question is, which is most likely to be successful? Here it would seem that the Patterson method clearly gets the nod. If one can get good diffraction patterns (which Franklin had succeeded in doing), the technique, though laborious, should in principle eventually yield an unambiguous structure. On the other hand, if the pictures are fuzzy (like Astbury's original X-rays were - because he was working with a mixture of two forms of DNA) then the probability of getting anything useful is low.
The big disadvantage of the Patterson projection method was the fact that it was slow and complicated to use, especially in an era in which all calculations were done by hand. Furthermore, it provided no straight forward way to narrow the search by introducing chemical considerations such as bond angles, what bonds with what, etc.
The big disadvantage of the Pauling method was that it might not yield a unique structure. Pauling himself, misled in part by Astbury's bad X-ray data, proposed a 3-strand model; Watson and Crick fiddled with a couple of other 2-strand models before settling on their dyadic helix. But if the final structure had not been so compellingly elegant (and there was no reason at the time these pursuit decisions were being made to assume it would be), the model-building strategy might have been inconclusive for a long time.
The advantage of the Pauling method was that it relied on, and encouraged the collection of, information about the chemical properties of DNA and its constituents. And if one were lucky, it just might turn out that relatively simple stereochemical considerations would place sufficient constraints that the structure would be uniquely determined.
So we would expect strategy A to appeal to scientists who are already trained in crystallography, who are not under severe time pressure (whether it be external or self-imposed), and who are either at a stage in their career where they would have a lot to lose by proposing a structure which turned out to be wrong or innately cautious.
Strategy B would be more suitable for scientists who enjoy speculating and have relatively little to lose if their model-building attempts do not bare fruit, who have access to good biochemical information, but are not experts in X-ray diffraction, and who are temperamentally ambitious or impatient or both.
In the decision-situation which I have described, there is no clearly superior strategy, so it is not surprising to find that both paths were followed. Sometimes accounts of the discovery of the DNA imply that because Watson and Crick were the first to discover the structure of DNA, there was something inherently superior about the way in which they did science - that there is a grand methodological moral here.
I would draw a different moral. Since evaluations of pursuit strategies are always just guesses, albeit informed guesses, science as a whole benefits by diversifying the search paths. And sometimes, as happened in the double-helix case, the alternate paths converge on the same result which certainly increases our confidence that it is correct. Thus Franklin and Gosling were immediately able to test the Crick-Watson structure against their data and show that evidence from diffraction studies supported it (p. 180). In this particular example, in retrospect there was no compelling scientific reason why a mixed strategy should not have been used all along. And as a matter of fact, Watson learned quite a bit about X-ray diffraction and Crick knew a lot of crystallography even though he disdained the field's traditional methodology. On the other hand, English crystallographers at this time were gradually learning to incorporate stereo-chemical considerations into their structures, partly after Bragg endorsed an erroneous structure for the poly peptide chain which allowed free rotation around conjugated double bonds (p. 86). (However, Bragg's embarrassment may also have had the effect of encouraging crystallographers to be more cautious!)