IX. Giere's Naturalistic Account of Satisfactory Modeling
Noretta Koertge
Following Kuhn, all philosophical accounts of the development of science have paid more or less close attention to actual episodes of scientific inquiry. As we have seen, Lakatos and his students launched an historiographic research programme as a means of testing his scientific methodology and both Laudan and Hull appeal to data from past or contemporary scientific practice. Giere also refers to case studies, but he supplements his evidential base with experimental results from cognitive psychology. On his interpretation, both of these sources suggest that scientists do not try to make assessments of the degree of verisimilitude of universal scientific theories or how likely they are to be true. Rather they make qualitative judgments about whether a particular theoretical model provides a satisfactory fit to a set of empirical data. Let us look at the major features of Giere's approach in more detail.
a. Naive Statistical Reasoning Patterns
Nineteenth century logicians such as Boole believed that logic embodied the laws of thought - or at least idealizations of them - but any number of cognitive studies in this century (starting with Bartlett in the 1930's) indicate otherwise. To cite just three examples:
(i) People often cannot correctly pick out what kinds of evidence is relevant to testing the truth of statements of the form: "If an envelope is sealed, then it has a stamp on the other side."
(ii) In certain contexts, people often do not realize that the probability of a conjunction, such as "Linda is a bank teller and a feminist" cannot be greater than the probability of either conjunct.
(iii) People frequently neglect base rates when making diagnostic inferences. Thus, if I can't tell whether the cab which hit me was green or yellow, subjects will conclude that there is a 50% chance it was yellow even when the problem explicitly says that 85% of the cabs in my town are yellow.
There is a pattern to the errors people make and although formal training in logic, probability and statistics reduces their frequency somewhat, they are remarkably firmly fixed. Cognitive psychologists have tried to codify some of these patterns. Tversky and Kahneman, for example, suggest people may attempt to solve many statistical problems by using a "representativeness heuristic" whereby probabilities are based on estimates of degrees of similarity instead of on relative frequencies.
Giere hypothesizes that in their informal inferences about the merits of scientific theories, scientists are more likely to follow the patterns exemplified by the subjects of these psychological experiments rather than the rules of correct reasoning found in logic or philosophy of science books. (Presumably, scientists do follow the more sophisticated inference rules when proving theorems or making actuarial calculations within their scientific disciplines.)
In particular, Giere rejects the Bayesian account of theory appraisal according to which the probability that a theory is true conditional on a piece of evidence depends on both the prior plausibility of the theory and the extent to which the theory explains the evidence compared to the explanatory power of rival theories. As we will see below, Giere believes that a simpler evaluative procedure is employed.
b. Satisficing Decision Rules
On the classical account of rational decision making, the agent should lay out all her options, give a complete description of the possible consequences of each action, assign each outcome a subjective value (its utility), access how probable each of the consequences is, and then choose that option which corresponds to the maximum expected utility.
Critics of this model of rationality, such as Herbert Simon, have argued that not only is it impractical, but also that setting it up as a regulative ideal can lead to bad consequences. It is generally hopeless to try to do a complete search for the best option. We cannot foresee all of the possible unintended consequences of our actions and we generally have no way of assigning probabilities to them. (And as we have seen in the above section, even when probabilities are given, people have trouble reasoning with them.)
So, Simon proposes instead a model of "bounded rationality": When trying to make a decision, we should consider actions serially and stop our search when we happen upon one which is "good enough". What level of utility is considered to be satisfactory depends on the context. Simon's so-called "satisficing" model seems particularly useful for games such as chess where it is obviously impossible to pursue an optimizing strategy for every more. As we will see below, Giere does not adopt all of the details of Simon's model, but he does follow his emphasis on bounded rationality.
c. Theories as Cognitive Models
There is a tension in the history of philosophy between those who describe the growth of knowledge mainly in terms of the
refinement of concepts and those who tend to talk about propositions or theories. People who talk a lot about concepts are generally more interested in analyzing their meaning than ascertaining exactly where they apply. Talk about theories, on the other hand, tends to generate questions about generality of the claims and whether they are true or false.
But the surface distinction (marked by the difference between single words and complete sentences) is blurred when we speak of "conceptual schemes" or explaining phenomena by "subsuming them under a concept" or showing they fit into a pattern. Hempel persuasively argues that the process of diagnosing a disease, to call Johnny's red spots a case of measles, is also to activate many general laws about what caused the spots, how long they will persist, what concomitant symptoms are to be expected, etc. So our concept of measles somehow incorporates a theory of disease. And, of course, theories are expressed using concepts. So what is really at issue here?
Recently the debate between the "conceptualists" and the "propositionalists" has taken what to my mind is a bizarre turn, and as a consequence, the lines have been more clearly drawn. Here is a brief summary of these developments. As we saw in Chapter I., the logical positivists wished to present scientific theories as sets of axioms, in which the meanings of any theoretical terms would be given via correspondence rules linking them to observables. But this approach to meaning ran into all sorts of problems, especially for realists who did not wish to claim that electrons were nothing but a logical construction from clicks in Geiger counters and tracks in bubble chambers.
In an attempt to side-step the problem of meaning, various philosophers (e.g., Suppe and van Fraassen) proposed that scientific theories be viewed as complex formal models which map onto physical systems. The model may contain generalizations about how the system will behave over time (which may be called laws), but the model itself will not make any claims about which systems it applies to. Thus, if we construe scientific theories as models, like concepts they cannot be true or false, but only applicable or inapplicable.
Let me illustrate the approach with a very simple example. Consider Kepler's discovery of what are traditionally called the Laws of Planetary Motion. If we were to cast this discovery in propositional form, it would read something like this:
"All planets travel in ellipses around the sun, such that Rv is a constant and T2/R3 is a constant" (where R is the distance from the sun to the planet, R is the mean radius, v is the velocity, and T is the period).
In order for this law to be falsifiable, one would have to specify what counts as a planet in a manner which does not depend on the values of R, v, and T. One could then look at the orbits of Saturn, Jupiter, etc. to evaluate the claim's truth, falsity or degree of verisimilitude.
On the model-theoretic approach one might define the concept of a Keplerian system as follows:
"A Keplerian system consists of planets moving in ellipses around the sun such that Rv is a constant and T2/R3 is also a constant."
One would then determine whether this model actually fits the system consisting of our sun, Saturn, Jupiter, etc.
So far, it doesn't make much difference whether we cast Kepler's laws in propositional or conceptual form. But now consider the following two developments, one hypothetical, one quasi-historical:
(i) Suppose Kepler had found that these equations did not come close to describing the motion of Mercury, although they worked well for all the other planets. (Recall his battle with Mars which led to his disenchantment with the Ptolemaic approach.) Then on the propositional approach Kepler would have had to say that his laws of planetary motion were refuted and he would either have had to revise them or else come up with a non-ad-hoc reason for considering Mercury not to be a planet.
Kepler's options on the conceptual construal of theories are different. Here all he would have to say is that the Keplerian model did not apply to the sun-Mercury system. He could then introduce a second model which did describe Mercury's path or he could go back to the drawing board and search for a single new model which fit all the planets.
(ii) After Galileo discovered the moons of Jupiter, the question arose as to whether their motions would exhibit the same regularities as did the sun's planets. To answer this question on the propositional approach, one needs to look carefully at the characteristic properties of planets (such as their mass and positions relative to the sun), if necessary rewrite the antecedent of the law to make these explicit, and then since the satellites of Jupiter exhibit these characteristics, one would predict the laws should hold for other systems where small masses orbit around a larger central mass. Thus, as Newton noted in the Principia evidence about their orbits corroborates the Newtonian extension of Kepler's Laws.
On the model-theoretic approach, on the other hand, one notes similarities between the traditional planets and the new satellites and so one hopes the Keplerian model will apply, but there is no process of logical prediction. It is conceptually economical to be able to use the same model for more than one physical system, but this happy circumstance is not to be expected in general.
The debate between the propositionalists and conceptualists is generally taken to be about which is the most "natural" way to represent real live scientific theories. For example, the model-theoretic approach has been particularly popular amongst philosophers of biology, who talk about the adaptationist models and the randomdrift model as each describing certain cases of evolution. Applied scientists such as psychotherapists, often adopt an eclectic approach, using Freudian or Maslowian or Skinnerian models in an opportunistic fashion without attempting to find a unified theory or to specify ahead of time the circumstances in which each particular approach should apply. However, there has been very little attention paid to the methodological correlates of each view or the issues of how we are to re-interpret explanation and prediction on the model-theoretic account.
Giere believes the model-theoretic approach also gives the best account of physics, although some of his evidence (such as the case of the harmonic oscillator) are surely examples of models in quite a different sense, viz. descriptions of initial conditions. It is, of course, the improvement in models of boundary and initial conditions (e.g. should we represent the earth as an oblate sphere instead of a mass point?), that Lakatos was talking about in his account of progressive research programs. Giere also claims that the model-theoretic approach allows us to give a better account of imperfect theories. He cites Popper's problems in clarifying the notion of degrees of verisimilitude or approximation to the truth, apparently concluding that the notions of degrees of isomorphism or approximate fit of model to the world are clearer! In any case, Giere will argue, following Simon's satisficing approach, that all scientists really need to decide is whether the fit of the theoretical-model is "good enough" and what is satisfactory will depend on the context. We will analyze exactly how the context enters into the decision procedure later. Let us now at last look at Giere's own philosophical model of scientific appraisal.
d. Giere's Decision Rules
Giere's central model of scientific decision-making applies to situations of the following structure: Two rival theoretic models have been proposed to fit a single physical system. How should (and do) scientists choose between them? As examples, Giere discusses the conflict between Schroedinger and Dirac models of nuclear potentials and the debate between advocates of continental drift and "stabilists".
Giere sets up the problem following standard decision theory. The scientist has two options - choose model A or choose model B. (Giere does not dwell on the distinction between pursuit and acceptance.) He then considers the following two possibilities: either Model A fits best [sic] or Model B fits best [sic]. (He does not include the possibility that neither model fits well.) This leads to a simple 2 X 2 decision matrix with 4 outcomes:
Figure 1: Skeleton Matrix
Because he is following Simon's satisficing approach, Giere does not need to assign probabilities or cardinal utilities to the outcomes. All that we need do is classify them as satisfactory or unsatisfactory. Although recognizing that scientists may have strong ideological or professional interests which lead them to favor one model over the other, he adds a requirement of "minimal open-mindedness" which generates the following assignments:
Fig. 2: Matrix of "Minimally Open-Minded" Scientist
So even the most partisan advocate of A will concede that the option of choosing A would be unsatisfactory should it transpire that B is the better model.
However, according to a pure satisficing approach (which does not try to assign even rough probabilities to the outcomes), neither option in the above matrix is satisfactory in all of its possible outcomes. But the impasse can be broken, Giere says, if certain kinds of experimental evidence becomes available.
What Giere now proceeds to do is to recommend the sort of experimental test which is traditionally called a "crucial experiment" and he even gives the sort of probability specifications which Popper used in his account of severe testing and which Bayesians call "likelihoods". However, Giere tries to present them as a natural extension of the satisficing approach. We will decide later how satisfactory his philosophical gloss is, but first his description of the ideal experiment to discriminate between the rival models.
Set up an experiment such that:
(i) if Model A is a good fit, then the results RB will be highly probable;
(ii) if Model B is a good fit, then the results RB will be highly probable;
(iii) if Model A is a good fit, then the results RB will be highly improbable;
(iv) if Model B is a good fit, then the results RA will be highly improbable.
Once such an experiment is carried out, Giere claims the scientist is no longer faced with an inconclusive decision matrix. If the experimental result is RA, we can assess the outcomes as follows:
Fig. 3: Matrix, given RA
Now Giere argues, option A gives us such a high probability of a satisfactory outcome that the choice of Model A becomes satisfactory. A parallel argument is obtained if RB is the experimental result. So after such an experiment is conducted, the previous stale-mate is resolved and the scientist no longer need suspend judgment.
e. Giere's Meta-position
In keeping with the naturalistic approach to philosophy of science, Giere believes his own account of science should be appraised in exactly the same way that more traditional scientific theories are evaluated. So, following in the tradition of Kuhn, Lakatos and Laudan, Giere refers to historical case-studies as evidence for his account. However, since he espouses the model-theoretic conception of theories, he does not need to argue that his satisficing model applies everywhere - all he needs to do is show that it fits some important instances, and Giere does indeed produce two very interesting case studies, one of which relies on his own interviews with participating scientists.
However, Giere departs from a consistent reflexive stance in two ways. First, a minor departure: Early on in the book, he says that his aim is to provide a unified theory of scientific decision making. In the event, all he offers is a single model of a special case of scientific reasoning, the choice between two models, given an unambiguous crucial experiment. There are, of course, many other aspects of scientific reasoning, such as choice of problem, the use of heuristics, resolution of inconsistencies within or between fields, ethical and technological decision making, and as we will see below, there is reason to doubt that Giere's model as it stands can be extended to cover all of these cases.
The second discrepancy is more serious. In providing a single philosophical model of scientific decision making, Giere wisely chooses the case of a crucial experiment. As Popper emphasized, it is relatively easy to find confirmations for a theory. (Or to put a similar point in model-theoretic language, given an abstract model, it is relatively easy to find some physical system which fits it approximately.) But what is not easy to do is to find a theory and design an experiment such that our theory correctly predicts the actual result, but all extant rival theories fail. It is this kind of severe testing which makes science a critical enterprise.
What if we extend this methodology to the meta-level? Then, on his own account, what Giere should have searched for is a case where his model gives predictions which fit, but rival philosophical models give predictions which fail - a philosophical crucial experiment. But this he has failed to do. To see why, let us briefly compare Giere's model with a standard Bayesian analysis of decision-making.
f. A Bayesian Analysis of Crucial Experiments
Let us quickly "translate" Giere's requirements for crucial experiments (i) - (iv) above into standard probability notation. Thus the fact that RA is highly probable if Model A fits best becomes:
(i) p(RA,A) = hi and the remaining requirements become: (ii) p(RB,B) = hi
(iii) p(RB,A) = lo
(iv) p(RA,B) = lo
Let us now write down the probabilities which Giere incorporates into the satisficing matrix. Suppose the actual experimental result is RA. Then, says Giere, the scientist should conclude that it is highly probable that the world is such that Model A fits best, viz.
p(A,RA) = hi.
Likewise for the other cells of the decision matrix.
But the inference which Giere describes, from p(RA,A) = hi to p(A,RA) = hi, is rather like arguing from "most rapists are men" to it is very likely to be true that "most men are rapists". The inductive conclusion does not follow, unless one adds some strong assumptions about prior probabilities. And pace Giere, there is nothing in the remaining three requirements which sanctions this inductive inference.
It is easy to see this if we write out Bayes' theorem for this case and insert the information contained in the four requirements.
The value which we need to insert in the first cell of the decision matrix can be computed as follows:
(l) p(A,RA) = p(A) x p(RA,A)
p(A) x p(RA,A) + p(B) x p(RA,B)
Inserting the values specified by the requirements we get:
(2) p(A,RA) = p(A) x hi
p(A) x hi + p(B) x lo
And it's easy to see that the value of the posterior probability, p(A,RA), depends on the relative values of the prior probabilities, p(A) and p(B). [To pursue the simpler parallel case, "most B's are A's" does follow from "most A's are B's" if the classes of A's and B's are similar in size.]
Let us now look at two special cases of equation (2). First, what happens if we "ignore priors" as do many of the subjects in Tversky and Kahneman's experiments? This is tantamount to assuming a "flat distribution" of priors, namely that p(A) = p(B) = 0.5. In this case, (2) becomes:
(3) p(A,RA) = .5 x hi >> .5
.5 x hi + .5 x lo
In this case, Giere's model and the Bayesian model coincide.
Now let us consider the case where Model A correctly predicts the result RA and Model B does not, but there are prior reasons for thinking it unlikely that Model A provides the best fit. If p(A) is lo, while p(B) is hi, equation (2) becomes:
(4) p(A,RA) = lo x hi ~ .5
lo x hi + hi x lo
In such a case, Giere's model which takes no cognizance of priors tells scientists to conclude that it is highly probable that Model A is the best fit while the Bayesian model recommends that they suspend judgment.
So if we are to apply Giere's own account reflexively, we should look for a historical episode corresponding to equation (4) above. Many of Giere's examples would appear to involve fairly flat priors (he, of course, does not analyze priors) and hence do not discriminate between his model and the Bayesian approach because both assign large posterior probabilities. However, there is one case described by Giere that does fit equation (4). This is the situation regarding the continental drift hypothesis (Model A) early on before any spreading mechanism had been proposed. According to Giere, at this point the evidence (RA) regarding the complementary shapes of the continents plus the wandering positions of the magnetic poles is very probable on the continental drift hypothesis (A) but very improbable on a stabilist model (B). Why then did scientists not follow Giere's satisficing rule and choose the drift hypothesis? Well, because there were other geophysical reasons for considering it to be implausible. Giere quotes a scientist's conclusion after an analysis of the forces that would be required: "the puny secular stresses we have discussed appear to be wholly inadequate for the task." (Giere, p. 236) A Bayesian would interpret this phrase as revealing that the scientist believed p(A) to be very low and predict that the scientist would assign the continental drift hypothesis a low posterior probability despite its amazing ability to account for the evidence summarized in RA.
Giere, however, interprets the scientist as saying that "large-scale drift is physically impossible" [his italics] (p. 236) and concludes that half of the decision matrix should be "simply eliminated" (p. 236) even by a minimally open-minded scientist! He presents the following Pickwickian argument. If Model A is physically impossible, then the outcome "Model A fits best" must be eliminated, hence to choose A could not lead to a satisfactory outcome for anyone who is open-minded enough to want their model to fit the world. A Bayesian, of course, would argue that a minimally open-minded scientist should never assign the value zero to a prior probability of a hypothesis because this makes it impossible ever to raise the probability of such a hypothesis regardless of how many subsequent arguments are put forward in its favor.
This is not the occasion to do a detailed appraisal of the merits and demerits of Bayesianism. Suffice it to say, once Giere has deviated from strict Simonian satisficing by introducing probabilities, he invites a comparison with a Bayesian approach which requires that probability assignments obey theorems of the probability calculus. Giere, however, would probably reply that cognitive science experiments show that college sophomores, graduates in business school, and medical interns routinely and systematically violate the laws of logic and probability, so why shouldn't a naturalistic account of science do likewise? Let us now look in more detail at the arguments for a naturalistic philosophy of science.
g. A Plea for a Sophisticated Naturalism
In a lovely monograph called Minimal Rationality, Cherniak produces an elegant and sustained argument which could be crudely summarized as follows: It's utopian (in the pejorative sense) to define rational agent in a way which requires them to carry out mental processes too complex to be performed by a computer as big as the universe. "Suppose that each line of the truth table...could be checked in the time it takes a light ray to traverse the diameter of a proton...a belief system containing only 138 logically independent propositions would overwhelm the time resources of this super-machine." (p. 93) I am persuaded that we want any theory of scientific method to take into account the finite cognitive resources of human beings.
I also believe that recent studies of human inference studies, such as those cited by Giere are extremely valuable, not just to instructors of critical thinking courses who need to know students' intuitive reasoning patterns in order to correct them, but also to historians and philosophers of science. However, we must be careful how we use this data. Here's a simple example of the misuse of psychological studies. A few years ago it was fashionable to argue for a strong theory-and-value-ladeness-of-observation thesis by citing some striking psychological experiments. Kuhn describes people's inability to detect reversed-colors on a deck of playing cards and there was another popular experiment which seemed to indicate that poor kids overestimated the diameters of fifty cent pieces. (I believe this latter experiment has since been discredited, but that is not my point.) Such studies were used to argue that scientists tend to see what they expect to see and what they want to see. True enough - granted as soon as asked.
But what the above story leaves out is the sustained and systematic attempts which scientists make to protect themselves against this pervasive human tendency. The whole methodology of placebos and double-blind experiments was explicitly designed for just this reason. And the almost obsessive desire of scientists to measure things with instruments instead of relying on human judgment is in part a response to the same phenomenon. The psychological studies help highlight not only the differences between (some) casual observations and (some) scientific observations, but they also help explain why scientific methodology has developed in the way it has.
I suggest that results such as those of Tversky and Kahneman should also not be taken over wholesale as a model of scientific reasoning. True, their subjects use quick and dirty heuristic methods to solve problems in logic, probability and statistics, and it is valuable to know what these patterns are. But what needs to be added is that at least some of these psychological subjects are embarrassed when they discover the mistakes which their quick and dirty methods have produced. For example, consider a test question which Giere cites:
"If a test to detect a disease whose prevalence is l/1,000 has a false positive rate of 5 percent, what is the chance that a person found to have a positive result actually has the disease?" (p. 153)
Giere concludes from the fact that only 11 of the 60 subjects at Harvard Medical School gave the correct answer (which is 2% - half of them guessed over 50%), that physicians aren't Bayesians. True, their casual responses weren't Bayesian. However, physicians as well as other scientists, have taken the leadership in arguing that the AIDS testing program (as well as various urine tests for drugs) is going to produce an awful lot of anguish from false positives if it is used on a population in which the incidence of the condition is low. Upon reflection, everyone agrees that base rates (prior probabilities) must not be neglected in such circumstances.
So a sophisticated, more complete, naturalism would describe not only the casual responses of subjects on questionnaires, but also their reflective responses after they have had a chance to critically discuss the issue with colleagues (and even professional statisticians!).
[It is sad to note that on Giere's own decision rule (which neglects prior), scientists are enjoined to conclude that the disease model is a satisfactory fit for every person who tests positive! Here is now the analysis would go.
Let A be the disease model and B be a model of a person who doesn't have the disease. Let RA be positive test results and RB be negative test results.
Now the diagnostic tests satisfies Giere's conditions:
(i) prob (RA,A) = hi
(ii) prob (RA,B) = lo, etc.
So according to his decision rule, given the evidence of a positive test result the scientist should choose the disease model as the best fit.]