|
September 29, 2003
Elliott Shore (Information Services)
"Conforming to the Count: How We Decide to do the Things we Do"
Summary
Prepared by Anne Dalke
Additions, revisions, extensions are encouraged in the Forum
Participants
Elliott invited us to join him in a conversation about something he "didn't understand and would like to think about together": two themes which he described as different ways of "conforming to the count": - following the leader, where the leader matters (that is: where he "counts," either in terms of "having data, or by "counting" himself, in terms of his importance). Elliott offered here two examples from his life as an administrator: the query in a library meeting about how many of the participants use Blackboard (after which those who didn't went out and bought it; that is, they made a decision based on what most of the people in the room were doing); and the U.S. News and World Report rankings (about which Elena Bernal of BMC's Institutional Research Office will lead a Brown Bag session on October 27): we collect data in accord with how the rankings count it, and make them "matter because we think they do."
- because we can count it, it matters. Elliott's first anecdote in this category was drawn from the Princeton Institute for Advanced Study, where the molecular biology department devised a strategy, later adopted by the entire university, for measuring how much time each department spent doing work for others, and then charged them for same. His second example was grading: how do we devise schemes of counting in our courses, in order to produce grades in them?
Elliott then asked participants in the group to give examples of our own experiences both of "following the leader" (or the crowd) in deciding what to value, based on what others counted; and of thinking that what can be counted is indeed what matters. (In the course of constructing the lists which followed, the distinctions between these two categories became somewhat blurred.)
examples of counting what "the leader" or "the crowd" counts:
- "the budweiser effect": p.c. vs. mac, beta vs. vhs (tape mediums)
- fitting onself into presciptions based on what measures funding agencies will recognize
- the buzz word "assessment": quantifying all services offered (and so, perversely, "justifying" them, using numbers)
- finding a common font (such as a Greek, on the computer): developing uni-codes in order to communicate with a broader range of scholars
- asking what our peer institutions are doing, and using that information to establish the standard against which we measure ourselves
- the pressure to pass along students, as short-term insurance for a department
- trying to raise our retention numbers (by helping students be happy and learn, or by "cutting them more slack"?)
- exit polling on the east coast when the west is not done voting
- I.Q.
- medical treatment and definitions of health dependent on "normal values"
- quantification after the fact: where we live and what venues we select for our children's education, based on choices made by the people "who matter to us"
- qualifications of doctors and hospitals, based on published statistics about (for instance) hospital death rates (perhaps better hospitals treat more serious conditions? offer more heroic--more risky, less predictably successful--treatment?)
- ranking the pain (of a hip, broken leg, childbirth) on a scale of 1-10
|
examples of valuing something because it can be counted:
- basing tenure on peer-reviewed publications
- the clinical syndrome of ostopenia, defined (because now measurable) as bone density one standard deviation below the mean
- course enrollment numbers (the more students enrolled, the better?)
- counting/making lists of one's own publications and communicating them to others (in public forums such as the Annual Report)
- GPAs
- tracking systems
- demographic social history (births, deaths, numbers of illicit children) became so narrow it disappeared as a discipline: its conclusions were so dependent on numbers that all cultural deviations were ignored
- amount of money spent by the library on each student (which neglects to tell an important part of the story)
- Gross National Product as an economic indicator (invented in the 1920's-'30's) to describe how well off we are
- ethnic categories in the census, which change every few years (in the most recent census, people self-selected as many categories as they wanted)
- grading: how differentiate cogent, compelling students from the direction-followers? how to identify the real achievers by means of what is quantifiable?
- how we count each other as faculty, based on our "inflated" grading scales
- outcomes assessments
- time limited tests, as useful predictors of the ability to read Greek texts
- analysis of variants in ANOVA: measuring the heritability of genes
- heritability of intelligence, measured by a variance from a norm
- consumer confidence survey: the purchasing managers' index of "happiness," on which stock market moves are hedged
- how badly the social science/science index is made (with statistics skewed by authors, for instance, citing themselves in their own articles)
|
During the discussion generated by our list-making, it was observed that, in all of these schemes, not everyone can be good; in each of them, someone has to fail. Success needs redefining! Many of our choices of behavior are based on unconscious assumptions, which we were here making conscious. What is counted first usually becomes the measure, the benchmark, the normative value for defining where you fall in the count; that "pioneer" standard" is often not re-evaluated. There are sometimes rational reasons for "following the crowd," for communicating and drawing on shared data: we need not invent the wheel in every decision we make. We also assume that there is safety in numbers, that we can't be faulted if we follow the crowd. But who judges the relative value or risk of any given act? The "budweiser effect" evolved because it disenfranchized the least number of people: because the brew was bland, not remarkable or particularly distinguishable, most people would drink it. How entrenched are these systems of valuation, and how embedded, in what we count? If any convergence represents a diminishment of novelty, some will respond by choosing to do something different (and by so doing may define, for instance, a new advertising niche). We considered the backlash against basing evaluations on large numbers--in which "doing it differently" becomes just a subset of "doing it the same." You can actually gain market share by doing something which doesn't fit the norm, and certify change by getting credibility by (for instance) hiring a consulting firm whose numbers can demonstrate the reasonableness of such actions. Some will always seek security and connectedness; others are interested in novelty and difference. But whatever our preferences, we posited that value should and CANnot be generated by the count. Wherefrom, then, does value arise? Is the sphere of value always the same as the sphere of risk?
Our discussion of the relationship between value and quantification will continue next Monday @ noon, in a presentation by Paula Viterbo, a post-doctoral fellow in History of Science at Bryn Mawr.
Return to Brown Bag Home Page
|
