Serendip is an independent site partnering with faculty at multiple colleges and universities around the world. Happy exploring!

Assessing Assessment

Howard Hoffman, On Life: "We tend to see only what we are prepared to comprehend."

In Spring 2011, six faculty members at Bryn Mawr College (Jody Cohen, Anne Dalke, Wil Franklin, and Alice Lesnick, Sara Nath and J.C. Todd) began meeting under the aegis of the Teaching and Learning Initiative to discuss course-based assessment, with a particular focus on modes of student self-assessment. Our conversation will again be supported in Spring 2012. We have created this page as an archive of our discussions and of resources, as well as a site for further discussion, to which everyone with an interest in these questions is warmly invited, in the forum area below and on the individual pages.

Our Motivating Questions

Our particular interest is in the question of self-assessment and its relation to broader conversations about assessment at the College and beyond.

Our First Semester

We are reading and discussing a variety of texts, reframing the conversation about self-assessment as part of the process of change and innovation in education, and interweaving those conversations with our own teaching experiences.

Future Discussions

Our seminar will continue over the next semester, with hopes of bringing our learning to the community at the College.

Resources

A compilation of resources for assessing assessment

Comments

bullet points

Submitted by Anne Dalke on Mon, 02/27/2012 - 1:24pm.

I just want to record here a few bullet points, to highlight important moments (for me)
in some of the rich conversations we've been having lately:
these all involve questions about
* anonymity in student evaluations: intended to encourage frank, open responses, they can also invite a mean-spirited, irresponsible sort of nastiness
* audience in student evaluations: how different it would be if these were framed as letters to us, real individuals who have been in relationship w/ our students for a semester, wanting feedback on what worked and didn't--rather than (as they are now) queries about whether "the instructors" have met their obligations (or not); what might happen if the audience were re-framed from "them" (dept, appointments committee, etc.) to "me" (the teacher who has been working w/ the students)
* temporality: how does the timing of evaluations (which are filled out before the term--and its major work--has been finished; @ a time when students are feeling particularly pushed and harried) affect the sort of reports they elicit?
* reading the evaluations through the lens provided by the course (for example, if a student who has taken a course about class issues evaluates the professor as "elitist," it could be said that she is actually applying the framework of the course to critique activities taking place w/ in…this opens up a very useful way of understanding her evaluation, as more than a put-down of the instructor)
* reading the evaluations collectively, with one's "posse," rather than alone, as an individual subject to criticism and judgment, gives a heartening sense of the larger shared project of education.

The Art of Distraction

Submitted by Anne Dalke on Mon, 02/20/2012 - 8:34pm.

Hanif Kureishi, in The New York Times Sunday Review (February 19, 2012):
"the attempted standardization of a human being and of a notion of achievement ... is limiting, prescriptive and bullying .... A flighty mind might be going somewhere .... From this point of view — that of drift and dream; of looking out for interest; of following this or that because it seems alive — Ritalin and other forms of enforcement and psychological policing are the contemporary equivalent of the old practice of tying up children’s hands in bed, so they won’t touch their genitals ....

As we as a society become ... more regulated and conformist, our ideals of competence become more misleading and cruel, making people feel like losers. There might be more to our distractions than we realized we knew. We might need to be irresponsible. But to follow a distraction requires independence and disobedience; there will be anxiety in not completing something, in looking away, or in not looking where others prefer you to. This may be why most art is either collaborative ... or is made by individual artists supporting one another in various forms of loose arrangement, where people might find the solidarity and backing they need.

This article spoke strongly to me (and strongly, I think, to a number of the issues raised in Margaret Price's book). Plus I loved the illustrations!

bullying dreaming

Submitted by alesnick on Tue, 02/21/2012 - 7:26am.

Thanks, Anne, for posting this. It speaks strongly to me, as well. I wonder if it helps make sense of the current cultural obsession with bullying in schools -- as if somehow we are aware that something is being bullied, so we warn kids not to tolerate the bully and not to be the bully, but really the bully is this awful paradigm of unrelenting competence and achievement (with terrific anxiety as the fine or the punishment for transgression) that's the bully.

I also think Kureishi's text here speaks to working for social justice. We were talking in my senior sem last night with a panel of alumni about the importance of a (very) longterm view of one's impact. The kind of self-trust Kureishi evokes is a necessary part of this.

Badges for Lifelong Learning!

Submitted by Anne Dalke on Tue, 02/21/2012 - 6:39pm.

So who knew (not me, that's for sure!)....
that one fairly recent innovation in self-directed assessment are self-constructed, peer-evaluated Badges for Lifelong Learning, open-source creations that attempt to de-hierarchialize notions of achievement -- but that have also been around long enough to generate the critique that such systems, if standardized, promote normative models of "success" and "achievement," rather than supporting personal growth and transformation (and so, in Alice's inimitable words, continue to "bully," in support of "this awful paradigm of unrelenting competence and achievement").

treasure of sierra madre?

Submitted by alesnick on Tue, 02/21/2012 - 9:34pm.

isn't that the movie that ends with the statement about not needing any stinking badges? i have heard of this, can't remember where (somewhere online) and i do get the critique -- i think we need a stronger counter to hierarchy than kinder, gentler certifications. i actually think we need new minds (in kevin kelly's terms -- in a recent book called What Technology Wants) capable of new ways of thinking to explore the universe . . . including how in the world to deconstruct and work outside of hierarchies! just saying.

Lord of the Flies

Submitted by Wil Franklin on Wed, 02/22/2012 - 11:43am.

Is this the same human condition played out in Lord of the Flies or reality TV like Survivor? If I am reading this correctly, de-centralized, community-based assessment can be just as stifling as top-down, authoritative assessment. Is that always the case? How does the diversity or composition of the community affect the assessment structures that emerge? Is it inevitable that all goal oriented work succumbs to this predictable fate? Is there a tipping point level of diversity at which community-based assessment takes a more liberating form? Is there a goal that is immune to this fate? And if, as I suspect (based on Bayesian computational models Paul and I worked on) the tipping point must be extremely lopsided towards diversity, then how does that inform the community values we try to set up in our classrooms? If we desire unfettered possibilities for our students then is there any left to do besides deconstructing power structures and celebrating diversity?

"The Burkean parlor!"

Submitted by Anne Dalke on Sat, 02/11/2012 - 10:22am.

I promised Wil a description of "the Burkean parlor." Here 'tis, and --still mulling over Margaret's visit--I'm thinking now, if what we are really doing is nurturing this "unending conversation," what is the role of assessment here?

Imagine that you enter a parlor. You come late. When you arrive, others have long preceded you, and they are engaged in a heated discussion, a discussion too heated for them to pause and tell you exactly what it is about. In fact, the discussion had already begun long before any of them got there, so that no one present is qualified to retrace for you all the steps that had gone before. You listen for a while, until you decide that you have caught the tenor of the argument; then you put in your oar. Someone answers; you answer him; another comes to your defense; another aligns himself against you, to either the embarrassment or gratification of your opponent, depending upon the quality of your ally's assistance. However, the discussion is interminable. The hour grows late, you must depart. And you do depart, with the discussion still vigorously in progress.

Life as a Burkean parlor

Submitted by Wil Franklin on Mon, 02/13/2012 - 11:56am.

Wow! My mind instantly went to my 2 year old son trying to figure out the “tenor of the argument” called life. And as he rips apart everything he gets his hands on or tosses plates and silverware off the dinner table, I recommit myself to “educating” (not disciplining) him. He needs to be educated into the “tenor of the argument”. At two years old, he is finally able to understand the conversation (both literally and figuratively in the sense of the culture he needs to learn). If life is a Burkean parlor discussion, which in many interesting ways it is, then all of us are thrown into an unknown and unintelligible parlor conversation. A classroom is a microcosm of life or the Burkean parlor discussion, of course. Teachers set the tenor to greater or lesser extents and they choose which aspects of the conversation to value. One might value students that “put their oar in” or one might value the fidelity to which the new viewpoint fits with the tenor. In this sense, I can understand Price’s use of criterion/rubrics/standards. They make the “tenor of the argument” more transparent. But, what if we value “putting ones oar in”? What if I am concerned less about the faithfulness of the argument and more about the generativity of the argument?

Perhaps, I am finally seeing a loop between faithfulness to the tenor and generativity. I really value generativity, but at the same time tweets/facebook posts and even some blogs are not necessarily generative. Not all utterances are of value. To be generative, some link and connection to the tenor of the argument must be made – to be able to transport the others in the discussion across the bridge to the new understandings and new viewpoint.

So what is the tenor of my course? To the extent that I want to make assessment meaningful, not absurd, I need to be clear to my students my objectives, how they are open or not to their own objectives and what I value in the work of the class. An introductory biology lab has a very distinct tenor, even if I insist that students own it. Is it enough to own it? Doesn’t the Burkean parlor discussion called biology have certain constraints? I dislike constraints. Could I choose to focus on the fact that as a student “puts their oar in” it changes the tenor of the discussion?

I guess I am still ruminating on David Brooks’ “How to fight the Man”, and the failings of what he called “rebellion without a rigorous alternative”. I think the strain between valuing “faithfulness to the tenor” and “putting ones oars in” is equivalent to Brooks’ “Most professors would like their students to be more rebellious and argumentative. But rebellion without a rigorous alternative vision is just a feeble spasm.” I want my students to be rebellious, but not feeble. Can I assess them on that? Is that too much to ask? That is not a rhetorical question. Perhaps, not everyone “desires” to be rebellious? For many, just figuring out the tenor is hard enough.

structure!

Submitted by Anne Dalke on Fri, 02/10/2012 - 8:30am.

I attended Margaret Price's talk @ Haverford y'day afternoon: "Ways to Move: Mental Disability and the Kairotic Space of the Classroom," which included a number of revelations for me. The first was her beginning with a query about--and invitation to increased--access: "if you prefer to apprehend this visually rather than orally, here are copies of my script; I'm offering a visual channel…also, do what you need to take care of your own bodyminds, while I'm talking; if you need to attend to something else (go to the bathroom, check a message, lie on the floor), then please do it. Lecture spaces not terribly accessible; let's open up the possibility that we can make them more so...." So that was the first revelation: “True access isn't accommodations added on to a classroom environment, but built into its infrastructure, part of its foundation."

I liked this alot, in part because it offered such a clear foil to my own preferences (and practices!): when I'm leading a class discussion, I want everyone focused on our shared exploration. Checking phones, bathroom breaks, any loss of attention to MY agenda (which I'm hoping, of course, will become--and trying to make into--our shared agenda) is an irritant to me.

I realize of course that inviting each learner to attend to her own needs runs a very real risk that we will all disable one another; and I realize further that there needs to be some space for pushing back (this became a topic during the Q&A): making class more flexible and responsive to individual needs (of a student who says, for instance, “I prefer to listen, to process info, rather than contribute to the conversation," etc.….) creates a tension between respecting claims re: what works for her and what might work better for the group as a whole. Where's the room, here, to push back, to say, "I don’t think you are learning best that way….?" And what about practices that seem to us to be inimical to learning?

Anyway, for me the bottom line here was the caution not to try and resolve such points of tension, but to linger on them, open them for exploration. This was related to another good question Margaret asked us: "What would academia look like if we built in more interstices, more time when 'nothing' happened?"

My other revelation came when Margaret described her topic, "kairotic spaces," as “spaces that scare me a lot.” These are spaces in which there is a
*real-time unfolding of events
*impromptu communication (required or encouraged)
*contact (in-person or tele-present)
*a strong social element
*high stakes

Key here is the pairing of spontaneity w/ high levels of professional/academic impact--a combination which, she argued, "intersects problematically w/ mental disability.... I struggle a lot w/ the spaces of academia, especially spontaneous ones…when expectations are less clear, I find it very difficult…I don’t know how to perform," she said.

So the revelation for me was that so many of my own pedagogical innovations--which tend generally in the direction of less structure, more fluidity--might be particularly disabling for brains that operate differently than mine, which actually find not just more comfort, but more freedom to do/be, within structures of clear expectation.

Finally, I thought that, perhaps, the medium was the message here, that some of our own difficulties with talking w/ Margaret were being described/analyzed/explained in this talk she gave after.

"every tub must sit on its own bottom"

Submitted by Anne Dalke on Wed, 01/25/2012 - 9:16pm.

All the work that Kathleen Fitzpatrick’s Planned Obsolescence does in re-thinking "peer review" (to peer-to-peer review) seems to me quite relevant to our explorations. For a taste:

"We cling to a profound individualism when thinking about scholarly productivity…no matter how much we might claim to privilege collaboration, the multi-author dissertation remains literally unthinkable; when it comes to assessment, every tub, as it were, must sit on its own bottom."

thinking about mental health

Submitted by Anne Dalke on Sat, 01/21/2012 - 12:43pm.

here was my draft plan:

1) I send all faculty notices of Price's visit and public talk @ 4:15 on Thursday, Feb. 9:
"Ways to Move: Disability and the Kairotic Space of the Classroom."

2) Over the weekend, I send a follow-up note including the passage, below,
and invite faculty members to write back their answers to these questions:

Are there certain behaviors (perhaps not included in the list below) which your classes cannot accommodate,
which do not “fit” with the ways you organize learning experiences for your students?
How do you respond when a student exhibits such behavior? What are your bottom lines?

On p. 5 of her Introduction to "Mad @ School," Margaret Price suggests "that some of the most important common topoi of academe intersect problematically with mental disability. These include
*Rationality
*Criticality
*Presence
*Participation
*Resistance
*Productivity
*Collegiality
*Security
*Coherence
*Truth
*Independence

For instance, what does "participation" in a class mean for a student who is undergoing a deep depression and cannot get out of bed? Or a student who experiences such severe anxiety, or obsession, that he can barely leave his dorm room or home? What about a student on the autism spectrum who has difficulty apprehending the subtle social cues that govern classroom participation, the difference between "showing engagement" and "dominating the conversation," the sorts of spontaneous oral performances that are considered "smart"? What does "collegiality" mean for a faculty member who has these same difficulties? What happens to the "productivity" of an academic writer who struggles to achieve the linear coherence that most academic writing demands? Or whose disability affects the many self-directed stages of writing and revising--initiation, organization, seeking and applying feedback, completion? Why, indeed, is "coherence' one of the most-often emphasized features of a thesis-driven academic argument; does the demonstration of coherence indicate a stronger mind?"

3) We kick off the conversation in the Wed, Feb. 15th faculty meeting by
* projecting/reading some of these responses (5 minutes)
* breaking people into small groups to process this information/
perhaps add more stories to the data-collection we've done so far (15 minutes)

4) We return to the large group to say (20 minutes):
In beginning this discussion by naming what some of our students cannot do, we have been having what educational researcher Eve Tuck calls a "damage-centered" conversation, establishing "harm or injury in order to achieve reparation." Let's try to turn now to what Tuck terms "desire-based" work: "Desire is about longing .... desire reaches for contrasting realities." So let's switch this up:

What do we think is valuable about the fact that our students (and our) brains work differently from one another?
How might we begin to incorporate such added values into our classes?

The bottom line question, of course, is what do we want to get out of this conversation?
What might be next steps, beyond it?

conversation since has highlighted a # of problematics w/ this plan.....

Absurd Assessment

Submitted by Wil Franklin on Thu, 01/26/2012 - 11:36am.

A desire-based paradigm for education has profound implications - in my mind. A basic assumption of such a perspective is that humans are complex, contradictory agents longing for self-determination. If an educator is to take this assumption seriously, then (I think) it means the "silent curriculum" - the teaching and learning that happens between the cracks, outside of written curriculum and lessons - may be the most important work. It is the work of interacting with individuals. It is being open to various needs. It is not so much any particular content. And, if it is not any particular content, then what are we to assess?

I'm not sure to what extent the rest of our assessment group is concerned with the above, but I cannot help but wonder what I am trying to assess in my courses. Or to put it another way, to the extent that I am assessing content, I worry that I am not assessing anything of any value, important or even meaning. Are my assessments absurd? Are they inconsistent with my deeply held belief that humans are complex, contradictory and unique agents?

Is their a chance we could push our colleagues at the general faculty meeting on this line of questioning? I don't think my worries pertain to all the faculty. I do not wish to call attention to any real or imagined absurdity. But, I do suspect their is a lot of absurd/inconsistent assessment going on.

On meeting on-line

Submitted by Anne Dalke on Sat, 01/21/2012 - 12:36pm.

Does our group having access to this on-line space relieve pressure, or add to it, since now we have another obligation, another place where we need to be "present"? This question is related, of course, to our in-person discussion this week about mental health issues, and "education as life," or "life-friendly." And reminds me, too, of a digital humanities conversation in the fall, when an alum said that, for students, increasing on-line availability (of texts, of forums, of conversation) means being "on" all the time...there's no downtime, for instance, when a TTH class "meets virtually" for Sunday night blogging...

talking about online talk...

Submitted by jccohen on Sat, 01/21/2012 - 1:32pm.

thanks, anne, for raising that question about online availability. for me it does feel like upping our time/expectations, and i'm not sure to what degree i'll want to participate in this additional mode. i love talking w/you all and i'm increasingly aware of the value of online forums as ways to extend conversations, andbut yes, it's more time and another place to remember to put attention. on the other hand, i don't feel so pressed about how we spend our in-person time together. the faculty meeting seems immediate so it makes sense to me to do that next time, and then the evals as much as we can, and/or could do that the next time. i also like for us to be somewhat open to issues coming up for any of us that we want feedback on in the moment. i trust us to use our time together well/richly and fruitfully, since we have. how about having this online space available but w/o pressure--could we decide to do it that way?

relating resource and expectation

Submitted by alesnick on Sat, 01/21/2012 - 3:01pm.

hi all -- i like the idea to have the online space available w/o pressure, and am interested to hear you say, jody, that you are increasingly aware of the value of online forums to extend conversations. this keys to big questions in ed about how to hold high expectations of self and others but not use pressure to hold them there/up/in . . . and not allow them to generate pressure we don't choose or want. how to use/create resources and still not have to take them (all) on/seriously/all the time . . . today at the gym I listed to this rsa talk about re-envisioning efficiency. really interesting perspectives on how to center social/public programming/justice work on human complexity, the essential keystone of relationships, rather than bow to reductive scalings of same. a line that stays with me: "successful people get things done in their own way." and the idea that successful projects have distinct personalities behind them. to me, this speaks to our assessment project, and also our more recent initiative to refuse overdrive in our work/modeling with/for students.

New Questions

Submitted by Wil Franklin on Wed, 01/18/2012 - 4:54pm.

Having just read over and commented on another round of student self evaluations, I have come away amazed again at the window I was given – a window into learning that I have no way of capturing in a test. This latest round has re-affirmed the power and usefulness of self-assessment; if understanding an individual is valued over judging an individual. On a practical note, the self-assessments supplied far more questions than answers and I bring some of the following to our new round of discussion.

Some excerpts and subsequent questions from self assessment in Bio 111 Lab, F 2011:

Many Bryn Mawr students are obsessed with their grades. I’m not sure if this is a personal thing or if it relates more to expectations from parents, but nonetheless many people here seems obsessed with getting one hundred percent in all of their classes. Anything less seems unacceptable to them. I think this is what the problem stems from. People feel as though they have not done well unless they are told by a teacher that they have one-hundred percent, and they do not feel comfortable with self grading, because they are not getting the acknowledgement that they seek. Personally however, I like it. I think this gives us a chance to experiment and to put ourselves out there without the stress of failing the class because of one unclear assignment.

Can we supply acknowledgement, without grading? Is there a place for acknowledgement in the paradigm of self assessment? Do humans need acknowledgement from others? How much? How do we wean them off the need for outside acknowledgement? Do we even want to?

“Real learning is falling over and picking yourself backup. Science is failure after failure after failure.” Is it correct to say that I was confused more often then not this quarter? Yes, it is. However, I think I am now beginning to really understand when others say it is okay to experience failure if it means you learn from it.
Therefore one thing that I am really satisfied with myself in this quarter is that I became a better self-learner as I am getting used to building up my own “network” by finding the relations among little pieces of knowledge. I used to regard assignments as “burdens”, the only thing I thought when I did my assignment was to finish it. However, I gradually discover that all the assignments actually offer me opportunities to think independently and deeply. Since then, I began to treat every assignment as a chance for self-learning
I felt more comfortable this quarter to speculate and inquire. I did not pretend to understand everything, a facade I think I picked up in high school. (In high school, we were given everything we needed and not expected to think beyond that; therefore, not knowing could only mean a low grade.) I have been learning in lab to embrace that which I don’t understand or know as something I can improve on and eventually claim as knowledge.

These three examples seem to fall under the heading of “becoming more self-aware”. Do these new lessons on self awareness count in a science course about facts? Do these students deserve a merit grade for learning a life lesson about failure, or assignments as opportunities? Does it count if these students learned these lessons by understanding the relationship between inquiry, science and learning?

Looking forward to exploring anew.

responses/not to grading/not

Submitted by jccohen on Tue, 01/17/2012 - 9:00pm.

Just picked up and read my course evals from last semester, and was so struck by students' responses--and not--to the not-grading I practiced in both classes. In my Esem evals there was no mention of grades at all, and in fact no one really talked about my role as an assessor per se. They wrote about their own challenges and growth as 'readers, writers, and thinkers' (how it's framed in the form). Also, in that eval form students are asked to comment on our one-to-one writing conferences, and a number of folks talked about how these conversations, and specifically for some my questions, supported them in figuring out what they wanted to say and how they could say it. In light of our assessment group, I was especially glad that a few talked about learning to be okay with and value uncertainty in their speaking and writing:) I also did not give grades in my Ed class, and a good handful of students expressed their discomfort with this. They talked about it in terms of not knowing 'what was happening' in terms of how they were doing in the class, one said this was a way 'most people' were used to knowing what was going on (implication: they needed it), one said she didn't feel comfortable asking about grades b/c she felt this wasn't the response the instructors wanted. This is a moment to say that I co-taught this class with a student, a whole other and complicated subject, but I'm guessing that may have contributed somehow to this discomfort about not getting graded. But really the contrast that struck me was about the age/college level of the students. The ESem was all freshwomen, of course, and the Ed class tended toward juniors and seniors. This left me with the uncomfortable question of what we (the big 'we' of this institution and beyond) are teaching our students about assessment over their time in the colleges. I've not-graded Ed classes before, including senior sem, and haven't heard this clearly expressed discomfort, and we all know that classes have different characters... but I'm struggling with what I should take away from this and how it might impact my work with assessment this semester. Look forward to talking w/you all on Thurs.!

Sustainability

Submitted by Anne Dalke on Fri, 01/13/2012 - 7:57pm.

I'm picking up here on the word 'sustainability,' from Alice' s so-interesting post about learning from failure…..and (inspired by her painting) turning it first into an image.

I want to juxtapose the image/concept with a wonderful line that Jody just shared with me, from Elizabeth Ellsworth's book, Teaching Positions: Difference, Pedagogy and he Power of Address: "Pedagogy, when it 'works,' is unrepeatable and cannot be copied, sold or exchanged--it's 'worthless' to the economy of educational accountability."

And so my associative mind is puzzling now about the relationship between accountability and sustainability. How to make sustainable that which is not repeatable? How to find forms of accounting/assessing that are not putting into a box the unique and unrepeatable acts of true teaching and learning? The acknowledgement of what we don't/can't know, the unpredictability and surprise of our educating ourselves and others?

I return to this working group, after the end of another semester, w/ a whole range of new experiences under my belt/in my heart and mind. A strong theme, in the on-line evaluations students did for the core course in gender studies,
was the emergence of friendship among us all:

* "Thanks for a semester filled with learning, growth, and the formation of new friendships."

* All in all, the community - intellectual and personal - I felt that we created in this class was astounding. I really felt like we were all so responsive to each other, intellectually, emotionally, in lots of ways...

* It is the first safe space I have ever experienced that emerged through the process of creating right relationships among peers, rather than through a preestablished rule. It is the first class I have seen where emotion and empowerment were woven into intellectual pursuits-- it went beyond interdisciplinary-- it left disciplines all together and played with human experience in a very raw way.

*each of us is both a particle and a wave …We bounce off each other, diffract through one another, imprint ourselves on each other …We are wave-like in that there is something of our presence that is not necessarily visible or finite - we change the world we live in and the people we live with. But there is also a way in which we are discrete – we have a physical presence, we are in a certain way tangible, borders can be drawn around us….the wave and particle … like electrons we are both at once.

So what I'm thinking about now is how to re-figure assessment as a friendly act, part of on-going, collaborative learning, what friends do for-and-with one another all the time ("how's this dress look?" "you've been drinking, let me drive," "what a surprise, your showing up just now!" etc. etc.)--giving one another feedback, holding one another accountable, giving thanks and affirmation….

…that seems to me a sustainable practice!

Can't wait to go exploring further w/ you all in this direction.

counter narrative?

Submitted by Anne Dalke on Mon, 01/16/2012 - 7:05pm.

I have just finished reviewing a portfolio for a teacher certification candidate, which includes the wonderful story of how a test once "failed her brother"--failed, that is, to find out what he knew. I'm thinking that maybe the current College evaluation forms are failing our students, failing to offer them the opportunity to show what they are learning?

I've just now read these forms for both my fall classes, and am aghast @ the great divide between what they report and the students' self-assessments. Reading the first set gives very little sense of how successful these classes were, for almost all the students in them; reading the second set provides both testimony to individual growth and community collaborative learning. I'm flabbergasted @ the difference, and --trying to make sense of it-- am wondering how much of it has to do w/ the way that the College evals ask for the teachers to be judged, rather than inviting students to think about their own learning? And asking them to do so on their own terms, as the self-evals do?

re-anchoring

Submitted by alesnick on Tue, 01/10/2012 - 12:40pm.

I am wondering what my grounding is in the question of assessment. What do I really think it is, and is for? The way I answer this changes with the way I define learning. Last summer, working in a new way with Paul, I wrote that learning is, a matter of questioning and changing the boundary between what is and what is going to be – seeing this relationship, rather, as completely open. It is a matter of each individual’s traversing and dissolving this boundary by drawing vision and purpose out of our lives and daring to realize them, which realizing renews the universe, over and over in a neverending creative process.

I just watched a Ted Talk by David Damberger, Engineers Without Borders, Canada, on "What Happens When an NGO admits failure." EWB now publishes an annual "Failure Report" because, "We . . . believe that it’s important to publicly celebrate these failures, which allows us to share the lessons more broadly and create a culture that encourages creativity and calculated risk taking. This is a culture we value within EWB, and also try to work with our partners in Africa to create in their organizations."

According to Damberger, projects he helped create failed because human factors for their sustainability were not considered. Since he and his organization want to solve problems, they need to learn from the failures to keep going and do better.

Here, the assessment is a process of learning, returning to the village where the water taps were installed, asking people questions . . . and it is part of a cycle of action and revision.

re-entry

Submitted by alesnick on Mon, 01/09/2012 - 6:10pm.

happy new year! i'm returning to teaching after a fall sabbatical, during which i spent some time working with a paint brush rather than a pen [computer]. i was thinking to say that this is a double re-entry, into teaching and into this group, but while the group hasn't met since last spring all together, i've been in active conversation with it as an imagined whole and with most of its members with reference to it. so not re-entry, but a different continuity.

Says the paintbrush to the pen: with me, it's a quicksilver way.

Photo Book Painting

Says the pen: not so fast; I'm nimble, too.

paintbrush: yeah but with me it's about the merge, meld, melt, mirage, but you, merely 0's and 1's. this is the problem with human measurement systems, the fixation on 0's, 1's.

pen: yeah, but they need those to make you in large numbers, and to transport you across bridges and down highways to all the little children.

paintbrush: oh, please. do you know any new songs? how does anyone know i'm any good? is it my catalytic validity (patti lather) -- the changes i spur? is it how i make people feel, or talk, or paint next? yes, it think it's that. could you help the humans make an assessment system that captures that?

pen: why should i?

paintbrush: to crowd out the reductive kinds -- the ones that render you a sword or spit.

pen: but why should i help them do assessment at all? what if i refuse? what if, like bartleby, i would prefer not to?

paintbrush: then how will they decide which of my canvasses gets into the museum?

pen: oh, look who's all "know the right answer now." what if i want instead to map a world in the question of how a person is doing at learning something doesn't make sense except in personal, local, emergent situations?

paintbrush: no one will want to use your map because they'll suspect it leads to a lonely, hungry dead-end.

pen: this is depressing. can you bring some color to this convo?

paintbrush: sure thing.

Blessings....

Submitted by Anne Dalke on Wed, 06/15/2011 - 11:20am.

As blessing for getting us going, last semester, I sent my colleagues the last three stanzas from a blessing that Kim Stafford (son of the poet William) provided @ the Oregon State Capital inaugural ceremonies last January. Substitute "education" for "democracy" and "justice" and "governance" to get my sense of what we were going to be up to: enlarging the scope of education (and assessing education) far beyond counting:

...For democracy is not counting, but collective wisdom.
Justice is not judgment, but investigation of the good.
Governance is informed affection for all our people.

We shall be kindred in this work. Clouds rise from the sea
to carry their treasure to us all, to our cleft of forest, our
river’s bend, our rim of sage where meadowlark calls.

Good governance, in my life and in each of yours,
lives in how we listen, one to another: to rivers wild,
to easy rain, to fervent questions from a child.

This is what I think our work together is up to, and why it so inspires me! Our focus isn’t pedagogical in the conventional sense; we ranged quite broadly this past semester, really enlarging our original topic of “assessment” to think contextually about what education is, how to “deliver” it, how to “receive” it, asking repeatedly “what do we want them to get better/good @”? I love Sara’s answer to this question: “I want them to be certain of one thing: the uncertainty, to learn to tolerate the ambiguity and be okay w/ that. I want them to realize there is not just one way to do something.” And I love (and instantly appropriated!) Alice’s closing ritual of asking students to “think of something you brought in, and are leaving behind, and then of something you have picked up, and are taking away--and make it concrete."

A good deal of our conversation had to do with student self-assessment; we talked about the possibility of evaluation on the level of the group, rather than always focusing on individual accomplishment. We discussed surface and depth; distance and upcloseness; meditation and restraint; adaption and collaboration. We talked about a “pedagogy of surprise” (for what? surprised to what end? to “shake up the pattern”?). We discussed (and dismissed) “gatekeeping for weakness,” and worried about the insistently time-driven dimension of assessment. We talked about the paradox of wanting to be “totally accepting of where our students are, and wanting them to aspire to something beyond”—-that old tension between support and challenge.

As noted above, we had particularly rich conversations seeded by Margaret Price’s new book, Mad @ School, and by The New Yorker article on “The Poverty Clinic,” which explores the possibility of childhood trauma as a major indicator of adult health problems, and so prodded us to think way beyond the box in terms of what we are assessing, and why. Does the very structure in which we are embedded instigate mental health problems? How essential is it (and why is it essential?) that students be able to “make manifest” -- that is, to reveal publicly -- what they have learned? How can we work toward a more diverse range of manifestations? Some of my students, this past semester, have begun to experiment in this direction; see

The Psychology of Serendip: A collaborative project showing the progression of two students in discovering our own interpretations of learning.
A GIST Portfolio: An attempt at exploring my journey through Gender Information Science and Technology at Bryn Mawr college with the goal of assessing individual growth, learning, and contribution to knowledge.
Gender, Information, Science and Technology: Portfolio: an accumulation of my learning experience in GIST, Bryn Mawr Spring 2011.
My GIST Portfolio

Such initiatives, I hope, can begin to alter the environment in which such students can conduct their own assessments, as reflective meditative communications, making themselves "present" in ways that make most sense to themselves.