Evaluations

I recently did some of the tests on yourmorals.org. They ask you a number of questions to get at how much you care about certain foundational moral values: Harm, Fairness, Loyalty, Authority, and Purity. On the "Moral Foundations Questionnaire" I came out very close to the average among political liberals for caring about not causing harm, for valuing fairness, and for not believing authority has inherent value. I was in between the conservative and liberal averages on valuing loyalty, and I seem to really dislike things I find disgusting--even though I don't very actively seek to constrain society (or my friends) to my particular standard of purity on the things asked about in the questions. These numbers were fairly predictable to me, except for how much I seem to value purity (really more a weak stomach when it comes to things I find disgusting). But there was a second, shorter questionnaire that just asked you outright to rate how much you care about different moral values relative to other people. To judge by my self-reporting on "What is your moral decision type?, one would imagine that I care less about moral values than the average person--let alone people with truly strong moral sensitivities. However, I don't think people who know me would evaluate me as a weakly moral, apathetic, or middle of the road person. So what's going on?

It's a simple case of awareness of personal bias. I'm not imagining, here, that I can correct for my personal bias, but in answering the "moral decision type" questions, knowing that somebody has to be average and that few people are at the extremes, I tried to think of the group of people I really know and regularly associate with. In most cases, I figured my moral sensibilities were average or just barely one side of average. Looking at the aggregate results, most people don't feel that way. Once again, most people rate themselves as significantly above or below average, depending on the trait and whether they feel above or below is admirable or not. Consequently, my self evaluation placed me as much more "average" than the average self-evaluation, and closer to my values from the "Moral Foundations Questionnaire" than the average self-evaluation. But I didn't get all that close.

Another test I did was the "Implicit Happiness" test. I rated my life as fairly satisfying, and as having met many of my hopes and aspirations. This placed me noticeably above the average self-evaluation for life satisfaction, but I know that I feel sad and depressed quite a lot. After taking the Implicit Happiness test, it showed me as noticeably below average on personal happiness. Again, I wasn't surprised. My unhappiness is in the way my brain and body work, not in an unfulfilling life. Maybe I'll be able to modify the brain chemistry with time and tools like meditation, but I haven't succeeded yet.

I tell these stories just as a personal reminder of how poor we are at statistical evaluations. Sometimes we get it about right, sometimes we are way off, and sometimes we aren't asking the right questions. That brings me to my most recent difficulty with evaluations. I've started writing letters of recommendation for students. It is clear to me that average recommendations don't get people into programs, even if an average applicant (the poor applicants have typically been weeded or self-selected out) is amply deserving of entrance, and likely to do very well. I also want to save room at the top for recommending the occasional, truly exceptional student. So I get these questionnaires, and they ask me to rank students in the top 5%, next 10%, next 20%, etc. on various attributes. It's a terrible method, but there may not be a better one. Top X% among what group? All students at my school? All biology majors? Biology majors that are likely to apply for the program? All human beings? So my emotions tell me, this student is a bit above average for my classes. He or she is determined, consistent, conscientious, and will make a good doctor, pharmacist, dentist, or whatever if he or she receives the proper training. I have no doubt I would prefer my student in these positions over some professionals I have met in the same field, or at least not less. So I inflate the numbers a little. I sit here wishing all of us recommenders had better developed statistical intuition, because I feel like I'm bending the truth, but the reality is, I'm making what I know to be an approximately accurate adjustment for the poor statistical intuition of the majority of recommenders. I'm not claiming more than I can justify. I'm not sending unprepared students places they don't belong, but I am using statistical intuition to ignore statistics and acknowledge the human biases of recommenders and evaluators. If my student has the same GPA and other quantifiables as another, I'm not going to doom his or her application by giving an above average rating when I can feel OK about an excellent. I won't call him "outstanding" or her "top of the class" if they aren't, but I won't hold back on honest praise.