The Trouble with Physics--Seers and Craftspeople Part 1

The following is an excerpt from the conclusion of the book The Trouble with Physics by Lee Smolin. I have thought for months about how to synthesize this, and failed. Here it is, in full. It is largely a long quote, but a few sentences or paragraphs have been removed to condense it as much as possible, and I have paraphrased at a couple of points. I reviewed and expanded on some thoughts from other portions of the book here and here (essentially the same post). Now for Smolin:

"Science has succeeded because scientists comprise a community that is defined and maintained by adherence to a shared ethic. It is adherence to an ethic, not adherence to any particular fact or theory, that I believe serves as the fundamental corrective within the scientific community.
"There are two tenets of this ethic:

1. If an issue can be decided by people of good faith, applying rational argument to publicly available evidence, then it must be regarded as so decided.
2. If, on the other hand, rational argument from the publicly available evidence does not succeed in bringing people of good faith to agreement on an issue, society must allow and even encourage people to draw diverse conclusions.

"I believe that science succeeds because scientists adhere, if imperfectly, to these two principles. To see whether this is true, let us look at some of the things these principles require us to do.

• We agree to argue rationally, and in good faith, from shared evidence, to whatever degree of shared conclusions are warranted.
• Each individual scientist is free to develop his or her own conclusions from the evidence. But each scientist is also required to put forward arguments for those conclusions for the consideration of the whole community. These arguments must be rational and based on evidence available to all members. The evidence, the means by which the evidence was obtained, and the logic of the arguments used to deduce conclusions from the evidence must be shared and open to examination by all members.
• The ability of scientists to deduce reliable conclusions from the shared evidence is based on the mastery of tools and procedures developed over many years. They are taught because experience has shown that they often lead to reliable results. Every scientist trained in such a craft is deeply aware of the capacity for error and self-delusion.
• At the same time, each member of the scientific community recognizes that the eventual goal is to establish consensus. A consensus may emerge quickly, or it may take some time. The ultimate judges of scientific work are future members of the community, at a time sufficiently far in the future that they can better evaluate the evidence objectively. While a scientific program may temporarily succeed in gathering adherents, no program, claim, or point of view can succeed in the long run unless it produces sufficient evidence to persuade the skeptics.
• Membership in the community of science is open to any human being. Considerations of status, age, gender, or any other personal characteristic may not play a role in the consideration of a scientist's evidence and arguments, and may not limit a member's access to the means of dissemination of evidence, argument, and information. Entry to the community is, however, based on two criteria. The first is the mastery of at least one of the crafts of a scientific subfield to the point where you can independently produce work judged by other members to be of high quality. The second criterion is allegiance and continued adherence to the shared ethic.
• While orthodoxies may become established temporarily in a given subfield, the community recognizes that contrary opinions and research programs are necessary for the community's continued health.

"When people join a scientific community, they give up certain childish but universal desires: the need to feel that they are right all the time or the belief that they are in possession of the absolute truth. In exchange, they receive membership in an ongoing enterprise that over time will achieve what no individual could ever achieve alone. They also receive expert training in a craft, and in most cases learn much more than they ever could on their own. Then, in exchange for their labor expended in the practice of that craft, the community safeguards a member's right to advocate any view or research program he or she feels is supported by the evidence developed from its practice.
"I would call this kind of community, in which membership is defined by adherence to a code of ethics and the practice of crafts developed to realize them, an ethical community. . .
". . . some ethical communities exist to preserve old knowledge rather than to discover new truths. . . . [S]cience in its modern form evolved from monasteries and theological schools--ethical communities whose aim was the preservation of religious dogma. . . .
". . . I would like to introduce a second notion, which I call an imaginitive community. This is a community whose ethic and organization incorporates a belief in the inevitability of progress and an openness to the future. The openness leaves room, imaginatively and institutionally, for novelty and surprise. Not only is there a belief that the future will be better, there is an understanding that we cannot forecast how that better future will be reached.
". . . .
"An imaginative community believes that the future will bring surprises, in the form of new discoveries and new crises to be overcome. Rather than placing faith in their present knowledge, its members invest their hopes and expectations for the future in future generations, by passing along to them the ethical precepts and tools of thinking, individual and collective, that will enable them to overcome and take advantage of circumstances that are beyond the present powers of imagination.
"Good scientists expect that their students will exceed them. Although the academic system gives a successful scientist many reasons to believe in his or her own authority, any good scientist knows that the minute you succumb to believing that you know more than your best students, you cease to be a scientist.
"The scientific community is thus both an ethical and an imaginative community.
"What should be abundantly clear from this description is that controversy is essential for the progress of science. . . .
". . . When scientists come to agreement too soon, before they are compelled to by the evidence, science is in danger.
"If we think we know the answer, we will try to make every result fit that preconceived idea.
". . . I would suggest that enough scientists adhere to enough of the ethic that in the long run progress continues to be made, despite the fact that time and resources are wasted in the promotion and defense of orthodox and fashionable ideas that later turn out to be wrong. The role of time must be emphasized.
". . . .
"Adherence to the shared ethic is never perfect, so there is always room for improvement in the practice of science. This seems especially true today, when fashion appears to be playing too large a role, at least in physics. You know this is happening whenever there are bright young recent PhDs who tell you privately that they would rather be doing X but are doing Y because that is the direction or technique championed by powerful older people, and they thus feel the need to do Y to get funding or a job. Of course, in science as in other areas, there are always a few who choose to do X in spite of the clear evidence that the doers of Y are better rewarded in the short term. . . . Thus the progress of science may be slowed by orthodoxy and fashion, but as long as there is room for those who do X instead of Y, it cannot be stopped completely.
". . . . While the progress of science relies on the possibility of achieving consensus in the long term, the decisions an individual scientist makes as to what to do, and how to evaluate the evidence, are always based on incomplete information. Science progresses because it is built on an ethic recognizing that in the face of incomplete information we are all equal. No one can predict with certainty whether an approach will lead to definite progress or years of wasted work. All we can do is train students in the crafts that experience has shown to lead most often to reliable conclusions. After that, we must leave them free to follow their own hunches and we must make time to listen to them when they report back. As long as the community continually opens up opportunities for new ideas and points of view and adheres to the ethic that in the end we require consensus based on rational argument from evidence available to all, science will eventually succeed.
"The task of forming the community of science will never be finished. It will always be necessary to fight off the dominance of orthodoxy, fashion, age, and status. There will always be temptations to take the easy way, to sign up with the team that seems to be winning rather than try to understand a problem afresh. At its finest, the scientific community takes advantage of our best impulses and desires while protecting us from our worst. The community works in part by harnessing the arrogance and ambition we each in some degree bring to the search. Richard Feynman may have said it best: Science is the organized skepticism in the reliability of expert opinion."

"The one thing everyone who cares about fundamental physics seems to agree on is that new ideas are needed. From the most skeptical critics to the most strenuous advocates of string theory, you hear the same thing: We are missing something big.
. . . .
"It goes without saying that people who are good at asking genuinely novel but relevant questions are rare, and that the ability to look at the state of a technical field and see a hidden assumption or a new avenue of research is a skill quite distinct from the workaday skills that are a prerequisite for joining the physics community. It is one thing to be a craftsperson, highly skilled in the practice of one's craft. It is quite another to be a seer.
"The distinction does not mean that the seer is not a highly trained scientist. The seer must know the subject thoroughly, be able to work with the tools of the trade, and communicate convincingly in its language. Yet the seer need not be the most technically proficient of physicists. . . . There is only one person I can think of who was both a visionary and the best mathematician of his day: Isaac Newton; indeed, almost everything about Newton is singular and inexplicable.
. . . .
"Master craftspeople and seers come to science for different reasons. Master craftspeople go into science because, for the most part, they have discovered in school that they're good at it. . . .
"Seers are very different. They are dreamers. They go into science because they have questions about the nature of existence that their schoolbooks don't answer. If they weren't scientists, they might be artists or writers or they might end up in divinity school. It is only to be expected that members of these two groups misunderstand and mistrust each other.
"A common complaint of the seers is that the standard education in physics ignores the historical and philosophical context in which science develops. . . .
"Of course, some people are mixtures of both. No one makes it through graduate school who is not highly competent on the technical side. But the majority of theoretical physicists I know fall into one or the other group. . . .
"When I first encountered Kuhn's categories of revolutionary and normal science as an undergraduate, I was confused, because I couldn't tell which period we were in. If I looked at the kinds of questions that remained open, we were clearly partway through a revolution. But if I looked at how the people around me worked, we were just as obviously doing normal science. There was a paradigm, which was the standard model of particle physics and the experimental practices that had confirmed it, and it was normally progressing.
"Now I understand that the confusion was a clue to the crisis I have been exploring in this book. We are indeed in a revolutionary period, but we are trying to get out of it using the inadequate tools and organization of normal science.
. . . .
"I have nothing against people who practice science as a craft, whose work is based on the mastery of technique. This is what makes normal science so powerful. But it is a fantasy to imagine that foundational problems can be solved by technical problem solving within existing theories. It would be nice if this were the case--certainly, we would all have to think less, and thinking is really hard, even for those who feel compelled to do it. But deep, persistent problems are never solved by accident; they are solved only by people who are obsessed with them and set out to solve them directly. These are the seers, and this is why it is so crucial that academic science invite them in rather than exclude them.
"Science has never been organized in a way that is friendly to seers. . . .
". . . But who are the seers? They are by definition highly independent and self-motivated individuals who are so committed to science that they will do it even if they can't make a living at it.

Many of the seers have to make careers outside of the establishment, and are only welcomed back much later, if at all.

"I wish I could have an honest conversation about risk with the National Science Foundation. Because I'm sure that 90 percent of the grants they give in my field fail, when measured against the real standard: Do those grants lead to progress in science that would not have occurred if the person funded did not work in the field?
"As every good businessperson knows, there is a difference between low-risk/low-payoff and high-risk/high-payoff strategies, starting with the fact that they are designed with different goals in mind. When you want to run an airline or a bus system or make soap, you want the first. When you want to develop new technologies, you cannot succeed without the second.
"What I wouldn't give to get university administrators to think in these terms. They set up the criteria for hiring, promotion, and tenure as if there were only normal scientists. Nothing should be simpler than just changing the criteria a bit to recognize that there are different kinds of scientists, with different kinds of talents. Do you want a revolution in science? Do what businesspeople do when they want a technological revolution: Just change the rules a bit. Let in a few revolutionaries. Make the hierarchy a bit flatter, to give the young people more scope and freedom. Create some opportunities for high-risk/high-payoff people, so as to balance the huge investment you made in low-risk, incremental science. The technology companies and investment banks use this strategy. Why not try it in academia? The payoff could be discovering how the universe works."