|
|
An
Angel Weighing a Soul, circa 1348-55 by
Ridolfo Guariento di Arpo
|
|
|
An
Angel Weighing a Soul, circa 1348-55 by
Ridolfo Guariento di Arpo
|
| The evaluation of scientific accomplishments and the appraisal of the creative potential of members of academic institutions, in their funding largely dependent on public resources, is a bothersome necessity, but helpful in maintaining a certain level of quality and efficiency in the output of white-collar workers. However, mental achievements are more difficult to measure than the output of assembly-line components or the words per minute in type-writing. How to measure mental creativity or the brilliance of ideas? By competitors, as it is the usual habit in the natural science disciplines? And based on what grounds? On grounds of publications in journals, again peer-reviewed? It appears as some kind of general remedy to resort to peer-reviews when it comes to decisions relevant to funding or careers. But what may be helpful in one domain of wisdom may be at least problematic in others. |
| First of all, competitors tend to devaluate the records of their potential rivals. Fortunately, in the natural sciences not much room is left for subjectivity and prejudice. A 'good paper' reports on unambiguous facts and measurements, and on their cautious interpretation. Here, any review process has to be as rigorous as possible, and any diligent and careful researcher has a good chance to survive even the most severe survey, provided his or her data were reliable. But what about 'data', what about 'subjectivity' and 'prejudice' in the humanities? Do the humanities collect 'hard facts'? Here, it is not the main interest, what is written in a publication, but rather how it is written and who is the writer. In other words: subjectivity is on the central stage in the humanities. The humanities don't discuss hypotheses and theories like the natural sciences do; they discuss authors. |
| Second, it is relatively easy to judge the value of a natural science report. You just have to be familiar with the methods used, and with some formalism widely agreed upon. However, in the humanities you have to deal with publications of a radically different type. In the humanities, scientists spend most of their time with reading and writing, not with experimental work. To elaborate on some essential thoughts easily can take the form of several dozens of pages or even of whole books. Citations in the humanities much more often are to books than to journal articles, very much in contrast to citations in the natural sciences. Therefore, while arguments can be exchanged easily in the natural sciences by citing a few papers, given the justified assumption that all discussants are familiar with the main results, discussions in the humanities are of a completely different kind. |
| In the humanities, discussions are an integral and genuine part of the creative process. Performing a lecture, giving a talk, participating in a round table conference, setting up a seminar, even hearing an exam is doing scientific work. The results of such work can never be predicted with 100% certainty. It depends on the course of inter-human communication. In the humanities, no scientist can be absolutely sure about his / her topic. He / she has always to engage in some kind of discourse, with the results depending on immediate reception and reflection. Very often, their style of presentation may appear - especially to natural scientists - like circumstantial story telling (for natural scientists, a totally non-scientific style). |
| When it comes to fund raising and the allotment of limited resources, some natural scientists are quick with their call for strict 'evaluation'. What they mean by that is easy: They want to have counted, how often their papers are cited. In times of globalization, most citations are obtained to international journals. A criterion like this may allow for the comparison of natural scientists with each other (after some adjustment for the respective discipline), but for sure it does not allow to compare natural science outputs to creations in the humanities. By virtue of their inherent subjectivity, creations in the humanities cannot be subjected to peer review as successfully done so in more primitive disciplines. 'Measure, what can be measured. And make measurable, what cannot.' This was the motto of Galileo Galilei marking the beginnings of natural science. But it may not always be possible or even purposeful to measure. There is more to the human mind than can be weighed or counted. |
| Should we conclude, then, that creative output cannot be measured at all in the humanities? It can (at least to some extent), but not under the same premises as in the natural sciences. (1) In the humanities, since language is so central to productivity, scientists should never be judged solely by their foreign language output. (2) Never subject a scholar in the humanities to a review process by peers; rivalry will seduce most referees to direct their subjectivity against the subjectivity of the candidate. (3) If you want a thorough evaluation in a discipline of the humanities, you have to invest more time and more money than in a discipline of the natural sciences. Each referee has do deal with the candidate’s oeuvre in detail, by really reading significant parts of it, and not just by counting impact factors as it may be sufficient in a natural science discipline. (4) Never base a serious decision, be it on funding or on career, on the opinion of only 2 (or even just 1) referee. (5) Be sure that the referees are not active in the same field as the candidate, but in various disciplines related to the candidate’s discipline. (6) Allow for a second round in the review process, confronting all (anonymous) referees with all exposés and with the candidate’s reply to them. |
| Whereas in the natural sciences we increasingly tend to have creativity measured by machines and not by (fallible) humans, when it comes to real human cognitive activities, the output of minds can no longer be measured by machines, but only by other human minds, with the investment of all cognitive and social ingenuity the human mind is capable of. |
