You shall not pass! The role of a reviewer

 

What does a reviewer do? At first glance, the task of a reviewer is very simple: he is supposed to separate the wheat from the chaff, i.e., separate bad publications from good ones that can be published.

But what is the difference between good and bad publications? As is often the case, the decision is rarely clear, because there are many gradations between good and bad.

That’s why I always think carefully before accepting any request for a review: am I competent on the topic? If so, then my first glance is for the science presented here: is it consistent? are the methods used suitable? The best-written manuscript is worthless if its science were incorrect or not credible…

 

Some frequent points of attention

If the scientific results presented are correct or appear to be correct, I look at the presentation in detail. Are the illustrations, equations and diagrams presented meaningful and interesting, ideally without having to read the text? Even though English has supposedly established itself as the language of science, we also have to think about readers who have little knowledge of it (and the question arises all the more when a text is published in another language). Yes, it is still possible to publish in languages other than English, and I am happy to support this as much as I can, despite my limited knowledge of foreign languages. For me, this also contributes to a more inclusive scientific community.

At this point, you should also ensure that the labels are legible: not everyone can read a 5pt font!

Another important point to examine is the section of the article announcing the results. How are the results presented described? How are the new results placed in their scientific context? Does the author know the literature well? Have they overlooked important aspects any important points that are already known but are relevant to the subject, and which not all readers may be aware of? From my point of view, it is essential for the author to be familiar with older literature. I don’t know of any topic in chemistry on which there is no important literature from the 20th century. And of course, it is crucial that the author also knows about literature other than their own. If a very large number of the quotes come from the author or their team, you should look twice and ask yourself some questions! Of course, introduction need not be an exhaustive overview of the subject, but the reader must be guided, and the context of the study presented must be outlined.

I also like to pay attention to the title chosen for the article. Of course, we don’t ask authors to write advertising texts, but a catchy or even humorous title is always welcome. Which article would you rather read: “Comptes Rendus Chimie, a chemical journal since 1835″, or “Comptes Rendus Chimie, a journal with a long publishing tradition, prepared for the 21st century”?

Of course, even a scrupulous verification of all these points cannot completely prevent fabricated studies from being published, provided their authors have put enough criminal genius into them. Likewise, no matter what you do to protect your home, burglaries happen… However, questions remain: how likely is this to occur in Comptes Rendus Chimie? Why would an author take such risks, and what does he gain from it? Doesn’t it take more effort to invent a study from scratch, rather than do serious science?

 

Writing a report

Now comes the most difficult part: writing a report and evaluating the entire manuscript. If a publication is very good, it’s easy: there’s nothing to add or remove, and the article can be said to be “publishable as is”.

Things get more complicated if the manuscript is only half-good, or even completely bad. Why is this? Because authors are proud of their work, and no one wants to hear that everything they’ve done is rubbish. In addition, my aim is to help authors understand what the problems in their manuscript are, so that they can solve them where possible, or do everything right in their next work. We learn from our mistakes, but only if they are highlighted to us politely, kindly, and respectfully. We were supposed to have experienced this at school: to learn something, we need the teacher to assess where we’re at, and explain where we’ve gone wrong. It’s nothing different here. Almost every scientist knows exemples of reports which, in the best-case scenario, did not help them at all or only caused anger and led to the wholesale rejection of any attempt at improvement.

 

Scientific evaluation in a competitive environment

In this context, what is so special about Comptes Rendus Chimie? Firstly, the organization that publishes them, the Académie des Sciences, and secondly, the journal’s reputation. Why is this? Because the Académie des Sciences does not seek to make money from its publications. The Académie des Sciences serves science itself, and therefore, everyone on earth. To this end, it has chosen to make its publications entirely open access, and without authors having to pay to be published.

This is why, even if Comptes Rendus Chimie is not (yet?) among the leading international scientific journals in chemistry, it is possible for all of us – authors, reviewers and editors – to work together to improve this, if we so wish. I’d still prefer a journal committed to publishing robust, reliable science, based on trusted data, and powered by scientists committed to these values of quality and relevance.

This attitude is quite unpopular nowadays, as many researchers prefer to publish in journals that excel in quantitative bibliometric indicators, each more obscure and complicated to calculate than the last, or place less importance on the robustness of scientific evaluation. In the short term, of course, it’s easier to promote oneself and distinguish oneself from one’s colleagues. However, this is also dangerous, because what light does it shed on one’s own work, in a context where more and more articles are retracted after publication when shortcomings are detected, and where authors and publishers manage to cheat with the metrics?

That’s why I do hope that Comptes Rendus Chimie persists in its attachment to the traditional principles of science, and continues to offer publication opportunities to all scientists, and in particular to researchers with fewer resources than in France or Germany. For the only quality criterion for a not-for-profit journal may be the intrinsic quality of the articles published, and no one can say in advance what will be of importance to the science of tomorrow or be forgotten by history. At least, I don’t think I’m capable of doing that, either as a reviewer or in my own research.

 

Reviewer: a delicate position

The reviewer is not the author of the article, let alone the person behind the research described. But of course, they always know everything better than the authors themselves, and have brilliant ideas for improving the manuscript. Science is like a sports match, for example, a France-Germany soccer game: don’t the spectators know the rules of the game better than the coaches of the national teams, or even the referee himself?

Of course, in their report, the evaluator must share their ideas with the authors, and hope that they will benefit from them. However, it is fundamental that the evaluator does not base his expertise and the decision rendered on his own ideas, but on the science as presented in the manuscript. Nothing more.

Another hot topic in the scientific community is “gender and diversity”. Does it have a place in the evaluation process? Personally, I don’t think so. Firstly, because unless you know the authors personally, it’s often impossible to draw conclusions about their gender on the basis of their first name alone (in Germany, Andrea is a feminine name, but in Italy, it’s masculine; and what about an Italian working in Germany?). Secondly, because one wonders how the gender of an author or their self-image can have an impact on the quality of the research presented… As far as I’m concerned, I haven’t found any correlation yet! The same applies to the ethnicity of the authors.

In my opinion, the question of the affiliation is much more problematic. Given the ratio of good to bad manuscripts already submitted by authors from the Free University in Duckburg, it’s sometimes difficult to remain entirely neutral when a new article arrives.

But because, after all, reviewers are scientists, they don’t hesitate to examine themselves critically. Authors, reviewers and editors are first and foremost researchers, and as such, they are (most of the time) not responsible for the political decisions made in their home countries, or for the historical events of the last thousand years… 

What about personal feelings, and possible sympathy/antipathy for the authors? Of course, we scientists are human beings. It’s not uncommon for us to know each other, or at least each other’s work. Obviously, we’re flattered if our work is cited, and we’re more disposed towards our personal scientific heroes, our absolute role models, or Nobel Prize winners. But we are (and should be) acutely aware of these human problems. For my part, I would feel ready to refuse to review a manuscript for such reasons, especially if I didn’t like one of the authors at all (for whatever reason). And I have no regrets about this, because the job of reviewer is a voluntary one, with no financial compensation whatsoever: it’s thanks to this that we can remain independent and objective.

My vacation on the Côte d’Azur is certainly not financed thanks to my reports for Comptes Rendus Chimie. But perhaps a holiday on the Côte d’Azur is tempting because Comptes Rendus Chimie makes you feel positive about France and makes you curious about the country that has had a young, veteran, and charming ambassador for chemistry since 1835.

 

Ralph Puchta
Friedrich-Alexander-Universität, Erlangen, Germany

 

This article is also available in French on this website.

---

Credit for cover illustration : A reviewer at the National Institutes of Health evaluates a grant proposal. Image provided by the Center for Scientific Review (source). Public domain.

---

---