International scientific community agree on need for quality indicators for new (Open Access) journals

25 OCT 2012

An international group of top experts are willing to contribute to the development of a new tool for assessing the quality of new (Open Access) journals.

Possible quality indicators provide good basis for discussion

Research funders, editors-in-chief of prestigious academic journals, and publishers discussed the results of studies into two possible quality indicators for young (Open Access) journals at a two day colloquium in Rotterdam, the Netherlands. Although these quality indicators are not yet ready to use, they provided a good basis for discussing how to ensure greater insight into the quality of new journals.

Lack of reliable indicators

According to a number of surveys, the majority of scientific and scholarly communities are in favour of Open Access to scientific publications. However, most researchers do not yet publish their work in Open Access journals. Many of these journals are relatively new and so - in the perception of the scientific community - do not yet have an easily discernible impact marker. This makes it difficult for researchers to determine which of the newly available Open Access journals are of sufficiently high quality to be worthy of their publications.

There is therefore an urgent need for valid and reliable ways to judge the quality of Open Access journals in the natural sciences, social sciences, and humanities. The European SOAP project also indicates that the lack of reliable quality indicators is one of the main barriers to authors publishing in Open Access journals.

New indicators

The aim of the initiators - SURF and Amsterdam University Press, with support from the Netherlands Organisation for Scientific Research (NWO) and the Royal Netherlands Academy of Arts and Sciences (KNAW) - was to investigate potential tools for providing greater insight into the quality of journals. CWTS focused on an indicator for the editorial boards of journals. Jelte Wicherts of Tilburg University set up a tool to indicate the quality of the peer review process.

Franciska de Jong, a member of the General Board of NWO, the largest funder of scientific research in the Netherlands, stated: "Uncertainty is inherent to the enterprise of starting a new journal and we should help the initiators of those journals to turn their prestige as academics into the trust of authors."

Transparency of the peer review process

Those attending agreed on the need for new tools to measure quality. The transparency of the peer review process is considered to be an important issue. A set of criteria are needed with which a journal must comply so as to provide sufficient insight into how its peer review process is organised. That information must be easily accessible for authors. Many journals do in fact have that information on their website, but it is hidden away in the midst of large quantities of text.
Transparency is needed so that information can quickly be found about the journal's aim and target group, the assessment criteria applied by reviewers, and who is ultimately responsible for the final decision on whether or not to include a publication in the journal. A "hands-on" session at the colloquium revealed that a lot of journals still fail to provide this kind of information, or do not make it easily accessible on their website.

Editorial board

CWTS investigated whether the editorial board can be the yardstick for assessing the quality of a journal. The study showed a correlation between the impact of the members of the editorial board and the impact of the journal. At the same time, there is a large variety of roles of editorial boards. Hence, there is no clearcut relation between the editorial board impact measure and journal quality. The experts also stressed that editorial boards may say more about the prestige of a journal than about its quality.

Follow up

CWTS and Jelte Wicherts will be publishing their studies and will include the comments and feedback provided by the experts during the colloquium. In the meantime, the organising committee will get together and determine the follow-up, supported by the participants who stated that they are willing to continue this discussion and development.

European support

Neelie Kroes, the EU Commissioner for Europe's Digital Agenda, strongly supports the initiative for the development of new quality indicators. She addressed the international experts at the Rotterdam Colloquium on Quality of Young (Open Access) Journals in a dedicated video message.



The Commission has proposed that Open Access should be the general principle for all publications covered by the next 80 billion euro European Research Programme, Horizon 2020. Commissioner Kroes states that - rather than "Brussels" imposing rules and guidelines - it should be the scientific community itself that finds ways to encourage Open Access, whether this involves using the leverage of peer review, the composition of editorial boards, or bibliometric indicators. "Ultimately," she says, "we want prestigious Open Access journals that attract the finest research and the finest scientists and help young Open Access journals to grow into established and esteemed publications. But how do we build trust and stimulate their use? It is up to the scientific community itself to determine what will work best."

Latest modifications 10 Sep 2013