Ethical considerations around watchlists (COPE Forum, March 2024)

Apr 2, 2024 | Scholarly publishing

The March 2024 COPE Forum began with a discussion on ‘ethical considerations around watchlists’, hosted by Daniel Kulp (Chair, COPE Council).

During the discussion, the benefits and potential risks associated with watchlists were explored from the position of both individuals and journals, as well as how publishers should use them. In principle, a watchlist could be used as a quality control tool; usually kept private as a measure of accountability. If, however, the list was exposed/leaked, the publisher’s trustworthiness and maker of transparency can be held to account.

It is important to note what is included on a watchlist – is information recorded about an individual, including personal data? Do COPE offer any guidance on what kind of criteria individuals should meet to be on a watchlist? And how can an individual defend themselves or “remove” themselves from a list? These were some of the key questions raised during the conversation regarding individual authors or account holders. The threat is much greater to be named and listed; as careers, research programs and scholarly publications can all be potentially be disputed/disrupted if this extremely sensitive information is exposed in the public domain.

At an individual level, abhorrent practices to circumnavigate peer-review processes could be observed through “flagging” an author’s account – for example, to collate information about suspicious activity. However individuals have a higher right to privacy and such lists need to be regulated or kept private within the publishing house. Academic publishing is witnessing a systematic effort to undermine peer review; the rise of papermills, reviewer cartels and bad actors exploiting the system often for financial gain mean that individuals should be aware of this and make due considerations during submission.

Sharing nefarious behaviours is important and if publishers share systematic problems regarding behaviour amongst one another, they can collectively disrupt the ways papermills and peer-review rings operate. COPE has guidance on how EICs can share information. However, how can we share this information more broadly? The STM Integrity Hub offers a duplicate submissions check across publishers, which could be a useful tool to manage misconduct. This is being developed and the STM Integrity Hub are exploring managing this in an ethical and legal way.

One of the biggest risks with watchlists posed at an individual level is that account holders are added incorrectly or without proper reason. This could be very detrimental to an individual in terms of their career and may also possibly be defamatory and could have legal ramifications. Watchlists must aim to be transparent but not give away details of what we are looking for. The risk is higher for individuals as it can directly impact future employment and ability to do research. Should there be mitigation circumstances for appeal for minor infractions? A similar name could be a risk, for example.

In the absence of an agreed decision amongst the industry on what should and shouldn’t be on a watchlist, the main risk appears to be a legal one. STM have already advised that the industry needs to be careful regarding the type of data which is shared. Legally underpinning this stage is crucial to ensure an individual’s privacy and GDPR-compliance is maintained. It was suggested that if private watchlists are maintained, legal counsel should be able to advise how to evoke sanctions or prevent someone from publishing. The publisher however must be mindful of any reputational damage or litigation brought against them.

If an individual finds themselves on a watchlist they do of course have the right to reply, to ensure that the process is fair and transparent and that they can defend themselves and their position. It was raised that it would be good practice to notify an individual that concerns have been identified, which will allow them to ask for an appeal. It is important to consider and understand what the watchlist represents.

As the risks increase and the systematic manipulation of peer review is attacked, the ways in which this data is captured in order to identify patterns of misconduct has to be carried out quickly, ethically and legally in order for the publisher to maintain integrity and be commercially viable as well as managing its own trustworthiness.

ALPSP: AI = Friend or Foe for Protecting Research Integrity?

Feb 6, 2024 | Scholarly publishing

The ALPSP panel for this discussion on AI and the impact it is having within academic publishing was made up of Nicola Davies, IOP Publishing (Co-chair), Helene Stewart, Clarivate (Co-Chair), Meurig Gallagher, University of Birmingham (Speaker), Matt Hodgkinson, UKRIO (Speaker) and Jennifer Wright, CUP RI Manager (Speaker).

It was fascinating to see how the conversation around AI has moved on within a few months regarding this technological advancement. Institutions, publishers and journal stakeholders all have a concept of AI and are developing policies and guidance about how we should be using it and are underpinning the “what for?”.

Many of us by now will have tried asking a Large Language Model (LLM) to write a paragraph or create an image using Artificial Intelligence. It’s brilliant to watch how quickly tasks can be created, large blocks of text are generated at an immense speed, and right before us we see how quickly human intelligence can be mimicked.  This notion was highlighted in Meurig Gallagher’s presentation; that essentially AI is trying to act as human as possible using the instructions it has been provided with. However, when these tools are posed with mathematical equations it does not have the knowledge to apply the learning and therefore can spectacularly fail! These “gaps” therefore build into the guidance stakeholders need to be aware of when creating policy around AI – it cannot be relied upon solely to do the work. Matt Hodgkinson developed this further and shared many caveats that researchers and general users might come up against when using chatbots:

• Many LLMs are unvalidated for scholarly uses.
• References should be fact-checked as they can be falsified, therefore it is important to check sources and supporting literature.
• The quality of evidence is not assured.
• Outputs may be based on using out-of-date information based on “old” training material.

The ominous but noteworthy warning was circulated that “if you are not an expert, you will be fooled by fluent but incorrect outputs”.  Therefore, all of us involved in scholarly publishing need to be mindful of these contributions and check author statements within articles to assess whether an LLM has been used. Of course, one of the largest threats we are witnessing is the output of paper mills and their use of AI could lead to the tool’s collapse as its knowledge bank is infiltrated with “fake” data, which if left undetected will pollute the pool where the data is extracted from.

Nonetheless the principles of Research Integrity can be applied to the use of AI-generated content and Matt shared this slide to disclose how these principles are applied:

UKRIO presentation at ALPSP 2024

Dr Jennifer Wright from Cambridge University Press shared with the audience how to implement transparency which is really the crux of what many VEOs are looking at. It was suggested that AI declarations should be included within image captions, acknowledgements, and methodologies if applicable and the details that should be shared include the type of model that was used, eg: CHATGPT, and how and when it was accessed. It is also important to include any additional COI statements because of the use of the model. Looking forwards, Dr Wright elaborated on future considerations and posed some important questions around reporting standards: What will the impact be of AI on the scholarly record? How could/should/will research and publication practices change? How will concepts such as retractions be enforced? Can a bot retrain itself?

The challenges are still clearly evident with AI. However the more we progress and understand how it can be used, trust markers can be identified to validate the outputs. As long as scholars use and do not abuse the tech, we could watch something incredible unfold!