Largest Research Integrity Survey Ever Fails As Universities Refuse To Collaborate | Science

By Jop de Vrieze25 November 2020, 3.30 pm

The world’s largest multidisciplinary survey of research integrity risks failing to achieve its goals after two-thirds of invited institutions refused to cooperate, citing the sensitivity of the topic and fearing negative publicity. This led the researchers to lead the Dutch National Research Integrity Survey on their own to scrape many email addresses and solicit responses. The survey will close on December 7, but the team collected responses from less than 15% of 40,000 targeted participants.

“It was supposed to be a collaborative effort, but it ended up like a satellite in the Solar System trying to send signals,” says Gowri Gopalakrishna, a postdoctoral researcher at Amsterdam University Medical Center (AUMC) who is coordinating the € 800,000 survey.

Lex Bouter, who studies research methods and integrity at the Free University of Amsterdam (VU), began planning the survey in 2016 to address the lack of data on questionable research practices and scientific misconduct. He wanted to ask all academics working in the Netherlands not only how they conduct their research, but also about work habits, pressures and other aspects of academic life. Bouter, the former president of the VU himself, assured heads of other universities that the investigation would not generate an institutional ranking of misconduct.

But at a meeting in December 2019, some college presidents argued that a poll wouldn’t be suitable for such a sensitive topic, Bouter recalls. Others found the survey too focused on misbehavior, such as falsifying data or selecting results. “I thought he was biased,” says Henk Kummeling, president of Utrecht University, who declined to attend. “If you ask only questionable research practices, you already know what you will get from the survey.”

Eventually, five of the 15 universities in the Netherlands agreed to cooperate, provided they could have a say in the design and content of the survey. The questionnaire has been expanded to include more questions on desirable scientific practices such as data sharing and open science. After the changes, the presidents of the five participating universities tried unsuccessfully to get others to participate.

In a statement sent to Science, Leiden University President Carel Stolker says her institution refused to participate because the survey is methodologically weak, but did not provide details. Kummeling says his university turned it down because it “didn’t feel the results would be useful for future policy,” but denies it was because of the sensitivity of the topic. “Everything about integrity is sensitive,” he says. “But I have no problem mapping very clearly what’s going on.”

But according to Gopalakrishna, the institutions worried about the bad publicity. “We tried to make the poll more balanced, but we couldn’t shake off the negative image,” he says. Jeroen de Ridder, a philosopher of science at VU who is not involved in the study, says he is disappointed that a unique opportunity is being missed to study the integrity of research across disciplines. He denies that the survey has methodological flaws: “This has become the most accurate and complete survey that anyone could want,” says de Ridder.

Participating universities helped Gopalakrishna by providing staff email addresses and sending reminders to fill in the questionnaire. To get answers from researchers at the other 10 institutions, Gopalakrishna and his colleagues had to scrape email addresses and send out surveys without warning. “This has resulted in a lot more bounced and unopened emails,” he says.

Even some researchers from collaborating institutions, wary of phishing scams, did not open their emails, says AUMC epidemiologist Gerben ter Riet, a co-investigator of the survey. “Researchers are inundated with emails and inundated with polls,” he says. “It is extremely difficult to stand out with something legitimate.”

As of November 24, 13.3% of researchers affiliated with collaborating institutes had responded to the request, compared with 9.5% of those from non-collaborating institutes.

After December 7, the team will need to determine how representative the champion is. Bouter still expects it to be of value. “I’m a little disappointed with the process, but with over 5,000 completed surveys this is still the largest survey to date.”