With my modest experience as an expert witness in courts, I am against this. It is very difficult to make judges understand, for example, how science works - one can only be so sure, and we go ahead with that, for example, in medicine, as well as cryptography. Judges have a hard time appreciating this nuance.
I agree that something stricter should be done, but it should not be about bringing the legal system into play. I see a fundamental issue with bringing science to trial courts, where rhetoric, appeals to emotions, and other different priorities are paramount, not technicalities about overenthusiastic interpretations, data fudging, p-hacking, empirical anomalies and wilful data manipulation.
Science works by different norms of truth (I would call this statistical) than the judicial system does (beyond reasonable doubt/preponderance of evidence). I believe an international peer scientific committee ostracising a person from publication for X number of years, or forever, might be a better measure than a criminal trial and punishment in open court.
I think if it were to make sense, it would have to be similar standards to perjury: wilfully making a materially false statement about your data or process in a published paper. I.e. only targeting outright fraud in statements of fact about what you did and observed. The conclusions from that data would not make sense to include in that umbrella: firstly because as you mention they are difficult to judge and are in fact for pretty much any paper up for debate, and secondly because that's the part that is expected to be assesed by the process of peer review and pulishing anyway.
I'd say that at the moment there's a bit of an issue with the way the community handles this kind of thing, in a way which is akin in structure (I'm not comparing severity/morality) to sexual assault in many communities (science also among them): it's sadly common that someone is widely known or suspected within their field to engage in scientific fraud, but it's only known within that because that person has enough power to make it dangerous to overtly make an accusation, as well as a general fear that it will discredit the field in general. And someone with a bad reputation there still often gets to engage with the community. It seems that only in the really high-profile cases are there actual consequences, and even then they often only come out long after the offender has retired.
(I'm not entirely convinced criminalising it will actually reduce the problem, though. The idea that harsher punishments = less misbehaviour is a bit of a fallacy in part because people who do this don't expect to be caught)
Courts and judges may not understand how science works, but they understand how false statements work. They have centuries of experience with that. Libel, slander, lying under oath, false testimony, perjury... they've seen it all, over and over and over and over.
Willful data manipulation? They may see it more often with financial data, but they've seen it, rather often. And they aren't finance experts either, but they still deal with it competently.
The alternative is that science exists outside the legal system. A scientist can engage in misconduct in a way that gets believed for a while and results in multiple deaths, with no legal consequences? That can't be right.
The legal system is incompatible with this very concept. The legal system works on precedent. Once something is established by the court, it is treated as fact, forever, with an EXTREMELY high bar to get changed. US courts still impose polygraph testing (at significant cost to those forced to take them) on people on probation even though polygraphs are no longer allowed to convict and just plain don't work.
It's not about crimes against science any more than financial fraud is about crimes against money. Science fraud is about falsification, same as financial crime.
I couldn't agree more. Here's a hack to do it without any change in legislation. Scientific journals should require the standard boiler-plate of an affidavit form to be added to the article itself and signed and witnessed as part of the publication process. In this manner, every submitted article is also a valid affidavit admissible in court, and knowingly making a statement which is false or misleading in a material particular while under oath is the crime of perjury.
As I once said:
"Perjury must be a crime. There is only one sin in science, and that sin is faking data, and faking evidence is faking data. Perjury is surely a crime."
I'll leave out the unfortunate context in which this needed saying.
Maybe we should treat faking scientific evidence more as a sin than a crime.
E.g. no fines or prison, but "banishment" from scientific circles until the perpetrator repents publicly, explains all the details and asks for absolution.
This would be somewhat shameful, but people fear shame more than death, and there would also be a path towards restoration + more knowledge of how the fraud actually worked and what led to it.
I think it would lead to worse outcomes, and is likely to end up based more on whether people like the person accused, or like their results than on what they actually did.
I agree that it is a tricky mechanism and that there is a potential for abuse. Social network-like pseudonymous mob shaming shouldn't be replicated elsewhere.
That said, what the culprits do is really very shameful and professional organizations used to treat shameful behavior of their members in this way, rather than handing them over to standard public courts.
At the very least, if other scientists are publicly involved, the public gets an impression that the community doesn't "wash its collective hands" over its own bad actors and tries to remedy the problem actively.
Maybe both should be combined. "A trial by your peers" would be trial by a non-judicial panel made of other scientists (who are the real peers of the culprit), and it could only mete our punishments of specific and relevant type: research bans, public apologies etc.
I certainly don't think that people should be jailed for scientific misconduct. Prisons are useful in isolating dangerous people from the rest of the society, but their restorative effect is minuscule even in Europe, much less in the USA. And fraudulent scientists don't have to be isolated from society, unlike rapists or muggers. They just have to lose their credibility.
Perjury is in a sense a more offensive crime than the petty acts that fills prisons everywhere, for it is the one crime that makes criminals of us all. And it is far worse when it is done by a member of a professional class that enjoys the benefit of being assumed credible rather than criminal. Faking data is in a sense much worse than overtly punching someone in the face, because the number of lives I can adversely affect by punching someone in the face is just one, and the number of people I can probably get away with punching in the face before being caught is also just one.
I see your point about wanting to reserve prison for strictly violent offenses, but there is such a thing as serious yet non-violent crime, which is the entire reason why low-security prison is there.
One of the problem with jailing people is that their work ability is no longer available to the rest of the society, or only in a reduced form.
Scientists, even dishonest ones, are very highly educated people with important and hard to replace skills. Locking those skills away is a huge loss, much higher than when you lock up a run-of-the-mill Ponzi schemer.
I would prefer some punishment that would still make use of their talent and skills to benefit humanity. Perhaps they should be required to design better research methods that prevent their own sort of fraud etc.
I am merely presenting it in an honest manner as having more to it than just rhetoric made up on the spot from the comfort of my keyboard, while also not insinuating the credibility of being able to quote other people who agree.
I am sympathetic to the fear that this would lead to scientists being thrown in prison for technicalities or even misunderstandings. Nobody wants that.
We also don’t want a cottage industry of performative, ladder climbing researchers siphoning funding from real ones.
At a minimum, the public could stop funding researchers with faked data or images. It’s unacceptable that the NIH keeps funding known frauds. If you doctor images, you’re done.
Technically, sanctions are already applied by federal funders (e.g. multiple-year funding bans) [1]... although in practice retractions don't happen, so researchers still benefit if you don't do due diligence about your authors (which I don't think is easy anyway.)
Another way of looking at it is what if this was just a legit unintentional scientific error. The scientific process is not suppose to be so fragile that a paper by an individual causes it to go astray for 16 years.
Fraud is already illegal, and there isn't really any argument made here that scientific fraud should be handled differently from the generic type, indeed most of the author's "response to objections" are that scientific fraud should be treated exactly the same as other fraud.
A fair point here is the impact of the crime. As the author puts it: "if it delayed a successful treatment by just 1 year, I estimate that it would have caused the loss of 36 million QALYs (Quality Adjusted Life Years), which is more than the QALYs lost by Americans in World War II."
Of course these are just the wildly successful instances of such fraud. Many such fraud probably represent just one more quickly ignore paper. But still, the crime matters because it has this potential to derail a field. The magnitude of the crime is not represented by calling it "fraud".
This would close down whole departments and wouldn't solve the problem of the wrong ranking system by citation.
A more important step towards truth would be to rank research based on accuracy of predictions and independent experiment reproductions.
Science itself would have been considered a form of intellectual misconduct at one point. It would be important to be very careful here with definitions and scope, Eg, limiting it to fake data
The independent scientific misconduct board Denmark implemented seems like a good first step [1].
Given it relies on institutions (I assume this includes universities) sending it complaints, I’m not sure how we get over the conflict of interest mentioned.
> Sylvain Lesné, the lead author on the Alzheimer’s paper, remains a professor at the University of Minnesota and still receives NIH funding
Okay, maybe you permit anonymous complaints. If the anonymous complaint results in an adverse finding, the institution is penalised. If not, just the researcher.
Penalties should include fines. But also a term during which they are blacklisted from NIH funding.
If one fraudulent paper can derail a field that much, it's only because we have other, larger problems. Such as funding hegemony with the big researchers and big labs.
TFA tries to claim that this single paper misled the field for 16 years, but that’s an incredibly strong claim for which it offers no real evidence. In fact, in the first few paragraphs the author admits that there were many other papers that drove scientists to pursue this hypothesis. The fraud is clearly tragic, but the tenuous arguments made in this piece are pretty flimsy.
The nature of science is that there are a lot of papers. Not all are replicated but generally “we pursued this direction for 16 years” type of hypothesis are based on many results, which acts as an error correcting code on mistakes and fraud — and is exactly how the system is supposed to work.
Blog post author here. The paper was the 4th most cited paper in Alzheimer's research since 2006. So I feel reasonably confident that if it had never been written, some researchers at the margin would have chosen to work on other hypotheses instead, and perhaps those other avenues would have been more fruitful.
How much time could have been saved towards an effective treatment? It could be as high as a decade, but of course more likely it was zero years. I averaged it out to 1 year.
Now suppose you think that 1 year is orders of magnitude too high, and that in expectation it averages out to a 1 day delay. Even then, I estimate 100,000 QALYs would be lost, making this a tragically high impact case of misconduct.
---
Final point: Nobody doubts that science is error correcting. The point is that the errors are corrected far too slowly and many never get corrected at all. It's incredibly hard to develop good theories when you know that 30-50% of the results in your lit review are false.
...and no incentives for publishing negative results, or doing replication work...
Still these people are criminals and clearly need to be treated as such. To say there are larger problems doesn't at all diminish the need to deal with criminals as we do in ALL walks of life.
There are other deterrents possible than this author’s vendetta has
Maybe civil charge to pay back the public funding, in the case of there being public funding. Turning researchers into debt serfs may reach better compliance and sustain access to their cognitive abilities on behalf of the public, as opposed to prison
Forcing the slabos (I created that word just because we needed one) for doing science for free in advance, and creating incentives to blackmail them to return the money later is the wet dream of any politician.
In the end there could be the undesired collateral effect of nobody wanting to do science anymore by the perceived risk. Only the very rich people could afford the lawfare
As somebody that was required to pay an extra of 700€ before to be allowed to read my thesis, in concept of "insurance" (that, surprise, surprise, the university never provided but still billed) I can tell you that this is not very realistic.
It just adds a new member in the table sucking researchers dry. Who will pay for this insurance?, the grants earned by this scientific? Then this is less money that goes to the researcher one year later. Those that hire researchers will need to either increase the funds or to allow some free hours in the contract so the researcher can double jobs working in McDonalds.
Not knowing much about the jurisprudence here, why would wire fraud not already make scientific misconduct illegal, with the determination about standards of evidence (not necessarily with intent) left to the courtroom?
One side effect would probably be, that anti-science rulers would turn this into a weapon against unliked scientific results. Think vaccines. The Damocles sword of endless litigation would hang over every scientific research concerning public policy.
Maybe it's overall better to let science sort out science things.
Sounds familiar to how the Catholic Church handled the abuse scandal… which was not very well at all. Science has proven, like many other rotten institutions, that it cannot police itself.
Probably a complex question. Government funders, universities, and journals seem to have a civil case against science frauds. Some might rise to criminal fraud levels. But there is no SEC for science. Maybe tucked away somewhere there are criminal investigators in a science agency, as in the SEC?
Imagine if all research had to go through an HR style cover your ass system before any of it could be shared. You would basically shut down all multi-institution research.
That's more on the field than the fraud, I fear. Papers can make all kinds of serious errors without any fraud, just due to ignorance, incompetence, or error.
If one fraudulent actor is massively setting back a field then there is too much trust in published results.
With my modest experience as an expert witness in courts, I am against this. It is very difficult to make judges understand, for example, how science works - one can only be so sure, and we go ahead with that, for example, in medicine, as well as cryptography. Judges have a hard time appreciating this nuance.
I agree that something stricter should be done, but it should not be about bringing the legal system into play. I see a fundamental issue with bringing science to trial courts, where rhetoric, appeals to emotions, and other different priorities are paramount, not technicalities about overenthusiastic interpretations, data fudging, p-hacking, empirical anomalies and wilful data manipulation.
Science works by different norms of truth (I would call this statistical) than the judicial system does (beyond reasonable doubt/preponderance of evidence). I believe an international peer scientific committee ostracising a person from publication for X number of years, or forever, might be a better measure than a criminal trial and punishment in open court.
I think if it were to make sense, it would have to be similar standards to perjury: wilfully making a materially false statement about your data or process in a published paper. I.e. only targeting outright fraud in statements of fact about what you did and observed. The conclusions from that data would not make sense to include in that umbrella: firstly because as you mention they are difficult to judge and are in fact for pretty much any paper up for debate, and secondly because that's the part that is expected to be assesed by the process of peer review and pulishing anyway.
I'd say that at the moment there's a bit of an issue with the way the community handles this kind of thing, in a way which is akin in structure (I'm not comparing severity/morality) to sexual assault in many communities (science also among them): it's sadly common that someone is widely known or suspected within their field to engage in scientific fraud, but it's only known within that because that person has enough power to make it dangerous to overtly make an accusation, as well as a general fear that it will discredit the field in general. And someone with a bad reputation there still often gets to engage with the community. It seems that only in the really high-profile cases are there actual consequences, and even then they often only come out long after the offender has retired.
(I'm not entirely convinced criminalising it will actually reduce the problem, though. The idea that harsher punishments = less misbehaviour is a bit of a fallacy in part because people who do this don't expect to be caught)
Courts and judges may not understand how science works, but they understand how false statements work. They have centuries of experience with that. Libel, slander, lying under oath, false testimony, perjury... they've seen it all, over and over and over and over.
Willful data manipulation? They may see it more often with financial data, but they've seen it, rather often. And they aren't finance experts either, but they still deal with it competently.
The alternative is that science exists outside the legal system. A scientist can engage in misconduct in a way that gets believed for a while and results in multiple deaths, with no legal consequences? That can't be right.
The legal system is incompatible with this very concept. The legal system works on precedent. Once something is established by the court, it is treated as fact, forever, with an EXTREMELY high bar to get changed. US courts still impose polygraph testing (at significant cost to those forced to take them) on people on probation even though polygraphs are no longer allowed to convict and just plain don't work.
"It is very difficult to make judges understand, for example, how science works"
Isn't that true about white collar crime in general? Some frauds or tax evasion schemes are very elaborate. We still (try to) prosecute them.
It's not about crimes against science any more than financial fraud is about crimes against money. Science fraud is about falsification, same as financial crime.
I couldn't agree more. Here's a hack to do it without any change in legislation. Scientific journals should require the standard boiler-plate of an affidavit form to be added to the article itself and signed and witnessed as part of the publication process. In this manner, every submitted article is also a valid affidavit admissible in court, and knowingly making a statement which is false or misleading in a material particular while under oath is the crime of perjury.
As I once said:
"Perjury must be a crime. There is only one sin in science, and that sin is faking data, and faking evidence is faking data. Perjury is surely a crime."
I'll leave out the unfortunate context in which this needed saying.
Maybe we should treat faking scientific evidence more as a sin than a crime.
E.g. no fines or prison, but "banishment" from scientific circles until the perpetrator repents publicly, explains all the details and asks for absolution.
This would be somewhat shameful, but people fear shame more than death, and there would also be a path towards restoration + more knowledge of how the fraud actually worked and what led to it.
So mob justice instead of trial?
I think it would lead to worse outcomes, and is likely to end up based more on whether people like the person accused, or like their results than on what they actually did.
Social shame is a tricky mechanism.
I agree that it is a tricky mechanism and that there is a potential for abuse. Social network-like pseudonymous mob shaming shouldn't be replicated elsewhere.
That said, what the culprits do is really very shameful and professional organizations used to treat shameful behavior of their members in this way, rather than handing them over to standard public courts.
At the very least, if other scientists are publicly involved, the public gets an impression that the community doesn't "wash its collective hands" over its own bad actors and tries to remedy the problem actively.
Maybe both should be combined. "A trial by your peers" would be trial by a non-judicial panel made of other scientists (who are the real peers of the culprit), and it could only mete our punishments of specific and relevant type: research bans, public apologies etc.
I certainly don't think that people should be jailed for scientific misconduct. Prisons are useful in isolating dangerous people from the rest of the society, but their restorative effect is minuscule even in Europe, much less in the USA. And fraudulent scientists don't have to be isolated from society, unlike rapists or muggers. They just have to lose their credibility.
Perjury is in a sense a more offensive crime than the petty acts that fills prisons everywhere, for it is the one crime that makes criminals of us all. And it is far worse when it is done by a member of a professional class that enjoys the benefit of being assumed credible rather than criminal. Faking data is in a sense much worse than overtly punching someone in the face, because the number of lives I can adversely affect by punching someone in the face is just one, and the number of people I can probably get away with punching in the face before being caught is also just one.
I see your point about wanting to reserve prison for strictly violent offenses, but there is such a thing as serious yet non-violent crime, which is the entire reason why low-security prison is there.
One of the problem with jailing people is that their work ability is no longer available to the rest of the society, or only in a reduced form.
Scientists, even dishonest ones, are very highly educated people with important and hard to replace skills. Locking those skills away is a huge loss, much higher than when you lock up a run-of-the-mill Ponzi schemer.
I would prefer some punishment that would still make use of their talent and skills to benefit humanity. Perhaps they should be required to design better research methods that prevent their own sort of fraud etc.
Pretty incredible to quote yourself in your own post.
I am merely presenting it in an honest manner as having more to it than just rhetoric made up on the spot from the comfort of my keyboard, while also not insinuating the credibility of being able to quote other people who agree.
I am sympathetic to the fear that this would lead to scientists being thrown in prison for technicalities or even misunderstandings. Nobody wants that.
We also don’t want a cottage industry of performative, ladder climbing researchers siphoning funding from real ones.
At a minimum, the public could stop funding researchers with faked data or images. It’s unacceptable that the NIH keeps funding known frauds. If you doctor images, you’re done.
Technically, sanctions are already applied by federal funders (e.g. multiple-year funding bans) [1]... although in practice retractions don't happen, so researchers still benefit if you don't do due diligence about your authors (which I don't think is easy anyway.)
[1] https://retractionwatch.com/2023/12/21/what-analyzing-30-yea...
Another way of looking at it is what if this was just a legit unintentional scientific error. The scientific process is not suppose to be so fragile that a paper by an individual causes it to go astray for 16 years.
Yes, we can generate an infinite number of interpretative frames. But what principle do we have to sort good from bad?
Fraud is already illegal, and there isn't really any argument made here that scientific fraud should be handled differently from the generic type, indeed most of the author's "response to objections" are that scientific fraud should be treated exactly the same as other fraud.
A fair point here is the impact of the crime. As the author puts it: "if it delayed a successful treatment by just 1 year, I estimate that it would have caused the loss of 36 million QALYs (Quality Adjusted Life Years), which is more than the QALYs lost by Americans in World War II."
Of course these are just the wildly successful instances of such fraud. Many such fraud probably represent just one more quickly ignore paper. But still, the crime matters because it has this potential to derail a field. The magnitude of the crime is not represented by calling it "fraud".
This would close down whole departments and wouldn't solve the problem of the wrong ranking system by citation. A more important step towards truth would be to rank research based on accuracy of predictions and independent experiment reproductions.
Science itself would have been considered a form of intellectual misconduct at one point. It would be important to be very careful here with definitions and scope, Eg, limiting it to fake data
The independent scientific misconduct board Denmark implemented seems like a good first step [1].
Given it relies on institutions (I assume this includes universities) sending it complaints, I’m not sure how we get over the conflict of interest mentioned.
> Sylvain Lesné, the lead author on the Alzheimer’s paper, remains a professor at the University of Minnesota and still receives NIH funding
Okay, maybe you permit anonymous complaints. If the anonymous complaint results in an adverse finding, the institution is penalised. If not, just the researcher.
Penalties should include fines. But also a term during which they are blacklisted from NIH funding.
[1] https://ufm.dk/en/research-and-innovation/councils-and-commi...
If one fraudulent paper can derail a field that much, it's only because we have other, larger problems. Such as funding hegemony with the big researchers and big labs.
TFA tries to claim that this single paper misled the field for 16 years, but that’s an incredibly strong claim for which it offers no real evidence. In fact, in the first few paragraphs the author admits that there were many other papers that drove scientists to pursue this hypothesis. The fraud is clearly tragic, but the tenuous arguments made in this piece are pretty flimsy.
The nature of science is that there are a lot of papers. Not all are replicated but generally “we pursued this direction for 16 years” type of hypothesis are based on many results, which acts as an error correcting code on mistakes and fraud — and is exactly how the system is supposed to work.
Blog post author here. The paper was the 4th most cited paper in Alzheimer's research since 2006. So I feel reasonably confident that if it had never been written, some researchers at the margin would have chosen to work on other hypotheses instead, and perhaps those other avenues would have been more fruitful.
How much time could have been saved towards an effective treatment? It could be as high as a decade, but of course more likely it was zero years. I averaged it out to 1 year.
Now suppose you think that 1 year is orders of magnitude too high, and that in expectation it averages out to a 1 day delay. Even then, I estimate 100,000 QALYs would be lost, making this a tragically high impact case of misconduct.
--- Final point: Nobody doubts that science is error correcting. The point is that the errors are corrected far too slowly and many never get corrected at all. It's incredibly hard to develop good theories when you know that 30-50% of the results in your lit review are false.
...and no incentives for publishing negative results, or doing replication work...
Still these people are criminals and clearly need to be treated as such. To say there are larger problems doesn't at all diminish the need to deal with criminals as we do in ALL walks of life.
There are other deterrents possible than this author’s vendetta has
Maybe civil charge to pay back the public funding, in the case of there being public funding. Turning researchers into debt serfs may reach better compliance and sustain access to their cognitive abilities on behalf of the public, as opposed to prison
Forcing the slabos (I created that word just because we needed one) for doing science for free in advance, and creating incentives to blackmail them to return the money later is the wet dream of any politician.
In the end there could be the undesired collateral effect of nobody wanting to do science anymore by the perceived risk. Only the very rich people could afford the lawfare
Insurance mitigates
As somebody that was required to pay an extra of 700€ before to be allowed to read my thesis, in concept of "insurance" (that, surprise, surprise, the university never provided but still billed) I can tell you that this is not very realistic.
It just adds a new member in the table sucking researchers dry. Who will pay for this insurance?, the grants earned by this scientific? Then this is less money that goes to the researcher one year later. Those that hire researchers will need to either increase the funds or to allow some free hours in the contract so the researcher can double jobs working in McDonalds.
The principle of moral hazard suggests that paying for insurance makes people more likely to use it.
we’re not going to call it research paper fraud insurance, just professional liability insurance
they are still going to gave the threat of a public records civil charge from the government
The average researcher probably makes under $6 million in their lifetime, before tax. That money is probably never being paid back unfortunately.
The threat of permanently withholding future funding is probably enough to curb casual fraud.
[dead]
Not knowing much about the jurisprudence here, why would wire fraud not already make scientific misconduct illegal, with the determination about standards of evidence (not necessarily with intent) left to the courtroom?
One side effect would probably be, that anti-science rulers would turn this into a weapon against unliked scientific results. Think vaccines. The Damocles sword of endless litigation would hang over every scientific research concerning public policy.
Maybe it's overall better to let science sort out science things.
Sounds familiar to how the Catholic Church handled the abuse scandal… which was not very well at all. Science has proven, like many other rotten institutions, that it cannot police itself.
Isn't it already fraud? Usually involving the postal service too?
I do not want this to happen, evem though the scientists were eventually exonerated: https://www.science.org/content/article/why-italian-earthqua...
Publishing a paper isn't necessarily engineering a product, but decisions are made from data, and there are corresponding effects.
Reproducibility is the test suite of science. Make experiments reliably, repeatably testable.
Wonder if computable research will become a requirement for publication. Will that make a slow process slower?
Existing fraud laws should be sufficient, no?
Probably a complex question. Government funders, universities, and journals seem to have a civil case against science frauds. Some might rise to criminal fraud levels. But there is no SEC for science. Maybe tucked away somewhere there are criminal investigators in a science agency, as in the SEC?
Imagine if all research had to go through an HR style cover your ass system before any of it could be shared. You would basically shut down all multi-institution research.
Men with guns have no place in advancing knowledge.
People who fake their results are not advancing knowledge. In the case of the Alzheimer’s paper, it held back knowledge for a very long time.
Beyond that though, these are cases of blatant fraud that steals money from the taxpayer.
That's more on the field than the fraud, I fear. Papers can make all kinds of serious errors without any fraud, just due to ignorance, incompetence, or error.
If one fraudulent actor is massively setting back a field then there is too much trust in published results.
[dead]