False claims in a widely-cited paper

A couple months ago we had a post, This paper in Management Science has been cited more than 6,000 times. Wall Street executives, top government officials, and even a former U.S. Vice President have all referenced it. It’s fatally flawed, and the scholarly community refuses to do anything about it. which was about, ummm, a fatally flawed but very influential paper in Management Science.

The paper in question claimed to find that “High Sustainability companies significantly outperform their counterparts over the long-term, both in terms of stock market and accounting performance,” and I conjecture that one reason for the paper’s great success was that it was pushing a feel-good message that would be popular all over the political spectrum: for the left, it’s evidence in favor of environmental and social sustainability; for the right, it’s an example of the success of the free market, implying that if you care about sustainability, you can get it without government regulation; and, for the center, it’s a message that the system works. It fits in just fine with the baseline smug business-school ideology that firms do well by doing good.

The above story came from my occasional collaborator Andy King, a business school professor himself but of a more disagreeable variety (just as I’m a disagreeable social scientist).

A couple days ago King sent me a followup email:

I would love to get your thoughts and advice on correcting a misreported study.

The publication in question is Eccles, Ioannou, and Serafeim (2014), “The Impact of Corporate Sustainability on Organizational Processes and Performance,” published in Management Science. It is cited roughly 2,000 times per year and has had considerable influence on investment practice and public policy. It is the most cited publication in MS since 2006.Unfortunately, the method described in the paper is not the method the authors actually used. The authors finally acknowledged this in September 2025, after two years of pressure. Yet they have refused to submit a corrigendum.

I have been in contact with the journals, Management Science, but their policies allow only authors to request corrections. They did allow me to submit a comment for review, since they judged the authors non-responsive, but it must go through a lengthy review process.

I have also contacted Research Integrity Offices, as I believe this constitutes an ongoing violation: the authors are knowingly refusing to correct an acknowledged misreport in their study.– London Business School (Ioannou) claims there is no violation because he did not conduct the analysis. (To me this seems irrelevant to the issue of correcting a misreport.)

– Harvard Business School (Serafeim’s employer) has declined to disclose the existence or outcome of any internal review.

– Oxford (where Eccles is currently affiliated) claims Harvard is responsible for Eccles’s actions, since the research occurred when he was at HBS.

– I contacted the UK RIO, but they say they are powerless.

Do you have any ideas about what else I can try?

Also, are things generally this bad, or is it just research from business schools?

My response: Yeah, I’ve pretty much given up on Research Integrity Offices and similar organizations after the two experiences described here (University of California professor does blatant data misrepresentation, no consequences) and here (Cornell professor commits tons of research fraud, eventually he’s forced to leave but it takes a long time, and the university does not respond to outside concerns). Or, closer to home, there’s this story of Columbia University continuing to deny that they misreported their U.S. News data. And the Rutgers political science professor discussed here who got an award from the American Political Science Association for a book with plagiarized material . . . and after the APSA was informed of the plagiarism, they refused to take the award away or even have it shared with the people whose work had been copied.

As I wrote about a couple of these cases:

What’s really bad is when the cheaters do a Lance Armstrong and attack the people who reveal the problem. When engaging in this attack on truth-tellers, the cheaters often play the Javert card, acting as if it’s completely fine to plagiarize, and that their critics are obsessed weirdos. It’s as if all the people that matter are buddies at a country club, and they have to deal with impertinent caddies who call them out on every damn mulligan. They may get even more annoyed at people like us who are members of the club but still side with the caddies.

So, yeah, really disgusting that these guys are still teaching at major business schools.

I think the ultimate solution would be to put all these people into a newly created university, Second Chance U. It could be a pretty amazing place, including all the people mentioned above, along with the mathematician who wrote a chess book that took material for online sources without attribution (not plagiarism, in that plagiarism applies to the wording, not to content, but still way uncool), the disgraced primatologist, the other disgraced primatologist, Dr. Anil Potti, Laurence Tribe, Lawrence Summers, any other Larrys we can dredge up, and various poor unfortunates such as Dan Ariely, who through no fault of his own keeps ending up as a coauthor on papers with fake data. It would be the only university where students are absolutely encouraged to use chatbots to write their term papers!

OK, more seriously, in answer to Andy King’s question: No, I don’t know what to do. I’ll scream about it here, just as I keep screaming about Freakonomics pushing stupid science (see here and here for two of many examples), just as I keep screaming about that stupid physicist and his $100,000 per citation, etc etc etc. It doesn’t seem to be doing much, but that’s all I’ve got.

Speaking up about it is still net positive though. It may not affect whether they get fired, but should affect who hires them next, the grants they win in future, and the reputation of the institutions that harbour them. We don’t have many direct interventions, but the indirect ones have some effect. If not in correcting the public record, at least in reminding others who are tempted to fabricate that you will be deservedly branded as a fraud.

Unfortunately, the contrary evidence is large. The reputational damage does not seem to be having much impact on the journal since they seem unwilling to be accountable for what they publish when errors are pointed out. So, I don’t have much confidence that reputational damage to authors will be any more effective. Still I support efforts to expose such things, although I won’t go so far as to label it a “net positive.”

I still think the effect is cumulative and positive. It was public outcry, rather than internal bureaucratic processes, that did for Gino, for example.

Alright, not relevant, but I’m using Claude for the first time and actually reviewing the output. For survey data analysis, it gives useful tools, but it’s not able to answer the specific questions that the researcher has, no? I had to format the data and visualize it in a certain way, we’re not interested in hypothesis testing in this way. For generating code or information, this is a helpful tool, sure. But has anyone tried like, asking it a specific question with data in a strange format with a specific research question? This is sweet. At SAS I had to do everything manually. I’m toying around with it. I feel stupid I haven’t been using it to generate code, but I had never thought about it. And some problems at work were not related to generating text/code, or summarizing information. Not sure, anyone? I’m not worried about it “taking my job,” I was always coding recreationally, anyway. It’s satisfying. I can do other stuff for money.

The game changer is to use Claude Code, for example via the VScode extension. Give it a Qualtrics Survey File and a preregistration, and it implements your preregistered analysis. Ask it to visualize data, and it will write python code and can do sensible diagnostics (you can force it to use something else, but it has most training data for python). Integrate it with Quarto, and it can write a fully reproducible results section (with all output coming directly from code). Opus 4.6 released about six weeks ago, and I think it offers a substantial improvement for researchers. A lot of people may have tried it early and then given up… but the key with AI is to keep trying as new models get released. It’s hard to predict what they become better at.

ah ok i think my buddy (he’s a computer vision guy, LLM guy) was talking about this, he was saying use a pre-trained one and add additional “vocab” to the corpus… I saw Steve knock out like 10

PRs today, I’m wondering if the devs updated a corpus on the Stan math library, I’m telling on myself, Here. I’ve been in the SAS hole for a while. I’m also seeing a lot more complex Stan models. Did we make a dedicated Stan LLM? And if not, is this a good idea? I’m behind on this kinda research… I might post this on discourse but I don’t wanna keep @‘ing busy people with stupid questions.

What to do? Perhaps this case could provide a reason and some useful material to write a paper about the more general issues. It would not be a specific comment for Management Science about the specific case, but a paper about the more general problem of pointing out possible crucial problems or mistakes in a paper, and the difficulties with that process and possible correction. This specific case could provide useful information, and maybe some structure even, for writing such a paper should this not exist already.

Perhaps some suggestions can be made in the paper as well, for instance about how to deal with possible reactions by authors and journals in the process of pointing out possible problems or mistakes in a paper. Perhaps a proposal for taking certain steps, at certain points, in such a process can be depicted via some diagram and short descriptions (e.g. first contact authors, no reply: next step contact journal, no reply: next step contact integrity office). This could provide others in a similar situation with a possible guidelines and something to refer to, which can even be updated in the case of it being posted online or on some pre-print server.

Some possible suggestions can be made at the end of the proposed steps section or diagram if it led nowhere. For instance, if all the steps led to nothing, it could be suggested that posting a “comment on: Eccles et al., 2014”- style paper on a pre-print server or special website (perhaps something for Retraction Watch or something like that?) might be possible and useful. In case of it being a more official website than a pre-print server, such a (section of a) website will only publish the “comment on: Eccles et al., 2014)” style paper when all the steps (e.g. first contacting authors, journal, etc.) have been taken. So, it can be considered to be a “this is all we can do now” -action. The possible place where this paper will be published can create some more credibility for itself and the process by verifying (to some extent at least) the various steps taken thus far. It might have some effects when posted in such a place when scientists know about the existence of such a place. It can put some pressure on the authors or journal to finally correct things or to prevent such a paper being posted at such a place (that could be seen as negative attention for the journal and something to prevent).

I agree with the idea of writing a more general paper. The comments on this site are helpful to that end.

In terms of posting online, both my replication (https://tinyurl.com/38xp3cp8) and my response to their claim admitting the misreporting (https://tinyurl.com/msne74ep) and their explanation are online.

I have a direct comment in MS under early review.

A

I just checked the posted links, and if I am not mistaken your link to the replication SSRN pre-print is for a manuscript that has been more officially published at Journal of Management Scientific Reports in 2025 (?). It seems that three papers regarding this all are published now at that journal within a few months: your replication in 2025, their explanation in 2026, and your words matter-response to that in 2026. If this is (largely) correct, the more official publishing of your criticisms and their reply might provide even more material for a possible general paper.

And it could even provide additional ways to incorporate this material concerning this case. Maybe one could track citations of the orginal 2014 paper from 2026 onwards, for instance, and see whether the papers citing the original 2014 paper also cite your criticisms and the reply by the original authors. That might show how even publishing criticism in an official journal may not result in that information being referred to next to the original criticized paper when that latter one is being cited.

That in turn might tie in with work showing that retracted paper are still being cited (which I believe exists), in the sense that different forms of corrections (retraction or critical published paper) may not have an effect on the use and interpretation of the original paper.

I am just one person, and I have been working on this for years. Might you consider writing something?

I’m actually 3 kids in a trenchcoat, but still there’s only so much that I can do either.

Quote from above: “I am just one person, and I have been working on this for years. Might you consider writing something?”

I am currently writing a manuscript which I hope to soon post on SSRN to go with my other ones posted there. The most recent one on there is titled “Why psychopathy might be present and prosperous in present-day psychological science” and cites the Steen (2010) paper I mentioned elsewhere in the comments here if I remember correctly. I thought that would be my last manuscript, as I am not even an official scientist with a position in academia. I hope to finish the manuscrtipt I am working on now soon, and after that I intended to have a break from writing.

I do like to contribute to projects or initiatives which I think might be important or useful or whatever, so if you would like me to think some more about things I can try but I am not sure I have more to offer than the thoughts I shared already here. I don’t have much experience with academia or publishing so from that perspective my knowledge about things is very minimal. I can just think about stuff in a more general way I reason.

Concerning your comment: it is perhaps precisely because you have been working on this for years that you might be the perfect person to initiate and first-author a possible paper. You may have a unique combination of knowledge and experience and material through all of this I reason that may be interesting to further ponder. It’s not something that should feel like another task or another thing to add to a possible to-do-list I reason, it was just a thought that I like to share should it be useful for something.

The Lance Armstrong analogy is apt in many ways. Many cyclists at the time were in on the doping, and I suppose they felt that since “everyone” is doing it, or since others who are also doing it are getting away with it, why me? It’s not fair! It’s a vendetta against me that you’re going after me. Why don’t you go after the other ones?

All these examples reflect poorly on “peer review”. It’s hard to imagine how the peer reviewers missed that the published results were not generated by the method described. If it is easily missed, that really does point out the impotency of such reviews, doesn’t it?

Andrew- Perhaps King could write something up for Econ Journal Watch, of which you’re a coeditor. So many journals are bad actors in cases like this that some outside venue like that is needed both for formal corrections and to pressure the original journals to change their behavior. We need more “journal watch” journals, especially in medical and biological science.

I will try this. Suggestions welcome.

Have you noticed problems with Summers’ academic work?

I’ve noticed problems with just about everything else he’s done or said.

“I’ve noticed problems with just about everything else he’s done or said.”

That brings me to another option to put in the possible paper about how to deal with unresponsive authors and journals: the case-study paper about the work of a certain author.

It could be sort of like what you sometimes see when all the work from a fraudster is inspected and strange things or patterns become more clear. In this case however, there is just something wrong or strange, and not official fraud is suspected but just sub-optimal work. However, in its entirety, certain patterns may become clear.

The reference list could use the asterix thing you sometimes see with meta-analyses if I am not mistaken, to show all the papers the case-study author was involved with. Such a case study might provide useful information about how certain sub-optimal work influences things, how sub-optimal work may connect to other sub-optimal work, and how much might be connected to certain authors.

This case gets me thinking about the problem of retraction-worthy work more generally. Has anyone done a diagnostic overview using a large sample of cases? Something like, here are the common ways errors and misrepresentations show up. Or here are some standard tests you can employ to make serious problems more detectable. Maybe it could be useful for reviewers, or even journal editors.

I know of a paper by Steen (2010) titled “Retractions in the scientific literature: do authors deliberately commit research fraud?”

An excerpt:

“This study reports evidence consistent with the ‘deliberate fraud’ hypothesis: authors of fraudulent retracted papers appear to target journals with a high IF (table 2), often have several other retracted papers (tables 2 and 4), tend to diffuse responsibility across $4 co-authors (table 2, figure 1), delay retracting fraudulent papers (table 2) and collaborate with co-authors who also have other retracted papers (table 4).” (p. 114)

Perhaps journals can simply check all the authors’ names for previous retracted work, should this not be done already and should this make sense. That might provide a real continuous and negative consequence of having a retracted paper, whatever the reason (error or fraud). If journals would check this, the extra possible scrutiny that may be a result of this checking might be something that is perceived as negative by the author with one or more previsouly retracted papers. And the possible extra scrutiny could make it less likely that an error in one retracted paper turns into fraud in another retracted paper (should there be a correlation in certain cases), or that a fraudster could (co-) author more retracted papers. But I guess in the case of papers retracted due to fraud the chances are that the author will not be submitting any new work because they got fired or quit.

I think I have heard that possible plagiarism in submitted manuscripts is sometimes (or mostly?) being checked by many journals (?). That might be one thing that is presently being done by journals.

Honestly, when I encounter an unethical work problem, I find no source to report it or someone to care enough to investigate it. We live in a very difficult time where power is given to the fittest, the richest, the most outspoken, the native….and the list goes on. I am tired.

Off topic, but did you notice that Marc Hauser’s in the Epstein files? He asks Epstein for advice about how to handle the misconduct investigation again him and later for advice on his new business which involves helping at-risk youth (sounds legit!)

Hauser: jeffrey good meeting with the dean. i spoke about my responsibility, about feeling awful for the pressure, and my openness to doing whatever i could to help reduce the pressure on him, the university, and corporation. i mentioned that i had heard about pressure to fire me, and i wanted to do what i could to hold my job as i love harvard, and want to continue to contribute. he said that he had been explaining to all those who had asked about dismissal that harvard had followed their procedures, and the decision about the case was final. he said he was sticking to his decision. we repeatedly came back to the issue of what i could do, and he repeatedly supported my staying quiet. he understands that i am not in the lab doing research, and has conveyed this.

i had a thought: is there any way you might propose a meeting between me and one ore more of the board members that you know? i realize the odds are low here, but perhaps if some humanity was inserted into this, and then realized how crushed i am by all of this, that i am not sitting around at home having a great time and plowing ahead as if there were no sanctions, they would accept this.

Epstein: no chance, let him take the heat

Anon:

Yeah, we have a whole bit about Hauser and Epstein in a post from a couple months ago.

But I hadn’t caught the connection between Epstein and the at-risk youths. How horrible!