Research practices and research misconduct

doi:10.21203/rs.3.rs-58772/v1

Download PDF

Research

Research practices and research misconduct

https://doi.org/10.21203/rs.3.rs-58772/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 11 Dec, 2020

Read the published version in Research Integrity and Peer Review →

You are reading this older preprint version

Read the latest preprint version →

Background: Research on research integrity has tended to focus on frequency of research misconduct and factors that might induce someone to commit research misconduct. A definitive answer to the first question has been elusive, but it remains clear that research misconduct occurs too often. Answers to the second question are so diverse, it might be productive to ask a different question: What about how research is done allows research misconduct to occur?

Methods: With that question in mind, research integrity officers (RIOs) of the 62 members of the American Association of Universities were invited to complete a brief survey about their most recent instance of a finding of research misconduct. Respondents were asked whether one or more good practices of research (e.g., openness and transparency, keeping good research records) were present in their case of research misconduct.

Results: Twenty-four (24) of the respondents (39% response rate) indicated they had dealt with at least one finding of research misconduct and answered the survey questions. Over half of these RIOs reported that their case of research misconduct had occurred in an environment in which at least nine of the ten listed good practices of research were deficient.

Conclusions: These results are not evidence for a causal effect of poor practices, but it is noteworthy that good research practices such as those listed would make it more difficult if not impossible for someone in such an environment to commit research misconduct.

Other Public Policy

Good Practices of Research

Responsible Conduct of Research

Research Integrity Officer

Research Misconduct

Much of the literature on research misconduct has focused on the question of why a researcher might choose to engage in “fabrication, falsification, or plagiarism” (U.S. definition of research misconduct: Office of Science and Technology Policy 2000). When cases of research misconduct reached the public eye in the 1980s, the scientific community saw such behavior as rare and likely the result of a few bad apples (Broad 1981). More recently, others argued that the rate of serious misbehavior is as high as 1 in 3 scientists (e.g., Martinson et al. 2005), perhaps an expected outcome of the research environment. Rates of research misconduct per se are probably much closer to 2% (Fanelli et al. 2009). Even if just 2%, it would be hard to argue that these are simply a few damaged individuals. Whether the number is high or low, many have emphasized the importance of pressure in science for funding and publication (e.g., Mumford et al. 2001; Davis et al. 2007; Tijdink et al. 2014). On the other hand, Fanelli et al. (2015) concluded based on a retrospective meta-analysis of retractions that pressures to publish were less predictive of research misconduct than a variety of other environmental factors. One important factor might be the perceived fairness of the system (Keith-Spiegel et al. 2005; Martinson et al. 2006, 2010).

Other studies of those who have been found to have committed research misconduct not surprisingly provide a more complicated picture in which possible factors resulting in misconduct are diverse (Davis et al. 2007; Wright et al. 2008; DuBois et al. 2013). Interestingly, many of the factors identified might be best classified as rationalizations (e.g., “I am a good person so the fact I did this must be because of some external factor such as I was forced to do so by others or the pressures of science”) or symptoms of less than ideal research environments (e.g., lack of mentoring). While these results are of interest, it is important to note that they are extracted from materials submitted in support of findings of research misconduct. Unfortunately, the primary charge to such reviews is typically only to determine if research misconduct occurred, not why it occurred. For example, institutions receiving funding from the Public Health Service are asked only to provide a “statement of findings”:

“Statement of findings. For each separate allegation of research misconduct identified during the investigation, provide a finding as to whether research misconduct did or did not occur…” (Code of Federal Regulations 2011).

Despite the many possible factors that may have precipitated research misconduct, a response often cited as a solution is education (e.g., see Resnik and Master 2013; Yeo-The and Tan 2020). In fact, institutions receiving funding from the Public Health Service are charged with fostering “a research environment that promotes the responsible conduct of research, research training, and activities related to that research or research training, discourages research misconduct…” (Code of Federal Regulations 2011b). However, if it is in fact just a few bad apples, or research misconduct is ubiquitous or a reaction to pressures, it isn’t readily apparent how education or training would be cost effective, much less useful at all. If education is a proposed solution, then presumably it is assumed that the problem is that scientists are insufficiently armed with knowledge or skills to protect them from committing research misconduct. While that is possible, it seems a tenuous conclusion given that the history of science prior to the 1980s was marked by the absence of such training.

It might be of use to reframe the question about research misconduct. Instead of asking “Why would a researcher commit misconduct?”, perhaps it would be of value to ask “How is it possible that a member of the scientific community can commit misconduct?” In short, shouldn’t the very nature of how research is done protect from the risk of misconduct? It may be worth considering the possibility that simply engaging in good practices of research would be a strong protection from research misconduct occurring – independent of the factors contributing to the misconduct. For example, if research is normally conducted in an environment in which researchers are open and transparent with one another, or if the responsibilities of authorship are emphasized at least as much as the credit, or if research records are kept sufficiently so that it is possible to reconstruct what had been done, then it seems any one of these good research practices would make it difficult if not impossible for a member of such a group to commit research misconduct.

The goal in this study is to supplement our understanding of the circumstances under which research misconduct typically occurs by obtaining perspectives of people overseeing research misconduct investigations about the characteristics of research practices in the environment in which research misconduct has occurred.

This project was reviewed and approved for exempt status (45 CFR 46.101(b), category 2) by the UC San Diego Institutional Review Board (#170981XX).

A survey instrument was developed for an audience of research integrity officers (RIOs), individuals with responsibility for receiving and responding to allegations of research misconduct. [NOTE: These individuals are referred to as “respondents” in this manuscript, a term which is sometimes used elsewhere to refer to an individual accused of having committed research misconduct.] The survey was designed to request perceptions of the extent to which research misconduct occurred in the context of good practices of research. The survey was constructed so as to first verify that the respondent did have RIO responsibilities and had “firsthand knowledge of one or more cases of findings of research misconduct in [their] institution.” Those confirming were asked for their judgments regarding 13 factors possibly relevant to the most recent case of a research misconduct finding (Tables 1 and 2). Each statement was scored for degree of agreement (Strongly disagree, Disagree, Neither agree nor disagree, Agree, Strongly Agree) or to be marked as something they didn’t know or was not applicable in the given case. The survey concluded with a request for their overall perception comparing “…the research practices [they] noted above to those in other research groups that have not experienced allegations or findings of research misconduct?” and an opportunity to submit an open-ended response to the prompt for “…additional comments or thoughts either about circumstances associated with research misconduct or this survey...” No personally identifiable information was collected and respondents were advised to not include any such information in open-ended responses.

Respondents to the survey were the RIOs associated with the 62 Association of American Universities (AAU) institutions. For each of these institutions, the RIO or equivalent was identified from the institution’s Web pages. On August 16, 2017, a single, personal e-mail invitation was sent to each of the 62 RIOs requesting their completion of the survey. The survey was closed on September 16, 2017.

Responses were tabulated for reporting as percent of respondents agreeing or strongly agreeing with the statements listed.

A Research Integrity Officer or the equivalent (RIO) was identified for each of the 62 AAU institutions, but it was rarely easy to identify the right person from information on the Web. In many cases, it was necessary to take advantage of the author’s knowledge of the field and alternative key words.

Responses were received from 31 individuals (50% of those invited), all of whom self-identified as having responsibility or oversight for addressing allegations of research misconduct. Twenty-eight (28) of the respondents indicated they had dealt with at least one finding of research misconduct, but four of those did not answer the subsequent questions. All summaries below are based on the remaining 24 respondents (39% response rate).

Table 1. Good practices of research. Percent agreement that selected examples of good practices in research were present in the context of the case of research misconduct. Respondents were not included in the calculated percentages if they noted that they did not know about a particular item or that it was not applicable. The number of total respondents (N) for each question is included in parentheses.

	Agree or Strongly Agree
…open and transparent with each other about their work.	19% (N=21)
…had a good understanding of statistical methods or sought out the necessary expertise.	42% (12)
…considered authorship to be both a credit and a source of responsibility.	39% (18)
…felt empowered to speak up if something didn’t seem right or they had questions.	24% (17)
…leader of the research group/team was a good manager of:
people.	22% (23)
budgets.	33% (12)
the research operations.	43% (23)
the research data.	27% (22)
…designed research studies to protect themselves from the risk of bias.	25% (20)
…kept research records sufficient for others to reconstruct what had or had not been done.	39% (23)

Table 2. Responsible conduct of research training. Percent agreement about whether the person found responsible for committing research misconduct had received one or more forms of responsible conduct of research (RCR) training. Respondents were not included in the calculated percentages if they noted that they did not know about a particular item or that it was not applicable. The number of total respondents (N) for each question is included in parentheses.

	Agree or Strongly Agree
… received adequate mentoring in the responsible conduct of research.	40% (N=20)
[took] one or more in person courses in responsible conduct of research.	64% (14)
[took] one or more online courses in responsible conduct of research (e.g., CITI).	50% (10)

For those cases in which the RIOs rendered judgments, every one of the listed good practices of research were agreed to be present by less than half (19-43%) of the respondents (Table 1). The three categories of practices least frequently seen as characterizing the research environments for findings of research misconduct were that the researchers: (1) were open and transparent with each other about their work (19% agree or strongly agree), (2) felt empowered to speak up if something didn’t seem right or they had questions (24%), and (3) were in a group in which the group leader was a good manager of people (22%) – all examples of people/social issues. The next two areas of greatest concern were both matters of data: (4) designing of research studies to protect from the risk of bias (25%) and (5) effective management of the research data (27%). Conversely, the statements with the highest rates of agreement (Table 2) were that researchers responsible for committing research misconduct had taken one or more in person courses in responsible conduct of research (64%) or online courses in responsible conduct of research (50%).

Over 25% of respondents judged that their case of research misconduct occurred in a context devoid of any of the ten good practices of research listed in Table 1. An additional 26% of respondents concluded that their case was characterized by only one of the ten good practices. Over 90% of the respondents concluded that the context for their research misconduct case was characterized by only five of the ten possible practices. Overall, 90% of the RIOs rated research practices to be worse or much worse than average in the environments in which they had a finding of research misconduct.

The results from this study are consistent with the hypothesis that research misconduct occurs in circumstances in which good practices of research are absent. The premise of this study came from an insight from teaching responsible conduct of research that many recommended practices (e.g., being open and transparent, understanding and appropriate use of statistics, or keeping good records) would in theory make it harder for someone to commit research misconduct. In fact, it is easy to imagine that any one of the listed practices (Table 1) might be a serious impediment to such misconduct. While the present study was not a prospectively designed double blind trial – a study that would be difficult if not impossible to conduct – it does provide some insight into the circumstances present in a few recent cases of research misconduct in some of the leading academic universities of North America.

As summarized above, the environment in which these cases of research misconduct occurred was characterized by deficits in one or more of the selected practices noted in Table 1. In only one of the cases did the respondent note that all of these research practices were in place. Otherwise, more than half of the respondents were able to identify only one of the ten listed good practices as characterizing the circumstances in which the case of research misconduct had occurred. It is noteworthy that these items for the most part are not extraordinary expectations, but rather approaches to research that one might assume would be typical. Fortunately, 90% of the respondents believed that typical research practices were no worse if not better than those seen in these cases of misconduct.

While the list of good practices of research listed above is by no means comprehensive, it is perhaps not surprising that the practices most frequently seen as deficient could all be categorized as people/social issues: being open and transparent, feeling empowered to speak up if needed, and having a group in which the leader is a good manager of people. The next two practices most frequently seen as deficient were very clearly related to data: designing of research studies to protect from bias and effective data management. In all five of these cases, it is easy to see how much harder it would be for research misconduct to have occurred if these practices had been in place.

While protecting from the risk of research misconduct is an important goal, it is not the only possible benefit of improving standard practices of research. As noted by one of the respondents, poor management and oversight can create an environment in which there is an increased risk of questionable research practices:

“The PI managed a large group of researchers, but did not adequately manage or oversee their work to the extent where they were able to identify [questionable] research practices, nor were they able to produce proper research records when requested. Although it was misconduct, it stemmed mostly from careless management, mentoring, and research practices rather than an outright effort to deceive.”

In addition to the intended primary focus on good practices of research as a protection from research misconduct, three incidental observations are noteworthy. First, despite the possible relevance of these good practices to the risk of research misconduct, 8% of the possible answers about these good practices were marked as “Don’t know or don’t remember” (data not shown). It seems as a minimum that such information would be an opportunity to better understand the circumstances that allowed research misconduct to occur. More importantly, this could be a window allowing for better design of options to strengthen the research enterprise and to protect against future cases. Perhaps questions about research practices should more routinely be part of all research misconduct investigations.

A second incidental finding from this study is a reminder that we should not put too much stock in the protective value of training in responsible conduct of research. As summarized above, nearly two thirds of those who had committed research misconduct had taken an in person course in responsible conduct of research and half had taken an online course in responsible conduct of research. Admittedly, the quality of courses may be a factor (Antes et al. 2009), but this finding is consistent with other studies that have emphasized the importance of what happens in the research environment rather than courses per se (Anderson et al. 2007).

A third and final incidental finding from this project is that many institutions likely have room to improve in transparency about where and how to report allegations of research misconduct. Despite the author’s familiarity with the field, it was often challenging to find the right individual. Specific examples were not recorded at the time the study was conducted, but as recently as June 2020 Google searches for “Research Integrity Officer” at the 62 AAU institutions resulted in nine instances (=15%) with no hits. Unfortunately, even those with pages including the words “Research Integrity Officer” provided information about roles and/or responsibilities for the position but failed to clearly identify where the RIO could be found and how to lodge an allegation of research misconduct. To the extent this is true, it is another opportunity for improvement.

This study has a number of limitations that should be noted. First, it is far from comprehensive. Respondents represented only a sampling of North American universities. The survey asked only about the one, most recent case of research misconduct. The questions asked about good practices of research are only a sampling of the many practices that might be relevant to the responsible conduct of research. Nonetheless, the survey answers provided are sufficient to illustrate the principle that such factors are worth considering as being permissive if not supportive of cases of research misconduct.

One oversight of this study was the failure to distinguish between different forms of research misconduct. Although fabrication, falsification, and plagiarism are all considered research misconduct according to a government-wide definition in the US (Office of Science and Technology Policy 2000), plagiarism differs from the other two categories in that it does not necessarily involve a misrepresentation or manipulation of research data. To a large extent, fabrication and falsification were the intended focus for the present survey. Many of the questions clearly represent factors less likely to be relevant to plagiarism than to fabrication or falsification. It isn’t clear how many if any of the respondents were referencing plagiarism cases, but at least one noted the concern and chose to address an earlier case that was not about plagiarism:

“The most recent case involved plagiarism, but I realized that it didn't fit well with your questions, so I switched to an earlier case.”

Although the premise of this study is to call attention to the possibility that the simple adoption of good practices of research would create an environment inhospitable to committing research misconduct, there are clearly cases in which that may be much less relevant. At least two such cases were cited in this study. The first is an instance in which the individual presumably recognized that they were violating accepted norms, but had no respect for them at all:

“The [accused] in this case had no moral compass. They created a story that they felt would play well with the committee members, colleagues and superiors out of whole fabric. It was only when this pathology became apparent, that the members of the committee and panel could step outside of norms of responsible conduct [to recognize] someone who did not respect those norms.”

The second case was arguably someone who perhaps had no connection to questions of morality at all:

“Because this survey was based on my most recent research misconduct case, I am afraid it [is] not very accurate. That case was a confession by someone who systematically changed data, sometimes for no apparent reason.”

Although both of the above cases might be instances in which the individual would not have been swayable by the good practices of their colleagues, it does still seem plausible that (for example) a collaborative and transparent research environment would have made it harder for them to commit their misconduct.

Perceptions of respondents in this study are consistent with an association between committing research misconduct and an environment in which research practices fall short. While these results are not evidence for a causal effect of poor practices, it is noteworthy that good research practices such as those listed would make it more difficult if not impossible for someone in such an environment to commit research misconduct. This perspective aligns with the recognition that an important factor in crime is having the opportunity (Adams and Pimple 2005). This way of thinking about prevention of research misconduct has several notable features. First, fostering a research environment in which most members of the community adhere to good practices of research might fail to dissuade any one individual from committing research misconduct, but such an environment would make it more difficult for them to do so with impunity. Second, this approach is independent of the reasons for which someone might commit research misconduct. Good research practices would prevent, highlight, and/or mitigate the act of misconduct regardless of the reason it was committed. Finally, whether or not research misconduct would be diminished by increased emphasis on good practices of research, it seems clear that the integrity and reproducibility of science could only be helped by improving the way in which we do science. Given the current increased focus on research rigor and reproducibility (e.g., Baker 2016), it is worth asking if we can better design programs (including RCR training) that will foster a climate in which good practices of research will be more widely adopted, thereby decreasing the risk of research misconduct, but at the same time increasing the likelihood of high quality research.

Ethics approval: This research used survey procedures. The UC San Diego Institutional Review Board (#170981XX) has confirmed that no ethical approval is required and the project is exempt under 45 CFR 46.101(b), category 2.
Consent for publication: Not applicable.
Availability of data and material: The survey and all anonymized survey responses are available on request.
Conflicts of interest/Competing interests: The author declares he has no conflicts of interest.
Funding: National Institutes of Health (NCATS/NIH UL1TR001442) and the National Science Foundation (CCE STEM #1835029).
Author’s contributions: MK is solely responsible for the conception and design; data acquisition, analysis, and interpretation; and drafting, revising, and final approval of the manuscript.
Acknowledgements: This project was supported in part by grants from the National Institutes of Health (NCATS/NIH UL1TR001442) and the National Science Foundation (CCE STEM #1835029). Elizabeth Heitman (University of Texas Southwestern Medical Center, Dallas, TX), Francis Macrina (Virginia Commonwealth University, Richmond, VA), Brian Martinson (Health Partners Institute, Minneapolis, MN), Michael Mumford (Oklahoma University, Norman, OK), and Daniel Vasgird (West Virginia University, Morgantown, WV) are thanked for their advice during the preparation of the survey for this project. The opinions expressed in this article are the author’s own and do not necessarily reflect those of these individuals nor the National Institutes of Health, the Department of Health and Human Services, the National Science Foundation, or the United States government.

Adams, D. & Pimple, K. (2005). Research misconduct and crime lessons from criminal science on preventing misconduct and promoting integrity. Accountability in Research, 12, 225–240.
Anderson, M.S., Horn, A.S., Risbey, K.R., Ronning, E.A., De Vries, R., & Martinson, B.C. (2007). What Do Mentoring and Training in the Responsible Conduct of Research Have To Do with Scientists’ Misbehavior? Findings from a National Survey of NIH-Funded Scientists. Academic Medicine, 82(9), 853–860.
Antes, A.L., Murphy, S.T., Waples, E.P., Mumford, M.D., Brown, R.P., Connelly, S., & Devenport, L.D. (2009). A Meta-Analysis of Ethics Instruction Effectiveness in the Sciences. Ethics & Behavior, 19(5), 379-402. https://doi.org/10.1080/10508420903035380
Baker, M. (2016). Is there a reproducibility crisis? Nature, 533, 452-454.
https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970
Broad, W. J. (1981). Congress Told Fraud Issue 'Exaggerated'. Science, 212, 421.
Code of Federal Regulations (2011). Institutional investigation report. The Institutional Investigation § 93.313(f), p. 660. https://www.govinfo.gov/app/details/CFR-2011-title42-vol1/CFR-2011-title42-vol1-part93-subpartC
Code of Federal Regulations (2011b). Compliance and Assurances. Subpart C- Responsibilities of Institutions. § 93.300(c), p. 655. https://www.govinfo.gov/app/details/CFR-2011-title42-vol1/CFR-2011-title42-vol1-part93-subpartC
Davis, M.S., Riske-Morris, M., & Diaz, S.R. (2007). Causal Factors Implicated in Research Misconduct: Evidence from ORI Case Files. Science and Engineering Ethics, 13, 395–414. https://doi.org/10.1007/s11948-007-9045-2
DuBois, J.M., Anderson, E.E., Chibnall, J., Carroll, K., Gibb, T., Ogbuka C., & Rubbelke T. (2013). Understanding research misconduct: a comparative analysis of 120 cases of professional wrongdoing. Accountability in Research, 20(5-6), 320-338. https://doi.org/10.1080/08989621.2013.822248
Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One. 4, e5738. https://doi.org/10.1371/journal.pone.0005738.
Fanelli, D., Costas, R., & Larivière, V. (2015). Misconduct policies, academic culture and career stage, not gender or pressures to publish, affect scientific integrity. PLoS One, 10, e0127556.
Keith-Spiegel, P., & Koocher, G. P. (2005). The IRB paradox: Could the protectors also encourage deceit? Ethics & Behavior, 15, 339–349.
Martinson, B. C., Anderson, M. S., Crain, A. L., & De Vries, R. (2006). Scientists’ perceptions of organizational justice and self-reported misbehaviors. Journal of Empirical Research on Human Research Ethics, 1, 51–66.
Martinson B.C., Anderson M.S., & De Vries R. (2005). Scientists behaving badly. Nature, 435(7043), 737-738. https://doi.org/10.1038/435737a
Martinson, B. C., Crain, A. L., De Vries, R., & Anderson, M. S. (2010). The importance of organizational justice in ensuring research integrity. Journal of Empirical Research on Human Research Ethics, 5, 67–83.
Mumford, M. D., Connelly, M. S., Helton, W. B., Strange, J. M., & Osburn, H. K. (2001). On the construct validity of integrity tests: Individual and situational factors as predictors of test performance. International Journal of Selection and Assessment, 9, 240–257.
Office of Science and Technology Policy (2000). Federal Policy on Research Misconduct. Federal Register, 65(235), 76262.
https://www.govinfo.gov/content/pkg/FR-2000-12-06/pdf/00-30852.pdf
Resnik, D.B. & Master, Z.M. (2013). Policies and Initiatives Aimed at Addressing Research Misconduct in High-Income Countries. PLOS|Medicine. https://doi.org/10.1371/journal.pmed.1001406
Tijdink, J.K., Verbeke, R., & Smulders, Y.M. (2014). Publication pressure and scientific misconduct in medical scientists. Journal of Empirical Research on Human Research Ethics, 9(5), 64-71. https://doi.org/10.1177/1556264614552421
Wright, D.E., Titus, S.L., & Cornelison, J.B. (2008). Mentoring and Research Misconduct: An Analysis of Research Mentoring in Closed ORI Cases. Science and Engineering Ethics, 14, 323–336. https://doi.org/10.1007/s11948-008-9074-5
Yeo-Teh, N.S.L. & Tang, B.L. (2020) Research ethics courses as a vaccination against a toxic research environment or culture. pp. 1-11. © The Authors 2020, Sage Publications. https://doi.org/10.1177/1747016120926686

Download PDF

Journal Publication

published 11 Dec, 2020

Read the published version in Research Integrity and Peer Review →

Editorial decision: Major revision
04 Oct, 2020
Review #2 received at journal
08 Sep, 2020
Reviewer #2 agreed at journal
04 Sep, 2020
Review #1 received at journal
03 Sep, 2020
Reviewer #1 agreed at journal
18 Aug, 2020
Reviewers invited by journal
17 Aug, 2020
First submitted to journal
13 Aug, 2020
Editor assigned by journal
13 Aug, 2020
Submission checks completed at journal
12 Aug, 2020
Editor invited by journal
12 Aug, 2020

You are reading this older preprint version

Read the latest preprint version →

Research practices and research misconduct

Status:

Journal Publication

Version 1

Abstract

Introduction

Methods

Results

Discussion

Conclusion

Declarations

References

Status:

Journal Publication

Version 1