Study design and ethics
This infodemiological qualitative study aimed to identify and characterize false oral health information found on the Internet. For that, websites published in English were retrieved by Google Advanced Search, and screened by 2 independent investigators, to compile misleading messages through thematic content analysis [18,19]. Based on a conceptual framework, content items were divided in 4 groups: misinformation; misinformation with financial, social and/or psychological interests produced or propagated by laypeople; misinformation with financial, social and/or psychological interests produced or propagated by dental professionals; and misinformation with political interests [5,10,17,19-24]. To determine the spread of content items, social media (Facebook and Instagram) and a fact-checking tool (Snopes) were respectively screened to identify corresponding posts and warnings.
This study did not require institutional review board approval from the Council of Ethics in Human Research of the blinded for peer review because federal regulations do not apply to research using publicly available data that does not involve human subjects.
Conceptual framework
We analyzed current concepts, terminologies, and taxonomies of information disorder to guide the investigations performed in this study. The definitions of terms were synthesized based on the most accepted theories, as follows: a) misinformation: false information determined based on a grounding of truth and applies only to informationally-oriented content; b) fake news: intentionally misleading and biased representational information for the benefit of the messenger sender, which contains false information, with or without a blend of one or more components of omitted important information, a decontextualized content, misleading headlines or clickbait; c) disinformation: information that is false and deliberately created to harm a person, social group, organization or country; d) mal-information: the sharing of accurate information with the intention to cause harm; and e) conspiracy theories: attempts to explain the ultimate causes of significant social and political events and circumstances with claims of secret plots by two or more powerful actors [10,17,19,22-25]. Moreover, misleading online content can be supported by distinct types of interests.
Regarding the aforementioned concepts, content items were characterized according to 3 criteria:
- The detection of misleading content (yes or no), including non-evidence-based information as proposed by Swire-Thompson and Lazer [5]. This process regarded the message contexts to identify possible misinterpretation or imprecise use of information;
- The identification of interests (yes or no) behind the content, authors, and websites. The interests were classified as informational (warnings against deceitful content and beliefs), financial (profiting from information disorder through advertising), political (attempts to influence public opinion in relation to political positions), social (connecting with groups online or off), and/or psychological (seeking prestige or reinforcement), as stated by Wardle and Derakhshan [10];
- The definition of the scientific background of the producer(s) or propagator(s) of content (dentists/oral health companies or others), aligned to the technical knowledge that enable specialists to the identification of misleading oral health information, concerning the professional responsibility of promoting those subjects on digital environments.
Figure 1 depicts the decision tree used to divide oral health misleading content items according to groups with distinct characteristics of interest and authorship, as follows: G1 – misinformation (informationally-oriented content); G2 – misinformation with financial, social and/or psychological interests produced or propagated by laypeople; G3 – misinformation with financial, social and/or psychological interests produced or propagated by dental professionals; and G4 – misinformation with political interests. This structure considered the objective contextual analysis that could be associated with the intentional production or propagation of false information, since the determination of intentionality is founded on subjective factors, making difficult the standardization of analysis. According to Poe's law, the clues left by newsmakers are often inadequate to differentiate the honest and dishonest mistakes [3,26].
Data collection
Between February 4 and 23, 2021, websites published in English were retrieved using Google Advanced Search, the search engine market leader in English-speaking countries [27]. The systematization of data collection is summarized in Figure 2. Initially, an exploratory analysis about the main dental fields was made through predetermined search strategies using Boolean operators. The queries were created by the intersection of a “specific field” AND (“fake news” OR disinformation). Previously, the browser was defaulted by idiom in a Microsoft Edge at Windows 10 system, with all cookies and history cleared to minimize the influence of personal preferences on data collection. The first 10 websites (front page) were accessed for each search strategy, replacing repetitions and sponsored advertisements. It is noteworthy that 95% of users who search for health information on Google only follow links on the first page [28]. All links selected were registered in Archive Today to ensure that websites remain unchanged and saved in a public repository for further analysis [29,30].
Ten main dental fields (general dentistry, cariology, oral medicine, restorative dentistry, endodontics, prosthodontics, orthodontics, pediatric dentistry, periodontics, and surgery) were applied in the first stage. Only the queries with general dentistry and cariology did not retrieve websites. Thus, between March 1 and April 22, 2021, 2 independent investigators (ML and OSJ) manually analyzed the first 80 websites through thematic content analysis to identify misleading information. For that, the investigators coded each sentence of webpages as 0 (information) or 1 (misleading content) [18,19]. Each misleading content item and its respective authors’ contextualization were tabulated in association with a particular dental field. Then, these findings were gathered in a single table to eliminate possible repetitions.
Afterwards, in the second stage, 7 themes known related to general dentistry and cariology (dental caries, floss, fluoride, mouthwash, toothache, toothbrushing, and toothpaste) and 3 themes usually observed among fields without the identification of misleading content (oral cancer, root canal, and teeth whitening) were explored to retrieve more 100 websites. Additionally, 20 new themes that appeared in the first and second stages of analysis led to the collection of more 200 websites (third stage). After that, 3 other new themes emerged from the third stage, requiring the retrieval of additional 30 websites (fourth stage), when it was achieved the saturation of themes. The screening resulted in 410 websites with the identification of 369 non-repeated potentially misleading content items.
Identification and characterization of false oral health information
Between April 27 and July 5, 2021, other 2 independent investigators (ML and TC) manually analyzed potentially misleading content items, considering their context on websites. The items divergently qualified by examiners were re-assessed until consensus. Fifty-one items were excluded from the analysis because they were related to supportive results of systematic reviews, clinical trials, or cohort studies published in international journals indexed in databases Cochrane, Embase Search, Clinical Trials, Ovid, PubMed, Scopus, and/or Web of Science. Then, each one of 318 misleading content items was allocated in a group, as previously described (Table 1).
The publication of misleading content on Instagram and Facebook was verified using the CrowdTangle [31], taking into account the popularity of posting social media worldwide [32]. This platform permits data analysis of public accounts and pages. Also, the Snopes fact-checking tool was accessed to determine whether false oral health information had already been debunked previously [33]. The main words of each misleading content item were applied to search posts and warnings using Boolean operators in both platforms. For example, the item described as “fluoride may cause cancer” was synthesized as fluoride AND cancer.
For more details, please consult the public data repository [30].
Data analysis
Statistical analysis was performed using the Statistical Package for Social Sciences (v. 21.0; SPSS; Chicago, IL, USA). Thematic nodes and subnodes that emerged from the content analysis were represented by a conceptual map. Also, data were evaluated by descriptive analysis and Cramer’s V test to compare distinct categories of misleading content items according to the frequencies of types of interest (informational, financial, social, psychological, and political), detection on social media (Facebook and Instagram), and fact-checking agency (Snopes). For all analyses, p values <0.05 were considered significant.