We report this manuscript in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA statement) guideline [24] (Supplementary material 1).
Search strategy
A systematic library search was conducted by one of the authors (ASM) in consultation with a librarian across seven electronic databases (CINAHL, Embase, PsycINFO, PubMed, Scopus, Web of Sciences and LIVIVO). Gray literature sources was searched in the System for Information on Grey Literature in Europe (SIGLE) (http://www.opengrey.eu/). A hand-search of relevant bibliographies was performed to identify potential studies that were excluded. The key words used in the search included; oral, dental, Health, Literacy, tool, instrument, questionnaire, Psychometric, validity, reliability. Only peer-reviewed articles that were written in English were considered. The full search strategy for each database is provided in Table 1.
Table 1. Library Search Strategy
Database
|
Search terms
|
Results
|
CINAHL
|
(TI (Oral* OR dental*) OR AB (Oral* OR dental* ) ) AND (TI (Health AND Literacy) OR AB (Health AND Literacy) ) AND (TI (tool OR instrument OR questionnaire) OR AB (tool OR instrument OR questionnaire)) AND (TI (Psychometric OR validity OR reliability) OR AB (Psychometric OR validity OR reliability))
|
349
|
EMBASE
|
('Oral*':ab,ti OR 'dental*':ab,ti OR 'Oral '/exp OR 'dental'/exp) AND (('health':ab,ti AND 'literacy':ab,ti OR ('health '/exp AND 'literacy'/exp)) AND ('tool ':ab,ti OR 'instrument':ab,ti OR 'questionnaire':ab,ti OR 'tool'/exp OR 'instrument'/exp OR 'questionnaire'/exp) AND ('Psychometric':ab,ti OR ' validity':ab,ti OR ' reliability':ab,ti OR 'Psychometric'/exp OR 'validity'/exp OR 'reliability'/exp)
|
458
|
PSYCINFO
|
(TI (Oral* OR dental*) OR AB (Oral* OR dental* )) AND (TI (health AND literacy) OR AB (health AND literacy)) AND (TI (tool OR instrument OR questionnaire ) OR AB (tool OR instrument OR questionnaire )) AND (TI (Psychometric OR validity OR reliability ) OR AB (Psychometric OR validity OR reliability ))
|
214
|
PUBMED
|
(((((Oral[MeSH Terms]) OR dental[MeSH Terms])) AND ((Health[MeSH Terms]) AND Literacy[MeSH Terms])) AND (((tool[MeSH Terms]) OR instrument[MeSH Terms]) OR questionnaire[MeSH Terms])) AND (((Psychometric[MeSH Terms]) OR validity[MeSH Terms]) OR reliability[MeSH Terms]) OR (((((((Oral[Title]) OR dental[Title]) OR Oral[Abstract]) OR dental[Abstract])) AND ((((Health[Title]) AND Literacy[Title]) OR Health[Abstract]) AND Literacy[Abstract])) AND ((((((tool[Title]) OR instrument[Title]) OR questionnaire[Title]) OR tool[Abstract]) OR instrument[Abstract]) OR questionnaire[Abstract])) AND ((((((Psychometric[Title]) OR validity[Title]) OR reliability[Title]) OR Psychometric[Abstract]) OR validity[Abstract]) OR reliability[Abstract])
|
495
|
SCOPUS
|
( TITLE-ABS-KEY ( oral* OR dental* ) ) AND ( TITLE-ABS-KEY ( "health" AND literacy" ) ) AND ( TITLE-ABS-KEY (tool OR instrument OR questionnaire ) ) AND ( TITLE-ABS-KEY (Psychometric OR validity OR reliability ) )
|
341
|
LIVIVO
|
( TI=((( MESH=Oral OR MESH=dent* ) AND MESH=Health AND MESH=Literacy ) AND (MESH=tool OR MESH=instrument) OR MESH=questionnaire) ) OR ( ( ( TI=(Oral* OR dent*) ) AND TI=(Health AND Literacy) ) AND TI=(tool OR instrument OR questionnaire) ) AND (Psychometric OR validity OR reliability)
|
453
|
WEB OF SCIENCE
|
(((TI=(Oral* OR dent*) AND TI=(Health AND Literacy) AND (tool OR instrument OR questionnaire))) OR ((((TS=(Oral* OR dent*) AND TS=(Health AND Literacy) AND (tool OR instrument OR questionnaire) AND TS=(Psychometric OR validity OR reliability))))
|
270
|
Open Gary
|
((( oral* OR dental*) AND ( (health AND literacy ) AND (tool OR instrument OR questionnaire) AND (tool OR instrument OR questionnaire)))
|
0
|
Bibliography
|
|
15
|
|
Total
|
2595
|
Studies that fulfilled the following criteria were included: (1) Studies whose results have assessed one or more of the following psychometric properties: internal consistency, reliability, measurement error, content validity, face validity, structural validity, hypothesis testing, cross-cultural validity, criterion validity, or responsiveness; (2) all studies published between each database's inception and January 2019 that have been design, develop, or psychometric to measure oral and dental health literacy; and (3) studies published in English language. Studies were excluded if they were (1) conference abstracts, systematic review and meta-analysis, and other studies that did not meet the inclusion criteria as well (2) Protocol studies related to psychometrics (Studies with no results).
Screening, Data extraction
Search strategies were performed by two trained authors (ASM and SR). The authors were the same at all stages of the study. In the first stage, titles and abstract of the articles were evaluated. In the second stage, the full text of the articles was independently reviewed by two authors. To assess agreement between reviewers for study selection, we used the Kappa (K) statistic, which measures agreement beyond chance [25]. A Kappa value > 0.6 is considered substantial agreement and a Kappa value >0.8 is considered almost perfect agreement [26].The quality of each article was quantified by a score of 0 or 1 (low or high).
Assessment of risk of bias
The COSMIN Risk of Bias checklist was used to assessment the methodological quality of the included studies on measurement properties. This checklist consists of several boxes, each pertaining to a specific measurement property and containing several questions/standards about the design requirements and statistical methods of the studies. For each measurement property in each study, the COSMIN item with the lowest score will indicate the overall methodological quality (i.e., worst-score-counts method) [27, 28].
In this study, the End Note software was used to organize the references. Data extraction included author, year, target population, sample size, location of the study, complete instrument name, report, time management (min), number of questions and scales, and rating. One part of the data extraction is related to the process of qualitative evaluation of the tools which is discussed below. The searches conducted From February to April 2019. The authors entered the data existed in the articles into Excel software based on the items in the data extraction section.
The process of qualitative evaluation of the studies related to OHL tools
At this stage, the full texts selected related to OHL tools at the screening stage were evaluated by two authors (ASM and MGH) independently, and on the basis of two factors. Differences in judgment were resolved through a consensus procedure.
1) Evaluate aspects of OHL: To examine the specific skills and competencies measured by the different tools we used the taxonomy of skills identified by Sørensen et al., 2012 in their content analysis of health literacy definitions. This process evaluates the tools based on different dimensions, including the reading dimension (basic skills for reading based on the International Student Assessment [PISA]), interactive dimension (the ability to communicate about health issues), perceptual dimension (the ability to extract meaning from information sources), and computational dimension (the ability to perform numeric tasks and mathematic operations). The remaining dimensions includes; information search (which requires the ability to find information on health for health management), performance (the ability to use and process, or act upon health information and informed decision), assessment (ability to filter, change and evaluate information), and responsibility (the ability to take responsibility and make decision on health and Health care), [23].
2) Qualitative assessment of methodology and psychometric properties. To evaluate the psychometric section, the COSMIN checklist (the consensus-based standards for the selection of health measurement instruments) was used [29]. This tool examines the quality of studies in 4 areas, 12 domains and 114 items. The 12 domains include; internal consistency, reliability, measurement error, content validity, structural validity, hypothesis testing, cross-cultural validity, criterion validity, responsiveness of theory methods (if applied), interpretability, and generalizability of the tool’s properties. Since there is no gold standard for the oral and dental health literacy tools[22], the domain of Criterion validity was not considered. All 114 items were evaluated according to the poor, fair, good, and excellent scale. Taking the lowest rating for each item in one box, an overall quality score (poor, fair, good, excellent) is obtained for each measurement property separately[30, 31].
Supplementary material 2 presents the ratings of the quality of each instrument based on the COSMIN checklist as well as three categories of ‘adequate’, ‘not adequate’, and ‘unclear’[29].
The strength of evidence assessment
To evaluate the instruments, the strength of evidence for each was rated on a scale graded as strong, moderate, limited, conflicting or unknown. The criteria for rating were the methodological and measurement quality, the number and consistency of results among the body of research using the instrument. Strong evidence was marked by several articles with high quality methods or one published paper of an excellent quality and a report of consistency of the properties. Moderate level was characterized by several articles with fair methods or one published paper of good quality. Limited rating would characterize an instrument with one article of fair quality. Conflicting level would describe an instrument that had mixed findings. Unknown rating was for an instrument with several papers of low quality methods or simply no published paper.