This Australian study supports the poor accuracy of self-reported RA as a sole measure of RA diagnosis, and strengthens the argument for finding a more accurate, yet practical, way to classify RA at a population level. The prevalence of unrefined self-reported RA in our study of 5.4% is much higher than would be expected for true RA, even among an all-female cohort aged over 34 years, in which the prevalence of RA is higher than in males or younger age groups (3, 30). A study of 7443 post-menopausal women aged 50–79 years in the USA found a validated prevalence of 0.6% (9) and a study of French women aged 40–65 years found a validated prevalence of 1% (10), and we would expect our true prevalence to be similar.
We explored the use of available self-reported and administrative data to improve the accuracy of case-finding methods for RA. A question on medications is a relatively simple addition to population surveys and has been proposed to improve validity of self-reported disease. In the Black Women’s Health Study, the positive predictive value (PPV) of self-reported RA increased to 76% in women who reported taking DMARDs and to 61% in women who reported taking non-steroidal anti-inflammatory drugs (NSAIDs), compared to only 29% in women who did not report taking any related medications. When women using only prednisolone, or those reporting other rheumatic conditions, were excluded, and only those taking DMARDs were included, the PPV increased to 88% (11). This suggests use of DMARDs as a case-finding method is likely to be relatively accurate. The current study developed two self-reported medication case definitions in keeping with this previous literature and found that excluding those taking only prednisone/prednisolone gave a prevalence of 1.2%, which is closer to the expected prevalence in our population (31, 32). In contrast to the Black Women’s Health study, the effect of adding non disease specific medications like NSAIDs and steroids did not appear helpful in this study. This would be expected due to the breadth of indications for use of NSAIDs or steroids and therefore a lack of specificity for an RA diagnosis.
The PBS database provides a more complete and objective measure of medication prescribing than self-report. The PBS is one of the few medication reimbursement schemes in the world that provides whole population coverage. Additionally, in Australia nearly all medications for RA require prescription for access and are used according to PBS restrictions, meaning they are recorded on the PBS database. Limitations of the PBS database for this study are that prior to April 2012 payments below the co-payment threshold at which the PBS would cover part of the cost (up to $35.40) were not recorded, meaning that methotrexate, hydroxychloroquine, azathioprine and some other older conventional DMARDs were not captured, and medications dispensed solely to an inpatient in a public hospital are also not included, which we would expect to lead to some false negatives. Use of PBS dispensed DMARDs as a sole method for case-finding (PBS-strict definition) gave a slightly higher than expected prevalence of 2.8%, which is not unexpected given the likely inclusion of some individuals taking DMARDs for other rheumatic or immune conditions, as the PBS did not record the indication for use. This is supported by the prevalence of 1.9% once individuals who had been admitted with, or had medications consistent with, other rheumatic/immune conditions were excluded. There was a discrepancy between self-reported and PBS strict medication definitions, with a much lower prevalence by the self-reported definition. This is likely influenced by a lower number of women answering this question, and under-recording of medications, such as methotrexate or injectable DMARDs that are not taken daily. We chose to apply the dispensed medication definitions to the total group rather than applying it in patients who additionally had self-reported RA, as we believe the established limitations of self-report as a diagnostic criterion would mean using this as a starting point for our case-definitions would go against the aims of this study. In addition, by using purely administrative data for the definitions and not requiring self-report/survey data, we have created a tool to approximate cases at a population level without use of intensive resources, making this of greater practical use.
Admission and ED data are likely to be the most specific measure of RA, and are of similar specificity to medical record review, which is usually held to be the gold standard. A recent study from Western Australia found that RA classified by ICD-10 discharge codes in hospital records had a sensitivity of 90% and PPV of 91% compared to rheumatologist medical record review (23). In our study, the available data covered all public hospitals and EDs, and additionally covered private and day hospitals in some states, including NSW (the most populous Australian state). The accuracy of this definition is also supported by the strong correlation between admission/ED and PBS dispensed RA medication. The major limitation to using hospital data is poor sensitivity, as most people with RA do not require hospital treatment, and in Australia we do not have a population database that records diagnoses associated with public outpatient visits. This definition will therefore underestimate true RA prevalence. This likely contributes to a bias towards only more severe cases being included, or towards patients with more comorbidities, as they are more likely to have required admission or ED review. Reassuringly, however, in the current study, 292 cases were identified by this method, giving a prevalence of 1.1%, which is around the expected value for true RA. Given this, using the admitted or ED group is likely a good compromise for a well-validated RA cohort without performing individual medical record review.
MBS codes were used to improve specificity of medication-defined and self-reported cases. The MBS requirements were not applied to hospital-defined cases as these were felt to represent physician-diagnosed, and thus confirmed, cases that did not require further validation. The MBS provides specific item numbers for some diagnostic and service items, but is not comprehensive. Codes were available for diagnostic tests specific to IBD and treatment specific to psoriasis and these were used to exclude individuals with these conditions, which can cause non-RA inflammatory arthritis that can also require DMARD therapies. The MBS codes could not be used to identify individuals that had consulted a rheumatologist as service codes for consultations do not include clinician speciality.
The main limitation to our study is that we were unable to use medical record review, blood tests or physician review as the gold standard comparator due to restrictions imposed by the ethics approvals of the ALSWH and the survey data collection process. For this reason we were unable to statistically compare our case methods for accuracy relative to a gold or reference standard. However, access to the admission/ED data does provide a relatively well-validated group for comparison, as the diagnosis codes are provided based on clinician review. We also acknowledge that there is the potential for exclusion of some participants with true RA during the refinement process if these individuals had other concomitant inflammatory conditions. Our generalisability is limited to only women; however this cohort has been deliberately sampled to be representative of the total female Australian population (in the included age groups) so generalisability should be high within the female population.
We propose two final case-definitions for use in further study of RA and its risk factors using the ALSWH data, a “Documented RA” group, using admitted/ED patients, and a “Treated RA” group, using ‘refined’ case definitions of PBS dispensed medications. Additionally, these definitions should be used to improve the national monitoring of RA in Australia, and with adjustments for local data sources, in many other nations. While we acknowledge that none of the methods to estimate RA cases is perfect, and the lack of comparison with a reference standard, the present study provides an algorithm for identifying RA cases that strikes a balance between improving accuracy and practicality/resource use. This provides a solution to the need for a more standardised and pragmatic method for RA definition to use in large studies and at a population level. If self-reported data are used, refining such a definition by excluding likely false-positive cases with the methods described above is likely to improve performance significantly.