The pursuit of simple, yet fair, unbiased, and objective measures of researcher performance has occupied bibliometricians and the research community as a whole for decades. However, despite the diversity of available metrics, most are either complex to calculate or not readily applied in the most common assessment exercises (e.g., grant assessment, job applications). The ubiquity of metrics like the h-index (h papers with at least h citations) and its time-corrected variant, the m-quotient (h-index ÷ number of years publishing) therefore reflect the ease of use rather than their capacity to differentiate researchers fairly among disciplines, career stage, or gender. We address this problem here by defining an easily calculated index based on publicly available citation data (Google Scholar) that corrects for most biases and allows assessors to compare researchers at any stage of their career and from any discipline on the same scale. Our ε′-index violates fewer statistical assumptions relative to other metrics when comparing groups of researchers, and can be easily modified to remove inherent gender biases in citation data. We demonstrate the utility of the ε′-index using a sample of 480 researchers with Google Scholar profiles, stratified evenly into eight disciplines (archaeology, chemistry, ecology, evolution and development, geology, microbiology, ophthalmology, palaeontogy), three career stages (early, mid-, late-career), and two genders. We advocate the use of the ε′-index whenever assessors must compare research performance among researchers of different backgrounds, but emphasize that no single index should be used exclusively to rank researcher capability.
Article
WITHDRAWN: A fairer way to compare researchers at any career stage and in any discipline using open-access citation data
https://doi.org/10.21203/rs.3.rs-131125/v1
This work is licensed under a CC BY 4.0 License
Journal Publication
published 09 Sep, 2021
Version 1
posted
You are reading this older preprint version
25 September, 2024. Research Square has withdrawn this preprint as it was requested after the work was already published at a peer reviewed journal, which is in conflict with our editorial policy regarding postprints.
Editorial notes are used to provide important context regarding the topic of a preprint or to alert readers to potential issues concerning that preprint or a downstream publication associated with it. For more information on editorial notes, see our Editorial Policies.
Deriving a fair, unbiased, and easily generated quantitative index serving as a reasonable first-pass metric for comparing the relative performance of academic researchers is — by the very complexity, diversity, and intangibility of research output across academic disciplines — impossible1. However, that unachievable aim has not discouraged bibliometricians and non-bibliometricians alike from developing scores of citation-based variants2,3 in an attempt to do exactly that, from the better-known h-index4,5 (h papers with at least h citations), m-quotient4,5 (h-index ÷ number of years publishing), and g-index6 (unique largest number such that the top g papers decreasingly ordered by citations have least g2 citations), to the scores of variants of these and other indices — e.g., h2-index, e-index7, χ-index8, hm-index9, gm-index10, etc. 3. Each metric has its own biases and strengths11–13, suggesting that several should be used simultaneously to assess citation performance. For example, the arguably most-popular h-index down-weights quality relative to quantity14, ignores the majority of accumulated citations in the most highly cited papers15, has markedly different distributions among disciplines16, and tends to increase with experience17. The h-index can even rise following the death of the researcher, because the h-index can never decline2 and citations can continue to accumulate posthumously.
Despite their broad use in everything from assessing candidates applying for academic positions, comparing the track records of researchers applying for grants, to applications for promotion3,18 inter alia, single-value citation metrics are rarely meant to (nor should they) be definitive assessment tools3. Instead, their most valuable (and fair) application is to provide a quick ‘first pass’ to rank a sample of researchers, followed by more detailed assessment of publication quality, experience, grant successes, mentorship, collegiality and all the other characteristics that make a researcher more or less competitive for rare positions and grant monies. But despite the many different metrics available and arguable improvements that have been proposed since 2005 when the h-index was first developed4,5, few are used regularly in these regards. This is because they are difficult to calculate without detailed data of a candidate’s publication history, they are not readily available on open-access websites, and/or they tend to be highly correlated with the h-index anyway19. It is for these reasons that the admittedly flawed20,21 h-index and its experienced-corrected variant, the m-quotient, are still the dominant (h-index much more so than the m-quotient)2 metrics employed given that they are easily calculated2,22 and found for most researchers on open-access websites such as Google Scholar23 (scholar.google.com). The lack of access and detailed understanding of the many other citation-based metrics means that most of them go unused3, and are essentially valueless for every-day applications of researcher assessment.
The specific weaknesses of the h-index or m-quotient make the comparison of researchers in different career stages, genders, and disciplines unfair because they are not normalized in any way. Furthermore, there is no quantitatively supported threshold above or below which assessors can easily ascertain minimum citation performance for particular applications — while assessors certainly use subjective ‘rules of thumb’, a more objective approach is preferable. For this reason, an ideal citation-based metric should only be considered as a relative index of performance, but relative to what, and to whom?
To address these issues and to provide assessors with an easy, rapid, yet objective relative index of citation performance for any group of researchers, we designed a new index we call the ‘ε-index’ (the ‘ε’ signifies the use of residuals, or deviance from a trend) that is simple to construct, can be standardized across disciplines, is meaningful only as a relative index for a particular sample of researchers, can be corrected for career breaks (see Methods), and provides a sample-specific threshold above and below which assessors can determine whether individual performance is greater or less than that expected relative to the other researchers in the specific sample.
With the R code we provide, an assessor need only acquire four separate items of information from Google Scholar (or if they have access, from other databases such as Scopus — scopus.com) to calculate a researcher’s ε-index: (i) the number of citations acquired for the researcher’s top-cited paper (i.e., the first entry in the Google Scholar profile), (ii) the i10-index (number of articles with at least 10 citations), (iii) the h-index, and (iv) the year in which the researcher’s first peer-reviewed paper was published. While the last item requires sorting a researcher’s outputs by year and scrolling to the earliest paper, this is not a time-consuming process. We demonstrate the performance of the ε-index using Google Scholar citation data we collected for 480 researchers in eight separate disciplines spread equally across genders and career stages to show how the ε-index performs relative to the m-quotient (the only other readily available, opportunity-corrected citation index available on Google Scholar) across disciplines, career stages, and genders. We also provide a simple method to scale the index across disciplines (ε′-index) to make researchers in different areas comparable despite variable citation trends within their respective areas.
Despite the considerable variation in citation metrics among researchers and disciplines, there was broad consistency in the strength of the relationships between citation mass (Arel) and loge years publishing (t) across disciplines (Fig. 1), although the geology (GEO) sample had the poorest fit (ALLR2 = 0.43; Fig. 1). The distribution of residuals ε for each discipline revealed substantial difference in general form and central tendency (Fig. 2), but after scaling, the distributions of ε′ became aligned among disciplines and were approximately Gaussian (Shapiro-Wilk normality tests; see Fig. 2 for test values).
After scaling (Fig. 3a), the relationship between ε′ and the m-quotient is non-linear and highly variable (Fig. 3b), meaning that m-quotients often poorly reflect actual relative performance (and despite the m-quotient already being ‘corrected’ for t, it still increases with t; Supplementary Information Fig. S1). For example, there are many researchers whose m-quotient < 1, but who perform above expectation (ε′ > 0). Alternatively, there are many researchers with an m-quotient of up to 2 or even 3 who perform below expectation (ε′ < 0). Once the m-quotient > 3, ε′ reflects above-expectation performance for all researchers in the example sample (Fig. 3b). The corresponding ε′ indicate a more uniform spread by gender and career stage (Fig. 3c) than do m-quotients (Fig. 3d). Another advantage of ε′ versus the m-quotient is that the former has a threshold (ε′ = 0) above which researchers perform above expectation and below which they perform below expectation, whereas the m-quotient has no equivalent threshold. Further, the m-quotient tends to increase through one’s career, whereas ε′ is more stable. There is still an increase in ε′ during late career relative to mid-career, but this is less pronounced that that observed for the m-quotient (Fig. 4).
Examining the ranks derived from ε′ across disciplines, genders and career stage (Fig. 5), bootstrapped median ranks overlap for all disciplines (Fig. 5a), but there are some notable divergences between the genders across career stage (Fig. 5b). In general, women ranked slightly below men in all career stages, although the bootstrapped median ranks overlap among early and mid-career researchers. However, the median ranks for late-career women and men do not overlap (Fig. 5b), which possibly reflects the observation that senior academic positions in many disciplines are dominated by men24–26, and that women tend to receive fewer citations than men at least in some disciplines, which often tends to compound over time27–30. The ranking based on the m-quotient demonstrates the disparity among disciplines (Fig. 5c), but it is perhaps somewhat more equal between the genders (Fig. 5d) compared to the ε′ rank (Fig. 5b), despite the higher variability of the m-quotient bootstrapped median rank.
However, calculating the scaled residuals across all disciplines for each gender separately, and then combining the two datasets and recalculating the rank (producing a gender-‘debiased’ rank) effectively removed the gender differences (Fig. 6).
Todeschini and Baccini31 recommended that the ideal author-level indicator of citation performance should (i) have an unequivocal mathematical definition, (ii) be easily computed from available data (for a detailed breakdown of implementation steps and the R code function, see github.com/cjabradshaw/EpsilonIndex; we have also provided a user-friendly app available at cjabradshaw.shinyapps.io/epsilonIndex that implements the code and calculates the index with user-provided citation data), (iii) balance rankings between more experienced and novice researchers (iv) while preserving sensitivity to the performance of top researchers, and (iv) be sensitive to the number and distribution of citations and articles. Our new ε-index not only meets these criteria, it also adds the ability to compare across disciplines by using a simple scaling approach, and can easily be adjusted for career gaps by subtracting research-inactive periods from the total number of years publishing (t). In this way, the ε-index could prove invaluable as we move toward greater interdisciplinarity, where tenure committees have had difficulty assessing the performance of candidates straddling disciplines32,33. The ε-index does not ignore high-citation papers, but neither does it overemphasize them, and it includes an element of publication frequency (i10) while simultaneously incorporating an element of ‘quality’ by including the h-index.
Like all other existing metrics, the ε-index does have some disadvantages in terms of not correcting for author contribution — such as the hm-index9 or gm-index10 — even though these types of metrics can be cumbersome to calculate. Early career researchers who have published but have yet to be cited will not yet be able to calculate their ε-index, as they will not have an h-index score, so would require different types of assessment. Another potential limitation is that the ε-index alone does not correct for any systemic gender biases associated with the many reasons why women tend to be cited less than men24–30,34, but it does easily allow an assessor to benchmark any subset of researchers (e.g., women-only or men-only) to adjust the threshold accordingly. Thus, women can be compared to other women and ranked accordingly such that the ranks are more comparable between these two genders. Alternatively, dividing the genders and benchmarking them separately followed by a combined re-ranking (Fig. 6) effectively removes the gender bias in the ε-index, which is difficult or impossible to do with other ranking metrics. We certainly advocate this approach when assessing mixed-gender samples (the same approach could be applied to other subsets of researchers deemed a priori to be at a disadvantage).
The ε-index also potentially suffers from the requirement of the constituent citation data upon which it is based being accurate and up-to-date35,36. Regardless, should an assessor have access to potentially more rigorous citation databases (e.g., Scopus), the ε-index can still be readily calculated, although within-sample consistency must be maintained for the ranks to be meaningful. We also show that the distribution of the ε-index is relatively more Gaussian in behaviour than the time-corrected m-quotient, with the added advantage of identifying a threshold above and below which individuals are deemed to be performing better or worse than expected relative to their sample peers. While there are potentially subjective rules of thumb for thresholds to be applied to the m-quotient, the residual nature of the ε-index makes it a more objective metric for assessing relative rank, and the ε-index is less-sensitive than the m-quotient regarding the innate rise of ranking as a researcher progresses through her career (Fig. 4).
We reiterate that while the ε-index is an advance on existing approaches to rank researchers according to their citation history, a single metric should never be the sole measure of a researcher’s productivity or potential. Nonetheless, the objectivity, ease of calculation, and flexibility of its application argue that the ε-index is a needed tool in the quest to provide fair initial appraisals of a researcher’s publication performance.
Researcher samples. Each co-author assembled an example set of researchers from within her/his field, which we broadly defined as archaeology (S.A.C.), chemistry (J.M.C.), ecology (C.J.A.B.), evolution/development (V.W.), geology (K.T.), microbiology (B.A.E.), ophthalmology (J.R.S.), and palaeontogy (J.A.L.). Our basic assembly rules for each of these discipline samples were: (i) 20 researchers from each stage of career, defined here arbitrarily as early career (0–10 years since first peer-reviewed article published in a recognized scientific journal), mid-career (11–20 years since first publication), and late career (> 20 years since first publication); each discipline therefore had a total of 60 researchers, for a total sample of 8 × 60 = 480 researchers across all disciplines. (ii) Each sample had to include an equal number of women and men from each career stage. (iii) Each researcher had to have a unique, publicly accessible Google Scholar profile with no obvious errors, inappropriate additions, obvious omissions, or duplications. The entire approach we present here assumes that each researcher’s Google Scholar profile is accurate, up-to-date, and complete.
We did not impose any other rules for sample assembly, but encouraged each compiler to include only a few previous co-authors. Our goal was to have as much ‘inside knowledge’ as possible with respect to each discipline, but also to include a wide array of researchers who were predominantly independent of each of us. The composition of each sample is somewhat irrelevant for the purposes of our example dataset; we merely attempted gender and career-level balance to show the properties of the ranking system (i.e., we did not intend for sampling to be a definitive comment about the performance of particular researchers, nor did we mean for each sample to represent an entire discipline). Finally, we completely anonymized the sample data for publication.
Citation data. Our overall aim was to provide a meaningful and objective method for ranking researchers by citation history without requiring extensive online researching or information that was not easily obtainable from a publicly available, online profile. We also wanted to avoid an index that was overly influenced by outlier citations, while still keeping valuable performance information regarding high-citation outputs and total productivity (number of outputs).
For each researcher, the algorithm requires the following information collected from Google Scholar: (i) i10-index (the number of publications in the researcher’s profile with at least 10 citations, which we denoted i10); one condition is that a researcher must have i10 ≥ 1 for the algorithm to function correctly; (ii) h-index — the researcher’s Hirsch number 4: the number of publications with at least as many citations, which we denoted h; (iii) the number of citations for the researcher’s most highly cited paper (denoted cm); and (iv) the year the researcher published her/his first peer-reviewed article in a recognized scientific journal (denoted Y1). For the designation of Y1, we excluded any reports, chapters, books, theses or other forms of publication that preceded the year of the first peer-reviewed article; however, we included citations from the former sources in the researcher’s i10, h, and cm.
Ranking algorithm. The algorithm first computes a power-law-like relationship between the vector of frequencies (as measured from Google Scholar): i10, h, and 1, and the vector of their corresponding values: 10, h, and cm, respectively. Thus, h is, by definition, both a frequency (y-axis) and value (x-axis). We then calculated a simple linear model of the form y ~ α + βx, where
$$\:y={\text{log}}_{e}\left[\begin{array}{c}{i}_{10}\\\:h\\\:1\end{array}\right]\:\text{a}\text{n}\text{d}\:x={\text{log}}_{e}\left[\begin{array}{c}10\\\:h\\\:{c}_{m}\end{array}\right]$$
(y is the citation frequency, and x is the citation value) for each researcher (Supplementary Information Fig. S2). The corresponding \(\:\widehat{\alpha\:}\) and \(\:\widehat{\beta\:}\) for each relationship allowed us to calculate a standardized integral (area under the power-law relationship, Arel) relative to the researcher in the sample with the highest cm. This implies all areas were scaled to the maximum in the sample.
A researcher’s Arel therefore represents her/his citation mass, but this value still requires correction for individual opportunity (time since first publication, t = current year – Y1) to compare researchers at different stages of their career. This is where career gaps can be taken into account explicitly for any researcher in the sample by subtracting ai = the total cumulative time absent from research (e.g., maternity or paternity leave, sick leave, secondment, etc.) for individual i from t, such that an individual’s career gap-corrected 
. We therefore constructed another linear model of the form Arel ~ γ + θloget across all researchers in the sample, and took the residual (ε) of an individual researcher’s Arel from the predicted relationship as a metric of citation performance relative to the rest of the researchers in that sample (Supplementary Information Fig. S3). This residual ε allows us to rank all individuals in the sample from highest (highest citation performance relative to opportunity and the entire sample) to lowest (lowest citation performance relative to opportunity and the entire sample). Any researcher in the sample with a positive ε is considered to be performing above expectation (relative to the group and the time since first publication), and those with a negative ε fall below expectation. This approach also has the advantage of fitting different linear models to subcategories within a sample to rank researchers within their respective groupings (e.g., such as by gender; Supplementary Information Fig. S4). An R code function to produce the index and its variants using a sample dataset is available from github.com/cjabradshaw/EpsilonIndex.
Discipline standardization. Each discipline has its own citation characteristics and trends 16, so we expect that the distribution of residuals (ε) within each discipline to be meaningful only for that discipline’s sample. We therefore endeavored to scale (‘normalize’) the results such that researchers in different disciplines could be compared objectively and fairly.
We first scaled the Arel within each discipline by dividing each i researcher’s Arel by the sample’s root mean square:
$$\:{A}_{{\text{rel}}_{i}}^{{\prime\:}}=\frac{{A}_{{\text{rel}}_{i}}}{\sqrt{\frac{\sum\:_{i=1}^{n}{A}_{{\text{rel}}_{i}}}{n-1}}}$$
where n = the total number of researchers in the sample (n = 60). We then regressed these discipline-scaled \(\:{A}_{\text{rel}}^{{\prime\:}}\) against the loge number of years since first publication pooling all disciplines together, and then ranked these scaled residuals (ε′) as described above.
Data and code availability
Example code and data to calculate the index are available at: github.com/cjabradshaw/EpsilonIndex. A R Shiny application is also available at: cjabradshaw.shinyapps.io/epsilonIndex, with its relevant code and example dataset available at: github.com/cjabradshaw/EpsilonIndexShiny.
Acknowledgments
We acknowledge our many peers for their stewardship of their online citation records. We acknowledge the Indigenous Traditional Owners of the land on which Flinders University is built — the Kaurna people of the Adelaide Plains.
Author Contributions
C.J.A.B conceptualized the paper, and C.J.A.B. designed the research. All co-authors provided sample data from their disciplines. C.J.A.B. wrote the code and did the analysis. C.J.A.B. wrote the paper, with contributions from all other authors.
Competing interests
The authors declare no competing interests.
Additional information
Supplementary information is available for this paper at https://doi.org/xxx.
Correspondence and requests for materials should be addressed to C.J.A.B.
- Phelan, T. J. A compendium of issues for citation analysis. Scientometrics 45, 117-136, doi:10.1007/BF02458472 (1999).
- Barnes, C. The h-index debate: an introduction for librarians. J. Acad. Libr. 43, 487-494, doi:10.1016/j.acalib.2017.08.013 (2017).
- Wildgaard, L. in Springer Handbook of Science and Technology Indicators (eds Wolfgang Glänzel, Henk F. Moed, Ulrich Schmoch, & Mike Thelwall) 361-396 (Springer International Publishing, 2019).
- Hirsch, J. E. An index to quantify an individual's scientific research output. Proc. Natl. Acad. Sci. U.S.A. 102, 16569-16572, doi:10.1073/pnas.0507655102 (2005).
- Schubert, A. & Schubert, G. in Springer Handbook of Science and Technology Indicators (eds Wolfgang Glänzel, Henk F. Moed, Ulrich Schmoch, & Mike Thelwall) 301-334 (Springer International Publishing, 2019).
- Egghe, L. How to improve the h-index. The Scientist 20, 15 (2006).
- Zhang, C.-T. The e-Index, complementing the h-Index for excess citations. PLoS One 4, e5429, doi:10.1371/journal.pone.0005429 (2009).
- Fenner, T., Harris, M., Levene, M. & Bar-Ilan, J. A novel bibliometric index with a simple geometric interpretation. PLoS One 13, e0200098, doi:10.1371/journal.pone.0200098 (2018).
- Schreiber, M. A modification of the h-index: The hm-index accounts for multi-authored manuscripts. J. Informetr. 2, 211-216, doi:10.1016/j.joi.2008.05.001 (2008).
- Schreiber, M. How to modify the g-index for multi-authored manuscripts. J. Informetr. 4, 42-54, doi:10.1016/j.joi.2009.06.003 (2010).
- Thompson, D. F., Callen, E. C. & Nahata, M. C. New indices in scholarship assessment. Am. J. Pharm. Educ. 73, 111-111, doi:10.5688/aj7306111 (2009).
- Bornmann, L., Mutz, R. & Daniel, H.-D. Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. J. Am. Soc. Inf. Sci. Tec. 59, 830-837, doi:10.1002/asi.20806 (2008).
- Bornmann, L., Mutz, R., Hug, S. E. & Daniel, H.-D. A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. J. Informetr. 5, 346-359, doi:10.1016/j.joi.2011.01.006 (2011).
- Costas, R. & Bordons, M. The h-index: advantages, limitations and its relation with other bibliometric indicators at the micro level. J. Informetr. 1, 193-203, doi:10.1016/j.joi.2007.02.001 (2007).
- Anderson, T. R., Hankin, R. K. S. & Killworth, P. D. Beyond the Durfee square: enhancing the h-index to score total publication output. Scientometrics 76, 577-588, doi:10.1007/s11192-007-2071-2 (2008).
- Batista, P. D., Campiteli, M. G. & Kinouchi, O. Is it possible to compare researchers with different scientific interests? Scientometrics 68, 179-189, doi:10.1007/s11192-006-0090-4 (2006).
- Kelly, C. D. & Jennions, M. D. The h index and career assessment by numbers. Trends Ecol. Evol. 21, 167-170, doi:10.1016/j.tree.2006.01.005 (2006).
- Hirsch, J. E. Does the h index have predictive power? Proc. Natl. Acad. Sci. USA 104, 19193, doi:10.1073/pnas.0707962104 (2007).
- Bornmann, L. Redundancies in h index variants and the proposal of the number of top-cited papers as an attractive indicator. Measurement 10, 149-153, doi:10.1080/15366367.2012.716255 (2012).
- Costas, R. & Franssen, T. Reflections around ‘the cautionary use’ of the h-index: response to Teixeira da Silva and Dobránszki. Scientometrics 115, 1125-1130, doi:10.1007/s11192-018-2683-0 (2018).
- Abramo, G., D’Angelo, C. A. & Viel, F. The suitability of h and g indexes for measuring the research performance of institutions. Scientometrics 97, 555-570, doi:10.1007/s11192-013-1026-4 (2013).
- Bhattacharjee, Y. Impact factor. Science 309, 1181 (2005).
- Delgado López-Cózar, E., Orduña-Malea, E. & Martín-Martín, A. in Springer Handbook of Science and Technology Indicators (eds Wolfgang Glänzel, Henk F. Moed, Ulrich Schmoch, & Mike Thelwall) 95-127 (Springer International Publishing, 2019).
- Tregenza, T. Gender bias in the refereeing process? Trends Ecol. Evol. 17, 349-350, doi:10.1016/S0169-5347(02)02545-4 (2002).
- Larivière, V., Ni, C., Gingras, Y., Cronin, B. & Sugimoto, C. R. Global gender disparities in science. Nature 504, 211-213 (2013).
- Howe-Walsh, L. & Turnbull, S. Barriers to women leaders in academia: tales from science and technology. Stud. High. Educ. 41, 415-428, doi:10.1080/03075079.2014.929102 (2016).
- Aksnes, D. W. Characteristics of highly cited papers. Res. Eval. 12, 159-170, doi:10.3152/147154403781776645 (2003).
- Maliniak, D., Powers, R. & Walter, B. F. The gender citation gap in international relations. Intl. Organ. 67, 889-922, doi:10.1017/S0020818313000209 (2013).
- Beaudry, C. & Larivière, V. Which gender gap? Factors affecting researchers’ scientific impact in science and medicine. Res. Policy 45, 1790-1817, doi:10.1016/j.respol.2016.05.009 (2016).
- Atchison, A. L. Negating the gender citation advantage in political science. PS-Polit. Sci. Polit. 50, 448-455, doi:10.1017/S1049096517000014 (2017).
- Todeschini, R. & Baccini, A. Handbook of Bibliometric Indicators: Quantitative Tools for Studying and Evaluating Research. (Wiley-VCH, 2016).
- Austin, J. Interdisciplinarity and tenure. Science 10 January (2003).
- Evans, E. Paradigms, Interdisciplinarity, and Tenure PhD thesis, Stanford University, (2016).
- Carter, T. E., Smith, T. E. & Osteen, P. J. Gender comparisons of social work faculty using H-Index scores. Scientometrics 111, 1547-1557, doi:10.1007/s11192-017-2287-0 (2017).
- Teixeira da Silva, J. A. & Dobránszki, J. Multiple versions of the h-index: cautionary use for formal academic purposes. Scientometrics 115, 1107-1113, doi:10.1007/s11192-018-2680-3 (2018).
- Teixeira da Silva, J. A. & Dobránszki, J. Rejoinder to “Multiple versions of the h-index: cautionary use for formal academic purposes”. Scientometrics 115, 1131-1137, doi:10.1007/s11192-018-2684-z (2018).
There is NO Competing Interest.
- BradshawNatCommSI.docx
Supplementary figures S1-S4