This section presents the bibliometric and network descriptive results obtained from the two datasets. In Section 3.1, we provide some basic information about the shared amount of literature between the two datasets. In Section 3.2, we present a so-called performance analysis where we include the number of publications, the absolute number of citations, and the weighted number of citations of the two datasets (Ayoko et al., 2022). In Section 3.3, we present the core visual mapping of different bibliometric indicators (VOSviewer) together with the quantitative measures calculated on the full network (Gephi). We comment in detail and discuss similarities and differences in the next section.
3.1 Shared literature
The two datasets contain 156 titles in common. Instead, the remaining 503 articles in the direct database and 417 articles in the indirect database have different titles. This means that only 14,5% of the total number of unique research articles (1,076) belongs to both datasets. This first indicator shows that the two datasets share a relatively small number of articles.
3.2 Performance analysis
Following Ayoko et al. (2022) we calculate four performance measures for the two datasets:
-
The total number of publications per year.
-
The absolute number of citations per year.
-
The 20 most cited articles according to the total number of citations an article has received so far6 (Scopus).
-
The 20 most cited articles according to the weighted citations indicator, which is calculated as the ratio between the number of citations a article received and the total number of citations received by all the articles published within the same year. This indicator is meant to normalize the citations over the time dimension since it is more likely that articles published earlier in time have a higher absolute number of citations compared to articles published in more recent years.
Figure 3 presents the total number of publications per year for each dataset. We can observe that the path of the two datasets is approximately the same until 2012. After that year, the publications identified through the direct method (160, 2017 excluded) were twice compared to the ones identified through the indirect method (82). However, this might be a structural component since any literature review always comes after the publication of the primary sources, as indicated by the significant drop in 2017 for the indirect method as well.
Figure 4 shows the absolute number of citations per year for each dataset. In this case, the situation is different compared to the number of publications. The total (average) number of citations (per year) of the articles identified through the direct method is 26,785 (582), and the total number of citations of the articles identified via the indirect method is 114,851 (2,496). Moreover, it is possible to observe two main picks of citation of the indirect articles in 1995 and 1998.
Table 2 ranks the 20 most relevant articles according to the total number of citations they have received so far7 according to Scopus. It is possible to observe that the most cited direct article is Olson and Olson (2000), cited 1,119 times. The most cited indirect article is Baumeister and Leary (1995), which has a total of 10,727 citations. Observing Table 2, it is possible to infer three empirical regularities: 1) the 20 most relevant articles of the indirect method have about ten times more citations than the articles in the same ranking identified through the direct method; 2) the most cited research article of the direct method has almost the same number of citations of the twentieth most cited article of the direct method; 3) there are no articles in common between the two rankings. The differences observed in Fig. 3 and Table 2 might be explained by the fact that the review articles probably include in their references (indirect method) seminal works or methodological contributions that are not picked through the direct method that uses specific keywords.
Table 2
– Most relevant articles ranked according to the absolute number of citations
N° | Direct method | Citations | Indirect method | Citations |
---|
1 | Olson and Olson (2000) | 1,199 | Baumeister and Leary (1995) | 10,727 |
2 | Gajendran and Harrison (2007) | 706 | Nahapiet and Ghoshal (1998) | 9,633 |
3 | Gilmore and Pine (1997) | 600 | Meyer and Alien (1991) | 4,226 |
4 | Baltes et al. (1999) | 485 | Hackman and Oldham (1976) | 3,971 |
5 | Kossek, Lautsch and Eaton (2006) | 448 | Davis, Bagozzi, and Warshaw (1992) | 3,611 |
6 | Cascio (2000) | 351 | Hackman and Oldham (1975) | 3,554 |
7 | Lapierre and Allen (2006) | 335 | Goldberg (1990) | 3,497 |
8 | Allen et al. (2013) | 304 | Goodhue and Thompson (1995) | 2,908 |
9 | Malone and Laubacher (1998) | 267 | Harter, Schmidt, and Hayes (2002) | 2,101 |
10 | Straub and Karahanna (1998) | 254 | Quinn and Rohrbaugh (1983) | 1,608 |
11 | Hill et al. (1998) | 251 | Kimberly and Evanisko (1981) | 1,478 |
12 | Hornung, Rousseau and Glaser (2008) | 234 | Sproull and Kiesler (1986) | 1,446 |
13 | Staples, Hulland, and Higgins (1999) | 232 | Akerlof (1982) | 1,425 |
14 | Cooper and Kurland (2002) | 230 | Trist and Bamforth (1951) | 1,421 |
15 | Rau and Hyland (2002) | 223 | Ashforth, Kreiner, and Fugate (2000) | 1,285 |
16 | Golden, Veiga, and Dino (2008) | 218 | Morgeson and Humphrey (2006) | 1,236 |
17 | Shockley and Allen (2007) | 202 | Cramton (2001) | 1,232 |
18 | Leslie et al. (2012) | 198 | Faraj and Sproull (2000) | 1,116 |
19 | Gittell (2000) | 194 | Thomas and Ganster (1995) | 1,099 |
20 | Sullivan and Lewis (2001) | 191 | Allen (2001) | 1,074 |
However, as already mentioned, the total number of citations can be biased depending on the publication year. Indeed, ceteris paribus, it is more likely that articles published earlier in time have more citations than articles published more recently. A common practice to mitigate the time effect is to take the ratio between the absolute number of citations of each article and the total number of citations of all the articles published in the same year within the database. Table 3 shows the 20 most relevant articles ranked according to the weighted number of citations. In this case, it is possible to observe that: 1) several articles published during the second half of the 20th century have a score equal to 1.00 because they are the only articles published in that specific year (within the database); 2) there are two articles in common (Gilmore and Pine, 1997; Golembiewski et al., 1974).
Table 3
– Most relevant articles ranked according to the weighted number of citations
N° | Direct method | Weighted citation | Indirect | Weighted citation |
---|
1 | Golembiewski, Hilles and Kagno (1974) | 1.00 | Golembiewski, Hilles and Kagno (1974) | 1.00 |
2 | Metzger and Von Glinow (1988) | 1.00 | Golembiewski and Proehl (1978) | 1.00 |
3 | Winett and Neale (1981) | 1.00 | Trist and Bamforth (1951) | 1.00 |
4 | Chess and Lyman (1969) | 1.00 | Lodahl and Kejnar (1965) | 1.00 |
5 | Gordon and Van Arsdale (1986) | 1.00 | Pierce and Newstrom (1980) | 1.00 |
6 | Cooper and Davidson (1983) | 1.00 | Hackman and Oldham (1976) | 0.99 |
7 | Gray, Nilles and Lopez (1977) | 1.00 | Sproull and Kiesler (1986) | 0.98 |
8 | Olson (1989) | 0.98 | Davis, Bagozzi and Warshaw (1992) | 0.97 |
9 | O'Conaill, Whittaker and Wilbur (1993) | 0.72 | Hackman and Oldham (1975) | 0.97 |
10 | Katz (1987) | 0.59 | Kimberly and Evanisko (1981) | 0.95 |
11 | Hartman, Stoner and Arora (1991) | 0.58 | Quinn and Rohrbaugh (1983) | 0.93 |
12 | Olson and Olson (2000) | 0.58 | Nahapiet and Ghoshal (1998) | 0.87 |
13 | Pierce and Newstrom (1982) | 0.55 | Harris and Schaubroeck (1988) | 0.86 |
14 | Yap and Tng (1990) | 0.55 | Goldberg (1990) | 0.85 |
15 | Gilmore and Pine (1997) | 0.54 | Meyer and Alien (1991) | 0.79 |
16 | Thompson and Scalpone (1985) | 0.50 | Baumeister and Leary (1995) | 0.68 |
17 | Elling (1985) | 0.50 | Akerlof (1982) | 0.66 |
18 | Hirschhorn and Gilmore (1992) | 0.47 | Wiley (1987) | 0.57 |
19 | Winett, Neale and Williams (1982) | 0.43 | Gilmore and Pine (1997) | 0.57 |
20 | Olszewski and Mokhtarian (1994) | 0.42 | Huber (1984) | 0.55 |
3.3 Science mapping and network comparison
In the following analysis, we present two types of related results: a) a reduced visual map of the most relevant (core) constituents under investigation using the software VOSviewer; b) a calculation of the network properties on the whole set of connected8 constituents using the software Gephi. This double perspective (zoom-in and overview, respectively) provides more information on the basis of which it is possible to draw a conclusion on the analogy or dissimilarity between the datasets.
Bibliographic coupling of documents
Bibliographic coupling (Kessler 1963) produces a measure of similarity between documents, authors, or journals based on the references they have in common. Assuming that related and similar constituents draw upon the same knowledge basis, the bibliographic coupling indicator determines the similarity between constituents. Essentially, the higher the number of quoted references in common between constituents, the stronger their similarity in terms of content. However, the bibliographic coupling indicator is a static metric because the relationship between two constituents cannot change over time.
Table 4
– Bibliographic coupling of articles, full network properties
Measure | Direct | Indirect |
---|
Number of connected nodes | 534 (out of 659) | 493 (out of 573) |
Number of edges | 20,789 | 16,858 |
Number of clusters | 14 | 10 |
Average degree | 77.86 | 68.38 |
Graph density | 0.146 | 0.139 |
Modularity | 0.293 | 0.348 |
Average clustering coefficient | 0.564 | 0.507 |
Average shortest path length | 2.129 | 2.039 |
Betweenness centrality | 3,585.40 Stavrou and Ierodiakonou (2011) | 2,886.53 Gajendran and Harrison (2007) |
Closeness centrality | 0.659 Gajendran and Harrison (2007) | 0.658 Gajendran and Harrison (2007) |
Figure 5 visually compares the bibliographic coupling networks of the 20 most relevant articles (according to VOSviewer) between the direct and the indirect methods. It is possible to observe that the central node of the networks coincides (Gajendran and Harrison 2007). This is also confirmed by the betweenness centrality and closeness centrality of Table 49. From the combined information of Fig. 5 and Table 4, we can assert that the nodes (articles) of the direct method tend to cluster more together and be more connected than those retrieved through the indirect method.
This is plausible if we consider that all the articles of the direct method share some commonalities in terms of topic (they were all retrieved through the same string of keywords), whereas the indirect database contains also seminal and methodological articles that might not be directly related to FWAs.
Another measure produced by the bibliographic coupling algorithm of VOSviewer is the total link strengths. The total link strength is an indicator of importance that measures how much an article is embedded in the knowledge structure of the field (of FWAs). The higher the link strength, the more relevant the article is within the network. Table 4 shows the 20 most relevant articles according to the total link strength calculated on the whole network.
Table 5
– Most relevant articles ranked according to the total link strength (tls)
N° | Direct method | tls | Indirect method | tls |
---|
1 | Gajendran and Harrison (2007) | 962 | Gajendran and Harrison (2007) | 883 |
2 | Golden (2007) | 805 | Golden (2007) | 758 |
3 | Golden and Fromen (2011) | 766 | Beauregard and Henry (2009) | 718 |
4 | Gajendran, Harrison and Delaney-Klinger (2015) | 744 | Golden and Fromen (2011) | 671 |
5 | Allen et al. (2013) | 731 | Siha and Monroe (2006) | 667 |
6 | Martinez and Gómez (2013) | 723 | Golden (2006) | 649 |
7 | Golden (2012) | 721 | Golden and Veiga (2008) | 604 |
8 | Taskin and Bridoux (2010) | 715 | Gajendran, Harrison, and Delaney-Klinger (2015) | 593 |
9 | Sardeshmukh, Sharma, and Golden (2012) | 713 | Taskin and Bridoux (2010) | 591 |
10 | Siha and Monroe (2006) | 699 | Golden and Veiga (2005) | 587 |
11 | Golden (2006) | 664 | Golden (2012) | 582 |
12 | Golden (2009) | 650 | Allen et al. (2013) | 555 |
13 | Peters and Heusinkveld (2010) | 628 | Golden and Raghuram (2010) | 550 |
14 | Golden and Veiga (2005) | 628 | Golden, Veiga, and Dino (2008) | 547 |
15 | Thompson, Payne, and Taylor (2015) | 614 | Sardeshmukh, Sharma, and Golden (2012) | 539 |
16 | Lautsch, Kossek and Eaton (2009) | 601 | Batt and Valcour (2003) | 505 |
17 | Golden and Raghuram (2010) | 600 | Golden, Veiga and Simsek (2006) | 505 |
18 | Campbell and Heales (2016) | 581 | Martinez and Gómez (2013) | 487 |
19 | Mayo et al. (2016) | 559 | Kelly et al. (2014) | 479 |
20 | Golden, Veiga, and Dino (2008) | 555 | Pérez Pérez et al. (2004) | 463 |
The bibliographic coupling analysis in terms of total link strength produces two results: 1) the first two articles in the ranking of the indirect and the indirect methods are the same, and they have a comparable total link strength (Gajendran and Harrison, 2007; Golden, 2007); 2) out of 20 articles, 14 references are shared between the two datasets, showing a deep similarity in the core literature. According to Ayoko et al. (2021), joining the titles from the three ranking methods (Table 2 and Table 3 of the performance analysis and Table 5 based on bibliographic coupling), it is possible to observe that the direct method generates 55 unique articles (five duplicates), and that the indirect method generates 49 unique articles (11 duplicates). Moreover, 15 titles, that is 16,8% of the total number of the core research articles (89), belong to both sets obtained by joining the ranking methods. This percentage is comparable to the one of shared literature (Section 3.1).
Bibliographic coupling of sources
Through the bibliographic coupling indicator, it is also possible to relate the fields of research represented by the journals in which the articles are published. In the direct database, it is possible to count 318 total sources, and in the indirect database, it is possible to count 230 total sources. For the visual mapping (VOSviewer) only the journals with at least five published articles were selected.
Table 6
– Bibliographic coupling of sources, full network properties
Measure | Direct | Indirect |
---|
Number of connected nodes | 260 (out of 318) | 197 (out of 230) |
Number of edges | 7,768 | 5,743 |
Number of clusters | 13 | 8 |
Average degree | 59.75 | 58.30 |
Graph density | 0.231 | 0.297 |
Modularity | 0.170 | 0.175 |
Average clustering coefficient | 0.698 | 0.692 |
Average shortest path length | 1.913 | 1.776 |
Betweenness centrality | 1,951.54 New Technology, Work and Employment | 925.02 Journal of Organizational Behavior |
Closeness centrality | 0.799 The International Journal of Human Resource Management10 | 0.841 Journal of Organizational Behavior |
From Fig. 6 and Table 6, we observe the following. The 26 core sources of the direct method are the outlet of 203 publications (30.8% of the total direct articles). New Technology, Work and Employment (24 publications and 1,338 citations) and the International Journal of Human Resource Management (24 publications and 769 citations) constitute the core sources. Out of 13,249 total citations received by the journals meeting the threshold of five publications, the Journal of Applied Psychology is the one with the highest number of total citations, 1,831. The 25 core sources of the indirect method are the outlet of 245 publications (42.7% of the total indirect articles). The Journal of Organizational Behavior (22 publications and 1,945 citations), followed by Organization Science (20 publications and 5,161 citations) and Human Relations (20 publications and 3,036 citations) constitute the core sources. Out of 57,927 total citations received by the journals meeting the threshold of five publications, the Journal of Applied Psychology is the one with the highest number of total citations, 12,397. Out of 39 unique sources in the core network (Fig. 6), 12 (30.7%) are in common.
A comparison between the source networks indicates that their underlying structure is quite different, with the network of the indirect method being less differentiated and more compact. This result is expected because an author, when writing a review article, is often led by her experience in terms of identifying potentially relevant sources.
Bibliographic coupling of authors
The bibliographic coupling indicator can also produce a network relationship between authors11 based on the references they share in the articles they authored. This analysis can reveal authors who, despite not directly collaborating, deal with the same areas of research because of referencing to the same knowledge basis. For the visual mapping (VOSviewer) only the authors with at least three published articles were selected. This filter restricts the sample to 46 authors (out of 1,281) in the direct method database and 48 (out of 1,063) in the indirect method database.
Table 7
– Bibliographic coupling of authors, full network properties
Measure | Direct | Indirect |
---|
Number of connected nodes | 1,077 (out of 1,281) | 945 (out of 1,063) |
Number of edges | 93,343 | 78,557 |
Number of clusters | 29 | 19 |
Average degree | 173.39 | 166.25 |
Graph density | 0.161 | 0.176 |
Modularity | 0.367 | 0.426 |
Average clustering coefficient | 0.645 | 0.596 |
Average shortest path length | 2.058 | 1.981 |
Betweenness centrality | 13,068.02 Golden T.D. | 12,402.87 Golden T.D. |
Closeness centrality | 0.739 Golden T.D. | 0.758 Golden T.D. |
In the restricted network only 17,5% (14 out of 80) of the authors are present in both lists. Moreover, both Fig. 7 and Table 7 underscore the centrality of Timothy Golden in FWAs research. This centrality is further confirmed by Table 8, which ranks the five most relevant authors based on VOSviewer total link strength. However, despite these shared aspects regarding influential authors, the two networks exhibit significant heterogeneity concerning authors and their connections.
Table 8
– Most relevant authors ranked according to the total link strength
| Direct | Indirect |
---|
N° | Author | Docs | Citations | tls | Author | Docs | Citations | tls |
1 | Golden, T.D. | 10 | 1,123 | 14,117 | Golden, T.D. | 10 | 1,196 | 13,344 |
2 | Allen, T.D. | 8 | 1,149 | 9,896 | Allen, T.D. | 11 | 2,255 | 11,280 |
3 | Peters, P. | 8 | 300 | 6,437 | Kelly, E.L. | 6 | 961 | 7,513 |
4 | Masuda, A.D. | 3 | 318 | 5,628 | Kossek, E.E. | 6 | 992 | 6,422 |
5 | Shockley, K.M. | 4 | 611 | 5,421 | Moen, P. | 4 | 740 | 6,092 |
Co-citation analysis of documents
Co-citation occurs when two references are cited together by another third paper (Small 1973). This indicator basically measures the frequency with which two documents are cited simultaneously by other documents. Focusing on the similarities in the references12 (secondary sources) of the main database co-citation analysis depicts the structure of the background knowledge of the main set of publications. This measure is dynamic since the co-citation relation between two papers can change over time after new contributions are published because these might connect new documents or areas of research. For the network visualization (Fig. 8) we consider only references cited at least 10 times.
Table 9
– Co-citation of articles, full network properties
Measure | Direct | Indirect |
---|
Number of connected nodes | 20,456 (out of 27,170) | 22,717 (out of 27,118) |
Number of edges | 755,119 | 988,176 |
Number of clusters | 128 | 127 |
Average degree | 73.82 | 86.99 |
Graph density | 0.004 | 0.004 |
Modularity | 0.878 | 0.901 |
Average clustering coefficient | 0.968 | 0.969 |
Average shortest path length | 3.681 | 3.962 |
Betweenness centrality | 17,933,175.35 Bailey and Kurland (2002) | 20,992,324.32 Bailey and Kurland (2002) |
Closeness centrality | 0.426 Bailey and Kurland (2002) | 0.401 Bailey and Kurland (2002) |
In the core network, among the 18 unique titles, 6 references (33%) are common, indicating a substantial overlap. Bailey and Kurland (2002) emerged as a cornerstone article in both networks. Additionally, when examining the references of the references, Table 9 illustrates very similar full network properties, including, again, Bailey and Kurland (2002) as a central node.
Co-citation analysis of sources
In the case of sources of documents, the co-citation network represents the structure of the relationship between the journals that have been a common reference point of the main databases. This network provides a deeper view of the underlying literature on the basis of the two sets of articles identified through the two methods. For the network visualization (Fig. 9) we fixed a minimum threshold of 50 citations for a source to be included in the analysis. This means that to be taken into consideration, a source (journal) has to be cited through one or more documents published in the same outlet at least 50 times in total. Out of 10,878 (9,848) sources at the basis of the dataset identified through the direct (indirect) method, only 55 (54) are above the selected threshold.
Table 10
– Co-citation of sources, full network properties
Measure | Direct | Indirect |
---|
Number of connected nodes | 10,747 (out of 10,878) | 9,810 (out of 9,848) |
Number of edges | 407,157 | 445,090 |
Number of clusters | 81 | 70 |
Average degree | 75.77 | 90.74 |
Graph density | 0.007 | 0.009 |
Modularity | 0.390 | 0.440 |
Average clustering coefficient | 0.907 | 0.897 |
Average shortest path length | 2.473 | 2.424 |
Betweenness centrality | 3,640,633.29 Journal of Applied Psychology | 2,991,250.54 Academy of Management Review |
Closeness centrality | 0.619 Journal of Applied Psychology | 0.640 Academy of Management Review |
In both the core networks the Journal of Applied Psychology plays a relevant role, and, most importantly, they share 43 sources in common, which is almost 65% of the unique journals (66). Through the references of the direct method dataset the Journal of Applied Psychology is cited 827 times, the Journal of Vocational Behavior 535 times, and the Academy of Management Journal 460 times. Through the references of the direct method dataset, the Journal of Applied Psychology is the first one in the ranking and is cited 1,009 times, the Academy of Management Journal is quoted 610 times, and the Journal of Vocational Behavior is cited 482 times. In the indirect method, the relevance of the Academy of Management Review is also highlighted by the full network properties (Table 10)
Co-occurrences of author keywords
Another measure to establish a relationship between documents is the text content of the articles themselves. The analysis of the co-occurrence of (key)words (Caputo et al. 2021) is based on the idea that similarities can be identified by the associations established between articles’ (key)words (López-Fernández et al., 2016). This semi-content analysis is also useful for identifying and mapping thematic areas – clusters – that make up the theoretical building blocks for the field under inquiry (Manesh et al., 2020). In our analysis, we consider only the authors’ keywords. For the network visualization (Fig. 10) we fixed a minimum threshold of three times that a keyword must appear among all the considered documents. In the direct method list 108 out of 1,416 keywords met the threshold. Out of 1,033 keywords present in the list of articles identified through the indirect method, 77 keywords met the threshold. This is expected because the direct articles focus more on the topic reflected by the string of words used for the search criteria.
Table 11
– Co-occurrences of author keywords, full network properties
Measure | Direct | Indirect |
---|
Number of connected nodes | 1,252 (out of 1,416) | 852 (out of 1,033) |
Number of edges | 4,429 | 3,150 |
Number of clusters | 71 | 54 |
Average degree | 7.07 | 7.39 |
Graph density | 0.006 | 0.009 |
Modularity | 0.692 | 0.698 |
Average clustering coefficient | 0.886 | 0.882 |
Average shortest path length | 3.655 | 3.540 |
Betweenness centrality | 307,526.21 Telework | 158,054.68 Telecommuting |
Closeness centrality | 0.474 Telework | 0.491 Telecommuting |
In the full direct network, ‘telework’ appears 86 times, ‘telecommuting’ 74 times, ‘gender’ 26 times, ‘flexibility’ 23 times, and ‘teleworking’ 20 times. In the full indirect list, ‘telecommuting’ appears 69 times, ‘telework’ 41 times, ‘job satisfaction’ 22 times, ‘teleworking’ 20 times, and ‘work-family conflict’ 14 times. Out of 141 unique keywords, a total of 44 (about 31.2%) keywords were in common. As evident from the full network properties, it is notable that the keywords 'telework' and 'telecommuting' serve as central nodes in the direct and indirect network, respectively.