The COSTA technique has been used in a variety of contexts including educational research, health research, and organizational research. In educational research, the COSTA technique has been used to analyze data from interviews with teachers and students. In health research, the COSTA technique has been used to analyze data from interviews with health care professionals. In organizational research, the COSTA technique has been used to analyze data from interviews with employees in order to identify patterns in the organizational culture. Furthermore, the COSTA technique is a qualitative data analysis technique that is based on the principles of grounded theory. It is an iterative process of data analysis that involves coding, sorting, and searching for patterns in the data. The COSTA technique has been used in a variety of contexts including educational research, health research, and organizational research. It is a useful tool for identifying patterns and meaning in qualitative data.
5.2.1 Key Components of the COSTA QDA (Coding, Ordering, searching for themes, Testing Analysis)
COSTAQDA is a unique qualitative data analysis software that consists of five key components: Coding, Organizing, Searching, and Testing Analysis. Researchers can use the coding component to categorize and label text segments, the organizing component to manage and structure their data, the searching component to locate specific themes or patterns, and the testing analysis component to conduct various analyses and derive insights. By integrating these components, COSTA QDA facilitates a systematic and rigorous analysis of qualitative data, enabling researchers to make informed conclusions and derive meaningful insights from their research. The following section provides a further detailed account of each of the COSTA QDA stages.
A "code" in qualitative research refers to a linguistic or visual symbol, typically a word or brief phrase, that is used to represent and encapsulate the essential and evocative attributes of a specific part of data (Onwuegbuzie, Frels, & Hwang, 2016; Saldaña, 2016). Creation of categories is essential for anchoring inductive codes that are generated directly from the source documents, be they literature pieces or interview transcripts. Researchers may decide to use anchor codes derived from key variables of their study title. These are also known as deductive codes or even theoretical codes (Nowell, et al., 2017). The first step on the COSTAQDA is to create a project as soon as the system access is procured. At this level, the researcher is concerned with three critical steps, which are pro-forma categories (deductive codes/anchor codes. apriori codes). Once there are created on the software, the next step is to upload articles that have been already coded. Articles could be literature pieces such as published or grey (gray) literature such as reports and dissertations. The researcher would have separated the codes through the process of reduction (Mezmir, 2020). This stage is represented in Fig. 2 below.
Once the project is created and categories are on display, the software allows invitations to collaborators (Church, et al., 2019). This enhances the concept of reliability and consistency of methods used in analysis when it is established that peers interacted with data through verification process, which establishes the principle of credibility according to Guba and Lincoln (1984). This stage enables intuitive, multiple and geographically dispersed real-time interactions on a cloud-based reality. This is represented by Fig. 3 below.
Once the collaborators have been invited, then the researcher may start uploading the articles in case of literature review-based study or transcripts in case of primary research. Literature needs to be coded on the actual document before uploads are done. The uploading of coded documents helps other researchers to verify where in the document corpus a particular inductive code was created. Researchers have a capability to edit naming conventions or even to delete an uploaded document. Careful consideration is to make sure that once codes have been uploaded, as per Fig. 5 below, the researchers may not delete any document as it may cause instability to the coding system. Once articles/transcripts are loaded on the system, then they may also be viewable to others with rights of access to the project. This functionality is represented in Fig. 4 below.
The final stage under the Coding stage of the COSTA technique is the actual code input on the system. This can be done by individually adding the codes through the “Add Codeinput: tab or by data importation from a large file. At this stage, researchers and their collaborators ensure that each code is assigned to a category and linked to an article. This is represented in Fig. 5 below.
The process of conducting a literature review and qualitative research involves the systematic analysis and interpretation of a vast amount of textual data. In this context, the Article Matrix, as depicted in Fig. 6, plays a crucial role in providing a visual representation of the distribution of codes across different categories and articles. This tool not only aids in organizing and summarizing the findings but also serves as a foundation for constructing a robust argument and proposition in research.
At the core of the Article Matrix lies the ability to confirm the development of codes per category and per article. This level of granularity is instrumental in understanding the nuances and patterns within the data. By categorizing codes based on themes or topics, researchers can discern the prevalent trends, recurring concepts, and underlying issues addressed in the literature. This systematic approach fosters a comprehensive understanding of the subject matter and facilitates the identification of gaps or areas requiring further exploration.
The matrix's capacity to display the distribution of codes across articles provides a clear overview of the depth and breadth of the literature under review. Each cell in the matrix corresponds to a specific code within a particular article, allowing researchers to easily trace the prevalence of a code across multiple sources. This not only aids in evaluating the richness of evidence supporting a particular concept but also enables the identification of key sources that contribute significantly to the development of specific codes.
In essence, the Article Matrix serves as a dynamic tool for literature review analysis, enabling researchers to gauge the strength of key sources. A high concentration of codes within specific articles suggests a robust foundation for a particular argument, indicating a consensus or recurring emphasis in the literature. Conversely, a scattered distribution may highlight diverse perspectives or gaps in the existing body of knowledge, prompting researchers to explore these variations and contribute fresh insights to the discourse.
Moreover, the matrix assumes a pivotal role in guiding interactions with research participants during interviews. As researchers engage with individuals to gather qualitative data, the Article Matrix becomes a reference point for identifying key players and foundational works in the field. This knowledge not only enhances the quality of interviews but also ensures that the conversations are informed by a deep understanding of the existing literature, fostering more meaningful and insightful exchanges.
The visual representation of code proportions per category through a pie chart adds another layer of analytical depth to the research. Pie charts succinctly convey the distribution of codes, offering a quick and intuitive understanding of the relative emphasis on different categories within the literature. This visualization aids researchers in identifying dominant themes and allows for a comparative analysis of the significance assigned to various aspects of the research topic.
The Article Matrix is an indispensable tool in the arsenal of a researcher conducting literature reviews and qualitative research. Its ability to showcase the distribution of codes per category and article, coupled with the visual representation of code proportions through a pie chart, elevates its utility in unravelling the complexities within the data. As a linchpin in literature review analysis, the matrix not only informs the construction of cogent arguments and propositions but also enhances the depth and precision of interactions with research participants, contributing to the overall rigor and robustness of the research endeavour.
Ordering
The process of data analysis in qualitative research is a multifaceted journey, and the crucial stage of ordering adds a layer of structure and meaning to the plethora of information gathered. Figure 7, which illustrates the Code Input stage, encapsulates the essence of ordering through the development of proforma categories, also known as A-priori codes or inductive codes. This pivotal stage not only serves as a reference material for the entire coding activity but lays the foundation for a nuanced and insightful analysis.
The ordering process begins with the meticulous creation of categories during the Code Input stage. These categories are not arbitrary; instead, they are informed by a proforma, a predefined structure that helps guide the assignment of codes. The use of A-priori codes implies a pre-established framework based on existing theories or concepts relevant to the research domain. This structured approach aids in organizing the data in a meaningful way, aligning with established theoretical frameworks and ensuring that the analysis is anchored in relevant literature.
The significance of ordering becomes apparent as researchers delve into the Code Input stage. Here, the initial codes are assigned to the pre-defined categories, contributing to the development of patterns and identifying similarities within the data. This systematic arrangement of information is instrumental in discerning the underlying structures and connections that may not be immediately apparent in the raw data. By imposing an order through the application of inductive codes, researchers enhance their ability to extract meaningful insights and develop a coherent narrative from the complexity of qualitative data.
The creation of categories and the subsequent ordering process are not isolated activities but are intricately linked to the coding stage discussed earlier. Coding involves the assignment of labels or tags to segments of data, and the creation of categories serves as a guide for this coding process. The ordered framework provided by the A-priori codes ensures that the coding is not arbitrary or haphazard but follows a systematic and purposeful approach. This synergy between coding and ordering is vital for maintaining the integrity of the analysis and facilitating a more profound understanding of the data.
In essence, the importance of ordering lies in its role as a catalyst for pattern development. As codes are input into the system, the structured framework created through A-priori codes allows for the identification of recurrent themes and emerging patterns. This ordered approach enables researchers to navigate through the complexities of qualitative data with a clear roadmap, enhancing their ability to make sense of the information at hand.
Furthermore, ordering contributes to the transparency and replicability of the research process. The use of pre-defined categories and inductive codes provides a systematic and standardized methodology that can be shared and understood by other researchers in the field. This not only adds credibility to the research but also allows for the validation of findings by peers, fostering a culture of robust and reliable qualitative research practices.
Finally, the ordering process, as illustrated in Fig. 7 during the Code Input stage, is a linchpin in qualitative data analysis. Its role in proforma development of categories, guided by A-priori codes, not only serves as a reference material but significantly enhances the pattern development during the subsequent stages of analysis. The synergy between coding and ordering ensures a systematic and purposeful approach to data analysis, ultimately leading to a more profound understanding of the research subject. As a structured framework, ordering contributes to the transparency and replicability of the research process, reinforcing the credibility of qualitative research endeavours.
Searching for themes
This stage involves searching for sub-themes from codes assigned into a category. The stage of searching for themes within the context of thematic analysis is a critical juncture where the researcher navigates the intricate landscape of codes and categories to distil meaningful patterns and insights. Thematic analysis, characterized by its inductive approach, involves moving beyond isolated cases towards broader interpretations, allowing themes to emerge organically from the data rather than being predetermined before data collection, as emphasized by Patton (1980).
At the heart of this stage is the dynamic process of connecting codes assigned to a category and identifying underlying sub-themes. Codes, in their essence, serve as the foundational elements of analysis. Metaphorically, they are like individual puzzle pieces scattered across a table, waiting to be assembled into a coherent picture. However, understanding the complete narrative requires more than the interpretation of individual codes; it necessitates the recognition of how these codes interrelate and contrast with one another.
The transition from codes to categories represents a crucial step in this analytical journey. Codes, when grouped into categories, form a structured framework that captures the (inter)relationships and contrasts within the data. Categories, therefore, act as aggregators, bringing together individual codes that are related either analytically or conceptually. This aggregation is not just a clustering exercise; it is an important intermediate step in the production of themes. Categories serve as lenses through which researchers can discern overarching patterns and meaning, laying the groundwork for the subsequent identification of themes.
Once categories have been developed, the researcher embarks on the task of producing themes. This involves a twofold process that demands a holistic perspective. Firstly, the researcher must bring together various related categories, recognizing the similarities, differences, and relationships that exist across them. This integrative step requires a comprehensive understanding of the data, allowing for the identification of overarching patterns that transcend individual categories. It is the researcher's ability to synthesize information and discern connections that sets the stage for the thematic development.
The second step in producing themes involves assigning a statement that encapsulates the essence of the interconnected categories. This theme name is not merely a label; it is a concise and inclusive representation of the content, relationships, and nuances embedded in the underlying categories. It serves as a conceptual umbrella that captures the overarching narrative within the data, responding to any observed similarities or differences. Crafting theme names is an art that requires precision and responsiveness to the intricacies of the data, ensuring that the themes resonate with the richness of the qualitative material.
Themes, by their nature, align with the conceptual or analytic goals of the study. They are not arbitrary constructs but are designed in direct response to the primary research questions or focus of the study. As the culmination of the thematic analysis process, themes offer a distilled and coherent representation of the data, providing a conceptual framework that contributes to a deeper understanding of the phenomena under investigation.
In essence, the search for themes is a transformative process where the researcher, armed with a rich tapestry of codes and categories, navigates the complexities of qualitative data to unveil meaningful patterns. It is a creative and reflexive endeavour that requires a keen eye for relationships and a nuanced understanding of the intricacies embedded in the data. The resulting themes not only encapsulate the richness of the research findings but also serve as a foundation for constructing a compelling and contextually relevant narrative.
The flow chart on the left becomes populated at the stage when initial codes within categories are assigned to sub-themes. It is through this stage that reports may also be generated in either pdf or csv formats.
Testing Analysis
This occurs when themes are created by the principal researcher and then verified by collaborators through agreements. Testing analysis is very important in establishing the rigor applied to methods that bring about conclusions in a qualitative study. This allows the research activity to provide assurance of coding consistency among collaborating coders to verify methods of establishing sub-categories by using Cohen’s Kappa statistic (Niedbalski & Slezak, 2023).
Inter-Coder Reliability (ICR) is a statistical measure used in qualitative research to assess the consistency and agreement among different coders or researchers involved in the analysis of the same set of qualitative data. It plays a crucial role in ensuring the rigor and credibility of qualitative research methods by providing a quantifiable metric for the reliability of the coding process. Cohen's Kappa is a specific statistical measure commonly employed to calculate Inter-Coder Reliability.
Cohen's Kappa is a statistic that assesses the agreement between two or more coders beyond what would be expected by chance alone. It takes into account both the observed agreement between coders and the agreement that would be expected due to chance. The formula for Cohen's Kappa produces a score ranging from − 1 to 1. A score of 1 indicates perfect agreement, 0 indicates agreement expected by chance, and negative values suggest less agreement than would be expected randomly.
In the context of qualitative research, where the interpretation of data can be subjective, assessing inter-coder reliability is essential for establishing the credibility of the coding process. When multiple researchers are involved in coding, ICR helps ensure that their interpretations and application of codes align consistently. It provides a quantitative measure to ascertain the level of agreement between coders, adding an objective layer to the inherently subjective nature of qualitative analysis.
The calculation of Inter-Coder Reliability using Cohen's Kappa involves comparing the codes assigned by each coder to the same segments of data. If there is a high level of agreement beyond what could be expected by chance, it indicates that the coding process is reliable and consistent among different coders. On the other hand, a low agreement may suggest a need for further clarification of coding instructions or a refinement of the codebook to enhance consistency.
The importance of Inter-Coder Reliability goes beyond the statistical measurement itself. It is a fundamental aspect of ensuring the methodological rigor and credibility of qualitative research. Peer debriefing, a common practice in qualitative research, involves researchers engaging in discussions and reflections with their peers to validate and enhance the trustworthiness of the research findings. Inter-Coder Reliability serves as a quantitative validation of the qualitative coding process, providing a tangible basis for peer debriefing discussions.
By using statistics to assess coding consistency, researchers can address potential biases, clarify ambiguous coding criteria, and enhance the overall rigor of the study. It also adds transparency to the research process, allowing other researchers to understand and evaluate the reliability of the coding framework. This is particularly important in collaborative research efforts where multiple coders may bring different perspectives and interpretations to the data.
Finally, Inter-Coder Reliability, often measured using Cohen's Kappa, is a crucial component of ensuring the credibility of qualitative research methods. It provides a quantitative assessment of the consistency and agreement among coders, offering a solid foundation for establishing the rigor of the coding process. This statistical approach, when combined with peer debriefing and other qualitative validation methods, contributes to the overall trustworthiness and reliability of qualitative research findings. Figure 9 represents the functionality for ICR calculation, while Fig. 10 represents a final product of a qualitative research activity – the themes.