Teamwork assessment in undergraduate Engineering courses

doi:10.21203/rs.3.rs-4984098/v1

Download PDF

Research Article

Teamwork assessment in undergraduate Engineering courses

https://doi.org/10.21203/rs.3.rs-4984098/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

This study explores the effective design and development of a rubric that enhance learning and collaboration in project-based learning (PBL) environments. By gathering surveys and feedback from 152 Engineering undergraduate students and 12 staff members, we gained insights into the key elements and features of a rubric suitable for assessing teamwork skills in PBL contexts. The students emphasised the fairness and transparency on how the rubric was communicated and explained in addressing all assessment requirements, so that these were clearly understood by both academics and students. The academics noted the important elements of good teamwork, including anticipation, initiative, helping others, communicating, problem-solving, and actively seeking and suggesting solutions that show critical thinking and innovation. The outcome of our study supports the longer-term development of rubrics and their broader application as tools for assessing teamwork. These insights benefit researchers examining rubrics and seeking to understand academics' and students' perspectives.

assessment rubric

teamwork

project-based learning

engineering design

Reports have emerged over the past decade that identify the “21^st-century skills” as those students need to succeed in the global labour market [1-4]. These skills include problem solving, critical thinking, teamwork, communication and digital skills [3, 5-7]. Previous studies have reported that many learning activities or assessment tasks using small groups enable students to develop cooperative work habits. In addition, during project-based learning, students engage with a variety of learning strategies such as problem-based, collaborative and independent learning, which are then transferable to their future professional endeavours [8-11]. Hence, many universities have incorporated a project-based learning (PBL) approach into their curricula and assessment processes to enable students to cultivate these 21^st-century skills.

To date, marking rubrics have been widely adopted in higher education to provide feedback on oral presentations, essays, theses writing and peer assessments [12-16]. Research has found that using such rubrics assists staff in improving the efficiency of the marking process. For the students, the use of rubrics reassures them of the fairness and transparency of the marking system [16, 17]. Rubrics have also been found to support student learning and improve student performance [17, 18]. However, there is a lack of consensus in higher education institutions on appropriate rubrics for assessing and providing feedback on teamwork and, specifically, between what a rubric says and how a reader (student or academic staff) interprets it in order to assess the quality of both individual work and the team processes in a group project. If students are expected to develop the required collaborative skills to function effectively in the 21^st-century workplace, the development of such rubrics is of vital importance, for both academics and students, to ensure that these requisite skills have been acquired.

In the evolving landscape of educational assessment, our study introduces a critical element: a specially designed rubric. While acknowledging the significance of PBL and teamwork in contemporary education, this study explored the role of rubric in these contexts. It aims to seek to understand the perspectives of both academic staff and students thought about it and on its effectiveness and utility.

Project-based learning (PBL) and assessing teamwork

Teaching 21^st-century or transferable skills in a classroom is challenging. PBL is becoming an increasingly popular approach that universities are using to encourage students to work in groups to solve real-world problems. Through PBL, students can be required to work collaboratively to conduct research, gather information to devise a solution to a problem, reflect on their learning and, finally, communicate their findings. Thus, the main features of PBL may include a team project, associated learning activities (e.g. group formation, task construction, practical classes, problem solving, people and time management).

When students from diverse backgrounds work together, disagreement is unavoidable, typically because values, perspectives and opinions are different and have not been openly discussed. The effective incorporation of diverse values and perspectives remain challenge in PBL, necessitating skills in effective communication, people- and time-management skills and be able to develop a fair sharing of tasks [19, 20]. To fully realise the benefits of teamwork in PBL, it is essential to have robust assessment measures. These measures should evaluate student’s teamwork ability and provide meaningful feedback to guide and support their learning process [21].

In order to assess teamwork performance and final outcomes, various tools have been considered, such as an observational approach [22, 23], questionnaires [24, 25], peer assessment [20, 26] and rubrics [27-29]. The effectiveness of these tools varies and their limitations must be considered to ensure accurate measurement of the students’ knowledge and performance [14, 30, 31].

Rubrics as a focal point of study

Rubrics, with their structured and standardised format, are designed to provide clear benchmarks for evaluating performance and contribution; yet, they are not without their challenges in their application, particularly when misinterpreting or oversimplifying complex processes involved in teamwork [30, 31]. In the context of assessing teamwork in PBL, rubrics typically comprise three key elements (see Figure 1): 1. skill or performance categories, which define the specific aspects of teamwork being assessed; 2. scoring or achievement level for each category, offering a graded scale to measure performance; and 3. descriptors for each score, detailing the criteria met at each level of achievement [32-36].

Figure 1. Example of the key elements for a rubric that contains the performance category, scoring levels and descriptors.

When properly used with clear key elements, good rubrics not only clarify expectations of teaching staff of student performance [15, 37, 38], but also provide staff with a systematic approach and coherence in the marking process by avoiding arbitrary grading and subjective assessment of students’ work. The adoption of rubrics can assist staff in improving the efficiency of the marking process, and aid in reducing variability in grading [39, 40]. Furthermore, rubrics are useful as tools for staff to communicate expectations to students and provide feedback that informs the students of what they did well and which areas need improvement [39, 41].

Various authors have found that undergraduate and graduate students used rubrics to evaluate their own work in order to obtain immediate feedback, particularly in estimating their own grades and improving the quality of their work before submitting assignments [15, 29, 35, 39, 42]. In addition, students were also able to utilise the feedback from one rubric in subsequent assignments. Students also regarded the marking process as fairer and more transparent when rubrics were used.

Despite these benefits, challenges persist in the application of rubrics for assessing teamwork skills and the output quality of the output of student team [26-29]. Few studies have highlighted issues such as grading inconsistencies [26] and challenges in using an instructor/student co-constructed rubric for assessment, especially in dealing with peer and failure in communicating ideas effectively to their peers [29].

These challenges underline the necessity for both academic and students to have unified understanding of about the overall purpose of the rubric and the way how the individual elements of the rubrics are interpreted [35, 43-45]. Furthermore, there is generally a lack of consensus among staff and students on the scope and key elements of the rubrics [35, 37]. Therefore, in order to maximize the effectiveness of rubrics, it is clear that both students and academic staff need to have a common understanding of any rubric’s assessment criteria.

Rationale for the present study

The literature suggests teamwork is one of the essential employability skills in which students must succeed in an increasingly complex and fast-changing global market. In the teaching and learning space, several approaches and tools have been developed for assessing students’ comportment and performance in teamwork tasks. One of these tools is the rubric. However, to date, there is a lack of consensus on the key characteristics of a good rubric for assessing and providing feedback on teamwork. Hence, this study sought to investigate academics’ and students’ attitudes toward a new rubric for assessing collaborative teamwork. The tasks that students undertook in the teamwork were designed using a PBL approach.

Recognising the increasing importance of teamwork skills in today's global market. It aims to rigorously assess and refine an educational rubric, establishing it as a vital tool for evaluating and enhancing student performance in teamwork-oriented tasks. This research is designed to investigate the optimal design and development of rubrics within educational assessments. It focuses particularly on their application in PBL environments, where effective learning and collaboration are common. By doing so, the study seeks to bridge the gap between theoretical rubric research and practical application in academia. The rationale for this study has been strengthened to reflect its dual objectives: to offer academics practical insights for utilising rubrics in enhancing student collaboration, and to contribute to the broader research on rubric development and implementation. Specifically, this study sought to answer the following questions:

RQ1: What are the key elements of a rubric that guide and improve student learning and performance in teamwork activities?
RQ2: What are the key features of a rubric as an instrument to provide students with feedback that can be used to improve performance in future assessments?

Rubric development

This study involved adapting an online rubric, designed by the University of Wisconsin, which covered students’ participatory experiences of, and processes used during, teamwork, as well as the associated learning activities such as “conduct research” and “solve problems”. This adaptation involved a group of academic staff with different expertise, but with similar experience in delivering PBL, to develop a draft rubric (see Appendix A). This adaption process involved modifying the rubric considering technical and pedagogical aspects of the design and implementation specific of our objectives. For example, we changed the scoring system to suit our project, and we refocused the skills being assessed to include professionalism, work ethic, and communication skills. The draft rubric contained a performance category for teamwork, four scoring or achievement levels (exemplary, proficient, marginal and unacceptable) and their corresponding criteria or descriptors.

Data collection and instrument

The study was conducted in the School of Engineering, RMIT University, Melbourne, Australia, from July to December 2018. The participants, including both students and academic staff, were invited to participant in a survey using an anonymous written questionnaire. The questionnaire consisted open-ended questions specifically designed to gather feedback that aligns with the research questions. The following are the example of survey questions relevant the findings of this paper:

What is your overall opinion of the example rubric?
How did you find the definitions of what key performance looks like at each level, particularly for excellent performance? What are the qualities that distinguish an excellent response from other levels?
How useful do you think the rubrics are in providing meaningful and timely feedback, particularly in terms of identifying areas needing work and strengths for improving performance?
How could the above rubrics be made better in terms of enhancing their role as a feedback tool?

Participants

The student participants were drawn from the first-, second- and third-year Biomedical Engineering design courses/subjects (i.e., three separate courses/subjects) and a first-year course in Computer-Aided Design in Mechanical, Automotive and Advanced Manufacturing and Mechatronics Engineering. These courses were specifically chosen because they significant emphasis on teamwork in assessments, with at least 25% of the assessment related to teamwork. In these courses, students are asked to form groups (between three-to-five members per group) to carry out design projects (see Table 1 for the proportion of teamwork assign for each course).

Table 1. Design courses in study, number of enrolment and proportion of grade for teamwork assessment

Course/Subject title	Program(s)/Degree(s)	Expected enrolment	% of group assessment
Introduction to Biomedical Engineering and Design	Biomedical Engineering	35	60
Biomedical Computer Aided Design	Biomedical Engineering	30	25
Biomedical Engineering Design and Practice 2	Biomedical Engineering	27	70
Computer Aided Design	Engineering Degrees (Mechanical, Automotive, Advanced Manufacturing and Mechatronics)	150	25

The student surveys were administered by a research officer, who was not teaching the courses, at the end of a two-hour class in Week 5 of a twelve-week teaching period. The survey took five-to-ten minutes to complete so as not to disrupt the classes. If the students decided to take part, they were asked to anonymously complete a checklist of basic demographic information and provide responses to a set of open-ended questions relating to the rubric which was provided with the survey. The main analysis used data from a total of 152 (62.8%) students (from the four courses in Table 1) participated in the survey (119 males and 33 females), with an average age of 19.9 ± 3.3 years (range 17-41 years).

All staff members in the School of Engineering, particularly those who involved in teaching the relevant courses, were invited to participate in the survey. To conduct the staff survey, our research team made initial contact with them either face-to-face or by email. We provided them with a briefing on the study and sent each interested staff member a copy of an information sheet and consent form. Staff members who chose to participate were then asked to complete a questionnaire and return it to the research team. A total of 12 staff participated in the survey: 11 were permanent staff and one held a casual position. On the average, they had nine years of experience in teaching and coordinating courses at the tertiary level.

Ethics

Prior to conducting the survey, approval was obtained from the RMIT Human Research Ethics Committees (SEHAPP 47-18). Each participant was provided with the standard Participant Information Sheet and Consent form. Their consent was collected after obtaining their signature on the consent form. This ensured that ethical guidelines were followed and confirmed that participants were well-informed about the study and voluntarily agreed to take part. We also emphasised to the students that their participation was voluntary and would not affect their assessment grades and progress if they chose to withdraw from the study. They were offered the opportunity to enter into a draw for an iPad; or book and movie vouchers; as a small incentive.

Data coding and analysis

The project assistant entered all data for both the student and staff surveys into Excel spreadsheets. Survey data were then analysed using the NVivo 12 software package. Based on the approach used by Saldana [46], the transcriptions were analysed in an iterative process using the following steps. First, we read through all the participants’ responses and familiarised ourselves with their responses. Then, we conducted a preliminary analysis during which codes were generated [47]. Our research team reread the codes to identify emergent themes related to the codes. We continuously reviewed the major themes to determine whether they reflected the codes accurately. After that, we analysed the themes to identify the key findings.

Findings

For the initial reading of the data simple coding was used. In terms of the participants’ overall perceptions of the rubric a simple positive/negative response was coded. In terms of how the participants found the definitions of the different achievement levels, the data were coded in terms of whether the participants found the definitions useful and whether they found the levels reflected their own understanding of what, for example, an “excellent” achievement was. In terms of using the rubric to provide feedback, the data were coded in terms of whether the participants found the rubric useful or not. A deeper coding was then performed to provide a more detailed analysis. Through this analysis, 55.7% of students and 58% of staff have identified a number of issues that could be improved. These are presented below.

Key elements of teamwork rubrics for enhancing learning and performance

Both staff and students commented that the clarity and structure of the rubric could be improved with the use of clear guidelines, clear categories, objective criteria, scoring system, and robustness. These perceptions are listed below.

Clear guidelines

The students (10%) and academic staff (17%) indicated that the rubric needed clearer instructions and more specific measures. For example, some students stated:

There must be an appendix or detailed note to explain each and every part of the rubric more clearly. [Surveyed student #35]

Easy to understand what I am marked on, could use some more elaboration for a few of them. [Surveyed student #57]

Pretty useful but is also a bit vague, maybe a teacher should go through the rubric to be more specific about the expectations required to get the highest marks. [Surveyed student #122]

Comments were offered by staff members on the rubric lacking objective measures, leading to students possibly thinking it was too abstract and subjective:

Overall, it is good and detailed. Some of the items can be more specific and measurable, e.g. 'sometimes', 'occasionally', can be quantified in some time frame. My experience with students is that they could like to see few examples under each category or else, they feel a bit disconnect thinking it is too abstract or subjective. [Academic #9]

Our participants, both staff and students, highlighted the importance of providing clearer definitions for the items being measured. This called for more detailed and explicit guidance within the rubric, with suggestions to include appendices or notes that further explain the criteria. Feedback indicated that certain aspects of the rubric are perceived as too abstract or vague, leading to requests for more objective, quantifiable measures and less ambiguous language. Both students and staff expressed the need for concrete examples under each criterion to better understand and meet the expectations set in the rubric.

Clear performance categories

Students’ responses (10%) revealed that the rubric needed to be improved in terms of its categories. For example, some students stated:

Combine professionalism into ethic and group [Survey student #100]

All of them are good but professionalism would be part of other categories like work ethic and communication. [Survey student #144]

In a similar vein, 25% of the staff suggested some of the performance categories in the rubric needed to be condensed to avoid repetition, for example one staff member stated:

The group/teamwork section should go under ethics instead as it seems to always have a positive attitude/impression. [Academic #10]

In addition, another staff suggested that to ensure the rubric is an effective tool to cultivate the targeted professional competencies such as work ethic, communication and problem solving, a few examples for each category should be provided as guides:

Look good. My experience with students is that they would like to see few

examples under each category or else, they feel a bit disconnect thinking it is too abstract or subjective. [Academic #7]

Two out of the twelve academic staff (16.7%) emphasised that the quality of what the student groups produce should also form an assessment category in the rubric:

Perhaps contain a section assessing the quality of the material they produce. I have found that most group hostility surrounds individuals within a group having different opinions on quality Inevitably, a few members end up doing more work than the others to lift the final product to a quality they deem acceptable. [Academic #2]

Analysis of these comments revealed several key areas for improvement. Both students and some staff members suggested that certain categories in the rubric, such as 'professionalism', 'ethics' and 'group/teamwork', overlap and could be combined. Staff member also recommended to include criteria for assessing the quality of group outputs and the dynamics within the team. These comments are of interest as they related to the potential interrelation between team functioning and the quality of the final product.

Objective criteria

Some students (32%) frequently indicated that the rubric served its purpose well. They noted that it guided them on how team members should work together and behave professionally. They also thought that the scoring/achievement scale was useful in helping them to achieve the desired grade. However, some students (15.6%) found that the descriptor or needed further clarification:

Sometimes key performance can be hard to define thus unclear. Good descriptors for basics but less definitions for high marks. [Surveyed student #22]

For the general criteria, it is okay, however in certain areas, its specific like individual work, it needs more clarity [Surveyed student #50]

Key performance includes staying on task, respectful etc. Excellent performance includes hard working, actively working with team members. [Surveyed student #101]

Some staff members (25%) thought the rubric provided a clear outline on how students were to be assessed and what performance was expected of them:

Seems ok to me, particularly given there is some range on the number awarded to each.

The advice I was given was to ensure the descriptions contained adverbs and adjectives which were demonstratable. This eliminates room for debate from students over their marks I think it looks good for student assessment. [Academic #2]

The feedback indicated the importance of developing a rubric with clear, specific and demonstrable criteria that can objectively assess both general competencies and specific skills or tasks. Students expressed a need for more detailed guidance on specific aspects of performance, particularly those that distinguish average from excellent work. Academic staff emphasised the importance of using clear, demonstrable adjectives and adverbs in the rubric. This approach is considered as key to minimising subjectivity and debates over grading, highlighting a preference for tangible and observable criteria in assessments.

Scoring System

Two staff (17%) and 18 students (12%) thought that the scoring ranges for the quality levels of each assessment criterion should be more even. In addition, some students suggested that criteria with higher weight or value should have more description:

Some ranges in my opinion are too big (24-30) to fit one description. Make the grade out of 10 or increase amounts of intervals. [Surveyed student #39]

Each category should be given a separate score, the teamwork section had 4 sections but only one score, individual scores would help. [Surveyed student #23]

Perhaps keep it concise and reduce the weighting on some areas to keep it equal. [Surveyed student #49]

This indicates, again, that a more detailed description of each achievement level was required. Similarly, staff members also thought that the rubric should provide a clearer outline of all levels of performance. As one staff member commented:

It does not actually tell the student (group) why a particular grade was awarded and the difference between say a 6 to 15 for group/teamwork. A smaller range associated with particular behaviours. e.g. where there is a range of say 6-15 for a particular subset within one aspect. [Academic #3]

Another staff member stated:

I think that the ‘exemplary’ category should highlight qualities of someone who has gone above and beyond. Otherwise, many people will end up in this bracket just by doing the work and complying in all these areas (even if they haven't gone above and beyond). This is the issue I currently have with my rubrics and end up with lots of HDs just because they comply, e.g. in ‘communication’ category, could add something like 'helps others who struggle communicating and problem solving, actively seeks and suggests solutions that show critical thinking and innovation. [Academic #8]

In addition to wanting better descriptors for each achievement level, students (16%) and staff (25%) also recommended that the descriptors needed to be as simple and specific as possible. They found the many layers of descriptors within an assessment task rather confusing, and this made it difficult to determine the achievement level of a single group:

It is a good step but the line between each section is vague and can be interpreted differently. [Surveyed student #99]

I don't think there should be divisions within proficiency as this makes it harder to score. [Academic #5]

In terms of scoring system in the rubric, several suggestions were drawn from the feedback. These included: 1. scoring accuracy, which involved narrowing the scoring ranges to allow for better precision in assessment, provide separate scores for different point of criteria that offer clearer feedback on specific areas of performance, and ensuring balanced weighting across categories to maintain balance and fairness in the evaluation. 2. clarity and distinctiveness, which involved defining exemplary performance reflecting above and beyond basic compliance, and 3. simplifying scoring levels, which involved reducing the number of divisions within each proficiency level to avoid complicating the scoring process.

Robustness

Two staff members (17%) indicated that to avoid students arguing over their marks the rubric required a more precise summative rating scale:

Seems okay to me, particularly given there is some range on the number awarded to each. The advice I was given was to ensure the descriptions contained adverbs and adjectives which were demonstratable. This eliminates room for debate from students over their marks. [Academic #2]

It does not actually tell the student ([or] group) why a particular grade was awarded and the difference between say a 6 and 15 for group/team work. [Academic #3]

When considered the insights from academic staff directly involved in its application, it is essential to develop a robust assessment tool. Their feedback highlighted the need for precise and observable descriptors that eliminate ambiguity. They also recommended providing clear and explicit explanations for each performance level, thereby offering students and groups a clear understanding of expectations and justifications for their assessments.

Key features of rubrics in providing feedback in enhancing student performance

Just over half of the respondents (52% of students and 50% of staff) indicated that the rubric could be a source and mechanism for providing feedback to improve student performance. However, some of the staff and students suggested additional features to complement the rubric. These additional features could be grouped into: incorporate self- and peer-assessment; the need for descriptions that guide subsequent improvement; provide spaces for specific written comments; and marking time and feedback. These findings are presented below.

Incorporate self- and peer-assessment

A number of students (17%) and staff (25%) suggested including self- and peer-assessments in the rubric to allow students to assess their own contributions as well as those of their peers.

Also, one student made a suggestion to allow peers to provide each other with feedback to ensure all team members were clearly and fairly assessed, and each student would receive a fair mark based on the whole team’s achievement:

Giving students opportunity to give feedback about other students but making sure that everyone gets a fair mark. [Surveyed student #89]

One of the comments from an academic who had prior experience with peer assessment noted the importance of peer-assessment in encouraging students to participate in group tasks and take initiative to help others.

These insights revealed key features of rubrics related to peer assessment; they should enable students to provide feedback about their peers, ensure fairness and equity in marking, and encourage active participation and mutual support.

Need for descriptions that guide subsequent improvement

Twenty-five (16%) students believed that the rubric was not sufficiently explanatory in helping them identify their areas of strength and weakness, and that the rubric had limited use in guiding them to improve their learning. For example, some students commented:

Not very good. They are good for marking and identifying areas of strength or weakness; However, they give very vague and generic feedback. [Surveyed student #30]

Pretty useful but it also a bit vague, maybe a teacher should go through the rubric to be more specific about the expectations required to get the highest marks. [Surveyed student #104]

If the feedback points our something that need to be improved, I will be able to see the problem right away and to do better next time. [Surveyed student #66]

Students identified to several key features of rubrics crucial for feedback that enhances their performance. They are looking for rubrics with detailed and specific descriptors that clearly outline expectations for each level of achievement, particularly for higher performance brackets. They also appreciate feedback that helps them to identify and subsequently address areas for improvement.

Provide specific written comments

Twenty (13%) students and two staff (17%) indicated that supplementing the rubric with written feedback could have a positive impact on students’ learning. As one student stated:

Put a footnote which specifies what needs to be improved on. [Surveyed student #21]

Another added:

The rubric is too broad and may require additional comments to articulate on feedback. [Surveyed student #135]

One of the staff also argued that the rubric needed written comments to address the grading for the overall assessment of the tasks related to teamwork and PBL:

I think it needs to be supplemented by overall feedback section that provides specific example on the overall assessment of the tasks. [Academic #7]

To effectively support student performance improvement, rubrics should provide clear, detailed and actionable feedback that goes beyond numerical or categorical grades. It is crucial to articulate specific areas for improvement, providing contextual examples and holistic comments. These features ensure that rubrics serve not only as tools for assessment but also valuable resources for learning and development.

Marking time and feedback

An important finding was the students’ suggestion (5%) that, after assignments have been marked, the feedback and grade should be returned to them speedily, noting that this would help in their learning by reflecting on their weaknesses and strengths while the whole event is still fresh in their minds. One student commented:

They would be useful if they were sent out a couple days after submitting work while

the stuff you did is still fresh in your head, else you forget everything. [Surveyed student #99]

Similarly, academic staff (17%) acknowledged that the marked assignment could provide meaningful feedback in terms of areas needing more work and those areas that were already at a high standard, but the timing was important.

…but I guess this depends on when they [the students] receive the outcomes. [Academic #3]

In summary, these insights highlighted timing is a critical aspect of an effective rubric-based feedback system. Timely feedback, especially when coordinated with the release of grades, can greatly enhance the learning experience. Providing feedback while students’ memories of their work are still fresh allows them to reflect on their performance and draw meaningful connections with their work.

The results indicated common concern from both academic staff and students. These concerns can be broadly categorised into two principal issues: 1. Improving clarity and structure of the rubric, and 2. Incorporating feedback mechanisms within rubric for student development.

In relation to RQ1, students and academic staff reported a number of issues in relation to the clarity and structure of the rubric. The students and staff noticed different things. The students suggested that some of the categories could be combined to avoid duplication, while the academic staff wanted to add categories that would assess the innovation and creativity demonstrated by the teamwork. Staff also provided additional definitions for good teamwork such as “initiative”, “helping others who struggle communication” and “actively seeks and suggest solutions that show critical thinking and innovation”.

In terms of the detailed descriptors of each achievement level, the students suggested that providing an example of a marked report and its associated rubric item would help them in interpreting the rubric—a finding consistent with those made by Torrance [38]. Our intention will be to provide students with examples that clarify the differences between different grades. For example, a statement that is asserted by a student would receive a lower grade than one that is backed up with data or a citation and reference. A combination of both original data and additional references would receive an even higher grade.

In particular, the students disliked vague and unclear rubric descriptors that did not inform them on what exactly was needed for their assessment. The academic staff also emphasised the importance of having clearer descriptions that contained adverbs and adjectives that are demonstrable and measurable. For example, both staff and students found that the use of terms such as “always”, “often” and “occasionally” were open to interpretation. They suggested that the use of clearer language would emphasise the objectivity in grading and eliminate unnecessary appeals from students on their results. This is consistent with the view of Dawson et al. [37] who found that objective criteria helped avoid arbitrary grading and subjective assessments of students’ work.

In terms of the allocation of marks to each achievement level, both staff and students agreed that the scoring ranges for each achievement level should be equal, rather than having some levels with larger ranges than others.

Participants also suggested that the rubric should be accompanied by clearer guidelines or a general introduction that explains how the rubric will be applied and what its purpose is. Indeed, 20% of the participants, both staff and students, did not have a clear understanding that the rubric was designed to assess the whole team based on their collective effort.

In relation to RQ2, student and staff reported several issues in incorporating feedback mechanisms within rubric. There were a number of staff and students who interpreted the rubric as being used for peer- and self-assessment, i.e. a rubric for individual contributions, rather than a rubric for a team’s performance (as it was designed). Once the misunderstanding was clarified, the participants suggested that additional categories could be added to the rubric that covered both peer- and self- assessment to aid the students in developing their professional practice [26]. However, there were noticeable differences between the staff and the students’ suggestions here. The staff were more concerned with the individual contributions of each student toward the quality and creativity evidenced in a team’s final product. In particular, staff were concerned with an uneven distribution of labour within the student teams and about “social loafing”. In contrast, the students were more concerned with their groups’ dynamics and communication as well as the distribution of workload.

We agree on the potential in using the rubric in conjunction with self- and peer- assessment, as rubrics can promote high levels of learning when students reflect on their own efforts and the work of their peers [16, 44]. Such peer-assessment might include the following criteria: 1. clear allocation of roles and responsibilities in teamwork; 2. agreed-upon levels of participation, initiative and helping others, 3. effectiveness of communication and management of group dynamics, 4. clarification of shared knowledge and expectations; and 5. moderation of standards and quality of material each student produced. Therefore, we plan to include a form on which each group member can summarise their relative contributions to the project. They will also be asked to provide a reflective statement (either individual or as a group) to summarise the group process.

Both staff and students also agreed that the value of the rubric would be enhanced with a “free response” section, where the assessor could provide written feedback not covered by the rubric’s assessment items. Such a section was perceived as being helpful for the students in identifying their areas of strength and the areas in which they could improve. This has been noted by other researchers, notably Dawson et al. [37] and Velasco-Martínez et al. [41], who reported that quality written comments enabled the students to improve their level of achievement and cultivate life-long learning skills.

The students suggested that the fairness and transparency of the rubric could be enhanced by the academic staff spending more time explaining how the rubric worked. Without such guidance, the students thought the implementation of the rubric was problematic. In addition, because teamwork was often undertaken during the course of a semester, it was important that the academic staff provide constant communication with the students about the assessment requirements. In other words, the rubric needed to be supplemented with further detail from the academics on a continuous basis.

There were significant differences in how the staff and the students used the rubric. Staff members regarded the rubric as a means of assessment; the students considered the rubric more as an instrument used to ensure each group member was held accountable for their work and their individual contribution to the team. The students also used the rubric as a checklist that detailed what was expected of each team member. These finding are consistent with the idea that assessors should, first, use rubrics to evaluate and judge students’ work [39], secondly, students should use rubrics to focus on their own work [15] and, thirdly, ensure the accountability of their peers [35].

Many students said that, despite having a clear and standardised rubric for a particular piece of assessment, they still found it difficult to apply the rubric to their own work. They suggested that academic staff should provide an example of a marked assessment report with its associated rubric. The marked example then could clarify the interpretation of the rubric and the students would then be able to shape their work to a higher standard for subsequent assessments. The provision of examples of student’s work was also regarded as one way to clarify the descriptors of each achievement level.

Both staff and students, mentioned that the feedback and comments which are returned to the students in a format that parallels the rubrics, should be distributed as close as possible to the submission date of the group project. This would allow the students to reflect more on their results while the assessment task was still fresh in their minds. Staff also considered the timeliness of the feedback to be important. These findings are consistent with Dawson et al. [37] who noted that effective feedback needed to be timed appropriately.

In addition to timeliness, the staff also suggested that the rubric needed to include feedback that could be actionable in future assessments and that an overall ‘comments’ section would enhance the rubric’s usefulness in terms of a feedback mechanism to the students. Dawson et al. [37], again, suggested that the modalities, i.e., the forms in which feedback information was provided, can change the usefulness of the feedback.

Refinement and implementation of the revised rubric

In relation to the suggestions to revise the categories and scoring scheme detailed in the rubric, we noted that the sample rubric contained six categories of performance criteria and four scoring levels (see Appendix A). Literature suggests optimal criteria range from four to nine, with three to six scoring levels per criterion [36, 45]. The revised rubric, which is outlined in Appendix B, was developed based on valuable feedback from both staff and students (see Table 2). It incorporated more specific criteria and clearly defined scoring levels, accompanied by illustrative examples for each level. These refinements aimed to enhance the rubric's effectiveness and provide a comprehensive framework for assessing teamwork skills.

Table 2. General themes arising from qualitative evaluation of the example rubric and modifications

Theme area	Participant’s comment	Modifications
Clear guidelines	The rubric wasn't very clear on what/who I was assessing. [Student #104]	Provided a guideline
Clear assessment categories	Simplify: each criterion to be marked 1-5 or 0-5; fewer categories: professionalism + work ethic + communication + info sharing. [Student #33]	Revised the category
Objective criteria	Key defining words such as 'some', 'often', 'always' between definitions helped define scoring for key performance. [Student #115]	Measurable items specified
Scoring system	The point system seemed a bit arbitrary. It could be made more consistent. [Student #74]	Revised scoring
Incorporate self- and peer-assessment	Maybe [include] a separate rubric for self and peer assessment. [Student #129]	Included a statement of individual contribution and reflective summary on the group process
Need for robustness and details	For general criteria, it is fine, but in certain areas such as individual work it needs more clarity. [Student #54]	Further details and descriptions added
Provide written comments	Works well so far as comments are also provided for improvement. [Student #123]	Comments section added
Return marked rubric timely	They're not very timely as they don't help current project since they're filled out afterwards. [Student #137]	Post the grade on the online learning management system

This refined rubric demonstrated enhanced clarity and effectiveness in assessing teamwork activities. This was evident through improved grading consistency and alignment with performance expectations. These findings aligned with the research conducted by Pang et al. [17], further confirming the rubric's validity as a clear and process-oriented assessment tool.

Limitations and future direction of research

This study has limitations which suggest opportunities for future research. First, this study was conducted in a university undergraduate Engineering training context with a specific example of a rubric provided for feedback on teamwork. The teamwork tasks and activities that students participated in were designed as part of a PBL activity. Different projects—projects with experiments, design and simulation, and projects with management—may have different requirements. As such, certain findings may not be readily generalisable to other academic disciplines or rubric uses. Nevertheless, we believe that these findings provide valuable information for the development of robust rubrics for assessing teamwork.

Secondly, as reported previously [11], as the first year students were transitioning from high school to university, they might have limited PBL experience and lack of exposure to the more experiential collaborative form of teaching and learning in higher education. However, their experiences were investigated in this study and reported in the paper. Their inexperience might limit generalisability and validity to a senior year cohort of undergraduate degrees.

Thirdly, this study did not focus on what might be needed in higher levels of undergraduate studies. Any effects of the levels of experience of the staff were also not directly assessed, however, most of the staff did have experience in using PBL, which could change the responses to the rubric and its utility. Finally, cultural factors, such as not having English as a first language, were not examined in detail in the reporting of the results. However, future work may also include the effects of culture on interpretation of the rubric.

Implications

PBL is a commonly used pedagogical approach in Engineering education, and it often requires students working collaboratively in teams to complete tasks. Every semester, students might be taking more than one course that require them to work in groups to solve real-world or simulated problems. However, because there is no consensus among educators on a standard rubric to be used, students might be provided with different rubrics with different assessment criteria for each course related to assessing teamwork. Adding to this confusion are the general discrepancies between staff and student perceptions of the purpose of rubrics [37, 45]. Hence, it is important to understand how students and staff interpret the key elements in any rubric and, based on their comments, our efforts will lead to a more robust rubric in future.

Findings from this study revealed some essential steps in developing a more robust form of assessment tool for teaching and learning that was agreeable to both academic staff and students. In this study, staff and students worked together to understand each other and each other’s expectations of the teaching, the learning and the assessment. For the staff, it was important for them to know how to assess teamwork more effectively. For the students, they needed to be able to use the assessment rubric as another tool in moving on from their training to become a professional. The interaction between the staff and students that the rubric engendered did enable both staff and students to reflect on what the other might want and what each of them could do. Such interactions offer the potential for a more democratic and engaged discussion between staff and students on the use of rubrics, which not only benefits student learning, but also assists staff in efficient marking.

Ultimately, this study contributed to the broader dialogue on rubric development. It provided a model that demonstrated how collective feedback can result in creation of more effective and universally applicable assessment tools in the field of education.

This study highlights the importance of developing clear criteria for scoring rubrics used in evaluating students’ teamwork, in which we utilised the PBL approach to engage and connect students to their learning through the active exploration of real-world challenges and problems. The performance and outcomes of students’ teamwork were assessed in the format that parallels the rubric’s categories, scores, and key descriptors. A well-accepted and well-defined scoring rubric will allow standardised evaluations of performance to enhance assessor consistency in grading assignments and provide essential feedback to students that allows them to reflect on their learning and professional practice development.

Overall, the results showed that students considered the rubric worked well for its intended purpose. However, further improvements may include providing definitive and objective measures to avoid vague interpretation, revision of the performance categories and scoring levels, addition of a section for detailed written feedback identifying errors and areas for improvement, and complementing the rubric with self- and peer-assessments to ensure the effectiveness in and fairness of teamwork processes. From the staff perspective, the rubric was an effective instrument to measure students’ performance. That said, it could also contain elements to measure the quality and creativity contributed by each student during the teamwork. Insights gained from this study may in future help staff develop a more reliable and valid rubric for assessing both teamwork and the quality of individual work within a team.

Acknowledgment

The authors thank Andoh Afful for assisting the study, which included conducting the surveys. The authors would also like to express their sincere gratitude to the participants for volunteering to provide their invaluable comments and perspectives.

Funding details

This work was supported by the Scheme for Teaching and Learning Research, College of Science Engineering and Health, RMIT University, Melbourne, Australia.

Disclosure statement

No potential conflict of interest was reported by the authors.

Conflict of Interest

The authors have no competing interests to declare that are relevant to the content of this article.

Funding details

This work was supported by the Scheme for Teaching and Learning Research, College of Science Engineering and Health, RMIT University, Melbourne, Australia.

Non-financial interests

None.

Author contributions

Conceptualization: Toh Yen Pang, Alexandra Kootsookos, Kate Fox and Elena Pirogova;

Methodology: Toh Yen Pang, Alexandra Kootsookos;

Formal analysis and investigation: Toh Yen Pang and Alexandra Kootsookos;

Writing - original draft preparation: Toh Yen Pang, Alexandra Kootsookos;

Writing - review and editing: Alexandra Kootsookos, Kate Fox and Elena Pirogova;

Funding acquisition: Toh Yen Pang, Alexandra Kootsookos, Kate Fox and Elena Pirogova.

All authors read and approved the final manuscript.

Ethics approval and consent to participate

This study was performed in line with the principles of the Declaration of Helsinki. All authors confirm that we obtained approval from the College Human Ethics Advisory Network Committee (27 August 2018/SEHAPP 47-18). Informed consent was obtained from all individual participants included in the study.

Grainger P, Steffler R, de Villiers Scheepers MJ, Thiele C, Dole S. Student negotiated learning, student agency and General Capabilities in the 21st Century: The DeLorean Project. Australian educational researcher. 2018;46(3):425–47. doi: 10.1007/s13384-018-0287-6.
Larson LC, Miller TN. 21st Century Skills: Prepare Students for the Future. Kappa Delta Pi record. 2011;47(3):121–3. doi: 10.1080/00228958.2011.10516575.
Rosenberg S, Heimler R, Morote E-S. Basic employability skills: a triangular design approach. Education & training (London). 2012;54(1):7–20. doi: 10.1108/00400911211198869.
van Laar E, van Deursen AJAM, van Dijk JAGM, de Haan J. The relation between 21st-century skills and digital skills: A systematic literature review. Computers in human behavior. 2017;72:577–88. doi: 10.1016/j.chb.2017.03.010.
Bell S. Project-Based Learning for the 21st Century: Skills for the Future. The Clearing House. 2010;83(2):39–43.
Huq A, Gilbert D. All the world’s a stage: transforming entrepreneurship education through design thinking. Education & training (London). 2017;59(2):155–70. doi: 10.1108/ET-12-2015-0111.
Fletcher AJ, Sharif AWA, Haw MD. Using the perceptions of chemical engineering students and graduates to develop employability skills. Education for chemical engineers. 2017;18:11–25. doi: 10.1016/j.ece.2016.07.001.
Drake R, Goldsmith G, Strachan R. A novel approach to teaching teamwork. Teaching in Higher Education. 2006;11(1):33–46. doi: 10.1080/13562510500400115.
Gagnon LL, Roberge GD. Dissecting the journey: Nursing student experiences with collaboration during the group work process. Nurse Education Today. 2012;32(8):945–50. doi: https://doi.org/10.1016/j.nedt.2011.10.019.
Moni RW, Beswick E, Moni KB. Using student feedback to construct an assessment rubric for a concept map in physiology. Advances in Physiology Education. 2005;29(4):197–203. doi: 10.1152/advan.00066.2004.
Pang TY, Kootsookos A, Steiner T, Khatibi A, Crossin E. Collaborative Project: Exploring the Experiences of Engineering Students’ Engagement in Face-to-face and Virtual Peer Learning. Journal of Modern Education Review. 2017;7(3):12. doi: 10.15341/jmer(2155-7993)/03.07.2017/004.
Li J, Lindsey P. Understanding variations between student and teacher application of rubrics. Assessing Writing. 2015;26:67–79. doi: https://doi.org/10.1016/j.asw.2015.07.003.
Llaurado-Serra M, Rodríguez E, Gallart A, Fuster P, Monforte-Royo C, De Juan MÁ. Assessing the competences associated with a nursing Bachelor thesis by means of rubrics. Nurse Education Today. 2018;66:103-9. doi: https://doi.org/10.1016/j.nedt.2018.04.009.
Panadero E, Jonsson A. The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review. 2013;9:129–44. doi: https://doi.org/10.1016/j.edurev.2013.01.002.
Panadero E, Romero M. To rubric or not to rubric? The effects of self-assessment on self-regulation, performance and self-efficacy. Assessment in Education: Principles, Policy & Practice. 2014;21(2):133–48. doi: 10.1080/0969594X.2013.877872.
Panadero E, Romero M, Strijbos J-W. The impact of a rubric and friendship on peer assessment: Effects on construct validity, performance, and perceptions of fairness and comfort. Studies in Educational Evaluation. 2013;39(4):195–203. doi: https://doi.org/10.1016/j.stueduc.2013.10.005.
Pang TY, Kootsookos A, Fox K, Pirogova E. Does an assessment rubric provide a better learning experience for undergraduates in developing transferable skills? undergraduates in developing transferable skills?. Journal of University Teaching and Learning Practice. 2022;19(3):03. doi: https://ro.uow.edu.au/jutlp/vol19/iss3/03.
Company P, Contero M, Otey J, Plumed R. Approach for developing coordinated rubrics to convey quality criteria in MCAD training. Computer-Aided Design. 2015;63:101–17. doi: https://doi.org/10.1016/j.cad.2014.10.001.
Chan LH, Chen CH. Conflict from teamwork in project-based collaborative learning. Performance improvement (International Society for Performance Improvement). 2010;49(2):23–8. doi: 10.1002/pfi.20123.
Hastie C, Fahy K, Parratt J. The development of a rubric for peer assessment of individual teamwork skills in undergraduate midwifery students. Women and Birth. 2014;27(3):220–6. doi: https://doi.org/10.1016/j.wombi.2014.06.003.
Kokotsaki D, Menzies V, Wiggins A. Project-based learning: A review of the literature. Improving Schools. 2016;19(3):267–77. doi: 10.1177/1365480216659733.
Cooper S, Cant R, Connell C, Sims L, Porter JE, Symmons M, et al. Measuring teamwork performance: Validity testing of the Team Emergency Assessment Measure (TEAM) with clinical resuscitation teams. Resuscitation. 2016;101:97–101. doi: 10.1016/j.resuscitation.2016.01.026.
Salas E, Reyes DL, Woods AL. The Assessment of Team Performance: Observations and Needs. In: von Davier AA, Zhu M, Kyllonen PC, editors. Innovative Assessment of Collaboration. Cham: Springer International Publishing; 2017. p. 21–36.
Elmore L, Skelley J, Woolley T. Impact of adapted team-based learning methods on student self-assessment of professionalism, teamwork, and skills in a self-care course. Currents in pharmacy teaching and learning. 2014;6(4):488–93. doi: 10.1016/j.cptl.2014.04.002.
Pinho-Lopes M, Macedo J. Project-based learning in Geotechnics: cooperative versus collaborative teamwork. European journal of engineering education. 2016;41(1):70–90. doi: 10.1080/03043797.2015.1056099.
Vaughan B, Yoxall J, Grace S. Peer assessment of teamwork in group projects: Evaluation of a rubric. Issues in Educational Research. 2019;29(3):961–78.
Parratt JA, Fahy KM, Hutchinson M, Lohmann G, Hastie CR, Chaseling M, et al. Expert validation of a teamwork assessment rubric: A modified Delphi study. Nurse Education Today. 2016;36:77–85. doi: https://doi.org/10.1016/j.nedt.2015.07.023.
Cifrian E, Andrés A, Galán B, Viguri JR. Integration of different assessment approaches: application to a project-based learning engineering course. Education for chemical engineers. 2020;31:62–75. doi: 10.1016/j.ece.2020.04.006.
Zhao K, Zhou J, Dawson P. Using student-instructor co-constructed rubrics in signature assessment for business students: benefits and challenges. Assessment in education: principles, policy & practice. 2021:1–21. doi: 10.1080/0969594X.2021.1908225.
Alsina Á, Ayllón S, Colomer J, Fernández-Peña R, Fullana J, Pallisera M, et al. Improving and evaluating reflective narratives: A rubric for higher education students. Teaching and Teacher Education. 2017;63:148–58. doi: https://doi.org/10.1016/j.tate.2016.12.015.
Clary RM, Brzuszek RF, Fulford CT. Measuring Creativity: A Case Study Probing Rubric Effectiveness for Evaluation of Project-Based Learning Solutions. Creative education. 2011;2(4):333–40. doi: 10.4236/ce.2011.24047.
Dawson P. Assessment rubrics: towards clearer and more replicable design, research and practice. Assessment & Evaluation in Higher Education. 2017;42(3):347–60. doi: 10.1080/02602938.2015.1111294.
Martens KSR. Rubrics in program evaluation. Evaluation Journal of Australasia. 2018;18(1):21–44. doi: 10.1177/1035719X17753961.
Popham WJ. What's wrong - and what's right - with rubrics. Educational Leadership. 1997;55(2):72.
Reddy YM, Andrade H. A review of rubric use in higher education. Assessment & Evaluation in Higher Education. 2010;35(4):435–48. doi: 10.1080/02602930902862859.
Wolf K, Stevens E. The role of rubrics in advancing and assessing student learning. The Journal of Effective Teaching. 2007;7:3–14.
Dawson P, Henderson M, Mahoney P, Phillips M, Ryan T, Boud D, et al. What makes for effective feedback: staff and student perspectives. Assessment & Evaluation in Higher Education. 2019;44(1):25–36. doi: 10.1080/02602938.2018.1467877.
Torrance H. Assessment as learning? How the use of explicit learning objectives, assessment criteria and feedback in post-secondary education and training can come to dominate learning. Assessment in Education: Principles, Policy & Practice. 2007;14(3):281–94. doi: 10.1080/09695940701591867.
Chan Z, Ho S. Good and bad practices in rubrics: the perspectives of students and educators. Assessment & Evaluation in Higher Education. 2019;44(4):533–45. doi: 10.1080/02602938.2018.1522528.
Janssen G, Meier V, Trace J. Building a better rubric: Mixed methods rubric revision. Assessing Writing. 2015;26:51–66. doi: https://doi.org/10.1016/j.asw.2015.07.002.
Velasco-Martínez LC, Díaz-Barriga F, Tójar-Hurtado J-C. Acquisition and Evaluation of Competencies by the Use of Rubrics. Qualitative Study on University Faculty Perceptions in Mexico. Procedia - Social and Behavioral Sciences. 2017;237:869–74. doi: https://doi.org/10.1016/j.sbspro.2017.02.185.
Laura R-K. Rubric-Referenced Assessment in Teacher Preparation: An Opportunity to Learn by Using. Practical assessment, research & evaluation. 2010;15(8):1–9.
Bird FL, Yucel R. Improving marking reliability of scientific writing with the Developing Understanding of Assessment for Learning programme. Assessment & Evaluation in Higher Education. 2013;38(5):536–53. doi: 10.1080/02602938.2012.658155.
Fraile J, Panadero E, Pardo R. Co-creating rubrics: The effects on self-regulated learning, self-efficacy and performance of establishing assessment criteria with students. Studies in Educational Evaluation. 2017;53:69–76. doi: https://doi.org/10.1016/j.stueduc.2017.03.003.
Menéndez-Varela J-L, Gregori-Giralt E. The reliability and sources of error of using rubrics-based assessment for student projects. Assessment & Evaluation in Higher Education. 2018;43(3):488–99. doi: 10.1080/02602938.2017.1360838.
Saldana J. Coding Manual for Qualitative Researchers. SAGE Publications; 2009.
Varpio L, Ajjawi R, Monrouxe LV, apos, Brien BC, Rees CE. Shedding the cobra effect: problematising thematic emergence, triangulation, saturation and member checking. Med Educ. 2017;51(1):40–50. doi: 10.1111/medu.13124.

No competing interests reported.

Download PDF

Editorial decision: Revision requested
30 Oct, 2024
Reviews received at journal
08 Oct, 2024
Reviewers agreed at journal
07 Oct, 2024
Reviews received at journal
26 Sep, 2024
Reviewers agreed at journal
26 Sep, 2024
Reviewers invited by journal
04 Sep, 2024
Editor assigned by journal
04 Sep, 2024
Submission checks completed at journal
03 Sep, 2024
First submitted to journal
27 Aug, 2024

You are reading this latest preprint version

Teamwork assessment in undergraduate Engineering courses

Status:

Version 1

Abstract

Figures

Introduction

Literature Review

Project-based learning (PBL) and assessing teamwork

Rubrics as a focal point of study

Rationale for the present study

Materials And Methods

Discussion

Refinement and implementation of the revised rubric

Limitations and future direction of research

Implications

Conclusion

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1