4.1 Quantitative evaluation
Both instruction modes increased the participants’ understanding of LCD structure (Table 1). In the pre-test quiz, most scores were close to 0 (M2D − Pre = 0.35, MVR−Pre = 0.48) which means that the students had little prior knowledge about LCD. Thus, we expected that it would be suitable for comparing learning effects depending on learning media with students who had little prior knowledge. On the post-test quiz (Table 1), participants who learned in the VR environment scored 1.54 points higher than those who learned in the 2D environment (M2D − Post = 6.93, MVR−Post = 8.47). Considering the results of the normalized post-test, the score was 12% higher in the VR learning environment than in the 2D environments.
In the 3D reconstruction experiment (Table 2), the score of participants who learned in the VR environment was 0.7 points higher than that of those who learned in the 2D environment (2D = 3.65, VR = 4.35). This means that the score in the VR learning environment was about 13% higher when considering normalized score.
User evaluations using questionnaires were divided into five categories, and participants used a 7-point Likert scale to respond to each question. We used an independent t-test to statistically analyze the evaluation results of each educational method. All categories had Cronbach’s α > 0.8 (Table 3), which means that all questions were reliable and internally consistent.
Table 1
Mean M, and standard deviation SD and standard error SE of points (/13) earned during the quiz. (n = 20 for both media)
Medium | Pre-test [Points] | Post-test [Points] | Normalized Post-test | Gain |
2D-Pre | 0.35 | 1.56 | 0.35 | 6.93 | 0.75 | 0.17 | 0.53 | 18.80 |
VR-Pre | 0.48 | 1.73 | 0.39 | 8.47 | 0.60 | 0.14 | 0.65 | 16.65 |
Table 2
Mean M, standard deviation SD and standard error SE of the score (/7) on 3D reconstruction. (n = 20 for both media)
Medium | N | Score [Points] | Normalized Score |
2D | 20 | 3.65 | 2.08 | 0.47 | 0.52 |
VR | 20 | 4.35 | 1.98 | 0.44 | 0.65 |
4.2 Questionnaire evaluation
Mean, total mean, and total mean difference of each educational category were calculated for comparison with parent items (Table 4). Mean difference were also calculated for each of the 52 questions divided into five categories for comparison with child items (Table 5). The first category is composed of the answers to 12 questions that assess ‘Engagement’. Overall, the responses were more positive for learning with VR than with 2D media (overall M2D = 4.32, MVR = 5.15).
‘Attraction’ showed the most significant difference in the A2 item (M2D = 3.85, MVR = 5.45) and the lowest in A1 (M2D = 4.55, MVR = 5.75). ‘Time investment’ was also confirmed to be higher in VR than in 2D (M2D = 4.11, MVR = 5.31). The most significant difference was in T3. Finally, ‘Usability’ was evaluated as slightly easier for 2D than for VR (M2D = 4.83, MVR = 4.69), but the difference was not significant.
In ‘Immersion’, as well as ‘Engagement’, it was confirmed that the total mean was higher in VR than in 2D (overall M2D = 3.18, MVR = 4.49). Among the items of ‘Immersion’, ‘Emotional attachment’ showed the largest difference in the immersion category (M2D = 3.28, MVR = 4.98). Especially, in ‘Emotional attachment’ the largest difference was in E1 (M2D = 3.25, MVR = 5.90), and the smallest difference was in E4 (M2D = 4.35, MVR = 4.60).
In ‘Focus on attention’, the largest difference was in FA5 (M2D = 3.80, MVR = 5.30), and the smallest difference was in FA3 (M2D = 3.85, MVR = 4.70). For ‘Focus on attention’, although FA3 items individually had p > 0.05, total sum of ‘Focus on attention’ had p < 0.05. ‘Presence’ consisted of two items, and both items had M2D < MVR, but both had p > 0.05. For ‘Flow’, the largest difference was in F4 (M2D = 3.40, MVR = 5.15), and the item that showed the smallest difference was F3 (M2D = 2.95, MVR = 4.25). All items in ‘Flow’ had p < 0.05.
For ‘Motivation’, the mean values were higher in MVR than in M2D in all items except M9 and M10. The total mean for ‘Motivation’ is also higher in VR (overall M2D = 3.98, MVR = 4.70). Each item individually had p > 0.05, except M1, M2, M3, and M7 but the sums of all items had p < 0.05. The most significant difference was in M3 (M2D = 3.70, MVR = 5.65), and the most negligible difference was in M10 (M2D = 5.05, MVR = 5.05).
For ‘Cognitive benefit’ means of all items were higher for MVR than for M2D, and total mean for ‘Cognitive benefit’ is also
higher in VR (overall M2D = 4.44, MVR =5.09). All individual items had p > 0.05, but when pooled, they had p < 0.05. The item with the most significant difference was C3 (M2D = 4.25, MVR = 5.35), and the item with the most negligible difference was C4 (M2D = 4.50, MVR = 4.65).
Finally, for ‘Perceived learning effectiveness’ total means were higher for MVR than for M2D (M2D = 4.82, MVR = 5.43), exhibiting higher mean with VR in all of individual sub-items. Each individual item, except PL1 and PL5, had p > 0.05, but the total scores of all items had p < 0.05. The item with the most significant difference was PL5 (M2D = 3.75, MVR = 5.50), and the item with the most negligible difference was PL4 (M2D = 5.35, MVR = 5.40).
Table 3
Cronbach's alpha values for questionnaire
Research variable | Number of Items | Cronbach's Alpha (α) |
Engagement | 12 | 0.91 |
Immersion | 15 | 0.958 |
Motivation | 12 | 0.918 |
Cognitive benefits | 5 | 0.954 |
Perceived learning effectiveness | 8 | 0.898 |
Table 4
Mean, total mean, total mean difference of each educational evaluation category.
Category | Subcategory | Mean | p-value | Total mean | p-value (One-sided) | Total mean difference | Total mean difference |
2D | VR | 2D | VR | [Points] | [%] |
Engagement | Attraction | 4.03 | 5.44 | < 0.001 | 4.32 | 5.15 | < 0.001 | 0.83 | 11.9 |
Time investment | 4.11 | 5.34 | < 0.001 |
Usability | 4.83 | 4.69 | 0.260 |
Table 4
Immersion | Emotional attachment | 3.28 | 4.98 | < 0.001 | 3.18 | 4.49 | < 0.001 | 1.31 | 18.6 |
Focus of attention | 3.70 | 4.94 | < 0.001 |
Presence | 2.55 | 3.33 | 0.055 |
Flow | 3.20 | 4.70 | < 0.001 |
Motivation | | 3.98 | 4.70 | < 0.001 | 3.98 | 4.70 | < 0.001 | 0.72 | 10.3 |
Cognitive benefits | | 4.44 | 5.09 | 0.007 | 4.44 | 5.09 | 0.003 | 0.65 | 9.3 |
Perceived learning effectiveness | | 4.82 | 5.43 | < 0.001 | 4.82 | 5.43 | < 0.001 | 0.61 | 8.7 |
Table 5
Mean difference between VR-based and 2D Video-based learning method.
Category | Questionnaire | 2D | VR | t | p-value (One-sided) |
M | SD | M | SD |
Engagement (Attraction) | (A1) The contents we employed captured my attention Strongly. | 4.55 | 1.67 | 5.75 | 1.89 | 2.13 | 0.02 |
(A2) I liked the contents because it was novel. | 3.85 | 1.90 | 5.45 | 1.57 | 2.90 | 0.003 |
(A3) The topic of the contents made me want to find out more about it. | 3.90 | 1.37 | 5.30 | 1.69 | 2.88 | 0.0035 |
(A4) The space in which the contents took place was interesting. | 3.80 | 1.58 | 5.25 | 1.97 | 2.57 | 0.007 |
Total | 4.03 | 1.64 | 5.44 | 1.76 | 5.25 | < 0.001 |
Engagement (Time investment) | (T1) I wanted to spend the time to complete the contents successfully. | 4.00 | 1.49 | 5.50 | 1.49 | 2.97 | 0.0025 |
(T2) I wanted to spend time to participate in the contents. | 3.95 | 1.32 | 5.50 | 1.32 | 3.05 | 0.002 |
(T3) I wanted to spend time collecting the information provided. | 4.35 | 1.42 | 5.70 | 1.42 | 3.56 | 0.0005 |
(T4) I think that participating in this content was a waste of my time. * | 4.15 | 1.31 | 4.55 | 1.31 | 0.82 | 0.2075 |
Total | 4.11 | 1.37 | 5.34 | 1.63 | 5.05 | < 0.001 |
Engagement (Usability) | (U1) It was easy for me to use the contents. | 5.65 | 1.50 | 5.70 | 1.50 | 0.11 | 0.4585 |
(U2) I found the contents confusing. * | 3.60 | 1.60 | 3.40 | 1.60 | 0.35 | 0.364 |
(U3) I felt confident since I knew how to use the contents. | 5.65 | 1.63 | 5.50 | 1.63 | 0.31 | 0.3795 |
(U4) The content was unnecessarily complex. * | 4.40 | 1.60 | 3.95 | 1.60 | 0.80 | 0.214 |
Total | 4.83 | 1.78 | 4.69 | 1.97 | 0.63 | 0.2645 |
Immersion (Emotional attachment) | (E1) I was curious about how the content would progress. | 3.25 | 1.92 | 5.90 | 1.25 | 5.18 | < 0.001 |
(E2) I liked the content because it was novel. | 2.60 | 1.82 | 5.05 | 1.57 | 4.56 | < 0.001 |
(E3) I often felt interested in the content. | 2.90 | 1.89 | 4.35 | 1.23 | 2.88 | 0.0035 |
(E4) If interrupted, I looked forward to returning to the content. | 4.35 | 1.90 | 4.60 | 1.79 | 0.43 | 0.3355 |
Total | 3.28 | 1.96 | 4.98 | 1.57 | 6.06 | < 0.001 |
Immersion (Focus on attention) | (FA1) I was more involved with the content than with any other thoughts. | 4.10 | 1.48 | 5.10 | 1.80 | 1.92 | 0.0315 |
(FA2) I often forgot the passage of time in the environment. | 3.55 | 1.85 | 4.95 | 1.82 | 2.41 | 0.0105 |
(FA3) I was more focused on the content rather than on any external distraction. | 3.85 | 1.63 | 4.70 | 1.84 | 1.55 | 0.065 |
(FA4) During the session, hardly anything could distract me. | 3.20 | 1.54 | 4.65 | 1.76 | 2.78 | 0.0045 |
(FA5) Time went by quickly during the sessions. | 3.80 | 1.61 | 5.30 | 1.53 | 3.03 | 0.002 |
Total | 3.70 | 1.62 | 4.94 | 1.73 | 5.22 | < 0.001 |
Table 5
Immersion (Presence) | (P1) I felt that I was in a realistic environment, in which I could hardly separate what was virtual or real. | 2.25 | 1.55 | 2.85 | 1.73 | 1.16 | 0.1275 | |
(P2) The content felt so authentic that it made me think that the virtual characters/objects existed in reality. | 2.85 | 2.01 | 3.80 | 1.7 | 1.61 | 0.0575 | |
Total | 2.55 | 1.80 | 3.33 | 1.76 | 1.95 | 0.0275 | |
Immersion (Flow) | (F1) I did not have any irrelevant thoughts or external distractions during the session. | 3.00 | 1.34 | 4.60 | 1.73 | 3.27 | 0.001 | |
(F2) The content became the only thought occupying my mind. | 3.45 | 1.50 | 4.80 | 1.64 | 2.71 | 0.005 | |
(F3) I lost track of time as if everything stopped, and the only thing that I could think about was the content. | 2.95 | 1.47 | 4.25 | 1.94 | 2.39 | 0.011 | |
(F4) All of my senses were concentrated on the content. | 3.40 | 1.60 | 5.15 | 1.63 | 3.42 | 0.0005 | |
Total | 3.20 | 1.47 | 4.70 | 1.74 | 5.89 | < 0.001 | |
Motivation | (M1) I enjoyed this type of content very much. | 4.20 | 1.44 | 5.20 | 1.94 | 1.86 | 0.036 |
(M2) Learning in this type of environment was fun. | 3.90 | 1.62 | 5.50 | 1.93 | 2.84 | 0.0035 |
(M3) I would describe this type of content as interesting. | 3.70 | 1.63 | 5.65 | 1.76 | 3.65 | < 0.001 |
(M4) I was satisfied with my performance in this type of content-based learning environment. | 4.45 | 1.50 | 4.90 | 1.83 | 0.85 | 0.2005 |
(M5) I felt pressured while learning in this type of environment. * | 3.70 | 1.56 | 4.10 | 1.80 | 0.75 | 0.229 |
(M6) I did not try very hard while learning in this type of environment. * | 3.10 | 1.65 | 3.65 | 1.81 | 1.00 | 0.161 |
(M7) I was thinking how much I enjoyed learning in this type of environment. | 3.60 | 1.60 | 4.50 | 1.73 | 1.71 | 0.048 |
(M8) I felt competent after trying this type of tool for a while. | 4.15 | 1.42 | 4.75 | 1.52 | 1.29 | 0.1025 |
(M9) I was very relaxed while learning with this type of (VR/2D) tool. | 4.30 | 1.17 | 3.95 | 1.64 | 0.78 | 0.2215 |
(M10) I am very skilled at this type of content. | 5.05 | 1.57 | 5.05 | 1.73 | 0.00 | 0.5 |
(M11) This type of environment did not hold my attention. * | 3.55 | 1.79 | 4.45 | 1.73 | 1.62 | 0.057 |
(M12) I could not learn much using this type of tool. * | 4.05 | 1.54 | 4.65 | 1.84 | 1.12 | 0.1355 |
Total | 3.98 | 1.59 | 4.70 | 1.83 | 4.58 | < 0.001 |
Cognitive benefits | (C1) This type of content makes comprehension easier. | 4.80 | 1.36 | 5.30 | 1.69 | 1.03 | 0.1545 |
(C2) This type of content makes memorization easier. | 3.95 | 1.47 | 4.90 | 1.80 | 1.83 | 0.038 |
(C3) This type of content helps me to better apply what was learned. | 4.25 | 1.68 | 5.35 | 1.79 | 2.01 | 0.026 |
(C4) This type of content helps me to better analyze the problem. | 4.50 | 1.61 | 4.65 | 1.76 | 0.28 | 0.39 |
(C5) This type of content helps me to have a better overview of the content learned. | 4.70 | 1.66 | 5.25 | 1.94 | 0.96 | 0.171 |
Total | 4.44 | 1.56 | 5.09 | 1.78 | 2.75 | 0.0035 |
Perceived learning effectiveness | (PL1) I was more interested to learn the topics. | 4.50 | 1.32 | 5.40 | 1.43 | 2.07 | 0.0225 |
(PL2) I learned a lot of information in the topics. | 5.20 | 1.10 | 5.45 | 1.28 | 0.66 | 0.256 |
(PL3) I gained a good understanding of the basic concepts of the (VR/2D) content. | 5.20 | 1.15 | 5.50 | 1.32 | 0.77 | 0.224 |
(PL4) I learned to identify the main and important issues of the topics. | 5.35 | 0.99 | 5.40 | 1.19 | 0.15 | 0.443 |
(PL5) I was interested and stimulated to learn more. | 3.75 | 1.55 | 5.50 | 1.64 | 3.47 | 0.0005 |
(PL6) I was able to summarize and conclude what I learned. | 5.10 | 1.41 | 5.35 | 1.46 | 0.55 | 0.2925 |
(PL7) The learning activities were meaningful. | 5.05 | 1.23 | 5.65 | 1.35 | 1.47 | 0.075 |
(PL8) What I learned; I can apply in real context. | 4.40 | 1.76 | 5.20 | 1.44 | 1.58 | 0.062 |
Total | 4.82 | 1.4 | 5.43 | 1.37 | 3.95 | < 0.001 |
* These questions are measured with reverse scores, and measure negative items.