Guided and Unguided AI Use in Assessments: Student Engagement and Satisfaction
DOI:
https://doi.org/10.11594/ijmaber.07.09.05Keywords:
Artificial intelligence, GenAI, Guided use, Student engagement, Student satisfaction, Unguided useAbstract
The rapid adoption of generative artificial intelligence (GenAI) in higher education has changed how students plan, draft, revise, and complete assessment tasks. However, there is still limited evidence on how instructional guidance shapes students’ experiences with these tools. This study examined whether guided use of generative AI, defined as the use of structured prompting protocols, required human-in-the-loop critical reflection, and verification workflows, produces different outcomes from unguided use. A quantitative, quasi-experimental, cross-sectional comparison was employed involving 50 undergraduate students from information technology-related programs, with 25 students in the guided AI use group and 25 in the unguided AI use group. Student engagement and learning satisfaction were measured using multi-item Likert-scale subscales administered through an online post-assessment survey, and group differences were analyzed using independent-samples t-tests after assumption checks for normality and homogeneity of variance. Results showed that the guided AI use group reported significantly higher student engagement than the unguided group. The guided group likewise obtained significantly higher learning satisfaction than the unguided group. These findings indicate that AI does not improve educational outcomes through access alone. Instead, its value depends on pedagogical structure, explicit expectations, and instructor support. The study provides quantitative evidence that guided AI use can promote more active engagement and more satisfying assessment experiences than unguided use. It also offers practical implications for designing AI-supported assessment in higher education.
Downloads
References
Almufarreh, A. (2024). Determinants of Stu-dents’ Satisfaction with AI Tools in Edu-cation: A PLS-SEM-ANN Approach. Sus-tainability, 16(13), 5354. https://doi.org/10.3390/su16135354
Almulla, M. A. (2024). Investigating influencing factors of learning satisfaction in AI ChatGPT for research: University students perspective. Heliyon, 10(11). https://doi.org/10.1016/j.heliyon.2024.e32220
Baidoo-Anu, D., & Ansah, L. O. (2023). Educa-tion in the era of generative artificial in-telligence (AI): Understanding the poten-tial benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52–62. https://doi.org/10.2139/ssrn.4337484
Basch, C., Hillyer, G., Gold, B., & Yousaf, H. (2025). Understanding artificial intelli-gence knowledge and usage among col-lege students: Insights from a survey on classroom, coursework, and personal ap-plications. Review of Artificial Intelligence in Education, 6(i). https://doi.org/10.37497/rev.artif.intell.educ.v6ii.37
Borges, I., Sebastião, C., Caridade, C., & Pereira, V. (2026). From ChatGPT to classroom learning: Exploring the role of teacher mediation in AI-supported education. Australasian Journal of Educational Technology, 42(3).
https://ajet.org.au/index.php/AJET/article/view/11056
Cassim, F. A. K., & Yang. (2025, December 1). GAI pedagogical agents and teacher-guided prompt use in EFL debate train-ing. International Conference on Comput-ers in Education. ICCE 2025: The 33rd In-ternational Conference on Computers in Education.
Chan, C. K. Y., & Hu, W. (2023). Students’ voic-es on generative AI: Perceptions, bene-fits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20(1), 43. https://doi.org/10.1186/s41239-023-00411-8
Cohen, J. (1992a). A power primer (Pt. 155-159). Psychological Bulletin, 112(1). https://doi.org/10.1037/0033-2909.112.1.155
Cohen, J. (1992b). Statistical Power Analysis. Current Directions in Psychological Sci-ence, 1(3), 98–101. JSTOR.
Davis, F. D. (1989). Perceived usefulness, per-ceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008
Delello, J. A., Sung, W., Mokhtari, K., Hebert, J., Bronson, A., & De Giuseppe, T. (2025). AI in the classroom: Insights from educators on ssage, challenges, and mental health. Education Sciences, 15(2), 113. https://doi.org/10.3390/educsci15020113
Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1). https://doi.org/10.11648/j.ajtas.20160501.11
Ghasemi, A., & Zahediasl, S. (2012). Normality tests for statistical analysis: A guide for non-statisticians. International Journal of Endocrinology and Metabolism, 10(2), 486–489. https://doi.org/10.5812/ijem.3505
Guo, F., Zhang, L., Shi, T., & Coates, H. (2025). Whether and when could generative AI improve college student learning en-gagement? Behavioral Sciences, 15(8), 1011. https://doi.org/10.3390/bs15081011
Handelsman, M. M., Briggs, W. L., Sullivan, N., & Towler, A. (2005). A Measure of Col-lege Student Course Engagement. The Journal of Educational Research, 98(3), 184–192. https://doi.org/10.3200/JOER.98.3.184-192
Hooda, M., Rana, C., Dahiya, O., Rizwan, A., & Hossain, M. S. (2022). Artificial intelli-gence for assessment and feedback to enhance student success in higher educa-tion. Mathematical Problems in Engineer-ing, 2022(1), 5215722. https://doi.org/10.1155/2022/5215722
Hostetter, A. B., Call, N., Frazier, G., James, T., Linnertz, C., Nestle, E., & Tucci, M. (2024). Student and faculty perceptions of gener-ative artificial intelligence in student writing. Teaching of Psychology, 52(3), 319–329. https://doi.org/10.1177/00986283241279401
Kestin, G., Miller, K., Klales, A., Milbourne, T., & Ponti, G. (2025). AI tutoring outperforms in-class active learning: An RCT introduc-ing a novel research-based design in an authentic educational setting. Scientific Reports, 15(1), 17458. https://doi.org/10.1038/s41598-025-97652-6
Kim, J., Klopfer, M., Grohs, J. R., Eldardiry, H., Weichert, J., Cox, L. A., & Pike, D. (2025). Examining faculty and student percep-tions of generative AI in university cours-es. Innovative Higher Education, 50(4), 1281–1313. https://doi.org/10.1007/s10755-024-09774-w
Mishra, P., Singh, U., Pandey, C. M., Mishra, P., & Pandey, G. (2019). Application of stu-dent’s t-test, analysis of variance, and co-variance. Annals of Cardiac Anaesthesia, 22(4), 407–411. https://doi.org/10.4103/aca.ACA_94_19
Morgana, V., & Poli, F. (2026). Guided and un-guided GenAI tasks for learning academic spoken collocations. Journal of English for Academic Purposes, 80, 101638. https://doi.org/10.1016/j.jeap.2026.101638
Otto, S., Ejsing-Duun, S., & Lindsay, E. (2025). Disruptive tensions and emerging prac-tices: An exploratory inquiry into student perspectives on generative artificial intel-ligence in a problem-based learning envi-ronment. Education and Information Technologies, 30(13), 19111–19140. https://doi.org/10.1007/s10639-025-13533-5
Rasul, T., Nair, S., Kalendra, D., Robin, M., San-tini, F. de O., Ladeira, W. J., Sun, M., Day, I., Rather, R. A., & Heatcote, L. (2023). The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning & Teaching, 6(1), 41–56. https://doi.org/10.37074/jalt.2023.6.1.29
Sawilowsky, S. S., & Blair, R. C. (1992). A more realistic look at the robustness and Type II error properties of the t test to depar-tures from population normality. Psycho-logical Bulletin, 111(2), 352–360. https://doi.org/10.1037/0033-2909.111.2.352
Winer, B. J., Brown, D. R., & Michels, K. M. (1991). Statistical principles in experi-mental design (3rd ed.). McGraw-Hill.
Yang, H. (2025). Harnessing generative AI: Ex-ploring its impact on cognitive engage-ment, emotional engagement, learning retention, reward sensitivity, and motiva-tion through reinforcement theory. Learning and Motivation, 90, 102136. https://doi.org/https://doi.org/10.1016/j.lmot.2025.102136
Yeung, S. (2025). University students’ engage-ment with generative AI-supported au-tomated writing evaluation (AWE) feed-back. Journal of Second Language Writ-ing, 68, 101203. https://doi.org/10.1016/j.jslw.2025.101203
Yusuf, A., Pervin, N., & Román-González, M. (2024). Generative AI and the future of higher education: A threat to academic integrity or reformation? Evidence from multicultural perspectives. International Journal of Educational Technology in Higher Education, 21(1), 21. https://doi.org/10.1186/s41239-024-00453-6
Zhao, C. (2024). AI-assisted assessment in higher education: A systematic review. Journal of Educational Technology and Innovation, 6(4), 39–58. https://doi.org/10.61414/jeti.v6i4.209
Downloads
Published
Data Availability Statement
The data used in this study are not publicly available to protect the anonymity of the participants. Summary statistics and anonymized data tables are included in the manuscript.
Issue
Section
Categories
License
Copyright (c) 2026 Jelica R. Enriquez
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See the Effect of Open Access).
