What do students know about self-regulated learning with GenAI? A situational judgment test

Authors: Nick Naujoks-Schober & Marion Händel

Abstract

The rapid diffusion of generative artificial intelligence (GenAI) tools such as ChatGPT is reshaping students’ self‑regulated learning (SRL) practices, yet little is known about learners’ conditional knowledge for strategically employing these technologies. To address this gap, we developed and preliminarily validated a situational judgment test that measures students’ strategic knowledge of interacting with GenAI across two typical higher‑education learning scenarios. Following a multi‑step design, we (1) conducted a literature review to identify the most frequent cognitive, metacognitive, and resource‑oriented learning situations involving GenAI; (2) performed problem‑centered interviews with 18 students from eight disciplines to elicit real‑world strategies; and (3) refined these strategies into 36 items spanning the forethought, performance, and self‑reflection phases of SRL for two generic scenarios—planning a study project and creating a summary. Experts (N = 48 SRL scholars and AI specialists) rated the usefulness of each strategy on a six‑point Likert scale, yielding significant pairwise discriminations for the majority of items (28/36 for the summary scenario, 21/36 for the planning scenario). The situational judgment test thus captures conditional knowledge of SRL‑GenAI interaction, offering a robust instrument for assessing metacognitive competence beyond self‑report AI‑literacy measures. Findings will inform educators on students’ knowledge about SRL with GenAI and may guide interventions to foster effective, reflective AI‑enhanced learning.

1.      Introduction and aims

According to models of self-regulated learning (SRL), students should actively manage their learning processes using situation-specific learning strategies (Panadero, 2017). To employ the most effective strategy in a given context, students require conditional knowledge regarding their strategy use. To assess this knowledge, situational judgment tests compare students’ evaluations of strategy usefulness in realistic, hypothetical learning scenarios against expert benchmarks (Dörrenbächer-Ulrich et al., 2024; Pfost & Hübner, 2025).

However, the rapid rise of generative artificial intelligence (GenAI) applications, such as ChatGPT, is transforming learning scenarios and strategies associated with student SRL (Chiu, 2024). Current research discusses potential benefits of integrating GenAI into higher education for individualizing learning, for instance, through quizzes generated with students’ lecture notes (Dillon, 2024; Gärtner et al., 2024). Conversely, first studies highlight challenges in SRL with GenAI as students may delegate tasks too extensively to the AI (cognitive offloading; Lee et al., 2025) or insufficiently monitor their own learning and interaction with the AI (metacognitive laziness; Chardonnens, 2025; Fan et al., 2025).

According to the current state of research it is unclear whether students possess the strategic knowledge necessary to successfully interact with GenAI during self-regulated learning. Previous measurement instruments to assess AI literacy have focused either on students' self-reports (Almatrafi et al., 2024) or on their knowledge about GenAI specifications (Markus et al., 2025). We therefore aimed to develop and validate a situational judgment test to assess students’ knowledge of how to strategically interact with GenAI during their self-regulated learning.

2.      Method

To develop a situational judgment test, we followed a multi-step procedure: (1) We performed a literature search to identify scenarios of leaning with GenAI, (2) we conducted problem-centered interviews with students to select relevant scenarios and to identify strategies to interact with GenAI across different disciplines, and (3) we obtained expert ratings on the consequently developed situational judgment test.

Based on 60 publications, we extracted the three most frequent cognitive, metacognitive, and resource-oriented scenarios. Those scenarios were used for problem-centered interviews with students to analyze their strategy use during GenAI interactions. Using a maximum-variation sampling approach, interviews were held with 18 students across eight different study programs. Students were interviewed regarding their learning behavior when interacting with GenAI. Drawing on the Self-Regulated Learning Interview Schedule (Zimmerman & Pons, 1986), participants first rated the relevance of nine given scenarios (e.g., using GenAI to create a summary) regarding their own learning. Next, students described their strategic approach for those scenarios that they had rated as relevant. Both the content and usefulness of the reported strategies were coded using qualitative content analysis (Mayring & Fenzl, 2019), employing both inductive category development and deductive assignment to overarching learning strategy categories.

In the next step, we extracted strategies for the most relevant scenarios from the interviews. These strategies were iteratively refined according to low, middle, and high usefulness based on the degree of students’ (meta-)cognitive activity and the suitability of the GenAI. This final test was given to experts from the fields of self-regulated learning (N = 31) and/or artificial intelligence (N = 17) who rated the usefulness of the strategies on a 5-point Likert scale from 1 (very low) to 5 (very high).

3.      Results

Identification of scenarios. From the literature review, both discipline-specific and cross-disciplinary scenarios and strategies were identified. Especially the generic scenarios provided a solid foundation for the development of the situational judgment test, which is intended to be applicable to students across all fields.

Development of the situational judgment test. Two of the most relevant and generic scenarios for learning with GenAI were selected for the test, namely planning a study project and creating a summary. For each scenario, strategy options were identified from the interviews that differed in terms of their quality and usefulness for learning purposes. In line with Zimmerman's model (1986), each scenario was structured along the three phases of self-regulated learning. That is, the forethought phase before planning (p)/summarizing (s), the performance phase during p/s, and the self-reflection phase after p/s. For each scenario and each phase, six strategies were developed, resulting in 36 strategies differing in usefulness.

Usefulness of strategies (expert ratings). Initial descriptive observations of the expert ratings and two-factor variance analyses for Friedman ranks in connected samples indicate theoretically valid and significant pair comparisons of the strategies. Across all phases of the summary scenario, the expert ratings confirmed 28 out of 36 pair comparisons. In the planning scenario, however, the expert judgments showed only 21 valid pair comparisons. At the time of the conference, data from a student sample from different disciplines will be available, which allows for an initial validation of the test instrument.

4.      Theoretical and educational significance of the research

This research addresses the rapidly changing learning behavior shaped by GenAI, where SRL competencies are becoming increasingly vital for actively engaging in and maintaining an overview of one's own learning process. Additionally, by using typical scenarios in higher education and comparing student and expert ratings, the developed test goes beyond current self-reports of AI literacy and knowledge tests about GenAI. In doing so, the situational judgment test also captures an aspect of higher metacognitive skills through conditional knowledge of strategy use, which is often overlooked in other operationalizations of AI literacy (Almatrafi et al., 2024). As metacognitive monitoring and learners’ cognitive engagement seems central for self-regulated learning with GenAI interaction, the situational judgment test should provide a robust measure even against the rapid development of GenAI models.

Assessing conditional knowledge for strategic learning with GenAI also provides educators with an estimate of the learners’ existing competence in this regard. Based on students' answers, it will also be possible to identify how students consider GenAI useful across different learning scenarios. These insights reveal, for example, whether pure task outsourcing is considered more useful than targeted co-constructive processes, and whether students recognize their own (meta-)cognitive activity as a central element. Future research will focus on how students can be supported in their interactive process with GenAI.

5.      References

Almatrafi, O., Johri, A., & Lee, H. (2024). A systematic review of AI literacy conceptualization, constructs, and implementation and assessment efforts (2019–2023). Computers and Education Open, 6, 100173. https://doi.org/10.1016/j.caeo.2024.100173

Chardonnens, S. (2025). Adapting educational practices for Generation Z: Integrating metacognitive strategies and artificial intelligence. Frontiers in Education, 10, Article 1504726. https://doi.org/10.3389/feduc.2025.1504726

Chiu, T. K. F. (2024). A classification tool to foster self-regulated learning with generative artificial intelligence by applying self-determination theory: A case of ChatGPT. Educational Technology Research and Development, 72(4), 2401–2416. https://doi.org/10.1007/s11423-024-10366-w

Dillon, T. (2024). Korean university students’ prompt literacy training with ChatGPT: Investigating language learning strategies. English Teaching, 79(3), 123–157. https://doi.org/10.15858/engtea.79.3.202409.123

Dörrenbächer-Ulrich, L., Sparfeldt, J. R., & Perels, F. (2024). Knowing how to learn: Development and validation of the strategy knowledge test for self-regulated learning (SKT-SRL) for college students. Metacognition and Learning, 19(2), 1–45. https://doi.org/10.1007/s11409-024-09379-w

Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025). Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology, 56(2), 489–530. https://doi.org/10.1111/bjet.13544

Gärtner, C., Moraß, A., Koss, S., Garbusa, S., Matern, S., Innermann, I., & Forschungs- Und Innovationslabor Digitale Lehre, F. (2024). Einsatz, Nutzen und Grenzen von ChatGPT und anderen Large Language Modellen an den bayerischen HAWs (No. 5; Die Studien- und Schriftenreihe des Forschungs- und Innovationslabors Digitale Lehre – FIDL). FIDL – Forschungs- und Innovationslabor Digitale Lehre. https://doi.org/10.34646/THN/OHMDOK-1466

Lee, H.-P. (Hank), Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025). The impact of generative AI on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, Article 1121. https://doi.org/10.1145/3706598.3713778

Markus, A., Carolus, A., & Wienrich, C. (2025). Objective measurement of AI literacy: Development and validation of the AI competency objective scale (AICOS). Computers and Education: Artificial Intelligence, 9, 100485. https://doi.org/10.1016/j.caeai.2025.100485

Mayring, P., & Fenzl, T. (2019). Qualitative Inhaltsanalyse. In N. Baur & J. Blasius (Eds.), Handbuch Methoden der empirischen Sozialforschung (pp. 633–648). Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-21308-4_42

Panadero, E. (2017). A review of self-regulated learning: Six models and our directions for research. Frontiers in Psychology, 8, Article 422. https://doi.org/10.3389/fpsyg.2017.00422

Pfost, M., & Hübner, V. (2025). Assessment of strategic knowledge of learning from errors in higher education. Diagnostica, 71(2), 53–63. https://doi.org/10.1026/0012-1924/a000341

Zimmerman, B. J. (1986). Becoming a self-regulated learner: Which are the key subprocesses? Contemporary Educational Psychology, 11(4), 307–313. https://doi.org/10.1016/0361-476x(86)90027-5

Zimmerman, B. J., & Pons, M. M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23(4), 614–628. https://doi.org/10.3102/00028312023004614

Reflection silencers are installed in the exhaust system of stationary combustion engines to attenuate low-frequency noise by means of destructive interference. The acoustic properties of mufflers are experimentally determined by the standard two-load method, which only considers measurements without mean flow. In real engine operation, however, exhaust mass flow is always present. Measurements are significantly more complex and expensive if fluid flow is taken into account, which is why the available data is limited. Thus, the impact of mean flow on the attenuation of silencers is not clearly known yet. This work contributes to the state of the art by quantifying the influence of the Mach number on the transmission loss of double-tuned straight-through mufflers based on reproducible, noise corrected measurement results that include uncertainties. A frequency range between 20 Hz and 891 Hz is investigated at eleven different Mach numbers between 0 and 0.1 under ambient conditions. It is found that resonance peaks diminish with increasing Mach number, while other frequencies remain unaffected by mean flow. These findings can be transferred to operating conditions of stationary combustion engines and other exhaust systems. The experimental data will serve as a basis for the validation of analytical and numerical models in subsequent work.

In this work, the Theory of Porous Media (TPM) is employed to model percutaneous vertebroplasty, a medical procedure in which acrylic cement is injected into cancellous vertebral bone. Previously, isothermal macroscale models have been derived to describe this material injection and the arising mechanical interactions. However, the temperature of the injected cement is typically below the human body temperature, necessitating the extension of these existing models to the non-isothermal case. Following the modelling principles of the TPM and considering local thermal non-equilibrium conditions, our model introduces three energy balances as well as constitutive relations for thermal conduction and heat transfer. If restricted to local thermal equilibrium conditions, our model equations are in agreement with other TPM-based models. We observe that our model elicits physically reasonable behaviour in numerical simulations that employ parameter values and initial and boundary conditions relevant for our application. We claim our model to be thermodynamically consistent despite the employment of the Coleman and Noll procedure.

Empirische Rechtswissenschaft im Fokus: Dieses Buch bietet einen fundierten Überblick über Stand und Perspektiven interdisziplinärer Rechtswissenschaften. Von Verfassungs-, Europa-, Kartell- und Strafrecht über richterliches Entscheidungsverhalten bis hin zu Fragen von Politikberatung, Arbeitsmärkten und Gemeinschaftsgütern entfalten die Beiträge ein breites thematisches Spektrum. Zugleich zeigen sie, wie Verhaltenswissenschaften, datengetriebene Methoden und Künstliche Intelligenz unser Verständnis des Rechts vertiefen und verändern. Sie zeichnen dabei auch das Wirken von Christoph Engel, Direktor am Max-Planck-Institut zur Erforschung von Gemeinschaftsgütern, nach. 

While forests are known to increase turbulence and fatigue loads on wind turbines, the impact of smaller-scale vegetation such as tree rows has received limited attention. This study investigates speed-up effects caused by a tree row and their influence on wind turbine power and blade loads using Detached Eddy Simulation (DES) and OpenFAST. A realistically modeled tree row at the Risø campus is considered, including seasonal variations in leaf area index (LAI). The simulations reveal a speed-up region above the canopy, leading to a relative power increase of approximately 6% for the high-LAI case and about half that value for the low-LAI case. Aeroelastic simulations with the NREL 5 MW wind turbine confirm the power increase and show a rise in flapwise blade root bending loads. Second-order statistics within the rotor plane remain largely unchanged, indicating that the load increase is driven by mean flow acceleration rather than turbulence. These results demonstrate that tree rows can increase power and highlight the influence of local vegetation in wind turbine siting.

This paper experimentally examines how different greenfluencer message types (emotional, informational, social, and neutral) on Instagram affect purchase intention and attitudes toward sustainable fashion, with a particular focus on non-sustainable consumers. In an online experiment, 202 participants were randomly assigned to one of four fictitious Instagram posts and completed pre- and post-measures of attitudes and purchase intention. The results show that none of the message types improved purchase intention or attitudes; instead, most effects were negative. However, attitudes remained positively associated with purchase intention. The findings suggest that greenfluencer communication does not automatically promote sustainable consumer responses and is not more effective among non-sustainable consumers.

Background:
While border screening measures were widely adopted by countries during the COVID-19 pandemic, a lack of consensus on the utility of border screening created a gap in best practice for its implementation. As such, countries adopted a diversity of approaches, providing an opportunity to evaluate the configuration and evolution of border screening systems. The
article addresses three questions: (i) how did countries configure their border screeningsystems for COVID-19? (ii) In what contexts did countries rely on public or private providers of these services? (iii) what do policies and narratives reveal about the perceived role of border screening in global public health? The article contributes to long-standing debates over the
private sector’s role in public health and the perceived value of border screening measures.

Methods:
This article presents results from an international comparative study based on tracking the organisation of border screening in eight countries. Secondary data was collected between  July 2021 – June 2022 from official government websites and policy publications, private sector sources where relevant, and trusted media sources in each study country. The
countries included are Australia, Canada, Germany, Ireland, South Africa, South Korea, Spain, and the United Kingdom.

Results:
All study countries used private provision for pre-departure diagnostic testing for international travellers. In contrast, screening of arriving travellers was more diverse. Countries that opted for private sector post-arrival screening saw governance challenges around accreditation and monitoring of providers, while public service provision saw challenges in capacity and high
resource costs. Travel was often framed as a ‘luxury,’ allowing states to shift responsibility for obtaining tests onto individuals; especially in the context of individuals travelling from low income to high income countries.

Conclusions:
The different approaches countries followed for screening of departing and incoming travellers suggests wealthy countries were more oriented towards defending their populations against disease importation, rather protecting the international community from disease exportation. These findings provide an opportunity to reflect on the purpose and
implementation of border screening. We emphasise a need for further discussion on the efficacy of border screening from both perspectives, given the tendency for countries to rely on these measures