Guidelines on the Critical Use of Generative AI Systems in Education and Training Practices

Human Responsibility Remains Central

Gen-AI can generate alternatives, identify patterns, suggest solutions, structure information and detect inconsistencies. None of these capabilities transfers responsibility from the human user to the system.

The REBOOT AI Game-Driven Assistant T3.2.2 makes this distinction explicit. AI can generate a first version of a game and can subsequently assist in auditing that game. Nevertheless, the final judgement concerning pedagogy, ethics, narrative credibility and player experience remains with the game author and project team.

The ability of an AI system to generate or analyse information does not transfer human responsibility to the system.

Teachers, trainers, learners, educational designers and institutions remain responsible for the decisions made using AI-generated information. AI may advise, suggest, generate and identify inconsistencies, but humans must decide.

AI Output Should Be Treated as Provisional, Not Authoritative

One of the most valuable educational opportunities created by Gen-AI is precisely that its outputs should not automatically be trusted. REBOOT develops this idea directly in the AI-driven platform. The learner is expected to work with AI-generated information as evidence that must be evaluated, rather than as an answer that must be accepted. Every important piece of information passes through human verification before it becomes part of the game record.

Never equate fluency with truth.

A convincing AI answer may be accurate, partially accurate, incomplete, contextually inappropriate, an inference presented too strongly, based on questionable assumptions, or incorrect. Educational activities involving Gen-AI should therefore create opportunities for learners to ask:

The REBOOT Fact Log T3.2.3 demonstrates this principle particularly clearly. The quality of the final game depends on the quality of the learner's verification work, rather than on the quantity of information generated.

AI Should Support Thinking, Not Replace It

Gen-AI can significantly reduce the effort required to produce an answer. This is useful, but it can also create an educational problem if producing the answer removes the cognitive activity that the learner was expected to practise. REBOOT therefore makes a distinction between getting an answer and developing the capacity to reach a judgement.

The AI Game-Driven Assistant T3.2.2 is designed so that its role, limits and behaviour support rather than replace player reasoning. The Game Audit consequently asks a critical pedagogical question:

Before introducing Gen-AI into an educational activity, educators should ask: what cognitive work should remain with the learner? If the learning objective is argumentation, interpretation, comparison, judgement, problem solving, creative decision-making or critical thinking, AI should not remove the need to perform those activities. Instead, AI can create alternatives to compare, claims to verify, arguments to challenge, scenarios to evaluate, drafts to improve, contradictions to resolve, evidence to organise and perspectives to discuss. The objective is not to prevent AI from helping the learner, it is to ensure that help does not become substitution.

Critical AI Literacy Is Learned Through Interaction

REBOOT's training methodology suggests that critical AI literacy does not need to begin with a theoretical explanation of artificial intelligence. The Masterclass reference T3.1.4 describes a different pedagogical sequence. Participants first experience situations in which they must move, judge, classify, listen, negotiate, reveal, withhold and compare. Reflection follows the experience, so understanding develops from examining the consequences of action.

Instead of teaching learners only a list of AI limitations, educators can design situations in which learners discover those limitations themselves, for example by asking several Gen-AI systems the same question, comparing their answers, identifying contradictions, verifying factual claims, examining missing perspectives, rewriting or improving the outputs, explaining why they accepted or rejected particular information, and reflecting on what would have happened if the first AI answer had simply been accepted. Critical AI literacy then becomes an activity, not a definition.

Distinguish AI-Generated, AI-Assisted and Human-Created Content

The REBOOT Content Management Tool T3.2.1 is designed around an important distinction: fully AI-generated content, AI-assisted content, and content produced through traditional manual methods. This classification should become an explicit element of critical AI practice, not only a documentation-level principle. In the current version of the CMT interface, content origin is tracked at project level rather than as a field on every entry; adding a per-entry tag for this classification is recommended as a near-term enhancement, so the distinction is visible at the point of content entry, not only in retrospective documentation.

Not all use of AI represents the same level of human contribution.

AI-generated: the AI produces most or all of the initial content from instructions provided by the human.
AI-assisted: the human and AI participate iteratively, the AI may propose, analyse, restructure, expand or challenge material while the human substantially determines, evaluates and transforms the result.
Human-created: the substantive content is produced by the human without generative AI participating in its creation.

These categories are useful not to establish a hierarchy in which one method is automatically superior, but to make the production process visible. Learners and educators should be able to explain what the AI contributed, what the human contributed, who verified the result, and who made the final decisions.

AI Generation Is the Beginning of Authorship, Not Its End

The REBOOT AI Game-Driven Assistant T3.2.2 provides a particularly concrete example of AI-assisted creation. Its manual states that AI generation is a starting point rather than the end of authorship. After generation, the human author is expected to rewrite texts, correct questions and answers, change scores, replace media, delete material that the AI has overproduced, and introduce game-specific logic requiring human judgement. The resulting workflow is deliberately hybrid, AI provides breadth and first-pass construction, while humans remain responsible for narrative quality, pedagogy, accuracy and final experience design.

Submitting the first AI output should rarely represent completion of a learning task.

Educational activities can instead require learners to demonstrate transformation: Generate → Examine → Verify → Challenge → Modify → Justify → Produce. The important evidence of learning is therefore not simply the AI output, it is what the learner subsequently does with that output.

Verification Should Be Designed Into the Learning Activity

Critical use should not depend solely on telling students to "be careful with AI." Verification should become part of the structure of the activity. REBOOT provides an unusually strong example. The learner works with documents, testimonies, claims and AI-generated information that can remain confirmed, disputed or unresolved. The system deliberately creates situations in which no single source possesses the complete answer.

ERUDITA T3.2.3 expresses this principle through its distributed investigative structure:

Learners should be encouraged to use multiple sources rather than one AI response, and to distinguish between evidence, interpretation and conclusion. Verification can therefore become part of assessment. Instead of asking only "what is your answer," educators can also ask what the learner verified, what they rejected, what remained uncertain, and what changed their original interpretation.

Uncertainty Is Educationally Valuable

Traditional educational resources frequently present knowledge after uncertainty has already been removed. Gen-AI creates an opportunity to work differently. ERUDITA T3.2.3 deliberately distributes incomplete information between players. Learners must search, compare, communicate and make decisions. Learning emerges through interpreting evidence, confronting contradictions and reconsidering what they previously believed. The plot itself begins when ordinary records start to contradict the official explanation, and when fragments from different sources gradually form a pattern.

This approach is highly relevant to Gen-AI literacy, because AI systems frequently produce outputs containing different degrees of certainty. Critical education should therefore give learners permission to conclude "we do not know yet," "the available evidence is insufficient," or "there are competing interpretations." Recognising uncertainty is a cognitive competence, not a failure to answer.

Critical Use Includes the Capacity to Disagree With AI

A learner who knows how to prompt an AI system but does not know when to reject its answer is not yet critically AI literate. REBOOT makes disagreement part of the educational architecture through the Oracle, ERUDITA's AI facilitator T3.2.3. By design the Oracle deliberately provides one incorrect answer during play, precisely to incentivise cross-verification rather than passive acceptance, and this single, concrete mechanism is a working example of Principle 02 in practice: a designed reason to distrust fluent output.

The learner must retain epistemic authority over the AI.

The system may propose an answer, but the learner must be able to say this is incorrect, this is unsupported, this contradicts another source, this interpretation is too strong, this perspective is missing, this needs additional evidence, or I do not accept this conclusion. Critical Gen-AI competence therefore includes not merely effective prompting, but effective resistance to unreliable output.

Human Agency Must Remain Visible

REBOOT's game architecture deliberately prevents the virtual character or AI agent from carrying responsibility for the human player. In ERUDITA T3.2.3, the human is a supporter, investigator and decision partner. The virtual character possesses information and uncertainties, but the human player interprets evidence and decides what should happen.

The Oracle follows the same philosophy. It can know information and can answer questions, but it never interprets on the player's behalf, never initiates action, and stores everything it is told, so that it can be checked. It does not carry the player's responsibility.

AI can participate in a decision without becoming the decision-maker.

This is particularly important where educational decisions involve values, consequences for other people, sensitive information or ethical judgement.

Define What the AI Is Supposed to Do

One recurring lesson from the development of the REBOOT platform is that the phrase "AI-driven" is insufficient. The AI-driven Game Platform concept T3.1.1 distinguishes several fundamentally different functions: AI as a character, AI as a content generator, and AI as a grading or analysis layer. These functions carry different requirements, risks and potential failures, so the role of AI should be defined before the system is developed or introduced into an educational activity.

For education and training, this means defining whether AI is being used as a tutor, conversational partner, simulation character, source of information, brainstorming partner, content generator, feedback system, evaluator, translator, research assistant, creative collaborator or administrative tool. A conversational character should not necessarily possess the same authority as a tutor. A brainstorming tool should not be treated as a factual source. A content generator should not automatically become an evaluator of the same content. Clarity of purpose is therefore part of responsible AI design.

Define the Boundaries of AI Agents

The development of AI co-players in REBOOT raises a second important question: what should an AI agent know, and what should it be allowed to do? The Oracle's own design rules are a concrete, already-built answer. It never interprets, it never initiates, it stores everything it is told, and it deliberately gives one incorrect answer over the course of play to incentivise cross-verification. Similarly, the ERUDITA design T3.2.3 requires the author to determine what an agent knows, what it does not know, what it believes, what it may infer, and how its available information changes during the game.

This can be translated directly into educational practice. Before deploying an AI agent, educational institutions should define its purpose (why does this AI exist), its knowledge (what information can it access), its boundaries (what should it not answer or do), its authority (what decisions can it support but not make), its escalation path (when should a human teacher or trainer become involved), and its monitoring (how can problematic interaction be identified). An AI agent without defined boundaries is not simply flexible, it is poorly designed.

Human Oversight Must Be Operational, Not Symbolic

Saying that a "human remains in the loop" is insufficient unless the human actually has opportunities to examine and change what the AI does. REBOOT operationalises oversight in several ways. The AI Game-Driven Assistant T3.2.2 allows generated content to be edited directly in the Content Management Tool T3.2.1. Player-facing AI conversations can be reviewed through a control and audit function. The completed game can be audited for gaps, inconsistencies, repetition and weaknesses.

Human oversight therefore requires the ability to inspect, challenge, correct, override, delete, modify, test, and when necessary, stop the AI-mediated activity.

AI Systems Must Be Tested With Real Users

REBOOT also identifies the danger of confusing a convincing demonstration with an educational system that actually works. A scripted demonstration does not show what will happen when a real learner interacts unpredictably with an AI system. The materials therefore emphasise prototyping, testing and iteration, and warn against claiming capabilities that have not been demonstrated in practice, a lesson already confirmed by the April 2026 live session in Rome, where twelve students ran the full I AGREE META Game across ten exercises.

The Game Design Blueprint T3.1.3 reinforces this principle by defining design as player-centred, testable and iterative. Critical assumptions should be converted into hypotheses that can be observed through playtesting. For educational Gen-AI systems, evaluation should consequently examine not merely whether the technology works, but whether learners actually understand its role, question its outputs, retain agency, achieve the intended learning outcomes, encounter unforeseen difficulties, and use it in ways that were not anticipated by the designer.

Responsible AI Must Be Designed From the Beginning

Responsible use should not be added after an AI educational experience has already been created. The REBOOT Game Design Blueprint T3.1.3 explicitly defines responsible design as incorporating accessibility, privacy, safety, inclusion and ethical considerations from the beginning. Its quality criteria subsequently expand this to include safeguarding, representation, transparency and player wellbeing.

The AI-driven Game Platform concept T3.1.1 makes the same point from practical experience: where students interact freely with AI, particularly around sensitive topics, safety and moderation must be designed from the start rather than added shortly before deployment. The Guidelines therefore adopt the principle of responsibility by design. Before an AI-mediated educational activity begins, its designers should consider accessibility, privacy, safeguarding, inclusion, representation, transparency, wellbeing, moderation, and the sensitivity of the subject matter. These considerations should influence the design itself, rather than appear only as disclaimers.

Critical AI Education Is Also Ethical Decision-Making

Critical use of AI is not limited to determining whether information is factually correct. Many decisions require consideration of consequences, values and competing interests. ERUDITA T3.2.3 was deliberately designed so that ethical issues are not simply explained to the learner, they are embedded in decisions. The guiding principle of the plot is:

Learners progressively interpret incomplete evidence, confront uncertainty and eventually become responsible for a collective decision. The player does not simply receive a predetermined truth, the player participates in constructing enough understanding to become responsible for acting. Learners should not only be told principles such as fairness, responsibility and transparency, they should encounter situations where those principles have consequences and where different choices can be discussed, defended and reconsidered.

Regulatory and Institutional Alignment

The pedagogical principles above describe how REBOOT already treats AI critically in practice. This section places that practice against the current EU legal framework, so the Guidelines function as both a pedagogical and a compliance-aware document.

This section reflects the EU AI Act's transparency and high-risk provisions as they stand at the time of writing and should be reviewed as implementing guidance and national transposition measures develop.