PROCUREMENT · TOOL EVALUATION
Five Questions Every Learning Technologist Should Ask Before Adopting an AI Assessment Tool
You carry the implementation burden. These five questions go past the marketing and into how the tool behaves in your infrastructure.
By Eduface · July 2026 · 8 min read
When an institution decides to adopt an AI assessment tool, it is rarely the DVC who has to make it work on a Tuesday morning. That falls to you: configuring the LMS integration, troubleshooting grade passback errors, training reluctant staff, and fielding complaints when something does not behave as the vendor promised. That reality means the questions you ask during procurement matter more than anyone else’s sign-off.
What should learning technologists look for?
Learning technologists need to evaluate AI assessment tools on four practical dimensions: LMS compatibility, data compliance, lecturer experience, and institutional risk. The questions that matter are not about the AI’s marketing claims. They are about how the tool behaves in your specific infrastructure, under your institution’s legal obligations, and in the hands of lecturers who did not ask for more software.
1. Does the tool integrate with our LMS via LTI 1.3?
LTI 1.3 (Learning Tools Interoperability version 1.3) is the current standard for secure, authenticated connections between third-party tools and your LMS. It handles single sign-on, data exchange, and grade passback as a unified flow. Students submit work inside the LMS as normal; grades return to the gradebook automatically. Nobody needs a separate login and nobody has to export a CSV.
Tools that rely on LTI 1.1, custom plugins, or manual grade entry create ongoing maintenance problems. LTI 1.1 has weaker security and is being deprecated across major platforms. Custom plugins need updating with every LMS version release, which typically lands on your desk. Manual grade transfer introduces error, adds staff time, and breaks the student experience.
Before signing anything, ask the vendor to confirm their LTI version and walk you through a live grade passback demonstration on your specific platform. Canvas, Brightspace, Moodle, and Blackboard each handle LTI implementation slightly differently. Confirmation that it works with Moodle is not the same as confirmation that it works with your version of Moodle at your institution’s configuration.
Eduface integrates via LTI 1.3 with all four major platforms. Students submit through the LMS; grades pass back automatically without any workaround layer.
2. Where is student data processed, and does it leave the EU?
Student data is personal data under GDPR. That means your institution needs a lawful basis for processing it, a signed Data Processor Agreement with the vendor, and clarity on where the data goes once it leaves your systems.
The complication with AI assessment tools is that many of them route submission data through third-party AI APIs: OpenAI, Google Gemini, Azure OpenAI, and similar services. Those APIs are typically hosted on US infrastructure. Transferring personal data to the US requires either a Standard Contractual Clause arrangement or a transfer impact assessment under Chapter V of GDPR. In practice, many vendors have not done this work thoroughly, and if your institution is subject to an audit or a subject access request, the gap becomes your problem.
Ask vendors three direct questions: where are your servers located? Do you use any third-party AI APIs to process submission content? Do you have a DPA ready to sign? If you receive vague answers about industry-standard security practices or data is protected in transit, treat that as a red flag. A compliant vendor should be able to answer those questions in a single paragraph.
Eduface processes all data on its own GPU infrastructure in the Netherlands. No external AI APIs are used at any stage. Student submission data does not leave Eduface’s own servers.
3. What does the lecturer workflow look like day to day?
Adoption of any new tool depends almost entirely on whether lecturers will actually use it. The academic who spent twenty years marking essays in a Word document is not going to embrace an AI tool that requires three new platforms, a training course, and a process that takes longer than marking by hand.
The practical questions are: where does the lecturer review AI-generated feedback? Is it inside the LMS, inside the vendor’s own interface, or somewhere else? How long does the review process take per submission? Is the AI output accurate enough that approval is a quick scan-and-confirm, or does it typically require substantial editing?
A tool that generates feedback the lecturer has to rewrite from scratch is not saving time. It is creating a two-stage process where previously there was one. That is not a workflow improvement; it is a workflow replacement that costs more effort.
Eduface’s reviewer interface allows lecturers to review AI-generated feedback drafts before anything reaches students. Based on UK pilots, Eduface’s AI output aligns with lecturer assessments at 95% accuracy. In practice, that means most review sessions are confirm-and-release rather than edit-and-rewrite.
4. Is the human oversight genuine, or a compliance checkbox?
Under the EU AI Act (Regulation 2024/1689), AI systems used in student assessment are classified as high-risk applications under Annex III. Article 14 requires that high-risk AI systems be designed to allow effective human oversight: specifically, the ability to review, question, and override AI outputs before they affect someone.
The problem is that some tools interpret human oversight to mean that a human can, in theory, intervene. In practice, those tools release grades automatically on a timer unless a lecturer actively steps in to block them. That inverts the burden of oversight. The default should be human approval, not human intervention.
Ask the vendor: what happens to an AI-generated grade if the lecturer does not actively review it? If the answer is that it eventually releases automatically, that is not meaningful human oversight under the terms of Article 14. It is a liability risk for the institution and a reputational risk for the lecturer whose name is on the feedback.
Eduface holds every AI-generated grade as a draft. Nothing reaches the student without explicit lecturer approval. Human control is the default, not the fallback.
5. What evidence is there that the tool actually works?
Any vendor can claim their AI achieves high accuracy. The relevant question is: accuracy compared to what, measured how, in which disciplines, and verified by whom?
Ask for pilot data that includes named institutions, specific cohorts and disciplines, sample sizes, and a clear methodology for how AI-generated grades were compared against human marking. A vendor who cannot provide this is asking you to treat a marketing claim as evidence.
Also ask whether the tool appears on a recognised procurement framework. In the UK, the Jisc/CHEST framework involves vendor vetting across security, compliance, and technical standards. Inclusion on the framework is not a guarantee of quality, but it provides a baseline of institutional confidence that is absent from an unknown vendor’s self-reported accuracy figure.
Eduface has demonstrated 95% alignment with lecturer assessments in UK pilots, including work with Bath Spa University. Additional pilot partners include De Haagse Hogeschool, Tilburg University, Hogeschool Rotterdam, and UMCG. Detailed case study data is available on request. Eduface is also a Jisc/CHEST approved supplier.
Summary evaluation table
Question
What a good answer looks like
Red flags
LMS integration
LTI 1.3, automatic grade passback, no separate student login
Custom plugins, CSV exports, manual grade entry
Data residency
EEA servers, no third-party AI APIs, DPA available to sign
Vague security language, US-based processing, no DPA
Lecturer workflow
Quick review interface, high AI accuracy, familiar environment
Separate platform to learn, frequent heavy editing required
Human oversight
Grades held as drafts, explicit lecturer approval required
Auto-release unless lecturer actively intervenes
Evidence base
Pilot alignment data, named institutions, framework approval
Accuracy claims without data, no named pilots
Frequently asked questions
Is LTI 1.3 a requirement or just a preference?
For most institutions, it is effectively a requirement. LTI 1.3 provides the authenticated, secure connection your LMS security team will expect. Older versions introduce vulnerabilities that IT departments are increasingly unwilling to accept. If a vendor cannot support LTI 1.3, you should expect that integration to become a maintenance problem within one to two LMS update cycles.
How do I find out whether a vendor uses third-party AI APIs?
Ask them directly, in writing, and request it as a contractual disclosure. Many vendors are not transparent about this by default. If the vendor references AI providers like OpenAI or Google in their technical documentation, assume your institution’s data will pass through those systems unless the vendor explicitly confirms otherwise.
What counts as meaningful human oversight under the EU AI Act?
Article 14 of the EU AI Act specifies that human oversight must be effective: the human must be able to understand the AI’s output, detect failures, and intervene before the output has consequences. A rubber-stamp process does not meet that standard. For AI assessment, the minimum threshold is that a lecturer reviews each AI-generated grade before it is communicated to the student, with a genuine ability to modify or reject it.
Do I need to assess every vendor for GDPR compliance independently?
Yes. GDPR compliance is not transferable between vendors, and your institution retains responsibility as the data controller. Jisc/CHEST framework membership provides some vetting, but you still need a signed DPA with the vendor before going live, and you should review their data processing documentation before procurement sign-off.
What should I ask about the pilot evidence a vendor provides?
At minimum: which institutions ran the pilot, what disciplines and assessment types were included, what the sample size was, and how alignment was measured. Ask whether the pilot data has been reviewed by anyone independent of the vendor. Named institutions you can contact directly are significantly more credible than anonymised case studies.
Conclusion
The questions above will not make the procurement process shorter. They will make the tool you end up with far less likely to cause problems six months after go-live. If a vendor cannot answer these questions clearly, that is information worth having before you commit. Eduface was designed to answer all five of them.
Put Eduface through these five questions
Request a demo and we will walk through LTI 1.3, data residency, the reviewer interface, oversight, and the pilot data.