PK14 - AI-Assisted Automated Grading for Open-Ended Programming Assignments in Moodle

Supervisor: Kalchgruber

Problem Statement: Modern e-learning platforms like Moodle excel at automated Multiple Choice assessments but struggle with open-ended tasks, such as code logic explanations or architectural decisions. While existing plugins (like VPL) can check if code compiles, they cannot assess the quality, style, or conceptual understanding. Manual grading of these qualitative aspects is time-consuming and inconsistent.

Project Goal: The goal is to develop an AI-driven "Grading Assistant" that integrates with Moodle. The system will act as a "Second Reader," automatically fetching student submissions, evaluating them against a rubric using LLMs, and generating a transparent grading report for the lecturer to review.

Core Tasks:

  1. Moodle Integration: Use the Moodle REST API to programmatically retrieve student submissions (text or code files) and eventually upload the feedback.
  2. Data Privacy & Anonymization: Implement a pre-processing step to anonymize data (stripping PII/Student IDs) before sending content to an external LLM provider, ensuring GDPR compliance.
  3. Qualitative Analysis Pipeline: Develop a prompt engineering strategy to evaluate the submission based on a rubric (e.g., "Is the variable naming consistent?", "Does the comment explain the *why*?", "Is the recursion base case valid?").
  4. Reasoning & Justification: The system must output a structured "Defense of Grade," itemizing exactly why points were deducted. This prevents "Black Box" grading.
  5. Lecturer Dashboard (Human-in-the-Loop): Create a simple frontend where the lecturer sees the AI-suggested grade alongside the student work and can "Accept" or "Override" it with one click.

Research Focus: 
The research focus lies on an Agreement Analysis. The student must compare the AI-generated grades with those given by a human lecturer for the same set of submissions, and calculate the Inter-Rater Reliability (e.g., Cohen's Kappa or Pearson Correlation) to determine whether the AI is consistent and fair.

Prerequisites & Resources:

  • Experience with Python (FastAPI/LangChain) and basic knowledge of REST APIs.

Technologies: Moodle REST API; Python (LangChain, Pydantic); LLMs (GPT-4o / Claude 3.5); Streamlit or React (for the Dashboard)

Tags: EdTech, Moodle, Information System, Automated Assessment, Software Quality, Natural Language Processing, GDPR

Back