Problem Statement: The integration of Generative AI in education has created a "verification gap." High-quality written submissions and coding projects often fail to prove that a student has truly mastered the underlying theoretical concepts. While traditional oral exams are the most robust way to verify individual knowledge and critical thinking, they are difficult to implement at scale due to faculty time constraints.
Project Goal: Inspired by recent pedagogical research (e.g., at NYU), this project aims to develop a scalable, automated "AI Examiner" prototype. The system will conduct voice-based interviews to probe a student's understanding of the theoretical course content (concepts, models, and methodologies taught during the semester).
Core Functionality: The system should be designed as a modular framework. The student is free to choose the implementation strategy based on their available resources:
- Option A (Local): Use open-source models (Ollama, Whisper) for a free, privacy-first approach. Note: This requires a laptop with capable hardware (e.g., Apple Silicon or NVIDIA GPU) to ensure low latency.
- Option B (Cloud): Use external APIs (OpenAI, Anthropic, ElevenLabs) for maximum performance. Note: If this path is chosen, the student is responsible for possible API usage costs (est. €20-€50).
The prototype must include:
- Contextual Awareness: Ingest course materials (lecture slides/textbooks) to build a relevant RAG (Retrieval-Augmented Generation) pipeline.
- Voice-First Interaction: Facilitate a real-time dialogue using Speech-to-Text and Text-to-Speech technologies.
- Adaptive Questioning: Use LLMs to "drill down" into specific topics if a student's initial answer is vague, mimicking a human examiner.
- Automated Evaluation: Generate a grading report evaluating correctness, clarity, and depth of knowledge (potentially utilizing multi-model consensus).
Research Focus:
The research focus lies on an Effectiveness Analysis. The student must compare the AI-led oral exam results against traditional written assessment scores for the same topics to identify if the voice-based approach better detects deep conceptual knowledge vs. surface-level memorization.
Technologies: LLM Orchestration (e.g., LangChain, AutoGen); LLMs (flexible: local via Ollama/Llama.cpp or via APIs like GPT/Gemini/Claude); STT/TTS (e.g., Whisper, Web Speech API, or ElevenLabs); Backend: Python (FastAPI)
Tags: Information System; Natural Language Processing; EdTech; Voice Interaction; Knowledge Verification; Automated Assessment
