Fine-Tuning LLM for OBE Question Answering

Haw, Cong Ting (2026) Fine-Tuning LLM for OBE Question Answering. Final Year Project (Bachelor), Tunku Abdul Rahman University of Management and Technology.

Text Haw Cong Ting_Full Text.pdf Restricted to Registered users only Download (1MB)

Abstract

Outcome-based education requires that advice given to engineering lecturers be traceable to institutional sources and aligned with official accreditation policies. Existing educational chatbots often lack this level of domain grounding and do not consistently enforce the ethical and procedural constraints required by the Board of Engineers Malaysia. This thesis develops an auditable artificial intelligence advisory system by integrating parameter-efficient fine-tuning with aRetrieval Augmented Generation pipeline. To reduce reliance on synthetic data, a specialized Command Line Interface tool was used to extract question-answer pairs directly from three authoritative Malaysian accreditation manuals. The resulting dataset was reviews, curated and used to fine-tune five distinct reasoning architectures ranging from 8 billion to 32 billion parameters. Training was conducted on a single A100 GPU in a Google Colab environment using four-bit quantization and the Unsloth framework to improve hardware efficiency. Model performance was evaluated using an artificial intelligence judge across Ideal, Common, and Ethical Corner Case scenarios. The results indicate that the explicit reasoning architecture of the DeepSeek R1 Distill Llama 8B model achieved the highest overall compliance score of 85%. The explicit reasoning traces produced by the model appear to improve robustness under low-precision quantization. One possible explanation is that step-by-step reasoning distributes intermediate computations across multiple tokens, which may reduce the impact of numerical precision loss during inference. However, this effect is observational and was not isolated through controlled ablation, and therefore requires further investigation. Overall, the findings suggest that a localized and privacy-preserving artificial intelligence advisor can deliver accurate and traceable accreditation guidance, while achieving competitive performance compared to larger implicit architectures