exam-question-pipeline

Exam Question Pipeline

Use this skill to turn course materials into app-ready Exam-Lab question JSON without committing raw materials or confusing AI-generated questions with real past-exam questions.

Workflow

1. Create local-only workspace

   • Put raw PDFs, transcripts, audio, converted Markdown, and intermediate JSON under .local/exam-materials/{course-id}/.
   • Ensure .local/, raw material folders, and conversion output folders are in .gitignore.

2. Classify sources before generation

   • previous_exam: actual prior exam material. Use only for the matching exam scope, or as separated pattern/context evidence.
   • lecture_material: PPT/PDF/handout content used for formula and answer-key grounding.
   • transcript: lecture recording transcript. Use for scope, emphasis, and professor intent; do not use transcript text alone as formula authority.

3. Convert materials to Markdown

   • Use MarkItDown for fast readable slide text.
   • Use Docling when tables, images, layout, or extracted assets matter.
   • Treat OCR-heavy distribution tables as reference-only unless checked against official sources or source images.
   • Read references/material-conversion-policy.md when choosing tools or checking conversion quality.

4. Extract exam scope and intent

   • Record exact user clarifications about scope.
   • Separate midterm/final and year-specific evidence early.
   • If a transcript exists, extract only the relevant speaker lane and correct obvious domain-term misrecognitions before using it for intent.

5. Generate draft questions

   • Cover the full confirmed exam range.
   • Weight emphasized lectures or problem sets more heavily, but do not drop other in-scope materials.
   • Give needed critical values directly in calculation prompts unless the learning goal is table/distribution selection.

6. Normalize for Exam-Lab

   • Add source, answerStatus, exam, and metadata fields according to references/exam-lab-schema.md.
   • Use source: "ai" and answerStatus: "ai_draft" for AI-generated expected questions.
   • Use exam.kind: "past_exam" only for real prior exam questions.
   • Run scripts/normalize-exam-questions.mjs when converting a draft JSON into app-ready JSON.

7. Review and validate

   • Use references/question-generation-review.md for content-review checks.
   • Run scripts/validate-question-bank.mjs <json-file>.
   • Run app verification after upload:
     • npm test if tests exist.
     • npm run build.

8. Upload to app

   • Write the final JSON to src/lib/questions/{courseId}.json.
   • Import it from src/lib/questions/index.js.
   • If the UI lacks exam-session filtering, add filtering by question.exam.id instead of overloading source.

Output Rules

• Commit app-ready question JSON and process reports only.
• Do not commit raw PDFs, recordings, transcripts, conversion artifacts, or intermediate generation dumps.
• Always report remaining review risk for AI-generated answer keys.
• Keep real past exams and AI expected questions separated by both file location and metadata.

Resources

• references/exam-lab-schema.md: app JSON schema and exam/source conventions.
• references/material-conversion-policy.md: PDF/STT conversion and source handling rules.
• references/question-generation-review.md: generation and review checklist.
• scripts/validate-question-bank.mjs: deterministic JSON validation.
• scripts/normalize-exam-questions.mjs: inject/normalize Exam-Lab metadata.