Session A · online, live & self-paced

Integrating AI into Research & Pedagogy

An outcome-based, problem-driven workshop for researchers, lecturers and postgraduates — built so you can also work through it alone.

Session A

Ask well, find truly — how the machine works, prompting as a discipline, and grounded evidence discovery.

Session B

Write true, teach well — argument, data, outcome-based teaching materials, and disclosure that survives review.

You are scored

Every interaction you complete is recorded in an evidence ledger. You will see exactly what is assessed and why — and read your own scorecard at the end.

Ts. Dr. Mohd Nor Akmal Khalid · CAIT, FTSM, Universiti Kebangsaan Malaysia

Start here

How to use this deck

It works three ways: a live session, a self-paced module, and a reference you keep.

Move around

← → or swipe to change slides. O opens the slide map. N shows facilitator notes. T opens the sprint timer.

Switch language & theme

Top-left: EN / BM and Light / Dark. Keyboard: B for language, D for theme. Everything translates, including outputs.

Do, do not watch

Anything with a Copy button is meant to be pasted into a real tool. Anything with Show expected output tells you what good looks like before you run it.

Watch your ledger

The gold counter in the bottom bar is live. Click it (or press S) to see every scored item, what it is worth, and why it is assessed.

Self-paced route Alone? Set the timer to 12 minutes per module, finish every lab before advancing, and stop at each checkpoint until you score above 65%. That mirrors the live pacing almost exactly.

Session A · what this session covers

Three blocks, and what each one gives you

Every row is a section you can jump to with O. Your facilitator will set the pace.

Block	What we cover	You leave with
The Academic Prompter	How the machine actually works, why it invents, then R.T.F. and the three blocks R.T.F. leaves out	A refined prompt for your own field
Modul 1 — AI for teaching & content development	Constructive alignment, HOTS verbs, then build a lesson plan, a formative quiz set and a slide outline	Lesson plan · quiz set · slide outline · rubric
Modul 2 (1) — research & proposal writing	Frame the question, build the search, screen it, extract into a matrix, verify every citation	Literature matrix · draft problem statement

If you are here for one thing Teachers: Modul 1 is yours — its three labs produce material you can use next semester. Researchers: Modul 2 (1) feeds directly into Session B, where your matrix becomes a problem statement and then a proposal outline. Everyone needs The Academic Prompter first, because everything after it assumes you know why the machine invents references.

Why this workshop is built this way

Five theories, one design

Click each principle. The design is not decoration — it determines what you will be able to do Session B.

Andragogy

Problem-based

Outcome-based

Connectivism

Formative loop

Fun fact A 225-study meta-analysis of undergraduate STEM found that failure rates under traditional lecturing were about 1.5× those under active learning. The format you are in right now is not a stylistic preference.

Freeman S. et al. (2014). Active learning increases student performance in science, engineering, and mathematics. PNAS, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111

Biggs J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32, 347–364. https://doi.org/10.1007/BF00138871

Barrows H.S. (1986). A taxonomy of problem-based learning methods. Medical Education, 20(6), 481–486. https://doi.org/10.1111/j.1365-2923.1986.tb01386.x

Course hub · outcomes

What you will be able to do

Five course learning outcomes. Each one is written as something an observer could watch you do.

CLO	Outcome — by the end you can…	Bloom level	Evidence we collect
CLO1	Explain how a language model produces text, and predict where it will fail in an academic task.	Understand	Checkpoint A1, failure-mode cards
CLO2	Construct a structured, reusable prompt and iterate it to a usable output.	Apply	Prompt labs, iteration lab
CLO3	Design a documented, reproducible search strategy and verify every citation it returns.	Analyse	Search-string lab, verification drill
CLO4	Produce an outcome-aligned teaching or writing artefact using AI without surrendering authorship.	Create	Framework, rubric, draft labs
CLO5	Judge an AI-assisted workflow against integrity, privacy and disclosure standards.	Evaluate	Privacy sorter, disclosure builder, Checkpoint B2

Read this backwards If you only have time for one thing today, pick the CLO that matches the problem on your desk and do just those labs. The deck is designed to survive being taken out of order.

Course hub · assessment

What is assessed, and why

Every scored item on this deck, its weight, and the reason it exists. Nothing here is a mark on a transcript.

Why assess at all in a workshop? Because retrieval is what makes learning stick. Being asked to produce an answer — even a wrong one — strengthens memory far more than re-reading the same slide. The scoring here is a delivery mechanism for retrieval practice, not a gate.

Course hub · scoring

How the evidence ledger works

Five kinds of evidence, four bands, immediate feedback at every step.

Kind	What it measures	Feedback timing
quiz	Recall and discrimination under time pressure.	Instant, per option
lab	Whether you actually built the artefact, not just read about it.	On completion
engage	Depth of exploration — how much of a concept you opened up.	Progressive
judge	Calibrated judgement: can you tell a good source from a plausible one?	On submission
deliver	Committed, dated actions you will carry out of the room.	On completion

The four bands

…

The one rule Points are never awarded for agreeing with the deck. Several checkpoint items have a defensible second answer, and the feedback will say so. You are being scored on discrimination, not compliance.

Black P. & Wiliam D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7–74. https://doi.org/10.1080/0969595980050102

Hattie J. & Timperley H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

Roediger H.L. & Karpicke J.D. (2006). Test-enhanced learning. Psychological Science, 17(3), 249–255. https://doi.org/10.1111/j.1467-9280.2006.01693.x

Rate yourself

Where are you actually starting from?

Five outcomes, one honest answer each. Nobody else sees this — it stays in your browser. At the end of the session your scorecard puts these answers next to what you actually built.

CLO1I can explain how a language model produces text, and predict where it will fail

CLO2I can write a prompt that specifies task, context, constraints and verification

CLO3I can design a search strategy and verify every citation at its source

CLO4I can produce teaching or research material with AI without surrendering authorship

CLO5I can judge a workflow against integrity, privacy and disclosure requirements

Answer fast, and answer honestly There is no advantage to rating yourself high. The comparison at the end is between this answer and what you build — so an inflated rating only makes the gap look larger. Most people in most rooms overrate CLO3 and underrate CLO1, and finding out which way you lean is the point.

Session A · block 1 of 3

The Academic Prompter

Before any tool, the mechanism. You cannot write a good prompt for a machine whose failure modes you cannot predict.

What we cover

How a language model produces text, why it invents references, the grounded/ungrounded split, and R.T.F. with the three blocks it leaves out.

You will build

A prompt for your own field at three levels of construction, then a self-critique and rewrite protocol you can reuse on anything.

Why it comes first

Every later block assumes you know why a fluent answer can be entirely wrong. Skip this and the rest becomes tool training instead of judgement.

CLO1 · CLO2

Problem trigger

Twelve references, three of which do not exist

A real pattern, reconstructed. Work out what went wrong before we tell you.

The scenario

A postgraduate drafts a narrative review with a general-purpose chatbot. The prose is excellent. Twelve references are supplied, each with authors, journal, year, volume and a DOI. The supervisor spot-checks four. Two resolve. One resolves to a different paper with the same first author. One DOI returns nothing at all. The review is two days from submission.

Hold these three questions all morning

1. Why did this happen?

Not “the AI lied”. What mechanism produces a DOI-shaped string that is not a DOI?

2. Where should it have been caught?

There are at least four points in the workflow where a five-second check would have stopped it.

3. Who is accountable?

Under every major publishing standard, the answer is not ambiguous. We will return to this in Session B.

The numbers behind the story Across 84 AI-generated literature reviews containing 636 citations, 55% of GPT-3.5 references were fabricated outright; with GPT-4 that fell to 18%. Among references that were real, 24% still carried substantive errors. Better models reduced the problem. They did not remove it.

Walters W.H. & Wilder E.I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13, 14045. https://doi.org/10.1038/s41598-023-41032-5

Bhattacharyya M. et al. (2023). High rates of fabricated and inaccurate references in ChatGPT-generated medical content. Cureus, 15(5), e39238. https://doi.org/10.7759/cureus.39238

The Academic Prompter

What is actually happening when it answers

Five steps, one loop, repeated for every token. Click through all five.

Tokenise

Embed

Attend

Predict

Sample

New since 2025

Three changes matter for research work. Reasoning models spend extra computation generating hidden intermediate steps before answering, which sharply improves multi-step analysis but does not fix fabrication. Long context now routinely spans hundreds of thousands of tokens, so you can paste whole papers instead of summaries. Mixture-of-experts routing means a large model activates only part of itself per token, which is why capability and cost stopped rising together.

Vaswani A. et al. (2017). Attention is all you need. NeurIPS. arXiv:1706.03762 https://doi.org/10.48550/arXiv.1706.03762

Wei J. et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. NeurIPS. arXiv:2201.11903 https://doi.org/10.48550/arXiv.2201.11903

The Academic Prompter · grounding

Grounded or not grounded

If you remember one diagram from today, make it this one.

The two-second test Before trusting any AI answer, ask: could this tool have retrieved a document, or did it only generate text? If it never retrieved, no citation it gives you is evidence — it is a hypothesis about what a citation might look like.

New since 2025

Agentic deep research is the current frontier: instead of one retrieval pass, the system plans a search, reads results, notices what it missed, reformulates, and iterates for several minutes before writing. Tools built this way — Undermind, and the deep-research modes in the major assistants — behave much more like a diligent research assistant than a chatbot. They are still not a systematic review, and the coverage claim is theirs, not yours to repeat.

Lewis P. et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. NeurIPS. arXiv:2005.11401 https://doi.org/10.48550/arXiv.2005.11401

An instruction that makes the model mark its own uncertainty: “flag anything I must confirm against a primary source with [VERIFY]”, or “if you cannot do this from the material I gave you, say so instead of writing it”.

What it converts the tool into

Giving a model an explicit way to refuse turns it from a text generator into a check on your own thinking. Every lab in these two days uses this, and it is the single highest-leverage habit in the workshop.

R.T.F. today

Role and Constraints increasingly live outside the chat — in system instructions, saved projects and custom assistants. Write them once, and every conversation inherits them. Task, Context and Format stay in the message. If you find yourself retyping your role paragraph daily, move it into a project.

The Academic Prompter · R.T.F. applied

Six blocks. The first is compulsory.

Same task, three levels of construction. Copy each, run it, then read the annotations.

Lab A2.1 · from vague to reusableDo this nowTask: get a defensible summary of a mechanism you are about to write about.

What most people type. Note what the model has to guess.

Explain how metformin works.

7 words · 5 unstated assumptions

Expected output

What you get: a competent encyclopaedia paragraph pitched at an undefined reader, with no sources, no scope limit, and no indication of what is settled versus contested. Usable for orientation. Unusable in a manuscript.

Audience: guessed
Length: guessed
Evidence standard: none
Reusability: zero — you will retype this next time

Task

“Explain how metformin works” — present, and the only block that is. Everything else is left to the model.

This is the baseline you are measured against. Every block you add below is measurable improvement, not decoration.

Persona, task, context, format. Four blocks. This is the level most academics should live at.

ROLE: You are a clinical pharmacologist writing for a pharmacy research audience.

TASK: Explain the principal mechanism of action of metformin in type 2 diabetes.

CONTEXT: This will inform the introduction of a narrative review. My readers are postgraduates who know basic pharmacology but not this drug class in depth.

FORMAT: 180-220 words, third person, no bullet points. Separate clearly what is well established from what is still debated.

CONSTRAINT: Do not invent citations. Where a claim rests on specific evidence, say "[verify: ]" instead of naming a paper.

4 blocks · verification hook built in

Expected output

What you get: a paragraph at the right register and length, with the settled/contested line drawn explicitly, and inline [verify: ...] markers wherever the model would otherwise have been tempted to name a source.

Directly usable as a first draft you then evidence yourself
The verify markers become your search list for the next module
Re-runnable: change one line and it works for a different drug

Typical opening: “Metformin’s glucose-lowering effect is principally hepatic… [verify: magnitude of AMPK-independent contribution]”

Role

Sets vocabulary and depth. The single highest-yield line: it changes register, not just tone.

Task

One verb, one object. If you need two verbs, you need two prompts.

Context

Where the output will live and who reads it. This is what stops generic encyclopaedia prose.

Format

Length, person, structure. Also the settled-versus-debated split — a scholarly move most people forget to ask for.

Constraint

The anti-fabrication clause. You are converting an invisible risk into a visible to-do list.

The constraint block is the one that separates safe from unsafe academic use. Adopt it permanently.

Adds exemplars and self-critique. Use when the output goes somewhere that matters.

ROLE: Clinical pharmacologist writing for a pharmacy research audience.

TASK: Explain the principal mechanism of action of metformin in type 2 diabetes.

CONTEXT: Introduction of a narrative review; readers are postgraduates with basic pharmacology.

EXEMPLAR: Match the register of this sentence — "Hepatic gluconeogenesis is suppressed through mechanisms that remain incompletely resolved, with AMPK activation accounting for only part of the observed effect."

FORMAT: 180-220 words, third person, prose only. Explicitly separate established from contested.

CONSTRAINT: Never name a citation. Use "[verify: ]" markers instead.

SELF-CHECK: After writing, list (a) the three claims most likely to be wrong, (b) one important consideration you omitted, and (c) the question a hostile reviewer would ask first.

6 blocks · produces its own review notes

Expected output

What you get: the level-2 paragraph, plus a self-critique block that is often more valuable than the prose. Typical self-check output:

(a) Most likely wrong: the relative weighting of AMPK-dependent versus independent pathways; the claim about gut-mediated effects; any implied magnitude of HbA1c reduction
(b) Omitted: inter-individual variation in transporter genotype
(c) Reviewer’s first question: “Which of these mechanisms is demonstrated in humans rather than rodent models?”

That third answer is the one to act on. It is a free adversarial review before anyone else sees your draft.

Exemplar

One sentence in the voice you want beats three paragraphs describing that voice. Show, do not specify.

Self-check

Forces the model to argue against its own output. The reviewer question is the highest-value line in the entire prompt.

Reusability

Save this as a custom assistant. Swap the drug, keep the scaffold. You have built an instrument, not asked a question.

Level 3 is not always necessary. Use it when the output will be read by someone who can reject it.

New since 2025

Two shifts. System instructions and custom assistants mean the role and constraint blocks now live permanently outside the chat — write them once, and every conversation inherits them. Reasoning models respond differently to prompting: they do worse with “think step by step” (they already do) and better with a crisp statement of the success criteria. Tell a reasoning model what a good answer must satisfy, not how to think.

When you do not yet know how to ask. Let the model interrogate you first — especially useful for framing a research question.

I want to [describe your goal in one sentence].

Before you attempt it, ask me the five questions whose answers would most change how you approach this. Ask them one at a time and wait for my answer.

When you have all five, write the prompt I should have given you, and explain what each part does.

Watch for The questions it asks reflect common cases, not your case. Answer them honestly rather than conveniently, or you will get a prompt for someone else’s project.

The Academic Prompter · iteration

The first answer is a draft, not a verdict

Three moves, in order. Most people stop after zero.

DraftFirst output

→

CritiqueMake it argue against itself

→

ConstrainFix what broke, in writing

→

VerifyCheck outside the tool

Lab A2.2 · the three follow-ups that fix almost everythingDo this nowPaste these one after another into the same conversation as your draft.

# 1 — SELF-CRITIQUE
Critique what you just wrote against three tests:
(a) Is every factual claim one I could verify in a primary source?
(b) Which sentence would a specialist reader object to first, and why?
(c) What did you assume about my study that I never told you?

# 2 — CONSTRAINED REWRITE
Rewrite it, fixing everything you found in (a) and (b).
Rules: keep it under 200 words; delete any sentence you cannot defend;
mark every remaining evidence-dependent claim as [verify: ...].

# 3 — TRANSPARENCY PASS
List, as a table: each claim you kept, how confident you are (high/medium/low),
and what kind of source would settle it. Do not name sources.

3 turns · ~4 minutes

Expected output

Turn 1 typically surfaces two or three real problems and one invented one — you will recognise the invented one immediately, which is itself diagnostic.

Turn 2 shortens the text by 20–30% and the deletions are usually the sentences you were least sure about. Length reduction is a reliable proxy for unsupported claims removed.

Turn 3 gives you the table that becomes your search plan for the next module:

Claim	Confidence	Source type that settles it
Hepatic gluconeogenesis is the principal site of action	High	Any current pharmacology reference text
AMPK-independent pathways contribute substantially	Medium	Mechanistic study in human tissue, post-2015
Gut-mediated effects explain a large share	Low	Systematic review or a head-to-head human trial

Turn 1

Question (c) is the sleeper. Models routinely assume a design you never described, and that assumption then contaminates everything downstream.

Turn 2

“Delete any sentence you cannot defend” is the whole technique in one clause. Give the model permission to be shorter.

Turn 3

Converts prose into a verification workload. You now know exactly what to search for — and the low-confidence row is where your literature review actually begins.

Notice what this does: it converts an AI writing task into a human verification task with a defined scope. That is the whole methodology of this workshop in miniature.

Checkpoint A1 · immediate feedback

Four questions. Answer before you read on.

Every option explains itself the moment you click. Wrong answers are worth more than right ones here.

Q1A model gives you a reference with a correctly formatted DOI. What has it actually done?

Looked the reference up in a bibliographic database

Only if the tool retrieves. A pure language model has no database to look anything up in.

Generated the most probable DOI-shaped sequence of characters

Correct. The DOI is produced by the same next-token process as the prose around it. Format is not evidence.

Copied it verbatim from its training data

Sometimes, by coincidence — but it cannot tell you which case you are in, which makes it useless as a guarantee.

Assessed because this single misconception causes nearly every citation incident. CLO1.

Q2Which of these makes an AI answer grounded?

It cites sources

No — an ungrounded model cites sources too. That is exactly the trap.

It retrieved real documents before writing, and wrote only from them

Correct. Grounding is about the order of operations: retrieval precedes generation.

It sounds cautious and hedged

Assessed because tool selection for the rest of the day depends on it. CLO1, CLO3.

Q3You ask “why does intervention X improve outcome Y?” and get a fluent explanation. What is the main risk?

The explanation may be too technical

A formatting issue, easily fixed, and not what will get you into trouble.

Your question presupposed the effect, and the model complied

Correct — sycophancy. The neutral form is “what is the evidence for and against X improving Y?”

The model has no opinion on causation

It will readily produce causal language whether or not the evidence supports it. That is the problem, not the safeguard.

Assessed because question framing is the earliest and cheapest point of control. CLO1, CLO2.

Q4Which block improves an academic prompt most, per word spent?

A polite opening

Costs tokens, changes nothing measurable. Politeness is for people.

Role plus constraint

Correct. Role sets register and depth; constraint blocks the failure mode. Together they are perhaps twenty words.

A longer, more detailed description of what you want

Defensible — detail does help. But one exemplar sentence usually beats a paragraph describing that sentence.

Assessed because CLO2 is about construction efficiency, not prompt length. Note option 3 is defensible — feedback says so.

Session A · block 2 of 3

Modul 1 — AI for teaching & content development

The prompting block gave you a method. This block points it at your own syllabus and produces material you can use next semester.

What we cover

Constructive alignment, the Bloom verb ladder and which levels a capable model now completes perfectly, then assessment redesign.

You will build

An aligned lesson plan, a four-level analytic rubric, a formative quiz set that diagnoses, and a ready-to-export slide outline.

If you do not teach

Stay anyway. The rubric lab is the clearest demonstration in the workshop of how a constraint list changes an output, and supervisors use the same technique on thesis feedback.

CLO4 · CLO5

Modul 1 · alignment

Three things that must point the same way

Biggs called it constructive alignment. Most course outlines fail it in exactly one place, and it is almost always the same place.

OutcomeWhat they can DO after

→

ActivityPractising that exact thing

→

AssessmentEvidence of that thing

The usual break

Outcome says evaluate. Activity is a lecture. Assessment is a multiple-choice test of recall. Two of the three point at remembering; the outcome points at evaluating; and students correctly optimise for the assessment.

The diagnostic

Read your outcome verb, then look at your assessment. If a student could score full marks without ever performing that verb, the outcome is decoration. This test takes thirty seconds and fails most course outlines.

This deck, audited

CLO3 says design a search and verify citations. The activity was Lab A3.2, where you built a search string. The assessment was the verification drill, where you judged six real citations. Same verb, three times.

I can name the verb in each of my course outcomes
For each outcome, my students practise that verb at least once before being assessed on it
No student could pass my assessment without performing the outcome verb

Biggs J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32, 347–364. https://doi.org/10.1007/BF00138871

Krathwohl D.R. (2002). A revision of Bloom’s taxonomy: an overview. Theory Into Practice, 41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2

Modul 1

The verb is the outcome

KBAT/HOTS is not a topic you add. It is a verb you choose, and then an assessment that forces the learner to perform it.

Bloom level	Verbs that belong here	What the assessment must let them do	Robust to AI?
Remember	define, list, name, recall	Reproduce	No
Understand	explain, classify, summarise	Restate in their own terms	No
Apply	calculate, administer, implement	Work a case they have not seen	Partly
Analyse	differentiate, attribute, deconstruct	Take apart something real and messy	Yes
Evaluate	critique, justify, defend, judge	Choose between defensible options and say why	Yes
Create	design, formulate, construct	Produce something and defend the choices in it	Yes

I can name the verb in every outcome on my course outline
At least half of my outcomes sit at Analyse or above
For my highest outcome, the assessment actually requires that verb to be performed

Why the last column matters this year The two rows marked No are the ones a capable model completes perfectly in seconds. That does not make them worthless — recall is a prerequisite for everything above it — but it does mean they can no longer carry your summative assessment. Moving one outcome from Understand to Evaluate does more for integrity than any detection tool.

Krathwohl D.R. (2002). A revision of Bloom’s taxonomy: an overview. Theory Into Practice, 41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2

Modul 1 · lesson & rubric lab

Build one aligned session, then its rubric

Use a topic you actually teach. The second tab depends on the first.

Lab B3.1 · alignment & rubricDo this now

ROLE: Curriculum designer trained in constructive alignment (Biggs).

TASK: Build a 90-minute session plan for the topic below.

RETURN A TABLE with columns: Minutes | Outcome served | What learners DO | Evidence produced

RULES:
- Every row must serve a stated outcome. If a row serves none, delete it and say why.
- Learners must perform the outcome verb before being assessed on it
- No block of teacher talk longer than 12 minutes
- Include one formative check with immediate feedback, and state what a wrong answer tells me
- Assume learners can access AI tools during the session; design for that rather than around it

THEN, separately: name the weakest alignment in your own plan and explain the compromise.

TOPIC: [ ]
LEARNERS: [ ] (level, discipline, class size)
OUTCOMES (verbs matter): [ ]

Self-critique clause at the end

Expected output

A row from a good output:

| 25–45 | CLO2: construct a search strategy | In pairs, build a Boolean string for the assigned question, then swap and try to break each other’s | The two strings, plus each pair’s note on what broke |

Notice that the evidence column is not “participation”. It names a thing that exists afterwards and can be looked at.

And the self-critique:
“Weakest alignment: the 10-minute closing discussion serves no stated outcome. I kept it because closure aids retention, but it produces no evidence and should not be marked.”

That is a defensible compromise, stated openly. A plan with no compromises is usually a plan that has not been examined.

Evidence column

Forcing an evidence column exposes activities that feel good and produce nothing. It is the fastest audit of a lesson plan there is.

12-minute rule

An arbitrary constraint that reliably improves output. Without it you get a lecture with two questions appended.

“Design for AI access”

Changes the output completely. Assume the tool is in the room and the plan starts assessing judgement instead of production.

An outcome without evidence is an intention.

Paste the plan from Part 1 so the rubric marks what the session actually produced.

ROLE: Assessment designer.

TASK: Build a 4-level analytic rubric for the evidence produced by the plan below.

REQUIREMENTS:
- Criteria must map one-to-one onto the stated outcomes. No criterion without an outcome.
- Level descriptors must be OBSERVABLE. Ban the words: excellent, good, poor, adequate, some, appropriate
- Each descriptor states what the work DOES, so two markers reading it would agree
- Include one criterion on the quality of reasoning about AI use, marked on the reflection not on abstention
- Write the feedback sentence a student at each level should receive

THEN: identify which criterion is most likely to produce marker disagreement, and tighten it.

PLAN:
[paste Part 1 output]

Banned words do the heavy lifting

Expected output

Compare two descriptors for the same level:

Before: “Shows good understanding of search strategy.” — two markers will disagree, and neither can explain the mark to the student.

After: “The Boolean string contains at least two synonyms per concept block, uses controlled vocabulary where it exists, and the accompanying note identifies one limit the author chose not to apply and states why.” — checkable, and the student knows what to do differently.

The AI criterion, at the top level:
“The reflection names a specific task delegated, states what was verified and how, and identifies one output that was rejected with the reason.”

Note what this rewards: discrimination, not abstention. A student who used nothing and says so thoughtfully can also score here; a student who used everything and cannot say what they checked cannot.

Banned adjectives

“Good” and “adequate” are where marker disagreement lives. Removing them forces observable descriptors, which is the whole point of an analytic rubric.

Reflection, not abstention

Marking abstention rewards concealment. Marking reflection rewards judgement — and judgement is what you actually wanted to teach.

Feedback sentences

A rubric that generates the feedback sentence saves hours of marking and makes the feedback consistent across a teaching team.

A rubric two markers agree on is worth more than a beautiful one they do not.

Modul 1 · content development

Turn one syllabus topic into teaching material

Two deliverables, two tabs. Bring a real topic and paste your actual syllabus wording — described material always underperforms pasted material.

Lab A1.1 · formative quiz setDeliverable × 2

A formative quiz is not a test. Its job is to tell you what to reteach.

ROLE: Assessment designer in health sciences, trained in Bloom's revised taxonomy.

TASK: Write a 5-item formative quiz for the syllabus topic below.

FORMAT: For each item — stem, 3 options, the correct one marked, and a
feedback line FOR EVERY OPTION explaining why it is right or wrong.

RULES:
- At least 3 items must sit at Analyse or above. State the level of each item.
- Every distractor must be a real misconception a student in this field holds.
  No filler options, no obviously absurd choices.
- For each item, tell me what a wrong answer DIAGNOSES — what I should reteach.
- Write one item that a capable AI would answer correctly but a student who
  had not understood would still get wrong, and explain what makes it work.

CONSTRAINTS: No citations. Nothing beyond the syllabus text I supply.
If a topic cannot support an Analyse-level item, say so instead of forcing one.

SYLLABUS TOPIC (pasted verbatim):
[ ]
LEARNERS: [ level, discipline, class size ]

The diagnosis line is what makes it formative

Expected output

One item, in the shape you want:

Item 3 — Analyse. A patient on warfarin is started on a new antibiotic and the INR rises sharply within four days. Which single piece of information would most change your interpretation?
  a. The patient’s age — plausible but weak: age modifies risk, not mechanism. A student choosing this is reasoning about outcome, not cause.
  b. Whether the antibiotic inhibits CYP2C9 — correct: it separates a pharmacokinetic interaction from the alternative explanations below.
  c. Whether the patient’s diet changed — a real competing explanation, not a filler. Choosing this shows the student knows vitamin K matters but has not weighted the timing.

Wrong answer diagnoses: (a) reasoning from patient factors rather than mechanism — reteach the interaction pathway. (c) correct mechanism, wrong weighting of evidence — reteach using timing as a discriminator, not the pathway itself.

Notice what the diagnosis line gives you. Two students both score zero on this item and need completely different teaching. Without that line the quiz tells you a number; with it, the quiz tells you what to do on Monday.

“Real misconception”

This one instruction transforms the output. Without it you get one right answer and two obviously wrong ones, which measures nothing.

“What a wrong answer diagnoses”

The line that separates formative from summative. It converts a score into an instruction for your next session.

The AI-robust item

Asking the model to design an item its own kind would pass but an unprepared student would not is a fast, honest way to audit your existing question bank.

A quiz that tells you what to reteach is worth ten that tell you a number.

Ready-to-export means every slide has one idea and a stated purpose. This prompt refuses to write filler.

ROLE: Instructional designer who builds slides for active learning, not for reading aloud.

TASK: Produce a slide-by-slide outline for a 50-minute session on the topic below.

FORMAT — a table, one row per slide:
Slide no | Headline (a claim, not a label) | 3 bullets max | What the LEARNER does here | Time

RULES:
- Headlines must be claims. "Drug interactions" is a label; "Timing separates
  interaction from coincidence" is a claim. Labels are banned.
- Maximum 3 bullets and 12 words per bullet. If it needs more, split the slide.
- No slide may pass without a "what the learner does" entry. If a slide has none,
  delete it and tell me why you deleted it.
- Insert an activity or check at least every 12 minutes.
- Mark which slides are ANIMATION or DIAGRAM rather than text, and say what the
  diagram must show.

CONSTRAINTS: Use only the syllabus text below. Do not add facts, statistics
or citations of your own — mark any place I need to supply evidence as [MY SOURCE].

SYLLABUS TOPIC: [ ]
LEARNERS: [ ]

Headlines must be claims

Expected output

Three rows from a usable outline:

| 4 | Timing separates interaction from coincidence | INR rose day 4 · onset fits CYP inhibition · diet change would be slower | Rank three explanations by timing, in pairs | 6 min |

| 5 | DIAGRAM — the CYP2C9 pathway with the inhibition point marked. Must show enzyme, substrate, and where the antibiotic acts. Do not label the outcome; learners infer it. | — | Predict the INR direction before the reveal | 4 min |

| 6 | [MY SOURCE] Reported incidence in local practice | — | — | 2 min |

And the deletions it reports:
“Removed a slide titled ‘Introduction and objectives’. It had no learner action and its content is delivered by slide 4’s ranking task, which reveals the objectives by requiring them.”

The [MY SOURCE] markers are the important detail: the model has been forbidden from inventing the local incidence figure, so it leaves you a hole to fill rather than a plausible number to be embarrassed by.

Claims, not labels

A headline that states a claim gives the slide a job. A label gives it a topic, and topics are what produce slides nobody remembers.

Delete-and-explain

Forcing the model to report what it removed turns a generator into an editor, and the explanation usually teaches you something about your own structure.

[MY SOURCE]

The verification block from the R.T.F. extension, applied to teaching material. It is the reason nothing on your slides will be a fabricated statistic.

A slide without a learner action is a document that has been formatted badly.

Modul 1 · assessment redesign

What is still worth assessing?

Detection is a losing arms race with a false-accusation problem attached. Redesign is the durable answer.

Q1Which assessment is most robust to a student using a capable AI?

A 3000-word essay on a very obscure topic

Obscurity buys almost nothing. Grounded tools handle niche topics well, and the marker still cannot tell.

A submitted draft plus a recorded five-minute defence of three specific choices in it

Correct. The artefact may be AI-assisted; the defence cannot be outsourced in real time. You are assessing the reasoning, which is what the outcome claimed anyway.

A closed-book handwritten exam

Robust, certainly — but it assesses recall under time pressure. If your outcome verb was evaluate or design, you have solved the integrity problem by abandoning the outcome.

Assessed because the third option is the tempting one, and choosing it quietly narrows what your programme claims to teach. CLO5.

Q2A detector reports 88% probability that a student’s essay is AI-generated. What may you conclude?

That it very probably is

No. These scores are not calibrated probabilities, and false positives fall disproportionately on students writing in a second language.

Essentially nothing that could support an academic-misconduct finding

Correct. A detector output is a prompt to have a conversation, never evidence. The conversation — ask the student to explain a choice — is what actually informs you.

That the student should be referred to a panel

Referring on a detector score alone has produced documented injustices, and disproportionately affects non-native writers and neurodivergent students.

Assessed because a wrong decision here damages a named student. CLO5.

Q3You want students to use AI well rather than hide it. The most effective single change is to…

Ban it and state the penalty clearly

Drives use underground, where it cannot be taught or supervised. You lose visibility of exactly the behaviour you most need to shape.

Require a short AI-use statement and award marks for the quality of the reflection

Correct. Marking the reflection makes disclosure worth doing rather than merely required, and it turns the disclosure itself into evidence of metacognition.

Allow anything without disclosure

Removes the friction but also removes the learning. Students never articulate what they delegated or why, which is where the reflective gain lives.

Assessed because it is the policy choice most of you will actually have to make this year. CLO5.

Session A · block 3 of 3

Modul 2 (1) — research: finding and verifying evidence

Same method, different target. This block ends with the two artefacts that Session B builds on directly.

What we cover

Gap types, question frameworks, search-string construction, the specialised AI search and mapping tools, PRISMA screening, and citation verification.

You will build

A search strategy in three database syntaxes, a filled literature matrix, and a synthesis paragraph that ends on your gap.

Carry it forward

The matrix and the gap sentence are the inputs to Session B. Bring them; the proposal block starts by turning them into a problem statement.

CLO3

Modul 2 (1) · the gap

Five gaps, and none of them is “nobody has done this”

“Under-researched” is not a gap. Name the type and the justification writes itself.

Evidence gap

Population gap

Method gap

Context gap

Translation gap

Why panels reject gap statements Absence of literature is ambiguous evidence: it may mean nobody has looked, or it may mean everybody looked, found nothing, and did not publish. A named gap type forces you to say which one you are claiming.

Modul 2 (1) · question framing

Turn an interest into a searchable question

Three frames. Pick by study type, not by habit.

Lab A3.1 · frame your own questionDo this nowReplace the bracketed text with your actual topic before running.

For intervention and comparison questions. The default in clinical and pharmaceutical research.

ROLE: Research methodologist experienced in systematic review protocols.

TASK: Convert my rough interest into a PICO-framed question, then stress-test it.

MY INTEREST: [e.g. "whether pharmacist-led counselling helps older patients take their medicines properly"]

OUTPUT, in this order:
1. P / I / C / O as a four-row table, each stated in searchable terms
2. The full question written as one sentence
3. THREE alternative O (outcome) choices, with what each would commit me to measuring
4. The single element most likely to be criticised, and why
5. Whether this question is answerable with existing literature or requires new data

CONSTRAINT: Do not cite anything. If you need to reference evidence, describe the type of study instead.

Quantitative · intervention

Expected output

Element	Searchable form
P Population	Adults aged 65+ on four or more regular medicines
I Intervention	Structured pharmacist-led medication review with follow-up
C Comparator	Usual dispensing without structured review
O Outcome	Medication adherence at six months (validated scale or refill data)

Question: In adults aged 65+ taking four or more regular medicines, does structured pharmacist-led medication review, compared with usual dispensing, improve medication adherence at six months?

Alternative outcomes offered: hospital readmission (harder, more fundable, needs bigger n) · potentially inappropriate prescribing count (cheaper, proxy) · patient-reported confidence (soft, needs validated instrument).

Most criticised element: the comparator — “usual dispensing” varies so much between settings that it may not be a stable control.

Item 3

Alternative outcomes are the point of the whole exercise. Your choice of O determines your sample size, your budget, and whether the study is feasible at all.

Item 4

Free reviewer feedback before you have written anything. Fix the comparator now, not in revision.

Item 5

Distinguishes a review from a primary study. Many proposals fail because this question was never asked.

PICO is not paperwork. Each element becomes one Boolean block in the next lab.

When there is no intervention — exposure, prevalence, risk-factor and cohort questions.

ROLE: Epidemiologist.

TASK: Frame my topic as a PEO question (Population, Exposure, Outcome) and identify its threats to validity.

MY TOPIC: [e.g. "whether long-term proton pump inhibitor use is associated with nutrient deficiency"]

OUTPUT:
1. P / E / O table in searchable terms
2. The question as one sentence
3. The three most likely confounders, and how each would bias the result (direction, not just presence)
4. Whether a cross-sectional design would be sufficient, and if not, why not

CONSTRAINT: No citations. Describe study types instead of naming papers.

Observational · no comparator arm

Expected output

P adults on continuous acid-suppression therapy for 12 months or more · E proton pump inhibitor use, by duration and dose · O laboratory-confirmed deficiency in specified micronutrients.

Confounders, with direction:

Indication confounding — the underlying condition may itself impair absorption, biasing away from null (overstates the drug effect)
Age — correlates with both exposure and deficiency; biases away from null unless adjusted
Dietary intake — rarely measured; direction unpredictable, which is worse than a known bias

Cross-sectional sufficiency: no — it cannot establish that exposure preceded outcome, which is the entire claim.

Direction

Asking for the direction of bias, not just its presence, is the move that separates a real methods discussion from a checklist.

Item 4

Design adequacy is where most postgraduate proposals are weakest. Ask it early, in writing.

No comparator means no C. Forcing PICO onto an observational question produces a search strategy that misses half the literature.

For experience, perception and mixed-methods questions, where PICO distorts what you are asking.

ROLE: Qualitative health researcher.

TASK: Frame my topic using SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type).

MY TOPIC: [e.g. "how community pharmacists experience pressure to dispense against their clinical judgement"]

OUTPUT:
1. S / PI / D / E / R table
2. The question as one sentence, avoiding any causal language
3. Which qualitative designs would suit, and which would not, with one reason each
4. The search-term challenge: which of these five elements will be hardest to capture in database terms, and what to do about it

CONSTRAINT: No citations.

Qualitative · experience-focused

Expected output

S community pharmacists in urban practice · PI experience of pressure to dispense contrary to clinical judgement · D semi-structured interviews · E perceived professional autonomy and moral distress · R qualitative, interpretive.

Suitable designs: interpretative phenomenological analysis (suits lived experience of a bounded phenomenon) · framework analysis (suits policy-facing outputs). Unsuitable: grounded theory, unless you genuinely intend to build theory rather than describe.

Hardest element to search: PI — “pressure to dispense against judgement” has no controlled vocabulary term. Solution: search the sample and design terms tightly, accept low precision, and screen manually. Qualitative searching trades precision for recall by design.

Item 4

This is the honest answer nobody gives you: qualitative searches cannot be precise. Plan for manual screening from the start.

Choose the frame from the question, never the other way round. A forced frame is visible to every reviewer.

Modul 2 (1) · search strategy

A search strategy is a method, not a browsing session

If you cannot paste it into an appendix, it is not a strategy.

Lab A3.2 · build a reproducible search stringDo this nowRun this with the PICO you just built. It is the single most useful prompt in Session A.

ROLE: Medical librarian who builds systematic review search strategies.

TASK: Build a documented, reproducible search strategy from my PICO.

MY PICO:
P: [population]
I: [intervention]
C: [comparator]
O: [outcome]

PRODUCE:
1. For each PICO element, a concept block containing controlled vocabulary AND free-text synonyms,
   combined with OR. Include British and American spellings, plurals, and common abbreviations.
2. Combine the blocks with AND.
3. Translate the strategy into THREE syntaxes:
   - PubMed (MeSH terms with [mh], free text with [tiab])
   - Scopus (TITLE-ABS-KEY, quotation marks for phrases)
   - Web of Science (TS= topic search)
4. State clearly which MeSH terms you are NOT certain exist, and mark them "[CONFIRM IN MeSH BROWSER]".
5. Suggest two limits I should consider and one I should NOT apply, with reasons.
6. Give me a one-paragraph methods-section description of this strategy, written in past tense.

CONSTRAINT: Do not invent MeSH headings. If unsure, flag rather than guess.

Produces an appendix-ready artefact

Expected output

PubMed block (abridged):

("Medication Adherence"[mh] OR "medication adherence"[tiab] OR "treatment adherence"[tiab] OR compliance[tiab])
AND
("Pharmacists"[mh] OR pharmacist*[tiab] OR "pharmaceutical care"[tiab])
AND
("Aged"[mh] OR elderly[tiab] OR "older adult*"[tiab] OR geriatric*[tiab])

Flagged for confirmation: “Pharmaceutical Care” [CONFIRM IN MeSH BROWSER] — commonly used in text, not a MeSH heading in its own right.

Limits suggested: publication date from 2010 (practice model changed); human studies. Limit to avoid: English-only — it introduces language bias and, in this region, systematically excludes relevant national-language literature.

Methods paragraph returned: “PubMed, Scopus and Web of Science were searched from inception to [date] using controlled vocabulary and free-text terms for three concept blocks…”

Item 1

Controlled vocabulary AND free text. Using only MeSH misses recent papers not yet indexed; using only free text misses everything indexed under a different word.

Item 4

This line does the safety work. Models invent MeSH headings freely. Forcing a confirm-flag turns an invisible error into a visible task.

Item 5

The limit NOT to apply is the pedagogically important half. English-only filters are the most common silent bias in regional reviews.

Item 6

You now have the methods paragraph before you have run the search. Edit it after; do not write it from memory later.

Every flagged MeSH term is a two-minute check in the MeSH browser. That is the whole verification cost of this method.

Record it as you go Save date, database, exact string and hit count every single time. Reconstructing a search three months later is one of the most reliably miserable experiences in research.

Modul 2 (1) · the tool landscape

Three different jobs, three different tools

Almost every complaint that “the AI tool was useless” is a tool being asked to do a job it was not built for. Name the job first.

DiscoverFind claims and studies

→

MapSee how they connect

→

ManageStore, screen, cite

Discover — question in, evidence out

Elicit, Consensus, Undermind, Perplexity, SciSpace. You give a research question; the tool retrieves real papers and extracts an answer. Best when you know what you are asking. Weakest at telling you what you have not thought to ask.

Map — paper in, neighbourhood out

ResearchRabbit, Litmaps, Connected Papers, Inciteful. You give one good paper; the tool shows what it cites, what cites it, and who else sits in that cluster. This is how you find the work your keywords missed — and keyword misses are the invisible failure in every search.

Manage — record in, citation out

Zotero, Mendeley, EndNote, Rayyan. Not discovery tools and not meant to be. Their job is that every reference in your manuscript entered from a database record, which is what makes fabrication structurally impossible.

I can say which of the three jobs I am doing before I open a tool
I use at least one mapping tool, not only keyword search
Nothing reaches my manuscript except through a reference manager

The failure you cannot see A search returns what your terms matched. It never reports what it missed, and the papers you missed are disproportionately the ones using different vocabulary for your concept — often from a different discipline, which is exactly where the interesting work usually is. Mapping tools exist because keyword recall is invisible.

Modul 2 (1) · tools

The current landscape, sorted by one question

Does it retrieve before it writes? Click each tool.

Elicit

GROUNDED

Consensus

GROUNDED

Undermind

GROUNDED

Scite

GROUNDED

NotebookLM

GROUNDED

Citation graphs

GROUNDED

General assistants

GENERATIVE

Rayyan

GROUNDED

Two or three, not twelve The consistent advice across every serious current comparison is the same: most researchers need one discovery tool, one reference manager, and one general assistant. Tool sprawl costs more attention than it saves.

Modul 2 (1) · discovery & mapping

The tools that find what your keywords missed

All eight are grounded — they retrieve real records. That fixes existence, never applicability. Click each one.

ResearchRabbit

GROUNDED

Litmaps

GROUNDED

Connected Papers

GROUNDED

Inciteful

GROUNDED

Perplexity

GROUNDED

Liner

GROUNDED

SciSpace

GROUNDED

Semantic Scholar

GROUNDED

A workable free stack Perplexity to orient, Consensus or Elicit to answer, ResearchRabbit or Connected Papers to check for missed clusters, Semantic Scholar to verify anything suspicious, Zotero to hold it all. Nothing in that chain costs money, and every step has a different job.

Modul 2 (1) · lab

The same question, phrased for five different tools

Take one question from your own work. Each tab shows how to phrase it for that tool, and what a good return looks like.

Lab A3.3 · tool-matched phrasingDo this nowReplace the bracketed parts with your own topic. Run at least three tabs.

Built to answer yes/no and directional questions across many papers. Phrase it as a claim to be tested, never as a topic.

Does [intervention] improve [specific outcome] in [population]?

Then, separately, ask the opposite so you see both sides:

What evidence contradicts the claim that [intervention] improves [outcome] in [population]?

And a third, to expose the measurement problem:

How is [outcome] measured in studies of [intervention], and do results differ by measurement method?

Always run the contradiction query

Expected output

What you get: a consensus meter, plus individual papers tagged as supporting, contradicting or neutral, each with a one-line finding and its design.

What to do with it: the meter is an orientation device, not a result. Two things make it misleading — papers are weighted by count rather than by quality, and a claim can be supported in a population unlike yours. Open the contradicting papers first; they are usually fewer and always more informative.

Why the third query matters most. If supportive findings cluster in studies using one measurement method and null findings cluster in another, you have not found a mixed literature — you have found a measurement artefact, and that is a publishable gap. This single question has produced more research questions in this workshop than any other.

Claim, not topic

“Metformin and ageing” returns noise. “Does metformin extend healthspan in non-diabetic adults?” returns evidence.

The contradiction query

Confirmation is trivially easy to retrieve for almost any claim. Ask what would change your mind, every time.

A meter is orientation. The contradicting papers are the evidence.

Built for structured extraction across a set of papers. Its value is in the columns you name, not the question you ask.

Question: What is the effect of [intervention] on [outcome] in [population]?

Then add these custom columns, which are the actual point of the tool:
- Study design and sample size
- Exact population, including age range and setting
- How the outcome was measured (instrument or data source)
- Effect size WITH confidence interval
- Comparator used
- Funding source
- One stated limitation, in the authors' own framing

Filter to: [study design], [years], human studies only.

The columns are the tool

Expected output

What you get: a table, one row per paper, one column per field you named — which is your literature matrix, already half-filled.

The step nobody does: open three papers at random and check the extracted cells against the actual text. Extraction errors cluster in exactly the fields you care about most — effect sizes get pulled from the abstract rather than the results table, comparators get simplified, and confidence intervals occasionally attach to the wrong outcome.

Two columns worth adding that most people omit: funding source, because it costs nothing and reviewers ask; and how the outcome was measured, because that column is where your gap usually turns out to be hiding.

Custom columns

This is the difference between Elicit as a search box and Elicit as an extraction engine. Name six columns and it becomes the second.

Spot-check three

Extraction is retrieval plus interpretation, and the interpretation half is unverified until you check it.

A matrix you spot-checked is evidence. One you did not is a guess in a table.

Use it to convert a vague area into specific answerable questions — then run those properly elsewhere.

I am beginning a review on [broad topic] in [discipline and setting].

Restrict yourself to peer-reviewed academic sources.

Give me:
1. The five most active sub-questions researchers are currently arguing about in this area
2. For each: the dominant position, the main challenge to it, and one citation for each side
3. The terminology split — which terms different disciplines use for the same concept here
4. What a newcomer to this field most commonly gets wrong

Then state plainly: which of these five is least well served by existing evidence, and why.

Do not summarise the field in general. If you cannot support a point with a citation, omit it.

Item 3 is the one to keep

Expected output

What you get: a referenced orientation report in a minute or two — and, from item 3, the synonym list that will make your database search actually work.

Item 3 is the highest-value line on this slide. If nursing calls it medication concordance, pharmacy calls it adherence and health economics calls it persistence, then a PubMed search using only your own discipline’s term will silently miss two thirds of the evidence. That is the invisible recall failure from the previous slide, and this query is the cheapest way to find it.

What not to do with it: do not cite this report, and do not treat item 1 as a finished research agenda. It indexes the open web, so a well-optimised preprint and a Cochrane review can arrive looking equally authoritative. Take the questions and the vocabulary; verify everything else in a real database.

“Peer-reviewed only”

Without this constraint the open web dominates, and blog posts arrive dressed as evidence.

The terminology split

Feed these synonyms straight into your concept blocks in the search lab. This is the single most practical output of the whole tab.

“Omit if uncited”

The verification block again. It shortens the answer and raises its reliability at the same time.

Use it to find your vocabulary and your questions, never your evidence.

These take papers, not sentences. The “prompt” is your choice of seed, and choosing badly is the whole failure mode.

Not a text prompt — a seeding protocol. Do it in this order:

1. Seed with 2-3 papers you are certain are central. Use a recent one AND an
   older influential one; seeding only with recent work returns only recent work.
2. Expand to "similar work" and scan for a cluster you do not recognise.
   That cluster is the answer to "what did my keywords miss?"
3. Expand to "earlier work" to find the methodological source everyone cites.
4. Expand to "later work" to find who has already challenged your seed papers.
5. Export the ones you keep straight into Zotero. Do not retype anything.

Then answer these two in writing before you close the tab:
- Which cluster did I not know existed, and what vocabulary does it use?
- Which of my seed papers has been contradicted by later work?

Seed old and new, or you see only new

Expected output

What a good session looks like: you seed three papers, expand once, and find a cluster of eleven studies from a neighbouring discipline that uses a different term for your outcome. None of them appeared in your keyword search. Two of them are more relevant than anything you had.

That is the entire case for mapping tools, and it is why this step belongs before you finalise a search string rather than after. The cluster you find here becomes new synonyms in your concept blocks.

The step 4 question is the uncomfortable one. If a seed paper you have built your rationale on has been contradicted by later work, you want to know now — not from Reviewer 2. Check it against Scite for citation context before you rely on it.

Seed old and new

Recency bias in the seed propagates through the entire map. One older influential paper fixes it.

The unrecognised cluster

This is the deliverable. If everything on the map is already in your library, either your search was excellent or your seeds were too narrow.

Export, do not retype

Every manual retype is an opportunity for the error this whole module exists to prevent.

Keyword search finds what you asked for. Mapping finds what you should have asked for.

The only mode in which a general model cannot invent a source: it can only use what you uploaded.

[Upload the 10-20 papers you have already screened in.]

Across only the documents I have uploaded:

1. Group the findings by claim, not by paper. Where papers disagree, name the
   disagreement and say which methodological difference most likely explains it.
2. List every outcome measure used, and which paper used which.
3. Identify what NONE of these papers examined.
4. For every statement you make, cite the specific document and section.
   If something cannot be answered from these documents, say "not in the corpus"
   rather than answering from general knowledge.

Question 3 is your gap

Expected output

Why this is the safest tab on the slide: the model has no access to anything beyond your uploads, so a fabricated citation is not merely unlikely — it is impossible. Every claim points at a document you chose and can open in one click.

Question 3 is doing the real work. “What did none of these papers examine?” is your gap sentence, generated from your own screened corpus rather than from a model’s impression of a field. It is the same output as the extraction-matrix lab, arrived at from the other direction — and running both and comparing them is the strongest single check on a literature review that this workshop offers.

The one thing to stay alert to: it can still misattribute a claim to the wrong uploaded paper. Grounding fixes existence, never accuracy of attribution. Spot-check any claim you intend to build on.

“Only the documents I uploaded”

Stated explicitly, this closes the door on general knowledge leaking in and being mistaken for a finding from your corpus.

“Not in the corpus”

The refusal instruction, again. A model permitted to say nothing is a model you can trust when it says something.

A closed corpus is the only setting where fabrication is structurally impossible.

Modul 2 (1) · screening

Where AI helps, and where it must not decide

The PRISMA spine, with the AI-permissible zone marked.

IdentifyRecords from databases

→

DeduplicateAI-safe, mechanical

→

Title/abstractAI-assisted ranking

→

Full textHuman decision

→

IncludeRecorded with reasons

Where AI genuinely helps

Deduplicating across databases. Ranking a thousand abstracts so the likely includes surface first. Drafting your inclusion criteria into explicit, testable form. Extracting structured fields from papers you have already accepted. Translating an abstract so it is not excluded for language.

Where it must not decide

Final include/exclude calls. Risk-of-bias assessment. Judging whether a comparator matches yours. Any decision you would have to defend to a reviewer. If an AI ranking pushed a study to position 900 and you never read it, that is your exclusion, not the tool’s — and it is undocumented.

Report it or it did not happen If AI touched your screening, say so in the methods: which tool, which stage, what it was allowed to do, and how disagreements were resolved. Reviewers are increasingly asking, and an honest sentence is far cheaper than a query letter.

Page M.J. et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71

Ouzzani M. et al. (2016). Rayyan — a web and mobile app for systematic reviews. Systematic Reviews, 5, 210. https://doi.org/10.1186/s13643-016-0384-4

Modul 2 (1) · literature matrix

The matrix is what turns reading into an argument

Fill three rows now, from papers you already have open. The columns are the argument in disguise.

Study	Design & n	Population	Key finding	Limitation	Relevance to my gap

Lab A4.1 · matrix to paragraphDo this nowPaste your filled rows in place of the bracket.

ROLE: Experienced academic writer in the health sciences.

TASK: Write ONE synthesis paragraph from the extraction rows below.

RULES:
- Organise by FINDING, never by study. Do not write "Author A found... Author B found..."
- Group studies that agree; name the disagreement explicitly where they do not
- End with the sentence that states what remains unresolved — this is my gap
- Use [Author, year] placeholders only for rows I supplied. Invent nothing.
- 150-180 words, past tense for findings, present tense for the current state of knowledge

MY ROWS:
[paste your matrix here]

Organised by finding, not by study

Expected output

Shape of a good output:

“Structured pharmacist review has been associated with improved adherence across settings, though effect sizes vary considerably [A, 2019; B, 2021]. Trials reporting the largest benefits shared two features: follow-up contact beyond a single session, and adherence measured by refill data rather than self-report [A, 2019; C, 2022]. Where self-reported measures were used, effects were consistently smaller and in one case null [D, 2020] — a divergence that may reflect measurement rather than intervention. What remains unresolved is whether the benefit persists beyond six months in populations with high baseline polypharmacy, since no included study followed patients past that point.”

Notice three things:

No sentence begins with an author name
The disagreement is explained (measurement), not merely reported
The final sentence is the gap, and it is now a testable claim rather than a complaint about the literature

Rule 1

“Organise by finding” is the difference between a literature review and an annotated bibliography. It is also the most common reason reviews are sent back.

Rule 3

The unresolved sentence connects this module to your problem statement. The gap is an output of synthesis, not an input to it.

Rule 4

Placeholders only. The model cannot add a citation it was not given, which removes the fabrication risk entirely at this step.

This is the moment the review stops being a summary and becomes an argument.

Modul 2 (1) · references

The structural fix for fabricated citations

If every reference enters your library from a database record, fabrication becomes impossible rather than merely unlikely.

Zotero

GROUNDED

Mendeley

GROUNDED

EndNote

GROUNDED

Rayyan

GROUNDED

The rule that ends the problem No citation enters a manuscript except through the reference manager, and nothing enters the reference manager except by import from a database. Typing a reference by hand, or copying one out of a chat window, is the single behaviour that produces every incident in this module.

Modul 2 (1) · drill

Which of these would you cite without checking?

Mark each row Safe or Never, then check your answers. Two rows are harder than they look.

A reference the chatbot supplied, with a DOI that resolves to the exact paper named.

Safe. You verified it — the DOI resolving to the named paper is the check. Note that what made it safe was your action, not the tool.

A reference with correct authors and journal, but the DOI resolves to a different paper by the same first author.

Never. The most dangerous pattern on this slide, because three of four fields check out. The paper you meant to cite may not exist at all.

A claim from Consensus showing eleven supporting studies, which you have not opened.

Never — not yet. The studies exist; the tool retrieved them. But you have not checked whether their populations, comparators or outcomes match your claim. Grounding fixes existence, not applicability.

A paper you found in Scopus, imported to Zotero, and read the methods section of.

Safe. Retrieved from an index, imported as a record, and read. This is the whole workflow in one row.

A statistic quoted by an AI as coming from “a 2023 WHO report”, with no document named.

Never. An unnamed institutional source is the least verifiable citation form that exists. If you cannot name the document, you cannot cite it — and the number may have been generated alongside the attribution.

A key reference that Scite flags as having been cited by later work as “contrasting”.

Safe to cite — but not to cite uncritically. The paper exists and the flag is information, not disqualification. Read the contrasting work and position your claim accordingly. Citing contested evidence honestly is good scholarship; citing it as settled is not.

Why this drill carries the most weight Rows 2 and 6 are where careers get complicated. Row 2 has passed peer review more often than anyone would like; row 6 is how a contested or retracted finding propagates through a field for a decade.

Checkpoint A2 · immediate feedback

Four questions before we close Session A

Then read your scorecard on the next slide.

Q1Your search returns 40 papers. A colleague’s identical question returns 300. Most likely explanation?

They searched a better database

Possible, but it rarely accounts for a sevenfold difference on the same question.

Your concept blocks used controlled vocabulary only, with no free-text synonyms

Correct — and the papers you missed are disproportionately the recent ones, which are not yet indexed.

They set no date limit

Contributes, but a date limit is a deliberate, documented choice. Missing synonyms is an undocumented one.

Assessed because recall failures are invisible — you never see what you did not find. CLO3.

Q2A grounded tool returns nine papers supporting your claim. What have you established?

That the claim is well supported

No. You have established that nine documents exist and were retrieved — not that their populations, comparators or outcomes match yours.

That nine real documents exist which the tool judged relevant

Correct. Existence, plus relevance-as-judged-by-a-tool. Applicability is still entirely your work.

That the literature is settled on this point

Supporting papers are trivially easy to retrieve for almost any claim. Ask what contradicts it.

Assessed because the gap between “retrieved” and “applicable” is where most citation errors now live. CLO3, CLO5.

Q3A synthesis paragraph should be organised by what?

Chronology — oldest study first

Occasionally right, when the point is how thinking evolved. Usually it produces a list.

Finding — grouping studies that agree, naming those that do not

Correct. This is what turns a summary into an argument, and it is the most common revision request on review manuscripts.

Study — one paragraph per paper

That is an annotated bibliography. Useful as a working document, never as a review section.

Assessed because it is the structural habit that most improves review quality. CLO3, CLO4.

Q4Where may AI make the final call in screening?

Deduplication

Defensible — deduplication is mechanical and checkable. But it is not a screening decision, so it is not quite what the question asks.

Nowhere that you would have to defend to a reviewer

Correct. AI may rank, sort, translate and extract. The include/exclude decision stays with a named human, and the audit trail must show it.

Title and abstract screening, if the tool is accurate enough

Accuracy is not the issue — accountability is. An unread exclusion is your exclusion, undocumented.

Assessed because it is the bridge into Session B’s integrity module. CLO5.

End of half-day 1

Your Session A scorecard

A mirror, not a mark. The lowest rows tell you exactly what to redo.

If you are below 40%

You have read more than you have built. Go back and complete Lab A2.1 at level 2 and Lab A3.2 with your own topic. Twenty minutes will move you a full band.

If you are 40–64%

Solid understanding, thin practice. Open the ledger, find your two lowest rows, and do only those. Do not restart the session.

If you are above 65%

Before Session B, run the full A3.2 search on your real project and bring the output. Session B builds directly on it.

Before you close this tab Reveal your scorecard, then use Copy my report code or Download report file below it. That is the only way anything leaves this page — closing the tab clears everything. Keep the code: Session B can load it so you see both half-days in one view.

Before Session B Do one thing: verify every reference currently sitting in a draft you are working on. Not a new task — an audit of what you already have. Most people find at least one problem, and finding it now is free.

Session A · resources

Everything cited, with a live link

Every claim in Session A traces to one of these. Follow the DOIs — that is the habit the whole session is teaching.

Vaswani A. et al. (2017). Attention is all you need. NeurIPS. https://doi.org/10.48550/arXiv.1706.03762

Lewis P. et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. NeurIPS. https://doi.org/10.48550/arXiv.2005.11401

Wei J. et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. NeurIPS. https://doi.org/10.48550/arXiv.2201.11903

Walters W.H. & Wilder E.I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13, 14045. https://doi.org/10.1038/s41598-023-41032-5

Bhattacharyya M. et al. (2023). High rates of fabricated and inaccurate references in ChatGPT-generated medical content. Cureus, 15(5), e39238. https://doi.org/10.7759/cureus.39238

Hicks M.T. et al. (2024). ChatGPT is bullshit. Ethics and Information Technology, 26, 38. https://doi.org/10.1007/s10676-024-09775-5

Ji Z. et al. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 1–38. https://doi.org/10.1145/3571730

Gao C.A. et al. (2023). Comparing scientific abstracts generated by ChatGPT to real abstracts. npj Digital Medicine, 6, 75. https://doi.org/10.1038/s41746-023-00819-6

Page M.J. et al. (2021). The PRISMA 2020 statement. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71

Ouzzani M. et al. (2016). Rayyan — a web and mobile app for systematic reviews. Systematic Reviews, 5, 210. https://doi.org/10.1186/s13643-016-0384-4

Methley A.M. et al. (2014). PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Services Research, 14, 579. https://doi.org/10.1186/s12913-014-0579-0

Bramer W.M. et al. (2017). Optimal database combinations for literature searches in systematic reviews. Systematic Reviews, 6, 245. https://doi.org/10.1186/s13643-017-0644-y

Biggs J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32, 347–364. https://doi.org/10.1007/BF00138871

Barrows H.S. (1986). A taxonomy of problem-based learning methods. Medical Education, 20(6), 481–486. https://doi.org/10.1111/j.1365-2923.1986.tb01386.x

Krathwohl D.R. (2002). A revision of Bloom’s taxonomy: an overview. Theory Into Practice, 41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2

Black P. & Wiliam D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7–74. https://doi.org/10.1080/0969595980050102

Hattie J. & Timperley H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

Roediger H.L. & Karpicke J.D. (2006). Test-enhanced learning. Psychological Science, 17(3), 249–255. https://doi.org/10.1111/j.1467-9280.2006.01693.x

Freeman S. et al. (2014). Active learning increases student performance in science, engineering, and mathematics. PNAS, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111

Sweller J. (1988). Cognitive load during problem solving. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_1

Mayer R.E. & Moreno R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43–52. https://doi.org/10.1207/S15326985EP3801_6

If you want the mechanism

Read Vaswani first for architecture, then Lewis for why grounding changes the failure mode. Together they explain almost everything you saw fail today.

If you want the evidence on harm

Walters & Wilder for the citation numbers, Hicks for why “hallucination” is the wrong word, and Gao for how well machine text passes as human.

If you teach

Biggs for alignment, Black & Wiliam for why formative feedback outperforms grading, Hattie & Timperley for what makes feedback actually work.

If you supervise

PRISMA 2020 and Bramer are the two documents to hand a student before they touch a database. Most search problems are prevented there, not fixed later.

Reading the field as it moves

Model capabilities change faster than the peer-reviewed literature about them. Treat any specific benchmark number as perishable; treat the mechanism papers above as durable. The reason a model fabricates has not changed since 2023, even though how often it does has.