AI in
Clinical Research:
Dos & Don’ts
AI can draft. Only you can sign.
Ts. Dr. Mohd Nor Akmal Khalid
Center for Artificial Intelligence Technology (CAIT)
Faculty of Information Science and Technology, UKM
THU 22 OCT 2026 · 11:45–12:30
DEWAN NAFISAH ADEEB · O&G DEPARTMENT · HCTM
ChatGPT was asked to write 30 short medical papers. It listed 115 references.
How many were real and accurate?
Only 7%
were real and accurate.
- That was 2023. Models improved, so the problem moved: into published papers.
- Early 2026: about 1 in 277 new biomedical papers carried a fabricated reference (up from 1 in 2,828 in 2023).
In a proposal, every citation justifies a decision: the gap, the effect size, the sample size. A fake citation is a fake justification.
Two true stories from our own specialty
77.2%
ChatGPT’s score in an O&G OSCE, marked blind by 14 examiners. Human candidates averaged 73.7%.
It struggled when the scenario changed mid-station, and with local context.
46,042
women randomised to computerised CTG decision support or not. Poor neonatal outcome: 0.7% vs 0.7% (adjusted RR 1.01).
The software worked as designed. Outcomes did not change.
Passing an exam is not improving an outcome.
45 minutes, three parts
Plan
Question · Literature · Protocol
Protect
Consent · Patient data · Integrity
Analyse & publish
Analysis · Write-up · Disclosure
3 short demos
1 running example
1 traffic-light rule
0 paid tools needed
This session runs across today’s programme: it links back to the literature, PIS and sample-size talks, and forward to ethics, translation, registration, data analysis and EndNote.
The one rule for today
DELEGATE · TASKS
- polishing the English of your proposal
- stress-testing your PICO
- shorter, clearer PIS sentences
- drafting analysis code you can read
- rehearsing the questions a panel will ask
KEEP · JUDGMENT
- the question and its clinical relevance
- every citation, every number
- sample-size assumptions
- what the patient is told and agrees to
- the data, the analysis, the conclusion
Delegate tasks. Never delegate judgment.
ICMJE: AI tools cannot be authors because they cannot take responsibility for the work.
Dr Hana, MMed O&G trainee
Our running example · fictional
HER PROPOSAL
- Question
- Do WhatsApp reminders raise postpartum OGTT uptake after GDM?
- Design
- Single-centre RCT, HCTM
- People
- Women with GDM who deliver at HCTM
- Compared
- Scheduled reminders vs usual care
- Outcome
- OGTT done within 12 weeks postpartum
- Assumes
- uptake 40% → 60%
SIX STOPS · SIX CHANCES TO USE AI WELL OR BADLY
Question
PICO
Literature
Evidence
Protocol
Sample size
Consent
PIS in BM
Data
Analysis
Publish
Disclose
The whole talk on one slide
Go
USE FREELY · STILL READ IT
- edit your English
- critique your PICO
- simplify PIS wording
- explain a method or code
- rehearse viva questions
Go, then verify
USE · THEN CHECK
- literature search & summaries
- sample-size calculations
- BM translation drafts
- analysis code
- reporting-checklist checks
Stop
DO NOT, WITH OR WITHOUT AI
- invent or “fill in” data
- cite what you have not read
- upload identifiable patient data
- hide AI use · list AI as author
- AI-review confidential manuscripts
The colour depends on what is at stake if the output is wrong, not on which tool you use.
Plan with AI.
Think for yourself.
Question · Literature · Protocol
⏱ 11:52 – 12:05Don’t ask AI for a gap. Ask it to attack your question.
Here are 5 novel topics: 1. AI-based prediction of GDM 2. Probiotics for GDM prevention 3. Vitamin D and GDM 4. Exercise apps in pregnancy 5. Gut microbiome and GDM…
Generic · not checked for novelty · not linked to your clinic
1. “Usual care” is undefined: what do controls receive?
2. Contamination: mothers may share reminders in ward groups.
3. Justify the 12-week window against local practice.
4. Who sends reminders after the study ends?
5. Is uptake the outcome that matters, or diagnosis?
The gap comes from your clinic and the literature. AI is a sparring partner, not a source of ideas you can defend.
Three references. One is fake. One is wrong.
CHECKED IN PUBMED
[1] Real and accurate. Found in PubMed; details match.
[2] Real paper, wrong journal and year. A “Frankenstein” citation.
[3] Does not exist. Plausible authors, plausible journal, exactly the study Hana hoped to find.
FIVE CHECKS FOR EVERY CITATION
A literature workflow you can defend
Search
PubMed / Scopus with a written, repeatable search string.
YOU · DATABASEManage
Import into EndNote, de-duplicate, screen.
TOMORROW · 10:15Interrogate
NotebookLM over your selected PDFs: answers cite your sources.
AI · GROUNDEDRead
The 10–15 papers your protocol leans on, in full.
YOU · ALWAYSDon’t treat a chatbot’s list as a literature search: not reproducible, not complete, not checkable.
Do use grounded tools to find where a paper says something, then read that passage.
The number looked right. The assumption was hidden.
Using the formula for comparing two proportions, you need 77 participants per group (154 in total).
zα = 1.645, zβ = 0.842
RECHECKED · OpenEpi / G*Power / by hand
z = 1.645 is a one-sided test. Nobody asked for that.
Two-sided: 97 per group · 107 with continuity correction
+15% loss to follow-up → 115 per group, 230 women
AI’s answer was a fifth too small. Ask it to state every assumption, then recompute in a validated tool: the method from the previous session.
Amir, Mei Ling, Hana, +14
AI says a sample size of 30 is enough. Central limit theorem 😎23:41
Mine wrote 40 references in 10 seconds. Submitting the proposal tonight 🚀23:44
Speed is not the bottleneck. Trust is.
A proposal written in one night can be returned in one committee meeting.
Which part of your proposal would you be uncomfortable defending if a panel asked, “Where did this come from?”
Protect the patient,
the data, the record.
Consent · Patient data · Integrity
⏱ 12:05 – 12:15AI can make consent readable
13.9
GRADE · BEFORE
→
8.9
GRADE · AFTER
15 surgical consent forms from 15 academic centres, simplified by GPT-4. Physicians and a lawyer judged content and legal sufficiency preserved.
Because humans checked every form.
BUT · HANA’S PIS, ONE SENTENCE, “SIMPLIFIED”
ASAL · REC-APPROVED WORDING
Penyertaan anda dalam kajian ini adalah secara sukarela. Anda boleh menarik diri pada bila-bila masa tanpa menjejaskan rawatan anda.
VERSI AI · “LEBIH MUDAH”
Anda perlu hadir untuk semua ujian susulan sehingga kajian tamat.
Do: shorten sentences, check reading level, draft the BM version.
Don’t: let AI touch voluntariness, risks, confidentiality or contacts unchecked.
The ethics committee approves the final text, not the AI. An AI translation counts as one forward translation: back-translation and pre-testing still apply.
What never goes into a public AI tool
🤰
Patients
Case notes, IC or MRN numbers, CTG and ultrasound images with identifiers, lab results, chats with participants.
📄
Confidential work
Proposals under review, manuscripts and grants you are reviewing, unpublished results.
🏥
Institutional
Hospital data, audit reports, anything under an ethics or data-sharing agreement.
Health data is sensitive personal data under Malaysia’s PDPA 2010. Your consent form almost certainly did not ask permission to send it to a chatbot.
Instead: de-identify · test prompts on synthetic data · use UKM-managed accounts · keep the real analysis on your own machine.
| ID | Sex | Age | Pre | Post |
|---|---|---|---|---|
| 001 | F | 47 | 0.52 | 0.18 |
| 002 | M | 38 | 0.31 | 0.22 |
| 003 | F | 27 | 0.74 | 0.15 |
| 004 | M | 58 | 0.28 | 0.25 |
| 005 | F | 37 | 0.66 | 0.19 |
| … | … | … | … | … |
Too many ages ending in 7 or 8 · pre and post values unrelated
AI faked a whole trial in minutes
- Researchers asked GPT-4 with data analysis to create a dataset for 300 “patients” comparing two corneal transplant procedures.
- It made one procedure look better. Real evidence showed no difference.
- The flaws were visible only on close forensic checks.
Generating, “filling in” or adjusting data is research misconduct, with or without AI. Keep source documents, CRFs and an audit trail.
Analyse, write,
disclose.
Analysis · Write-up · Disclosure
⏱ 12:15 – 12:27Right code, wrong conclusion
Reminder 71/115 (61.7%) · Usual care 49/115 (42.6%)
RD = 19.1 points (95% CI 6.5 to 31.8) · p = 0.004
✦ “This proves WhatsApp reminders prevent type 2 diabetes after GDM.”
WATCH FOR
Code you can read, rerun and keep with the protocol
Over-claim: screening uptake ≠ diabetes prevented
Real data stays local. Cross-check in SPSS or R: tomorrow 08:15
Test the tool like a drug
DULU · EARLY 11TH CENTURY
Ibn Sīnā’s rules for testing a drug: test it in humans, on a single condition, and the effect must appear in all or most cases.
KINI · 2017
INFANT tested CTG decision-support software the same way: in 46,042 women, against a clear outcome. It did not help.
An AI tool for patients is an intervention. It earns trust the way any intervention does: with evidence.
If AI is the subject of your study
Use the matching EQUATOR guideline from the protocol stage, not after.
| Your study | Guideline | O&G example |
|---|---|---|
| Trial of an AI intervention · protocol | SPIRIT-AI 2020 | AI alert for abnormal CTG |
| Trial of an AI intervention · report | CONSORT-AI 2020 | same trial, results paper |
| Prediction model, including machine learning | TRIPOD+AI 2024 | PPH or pre-eclampsia risk score |
| Study that uses a large language model | TRIPOD-LLM 2025 | LLM summarising discharge notes |
| First live clinical use of an AI tool | DECIDE-AI 2022 | AI-assisted fetal biometry in clinic |
| Chatbot giving health advice | CHART 2025 | chatbot answers to antenatal questions |
What journals and funders expect
ALLOWED · DISCLOSED
- language editing
- AI-assisted analysis you verified
- AI as the object of study, reported fully
NOT ALLOWED
- AI listed as an author
- AI cited as a primary source
- undisclosed AI use
AS A REVIEWER
- no uploading manuscripts or grants to AI
- NIH bans AI in its peer review
- follow each journal’s reviewer policy
ICMJE: humans are responsible for anything submitted that used AI. Always read your target journal’s own policy: they differ, especially on AI images.
Disclose specifically, not vaguely
“AI was used to assist in writing” tells an editor nothing.
Writing help → acknowledgments or AI statement. Analysis or figures → Methods. UKM students: Appendix B2 template · GAIDeT
TASKS DELEGATED TO AI
TOOLS
Five dos, five don’ts
✓ DO
- Ask AI to critique your question, not invent it
- Check every citation against the original
- Recompute every number in a validated tool
- De-identify; test on synthetic data
- Keep an AI-use log and disclose specifically
✗ DON’T
- Paste identifiable patient data into public AI
- Cite anything you have not read
- Let AI generate or fill in data
- Let AI change what consent means
- Use AI on confidential reviews, or as an author
AI can draft. Only you can sign.
Hana’s route: what AI did, what she did
Question
PICO
AI: listed 5 weaknesses
Hana: defined usual care
Literature
Evidence
AI: gave 3 references
Hana: kept 1, rejected 2
Protocol
Sample size
AI: said 77 per group
Hana: recomputed 115
Consent
PIS in BM
AI: simplified the text
Hana: restored “sukarela”
Data
Analysis
AI: wrote the code
Hana: cut the over-claim
Publish
Disclose
AI: edited the English
Hana: wrote the declaration
Six stops. AI saved her time. Every decision was hers.
A research toolbox at zero cost
Terima kasih
Thank you
akmal@ukm.edu.my · akmal.app
AI can draft. Only you can sign.
References & provenance
Ali, R., Connolly, I. D., Tang, O. Y., et al. (2024). Bridging the literacy gap for surgical consents: an AI-human expert collaborative approach. npj Digital Medicine, 7. doi:10.1038/s41746-024-01039-2
Bellamy, L., Casas, J. P., Hingorani, A. D., & Williams, D. (2009). Type 2 diabetes mellitus after gestational diabetes: a systematic review and meta-analysis. The Lancet, 373(9677), 1773–1779.
Bhattacharyya, M., Miller, V. M., Bhattacharyya, D., & Miller, L. E. (2023). High rates of fabricated and inaccurate references in ChatGPT-generated medical content. Cureus, 15(5), e39238. doi:10.7759/cureus.39238
Brocklehurst, P., Field, D., Greene, K., et al. (2017). Computerised interpretation of fetal heart rate during labour (INFANT): a randomised controlled trial. The Lancet, 389(10080), 1719–1729. doi:10.1016/S0140-6736(17)30568-8
Collins, G. S., et al. (2024). TRIPOD+AI statement. BMJ, 385, e078378.
COPE Council. (2023). Authorship and AI tools: COPE position statement. publicationethics.org
Cruz Rivera, S., Liu, X., Chan, A.-W., et al. (2020). Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nature Medicine, 26, 1351–1363.
Gallifant, J., et al. (2025). The TRIPOD-LLM reporting guideline for studies using large language models. Nature Medicine.
Huo, B., et al. (2025). Reporting guideline for chatbot health advice studies: the CHART statement. BMJ Medicine.
ICMJE. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals (current version): AI-assisted technology. icmje.org
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., & Denniston, A. K. (2020). Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nature Medicine, 26, 1364–1374.
Nasser, M., Tibi, A., & Savage-Smith, E. (2009). Ibn Sina’s Canon of Medicine: 11th century rules for assessing the effects of drugs. J R Soc Med, 102(2), 78–80.
National Institutes of Health. (2023). The use of generative artificial intelligence technologies is prohibited for the NIH peer review process (NOT-OD-23-149).
NUS OBGYN-AI Collaborative Group (Li, S. W., Kemp, M. W., Logan, S. J. S., et al.). (2023). ChatGPT outscored human candidates in a virtual objective structured clinical examination in obstetrics and gynecology. AJOG, 229(2), 172.e1–172.e12.
Personal Data Protection Act 2010 (Act 709), Malaysia, as amended by the Personal Data Protection (Amendment) Act 2024.
Suchikova, Y., Tsybuliak, N., Teixeira da Silva, J. A., & Nazarovets, S. (2025). GAIDeT: Generative AI Delegation Taxonomy. Accountability in Research.
Taloni, A., Scorcia, V., & Giannaccare, G. (2023). Large language model advanced data analysis abuse to create a fake data set in medical research. JAMA Ophthalmology. doi:10.1001/jamaophthalmol.2023.5162
Topaz, M., et al. (2026). Fabricated citations: An audit across 2.5 million biomedical papers. The Lancet, 407(10541), 1779–1781.
Universiti Kebangsaan Malaysia. (2025). Guideline on Responsible Use of Generative AI in Teaching, Learning and Research Supervision.
Vasey, B., Nagendran, M., Campbell, B., et al. (2022). Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine, 28, 924–933.