Evidence Brief · OneRX

ChatGPT for Doctors: When It Is Safe, When It Is Not

Published 26 July 2026 · Reading time 10 min
By the OneRX Clinical Content Team · Reviewed against source publications prior to release
For healthcare professionals only

On this page: Why prohibition failed · Green tasks · Amber tasks · Red tasks · The habit that matters most · Where OneRX differs · The decision rule · FAQ · References

Hospital policies banning general AI tools have mostly failed, and they have failed for an unglamorous reason.

The tools are useful, they are free, they are on the phone in the pocket of every resident in the building, and nobody is going to stop using something that turns a forty-minute discharge summary into a six-minute one on the strength of a circular email.

So the question worth answering is not whether clinicians should use these tools. That has been settled by behaviour. The question is which tasks a general model can safely take and which it cannot, and the honest answer sorts by task rather than by tool.

This is the operational companion to Is ChatGPT Reliable for Medical Advice? — that piece covers what the evidence shows about model reliability. This one covers what to do about it on Monday morning.

Green: use freely

The unifying feature of everything in this category is that you supply the substance and the model handles form, so an error is visible on reading rather than buried in plausible prose.

Amber: complete verification, not partial

Here the model supplies substance, but you can check it quickly against a source.

The critical word is complete. Verification cannot be selective, and this is the part clinicians consistently get wrong.

In the largest oncology assessment of its kind, non-concordant recommendations appeared intermingled with concordant ones inside the same response — 34.3% of outputs contained at least one recommendation that did not match NCCN, sitting alongside content that did (Chen et al, JAMA Oncology 2023;9(10):1459–1462).

That study tested GPT-3.5 and the error rates have improved substantially since. The structural finding has not. If correct and incorrect content are interleaved, checking two lines out of five means you verify the two that were right.

There is a second reason partial checking fails. In current models, residual errors concentrate in complex conditional treatment algorithms rather than distributing evenly across a response (Meyer et al, 2026).

The questions most worth asking are the ones least worth trusting, which is precisely the opposite of what a spot check assumes.

Red: do not

Tasks where an error would be silent, the stakes are high, or you would have to defend the decision afterwards.

The habit that matters more than the tool

If you change one thing after reading this, change this one: do not put your own recollection into the question.

Researchers at Mount Sinai built 300 physician-validated clinical vignettes, each containing exactly one fabricated element — an invented laboratory test, a physical or radiological sign that does not exist, or a made-up condition. Everything else in each vignette was clinically sound. Six leading language models were asked to work with them.

The models repeated or elaborated on the planted fiction in up to 83% of cases. A dedicated mitigation prompt, explicitly warning that the vignette might contain false information, roughly halved the rate. It did not eliminate it (Omar et al, Communications Medicine 2025;5(1):330).

Translate that into a corridor between two clinics. You half-remember a threshold — an eGFR cutoff, a CPS score, a nodal count — and you type it into the question because typing it feels like giving context.

"In a patient with CPS 12, is pembrolizumab..."

The model does not challenge the premise. It accepts it and builds on it, fluently, in the same confident register it uses when it is entirely correct.

The tool has now amplified the exact error you consulted it to resolve, and nothing in the output looks any different.

So: describe the clinical picture and let the system retrieve the threshold. The moment you supply the number yourself, you have contaminated the answer before it started.

Where OneRX differs

The three properties that separate a general chatbot from clinical decision support software are not about medical knowledge. They are about provenance, currency and auditability, and each maps to a documented failure.

That last row is the one clinicians underweight until the day it matters. If a decision is questioned eighteen months on, "the AI suggested it" is not a defence, and with a general chatbot you cannot even reconstruct what it suggested or on what basis.

NMC obligations on standard of care and record retention do not soften because software was in the workflow, and CDSCO's software guidance points the same direction — traceability of the decision process is the expectation for anything used clinically.

In building retrieval for Indian practice, the failure we encounter most often is not wrong medicine. It is correct medicine for a drug that is not on the shelf, not on the formulary, or not within reach of the patient in front of you.

That is why 1,400+ doctors across 8+ countries use OneRX at the point of care rather than a general model.

The decision rule

One question, before using a general model for anything clinical.

Questions clinicians actually ask

Is there a safe alternative to ChatGPT for doctors?

Retrieval-grounded clinical decision support platforms address the core failure mode by fetching documents before answering, so citations resolve and guideline versions can be named. For Indian practice, the additional requirement is regulatory, pricing and coverage context attached to the clinical answer.

Can I use ChatGPT for discharge summaries?

For structuring content you supply, yes. You are the source and errors surface on review. Avoid entering identifiable patient data into consumer platforms, given DPDP Act obligations.

Why can I not simply verify the answer myself?

You can, but verification has to be complete rather than selective, because non-concordant content appears interleaved with correct content in the same response (JAMA Oncology 2023;9(10):1459–1462), and residual errors concentrate in the complex conditional questions rather than spreading evenly.

What is the single most dangerous way to use a medical chatbot?

Embedding an unverified specific in your question. Models elaborated on planted false details in up to 83% of tested vignettes, and an explicit mitigation prompt only halved that (Communications Medicine 2025;5(1):330).

What does auditability mean in practice?

That you can reconstruct, months later, which source produced a recommendation and what guideline version it reflected. This is relevant to NMC standard-of-care obligations and aligns with CDSCO's traceability expectations for clinical software.