A
physician
signs
a
progress
note
generated
from
an
ambient
recording.
Another
accepts
a
predictive
risk
score
embedded
in
the
electronic
health
record.
A
third
submits
an
artificial
intelligence
(AI)-drafted
appeal
of
an
insurance
denial
under
her
own
name.
In
each
case,
the
organization
calls
the
technology
“assistive.”
But
who
exercised
judgment?
That
question
will
define
the
next
stage
of
healthcare
AI,
and
we
are
not
asking
it
clearly
enough.
The
loudest
fears
about
AI
in
medicine
imagine
a
dramatic
handoff:
a
hospital
announcing
that
algorithms
now
diagnose
and
physicians
are
obsolete.
That
is
not
the
danger
I
worry
about
most.
The
quieter
and
more
likely
one
is
that
clinicians
will
continue
to
carry
legal,
ethical,
and
professional
responsibility
for
decisions
while
gradually
losing
the
time,
information,
authority,
and
discretion
required
to
make
them.
We
should
stop
treating
“AI
use”
as
a
single
category.
Ambient
documentation
tools
shape
the
clinical
record.
Predictive
systems
direct
attention
toward
some
risks
and
away
from
others.
Generative
tools
draft
prior-authorization
appeals,
patient
instructions,
and
discharge
summaries.
Message-triage
systems
decide
what
a
clinician
sees
first.
Each
of
these
changes
the
encounter
in
different
ways,
and
each
raises
distinct
questions
about
consent,
review,
and
accountability.
Lumping
them
together
obscures
the
hard
questions.
The
common
thread
is
a
widening
gap
between
where
responsibility
sits
and
where
control
lives.
Medicine
has
long
tied
accountability
to
identifiable
professionals:
physicians
sign
notes,
enter
orders,
and
answer
for
the
consequences.
AI
complicates
that
arrangement
in
a
specific
and
troubling
way.
A
clinician
may
sign
a
note
that
was
largely
machine-generated,
technically
accurate
yet
stripped
of
the
uncertainty,
hesitation,
or
social
context
that
made
the
encounter
clinically
important.
A
physician
may
accept
a
predictive
recommendation
because
it
looks
objective,
because
the
schedule
leaves
no
time
for
independent
review,
or
simply
because
overriding
it
requires
extra
documentation
while
accepting
it
requires
a
single
click.
A
denial
appeal
may
go
out
under
a
clinician’s
name
containing
arguments
she
never
had
time
to
examine.
In
every
case,
formal
accountability
stays
human
while
practical
control
becomes
technological
and
organizational.
That
is
an
unstable
and
ethically
questionable
arrangement.
A
person
cannot
be
meaningfully
responsible
for
a
decision
unless
she
has
the
authority,
information,
and
opportunity
to
make
it.
This
is
where
automation
bias
becomes
dangerous.
The
worry
is
not
that
clinicians
will
blindly
obey
algorithms.
It
is
that
organizational
conditions
—
time
pressure,
staffing
shortages,
fragmented
records
—
will
make
independent
review
feel
inefficient
and
eventually
optional.
When
AI-generated
documentation
is
usually
correct,
reading
every
sentence
starts
to
seem
like
wasted
effort.
When
a
recommendation
is
built
into
the
record
rather
than
offered
as
one
opinion
among
many,
it
acquires
an
authority
it
hasn’t
earned.
Over
time,
clinicians
risk
becoming
reviewers
of
machine
output
rather
than
originators
of
professional
judgment.
A
visibly
absurd
error
is
easy
to
catch.
A
polished,
coherent,
clinically
plausible
error
can
enter
the
record,
influence
later
decisions,
and
gain
clout
through
repetition.
None
of
this
is
an
argument
against
the
technology.
Ambient
tech
can
reduce
the
documentation
that
drives
burnout
and
let
physicians
look
at
patients
instead
of
screens.
Predictive
tools
can
identify
deterioration,
adverse
drug
events,
or
patients
who
need
follow-up.
AI-assisted
appeals
can
help
clinicians
push
back
on
an
insurance
system
built
to
exhaust
them
—
the
same
system
that
often
uses
AI
to
generate
the
denials
in
the
first
place.
These
benefits
are
real.
But
efficiency
alone
cannot
tell
us
whether
a
tool
is
governed
well,
and
prediction
is
not
the
same
as
judgment.
A
model
is
built
from
past
data,
selected
variables,
and
a
defined
outcome;
it
can
tell
you
what
has
happened
before,
but
it
doesn’t
have
the
clinical
acumen
to
reliably
say
what
will
happen
next
—
much
less
what
should
happen
for
the
patient
in
front
of
you.
A
risk
score
should
start
a
conversation,
not
end
one.
Two
safeguards
deserve
particular
emphasis.
First,
consent
must
be
specific.
Patients
cannot
meaningfully
consent
to
“AI”
in
the
abstract.
They
should
know
whether
a
visit
is
being
recorded,
whether
the
recording
is
retained,
what
the
system
produces,
who
reviews
it,
whether
the
output
will
influence
diagnosis,
treatment,
discharge,
or
insurance
authorization,
and
whether
declining
will
affect
their
care.
Broad
disclosure
that
“AI
may
be
used”
protects
the
institution
more
than
it
informs
the
patient.
Second,
clinicians
need
to
pay
attention
to
what
a
tool
cannot
see.
Tone,
contradiction,
family
dynamics,
and
the
quiet
sense
that
something
is
not
right
rarely
survive
a
clean
automated
summary,
yet
they
are
often
what
matters
most.
What
healthcare
organizations
need
is
a
plain
standard:
assistive,
not
autonomous.
In
practice,
that
means
every
consequential
AI-supported
process
has
a
named
accountable
human
and
a
defined
scope
—
whether
the
system
drafts,
recommends,
predicts,
or
decides.
It
means
human
review
that
involves
more
than
clicking
“approve,”
with
enough
time
and
information
to
change
or
reject
the
output.
It
means
visible
disclosure
to
patients
and
clinicians,
a
practical
way
to
correct
errors
before
they
spread
through
the
record,
and
protected
override
authority
so
a
clinician
can
depart
from
a
recommendation
without
retaliation
or
a
mountain
of
extra
paperwork.
And
it
means
governance
that
evaluates
more
than
technical
accuracy
—
whether
consent
is
meaningful,
whether
the
tool’s
effects
land
equitably
across
patients,
how
it
changes
daily
workflow,
and
what
it
does
to
the
clinical
relationship.
It
also
requires
honest
monitoring
of
whether
clinicians
are
actually
reviewing
outputs
or
just
approving
them
under
pressure.
The
public
debate
on
healthcare
AI
tends
to
split
between
enthusiasm
and
alarm.
The
more
useful
position
sits
in
between.
AI
can
help
clinicians
see
more,
remember
more,
and
reason
more
effectively.
It
should
not
be
used
to
simply
manufacture
the
appearance
of
human
judgment.
A
clinician’s
signature
cannot
become
a
ceremonial
act
placed
beneath
a
machine-produced
decision.
If
physicians
and
other
clinicians
are
going
to
remain
accountable,
they
must
also
remain
meaningfully
in
control.
AI
should
be
assistive,
not
autonomous
—
and
clinical
judgment
must
remain
more
than
the
final
click.
Please
enable
JavaScript
to
view
the