The question of whether AI in healthcare is going to be overseen by a patchwork of state regulations or some overarching federal rules remains unclear. But states are continuing to pass laws to protect consumers and health system employees. California Gov. Gavin Newsom just signed several bills into law that address AI integration into healthcare and medical confidentiality, while vetoing a few others.
One new law (AB 1979) requires health facilities, clinics, and physician offices to take reasonable steps to ensure that licensed healthcare professionals retain independent professional judgment when their patient care is informed by clinical decision support systems. The new law bans entities from deploying AI tools to independently perform clinical functions that state law strictly reserves for licensed human professionals.
It also updates the Confidentiality of Medical Information Act to protect medical records and personal data when accessed through direct-to-consumer health chatbots.
“Technology should not replace human decision-making in healthcare,” said bill author Assemblymember Mia Bonta, in a statement. “AI is rapidly integrating into our healthcare system and reshaping our personal experience with it. Protecting patient safety, keeping our professional workforce engaged, and preserving the integrity of healthcare requires that we not rely exclusively on AI to do things quicker and cheaper. AB 1979 does that by ensuring medical records stay protected when accessed through direct-to-consumer health chatbots, and that licensed healthcare professionals retain the ability to exercise professional judgment when a clinical decision support system informs their care of a patient.”
Another law (SB 503) requires developers and deployers of clinical decision support systems to identify tools that present a risk of biased impacts, proactively mitigate those risks and monitor systems once deployed across patient populations.
Newsom vetoed a bill (AB 2575) that sponsors say would have protected healthcare workers from retaliation for overriding AI recommendations in hospitals.
The California Nurses Association (CNA) claims the veto of AB 2575 gives hospital corporations a green light to discipline nurses who override unsafe AI to protect their patients, enabling tech companies to blame healthcare workers when tech products cause harm. ”The governor knows AI should never replace a nurse’s clinical judgment, but he vetoed the bill that would have protected us for using our judgment,” said Sandy Reding, R.N. and president of CNA, in a statement. “That makes no sense to any nurse at the bedside. Today, a nurse who sees that the algorithm is wrong and acts to save her patient can still be disciplined or fired by her employer.”
In a CNA survey of more than 2,000 registered nurses, two-thirds said automated acuity scores measuring patients’ care needs did not match their own assessments of their patients.
Newsom vetoed another bill that would prohibit offering psychotherapy services when the services are provided through the use of companion chatbots. The bill would prohibit allowing AI to make therapeutic decisions or detect emotions or mental states without review and approval by a licensed professional.
