Elon Musk told X users to upload their medical information to train AI bot Grok Gaylord Contreras | usagoldmines.com

“Try submitting x-ray, PET, MRI or other medical images to Grok for analysis,” Musk wrote on X last month. “This is still early stage, but it is already quite accurate and will become extremely good. Let us know where Grok gets it right or needs work.”

It turns out, Grok needs work. 

The AI successfully analyzed blood test results and identified breast cancer, according to some users. But it also grossly misinterpreted other pieces of information, according to physicians who responded to Musk’s post. In one instance, Grok mistook a “textbook case” of tuberculosis for a herniated disk or spinal stenosis. In another, the bot mistook a mammogram of a benign breast cyst for an image of testicles.

Musk has been interested in the relationship between health care and AI for years, launching the brain-chip startup Neuralink in 2022. The company successfully implanted an electrode that allows a user to move a computer mouse with their mind, Musk claimed in February. And xAI, Musk’s tech startup that helped launch Grok, announced in May it had raised a $6 billion investment funding round, giving Musk plenty of capital to invest in health care technologies, though it’s uncertain how Grok will be further developed to address medical needs.

“We know they have the technical capability,” Dr. Laura Heacock, associate professor at the New York University Langone Health Department of Radiology, wrote on X. “Whether or not they want to put in the time, data and [graphics processing units] to include medical imaging is up to them. For now, non-generative AI methods continue to outperform in medical imaging.”

X did not respond to Fortune’s request for comment.

The problems with Dr. Grok

Musk’s lofty goal of training his AI to make medical diagnoses is also a risky one, experts said. While AI has increasingly been used as a means to make complicated science more accessible and create assistive technologies, teaching Grok to use data from a social media platform presents concerns about both Grok’s accuracy and user privacy.

Ryan Tarzy, CEO of health technology firm Avandra Imaging, said in an interview with Fast Company that asking users to directly input data, rather than source it from secure databases with de-identified patient data, is Musk’s way of trying to accelerate Grok’s development. Also, the information comes from a limited sample of whoever is willing to upload their images and tests—meaning the AI is not gathering data from sources representative of the broader and more diverse medical landscape.

Medical information shared on social media isn’t bound by the Health Insurance Portability and Accountability Act (HIPAA), the federal law that protects patients’ private information from being shared without their consent. That means there’s less control over where the information goes after a user chooses to share it.

“This approach has myriad risks, including the accidental sharing of patient identities,” Tarzy said. “Personal health information is ‘burned in’ too many images, such as CT scans, and would inevitably be released in this plan.”

The privacy dangers Grok may present aren’t fully known because X may have privacy protections not known by the public, according to Matthew McCoy, assistant professor of medical ethics and health policy at the University of Pennsylvania. He said users share medical information at their own risk.

“As an individual user, would I feel comfortable contributing health data?” he told the New York Times. “Absolutely not.”