Is AI reliable? MSU researcher who created tool says ‘no’

EAST LANSING, Mich. (WILX) - Your AI assistant might be lying to you — and a new tool from Michigan State University can tell you when.

A new study from MSU set out to find whether AI-generated answers are reliable as the technology becomes more prevalent.

Millions of Americans use AI every week, and some even use it daily.

Chloe Loftus, a student at MSU, uses it as she studies to become a teacher.

“My students, they use it for some projects, for sure. But I use it even more with like lesson planning, creating templates for them, even like blocking off my day,” Loftus said.

Maya Kolton, also an MSU student, calls AI “amazing as a tool.” She uses it for her own studies, particularly when writing.

But Loftus and Kolton have both gotten answers that are either not what they’re looking for or completely wrong.

So, how reliable is artificial intelligence?

It’s a problem Reza Khan Mohammadi, a research assistant at Michigan State University, has been working to address.

“You can get these large language models, or AI models, better, but how can we really trust them? Can we even trust them, and if so, when?” Khan Mohammadi said.

He is working on a tool he calls a “trust meter” for AI-generated answers. It was developed by researchers asking simple questions multiple times to see if answers remained consistent.

“If an argument that a large language model makes is basically a confident one and an honest and true one, in that scenario, if we stress the model out, the model’s answer is not going to change,” he said.

However, if the answer changes between attempts, that would indicate the AI answer is not reliable.

The tool has also been used in more high-stakes fields, including searching out both medical and financial information.

Khan Mohammadi says AI use for lower risk questions like looking for a new movie to watch, can be very helpful, but overall, he would not call AI ‘reliable.’

“At the end of the day, it comes down to the task, but if I want to give one answer, I would say no,” he said.

Mahdu Anderson, a resident of Bath Township, doesn’t often use AI because it hasn’t been around long enough to earn her trust. She says she would still be skeptical of answers, even with the trust meter tool.

“I think I’d rather have something that indicates with the source it was from and that sort of thing. We still need to verify all the information we have,” Anderson said.

See News 10’s Bobby Cushing’s full interview with Khan Mohammadi:

Subscribe to our News 10 newsletter and YouTube page to receive the latest local news and weather. Looking to hire people, or grow your business through advertising? Gray Digital Media is your one-stop marketing solution. Learn more.