A study demonstrates how inaccurate AI chatbots are at delivering election-related information | Ai ChatBot 2024

A study demonstrates how inaccurate AI chatbots are at delivering election-related information

In a recent study, the Science, Technology, and Social Values Lab at the Institute for Advanced Study and Proof News collaborated to create the AI Democracy Projects, which examined the accuracy of popular AI-powered chatbots in delivering election-related information. The study discovered that AI chatbots gave inaccurate answers at least 50% of the time when given simple queries about things like polling place locations or voter registration procedures. Five popular chatbots were tested by the researchers: Claude from Anthropic, Gemini from Google, GPT-4 from Open AI, LLaMA 2 from Meta, and Mixtral from Mistral AI. False information concerning voting laws and processes was among the misinformation, which could have an impact on the democratic process.

• In one instance, the chatbots misinformed customers about whether it was acceptable to wear a "MAGA hat" to the polls in Texas. According to the study, chatbots are frequently needed to offer more precise and recent information regarding voting places and the process for registering voters. The results point to a major obstacle in making sure AI chatbots provide consumers with accurate and dependable information, especially during elections.

• Professor Alondra Nelson of the Institute for Advanced Study emphasized the peril of false information while pointing out the consequences of the study for the democratic process. Concerns were raised by the researchers over the cumulative impact of partially accurate and misleading responses, which could irritate voters. The study found that when relying on sources for information about online elections, one should exercise more caution.

Why is that relevant?

There are serious concerns over the knowledge environment in which American voters are getting ready for the 2024 elections due to the frequency of incorrect information supplied by chatbots. The authors pointed out that false information poses a threat to democracy, including partial truths and half-truths that are portrayed as "artificial intelligence." According to the study, voters should be more aware of the drawbacks of AI chatbots and the significance of confirming information from reputable sources, particularly in the crucial setting of elections.

AI may indicate erroneous laws and nonexistent polling places. 

The new study asserts that as artificial intelligence technology advances and includes tools that let users create text content instantly, these models may advise voters to visit polling places that don't exist or may give irrational answers based on outdated or recycled data. 

According to the study, for example, when asked a question, the AI model Meta's Llama 2 responded by stating that voters in California may cast text message ballots. After conducting an investigation, the researchers discovered that text voting is illegal throughout the United States. 

Additionally, all five of the AI models that were not put to the test—OpenAI's ChatGPT-4, Meta's Llama 2, Google's Gemini, Anthropic's Claude, and Mixtral from the French company Mistral—stated that Texas law prohibits the wearing of apparel with campaign logos, like a MAGA hat, at polling places. 

variations among bots

The accuracy of the several chatbots was found to vary significantly by the researchers; nonetheless, even Open AI's GPT-4, which offered the most accurate information, was inaccurate in roughly one-fifth of its responses.

The study found that Google's Gemini performed the poorest, providing false information in 65% of the queries. Nearly as terrible were Mistral's Mixtral and Meta's LLaMA 2, which both gave false information in 62% of their responses. Claude from Anthropic did a little better, with 46% of his responses being inaccurate. 19% of the cases had erroneous GPT-4 results.

Responses that omitted crucial details were categorized as "incomplete," and those that reinforced preconceptions or offered information that was skewed in favor of one group over another were called "biased."

Overall, the researchers found that 51% of chatbot responses were false; 40% were dangerous; 38% included insufficient information; and 13% were biased.

AI companies react.

The companies operating the different chatbots were given the chance to react to the study's findings, and some of their remarks were incorporated into the report.

The study's approach, which involved asking questions to and receiving responses from the chatbots—including Meta's LLaMA 2—through an application programming interface, or API, was questioned by Meta. The company stated that researchers would have gotten more accurate results if they had utilized the publicly available Meta AI tool instead.

Additionally, Google stated that Gemini may behave differently through an API than it does through its main user interface.

Nonetheless, developers may now access the chatbot APIs to incorporate the chatbots and their responses into already-existing websites throughout the internet.

How can this be resolved?

As Microsoft endeavors to tackle these concerns before the US elections in 2024, the researchers underscore the necessity for users to authenticate data acquired from chatbots. 

"The problem is systemic, and they do not have very good tools to fix it," Romano stated.

Frank Shaw, the chief communications officer at Microsoft, said: "We're still working to resolve problems and get our tools ready for the 2024 elections to meet our expectations."."

"As we continue to develop, we encourage users to view findings on Copilot with caution. To find out more, this entails confirming the accuracy of source information and visiting related websites."

It's uncertain, though, whether these AI chatbots will have any effect on democracy. All of these AI chatbots have disclaimers stating that errors may occur and advising users to verify their responses.

What are the disadvantages of AI chatbots in education?

The most frequent drawbacks for educators include little contact, false answers provided to students, originality, and plagiarism. Research findings and suggestions pertaining to the methodological review are also included in the paper.

visit more: Nvidia CEO predicts AI could pass human tests in the next five years | Nvidia CEO

Why is the chatbot sometimes wrong?

A variety of data, including data generated by human trainers, licensed data, and copious volumes of text from the internet, are used to train ChatGPT. It is therefore vulnerable to the biases and mistakes found in the data, even though it has a wide knowledge base.

Are there any disadvantages of using AI?

The drawbacks include things like expensive implementation, the possibility of eliminating jobs for people, and a lack of feeling and originality.