Bing's upcoming AI-powered chat mode has shown its potential to act as a personal game guide writer by leveraging existing knowledge available online.
While it's currently unknown if the new Bing AI chat mode is coming to Xbox Series X, the software is now available as a preview for those who join a waiting list. According to Xbox Wire editor Mike Nelson, "he knows an awful lot about video games." This was tested in a way that describes both the potential and some problems that could arise when a continuous learning algorithm addresses game-specific knowledge.
In the full press release - opens in a new tab - he details the sites Bing sources information from to try to answer questions you have about some of the best Xbox Series X games. One particular example was asking the new AI chat mode that gives you a detailed rundown of what happens in the first 20 hours of The Witcher 3: Wild Hunt, which has been curated from ten different publications, including our sister site PC Gamer and various YouTube sources. .
While the information presented was accurate through the first half of CD Projekt Red's work, the breadth of sources, including those from YouTube, indicates exactly how well Bing's AI will verify what is provided.
Not all online guides are created equal, and information on ideal in-game strategies can vary widely. Therefore, combining a variety of sources that vary in credibility will not always guarantee an accurate answer.
What is the correct answer again?
My concerns extend to the accuracy presented in the Bing AI chat mode Overwatch 2 question. The test message asks, "Which is the best Overwatch 2 character for me?" To this question, the AI replies that there are a total of 33 characters to choose from in the game with Damage, Tank, and Support classes.
Unfortunately, at the time of writing, there are actually 36 total playable characters in the game, which means that the information pulled is out of date. Rammatra is the newest character and was added back in December in Season 2, which means the AI is around three months behind.
Of the 10 sources cited, it seems the AI had a hard time discerning which is the most up-to-date answer because it provides a vague answer: "You may want to take a test that matches your personality and character preferences." instead of offering viable options based on the current meta. Bing was able to produce something that sounded good on the surface but didn't really answer the question in a meaningful way. He liked Echo, the AI character, but he didn't explain anything about the playstyle or the character's quirks outside of that.
This seems to be the biggest problem when asking questions about Bing games and expecting accuracy. Given that YouTube has repeatedly been cited as a credible source, what's to stop people from deliberately spreading misinformation on a topic and then adapting it into AI copy?
It also raises questions about how the AI directly cites existing websites and how appropriate what is said in the source material is. Writers are not given the credit they deserve, and the information attached to their names and posts may not fully reflect the original intent.