Can we talk about how AI chatbots keep giving outdated advice?

I was asking ChatGPT about the best way to set up a smart home hub last night, and it recommended a protocol that Google stopped supporting in 2022. Then I tried another bot for recipe help and it told me to use an ingredient that got recalled three years ago. How do these models not refresh their knowledge bases more often? Has anyone else run into this or found a way to check if the info is actually current?

3 comments

3 Comments

spencer7821mo agoMost Upvoted

Its actually way worse than people realize because the models dont just have old data, they get actively poisoned by bad sources. Last week I asked Claude about car maintenance tips and it cited a forum post from 2019 that recommended a fix that would literally void your warranty on a 2024 model. The scary part is these bots dont know theyre pulling from outdated sources, they just see the most repeated answer as the right one. Even the real-time search features are hit or miss since they scrape from random blogs that havent been updated in years.

jamie9401mo ago

Got the same problem asking about home security cameras and it suggested a model that got discontinued like two years ago. The bots don't know when things change, they just remember the popular answer from the training data. Even the ones claiming to search live pull from sketchy blogs or old reddit threads. I've basically started treating every bot answer as a starting point and then double checking the date myself on actual manufacturer sites. It's annoying but it's the only way to not get burned by outdated advice.

murphy.aaron1mo ago

Wait hold on, I always thought the data cutting off thing was mostly a meme but this is actually making me rethink it. I used to just take whatever the bot said and run with it, but seeing how bad the smart home and car advice gets I'm starting to think I've been lucky not to mess something up.