Think Twice Before Using Chat GPT for Local Content

We have been testing using ChatGPT to help write content for local businesses and wanted to share what we’ve found so far.  One of our first tests was for The Gorfinkle Real Estate Group.

The Test: 

We asked ChatGPT for a list of neighborhoods in Braintree, MA. It returned a list of 13 “neighborhoods.”

Now the first issue is that Braintree only has three neighborhoods. We asked Dan Gorfinkle about this. He said, “I confirmed with a lifelong Braintree resident and found the following neighborhoods, Braintree Highlands, East Braintree, and South Braintree. The other neighborhoods on the list are not correct.”

So, what on Earth are the other ten things Chat GPT listed? Well, “Five Corners” and “South Braintree Square” appear to be shopping centers that are not neighborhoods that someone would live in.  Washington Street is literally a street, not a neighborhood.

Yet, when we asked Chat GPT to write an article on the top neighborhoods to live in Braintree, MA, it listed “Braintree Square” and said people would want to buy homes there.

Next, we asked ChatGPT for some things to do in Braintree Highlands, which is one of the supposed neighborhoods it listed.

It listed a golf course 28 minutes away as the top item. When we asked Gorfinkle about this, he said that residents of Braintree would not go to this golf course as it is too far away. Braintree actually has a golf course right in town that has a similar rating.

Next, we asked ChatGPT to list schools in South Braintree.

The first result, South Braintree Elementary School, does not exist. A full list of schools can be found here in the navigation menu, and this fictional school is not listed. We also confirmed this with Gorfinkle.

Finally, we asked it for details about a local park in Braintree known as Sunset Lake Park.

Chat GPT said it had a basketball court, baseball field, and soccer field. None of this information was listed on their website or on street view. Gorfinkle stated, “they have what’s listed on the site and if you wanted a basketball court or soccer field you would need to go to a nearby school which is on the other side of the lake.”

We tried to figure out where Chat GPT was getting all this wrong information by asking it to cite its sources, but it could not tell us.

Conclusion: 

Given how many facts it got wrong, we would suggest strongly thinking twice before using it for local content.  Want to talk more about ways you should be using AI to help with SEO? Make sure you come to our next LocalU event in Dallas.