The PIGEON neural network learned to determine the location of objects in photographs

The PIGEON neural network learned to determine the location of objects in photographs

Photo from 2012: when determining the location of a campsite in Yellowstone, the neural network was wrong by 22 km.

Stanford graduate students have developed a neural network Predicting Image Geolocations (or PIGEON for short), which is able to determine the location of a photograph. PIGEON has an accuracy of 40 km and correctly names the country 95% of the time.

Lukas Haas, Michal Skreta and Silas Alberti were united by their passion for the online game GeoGuessr, in which participants are invited to determine the geolocation of objects depicted in photographs. GeoGuessr has over 50 million players participating in world championships, streaming on Twitch and popular YouTube channels. Stanford graduate students thought they were pretty good at GeoGuessr themselves. Moreover, they wondered if they could create an AI player that would perform better than humans. They turned to the already existing CLIP image analysis system and Google Street View images, creating their own dataset from 500,000 images of the service.

As NPR describes, “the team added additional elements to the program, including one that helped the AI ​​classify images based on their position on the globe.”

To test PIGEON, its creators arranged a sparring match between their neural network and GeoGuessr legend Trevor Reinbolt: “We weren’t the first AI to play against Reinbolt. We’re just the first AI to defeat Reinbolt.”

PIGEON has many usage scenarios, from locating locations on old images to quick surveys (e.g. for the presence of invasive plant species). The downside is that PIGEON can reveal information about people that they never intended to share. According to Jay Stanley, a senior policy analyst at the American Civil Liberties Union who studies technology, companies and governments will be able to use AI to learn which countries and regions people have visited; and such developments will help criminals in pursuing their victims. And removing geotags from photos will no longer protect privacy.

Stanford graduate students are aware of the risks their development carries. They co-authored the paper with their professor, Chelsea Finn, but refrained from releasing the full model precisely because of these concerns.

Google already has a feature known as “location estimation” that uses AI to determine the geolocation of a photo. Currently, it uses only a directory containing approximately one million points of interest, rather than the 220 billion street images that Google has collected. The company told NPR that users can turn off the feature.

Related posts