What is this?
GeoVisual Search is a way to find visually similar things all over the globe. Just click anywhere on the map and a red box will show up (we call that a “tile”). We’ll search for other places on the map that look like that. To do this, we use deep learning, a form of artificial intelligence that is loosely inspired by the structure of the brain.
Why didn’t it find all of the windmills?
Every time you click on a tile, we search the entire map for visually similar tiles. We return up to 1000 results. For something like windmills or farms, there are clearly many more than 1000 so we just return the top results.
Also, remember that GeoVisual Search isn’t actually trying to get an accurate count of objects like windmills and smokestacks. Instead, we’re just looking for things that are visually similar. However, the research we’ve done on teaching the computer visual patterns is an important step on the road to counting objects accurately.
How does it decide what results to return?
We use a type of artificial intelligence called deep learning, which is loosely inspired by neurons and the structure of the brain. For every tile on the map we run it through a deep learning algorithm that creates a “fingerprint” for that tile. Basically, you can think of it as abstracting some of the qualities of that tile in a way that allows the computer to begin representing the image like a human does: with colors, edges, and other features of the image. When you click on something, we compare every other image to that fingerprint and try to return the ones that look like each other.
It isn’t actually looking for other windmills or whatever object you’re clicking on - it’s just looking for things that look like that tile.
The results aren’t always perfect - sometimes we return things that don’t look visually similar or that weren’t what you’re looking for. Those are the kinds of cases we’ll be working on to improve accuracy.
What’s your accuracy?
We don’t actually measure accuracy for visual similarity. While working on this release, we’ve sampled tens of thousands of searches and tuned our machine learning until we were reasonably happy. We’ve also tested out getting human judgments (i.e., put two images in front of a human and ask if they are similar) as more data to fine tune and test our algorithms.
Since we don’t count windmills or any other object yet, we don’t measure their accuracy.
We’ll be exploring many ways of measuring and improving our results this year.
Some of the results aren’t very good, what’s going on?
It really helps if you click on something visually distinct. For example, one of the reasons that windmills work so well is that the windmill pops out of the rest of the image. Even for a human, it would be hard to show all of the visually similar places on earth where there’s grassland or forest.
However, sometimes we just get it completely wrong. Sometimes the search results are interesting, like when searching for suburbs shows very similar formations around foothills; but other times we’re just completely off. We’re working on making computers better and better at finding patterns in satellite imagery, so expect results to get better and better.
Can I change the size of the box?
For this demo, the box is a single size. However, in the future we’ll allow you to define an arbitrary shape and size.
What’s next for GeoVisual Search?
GeoVisual Search is just a demo. But it’s a really cool and visceral way to see the power of computer intelligence applied to satellite imagery. Plus, it’s helped us to set up the infrastructure to start building more and more interesting applications on top of our core platform.
Our research will start to focus on object detection at scale: how do we look for wind turbines, derricks, oil tanks, buildings, and other important objects all over the planet. For these objects, we’ll use the underlying principles of visual similarity to teach the computer what a wind turbine looks like in all of its forms and then try to do an accurate count of all the turbines globally. Obviously this is a very difficult task, but we think we’ve got the science to tackle this problem.
Once we’ve counted objects, we can start looking at maps through time and see what changes - how many new wind turbines are there and where are they, for example.
If you have ideas about what you’d like to do with GeoVisual Search today and have a team of developers who are experts at machine learning and/or geospatial data, drop us a line for early access to our underlying platform.
When can customers buy this?
GeoVisual Search is a demo. We’re planning to evolve the product this year and look for early customers who might want to build upon this functionality. However, we don’t right now have a release date.