In this article, you will learn how you can use Overpass Turbo in your GEOINT investigation. The guide starts with [[How to Use Overpass Turbo#^8f3d63|description of the basic elements of OSM]] that you need to understand when working with the tool. Next comes the [[How to Use Overpass Turbo#^2e0a87|description of the Overpass Query Language]], which is needed to compose queries to the OSM database. In the next part you will learn about the [[How to Use Overpass Turbo#^bcc759|methodology]] that should be used when working with Overpass Turbo to solve GEOINT case. The methodology includes 2 main methods of query construction: [[How to Use Overpass Turbo#^58ddb0|sequential]] and [[How to Use Overpass Turbo#^2ce4c9|trilateration]]. The final section presents three examples of solving GEOINT cases with Overpass Turbo, showing [[How to Use Overpass Turbo#^3feb38|examples]] of applying the [[How to Use Overpass Turbo#^7a771d|sequential]] and [[How to Use Overpass Turbo#^0dccad|trilateration]] methods, and their [[How to Use Overpass Turbo#^49e0eb|combination]]. Each example is attached with a code that you can test by yourself. ![[pic15.png]] >[!abstract] >- [[How to Use Overpass Turbo#^8f3d63|OpenStreetMap]] > - [[How to Use Overpass Turbo#^b12c43|Node]] > - [[How to Use Overpass Turbo#^bf31a1|Way]] > - [[How to Use Overpass Turbo#^dc7c22|Relation]] > - [[How to Use Overpass Turbo#^3924b4|OSM Tags]] >- [[How to Use Overpass Turbo#^2e0a87|Overpass QL overview]] > - [[How to Use Overpass Turbo#^969a06|Statements]] > - [[How to Use Overpass Turbo#^1720d5|Execution state]] > - [[How to Use Overpass Turbo#^0affcf|Sets]] > - [[How to Use Overpass Turbo#^c82faf|Language structure]] > - [[How to Use Overpass Turbo#^957c2a|Brackets]] >- [[How to Use Overpass Turbo#^bcc759|Methodology of use]] > - [[How to Use Overpass Turbo#^9a27b0|Object extraction and recognition]] > - [[How to Use Overpass Turbo#^7665c2|Identifying the search region]] > - [[How to Use Overpass Turbo#^860685|Understanding of regional OSM tags specifics]] > - [[How to Use Overpass Turbo#^7911b8|OSM tag selection]] > - [[How to Use Overpass Turbo#^91a218|Query creation]] > - [[How to Use Overpass Turbo#^58ddb0|Sequential method]] > - [[How to Use Overpass Turbo#^2ce4c9|Trilateration method]] >- [[How to Use Overpass Turbo#^3feb38|Examples of use]] > - [[How to Use Overpass Turbo#^7a771d|Example #1: Sequential method]] > - [[How to Use Overpass Turbo#^0dccad|Example #2: Trilateration method]] > - [[How to Use Overpass Turbo#^49e0eb|Example #3: Combination of methods]] **Overpass API** — an API that allows extracting certain data from the OpenStreetMap (OSM) database by user query. Queries to the Overpass API can be created in [Overpass QL (Query Language)](https://wiki.openstreetmap.org/wiki/Overpass_API/Overpass_QL) form, e.g. using [Overpass Turbo](https://overpass-turbo.eu/). **Overpass Turbo** — web interface for creating queries and displaying the result of Overpass API. With Overpass QL (Query Language) we can request and analyze the OSM objects we need by filtering and refining with specific OSM tags. ## OpenStreetMap ^8f3d63 **[Open Street Map (OSM)](https://www.openstreetmap.org/)** — a project that creates and provides free geographic data, giving everyone the opportunity to create maps of the world. When filling a map, elements are created. There are 3 basic elements: **nodes**, **ways**, **relations**. ### Node ^b12c43 ![[pic5.png|100]] A node represents a specific point on the earth's surface defined by its latitude and longitude. Each node comprises at least an id number and a pair of coordinates. - nodes can be used to define standalone point features. - nodes are also used to define the shape of a way. - a node can be included as a member of a relation. ### Way ^bf31a1 ![[pic6.png|100]] A way is an ordered list of 1-2000 nodes that define a polyline. Ways are used to represent linear features such as rivers and roads. - ways can also represent the boundaries of areas (solid polygons) such as buildings or forests. - note that closed ways occasionally represent loops, such as roundabouts on highways, rather than solid areas. - areas with holes, or with boundaries of more than 2,000 nodes require a more complex multipolygon relation data structure. ### Relation ^dc7c22 ![[pic7.png|100]] A relation is a multi-purpose data structure that documents a relationship between two or more data elements (nodes, ways, and/or other relations). Examples include: - a route relation, which lists the ways that form a major highway, a cycle route, or a bus route. - a turn restriction that says you can't turn from one way into another way. - a multipolygon that describes an area with holes. ### OSM Tags ^3924b4 ![[pic8.png|100]] Each object can be described by one or more tags. Each tag has a `key=value` structure that allows describing and categorizing the object. The key, therefore, is used to describe a topic, category, or type of feature (e.g., `highway` or `name`). Keys can be qualified with prefixes, infixes, or suffixes usually, separated with a `:`. The value provides detail for the key-specified feature. - free form text (e.g., `name="Jeff Memorial Highway"`) - one of a set of distinct values (e.g., `highway=motorway`) - multiple values from an enumeration (e.g., `opening_hours=Tu-Fr 08:00-18:00;Mo 09:00-18:00;Sa 09:00-12:00;closed Aug`) - number (e.g., `distance=326`) The value is obligatory for the tag, even if the key is self-explanatory (e.g., `building=yes`). Here are a few examples used in practice: - `highway=residential` a tag with a key of `highway` and a value of `residential` which should be used on a way to indicate a road along which people live. - `name="Park Avenue"` a tag for which the value field is used to convey the name of the particular street - `maxspeed=50` a tag whose value is a numeric speed and speed unit. - `maxspeed:forward=*` a key that includes a namespace for `maxspeed` to further distinguish its meaning. - `name:de:1953-1990=Ernst-Thälmann-Straße` a tag with the `name` key suffixed namespaces to specify the German name which was valid in some years. Thus, OSM is a huge database containing geographical objects with descriptions contained in tags. ## Overpass QL overview ^2e0a87 ### Statements ^969a06 Queries consist of statements: - each statement ends with `;` - statements are executed sequentially - each statement changes the execution state ### Execution state ^1720d5 The execution state consists of: - default set labeled as `_` - user-created sets ### Sets ^0affcf Sets are storages to which data is written after the execution of a statement: - sets can be used for writing and reading data - a set can contain any combinations and quantities of OSM elements (nodes, ways, relations) - by default the result of the operator is written to the default set `_` - user can create his own sets ### Language structure ^c82faf Statements can be divided into three categories: - [Settings](https://wiki.openstreetmap.org/wiki/Overpass_API/Overpass_QL#Settings) — optional statements that set parameters for query execution ( e.g., `[out:json][timeout:25];`) - [Block statements](https://wiki.openstreetmap.org/wiki/Overpass_API/Overpass_QL#Block_statements) — statements which group Overpass QL statements together (e.g., `(statement_1; - statement_2;)[->.result_set];`) - [Standalone queries](https://wiki.openstreetmap.org/wiki/Overpass_API/Overpass_QL#Standalone_statements) — complete statements consisting of filters which can change the composition of sets (e.g., `rel["boundary"="postal_code"][postal_code=65343];`) ### Brackets ^957c2a Brackets are also used in Overpass QL: `( )`, `[ ]` and `{ }` ## Methodology of use ^bcc759 A plan for creating a query: - [ ] Object extraction and recognition - [ ] Identifying the search region - [ ] Understanding of regional OSM tags specifics - [ ] OSM tag selection - [ ] Query creation Let's look at each phase of the plan separately. ### Object extraction and recognition ^9a27b0 We need to extract all objects from the given perspective, recognize and analyze their characteristics. The list of objects will be used to select OSM tags. ### Identifying the search region ^7665c2 Overpass Turbo resources are limited, which doesn't allow us to request the entire OSM database at once, so we need to narrow the search area on the map. > [!tip]+ Use the planning techniques in the **Instructions** section to identify a possible search region ### Understanding of regional OSM tags specifics ^860685 There are regionally specific OSM tags: - language used in the region (e.g., `taxon:it="Pioppo cipressino"`) - regional technical designations (e.g., `railway:jkv=yes`) > [!tip]+ Use the map of the selected region on [OpenStreetMap](https://www.openstreetmap.org/) to learn about region-specific OSM tags ### OSM tag selection ^7911b8 Select OSM tags in accordance with the selected objects, their parameters and specifics of the region. > [!tip]+ Use the **Objects** section with OSM tags categorized by object type to find the tag you need ### Query creation ^91a218 Using statements and sets, create an Overpass Turbo query. When we need to find a place where there is a combination of all the objects we see in the photo, we can use two query methods that can be combined: Sequential method and Trilateration method. For both methods, we need to select a key element that will be marked on the map. #### Sequential method ^58ddb0 The sequential method is that we create a sequence of elements by chaining them together. With each statement we reduce the number of results, which reduces the resources spent on query execution. Let's imagine that we have 5 elements at a certain distance from each other. The #5 element is the key element. Let's determine the approximate distance between them. ![[sequential-method-1.png|500]] Then we move sequentially from one element to another, narrowing the area: - in the radius of each element of type #1 find all elements of type #2 - in the radius of each element of type #2 find all elements of type #3 - in the radius of each element of type #3 find all elements of type #4 - in the radius of each element of type #4 find all elements of type #5. Thus, we get the list of elements of type #5 that are at the distance we set from all elements at the same time. ![[sequential-method-9.png|500]] #### Trilateration method ^2ce4c9 The trilateration method allows to get data about all elements in a given region and at the same time narrow the search area by determining the distance from each element to the key element. This method requires more resources because it requires storing information about all elements of a certain type. Let's imagine that we have 4 elements at a certain distance from the key element #5. ![[trilateration-method-1.png|500]] Get the location of all elements in the given region: - get the set of elements of type #1 - get the set of elements of type #2 - get the set of elements of type #3 - get the set of elements of type #4 We know the approximate distance from the key element #5 to the others, so we find all the key elements that are at the given distance: - set the distance from elements of type #1 to elements of type #5 - set the distance from elements of type #2 to elements of type #5 - set the distance from elements of type #3 to elements of type #5 - set the distance from elements of type #4 to elements of type #5 Thus, we get the list of elements of type #5 that are at the distance we set from all elements at the same time. ![[trilateration-method-9.png|500]] ## Examples of use ^3feb38 Let's look at some simple examples of using Overpass Turbo when the input data is a photo. Let's imagine that we know in advance the region in which we are going to search. ### Example #1: Sequential method ^7a771d The photo was taken in Italy, in the town of Legnago. We need to find the place where the photo was taken. ![[Street View 360-1-edit.png]] After exploring the area and most frequently marked objects on the OSM we mark them out in the photo. We can see a communications tower, power lines, a residential building and a parking area. Let the parking be the key element: ![[Street View 360-1-edit-1.png]] Let's set the city of Legnago as the search region: `{{geocodeArea:Legnago}}->.searchArea;` Let's find all communications towers in the region and write this data to the `.tower` set: `node["man_made"="tower"]["tower:type"="communication"](area.searchArea) -> .tower;` Let's find all such power lines that are within 200 meters of the communications towers that were found earlier and record this data in the `.tower_line` set: `way["power"="line"](around.tower:200) -> .tower_line;` Let's find such buildings that are 50 meters away from the power lines that were found earlier and record this data in the `.tower_line_building` set: `way["building"="yes"](around.tower_line:50) -> .tower_line_building;` Let's find such parkings that are 30 meters away from the buildings that were found earlier. The result will be recorded in the default set `_`: `way["amenity"="parking"]["parking"="street_side"](around.tower_line_building:30);` Output the result using the `out` operator. Expand the query result using the `>` operator. > [!info]+ >Recurse down (`>`) — an operator that takes an input dataset and converts it into an output dataset that includes: >- all nodes that are part of ways belonging to the input dataset (if the dataset contains part of a way, such a way will be displayed in its entirety, even outside the frame); >- all nodes and ways that are members of a relation belonging to the input dataset (if the dataset contains a relationship, the relation will be displayed in its entirety, including nodes and ways, even outside the frame); >- all nodes that are part of ways in the output set Display the result on the map again using the `skel` parameter. > [!info]+ >`skel` — parameter allowing to output minimal information: >- for nodes: id and coordinates >- for ways: id and identifiers of the constituent nodes. >- for relations: id of the relation, as well as id, type and role of all its constituent members. `out;` `>;` `out skel;` We get the following query: `{{geocodeArea:Legnago}}->.searchArea;` `node["man_made"="tower"]["tower:type"="communication"](area.searchArea) -> .tower;` `way["power"="line"](around.tower:200) -> .tower_line;` `way["building"="yes"](around.tower_line:50) -> .tower_line_building;` `way["amenity"="parking"]["parking"="street_side"](around.tower_line_building:30);` `out;` `>;` `out skel;` We get one result: ![[pic13.png]] The place we're looking for: `45.196210264081444, 11.288655071344511` ![[pic14.png]] ### Example #2: Trilateration method ^0dccad The photo was taken in Spain, in the Castilla-La Mancha autonomous community or Madrid. We need to find the place where the photo was taken. ![[Street View 360-2-edit.png]] After exploring the area and the most frequently marked objects on the OSM, we mark them out in the photo. We can see a parking, garbage containers, a bridge over a river, the name of a supermarket, a crosswalk, and a give way road sign. Let the road sign be the key element: ![[Street View 360-2-edit-1.png]] Let's set Castilla-La Mancha and Madrid as the search region: > [!info]+ > `( )` — an operator that combines sets of data into one. `(` `{{geocodeArea:"Castilla-La Mancha"}};` `{{geocodeArea:"Madrid"}};` `) -> .searchArea;` OSM tags for the same supermarket may vary, so let's list the possible variants. Get all Ahorramás supermarkets in the selected region and write this data to the `.shop` set: > [!info]+ > `nwr` — an operator that combines all three elements (node, way, relation). `(` `nwr["building"="commercial"]["name"~"Ahorram(a|á)s"](area.searchArea);` `nwr["building"="commercial"]["brand"~"Ahorram(a|á)s"](area.searchArea);` `nwr["shop"="supermarket"]["name"~"Ahorram(a|á)s"](area.searchArea);` `nwr["shop"="supermarket"]["brand"~"Ahorram(a|á)s"](area.searchArea);` `nwr["name"~"Ahorram(a|á)s"](area.searchArea);` `) -> .shop;` Get all the bridges in the selected region and write this data to the `.bridge` set: `nwr["bridge"="yes"](area.searchArea) -> .bridge;` OSM tags for rivers may vary, so let's list the possible variants. Let's get all rivers in the selected region and write this data to the `.water` set: `(` `nwr["waterway"="river"](area.searchArea);` `nwr["water"="river"](area.searchArea);` `) -> .water;` OSM tags for crosswalks may vary, so let's list the possible variants. Let's get all crosswalks in the selected region and write this data to the `.crossing set: `(` `nwr["crossing"="marked"](area.searchArea);` `nwr["highway"="crossing"](area.searchArea);` `) -> .crossing;` OSM tags for parkings may vary, so let's list the possible variants. Let's get all parking lots in the selected region and write this data to the `.parking` set: `(` `nwr["amenity"="parking"](area.searchArea);` `nwr["parking"="street_side"](area.searchArea);` `nwr["amenity"="recycling"](area.searchArea);` `nwr["recycling_type"="container"](area.searchArea);` `) -> .parking;` Let's find such give way signs, which are simultaneously located in the radius: - 100 meters from a Ahorramás supermarket - 50 meters from a bridge - 50 meters from a river - 50 meters from a crosswalk - 50 meters from a parking `node["highway"="give_way"](around.shop:100)(around.bridge:50)(around.water:50)(around.crossing:50)(around.parking:50);` `out;` `>;` `out skel;` We get the following query: `(` `{{geocodeArea:"Castilla-La Mancha"}};` `{{geocodeArea:"Madrid"}};` `) -> .searchArea;` `(` `nwr["building"="commercial"]["name"~"Ahorram(a|á)s"](area.searchArea);` `nwr["building"="commercial"]["brand"~"Ahorram(a|á)s"](area.searchArea);` `nwr["shop"="supermarket"]["name"~"Ahorram(a|á)s"](area.searchArea);` `nwr["shop"="supermarket"]["brand"~"Ahorram(a|á)s"](area.searchArea);` `nwr["name"~"Ahorram(a|á)s"](area.searchArea);` `) -> .shop;` `nwr["bridge"="yes"](area.searchArea) -> .bridge;` `(` `nwr["waterway"="river"](area.searchArea);` `nwr["water"="river"](area.searchArea);` `) -> .water;` `(` `nwr["crossing"="marked"](area.searchArea);` `nwr["highway"="crossing"](area.searchArea);` `) -> .crossing;` `(` `nwr["amenity"="parking"](area.searchArea);` `nwr["parking"="street_side"](area.searchArea);` `nwr["amenity"="recycling"](area.searchArea);` `nwr["recycling_type"="container"](area.searchArea);` `) -> .parking;` `node["highway"="give_way"](around.shop:100)(around.bridge:50)(around.water:50)(around.crossing:50)(around.parking:50);` `out;` `>;` `out skel;` We get one result: ![[pic11.png]] The place we're looking for: `39.46218966002234, -3.604804126177369` ![[pic12.png]] ### Example #3: Combination of methods ^49e0eb The photo was taken in Germany, in the city of Hannover. We need to find the place where the photo was taken. ![[Street View 360-3-edit.png]] Having explored the area and the most frequently marked objects on OSM we highlight them in the photo. We can see a bicycle parking, a traffic light, recycling containers, a waste basket, a street lamp, tramlines, and another bicycle parking in the distance. Let the traffic light be a key element: ![[Street View 360-3-edit-1.png]] Let's set the city of Hannover as the search region: `{{geocodeArea:Hannover}} -> .searchArea;` OSM tags of tramlines may vary, so let's list the possible variants. Let's get all tramlines in the selected region and write this data to the `.railroad` set: `(` `nwr["electrified"="contact_line"](area.searchArea);` `nwr["railway"="tram"](area.searchArea);` `) -> .railroad;` Let's combine data about street lamp, waste basket, recycling container and nearby bicycle parking into one set. OSM tags for some objects may be different, so let's list possible variants. - get all street lamps in the selected region and put this data into the `.lamp` set. - get all the waste baskets that are located within 5 meters from the found street lamps and put this data into the `.lamp_basket` set. - get all the recycling containers that are within a radius of 10 meters from the found baskets and write this data to the `.lamp_basket_recycling` set. - get all the bicycle parking that are within 10 meters radius from the found containers. Let's write the obtained data to the `.lamp_basket_recycling_parking` set: `(` `(` `nwr["street_lamp"="yes"](area.searchArea);` `nwr["light_source"="lantern"](area.searchArea);` `nwr["highway"="street_lamp"](area.searchArea);` `) -> .lamp;` `nwr["amenity"="waste_basket"](around.lamp:5) -> .lamp_basket;` `(` `nwr["amenity"="recycling"](around.lamp_basket:10);` `nwr["recycling_type"="container"](around.lamp_basket:10);` `) -> .lamp_basket_recycling;` `nwr["amenity"="bicycle_parking"](around.lamp_basket_recycling:10);` `) -> .lamp_basket_recycling_parking;` Get all the bicycle parkings in the selected region and write this data to the `.parking` set: `nwr["amenity"="bicycle_parking"](area.searchArea) -> .parking;` Let's find such traffic lights, which are simultaneously located in the radius: - 30 meters from tramlines - 60 meters from bicycle parking with nearby waste baskets, recycling containers and street lamps. - 30 meters from bicycle parkings `node["highway"="traffic_signals"](around.railroad:30)(around.lamp_basket_recycling_parking:60)(around.parking:30);` `out;` `>;` `out skel;` We get the following query: `{{geocodeArea:Hannover}} -> .searchArea;` `(` `nwr["electrified"="contact_line"](area.searchArea);` `nwr["railway"="tram"](area.searchArea);` `) -> .railroad;` `(` `(` `nwr["street_lamp"="yes"](area.searchArea);` `nwr["light_source"="lantern"](area.searchArea);` `nwr["highway"="street_lamp"](area.searchArea);` `) -> .lamp;` `nwr["amenity"="waste_basket"](around.lamp:5) -> .lamp_basket;` `(` `nwr["amenity"="recycling"](around.lamp_basket:10);` `nwr["recycling_type"="container"](around.lamp_basket:10);` `) -> .lamp_basket_recycling;` `nwr["amenity"="bicycle_parking"](around.lamp_basket_recycling:10);` `) -> .lamp_basket_recycling_parking;` `nwr["amenity"="bicycle_parking"](area.searchArea) -> .parking;` `node["highway"="traffic_signals"](around.railroad:30)(around.lamp_basket_recycling_parking:60)(around.parking:30);` `out;` `>;` `out skel;` We get several results, each of which needs to be verified: ![[pic9.png]] The place we're looking for: `52.345014405539914, 9.822337721802075` ![[pic10.png]]