Geocoding Output

Overview

Beyond vector tiles, geolith can build a geocoding index from the same data with --output-format geocoding. It reads Overture places and addresses, and with --osm-pbf it adds named OpenStreetMap settlements, community-mapped venues, and house-number addresses. Four output formats are supported:

The output path defaults to the --output stem with the format's extension — .glx for glx, .tsv for pg-copy, .ndjson for photon and pelias. Override it with --geocoding-output.

geolith \
  --data ./data/overture \
  --osm-pbf ./data/planet-latest.osm.pbf \
  --natural-earth ./data/natural_earth.sqlite \
  --land-polygons ./data/land-polygons \
  --water-polygons ./data/water-polygons \
  --output ./data/planet.pmtiles \
  --output-format geocoding \
  --geocoding-output ./data/planet.glx

--bbox works the same as in tile builds — only features inside the box are indexed. Add --osm-pbf to also index OpenStreetMap data (see OpenStreetMap Data).

What Gets Indexed

OpenStreetMap Data

With --osm-pbf, the pipeline reads the PBF after the Overture files and adds what Overture lacks for geocoding: named settlements, house-number addresses, and venues mapped by the OSM community.

A place with a Wikipedia article takes the importance of the article instead (see Importance).

• Ways are placed at their centroid. Relations are not read.
• name:<lang> tags become multilingual names.
• A record that the admin divisions leave without a locality gets one: the OSM addr:city tag when the record is from OSM, else the nearest named place=city or place=town node within 30 km. The name is added to search_text, so Canara Bank, Hubballi finds the branch in Hubballi. Photon and Nominatim fill the city the same way.
• An OSM venue or address is dropped when an Overture record of the same layer lies within 50 m under a similar name (Jaro-Winkler similarity of the lowercased names of at least 0.9), unless it has a Wikipedia article. Settlements are always kept, because Overture places has none.
• OSM node, way and relation ids overlap, so the record id carries the element type in its top two bits. Pelias ids use the usual node/123 and way/123 source ids, and Photon's osm_id is the plain element id.
• --node-store and --tmpdir apply as in tile builds. With --bbox, the PBF is cut with osmium when it is installed; otherwise records outside the box are filtered after reading.

Importance

Search ranks places by text relevance plus importance, so "London" finds the city before a shop named after it. Importance follows Nominatim:

• With --wikimedia-importance <file>, an OSM place with a wikidata tag takes the importance of its Wikipedia article, from 0 to 1. The value grows with the links to the article: London 0.892, the Eiffel Tower 0.621. Use Nominatim's file (322 MB):

  wget https://nominatim.org/data/wikimedia-importance.csv.gz
  geolith … --output-format geocoding --wikimedia-importance wikimedia-importance.csv.gz
  

• Any other place takes the default of its type, 0.40001 - rank / 75, with Nominatim's search ranks: the values in the table above.
• An Overture place takes the venue default. Its confidence says whether the place exists, not how well known it is, so it stays in the addendum and does not set importance.
• An OSM venue or address with a Wikipedia article is kept even where an Overture record repeats it, because only the OSM record carries the article.
• A place's English name (name:en, or Overture names.common.en) is added to search_text when it differs from the primary name, so "Eiffel Tower" finds "Tour Eiffel".

glx Format

The default format writes one .glx file: the geocoding equivalent of a PMTiles archive. geolith search opens it and answers every Pelias endpoint from it — no database, no import step.

geolith … --output-format geocoding --geocoding-output ./data/india.glx
GLX_FILE=./data/india.glx geolith-search

The file holds:

• A term dictionary and posting lists with BM25 statistics, so search ranks places by text relevance plus importance — every word first, then about three quarters of the words when that finds too few.
• Stored word prefixes for autocomplete: posting lists for prefixes of 4 to 6 characters, and a short list of the most important places for each prefix of 2 or 3 characters.
• Per-place layer, country and position in the posting lists, so the layers, boundary.country and focus-point filters are applied before any place record is read.
• A spatial index: place positions in Hilbert order, grouped into blocks with bounding boxes, for /v1/reverse and focus search.
• Place records in zstd-compressed blocks of 1,024.

The writer sorts and merges on disk, so memory stays bounded at any size. The format and its reader are documented in the Search docs.

System Requirements

A geocoding build keeps its large data on disk, not in memory: the OSM node coordinates, the Overture records used to drop OSM duplicates, the posting lists, every per-place table and the admin boundary polygons. Memory holds bounded buffers and the bounding boxes of the admin boundaries. So a laptop can build a country, and the planet build stays near 5 GB of memory.

Both were measured on an Apple M-series machine with 12 cores and an NVMe SSD. The planet run reused a node store from an earlier run (--node-store), so its time leaves out the node pass, about 1.6 hours more. Its disk is the 102 GB node store, 121 GB of temporary files and output, and the 7.5 GB file of admin boundary polygons, all at the peak. The planet .glx file is 84 GB.

• Disk is the main cost. The OSM node store holds a slot for every node id up to the highest one in the file: 114 GB of slots even for one country, because OSM numbers nodes across the whole planet. It is a sparse file, so only the slots of real nodes take disk: 31 GB for India (measured on APFS). Use a filesystem with sparse files, such as APFS or ext4, and an SSD; on a hard disk the node pass is many times slower.
• The node store is removed after the OSM pass when it lives in the temporary directory. With --node-store <path> it stays, with a .meta.json beside it, so a later run on the same PBF skips the node pass (hours on a planet file). Delete it when you no longer need it.
• Memory. Each stage has a fixed budget: the node page cache (80% of the free memory, at most 4 GB), one 1 GB sort buffer and one 1 GB run of posting lists. The admin boundary index is the one part that grows with the area: about 9 GB for the whole planet.
• Time is mostly reading the OSM file.
• Open files. The build raises its own open-file limit when it can (up to 1,088). If the system does not allow it, it uses fewer dedup buckets.
• Temporary files go to --tmpdir (default: the system temporary directory). Point it at the disk with the most free space.

geolith --data ./overture --osm-pbf ./planet.osm.pbf \
  --output-format geocoding --geocoding-output ./planet.glx \
  --tmpdir /fast-ssd/tmp \
  --node-store /fast-ssd/planet-nodes.bin --node-cache

PostgreSQL COPY Format

pg-copy writes two files: <output>.tsv holding the rows, and <output>.sql holding the schema, for your own Postgres. Load the schema first, then the data. \COPY needs the table to exist. The table is named after the output file stem.

psql -d gazetteer -f ./data/geocoding.sql
psql -d gazetteer -c "\COPY geocoding FROM './data/geocoding.tsv' WITH (FORMAT text, HEADER true)"

The .tsv is PostgreSQL COPY text format, not CSV: columns are tab-separated, \N is NULL, and backslash escapes (\t, \n, \\) appear in text fields. Load it with FORMAT text. HEADER with FORMAT text needs Postgres 15 or later.

The schema creates the postgis extension, because the location index uses ST_MakePoint. The database must have PostGIS available.

Columns

Choosing the text index

--pg-index-flavor selects the index built over search_text. The table, the PostGIS location index, and the layer index are identical either way.

TIN is an inverted text index with BM25 ranking, the ==> operator, and the TINQL query language, which supports phrases, proximity, fuzzy terms, and boosts. It indexes the text column directly rather than a tsvector expression.

TIN is not self-hostable. If you run your own Postgres, pass --pg-index-flavor gin. For local development and CI against TIN-compatible SQL, PlanetScale publishes Lead, an AGPL-licensed extension that accepts the same queries — it is deliberately slow and unsuitable for production.

# self-hosted Postgres
geolith … --output-format geocoding --pg-index-flavor gin

Photon Format

The Photon output is GeoJSON Feature NDJSON — one feature per line, with no array wrapper:

{"type":"Feature","geometry":{"type":"Point","coordinates":[73.8567437,18.5204303]},"properties":{"osm_type":"N","osm_id":123456,"type":"city","name":"Pune","country_code":"IN","country":"India","state":"Maharashtra","city":"Pune","name:mr":"पुणे"}}

Key properties:

• osm_type is N, W, or R for OSM nodes, ways, and relations. Overture records default to N.
• osm_id is the plain OSM element id for OSM records, and geolith's feature hash otherwise.
• type is the layer mapped to a Photon type: house (address), amenity (venue), street, city (locality), county, state, or country.
• name is the primary name, with a name:<lang> property for every multilingual name.
• Admin fields (country, state, county, city, district) and address fields (street, housenumber, postcode) are emitted only when present. Coordinates carry 7 decimals.

Feed the file to Photon's importer to build the Photon index; see the Photon documentation for the import command for your deployment.

Pelias Format

The Pelias output is Elasticsearch bulk NDJSON — pairs of action + document lines compatible with the pelias-model document shape:

{"index":{"_index":"pelias","_id":"overture:venue:08f2a100d388cfff0200724f76a3f4e8"}}
{"source":"overture","layer":"venue","source_id":"08f2a100d388cfff0200724f76a3f4e8","name":{"default":"Blue Bottle Coffee","es":"Cafe Blue Bottle"},"phrase":{"default":"Blue Bottle Coffee","es":"Cafe Blue Bottle"},"center_point":{"lon":-122.4194,"lat":37.7749},"category":["cafe","coffee_shop"],"addendum":{"overture":"{\"confidence\":0.97,...}"}}

Key properties of the document shape:

• _id is the Pelias GID: {source}:{layer}:{source_id} with the original lowercased Overture GERS UUID.
• phrase is a literal copy of name; multilingual names live inside name as per-language keys.
• parent.* admin hierarchy values are arrays (country, country_a, region, county, locality, neighbourhood), with alpha-3 country codes in country_a.
• category is a lowercased array built from basic_category → taxonomy → categories in priority order.
• addendum.overture is a JSON-stringified string (the pelias-model codec convention) carrying confidence, category provenance, and contact fields.
• center_point coordinates are rounded to 6 decimals.
• Empty objects are omitted entirely — never emitted as {}.

Load the result into Elasticsearch with the _bulk API; Pelias then serves the index:

curl -s -H "Content-Type: application/x-ndjson" \
  -XPOST "http://localhost:9200/_bulk" \
  --data-binary @geocoding.ndjson

Buildings Index (Pelias only)

When Overture buildings data is present, the Pelias format additionally writes <stem>-buildings.ndjson targeting a separate overture-buildings index. Buildings use a flat Overture-schema document, not the Pelias document model:

{"index":{"_index":"overture-buildings","_id":"08b2a100d388cfff"}}
{"id":"08b2a100d388cfff","source":"overture","theme":"buildings","type":"building","centroid":{"lat":37.775,"lon":-122.415},"geometry":{"type":"Polygon","coordinates":[...]},"bbox_shape":{"type":"envelope","coordinates":[...]},"class":"residential","height":12.5,"num_floors":4}

• _id is the raw lowercased building ID (no source:layer: prefix).
• centroid is a {lat, lon} object (Elasticsearch geo_point).
• geometry is GeoJSON, stored as geo_shape.
• bbox_shape is an Elasticsearch envelope for fast bounding-box queries.
• Physical attributes (height, num_floors, class, subtype, roof/facade fields) are included when present.

Admin Hierarchy

Every record is enriched with its admin hierarchy by point-in-polygon lookup against Overture divisions/division_area. Put the theme=divisions/type=division_area Parquet files in --data; only areas that intersect --bbox are loaded.

When no country or dependency contains the point (a coastal place can lie outside its country's land polygon but inside its region's), country_code comes from the region, county, locality or neighbourhood that contains it, and country is the name of the country with that code.

• Only class = land areas are used. The maritime variants extend into territorial waters and would place coastal points in the sea area.
• Polygons are used exactly, with every part and every hole, so enclaves resolve to the enclave, not the country around it.
• When two areas of the same level contain a point, the smaller one wins.
• Names are the Overture names.primary value, so they are in the local language (for example Deutschland).
• The names are appended to search_text, so a query such as cafe Paris matches a venue in Paris.
• Addresses that no polygon covers fall back to the Overture country and postal_city fields.

For the whole planet the index holds about 1.26 million polygon parts and needs about 9 GB of memory. Without division_area in --data the pipeline still runs, and the admin fields are empty.

Measured Results (India)

A geolith search instance built from India data was benchmarked against Pelias, Photon, and Nominatim. Method and full tables: docs.search.geolith.app/benchmarks/engines.

The India index holds 5,466,755 rows: Overture places plus 301,576 OSM localities and 151,416 OSM addresses after dedupe.

A single-file .glx geocoding index — the PMTiles equivalent for geocoding — is coming next; see the Search docs.