# Geocoding Output

> Build a one-file .glx geocoding index, or Pelias, Photon and PostgreSQL exports, from Overture places and addresses plus OpenStreetMap settlements, venues and addresses.

## Overview

Beyond vector tiles, geolith can build a geocoding index from the same data with
`--output-format geocoding`. It reads Overture `places` and `addresses`, and with
`--osm-pbf` it adds named OpenStreetMap settlements, community-mapped venues, and
house-number addresses. Four output formats are supported:

| Format | Flag | Output | Consumer |
| --- | --- | --- | --- |
| glx | `--geocoding-format glx` (default) | one `.glx` file | [geolith search](https://search.geolith.app) — serves the file directly |
| PostgreSQL COPY | `--geocoding-format pg-copy` | `.tsv` + `.sql` | your own Postgres — `COPY` |
| Photon | `--geocoding-format photon` | `.ndjson` | Photon geocoder import |
| Pelias | `--geocoding-format pelias` | `.ndjson` | Elasticsearch `_bulk` API |

The output path defaults to the `--output` stem with the format's extension —
`.glx` for glx, `.tsv` for pg-copy, `.ndjson` for photon and pelias. Override it
with `--geocoding-output`.

```bash
geolith \
  --data ./data/overture \
  --osm-pbf ./data/planet-latest.osm.pbf \
  --natural-earth ./data/natural_earth.sqlite \
  --land-polygons ./data/land-polygons \
  --water-polygons ./data/water-polygons \
  --output ./data/planet.pmtiles \
  --output-format geocoding \
  --geocoding-output ./data/planet.glx
```

`--bbox` works the same as in tile builds — only features inside the box are
indexed. Add `--osm-pbf` to also index OpenStreetMap data (see
[OpenStreetMap Data](#openstreetmap-data)).

## What Gets Indexed

| Overture source | Geocoding layer |
| --- | --- |
| `places/place` | `venue` — requires a primary name; carries categories, confidence, and multilingual names |
| `addresses/address` | `address` — requires a street or house number; carries number/street/unit/postcode |
| `buildings/building` | Pelias only — separate flat-schema buildings index (see below) |

## OpenStreetMap Data

With `--osm-pbf`, the pipeline reads the PBF after the Overture files and adds
what Overture lacks for geocoding: named settlements, house-number addresses,
and venues mapped by the OSM community.

| OSM feature | Geocoding layer | Default importance |
| --- | --- | --- |
| `place=city` / `town` / `village`, `suburb` / `hamlet`, `quarter` / `neighbourhood` with a `name` | `locality` | 0.187 / 0.160 / 0.147 / 0.133 / 0.080 |
| `place=state`, `province` / `region` / `county`, `district` / `country` with a `name` | `region` / `region` / `county` / `country` | 0.293 / 0.160 / 0.240 / 0.347 |
| named `amenity`, `shop`, `tourism`, `leisure`, `historic`, `office`, `craft`, `healthcare`, `aeroway=aerodrome`, `railway=station`/`halt` | `venue` (keeps any `addr:*` fields) | 0.00001 |
| any other node or way with `addr:housenumber` or `addr:street` | `address` | 0.00001 |

A place with a Wikipedia article takes the importance of the article instead
(see [Importance](#importance)).

- Ways are placed at their centroid. Relations are not read.
- `name:<lang>` tags become multilingual names.
- A record that the admin divisions leave without a locality gets one: the
  OSM `addr:city` tag when the record is from OSM, else the nearest named
  `place=city` or `place=town` node within 30 km. The name is added to
  `search_text`, so `Canara Bank, Hubballi` finds the branch in Hubballi.
  Photon and Nominatim fill the city the same way.
- An OSM venue or address is dropped when an Overture record of the same
  layer lies within 50 m under a similar name (Jaro-Winkler similarity of the
  lowercased names of at least 0.9), unless it has a Wikipedia article.
  Settlements are always kept, because Overture `places` has none.
- OSM node, way and relation ids overlap, so the record id carries the element
  type in its top two bits. Pelias ids use the usual `node/123` and `way/123`
  source ids, and Photon's `osm_id` is the plain element id.
- `--node-store` and `--tmpdir` apply as in tile builds. With `--bbox`, the
  PBF is cut with `osmium` when it is installed; otherwise records outside the
  box are filtered after reading.

## Importance

Search ranks places by text relevance plus importance, so "London" finds the
city before a shop named after it. Importance follows
[Nominatim](https://nominatim.org/release-docs/latest/customize/Importance/):

- With `--wikimedia-importance <file>`, an OSM place with a `wikidata` tag
  takes the importance of its Wikipedia article, from 0 to 1. The value grows
  with the links to the article: London 0.892, the Eiffel Tower 0.621. Use
  Nominatim's file (322 MB):

  ```bash
  wget https://nominatim.org/data/wikimedia-importance.csv.gz
  geolith … --output-format geocoding --wikimedia-importance wikimedia-importance.csv.gz
  ```

- Any other place takes the default of its type, `0.40001 - rank / 75`, with
  Nominatim's search ranks: the values in the
  [table above](#openstreetmap-data).
- An Overture place takes the venue default. Its `confidence` says whether the
  place exists, not how well known it is, so it stays in the addendum and does
  not set importance.
- An OSM venue or address with a Wikipedia article is kept even where an
  Overture record repeats it, because only the OSM record carries the article.
- A place's English name (`name:en`, or Overture `names.common.en`) is added
  to `search_text` when it differs from the primary name, so "Eiffel Tower"
  finds "Tour Eiffel".

## glx Format

The default format writes one `.glx` file: the geocoding equivalent of a
PMTiles archive. [geolith search](https://docs.search.geolith.app) opens it and
answers every Pelias endpoint from it — no database, no import step.

```bash
geolith … --output-format geocoding --geocoding-output ./data/india.glx
GLX_FILE=./data/india.glx geolith-search
```

The file holds:

- **A term dictionary and posting lists** with BM25 statistics, so search ranks
  places by text relevance plus importance — every word first, then about three
  quarters of the words when that finds too few.
- **Stored word prefixes** for autocomplete: posting lists for prefixes of 4 to
  6 characters, and a short list of the most important places for each prefix
  of 2 or 3 characters.
- **Per-place layer, country and position** in the posting lists, so the
  `layers`, `boundary.country` and focus-point filters are applied before any
  place record is read.
- **A spatial index**: place positions in Hilbert order, grouped into blocks
  with bounding boxes, for `/v1/reverse` and focus search.
- **Place records** in zstd-compressed blocks of 1,024.

The writer sorts and merges on disk, so memory stays bounded at any size. The
format and its reader are documented in the [Search docs](/search/self-hosting/index-file).

## System Requirements

A geocoding build keeps its large data on disk, not in memory: the OSM node
coordinates, the Overture records used to drop OSM duplicates, the posting
lists, every per-place table and the admin boundary polygons. Memory holds
bounded buffers and the bounding boxes of the admin boundaries. So a laptop
can build a country, and the planet build stays near 5 GB of memory.

| Area | Memory (peak) | Free disk (peak) | Time |
| --- | --- | --- | --- |
| Country (India: 6.2 million places) | 4.9 GB | 32.5 GB | 16 min |
| Planet (652 million places) | 5.2 GB | about 231 GB | 7 h 35 min, with a saved node store |

Both were measured on an Apple M-series machine with 12 cores and an NVMe SSD.
The planet run reused a node store from an earlier run (`--node-store`), so
its time leaves out the node pass, about 1.6 hours more. Its disk is the
102 GB node store, 121 GB of temporary files and output, and the 7.5 GB
file of admin boundary polygons, all at the peak.
The planet `.glx` file is 84 GB.

- **Disk is the main cost.** The OSM node store holds a slot for every
  node id up to the highest one in the file: 114 GB of slots even for one
  country, because OSM numbers nodes across the whole planet. It is a
  sparse file, so only the slots of real nodes take disk: 31 GB for India
  (measured on APFS). Use a filesystem with sparse files, such as APFS or
  ext4, and an SSD; on a hard disk the node pass is many times slower.
- **The node store is removed after the OSM pass** when it lives in the
  temporary directory. With `--node-store <path>` it stays, with a
  `.meta.json` beside it, so a later run on the same PBF skips the node
  pass (hours on a planet file). Delete it when you no longer need it.
- **Memory.** Each stage has a fixed budget: the node page cache (80% of
  the free memory, at most 4 GB), one 1 GB sort buffer and one 1 GB run of
  posting lists. The admin boundary index is the one part that grows with
  the area: about 9 GB for the whole planet.
- **Time** is mostly reading the OSM file.
- **Open files.** The build raises its own open-file limit when it can (up
  to 1,088). If the system does not allow it, it uses fewer dedup buckets.
- **Temporary files** go to `--tmpdir` (default: the system temporary
  directory). Point it at the disk with the most free space.

```bash
geolith --data ./overture --osm-pbf ./planet.osm.pbf \
  --output-format geocoding --geocoding-output ./planet.glx \
  --tmpdir /fast-ssd/tmp \
  --node-store /fast-ssd/planet-nodes.bin --node-cache
```

## PostgreSQL COPY Format

`pg-copy` writes two files: `<output>.tsv` holding the rows, and
`<output>.sql` holding the schema, for your own Postgres. Load the schema first, then the data. `\COPY`
needs the table to exist. The table is named after the output file stem.

```bash
psql -d gazetteer -f ./data/geocoding.sql
psql -d gazetteer -c "\COPY geocoding FROM './data/geocoding.tsv' WITH (FORMAT text, HEADER true)"
```

The `.tsv` is PostgreSQL COPY **text** format, not CSV: columns are
tab-separated, `\N` is NULL, and backslash escapes (`\t`, `\n`, `\\`) appear in
text fields. Load it with `FORMAT text`. `HEADER` with `FORMAT text` needs
Postgres 15 or later.

The schema creates the `postgis` extension, because the location index uses
`ST_MakePoint`. The database must have PostGIS available.

### Columns

| Column | Type | Notes |
| --- | --- | --- |
| `id` | `BIGINT` | geolith's 64-bit feature hash, stored as signed |
| `name` | `TEXT` | primary name, or the formatted address |
| `search_text` | `TEXT` | name plus street, number and postcode; the column the text index covers |
| `lon`, `lat` | `DOUBLE PRECISION` | WGS84, 7 decimals |
| `layer` | `TEXT` | `venue`, `address`, and with `--osm-pbf` also `locality`, `county`, `region`, `country` |
| `country_code` | `TEXT` | ISO alpha-2 of the containing country (see below) |
| `country`, `region`, `county`, `locality` | `TEXT` | admin hierarchy from `divisions/division_area` (see below) |
| `street`, `postcode` | `TEXT` | addresses only |
| `importance` | `REAL` | 0.0 to 1.0 |
| `names` | `JSONB` | language code to name |

### Choosing the text index

`--pg-index-flavor` selects the index built over `search_text`. The table, the
PostGIS location index, and the layer index are identical either way.

| Flavor | Index | Needs an extension | Where it runs |
| --- | --- | --- | --- |
| `tin` (default) | `USING tin (search_text)` | `tin` | PlanetScale Postgres and Neki |
| `gin` | `USING GIN (to_tsvector('simple', search_text))` | none | any Postgres |

TIN is an inverted text index with BM25 ranking, the `==>` operator, and the
TINQL query language, which supports phrases, proximity, fuzzy terms, and
boosts. It indexes the `text` column directly rather than a `tsvector`
expression.

TIN is not self-hostable. If you run your own Postgres, pass
`--pg-index-flavor gin`. For local development and CI against TIN-compatible
SQL, PlanetScale publishes [Lead](https://github.com/planetscale/lead), an
AGPL-licensed extension that accepts the same queries — it is deliberately slow
and unsuitable for production.

```bash
# self-hosted Postgres
geolith … --output-format geocoding --pg-index-flavor gin
```

## Photon Format

The Photon output is GeoJSON Feature NDJSON — one feature per line, with no
array wrapper:

```json
{"type":"Feature","geometry":{"type":"Point","coordinates":[73.8567437,18.5204303]},"properties":{"osm_type":"N","osm_id":123456,"type":"city","name":"Pune","country_code":"IN","country":"India","state":"Maharashtra","city":"Pune","name:mr":"पुणे"}}
```

Key properties:

- **`osm_type`** is `N`, `W`, or `R` for OSM nodes, ways, and relations.
  Overture records default to `N`.
- **`osm_id`** is the plain OSM element id for OSM records, and geolith's
  feature hash otherwise.
- **`type`** is the layer mapped to a Photon type: `house` (address),
  `amenity` (venue), `street`, `city` (locality), `county`, `state`, or
  `country`.
- **`name`** is the primary name, with a `name:<lang>` property for every
  multilingual name.
- Admin fields (`country`, `state`, `county`, `city`, `district`) and address
  fields (`street`, `housenumber`, `postcode`) are emitted only when present.
  Coordinates carry 7 decimals.

Feed the file to Photon's importer to build the Photon index; see the
[Photon documentation](https://github.com/komoot/photon) for the import command
for your deployment.

## Pelias Format

The Pelias output is Elasticsearch bulk NDJSON — pairs of action + document lines compatible with the [`pelias-model`](https://github.com/pelias/model) document shape:

```json
{"index":{"_index":"pelias","_id":"overture:venue:08f2a100d388cfff0200724f76a3f4e8"}}
{"source":"overture","layer":"venue","source_id":"08f2a100d388cfff0200724f76a3f4e8","name":{"default":"Blue Bottle Coffee","es":"Cafe Blue Bottle"},"phrase":{"default":"Blue Bottle Coffee","es":"Cafe Blue Bottle"},"center_point":{"lon":-122.4194,"lat":37.7749},"category":["cafe","coffee_shop"],"addendum":{"overture":"{\"confidence\":0.97,...}"}}
```

Key properties of the document shape:

- **`_id`** is the Pelias GID: `{source}:{layer}:{source_id}` with the original lowercased Overture GERS UUID.
- **`phrase`** is a literal copy of `name`; multilingual names live inside `name` as per-language keys.
- **`parent.*`** admin hierarchy values are arrays (`country`, `country_a`, `region`, `county`, `locality`, `neighbourhood`), with alpha-3 country codes in `country_a`.
- **`category`** is a lowercased array built from `basic_category` → taxonomy → categories in priority order.
- **`addendum.overture`** is a JSON-stringified string (the `pelias-model` codec convention) carrying confidence, category provenance, and contact fields.
- **`center_point`** coordinates are rounded to 6 decimals.
- Empty objects are omitted entirely — never emitted as `{}`.

Load the result into Elasticsearch with the `_bulk` API; Pelias then serves the index:

```bash
curl -s -H "Content-Type: application/x-ndjson" \
  -XPOST "http://localhost:9200/_bulk" \
  --data-binary @geocoding.ndjson
```

## Buildings Index (Pelias only)

When Overture buildings data is present, the Pelias format additionally writes `<stem>-buildings.ndjson` targeting a separate `overture-buildings` index. Buildings use a flat Overture-schema document, **not** the Pelias document model:

```json
{"index":{"_index":"overture-buildings","_id":"08b2a100d388cfff"}}
{"id":"08b2a100d388cfff","source":"overture","theme":"buildings","type":"building","centroid":{"lat":37.775,"lon":-122.415},"geometry":{"type":"Polygon","coordinates":[...]},"bbox_shape":{"type":"envelope","coordinates":[...]},"class":"residential","height":12.5,"num_floors":4}
```

- **`_id`** is the raw lowercased building ID (no `source:layer:` prefix).
- **`centroid`** is a `{lat, lon}` object (Elasticsearch `geo_point`).
- **`geometry`** is GeoJSON, stored as `geo_shape`.
- **`bbox_shape`** is an Elasticsearch envelope for fast bounding-box queries.
- Physical attributes (`height`, `num_floors`, `class`, `subtype`, roof/facade fields) are included when present.

## Admin Hierarchy

Every record is enriched with its admin hierarchy by point-in-polygon lookup
against Overture `divisions/division_area`. Put the `theme=divisions/type=division_area`
Parquet files in `--data`; only areas that intersect `--bbox` are loaded.

| Overture `subtype` | Record field |
| --- | --- |
| `country` | `country`, `country_code` |
| `dependency` | `country`, `country_code`, only when no country contains the point |
| `region` | `region` |
| `county` | `county` |
| `locality` | `locality` |
| `neighborhood` | `neighbourhood` (Pelias and Photon only; `pg-copy` has no column) |

When no country or dependency contains the point (a coastal place can lie
outside its country's land polygon but inside its region's), `country_code`
comes from the region, county, locality or neighbourhood that contains it, and
`country` is the name of the country with that code.

- Only `class = land` areas are used. The `maritime` variants extend into
  territorial waters and would place coastal points in the sea area.
- Polygons are used exactly, with every part and every hole, so enclaves
  resolve to the enclave, not the country around it.
- When two areas of the same level contain a point, the smaller one wins.
- Names are the Overture `names.primary` value, so they are in the local
  language (for example `Deutschland`).
- The names are appended to `search_text`, so a query such as `cafe Paris`
  matches a venue in Paris.
- Addresses that no polygon covers fall back to the Overture `country` and
  `postal_city` fields.

For the whole planet the index holds about 1.26 million polygon parts and
needs about 9 GB of memory. Without `division_area` in `--data` the pipeline
still runs, and the admin fields are empty.

## Measured Results (India)

A geolith search instance built from India data was benchmarked against Pelias,
Photon, and Nominatim. Method and full tables:
[docs.search.geolith.app/benchmarks/engines](https://docs.search.geolith.app/benchmarks/engines).

| Metric | Result |
| --- | --- |
| Top-1 within 1 km — OSM cities and towns | 96% |
| Top-1 within 1 km — Overture places | 88.5% |
| Autocomplete (top-1 within 1 km) | 74% |
| Search near a point | 91% |
| OSM addresses | 74.5% |
| Load — search / reverse | 562 req/s / 3,392 req/s (100 connections) |

The India index holds 5,466,755 rows: Overture places plus 301,576 OSM
localities and 151,416 OSM addresses after dedupe.

A single-file `.glx` geocoding index — the PMTiles equivalent for geocoding —
is coming next; see the [Search docs](/search/self-hosting/index-file).