Tile-Generator

C++ toolset for generating optimized vector map tiles from OpenStreetMap PBF files, targeting ESP32-based GPS navigators.

Features

• Sparse Index (bitmap + rank): O(1) tile lookup over the bounding box. A 1-bit-per-cell coverage bitmap plus a popcount rank table locates each compact 8-byte entry — empty cells (sea, gaps) resolve with a single small read instead of wasting an index slot.
• High-Performance C++ Engine: OSM PBF parsing and tile generation using GEOS, GDAL, and Libosmium.
• Efficient Binary Format: Packed NPK2 containers with Delta+ZigZag+VarInt coordinate encoding.
• Memory-Mapped Storage: Uses mmap for feature storage, allowing processing of large PBF files with minimal RAM.
• Multi-threaded Processing: Parallel tile generation leveraging all available CPU cores.
• Z-Order Management: 4-pass rendering pipeline supported in binary format.

Generation Process

The generator operates in multiple passes to ensure topological consistency and optimal packaging:

1. Pass 1 (Relations): Scans PBF for administrative boundaries and water multipolygons.
2. Pass 2 (Features): Extracts nodes and ways, applying semantic filtering and layer assignment.
3. Pass 3 (Water): Integrates global water polygons from external Shapefiles (using GDAL/OGR).
4. Pass 4 (Tiles): Parallel clipping, simplification (GEOS), and NPK2 packaging.

Dependencies & Installation

Required Libraries

• Libosmium: PBF parsing.
• GEOS: Geometry operations (clipping, simplification, unions).
• GDAL/OGR: Shapefile support for ocean water polygons.
• Protozero: Low-level OSM data handling.
• C++17 Compiler: GCC 9+ or Clang.

Installation (Ubuntu/Debian)

sudo apt-get update
sudo apt-get install -y build-essential cmake \
    libosmium2-dev libgeos-dev libgdal-dev \
    libbz2-dev zlib1g-dev libexpat1-dev

Compilation

mkdir build && cd build
cmake ..
make -j$(nproc)

Usage

./nav_generator <osm_pbf_file> <output_dir> <features_json> [--zoom min-max]

Example

./nav_generator andorra-latest.osm.pbf ./NAVMAP features.json --zoom 10-15

Ocean Water Polygons (optional)

Download the pre-computed water polygons shapefile (once, ~540 MB):

wget https://osmdata.openstreetmap.de/download/water-polygons-split-4326.zip
unzip water-polygons-split-4326.zip

Generate with oceans:

./nav_generator input.pbf output features.json --zoom 6-17 \
    --water-shp water-polygons-split-4326/water_polygons.shp

The generator reads the PBF bounding box and only loads water polygons intersecting the extract area. See WATER.md for details.

Visualization & Debugging

The project includes a Python-based simulator to validate maps before flashing them to the ESP32.

Prerequisites

• Python 3.x
• Pygame: pip install pygame

Usage

python3 tile_viewer.py <output_dir> --lat <latitude> --lon <longitude> --config features.json

Map controls

Key / Action	Effect
Arrow keys / Mouse drag	Pan map
[ / ]	Zoom out / in
Scroll wheel	Zoom out / in
B	Toggle background color (white/black)
F	Toggle polygon fill
G	Toggle tile grid
S / L	Toggle stats / legend panels
P	Cycle routing profile (WALK → BIKE → CAR)
Q / Escape	Quit

A* routing overlay

If a ROUTE/ directory with profile subfolders (CAR/, BIKE/, WALK/) exists alongside the nav dir (generated by route_generator), the viewer supports interactive routing. The active profile defaults to WALK and is switchable in-app:

Action	Effect
Left-click on map	Set route origin (green marker)
Right-click on map	Set route destination (red marker) + compute A* route (blue track)
Route button / P	Cycle profile WALK → BIKE → CAR and recompute
R	Clear route and reload tiles

On each right-click (or profile change) the viewer loads every cell in the bounding box covering origin and destination (+1 cell margin) and runs A* over that subgraph, honouring turn restrictions and the turn penalty. The sidebar shows the map info on the left column and the route summary (active profile, origin/destination coordinates, nodes + distance in km, A* time) on the right column.

See docs/tile_viewer.md for the full routing usage and CLI flags (--route-dir, --route-profile).

Route Generator

route_generator builds A* routing graph files from the same OSM PBF. It runs independently from tile generation — use it with vector tiles, PNG tiles, or any map source.

A single run generates all three routing profiles at once:

ROUTE/CAR/ROUTE.bin
ROUTE/BIKE/ROUTE.bin
ROUTE/WALK/ROUTE.bin

Usage

route_generator <input.pbf> <output_dir>

Internally the graph is partitioned into 0.05°×0.05° subcells with an index — readers load only the cells needed for the route on-demand.

Example

# Build tile generator and route generator
cd build && cmake .. && make -j$(nproc)

# Generate all three routing profiles for Andorra
./route_generator andorra-251227.osm.pbf .
# → ./ROUTE/CAR/ROUTE.bin
# → ./ROUTE/BIKE/ROUTE.bin
# → ./ROUTE/WALK/ROUTE.bin

# Copy to SD card alongside tiles
rsync -av NAVMAP/ /media/sdcard/NAVMAP/
rsync -av ROUTE/  /media/sdcard/ROUTE/

For the full binary format and profile speed tables see docs/route_generator.md.

Docker

The image carries the full toolchain, so the first container run compiles both binaries into build/ when they are missing. The repository is mounted at its own path: every input and output stays exactly where it is today. Run the commands from the repository root.

docker compose build
docker compose run --rm generator nav_generator <input.pbf> <output_dir> <features_json> [--zoom min-max] [--water-shp <path>]
docker compose run --rm generator route_generator <input.pbf> <output_dir>

With Compose v1 the command is docker-compose; the file and the arguments are the same.

Web service

The web service draws the area on a map and runs the whole chain for it — download, merge, clip and the generators — so no osmium command has to be typed. It listens on 127.0.0.1:8080 only.

docker compose build web          # once, or whenever the Dockerfile changes
docker compose up -d web          # serves the page on 127.0.0.1:8080
docker compose logs -f web        # follow the service log
docker compose restart web        # after editing web/*.py
docker compose down

A change under web/static/ needs no restart (the browser picks it up on reload). If the daemon answers permission denied while trying to connect to the Docker daemon socket, run newgrp docker once or prefix the commands with sg docker -c "...".

See docs/web_service.md for the workflow on the page, the cache rules, the API and where the figures and the logs end up.

Viewer

The viewer service opens tile_viewer.py on the host display through the mounted X11 socket:

xhost +si:localuser:"$USER"
docker compose run --rm viewer python3 tile_viewer.py <nav_dir> --lat <lat> --lon <lon> [--config <features_json>]
xhost -si:localuser:"$USER"

Data sources

• Map data: © OpenStreetMap contributors, under the Open Database License.
• Region index and PBF extracts from Geofabrik.
• The map is drawn with Leaflet (BSD-2-Clause), vendored in web/static/vendor/.
• The base map is Natural Earth 1:10m (public domain), vendored as web/static/basemap.geojson; no setup step builds it, see docs/basemap.md to regenerate it.

Internal Format Details

• Tile container: NPK2 (Sparse index: coverage bitmap + rank table + compact 8B entries, O(1) lookup) — docs/bin_tile_format.md
• Tile internal format: NAV1 (Geometry + Text labels), 6-byte tile header
• Color palette: per-pack global RGB565 table; features store a 1-byte color index
• Feature header: 8 fixed bytes + varint coord_count/payload_size
• Coordinates: Web Mercator, 12-bit tile-relative space (0-4096)
• Routing graph: ROUTE.bin (header 32B + index 20B/cell + nodes 8B + edges 12B interleaved per cell; node coords as int16 cell-relative offsets; appended turn-restriction table via_node/in_edge/out_edge; surface bits in edge flags) — docs/route_generator.md

---

Author

Jordi Gauchía (jgauchia@jgauchia.com)