OpenStreetMap (OSM) is aA collaborative project to create a freely available, yet editable map of the world. In order to utilize this OSM in an offline environment, we will summarize the procedures for building a tile server and a routing server for the OSM. However, Ubuntu 22.04 LTS shall be the target of the environment build. Other environments (Ubuntu 20.04 LTS, Ubuntu 18.04 LTS, etc.) can be built using Switch2OSM (https://switch2osm.org/serving-tiles/(Please refer to the official page.)
This guideline also has been confirmed operation on AWS Graviton G5g host.
Tile server for OSM
Use switch2OSM（https://switch2osm.org/serving-tiles/(1) to build the environment.
Environment Setup (Ubuntu 22.04 LTS)
Install required software
| sudo add-apt-repository multiverse sudo apt updatesudo apt install screen locate libapache2-mod-tile renderd git tar unzip wget bzip2 apache2 lua5.1 mapnik-utils python3-mapnik \ python3-psycopg2 python3-yaml gdal-bin npm fonts-noto-cjk fonts-noto-hinted fonts-noto-unhinted fonts-unifont fonts-hanazono postgresql \ postgresql-contrib postgis postgresql-14-postgis-3 postgresql-14-postgis-3-scripts osm2pgsql net-tools curl libboost-all-dev \ build-essential autoconf libtool libxml2-dev libgeos-dev libgeos++-dev libpq-dev libbz2-dev libproj-dev munin-node munin protobuf-c-compiler \ libfreetype6-dev libtiff5-dev libicu-dev libgdal-dev libcairo-dev libcairomm-1.0-dev apache2-dev libagg-dev liblua5.2-dev liblua5.1-dev make \ cmake g++ libboost-dev libboost-system-dev libboost-filesystem-dev libexpat1-dev zlib1g-dev libbz2-dev libpq-dev libgeos-dev libgeos++-dev \ libproj-dev lua5.2 liblua5.2-dev autoconf libtool libxml2-dev libbz2-dev libgeos-dev libgeos++-dev libproj-dev gdal-bin libmapnik-dev mapnik-utils node-gyp nodejs |
|---|
Create postgis database
Specify the ubuntu login user name in <user name> and create the gis database.
| sudo -u postgres -i createuser <user name> createdb -E UTF8 -O <user name> gis |
|---|
Next, the necessary settings are implemented for the gis database.
psql gis=# CREATE EXTENSION postgis; gis=# CREATE EXTENSION hstore; gis=# ALTER TABLE geometry_columns OWNER TO <user name>; gis=# ALTER TABLE spatial_ref_sys OWNER TO <user name>;
exit |
|---|
Installation of osm2pgsql
Various tools exist to import and manage OpenStreetMap data into a database, but the common "osm2pgsql" is used.
| mkdir ~/src cd ~/src git clone git://github.com/openstreetmap/osm2pgsql.git cd osm2pgsql git checkout 0a1a458eb91ca613e4a2a9eb60d311a8f4f770d3 |
|---|
Compile osm2pgsql.
| mkdir build && cd build cmake .. make sudo make install |
|---|
Installing Mapnik
Check that mapnik has been installed correctly; if . If no errors occur on import, that's good.
| python >>> import mapnik >>> >>> quit() |
|---|
Install and render mod_tile
Install mod_tile and rendered. "mod_tile" is an Apache module that handles requests for tiles. "renderd" is a daemon that actually renders tiles when requested by "mod_tile". Compile the mod_tile source code as follows. Regarding "renderd" setup, it will be introduced in the Configure "renderd" section.
| cd ~/src git clone -b switch2osm https://github.com/SomeoneElseOSM/mod_tile.git cd mod_tile git checkout dd3c843697cfef8100c05312237d1d63ee44ffde ./autogen.sh ./configure make sudo make install sudo make install-mod_tile sudo ldconfig |
|---|
Style Sheet Structure
| cd ~/src git |
|---|
| clone https://github.com/gravitystorm/openstreetmap-carto.git |
|---|
| cd openstreetmap-carto git checkout a609fad9b35a96c3d81b761ba45f4c26f7719cc9 |
|---|
"carto -v" should display the version.
| sudo npm install -g carto carto -v carto project.mml > mapnik.xml |
|---|
Here, if the npm installation does not proceed, you can may need to configure the a proxy as follows.
Loading Map Data
Get data from https://download.geofabrik.de/
Only a small amount of test data (Nevada, USA) is loaded from Other map data can be downloaded, but for map data with large amounts of data, the download and subsequent processing is very slow for large amount of data. Decide the specific dataset you would like to use by yourself.
| mkdir ~/data cd ~/data wget https://download.geofabrik.de/north-america/us/nevada-latest.osm.pbf |
|---|
The following command inserts OpenStreetMap data into the database. This step is very disk I/O intensive.
-d option specifies the database to use.
-c option is the amount of memory allocated.
--number-processes means the number of CPU cores to use.
| osm2pgsql -d gis --create --slim -G --hstore --tag-transform-script ~/src/openstreetmap-carto/openstreetmap-carto.lua \ -C 2500 --number-processes 1 -S ~/src/openstreetmap-carto/openstreetmap-carto.style ~/data/nevada-latest.osm.pbf |
|---|
Most of the data used to create the map is taken directly from the OpenStreetMap data file downloaded above, but some shapefiles, such as low-zoom country boundaries, are still required. This process involves a considerable download and may take some time. The script downloads the data to the "data" directory under "openstreetmap-carto".
| cd ~/src/openstreetmap-carto/ scripts/get-external-data.py |
|---|
If the download does not proceed here, you can may probably need to set the proxy as follows.
| set HTTP_PROXY= |
|---|
| url_to_http_proxy set HTTPS_PROXY= |
|---|
| url_to_https_proxy |
|---|
Web server setup
Configure "renderd"
| Anchor | ||||
|---|---|---|---|---|
|
The "renderd" configuration file is "/usr/local/etc/renderd.conf".
| sudo vi /usr/local/etc/renderd.conf # modify following XML=/home/<user name>/src/openstreetmap-carto/mapnik.xml |
|---|
Apache Configuration
| sudo mkdir /var/lib/mod_tile sudo chown <user name> /var/lib/mod_tile sudo mkdir /var/run/renderd sudo chown <user name> /var/run/renderd |
|---|
Configure apache2 to load "mod_tile".
| sudo vi /etc/apache2/conf-available/mod_tile.conf # add following LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so |
|---|
Register mod_tile in /etc/apache2/conf-enable.
| sudo a2enconf mod_tile |
|---|
Next, add the following between the "ServerAdmin" and "DocumentRoot" lines
| sudo vi /etc/apache2/sites-available/000-default.conf # add following LoadTileConfigFile /usr/local/etc/renderd.conf ModTileRenderdSocketName /var/run/renderd/renderd.sock # Timeout before giving up for a tile to be rendered ModTileRequestTimeout 0 # Timeout before giving up for a tile to be rendered that is otherwise missing ModTileMissingRequestTimeout 30 |
|---|
Reload apache2 twice
| sudo service apache2 reload sudo service apache2 reload |
|---|
...
Confirm Operations
Check if the map is displayed in a web browser using the server IP on which the environment was built.
The following command executes the map data rendering process and checks if the map data is reflected on the Web browser.
| renderd -f -c /usr/local/etc/renderd.conf |
|---|
Once the map is displayed, navigate to the downloaded area (Nevada, USA) in your browser, and the map should reflect the map data, including roads and buildings.
Also, "~/src/mod_tile/extra/sample_leaflet.html".You can copy it under "/var/www/html" and display the map in your web browser. However,Confirmed to work only with firefox。. *Firefox has been confirmed working.
*The following command can be used to check the rendering logs.
| tail -f /var/log/syslog | grep " TILE" |
|---|
If the above procedure is confirmed to work, set up rendering processing in the background according to the following procedure.
| vi ~/src/mod_tile/debian/renderd.init # modify following RUNASUSER=<user name> |
|---|
| sudo cp ~/src/mod_tile/debian/renderd.init /etc/init.d/renderd sudo chmod u+x /etc/init.d/renderd sudo cp ~/src/mod_tile/debian/renderd.service /lib/systemd/system/ sudo /etc/init.d/renderd start sudo systemctl enable renderd |
|---|
Routing server for OSM
OSRM (Open Source Routing Machine) is a free routing application.
However, operation on EC2 has not been confirmed
Environment construction (Ubuntu 18.04 LTS)
Install OSRM from the following and build the environment.
https://github.com/Project-OSRM/osrm-backend.git
The following sites are helpful in building the environment.
- Ubuntu20.04 or 18.04 https://www.linuxbabe.com/ubuntu/install-osrm-ubuntu-20-04-open-source-routing-machine
- Ubuntu16.04 https://benjaminberhault.com//post/2018/12/08/set-up-an-osrm-server-on-ubuntu.html
Install the necessary software and compile osrm.
| sudo apt update sudo apt install build-essential git cmake pkg-config doxygen libboost-all-dev libtbb-dev lua5.2 \ liblua5.2-dev libluabind-dev libstxxl-dev libstxxl1v5 libxml2 libxml2-dev libosmpbf-dev \ libbz2-dev libzip-dev libprotobuf-dev |
|---|
| mkdir ~/osrm cd ~/osrm git clone https://github.com/Project-OSRM/osrm-backend.git -b 5.26 mkdir build cd build cmake ~/osrm/osrm-backend/ make sudo make install |
|---|
Next, create the OSRM routing data. If you have already finished downloading the data, simply copy it to "~/osrm/osrm-backend".
| wget -c https://download.geofabrik.de/north-america/us/nevada-latest.osm.pbf -P ~/osrm/osrm-backend |
|---|
Verify that TCP port 5000 is Listen listening with the osrm-routed command.
| cd ~/osrm/osrm-backend osrm-extract nevada-latest.osm.pbf --threads=10 osrm-partition nevada-latest.osrm osrm-customize nevada-latest.osrm osrm-routed --algorithm=MLD nevada-latest.osrm [info] starting up engines, v5.27.0 [info] Threads: 2 [info] IP address: 0.0.0.0 [info] IP port: 5000 [info] http 1.1 compression handled by zlib version 1.2.11 [info] Listening on; 0.0.0.0:5000 [info] running and waiting for requests Ctrl+c to exit |
|---|
If the above is confirmed to work, set this process to run in the background.
| sudo vi /etc/systemd/system/osrm-routed.service # add following[Unit] Description=Open Source Routing Machine Wants=network-online.target After=network.target network-online.target [Service] ExecStart=/usr/local/bin/osrm-routed --algorithm=MLD /home/<usr name>/osrm/osrm-backend/nevada-latest.osrm User=<user name> Group=<user group> Restart=always RestartSec=5s [Install] WantedBy=multi-user.target |
|---|
In "osrm-routed.service" above, <user name> and <user group> should match ~/osrm/osrm-backend.
| sudo systemctl start osrm-routed sudo systemctl enable osrm-routed |
|---|
Trouble Shooting
If the versions of Cmake and Boost do not match
What to do when an error occurs during osrm cmake
| cmake ~/osrm/osrm-backend/ Boost version: 1.65.1 Boost include path: /usr/include Detected version of Boost is too old. Requested version was 1.70 (or near |
|---|
Can be handled by updating cmake and boost versions
mkdir ~/tmp Upgrade #cmake version cd ~/tmp git clone https://gitlab.kitware.com/cmake/cmake.git Upgrade #boost version cd ~/tmp git clone https://github.com/boostorg/boost.git cd boost |
|---|
mkdir ~/tmp Upgrade #cmake version cd ~/tmp git clone https://gitlab.kitware.com/cmake/cmake.git Upgrade #boost version cd ~/tmp git clone https://github.com/boostorg/boost.git cd boost |