[#5] Update README for local build

From now on, we redistribute our data sources with our repository. Need
to highlight the licenses for all data sources used.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
This commit is contained in:
Stanislav Bogatyrev 2023-04-21 17:23:19 +03:00
parent 81c313d251
commit 65531482be

View file

@ -6,27 +6,28 @@
</p> </p>
--- ---
![GitHub release](https://img.shields.io/github/release/TrueCloudLab/frostfs-locode-db.svg)
![GitHub license](https://img.shields.io/github/license/TrueCloudLab/frostfs-locode-db.svg?style=popout)
# Overview # Overview
This repository contains instructions to generate UN/LOCODE database for FrostFS FrostFS uses UN/LOCODE in storage node attributes and storage policies. Inner
and raw representation of it. FrostFS uses UN/LOCODE in storage node attributes ring nodes converts UN/LOCODE into human-readable set of attributes such as
and storage policies. Inner ring nodes converts UN/LOCODE into human-readable continent, country name, etc.
set of attributes such as continent, country name, etc.
This repository tools generate UN/LOCODE database for FrostFS using data from
following sources:
- [UN/LOCODE](https://unece.org/trade/cefact/UNLOCODE-Download) database in CSV
format, licensed under the [ODC Public Domain Dedication and Licence (PDDL)](http://opendatacommons.org/licenses/pddl/1-0/)
- [OpenFlight Airports and
Countries](https://raw.githubusercontent.com/jpatokal/openflights/master/data/)
databases, licensed under the [GNU AGPL-3.0
license](https://github.com/jpatokal/openflights/blob/master/LICENSE)
- [OpenStreetMap](https://www.openstreetmap.org/)® open data, licensed under the [Open Data Commons Open
Database License](https://opendatacommons.org/licenses/odbl/) (ODbL)
# Build # Build
## Prerequisites ## Prerequisites
- Latest [frost] tool(https://github.com/TrueCloudLab/frostfs-node) - Latest [frostfs-cli](https://git.frostfs.info/TrueCloudLab/frostfs-node) tool
- [UN/LOCODE](https://unece.org/trade/cefact/UNLOCODE-Download)
database in CSV format
- [OpenFlight Airports](https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.dat)
database
- [OpenFlight Countries](https://raw.githubusercontent.com/jpatokal/openflights/master/data/countries.dat)
database
## Quick start ## Quick start
@ -40,18 +41,13 @@ $ make
## Building ## Building
First unzip file with GeoJSON continents from this repository. Then run frost-cli command to generate boltDB file.
``` ```
$ gunzip continents.geojson $ frostfs-cli util locode generate --help
```
Then run frost command to generate boltDB file.
```
$ frost util locode generate --help
generate UN/LOCODE database for FrostFS generate UN/LOCODE database for FrostFS
Usage: Usage:
frost util locode generate [flags] frostfs-cli util locode generate [flags]
Flags: Flags:
--airports string Path to OpenFlights airport database (csv) --airports string Path to OpenFlights airport database (csv)
@ -62,20 +58,20 @@ Flags:
--out string Target path for generated database --out string Target path for generated database
--subdiv string Path to UN/LOCODE subdivision database (csv) --subdiv string Path to UN/LOCODE subdivision database (csv)
$ ./frost util locode generate \ $ ./frostfs-cli util locode generate \
--airports airports.dat \ --airports airports.dat \
--continents continents.geojson \ --continents continents.geojson \
--countries countries.dat \ --countries countries.dat \
--in 2022-1\ UNLOCODE\ CodeListPart1.csv,2022-1\ UNLOCODE\ CodeListPart2.csv,2022-1\ UNLOCODE\CodeListPart3.csv \ --in 2022-2\ UNLOCODE\ CodeList.csv \
--subdiv 2020-2\ SubdivisionCodes.csv \ --subdiv 2022-2\ SubdivisionCodes.csv \
--out locode_db --out locode_db
``` ```
**Database generation might take some time!** **Database generation might take some time!**
You can test generated database with `frost`. You can test generated database with `frostfs-cli`.
``` ```
$ frost util locode info --db locode_db --locode 'RU LED' $ frostfs-cli util locode info --db locode_db --locode 'RU LED'
Country: Russia Country: Russia
Location: Saint Petersburg (ex Leningrad) Location: Saint Petersburg (ex Leningrad)
Continent: Europe Continent: Europe