george.bartolomey/frostfs-locode-db
1
0
Fork 0
forked from TrueCloudLab/frostfs-locode-db
Code Pull requests Activity Actions

[#6] Add automated database updater

Browse source 
A small parser of unece.org "Trade code download page" added to Makefile to fetch last
database's URL to download. Also, the forgejo action added. Action runs every
month to update databases and send PR to Forgejo.

Signed-off-by: George Bartolomey <george@bh4.ru>
This commit is contained in:
George Bartolomey 2024-07-03 16:39:00 +03:00
parent 7a76e91bdd
commit 3feec01b21
Signed by: george.bartolomey
GPG key ID: 35BC54839D73BFAD
2 changed files with 21 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

17
.forgejo/workflows/updatechecker.yml Normal file
View file

@ -0,0 +1,17 @@
on:
schedule:
- cron: "2 0 1 * *"
jobs:
checkupdates:
runs-on: docker
steps:
- uses: https://code.forgejo.org/actions/checkout@v3
- run: make update
- run: |
git config user.name "Snegurochka"
git config user.email "snegurochka@frostfs.info"
git switch -c update-dbs
git add .
git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@git.frostfs.info/TrueCloudLab/frostfs-locode-db
git commit -m "Automatic database update (UN/LOCODE version $(cat tmp/locode-version.txt))" && \
git push origin HEAD:refs/for/master -o topic=automatic-database-update

5
Makefile
View file

@ -63,7 +63,10 @@ data/countries.dat.gz: $(DIRS)
# IGNORE
# See https://unece.org/trade/cefact/UNLOCODE-Download
tmp/locode.csv.zip :$(DIRS)
wget -c https://service.unece.org/trade/locode/loc222csv.zip -O tmp/locode.csv.zip
DOWNLOADURL=$$(wget -O - https://unece.org/trade/cefact/UNLOCODE-Download \
| grep -oP '(?<=href=")\S+loc\d+csv\.zip'); \
echo "$$DOWNLOADURL" | grep -oP '(?<=loc)\d+' > tmp/locode-version.txt; \
wget -c "$$DOWNLOADURL" -O tmp/locode.csv.zip
data/unlocode-SubdivisionCodes.csv.gz: tmp/locode.csv.zip
unzip -p tmp/locode.csv.zip "*SubdivisionCodes.csv" | gzip > data/unlocode-SubdivisionCodes.csv.gz