TrueCloudLab/frostfs-locode-db
15
0
Fork 9
Code Issues 6 Pull requests 1 Releases 2 Activity Actions

[#6] Add automated database updater #10

Merged
realloc merged 1 commit from george.bartolomey/issue6pr into master 2024-07-05 07:35:00 +00:00 AGit
Conversation 0 Commits 1 Files changed 2 +21 -1
2 changed files with 21 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

17
.forgejo/workflows/updatechecker.yml Normal file
View file

@ -0,0 +1,17 @@
on:
schedule:
- cron: "2 0 1 * *"
jobs:
checkupdates:
runs-on: docker
steps:
- uses: https://code.forgejo.org/actions/checkout@v3
- run: make update
- run: |
git config user.name "Snegurochka"
git config user.email "snegurochka@frostfs.info"
git switch -c update-dbs
git add .
git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@git.frostfs.info/TrueCloudLab/frostfs-locode-db
git commit -m "Automatic database update (UN/LOCODE version $(cat tmp/locode-version.txt))" && \
fyrchik marked this conversation as resolved
fyrchik commented 2024-07-04 07:21:43 +00:00
Review
Copy link

How about mentioning having some version info in the commit message?

How about mentioning having some version info in the commit message?
george.bartolomey commented 2024-07-04 08:38:17 +00:00
Review
Copy link

What you do mean by "version"? Date of update or log of changed files?

What you do mean by "version"? Date of update or log of changed files?
fyrchik commented 2024-07-04 10:59:41 +00:00
Review
Copy link

I mean we have 27 December 2023 UN/LOCODE by Country version 2023-2 at the https://unece.org/trade/cefact/UNLOCODE-Download , either date or 2023-2 could serve as a version.

Or 232 in the loc232... filename, it seems related.

Not very important, if it is hard to extract, but IMO is useful.

I mean we have `27 December 2023 UN/LOCODE by Country version 2023-2` at the https://unece.org/trade/cefact/UNLOCODE-Download , either date or `2023-2` could serve as a version. Or `232` in the `loc232...` filename, it seems related. Not very important, if it is hard to extract, but IMO is useful.
george.bartolomey commented 2024-07-04 11:11:27 +00:00
Review
Copy link

I can introduce this by writing loc232 to tmp/version.txt. Also we have OpenFlights which doesn't have any releases. Is there any method to get version of OpenFlights databases?

I can introduce this by writing `loc232` to `tmp/version.txt`. Also we have OpenFlights which doesn't have any releases. Is there any method to get version of OpenFlights databases?
george.bartolomey commented 2024-07-04 11:35:23 +00:00
Review
Copy link

Done.

Done.
git push origin HEAD:refs/for/master -o topic=automatic-database-update
fyrchik marked this conversation as resolved
fyrchik commented 2024-07-04 07:20:52 +00:00
Review
Copy link

Should be either Automatically update databases or Automatic database update

Should be either `Automatically update databases` or `Automatic database update`
fyrchik commented 2024-07-04 13:20:06 +00:00
Review
Copy link

@george.bartolomey please, change this too

@george.bartolomey please, change this too
george.bartolomey commented 2024-07-04 14:51:37 +00:00
Review
Copy link

I changed this already.

I changed this already.
fyrchik commented 2024-07-04 15:05:50 +00:00
Review
Copy link

Oh, right, sorry. Forgejo usually prints outdated if the code in line has changed.

Oh, right, sorry. Forgejo usually prints `outdated` if the code in line has changed.

5
Makefile
View file

@ -63,7 +63,10 @@ data/countries.dat.gz: $(DIRS)
# IGNORE
# See https://unece.org/trade/cefact/UNLOCODE-Download
tmp/locode.csv.zip :$(DIRS)
wget -c https://service.unece.org/trade/locode/loc222csv.zip -O tmp/locode.csv.zip
DOWNLOADURL=$$(wget -O - https://unece.org/trade/cefact/UNLOCODE-Download \
| grep -oP '(?<=href=")\S+loc\d+csv\.zip'); \
echo "$$DOWNLOADURL" | grep -oP '(?<=loc)\d+' > tmp/locode-version.txt; \
wget -c "$$DOWNLOADURL" -O tmp/locode.csv.zip
data/unlocode-SubdivisionCodes.csv.gz: tmp/locode.csv.zip
unzip -p tmp/locode.csv.zip "*SubdivisionCodes.csv" | gzip > data/unlocode-SubdivisionCodes.csv.gz