catshop
Minimalistic automated (meta)data catalogue assembly and DCAT-AP compliant serving for static data files.
Star
Fork
Watch
39 4 4
Files
.tangled
config
scripts
shacl
static
.gitignore
.goreleaser.yaml
.pre-commit-config.yaml
assets.go
config.example.yml
docker-compose.yml
fetch_test.go
fetch.go
go.mod
go.sum
index.html
LICENSE
main.go
rdfutils.go
README.md
responseHandlers.go
utils.go
version.go
README.md 
 /\_/\            "setosa: 1"    
( €.€ )       NA
 (>   )>                      ["1998;2001;2010"]
  / \                                                  #0000FF

Catshop (WIP) #

Minimalistic automated (meta)data catalogue assembly and DCAT-AP compliant serving for static data files.

Requirements #

  • Catshop only works with sources that already serve DCAT-AP compliant dataset or catalogue metadata.
  • Datasets can either already minted, or contain a blank node for their URI.
  • Upstream catalogues have to be dereferencable.

Installation #

Docker #

Images are published to Docker Hub for every release. Mount your config and point CONFIG_PATH at it:

docker run -p 8083:8083 -v ./config.yml:/config.yml:ro -e CONFIG_PATH=/config.yml correlaid/catshop:latest

Credentials #

It is recommended to set the minio credentials via the environment variables: CATSHOP_MINIO_ACCESS_KEY and CATSHOP_MINIO_KEY_ID

Modes #

Currently, three modes are supported: reuse, mint and nest_cat

Dataset #

Reuse #

Use reuse for the case of a dataset already having it's data served and its own stable URI.

Mint #

Use mint if you want to serve both the data and the metadata with catshop and mint a URI.

Catalogue #

Nesting a catalogue #

Use nest_cat if upstreams root class already is catalog itself that will then be referenced here.

Contributing #

Setup #

Install pre-commit and golangci-lint, then run once:

pre-commit install

On every commit, the hooks run go mod tidy, gofmt, build, tests and golangci-lint, and check the commit message. Commits directly on main are refused.

Workflow #

  1. Create a branch from main, e.g. git switch -c fix/redirects.
  2. Commit your changes (see the message format below) and push the branch.
  3. Open a pull request into main. CI checks every commit message (run scripts/check-commits.sh main HEAD to check them yourself first) and runs tidy, gofmt, lint, tests and a GoReleaser build.
  4. Merge without squashing (tangled rebases the commits onto main), so each commit message counts for the next version.

Commit messages #

Messages follow Conventional Commits: type: description, with an optional scope (fix(fetch): ...). Reference issues in the footer:

git commit -m "fix: handle redirects across domains" -m "Close #16"

The type decides the next version:

Type Version bump
fix patch (v0.1.0 → v0.1.1)
feat minor (v0.1.0 → v0.2.0)
feat!, fix! (breaking) minor while on v0.x, major from v1.0.0 on
docs, refactor, test, ci, build, chore, perf, style, revert none

Releasing #

Run scripts/release.sh on an up-to-date main. It:

  1. computes the next version from the commits since the last tag,
  2. lists those commits,
  3. runs tests, golangci-lint and govulncheck, so a failing release never gets a tag,
  4. asks for confirmation, then pushes an annotated tag whose message lists the commits (git show v0.2.0 shows the release notes).

For a pre-release, pass a suffix: scripts/release.sh rc.1 tags e.g. v0.2.0-rc.1.

The tag triggers the release pipeline, which runs the same checks again, builds the image with GoReleaser and pushes it to Docker Hub as correlaid/catshop:0.2.0. Final releases also update latest; pre-releases don't.

The running version is shown in the startup log and in /healthz.