This patch suggests small improvements for the previously committed (very nice <3) unit test helper
* Use non-default ports for postgres + cockroach. This would help to run same services for ad-hoc unit test (for IDE development for example)
* Use the same flags for cockroach what we use for unit tests (mem store supposed to make tests faster)
* Use `docker-compose down -v` for removing all networks and annynmous volumes
* Remove connection limit (I have seen related problems in my jenkins runs)
* Omit unused databases to avoid getting failures (eg. omit psql when crdb is used)
* Do not drop cockroach database (full database will be dropped, anyway...): it also makes testing faster
Change-Id: Iadc04f8617a2825ea4f10072bd023a9c86883680
Our linting process depends heavily on custom tools and linting
configuration. To help improve this process for running on local
developer machines, we can run the various tasks in our existing CI
container.
Change-Id: I60407686ce9233cc4f16e3724c5e8d44367aa200
This change attempts to run tests as close to CI as possible. It
introduces a docker-compose file that deploys some supporting
services. While this change makes it possible to run tests locally,
it does not address the core time it takes to run tests end to end.
make test/postgres 1417.87s user 263.38s system 291% cpu 9:37.73 total
Resolves https://github.com/storj/dev-enablement/issues/10
Change-Id: I4d9adc125992c1b1c133e8fe108384fa3c66ff7a
We've had a lot of issues with alpine and currently there's a broken
network issue on alpine for users running on RPI arm32 architechture
which requires a workaround before docker is able to sync time between
the host and the container: https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0\#time64_requirements.
Since we're switching the base image of the storagenode to debian,
it's best to switch the base image of all our docker images to
debian as well for consistency; less drift across them and keeps
the push target consistent.
Change-Id: If3adf7a57dc59f19ef2221b892f340d919798fc5
Indentations are off in the push-storagenode-images target in the makefile which is possibly causing build to fail
Change-Id: Ia25b8f700f49c551e3f201c988747a83e04ad83c
This also fixes the build order. Unfortunately we need
to ensure that the web frontends are built before installing
Go binaries.
Fixes https://github.com/storj/storj/issues/4654
Change-Id: I5d1c83125fd3d1a454d3400b2cbdd44bd3f2250c
We are switching from alpine to debian due to a network issue
introduced in alpine 3.13+ which fails to verify certificates
due to not all armhf boards meet the time64 requirement:
https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0\#time64_requirements
Also, Debian does not have official imagess for arm32v6 architecture
so we are building with arm32v5 arch in the Makefile.
Change-Id: I3660c3f64b7c2b342dd4ccb876af5f4e3036ea9d
Go can now directly embed files without relying on external tools.
This makes code use go:embed and avoid the external tooling.
go:embed requires files to be present in the embedded directory,
hence we need to add .keep to "dist" folder. We also add one to
public/.keep, such that it won't be deleted when building storagenode.
Change-Id: I8bef81236be6829ed37ed4c16ef693677b93a631
Go can now directly embed files without relying on external tools.
This makes code use go:embed and avoid the external tooling.
go:embed requires files to be present in the embedded directory,
hence we need to add .keep to "dist" folder. We also add one to
public/.keep, such that it won't be deleted when building multinode.
Change-Id: I53ac3d5ac76e44f740d95221acf0da99fc256d42
Having the storagenode and storagenode-updater processes in one container
requires a process manager to properly handle the individual processes.
Using a process manager like supervisord requires that you package
supervisord and it configuration in the image, along with the storagenode
and storagenode-updater binaries.
Installing supervisord requires that we run apk to install it and its
dependencies at build time which makes it difficult to build multi-platoform
images; executing apk forces a requirement of the build system to run
foreign architechtures.
This change adds a dockerfile which will be used to build the base image
for the storagenode and has supervisord packaged. The base image will be
built manually using docker buildx, with QEMU binfmt support.
Updates https://github.com/storj/storj/issues/4489
Change-Id: I33f8f01398a7207bca08d8a4a43f4ed56b6a2473
Closes#4547
We do not build an docker image for the multinode dashboard,
which makes monitoring for docker-focused environments harder.
This adds the basic image and ties it into CI/CD.
Change-Id: I14c01a7f1f0019f6f5c1b8fd75dc424fc362b18d
Get storagenode and storagenode-updater binaries during
run of the container to not to release new docker image
on each new version of the storagenode binary.
Fixes https://github.com/storj/storj/issues/4176
Change-Id: I994c4942136a2cc7298eb0346238689eb406ae5b
Add new variable in the Makefile to specify the used NodeJS version so
it can be referenced wherever else in the file rather than hard-coding
it.
Having in to a variable makes more visible and quicker to find out which
NodeJS version the project uses.
Change-Id: I87abc3b3e5489407638618f8199aec7c2f40ca9f
Currently the admin UI Makefile target deletes the .gitignore file.
While it being unnecessary to my findings it also causes and dirty
git repository, since files have been modified.
Removing this command lets the build process continue to work as expected
and since we are not commiting any files or assets later on this should
be fine.
Change-Id: Ibc1b7a8e456394ca19ea39dd8389c2ec03f066fc
This change adds a script and the needed build logic to create a draft github release and then upload the created binaries to it in order to make the actual publishing a lot less error prone.
The logic is currently that it only does this for all github tags, but not for every main build/push. This is handled by the checks in the script itself.
Change-Id: Ie172a8e4a97200de901a26a055aa5a8a54b60a2a
Migrate the satellite admin UI web app from the Svelte template used to
generate a Svelte App scaffolding to SvelteKit.
There aren't any functional changes in the application, however, the
commit has a lot because:
1. SvelteKit uses a different directory layout and constraints to it, so
the files have been moved.
2. The files have changed its formatting due to the new default linter
configurations that SvelteKit uses.
3. The linter detected some issues with using `object` and `any` types
in Typescript, so they have been replaced by better general types
(e.g. Record).
The migration allows to use the new tooling rather than Rollup
directly, besides that will empower the future of it when it needs more
features (e.g. different routes, etc.).
Change-Id: Ifa6736c13585708337f6c5a59388077b784eaddd
Fix the Makefile target which build the satellite admin UI adding a
missed npm command which must run before the npm run build command.
Change-Id: I96c9b0bd2ad2d909fc680db2b22a6cf13931111a
Gateway-ST frequent release cycle has been resurrected, which means it's
safer to use the latest release tag in the storj repository's CI now.
Change-Id: I9df1c789a9b9418ba7cceaec9cfec3cc6c448284
Jenkins uses the same folder for different PR-s. Sometimes other runs do
not cleanup after themselves (e.g. timeout), hence add a cleanWs step to
ensure we delete files in the workspace.
gateway-st introduced a replace directive in go.mod, which does not work
with go install. Hardcode to the last version without the directive.
Using this fix to unblock ci builds.
Change-Id: I5e5d75bf47e30a5a8b6d835867c0c9176f25e08a
Embed web static files in multinode binary to be able to
release multinode as single binary.
Add make commands to build multinode binary with embeded web
assets.
Change-Id: I348aff7a7d847fae5c021cbf59abc7f892c0df80
Get storagenode and storagenode-updater binaries during
run of the container to not to release new docker image
on each new version of the storagenode binary.
Change-Id: Ic0eb4a9c18a98598dfd9b96c1d352c7399496fd2
Delete the docker-compose file which was sitting in the root of the
repository because we don't use it anymore.
Remove the Makefile targets which were call docker-compose and
referencing the file.
Change-Id: I6060fc7f06cd4b612ecb5d13ff80d67cfbc4e438
We already merged the multipart-upload branch to main. These two tools
make only sense if we are migrating a satellite from Pointer DB to
Metabase. There is one remaining satellite to migrate, but these tools
should be used from the respective release branch instead of from main.
Removing these tools from main will:
1) Avoid the mistake to use them from the main branch instead of from
the respective release branch.
2) Allow to finally remove any code related to the old Pointer DB.
Change-Id: Ied66098c5d0b8fefeb5d6e92b5e0ef5c6603df5d
