Commit Graph

244 Commits

Author SHA1 Message Date
Clement Sam
52a80b2c67 cmd/storagenode: use custom storagenode-base image
Updates https://github.com/storj/storj/issues/4489

Change-Id: I3585b50df598c69e58137f61b6a272dbd077c5be
2022-03-15 14:22:35 +00:00
Egon Elbre
dc0f7b5f77 storagenode,web/storagenode: use go:embed for assets
Go can now directly embed files without relying on external tools.
This makes code use go:embed and avoid the external tooling.

go:embed requires files to be present in the embedded directory,
hence we need to add .keep to "dist" folder. We also add one to
public/.keep, such that it won't be deleted when building storagenode.

Change-Id: I8bef81236be6829ed37ed4c16ef693677b93a631
2022-03-11 16:01:28 +02:00
Egon Elbre
28c9403702 multinode,web/multinode: use go:embed for assets
Go can now directly embed files without relying on external tools.
This makes code use go:embed and avoid the external tooling.

go:embed requires files to be present in the embedded directory,
hence we need to add .keep to "dist" folder. We also add one to
public/.keep, such that it won't be deleted when building multinode.

Change-Id: I53ac3d5ac76e44f740d95221acf0da99fc256d42
2022-03-11 11:10:37 +02:00
Clement Sam
26b722e414 Makefile: create manifest for storagenode-base image
Change-Id: I7c0c920f745717965800b6019fc9fba07b8479b6
2022-03-08 13:30:58 +00:00
Clement Sam
15a1428828 {cmd/storagenode,Makefile}: add storagenode base image Dockerfile
Having the storagenode and storagenode-updater processes in one container
requires a process manager to properly handle the individual processes.

Using a process manager like supervisord requires that you package
supervisord and it configuration in the image, along with the storagenode
and storagenode-updater binaries.

Installing supervisord requires that we run apk to install it and its
dependencies at build time which makes it difficult to build multi-platoform
images; executing apk forces a requirement of the build system to run
foreign architechtures.

This change adds a dockerfile which will be used to build the base image
for the storagenode and has supervisord packaged. The base image will be
built manually using docker buildx, with QEMU binfmt support.

Updates https://github.com/storj/storj/issues/4489

Change-Id: I33f8f01398a7207bca08d8a4a43f4ed56b6a2473
2022-03-04 14:34:58 +00:00
Sean Harvey
d4a6524673 Makefile: use arm64 executable for arm64v8 storagenode images
This change fixes regression (one line of 7e63afb) that overwrote the
previous fix (10d7a63).

Change-Id: I892b0c0445ac0fbe6eb30c131edc96fd1d33734d
2022-03-02 14:30:29 +01:00
Stefan Benten
f89e030c3f final touches
Change-Id: I8fcc986f71f592507dc3f35ce4352647da46d461
2022-02-21 13:08:48 +02:00
Stefan Benten
76be9e6efd cmd/multinode,Makefile: build docker image for multinode dashboard
Closes #4547

We do not build an docker image for the multinode dashboard,
which makes monitoring for docker-focused environments harder.
This adds the basic image and ties it into CI/CD.

Change-Id: I14c01a7f1f0019f6f5c1b8fd75dc424fc362b18d
2022-02-21 13:08:48 +02:00
Jeff Wendling
ce3a405644 makefile: remove uplink-image
Change-Id: I3552de12fa99892bcd7e83aa5b0fdddb268f7db5
2022-02-17 16:01:53 -05:00
Jeff Wendling
9061dd309f cmd/uplinkng: become cmd/uplink
Change-Id: If426c32219d32044d715ab6dfa9718807f32cb9f
2022-02-09 17:02:21 +00:00
Clement Sam
7e63afbef6 storagenode: docker image autoupdate binaries
Get storagenode and storagenode-updater binaries during
run of the container to not to release new docker image
on each new version of the storagenode binary.

Fixes https://github.com/storj/storj/issues/4176

Change-Id: I994c4942136a2cc7298eb0346238689eb406ae5b
2022-02-02 11:40:04 +00:00
Ivan Fraixedes
8e80a51b64 Makefile: Set NodeJS version to use in var
Add new variable in the Makefile to specify the used NodeJS version so
it can be referenced wherever else in the file rather than hard-coding
it.

Having in to a variable makes more visible and quicker to find out which
NodeJS version the project uses.

Change-Id: I87abc3b3e5489407638618f8199aec7c2f40ca9f
2022-01-13 08:52:26 +00:00
Stefan Benten
35113634f6 Makefile: do not remove files to keep state clean
Currently the admin UI Makefile target deletes the .gitignore file.
While it being unnecessary to my findings it also causes and dirty
git repository, since files have been modified.
Removing this command lets the build process continue to work as expected
and since we are not commiting any files or assets later on this should
be fine.

Change-Id: Ibc1b7a8e456394ca19ea39dd8389c2ec03f066fc
2022-01-07 20:36:50 +01:00
Stefan Benten
0334465306
Makefile: fix variable used to trigger release drafting (#4319)
Small fix to properly trigger the script.
2021-12-22 13:33:07 +01:00
Stefan Benten
e2481fb510 scripts,Jenkinsfile,Makefile: upload binaries to drafted github release
This change adds a script and the needed build logic to create a draft github release and then upload the created binaries to it in order to make the actual publishing a lot less error prone.
The logic is currently that it only does this for all github tags, but not for every main build/push. This is handled by the checks in the script itself.

Change-Id: Ie172a8e4a97200de901a26a055aa5a8a54b60a2a
2021-12-21 13:55:38 +00:00
Ivan Fraixedes
c10cc302b5
Makefile: Fix target for building Admin UI
Fix the Makefile target for building the Satellite Admin UI.

Change-Id: Id9087c313c451fa68796c9a495704ea3b3728a5c
2021-12-20 12:11:10 +01:00
Ivan Fraixedes
5573ece848 satellite/admin/ui: Migrate to SvelteKit
Migrate the satellite admin UI web app from the Svelte template used to
generate a Svelte App scaffolding to SvelteKit.

There aren't any functional changes in the  application, however, the
commit has a lot because:

1. SvelteKit uses a different directory layout and constraints to it, so
   the files have been moved.
2. The files have changed its formatting due to the new default linter
   configurations that SvelteKit uses.
3. The linter detected some issues with using `object` and `any` types
   in Typescript, so they have been replaced by better general types
   (e.g. Record).

The  migration allows to use the new tooling rather than Rollup
directly, besides that will empower the future of it when it needs more
features (e.g. different routes, etc.).

Change-Id: Ifa6736c13585708337f6c5a59388077b784eaddd
2021-12-17 10:27:13 +00:00
Stefan Benten
6799354748 ci: bump to go 1.17.5
Change-Id: I072bebaac8c12b31bc68ab4058b1991c4f6d5e12
2021-12-13 11:25:22 +01:00
Ivan Fraixedes
cb499cb7db
Makefile: Fix target builds the satellite admin UI
Fix the Makefile target which build the satellite admin UI adding a
missed npm command which must run before the npm run build command.

Change-Id: I96c9b0bd2ad2d909fc680db2b22a6cf13931111a
2021-11-12 13:29:55 +01:00
Ivan Fraixedes
ab425d4248 build: Consider the Admin UI assets
Add into the Makefile and Jenkins the operations to build the Admin UI
assets and lint them.

Change-Id: I687c1adec796fd591664e34eb73a465e53096b58
2021-11-12 08:48:28 +00:00
Artur M. Wolff
785eb93cd7 ci: install gateway@latest instead of @main
Gateway-ST frequent release cycle has been resurrected, which means it's
safer to use the latest release tag in the storj repository's CI now.

Change-Id: I9df1c789a9b9418ba7cceaec9cfec3cc6c448284
2021-11-01 12:16:38 +00:00
Michał Niewrzał
d3a0364f21 Makefile: add uplinkng to build binaries
We want uplinkng binaries easily accessible.

Change-Id: I83e1d90f9413ebd97e1a51d0435548bc0cac8ad8
2021-10-28 09:47:36 +02:00
Egon Elbre
c54bcd63a1 Makefile: bump node to v16
Change-Id: I0587168f373230710c7b50b9aefd34718f4a5b3e
2021-10-14 12:46:28 +00:00
Egon Elbre
db2cb33db9 ci: use correct Go version for the image
Change-Id: Ieed044780a38ebd209662ddacb24ba5e2998a319
2021-09-30 14:36:21 +03:00
Artur M. Wolff
cc9b845a83 ci: switch back to pulling gateway@main
Switch back to pulling gateway@main for continuous integration because
https://review.dev.storj.io/c/storj/gateway/+/5834 was merged. This
change is a partial revert of
https://review.dev.storj.io/c/storj/storj/+/5832.

Change-Id: I0a87f6f18a9a863fe92003b76c830335f9253ced
2021-09-28 15:01:04 +00:00
Egon Elbre
8ef03b0967 ci: cleanup ws before build, fix gateway install
Jenkins uses the same folder for different PR-s. Sometimes other runs do
not cleanup after themselves (e.g. timeout), hence add a cleanWs step to
ensure we delete files in the workspace.

gateway-st introduced a replace directive in go.mod, which does not work
with go install. Hardcode to the last version without the directive.
Using this fix to unblock ci builds.

Change-Id: I5e5d75bf47e30a5a8b6d835867c0c9176f25e08a
2021-09-23 15:57:47 +03:00
Egon Elbre
6e660cecdd Jenkinsfile: test cross-compile and bump deps
Change-Id: I47091de2e80bd96fcada616f75e1db07a59fb0c1
2021-09-16 18:59:31 +03:00
Egon Elbre
6b3718f33e Makefile: skip darwin build for the time being
Mac builds need to be reworked to work with latest Go version.

Change-Id: Ifbe37cf8fb41c5e0d806c518a3be32e093b448b0
2021-09-16 12:53:41 +03:00
Igor
cd973fcb78
Revert "Makefile: disable building storagenode for darwin" (#4186)
This reverts commit d867a7007f.
2021-09-08 19:56:29 +03:00
Egon Elbre
d867a7007f Makefile: disable building storagenode for darwin
Also a few cleanups to ensure it wasm is built when needed.

Change-Id: I7612d969dddbfc1d8589cf00666c1c60d6312140
2021-09-08 13:46:58 +00:00
paul cannon
fda316b461 Makefile: update channel name of #team-data
Avoids confusion when metrics need to be changed or updated.

Change-Id: I1d0bf939bd30cd66128eb04ccee5c4133447bfe9
2021-07-30 09:58:57 -05:00
Artur M. Wolff
10d7a63ab0 Makefile: include arm64 (instead of arm) executables in arm64v8 images
Change-Id: Ic9cd5cd7b052efcb75499c338dd60480f960c4fa
2021-07-30 14:01:48 +02:00
Yaroslav Vorobiov
6db6b76b27 multinode/console: embed web assets
Embed web static files in multinode binary to be able to
release multinode as single binary.
Add make commands to build multinode binary with embeded web
assets.

Change-Id: I348aff7a7d847fae5c021cbf59abc7f892c0df80
2021-07-12 18:51:24 +03:00
Yaroslav Vorobiov
f0d60a6ec2 Jenkins: add multinode UI stage
Change-Id: I77ed791ff9dbb5b046450ab625c1763b34691888
2021-05-21 15:49:31 +03:00
Yaroslav Vorobiov
c08ca361d8 cmd/storj-sim: add multinode process
Change-Id: I7e6c8db045daa87c5d89f49e58fc82453d4024c6
2021-05-14 01:40:13 +03:00
igor gaidaienko
d32ae0459b Revert "storagenode: docker image autoupdate binaries"
This reverts commit 244c488b10.
2021-05-13 14:21:06 +03:00
Yaroslav Vorobiov
244c488b10 storagenode: docker image autoupdate binaries
Get storagenode and storagenode-updater binaries during
run of the container to not to release new docker image
on each new version of the storagenode binary.

Change-Id: Ic0eb4a9c18a98598dfd9b96c1d352c7399496fd2
2021-04-21 17:56:20 +00:00
littleskunk
a6b3a565df jenkins/build: fix file extension for msi package 2021-04-20 17:27:44 +03:00
Ivan Fraixedes
5642a7a8dc docker-compose: Delete deprecated file
Delete the docker-compose file which was sitting in the root of the
repository because we don't use it anymore.
Remove the Makefile targets which were call docker-compose and
referencing the file.

Change-Id: I6060fc7f06cd4b612ecb5d13ff80d67cfbc4e438
2021-04-19 17:12:59 +00:00
Kaloyan Raev
a264a4422b cmd: remove metainfo-migrator and metainfo-verifier
We already merged the multipart-upload branch to main. These two tools
make only sense if we are migrating a satellite from Pointer DB to
Metabase. There is one remaining satellite to migrate, but these tools
should be used from the respective release branch instead of from main.

Removing these tools from main will:
1) Avoid the mistake to use them from the main branch instead of from
the respective release branch.
2) Allow to finally remove any code related to the old Pointer DB.

Change-Id: Ied66098c5d0b8fefeb5d6e92b5e0ef5c6603df5d
2021-04-08 14:50:29 +00:00
Michał Niewrzał
a5bc38b197 build: replace 'multipart-upload' branch occurences
We removed `multipart-upload` branches but we have few leftovers.

Change-Id: I74f637748a248cb286cd190a3a8f895ef85f3940
2021-03-31 11:06:47 +02:00
Egon Elbre
74b243d1ca certificate: move protobuf to storj/storj repository
Change-Id: I4689318304b54121a65e28860430d39d94ac5bd6
2021-03-29 12:59:49 +00:00
Kaloyan Raev
4b14839e14 make: bump-dependencies should update uplink from main branch
Change-Id: I140b15d8d67e1283af820b4a54b86c2b714e82ca
2021-03-29 11:48:39 +00:00
Michał Niewrzał
237782813b Merge remote-tracking branch 'origin/multipart-upload'
Change-Id: If6c5a450b238adab55d1e0dea67d01e5f5768a9f
2021-03-23 09:44:49 +01:00
Ivan Fraixedes
536990dcf6 Jenkins: Run integration test Redis unavailability
Make Jenkins to run the integration tests that verifies that the
satellite can operates when Redis is unavailable.

Change-Id: Idace3ffb84c788f2af2c6e24eec1d63fb40c263a
2021-03-22 09:56:30 +00:00
Michał Niewrzał
fa083a7f05 Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: Ib5ce5965b77b81c254d08c27ab30c7eccefbd4c6
2021-03-17 15:37:17 +01:00
littleskunk
dcdfa10370
jenkins/build: rename release binaries inside zip archive (#4060)
Co-authored-by: Ivan Fraixedes <ivan@fraixed.es>
2021-03-11 12:35:32 +01:00
Michał Niewrzał
67e26aafcd Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I9b183323cb470185be22f7c648bb76917d2e6fca
2021-03-10 08:53:38 +01:00
Stefan Benten
3a533640ce
Makefile: downgrade to go v1.15.7 (#4048) 2021-02-26 17:34:57 +01:00
Michał Niewrzał
d995fb497f Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I367da03351ab80f7343332420490dde9282aa47a
2021-02-23 12:31:31 +01:00