Commit Graph

265 Commits

Author SHA1 Message Date
Egon Elbre
d867a7007f Makefile: disable building storagenode for darwin
Also a few cleanups to ensure it wasm is built when needed.

Change-Id: I7612d969dddbfc1d8589cf00666c1c60d6312140
2021-09-08 13:46:58 +00:00
paul cannon
fda316b461 Makefile: update channel name of #team-data
Avoids confusion when metrics need to be changed or updated.

Change-Id: I1d0bf939bd30cd66128eb04ccee5c4133447bfe9
2021-07-30 09:58:57 -05:00
Artur M. Wolff
10d7a63ab0 Makefile: include arm64 (instead of arm) executables in arm64v8 images
Change-Id: Ic9cd5cd7b052efcb75499c338dd60480f960c4fa
2021-07-30 14:01:48 +02:00
Yaroslav Vorobiov
6db6b76b27 multinode/console: embed web assets
Embed web static files in multinode binary to be able to
release multinode as single binary.
Add make commands to build multinode binary with embeded web
assets.

Change-Id: I348aff7a7d847fae5c021cbf59abc7f892c0df80
2021-07-12 18:51:24 +03:00
Yaroslav Vorobiov
f0d60a6ec2 Jenkins: add multinode UI stage
Change-Id: I77ed791ff9dbb5b046450ab625c1763b34691888
2021-05-21 15:49:31 +03:00
Yaroslav Vorobiov
c08ca361d8 cmd/storj-sim: add multinode process
Change-Id: I7e6c8db045daa87c5d89f49e58fc82453d4024c6
2021-05-14 01:40:13 +03:00
igor gaidaienko
d32ae0459b Revert "storagenode: docker image autoupdate binaries"
This reverts commit 244c488b10.
2021-05-13 14:21:06 +03:00
Yaroslav Vorobiov
244c488b10 storagenode: docker image autoupdate binaries
Get storagenode and storagenode-updater binaries during
run of the container to not to release new docker image
on each new version of the storagenode binary.

Change-Id: Ic0eb4a9c18a98598dfd9b96c1d352c7399496fd2
2021-04-21 17:56:20 +00:00
littleskunk
a6b3a565df jenkins/build: fix file extension for msi package 2021-04-20 17:27:44 +03:00
Ivan Fraixedes
5642a7a8dc docker-compose: Delete deprecated file
Delete the docker-compose file which was sitting in the root of the
repository because we don't use it anymore.
Remove the Makefile targets which were call docker-compose and
referencing the file.

Change-Id: I6060fc7f06cd4b612ecb5d13ff80d67cfbc4e438
2021-04-19 17:12:59 +00:00
Kaloyan Raev
a264a4422b cmd: remove metainfo-migrator and metainfo-verifier
We already merged the multipart-upload branch to main. These two tools
make only sense if we are migrating a satellite from Pointer DB to
Metabase. There is one remaining satellite to migrate, but these tools
should be used from the respective release branch instead of from main.

Removing these tools from main will:
1) Avoid the mistake to use them from the main branch instead of from
the respective release branch.
2) Allow to finally remove any code related to the old Pointer DB.

Change-Id: Ied66098c5d0b8fefeb5d6e92b5e0ef5c6603df5d
2021-04-08 14:50:29 +00:00
Michał Niewrzał
a5bc38b197 build: replace 'multipart-upload' branch occurences
We removed `multipart-upload` branches but we have few leftovers.

Change-Id: I74f637748a248cb286cd190a3a8f895ef85f3940
2021-03-31 11:06:47 +02:00
Egon Elbre
74b243d1ca certificate: move protobuf to storj/storj repository
Change-Id: I4689318304b54121a65e28860430d39d94ac5bd6
2021-03-29 12:59:49 +00:00
Kaloyan Raev
4b14839e14 make: bump-dependencies should update uplink from main branch
Change-Id: I140b15d8d67e1283af820b4a54b86c2b714e82ca
2021-03-29 11:48:39 +00:00
Michał Niewrzał
237782813b Merge remote-tracking branch 'origin/multipart-upload'
Change-Id: If6c5a450b238adab55d1e0dea67d01e5f5768a9f
2021-03-23 09:44:49 +01:00
Ivan Fraixedes
536990dcf6 Jenkins: Run integration test Redis unavailability
Make Jenkins to run the integration tests that verifies that the
satellite can operates when Redis is unavailable.

Change-Id: Idace3ffb84c788f2af2c6e24eec1d63fb40c263a
2021-03-22 09:56:30 +00:00
Michał Niewrzał
fa083a7f05 Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: Ib5ce5965b77b81c254d08c27ab30c7eccefbd4c6
2021-03-17 15:37:17 +01:00
littleskunk
dcdfa10370
jenkins/build: rename release binaries inside zip archive (#4060)
Co-authored-by: Ivan Fraixedes <ivan@fraixed.es>
2021-03-11 12:35:32 +01:00
Michał Niewrzał
67e26aafcd Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I9b183323cb470185be22f7c648bb76917d2e6fca
2021-03-10 08:53:38 +01:00
Stefan Benten
3a533640ce
Makefile: downgrade to go v1.15.7 (#4048) 2021-02-26 17:34:57 +01:00
Michał Niewrzał
d995fb497f Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I367da03351ab80f7343332420490dde9282aa47a
2021-02-23 12:31:31 +01:00
Stefan Benten
f969e26827
Jenkinsfile, Makefile: bump to go v1.16 (#4041) 2021-02-17 16:54:04 +01:00
Kaloyan Raev
d0612199f0 Merge remote-tracking branch 'origin/main' into multipart-upload
Conflicts:
	go.mod
	go.sum
	satellite/metainfo/config.go
	satellite/metainfo/metainfo_test.go

Change-Id: I95cf3c1d020a7918795b5eec63f36112fdb86749
2021-02-01 14:32:12 +02:00
Jessica Grebenschikov
1f1d9fce58 satellite/console/wasm: add test to confirm wasm size isnt growing
Change-Id: I975a9f8ac3f6b98cc213140fdd7a99557efe14c8
2021-01-21 15:48:49 +00:00
Stefan Benten
ff102d6ffe
Makefile: update to go v1.15.7 (#4026) 2021-01-20 19:08:35 +01:00
Kaloyan Raev
6dff40f5c5 Merge remote-tracking branch 'origin/main' into multipart-upload
Conflicts:
	go.mod
	go.sum
	satellite/metainfo/metainfo.go

Change-Id: Ib5c49f3c911c58319855a171f9ce73657da976d9
2021-01-14 14:33:59 +02:00
Michał Niewrzał
ec88d21a3c Merge 'main' branch.
Change-Id: I6e8162d1a6caf75e89c9f9c9f9522730aebf83ae
2021-01-11 10:26:58 +01:00
Isaac Hess
db6b64c2ba Makefile: Custom gateway location for storj-sim
When running make install-sim we currently create a .build directory and
init a go module there. This change allows a user to pass in a custom
location for this to occur. If nothing is passed in it defaults to
current behavior.

Change-Id: I432dfc7bae412d8a8454a6b8b3dffece84a41147
2021-01-07 13:14:48 -07:00
Michał Niewrzał
ad3e3a38c5 Merge 'main' branch
Change-Id: Ia0db1b1f9ef3e0671d3f2208881b0abc3064e200
2021-01-04 12:13:45 +01:00
Stefan Benten
7f1871b8f1 all: switch from master to main 2020-12-28 22:59:06 +01:00
Michal Niewrzal
9a8959d429 Merge 'master' branch
Change-Id: Iba69ea73ca4d3f1cd4ae94243eaaae033c5324e8
2020-12-22 14:55:57 +01:00
Stefan Benten
d14f2e4164
Makefile,scripts: move from aarch64 to arm64v8 (#4008)
It turns out that alpine dropped support/updates for the aarch64 image.
Instead they have been using the arm64v8 notation for quite a while,
which resulted in breaking our recent aarch64 builds due to missing
dependencies/updates.

Both arches are exactly the same, aarch64 was created originally by Apple
and arm64 by GNU. The backends have been merged by now and the arm64 became
the de facto standard.
2020-12-21 14:50:57 +01:00
Stefan Benten
866ce478bf
build: update node to v14.15.3 (#4007) 2020-12-20 17:20:19 +01:00
Michal Niewrzal
2111740236 Merge 'master' branch
Change-Id: Ib73af0ff3ce0e9a1547b0b9fc55bf88704f6f394
2020-12-18 09:13:24 +01:00
Stefan Benten
e1456bc53f
Makefile: build wasm files only once (#4005) 2020-12-18 01:59:27 +01:00
Michal Niewrzal
57f374af24 Merge 'master' branch
Change-Id: Idf6b10ea7ca94e4d232e6a3b6a38ef2e646ba197
2020-12-15 08:26:53 +01:00
Kaloyan Raev
2bb010e7c5 cmd: remove segment reaper
It was designed to detect and remove zombie segments in the PointerDB.
This tool should be not relevant with the MetabaseDB anymore.

Change-Id: I112552203b1329a5a659f69a0043eb1f8dadb551
2020-12-14 09:36:37 +00:00
Stefan Benten
8fe829d5fd
build: add wasm bits to Dockerfile and bump to go v1.15.6 (#3992) 2020-12-11 02:23:39 +01:00
Kaloyan Raev
cc9e9ee1f5 storj-sim: use gateway from multipart-upload branch
Change-Id: I0886d277b3b757c8b00975a3e95c2d0d1228488b
2020-12-04 15:15:47 +02:00
Michal Niewrzal
7c384c8293 Merge 'master' branch
Change-Id: I1eefd5a56449e577820977d61fa4a22bdd4fc230
2020-11-16 10:02:54 +01:00
Ivan Fraixedes
c2ba5a9905 Makefile: Update Go version security patch
Update Go version to use the last path release 1.15.5 due to a security
fix.
https://groups.google.com/u/1/g/golang-nuts/c/c-ssaaS7RMI/m/5iS6JRtOAwAJ

Change-Id: I748df29e2309408ba1567ebf72652803ee4ec5bc
2020-11-15 00:36:54 +01:00
Stefan Benten
5f840aae6e
build: use go v1.15.4 (#3968) 2020-11-10 17:45:38 +01:00
Jessica Grebenschikov
99c88efbbf scripts/tests: fix gateway tests
Change-Id: I9a23ef08794043ad615066ae5929df9ff3a02d69
2020-10-27 08:21:28 -07:00
Stefan Benten
7161506b68 Makefile: handle msi packages correctly
With the current Makefile, both the msi and exe files get combined into one zip file.
This is not the expected behavior, which this change fixes.
It strips off only .exe from filenames going forward and leaves every other extension intact.

Change-Id: If8132b1427eec7a9e5ebd7ac6b8b3e9d12524080
2020-10-09 13:15:35 +02:00
Kaloyan Raev
bd177bff03 cmd/storj-sim: cleanup gateway setup
Remove usage of --non-interactive flag. It is not provided (and
necessary) by the multitenant S3 gateway anymore.

ACCESS_KEY and SECRET_KEY env vars are not provided anymore as they are
not generated by the multitenant S3 gateway.

Change-Id: I3ecfb92110e31ae13977de3899dad273daae6c1e
2020-10-06 14:22:47 +00:00
Stefan Benten
338ed90876 Makefile: simplify binaries-upload
This change makes the loop to create the zip files a little easier (especially to read).

In addition, it also omits the .exe part for windows zip files, to ensure we can properly
use the versioncontrol URL for both windows and unix rollouts.

Change-Id: I0cbabb2ca61eb3346d97b2071411885383e870f3
2020-10-06 10:35:04 +02:00
Yaroslav
a43afa7ea0 make: fix storagenode-updater binary target
Adds -tags=service when building storagenode-updater which adds
interaction with Windows SVC and Linux systemd to the binary.

Change-Id: I21b4b3e8718348a30ec0453b07135ff3de033a5a
2020-09-30 11:53:11 +00:00
Egon Elbre
d508c4c985 scripts: remove duplicate check config lock
We have configlock_test for checking changes so the separate script and
makefile target are not needed.

Change-Id: I2bbc1c21ad849c9b7ec8bba43c0e11e94e04f6a6
2020-09-29 10:03:34 +00:00
Matt Robinson
84b6b91ee1
build: Go 1.15.2 2020-09-17 07:57:20 +03:00
Matt Robinson
9996aeca8d
build: Go 1.15 (#3927) 2020-09-07 16:01:29 +03:00
Matt Robinson
c548475662
build: Go 1.14.7 (#3926)
Co-authored-by: Egon Elbre <egonelbre@gmail.com>
2020-08-12 12:42:08 +02:00
Egon Elbre
85aff88e7f ci: update to Go 1.14.6
Change-Id: I0926c543612080ecd671d7e3cb67f940258b8ecc
2020-07-20 13:17:49 +00:00
Michal Niewrzal
7e6f9cf5b2 storj: install latest gateway with make install-sim
Change-Id: I961eecaf7763911fee6b54c9e7c83beb84b797b4
2020-06-18 11:26:00 +02:00
Michal Niewrzal
4dd47a4601 storj: add bump-dependencies target
One target to update all dependencies in main module and in testsuite

Change-Id: I2d8512f5fd7f5bb814451988a3feef366b39eee1
2020-06-08 15:51:37 +00:00
Matt Robinson
eced8dcfec
build: Go 1.14.4 (#3904) 2020-06-03 17:35:35 +02:00
Egon Elbre
d9fec04f50 lib: remove the old implementation
The stable uplink library is located at storj.io/uplink.
The stable uplink C library is located at storj.io/uplink-c.

Change-Id: Ia69115d6384b668490efe7e8ab674d7e8104b4f4
2020-05-29 11:36:43 +00:00
Michal Niewrzal
48e67a29a9 cmd/storj-sim: remove old libuplink from storj-sim
Change-Id: I146a2d7947d1f358e7bdd8acba68eb3fee68539d
2020-05-29 08:39:37 +00:00
Matt Robinson
e34937317c
Call the right make target for the segment-reaper (#3860) 2020-04-22 08:34:24 +02:00
Matt Robinson
1e295a48e7
add container image for segment-reaper (#3855) 2020-04-21 17:48:40 +03:00
Michal Niewrzal
75961fe8ce cmd/gateway: remove gateway command from repository and adjust `make
install-sim`

Change-Id: Ie55989ec0d702f6037843a8b6664f4f4f6347446
2020-03-27 07:42:42 +00:00
Egon Elbre
0781a91ff4 linksharing: move to storj.io/linksharing
Change-Id: If3bca64f80de9c244414e2aff28bb8b3bf9e87a2
2020-03-19 16:49:00 +00:00
Kaloyan Raev
4f0bf3fe1d
build: cleanup more gateway targets from Makefile (#3802)
Change-Id: Ia95caa2187b3e9e056a83cbea4230788ed4e8abd

Co-authored-by: Michal Niewrzal <michal@storj.io>
2020-03-16 15:07:52 +01:00
Kaloyan Raev
ce1ae134a0 build: don't push gateway docker images
This is now done from the new storj/gateway repo by another Jenkins job.
Ideally, we should remove everything related to the gateway from this
Makefile, but we cannot do it at the moment due to an all-on-one test
that depends on the gateway.

Change-Id: Ibc722bcee20c9f58b4864583ee3245455b45c7ba
2020-03-11 18:41:19 +00:00
Egon Elbre
46228fee92 cmd/gateway: use proper module name
By using a require for storj.io/storj it will make the import
unambiguous. This means it is possible to have a module name
storj.io/storj/cmd/gateway.

Change-Id: I98439cbbaf433ae31309b7f80a19ced896018f65
2020-02-26 21:44:40 +02:00
Egon Elbre
64330c55b3 all: use pbgrpc
common/pb moved grpc to a separate package common/pb/pbgrpc.
This updates this repository to use it.

Change-Id: I2de2a190688871cf9cb61f7ea511f8a01e264e4e
2020-02-26 21:27:47 +02:00
Egon Elbre
e30f7b35b6 cmd/gateway: use a separate repository
Change-Id: Idbb0b2b6cf0e60c6d5d91218c24524d72285cf26
2020-02-24 10:03:03 +02:00
Matt Robinson
1b68d4548a
jenkins: use go 1.13.8 (#3781)
Change-Id: Ib369740c47dfa5dc0867053c212580fb7aee3655
2020-02-17 08:44:01 +02:00
littleskunk
68d5b1d6ec build/jenkins: allow -rc release tags
Change-Id: Ib1942a8029313debe672eaae6b4de835b74d5035
2020-02-10 12:01:18 +00:00
Egon Elbre
91a480f5a0 Jenkins: add storagenode npm checks
Change-Id: I93e3cc009c628e3c97a24541e7b01c75a342bda6
2020-02-03 20:35:33 +02:00
Egon Elbre
eaf3318a58 diagrams: satellite graph per process
Currently the whole satellite diagram can be quite overwhelming.
This change makes graphs for api, core and repair processes separately.

Change-Id: Iea906f51c3bcc46c71d7c8f6d8964034b317b3b4
2020-02-01 19:56:15 +00:00
Matt Robinson
7cbd83a399 jenkins: Remove os_arch from file inside zip file artifact
This changes the contents of the zip files, removing the _darwin_amd64
and other bits. Windows builds should still have filenames ending with
.exe.

Change-Id: I25a582034b59200c1d6034dd2925cdeaf06844e2
2020-01-29 16:29:14 +00:00
Matt Robinson
f6641aa434 jenkins: use go 1.13.7
Change-Id: I40dc231678aa7909efeb8586f2fd09713ef6f395
2020-01-29 13:42:14 +00:00
Matt Robinson
dd234aad4c
jenkins: bump go to 1.13.6
Change-Id: I7218029a2264d1ef9d9d1f853c470e22137b5844
2020-01-13 13:57:56 -05:00
Egon Elbre
fb4b11d13e
scripts: remove old scripts (#3742) 2020-01-07 13:28:41 +02:00
Egon Elbre
6615ecc9b6 common: separate repository
Change-Id: Ibb89c42060450e3839481a7e495bbe3ad940610a
2019-12-27 14:11:15 +02:00
Egon Elbre
ea455b6df0 all: remove code to default to grpc
We have moved to drpc so we don't need to have code for building
with grpc only.

Change-Id: I55732314dca0d5b4ce1132b68de4186a15d91b21
2019-12-20 20:12:04 +02:00
Egon Elbre
ef8bc88328 ci: use external repository
Change-Id: If26a005df45f6067240511d603fb4dd613f92b79
2019-12-19 12:05:49 +00:00
Matt Robinson
2986923244 build: update golang to 1.13.5
Change-Id: I7f8e29ac682a015d6a18f7989a5d0d726e2a0ce7
2019-12-18 19:47:41 +00:00
Jeff Wendling
ecd2ef4a21 all: build release fully dprc and test in mixed mode
Change-Id: I3bded3edf25a0b113601c8b12ecf1337f596649b
2019-11-15 10:03:18 -07:00
Matt Robinson
2f19b0a80e Unhook jenkins from the staging cluster while we sort it out (#3553) 2019-11-12 20:15:46 +01:00
Jeff Wendling
aaead2a29e
makefile: bump drpc and build storj-sim binaries like release (#3506)
Change-Id: I8cfd99226aefc2cfb5b18929799f56b3905cae53
2019-11-06 12:55:20 -07:00
Jeff Wendling
f32ff4ec79 release: use /usr/bin/env echo (#3505)
we don't want the shell builtin because we're using `-e`

Change-Id: Iaaabf02b1c4ece27e0cb99821df345b3fba7fb18
2019-11-05 22:40:06 +01:00
Jeff Wendling
12cead8f7b release: opt more binaries in to grpc (#3503) 2019-11-05 21:41:02 +01:00
Bryan White
841674e0e5 add goversioninfo to build-dev-deps make target for windows version info (#3501) 2019-11-05 21:14:10 +01:00
Matt Robinson
ce89c44c02 Rebuild faster by removing weird make calls (#3502) 2019-11-05 11:47:50 -07:00
Matt Robinson
e8f9a182da
Bump go version to 1.13.4 (#3450) 2019-11-04 13:24:36 -05:00
Jeff Wendling
abd27d496c satellite: make outgoing connections with grpc (#3457) 2019-11-02 16:38:53 +01:00
Matt Robinson
c8d64bd59e Make windows version more dynamic (#3169)
* Make the windows version dynamic

* Move the patch version to the right place
2019-10-28 12:23:22 +01:00
paul cannon
78000c0191 Makefile: handle empty/default $GOPATH (#3333)
Use `$(CURDIR)` in place of `$(shell pwd)`. This saves a
fork+exec, is more readable, and is more idiomatic. The only case where
those should have different values is when someone's configured `$SHELL`
is badly broken, and I'd argue in that case `$(CURDIR)` is preferable.

Change-Id: I6a3abac1e2e197f5332825ffad2bbd3da1fc4700
2019-10-22 12:44:13 +03:00
Matt Robinson
c0ffd59887 Go 1.13.3 (#3321) 2019-10-18 20:14:40 +02:00
Michal Niewrzal
ee55d14f35
mobile: create release script (#3197) 2019-10-17 08:39:34 -07:00
Egon Elbre
edf8318c0e
jenkins: update to golangci-lint 1.21.0 (#3289)
* jenkins: update to golint-1.21.0
* don't use underscores in var
* make handles private to satisfy linter

lib\uplinkc\universe.go:32:39: exported method Add returns unexported type storj.io/storj/lib/uplinkc._Ctype_long, which can be annoying to use (golint)
   func (m *Universe) Add(x interface{}) Handle {

* disable wsl explicitly
2019-10-17 10:42:22 +03:00
Natalie Villasana
cf430d2d73
scripts: add check-monitoring script to detect changes to monkit calls (#3114) 2019-10-15 13:00:14 -04:00
Matt Robinson
7485a1d0f5
Fix jenkins builds of storagenode console. (#3208) 2019-10-08 08:59:22 -04:00
Kaloyan Raev
616b2f521a installer/windows: add to private Jenkins build (#3206)
* Build Windows Installer stage

* Run the Windows Installer build on thw windows node

* Disable Build Images, Push Images and Upload steps

* checkout scm on windows agent

* Execute MSBuild

* Use bat instead of sh

* Test stash

* Copy artifacts between nodes

* Fix typo

* Sign Windows Installer

* Fix Makefile

* Enable binaries-upload

* Set Product Version to 0.0.0

* Build installer with Release configuration

* Enable back docker image stages and slack alerts

* Restore SNOboard shortcut after resolving conflicts

* Remove empty line from build.bat
2019-10-08 13:51:22 +02:00
JT Olio
37491d0d32 storagenode: embed the console into the binary and makefile (#3164)
* web/storagenode: add package-lock.json
* storagenode: compile console into binary
2019-10-08 10:52:19 +02:00
Michal Niewrzal
6770a9ce1a
mobile: move to lib/uplink-gomobile (#3177) 2019-10-06 15:34:12 -07:00
Jennifer Li Johnson
7ceaabb18e
Delete Bootstrap and Kademlia (#2974) 2019-10-04 16:48:41 -04:00
Michal Niewrzal
794254b0b2
mobile: build libuplink aar (#3165) 2019-10-04 01:56:42 -07:00
Matt Robinson
8989c0ae3b Add support for freebsd on amd64 (#3116) 2019-09-29 01:53:03 +02:00