Commit Graph

271 Commits

Author SHA1 Message Date
Kaloyan Raev
bd177bff03 cmd/storj-sim: cleanup gateway setup
Remove usage of --non-interactive flag. It is not provided (and
necessary) by the multitenant S3 gateway anymore.

ACCESS_KEY and SECRET_KEY env vars are not provided anymore as they are
not generated by the multitenant S3 gateway.

Change-Id: I3ecfb92110e31ae13977de3899dad273daae6c1e
2020-10-06 14:22:47 +00:00
Stefan Benten
338ed90876 Makefile: simplify binaries-upload
This change makes the loop to create the zip files a little easier (especially to read).

In addition, it also omits the .exe part for windows zip files, to ensure we can properly
use the versioncontrol URL for both windows and unix rollouts.

Change-Id: I0cbabb2ca61eb3346d97b2071411885383e870f3
2020-10-06 10:35:04 +02:00
Yaroslav
a43afa7ea0 make: fix storagenode-updater binary target
Adds -tags=service when building storagenode-updater which adds
interaction with Windows SVC and Linux systemd to the binary.

Change-Id: I21b4b3e8718348a30ec0453b07135ff3de033a5a
2020-09-30 11:53:11 +00:00
Egon Elbre
d508c4c985 scripts: remove duplicate check config lock
We have configlock_test for checking changes so the separate script and
makefile target are not needed.

Change-Id: I2bbc1c21ad849c9b7ec8bba43c0e11e94e04f6a6
2020-09-29 10:03:34 +00:00
Matt Robinson
84b6b91ee1
build: Go 1.15.2 2020-09-17 07:57:20 +03:00
Matt Robinson
9996aeca8d
build: Go 1.15 (#3927) 2020-09-07 16:01:29 +03:00
Matt Robinson
c548475662
build: Go 1.14.7 (#3926)
Co-authored-by: Egon Elbre <egonelbre@gmail.com>
2020-08-12 12:42:08 +02:00
Egon Elbre
85aff88e7f ci: update to Go 1.14.6
Change-Id: I0926c543612080ecd671d7e3cb67f940258b8ecc
2020-07-20 13:17:49 +00:00
Michal Niewrzal
7e6f9cf5b2 storj: install latest gateway with make install-sim
Change-Id: I961eecaf7763911fee6b54c9e7c83beb84b797b4
2020-06-18 11:26:00 +02:00
Michal Niewrzal
4dd47a4601 storj: add bump-dependencies target
One target to update all dependencies in main module and in testsuite

Change-Id: I2d8512f5fd7f5bb814451988a3feef366b39eee1
2020-06-08 15:51:37 +00:00
Matt Robinson
eced8dcfec
build: Go 1.14.4 (#3904) 2020-06-03 17:35:35 +02:00
Egon Elbre
d9fec04f50 lib: remove the old implementation
The stable uplink library is located at storj.io/uplink.
The stable uplink C library is located at storj.io/uplink-c.

Change-Id: Ia69115d6384b668490efe7e8ab674d7e8104b4f4
2020-05-29 11:36:43 +00:00
Michal Niewrzal
48e67a29a9 cmd/storj-sim: remove old libuplink from storj-sim
Change-Id: I146a2d7947d1f358e7bdd8acba68eb3fee68539d
2020-05-29 08:39:37 +00:00
Matt Robinson
e34937317c
Call the right make target for the segment-reaper (#3860) 2020-04-22 08:34:24 +02:00
Matt Robinson
1e295a48e7
add container image for segment-reaper (#3855) 2020-04-21 17:48:40 +03:00
Michal Niewrzal
75961fe8ce cmd/gateway: remove gateway command from repository and adjust `make
install-sim`

Change-Id: Ie55989ec0d702f6037843a8b6664f4f4f6347446
2020-03-27 07:42:42 +00:00
Egon Elbre
0781a91ff4 linksharing: move to storj.io/linksharing
Change-Id: If3bca64f80de9c244414e2aff28bb8b3bf9e87a2
2020-03-19 16:49:00 +00:00
Kaloyan Raev
4f0bf3fe1d
build: cleanup more gateway targets from Makefile (#3802)
Change-Id: Ia95caa2187b3e9e056a83cbea4230788ed4e8abd

Co-authored-by: Michal Niewrzal <michal@storj.io>
2020-03-16 15:07:52 +01:00
Kaloyan Raev
ce1ae134a0 build: don't push gateway docker images
This is now done from the new storj/gateway repo by another Jenkins job.
Ideally, we should remove everything related to the gateway from this
Makefile, but we cannot do it at the moment due to an all-on-one test
that depends on the gateway.

Change-Id: Ibc722bcee20c9f58b4864583ee3245455b45c7ba
2020-03-11 18:41:19 +00:00
Egon Elbre
46228fee92 cmd/gateway: use proper module name
By using a require for storj.io/storj it will make the import
unambiguous. This means it is possible to have a module name
storj.io/storj/cmd/gateway.

Change-Id: I98439cbbaf433ae31309b7f80a19ced896018f65
2020-02-26 21:44:40 +02:00
Egon Elbre
64330c55b3 all: use pbgrpc
common/pb moved grpc to a separate package common/pb/pbgrpc.
This updates this repository to use it.

Change-Id: I2de2a190688871cf9cb61f7ea511f8a01e264e4e
2020-02-26 21:27:47 +02:00
Egon Elbre
e30f7b35b6 cmd/gateway: use a separate repository
Change-Id: Idbb0b2b6cf0e60c6d5d91218c24524d72285cf26
2020-02-24 10:03:03 +02:00
Matt Robinson
1b68d4548a
jenkins: use go 1.13.8 (#3781)
Change-Id: Ib369740c47dfa5dc0867053c212580fb7aee3655
2020-02-17 08:44:01 +02:00
littleskunk
68d5b1d6ec build/jenkins: allow -rc release tags
Change-Id: Ib1942a8029313debe672eaae6b4de835b74d5035
2020-02-10 12:01:18 +00:00
Egon Elbre
91a480f5a0 Jenkins: add storagenode npm checks
Change-Id: I93e3cc009c628e3c97a24541e7b01c75a342bda6
2020-02-03 20:35:33 +02:00
Egon Elbre
eaf3318a58 diagrams: satellite graph per process
Currently the whole satellite diagram can be quite overwhelming.
This change makes graphs for api, core and repair processes separately.

Change-Id: Iea906f51c3bcc46c71d7c8f6d8964034b317b3b4
2020-02-01 19:56:15 +00:00
Matt Robinson
7cbd83a399 jenkins: Remove os_arch from file inside zip file artifact
This changes the contents of the zip files, removing the _darwin_amd64
and other bits. Windows builds should still have filenames ending with
.exe.

Change-Id: I25a582034b59200c1d6034dd2925cdeaf06844e2
2020-01-29 16:29:14 +00:00
Matt Robinson
f6641aa434 jenkins: use go 1.13.7
Change-Id: I40dc231678aa7909efeb8586f2fd09713ef6f395
2020-01-29 13:42:14 +00:00
Matt Robinson
dd234aad4c
jenkins: bump go to 1.13.6
Change-Id: I7218029a2264d1ef9d9d1f853c470e22137b5844
2020-01-13 13:57:56 -05:00
Egon Elbre
fb4b11d13e
scripts: remove old scripts (#3742) 2020-01-07 13:28:41 +02:00
Egon Elbre
6615ecc9b6 common: separate repository
Change-Id: Ibb89c42060450e3839481a7e495bbe3ad940610a
2019-12-27 14:11:15 +02:00
Egon Elbre
ea455b6df0 all: remove code to default to grpc
We have moved to drpc so we don't need to have code for building
with grpc only.

Change-Id: I55732314dca0d5b4ce1132b68de4186a15d91b21
2019-12-20 20:12:04 +02:00
Egon Elbre
ef8bc88328 ci: use external repository
Change-Id: If26a005df45f6067240511d603fb4dd613f92b79
2019-12-19 12:05:49 +00:00
Matt Robinson
2986923244 build: update golang to 1.13.5
Change-Id: I7f8e29ac682a015d6a18f7989a5d0d726e2a0ce7
2019-12-18 19:47:41 +00:00
Jeff Wendling
ecd2ef4a21 all: build release fully dprc and test in mixed mode
Change-Id: I3bded3edf25a0b113601c8b12ecf1337f596649b
2019-11-15 10:03:18 -07:00
Matt Robinson
2f19b0a80e Unhook jenkins from the staging cluster while we sort it out (#3553) 2019-11-12 20:15:46 +01:00
Jeff Wendling
aaead2a29e
makefile: bump drpc and build storj-sim binaries like release (#3506)
Change-Id: I8cfd99226aefc2cfb5b18929799f56b3905cae53
2019-11-06 12:55:20 -07:00
Jeff Wendling
f32ff4ec79 release: use /usr/bin/env echo (#3505)
we don't want the shell builtin because we're using `-e`

Change-Id: Iaaabf02b1c4ece27e0cb99821df345b3fba7fb18
2019-11-05 22:40:06 +01:00
Jeff Wendling
12cead8f7b release: opt more binaries in to grpc (#3503) 2019-11-05 21:41:02 +01:00
Bryan White
841674e0e5 add goversioninfo to build-dev-deps make target for windows version info (#3501) 2019-11-05 21:14:10 +01:00
Matt Robinson
ce89c44c02 Rebuild faster by removing weird make calls (#3502) 2019-11-05 11:47:50 -07:00
Matt Robinson
e8f9a182da
Bump go version to 1.13.4 (#3450) 2019-11-04 13:24:36 -05:00
Jeff Wendling
abd27d496c satellite: make outgoing connections with grpc (#3457) 2019-11-02 16:38:53 +01:00
Matt Robinson
c8d64bd59e Make windows version more dynamic (#3169)
* Make the windows version dynamic

* Move the patch version to the right place
2019-10-28 12:23:22 +01:00
paul cannon
78000c0191 Makefile: handle empty/default $GOPATH (#3333)
Use `$(CURDIR)` in place of `$(shell pwd)`. This saves a
fork+exec, is more readable, and is more idiomatic. The only case where
those should have different values is when someone's configured `$SHELL`
is badly broken, and I'd argue in that case `$(CURDIR)` is preferable.

Change-Id: I6a3abac1e2e197f5332825ffad2bbd3da1fc4700
2019-10-22 12:44:13 +03:00
Matt Robinson
c0ffd59887 Go 1.13.3 (#3321) 2019-10-18 20:14:40 +02:00
Michal Niewrzal
ee55d14f35
mobile: create release script (#3197) 2019-10-17 08:39:34 -07:00
Egon Elbre
edf8318c0e
jenkins: update to golangci-lint 1.21.0 (#3289)
* jenkins: update to golint-1.21.0
* don't use underscores in var
* make handles private to satisfy linter

lib\uplinkc\universe.go:32:39: exported method Add returns unexported type storj.io/storj/lib/uplinkc._Ctype_long, which can be annoying to use (golint)
   func (m *Universe) Add(x interface{}) Handle {

* disable wsl explicitly
2019-10-17 10:42:22 +03:00
Natalie Villasana
cf430d2d73
scripts: add check-monitoring script to detect changes to monkit calls (#3114) 2019-10-15 13:00:14 -04:00
Matt Robinson
7485a1d0f5
Fix jenkins builds of storagenode console. (#3208) 2019-10-08 08:59:22 -04:00
Kaloyan Raev
616b2f521a installer/windows: add to private Jenkins build (#3206)
* Build Windows Installer stage

* Run the Windows Installer build on thw windows node

* Disable Build Images, Push Images and Upload steps

* checkout scm on windows agent

* Execute MSBuild

* Use bat instead of sh

* Test stash

* Copy artifacts between nodes

* Fix typo

* Sign Windows Installer

* Fix Makefile

* Enable binaries-upload

* Set Product Version to 0.0.0

* Build installer with Release configuration

* Enable back docker image stages and slack alerts

* Restore SNOboard shortcut after resolving conflicts

* Remove empty line from build.bat
2019-10-08 13:51:22 +02:00
JT Olio
37491d0d32 storagenode: embed the console into the binary and makefile (#3164)
* web/storagenode: add package-lock.json
* storagenode: compile console into binary
2019-10-08 10:52:19 +02:00
Michal Niewrzal
6770a9ce1a
mobile: move to lib/uplink-gomobile (#3177) 2019-10-06 15:34:12 -07:00
Jennifer Li Johnson
7ceaabb18e
Delete Bootstrap and Kademlia (#2974) 2019-10-04 16:48:41 -04:00
Michal Niewrzal
794254b0b2
mobile: build libuplink aar (#3165) 2019-10-04 01:56:42 -07:00
Matt Robinson
8989c0ae3b Add support for freebsd on amd64 (#3116) 2019-09-29 01:53:03 +02:00
Matt Robinson
02f68d68d6 Put -s and -w in the right spot (#3135) 2019-09-27 17:38:02 +02:00
Matt Robinson
52497c253e Bump go version to 1.13.1 (#3130) 2019-09-26 17:51:09 +02:00
Matt Robinson
9962f0c3f0 build: make binaries smaller with -s and -w ldflags (#3125) 2019-09-26 14:08:18 +03:00
Egon Elbre
e0d5cbbbd7
jenkins: update golangci to v1.19.1 (#3119) 2019-09-25 19:35:56 +03:00
Michal Niewrzal
26526de548
add auto-updater to Makefile (#3076) 2019-09-19 05:25:46 -07:00
Egon Elbre
09447e24c8
jenkins: update to go1.13 (#2990) 2019-09-10 16:44:16 +03:00
Egon Elbre
a801fab66a
all: add archview annotations (#2964) 2019-09-10 16:24:16 +03:00
Jeff Wendling
28353823e8 jenkins: remove Jenkinsfile.gerrit (#2953) 2019-09-06 08:54:46 +03:00
Egon Elbre
c4ee7eb51c
jenkins: build and test web/satellite (#2908) 2019-09-05 16:32:46 +03:00
Bryan White
62df8ddb0c cmd/certificates: auth export improvements & certificates test script (#2897) 2019-09-04 19:58:38 +02:00
Matt Robinson
1b2a2d045f
[build]: Adjust arm63v6 and aarch64 images to match convention (#2845)
* Adjust arm32v6 and aarch64 docker images to match the hello-world image

* Update from master, fix potential bug in push-images target, and update storagenode deploy to handle arm64 image
2019-08-28 08:18:56 -04:00
Matt Robinson
ac58589615 Add aarch64 support and use go 1.12.9 (#2814) 2019-08-19 12:40:16 +03:00
Brandon Iglesias
5c61c79ab7 Start building binary for the link sharing service (#2790) 2019-08-14 15:04:56 -04:00
Egon Elbre
7fcbec48f7 go.mod: update grpc-go and go (#2777) 2019-08-14 06:18:10 -04:00
Alexander Leitner
f3db409103 lib/uplinkc: make .so smaller by removing symbol information (#2730) 2019-08-13 15:24:31 +03:00
Jess G
2a464bfdb5
Jg/1622 add backwards compatibility test (#2656)
* add stage to jenkins, add script for backwards compat tests

* debug backwards compat tests

* add setup in a script

* set env vars

* add more env ars

* mv to one script to debug

* debug api key problem

* add tmp dir

* rm print statements
2019-08-06 16:03:14 -07:00
Egon Elbre
c5de36ed1f
update to go 1.12.7 (#2493) 2019-07-09 11:56:08 -04:00
Egon Elbre
e83ebd7cde
jenkins: avoid using goimports and distribute load better (#2359) 2019-06-27 21:52:50 +03:00
Egon Elbre
86ec9b9570
jenkins: capability to use leakcheck (#2179) 2019-06-21 17:57:38 +03:00
Matt Robinson
89cd4d7bc0 Use a persistent cache and pkg directory (#2235) 2019-06-18 13:18:09 -04:00
Bryan White
76061c50c5
Uplink C bindings part 2 (#2209) 2019-06-15 13:23:12 +02:00
Egon Elbre
749846b42b
Update golangci-lint (#2159) 2019-06-10 11:52:09 +03:00
Egon Elbre
9ed8c96058 jenkins: check that lock file is updated properly (#1977) 2019-06-04 19:12:27 +02:00
Matt Robinson
4544542b43 I think this fixes the weird '-heads-' in the version tags (#2010) 2019-05-21 16:36:08 +02:00
Matt Robinson
cbb323ad52 Give the bootstrap image the business. (#2005) 2019-05-21 13:57:20 +02:00
Ivan Fraixedes
838476f164 Makefile: test-satellite-cfg-change improvements (#1915)
* Clarifies that the test-satellite-cfg-change is for being run by
  Jenkins as others are.
* Move the target for updating the satellite configuration lock file to
  a new Makefile group, called "Tooling".
* Change how it's run the script update-satellite-cfg-lock.sh for
  ensuring that the outcome is the same when running from different
  machines.
2019-05-09 16:25:44 +03:00
Matt Robinson
bcf9a809b6
Update golang version used for release builds (#1926) 2019-05-08 15:20:00 -04:00
Ivan Fraixedes
f8df461249 v3-1660: Create a test that breaks when satellite config file has changed (#1898)
* scripts: check for satellite configuration changes

Create a simple script that checks if the satellite configuration has
differed with the last changes.

* Makefile: add target to check satellite cfg changes

Add a Makefile target which executes the script that checks if the last
changes has made a change in the satellite configuration.

* ci: add satellite cfg check on integration stage

* FIXUP: use releae defaults rather than development ones

* FIXUP: show the message when config differs & some cleanups

* scripts: add script to update the satellite cfg lock

Add a script for allowing to update the satellite configuration lock
file and add Makefile target to run it.

* scripts/testsdata: update satellite cfg lock

Update the satellite configuration lock file with the last changes that
satellite has suffered upstream.
2019-05-07 17:20:04 -04:00
Matt Robinson
9f98450824 Fix Jenkins master builds (#1889)
* Missed a bit to make jenkins build the versioncontrol image correctly
2019-05-03 14:14:38 -04:00
Matt Robinson
43ae28ce2c Add image for versioncontrol server (#1875)
* First pass at building a version server container image

* Actually adding the Dockerfile this branch

* Add entrypoint to auto create config file
2019-05-02 23:25:04 +02:00
Egon Elbre
ba322abd9c
Remove Travis (#1868) 2019-04-30 21:18:32 +03:00
Matt Robinson
d843f80773 account for the new pod name (#1846) 2019-04-26 08:08:42 +03:00
Matt Robinson
ca2055abaf
Clean up the satellite image (#1768)
This is working on saturn and jupiter, plus travis is ok with it.
2019-04-16 17:06:59 -04:00
Matt Robinson
1b8976ab9a
Missed a part in the last attempt (#1760)
Again, appologies.
2019-04-15 15:04:02 -04:00
Matt Robinson
ac66260638
Manifests are derived from a specific image, not the magical latest tag (#1759)
Again, sorry for using my admin powers.
2019-04-15 14:30:36 -04:00
Matt Robinson
a81d13920f
Remove tracked branches for now (#1758)
@jtolds  made me do it
2019-04-15 13:49:08 -04:00
Philip Hutchins
04814a3f3d
Fixing upload and build binaries (#1751) 2019-04-12 13:28:04 -04:00
Matt Robinson
c4ba8831c3 Bump to 1.12.1 and use scripts/release.sh to build things (#1676) 2019-04-12 11:31:31 -04:00
Egon Elbre
0eee46524d update linter to v1.16 (#1741)
* update to v1.16

* use full version number

* fix linter issues

* fix comment

* nicer comment

* restart travis

* restart travis
2019-04-11 13:32:40 -04:00
Matt Robinson
e478d0855c
Build multiarch ARM32v6 and AMD64 images (#1686) 2019-04-08 16:01:20 -04:00
Stefan Benten
2cf86703a3
Add Versioning Server (#1576)
* Initial Webserver Draft for Version Controlling

* Rename type to avoid confusion

* Move Function Calls into Version Package

* Fix Linting and Language Typos

* Fix Linting and Spelling Mistakes

* Include Copyright

* Include Copyright

* Adjust Version-Control Server to return list of Versions

* Linting

* Improve Request Handling and Readability

* Add Configuration File Option
Add Systemd Service file

* Add Logging to File

* Smaller Changes

* Add Semantic Versioning and refuses outdated Software from Startup (#1612)

* implements internal Semantic Version library

* adds version logging + reporting to process

* Advance SemVer struct for easier handling

* Add Accepted Version Store

* Fix Function

* Restructure

* Type Conversion

* Handle Version String properly

* Add Note about array index

* Set temporary Default Version

* Add Copyright

* Adding Version to Dashboard

* Adding Version Info Log

* Renaming and adding CheckerProcess

* Iteration Sync

* Iteration V2

* linting

* made LogAndReportVersion a go routine

* Refactor to Go Routine

* Add Context to Go Routine and allow Operation if Lookup to Control Server fails

* Handle Unmarshal properly

* Linting

* Relocate Version Checks

* Relocating Version Check and specified default Version for now

* Linting Error Prevention

* Refuse Startup on outdated Version

* Add Startup Check Function

* Straighten Logging

* Dont force Shutdown if --dev flag is set

* Create full Service/Peer Structure for ControlServer

* Linting

* Straighting Naming

* Finish VersionControl Service Layout

* Improve Error Handling

* Change Listening Address

* Move Checker Function

* Remove VersionControl Peer

* Linting

* Linting

* Create VersionClient Service

* Renaming

* Add Version Client to Peer Definitions

* Linting and Renaming

* Linting

* Remove Transport Checks for now

* Move to Client Side Flag

* Remove check

* Linting

* Transport Client Version Intro

* Adding Version Client to Transport Client

* Add missing parameter

* Adding Version Check, to set Allowed = true

* Set Default to true, testing

* Restructuring Code

* Uplink Changes

* Add more proper Defaults

* Renaming of Version struct

* Dont pass Service use Pointer

* Set Defaults for Versioning Checks

* Put HTTP Server in go routine

* Add Versioncontrol to Storj-Sim

* Testplanet Fixes

* Linting

* Add Error Handling and new Server Struct

* Move Lock slightly

* Reduce Race Potentials

* Remove unnecessary files

* Linting

* Add Proper Transport Handling

* small fixes

* add fence for allowed check

* Add Startup Version Check and Service Naming

* make errormessage private

* Add Comments about VersionedClient

* Linting

* Remove Checks that refuse outgoing connections

* Remove release cmd

* Add Release Script

* Linting

* Update to use correct Values

* Move vars private and set minimum default versions for testing builds

* Remove VersionedClient

* Better Error Handling and naked return removal

* Straighten the Regex and string conversion

* Change Check to allows testplanet and storj-sim to run without the
need to pass an LDFlag

* Cosmetic Change to Dashboard

* Cleanup Returns and remove commented code

* Remove Version Check if no build options are passed in

* Pass in Config Values instead of Pointers

* Handle missed Error

* Update Endpoint URL

* Change Type of Release Flag

* Add additional Logging

* Remove Versions Logging of other Services

* minor fixes

Change-Id: I5cc04a410ea6b2008d14dffd63eb5f36dd348a8b
2019-04-03 21:13:39 +02:00
Matt Robinson
60035caade
Fix image tags (#1541) 2019-03-21 07:48:39 -04:00
Philip Hutchins
eeb1457478 Fixing tabs (#1423) 2019-03-07 19:58:55 +01:00
Philip Hutchins
af0d5add06
Adding fix for release branch latest tag pushing (#1420)
* Adding fix for release branch latest tag pushign

* Updating how we handle pushing latest branches

* Cleaning up and adding CUSTOMTAG where necessary

* Correcting comment on satellite-ui
2019-03-07 10:22:18 -05:00
Philip Hutchins
64f2eae392
Adding conditional around release branches so that we namespace latest container (#1416) 2019-03-06 20:52:23 -05:00
Philip Hutchins
1f452e67dc Adding dockerfile for running the web UI for Satellite (#1366)
* Adding dockerfile for running the web UI for Satellite

* Updating to work with Makefile and from root directory of repo

* Updating satellite ui build process to run in a more production like mode by generating the assets the pulling those into the satellite container

* Updates to allow external traffic to UI, updates to storagenode for identity creation, and logging for bug tracking

* Adding auto cert generation for storagenode

* removing satellite-ui-image from main images flow in Makefile and adding latest tag to docker build for it

* Adding solid defaults, tuning dockerfiles, and moving to standard logging methods

* Updating logging to be more standard

* Updating to logger.Debug

* Removing unused library and unused identity creation code

Change-Id: I956453037e303693ea37f94318180af0ab7984d5
2019-02-28 13:18:53 -07:00
Bryan White
aa99968c66 add inspector to makefile (#1269) 2019-02-08 10:53:40 +02:00
Bryan White
8edfd43abb
certificates config and --certs-dir fixes (#1093) 2019-01-22 13:35:48 +01:00
Timothy Adams
3cf571abe0
added targets for binaries to fix master (#1104) 2019-01-18 13:15:07 -05:00
Philip Hutchins
45f5648c89 adding identity and certificates to be built and pushed (#1097) 2019-01-18 10:37:52 -05:00
Egon Elbre
bbf81f2479 Consolidate identity management to identity cli commands (#1083)
* Consolidate identity management:

Move identity cretaion/signing out of storagenode setup command.

* fixes

* linters

* Consolidate identity management:

Move identity cretaion/signing out of storagenode setup command.

* fixes

* sava backups before saving signed certs

* add "-prebuilt-test-cmds" test flag

* linters

* prepare cli tests for travis

* linter fixes

* more fixes

* linter gods

* sp/sdk/sim

* remove ca.difficulty

* remove unused difficulty

* return setup to its rightful place

* wip travis

* Revert "wip travis"

This reverts commit 56834849dcf066d3cc0a4f139033fc3f6d7188ca.

* typo in travis.yaml

* remove tests

* remove more

* make it only create one identity at a time for consistency

* add config-dir for consitency

* add identity creation to storj-sim

* add flags

* simplify

* fix nolint and compile

* prevent overwrite and pass difficulty, concurrency, and parent creds

* goimports
2019-01-18 11:36:58 +01:00
Egon Elbre
c89609abbe
Remove captplanet (#1070) 2019-01-17 04:35:22 -05:00
Egon Elbre
9dbb6a101d
Rename storj-sdk to storj-sim (#1078) 2019-01-16 18:09:57 -05:00
Egon Elbre
b86c4dc5eb
add make install-sdk target (#1069) 2019-01-16 13:50:16 -05:00
Egon Elbre
387fb14e23
storj-sdk (#969) 2019-01-08 17:24:15 +02:00
Bryan White
b6611e2800
automate certificate signing in storage node setup (#954) 2019-01-04 18:23:23 +01:00
Matt Robinson
c0e6b62708
Test all-in-one (#900)
* Add test for aio

* Don't trust the user to have images built for a version

* Make travis run the aio test

* Add missing values to docker-compose, sort some things, consider the gateway image

* today's changes

* config changed, again

* more fixes

* Expose satellite port on localhost:7778

* Add retries and a timeout around the big-testfile test in AIO

* Another config value changed

* Make this error message a little more useful

* Fix nil condition
2019-01-03 14:54:27 -05:00
Bryan White
249244536a
CSR Service (part 2): cert signing rpc (#950)
* CSR Service:

+ implement certificate sign rpc method
+ implement certificate signer client/server
+ refactor `AuthorizationDB#Create`
+ refactor `NewTestIdentity`
+ add `AuthorizationDB#Claim`
+ add `Token#Equal`
+ fix `Authorizations#Marshal` when marshaling identities and certificates
+ tweak `Authorization#String` format
+ cert debugging improvements (jsondiff)
+ receive context arg in `NewTestIdentity`
+ misc. fixes
2019-01-02 12:39:17 -05:00
Bryan White
2016ce9fd6
Certificate revocation (#836)
* wip certificate revocation

* refactor tests

* wip testing

* testing

* review fixes

* integration fix attempt #1

* review fixes

* integration fix attempt #2

* linter fixes

* add copywrite

* integration fix attemp #3

* more testing

* more tests

* go mod tidy

* review fixes

* linter fixes
2018-12-13 21:01:43 +01:00
Bryan White
228aa34ff6
TLS extension processing (#771) 2018-12-07 14:44:25 +01:00
Matt Robinson
4237afb1c1
Attempt at fixing the all-in-one environment (#736) 2018-12-04 16:10:23 -05:00
Egon Elbre
3e461df470
Linting protobuf files (#676) 2018-11-30 17:02:01 +02:00
Matt Robinson
471f9004aa
Fix arm builds (#595) 2018-11-26 09:11:09 -05:00
Egon Elbre
42ff835e6e
Fail linter when copyright is missing. (#664) 2018-11-15 14:47:52 +02:00
Matt Robinson
c5f5915588
build the gateway binary too (#592) 2018-11-07 13:19:09 -05:00
Matt Robinson
d1e7534f9e
use GOPROXY if it's set in the environment (#495) 2018-10-17 15:06:26 -04:00
Matt Robinson
48eb8e5c8e
sign windows builds (#494) 2018-10-17 14:21:18 -04:00
Matt Robinson
bcd18a44bc
give our appearence on windows some love (#429) 2018-10-16 14:48:17 -04:00
Matt Robinson
00358fc368
clean up Makefile (#487) 2018-10-16 14:47:43 -04:00
Matt Robinson
a1f93285e2 figure out which deployments need to be patched during a deployment instead of guessing (#438) 2018-10-08 12:52:27 -04:00
Egon Elbre
e7e2d4d7c9
use golangci-lint and add missing error checks (#340) 2018-09-11 16:13:25 +03:00
Egon Elbre
6ee6f0fdf5
Remove duplicated mock store implementations (#337) 2018-09-11 10:27:12 +03:00
Matt Robinson
75c64035a2 Normalize the Docker Bits (#325)
* Normalize the dockerfiles and entrypoints

* fix formattting and docker-compose file

* add missing env var to satellite dockerfile

* Remove build args no longer in use

* More changes to make it all work
2018-09-07 15:46:08 -04:00
Matt Robinson
2c315db84b Fix the binaries and Docker Images (#306)
* Finally fix the binary builds for a couple os/arch combinations

* Make the file executable before adding it to the zip

* This cache hack wasn't suppose to make it into the PR

* Guard against calling the binary target by itself

* Order matters on OS X

* Vendored my image

* No 386 builds until storage is fixed.
2018-09-05 16:40:47 -04:00
Matt Robinson
0a78e741ec
Build and upload binaries (#296)
* First pass at building binaries

* Finished building images

* Reworked some bits to save binaries in a better place.

* First pass at uploading binaries

* Builds for freebsd fail currently

* Add ignore file for docker builds to not invalidate the cache as often

* Docker image for 1.11-alpine exists now

* Name windows binaries correctly

* The makefile is used in tests.

* Take a swing at updating the tests for 1.11

* Switch to something with glibc so the race detector works.

* Remove unused .PHONY targets
2018-08-31 11:21:44 -04:00
aligeti
743ec65204
hc to satellite rename (#282)
* hc to satellite rename

* rebase with master and fix spelling

* missed out rename

* rebase merge fix
2018-08-29 14:32:41 -04:00
Egon Elbre
23231a29b1
Reduce linter concurrency (#297)
Limit concurrency to reduce memory usage.
2018-08-28 18:00:18 +03:00
JT Olio
58c810519f
cross-compilation: fix darwin (#293)
* cross-compilation: fix darwin

* minio without fsnotify

* update Gopkg.lock

* make release target
2018-08-27 15:06:55 -06:00
Egon Elbre
0f5a2f4ef5 Enable more linters (#272)
* enable more linters

* Run gofmt -s

* run goimports

* run unconvert

* fix naked return

* fix misspellings

* fix ineffectual assigments

* fix missing declaration

* don't use deprecated grpc.Errof

* check errors in tests

* run gofmt -w -r "assert.Nil(err) -> assert.NoError(err)"

* fix directory permissions

* don't use nil Context

* simplify boolean expressions

* use bytes.Equal instead of bytes.Compare

* merge variable declarations, remove redundant returns

* fix some golint errors

* run goimports

* handle more errors

* delete empty TestMain

* delete empty TestMain

* ignore examples for now

* fix lint errors

* remove unused values

* more fixes

* run gofmt -w -s .

* add more comments

* fix naming

* more lint fixes

* try switching travis to go1.11

* fix unnecessary conversions

* fix deprecated methods

* use go1.10 and disable gofmt/goimports for now

* switch to 1.10

* don't re-enable gofmt and goimports

* switch covermode to atomic because of -race

* gofmt
2018-08-27 11:28:16 -06:00
aligeti
09da23737a
rename farmers to storagenode (#275)
* rename farmers to storagenode

* review changes

* merge conflicts
2018-08-24 21:52:58 -04:00
JT Olio
560e9fd9ec
uplink: make ul/uplink consistent (#286)
* uplink: make ul/uplink consistent

* pstore: make test more robust
2018-08-24 14:02:42 -06:00
aligeti
4edba19986 [Ready for review]Rename gw uplink (#273)
Rename gw uplink
2018-08-24 10:22:24 -06:00
Matt Robinson
3456a127cf
More fixes to the staging deploy process (#278) 2018-08-23 17:17:47 -04:00
Matt Robinson
631331236e
Fix staging deploy of satellite and storage-node (#277) 2018-08-23 17:04:51 -04:00
Matt Robinson
026997e725
All-in-one docker-compose project (#267)
* First pass at an all-in-one compose project

* Sort out dependencies so uplink can build in docker

* Added the final bits to make this work

* Add readme for all-in-one

* Fix sed to work on OS X and Linux

* Make the readme for all-in-one better

* Only bring up the satellite and uplink after fixing the mock overlay

* Use a dummy value for mock overlay for now

* Finishing touches

* Fix url form for aws cli tool

* Move files out of all-in-one directory to help with directory confusion.

* Added a make target to make all-in-one even better
2018-08-23 11:48:03 -04:00
Matt Robinson
5841065420 Attempt to fix jenkins builds (#260)
* Attempt to clean up docker image tags

* Clean only master

* Add back go version and actually use it
2018-08-21 22:54:56 -04:00
Matt Robinson
1687ce25c7 Fix the sha3 error with the build (#230)
* Fix the build
2018-08-15 14:49:30 -04:00
Maximillian von Briesen
f80ddf9358
Test captplanet with travis (#226)
* test captplanet in travis

* add comparison between uploaded and downloaded file

* test inline and remote files

* move captplanet testing to bash script
2018-08-14 12:58:16 -04:00
Philip Hutchins
c77d1b7f0b
Updating service names for docker tags (#207) 2018-08-08 13:48:59 -04:00
Dennis Coyle
5de5428d3a
First pass at node restrictions (#146)
* WIP First pass at node restrictions

* adjustments to storage clients

* fix cover

* undo previous import change

* adding copyright

* fix the tests

* check for keys

* moar fixes to tests

* linter

* change how we handle restrictions

* add generated pb.go file

* PR comments addressed

* MockKeyValueStore

* pr comments addressed

* missing )

* past my bedtime

* moar merge issues

* cleanup
2018-08-01 10:15:38 -04:00
Maximillian von Briesen
36c394c295
StatDB (#144)
* add statdb proto and example client

* server logic

* update readme

* remove boltdb from service.go

* sqlite3

* add statdb server executable file

* create statdb node table if it does not exist already

* get UpdateBatch working

* update based on jt review

* remove some commented lines

* fix linting issues

* reformat

* apiKey -> APIKey

* update statdb client apiKey->APIKey
2018-07-27 17:11:44 -04:00
JT Olio
4cff003ec9 fix docker makefile (#170)
* fix makefile
2018-07-26 16:01:38 -04:00
Matt Robinson
0cbbc69777 Add files for testing builds in docker (#161)
* Add files for testing builds in docker

* Make tests check for redis running before trying to start redis-server, which may not exist.

* Clean redis server before any tests use it.

* Add more debugging for travis

* Explicitly requiring redis for travis
2018-07-25 17:47:02 -04:00
Alexander Leitner
cde0ffaa52 Add dockerfile and yaml for setting up piecestore servers (#102)
* Add dockerfile and yaml for setting up piecestore servers

* Fix dockerfile for @aleitner (#115)

* Fix dockerfile for @aleitner

* Move files for @coyle

* Update yaml

* My linter had some errors so I resolved them

* Make jenkins do the needful

* Make piecestore-farmer look like overlay's build process

* Fix service spec to work in staging

* Make Jenkins push images, but not deploy them, yet.

* Modify entrypoint to fit new verbs

* Update piecestore-farmer entrypoint script to handle new app output
2018-07-19 15:42:50 -04:00