Commit Graph

190 Commits

Author SHA1 Message Date
Egon Elbre
ea455b6df0 all: remove code to default to grpc
We have moved to drpc so we don't need to have code for building
with grpc only.

Change-Id: I55732314dca0d5b4ce1132b68de4186a15d91b21
2019-12-20 20:12:04 +02:00
Egon Elbre
ef8bc88328 ci: use external repository
Change-Id: If26a005df45f6067240511d603fb4dd613f92b79
2019-12-19 12:05:49 +00:00
Matt Robinson
2986923244 build: update golang to 1.13.5
Change-Id: I7f8e29ac682a015d6a18f7989a5d0d726e2a0ce7
2019-12-18 19:47:41 +00:00
Jeff Wendling
ecd2ef4a21 all: build release fully dprc and test in mixed mode
Change-Id: I3bded3edf25a0b113601c8b12ecf1337f596649b
2019-11-15 10:03:18 -07:00
Matt Robinson
2f19b0a80e Unhook jenkins from the staging cluster while we sort it out (#3553) 2019-11-12 20:15:46 +01:00
Jeff Wendling
aaead2a29e
makefile: bump drpc and build storj-sim binaries like release (#3506)
Change-Id: I8cfd99226aefc2cfb5b18929799f56b3905cae53
2019-11-06 12:55:20 -07:00
Jeff Wendling
f32ff4ec79 release: use /usr/bin/env echo (#3505)
we don't want the shell builtin because we're using `-e`

Change-Id: Iaaabf02b1c4ece27e0cb99821df345b3fba7fb18
2019-11-05 22:40:06 +01:00
Jeff Wendling
12cead8f7b release: opt more binaries in to grpc (#3503) 2019-11-05 21:41:02 +01:00
Bryan White
841674e0e5 add goversioninfo to build-dev-deps make target for windows version info (#3501) 2019-11-05 21:14:10 +01:00
Matt Robinson
ce89c44c02 Rebuild faster by removing weird make calls (#3502) 2019-11-05 11:47:50 -07:00
Matt Robinson
e8f9a182da
Bump go version to 1.13.4 (#3450) 2019-11-04 13:24:36 -05:00
Jeff Wendling
abd27d496c satellite: make outgoing connections with grpc (#3457) 2019-11-02 16:38:53 +01:00
Matt Robinson
c8d64bd59e Make windows version more dynamic (#3169)
* Make the windows version dynamic

* Move the patch version to the right place
2019-10-28 12:23:22 +01:00
paul cannon
78000c0191 Makefile: handle empty/default $GOPATH (#3333)
Use `$(CURDIR)` in place of `$(shell pwd)`. This saves a
fork+exec, is more readable, and is more idiomatic. The only case where
those should have different values is when someone's configured `$SHELL`
is badly broken, and I'd argue in that case `$(CURDIR)` is preferable.

Change-Id: I6a3abac1e2e197f5332825ffad2bbd3da1fc4700
2019-10-22 12:44:13 +03:00
Matt Robinson
c0ffd59887 Go 1.13.3 (#3321) 2019-10-18 20:14:40 +02:00
Michal Niewrzal
ee55d14f35
mobile: create release script (#3197) 2019-10-17 08:39:34 -07:00
Egon Elbre
edf8318c0e
jenkins: update to golangci-lint 1.21.0 (#3289)
* jenkins: update to golint-1.21.0
* don't use underscores in var
* make handles private to satisfy linter

lib\uplinkc\universe.go:32:39: exported method Add returns unexported type storj.io/storj/lib/uplinkc._Ctype_long, which can be annoying to use (golint)
   func (m *Universe) Add(x interface{}) Handle {

* disable wsl explicitly
2019-10-17 10:42:22 +03:00
Natalie Villasana
cf430d2d73
scripts: add check-monitoring script to detect changes to monkit calls (#3114) 2019-10-15 13:00:14 -04:00
Matt Robinson
7485a1d0f5
Fix jenkins builds of storagenode console. (#3208) 2019-10-08 08:59:22 -04:00
Kaloyan Raev
616b2f521a installer/windows: add to private Jenkins build (#3206)
* Build Windows Installer stage

* Run the Windows Installer build on thw windows node

* Disable Build Images, Push Images and Upload steps

* checkout scm on windows agent

* Execute MSBuild

* Use bat instead of sh

* Test stash

* Copy artifacts between nodes

* Fix typo

* Sign Windows Installer

* Fix Makefile

* Enable binaries-upload

* Set Product Version to 0.0.0

* Build installer with Release configuration

* Enable back docker image stages and slack alerts

* Restore SNOboard shortcut after resolving conflicts

* Remove empty line from build.bat
2019-10-08 13:51:22 +02:00
JT Olio
37491d0d32 storagenode: embed the console into the binary and makefile (#3164)
* web/storagenode: add package-lock.json
* storagenode: compile console into binary
2019-10-08 10:52:19 +02:00
Michal Niewrzal
6770a9ce1a
mobile: move to lib/uplink-gomobile (#3177) 2019-10-06 15:34:12 -07:00
Jennifer Li Johnson
7ceaabb18e
Delete Bootstrap and Kademlia (#2974) 2019-10-04 16:48:41 -04:00
Michal Niewrzal
794254b0b2
mobile: build libuplink aar (#3165) 2019-10-04 01:56:42 -07:00
Matt Robinson
8989c0ae3b Add support for freebsd on amd64 (#3116) 2019-09-29 01:53:03 +02:00
Matt Robinson
02f68d68d6 Put -s and -w in the right spot (#3135) 2019-09-27 17:38:02 +02:00
Matt Robinson
52497c253e Bump go version to 1.13.1 (#3130) 2019-09-26 17:51:09 +02:00
Matt Robinson
9962f0c3f0 build: make binaries smaller with -s and -w ldflags (#3125) 2019-09-26 14:08:18 +03:00
Egon Elbre
e0d5cbbbd7
jenkins: update golangci to v1.19.1 (#3119) 2019-09-25 19:35:56 +03:00
Michal Niewrzal
26526de548
add auto-updater to Makefile (#3076) 2019-09-19 05:25:46 -07:00
Egon Elbre
09447e24c8
jenkins: update to go1.13 (#2990) 2019-09-10 16:44:16 +03:00
Egon Elbre
a801fab66a
all: add archview annotations (#2964) 2019-09-10 16:24:16 +03:00
Jeff Wendling
28353823e8 jenkins: remove Jenkinsfile.gerrit (#2953) 2019-09-06 08:54:46 +03:00
Egon Elbre
c4ee7eb51c
jenkins: build and test web/satellite (#2908) 2019-09-05 16:32:46 +03:00
Bryan White
62df8ddb0c cmd/certificates: auth export improvements & certificates test script (#2897) 2019-09-04 19:58:38 +02:00
Matt Robinson
1b2a2d045f
[build]: Adjust arm63v6 and aarch64 images to match convention (#2845)
* Adjust arm32v6 and aarch64 docker images to match the hello-world image

* Update from master, fix potential bug in push-images target, and update storagenode deploy to handle arm64 image
2019-08-28 08:18:56 -04:00
Matt Robinson
ac58589615 Add aarch64 support and use go 1.12.9 (#2814) 2019-08-19 12:40:16 +03:00
Brandon Iglesias
5c61c79ab7 Start building binary for the link sharing service (#2790) 2019-08-14 15:04:56 -04:00
Egon Elbre
7fcbec48f7 go.mod: update grpc-go and go (#2777) 2019-08-14 06:18:10 -04:00
Alexander Leitner
f3db409103 lib/uplinkc: make .so smaller by removing symbol information (#2730) 2019-08-13 15:24:31 +03:00
Jess G
2a464bfdb5
Jg/1622 add backwards compatibility test (#2656)
* add stage to jenkins, add script for backwards compat tests

* debug backwards compat tests

* add setup in a script

* set env vars

* add more env ars

* mv to one script to debug

* debug api key problem

* add tmp dir

* rm print statements
2019-08-06 16:03:14 -07:00
Egon Elbre
c5de36ed1f
update to go 1.12.7 (#2493) 2019-07-09 11:56:08 -04:00
Egon Elbre
e83ebd7cde
jenkins: avoid using goimports and distribute load better (#2359) 2019-06-27 21:52:50 +03:00
Egon Elbre
86ec9b9570
jenkins: capability to use leakcheck (#2179) 2019-06-21 17:57:38 +03:00
Matt Robinson
89cd4d7bc0 Use a persistent cache and pkg directory (#2235) 2019-06-18 13:18:09 -04:00
Bryan White
76061c50c5
Uplink C bindings part 2 (#2209) 2019-06-15 13:23:12 +02:00
Egon Elbre
749846b42b
Update golangci-lint (#2159) 2019-06-10 11:52:09 +03:00
Egon Elbre
9ed8c96058 jenkins: check that lock file is updated properly (#1977) 2019-06-04 19:12:27 +02:00
Matt Robinson
4544542b43 I think this fixes the weird '-heads-' in the version tags (#2010) 2019-05-21 16:36:08 +02:00
Matt Robinson
cbb323ad52 Give the bootstrap image the business. (#2005) 2019-05-21 13:57:20 +02:00
Ivan Fraixedes
838476f164 Makefile: test-satellite-cfg-change improvements (#1915)
* Clarifies that the test-satellite-cfg-change is for being run by
  Jenkins as others are.
* Move the target for updating the satellite configuration lock file to
  a new Makefile group, called "Tooling".
* Change how it's run the script update-satellite-cfg-lock.sh for
  ensuring that the outcome is the same when running from different
  machines.
2019-05-09 16:25:44 +03:00
Matt Robinson
bcf9a809b6
Update golang version used for release builds (#1926) 2019-05-08 15:20:00 -04:00
Ivan Fraixedes
f8df461249 v3-1660: Create a test that breaks when satellite config file has changed (#1898)
* scripts: check for satellite configuration changes

Create a simple script that checks if the satellite configuration has
differed with the last changes.

* Makefile: add target to check satellite cfg changes

Add a Makefile target which executes the script that checks if the last
changes has made a change in the satellite configuration.

* ci: add satellite cfg check on integration stage

* FIXUP: use releae defaults rather than development ones

* FIXUP: show the message when config differs & some cleanups

* scripts: add script to update the satellite cfg lock

Add a script for allowing to update the satellite configuration lock
file and add Makefile target to run it.

* scripts/testsdata: update satellite cfg lock

Update the satellite configuration lock file with the last changes that
satellite has suffered upstream.
2019-05-07 17:20:04 -04:00
Matt Robinson
9f98450824 Fix Jenkins master builds (#1889)
* Missed a bit to make jenkins build the versioncontrol image correctly
2019-05-03 14:14:38 -04:00
Matt Robinson
43ae28ce2c Add image for versioncontrol server (#1875)
* First pass at building a version server container image

* Actually adding the Dockerfile this branch

* Add entrypoint to auto create config file
2019-05-02 23:25:04 +02:00
Egon Elbre
ba322abd9c
Remove Travis (#1868) 2019-04-30 21:18:32 +03:00
Matt Robinson
d843f80773 account for the new pod name (#1846) 2019-04-26 08:08:42 +03:00
Matt Robinson
ca2055abaf
Clean up the satellite image (#1768)
This is working on saturn and jupiter, plus travis is ok with it.
2019-04-16 17:06:59 -04:00
Matt Robinson
1b8976ab9a
Missed a part in the last attempt (#1760)
Again, appologies.
2019-04-15 15:04:02 -04:00
Matt Robinson
ac66260638
Manifests are derived from a specific image, not the magical latest tag (#1759)
Again, sorry for using my admin powers.
2019-04-15 14:30:36 -04:00
Matt Robinson
a81d13920f
Remove tracked branches for now (#1758)
@jtolds  made me do it
2019-04-15 13:49:08 -04:00
Philip Hutchins
04814a3f3d
Fixing upload and build binaries (#1751) 2019-04-12 13:28:04 -04:00
Matt Robinson
c4ba8831c3 Bump to 1.12.1 and use scripts/release.sh to build things (#1676) 2019-04-12 11:31:31 -04:00
Egon Elbre
0eee46524d update linter to v1.16 (#1741)
* update to v1.16

* use full version number

* fix linter issues

* fix comment

* nicer comment

* restart travis

* restart travis
2019-04-11 13:32:40 -04:00
Matt Robinson
e478d0855c
Build multiarch ARM32v6 and AMD64 images (#1686) 2019-04-08 16:01:20 -04:00
Stefan Benten
2cf86703a3
Add Versioning Server (#1576)
* Initial Webserver Draft for Version Controlling

* Rename type to avoid confusion

* Move Function Calls into Version Package

* Fix Linting and Language Typos

* Fix Linting and Spelling Mistakes

* Include Copyright

* Include Copyright

* Adjust Version-Control Server to return list of Versions

* Linting

* Improve Request Handling and Readability

* Add Configuration File Option
Add Systemd Service file

* Add Logging to File

* Smaller Changes

* Add Semantic Versioning and refuses outdated Software from Startup (#1612)

* implements internal Semantic Version library

* adds version logging + reporting to process

* Advance SemVer struct for easier handling

* Add Accepted Version Store

* Fix Function

* Restructure

* Type Conversion

* Handle Version String properly

* Add Note about array index

* Set temporary Default Version

* Add Copyright

* Adding Version to Dashboard

* Adding Version Info Log

* Renaming and adding CheckerProcess

* Iteration Sync

* Iteration V2

* linting

* made LogAndReportVersion a go routine

* Refactor to Go Routine

* Add Context to Go Routine and allow Operation if Lookup to Control Server fails

* Handle Unmarshal properly

* Linting

* Relocate Version Checks

* Relocating Version Check and specified default Version for now

* Linting Error Prevention

* Refuse Startup on outdated Version

* Add Startup Check Function

* Straighten Logging

* Dont force Shutdown if --dev flag is set

* Create full Service/Peer Structure for ControlServer

* Linting

* Straighting Naming

* Finish VersionControl Service Layout

* Improve Error Handling

* Change Listening Address

* Move Checker Function

* Remove VersionControl Peer

* Linting

* Linting

* Create VersionClient Service

* Renaming

* Add Version Client to Peer Definitions

* Linting and Renaming

* Linting

* Remove Transport Checks for now

* Move to Client Side Flag

* Remove check

* Linting

* Transport Client Version Intro

* Adding Version Client to Transport Client

* Add missing parameter

* Adding Version Check, to set Allowed = true

* Set Default to true, testing

* Restructuring Code

* Uplink Changes

* Add more proper Defaults

* Renaming of Version struct

* Dont pass Service use Pointer

* Set Defaults for Versioning Checks

* Put HTTP Server in go routine

* Add Versioncontrol to Storj-Sim

* Testplanet Fixes

* Linting

* Add Error Handling and new Server Struct

* Move Lock slightly

* Reduce Race Potentials

* Remove unnecessary files

* Linting

* Add Proper Transport Handling

* small fixes

* add fence for allowed check

* Add Startup Version Check and Service Naming

* make errormessage private

* Add Comments about VersionedClient

* Linting

* Remove Checks that refuse outgoing connections

* Remove release cmd

* Add Release Script

* Linting

* Update to use correct Values

* Move vars private and set minimum default versions for testing builds

* Remove VersionedClient

* Better Error Handling and naked return removal

* Straighten the Regex and string conversion

* Change Check to allows testplanet and storj-sim to run without the
need to pass an LDFlag

* Cosmetic Change to Dashboard

* Cleanup Returns and remove commented code

* Remove Version Check if no build options are passed in

* Pass in Config Values instead of Pointers

* Handle missed Error

* Update Endpoint URL

* Change Type of Release Flag

* Add additional Logging

* Remove Versions Logging of other Services

* minor fixes

Change-Id: I5cc04a410ea6b2008d14dffd63eb5f36dd348a8b
2019-04-03 21:13:39 +02:00
Matt Robinson
60035caade
Fix image tags (#1541) 2019-03-21 07:48:39 -04:00
Philip Hutchins
eeb1457478 Fixing tabs (#1423) 2019-03-07 19:58:55 +01:00
Philip Hutchins
af0d5add06
Adding fix for release branch latest tag pushing (#1420)
* Adding fix for release branch latest tag pushign

* Updating how we handle pushing latest branches

* Cleaning up and adding CUSTOMTAG where necessary

* Correcting comment on satellite-ui
2019-03-07 10:22:18 -05:00
Philip Hutchins
64f2eae392
Adding conditional around release branches so that we namespace latest container (#1416) 2019-03-06 20:52:23 -05:00
Philip Hutchins
1f452e67dc Adding dockerfile for running the web UI for Satellite (#1366)
* Adding dockerfile for running the web UI for Satellite

* Updating to work with Makefile and from root directory of repo

* Updating satellite ui build process to run in a more production like mode by generating the assets the pulling those into the satellite container

* Updates to allow external traffic to UI, updates to storagenode for identity creation, and logging for bug tracking

* Adding auto cert generation for storagenode

* removing satellite-ui-image from main images flow in Makefile and adding latest tag to docker build for it

* Adding solid defaults, tuning dockerfiles, and moving to standard logging methods

* Updating logging to be more standard

* Updating to logger.Debug

* Removing unused library and unused identity creation code

Change-Id: I956453037e303693ea37f94318180af0ab7984d5
2019-02-28 13:18:53 -07:00
Bryan White
aa99968c66 add inspector to makefile (#1269) 2019-02-08 10:53:40 +02:00
Bryan White
8edfd43abb
certificates config and --certs-dir fixes (#1093) 2019-01-22 13:35:48 +01:00
Timothy Adams
3cf571abe0
added targets for binaries to fix master (#1104) 2019-01-18 13:15:07 -05:00
Philip Hutchins
45f5648c89 adding identity and certificates to be built and pushed (#1097) 2019-01-18 10:37:52 -05:00
Egon Elbre
bbf81f2479 Consolidate identity management to identity cli commands (#1083)
* Consolidate identity management:

Move identity cretaion/signing out of storagenode setup command.

* fixes

* linters

* Consolidate identity management:

Move identity cretaion/signing out of storagenode setup command.

* fixes

* sava backups before saving signed certs

* add "-prebuilt-test-cmds" test flag

* linters

* prepare cli tests for travis

* linter fixes

* more fixes

* linter gods

* sp/sdk/sim

* remove ca.difficulty

* remove unused difficulty

* return setup to its rightful place

* wip travis

* Revert "wip travis"

This reverts commit 56834849dcf066d3cc0a4f139033fc3f6d7188ca.

* typo in travis.yaml

* remove tests

* remove more

* make it only create one identity at a time for consistency

* add config-dir for consitency

* add identity creation to storj-sim

* add flags

* simplify

* fix nolint and compile

* prevent overwrite and pass difficulty, concurrency, and parent creds

* goimports
2019-01-18 11:36:58 +01:00
Egon Elbre
c89609abbe
Remove captplanet (#1070) 2019-01-17 04:35:22 -05:00
Egon Elbre
9dbb6a101d
Rename storj-sdk to storj-sim (#1078) 2019-01-16 18:09:57 -05:00
Egon Elbre
b86c4dc5eb
add make install-sdk target (#1069) 2019-01-16 13:50:16 -05:00
Egon Elbre
387fb14e23
storj-sdk (#969) 2019-01-08 17:24:15 +02:00
Bryan White
b6611e2800
automate certificate signing in storage node setup (#954) 2019-01-04 18:23:23 +01:00
Matt Robinson
c0e6b62708
Test all-in-one (#900)
* Add test for aio

* Don't trust the user to have images built for a version

* Make travis run the aio test

* Add missing values to docker-compose, sort some things, consider the gateway image

* today's changes

* config changed, again

* more fixes

* Expose satellite port on localhost:7778

* Add retries and a timeout around the big-testfile test in AIO

* Another config value changed

* Make this error message a little more useful

* Fix nil condition
2019-01-03 14:54:27 -05:00
Bryan White
249244536a
CSR Service (part 2): cert signing rpc (#950)
* CSR Service:

+ implement certificate sign rpc method
+ implement certificate signer client/server
+ refactor `AuthorizationDB#Create`
+ refactor `NewTestIdentity`
+ add `AuthorizationDB#Claim`
+ add `Token#Equal`
+ fix `Authorizations#Marshal` when marshaling identities and certificates
+ tweak `Authorization#String` format
+ cert debugging improvements (jsondiff)
+ receive context arg in `NewTestIdentity`
+ misc. fixes
2019-01-02 12:39:17 -05:00
Bryan White
2016ce9fd6
Certificate revocation (#836)
* wip certificate revocation

* refactor tests

* wip testing

* testing

* review fixes

* integration fix attempt #1

* review fixes

* integration fix attempt #2

* linter fixes

* add copywrite

* integration fix attemp #3

* more testing

* more tests

* go mod tidy

* review fixes

* linter fixes
2018-12-13 21:01:43 +01:00
Bryan White
228aa34ff6
TLS extension processing (#771) 2018-12-07 14:44:25 +01:00
Matt Robinson
4237afb1c1
Attempt at fixing the all-in-one environment (#736) 2018-12-04 16:10:23 -05:00
Egon Elbre
3e461df470
Linting protobuf files (#676) 2018-11-30 17:02:01 +02:00
Matt Robinson
471f9004aa
Fix arm builds (#595) 2018-11-26 09:11:09 -05:00
Egon Elbre
42ff835e6e
Fail linter when copyright is missing. (#664) 2018-11-15 14:47:52 +02:00
Matt Robinson
c5f5915588
build the gateway binary too (#592) 2018-11-07 13:19:09 -05:00
Matt Robinson
d1e7534f9e
use GOPROXY if it's set in the environment (#495) 2018-10-17 15:06:26 -04:00
Matt Robinson
48eb8e5c8e
sign windows builds (#494) 2018-10-17 14:21:18 -04:00
Matt Robinson
bcd18a44bc
give our appearence on windows some love (#429) 2018-10-16 14:48:17 -04:00
Matt Robinson
00358fc368
clean up Makefile (#487) 2018-10-16 14:47:43 -04:00
Matt Robinson
a1f93285e2 figure out which deployments need to be patched during a deployment instead of guessing (#438) 2018-10-08 12:52:27 -04:00
Egon Elbre
e7e2d4d7c9
use golangci-lint and add missing error checks (#340) 2018-09-11 16:13:25 +03:00
Egon Elbre
6ee6f0fdf5
Remove duplicated mock store implementations (#337) 2018-09-11 10:27:12 +03:00
Matt Robinson
75c64035a2 Normalize the Docker Bits (#325)
* Normalize the dockerfiles and entrypoints

* fix formattting and docker-compose file

* add missing env var to satellite dockerfile

* Remove build args no longer in use

* More changes to make it all work
2018-09-07 15:46:08 -04:00
Matt Robinson
2c315db84b Fix the binaries and Docker Images (#306)
* Finally fix the binary builds for a couple os/arch combinations

* Make the file executable before adding it to the zip

* This cache hack wasn't suppose to make it into the PR

* Guard against calling the binary target by itself

* Order matters on OS X

* Vendored my image

* No 386 builds until storage is fixed.
2018-09-05 16:40:47 -04:00
Matt Robinson
0a78e741ec
Build and upload binaries (#296)
* First pass at building binaries

* Finished building images

* Reworked some bits to save binaries in a better place.

* First pass at uploading binaries

* Builds for freebsd fail currently

* Add ignore file for docker builds to not invalidate the cache as often

* Docker image for 1.11-alpine exists now

* Name windows binaries correctly

* The makefile is used in tests.

* Take a swing at updating the tests for 1.11

* Switch to something with glibc so the race detector works.

* Remove unused .PHONY targets
2018-08-31 11:21:44 -04:00