Commit Graph

81 Commits

Author SHA1 Message Date
Michal Niewrzal
ae6bbd8fef
Uplink CLI setup wizard (#1707) 2019-04-10 09:23:20 -04:00
Egon Elbre
1330070209
fix potential truncation of int64 (#1718) 2019-04-10 06:36:27 -04:00
Stefan Benten
2cf86703a3
Add Versioning Server (#1576)
* Initial Webserver Draft for Version Controlling

* Rename type to avoid confusion

* Move Function Calls into Version Package

* Fix Linting and Language Typos

* Fix Linting and Spelling Mistakes

* Include Copyright

* Include Copyright

* Adjust Version-Control Server to return list of Versions

* Linting

* Improve Request Handling and Readability

* Add Configuration File Option
Add Systemd Service file

* Add Logging to File

* Smaller Changes

* Add Semantic Versioning and refuses outdated Software from Startup (#1612)

* implements internal Semantic Version library

* adds version logging + reporting to process

* Advance SemVer struct for easier handling

* Add Accepted Version Store

* Fix Function

* Restructure

* Type Conversion

* Handle Version String properly

* Add Note about array index

* Set temporary Default Version

* Add Copyright

* Adding Version to Dashboard

* Adding Version Info Log

* Renaming and adding CheckerProcess

* Iteration Sync

* Iteration V2

* linting

* made LogAndReportVersion a go routine

* Refactor to Go Routine

* Add Context to Go Routine and allow Operation if Lookup to Control Server fails

* Handle Unmarshal properly

* Linting

* Relocate Version Checks

* Relocating Version Check and specified default Version for now

* Linting Error Prevention

* Refuse Startup on outdated Version

* Add Startup Check Function

* Straighten Logging

* Dont force Shutdown if --dev flag is set

* Create full Service/Peer Structure for ControlServer

* Linting

* Straighting Naming

* Finish VersionControl Service Layout

* Improve Error Handling

* Change Listening Address

* Move Checker Function

* Remove VersionControl Peer

* Linting

* Linting

* Create VersionClient Service

* Renaming

* Add Version Client to Peer Definitions

* Linting and Renaming

* Linting

* Remove Transport Checks for now

* Move to Client Side Flag

* Remove check

* Linting

* Transport Client Version Intro

* Adding Version Client to Transport Client

* Add missing parameter

* Adding Version Check, to set Allowed = true

* Set Default to true, testing

* Restructuring Code

* Uplink Changes

* Add more proper Defaults

* Renaming of Version struct

* Dont pass Service use Pointer

* Set Defaults for Versioning Checks

* Put HTTP Server in go routine

* Add Versioncontrol to Storj-Sim

* Testplanet Fixes

* Linting

* Add Error Handling and new Server Struct

* Move Lock slightly

* Reduce Race Potentials

* Remove unnecessary files

* Linting

* Add Proper Transport Handling

* small fixes

* add fence for allowed check

* Add Startup Version Check and Service Naming

* make errormessage private

* Add Comments about VersionedClient

* Linting

* Remove Checks that refuse outgoing connections

* Remove release cmd

* Add Release Script

* Linting

* Update to use correct Values

* Move vars private and set minimum default versions for testing builds

* Remove VersionedClient

* Better Error Handling and naked return removal

* Straighten the Regex and string conversion

* Change Check to allows testplanet and storj-sim to run without the
need to pass an LDFlag

* Cosmetic Change to Dashboard

* Cleanup Returns and remove commented code

* Remove Version Check if no build options are passed in

* Pass in Config Values instead of Pointers

* Handle missed Error

* Update Endpoint URL

* Change Type of Release Flag

* Add additional Logging

* Remove Versions Logging of other Services

* minor fixes

Change-Id: I5cc04a410ea6b2008d14dffd63eb5f36dd348a8b
2019-04-03 21:13:39 +02:00
Egon Elbre
de15a4fdcf
remove utils.CombineErrors and utils.ErrorGroup (#1603) 2019-03-29 14:30:23 +02:00
Kaloyan Raev
30dfc2b20c
Remove PointerDB client (#1520) 2019-03-22 11:01:49 +02:00
Bill Thorp
66718cc5e6
Development defaults for configuration (#1430)
added --dev command line option, cfgstruct.DevFlag(), and cfgstruct.SetupFlag()
2019-03-12 08:51:06 -04:00
Benjamin Sirb
7e2e4b5397 Update README.md (#1400) 2019-03-05 09:54:49 +02:00
Cameron
018cdd4037
uplink ttl (#1365)
* add expires flag to uplink cp

* fix ttlDBs delete expired query
2019-02-28 12:51:24 -05:00
Michal Niewrzal
c7ad427d95
Move metainfo config (#1234)
* Move metainfo config

* move config to uplink

* reorganize code

* use satellite apikey

* comment update
2019-02-08 13:57:35 +01:00
Michal Niewrzal
383cb05e8e
Divide uplink and gateway params set (#1026)
* Divide uplink and gateway params set

* attempt to fix docker

* attempt to fix all in one

* test

* more reorganization

* fix compilation error

* fix imports order

* fix dependency

* rename structs

* keep minio params for now

* review comments

* remove manual flag check
2019-02-04 16:50:06 +01:00
Egon Elbre
d5346982c2
Delete provider package (#1177) 2019-01-30 22:47:21 +02:00
Egon Elbre
6132ce86b7
Remove utils.LogClose (#1169) 2019-01-29 22:42:27 +02:00
Egon Elbre
04d3699107
storj-sim configs (#1140) 2019-01-28 16:48:49 +02:00
Jennifer Li Johnson
856b98997c
updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
Egon Elbre
99d3b7a3c8
Fix import grouping (#1111) 2019-01-22 17:48:23 +02:00
Bryan White
322a2813f7
oops (#1110) 2019-01-22 15:34:40 +01:00
Egon Elbre
6c48d69c38
Disable mounting for bsd-like systems (#1109)
* Disable mounting for bsd-like
* Remove debug code
2019-01-22 15:28:57 +02:00
Bryan White
8edfd43abb
certificates config and --certs-dir fixes (#1093) 2019-01-22 13:35:48 +01:00
Egon Elbre
bbf81f2479 Consolidate identity management to identity cli commands (#1083)
* Consolidate identity management:

Move identity cretaion/signing out of storagenode setup command.

* fixes

* linters

* Consolidate identity management:

Move identity cretaion/signing out of storagenode setup command.

* fixes

* sava backups before saving signed certs

* add "-prebuilt-test-cmds" test flag

* linters

* prepare cli tests for travis

* linter fixes

* more fixes

* linter gods

* sp/sdk/sim

* remove ca.difficulty

* remove unused difficulty

* return setup to its rightful place

* wip travis

* Revert "wip travis"

This reverts commit 56834849dcf066d3cc0a4f139033fc3f6d7188ca.

* typo in travis.yaml

* remove tests

* remove more

* make it only create one identity at a time for consistency

* add config-dir for consitency

* add identity creation to storj-sim

* add flags

* simplify

* fix nolint and compile

* prevent overwrite and pass difficulty, concurrency, and parent creds

* goimports
2019-01-18 11:36:58 +01:00
paul cannon
681d77c41a
use _real_ cert-path to get node ID for metrics (#1058)
apparently, the presence of an "Identity" attribute in a config struct
does not imply that anything will fill it in or use it.

i tested all of these executables manually and individually this time.
2019-01-16 15:43:29 -05:00
paul cannon
248ee6438f
use Node ID for metrics ID (#1052)
This change removes automatic metrics reporting for everything going
through process.Exec(), and re-adds metrics reporting for those commands
which are expected to be long-lived. Other commands (which may have been
intermittently sending metrics before this, if they ran unusually long)
will no longer send any metrics.

For commands where it makes sense, a node ID is used as the metrics ID.
2019-01-15 10:02:54 -05:00
Kaloyan Raev
a63abf8fab
Save only user-specific flags to storage node's config.yaml (#1051) 2019-01-15 15:55:33 +02:00
Maximillian von Briesen
43ed9dea86
Remove overwrite flags (#1040)
remove overwrite flags from uplink, satellite, storagenode, captplanet, bootstrap

https://storjlabs.atlassian.net/browse/V3-1013
2019-01-14 10:57:58 -05:00
Philip Hutchins
b5f51014d3
Adding additional required values for satellite configuration (#1036)
* Adding other top level vars to be parsed into config

* Adding additional vars for parsing

* Fixing shell script conditionals in remaining entrypoints

* Adding null default for remaining env variable if statements
2019-01-11 22:47:18 -05:00
Michal Niewrzal
58b16c2ada
Fix for uplink setup command (#989) 2019-01-08 14:30:07 +01:00
Michal Niewrzal
4e857ea133
Add setup tag for config structs (#968)
* Fix ignored setup arguments

* fix linter errors

* small params cleanup

* fix integration tests

* cleanup in configs

* Add `setup` tag for config structs

* fix broken if statement

* cleanup captplanet config

* remove reduntant return

* add missing Signer config

* review comments

* local variable renamed

* remove unused var
2019-01-07 12:06:10 +01:00
Michal Niewrzal
4be58e2a76
Fix ignored setup arguments (#944)
* Fix ignored setup arguments

* fix linter errors

* small params cleanup

* fix integration tests

* cleanup in configs

* Restart Travis
2019-01-07 10:48:16 +01:00
JT Olio
2c916a04c3 pkg/provider: split into pkg/server, pkg/identity (#953) 2019-01-02 12:23:25 +02:00
Matt Robinson
5224e92646
Fix detection of existing config in docker images (#929)
* If the directory is somehow persistent, but the config file itself isn't created yet, create it

* Other improvements to thd storagenode Docker image

* Other improvements to thd satellite Docker image

* Other improvements to thd gateway Docker image

* Other improvements to thd uplink Docker image
2018-12-21 11:56:57 -05:00
Michal Niewrzal
f0662675bc
Add '--dir' param for all CLI parts (replace --base-path) (#833)
* Add '--dir' param for all CLI parts (replace --base-path)

* FindDirParam method moved

* fix compilation error

* make param global

* remove unused fields

* rename param

* remove config flag

* goimports
2018-12-14 22:14:59 +01:00
Kaloyan Raev
4cc49a8fa3
Better error handling of stream.Upload.Close (#811) 2018-12-10 16:35:22 +02:00
Kaloyan Raev
a4dec2da93
Fix "key not found" error on uploading files with uplink cli (#807) 2018-12-10 12:13:30 +02:00
Kaloyan Raev
72a20af569
Migrate Minio Gateway to the new Metainfo API (#775) 2018-12-07 20:31:29 +02:00
Bryan White
228aa34ff6
TLS extension processing (#771) 2018-12-07 14:44:25 +01:00
JT Olio
b6c936340b
clean up uplink config section (#786) 2018-12-06 17:18:11 -07:00
Kaloyan Raev
b7c45c7c16
Migrate Uplink CLI to Metainfo API (#745) 2018-12-04 18:46:02 +02:00
aligeti
c11a4385ae
upload filename fix (#760) 2018-12-04 10:02:12 -05:00
Michal Niewrzal
06c196d28c
Allow config generation into empty directory (#752)
* Allow config generation into empty directory

* unifed messages

* move main method to fpath

* fix linter problems
2018-12-04 12:42:16 +01:00
Michal Niewrzal
1206ef9126
Use OS related default setup directory (#741)
* Use OS related default setup directory

* added missing comment

* adjust test-captplanet.sh

* ApplicationDir moved to fpath
2018-12-03 16:51:56 +01:00
Egon Elbre
3e461df470
Linting protobuf files (#676) 2018-11-30 17:02:01 +02:00
Natalie Villasana
934e9c8dac
fixes Linux default base path for uplink (#730) 2018-11-28 14:17:58 -05:00
Natalie Villasana
d8b343ff3a
Fix default basepath location (#684) 2018-11-27 14:09:25 -05:00
Kaloyan Raev
8c56198090
Use designated error classes for common bucket and object errors (#691) 2018-11-21 13:17:28 +02:00
Michal Niewrzal
7e40caafcd
Write access for uplink mount command (#587)
* Write access for uplink mount command

* fix linter problems

* Fix closing wrong variable and debug logs

* goimports

* fix problem with missing getattr for large files

* better map initialization

* fixes after comments

* add support for making directory

* add support for removing directories

* optimize OpenDir and Rmdir methods

* use storj.JoinPaths

* fix linter issues

* remove useless error check
2018-11-21 11:19:54 +01:00
Michal Niewrzal
b1cab0fc5a
Fix for uplink cat writing additional bytes (#655)
* Fix for uplink cat writing additional bytes

* rename progress parameter
2018-11-15 16:21:58 +01:00
Kaloyan Raev
5cdb03515f
Configure path encryption in metainfo API (#644) 2018-11-14 11:26:18 +02:00
Michal Niewrzal
a62b413de2
Uplink mount better prefix handling (#541)
* Uplink mount better prefix handling

* fix formatting
2018-10-31 21:38:32 +01:00
Michal Niewrzal
79e1ed5d1f
Fix uplink mount for latest api (#539)
Fix uplink mount for changes in store.List
2018-10-26 16:22:20 +02:00
JT Olio
fdfb5f1da9
uplink: fix mount and cli/gateway logical merge conflict (#537)
Change-Id: Ib1661e1c9d3f06783854b2293579ab715d0f1c20
2018-10-25 16:00:15 -06:00
JT Olio
cd51787827 split gateway from uplink
Change-Id: I7504a630cb85539750e32780c1c2347742a9c9c4
Reviewed-on: https://review.gerrithub.io/430644
Reviewed-by: Kaloyan Raev <kaloyan@storj.io>
Tested-by: Kaloyan Raev <kaloyan@storj.io>
2018-10-25 15:01:44 -06:00