* Added the ability to pass timeout settings from cmd/uplink to libuplink.
* Removed commented out code.
* Updated 2min timeouts for the uplink CLI.
* Removed comment.
* Made transport defaultDialTimeout and defaultRequestTimeout public
* Added comments to describe where these defaults apply.
* Added a new defaults to libuplink and added tests.
* Added a new defaults to libuplink and added tests.
* uplink: Mark encryption key config field for setup
Set the "setup" property to the `EncryptionConfig.EncrptionKey` for
avoiding to save it in the configuration file.
This field is only meant for using in the command line parameters which
need to use a different encryption key than the one present in the key
file or use it when there is not set any encryption key file path.
* cmd/uplink: Setup non-interactive accept enc key
Change the uplink CLI setup command non-interactive to save the
encryption key into a file when it's passed through the flag
--enc.encryption-key
Previous to this change it wasn't possible to create an key file despite
of that the flag was provided, so it was useless on the setup command.
* cmd/uplink: Reuse logic to read pwd from terminal
Reuse the logic which is already implemented in the pkg/cfgstruct for
reading a password from the terminal on interactive mode, rather than
duplicating it in the setup command.
* cmd/gateway: Use encryption key file flags
The cmd/gateway was still using the `enc.key` configuration field which
doesn't exist anymore and its setup command wasn't using the
`enc.key-filepath` with combination of the `enc.encryption-key` for
generating a file with the encryption key.
This commit update the cmd/gateway appropriately and move to the uplink
package the function used by cmd/uplink to save the encryption key for
allowing to also be used by the cmd/gateway without duplicating the
logic.
* cmd/storj-sim: Adapt gateway config cmd changes
Adapt the cmd/storj-sim to correctly pass the parameters to the
cmd/gateway setup and run command.
* scripts: Don't pass the --enc.encryption-key flag
uplink configuration has changed to only support the
`--enc.encryption-key` flag for setup commands and consequently the
cmd/uplink and cmd/gateway don't accept this flag over other commands,
hence the test for the uplink had to be updated for no passing the
flag on the multiples calls that the test do to cmd/uplink.
* uplink: Remove func which aren't useful anymore
Remove the function which allows to user or load an encryption key
because it isn't needed anymore since the `--enc.encryption-key` flag is
only available for the setup command.
Consequently remove its usage from cmd/uplink and cmd/gateway, because
such flag will always be empty because in case that's passed Cobra will
return an error due to a "unknown flag".
* change BindSetup to be an option to Bind
* add process.Bind to allow composite structures
* hack fix for noprefix flags
* used tagged version of structs
Before this PR, some flags were created by calling `cfgstruct.Bind` and having their fields create a flag. Once the flags were parsed, `viper` was used to acquire all the values from them and config files, and the fields in the struct were set through the flag interface.
This doesn't work for slices of things on config structs very well, since it can only set strings, and for a string slice, it turns out that the implementation in `pflag` appends an entry rather than setting it.
This changes three things:
1. Only have a `Bind` call instead of `Bind` and `BindSetup`, and make `BindSetup` an option instead.
2. Add a `process.Bind` call that takes in a `*cobra.Cmd`, binds the struct to the command's flags, and keeps track of that struct in a global map keyed by the command.
3. Use `viper` to get the values and load them into the bound configuration structs instead of using the flags to propagate the changes.
In this way, we can support whatever rich configuration we want in the config yaml files, while still getting command like flags when important.
* added scopelint and correcte issues found
* corrected scopelint issue
* made updates based on Ivan's suggestions
Most were around naming conventions
Some were false positives, but I kept them since the test.Run could eventually be changed to run in parallel, which could cause a bug
Others were false positives. Added // nolint: scopelint
* first round cleanup based on go-critic
* more issues resolved for ifelsechain and unlambda checks
* updated from master and gocritic found a new ifElseChain issue
* disable appendAssign. i reports false positives
* re-enabled go-critic appendAssign and disabled lint check at code line level
* fixed go-critic lint error
* fixed // nolint add gocritic specifically
* add aws s3 benchmark script
* add s3 benchmark tests
* rearrange so smaller diff, fix spelling
* add configurable uplink config for s3-benchmark
* make new bucket w/unique name for each s3 test
* changes per CR
* update uplink cmd args
* add uplod/download benchmark, add script to run benchmarks
* fix lint err
* changes per CR comments
* export env var for linux
* fix lint
* add byte count to bench test
Uplink CLI was returning an error when the `--config-dir` flag was
used with interactive `setup` command because the directory used for
storing the encryption key file was the default configuration directory
rather than the value set in the flag.
This changes make the setup process to use the same directory, than the
one informed by the `--config-dir` flag value, for storing encryption
key file.
On the other hand, the default value for the encryption key file path
which was set with the purpose of showing it in the help message has
been removed because the cfgstruct doesn't show it unless that the flag
is set to the Cobra root CMD, so it was useless.
Also show the absolute path where the encryption key file has been saved when
the interactive setup command ends satisfactorily as it's equally done
with the configuration file.
* uplink: Add a new flag to set the filepath of the file which is used for
saving the encryption key and rename the one that hold the encryption key and
establish that it has priority over the key stored in the file to make the
configuration usable without having a huge refactoring in test-sim.
* cmd/uplink: Adapt the setup subcommand for storing the user input key to a file
and adapt the rest of the subcommands for reading the key from the key-file when
the key isn't explicitly set with a command line flag.
* cmd/gateway: Adapt it to read the encryption key from the key-file or use the
one passed by a command line flag.
* pkg/process: Export the default configuration filename so other packages which
use the same value can reference to it rather than having it hardcoded.
* Adapt several integrations (scripts, etc.) to consider the changes applied in uplink and cmd packages.
* cmd/uplink: add share command to restrict an api key
This commit is an early bit of work to just implement restricting
macaroon api keys from the command line. It does not convert
api keys to be macaroons in general.
It also does not apply the path restriction caveats appropriately
yet because it does not encrypt them.
* cmd/uplink: fix path encryption for shares
It should now properly encrypt the path prefixes when adding
caveats to a macaroon.
* fix up linting problems
* print summary of caveat and require iso8601
* make clone part more clear
* tie defaults to releases
this change makes it so that by default, the flag defaults are
chosen based on whether the build was built as a release build or
an ordinary build. release builds by default get release defaults,
whereas ordinary builds by default get dev defaults.
any binary can have its defaults changed by specifying
--defaults=dev
or
--defaults=release
Change-Id: I6d216aa345d211c69ad913159d492fac77b12c64
* make release defaults more clear
this change extends cfgstruct structs to support either
a 'default' tag, or a pair of 'devDefault' and 'releaseDefault'
tags, but not both, for added clarity
Change-Id: Ia098be1fa84b932fdfe90a4a4d027ffb95e249c6
* clarify cfgstruct.DefaultsFlag
Change-Id: I55f2ff9080ebbc0ce83abf956e085242a92f883e
* Initial Webserver Draft for Version Controlling
* Rename type to avoid confusion
* Move Function Calls into Version Package
* Fix Linting and Language Typos
* Fix Linting and Spelling Mistakes
* Include Copyright
* Include Copyright
* Adjust Version-Control Server to return list of Versions
* Linting
* Improve Request Handling and Readability
* Add Configuration File Option
Add Systemd Service file
* Add Logging to File
* Smaller Changes
* Add Semantic Versioning and refuses outdated Software from Startup (#1612)
* implements internal Semantic Version library
* adds version logging + reporting to process
* Advance SemVer struct for easier handling
* Add Accepted Version Store
* Fix Function
* Restructure
* Type Conversion
* Handle Version String properly
* Add Note about array index
* Set temporary Default Version
* Add Copyright
* Adding Version to Dashboard
* Adding Version Info Log
* Renaming and adding CheckerProcess
* Iteration Sync
* Iteration V2
* linting
* made LogAndReportVersion a go routine
* Refactor to Go Routine
* Add Context to Go Routine and allow Operation if Lookup to Control Server fails
* Handle Unmarshal properly
* Linting
* Relocate Version Checks
* Relocating Version Check and specified default Version for now
* Linting Error Prevention
* Refuse Startup on outdated Version
* Add Startup Check Function
* Straighten Logging
* Dont force Shutdown if --dev flag is set
* Create full Service/Peer Structure for ControlServer
* Linting
* Straighting Naming
* Finish VersionControl Service Layout
* Improve Error Handling
* Change Listening Address
* Move Checker Function
* Remove VersionControl Peer
* Linting
* Linting
* Create VersionClient Service
* Renaming
* Add Version Client to Peer Definitions
* Linting and Renaming
* Linting
* Remove Transport Checks for now
* Move to Client Side Flag
* Remove check
* Linting
* Transport Client Version Intro
* Adding Version Client to Transport Client
* Add missing parameter
* Adding Version Check, to set Allowed = true
* Set Default to true, testing
* Restructuring Code
* Uplink Changes
* Add more proper Defaults
* Renaming of Version struct
* Dont pass Service use Pointer
* Set Defaults for Versioning Checks
* Put HTTP Server in go routine
* Add Versioncontrol to Storj-Sim
* Testplanet Fixes
* Linting
* Add Error Handling and new Server Struct
* Move Lock slightly
* Reduce Race Potentials
* Remove unnecessary files
* Linting
* Add Proper Transport Handling
* small fixes
* add fence for allowed check
* Add Startup Version Check and Service Naming
* make errormessage private
* Add Comments about VersionedClient
* Linting
* Remove Checks that refuse outgoing connections
* Remove release cmd
* Add Release Script
* Linting
* Update to use correct Values
* Move vars private and set minimum default versions for testing builds
* Remove VersionedClient
* Better Error Handling and naked return removal
* Straighten the Regex and string conversion
* Change Check to allows testplanet and storj-sim to run without the
need to pass an LDFlag
* Cosmetic Change to Dashboard
* Cleanup Returns and remove commented code
* Remove Version Check if no build options are passed in
* Pass in Config Values instead of Pointers
* Handle missed Error
* Update Endpoint URL
* Change Type of Release Flag
* Add additional Logging
* Remove Versions Logging of other Services
* minor fixes
Change-Id: I5cc04a410ea6b2008d14dffd63eb5f36dd348a8b
* Divide uplink and gateway params set
* attempt to fix docker
* attempt to fix all in one
* test
* more reorganization
* fix compilation error
* fix imports order
* fix dependency
* rename structs
* keep minio params for now
* review comments
* remove manual flag check
