* hardcode debug ports for storj-sim
* add function to create port
* add 200 limit to sn
* remove iota and mv instance count
* remove singleIndex var
* remove err checking from port fx
* mv unneeded conversion
What: This change moves project-level bucket metadata encryption information to the volatile section, because it is unlikely to remain in future releases
Why: Ultimately, the web user interface will allow bucket management (creation, removal, etc), but not object management as that requires an encryption key for sure and we don't want to have users give the satellite their encryption keys.
At a high level, a (*Project) type should map to all of the things you can do inside the web user interface within a project, which by necessity cannot have an encryption key. So, we really don't want an encryption key in the non-volatile section of this library.
* releases: update alpha dockerfiles to embed release tags
Change-Id: I2978d3ed77d1b768be24ec9194bc404cd92e4700
* release script use plain shell
Change-Id: Ie4ca6633014e3bf00ac0c82e28905174a6ade95b
* removed /scripts from .dockerignore
Change-Id: Ib555236d62fb3dc354689284ac4a032d5b4b3aa8
* switch back to bash
Change-Id: Ie0958cde71cd396f44ac28c177f48df0bea7ba22
* fix go1.12 modules
Change-Id: If4c8364e7271f09febd1a6c8bfc731d723d344d9
* make sure the work tree is clean
Change-Id: Ie6e52af28e2bc52880d92a3b574d1f859fc8e88e
* internal/version: do version checks much earlier in the process initialization, take 2
Change-Id: Ida8c7e3757e0deea0ec7aea867d3d27ce97dc134
* linter and test failures
Change-Id: I45b02a16ec1c0f0981227dc842e68dbdf67fdbf4
Make separate "CreateCertificate" and "CreateSelfSignedCertificate"
functions to take the two roles of NewCert. These names should help
clarify that they actually make certificates and not just allocate new
"Cert" or "Certificate" objects.
Secondly, in the case of non-self-signed certs, require a public and a
private key to be passed in instead of two private keys, because it's
pretty hard to tell when reading code which one is meant to be the
signer and which one is the signee. With a public and private key, you
know.
(These are some changes I made in the course of the openssl port,
because the NewCert function kept being confusing to me. It's possible
I'm just being ridiculous, and this doesn't help improve readability for
anyone else, but if I'm not being ridiculous let's get this in)
* Initial Webserver Draft for Version Controlling
* Rename type to avoid confusion
* Move Function Calls into Version Package
* Fix Linting and Language Typos
* Fix Linting and Spelling Mistakes
* Include Copyright
* Include Copyright
* Adjust Version-Control Server to return list of Versions
* Linting
* Improve Request Handling and Readability
* Add Configuration File Option
Add Systemd Service file
* Add Logging to File
* Smaller Changes
* Add Semantic Versioning and refuses outdated Software from Startup (#1612)
* implements internal Semantic Version library
* adds version logging + reporting to process
* Advance SemVer struct for easier handling
* Add Accepted Version Store
* Fix Function
* Restructure
* Type Conversion
* Handle Version String properly
* Add Note about array index
* Set temporary Default Version
* Add Copyright
* Adding Version to Dashboard
* Adding Version Info Log
* Renaming and adding CheckerProcess
* Iteration Sync
* Iteration V2
* linting
* made LogAndReportVersion a go routine
* Refactor to Go Routine
* Add Context to Go Routine and allow Operation if Lookup to Control Server fails
* Handle Unmarshal properly
* Linting
* Relocate Version Checks
* Relocating Version Check and specified default Version for now
* Linting Error Prevention
* Refuse Startup on outdated Version
* Add Startup Check Function
* Straighten Logging
* Dont force Shutdown if --dev flag is set
* Create full Service/Peer Structure for ControlServer
* Linting
* Straighting Naming
* Finish VersionControl Service Layout
* Improve Error Handling
* Change Listening Address
* Move Checker Function
* Remove VersionControl Peer
* Linting
* Linting
* Create VersionClient Service
* Renaming
* Add Version Client to Peer Definitions
* Linting and Renaming
* Linting
* Remove Transport Checks for now
* Move to Client Side Flag
* Remove check
* Linting
* Transport Client Version Intro
* Adding Version Client to Transport Client
* Add missing parameter
* Adding Version Check, to set Allowed = true
* Set Default to true, testing
* Restructuring Code
* Uplink Changes
* Add more proper Defaults
* Renaming of Version struct
* Dont pass Service use Pointer
* Set Defaults for Versioning Checks
* Put HTTP Server in go routine
* Add Versioncontrol to Storj-Sim
* Testplanet Fixes
* Linting
* Add Error Handling and new Server Struct
* Move Lock slightly
* Reduce Race Potentials
* Remove unnecessary files
* Linting
* Add Proper Transport Handling
* small fixes
* add fence for allowed check
* Add Startup Version Check and Service Naming
* make errormessage private
* Add Comments about VersionedClient
* Linting
* Remove Checks that refuse outgoing connections
* Remove release cmd
* Add Release Script
* Linting
* Update to use correct Values
* Move vars private and set minimum default versions for testing builds
* Remove VersionedClient
* Better Error Handling and naked return removal
* Straighten the Regex and string conversion
* Change Check to allows testplanet and storj-sim to run without the
need to pass an LDFlag
* Cosmetic Change to Dashboard
* Cleanup Returns and remove commented code
* Remove Version Check if no build options are passed in
* Pass in Config Values instead of Pointers
* Handle missed Error
* Update Endpoint URL
* Change Type of Release Flag
* Add additional Logging
* Remove Versions Logging of other Services
* minor fixes
Change-Id: I5cc04a410ea6b2008d14dffd63eb5f36dd348a8b
* test-network-stalls tests... network stalls!
in particular, right now, it just tests whether an uplink correctly
times out after some amount of time when one of the nodes it's talking
to suddenly goes offline.
This tool is meant to be run under `storj-sim network test`.
Also included here:
* fix storj-sim-related test scripts on Mac
the default storj config dir on Mac has a space in it
('~/Library/Application Support/Storj'), which breaks everywhere it
shows up in an unquoted variable in a sh/bash script. easy enough to fix
as a one-off, but quoting bash vars avoids a dozen other potential
problems too.
change a few things using `head -c` to use `dd`. `head -c` works,
but is not as widely understood (as evidenced by each of these scripts
getting through code review, one at a time, with the comments not
matching the numbers actually used).
* storj-sim reports PIDs of worker processes to test
so that the tests can cause unfortunate "accidents" to befall the worker
processes in the course of the test, and find out whether everything
reacts correctly.
* define irreparable inspector protobuf
* add IrreparableDB method GetLimited
* fill out irreparable inspector API
* add IrreparableInspector server to satellite, fix small error
* refactor IrreparableDB to use pb.IrreparableSegment instead of irreparable.RemoteSegmentInfo
* Warn about permissions when creating identity
* Function to determine if directory is writeable
* Check if writable before authorizing
* Remove redeclatarion
* remove windows specific utils
* Nat nits
* Actually test if directory is writeable with file creation
* add private listener to grpc server
* add changes per init CR
* fix server.close
* add insecure grpc connection, update logs msg
* fix tests, move insecure client
* add private ports to storj-sim, add insecure client to other inspectors
* add ports to test so there arent conflicts
* fix lint err
* fix node started log msg, close public listener
* remove commented out line
* small identity refactor:
+ Optimize? iterative cert chain methods to use array instead of slice
+ Add `ToChain` helper for converting 1d to 2d cert chain
TODO: replace literal declarations with this
+ rename `ChainRaw/RestChainRaw` to `RawChain/RawRestChain`
(adjective noun, instead of nound adjective)
* add regression tests for V3-1320
* fix V3-1320
* separate `DialUnverifiedIDOption` from `DialOption`
* separate `PingNode` and `DialNode` from `PingAddress` and `DialAddress`
* update node ID while bootstrapping
* goimports & fix comment
* add test case
* Wiring up DumpNodes response for Inspector
* Finalize everything and test that it works
* Get Count and DumpNodes working for Overlay Cache
* WIP updating payment rollup to check statDB instead of overlay
* FIrst pass at updating statDB to take wallet and email
* Passing tests
* use pb.NodeOperator instead of Meta struct
* remove TODO
* revert go.mod
* Get SQL migration working correctly
* Changes Meta to Operator in NodeStats struct
* Adds update operator logic for statDB
* Fix db migrate tests - added v5 snapshot
* User friendly msg for missing snapshot version
* Passing tests
* Change node update to happen in discovery instead of in overlay
* Fix logic and update function calls
* Update comment on UpdateOperator interface method
* Update name of parameter
* Change type of argument to UpdateOperator
* Updates statDB tests
* Adding dockerfile for running the web UI for Satellite
* Updating to work with Makefile and from root directory of repo
* Updating satellite ui build process to run in a more production like mode by generating the assets the pulling those into the satellite container
* Updates to allow external traffic to UI, updates to storagenode for identity creation, and logging for bug tracking
* Adding auto cert generation for storagenode
* removing satellite-ui-image from main images flow in Makefile and adding latest tag to docker build for it
* Adding solid defaults, tuning dockerfiles, and moving to standard logging methods
* Updating logging to be more standard
* Updating to logger.Debug
* Removing unused library and unused identity creation code
Change-Id: I956453037e303693ea37f94318180af0ab7984d5
Adds a new `Info` method to the Kademlia endpoint that returns the following local node info:
* ID
* Type
* Metadata (email and wallet)
* Restrictions (free storage and bandwidth)
The new endpoint is exposed as `inspector kad node-info` command too.
this bandaid-fixes the identity tool issues (revocation
db defaults to /revocations.db due to the missing CONFDIR)
Change-Id: Ibdc8d1e9b64ec9a545bea592b0bd167ff5138117
* separate TLS options from server options (because we need them for dialing too)
* stop creating transports in multiple places
* ensure that we actually check revocation, whitelists, certificate signing, etc, for all connections.
this change removes the cryptopasta dependency.
a couple possible sources of problem with this change:
* the encoding used for ECDSA signatures on SignedMessage has changed.
the encoding employed by cryptopasta was workable, but not the same
as the encoding used for such signatures in the rest of the world
(most particularly, on ECDSA signatures in X.509 certificates). I
think we'll be best served by using one ECDSA signature encoding from
here on, but if we need to use the old encoding for backwards
compatibility with existing nodes, that can be arranged.
* since there's already a breaking change in SignedMessage, I changed
it to send and receive public keys in raw PKIX format, instead of
PEM. PEM just adds unhelpful overhead for this case.
* Divide uplink and gateway params set
* attempt to fix docker
* attempt to fix all in one
* test
* more reorganization
* fix compilation error
* fix imports order
* fix dependency
* rename structs
* keep minio params for now
* review comments
* remove manual flag check
* move payments command into satellite/main.go
* flag for db connection string in paymentsCmd
* refactor payments to satellite subcommand
* reports command, add payments arg descriptions
* report data prints to stdout unless --out is set
* fix small error in csv columns
* Consolidate identity management:
Move identity cretaion/signing out of storagenode setup command.
* fixes
* linters
* Consolidate identity management:
Move identity cretaion/signing out of storagenode setup command.
* fixes
* sava backups before saving signed certs
* add "-prebuilt-test-cmds" test flag
* linters
* prepare cli tests for travis
* linter fixes
* more fixes
* linter gods
* sp/sdk/sim
* remove ca.difficulty
* remove unused difficulty
* return setup to its rightful place
* wip travis
* Revert "wip travis"
This reverts commit 56834849dcf066d3cc0a4f139033fc3f6d7188ca.
* typo in travis.yaml
* remove tests
* remove more
* make it only create one identity at a time for consistency
* add config-dir for consitency
* add identity creation to storj-sim
* add flags
* simplify
* fix nolint and compile
* prevent overwrite and pass difficulty, concurrency, and parent creds
* goimports
* Add more info to SN logs
* remove config-dir from user config
* add output where config was stored
* add message for successful connection
* fix linter
* remove storage.path from user config
* resolve config path
* move success message to info
* log improvements
apparently, the presence of an "Identity" attribute in a config struct
does not imply that anything will fill it in or use it.
i tested all of these executables manually and individually this time.
This change removes automatic metrics reporting for everything going
through process.Exec(), and re-adds metrics reporting for those commands
which are expected to be long-lived. Other commands (which may have been
intermittently sending metrics before this, if they ran unusually long)
will no longer send any metrics.
For commands where it makes sense, a node ID is used as the metrics ID.
* Edit config on Setup
* Default to 1TiB storage space and 500GiB bandwidth
* Use human readable formats
* Use memory
* units of 1024 are measured with KiB/MiB etc
* pkg/cfgstruct: allow values to be configured with human readable sizes
Change-Id: Ic4e9ae461516d1d26fb81f6e44c5ac5cfccf777f
* Modify tests
* Removed comments
* More merge conflict stuff resolved
* Fix lint
* test fixin
Change-Id: I3a008206bf03a4446da19f642a2f9c1f9acaae36
* Remove commented code but secretly leave it in the histroy forever
* Move flag definition to struct
now kad inspector features exist on every server that has
kademlia running. likewise, overlay and statdb.
this means kad inspection features are now available on
storage nodes
Change-Id: I343c873552341de13302bfb7a5d79cccc84fc6b8
* Adding other top level vars to be parsed into config
* Adding additional vars for parsing
* Fixing shell script conditionals in remaining entrypoints
* Adding null default for remaining env variable if statements
* pkg/identity: use sha256 instead of sha3 for pow
Change-Id: I9b7a4f2c3e624a6e248a233e3653eaccaf23c6f3
* pkg/identity: restructure key generation a bit
Change-Id: I0061a5cc62f04b0c86ffbf046519d5c0a154e896
* cmd/identity: indefinite key generation command
you can start this command and leave it running and it will fill up your
hard drive with node certificate authority private keys ordered by
difficulty.
Change-Id: I61c7a3438b9ff6656e74b8d74fef61e557e4d95a
* pkg/storj: more node id difficulty testing
Change-Id: Ie56b1859aa14ec6ef5973caf42aacb4c494b87c7
* review comments
Change-Id: Iff019aa8121a7804f10c248bf2e578189e5b829d
* Add test for aio
* Don't trust the user to have images built for a version
* Make travis run the aio test
* Add missing values to docker-compose, sort some things, consider the gateway image
* today's changes
* config changed, again
* more fixes
* Expose satellite port on localhost:7778
* Add retries and a timeout around the big-testfile test in AIO
* Another config value changed
* Make this error message a little more useful
* Fix nil condition
* If the directory is somehow persistent, but the config file itself isn't created yet, create it
* Other improvements to thd storagenode Docker image
* Other improvements to thd satellite Docker image
* Other improvements to thd gateway Docker image
* Other improvements to thd uplink Docker image
* Add '--dir' param for all CLI parts (replace --base-path)
* FindDirParam method moved
* fix compilation error
* make param global
* remove unused fields
* rename param
* remove config flag
* goimports
* intial changes to migrate statdb to masterdb framework
* statdb refactor compiles
* added TestCreateDoesNotExist testcase
* Initial port of statdb to masterdb framework working
* refactored statdb proto def to pkg/statdb
* removed statdb/proto folder
* moved pb.Node to storj.NodeID
* CreateEntryIfNotExistsRequest moved pd.Node to storj.NodeID
* moved the fields from pb.Node to statdb.UpdateRequest
ported TestUpdateExists, TestUpdateUptimeExists, TestUpdateAuditSuccessExists TestUpdateBatchExists
* WIP possible discovery service impl
* Adds discovery service to CaptPlanet
* Updates the config and server for discovery service
* updates testplanet to use discovery package
* update satellite imports
* Removes unnecessary cache test
* linter fixes
* adds discovery startup to captplanet
* invoke refresh
* updates to discovery refresh cycle
* Make implementation more consistent with previous implementation
* add wait before trying to upload
* sleep a bit more
* remove kademlia bootstrap
* updates
* remove comments
* cmd/statreceiver: lua-scriptable stat receiver
Change-Id: I3ce0fe3f1ef4b1f4f27eed90bac0e91cfecf22d7
* some updates
Change-Id: I7c3485adcda1278fce01ae077b4761b3ddb9fb7a
* more comments
Change-Id: I0bb22993cd934c3d40fc1da80d07e49e686b80dd
* linter fixes
Change-Id: Ied014304ecb9aadcf00a6b66ad28f856a428d150
* catch errors
Change-Id: I6e1920f1fd941e66199b30bc427285c19769fc70
* review feedback
Change-Id: I9d4051851eab18970c5f5ddcf4ff265508e541d3
* errorgroup improvements
Change-Id: I4699dda3022f0485fbb50c9dafe692d3921734ff
* too tricky
the previous thing was better for memory with lots of errors at a time
but https://play.golang.org/p/RweTMRjoSCt is too much of a foot gun
Change-Id: I23f0b3d77dd4288fcc20b3756a7110359576bf44
* Merge bwagreement db into satellite master db
* adjust to recent tally changes
* linter problems
* linter problems
* returning db structs in more optimal way
* added pointer for assignment
* error message changed
* better param message
* adds channel for getting node out of lookup
* WIP adding the channels to lookups
* WIP adding channel to node lookups
* Wires up FindNodes method with channels
* WIP adds a test suite for lookup - tests are still failing
* WIP wires up use of testplanet for kademlia lookup tests
* WIP merging in node id changes
* Merges in pkg/storj node type changes
* Tests passing
* Lookup node working via Inspector now
* updates
* WIP working on getting tests passing
* WIP getting tests passing
* FindNode works
* Linter fix
* Adds copyrights to lookup_test
* removes a fmt.Printf I missed
* Removes commented out lines
* Pulls statdb stats into overlay cache whenever cache.Put() is called
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to filter based on node stats
* Updates overlay.FindStorageNodes()/overlayClient.Choose() to exclude duplicate IP addresses
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* preparing for use of `customtype` gogo extension with `NodeID` type
* review changes
* wip
* tests passing
* wip fixing tests
* more wip test fixing
* remove NodeIDList from proto files
* linter fixes
* linter fixes
* linter/review fixes
* more freaking linter fixes
* omg just kill me - linterrrrrrrr
* travis linter, i will muder you and your family in your sleep
* goimports everything - burn in hell travis
* goimports update
* go mod tidy
* initial commit of inspector gadget wireup
* change name of comman dline tool, setup grpc server
* Get inspector cli working with grpc client
* Wired up CountNodes command
* WIP getting buckets response working
* Added GetBucket command
* WIP working on get buckets command
* WIP working on bucket list
* Still WIP
* WIP getting bucket counts to work
* Some clean up of unnecessary changes
* List Buckets and Get Bucket are working
* Removing logs, getting ready for review
* initial commit of inspector gadget wireup
* change name of comman dline tool, setup grpc server
* Get inspector cli working with grpc client
* Wired up CountNodes command
* WIP getting buckets response working
* Added GetBucket command
* WIP working on get buckets command
* WIP working on bucket list
* Still WIP
* WIP getting bucket counts to work
* Some clean up of unnecessary changes
* List Buckets and Get Bucket are working
* Removing logs, getting ready for review
* Fix error return
* Trying to get tests passing
* Adds method on dht mock for tests
* Add dbx files back
* Fix package import error in dbx file
* Adds copyrights to pass linter
* tidy go mod
* Updates from code review
* Updates inspector to take flag arguments for address
* Format list-buckets output more prettier
* Wiring up PING in kad inspector tools
* initial commit of inspector gadget wireup
* change name of comman dline tool, setup grpc server
* Get inspector cli working with grpc client
* Wired up CountNodes command
* WIP getting buckets response working
* Added GetBucket command
* WIP working on get buckets command
* WIP working on bucket list
* Still WIP
* WIP getting bucket counts to work
* Some clean up of unnecessary changes
* List Buckets and Get Bucket are working
* Removing logs, getting ready for review
* initial commit of inspector gadget wireup
* change name of comman dline tool, setup grpc server
* Get inspector cli working with grpc client
* Wired up CountNodes command
* WIP getting buckets response working
* Added GetBucket command
* WIP working on get buckets command
* WIP working on bucket list
* Still WIP
* WIP getting bucket counts to work
* Some clean up of unnecessary changes
* List Buckets and Get Bucket are working
* Removing logs, getting ready for review
* Fix error return
* Trying to get tests passing
* Adds method on dht mock for tests
* Add dbx files back
* Fix package import error in dbx file
* Adds copyrights to pass linter
* tidy go mod
* Updates from code review
* Updates inspector to take flag arguments for address
* Format list-buckets output more prettier
* Write access for uplink mount command
* fix linter problems
* Fix closing wrong variable and debug logs
* goimports
* fix problem with missing getattr for large files
* better map initialization
* fixes after comments
* add support for making directory
* add support for removing directories
* optimize OpenDir and Rmdir methods
* use storj.JoinPaths
* fix linter issues
* remove useless error check
* Signature verification
* Clean up agreement sender to have less errors
* overlay address in captnplanet
* Refactor bandwidth.proto to not use streams
* Make sure the send worked
* Handle connection to satellite
* Save renter public key inside of renter bandwidth allocations
* Default diag to sqlite. Make configurable
* Separate bw server and dbm; regenerate dbx files
* Make sure test uses protobufs
* Demonstrate creating bandwidth allocations
* WIP starting to wire up the kademlia CLI tool
* WIP wiring up kad cli tools
* WIP starting to wire up the kademlia CLI tool
* WIP wiring up kad cli tools
* Got everything wired up
* WIP starting to wire up the kademlia CLI tool
* WIP wiring up kad cli tools
* merge in upstream
* WIP wiring up kad cli tools
* Got everything wired up
* WIP trying to get CLI to connect
* Inspector connects to overlay now
* Some refactoring
* Linter fixes
* Linter fixes
* Switch to pkg/process instead of using rootCmd.Execute
* add storeConfig struct and getSegmentStore helper for creating a segment store
* implement segment store in repairer, remove unnecessary repairer Repair method
* change repair method parameter from int to int32 to match type being passed in
* implement repairer service in captplanet
* rework Config, set Config defaults in captplanet/setup
The old paths.Path type is now replaced with the new storj.Path.
storj.Path is simply an alias to the built-in string type. As such it can be used just as any string, which simplifies a lot working with paths. No more conversions paths.New and path.String().
As an alias storj.Path does not define any methods. However, any functions applying to strings (like those from the strings package) gracefully apply to storj.Path too. In addition we have a few more functions defined:
storj.SplitPath
storj.JoinPaths
encryption.EncryptPath
encryption.DecryptPath
encryption.DerivePathKey
encryption.DeriveContentKey
All code in master is migrated to the new storj.Path type.
The Path example is also updated and is good for reference: /pkg/encryption/examples_test.go
This PR also resolve a nonce misuse issue in path encryption: https://storjlabs.atlassian.net/browse/V3-545
* implements CombineErrors on runCfg error chan
* implement error channel timeout and CombineErrors
* implements CombineErrors with time.After timeout
* changes d to duration in args
* changes comment to better reflect what collectErrors does
* appeases the linter gods
* Gets rid of timeout bug in collectErrors
* Initial Layout
* Commit to test File Handling OS independed
* Hide struct properties to prevent manual interaction
* Fix Linting Errors
* 1st Working Windows Version
* Add missing Error Handling
* Fix Linting Errors
* Remove dependencies
* Further Improvements
* Remove commented code
* Improve comments and error messages
* No pointers to FPath
* Improve comment
* Do not filepath.ToSlash URL path
* Extract helper functions for parsing local path and Storj path
* Minor Improvements based on PR Comments
* Fix Linting Error and make Regex private
* Improve Layout
* Rework FPath and add tests
* Add more tests cases for windows
* Use for-loop instead of goto
* Use FPath in all uplink commands
* Add guard checks
* Add Test Cases and add comments
* Let's do it right this time
* Oh travis...
* Handle redis URL
* Travis... why u gotta be like this?
* Handle when address does not use redis scheme
* Start repairer
* Match provider.Responsibility interface
* Simplify if statement
* Config doesn't need to be a pointer
* Initialize doesn't need to be exported
* Don't run checker or repairer on startup
* Fix travis complaints
* Creates cron-job for checker, adds it to captplanet and satellite
* removes datarepair from satellite & captplanet run
* Delete config.go
* removes unused datarepair imports
* adds comments to fix linter
* Loads cache from context for PointerDB access
* WIP adds overlay lookups to pointerdb requests
* Pointer lookup code is added for Get
* adds feature flag for pointerdb return
* refactors pointerdb code
* removes some unnecessary debug logs
* Fixes indent in config
* adds early return for non-remote pointers
* formats code, removes some comments
* Fixes tests broken by pointer proto changes
* adds error check and merges variable declaration
* removes commented out proto import
* adds error check to pdbclient
* merged the lasted master changes
* debug working of handling ctrl+c
* Handling of clean up of partially uploaded segments and pieces
* code cleanup per code comment
* updates based on code review comments
* setup repairer loop
* added read from queue
* Refactor to make things easier to import
* add more control flow to repairer
* add comment
* basic interval structure for running check/repair
* change function name GetNext to Dequeue
* better increment/decrement syntax
* export Repairer struct
* delete 'unreachable code'
* add mon.Task() to Repairer.Repair
* remove 24 hour interval
* set maxRepair on Config as well as Repairer
* add comment for Repairer struct, check err
* comment out runCfg.Repair in cmd/satellite/main.go because it is NI yet
* begin adding encryption for remote pieces
* begin adding decryption
* add encryption key as arg to Put and Get
* move encryption/decryption to object store
* Add encryption key to object store constructor
* Add the erasure scheme to object store constructor
* Ensure decrypter is initialized with the stripe size used by encrypter
* Revert "Ensure decrypter is initialized with the stripe size used by encrypter"
This reverts commit 07272333f461606edfb43ad106cc152f37a3bd46.
* Revert "Add the erasure scheme to object store constructor"
This reverts commit ea5e793b536159d993b96e3db69a37c1656a193c.
* move encryption to stream store
* move decryption stuff to stream store
* revert changes in object store
* add encryptedBlockSize and close rangers on error during Get
* calculate padding sizes correctly
* encryptedBlockSize -> encryptionBlockSize
* pass encryption key and block size into stream store
* remove encryption key and block size from object store constructor
* move encrypter/decrypter initialization
* remove unnecessary cast
* Fix padding issue
* Fix linter
* add todos
* use random encryption key for data encryption. Store an encrypted copy of this key in segment metadata
* use different encryption key for each segment
* encrypt data in one step if it is small enough
* refactor and move encryption stuff
* fix errors related to nil slices passed to copy
* fix encrypter vs. decrypter bug
* put encryption stuff in eestream
* get captplanet test to pass
* fix linting errors
* add types for encryption keys/nonces and clean up
* fix tests
* more review changes
* add Cipher type for encryption stuff
* fix rs_test
* Simplify type casting of key and nonce
* Init starting nonce to the segment index
* don't copy derived key
* remove default encryption key; force user to explicitly set it
* move getSegmentPath to streams package
* dont require user to specify encryption key for captplanet
* rename GenericKey and GenericNonce to Key and Nonce
* review changes
* fix linting error
* Download uses the encryption type from metadata
* Store enc block size in metadata and use it for download
* add mb command
* forgot colon
* add command descriptions
* use utils.ParseURL in commands
* return error message instead of minio.BucketAlreadyExists in mb
* ls command with bucket store functionality
* rb command with bucket store functionality
* rm command with bucket store functionality
* newline
* use print rather than errs for messages, add no buckets messsage
* cp command with bucket store functionality
* remove deprecated getStorjObjects function
* defer utils.LogClose(f) on instead of defer f.Close()
* Check for no buckets after for loop
* add checks for unspecified bucket in bucket store methods
* fix incorrect return types
* add no path error messages in object store methods
* split copy into helpers
* srcObj scheme check in download
* print buckets instead of appending to slice
* check if destObj.Host != srcObj.Host
* better method of handling destination name if not specified
* uplink rename
* final cleanups
* trailing slash fixes
* linting
* more linting
* helpful error messages
* Adjust startAfter after merging #328
* Improve output messages
* Improved error check for empty bucket and path
* No page limit on client side. Rely on server side limit.
* Better time formatting
* Fix paths in recursive list results
* Normalize the dockerfiles and entrypoints
* fix formattting and docker-compose file
* add missing env var to satellite dockerfile
* Remove build args no longer in use
* More changes to make it all work
* Finally fix the binary builds for a couple os/arch combinations
* Make the file executable before adding it to the zip
* This cache hack wasn't suppose to make it into the PR
* Guard against calling the binary target by itself
* Order matters on OS X
* Vendored my image
* No 386 builds until storage is fixed.
* First pass at building binaries
* Finished building images
* Reworked some bits to save binaries in a better place.
* First pass at uploading binaries
* Builds for freebsd fail currently
* Add ignore file for docker builds to not invalidate the cache as often
* Docker image for 1.11-alpine exists now
* Name windows binaries correctly
* The makefile is used in tests.
* Take a swing at updating the tests for 1.11
* Switch to something with glibc so the race detector works.
* Remove unused .PHONY targets
* add CopyObject method
* use utils.LogClose
* extract common code from GetObject to getObject helper
* remove rr.Range from getObject helper, create helper putObject
* return rr, err in getObject helper
* extract code from PutObject into putObject helper
* remove commented out text
* remove other commented out code
* WIP trying to get storj cp command to work with copyObject
* fix typo in rb and now it works
* use rr.Size() instead of srcInfo.Size
* Revert "WIP trying to get storj cp command to work with copyObject"
This reverts commit e256b9f9a0fda728d41eb5b9d7a98b5446825842.
* add CopyObject test
* rebase and fix merge conflicts
* check error in gateway-storj test
* fix typo
* wip ca/ident cmds
* minor improvements and commenting
* combine id and ca commands and add $CONFDIR
* add `NewIdenity` test
* refactor `NewCA` benchmarks
* linter fixes
* First pass at an all-in-one compose project
* Sort out dependencies so uplink can build in docker
* Added the final bits to make this work
* Add readme for all-in-one
* Fix sed to work on OS X and Linux
* Make the readme for all-in-one better
* Only bring up the satellite and uplink after fixing the mock overlay
* Use a dummy value for mock overlay for now
* Finishing touches
* Fix url form for aws cli tool
* Move files out of all-in-one directory to help with directory confusion.
* Added a make target to make all-in-one even better
* move mock overlay from client to server
this doesn't really change much, but it does allow you to
run a standalone gateway against captain planet. it still does
not allow you to run a standalone gateway against a standalone
heavy client
* pointerdb: small error fixes
* some cleanups
* fix tests
* captplanet standalone farmer setup
* Bandwidth Allocation
* utils.Close method changed to utils.LogClose
* Get build temporarily working
* Get/Put for PSClient should take payer bandwidth allocations rather than the NewPSClient function
* Update example client to reflect changes in client API
* Update ecclient to use latest PSClient, Make NewPSClient return error also
* Updated pieceranger tests to check for errors; sign method should take byte array
* Handle defers in store.go better
* Fix defer functions in psdb.go
* fun times
* Protobuf bandwidthallocation data is now a byte array
* Remove psservice package and merge it into pstore server
* Write wrapper for database calls
* Change all expiration names in protobuf to be more informative; add defer in retrieve; remove old comment
* Make PSDB tests implementation independent rather than method independent
* get rid of payer, renter in ecclient
* add context monitoring in store and retrieve
* WIP first commit
* change args check
* rename to cli, rename folder to storj, const defaultConfDir, use process.Exec()
* typos
* WIP create helper for getting minio objectlayer, changed setup command to try to match changes in setting up identity
* add rb command
* changes for setting up identity
* remove concurrency field from setupCfg
* rename variable returned by url.Parse to u across all commands for consistency
* add marker functionality to ls command
* add download functionality to cp command
* fix typo in rb init, use short variable names
