..although it ought to work for other storage.KeyValueStore needs as
well. it's just optimized to work pretty well for a largish hierarchy of
paths.
This includes the addition of "long benchmarks" for KeyValueStore
testing. These will only be run when -test-bench-long is added to the
test flags. In these benchmarks, a large corpus of paths matching a
natural ("real-life") hierarchy is read from paths.data.gz (which you
can get from https://github.com/storj/path-test-corpus) and imported
into a particular KeyValueStore. Recursive and non-recursive queries are
run on it to detect performance problems that arise only at scale.
This also includes alternate implementation of the postgreskv client,
which works in a less-bizarre way for non-recursive queries, but suffers
from poor performance in tests such as the long benchmarks. Once this
alternate impl is committed to the tree, we can remove it again; I just
want it to be available for future reference.
This is an old definition from the very early stage of development. It
is not used anymore.
Change-Id: I6a033e4006e6edfa7c18acc6ae91c9e4e1df0e6a
Signed-off-by: Kaloyan Raev <kaloyan@storj.io>
Reviewed-on: https://review.gerrithub.io/429582
Reviewed-by: JT Olio <hello@jtolio.com>
Tested-by: JT Olio <hello@jtolio.com>
* implements CombineErrors on runCfg error chan
* implement error channel timeout and CombineErrors
* implements CombineErrors with time.After timeout
* changes d to duration in args
* changes comment to better reflect what collectErrors does
* appeases the linter gods
* Gets rid of timeout bug in collectErrors
* Add s3 Scheme as valid URL Scheme
* Merge from Master
* Remove Test Case and fix Error
* Add invalid Test Case
* add valid s3 Test Case for good measure
* Improve Speed and Resource Handling
* Travis uses Go 1.11
* Use go modules instead of storj-vendor
* Automatic caching of downloaded dependencies
* Ensures that modules incompatible linters run with modules
* handle nil nodes in ec Put
* read and discard readers for nil nodes
* test 2 nil nodes, unique wont return false with nil nodes
* Discard reader data for nil nodes
* edit control flow
* add filter field into OverlayOptions message
* chooseFiltered method, add excluded parameter in populate method
* change excluded type to []dht.NodeID in ChooseFiltered, change comment
* change name filter to excluded_nodes in proto
* implement helper function contains
* delete ChooseFiltered and add its functionality into Choose method to keep original author's history, add excluded argument into Choose calls
* regenerate mock_client.go
* regenerate protobuf
* update test case to use new IDFromString function
* remove old kademlia test code
* begin adding path encryption
* do not encrypt/decrypt first element of path (bucket)
* add path encryption for delete and list
* use encrypted paths in streamstore.Meta
* fix listing with encrypted paths
* move encrypt/decryptAfterBucket to streamstore
* fix listing with no prefix
* remove duplicate logic for listing with no prefix
