Commit Graph

6 Commits

Author SHA1 Message Date
paul cannon
94651921c3 storage/testsuite: pass ctx in to bulk setup methods
to make them cancelable. Also,

* rename BulkDelete->BulkDeleteAll

this leaves room for a new method `BulkDelete(items storage.Items)` that
does a bulk deletion of a specified list of items, as opposed to
deleting _everything_. such a method would be used in the
`cleanupItems()` function found in utils.go, because when individual
deletes are fairly slow, that step takes way too long during tests.

* use BulkDelete method if available

nothing currently provides `BulkDelete(items storage.Items) error`,
but we made use of it with the Bigtable testing and code, and may make
use of it again when adding new kv backends.

* and eliminate the global context in test_iterate.go

Change-Id: I171c7a3818beffbad969b131e98b9bbe3f324bf2
2019-12-10 20:22:08 +00:00
Egon Elbre
ee6c1cac8a
private: rename internal to private (#3573) 2019-11-14 21:46:15 +02:00
Egon Elbre
db939d37ec
cover all the things (#1818) 2019-04-26 16:39:11 +03:00
Egon Elbre
ac18432dc5 Public Jenkins (#1779)
* initial test

* add parenthesis

* remove pipeline

* add few todos

* use docker image for environment

* use pipeline

* fix

* add missing steps

* invoke with bash

* disable protoc

* try using golang image

* try as root

* Disable install-awscli.sh temporarily

* Debugging

* debugging part 2

* Set absolute path for debugging

* Remove absolute path

* Dont run as root

* Install unzip

* Dont forget to apt-get update

* Put into folder that is in PATH

* disable IPv6 Test

* add verbose info and check protobuf

* make integration non-parallel

* remove -v and make checkout part of build

* make a single block for linting

* fix echo

* update

* try using things directly

* try add xunit output

* fix name

* don't print empty lines

* skip testsuites without any tests

* remove coverage, because it's not showing the right thing

* try using dockerfile

* fix deb source

* fix typos

* setup postgres

* use the right flag

* try using postgresdb

* expose different port

* remove port mapping

* start postgres

* export

* use env block

* try using different host for integration tests

* eat standard ports

* try building images and binaries

* remove if statement

* add steps

* do before verification

* add go get goversioninfo

* make separate jenkinsfile

* add check

* don't add empty packages

* disable logging to reduce output size

* add timeout

* add comment about mfridman

* Revert Absolute Path

* Add aws to PATH

* PATH Changes

* Docker Env Fixes

* PATH Simplification

* Debugging the PATH

* Debug Logs

* Debugging

* Update PATH Handling

* Rename

* revert changes to Jenkinsfile
2019-04-22 15:45:53 +02:00
Jennifer Li Johnson
856b98997c
updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
paul cannon
e2c0dd437a
offer PostgreSQL storage for pointerdb (#440)
..although it ought to work for other storage.KeyValueStore needs as
well. it's just optimized to work pretty well for a largish hierarchy of
paths.

This includes the addition of "long benchmarks" for KeyValueStore
testing. These will only be run when -test-bench-long is added to the
test flags. In these benchmarks, a large corpus of paths matching a
natural ("real-life") hierarchy is read from paths.data.gz (which you
can get from https://github.com/storj/path-test-corpus) and imported
into a particular KeyValueStore. Recursive and non-recursive queries are
run on it to detect performance problems that arise only at scale.

This also includes alternate implementation of the postgreskv client,
which works in a less-bizarre way for non-recursive queries, but suffers
from poor performance in tests such as the long benchmarks. Once this
alternate impl is committed to the tree, we can remove it again; I just
want it to be available for future reference.
2018-10-25 12:11:28 -05:00