What: we move api keys out of the grpc connection-level metadata on the client side and into the request protobufs directly. the server side still supports both mechanisms for backwards compatibility.
Why: dRPC won't support connection-level metadata. the only thing we currently use connection-level metadata for is api keys. we need to move all information needed by a request into the request protobuf itself for drpc support. check out the .proto changes for the main details.
One fun side-fact: Did you know that protobuf fields 1-15 are special and only use one byte for both the field number and type? Additionally did you know we don't use field 15 anywhere yet? So the new request header will use field 15, and should use field 15 on all protobufs going forward.
Please describe the tests: all existing tests should pass
Please describe the performance impact: none
* rename pkg/linksharing to linksharing
* rename pkg/httpserver to linksharing/httpserver
* rename pkg/eestream to uplink/eestream
* rename pkg/stream to uplink/stream
* rename pkg/metainfo/kvmetainfo to uplink/metainfo/kvmetainfo
* rename pkg/auth/signing to pkg/signing
* rename pkg/storage to uplink/storage
* rename pkg/accounting to satellite/accounting
* rename pkg/audit to satellite/audit
* rename pkg/certdb to satellite/certdb
* rename pkg/discovery to satellite/discovery
* rename pkg/overlay to satellite/overlay
* rename pkg/datarepair to satellite/repair
* add path implementation
This commit adds a pkg/paths package which contains two types,
Encrypted and Unencrypted, to statically enforce what is contained
in a path. It's part of a refactoring of the code base to be more
clear about what is contained in a storj.Path at all the layers.
Change-Id: Ifc4d4932da26a97ea99749b8356b4543496a8864
* add encryption store
This change adds an encryption.Store type to keep a collection
of root keys for arbitrary locations in some buckets. It allows
one to look up all of the necessary information to encrypt paths,
decrypt paths and decrypt list operations.
It adds some exported functions to perform encryption on paths
using a Store.
Change-Id: I1a3d230c521d65f0ede727f93e1cb389f8be9497
* add shim around streams store
This commit changes no functionality, but just reorganizes the code
so that changes can be made directly to the streams store
implementation without affecting callers.
It also adds a Path type that will be used at the interface boundary
for the streams store so that it can be sure that it's getting well
formed paths that it expects.
Change-Id: I50bd682995b185beb653b00562fab62ef11f1ab5
* refactor streams to use encryption store
This commit changes the streams store to use the path type as
well as the encryption store to handle all of it's encryption
and decryption.
Some changes were made to how the default key is returned in
the encryption store to have it include the case when the bucket
exists but no paths matched. The path iterator could also be
simplified to not report if a consume was valid: that information
is no longer necessary.
The kvmetainfo tests were changed to appropriately pass the
subtests *testing.T rather than having the closure it executes
use the parent one. The test framework now correctly reports
which test did the failing.
There are still some latent issues with listing in that listing
for "a/" and listing for "a" are not the same operation, but we
treat them as such. I suspect that there are also issues with
paths like "/" or "//foo", but that's for another time.
Change-Id: I81cad4ba2850c3d14ba7e632777c4cac93db9472
* use an encryption store at the upper layers
Change-Id: Id9b4dd5f27b3ecac863de586e9ae076f4f927f6f
* fix linting failures
Change-Id: Ifb8378879ad308d4d047a0483850156371a41280
* fix linting in encryption test
Change-Id: Ia35647dfe18b0f20fe13763b28e53294f75c38fa
* get rid of kvmetainfo rootKey
Change-Id: Id795ca03d9417e3fe9634365a121430eb678d6d5
* Fix linting failure for return with else
Change-Id: I0b9ffd92be42ffcd8fef7ea735c5fc114a55d3b5
* fix some bugs adding enc store to kvmetainfo
Change-Id: I8e765970ba817289c65ec62971ae3bfa2c53a1ba
* respond to review feedback
Change-Id: I43e2ce29ce2fb6677b1cd6b9469838d80ec92c86
What: This change moves project-level bucket metadata encryption information to the volatile section, because it is unlikely to remain in future releases
Why: Ultimately, the web user interface will allow bucket management (creation, removal, etc), but not object management as that requires an encryption key for sure and we don't want to have users give the satellite their encryption keys.
At a high level, a (*Project) type should map to all of the things you can do inside the web user interface within a project, which by necessity cannot have an encryption key. So, we really don't want an encryption key in the non-volatile section of this library.
* separate TLS options from server options (because we need them for dialing too)
* stop creating transports in multiple places
* ensure that we actually check revocation, whitelists, certificate signing, etc, for all connections.
