Jess G
f11bf46a11
Jg/1967 mv bucket metadata uplink ( #2505 )
...
* add bucketstore, add init uplink bucket
* update uplink to use bucket rpc
* fix tests
* wrap metainfo client errors
* add allowedBucket struct, fix tests
* update comment
* add paging
* updates per CR
* add test for pagination
* fix lint
* fix uplink test so its easier tyo understand
* fix gateway pagination bug
* changes per cr
* fix bug w/allowedBuckets, add test to catch
2019-07-12 08:57:02 -04:00
Egon Elbre
d52f764e54
protocol: implement new piece signing and verification ( #2525 )
2019-07-11 16:51:40 -04:00
Bill Thorp
0e463dccfd
7 day validity window for order limits ( #2520 )
...
* 7 day limit
2019-07-10 17:17:00 -04:00
Alexander Leitner
1c5db71faf
Change protobuf expirations to use time.Time ( #2509 )
...
* Change protobuf expirations to use time.Time instead of timestamp.Timestamp
2019-07-09 17:54:00 -04:00
Egon Elbre
910e4744ad
satellite/metainfo: optimize pointerdb size ( #2506 )
2019-07-09 16:36:18 -04:00
Jess G
f9696d6c5e
satellite/metainfo: add buckets RPC and database ( #2460 )
...
* add db interface and methods, add sa metainfo endpoints and svc
* add bucket metainfo svc funcs
* add sadb bucekts
* bucket list gets all buckets
* filter buckets list on macaroon restrictions
* update pb cipher suite to be enum
* add conversion funcs
* updates per comments
* bucket settings should say default
* add direction to list buckets, add tests
* fix test bucket names
* lint err
* only support forward direction
* add comments
* minor refactoring
* make sure list up to limit
* update test
* update protolock file
* fix lint
* change per PR
2019-07-08 15:32:18 -07:00
Stefan Benten
16156e3b3d
Ensure we force a segment size and account storage before committing them ( #2473 )
2019-07-08 18:24:38 -04:00
Alexander Leitner
3587e1a579
Change pointerdb pointer to use time.Time for Creation date ( #2483 )
2019-07-09 00:16:50 +02:00
aligeti
a554752ce0
adding expiration date check on satellite's CreateSegment ( #2476 )
...
* adding expiration check on satellite createsegment
2019-07-08 14:10:16 -04:00
Michal Niewrzal
094e1b8b90
Add 'Old' suffix to some metainfo methods/messages ( #2462 )
2019-07-08 15:33:15 +02:00
Michal Niewrzal
56fcf228fb
Enable bucket name validation ( #2449 )
2019-07-03 14:56:17 -04:00
aligeti
ae8b9698f9
Rename/remove EncryptionScheme -> EncryptionParame… ( #2363 )
...
* rename/remove EncryptionScheme -> EncryptionParameters
2019-07-03 14:07:44 -04:00
Michal Niewrzal
2ee5bada2c
Add pieceNum to PieceID derivation function ( #2193 )
2019-07-03 18:53:15 +02:00
Michal Niewrzal
61dfa61e3a
Add timestamp and piece size to piece hash ( #2198 )
2019-07-03 18:14:37 +02:00
Maximillian von Briesen
52e5a4eee3
pass logger into repairer and ecclient ( #2365 )
2019-07-02 13:08:02 +03:00
Jess G
ea55ae63e3
add bucket metainfo rpc ( #2383 )
...
* add bucket metainfo rpc
* updates per design doc
* rm bucketmeta rpc
* add changes per comments
* update Bucket name since same name already exists
* add placeholder methods to fulfill pb.MetainfoServer interface
* update proto lock
* add bucketListItem message
* why failing lint?
* add back ciphersuite but with correct types
* fix proto errs
* update time type
* add proto lock
* rm unneeded import
* update protolock package
2019-07-01 15:17:30 -07:00
Egon Elbre
385c046723
pkg/pb: rename Order2 to Order, OrderLimit2 to OrderLimit ( #2406 )
2019-07-01 18:54:11 +03:00
Egon Elbre
2b68a72428
internal/testplanet: ensure that metainfo connections get closed ( #2381 )
2019-07-01 17:35:10 +03:00
Michal Niewrzal
ff92a5bcf6
Improve RS validation message ( #2377 )
2019-06-29 17:05:46 +02:00
Egon Elbre
e83ebd7cde
jenkins: avoid using goimports and distribute load better ( #2359 )
2019-06-27 21:52:50 +03:00
Jeff Wendling
efcdaa43a3
lib/uplink: encryption context ( #2349 )
...
* lib/uplink: encryption context
Change-Id: I5c23dca3286a46b713b30c4997e9ae6e630b2280
* lib/uplink: bucket operation examples
Change-Id: Ia0f6e69f365dcff0cf11c731f51b30842bce053b
* lib/uplink: encryption key sharing test cases
Change-Id: I3a172d565f33f4e591402cdcb9460664a7cc7fbe
* fix encrypted path prefix restriction issue
Change-Id: I8f3921f9d52aaf4b84039de608b8cbbc88769554
* implement panics in libuplink encryption code
todo on cipher suite selection as well as an api concern
Change-Id: Ifa39eb3cc4b3443f7d96f9304df9b2ac4ec4085d
* implement GetProjectInfo api call to get salt
Change-Id: Ic5f6b3be9ea35df48c1aa214ab5d355fb328e2cf
* some fixes and accessors for encryption store
Change-Id: I3bb61f6712a037900e2a96e72ad4029ec1d3f718
* general fixes to builds/tests/etc
Change-Id: I9930fa96acb3b221d9a001f8e274af5729cc8a47
* java bindings changes
Change-Id: Ia2bd4c9c69739c8d3154d79616cff1f36fb403b6
* get libuplink examples passing
Change-Id: I828f09a144160e0a5dd932324f78491ae2ec8a07
* fix proto.lock file
Change-Id: I2fbbf4d0976a7d0473c2645e6dcb21aaa3be7651
* fix proto.lock again
Change-Id: I92702cf49e1a340eef6379c2be4f7c4a268112a9
* fix golint issues
Change-Id: I631ff9f43307a58e3b25a58cbb4a4cc2495f5eb6
* more linting fixes
Change-Id: I51f8f30b367b5bca14c94b15417b9a4c9e7aa0ce
* bug fixed by structs bump
Change-Id: Ibb03c691fce7606c35c08721b3ef0781ab48a38a
* retrigger
Change-Id: Ieee0470b6a2d07168a1578552e8e7f271ae93a13
* retrigger
Change-Id: I753d63853171e6a436c104ce176048892eb974c5
* semantic merge conflict
Change-Id: I9419448496de90340569047a6a16a1b858a7978a
* update total to match prod defaults
Change-Id: I693d55c1ebb28b5803ee1d26e9e198decf82308b
* retrigger
Change-Id: I28b74d5d6202f61aa3866fe407d423f6a0a14b9e
* retrigger
Change-Id: I6fd054885c715f602e2cef623fd464c42e88742c
* retrigger
Change-Id: I6a01bae88c72406d4ed5a8f13bf8a2b3c650bd2d
2019-06-27 17:36:51 +00:00
aligeti
7facda4422
value attribution integration with libuplink ( #2297 )
...
* integrates the value attribution with libuplink
2019-06-26 12:22:01 -04:00
Egon Elbre
b6ad3e9c9f
internal/testrand: new package for random data ( #2282 )
2019-06-26 13:38:51 +03:00
Michal Niewrzal
6b23380d6b
Make RS validation always enabled ( #2336 )
2019-06-26 12:27:36 +02:00
Jess G
e5c48fab74
fix ordersDB methods to take correct args ( #2314 )
...
* fix orderdDB methods to take correct args
* update tally to save projectID in correct format
* update var names in splitBucket test
* changes per CR comments
2019-06-25 08:58:42 -07:00
Egon Elbre
414648d660
Fix some metainfo.Client leaks ( #2327 )
2019-06-25 18:36:23 +03:00
Jeff Wendling
c35c8e4c24
allow reading bucket metadata with restricted keys ( #2321 )
...
Change-Id: I47d3a2f5f02744ae6c51d54963cdf2dff24134e2
2019-06-24 17:36:09 -06:00
JT Olio
e17908a266
metainfo: add project info rpc for getting a project salt using an api key ( #2311 )
...
The API exposed in #2302 needs to be able to get a project-level salt (https://github.com/storj/storj/pull/2302/files#diff-72b734f4a2e408a544fef071d2565fc8R200 ), see https://github.com/storj/storj/blob/master/docs/design/password-key-derivation.md also
I realized this morning that if we need it later this week, we need to get it into today's release.
Please describe the tests: tests will come as part of #2302
Please describe the performance impact: none
2019-06-24 11:15:45 -06:00
Michal Niewrzal
fdeb834801
Bucket name validation ( #2244 )
2019-06-24 11:52:25 +02:00
aligeti
d8e62bc06e
support value attribution endpoint ( #2231 )
...
Support Value attribution Endpoint
2019-06-21 15:14:34 -04:00
aligeti
043d603cbe
satellite rs config check with validation check set to false default ( #2229 )
...
* satellite rs config check with validation check
2019-06-21 14:15:58 -04:00
Kaloyan Raev
964c87c476
Fix checks around repair threshold ( #2246 )
2019-06-19 22:13:11 +02:00
Egon Elbre
8f4a6afc8a
Fixes for value attribution ( #2238 )
...
* Move value attribution to satellite/
* remove unnecessary conversion
2019-06-19 08:02:37 -04:00
aligeti
8e8a5eb89c
initial value attribution check development ( #2091 )
...
* initial value attribution check development
* code reveiew updates
2019-06-12 21:35:37 -04:00
Michal Niewrzal
b5ac4f3eac
Better metainfo Create/Commit request validation ( #2088 )
2019-06-05 18:41:02 +02:00
JT Olio
f1641af802
storage: add monkit task to missing places ( #2122 )
...
* storage: add monkit task to missing places
Change-Id: I9e17a6b14f7c25bbf698eeecf32785e9add3f26e
* fix tests
Change-Id: Id078276fa3de61a28eb3d01d4e751732ecbb173f
* import order
Change-Id: I814e33755b9f10b5219af37cd828cd75eb3da1a4
* remove part of other commit
Change-Id: Idaa4c95cd65e97567fb466de49718db8203cfbe1
2019-06-05 16:23:10 +02:00
JT Olio
ccb158c99b
pkg/auth: add monkit task to missing places ( #2123 )
...
What: add monkit.Task to a bunch of functions that are missing it
Why: this will significantly help our instrumentation, data collection, and tracing about what's going on in the network
2019-06-05 07:47:01 -06:00
JT Olio
29d16b4d68
satellite: add monkit task to missing places ( #2108 )
2019-06-04 13:55:37 +02:00
ethanadams
268dc6b7e4
Enable gocritic linter ( #2051 )
...
* first round cleanup based on go-critic
* more issues resolved for ifelsechain and unlambda checks
* updated from master and gocritic found a new ifElseChain issue
* disable appendAssign. i reports false positives
* re-enabled go-critic appendAssign and disabled lint check at code line level
* fixed go-critic lint error
* fixed // nolint add gocritic specifically
2019-05-29 09:14:25 -04:00
Michal Niewrzal
03bae48ad6
Remove unused parameter ( #2069 )
2019-05-29 11:15:15 +02:00
Michal Niewrzal
f731267e8c
Per-project usage limiting ( #2036 )
...
What: Changes to support custom usage limit for the project. With this implementation by default project usage limit is taken from configuration flag. If project DB field usage_limit will be set to value larger than 0 it will become custom usage limit and we will be used to verify is limit was exceeded.
Whats changed:
usage_limit (bigint) field added to projects table (with migration)
things related to project usage moved from metainfo endpoint to project usage type
accounting.ProjectAccounting extended with GetProjectUsageLimits() method
Why: We need to have different usage limits per project. https://storjlabs.atlassian.net/browse/V3-1814
2019-05-28 09:36:52 -06:00
Fadila
8f078d2841
Delete pending audit entries from containment DB when files get deleted ( #2050 )
2019-05-24 15:56:08 -04:00
Jeff Wendling
1bd52b9f90
server side macaroons ( #1945 )
...
What: Adds macaroon support to the server side
Why: So that api keys are now macaroons
2019-05-24 10:51:27 -06:00
Bill Thorp
91721f63ba
Bt/repair no nodes ( #1974 )
...
* handle cases where repair is equal to total
2019-05-17 15:02:40 -04:00
Jess G
8518618b7a
add postgres support to storj-sim ( #1908 )
...
* add flags to sotrj-sim for SA dbs
* add schema to postgres
* add createschema with parse to sa
* add metainfo db postgres support
* add kv default as bolt
* add debug log to see db source
* add env var for postgres to test-sim.sh
* fix lint errs
* dynamically add postgres to args
* add postgres to integration tests
* add sqlite and postgres integration jenkins
* fix db name
* merge integration tests into one step
* test integration tests w/psql
* try using different schema
* debug failure
* use correct host for running storj-sim
* rm sqlite integration
* add back integration
2019-05-14 08:13:18 -07:00
Jennifer Li Johnson
5395ff5fe6
Refactor accountingdb interface ( #1897 )
...
* splits accounting db into storagenodeaccounting and projectaccounting interfaces and renames methods to match
2019-05-10 15:05:42 -04:00
paul cannon
02be91b029
real-time tracking of space used per project ( #1910 )
...
Ran into difficulties trying to find the ideal solution for sharing
these counts between multiple satellite servers, so for now this is a
dumb solution storing recent space-usage changes in a big dumb in-memory
map with a big dumb lock around it. The interface used, though, should
allow us to swap out the implementation without much difficulty
elsewhere once we know what we want it to be.
2019-05-09 20:39:21 -05:00
Bill Thorp
ea978dd674
hopefully sensible satellite defaults ( #1888 )
...
* hopefully sensible satellite defaults
2019-05-07 10:44:47 -04:00
Michal Niewrzal
fe3dfc1587
Move pointerdb.Service to satellite ( #1826 )
2019-04-25 10:46:32 +02:00
Yaroslav Vorobiov
75870005b2
api keys change encoding to base32 ( #1813 )
2019-04-23 16:00:31 +03:00
