Commit Graph

1045 Commits

Author SHA1 Message Date
Jennifer Li Johnson
e285fe1997
Don't require encryption keys for project or bucket management (#2291) 2019-06-23 22:06:14 -04:00
Cameron
1283036e37
add storage node voucher request service (#2158)
* add voucher service on storage node

* config field tag syntax, go routines for requests

* hook up voucher service in storagenode/peer.go

* add voucher config to testplanet

* add voucher config to testplanet

* add voucher response status INVALID, ACCEPTED, REJECTED

* add a test for vouchers service

* handle no row from GetValid, test it

* add trust pool to voucher service

* use trusted list to get satellites

* verify vouchers upon receipt

* test VerifyVoucher
2019-06-21 18:48:52 -04:00
Jeff Wendling
81f1bc19dc add functions for password/root key derivation (#2294)
This commit adds two functions that implement the algorithms
described in the password key derivation design document. They
will be used during setup to derive bucket level root keys or
default passwords to use when buckets do not have their own
independent key.

Change-Id: Ie7fb2d8d549ba7465d0722716a2c1ac0ad907286
2019-06-21 18:21:16 -04:00
Bill Thorp
8f47fca5d3
Remove audit / uptime ratio fields (#2247)
* removed ratios
2019-06-21 13:14:53 -04:00
Ivan Fraixedes
3d6b25a043
[v3-1952 test 1 & 3] pkg/audit: Add DQ test for too many failed audits (#2265)
* pkg/audit: Add DQ test for too many failed audits
  
  Add an integration test which checks that a node which fails several
  audits gets disqualified but not before it reaches the audit reputation
  disqualification cut-off.

* internal/testplanet: Set DQ cut-off config values
  Set the values of the Overlay cache DQ cut-off configuration parameters
  used by testplanet.
2019-06-21 18:27:19 +02:00
ethanadams
4f2e893e68
Fix the way project_id is stored in bucket_storage_tallies and bucket_bandwidth_rollups (#2283)
* fixing issues where projectID is stored as the byte representation of a UUID string, instead of bytes of the UUID

* added test for spitBucketID
2019-06-21 11:38:37 -04:00
Maximillian von Briesen
bfcfe39313
Enable and fix statdb tests (#2270) 2019-06-21 11:23:15 -04:00
Natalie Villasana
169fc9594c
pkg/audit: fix maxRetries bug (#2271) 2019-06-21 11:10:03 -04:00
Maximillian von Briesen
d103dd2c4e
Include information about disqualified nodes in the storage node payment report (#2239) 2019-06-21 10:21:15 -04:00
JT Olio
aa25c4458f kvmetainfo: merge with storage/buckets (#2277) 2019-06-21 13:29:31 +02:00
JT Olio
76b54458e9 satellite: send external address in order limits (#2278) 2019-06-21 12:19:52 +03:00
JT Olio
06006effb3 lib/uplink: minor fixes (#2257) 2019-06-20 23:50:13 +02:00
Jess G
daf166d423
fix tally test flakiness (#2250)
* fix test flakiness

* update tests to rm uplink.upload

* rm metainfo change

* add test case of no objects in bucket
2019-06-20 12:15:13 -07:00
Natalie Villasana
edb3d1cbf8
pkg/overlay: update node selection config values for reputation (#2264) 2019-06-20 15:01:50 -04:00
Ivan Fraixedes
5b31086757
pkg/audit: Move test helper funcs to separated file (#2259)
Move 2 helper function used for test which relay on testplanet from the
test file where they were created to separated file to contain them
because they are not only used in the test file were initially they were
created.
2019-06-20 17:38:08 +02:00
Natalie Villasana
9386187fe6
add disqualification and new reputation system into overlay cache (#2227) 2019-06-20 09:56:04 -04:00
Fadila
1726a6c073
pkg/bloomfilter: implementation and benchmark results (#2113) 2019-06-20 12:27:20 +02:00
Kaloyan Raev
964c87c476 Fix checks around repair threshold (#2246) 2019-06-19 22:13:11 +02:00
Jess G
ddcf4fc2a3
add support to hide config settings (#2241)
* add hide support for config settings

* updates per CR to unit test

* check err for lint
2019-06-19 07:27:44 -07:00
Fadila
0eb8ac58c1
Create protocol buffer for garbage collection (#2222) 2019-06-19 15:56:46 +02:00
Egon Elbre
8f4a6afc8a Fixes for value attribution (#2238)
* Move value attribution to satellite/

* remove unnecessary conversion
2019-06-19 08:02:37 -04:00
Kaloyan Raev
ac65d19753
Avoid reporting audit failures for files deleted during audit (#2233) 2019-06-19 12:02:25 +03:00
Jess G
f0f59a5577 Always encrypt inline segments without padding (#2183) 2019-06-19 11:11:27 +03:00
Maximillian von Briesen
ad8cad4909
Expand the inspector tool to provide node id's for each segment, rather than just numeric totals (#2205) 2019-06-18 18:22:14 -04:00
Natalie Villasana
b30c35d306
change ReputationAuditOmega (et al.) to AuditReputationWeight (#2232) 2019-06-18 14:17:25 -04:00
Bill Thorp
119a8fd3cc removed fields (#2234)
* removed fields

* sql tweaks
2019-06-18 13:40:28 -04:00
aligeti
09940d4e0b
value attribution DB interface Insert & Get method support (#2200)
* value attribution DB interface methods support
2019-06-18 09:06:33 -04:00
Maximillian von Briesen
8398fae9b5
Add node churn and containment/reverify monkit stats (#2217)
* add counters for nodes that have/have not been seen in the past 24 hours/week

* add additional uptime counters

* add monkit stats for containment mode
2019-06-18 08:54:52 -04:00
Ivan Fraixedes
35c8648330
[v3-1914] Storage node disqualification: Change type from bool to timestamp (#2212)
* satellite/satellitedb: Alter nodes disqualification column
  Change the type of the 'disqualification' column of the nodes table from
  boolean to timestamp.
* overlay/cache: Change Disqualified field type
  Change the Disqualified field type the NodeDossier struct type from bool
  to time.Time to match with the disqualified type used by the DB layer.
* satellite/satellitedb: Update queries uses disqualified
  Update the queries which uses the disqualified column due to the column
  type has been changed from boolean to nullable timestamp.
* docs/design: Update disqualification due impl changes
  Update the disqualification design document to contain the architectural
  change required to be able to restore unfair disqualified nodes in case
  of an unexpected cause (bug, mistake, hard network disconnection, etc.).
2019-06-18 11:14:31 +02:00
Jennifer Li Johnson
d177761f51
Adds test coverage to CalculateBucketAtRestData (#2203) 2019-06-17 22:20:40 -04:00
Egon Elbre
f4f776d092 Use mail.test as domain in emails (#2224) 2019-06-18 02:28:40 +02:00
Kaloyan Raev
8e29ef8a6b Use zap.Stringer instead of zap.String (#2223) 2019-06-18 01:37:43 +02:00
Bill Thorp
f378125c8b
draft of removing stats from inspector (#2226) 2019-06-17 16:48:04 -04:00
JT Olio
e58a06bd0c config: update release values to match prod (#2192) 2019-06-15 18:19:19 +02:00
littleskunk
319cc77a34
increase audit timeout (#2208) 2019-06-14 13:53:49 +02:00
Kaloyan Raev
ebd9b375fc
Repair should not corrupt files (#2194) 2019-06-14 12:16:31 +03:00
Bill Thorp
6932dc92f3
NodeSelectionConfig inits and test cleanup (#2202)
* NodeSelectionConfig inits and test cleanup
2019-06-13 17:51:18 -04:00
ethanadams
4e1cc37a4c Fix bucket_storage_tally data (#2165) 2019-06-13 12:58:40 -04:00
JT Olio
df2fad15d8
pkg/process/logging: different defaults for release/dev (#2191)
* pkg/process/logging: different defaults for release/dev

Change-Id: I55be80430a31668fededf479b052e106ab18d9ce

* linting

Change-Id: I4e50d4c9569b7324c4704c14df7dd3228dbb7dd5

* Trigger Jenkins

* fix lock file

* use dev=debug and prod=info
2019-06-13 10:43:39 -06:00
JT Olio
fa09a7894b pkg/process: prometheus support (#2176) 2019-06-13 18:29:35 +02:00
Bill Thorp
703e5fee49
mostly removed things (#2199) 2019-06-13 12:06:37 -04:00
Bryan White
bc33964729 Uplink C bindings part 1 (#2196)
* uplink cbindings tooling
2019-06-13 11:09:05 -04:00
JT Olio
1ae5654eba kademlia/routing: add contexts to more places so monkit works (#2188) 2019-06-13 16:51:50 +02:00
Egon Elbre
1a1a084477
testcontext: sanitize folder name (#2195) 2019-06-13 15:46:08 +03:00
aligeti
8e8a5eb89c
initial value attribution check development (#2091)
* initial value attribution check development

* code reveiew updates
2019-06-12 21:35:37 -04:00
Natalie Villasana
5314c950cc
pkg/audit: add more reverify tests (#2144) 2019-06-11 16:14:40 -04:00
Jess G
d84cd719ff
add const stripesPerBlock const to calc blocksize (#2163)
* add const stripesPerBlock, update comments in Pad, add speed to progressbar

* change size of erasure share size, update comments

* missing copyright

* update tests with stripesPerBlock
2019-06-11 10:14:05 -07:00
Kaloyan Raev
252c8ac189
Add email to self node info (#2171) 2019-06-11 16:30:28 +03:00
Ivan Fraixedes
f624b213a3
reputation: Add configuration parameters (#2150)
* reputation: Add configuration parameters
 Add the configuration parameters which will be used by the algorithm
 which will calculate the storage node reputation.
 Because the reputation calculation is based on audit and uptime check
 results some configuration parameters are in pkg/audit, others in
 pkg/discovery and other in the satellite which will combine the both
 reputation results to obtain the storage node reputation for repair and
 uplink.
* satellite-config: Refresh lock file with new params
  Refresh the Satellite configuration yaml lock file with the new
  parameters added in this branch.
2019-06-11 12:14:01 +02:00
Kaloyan Raev
f0880b9b3c
Audit service should download erasure shares in parallel (#2164) 2019-06-11 11:00:59 +03:00