Commit Graph

2453 Commits

Author SHA1 Message Date
Maximillian von Briesen
fb10815229 Repair with hashes (#2925)
* add outline for ECRepairer

* add description of process in TODO comments

* begin download/getting hash for a single piece

* verify piece hash and order limit during download

* fix download piece

* begin filling out ESREpair. Get

* wip move ecclient.Repair to ecrepairer.Repair

* pass satellite signee into repairer

* reconstruct original stripe from pieces

* move rebuildStripe()

* calculate piece size differently, increment successful count

* fix shares slices initialization

* rename stripeData to segment

* do not pad reader in Repair()

* temp debug

* create unsafeRSScheme

* use decode reader

* rename file name to be all lowercase

* make repair downloader async

* declare condition variable inside Get method

* set downloadAndVerifyPiece's in-memory buffer to be share size

* update unusedLimits var

* address comments

* remove unnecessary comments

* move initialization of segmentRepaire to be outside of repairer service

* use ReadAll during download

* remove dots and move hashing to after validating for order limit signature

* wip test

* make sure files exactly at min threshold are repaired

* remove unused code

* use corrput data and write back to storagenode

* only create corrupted node and piece ids once

* add comment

* address nat's comment

* fix linting and checker_test

* update comment

* add comments

* remove "copied from ecclient" comments

* add clarification comments in ec.Repair
2019-09-06 15:20:36 -04:00
Jennifer Li Johnson
3387750280 storagenode/contact: create chore for nodes to ping satellites (#2877)
Creates a chore for nodes to announce themselves to their trusted satellites. Runs on startup and every hour thereafter
2019-09-06 12:14:03 -04:00
Egon Elbre
a3e0955e16
satellite/satellitedb: ensure that we process orders in order (#2950)
When transactions are handled in different orders there is a potential for a deadlock.
2019-09-06 17:49:30 +03:00
Yaroslav Vorobiov
c35ad5cbfc
storagenode/console: update api (#2969) 2019-09-06 15:01:03 +03:00
Jeff Wendling
28353823e8 jenkins: remove Jenkinsfile.gerrit (#2953) 2019-09-06 08:54:46 +03:00
Michal Niewrzal
587be8f206
auto-updater: download versions and archive with binary (#2922) 2019-09-05 23:10:05 +02:00
Natalie Villasana
6d363fb756
satellite/audit: create the audit queue, chore, and worker (#2888) 2019-09-05 11:40:52 -04:00
Bryan White
1fc0c63a1d
{cmd,pkg}/certificates: service refactor (#2938) 2019-09-05 17:11:21 +02:00
Bryan White
b222936d19
cmd/certificates: separate subcommand configs (#2932) 2019-09-05 16:56:57 +02:00
Egon Elbre
c4ee7eb51c
jenkins: build and test web/satellite (#2908) 2019-09-05 16:32:46 +03:00
Bogdan Artemenko
15e52e0985
web/satellite: Fixed usage tests. (#2963) 2019-09-05 16:10:46 +03:00
Bryan White
4bcaa59122 add list direction option to bucket_test.c (#2961) 2019-09-05 14:34:35 +02:00
Michal Niewrzal
61168493dc
uplink: don't stop deleting segments on first error (#2943) 2019-09-05 14:25:30 +02:00
Yehor Butko
d6bbc2a653
modules versions fixed (#2926) 2019-09-05 12:03:25 +03:00
Yehor Butko
bc8a531a22
dashboard refactored, tests added (#2929) 2019-09-05 11:41:39 +03:00
TopperDEL
e47f08afa0 lib/uplinkc: use ListDirection in struct BucketListOptions (#2946) 2019-09-05 09:25:27 +03:00
paul cannon
9821a21e5c satellite,storagenode,bootstrap: add contact service to peer (#2951)
* satellite,storagenode,bootstrap: add contact service to peer
2019-09-04 15:04:18 -04:00
paul cannon
adfa16188b pkg/contact: bare-bones service and endpoint (#2941)
* pkg/contact: bare-bones service and endpoint

* split contact package into satellite and node

* use new contact protobuf types
2019-09-04 11:29:34 -07:00
Bryan White
62df8ddb0c cmd/certificates: auth export improvements & certificates test script (#2897) 2019-09-04 19:58:38 +02:00
Cameron
af5fb8e9c5
satellite/vouchers: deprecate voucher endpoint, return 'please upgrade' error (#2940)
* voucher endpoint returns 'please upgrade' error, test
2019-09-04 13:21:02 -04:00
Jess G
f7bae57e5b
pk/pb: add initial proto for the new contact endpoints (#2948)
* add init contact proto

* make 2 svcs

* add geenerated proto code

* fux conflict with NodeRequest name

* use correct version on proto

* rm extra node fields

* add protolock

* update field names so they are better

* rm node id since we dont need it
2019-09-04 10:07:06 -07:00
littleskunk
05e9916b9c
storagenode/buildscript: Overwrite existing docker manifest file (#2934) 2019-09-04 18:55:03 +02:00
Yaroslav Vorobiov
dadd7327df
satellite/nodestats: return storage usage in Byte*hours (#2858) 2019-09-04 19:05:34 +03:00
Bogdan Artemenko
a4dddc65ba
web/satellite: Fixed Project Members deletion. (#2891) 2019-09-04 18:02:39 +03:00
Vitalii Shpital
534a74dac9 web/satellite: account billing page initial markup (#2928) 2019-09-04 17:33:25 +03:00
Yaroslav Vorobiov
f7403f97b0
storagenode/storageusage: add summary, rename timestamp to interval_start (#2911) 2019-09-04 17:13:43 +03:00
Yaroslav Vorobiov
758f7cb3dd
storagenode/bandwidth: remove bandwidth concerns from console, add satellite summary (#2923) 2019-09-04 17:01:55 +03:00
Ivan Fraixedes
83815ee7bf
satellitedb: always release savepoint processing orders (#2936)
* satellite/satellitedb: Always release savepoint
  Release the savepoint when processing orders in any case.
* satellite/satellitedb: Wrap errors exec savepoints
  Wrap the errors returned by the execution of savepoints operations when
  processing orders.
2019-09-04 15:02:20 +02:00
Stefan Benten
52cdca01a6 Add TopperDEL to the CLA list (#2947) 2019-09-04 12:28:00 +02:00
paul cannon
bf5a16b823 jenkins: work around git.apache.org failures (#2944) 2019-09-04 11:30:59 +03:00
Michal Niewrzal
ee614bf032
storagenode: add custom dial timeout for orders sending (#2939) 2019-09-03 17:32:28 +02:00
Ivan Fraixedes
4d65fdaea5
docs/design: Adapt SN Downtime tracking to be a blueprint (#2931)
Bluepints are replacing the design documents.
A blueprint has a wrapup section, this commit adds the section to the
document and replace any mention to design document by blueprint.
2019-09-03 16:44:10 +02:00
Ivan Fraixedes
d7d6e23a3e
pkg/server: Don't use Sugared logger (#2935)
Replace the usage of the Sugared logger by the normal one.
2019-09-03 11:39:26 +02:00
Michal Niewrzal
3fbe31aada
storagenode: Increase order sending request timeout (#2930) 2019-09-02 13:24:02 +02:00
Yingrong Zhao
10a896bf73
web/marketing: static asset path (#2872)
* use relative path instead of absolute path

* add template func baseURL

* add a method

* update storj-sim

* add comment
2019-08-30 18:43:53 -04:00
Michal Niewrzal
a6721ba92f
satellite/metainfo: Improve metainfo ListSegments (#2882) 2019-08-30 23:30:18 +02:00
Natalie Villasana
9a1b9f8431
uplink/ecclient: change delete logs from err to debug level (#2917) 2019-08-30 17:00:34 -04:00
Yehor Butko
7e702aa107
web/satellite: navigation, button and project members unit tests (#2907) 2019-08-30 14:34:51 +03:00
Michal Niewrzal
c680970e03
Storage node auto-updater skeleton (#2902) 2019-08-30 11:02:36 +02:00
JT Olio
48ba45eb55
pkg/process prometheus: forgot digits (#2920)
Change-Id: I1825d31dedcce45a80663d27261e2db0cb14c0f1
2019-08-29 17:51:03 -06:00
JT Olio
539f3857f1
web/satellite: downgrade typescript and pin stuff (#2918)
Change-Id: Ie14bd099fec1d6e7e30d4d0fe1d8878fdf2d4d31
2019-08-29 17:27:48 -06:00
Jess G
b20dcfd64c add type to prometheus metrics (#2916) 2019-08-29 23:42:38 +02:00
littleskunk
9d1910cb2b
storagenode/orderarchive: Reduce TTL from 45 to 7 days (#2915) 2019-08-29 22:38:09 +02:00
JT Olio
b3f9a8813d pkg/process: remove prometheus help (#2914)
the current prometheus help messages have enough unexpected
characters that they are breaking prometheus parsing. they
may also be triggering prometheus to expect more from us (type
annotations) than we have to offer.

we're really not adding a lot of value with these help messages,
so just take them out

Change-Id: I9b723447a294bb492a6292480e9f88634346a80b
2019-08-29 12:42:11 -07:00
Ivan Fraixedes
537769d7fa
storagenode/orders: Don't return error Archiving unsent (#2903)
Don't return error when archiving errors which aren't found in the DB
because it causes Storage Node send orders cycle to stop.

This was applied in the commit e47b8ed131
but the last call to orders.Archive function was missed so the errors
weren't returned when not found orders in the first call but they were
returned in the second call.

This commit address the second call for making handleBatches function
never returns error on not found orders.
2019-08-29 20:22:22 +02:00
Matt Robinson
243d280591
Clean up jenkins message to slack (#2912) 2019-08-29 13:25:49 -04:00
Egon Elbre
8a5db77e04
storagenode/retain: add comment (#2910) 2019-08-29 19:42:17 +03:00
littleskunk
7e37452abb open port for storage node dashboard API (#2899) 2019-08-29 11:47:43 -04:00
Matt Robinson
aa37334e7a
Send failures to slack for added visibility (#2909) 2019-08-29 11:35:02 -04:00
ethanadams
4ede12a2ab
satellite/orders: Fix for V3-2529: Release v0.19.0 storage nodes can't submit orders, duplicate key value violates unique constraint (#2900)
* V3-2529: Add DB savepoint to fix issue with postgres. Add test force a rejected order

Co-Authored-By: Ivan Fraixedes <ivan@fraixed.es>

* Update satellite/satellitedb/orders.go
2019-08-29 11:14:10 -04:00