JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,232 Commits 5 Branches 0 Tags 94 MiB
7d6973b5a2
Commit Graph

4232 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kaloyan Raev
2af7138928 web/satellite: change invoice link title to 'Invoice PDF' 
Change-Id: I820bb33f4ba88bb732d1c66ba3369ac4eac72f5c
 2020-05-07 18:14:40 +03:00
VitaliiShpital
9837230f1e web/satellite: project dropdown reworked 
Change-Id: I6267347693724ec34996af7625a917451aef631d
 2020-05-07 12:51:34 +00:00
Egon Elbre
ce6a500b0c satellite/overlay: support DistinctIP=false in selection cache 
Most of our tests and storj-sim are using DistinctIP.

Also fix bug with newNodeCount calculation.

Change-Id: I1a6d0efe7074908896a3322d19f487b929f0f0fc
 2020-05-07 11:04:32 +00:00
Yaroslav Vorobiov
d8bdc60e19 storagenode/console: remove last ping id and address 
Change-Id: I001e7d31d50aee7b9a07608158f8bd5c407c9c45
 2020-05-07 13:34:29 +03:00
Egon Elbre
90d859fbb8 private/testplanet: use drpc piecestore mock for testing 
Change-Id: Ia3f93f3c8b6584fb92f5d29025b7f0691120430e
 2020-05-07 10:54:49 +03:00
Egon Elbre
c5452a87ec private/testplanet: use drpc referral manager server 
Change-Id: I9e9e9a724c78c98859dd3e29416d766d8ffdca63
 2020-05-07 07:03:11 +00:00
Egon Elbre
2955c50bc1 satellite/overlay: fix data-race in node selection cache 
GetNodes returned references to nodes in the immutable state, however
some parts of code expect them to be modified.

Change-Id: I5be1866f95e0dbe062a6b6be60e29f2365c35faa
 2020-05-06 20:03:06 +03:00
crawter
693f224ffd storagenode/web: Segment.io integration 
Change-Id: I5b8054f39453bbfdebdbaaea1ffda4d5e706dab3
 2020-05-06 15:02:53 +00:00
Jess G
73b2cd3fd5
docs/blueprints: blueprint nodes cache (#3838) 
* init design for selecting nodes for uploads

Change-Id: I40920a88fc8c0c8591eaf3f4e711578ca02dad73

* spelling erro

Change-Id: I843945f356dbdfd5c8b7cefee91ab4172c41f3fb

* updates per comments

Change-Id: I2d6f0f9b4f22c4c7ae8ed80c4f533d2d9392badf

* add details

Change-Id: Id31ecfbe9f7ff068b127f04ba061a3fb9ad254cc

* mv redis to rationale section

Change-Id: I98d8565bb228555d92682d3b08c99c6056b0a84b

* add details about ReliabilityCache

Change-Id: I9f0ecf0a96a217c5e55c6a855b8e06e0a76ab574

* add option details

Change-Id: Icd229cbbdc263466d16856fdd82a021509a1c76a

* add opt for queue

Change-Id: I9d36e5e6f923440b6c22066a2aaf981aecc584d6

* final updates

Change-Id: Iee74985f3a45b3ef58ff23ef3ae26f609fb7f0d5
 2020-05-06 07:44:21 -07:00
Egon Elbre
4e94da3fda satellite/overlay: add feature flag for node selection cache 
Also distinguish the purpose for selecting nodes to avoid potential
confusion, what should allow caching and what shouldn't.

Change-Id: Iee2451c1f10d0f1c81feb1641507400d89918d61
 2020-05-06 16:13:47 +03:00
Fadila Khadar
e4da7d65f0 pkg/macaroons: fix action types comments 
Fix comments in ActionType definition

Change-Id: I29a300a0699697a10ebf5613ba36a2702aa67175
 2020-05-06 11:58:56 +02:00
Jeff Wendling
efee3563a8 storagenode/storagenodedb: fix joined_at migration 
Change-Id: I9091741bbec6e7a2fc6addc23249a40176ec040a
 2020-05-05 15:01:07 -06:00
Qweder93
b754806b23 storagenode/reputation: unknown_audit_reputation_alpha and beta added to db, and reputation endpoint 
Change-Id: I5d92268851bb9a202cc5d6b4d403467e6f692726
 2020-05-05 15:48:04 +00:00
Qweder93
8a443bc9f5 storagenode/console: api notifications tests added 
Change-Id: I472abc22746ea95841dcb0ae72dacee06fd98a2e
 2020-05-05 17:53:19 +03:00
Kaloyan Raev
b5433322b4 cmd/storj-sim: enable open user registration 
Change-Id: I695664279f42209245a82cc5bf103d037b844c32
 2020-05-05 13:03:02 +00:00
Qweder93
16cd9b06ec storagenode/heldamount: added api for heldamount history separated by periods 
Change-Id: I170010364269822848bc6cd051e0e0fb3df95d91
 2020-05-05 12:29:44 +00:00
VitaliiShpital
0ae0cea93b web/satellite: billing history UX improved 
added sorting header title
currency numbers are displayed with two digits now

Change-Id: I9531f112ff70ffb89b8a2dfe05ab02eb767dd107
 2020-05-05 12:07:33 +00:00
Jennifer Johnson
18078bf7ee satellite/audit: increases audit worker concurrency to 2 
Change-Id: Ibe3e3801b79accffbcfe9e2e02c96fc963894a7f
 2020-05-05 11:31:55 +00:00
NickolaiYurchenko
c5871b83b0 web/storagenode: no data state added to payout table 
Change-Id: I3f141debe2f7c04a20999c0d2e2a7cee1434f7d1
 2020-05-05 11:10:10 +00:00
Egon Elbre
b7495d414d mod: update sqlite to v3.31.1 
Change-Id: Ia2885f5385ffa227b8576c1d9e1c8bbba0c5d13d
 2020-05-05 10:52:36 +00:00
Egon Elbre
d98b8f6e23 satellite/metainfo,storage: use different limit for metainfo loop 
Change-Id: I5ef7233930679b977b33f7b3e1dda45c907dcfad
 2020-05-05 10:37:20 +00:00
Jeff Wendling
13baacc479 lib/uplink: better diagnostics with invalid encryption access 
Fixes #3841

Change-Id: I965b8fbb8fd221c740c2fc1421d66721c046f038
 2020-05-05 09:45:57 +00:00
Brandon Iglesias
324fd410a0
updating Aha link on our readme 2020-05-04 14:21:58 -04:00
Moby von Briesen
8f60cfc4fb satellite/overlay: Add flag for enabling/disabling disqualification from suspension mode 
Add a flag that allows us to easily switch disqualification from
suspension mode on or off. A node will only be disqualified from
suspension mode if it has been suspended for longer than the grace
period AND the SuspensionDQEnabled flag is true.

Change-Id: I9e67caa727183cd52ab2042b0a370a1bcaebe792
 2020-05-04 17:25:09 +00:00
Jeff Wendling
f93d764998 cmd/uplink: properly path escape shares 
Fixes #3829

Change-Id: I0f900a392239fcc064cea968fc7a5bc373ee26a1
 2020-05-04 17:00:25 +00:00
Egon Elbre
31b322f07e pkg/server: log apikey head 
Change-Id: Ib86987405ea73fe8a1b5cec93cf3b219e36074d1
 2020-05-04 16:36:02 +00:00
Ethan
acf53bea4d satellite/orders;accounting: Add monthly project download bandwidth rollup 
See https://storjlabs.atlassian.net/browse/SM-776

Change-Id: Ifd5cccea43c556fd59822d17344f399cfe9a7164
 2020-05-04 15:49:57 +00:00
Michal Niewrzal
22691bd47f satellite/accounting: test project limits reset after file delete 
https://storjlabs.atlassian.net/browse/USR-288

Change-Id: Ia63e58ef3b0f79b5edfb9a63f8edcdc17486b650
 2020-05-04 15:19:19 +00:00
Yingrong Zhao
0b80d60491 cmd/uplink: disable tlemetry if user opt out 
Change-Id: I793e6428d04e4f422d1f8292e99b233c70d9dd6e
 2020-05-04 14:46:49 +00:00
VitaliiShpital
2284008b8c web/satellite: onboarding tour: api keys and upload data steps 
Change-Id: I8ffa6d688a22c1568495a7e0e176096cadcd6eaa
 2020-05-04 13:26:12 +00:00
Qweder93
7a83473f00 storagenode/console: API /dashboard disk-space data separated for UsedForPieces, Trash, Available 
Change-Id: I7411da2c92c72a24af98e007efe476d8a023db82
 2020-05-04 12:36:18 +00:00
Michal Niewrzal
16081f53d2 storj: update to uplink 1.0.5 
Change-Id: Ie4f01356edc43c3ee2d124a69f4da1eb75a17618
 2020-05-04 08:09:18 +00:00
Egon Elbre
2fad150952 storagenode/piecestore: remove piecestore.Client.Delete 
This test was the last place using it. Replace it with a direct call so
we can remove the method from uplink piecestore.

Change-Id: I62e13028663a7e67aa2495f90ecc02d0d8657fbd
 2020-05-04 10:14:55 +03:00
Jeff Wendling
57eb8a17e2 storagenode: allow configuring database path independently 
Fixes #3852

Change-Id: I021c29c4dd7c393399f6abef41d8457514032833
 2020-05-04 06:04:31 +00:00
Egon Elbre
c630cf2490 storagenode/pieces: implement buffering for writing 
Currently uploads can cause a lot of IOPS, reduce this by introducing a
in-memory buffer on-top of the file.

Change-Id: I5f4e3e01c0a36258271d180b922107de447bcb59
 2020-05-04 06:01:32 +00:00
Stefan Benten
01e0ba2e0d
satellite/metainfo: move logging to debug for piece deletion (#3873) 2020-05-01 23:29:28 +02:00
Jeff Wendling
4ad01e8170 storagenode/storagenodedb: backfill reputation.joined_at 
it was being used in ways that implied it should be NOT NULL
even though it was possibly null. we used to get this data
from the satellite db's added_at column as seen in 30369b02,
so backfill it using that data where joined_at is NULL, and
then alter the table to constrain the column to be NOT NULL.

Fixes #3866.

Change-Id: If2d856189209740d985f71dada7b93525e625ef3
 2020-05-01 17:59:53 +00:00
Qweder93
6c4d3f133f storagenode/dashboard: trash added to avaliable space calculations 
Change-Id: Ia6f3af20dc98f569b86796ffa68428065d662c78
 2020-05-01 15:26:02 +00:00
Jeff Wendling
202e65e408 storagenode/storagenodedb: do correct generalized alter table procedure 
According to the docs at https://www.sqlite.org/lang_altertable.html
doing the steps

    1. Rename old table
    2. Create new table
    3. Copy data
    4. Drop old table

is incorrect and should be

    1. Create new table
    2. Copy data
    3. Drop old table
    4. Rename new into old

Additionally, each step was being run in a different transaction,
which could cause permanent failures if a problem happened during
the migration.

Avoid both of those problems by changing up some previous migrations
that ran in this way. Since they are semantically identical, it's
fine to change up these old migrations. It will help make newer
nodes coming up for the first time more robust.

Change-Id: I43fb004fa1b6cb2fe2554f9920925420da28fb4a
 2020-05-01 13:53:26 +00:00
Yaroslav Vorobiov
516b8cf2be storagenode-updater: add recovery for windows service restart 
Reimplement windows service restart part using svc, add recovery
for failed service startup. Added restart-service cmd, to execute
self restart in a separate process.

Addressed issues:
https://storjlabs.atlassian.net/browse/SG-49
https://storjlabs.atlassian.net/browse/SG-136
https://storjlabs.atlassian.net/browse/SG-137

Change-Id: Ic51d9a99e8c1c10800c6c60ff4e218321c674fea
 2020-05-01 09:07:05 +00:00
Jeff Wendling
bb28851964 test-sim-backwards: refactor, reduce node count to 9 in mixed mode 
This catches the bug that hit prod where we missed that a migration
was necessary for a column, but only on uploads, and only for nodes
that had not checked in yet.

So, this does more uploads, and stops a node from starting so that
we trigger that specific scenario. Hopefully it will catch other
similar ones in the future.

I confirmed that this locally caught the bug when the release under
test was v0.34.10 and HEAD did not include the fix for it.

Change-Id: If7d41e8241d6a042fa524b4aff956b0264ecb128
 2020-05-01 04:56:06 +00:00
Isaac Hess
5a85e8d749 satellite/audit: Fix flaky TestVerifierSlowDownload 
TestVerifierSlowDownload would sometimes not have enough nodes finish in
the allotted deadline period. This increases the deadline and also does
not assert that exactly 3 have finished. Instead, in keeping with the
purpose of the test, it asserts that the slow download is never counted
as a success and is always counted as a pending audit in the final
report.

Change-Id: I180734fcc4a499420c75164bad6253ed155d87de
 2020-04-30 15:58:47 -06:00
Yaroslav Vorobiov
2b00d78add satellite/payments: add documentation 
Change-Id: I75712ef282530b56ea7ca1f2bc31ee0af43dae33
 2020-04-30 21:12:16 +00:00
Matt Robinson
4a95ad0b2f
storagenode/piecestore: remove error from info log for download canceled (#3869) 2020-04-30 18:42:16 +03:00
Qweder93
0dfbdae614 private/date: MonthsCountSince removed, being unused anymore 
Change-Id: I666d29b91bc1283c1abb7c3a70b15417c1289f59
 2020-04-30 15:14:17 +00:00
Egon Elbre
2c47a9b1d9 pkg/server: log api key in debug traffic 
Change-Id: I677832200c9599c37c22cf742b396ca12542a803
 2020-04-30 17:42:32 +03:00
VitaliiShpital
4e5e6c8e5c web/satellite: onboarding tour: create project step 
Change-Id: If091fd7f703e61c402a6bbdd80d98b0ce7b2bfd6
 2020-04-30 16:50:19 +03:00
Qweder93
f54a4960a8 private/date: TestMonthCountSince temporary fix 
Change-Id: Ifc6f590f9fcdbd15ed84766cfe7a5809aa8a33f8
 2020-04-30 12:52:44 +00:00
Egon Elbre
00b2dfa313 storagenode/orders: fix TestDB and TestCleanArchive 
Tests were relying on too exact time values.

Change-Id: I83aa0e815b06372a65ec2d62b23df86051050d19
 2020-04-30 15:00:06 +03:00
Egon Elbre
418a4cc926 satellite/console: fix flaky TestProjectsList 
time.Now can return the exact time, ensure that it uses a later
timestamp in the listing.

Change-Id: I9bcbdf3d222804d949d3899cf3d1575f4de93f06
 2020-04-30 13:48:29 +03:00