Commit Graph

5259 Commits

Author SHA1 Message Date
JT Olio
b2ed7edd30 cmd/satellite: restore-trash parallel workers
Change-Id: Ic7466b21c20bda334e7ba4268a494e96b6528ac1
2021-02-18 19:11:19 +02:00
Michał Niewrzał
69151965b9 cmd/satellite: add logic to send RestoreTrash to nodes
Change-Id: I69815fa766a40a7266787e96bf9662854ae56937
2021-02-18 09:19:30 -07:00
JT Olio
3ae3389ddc cmd/satellite: restore-trash command
Change-Id: I80fc932c12147692d49cde277784871ac611fcad
2021-02-18 09:19:22 -07:00
Egon Elbre
adf687aebb satellite/metainfo/metabase: add full iterator
Add FullIterateObjects that iterates over all objects in the metabase.

Change-Id: I4b8dbda32dfce2d7729e9574af79e4f20faed98a
2021-02-18 16:51:35 +02:00
NickolaiYurchenko
50239e66f7 web/storagenode: zksync receipt convertation
Change-Id: I5b7b44c4e81a5a5dadf20db4c8ef852d71a01bd3
2021-02-18 09:30:54 +00:00
NickolaiYurchenko
212baf6145 web/storagenode: balance section markup, current month estimated payout frontend implementation
Change-Id: I8820c842e2773a114423eab8370b5c4dc8f73cdd
2021-02-18 09:52:58 +02:00
Jeff Wendling
3eb321fd33 storagenode/console/consoleapi: fix flaky test
Change-Id: I1af928953a19155e3c2e9f8c63e7162978e7b475
2021-02-17 19:48:55 +00:00
Jeff Wendling
9e7e753faf sn/sndb/payouts: migrate distributed == paid for periods < 2020-12
while satellites have also run this logic, old satellites that
no longer exist cannot and so the node cannot get the updated
data. this locally migrates it so that the calculations for
the undistributed amounts are correct.

there's also some tab/space whitespace and gofmt fixes.

Change-Id: I470879703314fe6541eaba5f21b47849781894f8
2021-02-17 13:07:10 -05:00
Stefan Benten
f969e26827
Jenkinsfile, Makefile: bump to go v1.16 (#4041) 2021-02-17 16:54:04 +01:00
Qweder93
dcf7b884ca storagenode/payouts: estimated payouts calculations fixed
Change-Id: I9d5d180aea1606350e0d9214fbf352bf9f76c184
2021-02-17 12:58:41 +00:00
Michał Niewrzał
dd9ad09301 satellite/metainfo: return RS value with DownloadSegment response
Until now we where using single RS per object but it turns out that we
need to be able to support RS per segment. We need to give uplink such information while downloading.

As an addition we are using RedundancySchemePerSegment flag for GetObject request to detect if
we should try to get RS from segment for this request response.

Change-Id: I209dad324496ff59b521b11d2343da61dcdbe7f5
2021-02-17 12:33:48 +00:00
Michał Niewrzał
f1169ad7bd satellite/metainfo: return RS value with DownloadSegment response
Until now we where using single RS per object but it turns out that we
need to be able to support RS per segment. We need to give uplink such information while downloading.

Change-Id: I6565b7c08962b3a1429f6079e7c2023a0a7c8b72
2021-02-17 12:39:53 +01:00
Egon Elbre
61f0fb67a9 satellite/metainfo/metabase: refresh alias cache only once
When there are concurrent refreshes to the cache and the entries are
missing, it could end up causing multiple database calls, even though
only one is needed.

Change-Id: I1ae7a124bbdd1570473cf3a032d375d2f25a8426
2021-02-17 10:00:04 +00:00
Fadila Khadar
259b030b3a satellite/metabase: add tests for BeginObject
adds tests to BeginObjectNextVersion and BeginObjectExactVersion
to check the behavior when an older or a newer committed version
exists.

The current behavior is: everything is committed.

Change-Id: Ia8facbe0dc038a5d214e4e56da3c8e4df2f18900
2021-02-17 09:02:26 +00:00
Michał Niewrzał
12402eb729 Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I38adf8218c1415c7ea1910f8bd6bed13544b0f03
2021-02-17 08:50:38 +01:00
VitaliiShpital
28433327cc web/satellite: remove upload step from create AG flow, close webworker on logout, styling fixes
WHAT:
remove upload info step from create access grant flow
close webworker on logout to always initiate new one on login
small styling fixes

WHY:
better user experience

Change-Id: I41ae0fe859b29889a9cced30f874d4a341e09ad2
2021-02-16 18:46:15 +00:00
Michał Niewrzał
6ebe06cd1b satellite/metainfo: fix for getting real RS values
Change-Id: Ib2cd01b2a68baab9f342dc0ff1ab8e5f12f4557f
2021-02-16 17:26:09 +01:00
Michał Niewrzał
8685c6c9cd private/testplanet: use better method in test
In ec client in uplink we have two methods Put and PutSingleResult. Logic is the same but result for PutSingleResult is combined into single var. We would like to remove unused Put method but to do this it needs to be replaced in test.

Change-Id: Ia65eff3ecc9e68d5b3180b557ea82fa31d3c969c
2021-02-16 13:24:01 +00:00
Fadila Khadar
5dd76522af gracefulexit: use GetSegmentByLocation instead of GetObjectLatestVersion
This enables the transfer of pieces from an on-going multipart upload.
Tests are also modified to take into account pending multipart uploads.

See https://storjlabs.atlassian.net/browse/PG-161

Change-Id: I35d433c44dd6e618667e5e8f9f998ef867b9f1ad
2021-02-16 10:49:36 +00:00
Egon Elbre
f7ad86521e satellite/overlay: fix data race in TestAuditHistoryBasic
Change-Id: I196f10973fe10b10b226ac3a63e62bf4fe9c256b
2021-02-16 12:32:19 +02:00
Michał Niewrzał
79d6294dbe metainfo-migration: fix data race
Change-Id: Idc266509fe7a6a717689f090a1409db09c8e9daa
2021-02-16 09:53:17 +01:00
NickolaiYurchenko
ed1caede97 web: size package fixed
Change-Id: Ie07666d6a9d7e72c61a3c2b842295da9076f9407
2021-02-15 17:15:08 +02:00
Malcolm Bouzi
4e3ac2c490 satellite/console: add new professional fields to console api
Change-Id: Ib7a69e117c7cf7f5eacd9f4ffc2c79fc94f01811
2021-02-12 17:21:09 -05:00
Kaloyan Raev
9bcffa3d02 cmd/metainfo-verifier: verifies if metainfo migration is done correctly
Change-Id: Id47f32fe098b5f4ac3b5e6829f5dd21e3a98c6da
2021-02-12 14:31:02 +00:00
Malcolm Bouzi
4b2e46a0c9 satellite/satellitedb: add employee size column to users
Change-Id: I21f5904331f0ceb92f494729c22a52c256a69163
2021-02-12 09:15:15 -05:00
Michał Niewrzał
b8513e2fcb satellite/metainfo: get encryption from StreamMeta for old uplinks
Old uplinks sends some additional information inside marshaled protobufs and we need to extract things like encryption parameters. Newer uplinks are passing it directly in request.

Change-Id: I0b575e68c3ed98481247fe38344e7d61cbd542ba
2021-02-12 13:15:49 +00:00
Jeff Wendling
fe5b6e1725 sat/console/consolewasm: restrict enc access to api key
Change-Id: Ie263ffc7343e59ee9a90996b259bb04258a78f29
2021-02-11 17:52:32 -05:00
Michał Niewrzał
341a4b7f59 metainfo-migration: extend test case
Change-Id: I9d1ac41a18bb08200ef20f7ff2f8df5531140f99
2021-02-11 19:28:41 +01:00
Yaroslav Vorobiov
2e1455bc55 multinode/multinodedb: add sqlite3 support
Change-Id: I023ffb75f836de2c33eb7dbee52072e1622448bd
2021-02-11 16:36:05 +00:00
Michał Niewrzał
5a23d9c167 metainfo-migrator: improve performance
Change-Id: I258605737c37dfdd8f0070dbadf0e3a9c6358cc3
2021-02-11 17:33:22 +01:00
Egon Elbre
2848bf488f satellite/metainfo/metabase: use alias pieces in segments table
This makes all tables automatically convert between aliases and piece
ID-s.

Change-Id: I27fa42c82bbb09e05e3327f85e13a000b48faffd
2021-02-11 16:40:37 +02:00
Egon Elbre
25f81f353c satellite/metainfo/metabase: add AliasPiece
This adds AliasPieces run length encoding. On average it should
make our pieces encoding:

   repair=50,optimal=85,total=90   152.0 bytes
   repair=16,optimal=37,total=50    65.4 bytes

Change-Id: I391a9183164828f05383a3cde9ab0e4549c2d440
2021-02-11 14:23:51 +02:00
Egon Elbre
63c7f8b7fc satellitedb/satellitedbtest: creating a database shouldn't auto-migrate
Some tests need to control the migration progress manually.

Change-Id: I776c69b6d56dc35c7cb88688c4b827d6bba4b7ac
2021-02-11 14:21:49 +02:00
Michał Niewrzał
908a96ae30 Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I075aaff42ca3f5dc538356cedfccd5939c75e791
2021-02-11 11:48:23 +01:00
Michał Niewrzał
dc2bec9f89 satellite/metainfo: collect uplink versions with monkit
This change adds metrics for uplink version and method which was used by this version.

Change-Id: I05fc425a024805cfcd6d8add810a2a0749405f4f
2021-02-10 17:47:13 +01:00
Egon Elbre
9cfaba2c5d private/tagsql: close Stmt debug tracker
Trackers check whether rows, transactions and statments are properly
closed when tests finish. Stmt wasn't closing the tracker, which
ended up causing false-positives.

Change-Id: I5ac57bc1579f903f19283b85a1996c06ce87741b
2021-02-10 16:49:43 +02:00
Qweder93
a86bd27513 storagenode/storagenodedb: paystubs distributed not null added
Change-Id: I021ba1c4ce4865f2575466d9b4d702775a5654cf
2021-02-10 12:51:09 +00:00
NickolaiYurchenko
4661bc9b84 web/storagenode: notification createdAt bug fixed
Change-Id: If989f3db1dfa947cd598b9aad906df074761b954
2021-02-10 11:51:44 +00:00
Qweder93
49884b5aad multinode/console: payouts added
Change-Id: I003b6a6286e39f43b3eb5e50de54b0a3d62f5abc
2021-02-10 09:45:16 +00:00
JT Olio
1ac17cfac5 uplink: link.tardigradeshare.io URLs should have an /s/ right after the domain
should be
https://link.tardigradeshare.io/s/<access>/<bucket>/<path>

legacy URLs have the /s/ missing but a redirect is issued

Change-Id: Ic2a3dc092ff68d7706fd888a9fbfc27716877c6e
2021-02-09 14:38:22 -07:00
Rafael Gomes
d9d6edde71 go.mod: bump monkit-jaeger to replace dashes by underscore
Change-Id: I51fd920901ff559e9def795e9183bbd6d06e4045
2021-02-09 15:24:00 -03:00
Cameron Ayer
4a797baa73 satellite/repair/repairer: a new set of rs_scheme tagged metrics
Change-Id: Ibecd9265da881247eeb85ba185ee8877a7243777
2021-02-09 14:19:22 +00:00
Moby von Briesen
520da5b1e4 scripts/tests/rollingupgrade: fix rollingupgrade test
Fix logic that determines whether satellite has the marketing web server
based on version number.

Change-Id: I34df745d16acf1ce1478fe992448b18565e6d363
2021-02-08 20:06:00 -05:00
Yingrong Zhao
babe01f229 scripts/tests/rollingupgrade: fix rollingupgrade test backwards
compatibility

Change-Id: I1551f993bc8ca5e6fac07da7710dd46dbb031e47
2021-02-08 21:37:24 +00:00
Yaroslav Vorobiov
1e328f3c30 satellite/compensation: add wallet features to invoice csv
Change-Id: I90a72c0a4c8d07604096913b6680263b6defc0a2
2021-02-08 22:13:48 +02:00
Yaroslav Vorobiov
966535e9de {storagenode,satellite}/nodeoperator: add wallet features
Change-Id: Iac7eb40a52b8fddcc573aebaad2e3a30a10cded9
2021-02-08 22:09:45 +02:00
VitaliiShpital
570dcfad3e web/satellite: add segment event for Generate Gateway Credentials button
WHAT:
we want to track how many people click Generate Gateway Credentials button

WHY:
user tracking

Change-Id: Iad6a38a552f5773bbea7c07bac3562365a5f57d9
2021-02-08 18:31:28 +00:00
VitaliiShpital
fcd18ef48a web/satellite: rework signup verify redirect flow
WHAT:
people who sign up on US2 are not redirected to verifying page. From now on we have to set verify URL to make redirect happen

WHY:
user experience

Change-Id: I96c51a2c4f9cb6376cbfea639675b32918b58bee
2021-02-08 18:00:57 +00:00
VitaliiShpital
fd095e604f web/satellite: fix for access grants creation datepicker
WHAT:
when user chooses the same date in datepicker it sets grant's NotBefore and NotAfter restrictions as the same date which makes access grant unusable

WHY:
bug fix

Change-Id: I416606610ebddfd1fa881cce89d2beeded48457f
2021-02-08 17:37:20 +00:00
Yingrong Zhao
89529237a2 web/satellite, web/marketing: remove referral program related UI code
Change-Id: Ia815a72c0d17760bb63d9a37bb18b88952209368
2021-02-08 11:07:35 -05:00