Commit Graph

5250 Commits

Author SHA1 Message Date
Qweder93
dcf7b884ca storagenode/payouts: estimated payouts calculations fixed
Change-Id: I9d5d180aea1606350e0d9214fbf352bf9f76c184
2021-02-17 12:58:41 +00:00
Michał Niewrzał
dd9ad09301 satellite/metainfo: return RS value with DownloadSegment response
Until now we where using single RS per object but it turns out that we
need to be able to support RS per segment. We need to give uplink such information while downloading.

As an addition we are using RedundancySchemePerSegment flag for GetObject request to detect if
we should try to get RS from segment for this request response.

Change-Id: I209dad324496ff59b521b11d2343da61dcdbe7f5
2021-02-17 12:33:48 +00:00
Michał Niewrzał
f1169ad7bd satellite/metainfo: return RS value with DownloadSegment response
Until now we where using single RS per object but it turns out that we
need to be able to support RS per segment. We need to give uplink such information while downloading.

Change-Id: I6565b7c08962b3a1429f6079e7c2023a0a7c8b72
2021-02-17 12:39:53 +01:00
Egon Elbre
61f0fb67a9 satellite/metainfo/metabase: refresh alias cache only once
When there are concurrent refreshes to the cache and the entries are
missing, it could end up causing multiple database calls, even though
only one is needed.

Change-Id: I1ae7a124bbdd1570473cf3a032d375d2f25a8426
2021-02-17 10:00:04 +00:00
Fadila Khadar
259b030b3a satellite/metabase: add tests for BeginObject
adds tests to BeginObjectNextVersion and BeginObjectExactVersion
to check the behavior when an older or a newer committed version
exists.

The current behavior is: everything is committed.

Change-Id: Ia8facbe0dc038a5d214e4e56da3c8e4df2f18900
2021-02-17 09:02:26 +00:00
Michał Niewrzał
12402eb729 Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I38adf8218c1415c7ea1910f8bd6bed13544b0f03
2021-02-17 08:50:38 +01:00
VitaliiShpital
28433327cc web/satellite: remove upload step from create AG flow, close webworker on logout, styling fixes
WHAT:
remove upload info step from create access grant flow
close webworker on logout to always initiate new one on login
small styling fixes

WHY:
better user experience

Change-Id: I41ae0fe859b29889a9cced30f874d4a341e09ad2
2021-02-16 18:46:15 +00:00
Michał Niewrzał
6ebe06cd1b satellite/metainfo: fix for getting real RS values
Change-Id: Ib2cd01b2a68baab9f342dc0ff1ab8e5f12f4557f
2021-02-16 17:26:09 +01:00
Michał Niewrzał
8685c6c9cd private/testplanet: use better method in test
In ec client in uplink we have two methods Put and PutSingleResult. Logic is the same but result for PutSingleResult is combined into single var. We would like to remove unused Put method but to do this it needs to be replaced in test.

Change-Id: Ia65eff3ecc9e68d5b3180b557ea82fa31d3c969c
2021-02-16 13:24:01 +00:00
Fadila Khadar
5dd76522af gracefulexit: use GetSegmentByLocation instead of GetObjectLatestVersion
This enables the transfer of pieces from an on-going multipart upload.
Tests are also modified to take into account pending multipart uploads.

See https://storjlabs.atlassian.net/browse/PG-161

Change-Id: I35d433c44dd6e618667e5e8f9f998ef867b9f1ad
2021-02-16 10:49:36 +00:00
Egon Elbre
f7ad86521e satellite/overlay: fix data race in TestAuditHistoryBasic
Change-Id: I196f10973fe10b10b226ac3a63e62bf4fe9c256b
2021-02-16 12:32:19 +02:00
Michał Niewrzał
79d6294dbe metainfo-migration: fix data race
Change-Id: Idc266509fe7a6a717689f090a1409db09c8e9daa
2021-02-16 09:53:17 +01:00
NickolaiYurchenko
ed1caede97 web: size package fixed
Change-Id: Ie07666d6a9d7e72c61a3c2b842295da9076f9407
2021-02-15 17:15:08 +02:00
Malcolm Bouzi
4e3ac2c490 satellite/console: add new professional fields to console api
Change-Id: Ib7a69e117c7cf7f5eacd9f4ffc2c79fc94f01811
2021-02-12 17:21:09 -05:00
Kaloyan Raev
9bcffa3d02 cmd/metainfo-verifier: verifies if metainfo migration is done correctly
Change-Id: Id47f32fe098b5f4ac3b5e6829f5dd21e3a98c6da
2021-02-12 14:31:02 +00:00
Malcolm Bouzi
4b2e46a0c9 satellite/satellitedb: add employee size column to users
Change-Id: I21f5904331f0ceb92f494729c22a52c256a69163
2021-02-12 09:15:15 -05:00
Michał Niewrzał
b8513e2fcb satellite/metainfo: get encryption from StreamMeta for old uplinks
Old uplinks sends some additional information inside marshaled protobufs and we need to extract things like encryption parameters. Newer uplinks are passing it directly in request.

Change-Id: I0b575e68c3ed98481247fe38344e7d61cbd542ba
2021-02-12 13:15:49 +00:00
Jeff Wendling
fe5b6e1725 sat/console/consolewasm: restrict enc access to api key
Change-Id: Ie263ffc7343e59ee9a90996b259bb04258a78f29
2021-02-11 17:52:32 -05:00
Michał Niewrzał
341a4b7f59 metainfo-migration: extend test case
Change-Id: I9d1ac41a18bb08200ef20f7ff2f8df5531140f99
2021-02-11 19:28:41 +01:00
Yaroslav Vorobiov
2e1455bc55 multinode/multinodedb: add sqlite3 support
Change-Id: I023ffb75f836de2c33eb7dbee52072e1622448bd
2021-02-11 16:36:05 +00:00
Michał Niewrzał
5a23d9c167 metainfo-migrator: improve performance
Change-Id: I258605737c37dfdd8f0070dbadf0e3a9c6358cc3
2021-02-11 17:33:22 +01:00
Egon Elbre
2848bf488f satellite/metainfo/metabase: use alias pieces in segments table
This makes all tables automatically convert between aliases and piece
ID-s.

Change-Id: I27fa42c82bbb09e05e3327f85e13a000b48faffd
2021-02-11 16:40:37 +02:00
Egon Elbre
25f81f353c satellite/metainfo/metabase: add AliasPiece
This adds AliasPieces run length encoding. On average it should
make our pieces encoding:

   repair=50,optimal=85,total=90   152.0 bytes
   repair=16,optimal=37,total=50    65.4 bytes

Change-Id: I391a9183164828f05383a3cde9ab0e4549c2d440
2021-02-11 14:23:51 +02:00
Egon Elbre
63c7f8b7fc satellitedb/satellitedbtest: creating a database shouldn't auto-migrate
Some tests need to control the migration progress manually.

Change-Id: I776c69b6d56dc35c7cb88688c4b827d6bba4b7ac
2021-02-11 14:21:49 +02:00
Michał Niewrzał
908a96ae30 Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I075aaff42ca3f5dc538356cedfccd5939c75e791
2021-02-11 11:48:23 +01:00
Michał Niewrzał
dc2bec9f89 satellite/metainfo: collect uplink versions with monkit
This change adds metrics for uplink version and method which was used by this version.

Change-Id: I05fc425a024805cfcd6d8add810a2a0749405f4f
2021-02-10 17:47:13 +01:00
Egon Elbre
9cfaba2c5d private/tagsql: close Stmt debug tracker
Trackers check whether rows, transactions and statments are properly
closed when tests finish. Stmt wasn't closing the tracker, which
ended up causing false-positives.

Change-Id: I5ac57bc1579f903f19283b85a1996c06ce87741b
2021-02-10 16:49:43 +02:00
Qweder93
a86bd27513 storagenode/storagenodedb: paystubs distributed not null added
Change-Id: I021ba1c4ce4865f2575466d9b4d702775a5654cf
2021-02-10 12:51:09 +00:00
NickolaiYurchenko
4661bc9b84 web/storagenode: notification createdAt bug fixed
Change-Id: If989f3db1dfa947cd598b9aad906df074761b954
2021-02-10 11:51:44 +00:00
Qweder93
49884b5aad multinode/console: payouts added
Change-Id: I003b6a6286e39f43b3eb5e50de54b0a3d62f5abc
2021-02-10 09:45:16 +00:00
JT Olio
1ac17cfac5 uplink: link.tardigradeshare.io URLs should have an /s/ right after the domain
should be
https://link.tardigradeshare.io/s/<access>/<bucket>/<path>

legacy URLs have the /s/ missing but a redirect is issued

Change-Id: Ic2a3dc092ff68d7706fd888a9fbfc27716877c6e
2021-02-09 14:38:22 -07:00
Rafael Gomes
d9d6edde71 go.mod: bump monkit-jaeger to replace dashes by underscore
Change-Id: I51fd920901ff559e9def795e9183bbd6d06e4045
2021-02-09 15:24:00 -03:00
Cameron Ayer
4a797baa73 satellite/repair/repairer: a new set of rs_scheme tagged metrics
Change-Id: Ibecd9265da881247eeb85ba185ee8877a7243777
2021-02-09 14:19:22 +00:00
Moby von Briesen
520da5b1e4 scripts/tests/rollingupgrade: fix rollingupgrade test
Fix logic that determines whether satellite has the marketing web server
based on version number.

Change-Id: I34df745d16acf1ce1478fe992448b18565e6d363
2021-02-08 20:06:00 -05:00
Yingrong Zhao
babe01f229 scripts/tests/rollingupgrade: fix rollingupgrade test backwards
compatibility

Change-Id: I1551f993bc8ca5e6fac07da7710dd46dbb031e47
2021-02-08 21:37:24 +00:00
Yaroslav Vorobiov
1e328f3c30 satellite/compensation: add wallet features to invoice csv
Change-Id: I90a72c0a4c8d07604096913b6680263b6defc0a2
2021-02-08 22:13:48 +02:00
Yaroslav Vorobiov
966535e9de {storagenode,satellite}/nodeoperator: add wallet features
Change-Id: Iac7eb40a52b8fddcc573aebaad2e3a30a10cded9
2021-02-08 22:09:45 +02:00
VitaliiShpital
570dcfad3e web/satellite: add segment event for Generate Gateway Credentials button
WHAT:
we want to track how many people click Generate Gateway Credentials button

WHY:
user tracking

Change-Id: Iad6a38a552f5773bbea7c07bac3562365a5f57d9
2021-02-08 18:31:28 +00:00
VitaliiShpital
fcd18ef48a web/satellite: rework signup verify redirect flow
WHAT:
people who sign up on US2 are not redirected to verifying page. From now on we have to set verify URL to make redirect happen

WHY:
user experience

Change-Id: I96c51a2c4f9cb6376cbfea639675b32918b58bee
2021-02-08 18:00:57 +00:00
VitaliiShpital
fd095e604f web/satellite: fix for access grants creation datepicker
WHAT:
when user chooses the same date in datepicker it sets grant's NotBefore and NotAfter restrictions as the same date which makes access grant unusable

WHY:
bug fix

Change-Id: I416606610ebddfd1fa881cce89d2beeded48457f
2021-02-08 17:37:20 +00:00
Yingrong Zhao
89529237a2 web/satellite, web/marketing: remove referral program related UI code
Change-Id: Ia815a72c0d17760bb63d9a37bb18b88952209368
2021-02-08 11:07:35 -05:00
Yingrong Zhao
3b49d3cddf satellite: remove referral program related code
This PR removes all back-end related referral program code including the
marketing portal.

We will have a separate PR for front-end code and database migration to
drop `offers` and `usercredits` table

Change-Id: If59f952cddfe0558a7dc03a0eac7cc1081517f88
2021-02-08 13:52:50 +00:00
Qweder93
dc6a8cbcf3 storagenode/suspension: fixed timings of downtime suspension notifications
Change-Id: Ie5b4f369efc0e66910b2f939aa8c5b11dc92946c
2021-02-08 15:24:29 +02:00
NickolaiYurchenko
e86041a5cf web: stylint config moved from package.json
Change-Id: Ic16ef4190947e5af447072f24055ca9294526956
2021-02-08 10:48:06 +00:00
NickolaiYurchenko
cd57606209 web/storagenode: size package refactored
Change-Id: I139478c760b3eb480990a1ff3aec6f8b90c218ac
2021-02-08 12:19:58 +02:00
Egon Elbre
0c223f288a satellite/metainfo/metabase: don't use random node ID-s
We will add a cache to nodes, so using completely random nodes wouldn't
show the actual performance.

Change-Id: I94f18283712812f05f7795efd3c7cf57499fa52c
2021-02-08 11:36:05 +02:00
Egon Elbre
d0a7ab60ec satellite/metainfo/metabase: add NodeAliasCache
This implements a node alias cache that automatically refreshes
and adds new nodes when needed.

Change-Id: I4af121f7307b7b7ac4f87ad465f02d4eeb17d42f
2021-02-08 11:36:03 +02:00
Natalie Villasana
3cc1a0b393 dbutil/cockroachutil: make crdb retry when tcp error
Adds checks in the existing NeedsRetry cockroachutil method
to see if the error is from the net package and a tcp error,
in which case it should retry. Also adds checks for if the
error contains EOF-- we don't retry in this case because it's
unclear if the query succeeded or failed.
These checks are needed to prevent our app from crashing when
it gets "connection reset by peer" and other tcp errors that
we've seen frequently coming from the metainfo loop.

Change-Id: I194d4b120082393cb6dbda2dd86b44f4696a66a4
2021-02-08 08:49:22 +00:00
Egon Elbre
0260966de4 satellite/metainfo/metabase: add NodeAliasMap
We need to keep an inmemory cache to avoid lookups into aliases table.
This adds the inmemory state of the cache.

Change-Id: Ief2b9bb19e10b46839b9208472dfc3035eb49af3
2021-02-08 10:28:22 +02:00
Malcolm Bouzi
8b49b210a1
Projects list view (#4035)
web/satellite: add new projects list management page
- add manage projects link to Projects navigation dropdown
- add create project cta to page
- add pagination of projects page
- render dates, number of users and project name to list table
- update snapshots
2021-02-05 15:57:03 -05:00