Commit Graph

5862 Commits

Author SHA1 Message Date
Egon Elbre
71eb184ef3 storagenode/piecestore: simplify TestTooManyRequests
Change-Id: I1452735bef206bc8cf9bde72fc503fec3ae5b067
2021-09-21 17:18:37 +03:00
Egon Elbre
36911b44a3 satellite/accounting/tally: make tests faster
Change-Id: I706399043479fc4c4ae3abc0bc70ae2db0318096
2021-09-21 17:18:37 +03:00
Egon Elbre
df09e7d1f7 satellite/metainfo: ensure storagenodes finish work for test
Some processing inside storagenodes is async compared to uplink upload
and download, hence we need to explicitly wait for storagenodes to
finish their pending work before flushing orders to the satellite.

Hopefully this fixes TestAttributionReport flakiness.

Change-Id: I77c651ab6471ae094b5c21d1ab3860c96cb0d039
2021-09-21 17:18:37 +03:00
Vitalii Shpital
8b91c55ec4 web/satellite, satellite/console: return old onboarding flow with feature flag
Added new feature flag for switching between different flows.
Got back old onboarding flow.

Change-Id: Idbf23ba8ff36900a47e3b4ebeb34b78351202e29
2021-09-21 11:37:30 +00:00
Michał Niewrzał
38366914b2 satellite/metainfo: add position to begin move results
BeginMoveObject response was missing Position.

Change-Id: I2a91f0b1ac39c476731a2755754e46970c8657be
2021-09-21 10:23:33 +00:00
Egon Elbre
32cee1e572 satellite/metabase/segmentloop: ensure we shutdown on ctx err
Not all errors from RunOnce can be retried. The context can be cancelled
with several different errors, e.g. timeout. Ensure we stop the loop
when context has errored, because none of the queries will succeed
when it has failed.

Change-Id: If3ff11f11a6f43c0d67633be1cfaf23e3e9e55f3
2021-09-21 10:02:00 +00:00
Vitalii Shpital
69d3a839fc web/satellite: fix for onb CLI upload step image
Fixed SVG image usage ID for onb CLI upload step

Change-Id: Ieb94902de5702594edb20817541a8d968c0ad4f1
2021-09-20 20:07:48 +03:00
Vitalii Shpital
91c69a8afc web/satellite: fix for encrypt your data screen
Fixed issue for Encrypt your data screen. It was impossible to proceed further into object browser after selecting Generate passphrase option

Change-Id: Ibddda3c6bd7fce34bd1c629bbbb2d87529964b8a
2021-09-20 19:34:34 +03:00
crawter
5e4b196b11 satellite/metainfo: finish move object
Second method needed to perform server-side move. It updates
metadata key and nonce and all segments key and nonces.

Change-Id: Ia43b26622a13048269f0ae9e1524b345db112adb
2021-09-20 16:21:01 +03:00
Vitalii Shpital
9153b221fd testsuite/ui/satellite: updated pnboarding CLI tests and improved selectors
Updated onb CLI flow test.
Updated tests to use aria-roledescription as selector.

Change-Id: I1442592e1938fbe730064977dc9b78266e403706
2021-09-17 15:11:47 +00:00
Vitalii Shpital
853a7536f6 web/satellite: reworked onb CLI flow to correspond v1.1
Removed useless screens and fixed routing.
Removed useless logic from common component.

Change-Id: I73a1af5803f866526d3ae327e832654501e4f8bd
2021-09-17 15:11:41 +00:00
Yingrong Zhao
0b500a30e4 satellite/audit: move audit metrics out of reporter
Since we are sharing the reporting logic between repair and audit. We
need to remove metric reporting logic in reporter.

Change-Id: Ib87295ab19079329e7438327d785a7f5c21d3b21
2021-09-16 17:58:56 +00:00
Malcolm Bouzi
252b78580d satellite/console: add status check to user authorization to ensure deleted accounts cannot perform actions
Change-Id: I2b37101335d154bf4bc93ec85c06ff62e555c9ec
2021-09-16 17:06:34 +00:00
Egon Elbre
09e1ff7fc4 web/satellite: fix promise usages
Change-Id: I6496ebdd57667e8222b2bfbe6174860997b6bb3f
2021-09-16 16:32:21 +00:00
Egon Elbre
6e660cecdd Jenkinsfile: test cross-compile and bump deps
Change-Id: I47091de2e80bd96fcada616f75e1db07a59fb0c1
2021-09-16 18:59:31 +03:00
Egon Elbre
b2d724962c cmd/storagenode-updater: avoid depending on the storagenode code
Change-Id: If81927aee43ccdd678a0e42f91632f85e9fdefab
2021-09-16 11:44:55 +00:00
Egon Elbre
6b3718f33e Makefile: skip darwin build for the time being
Mac builds need to be reworked to work with latest Go version.

Change-Id: Ifbe37cf8fb41c5e0d806c518a3be32e093b448b0
2021-09-16 12:53:41 +03:00
Yingrong Zhao
f0b73b4f45 mod: bump drpc, uplink and common
Updates:
- drpc to v0.0.25
- uplink to latest
- common to latest

This will fix the crash on UDP network temporary error for windows and a
rare deadlock during shutdown

Change-Id: I7e9c4ace22665c5471f19f9b8bf63aec4d68ceb3
2021-09-16 07:42:32 +00:00
Jeff Wendling
b160ec4c1b satellite/orders: bound RollupsWriteCache flushes
In the situation where the flushes take longer than the incoming
rate of writes, the RollupsWriteCache will take every connection
in the database pool and use them forever. Instead of doing that
and taking down satellite availability, bound the number of flush
operations that it will perform and drop incoming writes earlier
to keep memory usage constant.

Adds monitoring events for if any flushes or updates are lost.

Change-Id: I81b169b73501ee9b999f4b03d1e79645fc56f167
2021-09-15 19:14:39 +00:00
Vitalii Shpital
86845351c3 web/satellite: reworked upgrade to Pro account modal
Reworked the modal to correspond to latest designs.
Added prices and updated copies.

Change-Id: I6e59fdfe4222e02fcc35ac5d594bdc0bdf52a63c
2021-09-15 19:47:56 +03:00
littleskunk
3c3df0f362
storage/filestore: test interaction between VerifyDirReadable and VerifyDirWriteable 2021-09-15 18:57:56 +03:00
Jeremy Wharton
9dab10e1d9 web/satellite: Remove Advanced Report from GUI
Advanced Report is broken and it's causing support tickets.
The Advanced Report page must not be linked to until it is fixed.

Change-Id: I51d6afeee7584022ec920f311bb4c0d31174fd2c
2021-09-15 14:33:02 +00:00
Vitalii Shpital
2e6b4e90f8 web/satellite: add warning message to EncryptYourData component
Added warning copy to EncryptYourData component to let user know that this approach uses server-side encryption.

Change-Id: I864b8d5272e3d54a5ef0fafe464cd0e1ccaa9e83
2021-09-15 09:11:46 +00:00
Yingrong Zhao
e805dce704 satellite/satellitedb: drop audit reputation score related columns from
nodes table

Change-Id: I090dc69999531b8d33e979d42ae9aa4d04ff664e
2021-09-14 16:41:02 +00:00
crawter
07311cdd6f satellite/metainfo: begin move object added
First from two methods needed to perform server-side move. It gets
metadata key and nonce and all segments key and nonces and returns
all of that to uplink.

Change-Id: Ied2c79559e77d3f63091c4d61948f2d6a2147d67
2021-09-14 13:18:21 +00:00
Fadila Khadar
c00ecae75c satellite/gracefulexit: stop using gracefulexit_transfer_queue
Remove the logic associated to the old transfer queue.
A new transfer queue (gracefulexit_segment_transfer_queue) has been created for migration to segmentLoop.
Transfers from the old queue were not moved to the new queue.
Instead, it was still used for nodes which have initiated graceful exit before migration.
There is no such node left, so we can remove all this logic.
In a next step, we will drop the table.

Change-Id: I3aa9bc29b76065d34b57a73f6e9c9d0297587f54
2021-09-14 11:52:34 +00:00
crawter
e1a4195a35 satellite/metainfo/endpoint: MaxObjectKey length validation for BeginObject and CommitObject added
Change-Id: I6a766415a84b1e3c44ce65052ffa8ed4511d00dd
2021-09-13 15:07:16 +03:00
Egon Elbre
1aec831d98 satellite/audit,storage: increase sleep delay in TestMaxVerifyCount
Currently TextMaxVerifyCount flakes in some tests, try increasing the
sleep time to ensure that things are slow enough to trigger the error
condition.

Also pass ctx to all the funcs so we can handle sleep better.

Change-Id: I605b6ea8b14a0a66d81a605ce3251f57a1669c00
2021-09-10 15:30:37 +00:00
Vitalii Shpital
b472925c1e web/satellite: hot fix for Onb CLI flow's Create Bucket component's back button
There is a bug with back button of Create Bucket step of new onb CLI flow when proceeding further into flow and trying to come back.

Change-Id: I663a05021824164617ba933815973d8be856539f
2021-09-10 12:36:40 +00:00
Artur M. Wolff
7f595445ac satellite/metainfo: make subsequent auth validations not perform rate-limiting
Currently, requests that were successfully passed through the metainfo
endpoints rate-limiter might still fail in the middle of the
corresponding response. The problem is that we perform rate-limiting a
second time, which means other requests would influence whether the
current (already rate-checked) request will fail. This also has other
unintended effects, like responding with rpcstatus.PermissionDenied for
requests that were successfully rate-checked and did not lack
permissions but were rate-checked again in the middle of
(*Endpoint).BeginObject. This situation has been happening on the
gateway side and might affect other uplink clients. This change, where
appropriate, swaps subsequent validateAuth with validateAuthN that
performs rate-limiting once.

Change-Id: I6fc26dedb8c442dd20acaab5942f751279020b08
2021-09-10 12:07:06 +00:00
Vitalii Shpital
c06424cbf3 web/satellite: fixed inputs to have correct box-sizing
Fixed inputs to have correct box-sizing and fixed stylings for it.
Removed 2 unused components.
Fixed VInfo component's small styling issue.

Change-Id: I268b4b754be477d4e0d36b3946d62d65fd7d6cc8
2021-09-10 11:40:09 +00:00
Michał Niewrzał
84a383638b satellite/metabase: increase maximum batch size for loop
Current default is equal to hardcoded maximum of batch size
for loop listing. With this change we will bump maximum
batch size but current default won't change. We will be able to
increase it later without need for source code change.

Change-Id: I2744a87be28af4157f58ede73455682f61733bc1
2021-09-10 09:33:55 +00:00
Michał Niewrzał
495e530933 satellite/metainfo: drop metainfo.Service
Drop a service that in fact don't make sense and
its just a wrapper for direct DB methods.

Change-Id: I1cb76f3ecc2d8765964d919c88541179957645c1
2021-09-09 17:30:10 +02:00
Michał Niewrzał
67a33e4c06 satellite/metabase: drop unused IterateLoopStreams method
We were using this method for metainfo loop but now
segment loop is not using it so we can drop it.

Change-Id: I60c5b4f86a619259906d8c2ba76e665b8715be75
2021-09-09 14:12:44 +00:00
Michał Niewrzał
c258f4bbac private/testplanet: move Metabase outside Metainfo for satellite
At some point we moved metabase package outside Metainfo
but we didn't do that for satellite structure. This change
refactors only tests.
When uplink will be adjusted we can remove old entries in
Metainfo struct.

Change-Id: I2b66ed29f539b0ec0f490cad42c72840e0351bcb
2021-09-09 07:15:51 +00:00
Vitalii Shpital
9eaddee530 testsuite: tests for new CLI flow
Covered new CLI flow with tests

Change-Id: Ice5f4244383a376463d3aaae672dacd6ae1db14b
2021-09-08 20:09:23 +03:00
Vitalii Shpital
b86295c500 web/satellite: Final steps of onb CLI flow
Change-Id: I2c0a48b54347b81655b38f2cd2abd90cb8a84699
2021-09-08 20:09:23 +03:00
Vitalii Shpital
54c60fab47 web/satellite: onb cli flow's access grant flow
Added Generated AG, Install CLI and Import AG components

Change-Id: I5757ce4b82a17fdf807c346059ee26ff42be11ef
2021-09-08 20:09:23 +03:00
Vitalii Shpital
97a5c58ce4 web/satellite: cli flow's Skip Flow
Added API key, CLI setup and Generate AG components.
Also some common components were added to reuse in this flow.

Change-Id: Ic7b2e33b02b8d2c0e8f989a1ac59c8ebf05830d4
2021-09-08 20:09:23 +03:00
Vitalii Shpital
390c6b6fc6 web/satellite: cli flow's Encrypt Your Data component
Refactored Generate Passphrase screen to be common for onb and object flows.
Replaced initial Generate Passphrase screen to access grants wizard.
Moved download file functionality to utils.
Implemented Onboarding Encrypt your Data logic.

Change-Id: I46e780a5997c215d53ccd0254964b1c2b39c7c02
2021-09-08 20:09:23 +03:00
Vitalii Shpital
7a08c19c53 web/satellite: initial setup for onboarding CLI flow
Added components for the new flow.
Removed onboarding logic from regular access grant wizard.
Updated routes to include new flow.

Change-Id: I52ca6eb3da1e4e3a4b465ed58cd45398d10fd268
2021-09-08 20:09:23 +03:00
Igor
cd973fcb78
Revert "Makefile: disable building storagenode for darwin" (#4186)
This reverts commit d867a7007f.
2021-09-08 19:56:29 +03:00
Egon Elbre
d867a7007f Makefile: disable building storagenode for darwin
Also a few cleanups to ensure it wasm is built when needed.

Change-Id: I7612d969dddbfc1d8589cf00666c1c60d6312140
2021-09-08 13:46:58 +00:00
Jeremy Wharton
f55af3125c web/satellite: Replace checkbox reCAPTCHA with invisible reCAPTCHA
This change replaces the checkbox reCAPTCHA in the registration page
with an invisible reCAPTCHA.

Change-Id: I5b52e5fe35b551ba88af2404429b23bf38d4f9c1
2021-09-07 19:04:17 +00:00
Michał Niewrzał
03d638bbb3 cmd/uplink: add parallel upload
Use multipart upload to upload single object in parts in
parallel. Its using parallelism flag added earlier.

Change-Id: I45b531a5db43c86f0112a5e3bb4a83bc1d65650f
2021-09-07 15:12:09 +00:00
Egon Elbre
84539f5798 web/: disallow warnings and errors in tests
* all: fix missing Vue.use(Router)
* multinode: having same state and module names is not allowed
* multinode: simplify store definition
* storagenode: add or fix any annotations

Change-Id: If8a63ae46049fc77fc6cd7a7ad912d4ad74d23ac
2021-09-07 13:43:26 +00:00
Cameron Ayer
a7cda642a5 satellite/repair: add logging for irreparable segments in checker
If the checker sees an irreparable segment, log out some info
so we can see what the problem is

Change-Id: I76eda5270214205f4fefc646d6c391cc13ddcafd
2021-09-02 12:35:29 -04:00
Moby von Briesen
6d876acfbf satellite/console: UpdateProject changes
* Disabled updating project for paid tier users who have 0 storage or
bandwidth limits set (these users have been disabled and should not be
able to increase their limits through the UI)
* Better error handling
* Removed unnecessary type conversions

Change-Id: I1e07a1909a4ba877718aa944848f36382f7dbbe2
2021-09-01 20:30:26 +00:00
Egon Elbre
e5977ec849 web/: add custom linter for requiring @vue/component
Also ignore coverage folder for linting. I had to add a new
.stylelintignore file, because ignoreFiles property was not properly
working.

Change-Id: Iadd99b64eadd9c4103f750519263113ae8780ce1
2021-09-01 13:56:37 +00:00
Yaroslav Vorobiov
ee4361fe0d satellite/audit: fix segment stripes length calculation
GetRandomStripe function to randomly select a segment stripe to
audit was using `segment.EncryptedSize/segment.Redundancy.StripeSize()`.
Since integer divsion truncates it leads to skipping last stripe if
its size is less than stripe size. Use `Redundancy.StripeCount` to
get correct stripe count.

Change-Id: Ida09e035be30a21219ab3e1aedd66af8be707d1b
2021-09-01 13:25:20 +03:00