Commit Graph

7940 Commits

Author SHA1 Message Date
Clement Sam
cf7ce81d09 cmd/storagenode: refactor lazyfilewalker commands to satisfy the execwrapper.Command interface
Follow-up change for https://review.dev.storj.io/c/storj/storj/+/10335

Updates https://github.com/storj/storj/issues/5349

Change-Id: Iadf55bae84ebc0803a0766830e596c396dfb332b
2023-05-08 15:09:53 +00:00
Márton Elek
56dbe7738d cmd/storagenode: use the right subcommand for restarting windows services
Commit 3cf89633e9 is changed how the cobra subcommands are created for storagenode (with prefering local variables instead of package level variables).

However, there is a bug which makes it impossible to restart Storagenode services on Windows: the refactored code creates the rootCmd/runCmd twice: therefore the ctx of the running process is not exactly the same as the ctx which supposed to be stopped / cancelled.

This patch fixes this problem with re-using exising, initialized command instead of creating a new one for cancellation.

Fixes: https://github.com/storj/storj/issues/5845

Change-Id: Ib8a4b80d4574e448f65c8558e927c0908c9c5eed
2023-05-08 14:09:37 +00:00
Clement Sam
5977357a60 web/storagenode: hide payout rates for historical payout data
This is a temporal fix to buy us some time while we fix the issues.

We may revert later or maybe override with a later commit.

Updates https://github.com/storj/storj-private/issues/245

Change-Id: I27b2b6811d24d5948161553bf889fe45af013925
2023-05-08 12:53:37 +00:00
Michal Niewrzal
36e046375c satellite/repair/checker: remove segments loop parts
We are switching completely to ranged loop.

https://github.com/storj/storj/issues/5368

Change-Id: I8583549973cd36aa0e0c482c20d7a75cb7568ab3
2023-05-08 12:19:13 +00:00
Cameron
e8e6dd056a web/satellite: remove project member modal
Create modal for removing project members

issue: https://github.com/storj/storj/issues/5751

Change-Id: I621b8e1de54270faebb7577b8431903c62a6e45c
2023-05-08 11:36:56 +00:00
Wilfred Asomani
fb1a0cc784 web/satellite: redesign and allow enter passphrase skip
This change allows users to skip the enter passphrase modal and opt in
for that choice to be remembered. The enter passphrase modal is also
redesigned.

Issue: https://github.com/storj/storj/issues/5818
Issue: https://github.com/storj/storj/issues/5616

Change-Id: Ie1cf6602682af3e9625656455fa3c096a24b95d3
2023-05-08 10:52:49 +00:00
Clement Sam
291e639ac2 storagenode/pieces/lazyfilewalker: add execwrapper package
The execwrapper package wraps the exec.Cmd and has a Command
interface that mimics the behaviour of the exec.Cmd.
This is useful for testing the lazyfilewalker subprocesses
by stubbing instead of spawning a real subprocess.

Updates https://github.com/storj/storj/issues/5349

Change-Id: I14084139c76a531f2b6d7163f9aa35c3f5e192d7
2023-05-06 02:02:23 +00:00
Clement Sam
ec8bfe6b94 storagenode/pieces: capture logger time Key in zapwrapper
Updates https://github.com/storj/storj/issues/5349

Change-Id: I426f38c0ae0f93d498317e3f66ba4f5724620758
2023-05-06 02:02:23 +00:00
Jeff Wendling
6ac72e42cd all: bump storj.io/drpc
this includes a fix to cancellation when soft cancel
is enabled.

Change-Id: I1b9ab071f623153a2bd715888f97ca20277a76d7
2023-05-06 01:02:50 +00:00
Moby von Briesen
432b12d1c4 satellite/satellitedb: make value_attributions.partner_id nullable
There is still a reference to partner_id in a query, which we cannot
move until dropping the "not null" constraint for it. This change adds
that migration.

Related to https://github.com/storj/storj/issues/5432

Change-Id: I98802a6e1bd59f3d9214de3db6688d9daf664a70
2023-05-05 23:42:50 +00:00
Wilfred Asomani
e3573f44ea web/satellite: update project member invite modal
The project member invite modal has been updated to match the new figma
design.

Issue: https://github.com/storj/storj/issues/5744

Change-Id: I9ed7ad4ede0fb6f7f42f56b2d8386e12a5a5b6c1
2023-05-05 19:46:28 +00:00
Jeremy Wharton
f61230a670 satellite/console/dbcleanup: create console DB cleanup chore
A chore responsible for purging data from the console DB has been
implemented. Currently, it removes old records for unverified user
accounts. We plan to extend this functionality to include expired
project member invitations in the future.

Resolves #5790
References #5816

Change-Id: I1f3ef62fc96c10a42a383804b3b1d2846d7813f7
2023-05-05 19:11:53 +00:00
Cameron
2bb636684e web/satellite: update create bucket modal
issue: https://github.com/storj/storj/issues/5817

Change-Id: Ia86e0bad8de180fb4e6dde3ed194c9ce6f4f12f1
2023-05-05 17:50:58 +00:00
Wilfred Asomani
4ab433ba89 satellite/{web,console}: make duplicate member error readable
This change makes the error thrown when adding an existing member to a
project readable.

Issue: https://github.com/storj/storj/issues/5840

Change-Id: I4269495f9b7b09c77fbf1af1fc605e5c95bd7cbf
2023-05-05 16:46:12 +00:00
Egon Elbre
8b82dba602 storagenode/blobstore/filestore: add tracking of blobs
We've had issues with forgetting to close readers and writers.
Add leak tracking to find those pesky issues.

Change-Id: If6b0ad6e9958318a7e0affee9c6d0a1ece412b6d
2023-05-05 15:40:15 +03:00
Clement Sam
569c639cd5 web/storagenode: hide payout rates when no specific satellite is selected
We should not display payout rates when "All Satellites" option is
selected since some of the satellites may have different payout
rates.

Updates https://github.com/storj/storj-private/issues/245

Change-Id: I66cadf85aa8b57681ffbeac75c4ad97f8b057676
2023-05-05 01:00:41 +00:00
Wilfred Asomani
bf05040dd6 satellite/{console,db}: allow passphrasepromt toggling
This change adds the user's passphrase prompt setting to the
/account/settings endpoints.

Issue: https://github.com/storj/storj/issues/5616

Change-Id: I48d470d49e82096fd090b74da323b279e342546e
2023-05-04 19:26:54 +00:00
Paul Willoughby
d8081169cd satellite/metainfo: increase default MaxEncryptedObjectKeyLength
Allow a longer encrypted key length to reduce 'key length is too big'
errors in gateway-mt.  Gateway is enforcing an unencrypted key length
of 1024 bytes but when encrypted some keys are exceeding the current
limit.

updates https://github.com/storj/gateway-mt/issues/335

Change-Id: I861a2313e558c9f7d39569d21c7a3d429c83575c
2023-05-04 12:37:13 -06:00
Vitalii
a8ba7aa263 web/satellite: small styling updates for access grant flow
Fixed Copy and Download buttons.
Fixed wording styling.
Also fixed VButton's hover state if there is an icon.

Issue:
https://github.com/storj/storj/issues/5774

Change-Id: I9fbac12fe433817f4b5fc1d885f50e6f27e7f6a0
2023-05-04 17:33:06 +00:00
Vitalii
73898e0d86 web/satellite: make graph line more smooth
Made lines to be more smooth and also updates points styling

Issue:
https://github.com/storj/storj/issues/5826

Change-Id: I34229e772ceee5e48330f5662a584cd9dcd8c63d
2023-05-04 15:20:31 +00:00
Egon Elbre
511d74a76d satellite/metainfo: hardcode the prefix
This avoids a potential bug with regards to not returning the prefix
that the uplink expects.

Change-Id: I64683db57c5b22989db91b4f8e08530bf3b64ff1
2023-05-04 17:06:51 +03:00
Clement Sam
8a206f0aee web/multinode: fix wrong undistributed payout amount
Resolves https://github.com/storj/storj/issues/5849

Change-Id: I074f6ca9f9ac1e8ffa62649188a6059d9bcd4c25
2023-05-04 10:36:10 +00:00
Clement Sam
162af1e836 web/multinode: fix bandwidth graph disappearing on state change
Resolves https://github.com/storj/storj/issues/5847

Change-Id: I6731e7a39cd3ffedaff76b307c141a9e94d9b71f
2023-05-04 10:36:10 +00:00
Clement Sam
dab4288899 web/multinode: fix used space graph showing 0
Resolves https://github.com/storj/storj/issues/5566

Change-Id: Ib9c81fa2c92ed71ccf07e7a0c9b6f61c46294f09
2023-05-04 08:46:47 +00:00
Moby von Briesen
a229feac1e satellite/console: Pass pricing packages enabled config to frontend
Ensure that the value of "pricing packages enabled" flag on frontend is
the same as what is configured on the backend.

Change-Id: Id78771800a4973ebd3ad4e22f1953f6f71c75dd4
2023-05-03 18:13:38 -04:00
Wilfred Asomani
034431e4a8 web/satellite: update session timeout modals
This change opens up the "set session timeout" modal for users to set a
custom timeout duration if they haven't already.
It also fixes a few UI/UX issues on the modal including toggling on the
dashboard.

Issue: https://github.com/storj/storj/issues/5770

Change-Id: I0e71e191049b242e638ca36214d6dd33f78ae5fe
2023-05-03 13:59:50 +00:00
Clement Sam
e0542c2d24 storagenode: run garbage collection filewalker as a low I/O subprocess
Updates https://github.com/storj/storj/issues/5349

Change-Id: I7d810d737b17f0b74943765f7f7cc30b9fcf1425
2023-05-02 19:43:38 +00:00
Lizzy Thomson
25f2305e00 web/satellite: removed google fonts from satellite emails
update fonts in satellite emails by removing google fonts

Issue - https://github.com/storj/storj/issues/5690

Change-Id: I9243c172e52d780800300dca2ec78b5665979eda
2023-05-02 16:10:34 +00:00
Wilfred Asomani
bf630b7efa web/satellite: update team page design
This change the projects members (team) page to match the figma.

Issue: https://github.com/storj/storj/issues/5742

Change-Id: I5d4ef4478612a5c397b795c9b0b9b364fd16e4a4
2023-05-02 15:34:02 +00:00
Egon Elbre
ee28103b29 satellite/metabase: fix error message
The "object not found" included an additional prefix "metabase:", which
broke uplink error message detection.

Ideally, changing an internal error message shouldn't break the uplink.

Change-Id: I5ce7789cc11742d3435af1ec555bc96927f1bedc
2023-05-02 13:53:57 +00:00
Moby von Briesen
90692362c6 web/satellite: fix link
Link to end-to-end encryption page on docs.storj.io is incorrect. This
change fixes it.

Change-Id: I61c942e2d2b3bcd9ad0d51451027207a9eeb36f0
2023-04-28 18:51:22 -04:00
Jeff Wendling
80b3edf1d1 storagenode/piecestore: respect maximum chunk size requests
See https://review.dev.storj.io/c/storj/common/+/10297 for
more details.

Change-Id: Id5d19f029ae872780a554874592679191c1b5b2f
2023-04-28 16:31:41 -04:00
Vitalii
3f1166b5aa web/satellite: added update your session timeout banner
Added new banner to inform user that they can update their session timeout now.

Issue:
https://github.com/storj/storj/issues/5772

Change-Id: Icdf2164b80b12954d004537a4f31d30ef6bb12b8
2023-04-28 11:17:06 +00:00
Michal Niewrzal
98562d06c8 satellite/gc/bloomfilter: add sync observer
Current observer used with ranged loop is using massive amount of
memory because each range is generating separate set of bloom filters.
Each bloom filter can be up to 2MB of memory. That's a lot.

This change is initial change to reduce used memory by sharing bloom
filters between ranges and just synchronize access to them. This
implementation is rather simple and even naive but maybe it will be
enough without doing something more complex.

https://github.com/storj/storj/issues/5803

Change-Id: Ie62d19276aa9023076b1c97f712b788bce963cbe
2023-04-28 07:40:56 +00:00
Vitalii
30dee52256 web/satellite: rework delete access grant modal
Reworked delete access grant modal to be more simple.

Issue:
https://github.com/storj/storj/issues/5775

Change-Id: I415e75458736522f17914f252be531db5a2313f7
2023-04-27 21:26:38 +00:00
Vitalii
978b714128 web/satellite: rework open bucket modal styling
Reworked open bucket modal styling to correspond to create access grant flow

Issue:
https://github.com/storj/storj/issues/5819

Change-Id: I618b50472e45413a63a531ad959d9af69511d464
2023-04-27 16:10:27 +00:00
dlamarmorgan
b89e0e0272 payments/stripe: Remove apply coupon from generate invoice command
Remove the command to apply free tier coupons from the generate invoices
command. Applying free tier coupons should instead be done outside the
normal time window for invoice generation to save time during the
invoicing cycle.

Change-Id: If8fecc558411d5a6fff9d5689143d72f3b709e55
2023-04-27 08:24:12 -07:00
Michal Niewrzal
3cd79d987d satellite/gc/bloomfilter: extract BF upload logic
This is refactor/cleanup change before I will start working on adding
separate GC observer with optimized memory consumption.

https://github.com/storj/storj/issues/5803

Change-Id: I854cb3797802a32942c25f2765dbb72be88bacbd
2023-04-27 11:15:27 +02:00
Vitalii
48f920809f web/satellite: rework styling of manage project passphrase modals
Reworked styling of manage project passphrase modals to crrespond to create access grant flow.

Issue:
https://github.com/storj/storj/issues/5815

Change-Id: Iab02d2d94239d9e8c7f85a498fb3a3d83ece34e0
2023-04-26 20:39:11 +00:00
Vitalii
2e2978d647 web/satellite: rework styling of create project passphrase flow
Reworked styling of create project passphrase flow to be consistent with create access grant flow.

Issue:
https://github.com/storj/storj/issues/5815

Change-Id: I54db5b3c37a8cf96d939bfcb99cf0896af25c4cc
2023-04-26 20:03:49 +00:00
Wilfred Asomani
4929ae3c08 web/satellite: improve table pagination
This change modifies the existing table pagination to match the figma.

Issue: https://github.com/storj/storj/issues/5765

Change-Id: Ic9c57072509705c3cb946b1b6764fd09ba616ceb
2023-04-26 19:07:17 +00:00
Jeremy Wharton
7a17dc5e07 satellite/{console,satellitedb}: add methods for project invite table
This change immplements methods for interacting with the project member
invitations table.

Resolves #5766

Change-Id: I0090c50f9fde5bcdae4ebdaa72cdcaa84d341b54
2023-04-26 11:26:12 -05:00
Jeremy Wharton
defb9eae82 satellite/satellitedb: add table for project invitations
A table for storing pending project member invitations has been added.
This table is required to satisfy our new project invitation UX revamp.

References #5766

Change-Id: I6f948de66ed5b4dc81532564958ff7f48533cad2
2023-04-26 11:26:12 -05:00
Wilfred Asomani
774ac50528 web/satellite: migrate vue filters
This change removes vue filters declared in main.ts and replaces them
with exported functions in utils/strings.ts. This is toward migration
to Vue 3; it has no support for filters.

Issue: https://github.com/storj/storj/issues/5063

Change-Id: I0f0e73f5a4ba3300cd8804b083b18c05e614fa8c
2023-04-26 13:01:21 +00:00
Wilfred Asomani
b4ac006462 web/satellite: sync timeout modal with user session timeout
This change updates the session timeout modal to be triggered based on
the user's selected session timeout instead of the default config value
that is if the user has set a custom session timeout duration.

Change-Id: I1d3ace8542efe04c8ef4d729da6b6bec376d5ea4
2023-04-26 10:07:35 +00:00
dlamarmorgan
efcae857ba satellite/main,stripe/{client,service}: stripe balance invoice item cmd
Add a new billing command that will convert stripe customer balances into invoice items so that the charges can be processed normally by the invoicing workflow.

Change-Id: Iaa8350e7aca80a0f14e94eb8ef8b7d6ce0b5b3b8
2023-04-25 21:08:23 +00:00
dlamarmorgan
5fc493b276 metainfo: per-project and user-agent eventkit-based usage tracking
Add eventkit events for each satellite RPC type, tagged with user agent and public project id.

Change-Id: I838401683298d4594495de18563c55031e82f881
2023-04-25 15:13:43 +00:00
Márton Elek
1588bf9606 go.mod: bump monkit
Change-Id: Icc286378a8b01518510a4b4dd4d213d0c5d3b5b6
2023-04-25 14:26:11 +00:00
Vitalii
4d998970d4 satellite/admin: rework update user limits functionality
Fixed nil pointer dereference panic.
Updated naming conventions so that PUT request and GET response bodies are the same (bandwidth, storage and segment).
Allowed usage of notations like 150GB, 2TB for storage and bandwidth limits.
Updated tests.

Issue:
https://github.com/storj/storj/issues/5674

Change-Id: I7ac27c00721a9b4bf507afa34cb05c4475a809ad
2023-04-25 13:38:59 +00:00
Wilfred Asomani
260b71e70c satellite/{console,accountfreeze}: test freeze effects
This change adds more tests to the autofreeze chore and the freeze
service according to the testplan linked in the issue below.

Issue: https://github.com/storj/storj/issues/5738

Change-Id: Ib2afaa283961b2e7ef6fb6e5613ee083ac7d79eb
2023-04-25 13:04:37 +00:00