Commit Graph

8167 Commits

Author SHA1 Message Date
Clement Sam
a740f96f75 storagenode/pieces/lazyfilewalker: test zapwrapper
This add tests to the zapwrapper package and also adds a test
to verify the issue in https://github.com/storj/storj/issues/6006

Change-Id: Iec3f568e72683af71e1718017109a1ed52794b0b
2023-07-05 12:33:00 +00:00
Clement Sam
7ac2031cac web/multinode: fix wrong free disk space in allocation on dashboard
There are many case where the keywords `free` and `available`
are confused in their usage.

For most cases, `free` space is the amount of free space left
on the whole disk, and not just in allocation while
`available` space is the amount of free space left in the
allocated disk space.

What the user/sno wants to see is not the free space but the
available space. To the SNO, free space is the free space
left in the allocated disk space.

Because of this confusion, the multinode dashboard displays
the `free` disk space instead of the free space in the
allocated disk space https://github.com/storj/storj/issues/5248
While the storagenode dashboard shows the correct free space
in the allocation.

This change fixes the wrong free disk space. I also added a
few comments to make a distinction between the `free`
and `available` fields in the `DiskSpace*` structs.

Change-Id: I11b372ca53a5ac05dc3f79834c18f85ebec11855
2023-07-05 11:24:24 +00:00
Michal Niewrzal
21c1e66a85 satellite/overlay: refactor ReliabilityCache to keep more data
ReliabilityCache will be now using refactored overlay Reliable method.
This method will provide more info about nodes (e.g. country code) and
with this we are able to add two dedicated methods to classify pieces:
* OutOfPlacementPieces
* PiecesNodesLastNetsInOrder

With those new method we will fix issue where offline but reliable node
won't be checked for clumped pieces and off placement pieces.

https://github.com/storj/storj/issues/5998

Change-Id: I9ffbed9f07f4881c9db3bd0e5f0412f1a418dd82
2023-07-05 11:19:10 +02:00
Michal Niewrzal
f2cd7b0928 satellite/overlay: refactor Reliable to be used with repair checker
Currently we are using Reliable to get missing pieces for repair
checker. The issue is that now checker is looking at more things than
just missing pieces (clumped/off, placement pieces) and using only node
ID is not enough. We have issue where we are skipping offline nodes from
clumped and off placement pieces check.

Reliable was refactored to get data (e.g. country, lastNet) about all
reliable nodes. List is split into online and offline. This data will be
cached for quick use by repair checker. It will be also possible to
check nodes metadata like country code or lastNet.

We are also slowly moving `RepairExcludedCountryCodes` config from
overlay to repair which makes more sens for it.

This this first part of changes.

https://github.com/storj/storj/issues/5998

Change-Id: If534342488c0e440affc2894a8fbda6507b8959d
2023-07-05 10:56:31 +02:00
Márton Elek
500b6244f8
satellite/satellitedb: create table for node tags
Change-Id: I884bb740974e6b8241aa6b85faf266b85fe892d4
2023-07-05 09:38:53 +02:00
Clement Sam
1851d103f9 go.mod: bump storj.io/private
Updates https://github.com/storj/storj/issues/6006

Change-Id: I1f549d5642213e420f3e5d0df4ef972c77add713
2023-07-03 23:59:15 +00:00
paul cannon
032546219c satellite/admin: fix spelling of list-apikeys endpoint
Currently, any attempt to list the api keys associated with a project
from the admin UI results in a 404 NOT FOUND error.

This appears to be because there is no /api/projects/{project}/apiKeys
endpoint registered; it should have a lowercase k.

Change-Id: Ifbe4cd0f9ba12a6e37a0d9f64df91c264ced5558
2023-07-03 21:03:53 +00:00
Jeremy Wharton
1173877167 {satellite/console,web/satellite}: encode email in project invite URLs
This change properly encodes email addresses that are used as query
parameters in project invitation-related URLs.

Change-Id: Iaaf7b62b5ac3db3f0b0e000cc06fef8e315400a8
2023-07-03 18:07:19 +00:00
Vitalii
cb41c51692 web/satellite: abort request to get object count in 10 seconds
When entering passphrase to open bucket we make a ListObjectsV2Command request to figure out if there are objects encrypted with different passphrase.
If there are a lot of objects then this request takes forever to process.
By this change I added 10 seconds timeout for that request to not block user.

Issue:
https://github.com/storj/storj/issues/5954

Change-Id: I5243fba68d0b56dff2fb2b3a608a5e71860724c2
2023-07-03 17:33:11 +00:00
Márton Elek
d38b8fa2c4 satellite/nodeselection: use the same Node object from overlay and nodeselection
We use two different Node types in `overlay` and `uploadnodeselection` and converting back and forth.

Using the same object would allow us to use a unified node selection interface everywhere.

Change-Id: Ie71e29d60184ee0e5b4547eb54325f09c418f73c
2023-07-03 16:59:33 +00:00
Márton Elek
20a47034a5
cmd/tools: tag-signer utility to create signed node tags
Change-Id: I2983d688a109325a02fcd060ca1a2d4eb8e9e931
2023-07-03 18:10:08 +02:00
Márton Elek
01e33e7753 cmd/satellite: make satellite docker image compatible with storj-up
This patch makes satellite container images compatible with storj-up.

Which means that any official release can be easily tested locally.

It means that we need some binaries (like storj-up, dlv) and shall fragments part of the production image, but I think the risk is very low and the benefit is very high.

This is the first towards to unify all the images and make it possible to test/run the same components everywhere (storj-up/nigttly/prod).

https://github.com/storj/storj/issues/5946

Change-Id: Ib53b6213d94f93a793a841dedfe32cc59ef69b72
2023-07-03 15:02:27 +00:00
Márton Elek
8482b37c14
go.mod: bump to use latest common
Change-Id: Ie31e7779e86d13ca3c8acaacfe6ed1e23f2c9740
2023-07-03 13:32:35 +02:00
Wilfred Asomani
f131047f1a web/satellite: match projects table with the designs
This change updates the projects table on all projects dashboard to
more closely match the designs.

Change-Id: I547a83352fba8c3ad7958802db7b38b342b383e8
2023-06-30 12:05:47 +00:00
Vitalii
8d8f6734de satellite/{db, accounting}: added functionality to query settled bandwidth for given project
Added functionality to return only settled traffic from project_bandwidth_daily_rollups table for given month.
Updated {projectID}/usage-limits endpoint to return only settled bandwidth used.

This is a possible fix for this issue
https://github.com/storj/storj-private/issues/293

Change-Id: I12516dc898f449c2122e7442b8fbb88309a48ebe
2023-06-30 13:24:16 +03:00
Jeremy Wharton
c006126d54 web/satellite: add Back button to Bucket Details page
A button has been added to the Bucket Details page that returns the
user to the file browser.

Change-Id: Ic2868b1fc9e3b2b0e9785728dc7a116c828eced8
2023-06-30 08:53:36 +00:00
Wilfred Asomani
cd9feb6d09 web/satellite: split projects by whether user owns them
This change groups the list of projects in ProjectSelection.vue into
those owned by the current user and those that are not.

Issue: https://github.com/storj/storj/issues/5972

Change-Id: I20f4e68f84e4ece230afa53e7e7eed507da625af
2023-06-29 21:10:00 +00:00
Egon Elbre
2463b881c6 storagenode/piecestore: fix TestUpload
The test needs to wait for the upload information to be saved to the
database.

Fixes https://github.com/storj/storj/issues/6008

Change-Id: I1f258c923a4b33cbc571f97bad046cec70642a0b
2023-06-29 22:14:22 +03:00
Jeremy Wharton
0ca7d19fc7 web/satellite: show cursor as pointer when selecting session timeout
The cursor now appears as a pointer when interacting with the session
timeout modal's dropdown menu.

Change-Id: I309463b5f3af2df198a8bba5a9122e8a8f39de89
2023-06-29 14:42:21 +00:00
Jeremy Wharton
bd3aaf4f34 web/satellite: adjust mobile header padding and logo
The mobile header's horizontal padding has been adjusted to match the
padding of the page content. Additionally, the size of the logo in the
mobile header has been decreased.

Change-Id: I7559813455fb186f1402783db6022ebdaa59c7ae
2023-06-29 12:18:04 +00:00
Jeremy Wharton
f1fab9edfb web/satellite: update styling of session timeout selection modal
The styling of the session timeout selection modal has been altered to
more closely align with our designs.

Change-Id: Icaf2d16170908af4962075732c6df0f08d7dba2d
2023-06-29 11:43:56 +00:00
Michal Niewrzal
96d3c41c14 satellite/metabase: convert bucket name to bytes for queries
In case some invalid characters in bucket name we need to cast
bucket name to byte array for query argument. This change is
doing this for some missed cases.

Change-Id: I47d0d8e3c85a69bdf63de1137adcd533dcfe50a8
2023-06-29 10:43:35 +00:00
Jeremy Wharton
faf5b960ff satellite/console/dbcleanup: make chore clean up webapp sessions
The console DB cleanup chore has been extended to remove expired webapp
session records.

Resolves #5893

Change-Id: I455b4933552cfde86817a2ef8f9879dd7b0a121d
2023-06-29 09:53:18 +00:00
Vitalii
0d0e8cc8cf web/satellite: fix for too many objects banner in object browser
Fixed 'Too may objects' objects banner being displayed in a wrong place.
I made it simpler and more correct because it's impossible to know the exact count of objects for each path/prefix because s3 client can list only 1000 objects (for now) and our API at the same time calculates object count for the whole bucket (not for particular passphrase).
Added message that user can list all objects using Uplink CLI.

Also removed unused legacy css file.

Issue:
https://github.com/storj/storj/issues/5955

Change-Id: I4b3cff47763ebdb631119b690de876ecf6a22e9d
2023-06-29 11:05:35 +03:00
Jeremy Wharton
706cd0b9fb satellite/console: allow for adding unregistered project members
This change allows members without an account to be invited to a
project. The link in the invitation email will redirect these users to
the registration page containing custom text describing the invitation.

Resolves #5353

Change-Id: I6cba91e57c551ca13c7a9ae49150fc1d374cd6b5
2023-06-28 22:03:03 +00:00
dlamarmorgan
a010459520 satellite/payments/stripe/{client,server}: add set invoice status cmd
Change-Id: I6d9b4fe2ed4b17d63d858f7ceefd7c14adb062bb
2023-06-28 21:04:34 +00:00
Wilfred Asomani
14beb9ad85 web/satellite: update button hover state to match the figma
This change updates the buttons same as the "New Folder" button to
have hover state consistent with the design in the figma.

It also fixes an issue where table view will be shown by default even
if the user has less than 8 projects.

Issue: https://github.com/storj/storj/issues/5971

Change-Id: Ic8b99496e1990550a9ea1550c3c2bd80bf997aa9
2023-06-28 20:13:30 +00:00
Vitalii
6f078acb8d web/satellite: add gallery view caching
Add caching for gallery view previews and map.

Issue:
https://github.com/storj/storj/issues/5969

Change-Id: I6c9755aec6e1d4143005835adad212cafd46f649
2023-06-28 17:29:05 +00:00
Michal Niewrzal
578724e9b1 satellite/repair/repairer: use KnownReliable to check segment pieces
At the moment segment repairer is skipping offline nodes in checks like
clumped pieces and off placement pieces. This change is fixing this
problem using new version of KnownReliable method. New method is
returning both online and offline nodes. Provided data can be used to
find clumped and off placement pieces.

We are not using DownloadSelectionCache anymore with segment repairer.

https://github.com/storj/storj/issues/5998

Change-Id: I236a1926e21f13df4cdedc91130352d37ff97e18
2023-06-28 16:53:51 +00:00
Jeremy Wharton
ec780003f0 web/satellite: add responsiveness to upgrade notification
The upgrade notification has been updated to adapt to mobile screens
accordance with our designs.

Additionally, an issue where the notification would display "0B free
included" when displayed in the All Projects Dashboard has been fixed.

Change-Id: Ic13b9426ab5d6529c9d7b2ad8446a17da74905b1
2023-06-28 09:34:53 -05:00
Michal Niewrzal
e129841130 satellite/metabase: remove AOST from deleteInactiveObjectsAndSegments
By mistake AOST was added to query in deleteInactiveObjectsAndSegments
in DeleteZombieObjects. Delete statement is not supporting it.
Unfortunately unit tests didn't cover this case. This change removes
AOST from mentioned method and it adding AOST cases to unit tests.

Change-Id: Ib7f65134290df08c490c96b7e367d12f497a3373
2023-06-28 13:24:14 +00:00
Márton Elek
683f0c493f earthfile: build satellite-admin for nightly
Change-Id: Iafe7fe1867178517f359ca61d2178a68d1430894
2023-06-28 12:06:41 +00:00
Sean Harvey
b1523f82c8
satellite/admin: add owner full name on /api/apikeys/{apikey} endpoint
Updates storj/gateway-mt#321

Change-Id: I6759ec5dbba49261bb183e42d8cb333c326cb9e8
2023-06-28 16:01:15 +12:00
paul cannon
d06e4589ae bump storj.io/private
Change-Id: I0b773fe140bad485fc323701928eaf246638032c
2023-06-27 13:29:54 -05:00
Jeremy Wharton
7d44f99ce6 web/satellite: add My Projects button to All Projects Dashboard
This change adds a button to the header of the All Projects Dashboard
that navigates the user to the My Projects page.

Resolves #5996

Change-Id: Ie467e22c9039cf30fda1b8d815c1d6269f5ddf4f
2023-06-27 16:29:34 +00:00
Wilfred Asomani
30d0094c43 satellite/console: prevent unauthorized project mutation
This change further restricts projects members from modifying project
details by restricting the project edit graphql mutation; making it
check if the user performing the operation is the owner of the project.

Change-Id: Iaf10d16269ddc29437d3d5629db06e20cea3004e
2023-06-27 15:57:09 +00:00
Jeremy Wharton
bcce6023c3 web/satellite: show custom invitation text in registration page
This change makes the registration page to display custom text for
users that have been invited to a project.

References #5353

Change-Id: Ib20760f79ef29327b66316817010ca1dc00ff2ce
2023-06-27 15:15:43 +00:00
Vitalii
9374edfac9 web/satellite: added dropzone styling for object browser
Added greyed out dropzone styling for object browser.

Issue:
https://github.com/storj/storj/issues/5970

Change-Id: I9770a9d3fb90f6aaf659885f3c3cafed7af89e1d
2023-06-27 17:17:15 +03:00
Michal Niewrzal
98f4f249b2 satellite/overlay: refactor KnownReliable to be used with repairer
Currently we are using KnownUnreliableOrOffline to get missing pieces
for segment repairer (GetMissingPieces). The issue is that now repairer
is looking at more things than just missing pieces (clumped/off
placement pieces).

KnownReliable was refactored to get data (e.g. country, lastNet) about
all reliable nodes from provided list. List is split into online and
offline. This way we will be able to use results from this method to all
checks: missing pieces, clumped pieces, out of placement pieces.

This this first part of changes to handle different kind of pieces in
segment repairer.

https://github.com/storj/storj/issues/5998

Change-Id: I6cbaf59cff9d6c4346ace75bb814ccd985c0e43e
2023-06-27 13:27:23 +02:00
Egon Elbre
049953a7ce go.mod: bump storj.io/uplink
Change-Id: Ib535cb50195c5a0e6197e83e2e05865ba95bcaf8
2023-06-26 13:51:13 +00:00
Wilfred Asomani
79eb71841d satellite/{web,analytics}: add row actions to project members
This change adds row actions (delete,reinvite,copy) to the project
members table. It also adds analytics events for the actions.

Issue: #5762
Also fixes: #5941

Change-Id: I7fb7f88c7bd5ac2ce3e3d00530af4708ff220bd7
2023-06-26 12:45:02 +00:00
andriikotko
0421ef2fa1
docs/testplan: add testplan for inviting project members (#5811) 2023-06-26 13:20:16 +02:00
Jeremy Wharton
99f4a34a1d web/satellite: add button for resending expired project invitations
A button has been added to the Team page for resending expired project
member invitations. It appears when one or more of such invitations
have been selected.

Additionally, styling for certain search fields and the Team page's
header has been updated to align more closely with our designs.

Resolves #5752

Change-Id: I623fed5f50e60beca2f82136f8771dde5aa684f4
2023-06-23 23:46:15 -05:00
Jeremy Wharton
2ae75bcf4e satellite/console: prevent invite replies from deleting expired invites
This change prevents project member invitation responses from deleting
expired project invitations. Previously, accepting or declining an
expired invitation cause it to be destroyed.

References #5752

Change-Id: Id3917fb825bffc3e8a262d5b541b907678db1809
2023-06-23 19:05:52 -05:00
Jeremy Wharton
80c5a628cb satellite/console/dbcleanup: remove project invite cleanup
This reverts 9c75316 which allowed the satellite console DB cleanup
chore to delete expired project member invitations. We now want such
invitations to be accessible indefinitely.

References #5752

Change-Id: I489a7e19df825dd14376d3d260b70b3eef643e03
2023-06-23 21:15:36 +00:00
Jeremy Wharton
22f8b029b9 satellite/console: fix transaction error when inviting project members
The SQL transaction that inserted project invitations relied on the
error result of one of its statements in order to determine whether an
invitation should be updated. This was inappropriate since any errors
returned from a transaction statement should end the transaction
immediately. This change resolves that issue.

Change-Id: I354e430df293054d8583fb4faa5dc1bcf9053836
2023-06-23 20:17:37 +00:00
Wilfred Asomani
1b912ec167 satellite/{web,analytics}: add segment events for passphrase modals
This change sends new passphrase created event for when passphrase is
created with the method by which it was; entered/generated

Issue: #5918

Change-Id: Ib485b6ff7a968d4c84bf124e14c14c91478f0dfb
2023-06-23 18:30:46 +00:00
Moby von Briesen
7530a3a83d satellite/console: add CORS middleware to satellite UI and API
Add some basic handling to set cross-origin resource sharing headers for
the satellite UI app handler as well as API endpoints used by the
satellite UI.

This change also removes some no-longer-necessary CORS functionality on
the account registration endpoint. Previously, these CORS headers were
used to enable account registration cross-origin from www.storj.io.
However, we have since removed the ability to sign up via www.storj.io.

With these changes, browsers will prevent any requests to the affected
endpoints, unless the browser is making the request from the same host
as the satellite.

see https://github.com/storj/storj-private/issues/242

Change-Id: Ifd98be4a142a2e61e26392d97242d911e051fe8a
2023-06-23 17:46:44 +00:00
Wilfred Asomani
361f9fdba5 web/satellite: prevent unauthorized access to project settings page
This change further restricts projects members from accessing the
projects settings page by navigating to (all) projects dashboard when
/edit-project-details is visited or project is switched.
It also applies a white background to the project ownership tag to
improve contrast and visibility.

Change-Id: Ib855c4e3aa4be7ec9ec1e9b312041118442358ad
2023-06-23 17:02:07 +00:00
Wilfred Asomani
9d4787f5e7 web/satellite: use correct color for projects table icons
This change uses the correct color corresponding to the role of a user
on a project.

Change-Id: Ibd8f9ccae4486a8039f77bae5c2533b060e73be9
2023-06-23 15:40:21 +00:00