Commit Graph

8546 Commits

Author SHA1 Message Date
Andrii Kotko
a331610a8a
release v1.89.5 2023-10-05 15:45:40 +03:00
Egon Elbre
98e778e26c go.mod: bump uplink
This brings in 32bit arm bug fix.

Change-Id: I61b72abf2366643f58b1e670d2e02f4072aee0b5
2023-10-05 15:44:16 +03:00
Ethan Adams
20232e435a satellite/email: Add support for unauthenticated, cleartext SMTP connections
Change-Id: I11b4852122764c1ede188ca40d5edb14f2c4ee72
2023-10-05 10:33:11 +03:00
paul cannon
a9265ae543 satellite/{admin,console,satellitedb}: fix geofence removal
deleteGeofenceForProject wasn't able to work correctly, because
Console().Projects().Update() declines to update default_placement when
the input value is 0.

This introduces a Console().Projects().UpdateDefaultPlacement() method,
congruent to the method of the same name on Console().Users().
deleteGeofenceForProject now uses this new method, so that specifying a
new placement of 0 will work correctly.

Change-Id: I4589b36707f7e4f1cfdc66543520b0d4205c1a84
2023-10-05 10:33:11 +03:00
Sean Harvey
e98b94f114 satellite/satellitedb: fix DefaultPlacement overwritten on user
this fixes cases where it's possible to update a user and the
DefaultPlacement field gets overwritten to the zero value.

it also adds UpdateDefaultPlacement which can be used to set
DefaultPlacement directly. This is needed for the geofencing
endpoints in satellite admin to set the DefaultPlacement back
to zero to delete geofencing for a user.

Change-Id: If2c798dabfa6773ed6023fb8257bf00ec7bc2e68
2023-10-04 12:05:04 +03:00
Wilfred Asomani
00cd29cc6d satellite/{web,console}: return empty payments list on no wallet error
This change modifies wallet payments endpoints to return empty lists
instead of returning a 404 error if a wallet is not found for a user.

Change-Id: Ic765fecbc8183d14f179ce1d510ae512d8e0c4a9
2023-10-04 12:05:04 +03:00
Sean Harvey
68fe9a0a52 satellite/metainfo: fix inconsistency with auth check error
This fixes an inconsistency with error returned on copy and move
endpoints to match other endpoints. validateAuth() is already
wrapping the RPC status around the error, so this shouldn't be
doing it again.

This also ensures that rate limit errors for FinishCopyObject and
FinishMoveObject are correctly returned as rpcstatus.ResourceExhausted
so uplink can correctly map these to uplink.ErrTooManyRequests.

Change-Id: I6bf6185b456d6774b99d56cf3d7d8f8aa2afa0e8
2023-10-04 12:05:04 +03:00
Jeremy Wharton
b9c84d637b cmd/satellite: copy Vuetify distribution folder to satellite
This change ensures that files generated by building the Vuetify
project are copied to the satellite container.

References #6251

Change-Id: If56fe754d51f1487a3b3c2cf98c40e3010539121
2023-09-29 19:24:51 +03:00
Wilfred Asomani
822c59638e web/satellite: patch listing token history issue
This change patches an issue where a user who has not claimed a wallet
would see the error "Can not list token payment history" on the all
projects dashboard.

Issue: https://github.com/storj/storj/issues/6358

Change-Id: I0783fae2c4441be4495b9c8bd82cf6dbe6eea557
2023-09-29 19:24:51 +03:00
Ivan Fraixedes
5c5306cef2 satellite/admin/back-office/ui: Delete yarn.lock file
The yarn.lock file came from the former repository where the back office
UI was developed.

It seems that our build process complains about some dirty state related
to this file. Because we don't use Yarn, we delete the file, hoping to
resolve the build issues.

Change-Id: I5febd8292657289d0fc67e08151c6c8b5ac8b5dc
2023-09-29 19:24:51 +03:00
Moby von Briesen
c14e4b1eb4 satellite/console: update CSP
Include *.storjsatelliteshare.io in the `connect-src` portion of the CSP
for the satellite UI.

Change-Id: Ic8c3d0cf892a3275866634cae3e9260d925e1c3e
2023-09-27 15:49:48 +00:00
Márton Elek
58b98bc335 satellite/repair: repair is configurable to work only on included/excluded placements
This patch finishes the placement aware repair.

We already introduced the parameters to select only the jobs for specific placements, the remaining part is just to configure the exclude/include rules. + a full e2e unit test.

Change-Id: I223ba84e8ab7481a53e5a444596c7a5ae51573c5
2023-09-27 14:54:06 +00:00
Vitalii
63645205c0 web/satellite: fix object browser pagination for the folders
Fix pagination for the folders which are not on the first page.
Also, fixed object count calculation inside folders.

Issue:
https://github.com/storj/customer-issues/issues/1055

Change-Id: I1d0fbb8856f13be6fb20698315a7e4d20b4affd9
2023-09-27 14:12:02 +00:00
Michal Niewrzal
e1215d5da8 satellite/overlay: add AOST to GetParticipatingNodes method
This method is sometimes ends with transaction error. Most probably
because it's trying to do full table scan on nodes table which is
heavily used. Adding AOST should help with DB contention.

Change-Id: Ibd4358d28dc26922b60c6b30862f20e7c0662cd1
2023-09-27 12:00:10 +00:00
Ivan Fraixedes
2a8e5aecfd
satellite/admin/back-office/ui: Don't ignore package-lock.json
When the new back office UI sources where copied from former repository
I didn't realize that the .gitignore had the package-lock.json file.

This commit remove the package-lock.json file, so it can be tracked, in
order to have reproducible builds.

The lack of the file caused the build to fail due to `npm ci` requires
it.

Change-Id: Ibe493d0cd5762afe5caabe9b77a333fd6daa5373
2023-09-27 13:17:30 +02:00
paul cannon
72189330fd satellite/gracefulexit: revamp graceful exit
Currently, graceful exit is a complicated subsystem that keeps a queue
of all pieces expected to be on a node, and asks the node to transfer
those pieces to other nodes one by one. The complexity of the system
has, unfortunately, led to numerous bugs and unexpected behaviors.

We have decided to remove this entire subsystem and restructure graceful
exit as follows:

* Nodes will signal their intent to exit gracefully
* The satellite will not send any new pieces to gracefully exiting nodes
* Pieces on gracefully exiting nodes will be considered by the repair
  subsystem as "retrievable but unhealthy". They will be repaired off of
  the exiting node as needed.
* After one month (with an appropriately high online score), the node
  will be considered exited, and held amounts for the node will be
  released. The repair worker will continue to fetch pieces from the
  node as long as the node stays online.
* If, at the end of the month, a node's online score is below a certain
  threshold, its graceful exit will fail.

Refs: https://github.com/storj/storj/issues/6042
Change-Id: I52d4e07a4198e9cb2adf5e6cee2cb64d6f9f426b
2023-09-27 08:40:01 +00:00
Jeremy Wharton
3d3785a605 web/satellite/vuetify-poc: prevent inappropriate sidebar closure
This change prevents the navigation sidebar from closing when an item
in the My Account dropdown menu is clicked and the display size is
larger than medium.

Resolves #6332

Change-Id: Id37c3d8ee7179805cfecbd3eac9257130e9acc5b
2023-09-26 19:31:12 +00:00
Moby von Briesen
f63f3f19ee satellite/analytics: Do not set lifecyclestage from Segment
We set lifecyclestage in Hubspot without Segment now. If Segment tries
to set lifecyclestage, it interferes with the desired behavior in
Hubspot.

Change-Id: I817c0324ecc69529d8ca7f617cb97d2f4e84aee8
2023-09-26 17:10:22 +00:00
Márton Elek
0affe03007 cmd/tools/tag-signer: make the output less noisy
This patch removes the following lines from the output (with disable tracing + set the log level to warn):

```
2023-09-25T15:30:58+02:00	INFO	process/tracing.go:73	Anonymized tracing enabled
2023-09-25T15:30:58+02:00	DEBUG	tracing collector	monkit-jaeger@v0.0.0-20220915074555-d100d7589f41/udp.go:128	started
2023-09-25T15:30:58+02:00	DEBUG	process/debug.go:37	debug server listening on 127.0.0.1:34803
```

Change-Id: Iccbf4fc3bde9436e0571943d0d85c51ebc766ef9
2023-09-26 16:30:48 +00:00
Vitalii
05a276ecc7 web/satellite/vuetify-poc: add buckets table to project dashboard
Added buckets table to the bottom of project dashboard view

Issue:
https://github.com/storj/storj/issues/6322

Change-Id: I059bf60631096956d55522da7d18cb8a9eaedc93
2023-09-26 15:49:47 +00:00
Vitalii
f5af0f2268 web/satellite/vuetify-poc: add geographic distribution dialog
Added geographic distribution dialog to object browser preview.

Issue:
https://github.com/storj/storj/issues/6321

Change-Id: Ia3d89a3c27ce8100a3a0a4e5fff91f0c41d0595e
2023-09-26 15:49:06 +00:00
Vitalii
f78cde93ac web/satellite/vuetify-poc: make project table search field be consistent with other search fields
Made search field styling be consistent with other search fields.

Issue:
https://github.com/storj/storj/issues/6320

Change-Id: I21e383cef522a9f76b437a8f9977eab72987766c
2023-09-26 15:09:41 +00:00
Cameron
8689f609d7 web/satellite: vuetify upload progress panel
vuetify snackbar with expansion panels displaying upload statuses.

https://github.com/storj/storj/issues/6257

Change-Id: Ife01616f5a07a4987153ef85331ff71f53b8cf78
2023-09-26 14:04:40 +00:00
Ivan Fraixedes
6555a68fa9 satellite/admin: Serve back-office static UI
Serve the front-end sources of the new back-office through the current
satellite admin server under the path `/back-office`.

The front-end is served in the same way than the current one, which is
through an indicated directory path with a configuration parameter or
embed in the binary when that configuration parameter is empty.

The commit also slightly changes the test that checks serving these
static assets for not targeting the empty file in the build folder.

build folders must remain because of the embed directive.

Change-Id: I3c5af6b75ec944722dbdc4c560d0e7d907a205b8
2023-09-26 13:18:29 +00:00
Ivan Fraixedes
48d7be7eab
private/apigen: Panic types defined in main package
The API generator was generating invalid code when types were defined in
a main package because the generated Go code was defining in import from
it.

This commit update the Go generator to panic with a explicit error
message if that situation happens.

The commit also add a new endpoint to the example with a named types
(i.e. no anonymous) to show that the Generator works fine with them.

Change-Id: Ieddd89c67048de50516f7ac7787d602660dc4a54
2023-09-26 14:04:33 +02:00
Ivan Fraixedes
7ab7ac49c8
private/apigen: Remove TypeScript errors from generated code
Remove the "duplicate identifier: 'path'" TypeScript errors from the
generated code.

Change-Id: I6b411a5cee720e2a16353034627954616c480f8a
2023-09-26 14:04:33 +02:00
Ivan Fraixedes
00484429d6
private/apigen: Generate valid TypeScript code with anonymous types
The API generator didn't generate valid TypeScript code when using
Go anonymous types.

This commit fixes that issue creating names for anonymous types.

Change-Id: Ice0748d8650686e3d3979523b8f218dc20eade5a
2023-09-26 14:04:33 +02:00
Ivan Fraixedes
a9901cc7d0
private/apigen: Add validations to improve usage
Add a few validations to panic with a nicer message or abort rather than
generating invalid code.

Also improve the panic message wrapping a standard error with errs2 at
the time that it's returned.

Change-Id: I1393933eb5f0bc3f86646bf4d0acfc64626efbe0
2023-09-26 14:04:33 +02:00
Ivan Fraixedes
822a13570e
private/apigen: Document Endpoint fields
Add document comments to tall the fields of the Endpoint struct.

Change-Id: Ibcd534f26474fc73fb378a9f30ce7d9401217557
2023-09-26 14:04:32 +02:00
Wilfred Asomani
d67bb4f2c5 web/satellite/vuetify-poc: show file delete confirmation dialog
This change extends the folder deletion modal to work for objects as
well.

Issue: https://github.com/storj/storj/issues/6299

Change-Id: I13e9ffa508c802480c0e3ed2ac630fa693b66fc7
2023-09-26 11:00:16 +00:00
Vitalii
4cbfa28e10 web/satellite/vuetify-poc: use 'demo-bucket' placeholder instead of prefilled value
Use placeholder instead of prefilled value for bucket creation dialog.

Issue:
https://github.com/storj/storj/issues/6319

Change-Id: I86c25926034adbe93a58df56e0bf18b60b41e568
2023-09-26 09:59:10 +00:00
Vitalii
4a9d5edbfc web/satellite/vuetify-poc: another round of small improvements
break text of download notification (e.g. when you download from preview)
hide right/left buttons in object preview if there are not next/previous items
add counter property to project name in "project create". Set max length so that user cannot type additional characters

Issue:
https://github.com/storj/storj/issues/6268

Change-Id: Icff95427a5c73c2fb5bb014ff09150283cc49e83
2023-09-26 11:37:21 +03:00
Ivan Fraixedes
9a06de9058 private/apigen: Add test for Types.All
Add a test for Types.All method because we may need to adjust the logic
in future commits and we want to detect what has changed for good or
bad.

Change-Id: I1db4bf67db3c87513cb9aeb7b942c6c132fc4dd1
2023-09-25 21:15:06 +00:00
Antonio Franco
fe0b5743b0
Af testplan/private cloud (#6265)
* doc(testplan): storj private cloud

* Update storj-private-cloud-testplan.md

this commit pushes the content to the template file and removes unused sections.
2023-09-25 18:44:39 +02:00
paul cannon
1b8bd6c082 satellite/repair: unify repair logic
The repair checker and repair worker both need to determine which pieces
are healthy, which are retrievable, and which should be replaced, but
they have been doing it in different ways in different code, which has
been the cause of bugs. The same term could have very similar but subtly
different meanings between the two, causing much confusion.

With this change, the piece- and node-classification logic is
consolidated into one place within the satellite/repair package, so that
both subsystems can use it. This ought to make decision-making code more
concise and more readable.

The consolidated classification logic has been expanded to create more
sets, so that the decision-making code does not need to do as much
precalculation. It should now be clearer in comments and code that a
piece can belong to multiple sets arbitrarily (except where the
definition of the sets makes this logically impossible), and what the
precise meaning of each set is. These sets include Missing, Suspended,
Clumped, OutOfPlacement, InExcludedCountry, ForcingRepair,
UnhealthyRetrievable, Unhealthy, Retrievable, and Healthy.

Some other side effects of this change:

* CreatePutRepairOrderLimits no longer needs to special-case excluded
  countries; it can just create as many order limits as requested (by
  way of len(newNodes)).
* The repair checker will now queue a segment for repair when there are
  any pieces out of placement. The code calls this "forcing a repair".
* The checker.ReliabilityCache is now accessed by way of a GetNodes()
  function similar to the one on the overlay. The classification methods
  like MissingPieces(), OutOfPlacementPieces(), and
  PiecesNodesLastNetsInOrder() are removed in favor of the
  classification logic in satellite/repair/classification.go. This
  means the reliability cache no longer needs access to the placement
  rules or excluded countries list.

Change-Id: I105109fb94ee126952f07d747c6e11131164fadb
2023-09-25 09:42:08 -05:00
Márton Elek
c44e3d78d8 satellite/satellitedb: repairqueue.Select uses placement constraints
Change-Id: I59739926f8f6c5eaca3199369d4c5d88a9c08be8
2023-09-25 10:14:25 +00:00
Márton Elek
b4fdc49194 satellite/repair/checker: persist placement information to the queue
Change-Id: I51c7fd5a2a38f9f6620c16eddaed3b4915ffd792
2023-09-25 09:33:46 +00:00
Márton Elek
18d5caad7e satellite/satellitedb: write/read placement information to/from repairqueue
Change-Id: Ie58f129feae7898850905940f94643605dcf56ae
2023-09-25 08:52:56 +00:00
Vitalii
7d0b8f6f8c web/satellite/vuetify-poc: center loader in file preview
Vertically centered loader and no preview states for gallery view.

Issue:
https://github.com/storj/storj/issues/6312

Change-Id: Icce4aff6d0927ee36e3a94886edf4cc31dd379bf
2023-09-23 05:11:51 +00:00
Vitalii
5689083393 web/satellite/vuetify-poc: fix limits values formatting on project dashboard
Fixed regular expression which should search and remove insignificant trailing zeros of limit values.

Issue:
https://github.com/storj/storj/issues/6311

Change-Id: I267c779e406a1933d43f09b497470bad9f8ab71c
2023-09-22 21:16:54 +00:00
Vitalii
52d337496f web/satellite: change 'Bandwidth' and 'Egress' labels to 'Download' on project dashboard/settings
Updated label to be 'Download' instead of 'Bandwidth' or 'Egress'

Issue:
https://github.com/storj/storj/issues/6310
https://github.com/storj/storj/issues/6315

Change-Id: Id075882e8dd1e243ca516907d9db420d6d066437
2023-09-22 20:36:22 +00:00
Vitalii
a43f17d9a3 web/satellite/vuetify-poc: use 'Files' header instead of 'Objects' for buckets table
Updated table header to be 'Files' instead of 'Objects'

Issue:
https://github.com/storj/storj/issues/6309

Change-Id: I65058b473481e8b7218db15767c5cb7e2e0c2d8d
2023-09-22 19:01:10 +00:00
Ivan Fraixedes
8381483f79
satellite/admin: Add back-office UI sources
Add the front-end sources of the new back-office.

The front-end doesn't have any business logic, it only has the pages and
the components, so it's purely UI.

The front-end was developed in a separate repository until was
completed.

Change-Id: I382e50789d6b929a67b8a0b887563ef48cb1473d
2023-09-22 18:02:55 +02:00
Dan Willoughby
e563de6c81 web/satellite: Update segment and free tier doc links
4 links updated

Change-Id: I71111079b58542e8c7e4240e2cb55414a7b35934
2023-09-22 11:33:58 +00:00
Jeremy Wharton
5f4cd92cc5 web/satellite/vuetify-poc: clear S3 data when switching projects
This change fixes an issue where the S3 client data was not cleared
when switching projects. This would cause errors to appear when
entering the bucket of a project you switched to.

Resolves #6295

Change-Id: Ib9da43ddf1d38eed6ca26ba73a24e38815617b3e
2023-09-22 00:49:22 -05:00
Jeremy Wharton
e92acca937 web/satellite/vuetify-poc: show loader when creating a folder
This change makes the New Folder dialog indicate its loading status.

Resolves #6307

Change-Id: I37c87ce78f5dfa0d2594e874f248c451fb0d710f
2023-09-21 23:08:24 +00:00
Jeremy Wharton
0205a08c20 web/satellite/vuetify-poc: use image icon for SVGs in object browser
This change makes SVG file entries in the object browser table use the
image icon. Previously, SVGs used the icon for unknown file types.

Resolves #6306

Change-Id: Ic2a8b6154dc222292f3048d967a0420d5872acd0
2023-09-21 22:27:45 +00:00
Jeremy Wharton
6a4abb7f14 web/satellite/vuetify-poc: make project stat cards clickable
When clicked, cards containing project stats will redirect the user to
the card's respective page.

Resolves #6305

Change-Id: I6a598ad2a8a6ab79f48f559eced55f8f8257a518
2023-09-21 21:47:22 +00:00
Jeremy Wharton
15c0c675b1 web/satellite/vuetify-poc: update "Learn More" links in page subtitles
This change updates the "Learn More" links in the Buckets page,
the Team page, and the project dashboard.

Resolves #6293

Change-Id: I2d9c0ca9b8bbd2991869648d231a8069868efdc0
2023-09-21 21:06:57 +00:00
Jeremy Wharton
123309c648 web/satellite: update vuetify dependency
This change updates the Vuetify dependency from 3.3.12 to 3.3.17.

Change-Id: Id570e28c105a10fb2baca25367a4a3e53ce4bf52
2023-09-21 15:55:40 +00:00