Commit Graph

6968 Commits

Author SHA1 Message Date
wilfredasomani
23e5ccb9c9 web/satellite: Fix styling on registration page
On small screens the top of the registration page is cut off. This change updates the styles to fix that

Change-Id: I73420cb30c5e34acf7aa6312c054655db410f5e2
2022-07-21 16:36:38 +00:00
Lizzy Thomson
c237468ac9 satellite/console: allow user to update project when limits are above paid defaults
When a user's bandwidth/storage limits are manually set to exceed the
paid tier defaults, attempting to update their project via the satellite
UI (e.g. to change the name/description) would result in an error.
This change modifies the limit checks for updating a project to remove
this issue.

https://github.com/storj/storj/issues/4892

Change-Id: I48853a3289b0ac51587f268a18c1b25743123fcf
2022-07-21 08:40:18 -06:00
Michał Niewrzał
d72f9525d4 satellite/metainfo/piecedeletion: use nodes cache
Piece deletion service was using KnownReliable method from
overlaycache to get nodes addresses to send delete request.
KnownReliable was always hitting DB because this method was
not using cache. This change is using new DownloadSelectionCache
to avoid direct DB calls.

Change is not perfect because DownloadSelectionCache is not as
precise as KnownReliable method and can select few more nodes
to which we will send delete request but difference should be
small and we can improve it later.

Updates https://github.com/storj/storj/issues/4959

Change-Id: I4c3d91089a18ac35ebcb469a56536c33f76e44ea
2022-07-21 00:43:24 +00:00
NickolaiYurchenko
ba009b846f web/satellite: register page made configurable
we want sign up page to be more configurable since we have to add partner branding to it.
added ability to add logo, different title and decpription and even html after common markup.
v-html rule disabled and sanitize plusin added to avoid xss injections.
tsconfig changed to be able to load modules from json files.

Change-Id: I9ff2cf6e5cf977b8c41ca48953b491bd361f26a1
2022-07-20 18:57:47 +03:00
Egon Elbre
82fede2132 satellite/satellitedb: batch delete storage node tallies
Currently we have a significant number of tallies that need to be
deleted together. Add a limit (by default 10k) to how many will
be deleted at the same time.

Change-Id: If530383f19b4d3bb83ed5fe956610a2e52f130a1
2022-07-20 15:42:35 +03:00
cl-mitch
a9de5ce6c3
web/satellite: Add PaymentMethodsTab to new billing flow (#4973)
Add PaymentMethodsTab.vue with stubbed components. Code will be added to the stubbed components in future PR.
2022-07-19 13:17:06 -05:00
Vitalii
b3f909051a web/satellite: fix for dev build
Fixed dev build to do only one build at a time.
Basically, it will do only one legacy type of build and ignore modern build.

Issue:
https://github.com/storj/storj/issues/4908

Change-Id: Ia90deac1f8bf4f609ec776a3580ab0dc8f22a84e
2022-07-19 19:03:04 +03:00
Vitalii
08120d1445 web/satellite: pass query param on satellite switch in signup form
If "partner" query param is set on signup screen we pass it on satellite switch.

Issue: https://github.com/storj/storj/issues/4955

Change-Id: I59039d6ae660d8790390e59650de0ce1e3bba1b4
2022-07-19 13:42:18 +00:00
Erik van Velzen
87e1528104 satellite/metabase: error if source object changed
An object copy/move is done by 2 DRPC calls. It's possible a new object was uploaded ore moved to the source location between these calls. For copy, in that case the segments end up with the wrong keys. This change adds an explicit check for that by comparing the streamId supplied by the user with the streamId in the database.

Fixes https://github.com/storj/storj/issues/4930

Change-Id: Id600456ce78fb4069b93644828a0b3eb85e23e16
2022-07-19 13:27:55 +02:00
Márton Elek
95d85c640c ci: use persisted cache files during verify phase
.

Change-Id: If8af41fd4e3df1d04fef50d9fa53d2038c7d6cc4
2022-07-19 09:50:09 +00:00
Qweder93
cf12802a12 satellite/metainfo: ListBuckets extended with value attribution assigned to bucket
We need to provide the ability to see bucket attribution on the gateway side
so customers can validate if bucket is attributed to them. Extendet metainfo.ListBuckets
request with UserAgent.

Fixes https://github.com/storj/storj/issues/4965

Change-Id: I5624874a7faa14cda06183ad44013e9ebb385b63
2022-07-18 18:23:50 +00:00
NickolaiYurchenko
0317c819dc web/satellite: logout added after password change
functionality moved from reset password where no cookie and causes error to proper place
added redirect with 2 sec delay after success

Change-Id: I9cfe7e88bed7cfaf04fc7dc7fde58702b0040c5c
2022-07-18 17:48:05 +03:00
Ivan Fraixedes
7aaab3c4c4 satellite/console/consoleweb: Trace HTTP endpoints
Trace all the requests that the HTTP API endpoints receive.

We want to trace them with Monkit because we want to break them down by
request type and response code for seeing if they succeeded or failed.

Also log them with DEBUG level with the IP client.

Change-Id: Ia7b013351c788f131e775818f27091f3014ea861
2022-07-18 13:27:50 +00:00
NickolaiYurchenko
d571b04ac8 web/satellite: new dashboard limit visual component added
also added vue filter to transform number of bytes into readable label

Change-Id: I416cf477fb94059c0f2ceffef755aa7fceacc352
2022-07-18 07:26:30 +00:00
NickolaiYurchenko
1f66bb3931 web/satellite: access grant empty validation added
after I copied and checked an acknowledgment i was able to delete the passphrase and create the access grant
now if I do this "copied" and "acknowledge" statuses if passphrase is changed

Change-Id: I57199a476c802eb3c44e4dddc43ef40fcedcee2f
2022-07-18 06:54:15 +00:00
paul cannon
726c95160b satellite/repair: avoid retrying GET_REPAIR incorrectly
We retry a GET_REPAIR operation in one case, and one case only (as far
as I can determine): when we are trying to connect to a node using its
last known working IP and port combination rather than its supplied
hostname, and we think the operation failed the first time because of a
Dial failure.

However, logs collected from storage node operators along with logs
collected from satellites are strongly indicating that we are retrying
GET_REPAIR operations in some cases even when we succeeded in connecting
to the node the first time. This results in the node complaining loudly
about being given a duplicate order limit (as it should), whereupon the
satellite counts that as an unknown error and potentially penalizes the
node.

See discussion at
https://forum.storj.io/t/get-repair-error-used-serial-already-exists-in-store/17922/36
.

Investigation into this problem has revealed that
`!piecestore.CloseError.Has(err)` may not be the best way of determining
whether a problem occurred during Dial. In fact, it is probably
downright Wrong. Handling of errors on a stream is somewhat complicated,
but it would appear that there are several paths by which an RPC error
originating on the remote side might show up during the Close() call,
and would thus be labeled as a "CloseError".

This change creates a new error class, repairer.ErrDialFailed, with
which we will now wrap errors that _really definitely_ occurred during
a Dial call. We will use this class to determine whether or not to retry
a GET_REPAIR operation. The error will still also be wrapped with
whatever wrapper classes it used to be wrapped with, so the potential
for breakage here should be minimal.

Refs: https://github.com/storj/storj/issues/4687
Change-Id: Ifdd3deadc8258f34cf3fbc42aff393fa545794eb
2022-07-18 05:11:56 +00:00
paul cannon
2f20bbf4d8 satellite/reputation: add a reputation write cache
This should lower the amount of database load coming from
reputation updates.

Change-Id: Iaacfb81480075261da77c5cc93e08b24f69f8949
2022-07-14 21:40:16 +00:00
Sean Harvey
582dce1b36 Makefile: bump to Go 1.17.12
Change-Id: Ifa163a113da8c21a72a6127e53b331b574eea8dc
2022-07-14 20:57:26 +00:00
Yaroslav Vorobiov
de5951331a satellite: wire storjscan chore to core process
Adds storjscan chore and storjscan client to satellite chore.
Storjscan chore interval added to storjscan config.

Change-Id: I13bb8d83b14ca90f4735c9a10c361ad24a56b256
2022-07-14 15:07:52 +00:00
Vitalii
69dc9a4731 satellite/console: added new email which is sent on unknown password reset
Added new email html template.
It is sent when user tries to reset password with unknown or unverified account.
Made a couple of minor config changes.

Issue: https://github.com/storj/storj/issues/4913

Change-Id: I730f48b3478e302d1e38e1f8a27c75f66a8ba6fd
2022-07-14 14:32:59 +00:00
cl-mitch
f5a569692b
web/satellite New billing coupons tab (#4890) 2022-07-14 17:32:39 +03:00
Jennifer Johnson
61649d2850 cmd/node-cleanup: command to delete uncontacted nodes
Some nodes were added to the nodes table due to a bug in quic
based storagenode contact code. This is a tool to clean up these nodes.

Delete with batch-size 1k seems to take ~400ms on local CockroachDB.

Change-Id: Ic0c1180528c27952e19c431fc9cc327292a10a5f
2022-07-12 12:15:49 +00:00
Yaroslav Vorobiov
6d95e34e39 satellite/payments: add payments method for deposit wallets
Add payments method to payments to DepositWallets interface.
Exposes payments retrieval API for a particular wallet to
other systems such as console billing API.

Change-Id: Ifcb3a35514aab50be00f6360007954980b5d8b38
2022-07-12 11:25:30 +00:00
Egon Elbre
51d4e5c275 satellite/{orders,overlay}: use cache for downloads
Use DownloadSelectionCache to avoid querying database for every
download.
This change only addresses downloads from users. The download selection
cache is not currently used for audit and repair.

Change-Id: I96a49e121dac0b4204f97592a63131edabd73fb5
2022-07-12 11:04:34 +00:00
Egon Elbre
48b0a65fbd satellite/overlay: use ReadCache in Download/UploadSelectionCache
sync2.ReadCache implements preemptive refreshing preventing stalling
while it's being updated.

Change-Id: Iee9ef36049b986f0e426c14a139b2bc9ac17fb53
2022-07-12 13:52:48 +03:00
cl-mitch
81ae137e37
web/satellite: added Billing Overview Tab for new Billing flow (#4900)
- Created screen for billing overview tab
 - Added total estimated charges and balance
 - Added breakdown of individual projects
 - Links to billing history and payment methods
 - Mobile Responsive
2022-07-11 15:35:32 -05:00
Egon Elbre
15dc797efd web/{multinode,satellite,storagenode}: revert go.mod
Adding go.mod into node_modules is not sufficient, because npm install
wipes them quite often out. Similarly, when running npm install locally
it will remove it, causing the git state to be dirty.

Rather than having them committed, add them after running npm install.

Change-Id: Iaf21a9c6e198dc31fe50345ec5dee85b44617176
2022-07-08 19:51:51 +03:00
Stefan Benten
c3171b4ba4
storagenode/retain: Move summary and start logs to info level (#4954)
We currently do not log the GC information/stats under normal circumstances.
This is not good for monitoring and troubleshooting.
2022-07-08 18:19:08 +02:00
Michał Niewrzał
43286dbc37 satellite/metainfo: use metaclient type instead of storj
This just cleanup change to unblock libuplink to reorganize types
which are aliases to storj types.

Change-Id: Id3edf13f1b0aef52d7606d545aa7a6594cf8d13f
2022-07-08 14:17:19 +02:00
Egon Elbre
96c3816313 build: add timeout to npm install
Add timeout to npm install and increase logging level.
npm install is still taking sometimes too long and it's not clear why,
verbose logging is not sufficient.

Change-Id: Ib72f9823f30c9744562e279c2a5481f096e38128
2022-07-07 17:59:16 +00:00
Egon Elbre
5c5e6f75e3 build: fail the build on timeout
When there's a timeout we need to handle it in `aborted` post action,
rather than failure.

Change-Id: I861dae2a95ed1818eed937df890f93865da12e75
2022-07-07 17:58:58 +00:00
kimbotsao
d09824d684 web/satellite: do not load stripe for environment unless enabled
Stripe script is only loaded when Stripe is enabled for adding
and using payments.

Change-Id: Ia2ed9dd138bf6a5a288adfb9259e7007822e634f
2022-07-07 16:46:15 +00:00
Egon Elbre
4e31c96836 satellite/console: don't use global log
Change-Id: I2464f2aca3cdb97c19de29dac59499cc9d5ff2bb
2022-07-06 16:50:04 +00:00
hovex023
6f4dc963b1
web/satellite: Billing history tab
* tabs and routing implemented
* added feature flag
* added billing history buildout
* Update to use real data for table population

Story: https://github.com/storj/storj/issues/4633

Co-authored-by: cl-mitch <mitch.george@compozelabs.com>
2022-07-06 11:42:16 -05:00
Egon Elbre
d0ca258925 build: increase npm install loglevel
The builds for npm still fail, however, `--timing` does not provide
sufficient data to debug the situation.

Change-Id: I7e618ba8cac775748ebea6145cd5c180d2dc7883
2022-07-06 18:13:53 +03:00
Lizzy Thomson
e26b1517ed web/satellite added middle truncation logic for filenames in file browser
Added middle truncation and extended filename on hover according to figma designs

Change-Id: I5559abc6e6d959491603c0d2d20ddfbfdc298eee
2022-07-06 13:43:22 +00:00
Cameron
023a1c6271 satellite/satellitedb: add public_id column and index to projects table
github issue: https://github.com/storj/storj/issues/4861

Change-Id: I3d1928ca51c825a4c1589db5ae55902e57f00c3b
2022-07-05 18:39:29 +00:00
Ivan Fraixedes
66f12992b0
cmd/storagenode: Don't abbreviate multinode in help
Don't abbreviate multinode in the command help message because there
isn't a need for it and the abbreviation isn't clear at all.

Change-Id: I7a1f2be6ae1f7d4b287c18c48b22c630549b731f
2022-07-05 19:29:12 +02:00
Clement Sam
ebd463b57c go.mod: update dependencies
Change-Id: I5360ea7b6487295cb7def759f417b31b133c34da
2022-07-05 18:41:39 +03:00
Artur M. Wolff
0f626a59c1 mod: bump storj.io/uplink
Change-Id: I6d51fd330ebb882564b6c0cd02d829de08082b46
2022-07-05 10:27:27 +00:00
Qweder93
417df4d73c satellite/metabase: don't return expired objects
Resolves: https://github.com/storj/team-metainfo/issues/112

Change-Id: If48f865826c22764fc1ff83745c34524cd396aac
2022-07-04 21:32:30 +03:00
Egon Elbre
d162788489 build: use npm install
`npm ci` deletes the node_modules directory, which also removes go.mod
from that folder.

Add --loglevel timing, so we can debug install slowness, whenever it
happens.

Change-Id: Ide613c4124bfdca9ae978876b2deed8abf86f987
2022-07-01 17:54:18 +03:00
Vitalii
53bddbdd0a web/satellite: fixes for bucket details page
Ask for encryption passphrase when user tries to navigate inside bucket using next flow: Buckets -> BucketDetails.
We don't ask for passphrase if user uses this flow: FileBrowser -> BucketDetails.
Don't allow empty passphrase in open bucket modal.
Fetch buckets using our API after new bucket creation flow (instead of S3 fetch).

Change-Id: Ia0894d6bb4a764c4ff0974fb16ed89bb82699807
2022-07-01 12:01:34 +00:00
Ivan Fraixedes
1fbc8f1f40 satellite/console/consoleweb: Delete old TODO comment
Delete an old TODO comment which doesn't apply anymore.

Closes https://github.com/storj/storj/issues/4958

Change-Id: Ie4be51afb6a39b3f5e5822ac04d1c8fd9d86ee3a
2022-07-01 11:31:19 +00:00
Egon Elbre
97e6e5a6a3 build: improve npm build
Add go.mod to node_modules folder, that way Go compiler doesn't
need to scan the node_module directories for any Go code.

Change-Id: I747909416490c847d6b4bfa3438fea66660fcd53
2022-07-01 10:51:53 +00:00
Egon Elbre
05e165283f storagenode/console/consoleapi: use fixed time.Now()
It seems the tests relied on time.Now(), which might cause some
discrepancies in calculations. Use a fixed time.Now() rather than
recalculating.

As a sidefix, remove "Test" prefix from t.Run. These are unnecessary.

Change-Id: I1de903fcf0fcf46fc8e3acf2463e17239b8e3cc6
2022-07-01 12:36:01 +03:00
Márton Elek
6c5edf436e build: execute web tests only when source is changed (verify)
Change-Id: I1036e8cfed8cc708ef28a41dcd69017028ed9ff4
2022-07-01 07:58:45 +00:00
Erik van Velzen
f23d5eb5a1 satellite/repair: remove superfluous conditional
Change-Id: If80ae0a1a4ee436763ed437fc77b0ed26db17a68
2022-06-30 18:09:17 +00:00
NickolaiYurchenko
5b550a192c web/satellite: access grant flow ui bugs fixed
Change-Id: If0e29728e3a410d629ffc6c39145ac0c9e572101
2022-06-30 10:00:26 +00:00
littleskunk
cf750716ea
satellite/console: enable new access grant flow (#4934) 2022-06-30 00:06:39 +02:00