Commit Graph

6829 Commits

Author SHA1 Message Date
Artur M. Wolff
0f626a59c1 mod: bump storj.io/uplink
Change-Id: I6d51fd330ebb882564b6c0cd02d829de08082b46
2022-07-05 10:27:27 +00:00
Qweder93
417df4d73c satellite/metabase: don't return expired objects
Resolves: https://github.com/storj/team-metainfo/issues/112

Change-Id: If48f865826c22764fc1ff83745c34524cd396aac
2022-07-04 21:32:30 +03:00
Egon Elbre
d162788489 build: use npm install
`npm ci` deletes the node_modules directory, which also removes go.mod
from that folder.

Add --loglevel timing, so we can debug install slowness, whenever it
happens.

Change-Id: Ide613c4124bfdca9ae978876b2deed8abf86f987
2022-07-01 17:54:18 +03:00
Vitalii
53bddbdd0a web/satellite: fixes for bucket details page
Ask for encryption passphrase when user tries to navigate inside bucket using next flow: Buckets -> BucketDetails.
We don't ask for passphrase if user uses this flow: FileBrowser -> BucketDetails.
Don't allow empty passphrase in open bucket modal.
Fetch buckets using our API after new bucket creation flow (instead of S3 fetch).

Change-Id: Ia0894d6bb4a764c4ff0974fb16ed89bb82699807
2022-07-01 12:01:34 +00:00
Ivan Fraixedes
1fbc8f1f40 satellite/console/consoleweb: Delete old TODO comment
Delete an old TODO comment which doesn't apply anymore.

Closes https://github.com/storj/storj/issues/4958

Change-Id: Ie4be51afb6a39b3f5e5822ac04d1c8fd9d86ee3a
2022-07-01 11:31:19 +00:00
Egon Elbre
97e6e5a6a3 build: improve npm build
Add go.mod to node_modules folder, that way Go compiler doesn't
need to scan the node_module directories for any Go code.

Change-Id: I747909416490c847d6b4bfa3438fea66660fcd53
2022-07-01 10:51:53 +00:00
Egon Elbre
05e165283f storagenode/console/consoleapi: use fixed time.Now()
It seems the tests relied on time.Now(), which might cause some
discrepancies in calculations. Use a fixed time.Now() rather than
recalculating.

As a sidefix, remove "Test" prefix from t.Run. These are unnecessary.

Change-Id: I1de903fcf0fcf46fc8e3acf2463e17239b8e3cc6
2022-07-01 12:36:01 +03:00
Márton Elek
6c5edf436e build: execute web tests only when source is changed (verify)
Change-Id: I1036e8cfed8cc708ef28a41dcd69017028ed9ff4
2022-07-01 07:58:45 +00:00
Erik van Velzen
f23d5eb5a1 satellite/repair: remove superfluous conditional
Change-Id: If80ae0a1a4ee436763ed437fc77b0ed26db17a68
2022-06-30 18:09:17 +00:00
NickolaiYurchenko
5b550a192c web/satellite: access grant flow ui bugs fixed
Change-Id: If0e29728e3a410d629ffc6c39145ac0c9e572101
2022-06-30 10:00:26 +00:00
littleskunk
cf750716ea
satellite/console: enable new access grant flow (#4934) 2022-06-30 00:06:39 +02:00
Egon Elbre
b8006e192b satellite/audit: use a larger delay in the test
The MinDownloadTimeout 950ms and delay of 1s were quiet close, possibly
causing flaky behavior in TestVerifierSlowDownload.

Change-Id: I4f6c1554a118b21427357642abe39986fd0af38d
2022-06-28 17:03:23 +03:00
Ivan Fraixedes
0051298eec satellite/console: Classify errors activation tokens
Classify errors related to invalid tokens for activating user accounts
for returning 400 status code rather than 500 status code.

Don't log all the errors with "error" level, only the ones related to
internal server errors and the rest log them with "debug" level because
they pollute the production satellite errors with errors that are
misguiding.

Change-Id: Id2bd737edba8550ce08965b51b8bf2540bd13ca4
2022-06-28 11:07:57 +00:00
Egon Elbre
65b5a0fe82 satellite/{overlay,satellitedb}: add AS OF to download selection
This should reduce the load caused by download selection queries.

Change-Id: Ic1a89d9c3eb5418f0792eb20ec2aece18dc63f2c
2022-06-28 05:18:01 +00:00
Erik van Velzen
74f4f6e765 satellite/metabase: fix copy to ancestor location
Previously copying an object to it's ancestor location (copy of copy)
broke the object and all copies.

This fixes this by calling the existing delete method rather than a
custom one when there is an existing object at the copy destination.

The check for existing object at destination has been moved to an
earlier point in FinishCopy.

metabase.DeleteObject exposes a transaction parameter so that it can be
reused within metabase.

Closes https://github.com/storj/storj/issues/4707

Uplink test at https://review.dev.storj.io/c/storj/uplink/+/7557

Change-Id: I418fc3337fa9f30146ccc1db456af168ae41c326
2022-06-28 03:53:01 +02:00
Jeff Wendling
3a63939fc9 satellite/metabase: add defensive check on copy object
- instead of closing over the outer err variable, potentially
  overwriting some errors or something, declare local variables.

- double check that we got the number of rows we expected to get
  and error otherwise. this prevents a possible source of inserting
  bogus rows into the database.

Change-Id: I30662be2727afe0a90e4215a182fedc2648d1169
2022-06-27 12:30:44 -04:00
Jeff Wendling
e00423e2e9 cmd/uplink: disable deleting local files
Change-Id: Id3687f3346f8ec9b5da5459a57d91d1d30bf5566
2022-06-27 10:30:04 -04:00
Vitalii
8bd9a079e4 web/satellite: fix bucket open/delete issue
Fixed bucket open and delete issue.

Issue: https://github.com/storj/storj/issues/4932

Change-Id: Idbd981007e537ffb4dd085c68c03a0edfd1cebec
2022-06-27 11:54:42 +00:00
Erik van Velzen
4da61aa257 satellite/metabase: segment_copies full table scan
Part of the delete query cause a full table scan of segment_copies. This
slowed down the system. This change should have the same semantics but
improved performance.

Part of https://github.com/storj/storj/issues/4898

Change-Id: I4afe23df05467eafc9c91591f47a7251a0f3dd31
2022-06-27 11:03:15 +00:00
Vitalii
5f1390e1d5 web/satellite: fixes for navigation sidebar
Show free/pro badge in collapsed nav side bar.
Use getter to indicate which badge to show instead of calculating once after first render.
Fix project overview label for smaller screens. The problem was with new project dashboard route name.

Change-Id: I6ec340b14fe7cf11ba96a0d4ae6771e830c2ed94
2022-06-27 10:53:02 +03:00
Yaroslav Vorobiov
91ea659298 satellite/payments/storjscan: payment API type add USD value
Change-Id: I6e06a5557df795b4f91dd52b191384d7e6b4b07c
2022-06-24 22:30:30 +00:00
JT Olio
18666b078a metabase: support disabling serverside copy when already enabled
Change-Id: I99b0b5597537f69fbdd8cab479e059ba4bf310bd
2022-06-24 17:56:43 +00:00
Erik van Velzen
77fea6137f satellite/metabase: copy inside transaction
Read the source object and write the destination object in the same
transaction, to prevent breaking the object because it was deleted
simultaneously.

This is probably the root cause of the metainfo loop halting from
2022-06-21 onwards, where 2 objects lost their root_piece_id during
copying.

Part of https://github.com/storj/storj/issues/4930

Change-Id: I9c45d56a7bfb48ecd5f4906ee1cca42922901e90
2022-06-24 01:32:09 +02:00
Márton Elek
e3ac0ae2c2
build: fix } typo in Jenkinsfile
Change-Id: I30f172069e7b07efc682b7407e9c48e3cb8c2f9e
2022-06-23 09:25:51 +02:00
Márton Elek
204367b74d
build: premerge/verify Jenkinsfile (subsets of Jenkinsifle.public)
Change-Id: I33c65a37ecd8fcb89729b2a8db3906c6b8890ca3
2022-06-23 09:15:43 +02:00
NickolaiYurchenko
b4e486f3fc web/satellite: common table component applied to bucket page
Change-Id: I579ee22ea441e3084f0d56462676d48870087624
2022-06-22 15:57:40 +00:00
Ivan Fraixedes
6e8e86a0b7 satellite/accounting: Fix typos in doc comments
Change-Id: I497bebe152848e023d927337c61e3eee73b87697
2022-06-22 14:45:21 +00:00
Vitalii
0dba785798 satellite/admin: return only owned projects for get user endpoint
Returns only user's own projects when we hit GET user endpoint.

Fix for this issue
https://github.com/storj/storj/issues/4820

Change-Id: I546268fa3e5983a72f11f998803da5455c0035b4
2022-06-22 13:09:36 +03:00
Paul Willoughby
de125eeb51 cmd/uplink: Fix typo in flag description
Change-Id: I6db38ecec2549d288285a9865f97badfe808b892
2022-06-21 11:55:02 -06:00
Ivan Fraixedes
fc4b0e41f0 scripts: Change Storagenode arm32v6 to arm32v5
We don't build "multinode" nor "storagenode" docker images for armv6
architecture, instead we build for armv5.

Fix the script that publish the manifest for those images for a specific
tag to use armv5 for not failing when executing.

Change-Id: I7d859d8718240e1cd0dae6489e7e5c3b4068ff6e
2022-06-21 11:54:38 +00:00
Egon Elbre
94ae196189 testsuite/ui: move go.mod
I would like to start moving some of the integration tests from scripts
folder into testsuite. This allows integration tests to easily have
other and fewer dependencies.

This will also be useful for https://github.com/storj/storj/issues/4823.

Change-Id: Idbdb5aefc0e42d091ad7f74dbdc096281c1dfcba
2022-06-21 14:12:35 +03:00
Ivan Fraixedes
f626cd1868 satellite/accounting: Correctly cache project bandwidth
Satellite caches the project bandwidth in Redis when it doesn't have it
because was not set or the key expired, however, it doesn't perform the
check and set if not exists in a transaction. It also uses the increase
function which increases the value if it exists otherwise it sets it.

This provokes that multiple concurrent request to the same project may
increase the total project by multiples of the bandwidth usage
registered in the database rather than setting it because they may check
if the key exists before any other has executed the increase and then
the first one executing it will set the value but the others will
increased causing that Redis has a wrong bandwidth usage value which is
N magnitude of the real one and making the satellite to deny the
downloading if it surpasses the project limit.

This commit changes the "update"" project bandwidth usage by an "insert"
but using a Redis function that only sets the value if the key doesn't
exists for solving the increase issue but also not overriding the value
due to may contain updates of other downloading requests which aren't
already registered in the DB.

Change-Id: I33e2fe462930b2fdb4061fc94002bd3544476f94
2022-06-21 08:31:41 +00:00
René Smeekes
fd042a92b9
storagenode/reputation: clarify wording on suspension notification (#4921)
Fixes https://github.com/storj/storj/issues/4920
2022-06-20 22:07:12 +02:00
Cameron
b7fd6a70fb web/satellite: fix ui bugs in bucket navigation
On bucket details page, make bucket name breadcrumb link back to the
page for that bucket. Additionally, add a pointer cursor to the
"buckets" breadcrumb.

Also remove settings dropdown from bucket details page as the only item
is "View Bucket Details" which is already the current page.

github issue: https://github.com/storj/storj/issues/4909

Change-Id: Ic24abba7fb447ca5a8ac1e8ba244ba98dec2fc92
2022-06-20 15:53:32 +00:00
NickolaiYurchenko
f7a3be5d20 web/satellite: common table component created
Change-Id: Ie45a2c37257f3311c767adcea1a22f284b66f24e
2022-06-19 14:52:42 +00:00
Cameron
43ed35e025 testsuite/ui/satellite: add ui tests for new access grant flow
also fix typo in web/satellite CreateAccessModal.vue

github issue: https://github.com/storj/storj/issues/4825

Change-Id: I6f66e604ed3d0899aa2ec83a0638c43d4b5e1e42
2022-06-17 21:11:30 +00:00
prerna-parashar
6a1d7c8747
satellite/analytics: Added tracks calls for product activity metrics (#4907)
satellite/analytics: Added tracks calls for product activity metrics
2022-06-17 12:57:10 -07:00
paul cannon
4e81a60838 satellite/overlay: fix TestNodeSelectionGracefulExit
This test had an effective config.Reputation.AuditCount = 0, meaning all
nodes that had _any_ positive audit results were considered vetted.
Because of that, only one node in the test setup was "new". And that
node was marked as being in GE, so could not be returned by node
selection.

The reason the tests still worked is because of the node selection rule
that says "if there are no new nodes at all, just get all reputable
nodes to satisfy the request".

This commit makes it so half of the nodes are vetted and half new, which
makes the test somewhat more interesting (and means we aren't
concentrating too much on testing details of behavior when AuditCount is
0).

Change-Id: I09157b7dc20ecaddd2a6e60cfe146e9186e3603b
2022-06-17 18:28:04 +00:00
Vitalii
b25faf30ab web/satellite: rework navigation sidebar styling
Reworked styling to correspond to newest designs.

Resolves those issues:
https://github.com/storj/storj/issues/4847
https://github.com/storj/storj/issues/4848
https://github.com/storj/storj/issues/4845
https://github.com/storj/storj/issues/4850

Change-Id: Ie3510b435815834adff5160541b5d9d4859763eb
2022-06-17 17:55:47 +00:00
Jeremy Wharton
5ce7d980af private/apigen: Make API generation deterministic
This change fixes the issue where the API generator would produce
different Go code for the same API definition upon each invocation
due to the random nature of map iteration.

Change-Id: I6770a10faf06311c24f541611c25d0b2b0f8e521
2022-06-17 12:06:08 -05:00
Qweder93
cc0dc2e262 satellite/metabase: FinishMoveObject cheks for empty metadata
To avoid regression with old versions of uplink objects move we need to
remove FinishMoveObject check for key and nonce, in SQL
in FinishMoveObject do check if metadata is nil then
don't set key and nonce. The same UPDATE clause should return metadata and
if metadata != nil we should do the same validation for key and nonce
to avoid putting broken key and nonce while doing move

Resolves: https://github.com/storj/team-metainfo/issues/108

Change-Id: If723dfad899e9235f53559b71ee1c7fe49deb8b8
2022-06-17 08:24:24 +00:00
Vitalii
e79e79d2e1 web/satellite: fix linksharing issue
Bucket creation/open flow wasn't storing credentials correctly inside Vuex store.

Change-Id: Ifc2eb8ca79a672dbc44d95089f7d6cee2e71bb57
2022-06-16 20:57:43 +03:00
Yaroslav Vorobiov
13be4ad0dd satellite/payments/storjscan: add chore to periodically ask for new payments
Change-Id: Ia6ad182184596fe1fe1f9a40f909a3af4be8feef
2022-06-16 09:38:37 +01:00
Jeremy Wharton
bd0d717f4c private/apigen: Remove unnecessary log message when generating API code
A message intended to diagnose issues in a previous change was
accidentally allowed to remain in finalized code and merged into
the codebase.

Change-Id: I7df662a81e3790de065beeb6db54be5280ac2fa1
2022-06-15 15:58:45 -05:00
Jeremy Wharton
905654956a private/apigen: Resolve import issues
This change resolves several issues:
* Imports can no longer be duplicated
* The import statement now conforms to Storj's code style requirements
* Packages are imported as needed, eliminating extraneous imports

Change-Id: Ie021e06bb0d472c65aa3575db94bc81ccac108e3
2022-06-15 19:51:54 +00:00
Fadila Khadar
ffcd54719a satellite/satellitedb: use tx instead of db.db in transactions
Change-Id: Icee8871af76233651f007e03173660905eafc0e3
2022-06-14 17:35:44 +00:00
JT Olio
bb1eb1bae5 storj: bump private
Change-Id: I3dd80fc19a1412ac1d8264080cbf20bc4f040f13
2022-06-14 11:54:28 -04:00
Cameron
240b70b828 satellite/console: use new type UpdateUserRequest as arg to db users.Update
The users.Update method in the satellitedb package takes a console.User
as an argument. It reads some of the fields on this struct and assigns
the value to dbx.User_Update_Fields. However, you cannot optionally
update only some of the fields. They all will always be updated. This means
that if you only want to update FullName, you still need to read the
user info from the DB to avoid updating the rest of the fields to zero.
This is not good because concurrent updates can overwrite each other.

This change introduces a new struct type, UpdateUserRequest, which
contains pointers for all the fields that are updated by satellite db
users.Update. Now the update method will check if a field is nil before
assigning the value to be updated in the db, so you only need to set the
field you want updated. For nullable columns, the respective field is a
double pointer. This allows us to update a column to NULL if the outer
pointer is not nil, but the inner pointer is.

Change-Id: I27f842d283c2711e24d51dcab622e57eeb9157f1
2022-06-14 09:28:03 -04:00
Fadila Khadar
3ae325462c satellite/metabase: fix move transaction
Change-Id: Id1b6c721e7373dffd3f40aafa15109fc2cb82474
2022-06-14 11:04:44 +02:00
Erik van Velzen
b828b6ec34 satellite/metabase: fix copy transaction
Change-Id: I4921d3e6bc17b0c6aff00eba7c76266cb36e78a0
2022-06-13 19:47:07 +02:00