Commit Graph

7132 Commits

Author SHA1 Message Date
Michal Niewrzal
6273ed035d satellite/metabase: make UploadID stable for different options
Multipart upload requires to have the same UploadID returned from
different requests (BeginUpload, ListUploads). Otherwise client won't
be able to find existing uploads. Main issue was that data needed to
construct UploadID is in System metadata which can be filtered out
by listing option.

This change is fixing how we are setting Status for listed objects and
it's forcing reading System metadata if we are reading pending objects.

Fixes https://github.com/storj/storj/issues/5298

Change-Id: I8dd5fbab4421a64dc3ed95556408ead4c829f276
2022-11-10 17:35:36 +00:00
Márton Elek
8c569866aa uplink/share: support share via insecure authservice protocol
This patch makes it possible to use `uplink share` in test environment (like storj-up) where authservice doesn't have full secure endpoint.

This supposed to be an undocumented feature (no flag, just a custom prefix) to avoid any confusion for regular users.

Change-Id: I256aefc944066e52c72224e7b6f1a593b5bc57f7
2022-11-10 15:16:37 +00:00
Egon Elbre
591be92cf9 .github: remove invalid codeowners
Change-Id: Id8594c5efa3987a78b42f407641b0e4b912cd22c
2022-11-10 15:52:48 +02:00
Malcolm Bouzi
3ae057261e web/satellite: make sure users are able to tab through all navigation dropdown items.
Change-Id: Ia6148d96cbe385e25bc1107b974288fb98000fe8
2022-11-09 20:15:07 +00:00
Vitalii
5f089873e0 web/satellite: fix auto open of first uploaded object preview
Fix for AHA moment when user uploads their first object in some particular bucket.

Change-Id: I3682762e6c73b10fd76e816bb752313bc735e9d2
2022-11-09 19:21:50 +00:00
Wilfred Asomani
bae4ee4c5f web/satellite: clear AB testing data on log out for mobile
this change adds a call to clear AB testing data for mobile screens on log out.
Currently, that data is cleared on only large screens; web/satellite/src/components/navigation/AccountArea.vue L156

Change-Id: I04508d33cf65a17d6ac0cc470811aa3bad04d80f
2022-11-09 18:24:55 +00:00
Jeremy Wharton
3c8facfe58 satellite/console,web/satellite: always notify when adding project members
Upon adding members to a project using the Add Team Member modal,
users are now notified that only email addresses belonging to an
account will receive a project invitation. This notification appears
regardless of whether every submitted email corresponds to an account.
Previously, users received an error message if any email address not
attached to an account was submitted.

Change-Id: Ia014c8311c1347e001b1c6c33de73ea61f20b0cb
2022-11-09 17:48:50 +00:00
Fadila Khadar
553c1558a5 satellite/satellitedb: add 'contained' column to nodes table
We want to be able to exclude contained nodes from nodes selection. For this, we add a 'contained' column to the nodes table to track the containment status.

Fixes https://github.com/storj/storj/issues/5231

Change-Id: Id78e645f172145adcb8664646e8ebf14e218b57b
2022-11-09 11:11:37 -06:00
paul cannon
1b5d953fd9 satellite/satellitedb: add queue for verifications
Since the auditor will be moving to a different process from the
metainfo loop, we need a way of communicating which segments have
been chosen for audit. This queue will be that communication, for now.

Contrast this with the queue for _re_verifications in commit 9c67f62f.

Refs: https://github.com/storj/storj/issues/5251
Change-Id: I9a269c7ef21e6c5e9c6e5e1f3db298fe159a8a79
2022-11-09 15:38:56 +00:00
Michal Niewrzal
f465fa927e satellite/console: drop redundant bucket interface
Change-Id: I1e132199f7c2f25eceb49a052ef4016d62db9705
2022-11-09 09:50:54 +01:00
Clement Sam
59b37db670 storagenode: overhaul QUIC check implementation
The current implementation blocks the the startup until one or none
of the trusted satellites is able to reach the node via QUIC.
This can cause delayed startup. Also, the quic check is done
once during startup, and if there is a misconfiguration later,
snos would have to restart to node.

In this change, we reuse the contact service which pings the satellite
periodically for node checkin. During checkin the satellite tries
pinging the node back via both TCP and QUIC and reports both statuses.
WIth this, we are able to get a periodic update of the QUIC status
without restarting the node.

Also adds the time the node was last pinged via QUIC to the tooltip
on the QUIC status tab.

Resolves https://github.com/storj/storj/issues/4398

Change-Id: I18aa2a8e8d44e8187f8f2eb51f398fa6073882a4
2022-11-09 03:15:57 +00:00
Yaroslav Vorobiov
7b13af1184 private/blockchain: address checksum hex
Updates blockchain.Address.Hex to return mixedcase
checksum hash.

Change-Id: I74c830b49eceb30876c8ef20d3adc4816de51a64
2022-11-08 18:19:07 +00:00
Wilfred Asomani
9a303b00bf satellite/analytics: replace hubspot api with oauth access token
Hubspot is migrating from using API keys for authentication to OAuth.
This change migrates our Hubspot integration to use OAuth tokens.
It modifies the EnqueueCreateUser code to not send empty HubspotUTK to hubspot, and to return error for failed requests.

see: https://developers.hubspot.com/changelog/upcoming-api-key-sunset

Change-Id: I422f00e3e3caeff3ff3d08ddec059502b9addaee
2022-11-08 16:10:32 +00:00
Igor
ec6a79af85
scripts/changelog.py: a few fixes (#5293) 2022-11-08 16:44:28 +02:00
NickolaiYurchenko
3c8f68fed9 web/satellite: pinia package added
added to replace vuex in future
buckets, projects and users modules pinia analogues created
have no pretty and straight-forward ways to work with option api + ts files so it is better to use with composition api components

Change-Id: Ia8acc491c0e76e01bf6d533747d186257680e5c9
2022-11-08 11:22:58 +00:00
Moby von Briesen
c07f016f57 satellite/satellitedb: Slight modifications to nodeevents table
* Mark node events table as "safely partitionable", meaning that it
  is/will not be queried relationally along with other tables. This way,
  we can safely use this table in Postgres rather than CockroachDB,
  where most of our other satellite tables are running.
* Add a dbx-generated delete function to the node events table, to allow
  us to easily delete entries created before a provided time. This
  allows us to keep the table clean, since there is no need to persist
  entries after emails have been sent.

Change-Id: I25e8a5c4092fe49dcfa6c8bb73f2043646bb611f
2022-11-07 17:16:59 +00:00
NickolaiYurchenko
8ebea4cf8d web/satellite: FileBrowser component migrated to use composition api
Change-Id: Id0bb007b790e0d2365eb78f2bc78f8c3da05bc70
2022-11-07 16:27:45 +00:00
Michal Niewrzal
e4fab975ad satellite/metainfo: fix types used in tests
Libuplink is using some aliases to storj package which we will
move directly to libuplink and remove from common/storj.
To make code compilable we need to fix places where we
are using aliased types directly to be able to update libuplink.

Change-Id: I7222a927af3b41e214d1c9204917f3ebce4727ce
2022-11-07 14:48:31 +01:00
Kaloyan Raev
e709112eeb satellite/metainfo: allow list permission for GetObject and GetObjectIPs
GetObject and GetObjectIPs are invoked by the Linksharing service to
display the shared object and its map. These two endpoint currently
require read permission.

There is a use case where an object can be shared with an access grant
that has only list permission. In such a case, the expectation is that
the linksharing service would still display the metadata of the shared
object (name, size, map), but the content would be still inaccessible.
See https://github.com/storj/gateway-mt/issues/209 for details.

This change allows GetObject and GetObjectIPs to require either read or
list permission to support the described use case.

Change-Id: I3477edc7bf8990e9848482890da047094c875d09
2022-11-07 10:59:43 +00:00
Jeremy Wharton
b34b5c4177 satellite/payments/stripecoinpayments: do not reference nil invoice
When a customer has no pending line items, an invoice will not be
generated for them and the Stripe client method responsible for
creating new invoices returns nil. This change adds a nil check to
account for this possibility to ensure that no panics are caused
by attempted processing of the invoice.

Change-Id: Id184d027d7447f0ef876db58601ab6cf63927fc5
2022-11-05 18:09:52 -05:00
Michal Niewrzal
b2dc8211d6 satellite/gracefulexit: observer cleanup
Some changes to make code cleaner and easier to adopt to new ranged
loop.

* removed unneeded mutex
* reorganize constructor args
* avoid creating the same redundancy scheme for each segment piece

Change-Id: I81f3f6597147fc515516949db3ce796a60b1c8a0
2022-11-05 12:10:44 +00:00
Moby von Briesen
cc858f4e91 satellite/console: Turn new billing screens on by default
Set default value for console.new-billing-screen to true.

Change-Id: I0f7f9663af4b4dc384d2bda946b4853593151cfa
2022-11-04 19:41:54 +00:00
Vitalii
a342edd1c2 web/multinode: fixed linter
Fixed web linter

Issue:
https://github.com/storj/storj/issues/5158

Change-Id: Ibc0b8e9f241e77e833645d9fa114bf5742c9f81c
2022-11-04 19:06:21 +00:00
Vitalii
8268933b56 web/storagenode: fix linter
Fixed web linter.

Issue:
https://github.com/storj/storj/issues/5158

Change-Id: Ia63e3e6e7352a99e902c9ed08a4bcfd75059e943
2022-11-04 18:32:26 +00:00
Clement Sam
10f59f8633 web/storagenode: update new suspension and audit threshold on dashboard
The new audit scoring sets disqualification threshold to <96:
https://forum.storj.io/t/new-audit-scoring-is-live/19466/24.

This changes applies this update to the dashboard,
same colors but different thresholds now:
<96 = disqualification
<99 = warning

previously:
<60 = disqualification
<95 = warning

Closes https://github.com/storj/storj/issues/5113

Change-Id: Iaf61e5034101fe5d4d70df6b72eb5f4ccb706e5b
2022-11-04 17:47:05 +00:00
Cameron
cb0c359b81 satellite/overlay: insert DQ node events for stray nodes
Change-Id: I99da11e506ab7f6bcebdb08a5815078a3297c932
2022-11-04 15:48:17 +00:00
Cameron
74ddfab810 satellite/overlay: insert DQ event into node events in overlay.DisqualifyNode
Also, return node email from overlaycache db DisqualifyNode to be used
in node events insertion

Change-Id: I41534cf01351c1690c3966a8055c5fe6fcf0d6a6
2022-11-04 15:18:31 +00:00
Cameron
f42e5049c9 satellite/satellitedb: add log line for node events insertions
Change-Id: I3271885cfd130128c240c9e9993e835fedb1ab7f
2022-11-03 16:09:23 -04:00
Jeremy Wharton
e34826c168 docs/blueprints: add automatic account freezing/unfreezing blueprint
This blueprint describes a way to allow user accounts to be
automatically restricted after failing to pay and unrestricted after
payment has been collected.

Change-Id: I4b64eb8ef6bf83603be7ad7d7bcb78f2d885c2a4
2022-11-03 16:56:28 +00:00
Egon Elbre
5ee27fc42a mod: bump storj.io/uplink
Change-Id: I9692c3c8dd266b29d23a782f6aff3594a6423de3
2022-11-03 14:06:17 +00:00
Erik van Velzen
337b72f310 satellite/metabase/rangedloop: uuid range pairs
Pair uuid's to create ranges. Will be used to parallelize the segment
loop.

Part of https://github.com/storj/storj/issues/5223

Change-Id: I73e2fb8a2cd379b840864449b6251b48feeb7b66
2022-11-03 11:15:56 +00:00
Cameron
68fe26ebe5 satelite/overlay: insert reputation events into node events
Insert reputation event into node events if reputation change occurs.

Change-Id: If1c5526092cb6834fe2faa6aa6e0306d4d88a4b7
2022-11-02 18:32:20 +00:00
Márton Elek
603bc019e1 build: use specific tag for nightly deployments
This patch is required to fix the nightly deployment:

 * We need to use the exact docker image tag what we built earlier
 * Migration should be full instead of snapshot (snapshot couldn't update existing, but older dbs)

Change-Id: Id2a2070638072a7b0021326326b0d53533817168
2022-11-02 17:57:20 +00:00
Vitalii
7ac1e38e28 web/satellite: reworked new Add coupon modal to use common VModal component
Reworked new Add coupon modal to use common VModal component to improve UX.

Issue:
https://github.com/storj/storj/issues/5104

Change-Id: Idd186ffd30a529761d15052bee4fac7cee4f5814
2022-11-02 19:16:25 +02:00
Vitalii
7f285c2eee web/satellite: reworked old add coupon code modal to use common VModal component
Reworked old Add coupon code modal to use common VModal component to improve UX.

Issue:
https://github.com/storj/storj/issues/5104

Change-Id: I4e1b38cfa2a88bacb4bcc58b837bc5b4319aca3d
2022-11-02 19:04:49 +02:00
Vitalii
db864a8605 web/satellite: reworked delete bucket modal to use common VModal component
Reworked delete bucket modal to use common VModal component to improve UX.

Issue:
https://github.com/storj/storj/issues/5104

Change-Id: I74acd07a0cb3f7e0231f88bf1255de9ac000ace5
2022-11-02 19:04:49 +02:00
Vitalii
8152920a51 web/satellite: reworked share object modal to use common VModal component
Reworked share object modal to use common VModal component to improve UX.

ISSUE:
https://github.com/storj/storj/issues/5104

Change-Id: Ie07550bb3e116651b08721298994eb0c867cf6f4
2022-11-02 19:04:49 +02:00
Cameron
865974950d satellite/overlay: insert node online events into node events table
Instead of sending emails at the time the node is seen to be back
online, we have decided to send the event to the node events table,
which will initiate the email sending process at some point.

Change-Id: Id756209498112579de8e78ee20ad2df54571a617
2022-11-02 16:26:19 +00:00
Cameron
f06da25c3d satellite/overlay: add nodeevents.DB to satellite overlay service
Add nodeevents.DB to satellite overlay service so we can insert node
events into the nodeevents DB.

Change-Id: I642c0ccc9941ecdb08cb22d5c8cf701959a55156
2022-11-02 15:56:37 +00:00
Michal Niewrzal
d21bbab2b2 satellite: fix metabase configuration wiring
New flag 'MultipleVersions' was not correctly passed from metainfo
configuration to metabase configuration. Because configuration was
set correctly for unit tests we didn't catch it and issue was found
while testing on QA satellite.

This change reduce number of places where new metabase flags needs
to be propagated from metainfo configuration to avoid problems with
setting new flags in the future.

Fixes https://github.com/storj/storj/issues/5274

Change-Id: I74bc122649febefd87f665be2fba628f6bfd9044
2022-11-02 15:17:34 +00:00
Egon Elbre
947cdf0fb4 ci: fixes for new Go version
We need to make exceptions for older uplink versions, because it does
not compile with newer Go versions due to quic dependency.

Change-Id: I3e073694f0942029c56740f0689088058ee068c3
2022-11-02 15:17:46 +02:00
Qweder93
5e5d6ecf6c satellite/{accounting, buckets}: added feature that allows to use custom SQL query instead of object loop for buckets accounting
since amount of objects is growing and looping through all of them
starts taking lot of time, we are switching for SQL query to do it
in chunks of tallies per bucket. 2nd part of issue fix.

Closes https://github.com/storj/team-metainfo/issues/125

Change-Id: Ia26bcac0a7e2c6503df9ebbf4817a636841d3284
2022-11-02 09:01:33 +00:00
Ethan
9a09d8920e satellite/gc: Upload bloomfilters with prefix and update LATEST when complete
Change he bloomfilter generation process to prefix the objects with a date and update the LATEST object with the prefix.  The sender will read the LATEST file to get the prefix to process.

Change-Id: Iae0d3c49015d57f391d87789fb799a7d774066bf
2022-11-01 21:24:46 +00:00
Ethan
4efde65c9e satellite/gc: Optionally run the GC bloomfilter process once, instead of in a loop
The current deployment strategy requires that the GC bloomfilter generation process executes only once and exits.

Change-Id: I952991f126596aa165d1f2e9fce6f8548c21bdba
2022-11-01 18:19:40 +00:00
Wilfred Asomani
4edef9e05c web/satellite: add AB test for new upgrade banner
This change adds AB testing for a new upgrade banner and sends a hit event when "Upgrade" is clicked.

Change-Id: Ie463e224af5b0cb74a601b68eedb2b34f9089fd7
2022-11-01 16:39:52 +00:00
Cameron
11e229cc7f satellite/nodeevents: implement node events DB
Implement node events DB with Insert and GetLatestByEmailAndEvent. Get
was changed to GetLatestByEmailAndEvent so we can verify items are being
inserted into the table without needing the ID, which is not available
to us in the tests.

Change-Id: I4abe63631c44774cd7e795fbab0cbab4d801db4c
2022-11-01 16:03:14 +00:00
Egon Elbre
4b05beb3f5 cmd/satellite: add 'repair-segment' command
This mostly wiring up the necessary systems.

Change-Id: I9e03b3240235ca8e4cc51ddf6a41e27d9dbc198c
2022-11-01 14:31:39 +00:00
Erik van Velzen
e09c8198d5 web/satellite/static/emails: set dcs support address
Helene: support@storj.io is for node operators. Support for DCS accounts
should go to supportdcs@storj.io.

Fixes https://github.com/storj/customer-issues/issues/354

Change-Id: Icfd437004defcee6d49f19e58bdea16e7e99b7d8
2022-11-01 00:06:24 +00:00
Lizzy Thomson
23917e51f2 satellite: invalid coupon code returns 400 status code
fixes: https://github.com/storj/storj/issues/4782

Change-Id: Idcc00202ed91a10c280f706e14d4901bf2f61541
2022-10-31 13:43:09 -06:00
Cameron
f76abb3b3d satellite/satellitedb: add dbx read method to node_events
Change-Id: I9cfcb467051921a9b54370e426e618a76ac4197e
2022-10-31 16:31:31 +00:00