Commit Graph

99 Commits

Author SHA1 Message Date
Márton Elek
afa5c54a35 satellite/satellitedb: add placement column to repair_queue
It makes it possible to run dedicated repair worker for different placement definitions.

https://github.com/storj/storj-private/issues/400

Change-Id: I376da867da5dbb4ab392d5f86c766f7543c32ee6
2023-09-12 11:29:18 +00:00
Márton Elek
500b6244f8
satellite/satellitedb: create table for node tags
Change-Id: I884bb740974e6b8241aa6b85faf266b85fe892d4
2023-07-05 09:38:53 +02:00
Jeremy Wharton
bb620e746b satellite/satellitedb: add index for project_members project_id column
An index has been added on the project_id column of the project_members
satellite database table so that we can retrieve members of a project
without performing a full table scan.

References #5855

Change-Id: I1cc30686f836c8fd1aa319247ce857a2392e7a52
2023-06-11 18:12:33 -05:00
Márton Elek
5c26661cd3 dbx/satellitedb: introduce new default_placement fields for users/projects
https://github.com/storj/storj/issues/5879

Change-Id: Ie6d50fd94c09f43c12f895444ed2a86f0820e01c
2023-06-08 10:02:10 +00:00
JT Olio
1437257dbf satellite: save and return which node features are enabled
current feature is if tcp fastopen was successfully enabled

Change-Id: Ide251863a9790b0fbebdf2e82dfd2afa8f25c408
2023-06-06 21:13:29 +00:00
Moby von Briesen
563239d555 satellite/satellitedb: Add value_attributions.partner_id back
There were some dbx-generated queries on the old satellite version that
still reference this column, so we need to add it back until the next
version.

Change-Id: I78b19336d9ca0384936d6cc11f5c50e579b4f2ab
2023-05-31 17:31:33 -04:00
Moby von Briesen
919278dbf9 satellite/satellitedb: drop partnerID columns
The last code referencing these columns was removed as of satellite
release v1.79, so it is safe to remove them now.

Resolves https://github.com/storj/storj/issues/5432

Change-Id: I2e9d641b2511a61e0b9482ef0f4955a73c290709
2023-05-24 14:38:57 +00:00
Jeremy Wharton
607b120116 satellite/satellitedb: add inviter ID column to project invites table
The project member invitations table has been modified to contain a
column for the ID of the user who sent the invitation. This ID is
required for us to return information about the inviter to the
satellite frontend.

References #5855

Change-Id: I928d987a8db2340f731ca65ce30173d4f90a9837
2023-05-17 15:46:40 +00:00
Michal Niewrzal
fa16d947c9 satellite/satellitedb: add index on owner_id column for projects table
We have method which is getting projects owned by specific user but it's
causing full table scan because we don't have index on owner_id column.

Change-Id: Icb71c9ac5b73104a52241ed8ba126c995c10811f
2023-05-09 08:57:32 +00:00
Moby von Briesen
432b12d1c4 satellite/satellitedb: make value_attributions.partner_id nullable
There is still a reference to partner_id in a query, which we cannot
move until dropping the "not null" constraint for it. This change adds
that migration.

Related to https://github.com/storj/storj/issues/5432

Change-Id: I98802a6e1bd59f3d9214de3db6688d9daf664a70
2023-05-05 23:42:50 +00:00
Jeremy Wharton
defb9eae82 satellite/satellitedb: add table for project invitations
A table for storing pending project member invitations has been added.
This table is required to satisfy our new project invitation UX revamp.

References #5766

Change-Id: I6f948de66ed5b4dc81532564958ff7f48533cad2
2023-04-26 11:26:12 -05:00
Cameron
549732228d satellite/satellitedb: add package info to customers db
Add columns package_plan and purchased_package_at to stripe_customers
table and add methods to update and select these values from console
service and payments accounts.

Change-Id: I1e89909055cc3054bfb7baa33c9dca3dfdc7336e
2023-03-31 15:51:30 +00:00
Wilfred Asomani
7c68b51c82 satellite/{console,db}: add onboardings columns
This change adds onboarding_start, onboarding_end and onboarding_step
columns to the user_settings table. the first two are used to determine
if a user should go through onboarding, the last will be used to as the
step a user got to before exiting onboarding without finishing.

Issue: https://github.com/storj/storj/issues/5660

Change-Id: I8070c880d0d2fc22086f24087c962f57c695cc50
2023-03-23 17:50:31 +00:00
JT Olio
2a63225b98 satellite/{contact,satellitedb}: preserve node message debounce support
Change-Id: I453ad35fda4e61d068db0c476dd86b50d7f2d705
2023-03-20 16:13:06 +00:00
Moby von Briesen
0980a2f9ce satellitedb: Add index to users table
Add a combined index on normalized_email,status to improve performance of
common "get user" query used for the satellite UI.

Change-Id: I24a20d7826e0a68a68c2f95b5847eb819921e7c0
2023-03-16 09:08:49 -04:00
Michal Niewrzal
d54ccfa92b satellite/satellitedb: change bucket_bandwidth_rollups primary key
In the past we set incorrect primary key for bucket_bandwidth_rollups
table and bucket name was first column of key instead of project id.
This change fixes this problem.

We will alter primary key manually for production satellites. That's
why migration code is checking if change was already applied.

https://github.com/storj/storj/issues/5332

Change-Id: I7dd555eb0c2f6fda1c90eaed08fa847b7f273bc7
2023-03-03 16:25:35 +00:00
Lizzy Thomson
54f0f47839 satellite/satellitedb: add passphrase_prompt column to user_settings
Add passphrase_prompt column to user_settings table to store a boolean
whether a user would like to be prompted to enter a passphrase when
entering a new project.

Issue https://github.com/storj/storj/issues/5613

Change-Id: I5d086df277aff458453343c7c38a379e7b1b8bf9
2023-03-01 14:53:18 -07:00
Lizzy Thomson
f733cb255b satellite/satellitedb: drop unused last_verification_reminder column
(updated to migration step 225) remove users table column
last_verification_reminder and update migrate_test.go

Issue: https://github.com/storj/storj/issues/4560

Change-Id: Icbd8b70a8243b75a11c481a3b87df0ebe9402c5f
2023-02-08 16:23:09 +00:00
Jeremy Wharton
49916b02f0 satellite/{console,satellitedb}: migration, methods for user settings
This change adds a database migration for a new table that stores
configurations for a user, the first of which is the session duration.
Database methods are implemented to interact with this table.

Resolves #5472

Change-Id: I01049265f385ea5de65907da1bc3bcf426d3c577
2023-02-07 21:48:29 +00:00
Michal Niewrzal
0b519fbd05 satellite/satellitedb: create index for interval_day for project_bandwidth_daily_rollup
To improve deletion of old entries in project_bandwidth_daily_rollup
we need index on `interval_day` column which is used to find those old
entries.

As an addition we are changing interval how often deletion is executed
from 7 to 1 day. We would like to have smaller portion of data to
delete.

Fixes https://github.com/storj/storj/issues/5465

Change-Id: Ie18ebe859887b93d6e4e6065a61fb9214c7ad27a
2023-02-03 10:02:58 +00:00
JT Olio
2753d5a32f satellite/overlay: keep track of noise info per node
Change-Id: Icef04c3e87dbf4bb57d3837274c323bf6dd2c81f
2023-02-01 23:03:35 -05:00
Michal Niewrzal
2876d5f20b satellite/satellitedb: drop project_bandwidth_rollups table
We are not using this table.

Change-Id: I3e4d5d0c2d8522b319caf48fea460b1cc35d5630
2023-01-12 10:23:48 +00:00
Moby von Briesen
e26654c2a9 satellite/satellitedb: Drop unused tables
The `coupons`, `offers`, `coupon_usages`, `coupon_codes`, and
`user_credits` tables are not used anymore. They exist due to
legacy billing code which has already been removed.

Change-Id: Ie93af90a06e5247a74015af2a78a223d69249565
2023-01-10 17:00:15 +00:00
Cameron
021c98c17a satellite/satellitedb: create table account_freeze_events
create table for account freeze events

github issue: https://github.com/storj/storj/issues/5396

Change-Id: Ia4f93ce5caacd02b161853e3261d237bc715f23d
2022-12-13 19:09:25 +00:00
Cameron
602f0cc132 satellite/satellitedb: add node events column last_attempted
Also add a method to read a node event by id.

Change-Id: I08ca010e18a08253015e0e1280f791d0cc04d239
2022-12-01 18:35:06 +00:00
Cameron
5f6b4900d2 satellite/satellitedb: add columns last_offline_email and last_software_update_email
Change-Id: I8b5023bbbba55de633ce5ac3faf3361352c202ef
2022-11-11 14:43:59 -05:00
Fadila Khadar
553c1558a5 satellite/satellitedb: add 'contained' column to nodes table
We want to be able to exclude contained nodes from nodes selection. For this, we add a 'contained' column to the nodes table to track the containment status.

Fixes https://github.com/storj/storj/issues/5231

Change-Id: Id78e645f172145adcb8664646e8ebf14e218b57b
2022-11-09 11:11:37 -06:00
paul cannon
1b5d953fd9 satellite/satellitedb: add queue for verifications
Since the auditor will be moving to a different process from the
metainfo loop, we need a way of communicating which segments have
been chosen for audit. This queue will be that communication, for now.

Contrast this with the queue for _re_verifications in commit 9c67f62f.

Refs: https://github.com/storj/storj/issues/5251
Change-Id: I9a269c7ef21e6c5e9c6e5e1f3db298fe159a8a79
2022-11-09 15:38:56 +00:00
Cameron
80bfa55b8e satellite/satellitedb: edit node_events migration to have different PK
Change node_events schema to use an id column as primary key rather than
node id because there can be multiple events per node id.

Change-Id: I518d8ef9ea658764876483e282a4058d3c4910f4
2022-10-28 22:59:28 +00:00
Cameron
6bf54bfaef satellite/satellitedb: add new table node_events
Add new table for node events. We can use this to notify node
operators of certain node events. Further, we can squash events for
multiple nodes with the same email into a single notification.

Change-Id: Icea6dd939df8fe4a98806bd79c014e21d239c43e
2022-10-28 19:16:25 +00:00
paul cannon
9c67f62fe3 satellite/satellitedb: add table for reverify queue
This table will be used as a queue for pieces that need to be reverified
(a regular audit timed out on the owning node, so now that node is
contained and we need to validate the piece before un-containing it).

Refs: https://github.com/storj/storj/issues/5228

Change-Id: I5dcd26b6adced8674cbd81884c1543a61ea9d4c8
2022-10-27 15:28:47 +00:00
Lizzy Thomson
ef04eb8bea satellite/satellitedb: add columns in projects table
add user_specified_usage_limit and user_specified_bandwidth_limit columns in projects table

github issue https://github.com/storj/storj/issues/5185

Change-Id: Ia8c9b190a2bdb684ac9d43af8f2159027d65aa88
2022-10-06 18:20:39 -06:00
paul cannon
a57cbc4e0b satellite/satellitedb: drop _gob columns
This concludes the saga that began with commit c053bdbd, migrating away
from using gob-encoded big.Float values in the database to using
integers with an implied number of decimal places.

Nothing is using or accessing or expecting the _gob columns to exist
anymore. The transition code and the migration chore are gone. All that
remains is to drop the columns.

Change-Id: I9b15ee52f7781510a6dc91cf7c54f7f9022b1210
2022-08-26 19:45:22 +00:00
Wilfred Asomani
c59d0d415c satellitedb: add signup_captcha column to users table
The signup_captcha column will hold the captcha scores of new users.

see: https://github.com/storj/storj/issues/5067

Change-Id: Ia322af29a3b5b019b417843272506a3dbd1397e4
2022-08-17 14:31:52 -05:00
paul cannon
0dcc0a9ee0 satellite/reputation: reconfigure lambda and alpha
This is in response to community feedback that our existing reputation
calculation is too likely to disqualify storage nodes unfairly with
extreme swings up and down.

For details and analysis, please see the data_loss_vs_dq_chance_sim.py
tool, the "tuning reputation further.ipynb" Jupyter notebook in the
storj/datascience repository, and the discussion at

    https://forum.storj.io/t/tuning-audit-scoring/14084

In brief: changing the lambda and initial-alpha parameters in this way
causes the swings in reputation to be smaller and less likely to put a
node past the disqualification threshold unfairly.

Note: this change will cause a one-time reset of all (non-disqualified)
node reputations, because the new initial alpha value of 1000 is
dramatically different, and the disqualification threshold is going to
be much higher.

Change-Id: Id6dc4ba8fde1be3db4255b72282207bab5491ca3
2022-08-17 18:52:53 +00:00
dlamarmorgan
8eec61d315 satellitedb: add timestamp index to billing table
Adding an index to the timestamp field of the billing transaction table to improve query performance. This should prevent having to do a full table scan when we query for the last billing transaction of a particular source and/or type.

Change-Id: I581f09494cc8662a12efba4302022a07121ba309
2022-08-16 22:50:02 +00:00
dlamarmorgan
ac241501ca satellitedb: add wallet address index to storjscan_wallet table
Adding an index to the wallet address field of the storjscan wallet table to improve query performance. This should prevent having to do a full table scan when we query for one or more wallet addresses for a given user in our queries.

Change-Id: Ic1b5d06c2258489e5464d186fed5270172f8cba5
2022-08-15 18:05:06 +00:00
kimbotsao
c3e7b1409d satellitedb: add salt to projects table
Change-Id: I53691849efc169c495566648c12ae3d639614c3d
2022-08-11 12:16:53 -04:00
dlamarmorgan
c8b72c8f71 satellite/{payments/billing,satellitedb}: refactor billing DB
Update the billing table to use generated IDs, to include the source and status fields, and to add metadata jsonb field that can be used for fields specific to a particular type of billing transaction. Additionally a new table was added to keep track of user balances

Change-Id: Ieb3a63aafd8fe21fc3386bafd43d52081b7d2838

satellite/{payments/billing,satellitedb}: refactor billing DB

Update the billing table to use generated IDs, to include the source and status fields, and to add metadata jsonb field that can be used for fields specific to a particular type of billing transaction. Additionally a new table was added to keep track of user balances

Change-Id: Ieb3a63aafd8fe21fc3386bafd43d52081b7d2838
2022-08-03 17:52:32 +00:00
Clement Sam
f2f046af7a satellite/satellitedb: add interval_end_time column to accounting_rollups table
Adding an interval_end_time column to the accounting_rollups table
to keep the last interval_end_time for each daily storage tallies.

Updates https://github.com/storj/storj/issues/4178

Change-Id: If7a8210c5e9fe2fc9df84b137a8b6e3db2471c58
2022-07-27 17:53:27 +00:00
Cameron
023a1c6271 satellite/satellitedb: add public_id column and index to projects table
github issue: https://github.com/storj/storj/issues/4861

Change-Id: I3d1928ca51c825a4c1589db5ae55902e57f00c3b
2022-07-05 18:39:29 +00:00
Yaroslav Vorobiov
da97fafb0c satellite/payments/storjscan: add payments DB
Change-Id: Ia6f0f62912c9a60d2c513459959634179eb9506c
2022-06-10 13:44:27 +01:00
dlamarmorgan
007d4190c2 satellite/{payments/billing,satellitedb}: Add payment billing DB
Add billing DB to the satellite. This DB will hold all transactions on the users account and can be used to compute the users current usable account balance.

Change-Id: I056416efc169e5e5e30c9f30cd8bc766b7bc8073
2022-05-27 08:56:31 +00:00
Márton Elek
5e8266d1de satellite/{payments/storjscan,satellitedb}: Add storjscan wallet DB
Add storjscan wallets DB to the satellite. For now this DB is a one to one mapping of the users account to a storjscan wallet that can be used by the account holder to make payments on their Storj account.

relates to https://github.com/storj/storj/issues/4347

Change-Id: I6e65b15817b90ceb75641244f9bf173c3b4228a7
2022-05-24 08:42:02 +00:00
paul cannon
aa728bd6ea satellite/satellitedb: add reputations.disqualification_reason
We added nodes.disqualification_reason recently, but we didn't add a
corresponding column in the reputations table (despite having a
corresponding `disqualified` column there).

Without this change, the (very useful and informative) assignments to
updateFields.DisqualificationReason in reputations.go have no effect.

Refs: https://github.com/storj/storj/issues/4601

Change-Id: I77404902ca64b56aed72f1de76b303fe82b76aab
2022-05-17 10:09:36 -05:00
Cameron
bfad64a6f0 satellite/satellitedb: add verification_reminders column to users table
When a new user registers, we send a verification request to their
email. Currently, if they do not verify their email, we take no further
action. We want to send these users reminders: one after about one day
and one after about 5 days. To do this we will use this new
verification_reminders column.

It will look something like this:

```
SELECT email FROM users
WHERE status = 0
   AND (
          (verification_reminders = 0 AND created_at < now() - 'INTERVAL 1d')
          OR (verification_reminders = 1 AND created_at < now() - 'INTERVAL 5d')
   )
```

Change-Id: If0620e08c97e9e337c9563481d665c5bd462693b
2022-05-16 16:37:16 +00:00
Moby von Briesen
24bbb7f236 satellite/satellitedb: Add webapp sessions table
Enables us to better manage sessions for satellite UI users.

Change-Id: I6695f8f1174d2e5fb970efddb0a8b14725e2b57e
2022-05-04 20:37:56 +00:00
Yaroslav Vorobiov
553ef9d640 satellite/satellitedb: suspended column removal migration
Last part of backwards compatible db migration to remove "suspended" column.
Removes exeption which removes "suspended" column in tests from `migrate_test.go`.
Adds DB migration to remove "suspended" column from 'nodes' and 'reputations' tables.

Change-Id: I02051279f6f4181e966c567919af0e774583f165
2022-04-20 17:59:11 +00:00
Vitalii
513ad99756 satellite/satellitedb: migrate wrongly affected users to have default limits
Migrate free tier users to have default limits if their limits were set to 0.
They were affected by incorrect working of Update user query.

Change-Id: I4c49c8d99b12dba2b9b0ab61b2175085976dcc95
2022-04-13 13:33:03 +00:00
Vitalii Shpital
b4cec4fd0b satellite/satellitedb: add columns to users table to control failed login attempts
Added failed_login_count and login_lockout_expiration columns to users table to control users failed login attempts.
We want to prevent brute forcing of user login so this is the first step.

Change-Id: I06b0b9f5415a1922e08cd9908893b2fd3c26bca0
2022-04-12 08:37:07 +00:00