Commit Graph

23 Commits

Author SHA1 Message Date
Yingrong Zhao
c8f4ff4135 satellite/satellitedb: migrate existing data from overlaycache and
audit_histories to reputations table

Change-Id: Ia47e5264b37d2b7585c8486a4e022f4559233bf0
2021-07-28 15:16:52 -04:00
Jeremy Wharton
0d8010e353 satellite/satellitedb: Implement multi-factor authentication db
Columns for MFA status, secret key, and JSON-encoded array of
recovery codes are added to the users table.

Change-Id: Ifed7e50ec9767c1670d9682df1575678984daa60
2021-07-13 18:18:36 -05:00
Michał Niewrzał
0ca7583282 satellite/accounting: add total for bytes and segments to tallies
We want to calculate bucket tally only from iterating objects.
Object currently has an info about totals for bytes and segments.
We need to adjust tallies to keep those totals. Older entries will
be untouched and code will use totals only if available. Change
is adding columns for totals to bucket_storage_tally table and
is adding general handling for them.

Next step is to start using total columns instead of inline/remote.
This will be done with next change.

Change-Id: I37fed1b327789efcf1d0570318aee3045db17fad
2021-07-01 08:52:32 +00:00
Michał Niewrzał
d53aacc058 satellite/repair: migrate to new repair_queue table
We want to use StreamID/Position to identify injured
segment. As it is hard to alter existing injuredsegments
table we are adding a new table that will replace existing
one. Old table will be dropped later.

Change-Id: I0d3b06522645013178b6678c19378ebafe485c49
2021-06-30 17:12:24 +02:00
Moby von Briesen
0ec3867ec0 satellitedb: add paid_tier column to users table
So that we can easily see whether a user is in the paid tier without
querying for payment methods.

Change-Id: I122566ddd0953203f852741fa12c71795bc1ec5c
2021-06-30 15:00:38 +00:00
Michał Niewrzał
8ce619706b satellite/audit: migrate to new segment_pending_audit table
Currently, pending audit is finding segment by segment location
(path) because we want to move audit to segmentloop and we will
have only StreamID and Position we need to add columns for those
fields. Altering existing table can cause issues while
migration and deployment. Cleaner choise is to make new table.
This change contains migration with new segment_pending_audit
table that will replace pending_audits table and adjustments
to use new table in the code.

Table pending_audits will be dropped with next release.

Change-Id: Id507e29c152da594bac1fd812c78d7ecf45ec51f
2021-06-28 13:19:49 +02:00
Fadila Khadar
a89c0763a0 satellite/gracefulexit: create table graceful_exit_segment_transfer_queue
table graceful_exit_segment_transfer_queue will be used to replace graceful_exit_transfer_queue. Currently, it uses the path of a segment to keep track of pieces to be transferred. As we want to use the segment metainfo loop, we will need to record stream_id and position of the segment instead of relying on object path.

This change also add a uses_segment_transfer_queue column to the graceful_exit_progress table to be able to know if a transfer has been initiated while using the old table.

Change-Id: Iafb1e8e65ba124e20de4a9ff76da181c3222de7e
2021-06-28 12:55:57 +02:00
Jeremy Wharton
07f8cff08c satellite/satellitedb: Add reputation table
The reputation table duplicates the reputation information in the
nodes table. It will be used for implementing the reputation
service.

Change-Id: I36c0318e8fa5f535e9d527df95b22a4f9eb365d4
2021-06-23 13:26:57 +00:00
Fadila Khadar
1d405f45ea satellite/orders: add egress_dead to project_bandwidth_daily_rollups
Migration step for adding a 'egress_dead' column to the project_bandwidth_daily_rollups.
It will be used to track bandwidth allocation that won't be consumned
as the corresponding order has already been processed and has a settled
bandwidth amount lower than the order limit (allocated bandwidth).

Change-Id: Ic07592e69292ae2076e69f6038bb0e0fae79b271
2021-06-03 16:05:56 +00:00
Moby von Briesen
f536ee3aaf satellite/satellitedb: Migrate non-expiring coupons to expire
Because of recent changes to how coupons for the free tier are handled
(see commit 4c0817bcfb), we no longer want all these $10
non-expiring coupons. After coupons are applied during invoice
generation, if a customer does not have any valid (non expired, non
consumed) coupons, a new promotional coupon is applied.

We could just wait for users to consume all $10 of the non-expiring
coupons, and the new promotional coupon would be applied for the
following billing cycle, but this gets tricky, because if in the final
month, the user is billed for $1 of usage, but only $0.5 of the $10
non-expiring coupon is remaining, the user will be charged for the
remaining $0.5. With the new promotional coupon of $1.65, expiring every
month, this would not be an issue.

So long story short, this commit migrates all non-expiring coupons to
expire within 2 billing periods (all existing non-expiring coupons in
prod were created in early April or later). That way, there is still
enough value in the $10 coupon that we don't have to worry about
customers exceeding it, and the coupons will expire. Then we'll
immediately apply the $1.65 coupon for the next month! And then
hopefully this unfortunate situation will come to a pleasant end.

Change-Id: I8a593948d8876c41a71d886b9a95d4e2c802b4f3
2021-05-25 19:25:02 +00:00
Fadila Khadar
16022105f1 satellite/orders: create project_bandwidth_daily_rollups table
Change-Id: I8f101c0714b5673450d4f2f01e297a9eddc6caf6
2021-05-25 09:05:47 +00:00
Cameron Ayer
be87c80fe1 satellite/satellitedb: drop columns total_uptime_count and uptime_success_count
These columns on the nodes table are not referenced anywhere and are
ready to be dropped

Change-Id: I3f3652f635108419e93ced12b0ac1e58bba27a32
2021-05-14 11:49:12 -04:00
Ivan Fraixedes
5f29a20937 satellite/satellitedb: Add missing indexes
The DBs of our production satellites have some indexes that we didn't
have in the migrations because at that time we weren't able to add them
because our migration test was not able to deal with Cockroach indexes
with the STORING clause.
We have recently modified the storj.io/private/dbutil/pgutil package to
support the CRDB STRORING clause, so we are adding the missing indexes
to our migrations for being able to have them if we have to recover a DB
from scratch or we deploy a new DB satellite.

Change-Id: I686ff84e5b4c02d9615f50fa531261363affefb8
2021-04-29 15:34:46 +00:00
Malcolm Bouzi
c1fdf1ad71 satellite/satellitedb: add have_sales_contact column on users
For business accounts we need to track the sales contact.
It will be a question to business accounts during onboarding.

Change-Id: I8d101ce1b52091478dfb0ddd875e1cc717d765d3
2021-04-26 13:37:26 +00:00
Moby von Briesen
04294e3e1c satellite/payments: Update coupon defaults for free tier
The new default promotional coupon is $10/month, and doesn't expire.

This change also migrates the coupon.duration column over to the new
coupon.billing_periods, and switches to rely completely on
billing_periods.

Change-Id: Ic3341e9fa4040449bab5e66ca4ee2640b095cf3d
2021-04-05 10:12:26 -04:00
Moby von Briesen
c334fd090e satellite/payments: Convert coupon duration to be nullable
* Add a nullable billing_periods column in the coupons table
* Add nullable billing_periods column to the currently unused
coupon_codes table
* Drop the duration column from the coupon_codes table
* Replace duration config type so that the default promotional coupon
can be configured to never expire

Zero downtime migration plan:
* Add billing_periods column to coupons and coupon_codes tables (this change)
* After one release, remove all references to the old duration column,
replacing with references to billing_periods. At this point, we can also
change the defult promotional coupon to never expire and migrate over
values from the old duration column.
* After another release, drop the duration column.

Change-Id: I374e8dc9fab9f81b4a5bc681771955662d4c007a
2021-04-01 16:28:23 +02:00
Cameron Ayer
05f8d2d0b1 satellite/satellitedb: filter offline suspended nodes from selection
Change-Id: I5a6f413453332238d579a7bf50eb30e9156f96c2
2021-03-27 23:36:46 +00:00
Moby von Briesen
27bcb46718 satellite/console: change default project and usage limits
This is one step for implementing the free tier:
* Change the default project limit from 10 to 3
* Move storage and bandwidth project usage limits from the metainfo
package to the console package (otherwise there is a cyclical
dependency, and metainfo doesn't use these values anyway)
* Change the default storage usage limit per project from 500gb to 50gb
* Change the default bandwidth usage limit per project from 500gb to 50gb
* Migrate the database so that old users and projects continue to have
the old defaults (10 projects/500gb usage)

Change-Id: Ice9ee6a738bc6410da18c336c672d3fcd0cab1b9
2021-03-24 14:24:27 +00:00
Cameron Ayer
37602d86d3 satellite/satellitedb: drop columns uptime_reputation_alpha and uptime_reputation_beta
Change-Id: I1e0c5f2f96a920070afb76851da98208654b02cf
2021-03-22 14:33:17 -04:00
Moby von Briesen
78e132cb74 satellite/satellitedb: Add coupon_codes table and allow optional link to coupons table
The coupon_codes table will allow for administrators to create new promo
codes associated with coupon information (amount, duration, etc...).

A user will be able to enter a promo code (aka coupon code) in order to
apply a new coupon to their account. The coupon in the coupons table is
linked to the template defined in the coupon_codes table.

Change-Id: I50e49fa92afbc6aa9d01d8a895c069efb59e472b
2021-03-19 20:46:28 +00:00
Cameron Ayer
8138939a2a satellite/satellitedb: remove bad migration step
Migration step 148 will cause errors because we missed some
references to the columns being dropped. Removing the step
altogether causes problems with backwards compatibility tests
because the change already exists in the latest release tag.
To circumvent, we change v148 to an empty migration.

Add methods FindTable and RemoveColumn in private/dbutil/dbschema

Change-Id: Ia527e95b88a88c5dc82800928ce6f8cfb879e334
2021-03-10 14:36:52 -05:00
Cameron Ayer
411a7ad0bc satellite/satellitedb: drop uptime_reputation_alpha and uptime_reputation_beta from nodes table
Change-Id: Ib46e783bf1a5c036394b4cac281382d0380bb1be
2021-03-01 15:30:51 +00:00
Egon Elbre
32afbd0d15 satellite/satellitedb: faster test database setup
Pregenerate the database schema we should use for most tests.
Currently, Cockroach is slow with regards to migration and it's
better if it happens in as few transactions as possible.

This reduces test time from ~21min to ~15min.

Change-Id: Ife8117053e6b9ecf3c93fe63677edf15d4d7c254
2021-02-22 21:13:00 +02:00