Commit Graph

1665 Commits

Author SHA1 Message Date
Moby von Briesen
78e132cb74 satellite/satellitedb: Add coupon_codes table and allow optional link to coupons table
The coupon_codes table will allow for administrators to create new promo
codes associated with coupon information (amount, duration, etc...).

A user will be able to enter a promo code (aka coupon code) in order to
apply a new coupon to their account. The coupon in the coupons table is
linked to the template defined in the coupon_codes table.

Change-Id: I50e49fa92afbc6aa9d01d8a895c069efb59e472b
2021-03-19 20:46:28 +00:00
Vitalii Shpital
c3ae122aa7 web/satellite: enter passphrase step for objects page
WHAT:
enter passphrase step for users who has already created passphrase

WHY:
to let users proceed to upload step

Change-Id: I084aec5b863981978cf190f99ee95154fbed9aab
2021-03-19 10:59:32 +00:00
Cameron Ayer
a04495713d satellite/audit: add missing logs for audit failure conditions
Among other conditions, nodes fail audits by returning incorrect
data and by reaching the max reverify count, but we weren't logging
these events. This commit adds the missing logs.

Change-Id: I80749a7e95e8cb97bc8dd7dac1e523e223114b7f
2021-03-18 17:33:11 +00:00
Ivan Fraixedes
4c1098e571 Redis: Update Redis package to last major version
Update the Redis dependency to use the last major production version.
The last version accepts a context parameter in all the network methods
so it allows us to pass it through them.

Change-Id: I34121b2ec3c2728602115c724933ad24c9e6e4fd
2021-03-18 14:19:49 +00:00
Egon Elbre
9491df76e7 satellite/satellitedb: optimize StoragenodeAccounting.SaveRollup
Change-Id: I758049872b4ea1ae22657dd9fcb47c228468b1d0
2021-03-16 18:11:40 +02:00
Vitalii Shpital
6a553ec9c5 web/satellite: change banner for beta satellites with URLs
WHAT:
beta satellite top banner's copy is changed to include support/feedback URLs

WHY:
so users using our beta satellite will be able to report feedback somewhere

Change-Id: Ibc349c8b3354b577275fcf1d2b75bfdd267729d9
2021-03-15 17:12:07 +00:00
Isaac Hess
1de6315b83 metainfo: Stub DownloadObject metainfo endpoint
This is the stub "not implemented" endpoint for the new DownloadObject
RPC.

Change-Id: Ifa724e20d89d839f2b6968d6283bc66a3bab8781
2021-03-15 10:52:57 -06:00
Cameron Ayer
8138939a2a satellite/satellitedb: remove bad migration step
Migration step 148 will cause errors because we missed some
references to the columns being dropped. Removing the step
altogether causes problems with backwards compatibility tests
because the change already exists in the latest release tag.
To circumvent, we change v148 to an empty migration.

Add methods FindTable and RemoveColumn in private/dbutil/dbschema

Change-Id: Ia527e95b88a88c5dc82800928ce6f8cfb879e334
2021-03-10 14:36:52 -05:00
Ivan Fraixedes
84b844a2a7 redis-server: Move testing type to specific testing pkg
Move a specific interface & types used for testing to be a private
subpackage with a name that clearly identifies it for testing purpose.

Change-Id: I646cf3b6f0a3b518a6f9a125998dc5a02df02db6
2021-03-10 06:09:46 +00:00
Ethan
7af245c3e7 satellite/payments: De-shadow customer page var in stripe mock
Fixes infinite loop when there are more records than the limit.

Change-Id: Ib61a8fcad6d08f4e0f1c5715d710acec607004b3
2021-03-03 20:51:43 -05:00
Jeff Wendling
376547c33c satellite/compensation: smaller txns for RecordPeriod
cockroach is having problems with huge transactions and
having them complete before timeouts or whatever, so
do smaller transactions.

because we can have partial recording of payments which
are not unique, we have to do a thing where we read and
check if it already exists before writing. this is not
concurrency safe.

Change-Id: Ia7d59499a43ce6d70cb2a23754edbdd1b643ef1a
2021-03-02 20:14:25 +00:00
Cameron Ayer
a44974a2f9 satellite/audit: fix pointless containment deletions
Previously if node was not found in containment, it was
given the status, 'skipped'.

We later try to delete skipped nodes from containment.

To fix this, add a new status called 'remove' to differentiate
nodes which should be skipped and nodes which should be deleted.

Change-Id: Ic09e62dc9723c89d0c9f968ce68c039114a9d74e
2021-03-02 13:40:18 -05:00
Natalie Villasana
c290e5ac9a satellite/orders: decrease FlushBatchSize default to 1000
The previous default FlushBatchSize of 10000 was causing major
slow down in select and insert statements on bucket_bandwidth_rollups.
We saw on the saltlake satellite that a FlushBatchSize of 1000 helped
reduce contention and query latency.

Change-Id: Ib95e73482219bc5aedc11925b1849fa5999774ba
2021-03-02 14:00:48 +00:00
Cameron Ayer
aeac6264cd sallite/satellitedb: add metric stray_nodes_dq_count
Add metric so we can see how many nodes are DQd due to
this.

Change-Id: Ie4bdd1375fb9bd948af14fed9a2962b783b6a526
2021-03-01 21:06:36 +00:00
Natalie Villasana
856db68fd9 satellite/gracefulexit: extend GE data cleanup to include exit_progress
The new 'consistency ge-cleanup-orphaned-data' cli command deleted
orphaned transfer queue items, but not entries in the
graceful_exit_progress table. This will delete orphaned entries
from the exit progress table too.

Change-Id: I5f927aac1f258490678deaf179be92ccfe10fcd8
2021-03-01 15:52:43 +00:00
Cameron Ayer
411a7ad0bc satellite/satellitedb: drop uptime_reputation_alpha and uptime_reputation_beta from nodes table
Change-Id: Ib46e783bf1a5c036394b4cac281382d0380bb1be
2021-03-01 15:30:51 +00:00
Michał Niewrzał
1af9400a23 satellite/satellitedb/dbx: remove unused methods
Turns out that many methods generated with dbx where not used at all.
Lets remove them.

As a next step we can think about dropping tables like:
* user_credit
* offer

Change-Id: Id6cda81a701348db2a6b8b26daa22ae9c4f87cb4
2021-02-25 13:50:50 +00:00
Vitalii Shpital
300e88f9a7 web/satellite: config flag for satellites in beta
WHAT:
config flag to indicate if satellite is in beta

WHY:
to avoid using hardcoded satellite names which may cause issues

Change-Id: If92eb7417c340bf343a9a91e2f6b11f0349020c5
2021-02-24 12:29:07 +02:00
Malcolm Bouzi
f3e75c754f satellite/console: add new professional tab fields to create user function in service
Change-Id: I746062ef6f938fa145a3eb29a491399c2cf35806
2021-02-23 18:25:49 +00:00
Egon Elbre
0178ec7771 satellite/satellitedb: clearer testdata version parsing
Change-Id: I93a23625b30472e826b962dab31ff3e1081b1b8f
2021-02-23 10:22:19 +02:00
Egon Elbre
32afbd0d15 satellite/satellitedb: faster test database setup
Pregenerate the database schema we should use for most tests.
Currently, Cockroach is slow with regards to migration and it's
better if it happens in as few transactions as possible.

This reduces test time from ~21min to ~15min.

Change-Id: Ife8117053e6b9ecf3c93fe63677edf15d4d7c254
2021-02-22 21:13:00 +02:00
Cameron Ayer
549033f2e6 satellite/satellitedb: don't include DQd and exited nodes in DQStrayNodes
Don't update DQ time of already DQd nodes. Don't DQ nodes who exited.

Change-Id: I4528a9ba9f8e278987165ad337a9b34dadb9788b
2021-02-19 15:12:30 -05:00
Egon Elbre
630718c392 satellite/gracefulexit: fix test DeleteAllFinishedTransferQueueItems
Using random piece num may generate the exact same key or piecenum.
Instead use fixed piecenum and key.

Change-Id: I54b7bc1a6698149bf99608dd46501ea963cec084
2021-02-19 18:15:22 +02:00
Egon Elbre
62a2d05ae6 satellite/accounting: try fix test ResetLimitsFirstDayOfNextMonth
TestProjectUsage_ResetLimitsFirstDayOfNextMonth seems to be flaky.
It doesn't seem to reproduce easily, however it should flush orders from
storagenodes to satellites, that way the orders chore flushing on the
satellite makes sense.

Change-Id: I586d9d8ce10b5f6320e79324f6128b4f8c2cac5f
2021-02-19 17:29:15 +02:00
Egon Elbre
1137620baf satellite/satellitedb: move tests to their domains
Testing interfaces is slightly clearer when it's in the package needing
the database rather than each individual implementation.

Change-Id: I10334c214a205f7e510b939b4359a2214c4e060a
2021-02-19 17:29:15 +02:00
JT Olio
b2ed7edd30 cmd/satellite: restore-trash parallel workers
Change-Id: Ic7466b21c20bda334e7ba4268a494e96b6528ac1
2021-02-18 19:11:19 +02:00
JT Olio
3ae3389ddc cmd/satellite: restore-trash command
Change-Id: I80fc932c12147692d49cde277784871ac611fcad
2021-02-18 09:19:22 -07:00
Michał Niewrzał
f1169ad7bd satellite/metainfo: return RS value with DownloadSegment response
Until now we where using single RS per object but it turns out that we
need to be able to support RS per segment. We need to give uplink such information while downloading.

Change-Id: I6565b7c08962b3a1429f6079e7c2023a0a7c8b72
2021-02-17 12:39:53 +01:00
Egon Elbre
f7ad86521e satellite/overlay: fix data race in TestAuditHistoryBasic
Change-Id: I196f10973fe10b10b226ac3a63e62bf4fe9c256b
2021-02-16 12:32:19 +02:00
Malcolm Bouzi
4e3ac2c490 satellite/console: add new professional fields to console api
Change-Id: Ib7a69e117c7cf7f5eacd9f4ffc2c79fc94f01811
2021-02-12 17:21:09 -05:00
Malcolm Bouzi
4b2e46a0c9 satellite/satellitedb: add employee size column to users
Change-Id: I21f5904331f0ceb92f494729c22a52c256a69163
2021-02-12 09:15:15 -05:00
Jeff Wendling
fe5b6e1725 sat/console/consolewasm: restrict enc access to api key
Change-Id: Ie263ffc7343e59ee9a90996b259bb04258a78f29
2021-02-11 17:52:32 -05:00
Michał Niewrzał
dc2bec9f89 satellite/metainfo: collect uplink versions with monkit
This change adds metrics for uplink version and method which was used by this version.

Change-Id: I05fc425a024805cfcd6d8add810a2a0749405f4f
2021-02-10 17:47:13 +01:00
Cameron Ayer
4a797baa73 satellite/repair/repairer: a new set of rs_scheme tagged metrics
Change-Id: Ibecd9265da881247eeb85ba185ee8877a7243777
2021-02-09 14:19:22 +00:00
Yaroslav Vorobiov
1e328f3c30 satellite/compensation: add wallet features to invoice csv
Change-Id: I90a72c0a4c8d07604096913b6680263b6defc0a2
2021-02-08 22:13:48 +02:00
Yaroslav Vorobiov
966535e9de {storagenode,satellite}/nodeoperator: add wallet features
Change-Id: Iac7eb40a52b8fddcc573aebaad2e3a30a10cded9
2021-02-08 22:09:45 +02:00
VitaliiShpital
fcd18ef48a web/satellite: rework signup verify redirect flow
WHAT:
people who sign up on US2 are not redirected to verifying page. From now on we have to set verify URL to make redirect happen

WHY:
user experience

Change-Id: I96c51a2c4f9cb6376cbfea639675b32918b58bee
2021-02-08 18:00:57 +00:00
Yingrong Zhao
89529237a2 web/satellite, web/marketing: remove referral program related UI code
Change-Id: Ia815a72c0d17760bb63d9a37bb18b88952209368
2021-02-08 11:07:35 -05:00
Yingrong Zhao
3b49d3cddf satellite: remove referral program related code
This PR removes all back-end related referral program code including the
marketing portal.

We will have a separate PR for front-end code and database migration to
drop `offers` and `usercredits` table

Change-Id: If59f952cddfe0558a7dc03a0eac7cc1081517f88
2021-02-08 13:52:50 +00:00
Jeff Wendling
e114cfe86d satellite/satellitedb: fix broken query
Change-Id: I6d412a673d75264bf9751c6f15b1fb0ab94e1394
2021-02-04 16:00:51 -05:00
Caleb Case
f011a5e8cc
Use WithNonce when creating caveats
This ensures the caveats are unique even when they contain the same
permissions and will result in unique macaroons. This is important to
ensure revocation doesn't impact more macaroons than intended.

Change-Id: I6354edd0119f2d85eaf580f2d1926a3de9151b88
2021-02-04 13:46:32 -05:00
Malcolm Bouzi
db3a3088f9
satellite/satellitedb: add professional user fields to db interface (#4034) 2021-02-04 10:00:15 -05:00
Egon Elbre
615586a471 {satellite,storagnode}/internalpb: bump gogo to v1.3.2
Change-Id: I585a3197630b9b675a4b540cf25612e1ed39b599
2021-02-04 11:09:13 +00:00
Yingrong Zhao
21dfd99883 bump storj.io/private to latest
Change-Id: If4f08cd8cd8cd7cb09cd7d7051d966b08fe96c2e
2021-02-04 00:54:28 +00:00
Ivan Fraixedes
9c9f481469 satellite/orders: Remove deprecated endpoint
Remove the orders Settlement endpoint because it isn't used and it was
already always returning an error.

Change-Id: I81486fbe7044a1444182173bc0693698ee7cfe7e
2021-02-03 23:47:07 +00:00
JT Olio
c8e42df2f8 satellitedb: reorder migrations 140-144
These changes are independently tracked on
https://github.com/storj/storj/tree/jt/migration-reorder

The point of this is to make the distributed column
migration, needed for SNO invoice generation, the very
next one, so we can release it as a point release.

Change-Id: I26e1c03629c4f079b9ad12485e2b71a715d82b3b
2021-02-03 18:28:42 +00:00
Yingrong Zhao
7e80badaf9 pkg/server,pkg/quic: accept an existing conn to create quic listener and
allow disabling tcp/quic

In order to have more control of a server so that we can
simulate connection failures in `testplanet`, this PR changes
quic.Listener to accept an existing UDPConn instead of relying on the
quic-go library to create the UDPConn.
This PR also adds two flags on the `server.Config` struct to allow
enabling/disabling tcp/tls listener and quic listener. By default, they
are both set to true.
    - `DisableTCPTLS`: internal flag, disables tcp/tls listener.
    - `DisableQUIC`: hidden flag, disables quic listener
By making the `DisableQUIC` a hidden flag, it allows storagenode operators to
have the ability to disable quic traffic in case their set up can't work
with udp traffic.

Change-Id: I853b12435d988b9c41ad9b873fd57480d792e378
2021-02-03 12:04:29 -05:00
Jeff Wendling
a754c3984b satellite/console/consolewasm: assert less about error
this changes from a satellite error to a local encryption
error with the upcoming permissions changes where we only
include keys for the paths that are allowed.

Change-Id: I7aa37cfbaee31a1e54afe0423b283b9f41d9345f
2021-02-03 10:36:58 -05:00
Ethan
9506b67ca2 satellite/projectaccounting: Improve performance of ProjectAccounting.getBuckets
Limit bucket name lookup to date range of the calling methods since we only need distinct bucket names for that time period.
Adds new index and removes an index specific to project ID since it is no longer needed.

Change-Id: Ic07bbfb1c32280e0c0e39f8da020b284e1e5d974
2021-02-03 14:05:12 +00:00
Kaloyan Raev
8d25c47897 satellite/repair: fix comment in TestRepairExpiredSegment
Change-Id: Ib91e81f6ba0a7f65daed157b78f7a1a108984930
2021-02-03 10:09:49 +02:00