Commit Graph

5649 Commits

Author SHA1 Message Date
Egon Elbre
10a0216af5 satellite/metainfo: use range for specifying download limit
Previously the object range was not used for calculating order limit.
This meant that even if you were downloading only a small range it would
account bandwidth based on the full segment.

This doesn't fully address the accounting since the lazy segment
downloads do not send their requested range nor requested limit.

Change-Id: Ic811e570c889be87bac4293547d6537a255078da
2021-06-01 09:36:55 +00:00
Fadila Khadar
aa49c8c44d satellite/orders: fix TestProjectUsageBandwidth
Change-Id: I3e9e81ea1ac2b0d7ef0936e4480a6b13c36bfcc5
2021-06-01 10:41:59 +02:00
Qweder93
91b7e24d55 multinode/payouts: naming refactoring
Renamed methods, error messages.

Change-Id: I7d7b6b092c05bbc5bf1322855efc5ccb9b312671
2021-05-31 19:47:12 +00:00
NickolaiYurchenko
20b98d31b8 web/multinode: expectations block added
added displaying of undistributed balance and current month expectations

contains changes from https://review.dev.storj.io/c/storj/storj/+/4867 to be workable

Change-Id: I9cb00d2db5b819a71ceeddf91d6863a6b4fa9034
2021-05-31 14:33:03 +00:00
Yaroslav Vorobiov
64e4225721 multinode/console: server reorder payouts API routes
Change-Id: Ia42bb3faae8a8782d841e5cba186dc73478581ac
2021-05-31 16:32:03 +03:00
Yaroslav Vorobiov
23f9beb635 multinode/console: held amount summary
Change-Id: Ia800748343e363d930ce0a0b9ab286b5abdc96af
2021-05-28 20:05:16 +03:00
Egon Elbre
0ef537a685 satellite/metabase/metaloop: limit max as of system time
When using a system time too far in the past, the query gets slower.

Change-Id: I7014d13a16b1ec44391549f829e7a57942f1fea7
2021-05-28 12:07:36 +03:00
Vitalii Shpital
a5dbc544f4 satellite/console: project member deletion bugs fixed
There was a bug when user tried to get project after removing themselves from it.

Also we made user select firstly created project only if they removed themselves from current selected project.

Change-Id: I4b28ebc1ab4a8c14d05ef702e034f2ab39225cc3
2021-05-27 14:54:54 +00:00
Vitalii Shpital
32dd4315af web/satellite: improve AG's selection checkmark
replaced png with svg of AG selection checkmark to improve it's visibility

Change-Id: I868d8c831a33d4cf47dcd40aaa49e1c31fe9cf0b
2021-05-27 16:09:37 +03:00
Michał Niewrzał
3af91e7a90 satellite/metabase: add iteration over segments
Method IterateLoopSegments can be used to iterate over all segments in metabase without touching objects.

Change-Id: I3cc0e783884b603b47ef3f8233e357aa8a391250
2021-05-26 09:59:33 +00:00
Malcolm Bouzi
37bc46c6f6 web/satellite: Fix for Payment Method area buttons on billing page not being aligned
Aligns 'Add STORJ' and 'Add Card' buttons

Change-Id: I6a9a92f59d7e2e903d0e4d683cb09cd8de6da987
2021-05-25 22:01:24 +00:00
Moby von Briesen
f536ee3aaf satellite/satellitedb: Migrate non-expiring coupons to expire
Because of recent changes to how coupons for the free tier are handled
(see commit 4c0817bcfb), we no longer want all these $10
non-expiring coupons. After coupons are applied during invoice
generation, if a customer does not have any valid (non expired, non
consumed) coupons, a new promotional coupon is applied.

We could just wait for users to consume all $10 of the non-expiring
coupons, and the new promotional coupon would be applied for the
following billing cycle, but this gets tricky, because if in the final
month, the user is billed for $1 of usage, but only $0.5 of the $10
non-expiring coupon is remaining, the user will be charged for the
remaining $0.5. With the new promotional coupon of $1.65, expiring every
month, this would not be an issue.

So long story short, this commit migrates all non-expiring coupons to
expire within 2 billing periods (all existing non-expiring coupons in
prod were created in early April or later). That way, there is still
enough value in the $10 coupon that we don't have to worry about
customers exceeding it, and the coupons will expire. Then we'll
immediately apply the $1.65 coupon for the next month! And then
hopefully this unfortunate situation will come to a pleasant end.

Change-Id: I8a593948d8876c41a71d886b9a95d4e2c802b4f3
2021-05-25 19:25:02 +00:00
Fadila Khadar
63cfc8fbe0 satellite/orders: use project daily bandwidth rollups
Replace GetProjectAllocatedBandwidth by GetProjectBandwidth which calculates
used bandwidth from allocated and settled bandwidth recorded in the
project_bandwidth_daily_rollups table.
For each day in the month, if the allocated bandwidth is expired, it uses the
settled bandwidth for computing used bandwidth.

Change-Id: Ife723c6d5275338f470619631acb25930d39ac3c
2021-05-25 18:28:58 +00:00
Michał Niewrzał
59eabcca24 satellite/orders: populate
project_bandwidth_daily_rollups table

We want to calculate used bandwidth better so we need to calculate it
from allocated and settled bandwidth. To do this we need first populate
this new table.

https://storjlabs.atlassian.net/browse/PG-56

Change-Id: I308b737bf08ee48ce4e46a3605697ab2095f7257
2021-05-25 18:07:22 +00:00
Moby von Briesen
02fc87e98b satellite/payments: Apply Stripe free tier coupon for new customers
Rather than applying our internal satellite implementation of coupons
when new accounts are created, use a configured Stripe coupon instead.
If no configuration is set, no coupon will be applied.

This change also removes logic for adding coupons to customers who pay
with crypto - they will already have the free tier coupon applied
anyway.

We will be phasing out our internal coupon implementation.

Change-Id: Ieb87ddb3412acbc74986aa9d18a4cbd93c29861a
2021-05-25 17:39:44 +00:00
Egon Elbre
92226d8ddb storagenode/storagenodedb: fix fillInBlobAccess
fillInBlobAccess was using a non-pointer receiver so the receiver wasn't
being modified. Luckily, this seems to be only being used in tests.

Change-Id: Ice01419933295562d558d48ba314d476660b67bd
2021-05-25 16:41:10 +00:00
Qweder93
099a31c69f multinode/payouts: paystubs for single period added
paystubs for single/all satellites for specific period added.

Change-Id: Ibb822aa11380bc70f4db8c81fa0b9b6f7a739078
2021-05-25 18:59:05 +03:00
Qweder93
1bc4f8fccd multinode/payouts: paystubs for all time added
paystubs for single/all satellites for all time added.

Change-Id: Ia665c69994f769cb0071363a617eaf87ef1a05f2
2021-05-25 18:19:23 +03:00
Egon Elbre
926bcd0f83 mod: bump private
Latest version contains a fix for pgxutil.Conn.

Change-Id: I200b1c11a1e536a57a7384c1a747a7216f8cd9ff
2021-05-25 15:28:16 +03:00
Egon Elbre
6e05b240dc private/lifecycle: log stack on slow shutdown
When services take long to shutdown it's useful to get a stack trace for
diagnosing the underlying problem.

Change-Id: Ic73a45741dfbe8fdddafd56a5b72121da886d133
2021-05-25 09:55:19 +00:00
Fadila Khadar
16022105f1 satellite/orders: create project_bandwidth_daily_rollups table
Change-Id: I8f101c0714b5673450d4f2f01e297a9eddc6caf6
2021-05-25 09:05:47 +00:00
Vitalii Shpital
c410eae177 web/satellite: better error message for duplicated bucket
WHAT:
better error message when trying to create a bucket with a name that already exists

WHY:
better user experience

Change-Id: I1f56439d6dc8503cdca03759b0cc180688aafd87
2021-05-25 08:24:00 +00:00
Vitalii Shpital
b33b0a38d6 web/satellite: remove redundant console errors on duplicated route changes
WHAT:
removed console errors when trying to navigate to current route

WHY:
there shouldn't be errors in console

Change-Id: Ifa6ccf2f336cab58ac5e27b5dae2167a3229f98a
2021-05-25 08:03:15 +00:00
Moby von Briesen
0ff3516f54 satellite/mailservice: Fix bug causing issues with test account creation
The context passed into SendEmail gets canceled before links are clicked
in the test environment. This change passes an un-cancelable context
from SendRenderedAsync so that email sending/link clicking can be
completed even if the parent context is canceled.

Change-Id: I88535d315900d7886877f0e14d1d052745402ac7
2021-05-25 06:19:53 +00:00
Egon Elbre
e7cf369cd8 satellite/metabase: use pgxutil.Conn helper
Clean up underlying pgx.Conn extraction with helper.

Change-Id: Ie43f663a6036c439e675df54fccd3605c4d8d69e
2021-05-25 05:56:35 +00:00
Jeff Wendling
e460dc51f7 cmd/uplinkng: implement object listing
Change-Id: Ib5f6964a0c42718913a680529bb66c6f475aeac9
2021-05-24 23:02:19 +00:00
Egon Elbre
cdcc67207c satellite/satellitedb: fix nil panic in UpdateCheckIn
Change-Id: If6ae2c3d9b7c269b0a9d652e68854091f668b5ec
2021-05-25 00:30:36 +03:00
Qweder93
79172777bd mnd/payouts: estimations replaced with expectations
Added expectations endpoint (estimations and distributed), added
coalesce to db query, so in case of empty payouts db 0 will be returned instead of error.

Change-Id: I535f14ef097876448d8949bc302895b25da2b6e7
2021-05-24 18:11:30 +00:00
Qweder93
83e82eb473 mnd/payouts: estimated payouts per satellite changed to per node
Change-Id: I231e95ece1f52accb17839a00bfc1588b6e2a658
2021-05-24 19:47:57 +03:00
Egon Elbre
515f9b54ff mod: bump monkit
Change-Id: I9be05b91cd9171ebc7d053cd087d6761150d44b6
2021-05-24 16:20:00 +00:00
Qweder93
f0dd7f3739 mnd/payouts: undistributed payouts added
Change-Id: Id61ac58fceca13b9a9e12a7e03c3d9624cce3fea
2021-05-24 18:20:08 +03:00
Vitalii Shpital
2896a2736c web/satellite: remove time caveats for object browser AGs
WHAT:
removed notBefore and notAfter caveats from object browser AGs

WHY:
caused a bug when user's clock is not synced correctly

Change-Id: I1af10418f1668003a09cba313eaed3b0764eabdc
2021-05-24 14:50:17 +00:00
NickolaiYurchenko
f2842a27e2 web/multinode: by node payouts page markup
Change-Id: I4d760a9965d28afc37f4cfa971d95254bfcb1d7e
2021-05-24 17:28:25 +03:00
Michał Niewrzał
c9d4674859 cmd: remove metabase-createdat-migration tool
We finished migration for all satellites and we can now remove this
code.

Change-Id: If92091ba94b51c319528dc95845042c9dfd6d074
2021-05-24 09:37:40 +02:00
Yaroslav Vorobiov
f0d60a6ec2 Jenkins: add multinode UI stage
Change-Id: I77ed791ff9dbb5b046450ab625c1763b34691888
2021-05-21 15:49:31 +03:00
Yaroslav Vorobiov
1ff2e1a474 web/multinode: update test snapshots
Change-Id: I8a7adeba1b48c754dc50e68c479f89903cd98486
2021-05-21 15:46:46 +03:00
Egon Elbre
2d3c40fd71 cmd/internal/asset: delete unused code
Change-Id: I44970a4f5ac42dc3a297263cd64150e94c40f62a
2021-05-20 18:30:19 +03:00
Vitalii Shpital
a9fcbf7284 web/satellite: bump object browser version
WHAT:
use new object browser version

Change-Id: Ieb473b8f4b1beff81a5c2661f55e8e3f946daa2e
2021-05-19 16:01:15 +00:00
Jeff Wendling
a1bf9ab6de cmd/uplinkng: initial commit with skeleton
Change-Id: I764618cc60c46882955e9b08b72b3c162aa4929f
2021-05-18 10:04:44 -04:00
Egon Elbre
10372afbe4 ci: fix lint errors
Change-Id: Ib5893440807811f77175ccd347aa3f8ca9cccbdf
2021-05-17 13:37:31 +00:00
JT Olio
1852773e3e satellite/contact: rate limit node checkins
Change-Id: Ied386a2350aa073de46443e5259b56d49ec61dbf
2021-05-17 08:15:04 +00:00
Egon Elbre
8f15f975a2 satellite/overlay: improve contended update checkin
Improve UpdateCheckIn on a contended row:

  name                             old time/op  new time/op delta
  UpdateCheckInContended-100x-32   2.29s ±55%   0.17s ±61%  -92.45%  (p=0.008 n=5+5)

Change-Id: I053ab9f1cff136c306e5fb57f5e355cdc0269a8c
2021-05-16 20:41:12 +03:00
Yaroslav Vorobiov
b426248936 scripts/tests: fix missing multinode binary
Change-Id: I288d06c9eaa814ddca1e4aaacd86c5277aba86cc
2021-05-14 22:47:05 +00:00
Qweder93
f2812d76cd multinode/payouts: satellite period/allTime summaries added
payout summaries for specific satellite period/allTime added

Change-Id: I144138304f01f23d5c4b10931988eaaced656aaf
2021-05-14 18:49:31 +00:00
Cameron Ayer
dfe85beac2 satellite/satellitedb/dbx: fix duplicated lint line
Change-Id: Ia6677417ace208689eb6a814f2e93448fe57047c
2021-05-14 13:48:51 -04:00
Cameron Ayer
be87c80fe1 satellite/satellitedb: drop columns total_uptime_count and uptime_success_count
These columns on the nodes table are not referenced anywhere and are
ready to be dropped

Change-Id: I3f3652f635108419e93ced12b0ac1e58bba27a32
2021-05-14 11:49:12 -04:00
Cameron Ayer
3ea7aa2c7a satellite/repair/repairer: log piece hash verification failures
Piece hash verification failures during repair download are considered
audit failures, but we are not logging these occurrences. Now we log
them.

Change-Id: If456cebcfda6af7a659be3d1fc74448e681fb653
2021-05-14 15:03:15 +00:00
NickolaiYurchenko
e0f3166343 web/multinode: store and payouts components tests added
WHAT: unit tests for vue components and vuex store and environment for it

WHY: to cover frontend part with tests

Change-Id: I2aeaadb200176d4ba0a1406068304785e95d92cd
2021-05-14 12:45:10 +00:00
NickolaiYurchenko
2ccbb32e14 web/multinode: payouts page markup
WHAT: total payouts table, period calendar, details area and payout history area markup
also this component fumctionally connected to api and vuex store

Change-Id: Id3abc87bc7545aa3fc0a7ef2e480a8ed73974b76
2021-05-14 12:06:16 +00:00
igor gaidaienko
6ee2210297 satellite/console: add test for time based permission
Add test with NotBefore and NotAfter restricted permission to verify that we don't have an access to bucket

Change-Id: I7ec98a5b02c0098ee7ec81034278398f4435f1cf
2021-05-14 11:39:12 +00:00