This change adjusts the label for STORJ deposit bonuses in billing
history to be more consistent with other labels.
Change-Id: I5e7179ae3ac52dafb0dcef084e9a7c4742491f9e
The DB query in GetAllocatedBandwidthTotal uses an exclusive range:
'WHERE interval_start > ?'
The value that is used for this condition is the first day of current the month,
00:00:00 UTC.
By using the exclusive '>', we exclude the entire first hour of the month from the
result set.
Change-Id: I3ed300f5230c7514dc9495a85e8166213cd0842e
this way we don't have to do 2 steps, and by using the ctid, postgres
is going to do two very efficient prefix scans.
Change-Id: Ia9d0546cdf0a1af67ceec9cd508d336a5fdcbdb9
also remove the continuation support from the queue, otherwise
we may end up sequential scanning the entire table to get
a few rows at the end.
then, in the core, instead of looping both to get a big enough
batch inside of the queue, as well as outside of it to ensure
we consume the whole queue, just get a single batch at a time.
also, make the queue size configurable because we'll need to
do some tuning in production.
Change-Id: If1a997c6012898056ace89366a847c4cb141a025
In jaeger, it shows that this function gets called repetitively in
a single request. Most of the time, it's less than 1ms. Therefore, it
doesn't add much value in our trace but create noises.
Change-Id: I20234f36bbcf0fc22f91e5e1a5634c0cad577ed0
Jira issue: https://storjlabs.atlassian.net/browse/USR-820
The bonus for depositing STORJ tokens is now added as to the Stripe
balance instead of the to `credits` DB table on the satellite.
Existing unspent bonuses in the `credits` DB table are still processed
as usual when generating invoices. They will be migrated to the Stripe
balance with a separate change.
The bonus is added to the Stripe balance with a separate Credit
transaction. The balance transactions for the deposit and the bonus can
be differentiate by their different description.
The billing history is modified to list the bonus from the Stripe
transactions list.
The workflow for depositing STORJ tokens to the Stripe balance is
improved to survive failures in the middle of the process.
Change-Id: I6a1017984eae34e97c580f9093f7e51ca417b962
Jira issue: https://storjlabs.atlassian.net/browse/USR-110
When generating invoices, if a coupon is not fully spent, but this is
its last valid month, it will be marked as expired in the database.
Change-Id: Ib1b2375f6703f9514d1ae089a387aa029b62d1fb
struct
This change is removing ProjectID from code. Next change will be about
dropping this colum from DB table.
Change-Id: Idb949e2829e2c304a2b6b011259c7cc7667082e1
the test as written fails if it's the last day of the month because
the two tallies that are added span two different periods.
Change-Id: I294397278508035ce640500fc3b1b0ffd82b9b71
projects
Currently, to generate invoices we are listing all projects. This is
problematic because we decided that elements like e.g. coupons should be
calculated per user. We want to have
more user oriented processing. With this change to prepare invoice
elements we are:
* listing Stripe customers
* listing projects for each customer
* processing projects to create project records
* calculating coupons per customer
* calculating credit spending per customer
Change-Id: I48b0063e44e4d301f97c87ecffb342649359968a
WHAT:
1. updated verification page URL in config
2. added list of partnered satellites to config
3. added logic for satellites dropdown on new signup/login pages
WHY:
1. signup/login flow was reworked in tardigrade.io repo (iframe removed, new pages etc.)
2. new config flag was added to check if satellite name matches at least one member of partnered satellites list to redirect user to verification page
3. new pages will have dropdown with partnered satellites list. Appropriate logic was added.
Change-Id: I33399ab66ca31f07b297a433f6b1f41da4cb6e66
Adds monkit tracing for ecrepairer.downloadAndVerifyPiece and
ecrepairer.putPiece so we can get more accurate estimates of node
performance during repair.
Change-Id: Ic05025bf3c493bb3d6f5d325d090c5b7c9e5465d
This will speed up the Put step of repair by not waiting to time out for
a handful of slow nodes, at the expense of a slightly less durable
pointer. It will still repair to the optimal threshold, but not every
node that is selected will end up in the pointer.
Change-Id: I02a0658e3fe6fc0383f26af0f50a065b8b11a651
the initial calculations for the historical values of comp_at_rest
were wrong. because our historical data only included total amounts
as well as compensation for bandwidth, the at rest value was
calculated as
at_rest = total - bandwidth
unfortunately, that calculation did not take surge pricing into
account correctly. the at rest and bandwidth values do not
include surge pricing, but the total that was used did. so what
we actually calculated was
no_surge_at_rest = surge_total - no_surge_bandwidth
which will create a value that is too large. this migration
fixes the calculation for imports that are old enough and
of a non-negligable difference.
Change-Id: I61eb0b670510f6d7fb8fc3de39ba79150fac10eb
..before they are transferred to another node and submitted to the
satellite as successful piece transfers, because if we submit an invalid
signature, the node will be marked as a cheater and disqualified
immediately.
These signatures should have been validated when the piece was
originally stored, but bitrot does happen and needn't be cause for an
immediate DQ.
Change-Id: I8b0ebd5812ea8a2e60766005b7251fbb74ef7857
Some satellites do not have payment configured (ex. Salt Lake, Europe North). In this case the StripeMock is used, which returns nil on invoice and charges List methods. This causes a panic.
https://storjlabs.atlassian.net/browse/SM-978
Change-Id: Iec1b0bfd9b383e6f793d03dd63a3dec60e0fd9f3
Jira issue: https://storjlabs.atlassian.net/browse/USR-719
Invoices now show the amount of used resources and their unit price.
This change also makes proper rounding to the nearest cent in few places
to resolve the "off-by-one-cent" issue observed in few invoices.
Change-Id: I2d70d6916b5cf4a9a9138c99c422f5f4d2deb35b
* add monkit stat new_remote_segments_needing_repair, which reports the
number of new unhealthy segments in the repair queue since the previous
checker iteration
Change-Id: I2f10266006fdd6406ece50f4759b91382059dcc3
Every now and then we see the repair error, "piece to add already exists".
With these new logs we should be able to verify if it is due to a change in
the pointer. These logs are only temporary
Change-Id: I029390cc4816668707546df14ed2cfe7ca192b0b
This attempts to add a README.md to help create consistent migrations
that maximize our test coverage and do not include unnecessary
statements.
It also adds a feature to have an `-- OLD DATA --` section as well
as a `-- NEW DATA --` section so that we can fix mistakes made in
previous snapshots (like a row that was forgotten to be added when a
table was created) without editing them going forward.
Change-Id: I28a786f8ef163cae1de1bb08f61af1e1104b0a88
What: As soon as a node passes the vetting criteria (total_audit_count and total_uptime_count
are greater than the configured thresholds), we set vetted_at to the current timestamp.
Why: We may want to use this timestamp in future development to select new vs vetted nodes.
It also allows flexibility in node vetting experiments and allows for better metrics around
vetting times.
Please describe the tests: satellitedb_test: TestUpdateStats and TestBatchUpdateStats make sure vetted_at is set appropriately
Please describe the performance impact: This change does add extra logic to BatchUpdateStats and UpdateStats and
commits another variable to the db (vetted_at), but this should be negligible.
Change-Id: I3de804549b5f1bc359da4935bc859758ceac261d
Most places now need the NodeURL rather than the ID and Address
separately. This simplifies code in multiple places.
Change-Id: I52621d8ca52296a8b5bf7afbc1001cf8bfb44239
