Commit Graph

521 Commits

Author SHA1 Message Date
Moby von Briesen
02fc87e98b satellite/payments: Apply Stripe free tier coupon for new customers
Rather than applying our internal satellite implementation of coupons
when new accounts are created, use a configured Stripe coupon instead.
If no configuration is set, no coupon will be applied.

This change also removes logic for adding coupons to customers who pay
with crypto - they will already have the free tier coupon applied
anyway.

We will be phasing out our internal coupon implementation.

Change-Id: Ieb87ddb3412acbc74986aa9d18a4cbd93c29861a
2021-05-25 17:39:44 +00:00
JT Olio
1852773e3e satellite/contact: rate limit node checkins
Change-Id: Ied386a2350aa073de46443e5259b56d49ec61dbf
2021-05-17 08:15:04 +00:00
Yaroslav Vorobiov
b426248936 scripts/tests: fix missing multinode binary
Change-Id: I288d06c9eaa814ddca1e4aaacd86c5277aba86cc
2021-05-14 22:47:05 +00:00
Yaroslav Vorobiov
c08ca361d8 cmd/storj-sim: add multinode process
Change-Id: I7e6c8db045daa87c5d89f49e58fc82453d4024c6
2021-05-14 01:40:13 +03:00
Ivan Fraixedes
7fb86617fc satellite/satellitedb: Use CRDB AS OF SYSTEM & batch for GE
Use the 'AS OF SYSTEM TIME' Cockroach DB clause for the Graceful Exit
(a.k.a GE) queries that count the delete the GE queue items of nodes
which have already exited the network.

Split the subquery used for deleting all the transfer queue items of
nodes which has exited when CRDB is used and batch the queries because
CRDB struggles when executing in a single query unlike Postgres.

The new test which has been added to this commit to verify the CRDB
batch logic for deleting all the transfer queue items of the exited
nodes has raised that the Enqueue method has to run in baches when CRDB
is used otherwise CRDB has return the error "driver: bad connection"
when a big a amount of items are passed to be enqueued. This error
didn't happen with the current test implementation it was with an
initial one that it was creating a big amount of exited nodes and
transfer queue items for those nodes.

Change-Id: I6a099cdbc515a240596bc93141fea3182c2e50a9
2021-05-07 13:09:19 -04:00
littleskunk
396cd5a683
satellite/payments: reduce object fee to 0 (#4104) 2021-04-29 22:31:32 +02:00
Moby von Briesen
4c0817bcfb satellite/payments: Populate new coupons during invoice generation
The previously configured never-expiring coupon does not refill every
month. Eventually, even though it never expires, it will run out. This
commit makes several small changes to address this issue for the free
tier:
* Change the config for the promotional coupon to be $1.65 for 1 month
(the change from $10 to $1.65 is due to our recent pricing changes)
* Update PopulatePromotionalCoupons (PPC for brevity) to add promotional
coupons to users with expired and consumed coupons (all users with a
project and no active coupons should get a new coupon when PPC is called)
* Call PPC at the end of the `create-invoice-coupons` stage of invoice
generation - after current coupons are processed and expired/exhausted.
* Remove legacy admin functionality for PPC from satellite/console - we
do not currently use it, but if we did, it should be in satellite/admin
instead.

Change-Id: I77727b97bef972df32ebb23cdc05055827076e2a
2021-04-28 18:07:05 +00:00
Moby von Briesen
b317f28fdb web/satellite: Update registration files
Allows us to remove the following files from satellite branding
repo, with an up-to-date single source of truth now in storj/storj:
* web/satellite/src/common/registrationSuccess.html
* web/satellite/src/common/registrationSuccess.scss
* web/satellite/src/views/register/registerArea.html
* web/satellite/src/views/register/registerArea.scss

The registrationSuccess files have been removed from all satellites in
the branding repository. The registerArea files have been removed only
from production satellites in the branding repository.

Importantly, this change enables the "resend email" functionality on
production satellites - previously, this functionality was available in
storj/storj, but not our branding repository.

Removes the config for VerificationPageURL, which redirected users away
from the satellite app to storj.io after creating an account. In order
for the email resend button to work, we cannot leave the app.

Adds a new config value for partner satellites, which replaces the
partner satellite names config. The new config includes name and
address. It is validated on setup/run to ensure it can be parsed.

Change-Id: I67db0702d9b9641f1a37b599f2929d56f3c33aca
2021-04-28 16:16:16 +00:00
Michał Niewrzał
02460fcc4c satellite/metainfo/expireddeletion: change chore interval to 24h
We want run deletion query more often to avoid long running DB operations.

Change-Id: I72c63bb8ec05cc3ab2e7ab63479ba6cdf830386a
2021-04-27 12:30:23 +00:00
Egon Elbre
dd5eb2616b satellite/metrics: join for monitoring
We don't want metrics to trigger the metaloop on it's own.

Change-Id: I648272a0e3043173b0a377697db1f172712bdda5
2021-04-23 12:21:48 +03:00
JT Olio
35d8a840d3 storj/storj: more domain changes
Change-Id: I643c38bdae6dc26c9346147d80a83ae9dde2eeae
2021-04-15 20:51:43 +00:00
JT Olio
3b09d6c308 storj/storj: update support request links
Change-Id: I7592df7fe4630508784b6cc92b751d3b567e911e
2021-04-14 20:52:45 +00:00
JT Olio
f3c8cac1ee storj/storj: more domain changes
Change-Id: I91ae3bc36be38c27269f9e6e678fa3616b8e4d53
2021-04-14 20:49:12 +00:00
Brandon Iglesias
32d76652c3
satellite/payments: Change customer pricing (#4085)
Co-authored-by: littleskunk <jens.heimbuerge@googlemail.com>
Co-authored-by: JT Olio <hello@jtolio.com>
Co-authored-by: Igor <38665104+ihaid@users.noreply.github.com>
2021-04-14 12:14:58 -06:00
JT Olio
afcc55fb47 storj/storj: some domain changes we can make now
Change-Id: I5622edea9dfcad788cac65e1754b331b9fed6154
2021-04-13 17:12:06 +00:00
Vitalii Shpital
0b59a165e9 web/satellite: remove google tag manager from satellite GUI
WHAT:
removed google tag manager from satellite GUI

WHY:
redundant

Change-Id: I9332b770b9050395bd2b44c2f3f3f410b5891fa4
2021-04-13 11:23:47 +00:00
littleskunk
6236deffb9
satellite/web: enable feature flag for new file browser and onboarding workflow (#4079) 2021-04-09 20:44:05 +02:00
Malcolm Bouzi
9591e75574 satellite/console: add feature flag for onboarding overview step pathway rendering
Change-Id: I2edfeb868e356b714c8d2535332713c1f64d35f9
2021-04-09 12:35:55 -04:00
Vitalii Shpital
b907aab0ad web/satellite: use config value for linksharing
WHAT:
use config value for linksharing instead of harded one

WHY:
better testing

Change-Id: I16fdb5b23198716b76d5332a1113daaa0eb86030
2021-04-09 14:56:26 +03:00
Vitalii Shpital
a3e6604883 satellite/console: config flag for enabling/disabling CSP
WHAT:
config flag for enabling/disabling CSP

WHY:
for easier testing

Change-Id: If04a98071d2b147d18b95809ef6bd4c95d6a2d1d
2021-04-09 14:44:04 +03:00
Moby von Briesen
c4a950a40c web/satellite: Remove client-side Segment analytics
We can be more precise and conservative by using the backend
satellite/analytics service. We also no longer need client-side Segment
scripts.

Change-Id: Ic5fb18bea2d388b586ad773e26027d69bde87294
2021-04-08 17:36:06 +00:00
Kaloyan Raev
a264a4422b cmd: remove metainfo-migrator and metainfo-verifier
We already merged the multipart-upload branch to main. These two tools
make only sense if we are migrating a satellite from Pointer DB to
Metabase. There is one remaining satellite to migrate, but these tools
should be used from the respective release branch instead of from main.

Removing these tools from main will:
1) Avoid the mistake to use them from the main branch instead of from
the respective release branch.
2) Allow to finally remove any code related to the old Pointer DB.

Change-Id: Ied66098c5d0b8fefeb5d6e92b5e0ef5c6603df5d
2021-04-08 14:50:29 +00:00
Vitalii Shpital
6ae2351389 web/satellite: import file browser component
WHAT:
import and instantiate file browser component

WHY:
to operate over folders and objects

Change-Id: Ib6fb4fdc2668d2f274df3d1b23f8cc0bb6a361ea
2021-04-07 22:53:20 +03:00
Moby von Briesen
04294e3e1c satellite/payments: Update coupon defaults for free tier
The new default promotional coupon is $10/month, and doesn't expire.

This change also migrates the coupon.duration column over to the new
coupon.billing_periods, and switches to rely completely on
billing_periods.

Change-Id: Ic3341e9fa4040449bab5e66ca4ee2640b095cf3d
2021-04-05 10:12:26 -04:00
Moby von Briesen
c334fd090e satellite/payments: Convert coupon duration to be nullable
* Add a nullable billing_periods column in the coupons table
* Add nullable billing_periods column to the currently unused
coupon_codes table
* Drop the duration column from the coupon_codes table
* Replace duration config type so that the default promotional coupon
can be configured to never expire

Zero downtime migration plan:
* Add billing_periods column to coupons and coupon_codes tables (this change)
* After one release, remove all references to the old duration column,
replacing with references to billing_periods. At this point, we can also
change the defult promotional coupon to never expire and migrate over
values from the old duration column.
* After another release, drop the duration column.

Change-Id: I374e8dc9fab9f81b4a5bc681771955662d4c007a
2021-04-01 16:28:23 +02:00
Michał Niewrzał
0b7c288155 test-sim: improve integration test scripts
* integration tests have now a trap that will display where script failed, line and message
* functions from uplink tests moved to utils.sh to reuse them

Change-Id: Ib2311775fd70ce784aa986328969d75eefc5ac36
2021-03-31 16:48:53 +00:00
Michał Niewrzał
a5bc38b197 build: replace 'multipart-upload' branch occurences
We removed `multipart-upload` branches but we have few leftovers.

Change-Id: I74f637748a248cb286cd190a3a8f895ef85f3940
2021-03-31 11:06:47 +02:00
Moby von Briesen
23c8331d1f satellite/analytics: Add flag for disabling analytics (default disabled)
Just in case we need to have an easy flag for turning analytics on or
off.

Change-Id: I219071200e0a4c156ba92cbbaa4de4ab75b2cb68
2021-03-29 16:23:47 +00:00
Michał Niewrzał
c8a3e0ff96 tests: fix calculation of previous month for billing integration test
Change-Id: I413629520aa52518395189c2ea56000cab67c948
2021-03-29 10:38:59 +02:00
Cameron Ayer
1a51049ac0 satellite/{overlay,satellitedb}: add flag to toggle suspending nodes for offline audits
This change introduces a new config flag,
--overlay.audit-history.offline-suspension-enabled,
to toggle suspending nodes for offline audits.

If the flag is set to true, nodes will be suspended if they meet the
requirements.

If the flag is false, nodes will not be suspended. If they are already
suspended and/or under review, these will be cleared.

Change-Id: Ibeba759c42d6e504f6b7598120d4fd4dab85ca74
2021-03-27 16:28:27 +00:00
Malcolm Bouzi
527b7eb56b web/satellite: ui changes to registration and billing pages to enable user to add promo codes to their account
- add Credit History table to billing acount page and set up ui for a user adding promo codes
- implement promo codes ui in registration form
- add feature flag to handle if coupon code ui should be rendered

Change-Id: I9fdeef7cffc7901958d3f9be335e1115b2471a2e
2021-03-26 11:57:11 -04:00
Michał Niewrzał
9bc2a9a4b7 rollingupgrade: change how metainfo-migration tool is installed
Change-Id: Iad1285a770f76c9cf09a872a4bf1cd39969ee117
2021-03-26 15:03:39 +00:00
Moby von Briesen
2a68e0767e scripts/tests/testversions: Fix testversions Episode 2
The previous fix did not remove the comment at the front of the config
values that were modified.

Change-Id: I05729c30a5398d049c6254afec01df00230d8deb
2021-03-25 18:06:13 -04:00
Moby von Briesen
63cd807c9f scripts/tests/testversions: Increase bandwidth/storage limits for testversions
A recent change made the default usage/storage limits for projects 50gb
rather than 500gb. This increases the default limit for testversions.

Change-Id: Ibea05c0d0760662e447b6455d560a2a640801c6c
2021-03-25 14:43:28 -04:00
Moby von Briesen
3db52491ec satellite/analytics: Add analytics service to satellite
* Set up basic structure of new service.
* Implement a basic analytics track event for user creation.

Change-Id: Ica8c785540b1ef9d848404af307a22f21d33c6aa
2021-03-25 13:28:51 -04:00
Michał Niewrzał
0ef0a3f967 rollingupgrade: add migration to metabase
For time of transition from pointerdb to metabase we need add migration step to rollingupgrade tests and comment few cases.

Change-Id: Ib12ae6aa14be35f9bf4ff3efb55cfc6957d4ceba
2021-03-25 10:52:35 +00:00
Moby von Briesen
27bcb46718 satellite/console: change default project and usage limits
This is one step for implementing the free tier:
* Change the default project limit from 10 to 3
* Move storage and bandwidth project usage limits from the metainfo
package to the console package (otherwise there is a cyclical
dependency, and metainfo doesn't use these values anyway)
* Change the default storage usage limit per project from 500gb to 50gb
* Change the default bandwidth usage limit per project from 500gb to 50gb
* Migrate the database so that old users and projects continue to have
the old defaults (10 projects/500gb usage)

Change-Id: Ice9ee6a738bc6410da18c336c672d3fcd0cab1b9
2021-03-24 14:24:27 +00:00
Michał Niewrzał
9d52451464 satellite/metainfo/expireddeletion: add config value to set list limit
We want to control how many expired objects we will query at once.

Change-Id: Iba7c79a6236c6b720b53bb8f078d55514690d8e5
2021-03-23 09:51:56 +00:00
Michał Niewrzał
237782813b Merge remote-tracking branch 'origin/multipart-upload'
Change-Id: If6c5a450b238adab55d1e0dea67d01e5f5768a9f
2021-03-23 09:44:49 +01:00
Moby von Briesen
d2b365bf9f satellite/payments: Disable paywall (free tier)
We are implementing the free tier, which will give all new users 3
projects, 50gb storage, and 50gb bandwidth per project. All users will
receive a recurring coupon to cover this amount of usage.

With the free tier, we no longer need a paywall. Users will not need to
enter a payment method unless they want to increase their project or
usage limits.

Change-Id: If3b026e91858e5f557a2758e366616cecc8f21c7
2021-03-22 17:11:20 -04:00
Cameron Ayer
864ad70fe2 satellite/overlay/straynodes: set --stray-nodes.enable-dq release default to true
Since we will enable this on all satellites, just set default to true

Change-Id: Ibc86a0afd0b0f57e86bd067abb9cdf06c295a467
2021-03-22 17:25:09 +00:00
Cameron Ayer
2607b16070 satellite/{overlay/straynodes,satellitedb}: rework DQNodesLastSeenBefore to return DQd node IDs and last contact successes
We would like to log Node IDs and last contact successes of nodes DQd
in this manner. We would also like to avoid returning an unbounded list
of items from the db. Therefore we change the query to select a limited
number of nodes that meet the DQ conditions and iterate until 0 rows are
returned. Each column of the query is already indexed.

Change-Id: Iaec2d9b56e7202b7c2028ba21750d40c8dd506ee
2021-03-22 13:01:30 -04:00
Ivan Fraixedes
fa8b0a29ea scripts: Do not use Docker for test-sim Redis test
Do not use Docker for running a Redis server in the test-sim that we
check that Satellite can still operate when Redis is unavailable because
we have to run this test in Jenkins and we don't want to empower a
container to be able to connect to the Docker socket of the host machine
for running sibling containers.

Change-Id: I6180e8ed804968c8ccb0783ed334acab38af9a0f
2021-03-22 08:43:53 +00:00
Vitalii Shpital
c3ae122aa7 web/satellite: enter passphrase step for objects page
WHAT:
enter passphrase step for users who has already created passphrase

WHY:
to let users proceed to upload step

Change-Id: I084aec5b863981978cf190f99ee95154fbed9aab
2021-03-19 10:59:32 +00:00
Michał Niewrzał
fa083a7f05 Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: Ib5ce5965b77b81c254d08c27ab30c7eccefbd4c6
2021-03-17 15:37:17 +01:00
Vitalii Shpital
6a553ec9c5 web/satellite: change banner for beta satellites with URLs
WHAT:
beta satellite top banner's copy is changed to include support/feedback URLs

WHY:
so users using our beta satellite will be able to report feedback somewhere

Change-Id: Ibc349c8b3354b577275fcf1d2b75bfdd267729d9
2021-03-15 17:12:07 +00:00
Michał Niewrzał
67e26aafcd Merge remote-tracking branch 'origin/main' into multipart-upload
Change-Id: I9b183323cb470185be22f7c648bb76917d2e6fca
2021-03-10 08:53:38 +01:00
Natalie Villasana
c290e5ac9a satellite/orders: decrease FlushBatchSize default to 1000
The previous default FlushBatchSize of 10000 was causing major
slow down in select and insert statements on bucket_bandwidth_rollups.
We saw on the saltlake satellite that a FlushBatchSize of 1000 helped
reduce contention and query latency.

Change-Id: Ib95e73482219bc5aedc11925b1849fa5999774ba
2021-03-02 14:00:48 +00:00
Fadila Khadar
cf0c88c679 cmd/uplink: fix no result when listing single file
uplink ls sj://bucket/file.ext gives a result if file.ext exists.

Change-Id: Ife741926acacf510e4e098009ce1724386c7522f
2021-02-26 11:43:02 +00:00
Vitalii Shpital
300e88f9a7 web/satellite: config flag for satellites in beta
WHAT:
config flag to indicate if satellite is in beta

WHY:
to avoid using hardcoded satellite names which may cause issues

Change-Id: If92eb7417c340bf343a9a91e2f6b11f0349020c5
2021-02-24 12:29:07 +02:00