Commit Graph

5616 Commits

Author SHA1 Message Date
Reef Loretto
0084164f3a
Fix link to whitepaper in README.md (#4107) 2021-05-03 11:27:07 +02:00
Vitalii Shpital
222d9b7f0d web/satellite: add script to compile and compress wasm module
WHAT:
compile, compress and place wasm module into the correct place

WHY:
easier usage/testing

Change-Id: I06e83ef1bed9fc62366e1400ce5ed8942bd96633
2021-04-30 15:48:56 +00:00
Egon Elbre
a8533042a3 mod: bump uplink
Renamed a private package in uplink and some of the tests need to be
updated.

Change-Id: I8735fc16c2066ecba669618f0c1dadf68273704d
2021-04-30 16:53:54 +03:00
Moby von Briesen
1b736104cb satellite/payments: Apply free tier coupons before preparing invoices
We are already adding the free tier coupons at the end of
InvoiceApplyCoupons, but there is a case where we will charge customers
who do not currently have a coupon the next time invoices are generated.
By applying the free tier coupon before preparing invoice project
records, we cover this case.

Once every customer has a coupon, it will be safe to remove this
functionality and only apply new coupons at the end of invoicing.

Change-Id: I65afbe5c0b84e63eeb1a0221e8d95311d87641a0
2021-04-30 13:26:25 +02:00
Egon Elbre
dae9464ee8 storagenode/trust: missed one error test
Change-Id: I3aff2e6b57ecc1b42f5c8eaca4dc762899cd53c5
2021-04-30 10:52:44 +03:00
littleskunk
396cd5a683
satellite/payments: reduce object fee to 0 (#4104) 2021-04-29 22:31:32 +02:00
Egon Elbre
2c657f594e satellite/metainfo: don't rely on exact error name
Change-Id: I7975a00c32891a43b4f1e6dc4f5847201844f2ec
2021-04-29 18:35:56 +00:00
Yingrong Zhao
65b17df58e bump storj.io/monkit-jaeger to latest
This will include the fix to flag errored functions in the jaeger traces

Change-Id: I1187ca0e2a9a390f5a1f3d8353edea84ea76ac91
2021-04-29 13:47:21 -04:00
Ivan Fraixedes
5f29a20937 satellite/satellitedb: Add missing indexes
The DBs of our production satellites have some indexes that we didn't
have in the migrations because at that time we weren't able to add them
because our migration test was not able to deal with Cockroach indexes
with the STORING clause.
We have recently modified the storj.io/private/dbutil/pgutil package to
support the CRDB STRORING clause, so we are adding the missing indexes
to our migrations for being able to have them if we have to recover a DB
from scratch or we deploy a new DB satellite.

Change-Id: I686ff84e5b4c02d9615f50fa531261363affefb8
2021-04-29 15:34:46 +00:00
Egon Elbre
961e841bd7 all: fix error naming
errs.Class should not contain "error" in the name, since that causes a
lot of stutter in the error logs. As an example a log line could end up
looking like:

    ERROR node stats service error: satellitedbs error: node stats database error: no rows

Whereas something like:

    ERROR nodestats service: satellitedbs: nodestatsdb: no rows

Would contain all the necessary information without the stutter.

Change-Id: I7b7cb7e592ebab4bcfadc1eef11122584d2b20e0
2021-04-29 15:38:21 +03:00
Malcolm Bouzi
30d2a2ea7b web/satellite: add total account balance to billing page
- added to top left of billing page
- credit history dropdown removed

Change-Id: I85a0db67d61a318c5ec29d67153ce4f47770bc17
2021-04-28 18:29:18 +00:00
Moby von Briesen
4c0817bcfb satellite/payments: Populate new coupons during invoice generation
The previously configured never-expiring coupon does not refill every
month. Eventually, even though it never expires, it will run out. This
commit makes several small changes to address this issue for the free
tier:
* Change the config for the promotional coupon to be $1.65 for 1 month
(the change from $10 to $1.65 is due to our recent pricing changes)
* Update PopulatePromotionalCoupons (PPC for brevity) to add promotional
coupons to users with expired and consumed coupons (all users with a
project and no active coupons should get a new coupon when PPC is called)
* Call PPC at the end of the `create-invoice-coupons` stage of invoice
generation - after current coupons are processed and expired/exhausted.
* Remove legacy admin functionality for PPC from satellite/console - we
do not currently use it, but if we did, it should be in satellite/admin
instead.

Change-Id: I77727b97bef972df32ebb23cdc05055827076e2a
2021-04-28 18:07:05 +00:00
Vitalii Shpital
2e5e4cb64f web/satellite: access grant wizard refactored
WHAT:
timeouts replaced with promises to make things work faster
removed cli step templates splitting
removed default date caveats for AG
fixed small back button bug

WHY:
improvement

Change-Id: Idc2608c467b7ab11ee4f0337d0b9fa3d282959a2
2021-04-28 17:41:10 +00:00
Moby von Briesen
b317f28fdb web/satellite: Update registration files
Allows us to remove the following files from satellite branding
repo, with an up-to-date single source of truth now in storj/storj:
* web/satellite/src/common/registrationSuccess.html
* web/satellite/src/common/registrationSuccess.scss
* web/satellite/src/views/register/registerArea.html
* web/satellite/src/views/register/registerArea.scss

The registrationSuccess files have been removed from all satellites in
the branding repository. The registerArea files have been removed only
from production satellites in the branding repository.

Importantly, this change enables the "resend email" functionality on
production satellites - previously, this functionality was available in
storj/storj, but not our branding repository.

Removes the config for VerificationPageURL, which redirected users away
from the satellite app to storj.io after creating an account. In order
for the email resend button to work, we cannot leave the app.

Adds a new config value for partner satellites, which replaces the
partner satellite names config. The new config includes name and
address. It is validated on setup/run to ensure it can be parsed.

Change-Id: I67db0702d9b9641f1a37b599f2929d56f3c33aca
2021-04-28 16:16:16 +00:00
Vitalii Shpital
bc95455135 web/satellite: delete bucket popup reworked
WHAT
added bucket name validation to delete bucket popup
removed bucket name auto-fill

WHY
better user experience

Change-Id: I5ba98050575a174ac22e5c32877cb2354cedb63f
2021-04-28 17:55:46 +03:00
Vitalii Shpital
2936b6755d web/satellite: wording updates
WHAT:
changed name of auto-created project from Untitled Project to My First Project
added estimated charges info for Free tier users

WHY:
better user experience

Change-Id: I6e5183a436989677ded928b665363b81f9878b34
2021-04-28 14:03:15 +00:00
Vitalii Shpital
75ba99b881 web/satellite: bump object browser component version with part size fix 2.0
WHAT:
another try to fix object browser part size issue
updated package-lock.json

WHY:
bug fix

Change-Id: I8104f3f84223f2e898fad6de63596a9841667ebc
2021-04-28 14:19:33 +03:00
Fadila Khadar
5af898fcc9 satellite/metabase: add migration step for index on pending objects
Add an index on (project_id, bucket_name) for objects with pending status

Change-Id: Ibd3c949a99e49e8e6bb884564331ec16f9184942
2021-04-28 09:54:10 +00:00
Qweder93
be7928d4bd crashcollect: removed redundant structure
Change-Id: Ied65a057baf4cbbe8a47685b0c52f3900363d4bd
2021-04-28 00:35:39 +03:00
Vitalii Shpital
917e4b81d3 web/satellite: regular header reworked
WHAT:
removed unnecessary templates splitting
fixed resources dropdown margins
fixes close navigation button responsiveness
aligned logo
added loader for projects dropdown and removed redundant request on dropdown closing
added satellite name to header's right side

WHY:
bug fixes/better user experience

Change-Id: I7ed30f882a1f5484efbf3e0e21ee5d637ef83bf2
2021-04-27 16:40:48 +00:00
Qweder93
a5c1e4b4a5 crashcollect: process for receiving panics
Added crashcollect server for tracking panics
on parent process and saving them in file.

Change-Id: I7926f9a16594227a3262e05d216199b7c2857385
2021-04-27 18:57:47 +03:00
Michał Niewrzał
02460fcc4c satellite/metainfo/expireddeletion: change chore interval to 24h
We want run deletion query more often to avoid long running DB operations.

Change-Id: I72c63bb8ec05cc3ab2e7ab63479ba6cdf830386a
2021-04-27 12:30:23 +00:00
Vitalii Shpital
606a501441 satellite/console: add tardigradeshare.io domain to media-src CSP whitelist
WHAT:
added tardigradeshare.io domain to media-src CSP whitelist

WHY:
we still support it

Change-Id: Id02daa0712cc84f5cb7edaafb21a143859531f26
2021-04-27 11:17:50 +00:00
Vitalii Shpital
f2e06ce882 web/satellite: added skip button for the onboarding flow
WHAT:
added skip button to onboardling flow

WHY:
better user experience

Change-Id: I80953de701e13730019efac38595116c9ce94689
2021-04-27 10:54:29 +00:00
Vitalii Shpital
64c82507c5 web/satellite: make project dashboard delay message more visible
WHAT:
replace info icon with a regular message

WHY:
better user experience

Change-Id: I7edaf1d139f824752eb3c96bcfbba7e5ef2aa09f
2021-04-27 10:54:25 +00:00
Vitalii Shpital
a84b8367dc web/satellite: changed Token to API Key label for CLI step of AG flow
WHAT:
label changed from Token to API Key

WHY:
to remove user's confusion

Change-Id: If30384bb3cf6b32ae7fede0a53af176c73965090
2021-04-27 10:35:01 +00:00
Malcolm Bouzi
c1fdf1ad71 satellite/satellitedb: add have_sales_contact column on users
For business accounts we need to track the sales contact.
It will be a question to business accounts during onboarding.

Change-Id: I8d101ce1b52091478dfb0ddd875e1cc717d765d3
2021-04-26 13:37:26 +00:00
Michał Niewrzał
7944df20d6 storj: use multipart API
Change-Id: I10b401434e3e77468d12ecd225b41689568fd197
2021-04-26 13:15:09 +00:00
Yaroslav Vorobiov
aa27a8f1dd private/crashreportpb: add protobuf for crashreport and crashcollect
Change-Id: I17da796030f63471ae0900142413523e8e12895d
2021-04-26 12:46:03 +00:00
Egon Elbre
948529a600 satellite/metabase/metabasetest: move test utils to a separate package
Change-Id: Iaa607996478497d5c97c6a6973d099b315b3abf0
2021-04-26 13:12:58 +03:00
Yingrong Zhao
307886ffe8 storagenode/piecestore: fix error handling in TestDownload
If a read error is returned, we want to preserve it instead of
overriding it with close error

Change-Id: Ie253a453c7b6b62598b89dffd7635b8266357074
2021-04-23 18:43:45 +00:00
Malcolm Bouzi
31c1137860 web/satellite: add login path
path will be used for a new link

Change-Id: Idb14ea5bd152cf169575b7223fc429eb44d7049e
2021-04-23 15:01:27 +00:00
Yingrong Zhao
08e1ce8930 cmd/connecttest: display more helpful error message and fix tls config
Change-Id: Ibedd643458ee8c84fd4668a23449d99eec43ba73
2021-04-23 14:27:34 +00:00
Egon Elbre
7802ab714f pkg/,private/: merge with private package
Initially there were pkg and private packages, however for all practical
purposes there's no significant difference between them. It's clearer to
have a single private package - and when we do get a specific
abstraction that needs to be reused, we can move it to storj.io/common
or storj.io/private.

Change-Id: Ibc2036e67f312f5d63cb4a97f5a92e38ae413aa5
2021-04-23 16:37:28 +03:00
Egon Elbre
c641ddcb54 pkg/lrucache: rename package
cache is really common variable and type name and we have already used
the package name alias in multiple places.

Change-Id: I6435785b7549b541d533de59ec94557b9bd11e04
2021-04-23 16:37:28 +03:00
Egon Elbre
4d37378129 storage/{cockroachkv,postgreskv}: delete unused code
Change-Id: I11da924508d207d8816bda206d28261931d9cd4e
2021-04-23 16:37:28 +03:00
Egon Elbre
8c62788b24 satellite/metabase/metaloop: stop timer properly
There's a rare chance that `Stop` returns false, however doesn't have
time triggered. Use a non-blocking drain to remove the token.

Change-Id: I1ae18a197424017f0ca76656602709a029b56bfd
2021-04-23 16:35:21 +03:00
Vitalii Shpital
d32515fa5d satellite/console: add storjshare.io domain to media-src CSP whitelist
WHAT:
whitelist .storjshare.io domain for media-src CSP

WHY:
to enable video preview for linksharing

Change-Id: Ib673602d31ca116e7ce1cee0eba17099a55d7dbc
2021-04-23 16:11:41 +03:00
Egon Elbre
a2e20c93ae private/dbutil: use dbutil and tagsql from storj.io/private
Initially we duplicated the code to avoid large scale changes to
the packages. Now we are past metainfo refactor we can remove the
duplication.

Change-Id: I9d0b2756cc6e2a2f4d576afa408a15273a7e1cef
2021-04-23 14:36:52 +03:00
Egon Elbre
dd5eb2616b satellite/metrics: join for monitoring
We don't want metrics to trigger the metaloop on it's own.

Change-Id: I648272a0e3043173b0a377697db1f172712bdda5
2021-04-23 12:21:48 +03:00
Egon Elbre
c1fbecb96b satellite/metabase/metaloop: add Monitor
We need some chores to join without triggering the loop.

For example it's fine to run metrics, only when something else is
running.

Change-Id: I9d8bd16f59c28c540c8d72971bc4e233a8660c02
2021-04-23 12:20:22 +03:00
Egon Elbre
fff21b330d cmd/metabase-verify: tool for verifying metabase state
Currently the tool verifies:
* validity of plain_offset
* whether plain_size is smaller than encrypted_size

Change-Id: I9ec4fb5ead3356a196392c26ca377fcdb367138e
2021-04-23 07:50:36 +00:00
Egon Elbre
6307875203 monkit: fix monkit lock
Previously check-monitoring ignored ScopeNamed.

Change-Id: I3d0d472e722cb19a65b471a914f5124cd224fc34
2021-04-23 06:36:40 +00:00
paul cannon
dcb73e46ca cmd/connect-test: add connect-test tool
This tool attempts to make TCP+TLS+DRPC and QUIC+DRPC connections to a
specified IP:port. If successful with either, it shows the node ID of
the opposite end of the connection.

This should be useful for SNOs who want to check that their packet
forwarding is working or that their node is listening for both
connection types.

Change-Id: I935dff037dd7f1106941a35567f6445230259d1a
2021-04-22 16:06:45 +00:00
Cameron Ayer
a0c5da6643 satellite/satellitedb: in stray nodes DQ, don't DQ nodes where last_contact_success = '0001-01-01 00:00:00+00'
When nodes check in for the very first time, if the satellite can't ping
them back, they are inserted into the nodes table with
last_contact_success of '0001-01-01 00:00:00+00'. If the stray nodes
chore runs before the node can fix their problem, they are DQd.

Solution: when DQing stray nodes, dont DQ where last_contact_success =
'0001-01-01 00:00:00+00'::timestamptz

Change-Id: I477a02d5ef85b2c930ed6b7d99a4d1995169bca8
2021-04-22 10:13:13 -04:00
Egon Elbre
4c9ed64f75 satellite/metabase/metaloop: move loop under metabase
Currently the loop handling is heavily related to the metabase rather
than metainfo.

metainfo over time has become related to the "public API" for accessing
the metabase data.

Currently updates monkit.lock, because monkit monitoring does not handle
ScopeNamed correctly. Needs a followup change to monitoring check.

Change-Id: Ie50519991d718dfb872ec9a0176a82e732c97584
2021-04-22 12:58:09 +03:00
Egon Elbre
ebc981291a satellite/accounting: use correct error in tests
rpcstatus codes are not part of the public uplink API
and the tests should not use them.

Change-Id: I9c96c557bf2dac5d6fff200fb217378524fd14ff
2021-04-22 07:25:42 +00:00
Malcolm Bouzi
52c3d7d642 web/satellite: add disabled state to onboarding overview RClone link
Change-Id: I4cef2d0772967f47e9fa063494de9f8903eb4f29
2021-04-21 19:34:55 +00:00
Yingrong Zhao
43f5888052 satellite/contact: only test PingBack failure case
The original test caused the testplanet to timeout due to contact
service stuck in a loop. We can change it to only test the failure case for
PingBack method instead of closing the TCP port on storagenodes.

Change-Id: Ic96aee637b39ae95050c6902c2bf9ca51fb586c3
2021-04-21 14:20:36 -04:00
Yaroslav Vorobiov
244c488b10 storagenode: docker image autoupdate binaries
Get storagenode and storagenode-updater binaries during
run of the container to not to release new docker image
on each new version of the storagenode binary.

Change-Id: Ic0eb4a9c18a98598dfd9b96c1d352c7399496fd2
2021-04-21 17:56:20 +00:00