Commit Graph

6192 Commits

Author SHA1 Message Date
Jeff Wendling
58bd85cbf4 uplinkng: some windows test fixes
Change-Id: Idbe66028ac77d45eff9ea1926e6e8bd66491ea19
2021-08-26 17:52:13 +00:00
Moby von Briesen
1549a6a68a web/satellite: Use OSP logo
Remove Storj DCS logo and replace it with the Storj OCS logo - the dcs
logo will replace this from our branding repo.

Change-Id: I01daa67380f04bf62df980a2c84b3e48bccd63f1
2021-08-26 16:52:15 +00:00
Vitalii Shpital
0376fcdaef integration/ui/satellite: fixed tests for onboarding flow
Fixed tests to pass

Change-Id: I90c61168a3a67188e586564a5df381f2f41e2834
2021-08-26 19:07:52 +03:00
Yingrong Zhao
445ebf86c0 satellite/satellitedb: allow setting rate limit for a project to 0
If a user starts to abuse our system, we need the ability to manually
disable their access

Change-Id: Ia31964cedb830323da69c1153a84d610cf8310cd
2021-08-26 13:45:01 +00:00
Michał Niewrzał
211a630982 cmd/uplink: add parallelism flag for single object download
Adds support for new uplink method DownloadObjectAt which
gives ability to download single object in parallel.

Change-Id: I8388653429992b0d24c383d17d7e90904203fe77
2021-08-26 13:13:38 +00:00
Michał Niewrzał
5c91ecd271 satellite/metainfo: endpoint cleanup
Two small cleanups:
* merging private commitObject, commitSegment,
makeInlineSegment with its public versions. We were
using it when pb.Pointer was still used.
* removing unused CreatePath method

Change-Id: Ib18b07473d91259335dab874559ef52412ab813d
2021-08-26 09:52:38 +00:00
Michał Niewrzał
c29734ef64 satellite/accounting/tally: remove commented metrics
This change removes metrics which were moved to metrics chore.

Change-Id: I5ddff428cae10ac5e94d8f45691d698958579a61
2021-08-26 09:29:55 +00:00
Moby von Briesen
7b4a09c1ea satellite/console: Allow basic headers in CORS preflight
Currently, post requests originating from www.storj.io are failing
because the content-type header is not permitted for the /register
endpoint.

Change-Id: I14f21c2a27d8f0c8bae3609a5bf4b1e69a72c119
2021-08-25 12:47:14 -04:00
Egon Elbre
6daad68735 web/satellite: don't require explicit RootGetter type
There's no need to define RootGetter type on StoreModule,
because Context already should have it defined.
Although the invocation to get that type is rather arcane.

Change-Id: I9525c0e4853f4c5211bdaf1645f0a27c26af8f53
2021-08-25 13:16:24 +03:00
Egon Elbre
ad0b19fb02 web/satellite: fix lint issues
Fix svg.d.ts definition.

Disable no-explicit-any in src/api, because wrangling all the GraphQL
result types properly is not that nice. We can either fix this later
manually, generate GraphQL types or remove the GraphQL endpoints.

Add annotations to src/store/. Currently it still uses any in places and
also defines more types than absolutely necessary. This is an
unfortunate side-effect of the vuex api. There does seem to be an
alternative package that handles them, but to minimize the number of
changes, we'll currently use these types. Due to those decisions it's
also not easily possible to have types instead of any in multiple
places.

StripeCardInput currently uses any, however, if we find the proper
declarations, we can replace them later.

Change-Id: I2ec8bf7fdd8023129d1f8739ce2b6d97de2a58d0
2021-08-25 06:01:19 +00:00
Moby von Briesen
101bbc4414 satellite/console: Fix flaky TestMFA test
Change-Id: Id4fa072f94a4102f3df1bd3aa99984bdd0724497
2021-08-24 17:42:59 +00:00
Egon Elbre
f0c3514b24 web/satellite: remove uses of v-html
Change-Id: Ief9d8af327c02212a6c138773af9f1dc8028a5b6
2021-08-24 13:29:35 +03:00
Egon Elbre
6f2258d870 web/storagenode: fix use of v-html
Change-Id: I0ebbefc7b562c712dd30158c6c688aa0c524abac
2021-08-24 13:02:42 +03:00
Michał Niewrzał
b7555980c4 satellite/metainfo: add zombiedeletion chore
This chore will be deleting zombie objects and it's segments.

Change-Id: Ia5d92dd6c3f52e178d4d43d9603732310761e027
2021-08-24 08:22:06 +00:00
Jeremy Wharton
6a6cc28fc1 satellite/console,private/web: Rate limit coupon code application
Rate limits application of coupon codes by user ID to prevent
brute forcing. Refactors the rate limiter to allow limiting based
on arbitrary criteria and not just by IP.

Change-Id: I99d6749bd5b5e47d7e1aeb0314e363a8e7259dba
2021-08-23 17:00:31 +00:00
Michał Niewrzał
0344790c20 satellite/metabase: delete zombie object that has no new segments for specific period of time
Added options flag to define after which object won't be marked as inactive. All segments CreatedAt
time needs to be bellow this flag to treat object as inactive.

Change-Id: Ib5cffc776c6ee1b62b51eb8595438f968b42528c
2021-08-23 08:25:38 +00:00
NickolaiYurchenko
1e02504be6 web/multinode: satellite, node, and usage request handling fix
Change-Id: Ica315fa9b78bd8e4d2d0db0f4f05f295b8991053
2021-08-20 21:33:26 +00:00
storj-antonio
8c8a3e217f testsuite/ui: move from integration
Move ui tests to testsuite/ and introduce a new go.mod.
This prevents cyclic references when building edge binaries for testing.

Change-Id: I613fdc21ef9eaffba2052f6ad973bfa18c7c86bd
2021-08-20 16:33:53 +00:00
Cameron Ayer
5a1a29a62e satellite/audit: fix containment bug where nodes not removed
When a node gets enough timeouts, it is supposed to be removed
from pending_audits and get an audit failure. We would give them
a failure, but we missed the removal. This change fixes it.

Change-Id: I2f7014e28d7d9b01a9d051f5bbb4f67c86c7b36b
2021-08-20 14:48:27 +00:00
Cameron Ayer
70296c5050 satellite/audit: change wording of audit worker error log
"audit failed" is already used when a node fails an audit. That makes
searching for this higher level audit worker error more difficult.
Additionally, the presence of errors from the audit worker doesn't
necessarily mean the audit failed. Reword the error message to
"error(s) during audit"

Change-Id: I0aab12c73c18d4bd962c5d8ac8a17cabcec022e6
2021-08-20 13:27:16 +00:00
Artur M. Wolff
e452f85163 satellite/metabase: sync batchSizeLimit and ListLimit constants
This change syncs batchSizeLimit and ListLimit constants to prevent
throwing away results returned while listing with a maximum returns
limit.

Change-Id: Ie2425542d945cb88653dcc34c079737bb32320d4
2021-08-20 11:01:46 +00:00
Vitalii Shpital
ca5215c481 web/satellite: rework passphrase step of objects flow
Combined generate/enter passphrase into 1 component.
Designs were significantly chenged.

Change-Id: I53a533af1b1aad1a270f16c432c6d28101dde966
2021-08-20 10:26:59 +00:00
NickolaiYurchenko
fafabaa501 web/satellite: redundant images deletion
Change-Id: I831804877d3b257f16a489e0d8b4226ac3cd1191
2021-08-20 08:51:58 +00:00
Jeff Wendling
2782e000ac cmd/uplinkng: parallel rm
Change-Id: I4f911ce1f384f13812986a34b96a94ded70cfb4e
2021-08-19 17:48:53 +00:00
Jeff Wendling
d57583dfd4 cmd/uplinkng: initial setup
Change-Id: If4df3ec8b3b554f5228d43e97503eb8a87525b23
2021-08-19 17:48:49 +00:00
Jeff Wendling
7eaa7490a1 cmd/uplinkng: secret prompting
Change-Id: I1b407b59559281c2242b55ebcf835d465db38e1d
2021-08-19 17:48:45 +00:00
Artur M. Wolff
a5371353bf satellite/metabase/segmentloop: fix rate.NewLimiter call
This change fixes the call to rate.NewLimiter to account for the
"infinite" case.

Change-Id: Ib3f914ca33ad8b981157fd224f077e6ad8d8c644
2021-08-19 17:23:13 +00:00
Egon Elbre
704cad8da4 satellite/console/consoleweb: add endpoint tests
This currently contains tests for both graphql and regular endpoints.

Co-authored-by: Antonio Franco <antonio@storj.io>
Change-Id: I28d7e629b1caa114438d6fbc3abcc079a8ca10a6
2021-08-19 12:22:45 +03:00
Jeremy Wharton
96e39018c7 satellite/console,web/satellite: Allow disabling MFA with recovery code
This change allows users to disable multi-factor authentication
with a recovery code. Previously, users could only disable MFA
with a passcode.

Change-Id: Iec20bf7d8f6781182b81d1f17d9641491dbc8460
2021-08-18 14:34:17 +00:00
Moby von Briesen
ef9a5210a4 satellite/payments: Add command to ensure free tier coupons
This command is intended to be run as part of invoice generation - it
iterates over Stripe customers, and applies the free tier coupon to any
customer who doesn't already have a coupon.

This way, we can ensure that all customers have at least the free tier
coupon before and after invoice generation, in case a different coupon
has expired.

Change-Id: I33a4aff9174049f9e051de53ef65298ca65ed688
2021-08-18 13:48:44 +00:00
Jeremy Wharton
24b39b7559 web/satellite: Warn user if number of MFA recovery codes is low
Adds a banner at the top of the dashboard warning the user if he
has 3 or less MFA recovery codes.

Change-Id: I3271f1fe33501eaeeeca343bf60ca899e80f0fa1
2021-08-18 13:18:34 +00:00
dlamarmorgan
cc083dbdc9 web/satellite,satellite/console: Allow paid tier users to edit limits
Added editable fields to the project details page for Storage Limit and Bandwidth limit. Leveraged existing types when possible.

Added fixed checking into the limits to prevent reducing limits beyond current usage, as well as limiting usage to less than the default paid tier maximum.

Change-Id: I07ce53470919a8a9d4dce56ade6904ede8daf34c
2021-08-18 00:07:10 +00:00
Moby von Briesen
1fa0cfbfe0 satellite/console: Add CORS headers for register endpoint
Set the Access-Control-Allow-Origin header if a request is made to the
registration endpoint from storj.io or www.storj.io. This allows the
client to make the request with no issues.

Change-Id: I0cd747ececfe877fb5f834d8292307fc14e3db5c
2021-08-16 17:46:08 -04:00
Sean Harvey
d22ecf9ec0 Jenkinsfile.public: pull latest ci image instead of building
Change-Id: Ia0e6863fdd00127336839d4d0b81306611aee177
2021-08-17 06:47:29 +12:00
Artur M. Wolff
2cd68bf4fb private/lrucache: import from common
Change-Id: Ia1f43d0440fef21122b071b05da59b4cf2689d6c
2021-08-16 10:04:32 +00:00
Bill Thorp
549e799bbe satellite: wrap notfound on failed deletes as DRPC errors
We're seeing BeginDeleteObject in metaclient returning object not found:
metabase: no rows deleted in the Gateway-MT mint tests.  There's a
client check for rpcStatus.NotFound, but the metabase endpoint isn't
wrapping the db error as a DRPC error.

Here's the chain:

  gateway.AbortMultipartUpload()
  project.AbortUpload()
  metainfoClient.BeginDeleteObject() <- understands DRPC errors
  endpoint.DeletePendingObject() <- where this code is
  db.DeletePendingObject() <- returns error

Change-Id: I93991de76487426df0a807b0d1e69fc975196a1a
2021-08-16 09:02:35 +00:00
Jeremy Wharton
ea772a8480 web/satellite: Allow users to see coupon in billing area
This change allows users to see the Stripe coupon applied to their
account in the billing area.

Change-Id: Ie1e810bfb2847f9b0c0bb827d5ca03c16cf5e818
2021-08-14 16:09:54 -05:00
Yingrong Zhao
6c34ff64ad satellite/satellitedb: remove referrence to audit information in
nodes and audit_history tables

This PR removes all code reference to audit_histories table and
```
audit_reputation_alpha, audit_reputation_beta,
unknown_audit_reputation_alpha, unknown_audit_reputation_beta,
```
columns from nodes table.

It also drops audit_histories table from the db since the code
that's referencing it currently are not being used.

Change-Id: Ifcda8db36afb3a333d487ff831f2fdefc8b02a4c
2021-08-13 21:11:28 +00:00
Jeremy Wharton
51ebc564d9 web/satellite,satellite/console: Overhaul password reset
Updates the password reset page to use the new theme.
Adds new endpoint '/api/v0/auth/reset-password'
for password reset.

Additionally, updates the link-clicking mail simulator to only
click links with a specified attribute. Otherwise, the password reset
cancellation link would be clicked before the password reset link
could be accessed, rendering testing impossible.

Change-Id: I8fde74ef7ad980880a7bf6558e3b9ed31509a393
2021-08-12 17:40:53 +00:00
Egon Elbre
f807518477 ci: try fix go build scanning node_modules directory
Add a go.mod to node_modules folder, that should prevent

  go build ./...

from scanning it, however, it's slightly hacky.

Change-Id: I073150704bd90afd8ccf38a969e0a1c725d25137
2021-08-12 18:11:11 +03:00
Egon Elbre
e43af5eb3d web/: enable more vue eslint rules
Fixes two things:

All button tags must have type field specified, otherwise it defaults to
submit, which is mostly not correct for our site.

Disallows unused refs.

Change-Id: I28bb44aab9c153f83fec16fc4023a453444c3926
2021-08-12 16:56:37 +03:00
Malcolm Bouzi
92c53afb84 satellite/{payments,console},web/satellite: Adds confirmation step if user already has coupon code applied and wants to replace it
Change-Id: I04d40d3b25bd67e29c043d651541ff300b5379ac
2021-08-11 20:04:23 +00:00
Vitalii Shpital
c5d5229716 web/satellite: update Overview step of onboarding flow
Updated the very first screen after sign up according to new designs

Change-Id: I03521e77ca2331e8ec4d9bf7094eccff7178bc58
2021-08-11 19:36:52 +03:00
Jeremy Wharton
9eb8dc0c79 web/satellite: Fix invalid font reference
Replaces all occurrences of the non-existent 'font_normal'
with 'font_regular'. References to 'font_normal' would always
fallback and never display the proper font.

Change-Id: I2cb98a5bbec33b45f7e4ec6ba67bc5be63b5b523
2021-08-11 15:48:59 +00:00
Artur M. Wolff
2f7e670996 satellite/metainfo: allow per-project rate limit of zero
Change-Id: I237c67dfa3d24ed4837175556f72b9c761644435
2021-08-11 11:45:58 +00:00
Egon Elbre
b8280bd057 web/satellite: enable vue/recommended linting
Change-Id: I73cd1e72f369a172f9416be9b1220cbbd6c77637
2021-08-11 11:10:51 +03:00
Egon Elbre
abf6bf9507 web/storagenode: enable vue/recommended linting
Change-Id: I0efefb9fdc1056ea3fb6d94951f8bb0231695aaa
2021-08-11 11:10:51 +03:00
Egon Elbre
5d8cbf0871 web/multinode: enable vue/recommended linting
Change-Id: Ice51a66d8c73596133e10aa969a6ca973441e46e
2021-08-11 11:10:51 +03:00
Jeff Wendling
59b8422318 cmd/uplinkng: implement revoke and normalize some language
this makes the distinction between an "access name" and
an "access value" and talks about which is expected for
commands. most are "access name or value".

Change-Id: I43c0043a17d37e89ab5f87388ae9e890a8b59958
2021-08-11 02:30:13 +00:00
Jeff Wendling
08d860570b cmd/uplinkng: parallelsm and a ton of fixes
this was just supposed to add parallel uploads/downloads
and it does do that, but i then found a bunch of bugs
with respect to path handling that i thought i had under
control. oops.

so this adds a ton of tests and tries to make the logic
in ulloc to be more consistent. almost all of the actual
file handling bits and knowledge happens in cmd_cp now
where it should belong.

additionally, the s3 command has the behavior that if your
bucket has the file s3://bucket/file, then executing
s3 ls s3://bucket/fi returns nothing. this change makes
uplinkng match that behavior even if i don't personally
like it.

a big portion of the weirdness is the concept introduced
that i've named "directoryish", which intends to capture
the behavior that if a user copies a file to that location
then the base name of the source should be appended on
rather than a direct copy. this concept is entirely a
based on the string value and not the actual filesystem
state. hence, the cp command is responsible for checking
if local paths are actually a directory, and adding a
trailing slash if necessary to make them "directoryish".
additionally, the empty key for a bucket and the empty
string for local paths are considered "directoryish".

Change-Id: I9120d18616fd813b29ff81beed4f5993caa99fb6
2021-08-11 02:30:06 +00:00