Commit Graph

6303 Commits

Author SHA1 Message Date
Cameron Ayer
251e432e5b satellite/satellitedb: drop contained from nodes and reputations
A few months ago we removed all references to the contained
column in nodes and reputations

bb21551a9c
and
56fe636123

But we never did the migration to drop the columns.
This commit will finally do that.

Change-Id: I82aa2f257b1fb14a2f1c4c4a1589f80895360ae4
2022-01-20 19:46:48 +00:00
Moby von Briesen
464b4b02fa satellite/console/consoleweb: Only replace some kinds of characters in name
The previous change (59648dc272) ends up removing a lot of characters
for valid non-English names. Instead, only replace URL characters such
as slashes, colons, and periods. Since someone may use these characters
to separate two parts of a name, e.g. Name1/Name2, replace these
characters with a hyphen.

Change-Id: I4cc3d1bdb05d525a83970cf1b42479414c9678e7
2022-01-20 11:13:26 -05:00
Clement Sam
2de6feefde satellite/contact: fix connection leak in PingMe endpoint
Fixes connection leak in pingMe endpoint.
includes other minor fixes.

Change-Id: I7c61f620565f46dd113d21a772de7c439be550e3
2022-01-20 14:20:24 +00:00
Clement Sam
8b40a071a0 storagenode: check if QUIC is properly configured
This change adds a check to confirm if UDP port if properly configured for QUIC

Resolves https://github.com/storj/storj/issues/4332
Partly resolves https://github.com/storj/storj/issues/4358

Change-Id: I9a66f26a115e48b4fcd168f50a7d0b4d81712f4e
2022-01-20 12:04:04 +00:00
Moby von Briesen
59648dc272 satellite/console: Filter special characters out of name
When a user is created, but before verification or forgot password email
is sent, remove any special characters in the provided name. This
protects us against certain phishing attacks.

Change-Id: Ieddd3479da20eb80b9f1b56eb86c8f46bca2642c
2022-01-20 00:18:56 +00:00
Clement Sam
9f3c1f9cda satellite/contact: add PingMe endpoint
Change-Id: I832a72fafeacf76ad64a0129bcc6582cc4f9290d
2022-01-19 17:52:33 +00:00
Michał Niewrzał
95750b9d32 satellite/console: verify limits while test
Additional test case to be sure that update operation went well.

Change-Id: I7dbf1e3110f716712ba9747a1915b54112b89801
2022-01-19 16:39:36 +00:00
NickolaiYurchenko
01287913ec web/satellite: graphql error logging added
most of project related errors fixed to show correct message
found out that update project mutation is broken

Change-Id: Ia3492d34e9a9ef0ce0f0ea4591bd48d5cdbb5f5a
2022-01-19 14:06:05 +00:00
Qweder93
228f465d45 satellite/metainfo: сombine checks for storage and segment limit
We need to combine methods from accounting.Service (ExceedsStorageUsage and ExceedsSegmentUsage)
to run checks concurrently.

Resolves https://github.com/storj/team-metainfo/issues/73

Change-Id: I47831bca92457f16cfda789da89dbd460738ac97
2022-01-19 10:44:04 +00:00
Michał Niewrzał
270c6e24f2 satellite/console: fix default limit for free-tier segment usage limit
Free-tier segment usage limit was defined as 150k, not 140k. This change
is correcting that.

https://github.com/storj/team-metainfo/issues/8

Change-Id: I71ec0961930b19fd09b2b996e01acd406a8dcf8f
2022-01-19 09:21:13 +00:00
NickolaiYurchenko
a81af2829e web/satellite: error while adding credit card handled
Change-Id: I995eb4f07a62c26052226ddbbf06ee332323a0dd
2022-01-18 17:53:07 +00:00
Mya
309ac41bfc satellite/satellitedb/dbx: add schema for oauth tokens
Change-Id: Ia980766a8f86b662545e7d8cae6ecf631e199809
2022-01-18 09:38:12 -06:00
Mya
83f7d3e045 satellite/satellitedb/dbx: add schema for oauth clients
Change-Id: Ib08c7074803805cca8ffaedbd89da54504842af7
2022-01-18 09:38:12 -06:00
Michał Niewrzał
36c07720bb satellite/metainfo: bound user agent metrics
We need to limit number of results for SeriesTag to avoid
overloading our metric system.

Change-Id: I3d9f4f4f6d489bf4ce87c6240f66d48b29b88abb
2022-01-18 12:42:09 +00:00
Jeremy Wharton
1007e6300e web/satellite: Distinguish between file and folder upload buttons
Previously, there was no difference between the "Upload File" and
"Upload Folder" buttons in the Object Browser aside from the text.
This change adds file and folder upload icons to their respective
buttons.

Change-Id: If9766c12420d9a9bc45348c0b9b0bdf6b42b8922
2022-01-16 02:15:00 +00:00
Mya
cf462fcf92 satellite/admin: update geofence endpoints to follow response conventions
Change-Id: Ieff91c16f91f9720e9c3d8ead0e13c47651d23f5
2022-01-13 12:33:10 -06:00
Michał Niewrzał
6bbe2fc692 cmd/uplinkng: fix flag/error messages
This change fixes some small issues with flag descriptions
and error messages.

Change-Id: I74a669d2566bf51ad871f16d64a92984ae75e712
2022-01-13 13:36:11 +00:00
Erik van Velzen
4f96a85642 cmd/uplink/share: register access via DRPC
Convert registering access with the edge services
from the HTTP protocol to DRPC protocol

Change-Id: Iba88dd0758c26f613cf501be9a20ead07d122d0b
2022-01-13 13:24:05 +00:00
Michał Niewrzał
5e9643e1b8 satellite/metainfo: cleanup validation
Refactoring to do few things:
* move simple validation before validations with DB calls
* combine validation check/update for storage and segment
limits together

Change-Id: I6c2431ba236d4e388791d2e2d01ca7e0dd4439fc
2022-01-13 13:00:40 +01:00
Michał Niewrzał
18ab464b51 satellite/metainfo: UpdateProjectSegmentUsage enabled only with segment validation
updating project segment usage entry to redis is enabled only if
segment validation is enabled in config value, so no incompatible
changes will be added while deployment.

Change-Id: I1288cb9ff0a8a00f095dc94e20d2f14393e9a613
2022-01-13 12:12:25 +01:00
Ivan Fraixedes
3a987ac00f satellite/admin: Fix client API & docs
The following 2 commits added 2 new query parameters to set the `burst`
and `segments` limits for a project and also to new fields to the
response JSON object body to the "get project limits" endpoint:

* c911360eb5
* b7b010adc9

However, the API documentation and the Typescript client API (used by
the UI) weren't updated with them.

Later, the commit dc6128e9e2 updated the
Typescript client API with the `segments` limit but it didn't update the
documentation to reflect it.

This commit brings all things that were missed in those previous
commits.

Change-Id: Iff12cdd4a0d3c448cd73b57a98d171ba468d2c98
2022-01-13 09:31:01 +00:00
Ivan Fraixedes
8e80a51b64 Makefile: Set NodeJS version to use in var
Add new variable in the Makefile to specify the used NodeJS version so
it can be referenced wherever else in the file rather than hard-coding
it.

Having in to a variable makes more visible and quicker to find out which
NodeJS version the project uses.

Change-Id: I87abc3b3e5489407638618f8199aec7c2f40ca9f
2022-01-13 08:52:26 +00:00
Stefan Benten
44a73b01d9 cmd/storagenode: use contact address instead of server address
Currently the address being used is most of the time just :28967, which is not the correct address to reach the node from the public on.
This change uses the designated contact external address value that contains the set and preferred way to reach the node.

Change-Id: I99e979c2541043755b81e65c36c4289bfa3f60f3
2022-01-12 15:15:09 +00:00
Michał Niewrzał
eb0d08d59b satellite/metainfo: collect versions of user tools
We want to know usage statistics for our main tools
like uplink-cli or rclone. Initially we will collect
only usage stats without relation to specific process
e.g. download or upload.

Change-Id: I203b1a6c07ae014e710368f77163f13fdf10763c
2022-01-12 13:49:15 +00:00
Yaroslav Vorobiov
e0b3c7152b satellite/nodestats: use overlay node for reputation status
Use overlay db node as primary source of truth for reputation status when node
query it's status via Endpoint.Get. If node was was disqualified in overlay reputation
status may be inconsistend with reputation table.

Change-Id: I1bf858a4b020324035847a9215ef29cf5432f6a4
2022-01-11 16:38:27 +00:00
Qweder93
b6625cadea satellite/metainfo: uncomment tests, renamed EncryptedPath into EncryptedObjectKey
moved tests back and renamed params that were renamed on uplink side

Change-Id: I4a9592bd955f2217c5809c235c394a4e78d337ea
2022-01-11 17:18:18 +02:00
Mya
bc3392cda4 cmd/uplinkng: fix clingy argument usage
Change-Id: I779a084edf14e0ee1c600f8e564dfc39aae073cb
2022-01-10 16:24:53 -06:00
Qweder93
332b673a05 satellite/metainfo: comment tests for rename
comment tests that contains fields needs
to rename on uplink side without breaking compatibility.
after rename tests will be moved back from comments.

Change-Id: I3bc4aff6ae7f6711ade956ac389f0d7e1a1ab91a
2022-01-10 17:46:32 +02:00
Qweder93
15c2b2cc10 satellite/metainfo: comment tests for rename
comment tests that contains fields needs
to rename on uplink side without breaking compatibility.
after rename tests will be moved back from comments.

Change-Id: I439783c62678c32805a85aa52bef1d2b767543a1
2022-01-10 14:44:43 +00:00
Qweder93
dc6128e9e2 satellite/admin: add update project segment limit on ui
Added possibility to update project segment limit on
satellite admin panel ui.

Resolves https://github.com/storj/team-metainfo/issues/5

Change-Id: I192d0c9adbab411556aa46673167b3ff7437d04b
2022-01-10 13:26:56 +00:00
Qweder93
8be655c487 satellite/metainfo: add segment limit validation, caching
We want to be able to limit the number of segments per project for users.
To limit this we need to check limit value associated with project
and value of used segments already in BeginMoveObject, BeginMoveSegment
and increment cache segments usage after each CommitSegment call.

Resolves https://github.com/storj/team-metainfo/issues/1

Change-Id: I6290e67c095a174b9d101c4521802d9bfe0453b8
2022-01-10 14:33:26 +02:00
Stefan Benten
f0ada87a9f
web/satellite: fix FileModal Share Link (#4341)
Co-authored-by: littleskunk <jens.heimbuerge@googlemail.com>
2022-01-07 22:21:05 +01:00
Stefan Benten
35113634f6 Makefile: do not remove files to keep state clean
Currently the admin UI Makefile target deletes the .gitignore file.
While it being unnecessary to my findings it also causes and dirty
git repository, since files have been modified.
Removing this command lets the build process continue to work as expected
and since we are not commiting any files or assets later on this should
be fine.

Change-Id: Ibc1b7a8e456394ca19ea39dd8389c2ec03f066fc
2022-01-07 20:36:50 +01:00
Yingrong Zhao
1f8f7ebf06 satellite/{audit, reputation}: fix potential nodes reputation status
inconsistency

The original design had a flaw which can potentially cause discrepancy
for nodes reputation status between reputations table and nodes table.
In the event of a failure(network issue, db failure, satellite failure, etc.)
happens between update to reputations table and update to nodes table, data
can be out of sync.
This PR tries to fix above issue by passing through node's reputation from
the beginning of an audit/repair(this data is from nodes table) to the next
update in reputation service. If the updated reputation status from the service
is different from the existing node status, the service will try to update nodes
table. In the case of a failure, the service will be able to try update nodes
table again since it can see the discrepancy of the data. This will allow
both tables to be in-sync eventually.

Change-Id: Ic22130b4503a594b7177237b18f7e68305c2f122
2022-01-06 21:05:59 +00:00
littleskunk
b21cbc85f1
satellite/orders: log level warn for remaining "bucketName or projectID not set" (#4326)
Co-authored-by: Stefan Benten <mail@stefan-benten.de>
2022-01-06 16:31:26 +01:00
nadimhq
606a73a8e8
docs/testplan: Adding a testplan for Project Dashboard (#4318)
This is a Testplan for the Project Dashboard that follows the Testplan template.
2022-01-06 07:58:23 -05:00
paul cannon
171f908d8b cmd/satellite: rename monkit stat with '_', not '-'
Change-Id: Idb7d2971fbef1145f281b728a492e40e04add155
2022-01-06 02:30:30 +00:00
dlamarmorgan
ab37b65cfc satellite/{accounting,orders,satellitedb}: group bucket bandwidth rollups by time window
Batching of the order submissions can lead to combining the allocated
traffic totals for two completely different time windows, resulting
in incorrect customer accounting. This change will group the batched
order submissions by projectID as well as time window, leading to
distinct updates of a buckets bandwidth rollup based on the hour
window in which the order was created.

Change-Id: Ifb4d67923eec8a533b9758379914f17ff7abea32
2022-01-05 20:24:48 +00:00
Egon Elbre
5d3085f38d testuite/ui/satellite: cleanup and skip tests
Each test doesn't need to be in a separate file.
Keep naming consistent such that relevant tests can be run together.
Also skip test that do not work at the moment.

No functional changes.

Change-Id: Icd314d383ec5b683dea8afaaf2790e2e4b306671
2022-01-05 17:16:42 +02:00
Egon Elbre
d06ba56bc6 web/satellite: fix invalid bucket name message
Change-Id: If69a032d94757e9830dcfad6c36932aae5098d5b
2022-01-05 17:16:35 +02:00
Egon Elbre
a42b9d1a48 all: fix uses of email.com
email.com is not a domain that should be used for examples
nor tests.

Change-Id: I654d4287d02633d5ed9740e81a79150470eeaf25
2022-01-05 16:29:19 +02:00
Egon Elbre
04d04c0306 web/satellite: remove old object browser
Change-Id: Ief16a3887444df093c644831d86a9b3e865e5867
2022-01-05 16:29:19 +02:00
nadimhq
0c3a7b4575 Update login_unverified_nonexistent_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
35712f4deb Update login_unverified_nonexistent_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
ba1bff5367 Update login_reset_password_verified_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
edf17754e5 Update login_reset_password_verified_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
14c64318a5 Update login_unverified_nonexistent_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
bc7a9593ac Update login_reset_password_unverified_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
bde3a884c4 testsuite/ui/satellite: email verification UI tests
Added UI tests surrounding email verification
2022-01-05 14:24:04 +01:00
Malcolm Bouzi
12cb733b16 satellite/satellitedb: add last_verifcation_reminder column to users table
We want to issue a reminder to users when they don't verify their email within 24hrs of registering. This change only adds a column to the users table.

Change-Id: I92e2baeabf179338ffec01574d4752c0ccdba88b
2022-01-04 19:38:04 +00:00