Commit Graph

6392 Commits

Author SHA1 Message Date
Mya
309ac41bfc satellite/satellitedb/dbx: add schema for oauth tokens
Change-Id: Ia980766a8f86b662545e7d8cae6ecf631e199809
2022-01-18 09:38:12 -06:00
Mya
83f7d3e045 satellite/satellitedb/dbx: add schema for oauth clients
Change-Id: Ib08c7074803805cca8ffaedbd89da54504842af7
2022-01-18 09:38:12 -06:00
Michał Niewrzał
36c07720bb satellite/metainfo: bound user agent metrics
We need to limit number of results for SeriesTag to avoid
overloading our metric system.

Change-Id: I3d9f4f4f6d489bf4ce87c6240f66d48b29b88abb
2022-01-18 12:42:09 +00:00
Jeremy Wharton
1007e6300e web/satellite: Distinguish between file and folder upload buttons
Previously, there was no difference between the "Upload File" and
"Upload Folder" buttons in the Object Browser aside from the text.
This change adds file and folder upload icons to their respective
buttons.

Change-Id: If9766c12420d9a9bc45348c0b9b0bdf6b42b8922
2022-01-16 02:15:00 +00:00
Mya
cf462fcf92 satellite/admin: update geofence endpoints to follow response conventions
Change-Id: Ieff91c16f91f9720e9c3d8ead0e13c47651d23f5
2022-01-13 12:33:10 -06:00
Michał Niewrzał
6bbe2fc692 cmd/uplinkng: fix flag/error messages
This change fixes some small issues with flag descriptions
and error messages.

Change-Id: I74a669d2566bf51ad871f16d64a92984ae75e712
2022-01-13 13:36:11 +00:00
Erik van Velzen
4f96a85642 cmd/uplink/share: register access via DRPC
Convert registering access with the edge services
from the HTTP protocol to DRPC protocol

Change-Id: Iba88dd0758c26f613cf501be9a20ead07d122d0b
2022-01-13 13:24:05 +00:00
Michał Niewrzał
5e9643e1b8 satellite/metainfo: cleanup validation
Refactoring to do few things:
* move simple validation before validations with DB calls
* combine validation check/update for storage and segment
limits together

Change-Id: I6c2431ba236d4e388791d2e2d01ca7e0dd4439fc
2022-01-13 13:00:40 +01:00
Michał Niewrzał
18ab464b51 satellite/metainfo: UpdateProjectSegmentUsage enabled only with segment validation
updating project segment usage entry to redis is enabled only if
segment validation is enabled in config value, so no incompatible
changes will be added while deployment.

Change-Id: I1288cb9ff0a8a00f095dc94e20d2f14393e9a613
2022-01-13 12:12:25 +01:00
Ivan Fraixedes
3a987ac00f satellite/admin: Fix client API & docs
The following 2 commits added 2 new query parameters to set the `burst`
and `segments` limits for a project and also to new fields to the
response JSON object body to the "get project limits" endpoint:

* c911360eb5
* b7b010adc9

However, the API documentation and the Typescript client API (used by
the UI) weren't updated with them.

Later, the commit dc6128e9e2 updated the
Typescript client API with the `segments` limit but it didn't update the
documentation to reflect it.

This commit brings all things that were missed in those previous
commits.

Change-Id: Iff12cdd4a0d3c448cd73b57a98d171ba468d2c98
2022-01-13 09:31:01 +00:00
Ivan Fraixedes
8e80a51b64 Makefile: Set NodeJS version to use in var
Add new variable in the Makefile to specify the used NodeJS version so
it can be referenced wherever else in the file rather than hard-coding
it.

Having in to a variable makes more visible and quicker to find out which
NodeJS version the project uses.

Change-Id: I87abc3b3e5489407638618f8199aec7c2f40ca9f
2022-01-13 08:52:26 +00:00
Stefan Benten
44a73b01d9 cmd/storagenode: use contact address instead of server address
Currently the address being used is most of the time just :28967, which is not the correct address to reach the node from the public on.
This change uses the designated contact external address value that contains the set and preferred way to reach the node.

Change-Id: I99e979c2541043755b81e65c36c4289bfa3f60f3
2022-01-12 15:15:09 +00:00
Michał Niewrzał
eb0d08d59b satellite/metainfo: collect versions of user tools
We want to know usage statistics for our main tools
like uplink-cli or rclone. Initially we will collect
only usage stats without relation to specific process
e.g. download or upload.

Change-Id: I203b1a6c07ae014e710368f77163f13fdf10763c
2022-01-12 13:49:15 +00:00
Yaroslav Vorobiov
e0b3c7152b satellite/nodestats: use overlay node for reputation status
Use overlay db node as primary source of truth for reputation status when node
query it's status via Endpoint.Get. If node was was disqualified in overlay reputation
status may be inconsistend with reputation table.

Change-Id: I1bf858a4b020324035847a9215ef29cf5432f6a4
2022-01-11 16:38:27 +00:00
Qweder93
b6625cadea satellite/metainfo: uncomment tests, renamed EncryptedPath into EncryptedObjectKey
moved tests back and renamed params that were renamed on uplink side

Change-Id: I4a9592bd955f2217c5809c235c394a4e78d337ea
2022-01-11 17:18:18 +02:00
Mya
bc3392cda4 cmd/uplinkng: fix clingy argument usage
Change-Id: I779a084edf14e0ee1c600f8e564dfc39aae073cb
2022-01-10 16:24:53 -06:00
Qweder93
332b673a05 satellite/metainfo: comment tests for rename
comment tests that contains fields needs
to rename on uplink side without breaking compatibility.
after rename tests will be moved back from comments.

Change-Id: I3bc4aff6ae7f6711ade956ac389f0d7e1a1ab91a
2022-01-10 17:46:32 +02:00
Qweder93
15c2b2cc10 satellite/metainfo: comment tests for rename
comment tests that contains fields needs
to rename on uplink side without breaking compatibility.
after rename tests will be moved back from comments.

Change-Id: I439783c62678c32805a85aa52bef1d2b767543a1
2022-01-10 14:44:43 +00:00
Qweder93
dc6128e9e2 satellite/admin: add update project segment limit on ui
Added possibility to update project segment limit on
satellite admin panel ui.

Resolves https://github.com/storj/team-metainfo/issues/5

Change-Id: I192d0c9adbab411556aa46673167b3ff7437d04b
2022-01-10 13:26:56 +00:00
Qweder93
8be655c487 satellite/metainfo: add segment limit validation, caching
We want to be able to limit the number of segments per project for users.
To limit this we need to check limit value associated with project
and value of used segments already in BeginMoveObject, BeginMoveSegment
and increment cache segments usage after each CommitSegment call.

Resolves https://github.com/storj/team-metainfo/issues/1

Change-Id: I6290e67c095a174b9d101c4521802d9bfe0453b8
2022-01-10 14:33:26 +02:00
Stefan Benten
f0ada87a9f
web/satellite: fix FileModal Share Link (#4341)
Co-authored-by: littleskunk <jens.heimbuerge@googlemail.com>
2022-01-07 22:21:05 +01:00
Stefan Benten
35113634f6 Makefile: do not remove files to keep state clean
Currently the admin UI Makefile target deletes the .gitignore file.
While it being unnecessary to my findings it also causes and dirty
git repository, since files have been modified.
Removing this command lets the build process continue to work as expected
and since we are not commiting any files or assets later on this should
be fine.

Change-Id: Ibc1b7a8e456394ca19ea39dd8389c2ec03f066fc
2022-01-07 20:36:50 +01:00
Yingrong Zhao
1f8f7ebf06 satellite/{audit, reputation}: fix potential nodes reputation status
inconsistency

The original design had a flaw which can potentially cause discrepancy
for nodes reputation status between reputations table and nodes table.
In the event of a failure(network issue, db failure, satellite failure, etc.)
happens between update to reputations table and update to nodes table, data
can be out of sync.
This PR tries to fix above issue by passing through node's reputation from
the beginning of an audit/repair(this data is from nodes table) to the next
update in reputation service. If the updated reputation status from the service
is different from the existing node status, the service will try to update nodes
table. In the case of a failure, the service will be able to try update nodes
table again since it can see the discrepancy of the data. This will allow
both tables to be in-sync eventually.

Change-Id: Ic22130b4503a594b7177237b18f7e68305c2f122
2022-01-06 21:05:59 +00:00
littleskunk
b21cbc85f1
satellite/orders: log level warn for remaining "bucketName or projectID not set" (#4326)
Co-authored-by: Stefan Benten <mail@stefan-benten.de>
2022-01-06 16:31:26 +01:00
nadimhq
606a73a8e8
docs/testplan: Adding a testplan for Project Dashboard (#4318)
This is a Testplan for the Project Dashboard that follows the Testplan template.
2022-01-06 07:58:23 -05:00
paul cannon
171f908d8b cmd/satellite: rename monkit stat with '_', not '-'
Change-Id: Idb7d2971fbef1145f281b728a492e40e04add155
2022-01-06 02:30:30 +00:00
dlamarmorgan
ab37b65cfc satellite/{accounting,orders,satellitedb}: group bucket bandwidth rollups by time window
Batching of the order submissions can lead to combining the allocated
traffic totals for two completely different time windows, resulting
in incorrect customer accounting. This change will group the batched
order submissions by projectID as well as time window, leading to
distinct updates of a buckets bandwidth rollup based on the hour
window in which the order was created.

Change-Id: Ifb4d67923eec8a533b9758379914f17ff7abea32
2022-01-05 20:24:48 +00:00
Egon Elbre
5d3085f38d testuite/ui/satellite: cleanup and skip tests
Each test doesn't need to be in a separate file.
Keep naming consistent such that relevant tests can be run together.
Also skip test that do not work at the moment.

No functional changes.

Change-Id: Icd314d383ec5b683dea8afaaf2790e2e4b306671
2022-01-05 17:16:42 +02:00
Egon Elbre
d06ba56bc6 web/satellite: fix invalid bucket name message
Change-Id: If69a032d94757e9830dcfad6c36932aae5098d5b
2022-01-05 17:16:35 +02:00
Egon Elbre
a42b9d1a48 all: fix uses of email.com
email.com is not a domain that should be used for examples
nor tests.

Change-Id: I654d4287d02633d5ed9740e81a79150470eeaf25
2022-01-05 16:29:19 +02:00
Egon Elbre
04d04c0306 web/satellite: remove old object browser
Change-Id: Ief16a3887444df093c644831d86a9b3e865e5867
2022-01-05 16:29:19 +02:00
nadimhq
0c3a7b4575 Update login_unverified_nonexistent_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
35712f4deb Update login_unverified_nonexistent_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
ba1bff5367 Update login_reset_password_verified_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
edf17754e5 Update login_reset_password_verified_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
14c64318a5 Update login_unverified_nonexistent_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
bc7a9593ac Update login_reset_password_unverified_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
bde3a884c4 testsuite/ui/satellite: email verification UI tests
Added UI tests surrounding email verification
2022-01-05 14:24:04 +01:00
Malcolm Bouzi
12cb733b16 satellite/satellitedb: add last_verifcation_reminder column to users table
We want to issue a reminder to users when they don't verify their email within 24hrs of registering. This change only adds a column to the users table.

Change-Id: I92e2baeabf179338ffec01574d4752c0ccdba88b
2022-01-04 19:38:04 +00:00
Malcolm Bouzi
6a059b7f9d web/satellite: remove all references to the old onboarding flow
Change-Id: I1f42fabdb407a7eb9c34f0341d332e66c3ce288d
2022-01-04 18:40:08 +00:00
Malcolm Bouzi
1dce461f55 web/satellite: makes sure files with uppercase extensions are able to be previewed in the object browser
Change-Id: I7f50cca64df8ac394e7b985368a8155b4a540889
2022-01-04 16:57:20 +00:00
Malcolm Bouzi
c884f327c8 web/satellite: add new events to segment
We'd like more data to help us figure out drop off points.

Change-Id: Ida624188b76d14e6349afb48e72a12a52553e61d
2022-01-04 14:08:23 +00:00
Moby von Briesen
5351c43757 satellite/analytics: track origin header and referer on signup
This allows us to distinguish between accounts created from the signup
page vs. from www.storj.io.
Also set a field `account_created=true` when we send so
that we can see when existing leads have created an account.

Change-Id: Ibef34825a08b6c68b8f2869625e576bb837520e5
2022-01-03 17:47:48 +00:00
Cameron Ayer
2a4ff6f469 satellite/satellitedb: select user_agent in apikeys method GetPagedByProjectID
The user_agent column was missing from the SQL query in
apikeys.GetPagedByProjectID. Add it.

Change-Id: I2201b0ecd1767194a6d2339dba25a0b58e1ee30a
2022-01-03 11:58:26 -05:00
Egon Elbre
c58a81e7c2 testsuite/ui: disable flaky UI tests
Currently UI tests are always failing.

Disable them for now, since they do not add any useful signal to the result.

Change-Id: Ifa4a5c70cb6954e355fa9f26912fb0ce03a62ddc
2022-01-03 15:54:15 +02:00
Moby von Briesen
1bd74658df satellite/console: Disable segment limit checking during project update
This is not supported on the client side yet.

Change-Id: Id2df75278efe4b3a48be21d0992ee835e804fc3a
2021-12-23 14:35:45 -05:00
nerdatwork
f6a33ec110
Fix social media sharing links (#4321) 2021-12-23 09:23:34 -05:00
Clement Sam
589c82f23e storagenode: display quic status on SNO dashboard
This change includes storagenode QUIC status on SNO dashboard.
If disabled, it displays warning for SNOs to foward their
UDP port for quic.

Change-Id: I8d28c9c0f5f1e90d80b7c18b9e1e7b78c5e45609
2021-12-23 12:22:24 +00:00
Michał Niewrzał
ff653f6f22 satellite/metainfo: stop using in tests unused method
We would like to remove it from uplink but to do that
we need to stop using it in storj tests.

Change-Id: I8f6af74d6b8b8e447ab8ed3a00517b40840dad58
2021-12-22 15:05:53 +01:00
Jeff Wendling
4426470173 cmd/uplinkng: rename parallelism flags, small fixes
this makes the flags match rclone nomenclature

fixes test-uplinkng to use the temporary config dir
instead of the machine default, and clean up some.

bumps clingy so that the command errors when an unknown
command is specified.

also fixes some printfs in share to use clingy stdout.
it still does some external actions that should be
passed through a ulext.External for mocking, but
that's ok for now.

Change-Id: Icc231e7e26393541c312396fec907b640b97718e
2021-12-22 13:13:30 +00:00