Commit Graph

6323 Commits

Author SHA1 Message Date
Qweder93
8be655c487 satellite/metainfo: add segment limit validation, caching
We want to be able to limit the number of segments per project for users.
To limit this we need to check limit value associated with project
and value of used segments already in BeginMoveObject, BeginMoveSegment
and increment cache segments usage after each CommitSegment call.

Resolves https://github.com/storj/team-metainfo/issues/1

Change-Id: I6290e67c095a174b9d101c4521802d9bfe0453b8
2022-01-10 14:33:26 +02:00
Stefan Benten
f0ada87a9f
web/satellite: fix FileModal Share Link (#4341)
Co-authored-by: littleskunk <jens.heimbuerge@googlemail.com>
2022-01-07 22:21:05 +01:00
Stefan Benten
35113634f6 Makefile: do not remove files to keep state clean
Currently the admin UI Makefile target deletes the .gitignore file.
While it being unnecessary to my findings it also causes and dirty
git repository, since files have been modified.
Removing this command lets the build process continue to work as expected
and since we are not commiting any files or assets later on this should
be fine.

Change-Id: Ibc1b7a8e456394ca19ea39dd8389c2ec03f066fc
2022-01-07 20:36:50 +01:00
Yingrong Zhao
1f8f7ebf06 satellite/{audit, reputation}: fix potential nodes reputation status
inconsistency

The original design had a flaw which can potentially cause discrepancy
for nodes reputation status between reputations table and nodes table.
In the event of a failure(network issue, db failure, satellite failure, etc.)
happens between update to reputations table and update to nodes table, data
can be out of sync.
This PR tries to fix above issue by passing through node's reputation from
the beginning of an audit/repair(this data is from nodes table) to the next
update in reputation service. If the updated reputation status from the service
is different from the existing node status, the service will try to update nodes
table. In the case of a failure, the service will be able to try update nodes
table again since it can see the discrepancy of the data. This will allow
both tables to be in-sync eventually.

Change-Id: Ic22130b4503a594b7177237b18f7e68305c2f122
2022-01-06 21:05:59 +00:00
littleskunk
b21cbc85f1
satellite/orders: log level warn for remaining "bucketName or projectID not set" (#4326)
Co-authored-by: Stefan Benten <mail@stefan-benten.de>
2022-01-06 16:31:26 +01:00
nadimhq
606a73a8e8
docs/testplan: Adding a testplan for Project Dashboard (#4318)
This is a Testplan for the Project Dashboard that follows the Testplan template.
2022-01-06 07:58:23 -05:00
paul cannon
171f908d8b cmd/satellite: rename monkit stat with '_', not '-'
Change-Id: Idb7d2971fbef1145f281b728a492e40e04add155
2022-01-06 02:30:30 +00:00
dlamarmorgan
ab37b65cfc satellite/{accounting,orders,satellitedb}: group bucket bandwidth rollups by time window
Batching of the order submissions can lead to combining the allocated
traffic totals for two completely different time windows, resulting
in incorrect customer accounting. This change will group the batched
order submissions by projectID as well as time window, leading to
distinct updates of a buckets bandwidth rollup based on the hour
window in which the order was created.

Change-Id: Ifb4d67923eec8a533b9758379914f17ff7abea32
2022-01-05 20:24:48 +00:00
Egon Elbre
5d3085f38d testuite/ui/satellite: cleanup and skip tests
Each test doesn't need to be in a separate file.
Keep naming consistent such that relevant tests can be run together.
Also skip test that do not work at the moment.

No functional changes.

Change-Id: Icd314d383ec5b683dea8afaaf2790e2e4b306671
2022-01-05 17:16:42 +02:00
Egon Elbre
d06ba56bc6 web/satellite: fix invalid bucket name message
Change-Id: If69a032d94757e9830dcfad6c36932aae5098d5b
2022-01-05 17:16:35 +02:00
Egon Elbre
a42b9d1a48 all: fix uses of email.com
email.com is not a domain that should be used for examples
nor tests.

Change-Id: I654d4287d02633d5ed9740e81a79150470eeaf25
2022-01-05 16:29:19 +02:00
Egon Elbre
04d04c0306 web/satellite: remove old object browser
Change-Id: Ief16a3887444df093c644831d86a9b3e865e5867
2022-01-05 16:29:19 +02:00
nadimhq
0c3a7b4575 Update login_unverified_nonexistent_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
35712f4deb Update login_unverified_nonexistent_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
ba1bff5367 Update login_reset_password_verified_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
edf17754e5 Update login_reset_password_verified_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
14c64318a5 Update login_unverified_nonexistent_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
bc7a9593ac Update login_reset_password_unverified_account_test.go 2022-01-05 14:24:04 +01:00
nadimhq
bde3a884c4 testsuite/ui/satellite: email verification UI tests
Added UI tests surrounding email verification
2022-01-05 14:24:04 +01:00
Malcolm Bouzi
12cb733b16 satellite/satellitedb: add last_verifcation_reminder column to users table
We want to issue a reminder to users when they don't verify their email within 24hrs of registering. This change only adds a column to the users table.

Change-Id: I92e2baeabf179338ffec01574d4752c0ccdba88b
2022-01-04 19:38:04 +00:00
Malcolm Bouzi
6a059b7f9d web/satellite: remove all references to the old onboarding flow
Change-Id: I1f42fabdb407a7eb9c34f0341d332e66c3ce288d
2022-01-04 18:40:08 +00:00
Malcolm Bouzi
1dce461f55 web/satellite: makes sure files with uppercase extensions are able to be previewed in the object browser
Change-Id: I7f50cca64df8ac394e7b985368a8155b4a540889
2022-01-04 16:57:20 +00:00
Malcolm Bouzi
c884f327c8 web/satellite: add new events to segment
We'd like more data to help us figure out drop off points.

Change-Id: Ida624188b76d14e6349afb48e72a12a52553e61d
2022-01-04 14:08:23 +00:00
Moby von Briesen
5351c43757 satellite/analytics: track origin header and referer on signup
This allows us to distinguish between accounts created from the signup
page vs. from www.storj.io.
Also set a field `account_created=true` when we send so
that we can see when existing leads have created an account.

Change-Id: Ibef34825a08b6c68b8f2869625e576bb837520e5
2022-01-03 17:47:48 +00:00
Cameron Ayer
2a4ff6f469 satellite/satellitedb: select user_agent in apikeys method GetPagedByProjectID
The user_agent column was missing from the SQL query in
apikeys.GetPagedByProjectID. Add it.

Change-Id: I2201b0ecd1767194a6d2339dba25a0b58e1ee30a
2022-01-03 11:58:26 -05:00
Egon Elbre
c58a81e7c2 testsuite/ui: disable flaky UI tests
Currently UI tests are always failing.

Disable them for now, since they do not add any useful signal to the result.

Change-Id: Ifa4a5c70cb6954e355fa9f26912fb0ce03a62ddc
2022-01-03 15:54:15 +02:00
Moby von Briesen
1bd74658df satellite/console: Disable segment limit checking during project update
This is not supported on the client side yet.

Change-Id: Id2df75278efe4b3a48be21d0992ee835e804fc3a
2021-12-23 14:35:45 -05:00
nerdatwork
f6a33ec110
Fix social media sharing links (#4321) 2021-12-23 09:23:34 -05:00
Clement Sam
589c82f23e storagenode: display quic status on SNO dashboard
This change includes storagenode QUIC status on SNO dashboard.
If disabled, it displays warning for SNOs to foward their
UDP port for quic.

Change-Id: I8d28c9c0f5f1e90d80b7c18b9e1e7b78c5e45609
2021-12-23 12:22:24 +00:00
Michał Niewrzał
ff653f6f22 satellite/metainfo: stop using in tests unused method
We would like to remove it from uplink but to do that
we need to stop using it in storj tests.

Change-Id: I8f6af74d6b8b8e447ab8ed3a00517b40840dad58
2021-12-22 15:05:53 +01:00
Jeff Wendling
4426470173 cmd/uplinkng: rename parallelism flags, small fixes
this makes the flags match rclone nomenclature

fixes test-uplinkng to use the temporary config dir
instead of the machine default, and clean up some.

bumps clingy so that the command errors when an unknown
command is specified.

also fixes some printfs in share to use clingy stdout.
it still does some external actions that should be
passed through a ulext.External for mocking, but
that's ok for now.

Change-Id: Icc231e7e26393541c312396fec907b640b97718e
2021-12-22 13:13:30 +00:00
Michal Niewrzal
da9e0188f8
Update README.md (#4320)
* remove alpha badge
* remove information about "beta phase" and "network under construction" from License section

Co-authored-by: Stefan Benten <mail@stefan-benten.de>
2021-12-22 14:12:58 +01:00
Stefan Benten
0334465306
Makefile: fix variable used to trigger release drafting (#4319)
Small fix to properly trigger the script.
2021-12-22 13:33:07 +01:00
Egon Elbre
5f39f20766 web/: add check for change to eslint import
Change-Id: Idf94c8e0c92d34e840db549e9e704eb3373762ac
2021-12-21 15:59:23 +00:00
Stefan Benten
e2481fb510 scripts,Jenkinsfile,Makefile: upload binaries to drafted github release
This change adds a script and the needed build logic to create a draft github release and then upload the created binaries to it in order to make the actual publishing a lot less error prone.
The logic is currently that it only does this for all github tags, but not for every main build/push. This is handled by the checks in the script itself.

Change-Id: Ie172a8e4a97200de901a26a055aa5a8a54b60a2a
2021-12-21 13:55:38 +00:00
Michał Niewrzał
db7985ca90 satellite/metainfo: set EncryptedMetadataEncryptedKey while listing objects
At some point we missed to add metadata key to list objects
response. Because of that uplink is taking key from pb.SteamMeta.
We need to clean it up.

Tests will be added on uplink side.

https://github.com/storj/uplink/issues/71

Change-Id: I3328e2f1b86bca15aeaf89f8e59cdca3c8e97742
2021-12-21 13:09:11 +00:00
Michał Niewrzał
87a9292127 satellite/metainfo/zombiedeletion: enable chore by default
We enabled this chore on different satellites
for testing. Works fine so we can enable this
by default for all configurations.

Change-Id: I987639685d8de5c7e5798adca30fe26bdac9e1d1
2021-12-21 13:26:18 +01:00
Stefan Benten
a6139f5a6b satellite/orders: log less decrypt order messages
Currently most of the satellite log is made up by this specific log
message and thus we should increase the log level. We already have
a monkit event tracking this case. In case we need to look into this
more we should just increase the satellite log level.

Change-Id: I27ebed3e6745701c66c83e8c52ddc836ad9d5f4e
2021-12-21 00:39:29 +01:00
nadimhq
a04b9ce372
testsuite/ui/multinode: adding more UI tests (#4301)
added tests for add node (check modal, cancel with separate buttons, add new node, add node that already exists) and more options (change display name, delete node(&cancel delete), copy node ID).
2021-12-20 14:30:48 -05:00
Ivan Fraixedes
c10cc302b5
Makefile: Fix target for building Admin UI
Fix the Makefile target for building the Satellite Admin UI.

Change-Id: Id9087c313c451fa68796c9a495704ea3b3728a5c
2021-12-20 12:11:10 +01:00
Ethan
185617326c satellite/email: Move MIME-Version header to the last element before Content-Type
Some MTAs are marking our emails as SPAM because of the current location of the header.

Change-Id: I3f12c36b184f69b643a8299f5ccbc609f4599ef9
2021-12-17 11:26:25 -06:00
Qweder93
b7b010adc9 satellite/admin: add project limit on segment count
Added possibility to set custom value of segments
per project for user/customer.

Change-Id: Ic6e1abe25269755e9e8819fbef065a87de359fce
2021-12-17 12:40:12 +00:00
Ivan Fraixedes
f87ce0cadf satellite/admin/ui: Fix component rendering
I introduced a change in the UI that was a better way to dynamically
render Svelte components
(https://review.dev.storj.io/c/storj/storj/+/5931/4..5) during the
review of the first version of it, however, while it works perfectly
on development mode it doesn't work when the assets are built for
production, failing silently, because the constructor name get renamed
due to the name mangling caused by the minifyer as stated in the
following issue: https://github.com/sveltejs/svelte/issues/6980

This commit use a different alternative not based on the constructor
name and it works fine with the production build.

Change-Id: I643c405f877a9206cf0e51b44d5138e5a9756a79
2021-12-17 11:00:31 +00:00
Ivan Fraixedes
5573ece848 satellite/admin/ui: Migrate to SvelteKit
Migrate the satellite admin UI web app from the Svelte template used to
generate a Svelte App scaffolding to SvelteKit.

There aren't any functional changes in the  application, however, the
commit has a lot because:

1. SvelteKit uses a different directory layout and constraints to it, so
   the files have been moved.
2. The files have changed its formatting due to the new default linter
   configurations that SvelteKit uses.
3. The linter detected some issues with using `object` and `any` types
   in Typescript, so they have been replaced by better general types
   (e.g. Record).

The  migration allows to use the new tooling rather than Rollup
directly, besides that will empower the future of it when it needs more
features (e.g. different routes, etc.).

Change-Id: Ifa6736c13585708337f6c5a59388077b784eaddd
2021-12-17 10:27:13 +00:00
Malcolm Bouzi
db0bd38d95 web/satellite: Add new user events to segment for link clicks
We'd like to have a better understading of what pages users are clicking on the most

Change-Id: Ib3983cd250a95106acb162c2284fc1d11f2a3152
2021-12-16 18:11:11 +00:00
Michał Niewrzał
60d03b9e42 satellite/metainfo: drop GetSegmentByLocation
Method is not used anymore. We need it during transition
from metainfo loop into segment loop.

Change-Id: Id32590ff9b02e06025efce1c32869e2cf9cd0bc5
2021-12-16 16:32:45 +01:00
Michał Niewrzał
3f0babffe8 satellite/{console,satellitedb}: add project segment limit to user
All limits we have for projects have also parent limits stored
with user data. New created project is first taking limits from
owner (user) limits.
This change is extending users table with project_segment_limit
column and adds functionality to get and set value for this
column.

Change-Id: Iff5e36c62b517652390b649fc05992475916ecff
2021-12-16 08:46:01 +00:00
Jeremy Wharton
9d13c649a2 satellite/{console,satellitedb}: Forbid creating users with used email
This change disallows creation of users possessing the same email.
If a user attempts to create an account with an email address
that's already used - whether it belongs to an active account or not -
he will be notified of unsuccessful account creation. If he attempts to
log in using an email address belonging to an inactive account,
he will be presented with a link allowing him to re-send the
verification email. Attempting to register with an email address
belonging to an existing account triggers a password reset email.

Change-Id: Iefd8c3bef00ecb1dd9e8504594607aa0dca7d82e
2021-12-15 17:24:35 +00:00
Clement Sam
137641f090 multinode/nodes: pass Node entity to Add method instead of set of a parameters
The Add method on the multinode DB interface accepts
a set of parameters which are already fields in the nodes.Node struct
excluding the name field.

When adding a new node, you're forced to call UpdateName() method
after calling the Add() method in order to save a node and update
the name.

This change allows passing the nodes.Node entity which includes
the name field. With this, a new node can be added together with
the name without invoking the UpdateName() method.

Change-Id: I281ec628dffaade35d6db4479a84f39636200072
2021-12-15 12:39:24 +00:00
nadimhq
596124f0f6
docs: Adding a testplan folder with a README and TEMPLATE for testplan (#4306)
Added a testplan folder in docs, with a README and TEMPLATE for testplan

We want community involvement with testplans, since we believe that testplans written as early as possible before implementation starts work like a checklist and as soon as we finish the implementation we can compare it to the test plan to check for any bugs. Also even before implementation is finished, if a developer can read a test plan beforehand they can prevent most bugs, hence why we believe that the earlier we write a test plan the more bugs we can prevent.

Co-authored-by: Igor <38665104+ihaid@users.noreply.github.com>
2021-12-15 07:32:36 -05:00