Commit Graph

4813 Commits

Author SHA1 Message Date
Jeff Wendling
2d2359667d satellite/orders: remove unused satelliteAddress field
Change-Id: I58091769472688433c48becc8dfc9029bddd87aa
2021-01-08 12:25:39 -05:00
Egon Elbre
ba5461562d satellite/orders: remove satellite address
SatelliteAddress in OrderLimit is not being used anymore and some
satellite addresses may consume too much bytes.

Change-Id: Ic7a0efe5b6211c2f3b91af67b293cde98b29d074
2021-01-08 16:57:36 +00:00
Egon Elbre
51731db121 satellite/orders: use smaller encrypted metadata
Avoid using project uuid string representation, because
it uses more bandwidth.

This reduces the encrypted metadata size from 118 -> 97 bytes.

Change-Id: Ic53a81b83acc065f24f28cd404f9c0b1fe592594
2021-01-08 16:40:31 +00:00
Yaroslav Vorobiov
6507f3ebc6 multinode/console: trusted satellites list api
Change-Id: I97bb9efb1d6cb7d456df0b86e66417c31018b762
2021-01-08 14:50:12 +02:00
Isaac Hess
db6b64c2ba Makefile: Custom gateway location for storj-sim
When running make install-sim we currently create a .build directory and
init a go module there. This change allows a user to pass in a custom
location for this to occur. If nothing is passed in it defaults to
current behavior.

Change-Id: I432dfc7bae412d8a8454a6b8b3dffece84a41147
2021-01-07 13:14:48 -07:00
JT Olio
35c61bd0e2 satellite: mark gateway credentials feature beta
Change-Id: I378d703bd17b842d56f7ca2c8bf24b47c4a07264
2021-01-06 15:18:43 -07:00
JT Olio
8907180e81 satellite: pass contact.external-address config to web ui
Change-Id: I54978aa34aa9eb98876fab6460a5737d718d6135
2021-01-06 10:11:20 -07:00
Bill Thorp
c0bc2123f5 uplink/cmd: fix uplink access register ability to use saved accesses
Jens noticed that 'uplink access register' wasn't working with named
accesses.

This was because GetNamedAccess was harcoded to use inspectCfg, which
in the case of 'uplink access register' wasn't being bound to the
config file.

Change-Id: I49403b45af28ad33408cfc5ec6545a395f0f080d
2021-01-06 15:08:51 +00:00
Egon Elbre
590bc529d2 cmd/storj-sim: fix local binary lookup
Change-Id: I6404fd05119c2a7113e4b0d7593a555327bef35f
2021-01-06 16:26:22 +02:00
Egon Elbre
b33d7a318e cmd/storj-sim: add -failfast
Fail all the processes immediately when one of the processes fails. This
is to make it more obvious that one of them has failed.

To disable failfast, use `-failfast=false`.

Change-Id: I2bbedf12fb653e42739d00273aa9ae515d34eda6
2021-01-06 16:26:22 +02:00
Kaloyan Raev
1e44be2900 go.mod: bump deps to uplink v1.4.4
Change-Id: I203259f0f324284ac2991e2cabf4bd471cd8ad5d
2021-01-06 13:07:17 +00:00
VitaliiShpital
2ad560e2dc web/satellite: satellite address field was added to continue in CLI step
WHAT:
satellite node url is now shown on continue in CLI step

WHY:
to make it usable

Change-Id: Iaf5fcbe3b9cfbebff904a16781ecd0225f3d4ba5
2021-01-05 21:32:17 +02:00
VitaliiShpital
6f065a8331 web/satellite: fix for all bucket names selecting
WHAT:
if all buckets are selected then we pass an empty array to web worker instead of all existing bucket names

WHY:
bug fix

Change-Id: I59b9e9ec1deb0abb3edeba7e2303e2b1d578935d
2021-01-05 16:13:00 +00:00
NickolaiYurchenko
f925d99d39 web/multinode: MyNodes page markup
Change-Id: Ie1bf57cde2dee6b72238ce7188a12dfb741bc031
2021-01-05 15:54:03 +00:00
NickolaiYurchenko
857155f5ac web/multinode: navigation panel added
Change-Id: Ia8d56cfce14476e1f4b59f4d13f239dda4a87b4f
2021-01-05 15:33:27 +00:00
NickolaiYurchenko
69a491aa6f web/multinode: add new node modal added
Change-Id: I337b39be065665cd531310171757ad8aab6be799
2021-01-05 15:10:54 +00:00
Yaroslav Vorobiov
5a43c86b68 multinode/console: list node satellite infos
Change-Id: Ic6cb8d1a6fd7637fdb7bf49e040c43ac30ab1bbf
2021-01-05 14:49:58 +00:00
Yaroslav Vorobiov
fb00d099cf multinode/console: list node infos
Change-Id: I5cac49feff2bac6fbd7ac61dfccffd672da8e8c0
2021-01-05 14:49:11 +00:00
NickolaiYurchenko
16a8b1da35 web/multinode: onboarding pages and common components
Change-Id: I71910108ff1cd40b7718dba1944589a2cc4852ef
2021-01-05 16:23:40 +02:00
NickolaiYurchenko
7703b4b691 web/multinode: common dropdown component added
Change-Id: I9e8b76abb73c561939daf9758ace0f82eed4f443
2021-01-05 13:17:31 +00:00
Egon Elbre
9cb4466eb0 cmd/storj-sim: use dev setup by default for consistency
Fixes bug when using release binaries together with storj-sim.

Change-Id: I077bedc1486ac85aa1f04fcc0ed4098cd313f2fc
2021-01-05 13:47:30 +02:00
Moby von Briesen
710b86849c storagenode/{console, reputation}: Add audit history to storagenode API
Change-Id: Id680b07feea380824e56496d545f46a94fdcc5d5
2021-01-04 17:37:45 +00:00
Moby von Briesen
a90d6fcad8 satellite/repair/checker: Use segment health on checker insert
Do not insert the number of healthy pieces for segment health anymore.
Rather, insert the segment health calculated by our new priority
function.

Change-Id: Ieee7fb2deee89f4d79ae85bac7f577befa2a0c7f
2021-01-04 11:48:17 -05:00
Moby von Briesen
6e2ef3b9ee Revert "satellite/satellitedb: Do not consider nodes with offline_suspended as reputable."
This reverts commit e24262c2c9.

Change-Id: I287deb2e52d03bbd698ed055f0f216b0b5bf2798
2021-01-04 14:28:37 +00:00
Michał Niewrzał
d4ebdba48c satellite/payments/stripecoinpayments: fix tests failing in 2021
We had some tests with hardcoded year 2020.

Change-Id: I0184c3ece819cb764eb305751a1d8d4056b6af17
2021-01-04 10:47:31 +01:00
Moby von Briesen
39260b6fdc storagenode/storagenodedb: Add audit_history to storagenode reputationdb
This change will give us the ability to make audit windows visible via
the storagenode api.

Change-Id: Ifb3d044ce9b456c16f8ea3b3b162ba894426477a
2020-12-31 11:00:42 -05:00
Moby von Briesen
edbee53888 satellite,storagenode: Pass audit history over GetStats endpoint
Full prefix: satellite/{overlay,nodestats},storagenode/{reputation,nodestats}

Allow the storagenode to receive its audit history data from the
satellite via the satellite's GetStats endpoint.

The storagenode does not save this data for use in the API yet.

Change-Id: I9488f4d7a4ccb4ccf8336b8e4aeb3e5beee54979
2020-12-30 19:13:26 +00:00
Rafael Gomes
8b2e4bfa7e satellite/metainfo/piecedeletion Remove spaces from metrics.
Change-Id: Iaf1d8a96a43087f2fcc579347f581e8a78a0fb58
2020-12-30 14:27:39 -03:00
paul cannon
7246368ca1 satellite/repair: clamp totalNodes to 100 or higher
Change-Id: I239418ed3671b1cee30b0b1797dc434244e72448
2020-12-30 10:39:14 -06:00
Moby von Briesen
641f01ed42 cmd/uplink: Actually overwrite accesses in uplink import
Previously, we were trying to overwrite accesses, which is a nested map
in the uplink config, by calling viper.MergeWithConfig with a nested
map. While this works for keys that don't exist already, it does not
overwrite already existing keys. In order to do that, we need to call
MergeWithConfig with "accesses.<accessname> -> value" rather than using
"accesses -> <accessname> -> value".

Change-Id: I74d7a9decf2078cdf2ff440eaf24821e30474b53
2020-12-30 15:25:53 +00:00
Bill Thorp
f65a1e8c1c uplink: use output formats for 'uplink access register' instead of writing to AWS file
The current uplink access register method has the ability to write to AWS credential files.
This is caused issues with repeat usage in recent AWS CLI code, and there was concern that
it was an unstable solution.  This version instead specifies various output formats "env"
and "aws". "env" formats the text so that it can be used with 'export'.  "Aws" generates
"aws configure" commands to persist the credentials to the AWS credential files as the
previous version could.

Example usages:

Setting ephemeral evironment variables in bash:
export $(uplink access register $(storj-sim network env GATEWAY_0_ACCESS)
--auth-service http://localhost:8000 --format env)

Setting persistant configs via AWS CLI in bash:
source <(uplink access register $(storj-sim network env GATEWAY_0_ACCESS)
--auth-service http://localhost:8000 --format aws --aws-profile storjsim)

Change-Id: I5d78d6462a3537780af3717a298bb2bebf9c2799
2020-12-30 13:34:01 +00:00
Moby von Briesen
825dc71227 satellite/{overlay, satellitedb}: Refactor audit history
* Separate audit history interface into its own file in the overlay
package
* Add overlay.AuditHistory struct so that internalpb.AuditHistory is
only used from within the database layer
* Add overlay.GetAuditHistory function for features that will require
access to detailed audit history information
* Do not return full audit history from UpdateAuditHistory - callers to
that function only need to know the online score and whether a full
tracking period has been completed
* Move audit history tests out of satellite/satellitedb, since they are
independent of database implementation

Change-Id: I35b0c4ac23bbaabd80624f8a9631c3cb1a1f33bd
2020-12-29 18:50:22 +00:00
Moby von Briesen
85ae13f11d satellite/satellitedb: Drop nodes_offline_times table.
Now that the deprecated downtime tracking service is removed
(3fc76f4ffe), we can safely remove
the nodes_offline_times table.

Change-Id: Ia7c6efe32ba104dff5a830af5f2beee3337eefe5
2020-12-29 18:17:50 +00:00
Moby von Briesen
e24262c2c9 satellite/satellitedb: Do not consider nodes with offline_suspended as reputable.
Nodes which are offline_suspended will no longer be considered for new
uploads. The current threshold that enters a node into offline
suspension is 0.6. Disqualification for offline suspension is still
disabled.

Change-Id: I0da9abf47167dd5bf6bb21e0bc2186e003e38d1a
2020-12-29 17:59:09 +00:00
Cameron Ayer
ca12e98d5d docs: change 'offline score' to 'online score' in downtime tracking doc
Change doc to reflect a change made during the implementation.

Change-Id: I31f6a2c08ae97cba7068975ab6c9dd276726273a
2020-12-29 15:31:20 +00:00
Stefan Benten
02d7638eda
Jenkinsfile.public: specify main branch for docker build
Docker build defaults to the master branch, when specifying a git repo.
With this change, it should be smart enough to use the proper main branch.
2020-12-29 16:56:36 +02:00
Stefan Benten
7f1871b8f1 all: switch from master to main 2020-12-28 22:59:06 +01:00
Ivan Fraixedes
2223877439 storgenode/piecestore: Log upload size on Upload
Add the upload size to the log lines of the storagenode Upload endpoint
to provides the information to Storage node operators.

Change-Id: Ife661d28be72c2bf02579093e21fa811566ac8dd
2020-12-28 10:28:09 +00:00
NickolaiYurchenko
73fb54fed2 web/multinode/app/store: fixed nodes mutation
Change-Id: Id999b309d73770c44e97caa07d901e821b09d05e
2020-12-24 17:43:01 +00:00
crawter
b0258f085f web/multinode/app: initial state design
Change-Id: If182d1c173e99be1df18daf5cc7083a0d188b3cb
2020-12-24 17:58:43 +02:00
NickolaiYurchenko
6e68e8e00e web/storagenode: held progress typo fix
Change-Id: Ifa77928d27815942990a5ce1b32a572f309d1799
2020-12-24 12:33:56 +02:00
Stefan Benten
ad58459198 satellite/admin: allow more than just "paid" invoice status during user deletion
Currently we do not allow anything other than the "paid" status for invoices when
trying to delete a user. However there can be a couple of other states that are
still fine to accept during deletion of a user. This change reverses the order to
check for the status that we do not want to allow.

Change-Id: I78d85af6438015c55100fa201ccffc731c91de1c
2020-12-23 16:40:44 +01:00
VitaliiShpital
7aea6feee3 web/satellite: access grants delete popup blinking bug fixed
WHAT:
when user deletes access grants, an empty list of being deleted grants appears for a second.
It is fixed now

WHY:
better user experience

Change-Id: Ibda50ad809dfc49f95c4c844487d9527c676fd76
2020-12-23 13:22:05 +00:00
VitaliiShpital
95053531fa web/satellite: access grant: result step back click bug fixed, small styling fixes
WHAT:
back click from result step now sends correct key to passphrase step.
Small styling fixes were also made

WHY:
bug fix

Change-Id: I62463d914496a654a45f01dfb45dd704edbaecb9
2020-12-23 13:21:58 +00:00
JT Olio
f787aa4647 go.mod: bump uplink to v1.4.3
Change-Id: Ia6ff4f3a84897aaebe7aef7680f17b94b59cb9de
2020-12-23 11:18:26 +00:00
JT Olio
7faaeed2bf satellite/access grant wizard: don't hardcode the satellites
Change-Id: Id9fbf68882cdb2fce846b7a2604cf965cc53ab1a
2020-12-22 21:24:45 -07:00
JT Olio
efde103dba accounting: rollup test is broken for the hour before midnight UTC
this change isn't the real fix. it's just ignoring the problem.

i don't know what the real fix is. is the problem with the test, or
is there actually a problem with the rollup code?

Change-Id: I552bdd947deadc212cc56efc5f818942b9827126
2020-12-22 14:14:52 -07:00
Ethan Adams
6070018021
satellite/overlay: use AS OF SYSTEM TIME with Cockroach
Query nodes table using AS OF SYSTEM TIME '-10s' (by default) when on CRDB to alleviate contention on the nodes table and minimize CRDB retries. Queries for standard uploads are already cached, and node lookups for graceful exit uploads has retry logic so it isn't necessary for the nodes returned to be current.
2020-12-22 21:07:07 +02:00
Jennifer Johnson
7cccbdb766 cmd/uplink/share: add -- to flags referenced in help statements
Change-Id: I86cd08d51c5306effec14f338f37c53c2743d6b2
2020-12-22 17:03:00 +00:00
Bill Thorp
1b0424cad6 uplink/cmd: Export a RegisterAccess method.
Gateway-MT requires integration tests, which would be aided by having an
exported RegisterAccess() method in uplink/cmd.

To support this change, a little of the Uplink cmd logic was shifted around
and a method was made public.  I also normalized finding the access
between accessInspect and accessRegister.

Change-Id: I29369296521c2cc179e27233f5451b95f46109d8
2020-12-22 14:52:40 +00:00