Commit Graph

64 Commits

Author SHA1 Message Date
Clement Sam
d800b51dd9 cmd/multinode: generate identity when not provided
Closes https://github.com/storj/storj/issues/4974

Change-Id: I7a07d846be7ac8f8f7d7e9ad61511ff84d2ab400
2022-10-12 12:01:14 +00:00
Egon Elbre
ff22fc7ddd all: fix deprecated ioutil commands
Change-Id: I59db35116ec7215a1b8e2ae7dbd319fa099adfac
2022-10-11 15:27:29 +00:00
Clement Sam
7e5025cac0 {storagenode,multinode/nodes}: use multinodeauth.Secret instead of []byte for APISecret
When enconding structs into JSON, byte slices are marshalled as base64
encoded string using the base64.StdEncoding.Encode():
ea9c3fd42d/src/encoding/json/encode.go (L833-L861)

We, however, expect API Secrets to be encoded as base64URL, so when
an marshalled secret (with byte slice type) is added to the multinode
dashboard, it fails with `illegal base64 data at input byte XX`.

This change changes the type of APISecret field in the
multinode/nodes.Nodes struct to use multinodeauth.Secret type instead
of []byte.
multinodeauth.Secret is extended with custom MarshalJSON and
UnmarshalJSON methods which implement the json.Marshaler and
json.Unmarshaler interfaces, respectively.

Resolves https://github.com/storj/storj/issues/4949

Change-Id: Ib14b5f49ceaac109620c25d7ff83be865c698343
2022-08-23 11:04:04 +00:00
Márton Elek
0cc054e931 build: turn off postgres unit tests (build time test)
Change-Id: Iecb018791d9533c354375f9b7865c83ba32092f5
2022-08-22 13:18:59 +00:00
Egon Elbre
5f7ea1358d private/web: make caching headers reusable
Move storagnode/console caching headers to private/web. Also,
start using them in multinode/console/server.

Change-Id: I1f0f3c9833a183476009737cece515ae7537fb83
2022-03-11 11:19:11 +02:00
Egon Elbre
28c9403702 multinode,web/multinode: use go:embed for assets
Go can now directly embed files without relying on external tools.
This makes code use go:embed and avoid the external tooling.

go:embed requires files to be present in the embedded directory,
hence we need to add .keep to "dist" folder. We also add one to
public/.keep, such that it won't be deleted when building multinode.

Change-Id: I53ac3d5ac76e44f740d95221acf0da99fc256d42
2022-03-11 11:10:37 +02:00
Clement Sam
137641f090 multinode/nodes: pass Node entity to Add method instead of set of a parameters
The Add method on the multinode DB interface accepts
a set of parameters which are already fields in the nodes.Node struct
excluding the name field.

When adding a new node, you're forced to call UpdateName() method
after calling the Add() method in order to save a node and update
the name.

This change allows passing the nodes.Node entity which includes
the name field. With this, a new node can be added together with
the name without invoking the UpdateName() method.

Change-Id: I281ec628dffaade35d6db4479a84f39636200072
2021-12-15 12:39:24 +00:00
Clement Sam
b8dd35ceaf private/testplanet: extend testplanet with multinode instance
Testplanet does not have a multinode instance, hence,
makes it difficult to run ui tests for the multinode dashboard.

Instead of using storj-sim for multinode ui tests,
it's better to use the same approach (testplanet) fir all UI tests.

This changes adds a multinode instance to testplanet.

Change-Id: I58aa8c4597e789275f9e7ea7059703c742903492
2021-10-13 10:19:45 +00:00
Qweder93
73cdefbc41 munltinode/console: node online statuses extended with StatusUnauthorized and StatusStorageNodeInternal
Change-Id: I12c90169f8959e5eafe2a64bf8b412f7afc48c1c
2021-07-21 15:00:41 +03:00
Yaroslav Vorobiov
6db6b76b27 multinode/console: embed web assets
Embed web static files in multinode binary to be able to
release multinode as single binary.
Add make commands to build multinode binary with embeded web
assets.

Change-Id: I348aff7a7d847fae5c021cbf59abc7f892c0df80
2021-07-12 18:51:24 +03:00
crawter
f79d697717 multinode/nodes: checking node availability before adding it
Change-Id: I7792ed359713f12fee9fb4ac0ea46703dfc8406e
2021-07-07 15:45:09 +00:00
Yaroslav Vorobiov
cbb4cd3fc3 multinode/reputation: add vetted at timestamp
Change-Id: Id35cb6cfdabf4bf2762e4a162cf3157afb0ff170
2021-07-07 18:11:54 +03:00
Qweder93
8f4505f532 multinode: handling offline nodes added
nodes/service and payouts/service returns node with status offline
or 0 in case if node is offline.

Change-Id: I74c03dcba9ddc9c05885ab329f80f3b14fe8c0fa
2021-07-07 13:25:32 +00:00
Yaroslav Vorobiov
818f6c6ea6 multinode/console: add summary to storage usage API
Change-Id: Ia8a1e598d667f25461f73f1626da22113cb7caeb
2021-07-07 15:00:05 +03:00
crawter
3b0b0ba3c4 multinode/nodes: node status extended with NotReachable
Change-Id: Ibd104945a79e07497f4963442c12a555ea02fbaf
2021-07-06 17:04:19 +03:00
Yaroslav Vorobiov
68627e7d80 multinode/console: add reputation satellite api
Change-Id: I7cef6c1c271607f7485f604d5b61587558a31878
2021-07-05 15:32:22 +00:00
Qweder93
cff3da7647 multinode/operators: using old api instead of new until nodes updated.
Change-Id: I210e300f11d03d705ff98e95ab0d9debad59c9d6
2021-06-24 18:37:04 +03:00
Qweder93
b9d7874dc2 multinode: old drpc api returned
drpc api with old names added to prevent breaking
backwards compatibility.

Change-Id: I1b8a41f2c1e0bd11ac83c86f0b1bfbfc1f07378f
2021-06-22 23:02:47 +00:00
crawter
d426d4f00d multinode/storage: disk space of concrete node and total disk space implemented
Change-Id: I50a96d3e9a77615e79b826a592c0a07c247e0520
2021-06-15 18:46:54 +03:00
Qweder93
5d70b6abef multinode/bandwidth: added monthly bandwidth summaries
montly bandwidth summaries for single/all nodes, single/all satellites added.

Change-Id: Ic384886c10622df74a4bd0645e2d7f2a85477644
2021-06-11 16:31:59 +00:00
Yaroslav Vorobiov
157d3d980e multinode/console: storage usage and total storage usage
Change-Id: I4970275daf4a4a9c5d02aea6a205891869dd4eff
2021-06-10 16:01:41 +00:00
Yaroslav Vorobiov
b23a782c6b multinode/console: held amount summary use satellite address
Change-Id: I7816a263579831eaaeb808e3cdd49b3b7b0a07b8
2021-06-10 15:55:43 +03:00
crawter
b445e7d397 multinode/operators: operator entity extended with nodeID and undistributed amount
Change-Id: I423fe79b0aefe09d0b0ddab490d7a39e2b3f3249
2021-06-08 20:01:09 +03:00
Qweder93
d8c11a79b9 multinode/payouts: paystub extended with disposed
added Disposed to paystub struct to count NetTotal value.

Change-Id: Iaae1f98a69b82166ba5594e8589c7d2a540bd07f
2021-06-01 16:28:20 +00:00
crawter
4af0037a67 multinode/console: operators controller added
Change-Id: I170371baec3c6996bd2af3c332620bd6fee3ed63
2021-06-01 19:01:14 +03:00
crawter
0b3ee4bda7 multinode/operators: domain entities and service created
Change-Id: I146229df0a279b95ce4d3ffec658509ccd6ff59a
2021-06-01 16:51:48 +03:00
crawter
4c704ea9a4 mnd/db/nodes: paginated list added
Change-Id: Ibb049c0ad9b09dd89852234b601d7577735c4d34
2021-06-01 13:19:48 +03:00
Qweder93
91b7e24d55 multinode/payouts: naming refactoring
Renamed methods, error messages.

Change-Id: I7d7b6b092c05bbc5bf1322855efc5ccb9b312671
2021-05-31 19:47:12 +00:00
Yaroslav Vorobiov
64e4225721 multinode/console: server reorder payouts API routes
Change-Id: Ia42bb3faae8a8782d841e5cba186dc73478581ac
2021-05-31 16:32:03 +03:00
Yaroslav Vorobiov
23f9beb635 multinode/console: held amount summary
Change-Id: Ia800748343e363d930ce0a0b9ab286b5abdc96af
2021-05-28 20:05:16 +03:00
Qweder93
099a31c69f multinode/payouts: paystubs for single period added
paystubs for single/all satellites for specific period added.

Change-Id: Ibb822aa11380bc70f4db8c81fa0b9b6f7a739078
2021-05-25 18:59:05 +03:00
Qweder93
1bc4f8fccd multinode/payouts: paystubs for all time added
paystubs for single/all satellites for all time added.

Change-Id: Ia665c69994f769cb0071363a617eaf87ef1a05f2
2021-05-25 18:19:23 +03:00
Qweder93
79172777bd mnd/payouts: estimations replaced with expectations
Added expectations endpoint (estimations and distributed), added
coalesce to db query, so in case of empty payouts db 0 will be returned instead of error.

Change-Id: I535f14ef097876448d8949bc302895b25da2b6e7
2021-05-24 18:11:30 +00:00
Qweder93
83e82eb473 mnd/payouts: estimated payouts per satellite changed to per node
Change-Id: I231e95ece1f52accb17839a00bfc1588b6e2a658
2021-05-24 19:47:57 +03:00
Egon Elbre
10372afbe4 ci: fix lint errors
Change-Id: Ib5893440807811f77175ccd347aa3f8ca9cccbdf
2021-05-17 13:37:31 +00:00
Qweder93
f2812d76cd multinode/payouts: satellite period/allTime summaries added
payout summaries for specific satellite period/allTime added

Change-Id: I144138304f01f23d5c4b10931988eaaced656aaf
2021-05-14 18:49:31 +00:00
Yaroslav Vorobiov
c08ca361d8 cmd/storj-sim: add multinode process
Change-Id: I7e6c8db045daa87c5d89f49e58fc82453d4024c6
2021-05-14 01:40:13 +03:00
Yaroslav Vorobiov
ea7fbdf843 multinode/multinodedb: add db migration
Change-Id: Ied1a7f3b951a25ab7a8edc25902c0479d2d08e87
2021-05-13 15:44:08 +00:00
Qweder93
19561698ba multinode/payouts: all satellites summaries added.
payout summaries for specific/all periods added.

Change-Id: I92087edec548c0418a0f543d643e59f5c7df9621
2021-05-12 18:43:47 +03:00
Qweder93
a11698f370 multinode/payouts: estimated payouts added
estimated payouts for specific/all satellites added.

Change-Id: I2530c9f1775593588e2a8f6c087ce6b4f9e354c4
2021-05-11 11:33:32 +00:00
Egon Elbre
0858c3797a satellite/{metabase,satellitedb}: deduplicate AS OF SYSTEM TIME code
Currently we were duplicating code for AS OF SYSTEM TIME in several
places. This replaces the code with using a method on
dbutil.Implementation.

As a consequence it's more useful to use a shorter name for
implementation - 'impl' should be sufficiently clear in the context.

Similarly, using AsOfSystemInterval and AsOfSystemTime to distinguish
between the two modes is useful and slightly shorter without causing
confusion.

Change-Id: Idefe55528efa758b6176591017b6572a8d443e3d
2021-05-11 12:40:36 +03:00
Egon Elbre
961e841bd7 all: fix error naming
errs.Class should not contain "error" in the name, since that causes a
lot of stutter in the error logs. As an example a log line could end up
looking like:

    ERROR node stats service error: satellitedbs error: node stats database error: no rows

Whereas something like:

    ERROR nodestats service: satellitedbs: nodestatsdb: no rows

Would contain all the necessary information without the stutter.

Change-Id: I7b7cb7e592ebab4bcfadc1eef11122584d2b20e0
2021-04-29 15:38:21 +03:00
Egon Elbre
a2e20c93ae private/dbutil: use dbutil and tagsql from storj.io/private
Initially we duplicated the code to avoid large scale changes to
the packages. Now we are past metainfo refactor we can remove the
duplication.

Change-Id: I9d0b2756cc6e2a2f4d576afa408a15273a7e1cef
2021-04-23 14:36:52 +03:00
Yaroslav Vorobiov
2e1455bc55 multinode/multinodedb: add sqlite3 support
Change-Id: I023ffb75f836de2c33eb7dbee52072e1622448bd
2021-02-11 16:36:05 +00:00
Qweder93
49884b5aad multinode/console: payouts added
Change-Id: I003b6a6286e39f43b3eb5e50de54b0a3d62f5abc
2021-02-10 09:45:16 +00:00
crawter
135d846aff mnd/console/server: endpoint with index.html added
Change-Id: Ic5154feaa995bf5c26c851024079f7e82612f306
2021-01-26 17:16:11 +00:00
Qweder93
a3b1059fe0 multinode/console: list node updated with total earned
Change-Id: I53a6ebd98acb7741f60d61178451575e7677c730
2021-01-12 23:01:49 +00:00
Yaroslav Vorobiov
6507f3ebc6 multinode/console: trusted satellites list api
Change-Id: I97bb9efb1d6cb7d456df0b86e66417c31018b762
2021-01-08 14:50:12 +02:00
Yaroslav Vorobiov
5a43c86b68 multinode/console: list node satellite infos
Change-Id: Ic6cb8d1a6fd7637fdb7bf49e040c43ac30ab1bbf
2021-01-05 14:49:58 +00:00
Yaroslav Vorobiov
fb00d099cf multinode/console: list node infos
Change-Id: I5cac49feff2bac6fbd7ac61dfccffd672da8e8c0
2021-01-05 14:49:11 +00:00