Commit Graph

32 Commits

Author SHA1 Message Date
Michal Niewrzal
fdf40a7526 storj: remove storj/private/version package which was moved to
`storj/private` repo

Change-Id: I81c3f5b9d5e4fe7bca760999eb045ee9734e5e2e
2020-03-24 14:31:33 +00:00
Jennifer Johnson
1c1750e6be removes bandwidth limiting
On satellite, remove all references to free_bandwidth column in nodes table.
On storage node, remove references to AllocatedBandwidth and MinimumBandwidth and mark as deprecated.

Protobuf message, NodeCapacity, is left intact for backwards compatibility.
Once this is released to all satellites, we can drop the column from the DB.

Change-Id: I2ff6c6537fc9008a0c5588e951afea58ede85838
2020-03-04 14:04:00 +00:00
Qweder93
484ec7463a storagenode: notifications on outdated software version
Change-Id: If19b075c78a7b2c441e11b783c3c09fed55060c7
2020-03-02 16:48:02 +00:00
Jeff Wendling
7999d24f81 all: use monkit v3
this commit updates our monkit dependency to the v3 version where
it outputs in an influx style. this makes discovery much easier
as many tools are built to look at it this way.

graphite and rothko will suffer some due to no longer being a tree
based on dots. hopefully time will exist to update rothko to
index based on the new metric format.

it adds an influx output for the statreceiver so that we can
write to influxdb v1 or v2 directly.

Change-Id: Iae9f9494a6d29cfbd1f932a5e71a891b490415ff
2020-02-05 23:53:17 +00:00
Jeff Wendling
71ff044edb storagenode/bandwidth: fix tests to not fail for 10 hours near the end of the month
Change-Id: I390569a8702164c42edddd3be020e93782227c2e
2020-01-31 16:25:52 -07:00
Isaac Hess
14fd6a9ef0 storagenode/pieces: Track total piece size
This change updates the storagenode piecestore apis to expose access to
the full piece size stored on disk. Previously we only had access to
(and only kept a cache of) the content size used for all pieces. This
was inaccurate when reporting the amount of disk space used by nodes.

We now have access to the total content size, as well as the total disk
usage, of all pieces. The pieces cache also keeps a cache of the total
piece size along with the content size.

Change-Id: I4fffe7e1257e04c46021a2e37c5adc6fe69bee55
2020-01-23 11:00:24 -07:00
Egon Elbre
6615ecc9b6 common: separate repository
Change-Id: Ibb89c42060450e3839481a7e495bbe3ad940610a
2019-12-27 14:11:15 +02:00
Vitalii Shpital
038ac58600
web/storagenode: minimal allowed version view implemented (#3583) 2019-11-26 18:08:24 +02:00
Vitalii Shpital
61c8bcc9a6
web/storagenode: egress chart implemented (#3574) 2019-11-20 16:37:57 +02:00
Nikolai Siedov
24318d74b3
storagenode/console: show satellite url in satellite selection (#3602) 2019-11-19 14:16:56 +02:00
Egon Elbre
ee6c1cac8a
private: rename internal to private (#3573) 2019-11-14 21:46:15 +02:00
Egon Elbre
93353df4d6
internal/sync2: make Fence accept context (#3393) 2019-10-28 16:04:31 +02:00
Bryan White
243ba1cb17
{versioncontrol,internal/version,cmd/*}: refactor version control (#3253) 2019-10-20 09:56:23 +02:00
Vitalii Shpital
78a71ad3b6
web/storagenode: node status updated (#3220) 2019-10-11 19:28:47 +03:00
Yaroslav Vorobiov
a11619e7f3
storagenode/console: use bandwidth monthly summary (#3183) 2019-10-04 09:29:25 -06:00
Yaroslav Vorobiov
4824ecdb8d storagenode/console: use bytes for remaining info (#3186) 2019-10-04 18:17:28 +03:00
littleskunk
b2e328f118 storagenode/dashboard: update online status (#3168) 2019-10-03 20:31:39 +02:00
Jennifer Li Johnson
724bb44723
Remove Kademlia dependencies from Satellite and Storagenode (#2966)
What:

cmd/inspector/main.go: removes kad commands
internal/testplanet/planet.go: Waits for contact chore to finish
satellite/contact/nodesservice.go: creates an empty nodes service implementation
satellite/contact/service.go: implements Local and FetchInfo methods & adds external address config value
satellite/discovery/service.go: replaces kad.FetchInfo with contact.FetchInfo in Refresh() & removes Discover()
satellite/peer.go: sets up contact service and endpoints
storagenode/console/service.go: replaces nodeID with contact.Local()
storagenode/contact/chore.go: replaces routing table with contact service
storagenode/contact/nodesservice.go: creates empty implementation for ping and request info nodes service & implements RequestInfo method
storagenode/contact/service.go: creates a service to return the local node and update its own capacity
storagenode/monitor/monitor.go: uses contact service in place of routing table
storagenode/operator.go: moves operatorconfig from kad into its own setup
storagenode/peer.go: sets up contact service, chore, pingstats and endpoints
satellite/overlay/config.go: changes NodeSelectionConfig.OnlineWindow default to 4hr to allow for accurate repair selection
Removes kademlia setups in:

cmd/storagenode/main.go
cmd/storj-sim/network.go
internal/testplane/planet.go
internal/testplanet/satellite.go
internal/testplanet/storagenode.go
satellite/peer.go
scripts/test-sim-backwards.sh
scripts/testdata/satellite-config.yaml.lock
storagenode/inspector/inspector.go
storagenode/peer.go
storagenode/storagenodedb/database.go
Why: Replacing Kademlia

Please describe the tests:
• internal/testplanet/planet_test.go:

TestBasic: assert that the storagenode can check in with the satellite without any errors
TestContact: test that all nodes get inserted into both satellites' overlay cache during testplanet setup
• satellite/contact/contact_test.go:

TestFetchInfo: Tests that the FetchInfo method returns the correct info
• storagenode/contact/contact_test.go:

TestNodeInfoUpdated: tests that the contact chore updates the node information
TestRequestInfoEndpoint: tests that the Request info endpoint returns the correct info
Please describe the performance impact: Node discovery should be at least slightly more performant since each node connects directly to each satellite and no longer needs to wait for bootstrapping. It probably won't be faster in real time on start up since each node waits a random amount of time (less than 1 hr) to initialize its first connection (jitter).
2019-09-19 15:56:34 -04:00
paul cannon
c139ed8ea1 storagenode/console: remove kademlia (#2942)
this is a trivial operation for storagenode/console, as it doesn't
really need or use kademlia in the first place.

What:

Removes kademlia from storagenode/console

Why:

We are in the process of getting rid of kademlia, and this is one place where it's particularly easy.

Please describe the tests:

Existing tests exercise storagenode/console behavior; if they continue to work, everything here should be tested satisfactorily.
Please describe the performance impact:

None
2019-09-11 16:41:43 -04:00
Egon Elbre
a801fab66a
all: add archview annotations (#2964) 2019-09-10 16:24:16 +03:00
Yaroslav Vorobiov
c35ad5cbfc
storagenode/console: update api (#2969) 2019-09-06 15:01:03 +03:00
Yaroslav Vorobiov
758f7cb3dd
storagenode/bandwidth: remove bandwidth concerns from console, add satellite summary (#2923) 2019-09-04 17:01:55 +03:00
Yaroslav Vorobiov
141af7e2f7
storagenode/console: refactor service and api (#2751) 2019-08-14 15:17:11 +03:00
Yaroslav Vorobiov
28a7778e9e
storagenode/nodestats: cache node stats (#2543) 2019-08-08 16:47:04 +03:00
paul cannon
17bdb5e9e5
move piece info into files (#2629)
Deprecate the pieceinfo database, and start storing piece info as a header to
piece files. Institute a "storage format version" concept allowing us to handle
pieces stored under multiple different types of storage. Add a piece_expirations
table which will still be used to track expiration times, so we can query it, but
which should be much smaller than the pieceinfo database would be for the
same number of pieces. (Only pieces with expiration times need to be stored in piece_expirations, and we don't need to store large byte blobs like the serialized
order limit, etc.) Use specialized names for accessing any functionality related
only to dealing with V0 pieces (e.g., `store.V0PieceInfo()`). Move SpaceUsed-
type functionality under the purview of the piece store. Add some generic
interfaces for traversing all blobs or all pieces. Add lots of tests.
2019-08-07 20:47:30 -05:00
Yehor Butko
5f194b4533
SNO Dashboard API (#2427)
* SNO Dashboard API
2019-07-17 14:42:00 +03:00
Yaroslav Vorobiov
524eb24c83 storagenode/nodestats: combine stats into single RPC call (#2455)
* change satellite nodestats endpoint
2019-07-08 17:33:43 +03:00
Yaroslav Vorobiov
9e8ecb6303
Storagenode nodestats at daily space usage (#2422) 2019-07-02 15:05:58 +03:00
Yaroslav Vorobiov
16cd1fde87 Storagenode add daily bandwidth usage query for SNO (#2348) 2019-07-02 11:53:39 +02:00
Yaroslav Vorobiov
a6db2dd332
Storagenode add nodestats client for SNO console (#2287) 2019-06-26 21:55:22 +03:00
Yehor Butko
96bc0ccfa4
SNO Dshboard initial api endpoint added (#2284)
* initial api endpoint added
2019-06-24 18:15:31 +03:00
Yehor Butko
e5fd0287e4
V3-1819 Storage node operator server and service started (#2112)
* V3-1819 Storage node operator server and service started
2019-06-20 14:52:32 +03:00