Rename the functions that are prefixed with 'New' which connect with
Redis by 'Open' to make clear that they perform network operations.
Change-Id: I1351e89a642e8e2c2586626646315ad0fb2c6242
Update the Redis dependency to use the last major production version.
The last version accepts a context parameter in all the network methods
so it allows us to pass it through them.
Change-Id: I34121b2ec3c2728602115c724933ad24c9e6e4fd
In order to prevent traffic amplification attack, QUIC allows
application to perform an address validation during connection
establishment.
(https://tools.ietf.org/id/draft-ietf-quic-transport-34.html#name-address-validation)
However, this adds another round trip during connection establishment.
In storj network, it does client authentication before servers starts
sending significant amount of data to any client. We believe the traffic
amplification attack isn't going to be significant when turning off
address validation in QUIC. AND it will provide us a significant
performance boost during connection establishment.
Change-Id: I7f9a0ca5ca770b715d08b1e8ce3022fbb2b85d42
This PR introduces a new connector that will try to establish a tcp and
a quic connection concurrently and returns the connection that's
ready first. If both connections are ready around the same time, it will
prioritize/return quic connection.
Change-Id: Ib87175321516125526aa01c96386bdc8d3b863b0
allow disabling tcp/quic
In order to have more control of a server so that we can
simulate connection failures in `testplanet`, this PR changes
quic.Listener to accept an existing UDPConn instead of relying on the
quic-go library to create the UDPConn.
This PR also adds two flags on the `server.Config` struct to allow
enabling/disabling tcp/tls listener and quic listener. By default, they
are both set to true.
- `DisableTCPTLS`: internal flag, disables tcp/tls listener.
- `DisableQUIC`: hidden flag, disables quic listener
By making the `DisableQUIC` a hidden flag, it allows storagenode operators to
have the ability to disable quic traffic in case their set up can't work
with udp traffic.
Change-Id: I853b12435d988b9c41ad9b873fd57480d792e378
This PR introduces a new listener that can listen for quic traffic on
both storagenodes and satellites.
Change-Id: I5eb5bc82c37dde20d3be2ec8fa5f69c18fae0af0
Due to the issues with the licensing and go version requirement by the
quic-go library, we can not introduce the library into libuplink at the
moment. Therefore, we decided to put our quic integration code into
storj/storj. It will unblock us from rolling out quic to storagenodes.
Change-Id: If9d765da10c45947a4e3a3a11e4679bab69bcf08
To avoid further name collisions, the very broad named package gets moved into
the consoleauth package where its also mainly being used.
Change-Id: Ie563c9700adbf0553baca2b7b8ba4a1d9c29d144
The stable uplink library is located at storj.io/uplink.
The stable uplink C library is located at storj.io/uplink-c.
Change-Id: Ia69115d6384b668490efe7e8ab674d7e8104b4f4
This will help to determine how many grpc calls are made to the
satellite.
Also remove the grpc funcs that have been added to upstream.
Change-Id: I91878f4fd10f9bfe601c94222c102eaaf4d35963
* debug
* traces
* cfgstruct
* process
Package `storj/private/version` will be removed as a separate change.
Change-Id: Iadc40faa782e6225513b28218952f02d9c240a9f
This moves grpc related tlsopts methods to private/grpctlsopts.
This allows to remove grpc dependency from tlsopts.
Change-Id: I25090b82b1e7a0633417ad600f8587b0c30ace73
Now that we are trying to identify the root cause of the satellite load limitations (i.e. currently the satellite has a max ability of 400 rps for uploads and we need this to be higher), we are using the golang diagnostic tools to collect insight into what the bottlenecks are. We currently have a debug endpoint to gather some cpu and mem data, but it could be useful to have continuous profiling. GCP stackdriver has support for continuous profiling so lets set that up and see if it is helpful to gather more data.
This PR adds support for [GCP continuous profiler](https://cloud.google.com/profiler) which allows enabling continuous cpu/mem profiling and the stats are sent to stackdriver in google cloud console.
To enable the continuous profiling for a storj component, do the following:
- prereq: the workload must be running in GKE and have Stackdriver Profiling IAM role permissions
- provide the config flag `debug.profilename` in the config.yaml file for the workload (i.e. satellite api process, etc). The profilename should be the workload name, for example "satellite-api".
- once the above config flag is provided, the profiler will be initialized and profiling stats will automatically be sent to GCP project where the workload is running and viewable in the Stackdriver Profile page in the console
The current implementation assumes the workload is running in GKE, however if we find if useful we can add support to enable this from anywhere. But for simplicity, its configured this way assuming the main goal is to enable in production systems.
Change-Id: Ibf8ebe2df7bf06fdd4951ee6a1e48854dd36ad47
this commit updates our monkit dependency to the v3 version where
it outputs in an influx style. this makes discovery much easier
as many tools are built to look at it this way.
graphite and rothko will suffer some due to no longer being a tree
based on dots. hopefully time will exist to update rothko to
index based on the new metric format.
it adds an influx output for the statreceiver so that we can
write to influxdb v1 or v2 directly.
Change-Id: Iae9f9494a6d29cfbd1f932a5e71a891b490415ff
Control Panel allows to control different chores and services.
Currently this adds controlling of cycles.
Change-Id: I734f1676b2a0d883b8f5ba937e93c45ac1a9ce21
By separating Server it allows Peers to directly embed the server
and provide customizations and hooks into rest of the services.
Change-Id: Ic1d68740fd494d2f82c1739bd990849c561b912b
Multiple time.Now calls in a short amount of time
may return the exact same time.
This adds a time.Sleep() to ensure we get different internal
timing.
Change-Id: I533b5eedfdcab13a0197976ccf4b9fa96d4c3bba
