Commit Graph

1869 Commits

Author SHA1 Message Date
JT Olio
d6c02fc657 storagenode/piecestore: add meters to upload/download rates (#2106) 2019-06-04 15:22:00 +02:00
VitaliiShpital
fbdff0fad8
nobuckets state implemented (#2110) 2019-06-04 16:04:17 +03:00
paul cannon
d15eaed588 add capability of logging all GRPC calls/payloads (#2067) 2019-06-04 14:55:24 +02:00
Egon Elbre
3b02607c1d go.mod: update grpc to latest version (#2096) 2019-06-04 14:38:31 +02:00
JT Olio
c4bb84f209 storagenode: add monkit task to missing places (#2107) 2019-06-04 14:31:38 +02:00
JT Olio
3fe8343b6c repairer: fix config comments (#2105) 2019-06-04 14:13:31 +02:00
Stefan Benten
e2c5d75b4c
Add VitaliiShpital (#2111) 2019-06-04 14:06:18 +02:00
JT Olio
29d16b4d68 satellite: add monkit task to missing places (#2108) 2019-06-04 13:55:37 +02:00
JT Olio
9c5708da32 pkg/*: add monkit task to missing places (#2109) 2019-06-04 13:36:27 +02:00
Egon Elbre
bc3463b9d3 Don't convert to float to avoid losing precision. (#2092) 2019-06-03 18:37:40 +02:00
Cameron
e077b0d380
rename VetNode to IsVetted (#2097)
* rename VetNode to IsVetted
2019-06-03 10:53:30 -04:00
Egon Elbre
b8e0ac6377
satellitedb/overlaycache: avoid tx leak in case of error (#2095) 2019-06-03 17:37:43 +03:00
Yaroslav Vorobiov
6809129e6f
Console add stripe service (#2080) 2019-06-03 16:46:57 +03:00
Egon Elbre
8c0c518621 pkg/storj: use proper unquoting of json data in NodeID and PieceID (#2094) 2019-06-03 15:29:07 +02:00
Yaroslav Vorobiov
79a0085103
Fix storage and object_count calculations on console (#2081) 2019-06-03 15:54:06 +03:00
Nikolay Yurchenko
5062436846
Account settings page rework (#2007) 2019-06-03 15:19:48 +03:00
Kaloyan Raev
2ab95b533e
Check errors for possible outcomes from audit's DownloadShares (#2072) 2019-06-03 12:17:09 +03:00
JT Olio
e60ff9dcbb
process/metrics: have metrics suffix default to dev/release status (#2073)
What: this will make it so release binaries default to whatever-release instead of whatever-dev in metrics collection

Why: So we can monitor release binaries with default configuration without getting drowned out by dev binaries
2019-05-31 16:47:48 -06:00
Jeff Wendling
140251882e
fix bug for setting flag only values in process setup (#2089)
* fix bug for setting flag only values in process setup

when the code was changed to directly load values into the config
structs, it was missed that some configuration is only defined
through flags, but can be loaded from config files still.

so, we need to propogate the settings to the flag only values.

* add test for setting propagation

* fix linting error
2019-05-31 21:15:50 +00:00
Maximillian von Briesen
294fddcece
Add audit monkit stats (#2087) 2019-05-31 16:46:25 -04:00
Fadila
5b730e3073
Make maxReverifyCount configurable (#2071)
* make max reverify count configurable
2019-05-31 17:23:00 +02:00
aligeti
4ad5120923
Checker service refactor (v3-1871) (#2082)
*  refactor the checker service

* monkit update
2019-05-31 10:12:49 -04:00
Natalie Villasana
6db9388082 add disqualified column to nodes table (#2086)
* add disqualified column to nodes table, update migrate script and testdata

* fix crazy formatting of postgres.v25.sql
2019-05-30 17:38:23 -04:00
Cameron
590b1a5a1d
Satellite voucher service (#2043)
* set up voucher service skeleton, basic test

* add VetNode db method

* basic test for VetNode

* encode and sign voucher functions

* fill out and sign vouchers

* test pass/fail voucher request

* match EncodeVoucher to other Encode functions
2019-05-30 15:52:33 -04:00
Jennifer Li Johnson
24c8132975
rewrites encode methods for readability (#2084) 2019-05-30 15:01:55 -04:00
Stefan Benten
4b75752d6b
Relocate IP Lookup to ensure it is always set (#2061) 2019-05-30 19:35:04 +02:00
Maximillian von Briesen
04c20b0ac0
Add monkit stats to piecestore upload/download (#2078) 2019-05-30 11:44:47 -04:00
aligeti
934ebf9cbf
Added the irreparable repair functionality (#1955)
* Added the irreparable repair functionality
2019-05-30 11:18:20 -04:00
Dennis Coyle
006509b758
Value Attribution design doc (#1996)
* first draft fo value attribution design doc

* Update value-attribution.md

* PR comments
2019-05-30 10:12:34 -04:00
Jeff Wendling
e74cac52ab
Command line flags features and cleanup (#2068)
* change BindSetup to be an option to Bind
* add process.Bind to allow composite structures
* hack fix for noprefix flags
* used tagged version of structs

Before this PR, some flags were created by calling `cfgstruct.Bind` and having their fields create a flag. Once the flags were parsed, `viper` was used to acquire all the values from them and config files, and the fields in the struct were set through the flag interface.

This doesn't work for slices of things on config structs very well, since it can only set strings, and for a string slice, it turns out that the implementation in `pflag` appends an entry rather than setting it.

This changes three things:

1. Only have a `Bind` call instead of `Bind` and `BindSetup`, and make `BindSetup` an option instead.
2. Add a `process.Bind` call that takes in a `*cobra.Cmd`, binds the struct to the command's flags, and keeps track of that struct in a global map keyed by the command.
3. Use `viper` to get the values and load them into the bound configuration structs instead of using the flags to propagate the changes.

In this way, we can support whatever rich configuration we want in the config yaml files, while still getting command like flags when important.
2019-05-29 17:56:22 +00:00
Brandon Iglesias
771271e7b8 updating help message on storage node for AllocatedBandwidth (#2074) 2019-05-29 13:40:04 -04:00
Nikolay Yurchenko
8918a14bbf add payment popup added (#2032) 2019-05-29 19:22:16 +03:00
ethanadams
16e3b77cf5
Enable Scopelint Linter (#2049)
* added scopelint and correcte issues found

* corrected scopelint issue

* made updates based on Ivan's suggestions

Most were around naming conventions
Some were false positives, but I kept them since the test.Run could eventually be changed to run in parallel, which could cause a bug
Others were false positives.  Added // nolint: scopelint
2019-05-29 09:30:16 -04:00
ethanadams
268dc6b7e4
Enable gocritic linter (#2051)
* first round cleanup based on go-critic

* more issues resolved for ifelsechain and unlambda checks

* updated from master and gocritic found a new ifElseChain issue

* disable appendAssign. i reports false positives

* re-enabled go-critic appendAssign and disabled lint check at code line level

* fixed go-critic lint error

* fixed // nolint add gocritic specifically
2019-05-29 09:14:25 -04:00
Yaroslav Vorobiov
20782d75bd Fix dates source on UsageReport page (#2070) 2019-05-29 12:00:33 +02:00
Michal Niewrzal
03bae48ad6
Remove unused parameter (#2069) 2019-05-29 11:15:15 +02:00
Jess G
fb86238acc
aws s3 performance tests (#2060)
* add aws s3 benchmark script

* add s3 benchmark tests

* rearrange so smaller diff, fix spelling

* add configurable uplink config for s3-benchmark

* make new bucket w/unique name for each s3 test

* changes per CR
2019-05-28 11:46:58 -07:00
Simon Guindon
dc2e6b9370 Changing 64bit atomics to 32bit for alignment on ARM. (#2066) 2019-05-28 11:20:46 -06:00
Michal Niewrzal
f731267e8c Per-project usage limiting (#2036)
What: Changes to support custom usage limit for the project. With this implementation by default project usage limit is taken from configuration flag. If project DB field usage_limit will be set to value larger than 0 it will become custom usage limit and we will be used to verify is limit was exceeded.

Whats changed:

usage_limit (bigint) field added to projects table (with migration)
things related to project usage moved from metainfo endpoint to project usage type
accounting.ProjectAccounting extended with GetProjectUsageLimits() method
Why: We need to have different usage limits per project. https://storjlabs.atlassian.net/browse/V3-1814
2019-05-28 09:36:52 -06:00
Maximillian von Briesen
c07162beef address potential divide by 0` (#2065) 2019-05-28 08:54:30 -06:00
Stefan Benten
8912d7149c
Fixes Auth Issue (#2064) 2019-05-28 16:32:51 +02:00
Maximillian von Briesen
5a4ff2c855 add repair monkit stats (#2045)
* add repair monkit stats

* rename values, use meter instead of counter, use success threshold instead of repair threshold

* Counter -> Meter

* add repair segment size

* update names and use ratios for healthy before/after repair

* restart jenkins
2019-05-28 16:10:26 +02:00
Natalie Villasana
aa6ff17b70 add Reverify to auditing (#2041)
Co-authored-by: Maximillian von Briesen <mobyvb@gmail.com>
Co-authored-by: Kaloyan Raev <kaloyan@storj.io>
2019-05-27 14:13:47 +03:00
Maximillian von Briesen
7a9742f47b size of raw key should be at least storj.KeySize (#2058) 2019-05-25 21:41:40 +02:00
Michal Niewrzal
4e8b5cb972 Fix flaky TestProjectUsageBandwidth (#2047) 2019-05-25 21:24:12 +02:00
René Smeekes
fcc52fc692 Use absolute size for unit calculation to account for negative numbers (#2056)
* Use absolute size for unit calculation to account for negative numbers

* import "math"

* fix for Abs

* Fix type mismatch

* Update size.go

* Adjust Tests to reflect correct Byte Value
2019-05-25 18:20:30 +02:00
Stefan Benten
20c9f9776c Add ReneSmeekes (#2057) 2019-05-25 18:01:16 +02:00
Jess G
6f23aeef18
add performance benchmarks (#2021)
* update uplink cmd args

* add uplod/download benchmark, add script to run benchmarks

* fix lint err

* changes per CR comments

* export env var for linux

* fix lint

* add byte count to bench test
2019-05-24 13:25:53 -07:00
Fadila
8f078d2841 Delete pending audit entries from containment DB when files get deleted (#2050) 2019-05-24 15:56:08 -04:00
Ivan Fraixedes
2cdc55d345 cmd/uplink: Fix a faulty test (#2054)
The test was not considering the proper length of a slice generated with
a random length.
2019-05-24 10:55:46 -07:00