Commit Graph

270 Commits

Author SHA1 Message Date
Egon Elbre
6502143e79
fix import ordering (#2322) 2019-06-25 12:46:29 +03:00
Yingrong Zhao
bbedff12a6 satellite: rearrange marketing package (#2268)
* move offer out of marketing package and remove marketing package

* fix imports

* fix rename errors

* remove offer service

* change package name from offers to rewards

* fix linting

* remove unused code and use appropriate comment
2019-06-24 16:51:54 -04:00
Stefan Benten
01beaa289a
Mask IP Addresses to subnets (#2305) 2019-06-24 17:33:18 +02:00
Cameron
1283036e37
add storage node voucher request service (#2158)
* add voucher service on storage node

* config field tag syntax, go routines for requests

* hook up voucher service in storagenode/peer.go

* add voucher config to testplanet

* add voucher config to testplanet

* add voucher response status INVALID, ACCEPTED, REJECTED

* add a test for vouchers service

* handle no row from GetValid, test it

* add trust pool to voucher service

* use trusted list to get satellites

* verify vouchers upon receipt

* test VerifyVoucher
2019-06-21 18:48:52 -04:00
aligeti
043d603cbe
satellite rs config check with validation check set to false default (#2229)
* satellite rs config check with validation check
2019-06-21 14:15:58 -04:00
Bill Thorp
8f47fca5d3
Remove audit / uptime ratio fields (#2247)
* removed ratios
2019-06-21 13:14:53 -04:00
Egon Elbre
69c2cbacdf internal/testplanet: improve test speed (#2280) 2019-06-21 18:41:39 +02:00
Ivan Fraixedes
3d6b25a043
[v3-1952 test 1 & 3] pkg/audit: Add DQ test for too many failed audits (#2265)
* pkg/audit: Add DQ test for too many failed audits
  
  Add an integration test which checks that a node which fails several
  audits gets disqualified but not before it reaches the audit reputation
  disqualification cut-off.

* internal/testplanet: Set DQ cut-off config values
  Set the values of the Overlay cache DQ cut-off configuration parameters
  used by testplanet.
2019-06-21 18:27:19 +02:00
Egon Elbre
5b030062c0
internal/testplanet: split planet.go file to avoid package naming conflicts (#2279) 2019-06-21 16:39:43 +03:00
Bryan White
5f47b7028d Uplink C bindings part 3 (#2258)
* add object upload and download
2019-06-21 08:24:06 -04:00
JT Olio
568b000e9b satellite: make order expiration configurable (#2251) 2019-06-21 13:38:40 +03:00
Egon Elbre
d435d4859d internal/testplanet: add missing consoleserver.Config (#2269) 2019-06-20 22:27:36 +02:00
Kaloyan Raev
964c87c476 Fix checks around repair threshold (#2246) 2019-06-19 22:13:11 +02:00
Egon Elbre
9d670b33b3
reduce expected disk size for storj-sim and testplanet (#2249) 2019-06-19 18:29:45 +03:00
Natalie Villasana
b30c35d306
change ReputationAuditOmega (et al.) to AuditReputationWeight (#2232) 2019-06-18 14:17:25 -04:00
Egon Elbre
f4f776d092 Use mail.test as domain in emails (#2224) 2019-06-18 02:28:40 +02:00
Bryan White
76061c50c5
Uplink C bindings part 2 (#2209) 2019-06-15 13:23:12 +02:00
Bill Thorp
6932dc92f3
NodeSelectionConfig inits and test cleanup (#2202)
* NodeSelectionConfig inits and test cleanup
2019-06-13 17:51:18 -04:00
Bryan White
bc33964729 Uplink C bindings part 1 (#2196)
* uplink cbindings tooling
2019-06-13 11:09:05 -04:00
Egon Elbre
1a1a084477
testcontext: sanitize folder name (#2195) 2019-06-13 15:46:08 +03:00
Egon Elbre
fdddaa2a47 Fix marketingweb code (#2177)
* set to only listen on 127.0.0.1, move static files to same location, better template handling

* handle error

* fix path in storj-sim

* revert template handling changes

* code shouldn't panic on invalid tempalte

* do not rewrite once writing has started

* write correct error code

* use filepath for path handling

* revert change

* fix

* fix mod tidy

* use correct error code for not found, avoid infinite loop on failure
2019-06-12 09:42:39 -04:00
Michal Niewrzal
7a28decfca Always use difficulty larger than 8 for identites (#2160)
Set difficulty for libuplink and testplanet larger than 8, because we use last byte in the node id to store the version number, so the difficulty must be larger than 8.
2019-06-10 14:00:53 +02:00
Stefan Benten
74484fc57e
Enforce our Minimum Requirements for Node Operators and sanity check them (#2155) 2019-06-10 12:14:50 +02:00
Egon Elbre
749846b42b
Update golangci-lint (#2159) 2019-06-10 11:52:09 +03:00
ethanadams
8f2dca8437 Re-enabling and fixing repairer tests (#2099)
* Disabled discovery service by changiing from Stop() to Pause()

Paused to solve race condition.  If discovery is running, it may mark a node "up" after they've been manually marked "down" in this test.

* Extend to the repair timeout

Fixes intermittent test failures when repairs were taking more than 2 seconds.

* Re-enabled test. Disabled discovery service by changiing from Stop() to Pause()

* Changed back to Stop.

* Revert "Changed back to Stop."

This reverts commit 46d410e72dfae63e0c44915be42784cc9a7b5abf.

* re-enabling TestIdentifyInjuredSegments

* Changed Pause to Stop.  Commented on timeout change

* testing...

* temporarily skipping audit tests

* changing back to discover Stop for testing via jenkins

* Revert "changing back to discover Stop for testing via jenkins"

This reverts commit 6aa8558b11a0053c30e0c8b2dbf0d6c0cb34ee6c.

* Changing back to Stop().  Depends on PR 2137

* Revert "temporarily skipping audit tests"

This reverts commit 1940ed9b315d663a0eb6c95521780cbcb48cb121.

* Removed reference to Graveyard since its been removed
2019-06-10 09:06:21 +02:00
JT Olio
469d485f62 dbutil: reduce defaults so sum is < 100 (#2157) 2019-06-09 21:04:22 +02:00
JT Olio
43d4f3daf5 discovery: remove graveyard (#2145) 2019-06-07 08:40:51 +03:00
Jess G
bf3d168cf6
makes sure all uplink cli configs get passed to libuplink, add stripeSize (#2103)
* makes sure all uplink cli configs get passed to libuplink, add stripSize

* update comment

* update defaults for uplink config blocksize

* changes per CR, update uplink config defaults

* pass shareSize from uplink config

* move block size validation to kvmeta pkg

* fix tests

* shareSize default 1k, rm config option blocksize

* rm printing err to stdout
2019-06-06 11:55:10 -07:00
Egon Elbre
0bf50c6825 sync2: prioritize Cycle stopping messages (#2137) 2019-06-06 10:45:52 -04:00
Egon Elbre
28a1201590 pkg/bwagreements: remove service (#2138) 2019-06-06 15:57:58 +02:00
JT Olio
d7f3a5f811
internal,lib,uplink: add monkit task to missing places (#2118)
* internal,lib,uplink: add monkit task to missing places

Change-Id: I490053eee4ed517502f9fe00c6394f0095bd13d0

* Include Monkit

* Add missing context

* Another missing ctx

* More ctx missing

* Linting

* go imports

Change-Id: Ibf0ed072eba339f027727ed8039f7bce1f223fa7

* fix semantic merge conflict

Change-Id: I67fb1f4e7b6cd5e89d69987ed7b3966b7d30ee37
2019-06-05 09:03:11 -06:00
JT Olio
f1641af802 storage: add monkit task to missing places (#2122)
* storage: add monkit task to missing places

Change-Id: I9e17a6b14f7c25bbf698eeecf32785e9add3f26e

* fix tests

Change-Id: Id078276fa3de61a28eb3d01d4e751732ecbb173f

* import order

Change-Id: I814e33755b9f10b5219af37cd828cd75eb3da1a4

* remove part of other commit

Change-Id: Idaa4c95cd65e97567fb466de49718db8203cfbe1
2019-06-05 16:23:10 +02:00
Michal Niewrzal
68c8ce63a7
Fix storing struct in context (#2126) 2019-06-05 16:02:29 +02:00
Stefan Benten
23213a7a29 Fix Config Comments in DButil Package (#2119)
* Fix Config Comments

* Updating lock file

* Update lock file

* Set Actual time.Duration

* Fix Issue
2019-06-04 17:53:38 -06:00
JT Olio
d02427e41a db: set max open conns, conn max lifetime, add db stat monitoring (#2117) 2019-06-04 23:30:21 +02:00
JT Olio
29d16b4d68 satellite: add monkit task to missing places (#2108) 2019-06-04 13:55:37 +02:00
Egon Elbre
bc3463b9d3 Don't convert to float to avoid losing precision. (#2092) 2019-06-03 18:37:40 +02:00
Kaloyan Raev
2ab95b533e
Check errors for possible outcomes from audit's DownloadShares (#2072) 2019-06-03 12:17:09 +03:00
Cameron
590b1a5a1d
Satellite voucher service (#2043)
* set up voucher service skeleton, basic test

* add VetNode db method

* basic test for VetNode

* encode and sign voucher functions

* fill out and sign vouchers

* test pass/fail voucher request

* match EncodeVoucher to other Encode functions
2019-05-30 15:52:33 -04:00
Stefan Benten
4b75752d6b
Relocate IP Lookup to ensure it is always set (#2061) 2019-05-30 19:35:04 +02:00
aligeti
934ebf9cbf
Added the irreparable repair functionality (#1955)
* Added the irreparable repair functionality
2019-05-30 11:18:20 -04:00
ethanadams
16e3b77cf5
Enable Scopelint Linter (#2049)
* added scopelint and correcte issues found

* corrected scopelint issue

* made updates based on Ivan's suggestions

Most were around naming conventions
Some were false positives, but I kept them since the test.Run could eventually be changed to run in parallel, which could cause a bug
Others were false positives.  Added // nolint: scopelint
2019-05-29 09:30:16 -04:00
Jess G
fb86238acc
aws s3 performance tests (#2060)
* add aws s3 benchmark script

* add s3 benchmark tests

* rearrange so smaller diff, fix spelling

* add configurable uplink config for s3-benchmark

* make new bucket w/unique name for each s3 test

* changes per CR
2019-05-28 11:46:58 -07:00
Simon Guindon
dc2e6b9370 Changing 64bit atomics to 32bit for alignment on ARM. (#2066) 2019-05-28 11:20:46 -06:00
Stefan Benten
8912d7149c
Fixes Auth Issue (#2064) 2019-05-28 16:32:51 +02:00
René Smeekes
fcc52fc692 Use absolute size for unit calculation to account for negative numbers (#2056)
* Use absolute size for unit calculation to account for negative numbers

* import "math"

* fix for Abs

* Fix type mismatch

* Update size.go

* Adjust Tests to reflect correct Byte Value
2019-05-25 18:20:30 +02:00
Jess G
6f23aeef18
add performance benchmarks (#2021)
* update uplink cmd args

* add uplod/download benchmark, add script to run benchmarks

* fix lint err

* changes per CR comments

* export env var for linux

* fix lint

* add byte count to bench test
2019-05-24 13:25:53 -07:00
Jeff Wendling
1bd52b9f90 server side macaroons (#1945)
What: Adds macaroon support to the server side

Why: So that api keys are now macaroons
2019-05-24 10:51:27 -06:00
Michal Niewrzal
7193b16e92
Java/Android libuplink bindings (#1918) 2019-05-24 10:13:01 +02:00
Cameron
4058c29ca4
filter duplicate node IPs (#1890)
* add last_ip field to dbx model node, generate dbx

* add last_ip to node proto, generate pb

* migrate

* resolve address in transport.DialNode, update lastIp in cache.UpdateAddress

* use net.SplitHostPort to isolate host address from port

* define DistinctIPs flag

* add test for GetIP

* select last_ip when querying for nodes

* if distinctIPs flag == true, query for nodes with distinct IPs

* some basic tests

* change last_ip to field 14 in proto

* remove comments

* check err

* change distinctIPs to distinctIP

* exclude IPs from newNodes in query for reputable nodes

* add index on last_ip

* only add to excludedIPs if flag is true

* test half new nodes returns distinct IPs

* fix alignment

* add test

* rework ip filter query, add retry logic, add switch for database driver

* add retry to SelectNewNodes

* change discovery intervals so IPs don't get overwritten

* remove TestGetIP

* edit updating node stats in test

* split exclude into nodeIDs and IPs

* separate non-distinct IP query into other function

* trigger checks

* remove else block
2019-05-22 16:06:27 -04:00