Commit Graph

255 Commits

Author SHA1 Message Date
Egon Elbre
f4f776d092 Use mail.test as domain in emails (#2224) 2019-06-18 02:28:40 +02:00
Bryan White
76061c50c5
Uplink C bindings part 2 (#2209) 2019-06-15 13:23:12 +02:00
Bill Thorp
6932dc92f3
NodeSelectionConfig inits and test cleanup (#2202)
* NodeSelectionConfig inits and test cleanup
2019-06-13 17:51:18 -04:00
Bryan White
bc33964729 Uplink C bindings part 1 (#2196)
* uplink cbindings tooling
2019-06-13 11:09:05 -04:00
Egon Elbre
1a1a084477
testcontext: sanitize folder name (#2195) 2019-06-13 15:46:08 +03:00
Egon Elbre
fdddaa2a47 Fix marketingweb code (#2177)
* set to only listen on 127.0.0.1, move static files to same location, better template handling

* handle error

* fix path in storj-sim

* revert template handling changes

* code shouldn't panic on invalid tempalte

* do not rewrite once writing has started

* write correct error code

* use filepath for path handling

* revert change

* fix

* fix mod tidy

* use correct error code for not found, avoid infinite loop on failure
2019-06-12 09:42:39 -04:00
Michal Niewrzal
7a28decfca Always use difficulty larger than 8 for identites (#2160)
Set difficulty for libuplink and testplanet larger than 8, because we use last byte in the node id to store the version number, so the difficulty must be larger than 8.
2019-06-10 14:00:53 +02:00
Stefan Benten
74484fc57e
Enforce our Minimum Requirements for Node Operators and sanity check them (#2155) 2019-06-10 12:14:50 +02:00
Egon Elbre
749846b42b
Update golangci-lint (#2159) 2019-06-10 11:52:09 +03:00
ethanadams
8f2dca8437 Re-enabling and fixing repairer tests (#2099)
* Disabled discovery service by changiing from Stop() to Pause()

Paused to solve race condition.  If discovery is running, it may mark a node "up" after they've been manually marked "down" in this test.

* Extend to the repair timeout

Fixes intermittent test failures when repairs were taking more than 2 seconds.

* Re-enabled test. Disabled discovery service by changiing from Stop() to Pause()

* Changed back to Stop.

* Revert "Changed back to Stop."

This reverts commit 46d410e72dfae63e0c44915be42784cc9a7b5abf.

* re-enabling TestIdentifyInjuredSegments

* Changed Pause to Stop.  Commented on timeout change

* testing...

* temporarily skipping audit tests

* changing back to discover Stop for testing via jenkins

* Revert "changing back to discover Stop for testing via jenkins"

This reverts commit 6aa8558b11a0053c30e0c8b2dbf0d6c0cb34ee6c.

* Changing back to Stop().  Depends on PR 2137

* Revert "temporarily skipping audit tests"

This reverts commit 1940ed9b315d663a0eb6c95521780cbcb48cb121.

* Removed reference to Graveyard since its been removed
2019-06-10 09:06:21 +02:00
JT Olio
469d485f62 dbutil: reduce defaults so sum is < 100 (#2157) 2019-06-09 21:04:22 +02:00
JT Olio
43d4f3daf5 discovery: remove graveyard (#2145) 2019-06-07 08:40:51 +03:00
Jess G
bf3d168cf6
makes sure all uplink cli configs get passed to libuplink, add stripeSize (#2103)
* makes sure all uplink cli configs get passed to libuplink, add stripSize

* update comment

* update defaults for uplink config blocksize

* changes per CR, update uplink config defaults

* pass shareSize from uplink config

* move block size validation to kvmeta pkg

* fix tests

* shareSize default 1k, rm config option blocksize

* rm printing err to stdout
2019-06-06 11:55:10 -07:00
Egon Elbre
0bf50c6825 sync2: prioritize Cycle stopping messages (#2137) 2019-06-06 10:45:52 -04:00
Egon Elbre
28a1201590 pkg/bwagreements: remove service (#2138) 2019-06-06 15:57:58 +02:00
JT Olio
d7f3a5f811
internal,lib,uplink: add monkit task to missing places (#2118)
* internal,lib,uplink: add monkit task to missing places

Change-Id: I490053eee4ed517502f9fe00c6394f0095bd13d0

* Include Monkit

* Add missing context

* Another missing ctx

* More ctx missing

* Linting

* go imports

Change-Id: Ibf0ed072eba339f027727ed8039f7bce1f223fa7

* fix semantic merge conflict

Change-Id: I67fb1f4e7b6cd5e89d69987ed7b3966b7d30ee37
2019-06-05 09:03:11 -06:00
JT Olio
f1641af802 storage: add monkit task to missing places (#2122)
* storage: add monkit task to missing places

Change-Id: I9e17a6b14f7c25bbf698eeecf32785e9add3f26e

* fix tests

Change-Id: Id078276fa3de61a28eb3d01d4e751732ecbb173f

* import order

Change-Id: I814e33755b9f10b5219af37cd828cd75eb3da1a4

* remove part of other commit

Change-Id: Idaa4c95cd65e97567fb466de49718db8203cfbe1
2019-06-05 16:23:10 +02:00
Michal Niewrzal
68c8ce63a7
Fix storing struct in context (#2126) 2019-06-05 16:02:29 +02:00
Stefan Benten
23213a7a29 Fix Config Comments in DButil Package (#2119)
* Fix Config Comments

* Updating lock file

* Update lock file

* Set Actual time.Duration

* Fix Issue
2019-06-04 17:53:38 -06:00
JT Olio
d02427e41a db: set max open conns, conn max lifetime, add db stat monitoring (#2117) 2019-06-04 23:30:21 +02:00
JT Olio
29d16b4d68 satellite: add monkit task to missing places (#2108) 2019-06-04 13:55:37 +02:00
Egon Elbre
bc3463b9d3 Don't convert to float to avoid losing precision. (#2092) 2019-06-03 18:37:40 +02:00
Kaloyan Raev
2ab95b533e
Check errors for possible outcomes from audit's DownloadShares (#2072) 2019-06-03 12:17:09 +03:00
Cameron
590b1a5a1d
Satellite voucher service (#2043)
* set up voucher service skeleton, basic test

* add VetNode db method

* basic test for VetNode

* encode and sign voucher functions

* fill out and sign vouchers

* test pass/fail voucher request

* match EncodeVoucher to other Encode functions
2019-05-30 15:52:33 -04:00
Stefan Benten
4b75752d6b
Relocate IP Lookup to ensure it is always set (#2061) 2019-05-30 19:35:04 +02:00
aligeti
934ebf9cbf
Added the irreparable repair functionality (#1955)
* Added the irreparable repair functionality
2019-05-30 11:18:20 -04:00
ethanadams
16e3b77cf5
Enable Scopelint Linter (#2049)
* added scopelint and correcte issues found

* corrected scopelint issue

* made updates based on Ivan's suggestions

Most were around naming conventions
Some were false positives, but I kept them since the test.Run could eventually be changed to run in parallel, which could cause a bug
Others were false positives.  Added // nolint: scopelint
2019-05-29 09:30:16 -04:00
Jess G
fb86238acc
aws s3 performance tests (#2060)
* add aws s3 benchmark script

* add s3 benchmark tests

* rearrange so smaller diff, fix spelling

* add configurable uplink config for s3-benchmark

* make new bucket w/unique name for each s3 test

* changes per CR
2019-05-28 11:46:58 -07:00
Simon Guindon
dc2e6b9370 Changing 64bit atomics to 32bit for alignment on ARM. (#2066) 2019-05-28 11:20:46 -06:00
Stefan Benten
8912d7149c
Fixes Auth Issue (#2064) 2019-05-28 16:32:51 +02:00
René Smeekes
fcc52fc692 Use absolute size for unit calculation to account for negative numbers (#2056)
* Use absolute size for unit calculation to account for negative numbers

* import "math"

* fix for Abs

* Fix type mismatch

* Update size.go

* Adjust Tests to reflect correct Byte Value
2019-05-25 18:20:30 +02:00
Jess G
6f23aeef18
add performance benchmarks (#2021)
* update uplink cmd args

* add uplod/download benchmark, add script to run benchmarks

* fix lint err

* changes per CR comments

* export env var for linux

* fix lint

* add byte count to bench test
2019-05-24 13:25:53 -07:00
Jeff Wendling
1bd52b9f90 server side macaroons (#1945)
What: Adds macaroon support to the server side

Why: So that api keys are now macaroons
2019-05-24 10:51:27 -06:00
Michal Niewrzal
7193b16e92
Java/Android libuplink bindings (#1918) 2019-05-24 10:13:01 +02:00
Cameron
4058c29ca4
filter duplicate node IPs (#1890)
* add last_ip field to dbx model node, generate dbx

* add last_ip to node proto, generate pb

* migrate

* resolve address in transport.DialNode, update lastIp in cache.UpdateAddress

* use net.SplitHostPort to isolate host address from port

* define DistinctIPs flag

* add test for GetIP

* select last_ip when querying for nodes

* if distinctIPs flag == true, query for nodes with distinct IPs

* some basic tests

* change last_ip to field 14 in proto

* remove comments

* check err

* change distinctIPs to distinctIP

* exclude IPs from newNodes in query for reputable nodes

* add index on last_ip

* only add to excludedIPs if flag is true

* test half new nodes returns distinct IPs

* fix alignment

* add test

* rework ip filter query, add retry logic, add switch for database driver

* add retry to SelectNewNodes

* change discovery intervals so IPs don't get overwritten

* remove TestGetIP

* edit updating node stats in test

* split exclude into nodeIDs and IPs

* separate non-distinct IP query into other function

* trigger checks

* remove else block
2019-05-22 16:06:27 -04:00
Egon Elbre
9c23c2d427 db: set max idle connections higher to avoid redialing all the time (#1991) 2019-05-21 17:30:06 +03:00
Jess G
8518618b7a
add postgres support to storj-sim (#1908)
* add flags to sotrj-sim for SA dbs

* add schema to postgres

* add createschema with parse to sa

* add metainfo db postgres support

* add kv default as bolt

* add debug log to see db source

* add env var for postgres to test-sim.sh

* fix lint errs

* dynamically add postgres to args

* add postgres to integration tests

* add sqlite and postgres integration jenkins

* fix db name

* merge integration tests into one step

* test integration tests w/psql

* try using different schema

* debug failure

* use correct host for running storj-sim

* rm sqlite integration

* add back integration
2019-05-14 08:13:18 -07:00
Simon Guindon
7ce9a60c85
Add additional error handling to email message sending. (#1639)
* Adding additional error handling to email message sending.

* Adding additional error handling to email message sending.

* Fixing issue with err being lost in the defer.

* Moving defers to be queued up earlier to ensure they get called.

* Fixed closing encoder to use defer pattern that matches the others.
2019-05-13 11:14:45 -04:00
Jeff Wendling
5ab8c9d052 transport: separate dial from request timeouts (#1940)
A previous change reused the same timeout for dialing as well as
requesting in order to speed up some tests. This change introduces
a distinct timeout so that the different operations can have different
timeouts.
2019-05-10 14:26:25 +03:00
Ivan Fraixedes
de2242a05a internal/sync2: Cycle minor impl & docs improvements (#1936)
* internal/sync2: Cycle minor impl & docs improvements

Add the following improvements to the Cycle type of the internal/sync2
package:

* Avoid that Close method hang if Start/Run has not been called.
* Ensure that internal ticker is always stopped.
* Add clarifications when methods calls panic.

* internal/sync2: add minimal package level docs

* Changes defer
2019-05-09 20:19:06 +02:00
Egon Elbre
a2b61fd67c
storage node collector (#1913) 2019-05-08 14:11:59 +03:00
Stefan Benten
891780990f
Add Identity to Version Control Server (#1876)
* Add Version Information for Identity Tool

* Adding Versions to JSON Output

* Restart CLA-Bot
2019-05-01 18:21:51 +02:00
Egon Elbre
db939d37ec
cover all the things (#1818) 2019-04-26 16:39:11 +03:00
Natalie Villasana
bcfb03836f kademlia Bootstrap function has exponential backoff (#1786)
* add exponential backoff to kademlia
2019-04-26 08:42:09 -04:00
Bill Thorp
a11dc76169
wired up online config (#1827)
* wired up online config
2019-04-26 08:15:06 -04:00
Ivan Fraixedes
ba152790b1
internal/textcontext: Fail test on error & docs (#1835)
* Make the `Dir` function to fail the test if the directories cannot be
  created.
* Improve the docs to clarify the behavior of the exported functions.
* Make the default timeout exported for being able to see its value in
  the documentation.
2019-04-26 12:21:32 +02:00
Egon Elbre
60c4c10c79
storagenode: delete psserver (#1837) 2019-04-26 08:17:18 +03:00
Jennifer Li Johnson
db6a2a46a9
set delete-tallies default to false (#1844) 2019-04-25 15:30:37 -04:00
Michal Niewrzal
fe3dfc1587
Move pointerdb.Service to satellite (#1826) 2019-04-25 10:46:32 +02:00
Michal Niewrzal
a585b97363
Download is hanging when one node is not responsive (#1764)
Change closes download connection in parallel and sets a timeout for communication between uplink and storage node
2019-04-25 10:17:26 +02:00