Commit Graph

1555 Commits

Author SHA1 Message Date
Bryan White
c607abf27c [V3-1147] Ensure certificate validation happens properly (#1403)
* add regression test & update transport tests

* separate client and server verificiation functions

* goimports
2019-03-06 09:42:34 -05:00
Egon Elbre
ead4a0ae95
Add new piece id definition (#1411) 2019-03-06 11:17:35 +02:00
Natalie Villasana
036b2f3b49
allocates 1kb for audit bandwidth agreements instead of 256kb (#1404) 2019-03-05 18:58:25 -05:00
aligeti
15034526cd Enhance the Storage Node to keep all BWA's for 90 days (#1374)
* Enhance the Storage Node to keep all BWA's for 90 days

* warn -> error

* rename enum
2019-03-05 17:20:46 -05:00
Jess G
3c9d83dbfe
convert psserver dashboard into an inspector (#1407)
* Convert psserver dashboard into an inspector

* remove dashboard stream, update ps.pb.mock

* fixes for lint errs
2019-03-05 15:48:37 -05:00
Bogdan Artemenko
467fe3f700
Implemented API keys fetch on login. (#1406)
Implemented API keys fetch on login and reload.
2019-03-05 13:51:59 +02:00
Yehor Butko
3e90b9e3b8
V3-1284 Node bootstrap web frontend (#1405)
* V3-1284 Node bootstrap web frontend
2019-03-05 13:19:43 +02:00
Yehor Butko
3e2c101bd8
V3-1152 Node bootstrap web backend (#1327)
* V3-1152 Node bootstrap
2019-03-05 12:38:21 +02:00
Benjamin Sirb
7e2e4b5397 Update README.md (#1400) 2019-03-05 09:54:49 +02:00
Bryan White
5742131941
tlsconfig refactor / cleanup (#1399) 2019-03-04 21:40:18 +01:00
Dennis Coyle
59233c6d75 Update .clabot (#1401)
Adding Dr. Ben
2019-03-04 21:21:49 +01:00
Bryan White
675e0ef683 [V3-1320] fix empty node ID verification non-error (#1395)
* small identity refactor:

+ Optimize? iterative cert chain methods to use array instead of slice
+ Add `ToChain` helper for converting 1d to 2d cert chain
  TODO: replace literal declarations with this
+ rename `ChainRaw/RestChainRaw` to `RawChain/RawRestChain`
  (adjective noun, instead of nound adjective)

* add regression tests for V3-1320

* fix V3-1320

* separate `DialUnverifiedIDOption` from `DialOption`

* separate `PingNode` and `DialNode` from `PingAddress` and `DialAddress`

* update node ID while bootstrapping

* goimports & fix comment

* add test case
2019-03-04 15:03:33 -05:00
aligeti
588e2a51d2 Handles zero bucket count and bucket with no files (#1391)
* handles zero bucket count and bucket with no files
2019-03-04 12:47:08 -05:00
Michal Niewrzal
cce84c9914
Cleanup errors handling around piece hash verification (#1382) 2019-03-04 15:29:35 +01:00
Kaloyan Raev
5fa7a4a7c6 Ensure ECClient upload timer is stopped when no more status is expected (#1397)
This change ensures that the upload timer of ECClient is always stopped after no more status is expected from uploaded pieces. It also ensures that the "Timer expired" message will be logged only if the context is not already cancelled.

This is to avoid confusing logs where a "Timer expired" message is logged significantly later and mixes with similar messages logged from the upload of the next file segments.
2019-03-04 14:48:13 +01:00
Stefan Benten
5f57d2c906
Reduce Log Messages on Production Settings (#1396)
* Reduce Tally Log Messages on Production Settings

* Reduce Logging in Piecestore Client as well
2019-03-04 11:12:43 +01:00
Yaroslav Vorobiov
a30ba4eca8
Add mail service to the satellite (#1302) 2019-03-02 17:22:20 +02:00
Kaloyan Raev
54f68347ad
Use DialNode for requesting node info (#1394) 2019-03-02 09:34:08 +02:00
JT Olio
01031fdcdc web ui: change default graphql endpoint (#1393)
this default graphql endpoint works whenever the satellite is hosting the npm built assets.
2019-03-01 21:35:31 +01:00
Dylan Lott
9abf191b01
Change payments to use statDB for wallet addresses rather than Overlay cache (#1368)
* Wiring up DumpNodes response for Inspector

* Finalize everything and test that it works

* Get Count and DumpNodes working for Overlay Cache

* WIP updating payment rollup to check statDB instead of overlay

* FIrst pass at updating statDB to take wallet and email

* Passing tests

* use pb.NodeOperator instead of Meta struct

* remove TODO

* revert go.mod

* Get SQL migration working correctly

* Changes Meta to Operator in NodeStats struct

* Adds update operator logic for statDB

* Fix db migrate tests - added v5 snapshot

* User friendly msg for missing snapshot version

* Passing tests

* Change node update to happen in discovery instead of in overlay

* Fix logic and update function calls

* Update comment on UpdateOperator interface method

* Update name of parameter

* Change type of argument to UpdateOperator

* Updates statDB tests
2019-03-01 10:46:34 -07:00
Cameron
3fd55c7034
remove unused date field in csv row (#1388) 2019-03-01 08:23:27 -05:00
Egon Elbre
3f3209c8d5
fixes to piecestore and psdb (#1380)
* replace direct reference with an interface in various places
* hide piecePath
* ensure psserver tests don't use path
* ensure psserver tests don't use sql queries directly
2019-03-01 07:46:16 +02:00
JT Olio
2efaa27318 accounting: support null wallet addresses (#1387) 2019-03-01 07:34:46 +02:00
Philip Hutchins
1f452e67dc Adding dockerfile for running the web UI for Satellite (#1366)
* Adding dockerfile for running the web UI for Satellite

* Updating to work with Makefile and from root directory of repo

* Updating satellite ui build process to run in a more production like mode by generating the assets the pulling those into the satellite container

* Updates to allow external traffic to UI, updates to storagenode for identity creation, and logging for bug tracking

* Adding auto cert generation for storagenode

* removing satellite-ui-image from main images flow in Makefile and adding latest tag to docker build for it

* Adding solid defaults, tuning dockerfiles, and moving to standard logging methods

* Updating logging to be more standard

* Updating to logger.Debug

* Removing unused library and unused identity creation code

Change-Id: I956453037e303693ea37f94318180af0ab7984d5
2019-02-28 13:18:53 -07:00
Dylan Lott
bacd683879
Adds DumpNodes method to Kademlia Inspector (#1328)
* Wiring up DumpNodes response for Inspector

* Finalize everything and test that it works

* remove changes to overlay inspector

* Replace list with iterate method in DumpNodes

* generate proto

* Merge in nodeinfo changes

* Update iterate function in DumpNodes

* Fix import order
2019-02-28 12:55:27 -07:00
Cameron
018cdd4037
uplink ttl (#1365)
* add expires flag to uplink cp

* fix ttlDBs delete expired query
2019-02-28 12:51:24 -05:00
Jennifer Li Johnson
6ecda190cb
Jj/payments cleanup (#1379)
* Removes date from payments csv and re-query of wallet addr

* removes dbx get wallet addr

* switch order in csv
2019-02-28 12:13:59 -05:00
Michal Niewrzal
4122c98cb7
Validate piece hash on satellite (#1359)
The satellite receives pieces signed hashes in Pointer. If signed hash cannot be validated then piece is removed from Pointer and not saved in DB.
2019-02-28 15:14:54 +01:00
Jess G
e82d3367f0
rename payment command to node usage (#1378) 2019-02-27 16:55:19 -05:00
Brandon Iglesias
b3d1ab32f5 adding JessicaGreben to our CLA bot (#1373) 2019-02-27 12:10:58 -07:00
aligeti
993af7ed37 fixes the bwa deletes to be done by serialnum, rather by signature (#1364) 2019-02-27 18:29:23 +02:00
Egon Elbre
e4706d12f2
disable vet for tests since it's also being done in when linting (#1370) 2019-02-27 16:04:45 +02:00
Egon Elbre
b7f8e309d9 add piecestore.Storage interface (#1369) 2019-02-27 10:56:16 +01:00
Egon Elbre
307cc640b6 add testcontext.Compile (#1356) 2019-02-27 09:25:28 +01:00
Bryan White
fde0020c68
dial node/address with bad server cert (#1342) 2019-02-26 19:35:16 +01:00
Jennifer Li Johnson
cefdff535a
Replace list calls with iterates in kademlia routing table (#1318)
* notes

* adds bool for option to skip self

* wip getKBucketID is still breaking TestAddNode

* fix (#1322)

* remove panic

* remove iteratebuckets

* implements getKBucketRange

* getNodeIDsWithinKBucket

* removes determineFurthestIDWithinK

* wip

* nodeIsWithinNearestK

* removes bucketIDToKey

* removes extra xor in test

* wip

* rebased

* uses create rt w opts

* lint

* fix lint
2019-02-26 11:07:53 -05:00
Egon Elbre
0b5e7d34b7 update golang.org/x packages and grpc (#1363) 2019-02-26 16:49:56 +01:00
aligeti
c6ad7644d2
Total file count through Monkit (#1351)
* segment, file, byte stats, total and per-bucket; checker: report segment health stats; reports the total num of lost files

* code review updates
2019-02-26 10:17:51 -05:00
Egon Elbre
d0b5b9b51b
reorder graphite metric string for easier parsing (#1362) 2019-02-26 15:47:31 +02:00
Bryan White
03747bad51
some certificates cmd cleanup: (#1361) 2019-02-26 09:55:52 +01:00
Bryan White
62a3749139
fix claims delete config (#1360) 2019-02-25 22:29:19 +01:00
Kaloyan Raev
1ec17653d4 Endpoint for local node info (#1355)
Adds a new `Info` method to the Kademlia endpoint that returns the following local node info:
* ID
* Type
* Metadata (email and wallet)
* Restrictions (free storage and bandwidth)

The new endpoint is exposed as `inspector kad node-info` command too.
2019-02-25 19:41:51 +01:00
Michal Niewrzal
81408a3c9e
Use SignedHash on client/uplink side (#1354)
* psclient receives storage node hash and compare it to own hash for verification
* uplink sends delete request when hashes don't match
* valid hashes are propagated up to segments.Store for future sending to satellite
2019-02-25 16:57:54 +01:00
Egon Elbre
bad733cfac
add Patrick Gerbes to cla (#1357) 2019-02-25 16:40:48 +02:00
Faris Huskovic
f75ec3ec82 Update README.md (#1353) 2019-02-25 15:09:44 +01:00
Bryan White
b652b6a522 use peer ca whitelist in testplanet (#1337) 2019-02-25 09:38:03 +02:00
Michal Niewrzal
6186b3f90a
Storage node hash calculation on upload (#1347)
Storage node is calculating hash of uploaded data and send it back to uplink with signature
2019-02-23 11:46:07 +01:00
Bill Thorp
373b301736
BWA aliases (#1333)
aliased RBAs and PBAs
2019-02-22 16:17:35 -05:00
Jennifer Li Johnson
b5447c6608
Routing table tests (#1270)
This PR includes a new package called testrouting, which implements a very algorithmically slow but hopefully easy-to-keep-operationally-correct in-memory routing table. The routing table also supports writing out its current structure as a DOT graph for visualization. testrouting is primarily meant to help in coming up with generic routing table integration tests.

This PR also includes a new routing table integration test suite that runs against all current routing table implementations. Our existing routing table passes a lot of the tests, but not all of them, still debugging why. I have confirmed the tests should pass with the visualization graphs though.
2019-02-22 13:39:29 -05:00
aligeti
4ccb80257d
Refactor how bandwidth allocations are stored on storage nodes (#1280)
* integrated with db migration framework

* updating the migrationdb test

* updating the migrationdb sql v0 and v1 files

* lint warnings

*  code review changes

* fix migration scripts, upated test data

* fixed the local/UTC timing issues

* code review updates

* updated code review comments
2019-02-22 10:51:39 -05:00