Commit Graph

149 Commits

Author SHA1 Message Date
Simon Guindon
d3885b7b78
Change crypto/rand to use math/rand in pgutil (#1589)
* Added retry logic and tests for handling if crypto/rand fails.

* Added retry logic and tests for handling if crypto/rand fails.

* Fixing linting error.

* Removing use of `crypto/rand` for use of `math/rand`.

* Changing code comment about why we ignore this error.
2019-04-02 12:52:25 -04:00
Alexander Leitner
59b3eb190e
API endpoint checking object and segment health from the satellite (#1546)
* Generate protobuf for checking object and segment health

* Initial cmd client for health check

* Register the health inspector on the satellite peer
2019-04-02 10:55:58 -04:00
Michal Niewrzal
f80750693c Store bandwidth from orders on satellite (#1586) 2019-04-01 16:14:58 -04:00
Kaloyan Raev
034c283eda Set last_contact_success when a node is initially added to overlay cache (#1632) 2019-04-01 12:42:06 -06:00
Cameron
6d43832c4f
record bucket data into bucket_storage_tally table (#1595)
* add MetadataSize to stats

* add logic for accumulating bucket stats in calculateAtRestData

* rename stats to BucketTally, move to accounting package

* define method on accountingDB for inserting bucketTallies

* insert bucketTallies into bucket_storage_tally table
2019-04-01 09:42:17 -04:00
Egon Elbre
872bd5d7c1
Remove pointerdb.Server (#1609) 2019-03-30 13:21:49 +02:00
paul cannon
3b12b5e85c
non-racy CreateEntryIfNotExists() (#1611)
This changes semantics slightly! with this change,
CreateEntryIfNotExists() will do a cache Update with every node passed
in, whether it exists or not. Update() already does a race-free upsert
operation, so that change removes the problematic race in
CreateEntryIfNotExists(). As far as I can tell, this semantic change
doesn't break any expectations of callers, and shouldn't affect
performance in a significant way, as we already have an awful lot of
round-trips to the db either way. But if I've misunderstood the
intention of the method, someone ought to catch it during review.
2019-03-29 15:07:07 -06:00
Yehor Butko
66a7d6bf3a
Satellite password check simplification (#1610) 2019-03-29 19:42:20 +02:00
Egon Elbre
9dbc7c719e
fix error handling in GetByProjectID (#1604) 2019-03-29 14:55:05 +02:00
Egon Elbre
de15a4fdcf
remove utils.CombineErrors and utils.ErrorGroup (#1603) 2019-03-29 14:30:23 +02:00
Yaroslav Vorobiov
0ce6d4ab81
Project api auth update (#1572) 2019-03-29 14:13:37 +02:00
Kaloyan Raev
9fb99c8484
Ensure serial number is not saved if error during order limit creation (#1602) 2019-03-29 11:53:53 +02:00
Kaloyan Raev
f9ba935286
Merge overlay_cache_nodes into nodes table (#1581) 2019-03-29 10:53:43 +02:00
Egon Elbre
be06fdfd6c Create orders.Service (#1593) 2019-03-28 22:09:23 +02:00
Jennifer Li Johnson
b3a8beb1bc
Updates accounting tables models and migration (#1584)
* go gen

* undo changes to bucket usage

* update locked

* spacing

* moves changes to migration v11

* minor changes to fix lint and test err

* change sql to fix errs
2019-03-27 22:46:49 -04:00
Yehor Butko
86bf3dee9f
V3-1307 combine first and last name to full name (#1569)
* V3-1307 combine first and last name to full name
2019-03-27 14:33:32 +02:00
Michal Niewrzal
bfdfebbde2
Satellite orders receiving (#1564)
This change adds satellite endpoint for receiving OrderLimits sent by storage node.
Change includes:
* wire up orders sender in storage node (also in testplanet)
* saving serial number for OrderLimit in serial_numbers table
* satellite endpoint for receiving, verifying and storing OrderLimit and Order serial number
* initial implementation for Orders DB
* basic test for sending orders to satellite
2019-03-27 11:24:35 +01:00
Natalie Villasana
5b48a48a79
adds test for correct download with non-critical amount of nodes offline (#1574) 2019-03-26 14:09:44 -04:00
Yaroslav Vorobiov
b2b86bee56
[V3-1366] Email project name bug (#1571) 2019-03-26 17:56:16 +02:00
Stefan Benten
3db903fe40
Move from Unique to Index on SatelliteDB (#1563)
* Move from Unique to Index

* Remove Index

* Make some more Indexes Unique and adjust migration

* Fix Migration Statements

* Fix Typo

* Fix Migration of older Table

* Exchange DROP statement

* Remove "if not exists"

* Revert Change in old Migration
2019-03-26 11:34:30 +01:00
Kaloyan Raev
d1639c4157 Merge statdb pkg into overlay pkg (#1570) 2019-03-25 18:25:09 -04:00
Yaroslav Vorobiov
cce6fa8fd3
added LOGIN auth scheme to satellite mail service (#1552) 2019-03-24 02:08:41 +02:00
Egon Elbre
94e79eda6d
remove overlay endpoint (#1521) 2019-03-23 10:06:11 +02:00
Natalie Villasana
0fa1d536e7
removes pingbackTimeout (#1556) 2019-03-22 16:06:57 -04:00
Egon Elbre
297b393dfd database for tracking information without storing orders / agreements (#1551)
* initial schema declaration

* fix typo

* split database

* fix syntax errors and generate code

* add queries for serial number manipulation

* update comment

* add migration script

* fix comments

* add new data

* fix bucket id
2019-03-22 11:54:22 -07:00
Egon Elbre
694b6dc1da
make tests run faster (#1553) 2019-03-22 15:14:17 +02:00
Natalie Villasana
61ee04d363
adds pingbackTimeout to kademlia endpoint (#1518) 2019-03-19 14:30:27 -04:00
Yehor Butko
c6f8d82978
V3-1319 Add an authorization token field for account creation backend (#1492)
* V3-1319 Add an authorization token field for account creation on the satellite UI backend
2019-03-19 19:55:43 +02:00
Natalie Villasana
a09f568fdb
audit service won't hang on bad storage node conns (#1503) 2019-03-19 13:37:26 -04:00
Michal Niewrzal
b05cf05649
Restrict slash in bucket name (#1524) 2019-03-19 15:37:28 +01:00
paul cannon
cd91a22e0f better way to check if err is ErrorCode_NoRows (#1453)
* better way to check if err is ErrorCode_NoRows
2019-03-18 20:15:27 -04:00
Egon Elbre
a24c74c502 fix message formatting (#1512) 2019-03-18 18:02:37 +01:00
Egon Elbre
05d148aeb5
Storage node and upload/download protocol refactor (#1422)
refactor storage node server
refactor upload and download protocol
2019-03-18 12:55:06 +02:00
Cameron
c7ffbe1c28
Add ability to view irreparable segments on satellite (#1448)
* define irreparable inspector protobuf

* add IrreparableDB method GetLimited

* fill out irreparable inspector API

* add IrreparableInspector server to satellite, fix small error

* refactor IrreparableDB to use pb.IrreparableSegment instead of irreparable.RemoteSegmentInfo
2019-03-15 16:21:52 -04:00
Bill Thorp
665fd33e3c
Repair queue isolation level fix (#1466)
Implemented custom SQLite and Postgres Repairqueue Dequeue handlers
2019-03-14 17:12:47 -04:00
Bill Thorp
52e829c6de
framework for deleting expired Orders from Satellite (#1436)
framework for deleting old bandwidth agreements
2019-03-12 16:57:21 -04:00
Dylan Lott
59f1e267c9
Removes concept of email from kademlia metadata (#1435)
* Removes concept of Email from Kademlia

* Removes kad email

* adds emails back to operator config for satellite

* replace operator configs in testplanet
2019-03-12 14:05:18 -06:00
Stefan Benten
12c5f42c30
Add Application Name to PostgreSQL Connection String (#1451)
* Add Application Name to PostgreSQL Connection String,
if not existing already

* Relocate Check

* Handle Application Name only for Postgres

* URL Encode

* Relocate Function into pgutil

* Fix Error, when ApplicationName is set

* Rename parameter

* Rename Check Value

* Straightline Comment

* Remove fmt Dependency

* Fix Linting Recommendation
2019-03-12 14:29:13 +01:00
Bill Thorp
1f4e1a3ab0
framework to delete raw accounting tallies (#1439)
framework to delete raw accounting tallies, renamed Query
2019-03-12 09:08:23 -04:00
littleskunk
b26e660ec3 add index created_at on bwagreements (#1438) 2019-03-11 08:09:50 +02:00
Yehor Butko
3ab696164f
Satellite console new activation flow (#1421)
Satellite console new activation flow
2019-03-08 16:01:11 +02:00
Jess G
193a70f0a6
add private listener to grpc server (#1398)
* add private listener to grpc server

* add changes per init CR

* fix server.close

* add insecure grpc connection, update logs msg

* fix tests, move insecure client

* add private ports to storj-sim, add insecure client to other inspectors

* add ports to test so there arent conflicts

* fix lint err

* fix node started log msg, close public listener

* remove commented out line
2019-03-07 13:19:37 -05:00
Yaroslav Vorobiov
9cf56c03fa
Bucket usage info rollup table creation and implementation (#1340) 2019-03-06 17:54:48 +02:00
Yehor Butko
af86d8e368
V3-1095 Update service to send invitations in project (#1412)
* V3-1095 Update service to send invitations in project
2019-03-06 17:42:19 +02:00
Yehor Butko
3e2c101bd8
V3-1152 Node bootstrap web backend (#1327)
* V3-1152 Node bootstrap
2019-03-05 12:38:21 +02:00
Yaroslav Vorobiov
a30ba4eca8
Add mail service to the satellite (#1302) 2019-03-02 17:22:20 +02:00
Dylan Lott
9abf191b01
Change payments to use statDB for wallet addresses rather than Overlay cache (#1368)
* Wiring up DumpNodes response for Inspector

* Finalize everything and test that it works

* Get Count and DumpNodes working for Overlay Cache

* WIP updating payment rollup to check statDB instead of overlay

* FIrst pass at updating statDB to take wallet and email

* Passing tests

* use pb.NodeOperator instead of Meta struct

* remove TODO

* revert go.mod

* Get SQL migration working correctly

* Changes Meta to Operator in NodeStats struct

* Adds update operator logic for statDB

* Fix db migrate tests - added v5 snapshot

* User friendly msg for missing snapshot version

* Passing tests

* Change node update to happen in discovery instead of in overlay

* Fix logic and update function calls

* Update comment on UpdateOperator interface method

* Update name of parameter

* Change type of argument to UpdateOperator

* Updates statDB tests
2019-03-01 10:46:34 -07:00
JT Olio
2efaa27318 accounting: support null wallet addresses (#1387) 2019-03-01 07:34:46 +02:00
Philip Hutchins
1f452e67dc Adding dockerfile for running the web UI for Satellite (#1366)
* Adding dockerfile for running the web UI for Satellite

* Updating to work with Makefile and from root directory of repo

* Updating satellite ui build process to run in a more production like mode by generating the assets the pulling those into the satellite container

* Updates to allow external traffic to UI, updates to storagenode for identity creation, and logging for bug tracking

* Adding auto cert generation for storagenode

* removing satellite-ui-image from main images flow in Makefile and adding latest tag to docker build for it

* Adding solid defaults, tuning dockerfiles, and moving to standard logging methods

* Updating logging to be more standard

* Updating to logger.Debug

* Removing unused library and unused identity creation code

Change-Id: I956453037e303693ea37f94318180af0ab7984d5
2019-02-28 13:18:53 -07:00
Jennifer Li Johnson
6ecda190cb
Jj/payments cleanup (#1379)
* Removes date from payments csv and re-query of wallet addr

* removes dbx get wallet addr

* switch order in csv
2019-02-28 12:13:59 -05:00