Commit Graph

1996 Commits

Author SHA1 Message Date
Kaloyan Raev
ebd9b375fc
Repair should not corrupt files (#2194) 2019-06-14 12:16:31 +03:00
Bill Thorp
6932dc92f3
NodeSelectionConfig inits and test cleanup (#2202)
* NodeSelectionConfig inits and test cleanup
2019-06-13 17:51:18 -04:00
ethanadams
4e1cc37a4c Fix bucket_storage_tally data (#2165) 2019-06-13 12:58:40 -04:00
JT Olio
df2fad15d8
pkg/process/logging: different defaults for release/dev (#2191)
* pkg/process/logging: different defaults for release/dev

Change-Id: I55be80430a31668fededf479b052e106ab18d9ce

* linting

Change-Id: I4e50d4c9569b7324c4704c14df7dd3228dbb7dd5

* Trigger Jenkins

* fix lock file

* use dev=debug and prod=info
2019-06-13 10:43:39 -06:00
JT Olio
fa09a7894b pkg/process: prometheus support (#2176) 2019-06-13 18:29:35 +02:00
Matt Robinson
2fdcd8c632 Give the uplink image the same business (#2168) 2019-06-13 18:14:53 +02:00
Bill Thorp
703e5fee49
mostly removed things (#2199) 2019-06-13 12:06:37 -04:00
Bryan White
bc33964729 Uplink C bindings part 1 (#2196)
* uplink cbindings tooling
2019-06-13 11:09:05 -04:00
JT Olio
1ae5654eba kademlia/routing: add contexts to more places so monkit works (#2188) 2019-06-13 16:51:50 +02:00
Yingrong Zhao
ec63204ab1 fix export name for DatePicker (#2185) 2019-06-13 16:04:48 +02:00
Faris Huskovic
c0fe061a67 add user credits table (#2085)
* add user credits table

* change primary key, change type for credit_type, and change relation kind of foreign keys from cascade to restrict

* modify table and query methods

* modify schema

* add dbx queries

* add migration file

* add orderby to read available credit entries
2019-06-13 09:52:33 -04:00
Egon Elbre
1a1a084477
testcontext: sanitize folder name (#2195) 2019-06-13 15:46:08 +03:00
ethanadams
380684e766 Removing bucket_id and adding project_id and bucket_name as primary key (#2182)
* removing bucket_id and adding project_id and bucket_name as primary key

* cleanup
2019-06-13 08:28:55 -04:00
aligeti
8e8a5eb89c
initial value attribution check development (#2091)
* initial value attribution check development

* code reveiew updates
2019-06-12 21:35:37 -04:00
JT Olio
ff7a9030eb lib/uplink: expose restrict on api keys (#2189) 2019-06-12 23:35:57 +02:00
Yingrong Zhao
474d9e7492
only parse templates for the first requests (#2180)
* only parse templates for the first requests

* fix race condition isse

* parse all templates before start the new server

* add comment for parseTemplates and remove unused code

* remove unused type
2019-06-12 16:27:07 -04:00
Stefan Benten
e85c56518a
Add Isaac (#2184) 2019-06-12 20:59:56 +02:00
Yingrong Zhao
6a826c53c5
fix satellite gui build issue (#2181)
* move import statement out from declare module

* delete unused component
2019-06-12 13:39:48 -04:00
Yingrong Zhao
a4d6e8c8ca split update methods into two for offer data (#2133)
* change Offers interface to separate Update method into two.

* Implement Finish and Redeem method to avoid concurrent updates

* Implement FinishOffer and RedeemOffer service methods

* add tests

* fix linting issue

* add tests for checking Finish and Redeem's results to work as expected

* fix linting error
2019-06-12 11:53:19 -04:00
JT Olio
14486b1885 satellite/orders: TODOs about transactions (#2134) 2019-06-12 17:00:29 +02:00
Egon Elbre
fdddaa2a47 Fix marketingweb code (#2177)
* set to only listen on 127.0.0.1, move static files to same location, better template handling

* handle error

* fix path in storj-sim

* revert template handling changes

* code shouldn't panic on invalid tempalte

* do not rewrite once writing has started

* write correct error code

* use filepath for path handling

* revert change

* fix

* fix mod tidy

* use correct error code for not found, avoid infinite loop on failure
2019-06-12 09:42:39 -04:00
Natalie Villasana
5314c950cc
pkg/audit: add more reverify tests (#2144) 2019-06-11 16:14:40 -04:00
Yingrong Zhao
df1401d952 add segment.io into satellite GUI (#1946)
* Inject segment snippet

* Add page tracking on components mounted

* Use router state to send tracking data to segment.io

Move router tracking into router file

* Create plugin for analytics

* Add plugin

* Add .env file for segment track id

* Replace custome plugin with vue-segment-analytics package

* Rename .env file to not be used for production

* Add DNT check before tracking

clean up code

* move tracker to be above page reload

* Inject segment snippet

* Add page tracking on components mounted

* Use router state to send tracking data to segment.io

Move router tracking into router file

* Create plugin for analytics

* Add plugin

* Add .env file for segment track id

* Replace custome plugin with vue-segment-analytics package

* Rename .env file to not be used for production

* Add DNT check before tracking

clean up code

* move tracker to be above page reload
2019-06-11 15:00:23 -04:00
Jess G
d84cd719ff
add const stripesPerBlock const to calc blocksize (#2163)
* add const stripesPerBlock, update comments in Pad, add speed to progressbar

* change size of erasure share size, update comments

* missing copyright

* update tests with stripesPerBlock
2019-06-11 10:14:05 -07:00
Yingrong Zhao
af66d9c6e4
Open a new port on satellite for admin GUI (#1901)
* Set up new port 8090 for in offers

Clean up commented code

Rename offers to offersweb

Remove unused code

Add todos for adding front-end templates

Add middleware for only allow local access

Add comment

Fix linting error

Remove commented code

Update storj-sim

Check request IP against Host IP

Use net pakcage to retrieve IP address

Rename service to marketing

* Add wrapper for all errors

* fix conflicts

* update the config file

* fix linting error

* remove unused packages

* remove global runtime var and add flag to storj-sim for mar static dir

* remove debugging lines

* add new config for test data and check if static dir flag is set before passing to mux

* change 'console' to 'marketing' for test data config

* fix linting errors

* update config flag

* Trigger Jenkins

* Trigger CLA
2019-06-11 11:00:59 -04:00
Kaloyan Raev
252c8ac189
Add email to self node info (#2171) 2019-06-11 16:30:28 +03:00
Ivan Fraixedes
f624b213a3
reputation: Add configuration parameters (#2150)
* reputation: Add configuration parameters
 Add the configuration parameters which will be used by the algorithm
 which will calculate the storage node reputation.
 Because the reputation calculation is based on audit and uptime check
 results some configuration parameters are in pkg/audit, others in
 pkg/discovery and other in the satellite which will combine the both
 reputation results to obtain the storage node reputation for repair and
 uplink.
* satellite-config: Refresh lock file with new params
  Refresh the Satellite configuration yaml lock file with the new
  parameters added in this branch.
2019-06-11 12:14:01 +02:00
Kaloyan Raev
f0880b9b3c
Audit service should download erasure shares in parallel (#2164) 2019-06-11 11:00:59 +03:00
littleskunk
5e7fed7541 improve logging (#2170)
Signed-off-by: littleskunk <jens.heimbuerge@googlemail.com>
2019-06-10 21:30:17 -06:00
Egon Elbre
61c8c3fb49
satellite/orders: batch update storage node allocations (#2146) 2019-06-10 17:58:28 +03:00
Michal Niewrzal
7a28decfca Always use difficulty larger than 8 for identites (#2160)
Set difficulty for libuplink and testplanet larger than 8, because we use last byte in the node id to store the version number, so the difficulty must be larger than 8.
2019-06-10 14:00:53 +02:00
Stefan Benten
74484fc57e
Enforce our Minimum Requirements for Node Operators and sanity check them (#2155) 2019-06-10 12:14:50 +02:00
Egon Elbre
749846b42b
Update golangci-lint (#2159) 2019-06-10 11:52:09 +03:00
ethanadams
8f2dca8437 Re-enabling and fixing repairer tests (#2099)
* Disabled discovery service by changiing from Stop() to Pause()

Paused to solve race condition.  If discovery is running, it may mark a node "up" after they've been manually marked "down" in this test.

* Extend to the repair timeout

Fixes intermittent test failures when repairs were taking more than 2 seconds.

* Re-enabled test. Disabled discovery service by changiing from Stop() to Pause()

* Changed back to Stop.

* Revert "Changed back to Stop."

This reverts commit 46d410e72dfae63e0c44915be42784cc9a7b5abf.

* re-enabling TestIdentifyInjuredSegments

* Changed Pause to Stop.  Commented on timeout change

* testing...

* temporarily skipping audit tests

* changing back to discover Stop for testing via jenkins

* Revert "changing back to discover Stop for testing via jenkins"

This reverts commit 6aa8558b11a0053c30e0c8b2dbf0d6c0cb34ee6c.

* Changing back to Stop().  Depends on PR 2137

* Revert "temporarily skipping audit tests"

This reverts commit 1940ed9b315d663a0eb6c95521780cbcb48cb121.

* Removed reference to Graveyard since its been removed
2019-06-10 09:06:21 +02:00
JT Olio
469d485f62 dbutil: reduce defaults so sum is < 100 (#2157) 2019-06-09 21:04:22 +02:00
Simon Guindon
3a7ebea76e
Moved monkit tracing after the limit nil check & fix node id formatting. (#2156)
* Moved monkit tracing after the limit nil check & fix node id formatting.

* Fixed to keep function traced even if limits is nil.
2019-06-07 18:34:16 -04:00
Natalie Villasana
16acb58546 rename verifier test files (#2154)
Despite the confusing git trail, this is two pure file renames, no other changes.
2019-06-07 16:02:36 -05:00
Cameron
23587bba0c
Storagenode vouchers table (#2121)
* add vouchers table with methods
2019-06-07 16:20:34 -04:00
Ivan Fraixedes
f5227abd36
uplink: enc.encryption-key flag is only available for setup command (#2090)
* uplink: Mark encryption key config field for setup
  Set the "setup" property to the `EncryptionConfig.EncrptionKey` for
  avoiding to save it in the configuration file.
  This field is only meant for using in the command line parameters which
  need to use a different encryption key than the one present in the key
  file or use it when there is not set any encryption key file path.
* cmd/uplink: Setup non-interactive accept enc key
  Change the uplink CLI setup command non-interactive to save the
  encryption key into a file when it's passed through the flag
  --enc.encryption-key
  Previous to this change it wasn't possible to create an key file despite
  of that the flag was provided, so it was useless on the setup command.
* cmd/uplink: Reuse logic to read pwd from terminal
  Reuse the logic which is already implemented in the pkg/cfgstruct for
  reading a password from the terminal on interactive mode, rather than
  duplicating it in the setup command.
* cmd/gateway: Use encryption key file flags
  The cmd/gateway was still using the `enc.key` configuration field which
  doesn't exist anymore and its setup command wasn't using the
  `enc.key-filepath` with combination of the `enc.encryption-key` for
  generating a file with the encryption key.
  This commit update the cmd/gateway appropriately and move to the uplink
  package the function used by cmd/uplink to save the encryption key for
  allowing to also be used by the cmd/gateway without duplicating the
  logic.
* cmd/storj-sim: Adapt gateway config cmd changes
  Adapt the cmd/storj-sim to correctly pass the parameters to the
  cmd/gateway setup and run command.
* scripts: Don't pass the --enc.encryption-key flag
  uplink configuration has changed to only support the
  `--enc.encryption-key` flag for setup commands and consequently the
  cmd/uplink and cmd/gateway don't accept this flag over other commands,
  hence the test for the uplink had to be updated for no passing the
  flag on the multiples calls that the test do to cmd/uplink.
* uplink: Remove func which aren't useful anymore
  Remove the function which allows to user or load an encryption key
  because it isn't needed anymore since the `--enc.encryption-key` flag is
  only available for the setup command.
  Consequently remove its usage from cmd/uplink and cmd/gateway, because
  such flag will always be empty because in case that's passed Cobra will
  return an error due to a "unknown flag".
2019-06-07 18:14:40 +02:00
Bill Thorp
2919aa2f79 Reputation / node selection doc (#2135) 2019-06-07 17:02:30 +02:00
Brandon Iglesias
29b3cec68c update to the reporting section to calculate bandwidth (#2151) 2019-06-07 10:54:58 -04:00
Ivan Fraixedes
44359e8bb3
pkg/kademlia: Fix bug using ref of var range loop (#2149)
Fix a bug in a range loop which used the reference of the variable assigned in the same loop to be appended in a slice, turning out that the slice will contain the same reference rather the a reference for each value of the ranged items.
2019-06-07 15:08:09 +02:00
Kaloyan Raev
8ddb1dee0d
Improve error analysis in audit service (#2125)
Co-authored-by: Natalie Ventura Villasana <nat@storj.io>
2019-06-07 15:38:41 +03:00
Ivan Fraixedes
cac4637670
satellite: Fix Peer.Run method doc comment (#2147) 2019-06-07 13:40:20 +02:00
Bogdan Artemenko
30784484c8
Fixed error notification on fail project member deletion. (#2148) 2019-06-07 13:56:07 +03:00
JT Olio
43d4f3daf5 discovery: remove graveyard (#2145) 2019-06-07 08:40:51 +03:00
Simon Guindon
a55df84bf7
Adding RS values to the info log and NodeID to tracing. (#2130) 2019-06-06 15:51:00 -04:00
Egon Elbre
03fece56de
Ensure Storage Nodes collect expired used serial numbers (#2143) 2019-06-06 22:06:31 +03:00
Jess G
bf3d168cf6
makes sure all uplink cli configs get passed to libuplink, add stripeSize (#2103)
* makes sure all uplink cli configs get passed to libuplink, add stripSize

* update comment

* update defaults for uplink config blocksize

* changes per CR, update uplink config defaults

* pass shareSize from uplink config

* move block size validation to kvmeta pkg

* fix tests

* shareSize default 1k, rm config option blocksize

* rm printing err to stdout
2019-06-06 11:55:10 -07:00
VitaliiShpital
f69f7b2d58 frontend bug fixes (#2127)
* frontend bug fixes

* removed redundant styles

* removed redundant media styles
2019-06-06 12:38:32 -04:00