Commit Graph

2950 Commits

Author SHA1 Message Date
Simon Guindon
476fbf919a
storagenode/storagenodedb: refactor SQLite3 database connection initialization. (#2732)
* Rebasing changes against master.

* Added back withTx().

* Fix using new error type.

* Moving back database initialization back into the struct.

* Fix failing migration tests.

* Fix linting errors.

* Renamed database object names to be consistent.

* Fixing linting error in imports.

* Rebasing changes against master.

* Added back withTx().

* Fix using new error type.

* Moving back database initialization back into the struct.

* Fix failing migration tests.

* Fix linting errors.

* Renamed database object names to be consistent.

* Fixing linting error in imports.

* Adding missing change from merge.

* Fix error name.
2019-08-21 10:32:25 -04:00
Vitalii Shpital
87ef5e3398
web/satellite: buckets api and store refactored (#2830) 2019-08-21 17:07:49 +03:00
Vitalii Shpital
1e099839dd
web/satellite: api keys component updated (#2819) 2019-08-21 16:47:07 +03:00
Nikolay Yurchenko
0f5d533781
web/satellite: Notification rework (#2781) 2019-08-21 16:21:23 +03:00
Jennifer Li Johnson
9df2ec6a37
docs/design: Kademlia Removal Design Proposal (#2686) 2019-08-20 15:48:35 -04:00
Yehor Butko
513563eff3
TabNavigation component fixed, unit tests added (#2834) 2019-08-20 18:23:56 +03:00
Egon Elbre
9ec0ceddf3
pkg/revocation: ensure we close revocation databases (#2825) 2019-08-20 18:04:17 +03:00
Egon Elbre
95080643b1
satellite/repair: fix data race (#2833) 2019-08-20 17:46:39 +03:00
Ivan Fraixedes
87f3b6c708 satellite/audit: Improve comments in verifier (#2829)
Improve some source code comments in the verifier.
2019-08-20 10:23:14 -04:00
Bryan White
25f0b13980 pkg/peertls: extension handling refactor (#2831) 2019-08-20 17:15:43 +03:00
Bogdan Artemenko
77f1555cf4
web/satellite: Moved endpoint url to config file. (#2832) 2019-08-20 16:31:01 +03:00
Egon Elbre
56383de298
satellite/metainfo: use status.Error and fix error codes (#2827) 2019-08-20 16:16:51 +03:00
Bogdan Artemenko
2e3ff4587a
web/satellite: ProjectMembers store and graphQL queries refactoring (#2820) 2019-08-20 15:57:43 +03:00
littleskunk
6615350188 initialize used space table with sum over pieceinfo (#2818) 2019-08-20 08:13:18 -04:00
Egon Elbre
3c41794429
pkg/kademlia: clearer error message (#2824) 2019-08-20 12:54:33 +03:00
Egon Elbre
95c88b803c
satellite/nodestats: use proper error codes (#2826) 2019-08-20 12:26:55 +03:00
Egon Elbre
2e1fc4bc52
docs/design: describe process in more detail (#2817) 2019-08-20 12:17:09 +03:00
Isaac Hess
25154720bd
lib/uplink: remove redis and bolt dependencies (#2812)
* identity: remove redis and bolt dependencies

* identity: move revDB creation to main files
2019-08-19 16:10:38 -06:00
Bryan White
8832a393e1
formatting fix (#2823) 2019-08-19 23:51:06 +02:00
Bryan White
863675605a scripts/test-sim-backwards: usability improvements (#2816)
* + add git worktree cleanup
+ specify commit in git worktree add

* fail backwards compatibility test if no postgres

* fix typo

* rm dir after worktree rm
2019-08-19 12:13:38 -07:00
Maximillian von Briesen
d83a965139
storagenode/piecestore: Add retain service on storagenode (#2785)
Add retain service on storagenode. This service runs retain jobs that have been queued by the storagenodes. Rather than running retain jobs during the grpc Retain() call, the grpc call queues a retain job to the retain service and returns immediately afterwards, removing a significant bottleneck in garbage collection.
2019-08-19 14:52:47 -04:00
Yehor Butko
bf681f32af
web/satellite/credits: api/store/component and tests refactored (#2811) 2019-08-19 21:12:23 +03:00
ethanadams
1a69ec8318
satellite/orders: document protocol and fix typos (#2813)
* Addressing comments from PR 2762
* Rebuild of orders.pb.go after comments added to proto file
* run update-satellite-config-lock for spelling fix.
2019-08-19 09:36:11 -04:00
Bogdan Artemenko
b77f582b29
web/satellite: Project members web client refactoring. (#2783) 2019-08-19 14:20:38 +03:00
Bryan White
6400d63a6c
satellite/satellitedb: Add piece count column to nodes table (#2795) 2019-08-19 12:58:13 +02:00
Vitalii Shpital
af41039eae
web/satellite: projects api key refactored (#2807) 2019-08-19 13:00:38 +03:00
Matt Robinson
ac58589615 Add aarch64 support and use go 1.12.9 (#2814) 2019-08-19 12:40:16 +03:00
Jess G
309ad2cb29
jenkins: Backwards Compatibility use Postgres (#2810)
* add postgres support

* fix indentation

* create db

* rm prod configs file
2019-08-18 07:51:33 -07:00
Jeff Wendling
057d30152c
uplink/storage/segments: seed download permuatation with timestamp (#2809) 2019-08-16 11:14:02 -06:00
Ivan Fraixedes
546d099cf5
storagenode/orders: An invalid one don't have to stop all (#2804)
When an unsent order stored in the DB cannot be unmarshalled due to an
unmarshal error the rest unsent orders must be processed as usual.

This changes will avoid that a Storage Node with unsent orders with
invalid protobuf serialized values get blocked without sending orders
until those invalid ones get removed from the DB.
2019-08-16 17:33:51 +02:00
Ivan Fraixedes
e47b8ed131
storagenode: No FATAL error when unsent orders aren't found (#2801)
* pkg/process: Fatal show complete error information
  Change the general process execution function to not using the sugared
  logger for outputting the full error information.
  Delete some unreachable code because Zap logger Fatal method calls exit
  1 internally.
* storagenode/storagenodedb: Add info to error
  Add more information to an error returned due to some data
  inconsistency.
* storagenode/orders: Don't use sugared logger
  Don't use sugar logger and provide better contextualized error messages
  in settle method.
* storagenode/orders: Add some log fields to error msgs
  Add some relevant log fields to some logged errors of the sender settle
  method.
* satellite/orders: Remove always nil error from debug
  Remove an error which as logged in debug level which was always nil and
  makes the logic that used this variable clear.
* storagenode/orders: Don't return error Archiving unsent
  Don't stop the process which archive unsent orders if some of them
  aren't found the DB because it cause the Storage Node to stop with a
  fatal error.
2019-08-16 16:53:22 +02:00
Vitalii Shpital
b1abbe5ce3
web/satellite: add team member placeholder changed (#2794) 2019-08-16 14:05:20 +03:00
Nikolay Yurchenko
5fe05df774
web/satellite: Buckets page rework (#2763) 2019-08-16 13:13:23 +03:00
ethanadams
8df683a265
Update satellite settlement endpoint to batch order processing into transactions. (#2762)
Update satellite settlement endpoint to batch order processing into transactions
2019-08-15 15:05:43 -04:00
Vitalii Shpital
989051328f
web/satellite: api key header updated, redundant components removed (#2770) 2019-08-15 21:21:46 +03:00
Cameron
497f10d7b1
add method CleanArchive to delete archived orders (#2796) 2019-08-15 12:56:33 -04:00
Maximillian von Briesen
189b268892
uplink/piecestore: Change where ignore cancel happens for closing downloads (#2786) 2019-08-15 10:32:05 -04:00
Alexander Leitner
9c4af157f5
docs/design: windows installer (#2780)
* Add design doc for windows installer
2019-08-15 08:10:55 -04:00
Michal Niewrzal
83461a7c01 satellite/metainfo: fix storing enc key/nonce (#2759) 2019-08-15 14:45:49 +03:00
Jeff Wendling
e22c0bff94 lib/uplinkc: fix flaky download test (#2791)
The download driver code loops through the downloaded_data buffer
but doesn't ensure that it always passes a valid pointer to the
Go side. In particular, if the malloc'd memory ends against an
unmapped page, and the test passes a pointer one past the end
of the memory region, and since the Go side always dereferences
the pointer when creating a slice, it will attempt to read
unmapped memory, causing a segfault.

This bug doesn't always present. Indeed, it depends on the details
of your system's memory allocator. I validated that this could be
a cause of observed crashes on OS X by using mmap and mprotect
to do the allocations ensuring that the page directly after the
memory we use was unmapped/protected. The crash happened exactly
as seen, and was fixed by changing this condition in the while
loop.

Change-Id: I685dac07ff9b904097375dbf850f387450858753
2019-08-15 11:32:29 +03:00
Jeff Wendling
2dab0ab466 pkg/process: only propagate missing keys to flags (#2784)
this avoids a problem where setting on a flag isn't sufficient
to express complex data structures like []string.

Change-Id: I06f13656996d658b4c7a957451cb253728a67eda
2019-08-14 14:41:41 -06:00
Yingrong Zhao
7db6851722
satellite/rewards: update current reward to be finished once redemption cap has reached (#2745)
* update offer once redemption cap has reached

* use transaction to get offer info before insert

* update offer status when redeemable capacity has reached

* fix format

* use pgutil to check constraint error

* change error message
2019-08-14 15:53:48 -04:00
Bryan White
1915b59af3 satellite/repair: monkit improvements (#2773) 2019-08-14 15:40:26 -04:00
Brandon Iglesias
5c61c79ab7 Start building binary for the link sharing service (#2790) 2019-08-14 15:04:56 -04:00
Egon Elbre
b3a3d8df06
go.mod: update grpc-go (#2778) 2019-08-14 21:43:21 +03:00
Yehor Butko
012775f874
web/satellite user api simplification (#2787) 2019-08-14 21:11:18 +03:00
Yaroslav Vorobiov
2c769fe9d9
satellite/overlaycache: add missing audit and uptime success count (#2788) 2019-08-14 20:53:39 +03:00
Brandon Iglesias
d28c8aad8f
design doc for design doc process (#2782) 2019-08-14 12:16:18 -04:00
Jess G
e1d8503f50
jenkins: add backwards compatibility test (#2774)
* add jenkins runs backwards compat test, fix sa config issue

* try to use current branch

* add jenkins stage condition of branch name

* try when with branch env var

* only run test on master branch

* fix != to ==

* add comment, fix indentation

* run for all branches

* make comment more specific
2019-08-14 08:34:12 -07:00
Yingrong Zhao
786828ea0a
satellite/console: bypass activation token logic for open source partner referral link (#2736)
* when there's partner id, we will not require an activation token for creating a new account

* create new token if user has a partner id on creation

* validate partner id first

* fix format

* remove unnecessary code

* display error message instead of reroute

* add more test

* add comments

* add comment
2019-08-14 11:27:22 -04:00