Commit Graph

3007 Commits

Author SHA1 Message Date
Egon Elbre
00b2e1a7d7 all: enable staticcheck (#2849)
* by having megacheck in disable it also disabled staticcheck

* fix closing body

* keep interfacer disabled

* hide bodies

* don't use deprecated func

* fix dead code

* fix potential overrun

* keep stylecheck disabled

* don't pass nil as context

* fix infinite recursion

* remove extraneous return

* fix data race

* use correct func

* ignore unused var

* remove unused consts
2019-08-22 13:40:15 +02:00
Yehor Butko
7c6fdd09d2
web/satellite: router refactoring (#2846) 2019-08-22 13:44:08 +03:00
Egon Elbre
1b9d163c95
satellite/marketingweb: WriteHeader should only be called once (#2850) 2019-08-22 13:08:49 +03:00
Kaloyan Raev
cff6dc2b2a
Rename design-doc-process.md to README.md (#2848)
This way the design doc process is directly visible when browsing the doc/design folder in the GitHub Web UI. As the number of docs grows it would be harder to find the doc with the design process.
2019-08-22 11:17:18 +03:00
Jeff Wendling
14e36e4d60
storagenode/nodestats: fix issue on 32 bit platforms (#2841)
* storagenode/nodestats: fix issue on 32 bit platforms

time.Duration is an int64, so casting it down to an int
can cause it to become negative, causing a panic.

Change-Id: I33da7c29ddd59be60d8deec944a25f4a025902c7

* storagenode/nodestats: fix lint issue in test

Change-Id: Ie68598d724d2cae0dc959d4877098a08f4eb9af7
2019-08-21 18:57:44 -06:00
Egon Elbre
2d69d47655
all: fix Error.New formatting (#2840) 2019-08-21 19:30:29 +03:00
Natalie Villasana
243cedb628
satellite/audit: implement reservoir struct and RemoteSegment observer method (#2744) 2019-08-21 11:49:27 -04:00
Simon Guindon
476fbf919a
storagenode/storagenodedb: refactor SQLite3 database connection initialization. (#2732)
* Rebasing changes against master.

* Added back withTx().

* Fix using new error type.

* Moving back database initialization back into the struct.

* Fix failing migration tests.

* Fix linting errors.

* Renamed database object names to be consistent.

* Fixing linting error in imports.

* Rebasing changes against master.

* Added back withTx().

* Fix using new error type.

* Moving back database initialization back into the struct.

* Fix failing migration tests.

* Fix linting errors.

* Renamed database object names to be consistent.

* Fixing linting error in imports.

* Adding missing change from merge.

* Fix error name.
2019-08-21 10:32:25 -04:00
Vitalii Shpital
87ef5e3398
web/satellite: buckets api and store refactored (#2830) 2019-08-21 17:07:49 +03:00
Vitalii Shpital
1e099839dd
web/satellite: api keys component updated (#2819) 2019-08-21 16:47:07 +03:00
Nikolay Yurchenko
0f5d533781
web/satellite: Notification rework (#2781) 2019-08-21 16:21:23 +03:00
Jennifer Li Johnson
9df2ec6a37
docs/design: Kademlia Removal Design Proposal (#2686) 2019-08-20 15:48:35 -04:00
Yehor Butko
513563eff3
TabNavigation component fixed, unit tests added (#2834) 2019-08-20 18:23:56 +03:00
Egon Elbre
9ec0ceddf3
pkg/revocation: ensure we close revocation databases (#2825) 2019-08-20 18:04:17 +03:00
Egon Elbre
95080643b1
satellite/repair: fix data race (#2833) 2019-08-20 17:46:39 +03:00
Ivan Fraixedes
87f3b6c708 satellite/audit: Improve comments in verifier (#2829)
Improve some source code comments in the verifier.
2019-08-20 10:23:14 -04:00
Bryan White
25f0b13980 pkg/peertls: extension handling refactor (#2831) 2019-08-20 17:15:43 +03:00
Bogdan Artemenko
77f1555cf4
web/satellite: Moved endpoint url to config file. (#2832) 2019-08-20 16:31:01 +03:00
Egon Elbre
56383de298
satellite/metainfo: use status.Error and fix error codes (#2827) 2019-08-20 16:16:51 +03:00
Bogdan Artemenko
2e3ff4587a
web/satellite: ProjectMembers store and graphQL queries refactoring (#2820) 2019-08-20 15:57:43 +03:00
littleskunk
6615350188 initialize used space table with sum over pieceinfo (#2818) 2019-08-20 08:13:18 -04:00
Egon Elbre
3c41794429
pkg/kademlia: clearer error message (#2824) 2019-08-20 12:54:33 +03:00
Egon Elbre
95c88b803c
satellite/nodestats: use proper error codes (#2826) 2019-08-20 12:26:55 +03:00
Egon Elbre
2e1fc4bc52
docs/design: describe process in more detail (#2817) 2019-08-20 12:17:09 +03:00
Isaac Hess
25154720bd
lib/uplink: remove redis and bolt dependencies (#2812)
* identity: remove redis and bolt dependencies

* identity: move revDB creation to main files
2019-08-19 16:10:38 -06:00
Bryan White
8832a393e1
formatting fix (#2823) 2019-08-19 23:51:06 +02:00
Bryan White
863675605a scripts/test-sim-backwards: usability improvements (#2816)
* + add git worktree cleanup
+ specify commit in git worktree add

* fail backwards compatibility test if no postgres

* fix typo

* rm dir after worktree rm
2019-08-19 12:13:38 -07:00
Maximillian von Briesen
d83a965139
storagenode/piecestore: Add retain service on storagenode (#2785)
Add retain service on storagenode. This service runs retain jobs that have been queued by the storagenodes. Rather than running retain jobs during the grpc Retain() call, the grpc call queues a retain job to the retain service and returns immediately afterwards, removing a significant bottleneck in garbage collection.
2019-08-19 14:52:47 -04:00
Yehor Butko
bf681f32af
web/satellite/credits: api/store/component and tests refactored (#2811) 2019-08-19 21:12:23 +03:00
ethanadams
1a69ec8318
satellite/orders: document protocol and fix typos (#2813)
* Addressing comments from PR 2762
* Rebuild of orders.pb.go after comments added to proto file
* run update-satellite-config-lock for spelling fix.
2019-08-19 09:36:11 -04:00
Bogdan Artemenko
b77f582b29
web/satellite: Project members web client refactoring. (#2783) 2019-08-19 14:20:38 +03:00
Bryan White
6400d63a6c
satellite/satellitedb: Add piece count column to nodes table (#2795) 2019-08-19 12:58:13 +02:00
Vitalii Shpital
af41039eae
web/satellite: projects api key refactored (#2807) 2019-08-19 13:00:38 +03:00
Matt Robinson
ac58589615 Add aarch64 support and use go 1.12.9 (#2814) 2019-08-19 12:40:16 +03:00
Jess G
309ad2cb29
jenkins: Backwards Compatibility use Postgres (#2810)
* add postgres support

* fix indentation

* create db

* rm prod configs file
2019-08-18 07:51:33 -07:00
Jeff Wendling
057d30152c
uplink/storage/segments: seed download permuatation with timestamp (#2809) 2019-08-16 11:14:02 -06:00
Ivan Fraixedes
546d099cf5
storagenode/orders: An invalid one don't have to stop all (#2804)
When an unsent order stored in the DB cannot be unmarshalled due to an
unmarshal error the rest unsent orders must be processed as usual.

This changes will avoid that a Storage Node with unsent orders with
invalid protobuf serialized values get blocked without sending orders
until those invalid ones get removed from the DB.
2019-08-16 17:33:51 +02:00
Ivan Fraixedes
e47b8ed131
storagenode: No FATAL error when unsent orders aren't found (#2801)
* pkg/process: Fatal show complete error information
  Change the general process execution function to not using the sugared
  logger for outputting the full error information.
  Delete some unreachable code because Zap logger Fatal method calls exit
  1 internally.
* storagenode/storagenodedb: Add info to error
  Add more information to an error returned due to some data
  inconsistency.
* storagenode/orders: Don't use sugared logger
  Don't use sugar logger and provide better contextualized error messages
  in settle method.
* storagenode/orders: Add some log fields to error msgs
  Add some relevant log fields to some logged errors of the sender settle
  method.
* satellite/orders: Remove always nil error from debug
  Remove an error which as logged in debug level which was always nil and
  makes the logic that used this variable clear.
* storagenode/orders: Don't return error Archiving unsent
  Don't stop the process which archive unsent orders if some of them
  aren't found the DB because it cause the Storage Node to stop with a
  fatal error.
2019-08-16 16:53:22 +02:00
Vitalii Shpital
b1abbe5ce3
web/satellite: add team member placeholder changed (#2794) 2019-08-16 14:05:20 +03:00
Nikolay Yurchenko
5fe05df774
web/satellite: Buckets page rework (#2763) 2019-08-16 13:13:23 +03:00
ethanadams
8df683a265
Update satellite settlement endpoint to batch order processing into transactions. (#2762)
Update satellite settlement endpoint to batch order processing into transactions
2019-08-15 15:05:43 -04:00
Vitalii Shpital
989051328f
web/satellite: api key header updated, redundant components removed (#2770) 2019-08-15 21:21:46 +03:00
Cameron
497f10d7b1
add method CleanArchive to delete archived orders (#2796) 2019-08-15 12:56:33 -04:00
Maximillian von Briesen
189b268892
uplink/piecestore: Change where ignore cancel happens for closing downloads (#2786) 2019-08-15 10:32:05 -04:00
Alexander Leitner
9c4af157f5
docs/design: windows installer (#2780)
* Add design doc for windows installer
2019-08-15 08:10:55 -04:00
Michal Niewrzal
83461a7c01 satellite/metainfo: fix storing enc key/nonce (#2759) 2019-08-15 14:45:49 +03:00
Jeff Wendling
e22c0bff94 lib/uplinkc: fix flaky download test (#2791)
The download driver code loops through the downloaded_data buffer
but doesn't ensure that it always passes a valid pointer to the
Go side. In particular, if the malloc'd memory ends against an
unmapped page, and the test passes a pointer one past the end
of the memory region, and since the Go side always dereferences
the pointer when creating a slice, it will attempt to read
unmapped memory, causing a segfault.

This bug doesn't always present. Indeed, it depends on the details
of your system's memory allocator. I validated that this could be
a cause of observed crashes on OS X by using mmap and mprotect
to do the allocations ensuring that the page directly after the
memory we use was unmapped/protected. The crash happened exactly
as seen, and was fixed by changing this condition in the while
loop.

Change-Id: I685dac07ff9b904097375dbf850f387450858753
2019-08-15 11:32:29 +03:00
Jeff Wendling
2dab0ab466 pkg/process: only propagate missing keys to flags (#2784)
this avoids a problem where setting on a flag isn't sufficient
to express complex data structures like []string.

Change-Id: I06f13656996d658b4c7a957451cb253728a67eda
2019-08-14 14:41:41 -06:00
Yingrong Zhao
7db6851722
satellite/rewards: update current reward to be finished once redemption cap has reached (#2745)
* update offer once redemption cap has reached

* use transaction to get offer info before insert

* update offer status when redeemable capacity has reached

* fix format

* use pgutil to check constraint error

* change error message
2019-08-14 15:53:48 -04:00
Bryan White
1915b59af3 satellite/repair: monkit improvements (#2773) 2019-08-14 15:40:26 -04:00