Commit Graph

2962 Commits

Author SHA1 Message Date
Nikolay Yurchenko
f83837bb03 web/satellite: billing banners (#3649) 2019-11-27 16:57:56 +02:00
Ivan Fraixedes
0bf7ac5b20 cmd/segment-reaper: Use bitmask type in observer (#3661)
bitmask type was created for making easy to understand the observer bit
logic.
This commit makes the changes in observer for using the bitmask type.
2019-11-27 13:11:07 +01:00
Ivan Fraixedes
7288d01781
cmd/segment-reaper: several refactorings (#3637)
* Move the observer implementation and the type definitions related with
  it and helper functions to its own file.
* Cluster struct type is used as a key for a ObjectsMap type.
  Observer struct type has a field of ObjectsMap.
  Cluster has a field for project ID.
  Observer processSegment method uses its ObjectMap field for tracking
  objects.

  However Observer processSegment clears the map once the projectID
  diverges from the one kept in the Observer lastProjectID field, meaning
  that it isn't needed to keep the projectID as part of the ObjectMap key.

  For this reason, ObjectMap can use as a key just only the bucket name
  and Cluster struct isn't needed.

  Because of such change, the ObjectMap type has been renamed to a more
  descriptive name.
* Make the types defined for this specific package not being exported.
* Create a constructor function for observer to encapsulate the map
  allocation.
* Don't throw away the entirely buckets objects map when an empty one
  is used to reuse part of the allocations.

  Encapsulate the clearing up logic into a method.
* Make the analyzeProject function to be a method of observer.
2019-11-27 10:28:43 +01:00
Isaac Hess
a6235d3962 storage/filestore: Monitor when we open files in trash
Change-Id: I817bf8349c2e1ba55e1490f06162af1099bebdb0
2019-11-26 14:38:49 -07:00
Yehor Butko
756b9b9e2b
satellite/payments: coupons and coupon usage (#3648) 2019-11-26 19:58:51 +02:00
Ivan Fraixedes
bf97ef06fc
storagenode: Add new endpoint to receive satellite requests for… (#3590)
* pkg/pg: Add new service function storage node

  Add a new service function to the storage node piece store for deleting
  pieces when satellites request them.

* storagenode/piecestore: Add endpoint to delete piece

  Add a new endpoint to receive from trusted satellites to delete a piece.

* private/testplanet: Fix storagenode mock

  Add to the storagenode mock the new endpoint method.

* proto.lock: Update it with the last protbuff changes

* storagenode/piecestore: Reuse test piece upload

  Extract the repeated logic from several tests functions for uploading a
  test piece to a test helper function.

* uplink/piecestore: Implement client side method

  Implement the client side method of the new piecestore RPC function.

* storagenode/piecestore: Add test DeletePiece endpoint

  Implement a test for the DeletePiece new endpoint method.
2019-11-26 18:47:19 +01:00
Yingrong Zhao
66f1a1680f
add completion receipt to exit-status cli command on storage node (#3650) 2019-11-26 12:32:26 -05:00
Ethan Adams
bba92911cc
fix calcuation of durability ration (#3656) 2019-11-26 12:04:48 -05:00
Nikolay Yurchenko
cf13cf6e00
web/satellite: referral links (#3655) 2019-11-26 18:54:42 +02:00
Jess G
854e5507ab
crdb uses namespaced db for each test (#3646)
* crdb uses namespaced db for each test

* add test for me test

* fix lint and tests

* updates per cr comments

* rm all replaceall
2019-11-26 08:39:57 -08:00
Isaac Hess
56f8fd2dd7
storagenode/pieces: Add EmptyTrash functionality (#3640)
* storagenode/pieces: Add EmptyTrash functionality

* storagenode/pieces: Fix err

* storagenode/pieces: Fix lint
2019-11-26 09:25:21 -07:00
Vitalii Shpital
038ac58600
web/storagenode: minimal allowed version view implemented (#3583) 2019-11-26 18:08:24 +02:00
Isaac Hess
2502c26154 Jenkinsfile: remove whitespace (#3244) 2019-11-26 16:47:46 +01:00
Cameron
7abad3c6bb
refactor sql to be compatible with pq and cockroach (#3647) 2019-11-26 09:26:22 -05:00
Vitalii Shpital
ce49c10418
web/storagenode: info containers of remaining bandwidth/diskSpace replaced (#3653) 2019-11-26 16:04:22 +02:00
Vitalii Shpital
b763c5319e
web/storagenode: disk space chart's tooltip reworked (#3652) 2019-11-26 15:54:19 +02:00
Michal Niewrzal
1e33995a89
cmd/segment-reaper: skeleton command for zombie deletion (#3599) 2019-11-26 05:01:31 -08:00
Egon Elbre
36fead0093 satellite/metainfo: add UserAgent support to endpoints (#3548) 2019-11-26 03:12:37 -08:00
Vitalii Shpital
59385aff66
web/storagenode: ingress chart implemented (#3618) 2019-11-26 11:19:57 +02:00
Yingrong Zhao
79a4fff6c7
satellite/referrals: set up referrals service and http endpoints (#3566) 2019-11-25 16:36:36 -05:00
Jeff Wendling
17b057b33e satellite/audit: monitor worker function
Change-Id: I94d1161deffe4ea9782abee1afbb5735f18aab44
2019-11-25 17:58:13 +00:00
JT Olio
01895d8bdd lib/uplink: explain safe versions (#3644)
Change-Id: Idf7391474ab10b0dab6ce97c773cabee59d5b664
2019-11-25 16:55:54 +01:00
Yaroslav Vorobiov
8a002e8c8e
satellite/accounting: separate project limit from project entity (#3632) 2019-11-25 16:18:04 +02:00
Nikolay Yurchenko
8234e24d13
web/satellite: token payments logic (#3581) 2019-11-25 14:59:41 +02:00
Michal Niewrzal
1aa2bc0a83
satellite/metainfo: reduce pointerDB access for CommitObject (#3589) 2019-11-24 13:14:51 -08:00
Ivan Fraixedes
48a557eb2b satellite/metainfo: Fix misspelling in comment (#3636) 2019-11-24 13:04:25 -08:00
JT Olio
031ba86de5
argon2: choose a steady parallelism value (#3630)
* argon2: choose a steady parallelism value

Change-Id: I6006da7d7980cda88f5f08ee759612df23a8132d

* whoops, not cruft

Change-Id: Ied9039f9a9be1d0f6ff3c7d5c4839a83fc7b4b1f

* fix broken test file

Change-Id: I07288cd6cef32ba387f2f003febff5c297e50997

* fix linting error

Change-Id: Icdbda8b709cc100a86f3859303c40edb8dff1e0f
2019-11-22 14:00:04 -07:00
Jess G
388f33b84d
satellitedb: add support to testplanet for cockroachdb (#3634)
* update migration steps, add crdb support to testplanet

* add crdb support

* have jenkins run a bares bones crdb compat test

* skip crdb tests

* skip crdb tests

* fix root_piece_id column

* write crdb store to tmp dir

* escape
2019-11-22 11:59:46 -08:00
Nikolai Siedov
e5934cc92a
satellite/console/auth: return in error handle added (#3639) 2019-11-22 21:44:57 +02:00
Matt Robinson
9af97d366a Make sed a little more cross platformable (#3629) 2019-11-22 11:17:02 -07:00
Vitalii Shpital
68cd6f4a67
web: ms edge support bug fixed (#3638) 2019-11-22 19:22:37 +02:00
Nikolai Siedov
ded7f6e2eb
web/satellite: registration/welcome message fixed, usage-report url fixed, storj-sim fixed (#3622) 2019-11-22 19:03:15 +02:00
Vitalii Shpital
37c379f7da
web/satellite: fonts changed to Inter (#3620) 2019-11-22 18:32:43 +02:00
Kaloyan Raev
bb16c07dc2
storagenode/updater: read identity location from storagenode's config.yaml (#3607) 2019-11-22 15:57:37 +02:00
Ivan Fraixedes
77d4add101
cmd/segment-reaper: Implement bitmask type (#3626)
Define a bitmask type for implementing its logic to detect if an index
is set or not.
2019-11-22 11:53:48 +01:00
littleskunk
8842b0c252 storagenode/gracefulexit: improve logging (#3633) 2019-11-21 21:10:02 -05:00
Yingrong Zhao
63e51df9a6
private/testplanet: add a mock referral manager server into testplanet (#3631) 2019-11-21 17:34:49 -05:00
Maximillian von Briesen
1339252cbe
satellite/gracefulexit: refactor concurrency (#3624)
Update PendingMap structure to also handle concurrency control between the sending and receiving sides of the graceful exit endpoint.
2019-11-21 17:03:16 -05:00
Natalie Villasana
b7a8ffcdff pkg/pb/referralmanager: update to add satellite ID to Get Tokens request (#3625) 2019-11-21 16:13:48 -05:00
Michal Niewrzal
d96df2691a satellite/metainfo: improve Loop comments (#3595) 2019-11-21 15:24:17 -05:00
Rafael Antonio Ribeiro Gomes
2739771761
storagenode: add bandwidth metrics (#3623)
* storagenode: add bandwidth metrics

* remove unecessary metric
2019-11-21 16:51:40 -03:00
Matt Robinson
976881f72b
satellite/console: Add security headers (#3615)
* satellite/console: Add X-Frame-Options and Referrer-Policy security headers

* Update to use CSP instead of XFO and include tardigrade.io

* Make FrameAncestors a config option

* Update satellite-config lock

* Make help text for FrameAncestors better
2019-11-21 11:15:22 -05:00
Yaroslav Vorobiov
87c7a2ff42
satellite/payments: token deposit accept cents (#3628) 2019-11-21 16:25:37 +02:00
Yaroslav Vorobiov
c72c443564
satellite/payments: add cents values to transaction info (#3614) 2019-11-21 15:23:16 +02:00
Nikolay Yurchenko
2030d67650
web/satellite: charges summary fix (#3619) 2019-11-21 14:50:03 +02:00
Nikolai Siedov
6a4389d3e1 satellite/console: apiKeys case-insensitive search added (#3621) 2019-11-21 00:56:55 +02:00
Yingrong Zhao
b995406ff9
satellite/satellitedb: separate uuid creation from db layer (#3600) 2019-11-20 14:16:27 -05:00
JT Olio
40012e5790 satellite/metainfo: continue instead of return (fixing my bad advice)
Change-Id: I57ad3de8e3f705429bad98ce976879c4d5e905c9
2019-11-20 09:37:48 -07:00
Ivan Fraixedes
c2e605e81e satellite/metainfo: Don't return error in loop when path has less than 4 parts (#3616)
* satellite/metainfo: Rollback path parts check in loop

We have to rollback the changes applied in checking the rawPath parts
from 4 to 3 because the production prointerDB is still storing buckets.

* satellite/metainfo: Don't return path parts less 4

Don't return an error in the metainfo loop iterator when a path doesn't
have 4 parts because it belongs to bucket metadata, not an actual
object.
2019-11-20 09:29:52 -07:00
Isaac Hess
6aeddf2f53
storagenode/pieces: Add Trash and RestoreTrash to piecestore (#3575)
* storagenode/pieces: Add Trash and RestoreTrash to piecestore

* Add index for expiration trash
2019-11-20 09:28:49 -07:00