Commit Graph

3106 Commits

Author SHA1 Message Date
littleskunk
9d1faeee58 storagenode/garbagecollection: increase MaxTimeSkew to be higher than satellite MaxCommitInterval
Change-Id: I86f8d0b44bea3aa005ff26d52588611c59df5e9a
2019-12-09 16:03:55 +00:00
Egon Elbre
56a3b62bef satellite/satellitedb: ensure migration tests run (#3706)
satellitedb migration tests ran against multiple base versions, however after the merging all the steps the base versions didn't exists anymore - which meant none of the migration tests were actually running.
2019-12-09 09:26:58 -06:00
Vitalii Shpital
6ef359aba2
web/storagenode: blurred checks hint added for all satellites state (#3709) 2019-12-09 16:10:33 +02:00
Vitalii Shpital
fe5cfeb61b
web/storagenode: disk space chart's tooltip size fixed (#3684) 2019-12-09 15:59:20 +02:00
Nikolai Siedov
c6776ae6bb
error messages fixed (#3712) 2019-12-09 15:20:44 +02:00
Vitalii Shpital
fa5288c254
satellitedb: bucket search fixed (#3594) 2019-12-09 14:46:30 +02:00
Nikolay Yurchenko
ea92c68600
web/satelllite: default cc delete disabling (#3695) 2019-12-09 14:25:38 +02:00
Michal Niewrzal
ffd570eb04 uplink/metainfo: remove additional GetObject from download
This change removes one additional metainfo.GetObject call during
download.

Change-Id: I86b4e5165f299069e3e18944fe79c697c6a514d3
2019-12-09 11:48:48 +00:00
Michal Niewrzal
27462f68e9
cmd/segment-reaper: add detecting and printing zombie segments (#3598) 2019-12-09 03:30:52 -08:00
Ivan Fraixedes
8d49a99ad8 uplink/metainfo: Fix doc comments
Fix some documentation comments in a few methods and apply our code
style conventions to all of them in this source file of the package.

Change-Id: I78186cb1e7d45020d6cf7bbea8a39d146ada1cb6
2019-12-09 08:47:52 +00:00
Michal Niewrzal
b17d40ffd4
segment-reaper: delete command logic (#3660) 2019-12-09 00:25:39 -08:00
Ivan Fraixedes
c400471bbc
satellite/metainfo: Fix some docs comments
Fix a documentation comment for one method and apply our code
conventions to some that I stumbled.

Change-Id: I3baf5d004a128dcd561c3e27c080aab345c64461
2019-12-06 19:14:35 +01:00
Jeff Wendling
1df7b360d7 satellite/metainfo: Use cockroachdb client for metainfo db
Change-Id: I3cf7a00de4f654eacaffbb494f4841c64a2d9ce6
2019-12-05 10:33:54 -07:00
Jeff Wendling
f15192ea40 storage/cockroachkv: initial client implementation
Change-Id: I72ae558739b2ca532f31cb64f480b43437b6b309
2019-12-05 16:11:23 +00:00
Jennifer Johnson
8e9532dbd9 satellitedb/repairqueue: use errs.New() instead of fmt.Errorf() to retain stack trace
Change-Id: I47f1985aaeace556e3e0f7a20d2718410936db17
2019-12-05 15:52:25 +00:00
paul cannon
378b863b2b private,satellite: unite all the "temp db schema" things
first, so that they all work the same way, because it's getting
complicated, and second, so that we can do the appropriate thing
instead of CREATE SCHEMA for cockroachdb.

Change-Id: I27fbaeeb6223a3e06d97bcf692a2d014b31465f7
2019-12-05 15:36:59 +00:00
Vitalii Shpital
97fa000ca9
web/satellite: usage data converting implemented (#3673) 2019-12-05 13:51:11 +02:00
Cameron
28a66747c9 satellite/satellitedb: cockroachDB compatible transaction in offersDB.Create (#3686)
* use crdb wrapper for create/update offer transaction, ensure times returned are UTC

* round to nearest microsecond to compare times
2019-12-04 21:54:43 -06:00
Jennifer Johnson
7c5f777a4f satellitedb/repairqueue: runs a different implementation of the query within Select() for postgres vs cockroach
Change-Id: Ie34dbdb9d870d7d9f8f269702b6b3bad0c55b98e
2019-12-04 21:32:02 +00:00
Yingrong Zhao
7af42e3c10
satellite/metainfo, satellite/repair, uplink/eestream: add metric for download failed due to not enough pieces available (#3665) 2019-12-04 16:24:36 -05:00
paul cannon
850c358087 private/dbutil/pgutil: make QuerySchema work on crdb
Adjust the pg_constraint query so that it works without a LATERAL JOIN,
since CockroachDB doesn't like that.

This isn't hooked up to a cockroach test yet, but that's coming.

Change-Id: I0df6b477d958996b673fc121eaa1f7c35e5cc504
2019-12-04 18:55:26 +00:00
Jennifer Johnson
e9eff54473 satellitedb/attribution: updates valueAttrQuery to work for both postgres and cockroach
Change-Id: I55d6c21950a66bf782596e9152622efd700c324a
2019-12-04 18:42:02 +00:00
Jennifer Johnson
ecb960f506 private/dbutil: distinguishes between db drivers and implementations to allow for different implementations of SQL queries.
Change-Id: I2dc8d1d371139aa8bc805e92a2b80b71f580fd64
2019-12-04 18:31:26 +00:00
Michal Niewrzal
b779826194 uplinkc: basic support for Scope (#3689) 2019-12-04 18:36:01 +01:00
Cameron
c00f688930 format db name in query to match actual db name (#3688) 2019-12-04 08:55:10 -05:00
Ethan Adams
9420fa9fc5 satellite/gracefulexit: Add graceful exit completed/failed receipt verification to satellite CLI (#3679) 2019-12-03 17:09:39 -05:00
Andrew Harding
2461ccd469 pkg/private/fpath: subsume AtomicWriteFile
AtomicWriteFile is useful primitive to use throughout the codebase

Change-Id: I338fc4505ba20d5aece09ddc257286f46298e083
2019-12-03 18:14:08 +00:00
paul cannon
4d2881b711 satellite/satellitedb: return latest coupon time as UTC
it doesn't necessarily _have_ to be UTC; the time is correct as returned
either way, but this will make it a little less prone to variance.

also, there is a test that depends on the time being returned in UTC.

Change-Id: Ia71e24ecd9973ba70a1cfb5621a3030a5c82d004
2019-12-03 17:36:41 +00:00
Jess G
e0b9b5b317
satellitedb: fixes for cockroachdb compatibility (#3682)
* crdb compat fixes

* update sn accounting query
2019-12-03 08:24:46 -08:00
Ivan Fraixedes
d69482e938
satellite/metainfo: Improve piece hash validation (#3671)
Improve the piece hash validation filtering out a piece when an order
limit is not found for it.

The commit also improves the documentation of an internal metainfo
method and rename the parameters of 2 methods for clarifying what they
are.
2019-12-03 14:36:32 +01:00
Cameron
52851026a7
satellite/console : remove code for creating user credit in CreateUser, skip TestUserCredits (#3680)
* remove code for creating user credit in CreateUser, skip TestUserCredits
2019-12-02 15:23:12 -05:00
paul cannon
b1fa7cdfbb
satellite/satellitedb/satellitedbtest: flag to run cockroach tests (#3674)
This will make it so we don't need to comment out those lines every time
we want to enable the cockroachdb tests during development.

Once it's ready this flag can go away.
2019-12-02 14:06:21 -06:00
Nikolay Yurchenko
90b631c8bf
web/satellite: referral links and registration (#3678) 2019-12-02 21:27:56 +02:00
Fadila
526a126a5a
Satellite: log the project information for upload, download, and list activities (#3651) 2019-12-02 15:39:19 +01:00
Fadila
24cacc8adf installer/windows: fix minor typo (#3676) 2019-12-02 14:48:50 +01:00
Ivan Fraixedes
42c61138e8
storage: Improve doc comments delete methods (#3591)
Improve the documentation of several methods involved in the delete
operation to make clear their behavior without having to inspect their
logic.
2019-12-02 12:18:20 +01:00
Vitalii Shpital
ae60b6533b
web/satellite: titles styling fixed (#3662) 2019-11-29 23:31:52 +02:00
Yehor Butko
ae05fa0874
satellitedb/accounting: fixed query (#3672) 2019-11-29 17:53:57 +02:00
Vitalii Shpital
d9a23b8727
web/satellite: password strength implemented on register page (#3669) 2019-11-29 14:52:21 +02:00
Vitalii Shpital
2a13a7764f
web/satellite: bundles size reduced (#3667) 2019-11-29 14:26:29 +02:00
Yehor Butko
0f523e82a6
missing line returned (#3670) 2019-11-28 23:42:04 +02:00
Yehor Butko
b10eee6e7f
satellitedb/accounting: project usage optimization (#3668) 2019-11-28 20:45:31 +02:00
Yaroslav Vorobiov
7e9b633dde
satellite/mailservice: move logging to send rendered async to cover template parsing (#3654) 2019-11-28 12:29:48 +02:00
Malcolm Bouzi
18a5e614d9 satellite/web: add segmentio plugin (#3405) 2019-11-27 11:57:59 -05:00
Nikolay Yurchenko
f83837bb03 web/satellite: billing banners (#3649) 2019-11-27 16:57:56 +02:00
Ivan Fraixedes
0bf7ac5b20 cmd/segment-reaper: Use bitmask type in observer (#3661)
bitmask type was created for making easy to understand the observer bit
logic.
This commit makes the changes in observer for using the bitmask type.
2019-11-27 13:11:07 +01:00
Ivan Fraixedes
7288d01781
cmd/segment-reaper: several refactorings (#3637)
* Move the observer implementation and the type definitions related with
  it and helper functions to its own file.
* Cluster struct type is used as a key for a ObjectsMap type.
  Observer struct type has a field of ObjectsMap.
  Cluster has a field for project ID.
  Observer processSegment method uses its ObjectMap field for tracking
  objects.

  However Observer processSegment clears the map once the projectID
  diverges from the one kept in the Observer lastProjectID field, meaning
  that it isn't needed to keep the projectID as part of the ObjectMap key.

  For this reason, ObjectMap can use as a key just only the bucket name
  and Cluster struct isn't needed.

  Because of such change, the ObjectMap type has been renamed to a more
  descriptive name.
* Make the types defined for this specific package not being exported.
* Create a constructor function for observer to encapsulate the map
  allocation.
* Don't throw away the entirely buckets objects map when an empty one
  is used to reuse part of the allocations.

  Encapsulate the clearing up logic into a method.
* Make the analyzeProject function to be a method of observer.
2019-11-27 10:28:43 +01:00
Isaac Hess
a6235d3962 storage/filestore: Monitor when we open files in trash
Change-Id: I817bf8349c2e1ba55e1490f06162af1099bebdb0
2019-11-26 14:38:49 -07:00
Yehor Butko
756b9b9e2b
satellite/payments: coupons and coupon usage (#3648) 2019-11-26 19:58:51 +02:00
Ivan Fraixedes
bf97ef06fc
storagenode: Add new endpoint to receive satellite requests for… (#3590)
* pkg/pg: Add new service function storage node

  Add a new service function to the storage node piece store for deleting
  pieces when satellites request them.

* storagenode/piecestore: Add endpoint to delete piece

  Add a new endpoint to receive from trusted satellites to delete a piece.

* private/testplanet: Fix storagenode mock

  Add to the storagenode mock the new endpoint method.

* proto.lock: Update it with the last protbuff changes

* storagenode/piecestore: Reuse test piece upload

  Extract the repeated logic from several tests functions for uploading a
  test piece to a test helper function.

* uplink/piecestore: Implement client side method

  Implement the client side method of the new piecestore RPC function.

* storagenode/piecestore: Add test DeletePiece endpoint

  Implement a test for the DeletePiece new endpoint method.
2019-11-26 18:47:19 +01:00