Commit Graph

16 Commits

Author SHA1 Message Date
Ivan Fraixedes
963db8c738 cmd/segment-reaper: Remove unneeded test cases
Remove unneeded observer's tests cases after the changes that enhanced
the segment-reaper not to be limited to objects up to 65 segments.

Also remove some tests cases that were already covered by the
TestObserver_processSegment_from_to test function.

Change-Id: I2385b406b50f7810ed22d3297b903a993f83fcfe
2020-05-20 10:10:51 +00:00
Ivan Fraixedes
cbdd3bfd7b cmd/segment-reaper: Rename test name to what it tests
Rename a test function to indicate the function that it actually tests
rather than indicating the function that calls the function under test.

Change-Id: I2c1fe165e3acf952993a9a65dfbfbdcda0d1cccc
2020-05-19 10:06:56 +00:00
Ivan Fraixedes
d27cb60f81 cmd/segment-reaper: Remove unneeded test logic
After the segment-reaper was enhanced to support objects with any number
of segments, the test helper function to generate test data doesn't have
to support to generate objects with a number of segments up to or over
the previous limit.

The commit remove that unnecessary test logic.

Change-Id: I2897c5a96fb133f61de2cdb0ef7d13ee5e0151e8
2020-05-18 16:39:48 +00:00
Ivan Fraixedes
c58dccb7a8
cmd/segment-reaper: Remove segments of skipped objects
Reorganize some operations in the observer.processSegment method:

* to minimal reduce the memory usage removing the segments of the
  objects marked as skipped. Skipped objects aren't discarded of the
  analysis stage so the segments aren't needed.
* returns earlier when an object is skipped because isn't needed a
  further processing.

Change-Id: I210a26c394477ee411ff7f640507dcc07733a47f
2020-05-12 16:12:44 +02:00
Bill Thorp
4b612a7906 cmd/segment-reaper: add support for >65 segments
Change the bitmask used by segment reaper to use []byte rather than uint64

This passes tests but I have literally no clue how to integration test this.

Change-Id: I393f4598b27cae6e427da2190dd3109bca721c34
2020-05-09 10:45:26 +00:00
Egon Elbre
11a44cdd88 all: don't depend on gogo/proto directly
Change-Id: I8822dea0d1b7b99e0b828e0373a0308a42dde2be
2020-04-08 17:32:15 +00:00
Egon Elbre
6492b13d81 all: remove old uuid
Change-Id: I3a137f73456f010c37d3933dbe12cbbb840b809f
2020-04-02 19:30:36 +03:00
Egon Elbre
0a69da4ff1 all: switch to storj.io/common/uuid
Change-Id: I178a0a8dac691e57bce317b91411292fb3c40c9f
2020-03-31 19:16:41 +03:00
Egon Elbre
c6f94ce9e4 satellite/metainfo: remove support for boltdb based pointerDB
By previous changes we can now remove testplanet.New and
also remove metainfo boltdb support.

Change-Id: I5bdfbbbb45967492728e705b34b2fedb4f28c381
2020-01-23 13:54:00 +02:00
Egon Elbre
6615ecc9b6 common: separate repository
Change-Id: Ibb89c42060450e3839481a7e495bbe3ad940610a
2019-12-27 14:11:15 +02:00
Ivan Fraixedes
9ac2c4d815 cmd/segment-reaper: Add test cases from/to processSegment
Add randomized test cases for testing the observer processSegment method
when the observer has a time range (from and to) set and there are
objects with segments whose creation date is outside of this range.

Change-Id: Ieac82a21f278f0850b95275bfdd2e8a812cc57a5
2019-12-13 17:23:32 +00:00
Michal Niewrzal
e1d99522b9 cmd/segment-reaper: test processSegment for single project
This is a test for `observer.processSegment` method from `segment-reaper
detect` command.

Change-Id: I14b7245848766b7aa01e5edffcf94c61a06a4c0a
2019-12-12 10:26:05 +00:00
Michal Niewrzal
b294569840 segment-reaper: fix for not analyzing last project in detect command
This change fixes issue where last project in DB was not analyzed to
find zombie segments

Change-Id: I9efd8a39a65f72d85c7aae325a0e4e3cc1cb9afb
2019-12-11 13:18:13 +00:00
Michal Niewrzal
011bb5d551 segment-reaper: test from/to detect parameters
Unit test for testing from/to parameters of `segment-reaper detect`
command.

Change-Id: I85c8839b676c61e7f046cb5ccc92d988ca51d6e3
2019-12-10 12:18:44 +01:00
Ivan Fraixedes
e961b1840c
cmd/segment-reaper: Implement unit test for observer (#3658)
Implement some unit test cases for the observer.processSegment method and fix a bug found by these tests.

A production snapshot is more certain but it's huge for having in the repository and run the test with it by the CI.
We want to have tests for the different cases to detect zombie segments and relaying on production data cannot guarantee to have all of them.

NOTE the test has been implemented with random values for not having always the same combination of segments list and the same values avoiding that the implementation gets stale due to the test. The issue about this is that it's harder to understand and we could get only sometimes failures in the CI in case that the implementation has some bug.

The random tests allow to eventually check cases that a static test may now cover because it is not expressed or because it is needed to implement a large number of cases.

Because we are worried that the test implementation is complex and we could have bugs on it despite that the same bugs should exist in both, the implementation and the test, moreover that we have to consider that the implementation and the tests have been written by different people. Because of that, we may replace entirely these random tests by a list of static ones.
2019-12-10 11:01:58 +01:00
Michal Niewrzal
27462f68e9
cmd/segment-reaper: add detecting and printing zombie segments (#3598) 2019-12-09 03:30:52 -08:00