Commit Graph

13 Commits

Author SHA1 Message Date
Ivan Fraixedes
9ac2c4d815 cmd/segment-reaper: Add test cases from/to processSegment
Add randomized test cases for testing the observer processSegment method
when the observer has a time range (from and to) set and there are
objects with segments whose creation date is outside of this range.

Change-Id: Ieac82a21f278f0850b95275bfdd2e8a812cc57a5
2019-12-13 17:23:32 +00:00
Michal Niewrzal
e1d99522b9 cmd/segment-reaper: test processSegment for single project
This is a test for `observer.processSegment` method from `segment-reaper
detect` command.

Change-Id: I14b7245848766b7aa01e5edffcf94c61a06a4c0a
2019-12-12 10:26:05 +00:00
Michal Niewrzal
b294569840 segment-reaper: fix for not analyzing last project in detect command
This change fixes issue where last project in DB was not analyzed to
find zombie segments

Change-Id: I9efd8a39a65f72d85c7aae325a0e4e3cc1cb9afb
2019-12-11 13:18:13 +00:00
Michal Niewrzal
011bb5d551 segment-reaper: test from/to detect parameters
Unit test for testing from/to parameters of `segment-reaper detect`
command.

Change-Id: I85c8839b676c61e7f046cb5ccc92d988ca51d6e3
2019-12-10 12:18:44 +01:00
Ivan Fraixedes
e961b1840c
cmd/segment-reaper: Implement unit test for observer (#3658)
Implement some unit test cases for the observer.processSegment method and fix a bug found by these tests.

A production snapshot is more certain but it's huge for having in the repository and run the test with it by the CI.
We want to have tests for the different cases to detect zombie segments and relaying on production data cannot guarantee to have all of them.

NOTE the test has been implemented with random values for not having always the same combination of segments list and the same values avoiding that the implementation gets stale due to the test. The issue about this is that it's harder to understand and we could get only sometimes failures in the CI in case that the implementation has some bug.

The random tests allow to eventually check cases that a static test may now cover because it is not expressed or because it is needed to implement a large number of cases.

Because we are worried that the test implementation is complex and we could have bugs on it despite that the same bugs should exist in both, the implementation and the test, moreover that we have to consider that the implementation and the tests have been written by different people. Because of that, we may replace entirely these random tests by a list of static ones.
2019-12-10 11:01:58 +01:00
Michal Niewrzal
27462f68e9
cmd/segment-reaper: add detecting and printing zombie segments (#3598) 2019-12-09 03:30:52 -08:00
Michal Niewrzal
b17d40ffd4
segment-reaper: delete command logic (#3660) 2019-12-09 00:25:39 -08:00
Ivan Fraixedes
0bf7ac5b20 cmd/segment-reaper: Use bitmask type in observer (#3661)
bitmask type was created for making easy to understand the observer bit
logic.
This commit makes the changes in observer for using the bitmask type.
2019-11-27 13:11:07 +01:00
Ivan Fraixedes
7288d01781
cmd/segment-reaper: several refactorings (#3637)
* Move the observer implementation and the type definitions related with
  it and helper functions to its own file.
* Cluster struct type is used as a key for a ObjectsMap type.
  Observer struct type has a field of ObjectsMap.
  Cluster has a field for project ID.
  Observer processSegment method uses its ObjectMap field for tracking
  objects.

  However Observer processSegment clears the map once the projectID
  diverges from the one kept in the Observer lastProjectID field, meaning
  that it isn't needed to keep the projectID as part of the ObjectMap key.

  For this reason, ObjectMap can use as a key just only the bucket name
  and Cluster struct isn't needed.

  Because of such change, the ObjectMap type has been renamed to a more
  descriptive name.
* Make the types defined for this specific package not being exported.
* Create a constructor function for observer to encapsulate the map
  allocation.
* Don't throw away the entirely buckets objects map when an empty one
  is used to reuse part of the allocations.

  Encapsulate the clearing up logic into a method.
* Make the analyzeProject function to be a method of observer.
2019-11-27 10:28:43 +01:00
Michal Niewrzal
1e33995a89
cmd/segment-reaper: skeleton command for zombie deletion (#3599) 2019-11-26 05:01:31 -08:00
Ivan Fraixedes
77d4add101
cmd/segment-reaper: Implement bitmask type (#3626)
Define a bitmask type for implementing its logic to detect if an index
is set or not.
2019-11-22 11:53:48 +01:00
Michal Niewrzal
8ea09d5556
cmd/segment-reaper: add iteration over DB (#3576) 2019-11-18 09:13:20 -08:00
Michal Niewrzal
4912922631
cmd: skeleton for segment-reaper detect command (#3562) 2019-11-14 07:57:27 -08:00