storj/storagenode/pieces
Márton Elek 9186365507 storagenode/pieces: more granular io and hashing statistic
This patch adds two new monkit metric:
 * piece_writer_io: the sum of the time, which is spent with io.Write during a piece upload (excluding the fs sync of the commit)
 * piece_writer_hash: the sum of the time, which is spent with hashing

The second is especially important. My storagenode (hosted on a cloud server) spend ~30 ms on hasing data, piece_write_io time is usually 5ms for me.

These metrics can help us to identify the reason of slownes on storagenode sides.

Both of these depend on the size of the piece. To make it more meaningfull without exploding the cardinality, I created a few size categories and classified the pieces based on these. Measurements shows that it can provide usefull results (>2MB uploads are usually 23-28 ms).

Change-Id: Ifa1c205a490046655bcc34891003e7b43ed9c0bc
2023-10-11 07:33:04 +00:00
..
lazyfilewalker storagenode/pieces/lazyfilewalker: test zapwrapper 2023-07-05 12:33:00 +00:00
cache_test.go {storagenode/pieces,cmd/storagenode}: refactor lazyfilewalker commands and tests 2023-06-02 00:11:53 +00:00
cache.go storagenode/blobstore: move blob store logic 2023-04-05 18:06:20 +00:00
db_test.go storagenode/storagenodedb/storagenodedbtest: pass ctx as an argument 2020-01-20 16:56:12 +02:00
deleter_test.go storagenode: run used-space filewalker as a low IO subprocess 2023-04-14 04:16:14 +00:00
deleter.go storagenode/pieces: send piece deletions to trash 2021-04-07 16:52:10 +00:00
doc.go storagenode: add doc.go files for sno packages 2021-09-29 08:24:56 +00:00
filewalker.go all: fix golangci failures 2023-08-09 11:44:44 +03:00
pieceexpiration_test.go storagenode/storagenodedb/storagenodedbtest: pass ctx as an argument 2020-01-20 16:56:12 +02:00
readwrite_test.go storagenode: run used-space filewalker as a low IO subprocess 2023-04-14 04:16:14 +00:00
readwrite.go storagenode/pieces: more granular io and hashing statistic 2023-10-11 07:33:04 +00:00
stat_test.go storagenode/pieces: more granular io and hashing statistic 2023-10-11 07:33:04 +00:00
stat.go storagenode/pieces: more granular io and hashing statistic 2023-10-11 07:33:04 +00:00
store_test.go storagenode/blobstore/filestore: add tracking of blobs 2023-05-05 15:40:15 +03:00
store.go storagenode/piecestore: implement trash recovery for download requests 2023-09-05 23:04:21 +00:00
trashchore.go storagenode/pieces: fix log line 2023-01-17 11:04:47 +02:00