storj/storagenode
Márton Elek 9186365507 storagenode/pieces: more granular io and hashing statistic
This patch adds two new monkit metric:
 * piece_writer_io: the sum of the time, which is spent with io.Write during a piece upload (excluding the fs sync of the commit)
 * piece_writer_hash: the sum of the time, which is spent with hashing

The second is especially important. My storagenode (hosted on a cloud server) spend ~30 ms on hasing data, piece_write_io time is usually 5ms for me.

These metrics can help us to identify the reason of slownes on storagenode sides.

Both of these depend on the size of the piece. To make it more meaningfull without exploding the cardinality, I created a few size categories and classified the pieces based on these. Measurements shows that it can provide usefull results (>2MB uploads are usually 23-28 ms).

Change-Id: Ifa1c205a490046655bcc34891003e7b43ed9c0bc
2023-10-11 07:33:04 +00:00
..
apikeys multinode/payouts: estimated payouts added 2021-05-11 11:33:32 +00:00
bandwidth storagenode: add doc.go files for sno packages 2021-09-29 08:24:56 +00:00
blobstore cmd/storagenode: add forget-satellite subcommand 2023-09-13 19:06:55 +00:00
collector storagenode/collector: delete expired piece info if file does not exist 2022-09-15 12:29:29 +00:00
console payments: don't redefine compensation rates twice 2023-06-10 12:40:43 +00:00
contact satellite,storagenode: propagate node tags with NodeCheckin 2023-07-05 13:45:42 +00:00
gracefulexit storagenode/gracefulexit: fix flaky test (hopefully) 2023-10-04 09:53:51 -05:00
healthcheck storagenode: accept HTTP calls on public port, listening for monitoring requests 2022-08-26 09:38:09 +00:00
inspector storagenode: add doc.go files for sno packages 2021-09-29 08:24:56 +00:00
internalpb all: fix deprecated ioutil commands 2022-10-11 15:27:29 +00:00
iopriority storagenode: run garbage collection filewalker as a low I/O subprocess 2023-05-02 19:43:38 +00:00
monitor storagenode/monitor: ignore shutdown errors 2023-08-16 11:53:58 +00:00
multinode {storagenode,web/multinode}: fix storage usage db/cache retrieval queries 2022-12-09 11:07:33 +00:00
nodestats storagenode/nodestats: add monkit metrics for reputation scores 2022-09-15 08:43:48 +00:00
notifications storagenode/console/consoleapi: use fixed time.Now() 2022-07-01 12:36:01 +03:00
operator storagenode: wallet features 2021-03-24 09:08:36 +00:00
orders satellite: move satellite/nodeselection/uploadselection => satellite/nodeselection 2023-07-07 10:32:03 +02:00
payouts {storagenode,web/multinode}: fix storage usage db/cache retrieval queries 2022-12-09 11:07:33 +00:00
pieces storagenode/pieces: more granular io and hashing statistic 2023-10-11 07:33:04 +00:00
piecestore storagenode/piecestore: notify if download piece was restored from trash 2023-10-05 18:50:01 +00:00
piecetransfer private/server,satellite/contact,misc: use new storj/common noise helpers 2023-02-07 09:53:45 -05:00
preflight storagenode/trust: ensure trust pool updates satellite status on Refresh 2023-10-03 16:54:00 +00:00
pricing all: golangci-lint v1.33.0 fixes (#3985) 2020-12-05 17:01:42 +01:00
reputation cmd/storagenode: add forget-satellite subcommand 2023-09-13 19:06:55 +00:00
retain {storagenode/pieces,cmd/storagenode}: refactor lazyfilewalker commands and tests 2023-06-02 00:11:53 +00:00
satellites storagenode/trust: ensure trust pool updates satellite status on Refresh 2023-10-03 16:54:00 +00:00
storagenodedb cmd/storagenode: add forget-satellite subcommand 2023-09-13 19:06:55 +00:00
storageusage storagenode: fix B*h to bytes disk usage conversion 2023-02-06 18:50:31 +00:00
trust storagenode/trust: ensure trust pool updates satellite status on Refresh 2023-10-03 16:54:00 +00:00
version storagenode/suspensions: added offline-suspension notificatio chore + tests 2020-10-29 19:44:22 +02:00
peer.go satellite,storagenode: propagate node tags with NodeCheckin 2023-07-05 13:45:42 +00:00