Commit Graph

8 Commits

Author SHA1 Message Date
paul cannon
41c5879f7c satellite: more detailed goroutine labels
This will apply an appropriate "subsystem" label to goroutines which are
part of the core, api, repairer, admin, or gc subsystems.

It will also label goroutines whose job it is to watch for slow shutdown
of lifecycle groups (there are a lot of these).

Finally, this will also label goroutines whose job it is to wait on the
toplevel errgroup of a subsystem.

Change-Id: I560b5fff4a0101300d6c9a67609c2d80d7424486
2022-05-11 17:50:55 +00:00
Egon Elbre
4031336cbd private/lifecycle: monitor unexpected shutdowns
Change-Id: I9af3a95a1b60c1572cd57bb5e9539d60aa0337bb
2021-06-25 19:26:23 +03:00
Egon Elbre
6e05b240dc private/lifecycle: log stack on slow shutdown
When services take long to shutdown it's useful to get a stack trace for
diagnosing the underlying problem.

Change-Id: Ic73a45741dfbe8fdddafd56a5b72121da886d133
2021-05-25 09:55:19 +00:00
Egon Elbre
e1f37ece08 private/lifecycle: warn on slow service shutdown
Adds a warning when service takes over 15s to shutdown.

Change-Id: I44307b4b7560ac2978f62a623894a4af4f5a7402
2020-11-06 15:01:54 +00:00
Egon Elbre
e0dca4042d all: add pprof labels for debugger
By using pprof.Labels debugger is able to show service/peer names in
goroutine names.

Change-Id: I5f55253470f7cc7e556f8e8b87f746394e41675f
2020-10-29 15:10:07 +00:00
Egon Elbre
77b53bd21c private/lifecycle: log fatal ending to a runner
Change-Id: If07b62dad7f4ac235dd51a3a217c2c56d30978ad
2020-09-03 16:54:40 +03:00
Jeff Wendling
7999d24f81 all: use monkit v3
this commit updates our monkit dependency to the v3 version where
it outputs in an influx style. this makes discovery much easier
as many tools are built to look at it this way.

graphite and rothko will suffer some due to no longer being a tree
based on dots. hopefully time will exist to update rothko to
index based on the new metric format.

it adds an influx output for the statreceiver so that we can
write to influxdb v1 or v2 directly.

Change-Id: Iae9f9494a6d29cfbd1f932a5e71a891b490415ff
2020-02-05 23:53:17 +00:00
Egon Elbre
e319660f7a private/lifecycle: implement Group
lifecycle.Group implements controlling multiple items such
that their startup and close works.

Change-Id: Idb4f4a6c3a1f07cdcf44d3147a6c959686df0007
2020-01-29 00:37:33 +00:00