storj/cmd/statreceiver
Isaac Hess 85b6316ce8 statreceiver: Update to filter on packet headers
The admission/v3 protocol now supports arbitrary key/value headers to be
included in each packet of metrics. This commit creates support for
this, so the lua config file can declare a filter taking into account
the key/value headers.

Change-Id: I41de8c018d33304ccf46ec221ae689d55c5fb1ee
2020-03-12 13:57:07 -06:00
..
luacfg lint: add staticcheck as a separate step (#3569) 2019-11-14 10:31:30 +02:00
common.go updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
copy.go updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
db.go updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
downgrade.go cmd/statreceiver: Change v2 known metrics to toml config file 2020-02-18 16:52:42 -07:00
downgrademetrics.toml cmd/statreceiver: Change v2 known metrics to toml config file 2020-02-18 16:52:42 -07:00
example.lua cmd/statreceiver: allow for packet filtering to packet destinations (#2019) 2019-05-22 16:19:32 -06:00
file.go updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
filter.go statreceiver: Update to filter on packet headers 2020-03-12 13:57:07 -06:00
graphite.go Fix some leaks and add notes about close handling (#2334) 2019-06-25 23:00:51 +03:00
influx.go cmd/statreceiver: Add instance tag to influx metric 2020-02-21 16:33:00 -03:00
main.go statreceiver: Update to filter on packet headers 2020-03-12 13:57:07 -06:00
parser.go all: use monkit v3 2020-02-05 23:53:17 +00:00
print.go statreceiver: Update to filter on packet headers 2020-03-12 13:57:07 -06:00
README.md Update README.md 2018-12-20 10:01:28 -07:00
sanitizer.go updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
schema.sql cmd/statreceiver: lua-scriptable stat receiver (#636) 2018-12-11 11:24:31 -07:00
statreceiver.lua statreceiver: Update to filter on packet headers 2020-03-12 13:57:07 -06:00
udp.go updates copyright 2018 to 2019 (#1133) 2019-01-24 15:15:10 -05:00
version.go statreceiver: add v2/v3 splitter and downgrade 2020-02-02 22:56:14 +00:00

statreceiver

This package implements a Lua-scriptable pipeline processor for zeebo/admission telemetry packets (like monkit or something).

There are a number of types of objects involved in making this work:

  • Sources - A source is a source of packets. Each packet is a byte slice that, when parsed, consists of application and instance identification information (such as the application name and perhaps the MAC address or some other id of the computer running the application), and a list of named floating point values. There are currently two types of sources, a UDP source and a file source. A UDP source appends the current time as the timestamp to all packets, whereas a file source should have a prior timestamp to attach to each packet.
  • Packet Destinations - A packet destination is something that can handle a packet with a timestamp. This is either a packet parser, a UDP packet destination for forwarding to another process, or a file destination that will serialize all packets and timestamps for later replay.
  • Metric Destinations - Once a packet has been parsed, the contained metrics can get sent to a metric destination, such as a time series database, a relational database, stdout, a metric filterer, etc.

Please see example.lua for a good example of using this pipeline.

Setup

If you use a relational database metric destination, make sure to instantiate the schema provided in schema.sql first.