public class LogsIngestionConfig extends Configuration
JsonProperty
is parsed directly from your logsIngestionConfigFile, which is
YAML. Below is a sample config file which shows the features of direct logs ingestion. The "counters" section
corresponds to counters
, likewise for gauges
and histograms
. In each of these three
groups, the pricipal entry is a MetricMatcher
. See the patterns file
here for
help defining patterns, also various grok debug tools (e.g. this one,
or use google)
All metrics support dynamic naming with %{}. To see exactly what data we send as part of histograms, see
FlushProcessor.processHistogram(MetricName, Histogram, FlushProcessorContext)
.
counters:
# For log line "alpha 42", increment counterWithValue by 42
- pattern: "alpha %{NUMBER:value}"
metricName: "counterWithValue"
# For log line "bravo", increment plainCounter by 1
- pattern: "bravo"
metricName: "plainCounter"
# For log line "found item in myhost-123", increment "dynamic_myhost_123" by 1
- pattern: "found item in %{WORD:name}-%{NUMBER:type}"
metricName: "dynamic_%{name}_%{type}"
# For log line "she sold 42 sea shells", increment "shells" by 42. See MYPATTERN below.
- pattern: "%{MYPATTERN}"
metricName: "shells"
valueLabel: "value"
# For log line "operation foo took 42 seconds in DC=oregon and AZ=2a", increment "foo.totalSeconds" with point
# tags "theDC=oregon theAZ=2a" by 42
- pattern: "operation %{WORD:op} took %{NUMBER:value} seconds in DC=%{WORD:dc}.*AZ=%{WORD:az}"
metricName: "%{op}.totalSeconds"
tagKeys:
- "theDC"
- "theAZ"
tagValueLabels:
- "dc"
- "az"
# For log line '127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "http://www.example.com/start.html" "Mozilla/4.08 [en] (Win98; I ;Nav)"',
# increment apacheBytes by 2326. See the patterns file.
- pattern: "%{COMBINEDAPACHELOG}"
metricName: "apacheBytes"
valueLabel: "bytes"
gauges:
# For log line "temperature 78", set the "temp" metric to 78
- pattern: "temperature %{NUMBER:value}"
metricName: "temp"
# For log line: "WARNING: [2878] (SUMMARY): points attempted: 859432; blocked: 0", set wavefrontPointsSent.2878 to 859432
- pattern: '%{LOGLEVEL}: \[%{NUMBER:port}\] %{GREEDYDATA} points attempted: %{NUMBER:pointsAttempted}'
metricName: "wavefrontPointsSent.%{port}"
valueLabel: "pointsAttempted"
histograms:
# For log line "histo 123.45", add 123.45 to the running distribution. Will send metrics like p99, mean, ...
- pattern: "histo %{NUMBER:value}"
metricName: "myHisto"
additionalPatterns:
- "MYPATTERN she sold %{NUMBER:value} sea shells"
Modifier and Type | Field and Description |
---|---|
List<String> |
additionalPatterns
Additional grok patterns to use in pattern matching for the above
MetricMatcher s. |
Integer |
aggregationIntervalSeconds
How often metrics are aggregated and sent to wavefront.
|
int |
configReloadIntervalSeconds
How often to check this config file for updates.
|
List<MetricMatcher> |
counters
Counters to ingest from incoming log data.
|
long |
expiryMillis
Metrics are cleared from memory (and so their aggregation state is lost) if a metric is not updated
within this many milliseconds.
|
List<MetricMatcher> |
gauges
Gauges to ingest from incoming log data.
|
List<MetricMatcher> |
histograms
Histograms to ingest from incoming log data.
|
boolean |
useWavefrontHistograms
If true, use
WavefrontHistogram s rather than Histogram s. |
Constructor and Description |
---|
LogsIngestionConfig() |
Modifier and Type | Method and Description |
---|---|
String |
patternsFile() |
void |
verifyAndInit() |
ensure, equals, hashCode, toString
public Integer aggregationIntervalSeconds
public List<MetricMatcher> counters
public List<MetricMatcher> gauges
public List<MetricMatcher> histograms
public List<String> additionalPatterns
MetricMatcher
s.public long expiryMillis
public boolean useWavefrontHistograms
WavefrontHistogram
s rather than Histogram
s. Histogram ingestion must be enabled on wavefront to use this feature. When
using Yammer histograms, the data is exploded into constituent metrics. See FlushProcessor.processHistogram(MetricName, Histogram, FlushProcessorContext)
.public int configReloadIntervalSeconds
public String patternsFile()
Grok
.public void verifyAndInit() throws ConfigurationException
verifyAndInit
in class Configuration
ConfigurationException
Copyright © 2017. All rights reserved.