{job=~”apache|syslog”} <- show me logs where the job is apache **OR** syslog
{job=”apache”} |= “11.11.11.11” #搜job=apache的11.11.11.11
( |= “text”, |~ “regex”, …)
{app=”loki”} |= “level=error” 少加一个label 但是速度不慢 略等于{app=”loki”,level=”error”}
{app=”loki”,level!=”debug”} 如果用不等于 性能可能就不及于{app=”loki”} != “level=debug” ，因为后者没有label，磁盘性能更高，更少的chunk
`w+` is the same as "\w+"
{container="query-frontend",namespace="tempo-dev"} |= "metrics.go" | logfmt | duration > 10s and throughput_mb < 500he query is composed of:
- a log stream selector
{container="query-frontend",namespace="loki-dev"}which targets thequery-frontendcontainer in theloki-devnamespace. - a log pipeline
|= "metrics.go" | logfmt | duration > 10s and throughput_mb < 500which will filter out log that contains the wordmetrics.go, then parses each log line to extract more labels and filter with them.
=: exactly equal.!=: not equal.=~: regex matches.!~: regex does not match.
Examples:
{name=~"mysql.+"}{name!~"mysql.+"}{name!~`mysql-d+`}
{job="mysql"} |= "error"{name="kafka"} |~ "tsdb-ops.*io:2003"{name="cassandra"} |~ `error=w+`{instance=~"kafka-[23]",name="kafka"} != "kafka.server:type=ReplicaManager"
When using |~ and !~, Go (as in Golang) RE2 syntax regex may be used. The matching is case-sensitive by default and can be switched to case-insensitive prefixing the regex with (?i).
The json parsers take no parameters and can be added using the expression | json in your pipeline. It will extract all json properties as labels if the log line is a valid json document. Nested properties are flattened into label keys using the _ separator. Arrays are skipped. For example the json parsers will extract from the following document: { "protocol": "HTTP/2.0", "servers": ["129.0.1.1","10.2.1.3"], "request": { "time": "6.032", "method": "GET", "host": "foo.grafana.net", "size": "55", }, "response": { "status": 401, "size": "228", "latency_seconds": "6.031" } } The following list of labels: "protocol" => "HTTP/2.0" "request_time" => "6.032" "request_method" => "GET" "request_host" => "foo.grafana.net" "request_size" => "55" "response_status" => "401" "response_size" => "228" "response_size" => "228"
The logfmt parser can be added using the | logfmt and will extract all keys and values from the logfmt formatted log line. For example the following log line: at=info method=GET path=/ host=grafana.net fwd="124.133.124.161" connect=4ms service=8ms status=200 will get those labels extracted: "at" => "info" "method" => "GET" "path" => "/" "host" => "grafana.net" "fwd" => "124.133.124.161" "service" => "8ms" "status" => "200"
Unlike the logfmt and json, which extract implicitly all values and takes no parameters, the regexp parser takes a single parameter | regexp "<re>" which is the regular expression using the Golang RE2 syntax. The regular expression must contain a least one named sub-match (e.g (?P<name>re)), each sub-match will extract a different label. For example the parser | regexp "(?P<method>\w+) (?P<path>[\w|/]+) \((?P<status>\d+?)\) (?P<duration>.*)" will extract from the following line: POST /api/prom/api/v1/query_range (200) 1.5s those labels: "method" => "POST" "path" => "/api/prom/api/v1/query_range" "status" => "200" "duration" => "1.5s"
复杂案例：
用括号提升优先级：
| duration >= 20ms or method="GET" and size <= 20KB
| ((duration >= 20ms or method="GET") and size <= 20KB)
line format 表达式：
{container="frontend"} | logfmt | line_format "{{.query}} {{.duration}}"
原标签替换：
label_replace() For each timeseries in v, label_replace(v instant-vector, dst_label string, replacement string, src_label string, regex string) matches the regular expression regex against the label src_label.
If it matches, then the timeseries is returned with the label dst_label replaced by the expansion of replacement. $1 is replaced with the first matching subgroup, $2 with the second etc.
If the regular expression doesn’t match then the timeseries is returned unchanged. This example will return a vector with each time series having a foo label with the value a added to it: label_replace(rate({job="api-server",service="a:c"} |= "err" [1m]), "foo", "$1", "service", "(.*):.*")
多重过滤：
{alexenv=~"$namespace",podName=~"$serviceName"} (显示实时日志)
grafana loki输出格式化：{job="logstash",alexerror="true"} |regexp `(?P<allmsg>(?s)(.+?)$)`|line_format "{{.allmsg}}"
logstash配置：
input { beats { port => 5044 } } filter { ruby { code => "event.set('alextime',event.get('@timestamp').time.localtime + 8*60*60)" } ruby { code => "event.set('alexyear',event.get('alextime').to_s.split(pattern='-')[0])" } ruby { code => "event.set('alexmonth',event.get('alextime').to_s.split(pattern='-')[1])" } ruby { code => "event.set('alexday',event.get('alextime').to_s.split(pattern='-')[2].slice(0..1))" } ruby { code => "event.set('alexhour',event.get('alextime').to_s.split(pattern=':')[0].slice(-2..-1))" } ruby { code => "event.set('alexpath',event.get('log'))" } ruby { #code => "event.set('blex',event.get('alexpath')['file']['path'])" #code => "puts event.get('alexpath')['file']['path'].split(pattern=':')" #code => "event.set('alexpath',event.get('alexpath')['file']['path'].split(pattern=':')[-1])" code => "event.set('alexpath',event.get('alexpath')['file']['path'].split(pattern=':')[-1].tr('\','/'))" } mutate { split => { "shortHostname" => "-" } add_field => { "podName" => "%{[shortHostname][0]}" "job" => "logstash" } } } output { file { path => "/nfs/%{[alexenv]}/%{podName}-%{alexyear}-%{alexmonth}-%{alexday}-%{alexhour}.log" codec => line { format => "%{message}"} } # stdout { } loki { url => "http://172.23.29.3:3100/loki/api/v1/push" batch_size => 112640 retries => 5 min_delay => 3 max_delay => 500 } }
正则表达式：
这个是正则表达式的模式修饰符。
(?i)即匹配时不区分大小写。表示匹配时不区分大小写。
(?s)即Singleline(单行模式)。表示更改.的含义，使它与每一个字符匹配（包括换行 符
）。
(?m)即Multiline(多行模式) 。 表示更改^和$的 含义，使它们分别在任意一行的行首和行尾匹配，而不仅仅在整个字符串的开头和结尾匹配。(在此模式下,$的 精确含意是:匹配
之前的位置以及字符串结束前的位置.)
(?x)：表示如果加上该修饰符，表达式中的空白字符将会被忽略，除非它已经被转义。
(?e)：表示本修饰符仅仅对于replacement有用，代表在replacement中作为PHP代码。
(?A)：表示如果使用这个修饰符，那么表达式必须是匹配的字符串中的开头部分。比如说"/a/A"匹配"abcd"。
(?E)：与"m"相反，表示如果使用这个修饰符，那么"$"将匹配绝对字符串的结尾，而不是换行符前面，默认就打开了这个模式。
(?U)：表示和问号的作用差不多，用于设置"贪婪模式"。
?: （？）单个问号是不捕捉模式
写法如：（?:）
如何关闭圆括号的捕获能力？
而只是用它来做分组，方法是在左括号的后边加上:?，
这里第一个圆括弧只是用来分组，而不会占用捕获变量，*/
"(?:\w+\s(\w+))"