text:
store: More than 2000 Connections/s to S3 Backend

In case of issues related to exact bucket implementation, please ping corresponded maintainer from list here: https://github.com/thanos-io/thanos/blob/master/docs/storage.md
pinging @bwplotka because of S3 backend
Thanos, Prometheus and Golang version used
prometheus --version
prometheus, version 2.12.0 (branch: HEAD, revision: 43acd0e2e93f9f70c49b2267efa0124f1e759e86)
  build user:       root@7a9dbdbe0cc7
  build date:       20190818-13:53:16
  go version:       go1.12.8

thanos --version
thanos, version 0.7.0 (branch: HEAD, revision: c6eaf68bec32aefd280318eaef6121d6ddb20d10)
  build user:       root@aa355daaf987
  build date:       20190902-15:31:30
  go version:       go1.12.5

What happened
Sometimes, Thanos Store is making more than 2000 connections per second to S3 backend. The firewall blocks any connections after that for 1 minute (DDOS prevention).
It occurrs as infrequent spikes and I can't yet relate to any event. I did not see any queries being done on Grafana or Thanos Query component shortly before or at the moment of those spikes, that could have caused them.
I updated on 03. September 2019 ~19:00 GMT+02:00 from v0.6.1 to Thanos 0.7.0, hoping that the newly merged

#1302 Thanos now efficiently reuses HTTP keep-alive connections

would fix this issue.
What you expected to happen
To not spike that much.
How to reproduce it (as minimally and precisely as possible):
I can't relate any event, so I'm not sure what to do to reproduce them.
Thanos Store Startup Configuration
/appl/thanos/bin/thanos store --data-dir=/appl/thanos/data-cache --grpc-address=0.0.0.0:7023 --http-address=0.0.0.0:7007 --log.level=debug --objstore.config-file=/appl/thanos/.s3.yml

Contents of --objstore.config-file
---
config:
  access_key: <REDACTED>
  bucket: p-prometheus-thanos
  encrypt_sse: false
  endpoint: <REDACTED_ON_PREMISE>
  http_config:
    idle_conn_timeout: 0s
    insecure_skip_verify: true
  insecure: false
  put_user_metadata: {}
  secret_key: <REDACTED>
  signature_version2: false
  trace:
    enable: false
type: S3

Full logs to relevant components
All Thanos components are started via systemd. Log output is transferred to /var/log/thanos.log via rsyslog. Hence the logs may also contain logs from Thanos Compactor & Query components.
Apparently the 2000/s connection threshold was reached at 13:04:11, but I can't see anything suspicious in the logs.
Logs

Sep  4 13:04:01 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:01.825817391Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:1316 postingsTouched:23049 postingsTouchedSizeSum:811328 postingsToFetch:0 postingsFetched:23049 postingsFetchedSizeSum:993152 postingsFetchCount:321 postingsFetchDurationSum:46153336980 seriesTouched:179783 seriesTouchedSizeSum:13566197 seriesFetched:179783 seriesFetchedSizeSum:97120224 seriesFetchCount:609 seriesFetchDurationSum:701108188367 chunksTouched:1288460 chunksTouchedSizeSum:116520069 chunksFetched:1288460 chunksFetchedSizeSum:191147868 chunksFetchCount:916 chunksFetchDurationSum:107864552498 getAllDuration:1063268359278 mergedSeriesCount:1 mergedChunksCount:32 mergeDuration:0}" err="rpc error: code = Unknown desc = send series response: rpc error: code = Internal desc = transport: transport: the stream is done or WriteHeader was already called"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.079653872Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 filtered out;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.086267514Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 queried;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.086699708Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567573140000 maxt=1567595055000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1567569600000-1567584000000 Resolution: 0"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.086880897Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567573140000 maxt=1567595055000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1567569600000-1567584000000 Resolution: 0"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.086963687Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567573140000 maxt=1567595055000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1567569600000-1567584000000 Resolution: 0"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.087027686Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567573140000 maxt=1567595055000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1567569600000-1567584000000 Resolution: 0"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.087291432Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:8 postingsTouched:0 postingsTouchedSizeSum:0 postingsToFetch:0 postingsFetched:0 postingsFetchedSizeSum:0 postingsFetchCount:0 postingsFetchDurationSum:0 seriesTouched:0 seriesTouchedSizeSum:0 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:0 chunksTouchedSizeSum:0 chunksFetched:0 chunksFetchedSizeSum:0 chunksFetchCount:0 chunksFetchDurationSum:0 getAllDuration:137693 mergedSeriesCount:0 mergedChunksCount:0 mergeDuration:4021}" err=null
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.611764342Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 filtered out"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.634286054Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 filtered out;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.646902717Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 filtered out;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.651589353Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 filtered out"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.659394176Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 filtered out"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.690827191Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 filtered out"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.694419501Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 filtered out"
Sep  4 13:04:02 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:02.719835135Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 filtered out"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.616888164Z caller=bucket.go:325 msg="error parsing block range" block=01DKW2PYFZ47Q954ZB76V5M78P err="read meta.json: open /appl/thanos/data-cache/01DKW2PYFZ47Q954ZB76V5M78P/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.616990476Z caller=bucket.go:325 msg="error parsing block range" block=01DKW3KJNF6C5MSTPRD4BSGZ8S err="read meta.json: open /appl/thanos/data-cache/01DKW3KJNF6C5MSTPRD4BSGZ8S/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.618902951Z caller=bucket.go:325 msg="error parsing block range" block=01DKW4E7E5QGWCF4BH7SWCFWGW err="read meta.json: open /appl/thanos/data-cache/01DKW4E7E5QGWCF4BH7SWCFWGW/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.618973884Z caller=bucket.go:325 msg="error parsing block range" block=01DKW56NPXANNEV0CMEX6K581X err="read meta.json: open /appl/thanos/data-cache/01DKW56NPXANNEV0CMEX6K581X/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.620408223Z caller=bucket.go:325 msg="error parsing block range" block=01DKW6181JVVW9H6QAVRAN74MJ err="read meta.json: open /appl/thanos/data-cache/01DKW6181JVVW9H6QAVRAN74MJ/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.62043776Z caller=bucket.go:325 msg="error parsing block range" block=01DKW6T8EM706WXN387FF3DSG6 err="read meta.json: open /appl/thanos/data-cache/01DKW6T8EM706WXN387FF3DSG6/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.622715992Z caller=bucket.go:325 msg="error parsing block range" block=01DKW7N21EF08YXVPDWE5WXSBY err="read meta.json: open /appl/thanos/data-cache/01DKW7N21EF08YXVPDWE5WXSBY/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.625997967Z caller=bucket.go:325 msg="error parsing block range" block=01DKW8E0CG7E66CT73MZH8MHGW err="read meta.json: open /appl/thanos/data-cache/01DKW8E0CG7E66CT73MZH8MHGW/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.626773435Z caller=bucket.go:325 msg="error parsing block range" block=01DKW97F57VPNCMFXM11SF34QJ err="read meta.json: open /appl/thanos/data-cache/01DKW97F57VPNCMFXM11SF34QJ/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.626800456Z caller=bucket.go:325 msg="error parsing block range" block=01DKWA11VMW67NPPYH93EA49AR err="read meta.json: open /appl/thanos/data-cache/01DKWA11VMW67NPPYH93EA49AR/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.628235615Z caller=bucket.go:325 msg="error parsing block range" block=01DKWAT5S8EBW0P67WMT4TZ19M err="read meta.json: open /appl/thanos/data-cache/01DKWAT5S8EBW0P67WMT4TZ19M/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.628262633Z caller=bucket.go:325 msg="error parsing block range" block=01DKWBKGN2X8NRMS0A3HW4XSXR err="read meta.json: open /appl/thanos/data-cache/01DKWBKGN2X8NRMS0A3HW4XSXR/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.62967897Z caller=bucket.go:325 msg="error parsing block range" block=01DKWCCP679PYV9VJE3S3E30EK err="read meta.json: open /appl/thanos/data-cache/01DKWCCP679PYV9VJE3S3E30EK/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.629729405Z caller=bucket.go:325 msg="error parsing block range" block=01DKWD4C7VR4M0CZMGZZ1KWEVC err="read meta.json: open /appl/thanos/data-cache/01DKWD4C7VR4M0CZMGZZ1KWEVC/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.631502923Z caller=bucket.go:325 msg="error parsing block range" block=01DKWDY4ZKRQDQ5KAWKTYE1TN5 err="read meta.json: open /appl/thanos/data-cache/01DKWDY4ZKRQDQ5KAWKTYE1TN5/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.631536558Z caller=bucket.go:325 msg="error parsing block range" block=01DKWEQ56MF8SX4F2QD8W4PTEJ err="read meta.json: open /appl/thanos/data-cache/01DKWEQ56MF8SX4F2QD8W4PTEJ/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.633667658Z caller=bucket.go:325 msg="error parsing block range" block=01DKWFH0HJ81Y22YM8K78F0MC5 err="read meta.json: open /appl/thanos/data-cache/01DKWFH0HJ81Y22YM8K78F0MC5/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.63370579Z caller=bucket.go:325 msg="error parsing block range" block=01DKWG9DPNKD2SAN3R07XZZJDV err="read meta.json: open /appl/thanos/data-cache/01DKWG9DPNKD2SAN3R07XZZJDV/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.637667472Z caller=bucket.go:325 msg="error parsing block range" block=01DKWH42KAQBFTWDMN531PHX36 err="read meta.json: open /appl/thanos/data-cache/01DKWH42KAQBFTWDMN531PHX36/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.637716685Z caller=bucket.go:325 msg="error parsing block range" block=01DKWHZ4YQYP67A0CAWP8FTEN7 err="read meta.json: open /appl/thanos/data-cache/01DKWHZ4YQYP67A0CAWP8FTEN7/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.638883728Z caller=bucket.go:325 msg="error parsing block range" block=01DKWJTWZRQMFHBP243E0N7DEJ err="read meta.json: open /appl/thanos/data-cache/01DKWJTWZRQMFHBP243E0N7DEJ/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.644267818Z caller=bucket.go:325 msg="error parsing block range" block=01DKWMDCF5JJH41N7DWF5YNHMC err="read meta.json: open /appl/thanos/data-cache/01DKWMDCF5JJH41N7DWF5YNHMC/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.644314139Z caller=bucket.go:325 msg="error parsing block range" block=01DKWN6SFNBCMHCZ172YF4P11P err="read meta.json: open /appl/thanos/data-cache/01DKWN6SFNBCMHCZ172YF4P11P/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.647090016Z caller=bucket.go:325 msg="error parsing block range" block=01DKWNZR9KYA4M8MZ1V99X3YB7 err="read meta.json: open /appl/thanos/data-cache/01DKWNZR9KYA4M8MZ1V99X3YB7/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.647137734Z caller=bucket.go:325 msg="error parsing block range" block=01DKWPSDRF734C4BG069MXE97V err="read meta.json: open /appl/thanos/data-cache/01DKWPSDRF734C4BG069MXE97V/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.647556818Z caller=bucket.go:325 msg="error parsing block range" block=01DKWQK8BP3652876G14BPZDT7 err="read meta.json: open /appl/thanos/data-cache/01DKWQK8BP3652876G14BPZDT7/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.647588262Z caller=bucket.go:325 msg="error parsing block range" block=01DKWRBNRDSENA8JBGTVXZWMFS err="read meta.json: open /appl/thanos/data-cache/01DKWRBNRDSENA8JBGTVXZWMFS/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.648839926Z caller=bucket.go:325 msg="error parsing block range" block=01DKWS41V3BZSAWERTW35EW7X3 err="read meta.json: open /appl/thanos/data-cache/01DKWS41V3BZSAWERTW35EW7X3/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.648880977Z caller=bucket.go:325 msg="error parsing block range" block=01DKWTKTJPNKTC591D98BF55PQ err="read meta.json: open /appl/thanos/data-cache/01DKWTKTJPNKTC591D98BF55PQ/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.64891254Z caller=bucket.go:325 msg="error parsing block range" block=01DKWVBER27PNTDGQ93PQJ1TBB err="read meta.json: open /appl/thanos/data-cache/01DKWVBER27PNTDGQ93PQJ1TBB/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.649270332Z caller=bucket.go:325 msg="error parsing block range" block=01DKWW48DEPE75JH5FX9SKZ3MW err="read meta.json: open /appl/thanos/data-cache/01DKWW48DEPE75JH5FX9SKZ3MW/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.651908257Z caller=bucket.go:325 msg="error parsing block range" block=01DKWWX5T37068WERGJ8T5DHZ9 err="read meta.json: open /appl/thanos/data-cache/01DKWWX5T37068WERGJ8T5DHZ9/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.652890668Z caller=bucket.go:325 msg="error parsing block range" block=01DKWXQ1Y2BJ33ETSXP3NSGPF9 err="read meta.json: open /appl/thanos/data-cache/01DKWXQ1Y2BJ33ETSXP3NSGPF9/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.652934816Z caller=bucket.go:325 msg="error parsing block range" block=01DKWYEX7527S9D7C22MDYB99W err="read meta.json: open /appl/thanos/data-cache/01DKWYEX7527S9D7C22MDYB99W/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.654246285Z caller=bucket.go:325 msg="error parsing block range" block=01DKWZ740QB25EMS2GS3RVFPVA err="read meta.json: open /appl/thanos/data-cache/01DKWZ740QB25EMS2GS3RVFPVA/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.654281239Z caller=bucket.go:325 msg="error parsing block range" block=01DKWZYR36D28VCRPZB6R2F4ZS err="read meta.json: open /appl/thanos/data-cache/01DKWZYR36D28VCRPZB6R2F4ZS/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.654741348Z caller=bucket.go:325 msg="error parsing block range" block=01DKX0PTHJTKS9FT7CGSTEECZP err="read meta.json: open /appl/thanos/data-cache/01DKX0PTHJTKS9FT7CGSTEECZP/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.654775989Z caller=bucket.go:325 msg="error parsing block range" block=01DKX1GPGMK05FH6X8PKJR6MDC err="read meta.json: open /appl/thanos/data-cache/01DKX1GPGMK05FH6X8PKJR6MDC/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.656274794Z caller=bucket.go:325 msg="error parsing block range" block=01DKX29VWKQFWTK0DT11AF1Y3Y err="read meta.json: open /appl/thanos/data-cache/01DKX29VWKQFWTK0DT11AF1Y3Y/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.662909148Z caller=bucket.go:325 msg="error parsing block range" block=01DKX3T9CDAJYJE4F7MNNEQB7K err="read meta.json: open /appl/thanos/data-cache/01DKX3T9CDAJYJE4F7MNNEQB7K/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.662967223Z caller=bucket.go:325 msg="error parsing block range" block=01DKX4KAH53E00FBXMN20M8TRV err="read meta.json: open /appl/thanos/data-cache/01DKX4KAH53E00FBXMN20M8TRV/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.664456651Z caller=bucket.go:325 msg="error parsing block range" block=01DKX5D4VQJP86K6GZQ59SB883 err="read meta.json: open /appl/thanos/data-cache/01DKX5D4VQJP86K6GZQ59SB883/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.664497005Z caller=bucket.go:325 msg="error parsing block range" block=01DKX65JY49W2227YAM3ETPSVB err="read meta.json: open /appl/thanos/data-cache/01DKX65JY49W2227YAM3ETPSVB/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.665737909Z caller=bucket.go:325 msg="error parsing block range" block=01DKX6YJHYKJAM9KNWWV1PG10Y err="read meta.json: open /appl/thanos/data-cache/01DKX6YJHYKJAM9KNWWV1PG10Y/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.665790354Z caller=bucket.go:325 msg="error parsing block range" block=01DKX7NK7VBWTP461Z2MREK6RA err="read meta.json: open /appl/thanos/data-cache/01DKX7NK7VBWTP461Z2MREK6RA/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.667483023Z caller=bucket.go:325 msg="error parsing block range" block=01DKX8H4SRSHV2CGDD92MVYGK3 err="read meta.json: open /appl/thanos/data-cache/01DKX8H4SRSHV2CGDD92MVYGK3/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.667571432Z caller=bucket.go:325 msg="error parsing block range" block=01DKX97Z7JQAF6TWM60FH4YQRA err="read meta.json: open /appl/thanos/data-cache/01DKX97Z7JQAF6TWM60FH4YQRA/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.6774169Z caller=bucket.go:325 msg="error parsing block range" block=01DKXATRSXTFZNA05WH09SQKGF err="read meta.json: open /appl/thanos/data-cache/01DKXATRSXTFZNA05WH09SQKGF/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.6784887Z caller=bucket.go:325 msg="error parsing block range" block=01DKXBP3MG04T0Q3VHYKFE1A4K err="read meta.json: open /appl/thanos/data-cache/01DKXBP3MG04T0Q3VHYKFE1A4K/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.678567294Z caller=bucket.go:325 msg="error parsing block range" block=01DKXCCXSFJKJJZYKFJQ856C2Z err="read meta.json: open /appl/thanos/data-cache/01DKXCCXSFJKJJZYKFJQ856C2Z/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.6800205Z caller=bucket.go:325 msg="error parsing block range" block=01DKXD8JHY0D6G85QDS32E0SJC err="read meta.json: open /appl/thanos/data-cache/01DKXD8JHY0D6G85QDS32E0SJC/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.680094276Z caller=bucket.go:325 msg="error parsing block range" block=01DKXE0858KVS3S99C0ZPK2TTF err="read meta.json: open /appl/thanos/data-cache/01DKXE0858KVS3S99C0ZPK2TTF/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.681733611Z caller=bucket.go:325 msg="error parsing block range" block=01DKXEWKHWE3EYHRWNWKFEJVFD err="read meta.json: open /appl/thanos/data-cache/01DKXEWKHWE3EYHRWNWKFEJVFD/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.681820288Z caller=bucket.go:325 msg="error parsing block range" block=01DKXFKQ2X2FDRF4GF9AEPMVE1 err="read meta.json: open /appl/thanos/data-cache/01DKXFKQ2X2FDRF4GF9AEPMVE1/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.683000294Z caller=bucket.go:325 msg="error parsing block range" block=01DKXG0GB0VK3SA10Y3SVPKX9Q err="read meta.json: open /appl/thanos/data-cache/01DKXG0GB0VK3SA10Y3SVPKX9Q/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.683101741Z caller=bucket.go:325 msg="error parsing block range" block=01DKXGG5SSD0VNK42ZXMV18R2Q err="read meta.json: open /appl/thanos/data-cache/01DKXGG5SSD0VNK42ZXMV18R2Q/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.685522779Z caller=bucket.go:325 msg="error parsing block range" block=01DKXH8ETY19VS5CVMW4M35Y47 err="read meta.json: open /appl/thanos/data-cache/01DKXH8ETY19VS5CVMW4M35Y47/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.687184281Z caller=bucket.go:325 msg="error parsing block range" block=01DKXJ496KH0PKWXPCRJQMVB95 err="read meta.json: open /appl/thanos/data-cache/01DKXJ496KH0PKWXPCRJQMVB95/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.687248467Z caller=bucket.go:325 msg="error parsing block range" block=01DKXJV8G3G0T2TZDQS9D90RWF err="read meta.json: open /appl/thanos/data-cache/01DKXJV8G3G0T2TZDQS9D90RWF/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.687490253Z caller=bucket.go:325 msg="error parsing block range" block=01DKXK8HG3BBBC5A78762G28V7 err="read meta.json: open /appl/thanos/data-cache/01DKXK8HG3BBBC5A78762G28V7/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.687526483Z caller=bucket.go:325 msg="error parsing block range" block=01DKXKQTF87NP5D7CV6FVCGBG5 err="read meta.json: open /appl/thanos/data-cache/01DKXKQTF87NP5D7CV6FVCGBG5/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.687551313Z caller=bucket.go:325 msg="error parsing block range" block=01DKXME248RYMBZYHSR72EWFJY err="read meta.json: open /appl/thanos/data-cache/01DKXME248RYMBZYHSR72EWFJY/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.688002804Z caller=bucket.go:325 msg="error parsing block range" block=01DKXNAQFVVJZ975CJS8DQJCVY err="read meta.json: open /appl/thanos/data-cache/01DKXNAQFVVJZ975CJS8DQJCVY/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.688040545Z caller=bucket.go:325 msg="error parsing block range" block=01DKXP14JD4P862G7Z33G7S552 err="read meta.json: open /appl/thanos/data-cache/01DKXP14JD4P862G7Z33G7S552/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.689238619Z caller=bucket.go:325 msg="error parsing block range" block=01DKXPX990NN6YVTA0YAR5BWDC err="read meta.json: open /appl/thanos/data-cache/01DKXPX990NN6YVTA0YAR5BWDC/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.68928131Z caller=bucket.go:325 msg="error parsing block range" block=01DKXQKKC38ZDRKMWJ6Z8E4R5K err="read meta.json: open /appl/thanos/data-cache/01DKXQKKC38ZDRKMWJ6Z8E4R5K/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.692860232Z caller=bucket.go:325 msg="error parsing block range" block=01DKXRFZWVHH6AT74XBFV9FF24 err="read meta.json: open /appl/thanos/data-cache/01DKXRFZWVHH6AT74XBFV9FF24/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.692919257Z caller=bucket.go:325 msg="error parsing block range" block=01DKXS736C8A2X4K1EV4GKG2WQ err="read meta.json: open /appl/thanos/data-cache/01DKXS736C8A2X4K1EV4GKG2WQ/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.694294313Z caller=bucket.go:325 msg="error parsing block range" block=01DKXSKQMY1GCGZK9BCF00VHTQ err="read meta.json: open /appl/thanos/data-cache/01DKXSKQMY1GCGZK9BCF00VHTQ/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.694348663Z caller=bucket.go:325 msg="error parsing block range" block=01DKXT3VW53TK5Y399NXR7WVZJ err="read meta.json: open /appl/thanos/data-cache/01DKXT3VW53TK5Y399NXR7WVZJ/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.694399894Z caller=bucket.go:325 msg="error parsing block range" block=01DKXTSR486MKAFX7KSSDCARDJ err="read meta.json: open /appl/thanos/data-cache/01DKXTSR486MKAFX7KSSDCARDJ/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.695490936Z caller=bucket.go:325 msg="error parsing block range" block=01DKXVP4DBSGSDTTT7PW77K836 err="read meta.json: open /appl/thanos/data-cache/01DKXVP4DBSGSDTTT7PW77K836/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.69554648Z caller=bucket.go:325 msg="error parsing block range" block=01DKXWD00FR17MKB8813X1WEWQ err="read meta.json: open /appl/thanos/data-cache/01DKXWD00FR17MKB8813X1WEWQ/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.695583925Z caller=bucket.go:325 msg="error parsing block range" block=01DKXWPNW2PSM79076BP1VFVBV err="read meta.json: open /appl/thanos/data-cache/01DKXWPNW2PSM79076BP1VFVBV/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.69633682Z caller=bucket.go:325 msg="error parsing block range" block=01DKXX8Q9EY4YZ2Q0W9TRXNNH0 err="read meta.json: open /appl/thanos/data-cache/01DKXX8Q9EY4YZ2Q0W9TRXNNH0/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.696397194Z caller=bucket.go:325 msg="error parsing block range" block=01DKXY08X7ZP15FPPHV5KKYBG9 err="read meta.json: open /appl/thanos/data-cache/01DKXY08X7ZP15FPPHV5KKYBG9/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.697324163Z caller=bucket.go:325 msg="error parsing block range" block=01DKXYDMYPFTJSZQWD0S6Z06PT err="read meta.json: open /appl/thanos/data-cache/01DKXYDMYPFTJSZQWD0S6Z06PT/meta.json: no such file or directory"
Sep  4 13:04:03 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:03.759713191Z caller=bucket.go:325 msg="error parsing block range" block=01DKXYX6PHKZ94564H3BKHSZRT err="download meta.json: get file: The specified key does not exist."
Sep  4 13:04:04 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:04.535192631Z caller=storeset.go:223 component=storeset msg="updating healthy stores" externalLabelOccurrencesInStores="map[string]int{\"\":1, \"{prometheus_server=\\\"p1-prom-mlsu001\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu002\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu101\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu102\\\"}\":1}"
Sep  4 13:04:05 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:05.721605303Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 filtered out"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.36082669Z caller=proxy.go:265 msg="store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 queried;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.361426887Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=458 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1556690400000-1567584000000 Resolution: 0"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.363022904Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=388 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1556776800000-1567584000000 Resolution: 0"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.364267779Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=232 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1563840000000-1567591200000 Resolution: 0"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.364869119Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=233 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1563840000000-1567591200000 Resolution: 0"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.368839678Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:1311 postingsTouched:0 postingsTouchedSizeSum:0 postingsToFetch:0 postingsFetched:0 postingsFetchedSizeSum:0 postingsFetchCount:0 postingsFetchDurationSum:0 seriesTouched:0 seriesTouchedSizeSum:0 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:0 chunksTouchedSizeSum:0 chunksFetched:0 chunksFetchedSizeSum:0 chunksFetchCount:0 chunksFetchDurationSum:0 getAllDuration:3159031 mergedSeriesCount:0 mergedChunksCount:0 mergeDuration:161251}" err=null
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.607700923Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567584000000 queried;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.608107262Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=458 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1556690400000-1567584000000 Resolution: 0"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.6095158Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=388 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1556776800000-1567584000000 Resolution: 0"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.610668258Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=232 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1563840000000-1567591200000 Resolution: 0"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.611323305Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=233 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1563840000000-1567591200000 Resolution: 0"
Sep  4 13:04:07 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:07.615449801Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:1311 postingsTouched:0 postingsTouchedSizeSum:0 postingsToFetch:0 postingsFetched:0 postingsFetchedSizeSum:0 postingsFetchCount:0 postingsFetchDurationSum:0 seriesTouched:0 seriesTouchedSizeSum:0 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:0 chunksTouchedSizeSum:0 chunksFetched:0 chunksFetchedSizeSum:0 chunksFetchCount:0 chunksFetchDurationSum:0 getAllDuration:3523156 mergedSeriesCount:0 mergedChunksCount:0 mergeDuration:137761}" err=null
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.542597492Z caller=storeset.go:223 component=storeset msg="updating healthy stores" externalLabelOccurrencesInStores="map[string]int{\"\":1, \"{prometheus_server=\\\"p1-prom-mlsu001\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu002\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu101\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu102\\\"}\":1}"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.934740449Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567591200000 queried;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.935459551Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595160000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.935571154Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595160000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.935620727Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595160000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.935666245Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595160000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.967809946Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567591200000 queried"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.968355993Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595160000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.968481427Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595160000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.968548639Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595160000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.968588354Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595160000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.978423886Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:6 postingsTouched:4 postingsTouchedSizeSum:2480 postingsToFetch:0 postingsFetched:0 postingsFetchedSizeSum:0 postingsFetchCount:0 postingsFetchDurationSum:0 seriesTouched:2 seriesTouchedSizeSum:144 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:2 chunksTouchedSizeSum:909 chunksFetched:2 chunksFetchedSizeSum:32000 chunksFetchCount:2 chunksFetchDurationSum:80589161 getAllDuration:42625891 mergedSeriesCount:2 mergedChunksCount:2 mergeDuration:42707}" err=null
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.981945814Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567591200000 queried"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.982554651Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595055000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.982654482Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595055000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.982699082Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595055000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:09 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:09.982747247Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595055000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.016340826Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:6 postingsTouched:4 postingsTouchedSizeSum:2480 postingsToFetch:0 postingsFetched:0 postingsFetchedSizeSum:0 postingsFetchCount:0 postingsFetchDurationSum:0 seriesTouched:2 seriesTouchedSizeSum:144 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:2 chunksTouchedSizeSum:261 chunksFetched:2 chunksFetchedSizeSum:32000 chunksFetchCount:2 chunksFetchDurationSum:84542553 getAllDuration:47616219 mergedSeriesCount:2 mergedChunksCount:2 mergeDuration:49307}" err=null
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.065048993Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567591200000 queried;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.065805146Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595070000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.065981684Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595070000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.0660504Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595070000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.066090992Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595070000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.098948841Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:6 postingsTouched:40 postingsTouchedSizeSum:9256 postingsToFetch:0 postingsFetched:0 postingsFetchedSizeSum:0 postingsFetchCount:0 postingsFetchDurationSum:0 seriesTouched:34 seriesTouchedSizeSum:2856 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:34 chunksTouchedSizeSum:4440 chunksFetched:34 chunksFetchedSizeSum:48915 chunksFetchCount:2 chunksFetchDurationSum:60042460 getAllDuration:32608881 mergedSeriesCount:34 mergedChunksCount:34 mergeDuration:185313}" err=null
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.219408921Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:6 postingsTouched:40 postingsTouchedSizeSum:9864 postingsToFetch:0 postingsFetched:0 postingsFetchedSizeSum:0 postingsFetchCount:0 postingsFetchDurationSum:0 seriesTouched:34 seriesTouchedSizeSum:2856 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:34 chunksTouchedSizeSum:4480 chunksFetched:34 chunksFetchedSizeSum:49248 chunksFetchCount:2 chunksFetchDurationSum:281613520 getAllDuration:236330596 mergedSeriesCount:34 mergedChunksCount:34 mergeDuration:188678}" err=null
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.505663312Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567591200000 queried;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.506087566Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595060000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.506220789Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595060000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.506327288Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595060000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.506377206Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595060000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.513850756Z caller=proxy.go:265 msg="store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567591200000 queried;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.514514925Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583900000 maxt=1567595200000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.514605393Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583900000 maxt=1567595200000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.514663865Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583900000 maxt=1567595200000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.514750824Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583900000 maxt=1567595200000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.590729551Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:6 postingsTouched:8 postingsTouchedSizeSum:92328 postingsToFetch:0 postingsFetched:2 postingsFetchedSizeSum:33160 postingsFetchCount:2 postingsFetchDurationSum:73810116 seriesTouched:12 seriesTouchedSizeSum:1020 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:12 chunksTouchedSizeSum:2469 chunksFetched:12 chunksFetchedSizeSum:38273 chunksFetchCount:2 chunksFetchDurationSum:159309916 getAllDuration:84120099 mergedSeriesCount:12 mergedChunksCount:12 mergeDuration:146592}" err=null
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.610679696Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:6 postingsTouched:8 postingsTouchedSizeSum:92328 postingsToFetch:0 postingsFetched:2 postingsFetchedSizeSum:33160 postingsFetchCount:2 postingsFetchDurationSum:138558400 seriesTouched:12 seriesTouchedSizeSum:1020 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:12 chunksTouchedSizeSum:1578 chunksFetched:12 chunksFetchedSizeSum:37314 chunksFetchCount:2 chunksFetchDurationSum:166602806 getAllDuration:95761588 mergedSeriesCount:12 mergedChunksCount:12 mergeDuration:94376}" err=null
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.656010866Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567591200000 queried"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.656460737Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595080000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.656547587Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595080000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.656595884Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595080000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.656693012Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595080000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.696182006Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567591200000 queried;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.696698902Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595060000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.696814577Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=1 MaximumResolution=0 mint=1567583940000 maxt=1567595060000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1567576800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.696866799Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595060000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.696920254Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=2 MaximumResolution=0 mint=1567583940000 maxt=1567595060000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1567576800000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.747435031Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567591200000 queried;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.748106809Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=458 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu002\"}" spans="Range: 1556690400000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.749503802Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=388 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu001\"}" spans="Range: 1556776800000-1567584000000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.750819581Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=232 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu101\"}" spans="Range: 1563840000000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.751536348Z caller=bucket.go:771 msg="Blocks source resolutions" blocks=233 MaximumResolution=0 mint=-62167219200000 maxt=253402300799000 lset="{prometheus_server=\"p1-prom-mlsu102\"}" spans="Range: 1563840000000-1567591200000 Resolution: 0"
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.790720575Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:6 postingsTouched:8 postingsTouchedSizeSum:33096 postingsToFetch:0 postingsFetched:2 postingsFetchedSizeSum:8808 postingsFetchCount:2 postingsFetchDurationSum:97910953 seriesTouched:10 seriesTouchedSizeSum:850 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:10 chunksTouchedSizeSum:1334 chunksFetched:10 chunksFetchedSizeSum:559184 chunksFetchCount:2 chunksFetchDurationSum:148653400 getAllDuration:93655875 mergedSeriesCount:10 mergedChunksCount:10 mergeDuration:80011}" err=null
Sep  4 13:04:10 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:10.901238208Z caller=bucket.go:871 msg="stats query processed" stats="&{blocksQueried:6 postingsTouched:14 postingsTouchedSizeSum:260248 postingsToFetch:0 postingsFetched:2 postingsFetchedSizeSum:62168 postingsFetchCount:2 postingsFetchDurationSum:145268553 seriesTouched:6 seriesTouchedSizeSum:510 seriesFetched:0 seriesFetchedSizeSum:0 seriesFetchCount:0 seriesFetchDurationSum:0 chunksTouched:6 chunksTouchedSizeSum:783 chunksFetched:6 chunksFetchedSizeSum:34126 chunksFetchCount:2 chunksFetchDurationSum:289129437 getAllDuration:244399430 mergedSeriesCount:6 mergedChunksCount:6 mergeDuration:80737}" err=null
Sep  4 13:04:11 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:11.432347989Z caller=proxy.go:265 msg="store Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried;store Addr: localhost:7023 LabelSets:  Mint: 1556690400000 Maxt: 1567591200000 filtered out;store Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu001:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu001\" ] Mint: 1567144800000 Maxt: 9223372036854775807 queried;store Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807 queried"
Sep  4 13:04:14 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:14.534581499Z caller=storeset.go:223 component=storeset msg="updating healthy stores" externalLabelOccurrencesInStores="map[string]int{\"\":1, \"{prometheus_server=\\\"p1-prom-mlsu001\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu002\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu101\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu102\\\"}\":1}"
Sep  4 13:04:17 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:17.743340617Z caller=proxy.go:409 err="failed to receive any data from Addr: p1-prom-mlsu002:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu002\" ] Mint: 1567144800000 Maxt: 9223372036854775807: context canceled" msg="returning partial response"
Sep  4 13:04:17 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:17.743431999Z caller=proxy.go:409 err="failed to receive any data from Addr: p1-prom-mlsu101:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu101\" ] Mint: 1567152000000 Maxt: 9223372036854775807: context canceled" msg="returning partial response"
Sep  4 13:04:17 p1-thanos-alsu001 thanos: level=warn ts=2019-09-04T11:04:17.743451208Z caller=proxy.go:409 err="failed to receive any data from Addr: p1-prom-mlsu102:7020 LabelSets: [name:\"prometheus_server\" value:\"p1-prom-mlsu102\" ] Mint: 1567152000000 Maxt: 9223372036854775807: context canceled" msg="returning partial response"
Sep  4 13:04:19 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:19.534640935Z caller=storeset.go:223 component=storeset msg="updating healthy stores" externalLabelOccurrencesInStores="map[string]int{\"\":1, \"{prometheus_server=\\\"p1-prom-mlsu001\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu002\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu101\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu102\\\"}\":1}"
Sep  4 13:04:24 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:24.534704177Z caller=storeset.go:223 component=storeset msg="updating healthy stores" externalLabelOccurrencesInStores="map[string]int{\"\":1, \"{prometheus_server=\\\"p1-prom-mlsu001\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu002\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu101\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu102\\\"}\":1}"
Sep  4 13:04:29 p1-thanos-alsu001 thanos: level=debug ts=2019-09-04T11:04:29.53792269Z caller=storeset.go:223 component=storeset msg="updating healthy stores" externalLabelOccurrencesInStores="map[string]int{\"\":1, \"{prometheus_server=\\\"p1-prom-mlsu001\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu002\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu101\\\"}\":1, \"{prometheus_server=\\\"p1-prom-mlsu102\\\"}\":1}"

Anything else we need to know
If this is of relevance: We do not downsample the data and we keep it for 7 months.
Thanos Compact Systemd startup command (on same server):
/appl/thanos/bin/thanos compact --data-dir=/appl/thanos/data-compactor --http-address=0.0.0.0:7008 --log.level=debug --objstore.config-file=/appl/thanos/.s3.yml --retention.resolution-raw=210d --wait

When querying
rate(thanos_objstore_bucket_operations_total{instance=~"p1-thanos-alsu001.*7007"}[30s])

I've around ~700/s, nothing near 2000/s.
When doing a tcpdump (sadly not at 13:04) I see lots of TCP ACK with NOOP & Timestamp Options, but no content really. Not sure, if those connections are causing the problem...?
Have you heard/encountered something familiar?
I'm not really sure how to further investigate. If you need further data, please tell me so and I'll see to it.
Environment:

OS (e.g. from /etc/os-release): RHEL7.6
Kernel (e.g. uname -a): Linux p1-thanos-alsu001 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 15 17:36:42 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Others:

Hi, thanks for looking into it. In fact I had a lot of those message. I have now doubled both values and set:
--index-cache-size=500MB --chunk-pool-size=4GB

I once read in slack:

sum it up (how large index-cache is for biggest blocks for 2w and assume how many you want to cache) and well -> you have the value

But I don't really know where I can see the size of the index-cache for biggest blocks. So I'm just going with the try and fail method.

That seems like a pretty good rule of thumb. They are talking about the sizes of index.cache.json files that are in the remote object storage.
Did increasing its size help you with this problem?

Hi, yes it seems setting higher values for caching did indeed help with this issue.
Thank you very much for helping. If you're ever in Switzerland, I'll invite you to some drinks 🍻

authors: GiedriusS muryoutaisuu

repo: thanos-io/thanos


text:
Proposal: Shortcut access for specific URLs

One minor change I've made is the ability to bind shortcuts to specific tab URLs.  This allows you to use ⌘⇧S for instance to open the most recent tab related to Slack (app.slack.com).  Right now the URL is lower-cased and then partially matched.
Would this be a feature everyone would want?  If so, I can package this up in a more user-friendly way and deliver it fairly quickly.
Assigning multiple shortcuts...

Assigning a partial URL for matching...

Please leave a 👍🏼 if this is a feature you would want.

source: gharchive/issue

repo: thatguychrisw/alfred-browser-tabs

license: MIT

license_type: permissive

license_source: github-api


text:
Add to Bower

Could you make Scalem a Bower package?
http://bower.io/#defining-a-package

You can now install through npm by typing npm install scalem.

I think it would make sense to make it a Bower package as well, not just npm: http://bower.io/docs/creating-packages/
This would make it usable via https://rails-assets.org/ as well.

OK, let me look into Bower since I haven't used it myself.

:+1:

You can now bower install scalem :-)

Sweet!

authors: manuelmeurer thdoan

repo: thdoan/scalem

license: mit

license_source: bigquery


text:
php artisan migrate:refresh returns error on fresh install

Laravel Version: 5.5
Voyager Version: 1.0
PHP Version:
Database Driver & Version:

Description:
After a fresh install of Laravel and Voyager, if I try to run php artisan migrate:refresh it returns error.
php artisan migrate:refresh --seed

  [Doctrine\DBAL\DBALException]
  Unknown database type enum requested, Doctrine\DBAL\Platforms\MySQL57Platform may not support it.

I ran into the same issue, but I managed to fix it by modifying a hack someone else made in this comment.
Adding this to my AppServiceProvider's boot method fixed it for me.
Schema::getConnection()->getDoctrineSchemaManager()->getDatabasePlatform()->registerDoctrineTypeMapping('enum', 'blob');

@marktopper, is this worth adding to the ServiceProvider in https://github.com/larapack/doctrine-support? Not sure if you already have something there or if it needs modifying

I will get this added to the Doctrine Support service provider.

This have been fixed in larapack/doctrine-support:v0.1.4.
https://github.com/larapack/doctrine-support/commit/02c9fc73324d9111bb03bb8b087653afde8972e0

Any word on this issue. Seems to still not be working correctly. Still getting "This type is not supported".

Not sure what word you're looking for.  The issue was closed 22 days ago with a comment of "please run composer update ...."

I did the composer update larapack/doctrine-support but still get the error "generic.exception: Enum type is not supported", and I am unable to add any new columns to a table that uses ENUM.

authors: BeetleJ adddz fletch3555 marktopper mortenscheel

repo: the-control-group/voyager


text:
Media Manager different behaviour from Post/Page upload

Laravel Version: 1.0.5
Voyager Version: 5.5.14
PHP Version: 7.1.7
Database Driver & Version: postgres 9.5.7

Description:
Uploading a .jpg that is originally 1,749kb  coverts the file to a .jpeg that is now 3,958kb
This is counter intuitive for a media manager. Why is it changing the file type? Why is it making the file larger rather than applying some sort of compression? Is there a way I can modify this behaviour?
Question 2 - Why are you hashing the file name? File names matter for SEO, I understand I can change it afterwards, but again, counter intuitive and something I'd have to train my users to do.
This only happens when working with the Media Manager upload itself. Uploading via a Post or Page significantly reduces the image size and does not change the file name.
Steps To Reproduce:
Upload a .jpg file through the Media Manager (not through a Post or Page interface)

This will be possible in version 1.2.
See (https://github.com/the-control-group/voyager/pull/3889).


text:
Large Image Uploads Being Scaled

Laravel Version: 5.4.36
Voyager Version: 1.0.5
PHP Version: 7.1.5
Database Driver & Version:

Description:
Large images that are uploaded are being scaled to have a width of 1800px. Is it possible to disable this? I am scaling very large images and it is degrading the quality considerably. In my opinion uploaded images should be left as is.
I've tried to disable it by setting width to null with;
{
	"quality": "100%",
	"resize": {
		"width": null,
		"height": null
	},
	"thumbnails": [
		{
			"name": "thumbnail",
			"crop": {
				"width": 200,
				"height": 40
			}
		}
	],
	"display": {
		"width": 6
	}
}

which doesn't work...
Steps To Reproduce:
Upload a large image

The same problem. Couldn't find an easy way to disable / change max width.

The Voyager Controller runs a check with isset() on the resize width and height: https://github.com/the-control-group/voyager/blob/v1.0.14/src/Http/Controllers/Controller.php#L334
isset() will return false if the value is null as well. So set the width and height to false, or a large number, and that should get you by.

Also, there's an open PR that should fix this.

authors: Gibbs Neutronas fletch3555 kalenjohnson


text:
[BUG] letters are showing instead of icons in voyager admin panel

Version information

Laravel: v5.8.*
Voyager: v^1.2
PHP: ^7.1.3
Database: MySQL v14.14

Description

The icons in voyager admin panel are not showing up. All I can see is letters like k, M, f etc. The images are showing up normally though.
Steps To Reproduce
Steps to reproduce the behavior:

Create new laravel app and install voyager.
Set up .env file, set app_url to http://localhost:8000
Login to voyager admin panel
Icons are not showing, everything else works.

Expected behavior
I expected to see icons in the admin panel
Screenshots

I solved it from here,
https://github.com/the-control-group/voyager/issues/4130


text:
Fix Select2 search on additional attribute

Use collection filter when an additional attribute is used as label.
Fixes #4561

Hello @MrCrayon
thanks for the PR.
I tested your code and now the oposite situation is not returning any results after trying  to filter select2.
Let me explain better.
After implement your code a select2 combo with a custom attribute is ok, but a select2 combo with a regular model collumn (single collumn) isn't returning results.
Thanks man.

@livrosroberto you are right, fixed. Thanks

Thanks @MrCrayon & @fletch3555

source: gharchive/pull-request

authors: DaLooK MrCrayon livrosroberto


text:
v2 release on @geit/goat as an install wrapper to renamed v2

For backwards dependency / easy update...
^2 wrapper => dependencies => the-goat/cli^2 + misc that was required in v1 and no longer in v2... With a deprecated warning to use @the-goat/cli
This entails you can still run npm i -g @geit/goat and get the latest (the-goat/cli@2) version..

Ah I have understood (by @Reggino) v2 is no longer actually active :-D

repo: the-goats/Goat


text:
Has flea issue with TarkovTraker

Describe the Issue
On the loot tiers page, when you filter for guns (as an example) and navigate to an item (a weapon in this case) and then back to the loot tiers page the guns that were advised to sell on flea market become all sell to trader.
Expected Behavior
Guns keep the recommendation to be sold on the flea market
To Reproduce
premises: You must be level 15+ on TarkovTraker to have flea unlocked, then:

Navigate with the top menu to settings
Switch TarkovTraker off
Switch Has Flea off
Switch TarkovTraker on
Navigate with the top menu to Loot tiers
Click on any item that is sell to flea market (filter for only guns for better clarity)
Navigate with the top menu to Loot tiers

Observe that all items are now sell to trader
If you now navigate with the top menu to settings you will see that, although greyed, the has flea switch is in off state and only after a few moments becomes on, this may be a clue to where the issue is.
Client
Desktop
Browser
Chrome
Relevant Console Log Output
No response
Extra Information
No response

This might be resolved in #253. I changed the way the item links work on the loot tier page so that it no longer causes the entire site to reload. Not sure if that solves it or not though.

Navigation with top menu is now seems fixed. But now if you reload the loot tier page the same happens: all items are now sell to trader

I think this issue is related to the fact that TarkovTracker progress was only updating if the user was on the settings or crafts pages. I've changed things so it updates every 30 seconds no matter which page the user is viewing. If all that is correct, then this issue should be resolved with #255, which is currently viewable here: https://better-quests-table.tarkov-dev.pages.dev/

Can confirm, this is due to the delay we have set for updating tarkov tracker progress (currently 5 minutes). If you sit on the loot tiers page for 5 minutes, you'll see that once tarkov tracker progress automatically updates, all the flea prices show up.

authors: Razzmatazzz Shebuka

repo: the-hideout/tarkov-dev


text:
403 Forbidden

when i try to search for a song; it suggests an open spotify app to play on as following:

and when i press enter this message occurs

OS: Elementary OS Hera 5.1.7 based on Ubuntu 18.04

First of all, thank you for reporting the bug!
There can be multiple problems here.
It seems the Spotify is running and API sees it, but it was not successful at starting the playback.
It is very tough to diagnose this bug, but my guess is that it is on the API side, not ours.
The immediate fix would probably be to just restart the client on your laptop and try the same again.
Please try and let me know if that helped :)
Also, just a small note, the search keyword is sp search QUERY, not  sp QUERY - check out sp help or sp ? for more info about that.

First of all, thank you for reporting the bug!
There can be multiple problems here.
It seems the Spotify is running and API sees it, but it was not successful at starting the playback.
It is very tough to diagnose this bug, but my guess is that it is on the API side, not ours.
The immediate fix would probably be to just restart the client on your laptop and try the same again.
Please try and let me know if that helped :)
Also, just a small note, the search keyword is sp search QUERY, not  sp QUERY - check out sp help or sp ? for more info about that.

Sorry for the extremly late reply.
I tried every possible solution to make it work but i failed. the same error prevailed. However, the searching sp search <query> gave me results as the following example.

It does not permit me to play (it returns 403 forbidden error).

Sorry for the extremly late reply.
I tried every possible solution to make it work but i failed. the same error prevailed. However, the searching sp search <query> gave me results as the following example.

It does not permit me to play (it returns 403 forbidden error).

unfortunately, you are right.... i tried to help you and debug it and that is what i found Issue on spotify developer community

unfortunately, you are right.... i tried to help you and debug it and that is what i found Issue on spotify developer community

Sorry for the extremly late reply.

No worries!
Sorry, I've assumed you have a premium account. As mentioned in the Troubleshooting section of README, play endpoint is only for paid Spotify subscribers. Unfortunately, that's a Spotify limitation.

Sorry for the extremly late reply.

No worries!
Sorry, I've assumed you have a premium account. As mentioned in the Troubleshooting section of README, play endpoint is only for paid Spotify subscribers. Unfortunately, that's a Spotify limitation.

repo: the-lay/ulauncher-spotify-api


text:
Import highlights not working

Logseq version: 0.7.5
plugin version: 1.16.0
Clicking on "Load Clippings File" opens up the file picker, but no action afterwards when the selection is done.

Can confirm this bug on both linux & windows, logseq: 0.7.9

I'm able to reproduce now, so hopefully I can get it fixed.

@theBenForce Problem's still there. Even have tried reload, reinstall the plugin, turn off other plugins. Here's the gif:

Plus: The plugin was worked early before(when I first time install), so that maybe could rule out the file system problem(where I storage the MyClipping.txt)and even the format recognize problem? Just in case, here's a screenshot about the format inside my clipping flie. Though it's in Chinese it should be has no difference with English version.

@Gitbreast could you share a few highlights from your clippings file? That may help me reproduce the issue.

@theBenForce Sure! Here's the end part of my clippings. My Clippings part.txt
How I got this file is: I copy the original file from my KP3, then delete about 9/10 of it, then ctrl+s and rename.
So that avoid changing the encode or something like that. BTW now it's this:

Thank you! I found the issue, it's because I'm looking for markers like "Location" and "Added on". I'll try to come up with a more robust way of parsing the metadata, but for now I'll just make sure it can parse without finding the metadata.

authors: Gitbreast petrushy theBenForce thelazyoxymoron

repo: theBenForce/logseq-plugin-my-highlights


text:
[WIP] refactor: use base queries to inherit

I made a separate kind for each braces so we can easily combine them when inheriting. Let me know what you think.

okay, I think I'm done.

Thanks a lot for the work!

authors: elianiva theHamsta

repo: theHamsta/nvim-treesitter-pairs


text:
Field nullable union type is throwing an error

Having a type like this:
#[Type]
class File {
	#[Field(name: 'sizes')]
	public function sizes(): ImageSizes|VideoSizes|null {}
}

throws an error saying null can't be converted to a GraphQL type. GraphQL spec does allow nullable unions, so this seems to be a bug with graphqlite.
For the above function phpDocumentor reports the following type:
new Nullable(
	realType: new Compound(
		types: [
			new Object_(),
			new Object_(),
			new Null_(),
		]
	)
)

so when NullableTypeMapperAdapter attempts to get a non-nullable version of that it goes through https://github.com/thecodingmachine/graphqlite/blob/master/src/Mappers/Root/NullableTypeMapperAdapter.php#L129
and returns the Compound type instance with Null_ being one of the types.
Fix should be trivial: return return $this->getNonNullable($type->getActualType()) instead of return $type->getActualType()
I can PR this change.

Seems like a bug - yea.  Since union types are a newer PHP feature, that was never fully incorporated.  A PR would be great, just need to make sure we have test coverage.

authors: oojacoboo oprypkhantc

repo: thecodingmachine/graphqlite


text:
Button to trigger modal links to the wrong modal ID (bs v4)

First of all thanks for this amazing project! Finally I can drop the jQuery library.
On the example page  the data-target of the Modal is linking to the wrong ID.

Ah yes, thanks. Will make required changes asap.

Updated, please check the new demo page and the new Wiki pages!

authors: Vuurvlieg thednp


text:
Small typo fixed

There was a small typo on line 24 $port was 25567 (not the default) I've changed $port to the default RCON value in Minecraft (25575) :)

Thanks!

authors: ImmanuelNL thedudeguy

repo: thedudeguy/PHP-Minecraft-Rcon


text:
Review SMAST resources chapter

Gavin can please review SMAST resources chapter.

Merged content into Chapter 12: UMassD Resources.

authors: AshleighNovak gavinfay

repo: thefaylab/lab-manual


text:
Create countdown timer

Countdown timer in kettle, printer, and shredder button.

Closed by #253.

repo: theglus/Home-Assistant-Config


text:
Refactor numerical/non numerical column

Is your feature request related to a problem? Please describe.

I noticed an inconsistency in the current codebase (mostly due to the uns entry of (non) numerical columns got introduced later on). Sometimes, there is custom code checking for numerical columns in different ways, which is now obviously obsolete.
Describe the solution you would like

Refactor code to use uns entries.

Seems like thats already fixed.

repo: theislab/ehrapy


text:
scGEN network.batch_removal error

Hi,
any ideas how to solve this issue?
adata format looks like following

Here are commands I ran,
adata_hvg = adata[:, adata.var["highly_variable"]].copy()
adata_hvg = remove_sparsity(adata_hvg)
adata_hvg.X = adata_hvg.X.astype('float32')
network = sca.models.scgen(adata = adata_hvg,
                           hidden_layer_sizes=[256,128])

network.train(n_epochs=epoch, early_stopping_kwargs = early_stopping_kwargs)
corrected_reference_adata = network.batch_removal(adata_hvg, batch_key="sample", cell_label_key="celltype",return_latent=True)
The error occured at the last command.

TypeError                                 Traceback (most recent call last)
Cell In[83], line 1
----> 1 corrected_reference_adata = network.batch_removal(adata_hvg, batch_key="sample", cell_label_key="celltype",return_latent=True)
File ~/anaconda3/envs/scarches/lib/python3.9/site-packages/scarches/models/scgen/vaearith_model.py:147, in scgen.batch_removal(self, adata, batch_key, cell_label_key, return_latent)
127 def batch_removal(self, adata, batch_key, cell_label_key, return_latent = True):
128     """
129     Removes batch effect of adata
130
(...)
145         adata of corrected gene expression in adata.X and corrected latent space in adata.obsm["latent_corrected"].
146     """
--> 147     corrected = self.model.batch_removal(adata, batch_key, cell_label_key, return_latent)
148     return corrected
File ~/anaconda3/envs/scarches/lib/python3.9/site-packages/scarches/models/scgen/vaearith.py:261, in vaeArith.batch_removal(self, adata, batch_key, cell_label_key, return_latent)
259 adata_latent = anndata.AnnData(latent_all)
260 adata_latent.obs = adata.obs.copy(deep=True)
--> 261 unique_cell_types = np.unique(adata_latent.obs[cell_label_key])
262 shared_ct = []
263 not_shared_ct = []
File ~/anaconda3/envs/scarches/lib/python3.9/site-packages/numpy/lib/arraysetops.py:274, in unique(ar, return_index, return_inverse, return_counts, axis, equal_nan)
272 ar = np.asanyarray(ar)
273 if axis is None:
--> 274     ret = _unique1d(ar, return_index, return_inverse, return_counts,
275                     equal_nan=equal_nan)
276     return _unpack_tuple(ret)
278 # axis was specified and not None
File ~/anaconda3/envs/scarches/lib/python3.9/site-packages/numpy/lib/arraysetops.py:336, in unique1d(ar, return_index, return_inverse, return_counts, equal_nan)
334     aux = ar[perm]
335 else:
--> 336     ar.sort()
337     aux = ar
338 mask = np.empty(aux.shape, dtype=np.bool)
TypeError: '<' not supported between instances of 'float' and 'str'

adata_hvg.obs['celltype'] = adata_hvg.obs['celltype'].astype(str)
solved the issue

repo: theislab/scarches


text:
Improve README

I can't get the container to work. I already use jwilder's one on one of my servers.

TBH, I don't really use this all that much anymore. I previously had a single IP + single host SSL certificate and wanted to secure multiple docker containers behind it. The setup I use now no longer requires this project.
I found my old init files for the docker containers I used then. Here is the setup process for the nginx container:
docker run -d --name nginx-proxy-subdirectory --restart=always -p 19080:80 -p 19443:443 -e EXTERNAL_HTTPS_PORT=19443 -v /var/run/docker.sock:/tmp/docker.sock:ro -v /volume1/docker/nginx-proxy-subdirectory/certs:/etc/nginx/certs -v /volume1/docker/nginx-proxy-subdirectory/vhost.d:/etc/nginx/vhost.d theit8514/nginx-proxy-subdirectory
This container runs both a HTTP and HTTPS port, and the HTTP port redirects to 443 by default. The EXTERNAL_HTTPS_PORT setting changes the port that is used for the redirect (if you use NAT and don't use external port 443, then this should be whichever port you have your NAT set to).
Then for each of the containers hosting services, I would add environment variables to which subdirectory that container should be hosted at:
 -e VIRTUAL_PORT=8081 -e NGINX_PROXY_SUBDIRECTORY=my_directory -e NGINX_PROXY_SUBDIRECTORY_MAP_TO_ROOT=0
If I accessed the host via https://localhost:19443/my_directory/, nginx would proxy the request to the docker container that has NGINX_PROXY_SUBDIRECTORY=my_directory. The VIRTUAL_PORT setting determines which port to use on the container, and NGINX_PROXY_SUBDIRECTORY_MAP_TO_ROOT setting determines if the subdirectory is appended to the internal request. In this case, the request would be proxied to http://<docker container ip>:8081/my_directory/


text:
Compact Mode Socket Closed

[E] Connection lost -> socket is already closed. Probable cause: incorrect token.  is the error i get when I try to launch, I have tried getting a new token still doesnt work.

Hello @SpaceDevz , this error occurs only on compact mode ? Also, there is no much I can do about this exception, the description is "WebSocketConnectionClosedException -> If remote host closed the connection or some network error happened, this exception will be raised.". This means that either your request is being blocked by Discord's API (like ip ban, dns issue) or there is some internal problem on your network.
Try enabling debug on your config and run again to see if any logs show up, and check if you removed the quotation marks on the token. Anyways, there is no way I can reproduce your error on my environment since it depends exclusively on the remote host.
Since when this problem start to happen ?

Hey!
Figured it out I had quotes around my token, because it said str above so I assumed u needed quotes. Do i need quotes aroudn the bait too?

No, for each parameter, just paste the value there (as plain text, no quotes). But this misunderstanding is my fault, I should sanitize the token value before connecting to the webgate. I'll correct this on the next update.
Thanks for reporting.

authors: SpaceDevz thejoabo

repo: thejoabo/virtualfisher-bot


text:
Switch a power-off icon for logging out

The current icon is a lock, commonly used for encryption stuff. As a matter of fact, when @JocelynDelalande and I started to work on OTR support for Shout (TBC on The Lounge :-) ), we used the lock icons to signify that the session was encrypted, unencrypted, ...

Before
After

I like it, makes more sense to me. :+1:

Yeah, makes sense to me, too. :+1:

:+1:

authors: MaxLeiter YaManicKill astorije xPaw

repo: thelounge/lounge


text:
Don't Cache Errors

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
When the service my resolver is reaching out to returns an error or times out and the empty result response is cached. Any subsequent requests then fetch from cache instead of going back out the service to try again.
Looking into the Redis DB, the fcqs is pointing to an fqc entry that is empty or (nil).

Describe the solution you'd like
A clear and concise description of what you want to happen.
It would be great if there was some mechanism built in or option to let the cache resolver know that it should not cache error responses.
Additional context
Add any other context or screenshots about the feature request here.
Apollo has a wrapper they provide to their KeyValue store to achieve something similar. I tried wrapping this around the Redis Cache instance directly, but that did not work. Maybe there is a way to achieve something similar? https://www.npmjs.com/package/@apollo/utils.keyvaluecache#errorsaremissescache

Closing this issue. Handling the response and throwing an error in my resolver made it so the empty response was no longer cached.

repo: thematters/apollo-response-cache


text:
QuickSearch of User & Tag

Blockers

https://github.com/thematters/matters-server/issues/2973

Description
defined in https://www.notion.so/Search-Rework-c66bd1db9c9c49d8a5a4b22cff5244b7
re-use current graphql search interface with a new flag (version=new),
backend will call same path as before if called without this new flag;
and all other request & response shape will be same as before
{
  search(input:{key:"@user", type:User, version: "<new-version>"}) {
    totalCount
    pageInfo{hasNextPage}
    edges{
      cursor
      node{
        ... on User {
          userName displayName ...
        }
      }
    }
  }
}

References

Product doc
Design

depends on #2973

repo: thematters/matters-server


text:
关于CNAME的问题

使用这个action的话，每次push自动化部署之后，都要重新create CNAME一下吗，force push之后public下的CNAME文件刷掉了好像

而且，这样提交一次好像每一篇文章的更新时间都改变了

如果填写了 CNAME，这个 action 会帮你自动创建 CNAME

文章更新时间全部变成一样这个问题，参与 此文 解决。

了解了，感谢解答！！！

authors: Extious XPoet


text:
Why did you change colors?

Hi all, thanks for the great work.
Changing to tailwind merge and improving theming is nice! But why would you completely change the colors, especially when https://flowbite.com is all still blue not cyan?
Obviously, this will cause users to not update at all (I understand the lib is in active development). I am lazy to go now and retheme every checkbox or select through the whole project :) Is there any way blue styles are coming back?

Hey @rootical,
Thanks for the kind words!
It's based on our strategy to give Flowbite React a unique identity just the same way we did with:

Flowbite
[Flowbite Svelte)[http://flowbite-svelte.com/]

However, together with @tulup-conner and @rluders we've already been discussing a way to very easily update the primary colors from your talwind.config.js file from cyan to blue or to red or any other HEX or RGBA color of your choosing - this will be properly be documented as well.
Hopefully that makes sense! We should make switching between colors a breeze so in a way updating it back to blue should happen seamlessly without having to dig into all the cyan color codes.
One suggestion from me personally would be to simply find and replace all -cyan- tags to -blue-.
Cheers,
Zoltan

when in Rome do as the Romans do... but it should have been a patch release, not a minor :) And doing such a significant change would be great to have documented config for getting back and not breaking the whole color scheme with the release, not after.
Looking forward to knowing how to quickly get back the color scheme, thank you!

Yeah, this threw me off as well.
After updating the modules suddenly all my button colors changed, I mean wtf why? You shouldn't have changed that without providing an easy way to restore the previous default (which you now are still only discussing at it seems).

I think that we should have multiple themes included by "default" for the library.

Standard
React
Svelte
etc.

Opinions? Suggestions? How it can be done?

authors: rgwebcode rluders rootical zoltanszogyenyi

repo: themesberg/flowbite-react


text:
Playground for trying out the components

Summary
Would like to have a playground environment where we can experiment with Svelte components, we can use sandpack for building the playground. We can also used it for reporting the issues.
Motivation
I believe having a playground for trying out components would greatly benefit developers.

Please create a fork from this stackblitz.

repo: themesberg/flowbite-svelte