text:
Expose more scrape_config parameters via relabeling

As suggested by @jimmidyson we should be able to set scrape_interval, scrape_timeout, and authentication parameters through relabeling.
Essentially allowing full scrape configuration through service discovery mechanisms requires having full coverage. To be more adaptive to changes and extensions we should find a more generic approach for this as manual mapping is error prone.

I wonder here if we're better pointing such use cases towards file_sd and procedurally generating config files. Something like auth is hard going to be hard to get right from a security standpoint.

If I can configure my whole scrape_config via my SD but still have to complete it manually in a prometheus.yml I won little. That's not some extraordinary use case we have to provide a fallback solution for, it's fundamental to all SD mechanisms.

If I can configure my whole scrape_config via my SD but still have to complete it manually in a prometheus.yml I won little.

I'd disagree here, if you feel the need to have things with that level of dynamism I'd argue that you've failed to establish standards across your organisation and are likely overthinking your monitoring. I think we should be encouraging standardisation, rather than every time I come across a new job having to determine what that team thought was a good name instead of /metrics. Or having to figure out what the scrape interval was, which is important information for debugging that you want to change rarely.
There is such a thing as having too generic a solution.

Then I have to ask myself why we are supporting relabeling of, for example, the metrics path at all.
The only way it's different from scrape interval is, that while I actually do have a fixed path across my org, this is certainly not the case for my scrape interval.

I wonder that about the metric path myself, but see it as mostly harmless and there's cases like where you have one scrape config covering many jobs and a handful of them are exceptions.
Interval is more problematic, as depending on the setup you can violate implicit assumptions we or our user's expressions make about timestamps (see recent rate discussion for example, we presume scrape interval is a constant) or take out a Prometheus server though overload. Anyone doing this would have to take some care that they were sufficiently protected, and as this would all be opt-in that's kinda okay.
It does open us up to things like on-the-fly or procedural changing of interval and inconsistent intervals across jobs, which my instinct is to actively discourage as I can see that being both alluring and causing a lot of complexity for little gain.

The interval being constant is a wrong assumption in general already.
It would be good to recall why we added the schema and metrics path in the first place then. I assume there was some exotic use case. Them changing within a job is a fairly unusual and can be worked around otherwise.
Schema and paths are not in the general scope of service discovery either. So I would then change the proposal of this issue remove those as relabelable parts.
I suppose the general idea was that you can configure multiple jobs within the same scrape config and want the flexibility there. But that's of no use if that doesn't carry over to authentication and scrape timings.
Having an indecisive approach just complicates operation. Generally users can easily adapt to however it is supposed to be done – as long as there's a single straightforward way.

The interval being constant is a wrong assumption in general already.

That's a little worrying, as anyone changing intervals on the fly has likely greatly overengineered their monitoring.
I believe that a given Prometheus server should have one scrape interval, and on rare occasions also a slower scrape.

It would be good to recall why we added the schema and metrics path in the first place then.  I assume there was some exotic use case.

For scheme https://github.com/prometheus/prometheus/pull/967 which points to https://github.com/prometheus/prometheus/issues/910#issuecomment-128305407 which implies because metrics path was relabelable.
Metrics path is relabelable due to https://github.com/prometheus/prometheus/pull/654, which appears to be an incidental change.
So no exotic use cases, I'd guess it was just the way you happened to implement that change.

I suppose the general idea was that you can configure multiple jobs within the same scrape config and want the flexibility there.

I've used metric_path that way, though I could also have used two scrape configs. We do say that if it can be done with configuration management then you should rather than adding features. There is an efficiency argument here, as having many duplicate SDs may be a resource hog - though if that's true then it's not a very good SD method.

Having an indecisive approach just complicates operation. Generally users can easily adapt to however it is supposed to be done – as long as there's a single straightforward way.

I'm strongly against scrape interval being relabelable, against auth on the basis that I don't think we can make it work in a secure way, params needs to stay and I'm mostly ambivalent on the rest.
So if we're going for consistency, then that means I'm for dropping scheme and metrics path and that the way to approach this is multiple scrape configs. This'd also reduce the complexity of relabelling a tad, which never hurts.

If you take away relabelling of metrics path then you're enforcing a consistent metrics path on all discovered targets in a job. In the case of Kubernetes that would be a PITA as multiple services are deployed with different requirements & scraped using the same job, using Kubernetes annotations to configure stuff like metrics path & scheme.

That's a little worrying, as anyone changing intervals on the fly has likely greatly overengineered their monitoring.

Not saying that it is a problem – but it doesn't become any more problematic when setting it through relabeling. How often this changes is up to the user either way.

So no exotic use cases, I'd guess it was just the way you happened to implement that change.

Thanks for digging up the relevant parts.
It was implemented as a label more for internal reasons – back then there wasn't even relabeling. Then relabeling come into play. From IRC logs I infer a use case came up and I eventually pointed out it is somewhat possible via relabeling.
I guess the whole story of relabeling internal labels started from there. And it's not bad in general – but doing it half-way is.

I've used metric_path that way, though I could also have used two scrape configs. We do say that if it can be done with configuration management then you should rather than adding features. There is an efficiency argument here, as having many duplicate SDs may be a resource hog - though if that's true then it's not a very good SD method.

I cannot think of any SD that would consume a notable amount of resources right now.
Dropping support would result in requiring more than one scrape configuration. But that's also what you need for the other non-relabelable options right now.
Job names currently being unique though being relabelable to non-uniqueness is a problem right now anyway. Unfortunately with some internal complications – those should disappear with #1064.
@jimmidyson yes, but one has to get a setup in place to handle this anyway, right? If targets require different authentication or scrape intervals that is currently not solvable. And based on @brian-brazil's arguments, it shouldn't be.
With that the whole idea of having a single scrape configuration for an entire cluster would fall apart, which from my point of view, also makes __metrics_path__ and __scheme__ relatively obsolete.
As said, it's a 0 or 1 thing.

If you take away the ability to relabel target config then you're pushing people to make their own ways to generate the dynamic config, whatever config management tool they decide to use. I would personally introduce the chance for service discovery to be able to set all target config which Prometheus gives more flexibility without the need for users to each roll their own ways of doing dynamic config.
In my dream (& we're very close to now with Kubernetes SD) I want users to just deploy Prometheus, expose metrics & profit. I would love Prometheus to be this simple to deploy & only introduce complexity when it's needed, such as sharding, federation, etc.
IMO we should expand SD capabilities via direct target config (or labels if you'd prefer), not remove functionality. Seeing as target config is a fixed set, I'd personally go with direct target config rather than the relabelling approach as I'm not sure what this gains us.

How would such a direct target configuration look like, e.g. in the case of Kubernetes SD? Relabeling is very verbose, but relatively infrequently touched and very flexible. My initial ideas to handle it more explicitly were not promising at all (too rigid vs. high implementation cost for each SD), but maybe you got another angle here.
I agree with @brian-brazil on the problem of authentication parameters. Those are the most likely to be equal across services though.
My main point is that the whole idea falls apart if the scrape intervals are not supported. That's where I do not see as much of a problem as Brian does. Yes, it should not change often – but that's up the user and I don't see it being more or less volatile in a Kubernetes service config (as an example) or the Prometheus config.

These settings (https://github.com/prometheus/prometheus/blob/master/config/config.go#L335-L356) would be what I would consider consistent properties of a target.
Auth parameters need to be thought about more, agreed, but that doesn't mean that an SD can't securely provide auth parameters. That would be SD specific though & security should be highlighted as a possible problem. But just because it is a potential problem doesn't mean it shouldn't be enabled.
Also agreed that it makes no difference where stuff like scrape interval is changed: if it's changed manually in config or changed via SD then it's changed regardless & the problems noted above will be experienced.

In my dream (& we're very close to now with Kubernetes SD) I want users to just deploy Prometheus, expose metrics & profit. I would love Prometheus to be this simple to deploy & only introduce complexity when it's needed, such as sharding, federation, etc.

I believe the same, there's two ways to approach the problem:

Give users complete flexibility of how they configure exposition, complexity cascades through the system
Standardise on one way to do exposition within an organisation/team. Complexity only for those not following the standard

I don't see a big benefit in allowing/forcing users to have to provide all these settings via kubernetes labels. Having your standard libraries automatically provide exposition in the expected place means you'd get a lot of this for free, without users needing to think about any aspect of it.

These settings (https://github.com/prometheus/prometheus/blob/master/config/config.go#L335-L356) would be what I would consider consistent properties of a target.

I'd consider those to be properties of the scrape config, rather than an individual target. They're all things that are going to be standard across a job or set of jobs. In the case of honor_labels for example that's only something that the Prometheus administrator should have control of, particularly in any form of shared Prometheus setup, so that one job can't accidentally take out another's metrics.

That would be SD specific though & security should be highlighted as a possible problem.

Anything available via SD is going to be available via the Prometheus web UI for everyone to see. Any protections around that are likely to be fragile (how do we know all the label names we need to protect if every team can chose their own?), so we can't realistically keep anything that's available to relabelling a secret.
We also need to make sure that the SD mechanism is designed to pass around secrets. Usually authentication/identity and credential distribution are completely separate concerns to service discovery, so the required security and controls won't be present.
Without a solid security story that works for non-experts and keeps them on the right path we shouldn't offer security features. For example encouraging users to put credentials for monitoring into an insecure SD is probably okay as monitoring tends not to be sensitive, but it'd be better to have no authentication in that case rather than giving a false sense of security. In the worst (and indeed expected) case that'd lead to users thinking it's okay to put important credentials in the insecure SD.
I don't think security is an area Prometheus should be getting into, getting this sort of thing right is extremely difficult and an entire project onto itself. This is an area where I think our stance should be that if you want to do something like this then you need to write an automatic config generator.

if it's changed manually in config or changed via SD then it's changed regardless & the problems noted above will be experienced.

To be clear, I'm not worried about someone changing it once every few years as is the usual case. I'm worried about someone changing it continuously, on an hourly or daily basis. Considering how rarely scrape intervals tend to change, some friction here is okay (and arguably beneficial, if it prevents overengineering in choices of scrape intervals). Scrape interval is a Prometheus-level setting, not a scrape config or target level setting.

Config has sane defaults, but is configurable via discovery. There is no forcing to use kubernetes labels, only if it differs from defaults. I don't see a problem with that. Could you explain how "complexity cascades through the system" by being able to configure targets?
I can understand the desire to standardise things like metrics path, but if you're trying to plug in Prometheus exposition to an existing app that already has a metrics endpoint used by other systems then you need to be able to change the Prometheus path. If we can't do that via SD config then it has to be done directly in config file, not optimal for a service you want to deploy & scrape all services.

Security needs discussion. We've seen quite a few people asking for it & we provide the ability to configure the scraper now, but only via config file. This might suffice for my use case tbh - it would mean all endpoints would need to have a single security configuration but I think that's fine.

Could you explain how "complexity cascades through the system" by being able to configure targets?

If something was fixed but now is configurable, then everything downstream needs to be able to handle that. Prometheus as we're discussing here is just one part of that, there will also be other automated systems looking for metrics. Humans need to care about this when debugging and when writing those automated systems, so that means training, documentation and libraries to be maintained.

if you're trying to plug in Prometheus exposition to an existing app that already has a metrics endpoint used by other systems then you need to be able to change the Prometheus path

It's my expectation that there's only going to be a handful of such paths (there were 2 in my previous job for example), so the config duplication won't be too bad.
Metrics path is the one I consider to have a good standalone argument for being relabelable, as there may be systems where the existing conventions result in paths varying by application (though opening up a control port in those cases would probably be a good idea). The other settings are effectively binary, so config duplication can handle them.

This might suffice for my use case tbh - it would mean all endpoints would need to have a single security configuration but I think that's fine.

That's how I'd handle it by default, monitoring tends to be no more than commercially sensitive. The main risk is usually accidental DoS rather than a breach.

Sorry for chiming in late. Slowly working through my piled up back-log...
It seems we have clarity that we don't want to revert the ability to change /metrics via relabeling.
Also, we all seem to agree to better not open the can of worms labeled “security”, so don't allow changing any auth related config parameter via relabeling.
Leaves us with the question where to draw the line in between.
If I understand @jimmidyson 's kubernetes use-case correctly, you'd set up a single Kubernetes SD scrape config for all the jobs in your cluster (i.e. job would be relabeled appropriately). In that case, changing scrape_interval and  scrape_timeout via relabeling is crucial as some jobs have the need for high-frequency scrapes while others are relaxed about that, and might expose a lot of metrics so that frequent scraping by default might not be an option. Also, for large scrape, we might need a custom timeout. It would be good if kubernetes user could set those parameters from within kubernetes and wouldn't need to exercise a separate tool for propagation into Prometheus.
And BTW: I see the Kubernetes/Prometheus combo as a broadly used set-up in the near future. So the kubernetes use-case matters a lot.

:+1: on closing this.
Metric path and scheme can be relabeled and theoretically be provided per target. So can URL parameters. This the furthest we can and should go to keep complexity manageable and prevent unintended usage.
It will never be possible to configure everything via SD and it most likely shouldn't.

What's the situation on auth parameters? In my case I simply want to use the snmp and blackbox exporter which are behind basic auth and configure it with relabling as described in our docs.
I don't think there is any way to pass in the credentials this way. Thoughts?

As I commented in the other thread, I think this effectively limits our ability to bring centralized monitoring to a moderately security conscious Kubernetes cluster.  A shared secret is not sufficient for tenancy, simply because that would allow any tenant to go fishing for anyone else's metrics.  It also prevents application authors for Helm and other out of the box solutions from pre-wiring for metrics because they can't anticipate a cluster wide secret.
I'd really like to make Prometheus capable of providing out of the box metrics gathering for multi-tenant Kubernetes clusters.  Without a way for the services themselves to declare their security, it's not really possible to have an out of the box story that isolates those services.

Have you considered a Prometheus per tenant? Any other way is going to be difficult to secure given our security model if you don't want tenants to be able to access each others endpoints.

I think we're seeing the separation of two types of Kubernetes clusters.  The simple, sparse tenancy cluster (one or more teams), which can absolutely get away with approaches suggested here.  And the other, which is hot dense clusters with thousands of nodes and potentially that many tenants, some of which have different security models, but all of which want to be able to easily expose and aggregate metrics.
The plan has been (up until now) to remove Heapster with Kubernetes, and provide a thin facade that would handle metrics tenancy in front of solutions like Prometheus or others.  In that model, the query / UI of Prometheus have tenancy enforced in front of it for the Kubernetes simple use cases (CPU / memory / disk + autoscaling), while potentially allowing admins / high privilege users to access prometheus directly.
For the dense cluster use cases, there's some tradeoff between efficiency and a limited security barrier.  I agree that the level of security provided by something shared like this has limitations, but I also think it parallels a set of tradeoffs made by clusters where tenancy is somewhat loose.  I.e., the goal for most users is to avoid accidental exposure of metrics that could be used to compromise other tenants (exposing personally identifying information, tenanted info, etc), but accepting that a determined attacker could potentially compromise some or all of those.

I'll note as well that there are other longer term efforts to bring centralized authorization and management across many services (things like SPIFFE and LOAS and network level isolation) where it becomes practical to move back to a central config.  But in the absence of those, some level of glue is truly missing.
I'm not adverse to carrying a patch for OpenShift / high tenancy Kube clusters to do some extra level of separation if necessary - I respect the concerns raised above about separation of responsibility.

some of which have different security models, but all of which want to be able to easily expose and aggregate metrics.

They can do that by each running their own Prometheus.

exposing personally identifying information

Such information should never be exposed to Prometheus. We're not designed to provide the level of security or integrity such data requires.

in front of it for the Kubernetes simple use cases (CPU / memory / disk + autoscaling)

That would all come from ~cadvisor, which is part of the cluster itself and thus should require at most one set of credentials.

accepting that a determined attacker could potentially compromise some or all of those.

The security is either there or is isn't. There is no security for kubernetes annotations accessed from Prometheus, we'll return them to anyone who has access to the http port. If you're happy with this, then an unauthenticated metrics endpoint provides the same security.
I think you're conflating 3 different use cases here, each with different threat models. Each is solvable individually, but can't be solved with one single approach.

There is no security for kubernetes annotations accessed from Prometheus, we'll return them to anyone who has access to the http port.

Our assumption is that in Kubernetes we would never expose prometheus like that to anyone except infrastructure admins.  There are many proxies and other solutions for that.

That would all come from ~cadvisor, which is part of the cluster itself and thus should require at most one set of credentials.

Today, but cadvisor is just one small part of the infrastructure.  There are edge proxies, local proxies, storage plugins, etc that all have metrics to contribute.  Using a shared secret for all of those means that all of those have to have the same level of security, which is not the case.

Such information should never be exposed to Prometheus. We're not designed to provide the level of security or integrity such data requires.

That can be things like service names, pod names, or other labels that are in use.  There is a spectrum of PII ranging from "not allowed to be stored outside of the EU" all the way down to "provides information that can be correlated with others".  I tend to think of this as the 80% case vs the 20% case - 80% of the applications want their metrics exposed, but don't have strong guarantees.  The 20% case has a number of levels, from "don't expose this to the public internet" to "don't expose this if an attacker gets loose on the network".

The security is either there or is isn't.

I think it's reasonable to say that Prometheus acting as a collector for an entire kubernetes cluster has a moderately complex config, a moderate set of roles / secrets that are separated, and that running multiple prometheus instances is totally reasonable for further isolation, and that even within that config there will have to be some separation of duties.  Anyone wanting to subdivide what metrics are visible must do that in front of prometheus, no disagreement, and I don't think it's prometheus' job to enforce that as such.
Service discovery that allows collectors to opt in to collection without preconfiguration by an admin, but only if no security at all is imposed, makes the tradeoff much worse the larger the kubernetes cluster gets.  I'm looking to have almost every component deployed on Kubernetes expose a scrape endpoint.  But not every component is going to tolerate being wide open, which just means that the "drop in, it works" scenario degrades back to "the admin has to go change the config every time a new infrastructure component is gathered".

There are edge proxies, local proxies, storage plugins, etc that all have metrics to contribute. Using a shared secret for all of those means that all of those have to have the same level of security, which is not the case.

I'd presume there's a limited number of those, such that specifying as many credentials as needed in the config is not completely unreasonable.
If you can use a common challenge response mechanism such as an ssl client cert, rather than a shared secret, then this problem goes away. I believe this is the direction things are generally going.

Service discovery that allows collectors to opt in to collection without preconfiguration by an admin, but only if no security at all is imposed,

You can have authentication, just not with credentials that aren't known in advance. If you need arbitrary on-the-fly credentials, you'd be looking at an approach similar to the CoreOS Prometheus operator to generate the config file.

makes the tradeoff much worse the larger the kubernetes cluster gets.

You'll run into scaling or social issues that require splitting out such a Prometheus as the cluster gets larger. Security is only one thing that could trigger this, and I'd guess one of the least common.

Ok - so to capture:

service discovery is only expected to be used in homogenous / predefined security domains - discovery will not grow to allow abstractions to handle that, because security / authentication is better handled above the tool rather than inside of it
dynamic reconfigured of prometheus via config is normal and expected, and therefore a higher level component should manage dynamic security if it so chooses
if a small N multitenant kubernetes cluster wants to allow secure self-registration, that coordination should be managed above
very high N tenancy is not something prometheus believes it can handle well via service discovery since it's a limited use case and has other challenges

On point 4 explicitly, for very high N tenancy (which is something we deal with often in OpenShift), would additional scrape config abstractions be accepted to keep config complexity low?  I.e. we could easily have 2-3k very small apps with limited metrics sets that have different auth boundaries - if we generate large config files, do you see Prometheus accepting limited concepts that reduce the need to generate large config files (or make them easy to parse)?  This is something we deal with often with HAProxy, where we have multi-megabyte generated config files (tens of thousands of backends) and so have worked with them to optimize their config loading and find gaps.

do you see Prometheus accepting limited concepts that reduce the need to generate large config files (or make them easy to parse)?

The general answer is that if you can in principle it via configuration management, we aren't going to duplicate that functionality. At 2-3k apps, you're well past the point where you must already have configuration management (and possibly past what one Prometheus can handle).

This is something we deal with often with HAProxy, where we have multi-megabyte generated config files (tens of thousands of backends) and so have worked with them to optimize their config loading and find gaps.

Unlike HAProxy, we aim for a seamless in-process config reload. If you've ideas around making config reloads less disruptive, we're open to that.

@fabxc @brian-brazil @beorn7 I just ran into this again and reread this issue and I really don't understand your concerns. Relabling allows overwriting every part of the url, except the UserInfo which would be required to set basic auth via relabling. I understand that you don't want to open the 'security can of worms', but the required changes are trivial and all implications are straight forward.
Yes, configuration management could solve that but frankly that's not a good answer for a official cloud native project. And looking at who all has interest in commercial use of prometheus, especially in Enterprise where some authentication is strictly required, I'm worried that this technical purism will cause people to build heavier and heavier workarounds, if not even fork/replace (been there you know).

It looks like there isn't even an workaround.
To be clear, maybe I miss something: If you want any minimal form of auth, you are required to introduce configuration management which discovers targets on it's own, rendering the SD in Prometheus useless.
That cfg mgmt would then render one static_config for each service endpoint with labels for that service endpoint. Maybe you can combine few with common labels, but it might be as many static_configs as service endpoints, giving you a huge config file. Possible to large to be human readable, let alone readable nicely in the /config.
Now you need to figure out where and how to run your cfg mgmt, probably you would have a volume in your pod for the config and use @jimmidyson's config reload. But you also need to monitor this, because you can't use the prometheus_sd_* metrics to do so. So you might end up running the node-exporter in the pod too and use the textfile collector for that.
Finally you need to figure out how to update your cfg mgmt sidecar. Maybe that works with some pull based system like chef, or you end up glueing something together. Possibly several days of work for that.
I mean, really? You can argue you need config mgmt anyway but that's a self-fulfilling prophecy. Prometheus is right now the only reason I need to think about it. Not my postgres databases, my gateways, my reverse proxies nor my backend service. To be fair, instead of introducing configuration management I'll rework all my applications to have a separate listener for prometheus metrics. But try telling that to a new users of prometheus with a straight face..

Finally you need to figure out how to update your cfg mgmt sidecar.

We generally combine confd and confd-sidecar for this, but we aren't using Kubernetes.

Don't force an ideology idea which works in a perfect world.
Let the user choose what's the best for his own.
If you want/need this feature you may have to use the Red Hat fork unless the Prometheus heroes realize what the community want.

@jkroepke Is there a redhat maintained fork with this functionality?

copying from https://github.com/openshift/origin/issues/17685#issuecomment-380386472
@discordianfish IIRC the patches were included in the openshift/prometheus image at some point but not anymore.

Well, I wouldn't be surprised if they fork Prometheus. It's no unheard of Red Hat forking popular projects and even introduce incompatible changes.

You could talk about that with one of their principal software engineers. ;o)

Still stand by my words ;)

authors: Rudd-O beorn7 brian-brazil discordianfish fabxc jimmidyson jkroepke simonpasquier smarterclayton

repo: prometheus/prometheus


text:
Hash collision between labels will lead to incorrect results

What did you do?
After reading the code related to stringlabel, I found that there are a lot of hash function calls, and it seems that the errors caused by hash collisions are not considered.
For example in funcLabelReplace:
https://github.com/prometheus/prometheus/blob/031d22df9e43f1c3014b8344337e133099902ae8/promql/functions.go#L1149-L1201
And there are many ContainsSameLabelset checks in the calculation of eval, here is also judged by hash, will there be the same problem?
https://github.com/prometheus/prometheus/blob/031d22df9e43f1c3014b8344337e133099902ae8/promql/value.go#L232-L249
What did you expect to see?
No response
What did you see instead? Under which circumstances?
I'm not sure if it's really a problem, but I really don't see how collisions are handled
System information
No response
Prometheus version
No response
Prometheus configuration file
No response
Alertmanager version
No response
Alertmanager configuration file
No response
Logs
No response

You're correct: there are a number of places in Prometheus where Labels hash collisions are not handled.
They seem to be quite rare in the wild; #5724 is the only example I can see reported here. By "birthday problem" math with 6 million series there is a 1 in a million chance of a collision.
Working on #12993 gave me the idea that we could produce a 256-bit hash for every series, then use that everywhere that the 64-bit hash is currently used. Collisions would be astronomically unlikely.
A (likely incomplete) list of places Labels.Hash() is used without any protection against collisions:

TSDB, to find a series in the head block.
In the PromQL engine, collecting the results of a range query.
ContainsSameLabelset, to check output of a PromQL subexpression.
DropMetricName, used by most PromQL functions.

There are also hashes of subsets of labels, which would not be helped by pre-computing a 256-bit hash.

Grouping in PromQL aggregations like sum by (foo, bar)

I must admit that the untreated hash collisions are still giving me the creeps.
My main concern is that we often use fast non-cryptographic hashes, which are probably much farther away from perfect pseudo-random distribution of hash values than cryptographic hashes. Pardon me if I use the wrong terminology here (I'm not a proper statistician), but the gist is that the probability of a collision is much higher than calculated via the birthday problem math if the hash values are correlated with the original values rather than randomly and equally distributed.
In prometheus/prometheus, we use xxHash, which is probably better than fnv64a, which we use in prometheus/client_golang. (See https://github.com/prometheus/client_golang/issues/220 for the hash collision handling added in client_golang eventually, but I'm not so sure anymore it deals with all cases properly.)
While properly dealing with all variations of hash collisions in a performance-preserving way would be cool, a sufficiently low chance of hash collisions is also "good enough". Note that the only real-world example reported by @bboreham above was fixed by improving the hash calculation (when combining different hash values).

Hello from the Bug Scrub!
Sounds like it's still relevant, but not easy. I like the idea of increasing bit size of a hash (or using more secure one) vs handling collisions, but proper proposal and experiment with benchmark is needed.

source: gharchive/issue

authors: bboreham beorn7 bwplotka wzharies

license_type: permissive

license_source: github-api


text:
Alert labels are not changed after update rule file and reloading prometheus.

Bug Report

Download prometheus version v2.0.0 and v2.3.2 (reproduced on both).
Extract these version to /tmp directory

tmp $ ls ./prometheus*
./prometheus-2.0.0.linux-amd64.tar.gz  ./prometheus-2.3.2.linux-amd64.tar.gz

./prometheus-2.0.0.linux-amd64:
console_libraries  consoles  LICENSE  NOTICE  prometheus  prometheus.yml  promtool

./prometheus-2.3.2.linux-amd64:
console_libraries  consoles  LICENSE  NOTICE  prometheus  prometheus.yml  promtool

Create new prometheus configuration file /tmp/prometheus.yml with following content:

global:
  scrape_interval:     30s
  evaluation_interval: 5s
  scrape_timeout:      5s

alerting:
  alertmanagers:
  - path_prefix: /reciver
    scheme: http
    static_configs:
    - targets:
      - 127.0.0.1:19999
    timeout: 10s

rule_files:
  - /tmp/rules/*.yaml

scrape_configs:

  - job_name: "Test metric"
    metrics_path: /
    honor_labels: true
    scrape_timeout: 30s
    static_configs:
      - targets:
        - 127.0.0.1:8080

Start simple metric generator like bellow:

#!/usr/bin/env python2

from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import SocketServer
import random

class Server(BaseHTTPRequestHandler):
    def _set_headers(self):
        self.send_response(200)
        self.send_header('Content-type', 'text/plain')
        self.end_headers()

    def do_GET(self):
        self._set_headers()
        self.wfile.write('my_awesome_metric{app="foobar"} ' + str(random.randrange(5,10)))

def run(server_class=HTTPServer, handler_class=Server, port=8080):
    server_address = ('', port)
    httpd = server_class(server_address, handler_class)
    print 'Starting...'
    httpd.serve_forever()

if __name__ == "__main__":
    from sys import argv

    if len(argv) == 2:
        run(port=int(argv[1]))
    else:
        run()

Start dummy alert manager like bellow (you can use nc linux utility for this):

package main

import (
	"flag"
	"fmt"
	"io"
	"log"
	"net/http"
	"os"
)

func handler(w http.ResponseWriter, r *http.Request) {
	_, err := io.Copy(os.Stdout, r.Body)
	defer r.Body.Close()
	if err != nil {
		panic(err)
	}
	fmt.Println("\n")
	w.WriteHeader(http.StatusOK)
}

func main() {
	port := flag.String("port", "19999", "port to listen on")
	path := flag.String("path", "/reciver/api/v1/alerts", "path to listen on")
	addr := fmt.Sprintf("0.0.0.0:%s", *port)
	log.Printf("Starting server %s%s", addr, *path)
	http.HandleFunc(*path, handler)
	log.Fatal(http.ListenAndServe(addr, nil))

Create directory /tmp/rule1 and place alert rule template and also place this rule to rule_files
directory with path /tmp/rules/ (from prometheus configuration file).

groups:
- name: my_group
  interval: 5s
  rules:
  - alert: my_alert
    expr: my_awesome_metric > 1
    for: 5s
    labels:
      rule_prefix: my.rule.prefix
      rule1_lavel: rule1_value

cp /tmp/rule1/rule.yaml /tmp/rules/rule.yaml

Create directory /tmp/rule2 and place alert rule template (notice different labels)

groups:
- name: my_group
  interval: 5s
  rules:
  - alert: my_alert
    expr: my_awesome_metric > 1
    for: 5s
    labels:
      rule_prefix: my.rule.prefix
      rule2_lavel: rule2_value

Start Prometheus with command

tmp $ ./prometheus-2.0.0.linux-amd64/prometheus --config.file=./prometheus.yml --web.listen-address=0.0.0.0:9100 --web.enable-lifecycle

Wait till new alerts appears in the output of dummy manager:

tmp $ ./dummy-manager
2018/09/03 17:48:56 Starting server 0.0.0.0:19999/reciver/api/v1/alerts
[{"labels":{"alertname":"my_alert","app":"foobar","instance":"127.0.0.1:8080","job":"Test metric","rule1_lavel":"rule1_value","rule_prefix":"my.rule.prefix"},"annotations":{},"startsAt":"2018-09-03T17:49:27.076313109+03:00","endsAt":"0001-01-01T00:00:00Z","generatorURL":"http://BY-LL-022:9100/graph?g0.expr=my_awesome_metric+%3E+1\u0026g0.tab=1"}]

[{"labels":{"alertname":"my_alert","app":"foobar","instance":"127.0.0.1:8080","job":"Test metric","rule1_lavel":"rule1_value","rule_prefix":"my.rule.prefix"},"annotations":{},"startsAt":"2018-09-03T17:49:27.076313109+03:00","endsAt":"0001-01-01T00:00:00Z","generatorURL":"http://BY-LL-022:9100/graph?g0.expr=my_awesome_metric+%3E+1\u0026g0.tab=1"}]

Overwrite rule.yaml within /tmp/rules directory by rule from /tmp/rule2/rule.yaml, see step 7. This rule has the same group and alert names but different labels. Perform reload with command:

$ cp /tmp/rule2/rule.yaml /tmp/rules/rule.yaml
$ curl -X POST http://localhost:9100/-/reload

What did you expect to see?
I expect to see json alert which contains new labels. UI shows correct labels but prometheus sends alert with old labels to alert manager.

What did you see instead? Under which circumstances?
I see alert with old labels instead of new ones which defined in the /tmp/rule2/rule.yaml file.
Environment

System information:
uname -srm
Linux 4.15.0-33-generic x86_64

Prometheus version:

tmp $ ./prometheus-2.0.0.linux-amd64/prometheus --version
prometheus, version 2.0.0 (branch: HEAD, revision: 0a74f98628a0463dddc90528220c94de5032d1a0)
  build user:       root@615b82cb36b6
  build date:       20171108-07:11:59
  go version:       go1.9.2
tmp $ ./prometheus-2.3.2.linux-amd64/prometheus --version
prometheus, version 2.3.2 (branch: HEAD, revision: 71af5e29e815795e9dd14742ee7725682fa14b7b)
  build user:       root@5258e0bd9cc1
  build date:       20180712-14:02:52
  go version:       go1.10.3

Alertmanager version:
Did not use.

Prometheus configuration file:
Please, see above.

This sounds like #4169. Can you try at master?

@brian-brazil, thank you for quick replay.
Will to reproduce today/tomorrow using latest master and post results here.

@brian-brazil am not able to get working configuration using Prometheus from master now (using the same configs and setups).
It seems like Prometheus does not see/use targets and alert managers from prometheus.yml configuration file (no targets alert managers via WEB UI).
It is possible workaround with absent missing targets, but it would be great to have JSON alert sent to dummy alert manager.

@krasoffski you're probably hitting https://github.com/prometheus/prometheus/issues/4551 (in short master is currently broken for SD). https://github.com/prometheus/prometheus/pull/4556 should fix it so you may want to build from there.

Can you retest with the latest v2.4.2 release as it should be fixed now?

Thanks @krasoffski

authors: brian-brazil krasoffski simonpasquier


text:
Implement ValueStream?

Hi,
I'm surprised to see that prometheus (or should I say, borgmon2) doesn't implement a valuestream function yet (allow transferring timeseries between prometheus instances). Which I thought was crucial to implement a multi-tier aggregation hierarchy,
Any thoughts?
BTW: you guys are awsome creating prometheus!

I've been playing around with multi-tier aggregation by exposing handcrafted "metrics" endpoints as console templates, building on this example. You can, instead of matching the name, specify any query (but need to then give it a new name). The effect is that one Prometheus can publish arbitrary (raw, aggregated or otherwise computed) timeseries that are then available to be scraped (albeit not at the standard endpoint) by any other prometheus.

@sunyc Could you explain what you mean by "valuestream"?

@matthiasr  which is obviously not very efficient...  And there is timestamping issue.
@beorn7  ValueStream is a RPC service that allows an prometheus instance scrape timeseries  from another prometheus . Which also retains original timestamp etc.
It has a number of usefulness, namely,  one could have a prometheus  that valuestream from several peers for aggregation etc etc etc.

I'm not too worried about timestamps … when aggregating for example request counts from many individual instances as one timeseries, there is no one timestamp anyway. And if the samples are just passed through 1:1 then why aggregate in the first place?

The Prometheus scrape protocol already contains a timestamp for that purpose. However, it is currently not possible to scrape multiple samples for one time series in one scrape.

@matthiasr  because scrape is not synchronized between  targets,  If there were no timestamp in the output the aggregation would produce wrong values..

Heya Yucong,
valuestreaming aka hierarchical federation is on our roadmap:
http://prometheus.io/docs/introduction/roadmap/#hierarchical-federation
Right now we only have the "hack" that Matthias mentioned, but soon[tm]
we'll have the real thing. I agree, it should be possible to transfer raw
samples with their original timestamps and uninterpolated values.
Cheers,
Julius
On Wed, Apr 15, 2015 at 5:27 PM, Yucong Sun notifications@github.com
wrote:

@matthiasr https://github.com/matthiasr because scrape is not
synchronized between targets, If there were no timestamp in the output the
aggregation would produce wrong values..
—
Reply to this email directly or view it on GitHub
https://github.com/prometheus/prometheus/issues/638#issuecomment-93449257
.

I would be happy to help (maybe we can also use go-grpc now), believe it or not, i spent quite a bit of of time on it in the past life .
LoL

I'll close this as a duplicate of https://github.com/prometheus/prometheus/issues/9 (updated that issue with the notion of "streaming").

@sunyc That'd be great in general. As a smaller starter project it'd probably be good to start with something small to get acquainted with the whole code and contributing business, and then go from there. We're all also hanging out on #prometheus on Freenode for discussions and inspiration :)

authors: beorn7 juliusv matthiasr sunyc


text:
promtool: Calculate coverage data for tests

Proposal
Make it possible to calculate whether the tests cover each rule and alert.
In order to do this some basic assumptions would have to be made, for example take a simple set of rules:

job:up:ratio calculates the "up ratio of a job"
Alert TooManyDown alerts if job:up:ratio < 0.5

The input would be just the up metric, potentially per job. This would mean job:up:ratio would have a selector like up, this would therefore be the expected input and something like this should exist in input_series for the test.
Even in this simple example there's some tricky bits, is it enough if an alert test covers job:up:ratio, or would it be better to count  the rule coverage differently from the alert coverage. (Seems like reporting them separately and the user can choose to interpret them as they like would work though.)
More tricky things are label_replace() and regexps on __name__, as it would be very tricky to follow the rule dependencies there but I think this isn't a huge problem -- in most cases it should be possible to follow a rule back to the input series via selectors. Where this fails the result is coverage is reported as less than it might actually be, potentially the coverage output could warn with something like regexp selector seen on __name__, coverage reported may be less than actual.
Coverage would need to be calculated per rule file, but not per test file (i.e. don't enforce anything like tests for a particular rule file have to be in a single test file -- we consider all the arguments to promtool test rules as a "suite"). However inputs would be per test file (i.e. if one test treats the output of recording rules from another file not under test, that would be fine).
I'm imaging the output could be something like:
rule-file.yaml:
    50% rule coverage
    50% alert coverage

Use case. Why is this important?
It would be good to be able to get an idea of how well unit tests cover the rules they are supposed to be testing. This brings PromQL closer to something you'd expect of a fully featured language.
I'm willing to work on this, maybe will prototype and see how it goes. I think it makes sense in promtool as it needs knowledge of how PromQL executes and can't quite be pushed to a PromQL language server (although maybe there are integration opportunities there, later).

It would be good to be able to get an idea of how well unit tests cover the rules they are supposed to be testing.

I'm a bit wary of this, as it would encourage getting 100% test coverage rather than spending time testing what is actually useful (which is usually only the alerts). For example if there are a pile of copy&pasted rules doing the same basic aggregation to different non-critical metrics, are writing and maintaining exhaustive unittests the best use of someone's time?
I'd prefer to encourage testing complex and non-trivial rules, over writing low-value unittests for simple rules.

I agree with this suggestion. Even with simple rules, I think unit tests can be valuable. I'm open to be corrected but as it stands, please consider this as an example:

Many targets are queried under a generic job and you've created a single alert used for a specific event.
As your incident management becomes more mature, your team starts creating runbooks and you realize that one of the targets in the job needs its own alert description, so a link to the runbook can be included in the notification.
You're required to now make a separate alert which specifies the target in the expression, from within the same job as above. You believe the alert to be simple to put in place (as it should be) and you do so without a unit test.
The next time the target is experiencing an issue, 2 alerts are triggered because you've forgot to add a !=target condition in the first alert's expression, to ignore the target you configured in step 3.

By configuring a input_series (at the very least) and an alert_rule_test  specifically for this job and target, we could have been advised through promtool test rules that a specific series was triggering more than 1 alert in our "alert_rules.yml" file.
While there could be another layer of abstraction handling this documentation scenario, I believe this justification holds a bit of weight. Maybe it's not documentation but rather a change in an alert label that routes the alert to a different team. In other words, for any reason that requires you to cherry-pick targets out of generic alerts. In an ideal world, exceptions wouldn't exist...
I hope I've demonstrated the scenario well enough, to help this be considered and I'm open to the idea that maybe I've presented an anti-pattern...

@dgl did the prototype go anywhere on this, or didn't have a chance to take a look?  I don't see a branch or PR mentioned here.

Since promtool does not have this feature, this is a short script I used to generate coverage. I adjusted it to be more generic and not specific to the folders I used. I did not test yhis adjusted version but it should work the same as what I did. Feel free to copy and use this for yourself until the feature is added into the tool.
TOTAL_ALERTS=$(ls rules/*.yaml | xargs -I {} yq e '.groups[] | .rules[] | .alert' {} | sort | uniq | wc -l | sed 's/ //g' || true)
ALERT_FILE_COUNT=$(ls rules/*.yaml | wc -l | sed 's/ //g' || true)
TEST_COUNT=$( (find -L tests -type f -name "*.yaml" -exec yq e '.tests | length' {} \; | awk '{s+=$1} END {print s}' 2> /dev/null || echo 0 ) | tr -d "\n" )
TEST_FILE_COUNT=$(find -L tests -type f -name "*.yaml" | wc -l | sed 's/ //g' || true)
TESTED_ALERTS=$(find -L tests -type f -name "*.yaml" -exec yq e '.tests[] | .alert_rule_test[] | .alertname' {} \; | sort | uniq | wc -l | sed 's/ //g' || true)
TEST_COVERAGE=$( [[ "$TOTAL_ALERTS" -eq 0 ]] && echo NaN || ( echo $(( $TESTED_ALERTS * 10000 / $TOTAL_ALERTS )) | sed 's/\(.*\)\([[:digit:]][[:digit:]]\)/\1.\2/' ) )

echo "$TOTAL_ALERTS alerts were found in $ALERT_FILE_COUNT files."
echo "$TEST_COUNT tests were found in $TEST_FILE_COUNT files."
echo "$TESTED_ALERTS/$TOTAL_ALERTS ($TEST_COVERAGE%) alerts have at least one test. (This does not mean the tests are good quality)"

Hello from the bug scrub.
@dgl this issue is quite old by now. Do you think it's still worth pursuing?

authors: PicksUpTheGauntlet beorn7 brian-brazil dgl jnovick stacybird


text:
Avoid unnecessary allocations in recording rule evaluation

When evaluating a rule, RecordingRule.Eval() uses a labels.Builder instance to build the label set for the recorded samples.
Builder.Labels() assumes that most of the time, the Builder is being used to remove or replace labels in the existing label set rather than add to them.
But in the case of a recording rule, we're likely adding or replacing labels from the rule definition, so Builder.Labels() ends up allocating two Labels instances (once here, and another later on in append() here).
Instead, during rule evaluation, we can calculate the upper bound of the length of the label set and allocate just one Labels instance ahead of time. This reduces allocations at the risk of slightly overallocating - if the rule overrides some of the labels returned in the query, we won't need quite so much space in the label set.
For one Mimir cluster, we've seen that Builder.Labels() is responsible for ~6% of allocations attributable to rule evaluation. (Note that in Mimir, the behaviour is slightly different as we perform a remote query and parse the JSON response, whereas in Prometheus, the query is performed locally.)
There's a similar improvement to be had in AlertingRule.Eval() as well - I will address this in a later PR.
Benchmark results
Note that this benchmark includes the cost of evaluating the query used in the benchmark's rule.
name                                                                                old time/op    new time/op    delta
RuleEval/no_labels_in_recording_rule,_metric_name_in_query_result-10                  7.07µs ± 2%    6.69µs ± 0%  -5.38%  (p=0.008 n=5+5)
RuleEval/only_new_labels_in_recording_rule,_metric_name_in_query_result-10            7.21µs ± 0%    6.93µs ± 0%  -3.95%  (p=0.008 n=5+5)
RuleEval/some_replacement_labels_in_recording_rule,_metric_name_in_query_result-10    7.05µs ± 0%    6.70µs ± 1%  -4.89%  (p=0.008 n=5+5)
RuleEval/no_labels_in_recording_rule,_no_metric_name_in_query_result-10               14.3µs ± 1%    14.4µs ± 0%    ~     (p=0.690 n=5+5)

name                                                                                old alloc/op   new alloc/op   delta
RuleEval/no_labels_in_recording_rule,_metric_name_in_query_result-10                  7.13kB ± 0%    6.44kB ± 0%  -9.66%  (p=0.008 n=5+5)
RuleEval/only_new_labels_in_recording_rule,_metric_name_in_query_result-10            7.13kB ± 0%    6.83kB ± 0%  -4.26%  (p=0.029 n=4+4)
RuleEval/some_replacement_labels_in_recording_rule,_metric_name_in_query_result-10    6.94kB ± 0%    6.44kB ± 0%  -7.16%  (p=0.008 n=5+5)
RuleEval/no_labels_in_recording_rule,_no_metric_name_in_query_result-10               15.2kB ± 0%    14.8kB ± 0%  -2.83%  (p=0.008 n=5+5)

name                                                                                old allocs/op  new allocs/op  delta
RuleEval/no_labels_in_recording_rule,_metric_name_in_query_result-10                     127 ± 0%       119 ± 0%  -6.30%  (p=0.008 n=5+5)
RuleEval/only_new_labels_in_recording_rule,_metric_name_in_query_result-10               127 ± 0%       121 ± 0%  -4.72%  (p=0.008 n=5+5)
RuleEval/some_replacement_labels_in_recording_rule,_metric_name_in_query_result-10       125 ± 0%       119 ± 0%  -4.80%  (p=0.008 n=5+5)
RuleEval/no_labels_in_recording_rule,_no_metric_name_in_query_result-10                  239 ± 0%       233 ± 0%  -2.51%  (p=0.008 n=5+5)

Fuzzing failed in the CI build, but I can't see any information about what exactly failed - is there some way to get more information?

Cc @bboreham

Are any further changes required? Would be great to get this merged.

I would like to first merge the label string behind a build tag

Can you rebase this?

Can you rebase this?

Yep, done.

Looks like the Windows build is failing due to non-deterministic order of series in query results - is this expected?

Looks like the Windows build is failing due to non-deterministic order of series in query results - is this expected?

Prometheus does not guarantee the order of results of an instant query, so yes.
Adding sort to the query is one option, or using more sophisticated code to check the result.

source: gharchive/pull-request

authors: bboreham charleskorn roidelapluie


text:
support modify alert-rule by http post/put/delete

privode http post/put/delete method to enable alert rule instead of config map
storage config in mysql
post / put / delete  http://127.0.0.1:9090/api/v1/alertRule
{
"Id": 0,     //mysql alert_rule id,modify or delete rule through id
"name": "example",
"expr": "sum (absent (example{job="example"})) by (job_name)==1",
"severity": "critical",
"for": "1m",
"sendInterval": "2m",
"description": "example no data alert"
}
"Id": the primary id of the alert-rule which storage in mysql, can modify or delete a rule through it
"for": an alert will stay with pending-state util a "for"-time expire and change to firing
"sendInterval": an alert state duration, will not send util the time expire
when modify a rule alert, the origin alert state will be hold
fetch all rule:  http://127.0.0.1:9090/api/v1/ruleList POST, will look up all the rules match the query below
{
"Id": 0,     //mysql alert_rule id,modify or delete rule through id
"name": "example",
"expr": "sum (absent (example{job="example"})) by (job_name)==1",
"severity": "critical",
"for": "1m",
"sendInterval": "2m",
"description": "example no data alert"
}
enable the mysql config in prometheus.yml
rule_mysql_config:
host: ""
port: ""
db: ""
user: ""
password: ""

Thanks for your PR. We purposefully do not attempt to maintain a database of configuration, nor do we depend on external databases. If you wish to build something like this you can do so on top of the yamls files we support.


text:
Adds link to valid metric names

One of our users today asked us if dashes were allowed in recording rule names.
We asserted that they were not, but also that we could not remember for certain.
After determining empirically that they are not allowed, I realized that the
documentation could be slightly clearer about valid rule names.
This PR simply adds a note to the documentation re-iterating that the rules must
be valid metric names - and more importantly, adds a link to where a user can
read what those are, in case they were not aware (or did not know where to find it).
Signed-off-by: Andrew Hayworth ahayworth@gmail.com

Thanks!

authors: ahayworth brian-brazil


text:
Laravel 5.5

Will there be an update to support Laravel 5.5?

Hi @kumold , let me check if it work and evaluate efforts on adding support for it. Will do that today, hopefully.

authors: SCIF kumold

repo: propelorm/PropelLaravel

license: mit

license_source: bigquery


text:
Utilities to Implement

[x] Object
[x] Array
[x] WeakMap
[x] WeakSet
[x] Map
[x] Set
[x] ArrayMap (Array.prototype.map, optimized)
[x] SignalAsyncData / load (wrap a promise)
[x] asyncFunction (wrap and invoke a function)
[x] @signal (wrapper around new Signal.State)
[x] localCopy / @localCopy
[ ] keepLatest / @keepLatest
[ ] dedupe / @dedupe
[x] @cached (computed, but on a getter)
[ ] localStorage / @localStorage

other storages as well?

[ ] fetch
[ ] Relays - https://www.pzuraq.com/blog/on-signal-relays

Blocked on framework-specific semantics and timing (cleanup)

throttle
debounce
Image
effect

[x] leaky microtaskqueue effect

waitUntil
resource

It would be nice to have a utility similar to MobX's reaction() function.
reaction() is very useful when you need to track the pervious value of a signal. You give it a tracked data function and an untracked effect function that's run when the value changes.
Something a bit like this:
export const reaction = <T>(
  data: () => T,
  effect: (value: T, previousValue: T) => void,
  equality = Object.is,
) => {
  const computed = new Signal.Computed(data);
  let previousValue = computed.get();
  let isReacting = false;
  const watcher = new Signal.subtle.Watcher(async () => {
    if (isReacting === true) {
      return;
    }
    isReacting = true;
    await 0;
    isReacting = false;
    const value = computed.get();
    if (!equality(previousValue, value)) {
      effect(value, previousValue);
      previousValue = value;
    }
  });
  watcher.watch(computed);
};

oo nice! that does seem useful -- I know quite a few folks that would like such a utility for caching or memoizing complex calculations (tho I don't know an exact use case, but I know it's desired!).
Would you be willing to submit a PR adding this utility along with a bunch of tests?
(so we can ensure behavior is retained as the polyfill (and spec) evolves?)

Reminds me of Vue's watch function that allows to declare multiple signals ("refs") to track:
https://vuejs.org/api/reactivity-core.html#watch

I have a DOM util in mind. Is it suitable here ?

ye!

Since everything here so far runs in any JS environment, I would personally tend towards putting DOM things in a separate package.

that is a fair point :thinking:
@yw662 can you open a new issue to talk more about your idea? you aren't the first to want to experiment with dom tools?
and we can make more repos for said experiments

authors: JosXa NullVoxPopuli justinfagnani yw662

repo: proposal-signals/signal-utils

license: MIT


text:
bold target label

improved gitmod script
fixes for commiting, echoing cmd before git commands, and doing a release PR
fix purge local branches

invalid PR from erroneous gh pr cmdline usage

repo: prosopo/workspaces


text:
Dropdown I am a...

Complete the links to tags for the various target audiences.

Also, noticed a funny issue loading with a slow connection.

I can work on this

Oh sorry I didn't noticed you were assigned to this, I'll look for something else haha

Hello @Squallac , that's not a problem, you're assigned now. I'll review it.
Keep it like your previous work, small and concise. Better to have smaller and quick iterations ;)
Let me know if you need any hands on deck!

Also, noticed a funny issue loading with ~a slow connection~ it stays there actually, in Firefox but not Safari..

I wasn't able to reproduce this issue, I tested with firefox and Chrome in Ubuntu using slow connection throttle. This might be an issue with the bundle size?

PR Open https://github.com/protect-earth/protect.earth/pull/42

@Squallac can you see the issue on https://new.protect.earth/ in FF?

@Squallac can you see the issue on https://new.protect.earth/ in FF?

Oh yeah I can see it! Let me take a look

I'd love to get this killed off. I keep bumping into it every time I preview the site. 😆

I was taking a look to the

Also, noticed a funny issue loading with ~a slow connection~ it stays there actually, in Firefox but not Safari..

I was taking a look to this issue, it looks like its not loading the css classes "ml-1 h-5 w-5" on the svg file.
Do you think this could be an issue with the tailwindCSS config @ijpatricio ? I'm not familiar with the config of tailwind

With your findings, I set the prop to not display the inner svg, and put it our own (the same) on the content slot, so that I could override the styles directly on the element.
Better done than perfect, when and if it will be a problem, we'll take care of that. For now it's done, I've made a quick PR.
Thank you @Squallac

Thank you @ijpatricio!

Also thank you @Squallac great teamwork.

authors: Squallac ijpatricio philsturgeon


text:
Refactor covariance: extract MeanTuple to external class so can be used by other operators

Covariance class contains an inner class MeanTuple to perform it's logic. Refactor this class to an external one.

MeanTuple has been created but operators do not make use of it.


text:
Launchpad "Masterclass" Curriculum Request

Problem Description
With the move to a 4 week curriculum, we want to be still able to educate folks about some of projects that are not in our core curriculum but people have a lot of interest in. We've previously been setting up Q&A sessions on an ad-hoc basis with the following teams:

Cryptoecon
Saturn
FVM
Compute Over Data
Filecoin Green
Marketing
Network Goods (they've been folded into the PLN section of the curriculum with their network funding Q&A session)

The goal here would be two things:

Cohort team to implement a rotation and cadence for live Q&A sessions
Curriculum team to build some supplemental curriculum and workshops around these teams.

What we could start with
-Crytpoecon: we already have a section for it in the curriculum we could just add workshop or a section on tokenomics
-Filecoin Green: we could add a section to the curriculum about filecoin green and add co.2storage to the dev tools section
Additional context
This is more of a broad issue ticket to get it on paper somewhere but I'm sure this will end up being multiple tickets and discussions. This will for sure need some discussion ideals/what to first/timeline etc with both teams.

@kt-wawro would be great to work with you to spec/ create an issue for each of the individual ones (e.g. one for CryptoEcon, one for COD, etc) and then plan which should be tackled in which order -- e.g. if you are going to ask COD to present next colo, we would prioritize making a page for that!

Info from Niki G FVM -
Best to check which dev tools should be featured here with respective dev advocates:
@truckerfling
@hammertoe
(also Zondax team maybe, for the solidity library they have built)
Filecoin. -
@Jenks
@Longfei
IPFS -
@Discordian
@Daniel Norman
Anything hackathon related -
@dawnkelly09
Bacalhau -
@Iryna
@AlisonWonderland (Ally)
Web.storage and NFT.storage -
@Aysun Demircan
Community tools:
Lighthouse -
@Nandit Mehra
Tableland - shared channel on Discord is fastest way to reach them, can add you
Livepeer - can check
Fleek/Ceramic/Pinata/Spheron/Moralis - are very low touch ways of using IPFS and we are recommending other ways listed above before any of these.
You should chat about how we are thinking about devtools with
@dawnkelly09
in maybe 1-2 weeks. They are changing very frequently.
1:09
All these chats can be done in the #devrel channel

repo: protocol/launchpad


text:
🛑 consensus-factory.io is down

In 13798f9, consensus-factory.io (https://consensus-factory.io) was down:

HTTP code: 0
Response time: 0 ms

Resolved: consensus-factory.io is back up in 5382cea after 6 minutes.

repo: protocol/upptime-pln


text:
🛑 earth23.sbs.tech is down

In c180558, earth23.sbs.tech (https://earth23.sbs.tech) was down:

HTTP code: 0
Response time: 0 ms

Resolved: earth23.sbs.tech is back up in cfcdfdd after 24 minutes.


text:
🛑 proto.school is down

In 95e3822, proto.school (https://proto.school) was down:

HTTP code: 0
Response time: 0 ms

Resolved: proto.school is back up in ca9e015 after 12 minutes.


text:
Topic messages: Smart filters

Add query parameter js-filter with java script  predicate function
Add Nashorn dependency
Filter messages before string inclusion using this function and record
Pass variables to function:
4.1 key (key object)
4.2 value (content object)
4.3 headers
4.4 offset
4.5 partition

UPD: design for the frontend part is already done on figma

Support message preview (partial selection of  json fields)
Support searching by message headers (requested in #780)

#1741

authors: Haarolean soffest

repo: provectus/kafka-ui


text:
timeoutexception when running with jmx enabled

Hello,
i am trying to setup kafka-ui and i receive this message:
org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1646050105263, tries=1, nextAllowedTryMs=1646050749797) timed out at 1646050749697 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: listNodes
the cluster report "offline"
it seems that this happen only if i enable jmx, without jmx the system work correctly.
jmx it's configured correctly on the host (checked with jmxterm) and there isn't network issue from kafkaui --> broker (checked again with jmxterm)
let me know if i can give you more details or you can point me into the right direction

hello, we have same problem. With jConsole i can connect without problem to broker - it also says that connection is insecure so there is no chance that ssl is active.

@ramarro123 @moravcik94
Could you please share your kafka-ui env variables/configmap/etc?

Hello,
I have the same problem and eventualy disabled the JMX port.
In my case, we have 5 clusters (5 environement) all configured alike (no ssl). For one of them, the JMX actually "works" ...  from time to time ... it is extremely slow and often timeout.
Maybe unrelated but the Oracle Java Mission Control Tool is also horribly slow to connect to the same JMX, while jconsole is rather quick.

@ramarro123 It is pretty strange, because TimeoutException you provided is not connected with jmx.
I can recommend to

set KAFKA_ADMIN-CLIENT-TIMEOUT=60000 (env variable) to exclude possibility of kafka api timeouts
add -Dsun.rmi.transport.tcp.responseTimeout=60000 to JAVA_OPTS.

-------stack trace------
[root@fbkppawx /opt/kafkaui]$ less logs/kafkaui.log
at java.rmi/java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:217)
at java.rmi/java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:162)
at com.sun.proxy.$Proxy115.newClient(Unknown Source)
at java.management.rmi/javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2105)
at java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:321)
at java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
at com.provectus.kafka.ui.util.JmxPoolFactory.create(JmxPoolFactory.java:31)
at com.provectus.kafka.ui.util.JmxPoolFactory.create(JmxPoolFactory.java:17)
at org.apache.commons.pool2.BaseKeyedPooledObjectFactory.makeObject(BaseKeyedPooledObjectFactory.java:62)
at org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1012)
at org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:356)
at org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:277)
at com.provectus.kafka.ui.util.JmxClusterUtil.getJmxMetrics(JmxClusterUtil.java:99)
at com.provectus.kafka.ui.util.JmxClusterUtil.lambda$getJmxMetric$3(JmxClusterUtil.java:82)
at java.base/java.util.Optional.map(Optional.java:258)
at com.provectus.kafka.ui.util.JmxClusterUtil.getJmxMetric(JmxClusterUtil.java:82)
at com.provectus.kafka.ui.util.JmxClusterUtil.lambda$getBrokerMetrics$0(JmxClusterUtil.java:73)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)
at reactor.core.publisher.FluxIterable$IterableSubscription.fastPath(FluxIterable.java:340)
at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:227)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:169)
at reactor.core.publisher.MonoCollect$CollectSubscriber.onSubscribe(MonoCollect.java:103)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:165)
at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:87)
at reactor.core.publisher.Mono.subscribe(Mono.java:4399)
at reactor.core.publisher.MonoZip.subscribe(MonoZip.java:128)
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157)
at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:165)
at com.provectus.kafka.ui.service.ReactiveAdminClient.lambda$describeCluster$18(ReactiveAdminClient.java:224)
at org.apache.kafka.common.internals.KafkaFutureImpl$WhenCompleteBiConsumer.accept(KafkaFutureImpl.java:177)
at org.apache.kafka.common.internals.KafkaFutureImpl$WhenCompleteBiConsumer.accept(KafkaFutureImpl.java:162)
at org.apache.kafka.common.internals.KafkaFutureImpl.complete(KafkaFutureImpl.java:221)
at org.apache.kafka.common.KafkaFuture$AllOfAdapter.maybeComplete(KafkaFuture.java:82)
at org.apache.kafka.common.KafkaFuture$AllOfAdapter.accept(KafkaFuture.java:76)
at org.apache.kafka.common.KafkaFuture$AllOfAdapter.accept(KafkaFuture.java:57)
at org.apache.kafka.common.internals.KafkaFutureImpl.complete(KafkaFutureImpl.java:221)
at org.apache.kafka.clients.admin.KafkaAdminClient$5.handleResponse(KafkaAdminClient.java:1882)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.handleResponses(KafkaAdminClient.java:1189)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1341)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1264)
at java.base/java.lang.Thread.run(Thread.java:830)
Caused by: java.net.ConnectException: Operation timed out
at java.base/sun.nio.ch.Net.connect0(Native Method)
at java.base/sun.nio.ch.Net.connect(Net.java:493)
at java.base/sun.nio.ch.Net.connect(Net.java:482)
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:588)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:339)
at java.base/java.net.Socket.connect(Socket.java:603)
at java.base/java.net.Socket.connect(Socket.java:552)
at java.base/java.net.Socket.(Socket.java:475)
at java.base/java.net.Socket.(Socket.java:249)
at java.rmi/sun.rmi.transport.tcp.TCPDirectSocketFactory.createSocket(TCPDirectSocketFactory.java:40)
at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:617)
... 45 common frames omitted
-------stack trace------
----config----
version: '2'
services:
kafka-ui:
image: provectuslabs/kafka-ui
container_name: kafka-ui
ports:
- "8083:8080"
restart: always
environment:
- TZ=Europe/Rome
- KAFKA_ADMIN-CLIENT-TIMEOUT=60000
- KAFKA_CLUSTERS_0_NAME=fabrick_kafka_tst
- KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=uxtequeue01.sg.gbs.tst:9092,uxtequeue02.sg.gbs.tst:9092,uxtequeue03.sg.gbs.tst:9092
- KAFKA_CLUSTERS_0_ZOOKEEPER=uxtequeue01.sg.gbs.tst:2181,uxtequeue02.sg.gbs.tst:2181,uxtequeue03.sg.gbs.tst:2181
- KAFKA_CLUSTERS_0_JMXPORT=5701
- KAFKA_CLUSTERS_0_JMXSSL=false
- KAFKA_CLUSTERS_1_NAME=fabrick_kafka_pre
- KAFKA_CLUSTERS_1_BOOTSTRAPSERVERS=uxppqueue01.sg.gbs.pre:9092,uxppqueue02.sg.gbs.pre:9092,uxppqueue03.sg.gbs.pre:9092
- KAFKA_CLUSTERS_1_ZOOKEEPER=uxppsndzookeep1.sg.gbs.pre:2181,uxppsndzookeep2.sg.gbs.pre:2181,uxppsndzookeep3.sg.gbs.pre:2181
#      - KAFKA_CLUSTERS_1_JMXPORT=5701
#      - KAFKA_CLUSTERS_1_JMXSSL=false
- JAVA_OPTS=-Xmx4096M -Xms1024M -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -Djute.maxbuffer=5242880 -Dsun.rmi.transport.tcp.responseTimeout=60000 -DLOG_PATH=/opt/kafkaui/logs/ -Dlogging.config=file:/opt/kafkaui/config/logback-kafkaui.xml
volumes:
- type: bind
source: /opt/kafkaui/logs
target: /opt/kafkaui/logs
- type: bind
source: /opt/kafkaui/config
target: /opt/kafkaui/config
- /etc/localtime:/etc/localtime:ro

@ramarro123
Kafka open source v2.4

Hi @Haarolean,
I'm a colleague of @ramarro123 and we are using Kafka open source v2.12-2.3.0 at the moment.

@ngmip @MgFbk thanks for the feedback!

The issue is confirmed and we're working on the fix. Thanks for your reports.

@ramarro123 @moravcik94 @ngmip @MgFbk the fix is available in master (and master-labeled docker image as well). Please let us know how it goes.

Hi @Haarolean,
thanks for update.
I've tested just now your new version: provectuslabs/kafka-ui:master
Is it right?
Unfortunately, I'm getting same errors explained in previous comments.
`2022-04-20 11:29:38.185 ERROR 1 --- [boundedElastic-4] c.p.kafka.ui.util.JmxClusterUtil         : Cannot get JMX connector for the pool due to:
java.rmi.ConnectException: Connection refused to host: 172.17.252.73; nested exception is:
java.net.ConnectException: Operation timed out
at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:623)
at java.rmi/sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:209)
at java.rmi/sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:196)
at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:132)
at java.rmi/java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:217)
at java.rmi/java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:162)
at com.sun.proxy.$Proxy113.newClient(Unknown Source)
at java.management.rmi/javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2105)
at java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:321)
at java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
at com.provectus.kafka.ui.util.JmxPoolFactory.create(JmxPoolFactory.java:31)
at com.provectus.kafka.ui.util.JmxPoolFactory.create(JmxPoolFactory.java:17)
at org.apache.commons.pool2.BaseKeyedPooledObjectFactory.makeObject(BaseKeyedPooledObjectFactory.java:62)
at org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1012)
at org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:356)
at org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:277)
at com.provectus.kafka.ui.util.JmxClusterUtil.getJmxMetrics(JmxClusterUtil.java:104)
at com.provectus.kafka.ui.util.JmxClusterUtil.lambda$getJmxMetric$3(JmxClusterUtil.java:87)
at java.base/java.util.Optional.map(Optional.java:258)
at com.provectus.kafka.ui.util.JmxClusterUtil.getJmxMetric(JmxClusterUtil.java:87)
at com.provectus.kafka.ui.util.JmxClusterUtil.lambda$getBrokerMetrics$0(JmxClusterUtil.java:77)
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106)
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:440)
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:527)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:830)
Caused by: java.net.ConnectException: Operation timed out
at java.base/sun.nio.ch.Net.connect0(Native Method)
at java.base/sun.nio.ch.Net.connect(Net.java:493)
at java.base/sun.nio.ch.Net.connect(Net.java:482)
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:588)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:339)
at java.base/java.net.Socket.connect(Socket.java:603)
at java.base/java.net.Socket.connect(Socket.java:552)
at java.base/java.net.Socket.(Socket.java:475)
at java.base/java.net.Socket.(Socket.java:249)
at java.rmi/sun.rmi.transport.tcp.TCPDirectSocketFactory.createSocket(TCPDirectSocketFactory.java:40)
at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:617)
... 30 common frames omitted`
Stack trace is repeated for each cluster node ip, where I'm trying to use JMX port.
No config changes on kafka node config.
Same as reported by @ramarro123:
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=false
-Djava.rmi.server.hostname=172.17.252.73
-Dcom.sun.management.jmxremote.rmi.port=5701
-Dcom.sun.management.jmxremote.port=5701

@Haarolean
I did not get a timeout but with the JMX port set, it needs a huge time to first get the information, nearly a timeout I think.

@MgFbk seems to be a different issue.
@ngmip does it affect UI?

@MgFbk does it work via any other tool like jconsole? It seems like a network issue of a sort

@Haarolean somewhat yes, for 4 minute after kafka-ui started the cluster is marked as "down", the topic list is empty and the consumers list needs like 10 sec to load (while it usually only needs one or two):

Then after what I assumes the successful yet slow JMX query the UI becomes OK :
The logs are quite showing this slowness :
2022-04-21 08:47:44,180 INFO  [main] o.s.b.w.e.n.NettyWebServer: Netty started on port 8080
2022-04-21 08:47:44,197 INFO  [main] c.p.k.u.KafkaUiApplication: Started KafkaUiApplication in 3.419 seconds (JVM running for 4.415)
2022-04-21 08:47:44,211 DEBUG [parallel-1] c.p.k.u.s.ClustersMetricsScheduler: Start getting metrics for kafkaCluster: uat
2022-04-21 08:47:44,222 INFO  [parallel-1] o.a.k.c.a.AdminClientConfig: AdminClientConfig values:
    ... kafka config omited for brevity ...
2022-04-21 08:47:44,258 INFO  [parallel-1] o.a.k.c.u.AppInfoParser: Kafka version: 2.8.0
2022-04-21 08:47:44,258 INFO  [parallel-1] o.a.k.c.u.AppInfoParser: Kafka commitId: ebb1d6e21cc92130
2022-04-21 08:47:44,258 INFO  [parallel-1] o.a.k.c.u.AppInfoParser: Kafka startTimeMs: 1650523664257
2022-04-21 08:51:41,643 DEBUG [boundedElastic-1] c.p.k.u.s.ClustersMetricsScheduler: Metrics updated for cluster: uat
2022-04-21 08:51:41,644 DEBUG [parallel-8] c.p.k.u.s.ClustersMetricsScheduler: Start getting metrics for kafkaCluster: uat
2022-04-21 08:55:39,120 DEBUG [boundedElastic-1] c.p.k.u.s.ClustersMetricsScheduler: Metrics updated for cluster: uat
2022-04-21 08:55:39,120 DEBUG [parallel-11] c.p.k.u.s.ClustersMetricsScheduler: Start getting metrics for kafkaCluster: uat

=> nearly 4 minutes to "getting metrics"

Hi @Haarolean,
thanks to comment

nearly 4 minutes to "getting metrics"

of @ngmip I've solved.
This morning, I've checked again KAFKAUI status for our TST CLUSTER, configured to use JMX_PORT.
It is running well and yesterday KO status was just temporary, time waited wasn't enough.
"Connection time out" exceptions were still there but just for some nodes of our cluster.
More deeply we have missing:
-Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT
on some nodes.
Restarted those nodes, everything is OK using JMX_PORT.
Thanks to all.
Great work.

I found this error when trying to connect to the kafka-operator from strimzi
2022-09-14 23:54:07,130 INFO  [parallel-4] o.a.k.c.a.AdminClientConfig: AdminClientConfig values:
        bootstrap.servers = [kafka-kafka-bootstrap:9092]
        client.dns.lookup = use_all_dns_ips
        client.id =
        connections.max.idle.ms = 300000
        default.api.timeout.ms = 60000
        metadata.max.age.ms = 300000

.......

2022-09-14 23:54:42,139 INFO  [kafka-admin-client-thread | adminclient-12] o.a.k.c.NetworkClient: [AdminClient clientId=adminclient-12] Disconnecting from node -1 due to socket connection setup timeout. The timeout value is 27857 ms.
2022-09-14 23:54:42,263 INFO  [kafka-admin-client-thread | adminclient-15] o.a.k.c.NetworkClient: [AdminClient clientId=adminclient-15] Disconnecting from node -1 due to socket connection setup timeout. The timeout value is 35132 ms.
2022-09-14 23:54:42,264 INFO  [kafka-admin-client-thread | adminclient-15] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=adminclient-15] Metadata update failed
org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: fetchMetadata
2022-09-14 23:54:42,939 INFO  [kafka-admin-client-thread | adminclient-14] o.a.k.c.NetworkClient: [AdminClient clientId=adminclient-14] Disconnecting from node -1 due to socket connection setup timeout. The timeout value is 34255 ms.
2022-09-14 23:54:42,939 INFO  [kafka-admin-client-thread | adminclient-14] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=adminclient-14] Metadata update failed
org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: fetchMetadata
2022-09-14 23:54:43,799 INFO  [kafka-admin-client-thread | adminclient-7] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=adminclient-7] Metadata update failed
org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata
2022-09-14 23:54:43,954 INFO  [kafka-admin-client-thread | adminclient-12] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=adminclient-12] Metadata update failed
org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata

so, I think this thread can be closed and resolved.
The only problem we currently have is that cluster stats collection and jmx metrics gathering are going one after another and  UI is not initialized before jmx metrics collected (see https://github.com/provectus/kafka-ui/issues/1689#issuecomment-1104792531).
To solve this we will not require jmx metrics to be collected before UI initialization and will do jmx collection in parallel with required cluster stats pulling. This will be implemented under https://github.com/provectus/kafka-ui/pull/2190 initiative

authors: Haarolean MgFbk batazor iliax moravcik94 ngmip ramarro123


text:
Add display for missing vote/pre-commit on a given block

Summary
Currently there is no way to determine which validator is missing in the total votes for a given block.
Problem Definition
Currently it is not possible to determine which validator missed a given block
https://explorer.provenance.io/block/1444909
In the above example the number of votes (6/7) is listed but there is no indication of this validator missed the vote (indeed as a related issue the number of missed blocks on a validator also appears to be incorrect... perhaps the missed block count is the direct number of blocks the validator was tasked with cutting vs the total number of pre-commits missed?)
Proposal
Update the block display with some type of indication for which validators properly voted on the block.  One possible solution would be to use a simple green check/red X icon next to the validator in the list of active validators for the block.

For Admin Use

[x] Not duplicate issue
[x] Appropriate labels applied
[ ] Appropriate contributors tagged
[ ] Contributor assigned/self-assigned

@minxylynx Can you take a look at the data here? in the example block above it says 6 of 7 validators voted, in the validators list each validator has didVote flag and they are all set to true. I'm assuming one of them should be false in this or that the validator count is off.
Here are the endpoints in question:
example page https://explorer.provenance.io/block/1444909
validatorCount on https://service-explorer.provenance.io/api/v2/blocks/height/1444909
didVote on https://service-explorer.provenance.io/api/v2/validators/height/1444909?page=1&count=30&sort=desc

@minxylynx Can you take a look at the data here? in the example block above it says 6 of 7 validators voted, in the validators list each validator has didVote flag and they are all set to true. I'm assuming one of them should be false in this or that the validator count is off.
Here are the endpoints in question: example page https://explorer.provenance.io/block/1444909 validatorCount on https://service-explorer.provenance.io/api/v2/blocks/height/1444909 didVote on https://service-explorer.provenance.io/api/v2/validators/height/1444909?page=1&count=30&sort=desc

I figured out what was happening, Its on my end. This shouldn't affect how the FE displays it.

closing as being worked in #167

authors: iramiller minxylynx webbushka

repo: provenance-io/explorer-frontend


text:
Extend marker withrawals

This PR allow any funds to be withdrawn by a WASM smart contract from a marker account (not just the marker denom funds).
Hold until the following PR is merged in Provenance.
https://github.com/provenance-io/provenance/pull/193

Blockchain PR was merged.

repo: provenance-io/provwasm


text:
Properties of Get-GPO object from pipeline not passing into job

What am I doing wrong here? Why are the properties not passing into the job?
$AllGPOs = Get-GPO -All | Sort-Object DisplayName
$AllGPOs[0..4] | Start-RSJob  -ScriptBlock {
    [pscustomobject]@{
    DisplayName  = $_.DisplayName
    GpoStatus    = $_.GpoStatus
    Owner        = $_.Owner
    CreationTime = $_.CreationTime
    Id           = $_.id
    }
} | Wait-RSJob | Receive-RSJob

The objects that come back only have the ID populated. However, doing it not as a job works.
$AllGPOs[0..4] | % { [pscustomobject]@{
    DisplayName  = $_.DisplayName
    GpoStatus    = $_.GpoStatus
    Owner        = $_.Owner
    CreationTime = $_.CreationTime
    Id           = $_.Id
    }
}

It appears that other objects go just fine. Get-Process and Get-AdComputer both pass all their properties.

That's weird. It's probably some quirkiness with that particular cmdlet. I'll check this out tomorrow and see if I can reproduce the issue and maybe see why it is happening.

It looks like it was what I suspected. Similar to some of the ActiveDirectory module commands, you will have to pipe your results to Select * to ensure that the data is passed correctly to an RSJob.
$AllGPOs[0..4] | Select * | Start-RSJob  -ScriptBlock {
     [pscustomobject]@{
     DisplayName  = $_.DisplayName
     GpoStatus    = $_.GpoStatus
     Owner        = $_.Owner
     CreationTime = $_.CreationTime
     Id           = $_.id
     }
 }  | Wait-RSJob | Receive-RSJob

When I did that, everything worked as expected. Without doing that, I saw the same issue with the lack of properties being displayed.

@cwhits Have you had a chance to look a this again with my suggestion?

Yep. That works.

authors: cwhits proxb

repo: proxb/PoshRSJob