text:
Distributed-mode (Discrepancy in total stats) : Locust master fails to acknowledge the "last stats" from slave during end of test

Description :
I am executing the locust tests under distributed mode in a Kubernetes cluster. At the moment I am testing only 1 master - 1 slave configuration. At end of the tests, it is found that locust master fails to acknowledge the last stats from slave. On completion, the Locust master prints the following statement "Time limit reached. Stopping Locust." and sends a quit signal to the slave/s with a wait time of  0.5 seconds to get reports from slaves. This is described as part of below code
main.py
` def spawn_run_time_limit_greenlet():
            logger.info("Run time limit set to %s seconds" % options.run_time)
            def timelimit_stop():
                logger.info("Time limit reached. Stopping Locust.")
                runners.locust_runner.quit()
            gevent.spawn_later(options.run_time, timelimit_stop)`

runners.py
def quit(self):
     for client in self.clients.all:
         self.server.send_to_client(Message("quit", None, client.id))
     gevent.sleep(0.5) # wait for final stats report from all slaves
     self.greenlet.kill(block=True)

However in most of the cases it is found that by the time slaves sends the report to master, its shutdown is already invoked stopping it from receiving any stats. Here is one example of the locust master-slave logs
As per master logs (as shown below), the signal was sent around 2020-06-13 21:04:13,968 and then it waited for exactly 0,5 seconds and the shutdown process began at 2020-06-13 21:04:14,469
[2020-06-13 21:04:13,968] locust-master-1-n9qxq/INFO/locust.main: Time limit reached. Stopping Locust.
[2020-06-13 21:04:14,469] locust-master-1-n9qxq/INFO/locust.main: Shutting down (exit code 1), bye.
[2020-06-13 21:04:14,470] locust-master-1-n9qxq/INFO/locust.main: Cleaning up runner...
[2020-06-13 21:04:14,971] locust-master-1-n9qxq/INFO/locust.main: Running teardowns...
As per the slave logs (as shown below), it received the message from master only at [2020-06-13 21:04:15,727] and by this time the master was already in shutdown process.
[2020-06-13 21:04:15,727] locust-slave-1-jqpq8/INFO/locust.runners: Got quit message from master, shutting down...
[2020-06-13 21:04:16,632] locust-slave-1-jqpq8/INFO/locust.main: Shutting down (exit code 0), bye.
[2020-06-13 21:04:16,632] locust-slave-1-jqpq8/INFO/locust.main: Cleaning up runner...
[2020-06-13 21:04:16,632] locust-slave-1-jqpq8/INFO/locust.main: Running teardowns...
As you can see, the master didn't rely on "acknowledgment signal from slave" rather it was depending on a wait time of 0.5 seconds which should not be the said criteria. This has resulted in loss of data wherein the master doesn't have the entire information of stats from the slave. So the aggregate results at master is less than the total requests tested by slave.
Expected behavior
The master should wait for the slave to send back the acknowledgement signal and the part of keeping time as the constant should be removed.
Environment

Python version: 3.8
Locust version: All the versions
Locust command line that you ran:
Locust master : locust --no-web --expect-slaves=1 -c 10 -r 2 --run-time=10m --csv=<> --logfile=<> -f /locustfile.py -H   --master
Locust slave : locust --no-web -f /locustfile.py -H -H    --slave --master-host= --master-port=

Hi! Can you do this consistently? What about if you run it on a single machine?
I've had no issues getting the last samples since that sleep was introduced. The size of the sleep is designed to account for any latency between master & slave, but nothing more, so if it takes a full second or so before the slaves get the message to shut down then it will not work. Are you doing anything particular in your locustfile that might block the slaves from receiving the message for that long?

@cyberw Hi, Thank you for the response. Correct me if I am wrong but by single machine do you mean running both master and slave on a single host? In that case, there is no issue at all the request-response between slave and master is quite fast.
However this issue seems to be coming up (like 7 out of 10 cases) where the master and slave are spinned as separate pods in the "same" Kubernetes namespace. So  definitely time should not be a criteria to account for latency rather master should have a different mechanism to poll the response from slave, only then it should go to shutdown.
I haven't added anything else apart from 2 API calls in locust file. It looks something like this
@seq_task(2)
def sample_task_one(self):
   <Call to an endpoint and capture results>

@seq_task(1)
def sample_task_two(self):
    <Call to an endpoint and capture results>

class UserBehavior(TaskSequence):
    tasks = {sample_task_one: 1, sample_task_two: 1}

class WebsiteUser(HttpLocust):
    task_set = UserBehavior
    wait_time = between(5.0, 10.0)

Correct me if I am wrong but by single machine do you mean running both master and slave on a single host? In that case, there is no issue at all the request-response between slave and master is quite fast.

Yes, that was what I meant.
When I added this sleep (quite recently actually, until then we were always dropping the last few requests on the workers :-/ ), I first tried to do it a more "safe" way, but ended up deadlocking somehow.
If you do have >0.5s latency between master and slave (introduced by k8s, underlying hardware, something else) then I'm afraid losing the last results is expected and unlikely to be fixed any time soon - unless you fix it yourself and make a PR :)
I guess for your purposes you could just fork / monkey patch it to be 5 seconds instead.

I haven't added anything else apart from 2 API calls in locust file. It looks something like this

Looks like your test is very "normal" (assuming your endpoint calls are just HttpUser requests and "capture result" doesnt actually do anything blocking)

@cyberw - is there any common way to inject config changes in Locust? Like prepare an ini/json config file that would override defaults in any way? Instead of doing monkey patching this possibly would help in this particular problem.

All of the "normal" configuration can be set using command line, environment variables or config file https://docs.locust.io/en/latest/configuration.html
Settings that havent been made configurable can, of course, not be configured using config file, which is why you're stuck with forking or monkey patching.
For this particular problem I think we should probably take another fixing the code (waiting for the last result set) instead of making the wait configurable (where I got stuck with a deadlock). But I dont have time, so someone else will have to do it.

authors: akash-work cyberw domik82

repo: locustio/locust


text:
Cut to only 5% free space on the top of the graphs

https://echarts.apache.org/en/option.html#yAxis.boundaryGap
Closes #1440

Cool! Thanks!

source: gharchive/pull-request

authors: benallard cyberw

license: mit

license_type: permissive

license_source: bigquery


text:
Feature/es module commonjs

ES module and CommonJS
Current approach is the following: Exporting dist with esm and commonjs for non-react jsx, which is logary, logary-plugin-browser, logary-plugin-node. logary-plugin-nextjs and logary-plugin-react is only using commonjs.
This means that when we run with-nextjs and with-nextjs-app we are using commonjs but when we are bundling logary-browser we are using esm with tree-shaking to get minimal bundle min file.
Here is some findings regarding es2015 usage as module rather than CommonJS
When using es2015 as module type in tsconfig.js you will get the following error:
SyntaxError: Cannot use import statement outside a module:

https://github.com/vercel/next.js/issues/9607

“When using type: module in package.json on Node v13.1.0 and an ESM next.config.js format, Next throws an error due to the config resolution logic relying on require.”

https://fredriccliver.medium.com/syntaxerror-cannot-use-import-statement-outside-a-module-69182014b8c6
https://github.com/vercel/next.js/discussions/15957
https://github.com/TypeStrong/ts-node/issues/922

“You’re trying to execute use it as an ES2015 module, but node.js currently understands CommonJS.”

The recommended approach here is to use “type”: “module” defined in package.json.
But when setting type as module it will result in the following error:
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module:
Here is a possible solution which does not work:

https://github.com/eslint/eslint/issues/12319#issuecomment-535799024

Another possible approach is to use both ES Module and CommonJS which is allowing the consumer to choose.

https://blog.logrocket.com/publishing-node-modules-typescript-es-modules/

“Node doesn’t yet support ES modules out of the box. It’d be great to publish a CommonJS version, too, so Node works with no extra effort. ES module support is coming in Node 13 and beyond, but it’ll be a while before the ecosystem catches up.”

Some more readings in the subject:
“they are already interpreted with a conflicting opinion. TS produces a JS file that the JS code it produces does not point to. ES6 is the closest thing to an officially correct way to do JavaScript, so if the ES6 code produced by TypesScript is wrong, this is a bug plain and simple. There is no need to wait for any proprosals and such, just fix TypeScript's bug. But today people feel that if they don't find fault with something and put it through 10 layers of proposal before acting, then they are not acting intellectually. Give me a break.”

https://github.com/microsoft/TypeScript/issues/16577

+1 but squash when you merge.

authors: haf ols

repo: logary/logary-js


text:
新增loggie crd定时同步

Proposed Changes:

Which issue(s) this PR fixes:
Fixes #637
Additional documentation:

// TODO: 暴露出配置

repo: loggie-io/loggie

license_source: github-api


text:
integration test failure: "Kafka Input can ingest 37 apache log lines from Kafka broker"

one of kafka's integration test is occasionally breaking the master or 5.x branches
see more on: https://travis-ci.org/elastic/logstash/builds/167292361
Failures:

  1) Kafka Input can ingest 37 apache log lines from Kafka broker
     Failure/Error: wait(timeout_seconds).for { @fixture.output_exists? }.to be true

       expected true
            got false
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/rspec-wait-0.0.9/lib/rspec/wait/handler.rb:13:in `handle_matcher'
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/rspec-wait-0.0.9/lib/rspec/wait/handler.rb:10:in `handle_matcher'
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/rspec-wait-0.0.9/lib/rspec/wait/handler.rb:9:in `handle_matcher'
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/rspec-wait-0.0.9/lib/rspec/wait/target.rb:30:in `to'
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/rspec-wait-0.0.9/lib/rspec/wait/target.rb:44:in `with_wait'
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/rspec-wait-0.0.9/lib/rspec/wait.rb:28:in `with_wait'
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/rspec-wait-0.0.9/lib/rspec/wait/target.rb:44:in `with_wait'
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/rspec-wait-0.0.9/lib/rspec/wait/target.rb:30:in `to'
     # ./specs/kafka_input_spec.rb:20:in `(root)'
     # /home/travis/.rvm/gems/jruby-1.7.25/gems/rspec-wait-0.0.9/lib/rspec/wait.rb:46:in `(root)'

@talevy pinging you as you're more familiar with the plugin

I'll take a look, thanks

assigned @suyograo since he said he is taking a look

Closing since its no longer an issue

source: gharchive/issue

authors: jsvd suyograo talevy

repo: logstash-plugins/logstash-input-kafka


text:
feature request: module federation

This template worked fine for our corporation for months, a huge thanks from me and the team! However, we have now come to a stage where we need to develop functionalities independently. I have heard and dug into this term "module federation" as a paradigm for frontend micro-service, and I was wondering if there's any possibility that an integration of the "vite-plugin-federation" library into this framework is possible.
TLDR: Any suggestion or possible upgrades in the future such that frontend microservice paradigm is possible?
Best regards,
JU PING

I've only skimmed vite-plugin-federation, but I think it probably won't work properly unless you unset manualChunks in vite.config.ts.
https://github.com/logue/vite-vue2-vuetify-ts-starter/blob/7dc52c7ec26eadbafa15a0af395c0b02a2d0b408/vite.config.ts#L111-L131

authors: logue ngjuping

license: MIT


text:
Update scalatest to 3.2.11

Updates org.scalatest:scalatest from 3.2.2 to 3.2.11.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalatest", artifactId = "scalatest" } ]

labels: test-library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #66.

repo: loicknuchel/TypedAPI


text:
hiding an object from graphql should automatically hide any field edges associated with it

e.g.
new object Foo with `hideFromGraphQL: true`

in object Bar
foo_id: UUIDType({
  schema:'Foo'
})

in graphql object bar_type.ts, we try and generate foo referencing object Foo when we shouldn't since hidden from graphql
workaround is
foo_id: UUIDType({
hideFromGraphQL:true,
  schema:'Foo'
})

we shouldn't need that

related, even with hideFromGraphQL:true, seem to have the field in graphql edit apis. i don't think we want that??

repo: lolopinto/ent


text:
Errors in log

Hi!
I have errors in log:

Add-on doesn't  work.
Actual Home Assistant with Supervisor
rpi3 B+
Bluetooth device used: Built-in
Thanks!

Make sure Bluetooth is enabled on the pi.
If it is, do you have any other process / application using Bluetooth?
Some users reported issues with the built in Bluetooth, can you try a Bluetooth dongle?

Same problem here!
Do i need to make any particular bluetooth side presets?
For example: should i launch bluetoothctl and trust the MAC address of the balance? Or put the dongle in discovery/discoverable mode?
I'm using a ugreen USB dongle in Home Assistant supervised... I don't think there is anything else that is using bluetooth but I don't even know how to verify it.
Thanks!

Same problem here!
Do i need to make any particular bluetooth side presets?
For example: should i launch bluetoothctl and trust the MAC address of the balance? Or put the dongle in discovery/discoverable mode?

Once you've got the MAC of the scale, you don't need anything specific

I'm using a ugreen USB dongle in Home Assistant supervised... I don't think there is anything else that is using bluetooth but I don't even know how to verify it.

Are you on a Pi or other device? By default hci0 will be used (most likely the built-in BT if you have a Pi) so you'd need to find out what hci port is used for the dongle and set this in the settings... You also need to ensure the dongle is BTLE compatible (BT 4.0 or higher)

I'm using home assistant supervised (with docker) on Debian 10.
The dongle is this: https://www.amazon.it/gp/product/B01LX6HISL/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1 and i'm using a custom PC, so no raspberry or other similar hardware.
In my Debian10 i can correctly see the dongle hci0 connectect and i can use bluetoothctl to scan bt devices and eventually connect to them.

@Stinocon @msslava
This would imply an issue with the pluepy library and/or OS or rights...
Have a look at this link for a few suggestions and let me know if any fixes your issue:
https://github.com/IanHarvey/bluepy/issues/218

So... After some trial and error now i have this:
-------------------------------------
2020-09-01 16:59:35 - Starting Xiaomi mi Scale...
2020-09-01 16:59:35 - Loading Config From Add-On Options...
2020-09-01 16:59:35 - Config Loaded...
2020-09-01 16:59:42 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-01 17:00:55 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-01 17:01:32 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-01 17:02:09 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-01 17:02:46 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-01 17:03:52 - Publishing data to topic miscale/Alessandra/weight: {"Weight":"14.10","BMI":"10.00","Basal Metabolism":"778.84","Visceral Fat":"1.00","TimeStamp":"2020-09-01-17:03:52"}
2020-09-01 17:03:52 - Data Published ...
2020-09-01 17:04:33 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-01 17:05:10 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-01 17:05:41 - Publishing data to topic miscale/Stefano/weight: {"Weight":"75.00","BMI":"22.89","Basal Metabolism":"1621.36","Visceral Fat":"10.20","Lean Body Mass":"60.70","Body Fat":"20.14","Water":"54.78","Bone Mass":"3.05","Muscle Mass":"56.85","Protein":"21.01","Body Type":"Balanced","Metabolic Age":"27","TimeStamp":"2020-09-01-17:05:41"}
2020-09-01 17:05:41 - Data Published ...
2020-09-01 17:06:22 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-01 17:07:35 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-01 17:08:05 - Publishing data to topic miscale/Stefano/weight: {"Weight":"75.00","BMI":"22.89","Basal Metabolism":"1621.36","Visceral Fat":"10.20","TimeStamp":"2020-09-01-17:08:05"}
2020-09-01 17:08:05 - Data Published ...
2020-09-01 17:08:47 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-01 17:09:24 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)

Apparently now is reading something... But error reamains. In addition... I can't see the data read inside Home Assistant...
This is the configuration of my addon:
HCI_DEV: hci0 MISCALE_MAC: 'E4:24:65:CA:95:CA' MQTT_PREFIX: miscale MQTT_HOST: 127.0.0.1 MQTT_USERNAME: mqttuseradmin MQTT_PASSWORD: mypass MQTT_PORT: 1883 TIME_INTERVAL: 30 MQTT_DISCOVERY: false MQTT_DISCOVERY_PREFIX: homeassistant USER1_GT: 71 USER1_SEX: male USER1_NAME: Stefano USER1_HEIGHT: 181 USER1_DOB: '1993-07-14' USER2_LT: 73 USER2_SEX: female USER2_NAME: secondperson USER2_HEIGHT: 180 USER2_DOB: '1995-02-01' USER3_SEX: female USER3_NAME: empty USER3_HEIGHT: 150 USER3_DOB: '1990-01-01'
I don't know if it was of any use but I had to first trust and then connect to the scale via bluetoothctl.

Thanks @Stinocon
Can you confirm what you changed so data now shows up?
Your add-on config doesn't look right. your USER1_GT value is 71 and your USER2_LT is 73.
Weights should not overlap.
With your config anything greater than 71 (incl anything between 71.0, and 73.0) will be allocated to User1, anything else (Less than 71) will be allocated to User2
Is this intended?
Last but not least, it looks as though the data is being posted to MQTT. Can you share your HA sensors config (since you've not enabled MQTT_DISCOVERY)
Have you also checked with a tool like MQTT Explorer that data is actually being published?
Last but not least, have a look at these 2 threads which may help:
https://github.com/IanHarvey/bluepy/issues/150
https://github.com/zewelor/bt-mqtt-gateway/issues/59

Thank you for your patience and understanding first of all.
I haven't changed anything except just throwing bluetoothctl on my Debian and trickle the MAC of the scale and then connect:
bluetoothctl
MACADDRESS trust
connect MACADDRESS

After this I tried to weigh myself by standing near the bluetooth receiver and I started to see data arriving in the addon logs. Unfortunately, however, as indicated I do not see anything in the sensor.
I created the sensor like this (inside my sensor.yaml):
  - platform: mqtt
    name: "Peso Stefano"
    state_topic: "miScale/Stefano/weight".
    value_template: "{{ value_json['Weight'] }}".
    unit_of_measurement: "kg"
    json_attributes_topic: "miScale/Stefano/weight".
    icon: mdi:scale-bathroom

As for the addon configuration I have inverted the weights, so now everything above 73 is attributed to user1 while everything below 71 is attributed to user2. Thanks!
Regarding MQTT: I had to disable the discovery because it actually gave me some problems, not knowing exactly which was the prefix of the MQTT discovery (I don't think to have set it in mosquitto) I preferred to disable it.
I didn't know about the tool that you indicated to me. I will surely do some tests!
About the linked issues... I can tell you that I tried to give the following commands:
sudo apt-get install libcap2-bin
sudo setcap 'cap_net_raw,cap_net_admin+eip' `readlink -f \`which python3\``.
sudo setcap 'cap_net_raw+ep' `readlink -f \`which hcitool\```.

As also indicated in the official HomeAssistant guide: https://www.home-assistant.io/integrations/bluetooth_le_tracker/ which I suppose is a way to overcome the problems of management of the bluetooth part by non-root users.
For now the easiest solution seems to me is to insert in crontab:
20 4 * * hciconfig hci0 reset
20 10 * * hciconfig hci0 reset
20 16 * * hciconfig hci0 reset
20 22 * * hciconfig hci0 reset

As indicated in https://github.com/zewelor/bt-mqtt-gateway/issues/59#issuecomment-518085563
Unfortunately it is the first bluetooth device that I try to insert, so I don't know how to behave with other devices.

After this I tried to weigh myself by standing near the bluetooth receiver and I started to see data arriving in the addon logs. Unfortunately, however, as indicated I do not see anything in the sensor.

If the addon logs shows data being pusblished, it would indicate that the addon managed to connect to the MQTT server and publish the message to the topic.
If you see nothing in HA, first try to check that the data is indeed received by the MQTT Server (try a tool like MQTT Explorer)
and if that does show up double check your sensor config... Topics are case sensitive...
As a side note, a different user reported that hciconfig hci0 reset fails every single time, but
hciconfig hci0 down
[wait 1 sec]
hciconfig hci0 up

does work. I've made a change (not yet published) to trial this method of reset.
Can you try to use a local add on and test this version?
you need to unzip and save under the addons folder on your Pi
Then install it via the supervisor (you probably will need to remove my "official" add on and instead install the one under local add-ons
xiaomi_mi_scale.zip

I installed your addon, configured and restarted Home Assistant.
I also tried to use MQTT Explorer and I see the messages arrive correctly:

Anyway, I don't see the sensor populating with data... Probably something wrong with the MQTT topic?
Now the addon configuration is as follows (partial - mqtt discovery enabled):
HCI_DEV: hci0
MISCALE_MAC: 'E4:24:65:CA:95:CA'
MQTT_PREFIX: miscale
MQTT_HOST: 127.0.0.1
MQTT_USERNAME:
MQTT_PASSWORD:
MQTT_PORT: 1883
TIME_INTERVAL: 90
MQTT_DISCOVERY: true
MQTT_DISCOVERY_PREFIX: homeassistant
USER1_GT: 73
USER1_SEX: male
USER1_NAME: Stefano
USER1_HEIGHT: 181

While the sensor configuration is as follows:
  - platform: mqtt
    name: "Peso Stefano"
    state_topic: "miscale/Stefano/weight"
    value_template: "{{ value_json['Weight'] }}"
    unit_of_measurement: "kg"
    json_attributes_topic: "miscale/Stefano/weight"
    icon: mdi:scale-bathroom

This is the MQTT Explorer sidebar:

In Add-on logs:
2020-09-02 11:22:56 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-02 11:29:19 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-02 11:34:01 - Publishing data to topic miscale/Stefano/weight: {"Weight":"75.50""Weight Unit":"kg","BMI":"23.05","Basal Metabolism":"1628.80","Visceral Fat":"10.45","Lean Body Mass":"60.69","Body Fat":"20.67","Water":"54.42","Bone Mass":"3.05","Muscle Mass":"56.84","Protein":"20.87","Body Type":"Balanced","Metabolic Age":"29","TimeStamp":"2020-09-02-11:34:01"}
2020-09-02 11:34:01 - Data Published ...
2020-09-02 11:35:42 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-02 11:38:54 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)
2020-09-02 11:43:42 - Bluetooth connection error: Failed to execute management command 'scanend' (code: 11, error: Rejected)

Thanks!

I had a typo error in the code after a recent change.
Try this pls
xiaomi_mi_scale.zip

Now we have different error!
2020-09-02 14:05:50 - Starting Xiaomi mi Scale... 2020-09-02 14:05:50 - Loading Config From Add-On Options... 2020-09-02 14:05:50 - Config Loaded... 2020-09-02 14:05:50 - Discovery Completed... 2020-09-02 14:05:50 - Bluetooth connection error: Failed to execute management command 'le on' (code: 17, error: Invalid Index) 2020-09-02 14:07:20 - Bluetooth connection error: Failed to execute management command 'le on' (code: 17, error: Invalid Index)

this better?
xiaomi_mi_scale.zip

Same error :(
-------------------------------------
2020-09-02 18:05:27 - Starting Xiaomi mi Scale...
2020-09-02 18:05:27 - Loading Config From Add-On Options...
2020-09-02 18:05:27 - Config Loaded...
2020-09-02 18:05:27 - Discovery Completed...
2020-09-02 18:05:27 - Bluetooth connection error: Failed to execute management command 'le on' (code: 17, error: Invalid Index)

Is other software using the bluetooth interface at the same time?

Is other software using the bluetooth interface at the same time?

Is other software (other home assistant addons or integrations) using the bluetooth interface at the same time?

I don't think so, but I wouldn't even formally know how to verify it.
To date I have no other extensions that manage the bluetooth, not even custom component in HACS, or other program.
Is there a way to verify this?

what do you see when you type hciconfig ?
(you may need sudo depending on how/where you type this from)
You should get something like this (keyword being UP RUNNING
hci0: Type: Primary Bus: USB
 BD Address: xx:xx:xx:xx:xx:xx ACL MTU: 310:10 SCO MTU: 64:8
 UP RUNNING
 RX bytes:1794 acl:0 sco:0 events:102 errors:0
 TX bytes:1772 acl:0 sco:0 commands:102 errors:0

Sorry for the delay:
hci0:   Type: Primary  Bus: USB         BD Address: 00:1A:7D:DA:71:11  ACL MTU: 310:10  SCO MTU: 64:8         UP RUNNING         RX bytes:676 acl:0 sco:0 events:47 errors:0         TX bytes:3157 acl:0 sco:0 commands:623 errors:0

It all looks ok
I have no idea why it won't work.
All I can suggest at this stage is to try a different USB BT Dongle.

0.1.8 released. Should improve things a bit, but it's not much different than the version posted above, at least from a BT point of view.
I'm afraid I have no solution for this which is really a bluepy issue,
Closing this thread

authors: Stinocon lolouk44 msslava stefangries


text:
Removing beta banners

No reason to have these now that we're moving to 1%.

:+1:

repo: lonelyplanet/rizzo


text:
the database is not syncing

nothing happens after startup (including no synchronization with the database), there are no obvious errors either. What could be the problem?
user23@host_nl:~/meilisync$ docker-compose up
Creating meilisync_meilisync_1 ... done
Attaching to meilisync_meilisync_1
meilisync_1  | 2023-11-19 16:22:45.955 | DEBUG    | meilisync.main:_:33 - plugins=None progress=Progress(type=<ProgressType.file: 'file'>) debug=True source=Source(type=<SourceType.postgres: 'postgres'>, database='pic_monitor', user='user23', host='my_host_ip', port=5432, password='my_db_password') meilisearch=MeiliSearch(api_url='http://my_host_ip:7700', api_key='meilisearch_api_key', insert_size=1000, insert_interval=10) sync=[Sync(plugins=None, table='photos', pk='id', full=True, index='photos', fields=None)] sentry=None
meilisync_1  | 2023-11-19 16:22:50.662 | INFO     | meilisync.main:_:82 - Full data sync for table "pic_monitor.photos" done! 49281 documents added.
meilisync_1  | 2023-11-19 16:22:50.662 | INFO     | meilisync.main:_:90 - Start increment sync data from "postgres" to MeiliSearch...
meilisync_1  | 2023-11-19 16:22:50.667 | DEBUG    | meilisync.main:_:93 - progress={'start_lsn': '0/F10DEB0'}

my config.yml:
debug: true
progress:
  type: file
source:
  type: postgres
  host: my_host_ip
  port: 5432
  user: user23
  password: "my_db_password"
  database: pic_monitor
meilisearch:
  api_url: http://my_host_ip:7700
  api_key: 'meilisearch_api_key'
  insert_size: 1000
  insert_interval: 10
sync:
  - table: photos # my table name in database pic_monitor
    index: photos # my index in meilisearch
    full: true

@rsymodo I'm running into the same thing – what was the fix?

authors: Nickersoft rsymodo


text:
[BUG] UI - The option available on clicking snapshots doesn't scroll with the screen.

Describe the bug
The option available revert, backup, delete on clicking snapshot doesn't scroll with the screen.
To Reproduce
Steps to reproduce the behavior:

Create a volume and attach it with a pod.
Write into the volume and take a snapshot.
Go to the volume detail page.
The option available on clicking snapshots doesn't scroll with the screen.

Expected behavior
The option should scroll with the page.
Environment:

Longhorn version: longhorn  master - 45630f4
Kubernetes version: 18.6
Node OS type and version: 18.04

Verified with longhorn-UI 02fb59c
Validation - Pass
The menu on clicking snapshot scrolls with the page scroll. The space between snapshot branches is wider now.

repo: longhorn/longhorn


text:
BMO will reply with the same answer as the last query

Hey there,
I use the BMO Chatbot plug-in with the different installed models: llama3.2, llama2, and codegemma. It worked fine for a couple of days. I am on Windows 11 and using Obsidian: v1.7.4. Regardless of the query or model, I get the same answer two days ago.
I've disabled and enabled the plugin again, restarted the machine several times, looked for similar issues online, and could not find an answer.
I am getting the same answer repeatedly.
The only thing left to do is uninstall the plugin and try to install it again. Since I cannot use the plugin, I'll do that, but I wanted to report it first.

I had some similar issues, I disable "Enable Reference Current Note" and start referencing my notes using [[name of your note]] , everytime i want to reference another note update [[name of your note]]


text:
'secure' cookie option

any reason you provided a default for the 'secure' option?  the cookies module dynamic default seems more intuitive than your hardcoding of false

Thank you, olsonpm.
It is a very good suggestion. I will remove the default value for secure option.

released v3.0.3

Wow - thanks so much.
btw, this module was well written and documented.  I had the pleasurable experience of writing code that just worked.

authors: longztian olsonpm

repo: longztian/koa-session-minimal


text:
希望添加命令行批量处理的功能

Describe the bug
指定列表文件的批处理功能
Expected behavior
是否允许我传入txt文件路径，txt文件路径中存在着我需要查杀的maya文件路径，一行一个，每一行逐步清理下来。
批处理是否还可以增加选项，可以让我选择是否需要清理完成后保存，还是在原maya文件边上创建一个backup文件夹放入其中，清理当前保存。

什么都不做
保存当前
备份一份在原maya文件边上的backup文件夹内，再保存当前

好的，晚上添加

已经支持

authors: loonghao uncleschen


text:
Question: Can I make multiple builds?

I would like to make multiple builds of my Slate documentation.  We have some pages we are attaching using includes that are necessary for developers of our APIs.  While we don't want to include other mark down files for the consumers of our APIs.

Can I make two index files?
Can I do both builds at the same time?

Yup! If you copy src/index.md.html to a new file, like src/foo.md.html, you'll get a second document at http://localhost:4567/foo.html. Does that solve your problem?

Almost.  After we have made some changes, I am executing the following from the command line:
bundle exec middleman build --clean
When I do so I get a nice build folder with the appropriate file structure and the index.html
build -> index.html
If I needed two version of "builds"
One for developers: with some code like the following that is included in the index.html.md file:
`
includes:

general/introduction
how-tos/overview
how-tos/DeveloperGuidelines
how-tos/GettingDevelopersStarted
how-tos/Example1
how-tos/Example2
and another build that will be used for non-developers:
includes:
general/introduction
how-tos/overview
how-tos/Example1
how-tos/Example2
`

I would like to see the following in my root folder structure when I run the following:
bundle exec middleman build --clean
root -> build-dev -> index.html
root -> build -> index.html
does that make my question clear?
I think what I would need to do is have two separate files (index.html.md) one for the Developers docs, and one for the non-developers, and then run (bundle exec middleman build --clean) this twice.  Would this be correct?

almost, I responded back to the closed issue request.
On Tue, Apr 11, 2017 at 1:50 PM Robert Lord notifications@github.com
wrote:

Closed #748 https://github.com/lord/slate/issues/748.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/lord/slate/issues/748#event-1038900874, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AX_rkDQrw-wCZMArkIAaSJ5e2fnZQiAYks5ru8uIgaJpZM4M6fbp
.

@javadrum I think going with two separate index.html.md files is the way to go here. Middleman, Slate's source Static Generator, doesn't allow you to customize a specific index-- it just renders the source markdown files.
What I would suggest is using something like Grunt or Gulp to run a task to swap your files out in a task, where you have say:

index-dev.html.md  - a copy of index.html.md with your dev-friendly docs
index-simple.html.md- a copy of index.html.md with your non-dev docs

Then, create a grunt task to copy index-dev.html.md to index.html.md, build the site, then copy the build folder to something like a /build-devfolder, then it could repeat the same for index-simple.html.md-- copy it over to index.html.md and then copy the "simple" build out to /build-simple.
You basically will have three build folders after you run a task:

build (which is the full build of the site for whichever was built last)
build-dev (which is the dev version)
build-simple (which is the simple version)

Then, it's just a matter of rsync-ing (or whatever your deployment method is) the correct build to your server in the correct location.
Let me know what you think!

That is perfect..   your answer is what I was envisioning, but I just wanted to make sure I wasn't missing something.

Just to further clarify, the steps would be, effectively, all shell scripts like this:
cp index-dev.html.md index.html.md
bundle exec middleman build --clean
cp /build /build-dev
cp index-simple.html.md index.html.md
bundle-exec middleman build --clean
cp /build /build-simple
echo "double build complete!"

authors: javadrum lord mbuttler

repo: lord/slate


text:
Add a write command

The command would simply write to a file. This can be useful for creating files without source templates, however the main idea is to allow the command to write to specific parts of a file, if they are for example json or yaml files.
# create a file
write: some stuff
to: myfile.txt

# add a dependency
write: '^6.1.0'
to: package.json
at: 'devDependencies/@typescript-eslint~1eslint-plugin'

# add a workflow step
write: |
      - name: check code style
        run: npm run lint
to: .github/workflows/publish.yml
after: jobs/run/steps/4

JSON Pointer format is used here. I suspect json-source-map and mapped-yaml (or equivalent libraries) can be used to precisely alter some JSON or YAML file while maintaining its formatting as much as possible.

This command could also benefit from the proposed filtering of files if it were to support glob patterns, specifically to allow the content to also be evaluated per file. This could basically remove the need for the proposed append and prepend commands.
write: '{{ license }}\n{{ content }}'
to: **/*.(jsx?|tsx?|cpp|java)
where:
  steps:
    - read: content
      from file: __file__
  eval: '{{ content | not matches: /.../ }}'

base command added via 8ce8644. additional proposals (filtering and JSON / YAML support) should be tracked in separate issues (as they also involve multiple commands).

repo: loreanvictor/tmplr


text:
Enact temps immediately

With the current timing of RL heartbeats and Share polling, the Loop app sits with a new Recommended Basal for 4 minutes before enacting it. It should instead do an enact immediately after generating the recommendation.

Ah yes, in the G4 + x22 case, it's not executing the BG polling and pump polling in a deterministic order: https://github.com/loudnate/Loop/blob/master/Loop/Managers/DeviceDataManager.swift#L85-L89
That should be changed.

authors: loudnate scottleibrand

repo: loudnate/Loop


text:
collecting monitors into groups

⚠ Please verify that this feature request has NOT been suggested before.

[X] I checked and didn't find similar feature request

🏷 Feature Request Type
UI Feature
🔖 Feature description
It would be nice to have a function to combine monitors into groups. I know there are tags, but it's a slightly different function.
✔ Solution
Creating groups to which monitors would be added. After clicking +, a list of monitors in a given group would expand.
❓ Alternatives
No response
📝 Additional Context
No response

+1 for this request.. For operators who have thousands of objects to monitor, it will be a very nice feature to have.
Also if possible, ability to move existing objects into groups when this feature is added

+1 for this request.. For operators who have thousands of objects to monitor, it will be a very nice feature to have.
Also if possible, ability to move existing objects into groups when this feature is added

Take a look at my pr. I think exactly what you both want ;)
https://github.com/louislam/uptime-kuma/pull/2693

@qristo76  a very similar feature has been implemented+merged.
Any version after 1.22.0-beta.0 supports this.

Could you close this Issue, if this has been resolved? (ony you or Louis can close issues)

How do you create a group?  I only have this option, but it isn't clear exactly how to create a new group.

How do you create a group?  I only have this option, but it isn't clear exactly how to create a new group.

Just create a new monitor and select "Group" in the type drop-down ;)

How do you create a group?  I only have this option, but it isn't clear exactly how to create a new group.

Just select "Group" in the type drop-down when creating a new monitor ;)

Dropdown menu not avaliable when creating new monitor

Oh, i see. Thanks!

I don't know how I missed this, thank you.
For completeness:

authors: CommanderStorm ErebusBat a1git julian-piehl mirivlad qristo76

repo: louislam/uptime-kuma


text:
badges at port does not work

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

Description
I try to retrieve a status - badges at a service, but I only get a N/A as answer!
Screenshots

Would be happy about a patch :)
👟 Reproduction steps
I tried to call the badges of Status with the ID of a created monitor with the check port.
👀 Expected behavior
I thought there is something like "Status UP" now
😓 Actual Behavior
Currently only a "N/A" is output!
See screenshot above
🐻 Uptime-Kuma Version
1.20.2
💻 Operating System and Arch
Ubuntu 20.04.5 LTS \
🌐 Browser
Firefox V110
🐋 Docker Version
Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1
🟩 NodeJS Version
-/-
📝 Relevant log output
No message in the logs!

root@Max1021Vweb:~# docker logs uptime-kuma
==> Performing startup jobs and maintenance tasks
==> Starting application with user 0 group 0
Welcome to Uptime Kuma
Your Node.js version: 16
2023-02-27T19:51:56Z [SERVER] INFO: Welcome to Uptime Kuma
2023-02-27T19:51:56Z [SERVER] INFO: Node Env: production
2023-02-27T19:51:56Z [SERVER] INFO: Importing Node libraries
2023-02-27T19:51:56Z [SERVER] INFO: Importing 3rd-party libraries
2023-02-27T19:51:58Z [SERVER] INFO: Creating express and socket.io instance
2023-02-27T19:51:58Z [SERVER] INFO: Server Type: HTTP
2023-02-27T19:51:58Z [SERVER] INFO: Importing this project modules
2023-02-27T19:51:58Z [NOTIFICATION] INFO: Prepare Notification Providers
2023-02-27T19:51:58Z [SERVER] INFO: Version: 1.20.2
2023-02-27T19:51:58Z [DB] INFO: Data Dir: ./data/
2023-02-27T19:51:58Z [SERVER] INFO: Connecting to the Database
2023-02-27T19:51:58Z [DB] INFO: SQLite config:
[ { journal_mode: 'wal' } ]
[ { cache_size: -12000 } ]
2023-02-27T19:51:58Z [DB] INFO: SQLite Version: 3.39.4
2023-02-27T19:51:58Z [SERVER] INFO: Connected
2023-02-27T19:51:58Z [DB] INFO: Your database version: 10
2023-02-27T19:51:58Z [DB] INFO: Latest database version: 10
2023-02-27T19:51:58Z [DB] INFO: Database patch not needed
2023-02-27T19:51:58Z [DB] INFO: Database Patch 2.0 Process
2023-02-27T19:51:58Z [SERVER] INFO: Load JWT secret from database.
2023-02-27T20:51:58+01:00 [SERVER] INFO: Adding route
2023-02-27T20:51:58+01:00 [SERVER] INFO: Adding socket handler
2023-02-27T20:51:58+01:00 [SERVER] INFO: Init the server
2023-02-27T20:51:58+01:00 [SERVER] INFO: Listening on 3001
2023-02-27T20:52:22+01:00 [AUTH] INFO: Login by token. IP=172.17.0.1
2023-02-27T20:52:22+01:00 [AUTH] INFO: Username from JWT: Maximilian1021
2023-02-27T20:52:22+01:00 [AUTH] INFO: Successfully logged in user Maximilian1021. IP=172.17.0.1
2023-02-27T20:52:47+01:00 [AUTH] INFO: Login by token. IP=172.17.0.1
2023-02-27T20:52:47+01:00 [AUTH] INFO: Username from JWT: Maximilian1021
2023-02-27T20:52:47+01:00 [AUTH] INFO: Successfully logged in user Maximilian1021. IP=172.17.0.1
2023-02-27T20:55:24+01:00 [AUTH] INFO: Login by token. IP=172.17.0.1
2023-02-27T20:55:24+01:00 [AUTH] INFO: Username from JWT: Maximilian1021
2023-02-27T20:55:24+01:00 [AUTH] INFO: Successfully logged in user Maximilian1021. IP=172.17.0.1

Did you add the monitor to a status page first?

This is a stock monitor... I just saw in the changelog that it was added and I wanted to include that right away Am 28.02.23, 11:06 schrieb Nelson Chan @.***>:
Did you add the monitor to a status page first?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Currently you have to add a monitor to a status page before you can access the badges.

Okay.. That could be the problem with me. I will try this tonight...

Sorry. I misunderstood your first answer. Thought you meant whether I have already created.
I put it in a status page and now it works!
Thanks


text:
Just updated docker container on unraid - nightly release - got Error: Connection terminated due to connection timeout

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

Description
log states this-
==> Performing startup jobs and maintenance tasks
==> Starting application with user 99 group 100
Welcome to Uptime Kuma
Your Node.js version: 16
2023-03-31T10:14:40+10:00 [SERVER] INFO: Welcome to Uptime Kuma
2023-03-31T10:14:40+10:00 [SERVER] INFO: Node Env: production
2023-03-31T10:14:40+10:00 [SERVER] INFO: Importing Node libraries
2023-03-31T10:14:40+10:00 [SERVER] INFO: Importing 3rd-party libraries
2023-03-31T10:14:41+10:00 [SERVER] INFO: Creating express and socket.io instance
2023-03-31T10:14:41+10:00 [SERVER] INFO: Server Type: HTTP
2023-03-31T10:14:41+10:00 [SERVER] INFO: Importing this project modules
2023-03-31T10:14:41+10:00 [NOTIFICATION] INFO: Prepare Notification Providers
2023-03-31T10:14:42+10:00 [SERVER] INFO: Version: 1.21.1-nightly-20230329043034
2023-03-31T10:14:42+10:00 [DB] INFO: Data Dir: ./data/
2023-03-31T10:14:42+10:00 [SERVER] INFO: Connecting to the Database
2023-03-31T10:14:42+10:00 [DB] INFO: SQLite config:
[ { journal_mode: 'wal' } ]
[ { cache_size: -12000 } ]
2023-03-31T10:14:42+10:00 [DB] INFO: SQLite Version: 3.41.1
Trace: Error: Connection terminated due to connection timeout
at Connection. (/app/node_modules/pg/lib/client.js:132:73)
at Object.onceWrapper (node:events:627:28)
at Connection.emit (node:events:513:28)
at Socket. (/app/node_modules/pg/lib/connection.js:57:12)
at Socket.emit (node:events:513:28)
at TCP. (node:net:301:12)
at process. (/app/server/server.js:1804:13)
at process.emit (node:events:513:28)
at emit (node:internal/process/promises:140:20)
at processPromiseRejections (node:internal/process/promises:274:27)
at processTicksAndRejections (node:internal/process/task_queues:97:32)
If you keep encountering errors, please report to https://github.com/louislam/uptime-kuma/issues
👟 Reproduction steps
Updfate docker container as it said there was an update for nightly tag
When it restarted, it no longer works.
Checked logs and got this as above.
change to latest tag and it is working again.
👀 Expected behavior
was working before update.
😓 Actual Behavior
no longer working
🐻 Uptime-Kuma Version
louislam/uptime-kuma:nightly
💻 Operating System and Arch
Unraid
🌐 Browser
chrome
🐋 Docker Version
louislam/uptime-kuma:nightly
🟩 NodeJS Version
No response
📝 Relevant log output
==> Performing startup jobs and maintenance tasks
==> Starting application with user 99 group 100
Welcome to Uptime Kuma
Your Node.js version: 16

2023-03-31T10:14:40+10:00 [SERVER] INFO: Welcome to Uptime Kuma
2023-03-31T10:14:40+10:00 [SERVER] INFO: Node Env: production
2023-03-31T10:14:40+10:00 [SERVER] INFO: Importing Node libraries
2023-03-31T10:14:40+10:00 [SERVER] INFO: Importing 3rd-party libraries
2023-03-31T10:14:41+10:00 [SERVER] INFO: Creating express and socket.io instance
2023-03-31T10:14:41+10:00 [SERVER] INFO: Server Type: HTTP
2023-03-31T10:14:41+10:00 [SERVER] INFO: Importing this project modules
2023-03-31T10:14:41+10:00 [NOTIFICATION] INFO: Prepare Notification Providers
2023-03-31T10:14:42+10:00 [SERVER] INFO: Version: 1.21.1-nightly-20230329043034

2023-03-31T10:14:42+10:00 [DB] INFO: Data Dir: ./data/
2023-03-31T10:14:42+10:00 [SERVER] INFO: Connecting to the Database
2023-03-31T10:14:42+10:00 [DB] INFO: SQLite config:
[ { journal_mode: 'wal' } ]
[ { cache_size: -12000 } ]
2023-03-31T10:14:42+10:00 [DB] INFO: SQLite Version: 3.41.1

Trace: Error: Connection terminated due to connection timeout

at Connection.<anonymous> (/app/node_modules/pg/lib/client.js:132:73)
at Object.onceWrapper (node:events:627:28)
at Connection.emit (node:events:513:28)
at Socket.<anonymous> (/app/node_modules/pg/lib/connection.js:57:12)
at Socket.emit (node:events:513:28)
at TCP.<anonymous> (node:net:301:12)
at process.<anonymous> (/app/server/server.js:1804:13)
at process.emit (node:events:513:28)
at emit (node:internal/process/promises:140:20)
at processPromiseRejections (node:internal/process/promises:274:27)
at processTicksAndRejections (node:internal/process/task_queues:97:32)
If you keep encountering errors, please report to https://github.com/louislam/uptime-kuma/issues

Please dont use nightly, it is an unstable version.

authors: louislam vw-kombi


text:
Group monitors

⚠⚠⚠ Since we do not accept all types of pull requests and do not want to waste your time. Please be sure that you have read pull request rules:
https://github.com/louislam/uptime-kuma/blob/master/CONTRIBUTING.md#can-i-create-a-pull-request-for-uptime-kuma
Tick the checkbox if you understand [x]:

[X] I have read and understand the pull request rules.

Description
Hello Guys,
I saw that many users requested a grouping feature... and I also wanted one. So I added it.
You can group services however and how deep you like.
The group always takes the state of the worsted child. If all are green and one in red. The whole group will be red.

This means you can set your notification at the parent level and get one message instead of 10 if something is broken.
Also, if you want to do maintenance, you can just set it at the parent and all children go into maintenance too.

The same happens if you pause the group.

To add a group, simply add a monitor and select "group" as its type.

To add monitors to this group, just edit the monitor and select the group in the "monitor group" dropdown.

If you create subgroups, it can look like this:

And of course, if one item in the subgroup fails, the subgroups fails and also the main group itself.
In text fields, subgroups are displayed using slashes like this Cloud / HTTPS.
And your state, which group is collapsed and which not, gets saved in your browser.
In your status page, you can now simply select the group to get displayed instead of every single service separately.
This makes the status page more simple and better for your normal users.

Fixes:

#2692
#2585
#1089
#639
#549
#324
#216
#29
#12

Type of change
Please delete any options that are not relevant.

User interface (UI)
New feature (non-breaking change which adds functionality)
This change requires a documentation update (maybe)

Checklist

[X] My code follows the style guidelines of this project
[X] I ran ESLint and other linters for modified files
[X] I have performed a self-review of my own code and tested it
[X] I have commented my code, particularly in hard-to-understand areas
(including JSDoc for methods)
[X] My changes generate no new warnings
[ ] My code needed automated testing. I have added them (this is optional task)

Absolutely incredible, I’ve so long to get this into Kuma thank you so much
Le sam. 28 janv. 2023 à 15:11, Peace @.***> a écrit :

⚠⚠⚠ Since we do not accept all types of pull requests and do not want
to waste your time. Please be sure that you have read pull request rules:
https://github.com/louislam/uptime-kuma/blob/master/CONTRIBUTING.md#can-i-create-a-pull-request-for-uptime-kuma
Tick the checkbox if you understand [x]:

I have read and understand the pull request rules.

Description
Hello Guys,
I saw that many users requested a grouping functions... and I also wanted
one. So I added it.
You can group services however and how deep you like.
The group always takes the state of the worsted child. If all are green
and one in red. The whole group will be red.
[image: image]
https://user-images.githubusercontent.com/32363424/215270385-8d1a43c8-b462-42b7-835f-2a3fcec0d9f4.png
This means you can set your notification at the parent level and get one
message instead of 10 if something is broken.
Also, if you want to do maintenance, you can just set it at the parent and
all children go into maintenance too.
[image: image]
https://user-images.githubusercontent.com/32363424/215270471-f7cf61f0-d46a-47f5-ad52-8a36df89420e.png
The same happens if you pause the group.
[image: image]
https://user-images.githubusercontent.com/32363424/215270886-900ea21b-e4d5-4738-a595-6f4058de1801.png
To add a group, simply add a monitor and select "group" as its type.
[image: image]
https://user-images.githubusercontent.com/32363424/215270547-83ade9a8-4635-4fda-97c4-3282c304a86b.png
To add monitors to this group, just edit the monitor and select the group
in the "monitor group" dropdown.
[image: image]
https://user-images.githubusercontent.com/32363424/215270595-ebcc518a-55df-4d7c-a8d6-7084d1cc0046.png
If you create subgroups, it can look like this:
[image: image]
https://user-images.githubusercontent.com/32363424/215270701-182828ad-6e39-4273-a849-47520f1cdd1b.png
And of course, if one item in the subgroup fails, the subgroups fails and
also the main group itself.
In text fields, subgroups are displayed using slashes like this Cloud /
HTTPS.
And of course your state, which group is collapsed and which not, gets
saved in your browser.
In your status page, you can now simply select the group to get displayed
instead of every single service separately.
This makes the status page more simple and better for your normal users.
[image: image]
https://user-images.githubusercontent.com/32363424/215270947-764d6205-39fb-4244-b422-6533683301f4.png
[image: image]
https://user-images.githubusercontent.com/32363424/215270957-57f532a8-8f10-4efa-8535-84667f6006c7.png
Fixes:

#2692 https://github.com/louislam/uptime-kuma/issues/2692
#2585 https://github.com/louislam/uptime-kuma/issues/2585
#1089 https://github.com/louislam/uptime-kuma/issues/1089
#639 https://github.com/louislam/uptime-kuma/issues/639
#549 https://github.com/louislam/uptime-kuma/issues/549
#324 https://github.com/louislam/uptime-kuma/issues/324
#216 https://github.com/louislam/uptime-kuma/issues/216
#29 https://github.com/louislam/uptime-kuma/issues/29
#12 https://github.com/louislam/uptime-kuma/issues/12

Type of change
Please delete any options that are not relevant.

User interface (UI)
New feature (non-breaking change which adds functionality)
This change requires a documentation update (maybe)

Checklist

My code follows the style guidelines of this project
I ran ESLint and other linters for modified files
I have performed a self-review of my own code and tested it
I have commented my code, particularly in hard-to-understand areas
(including JSDoc for methods)
My changes generate no new warnings
My code needed automated testing. I have added them (this is
optional task)

You can view, comment on, or merge this pull request online at:
https://github.com/louislam/uptime-kuma/pull/2693
Commit Summary

645fd94
https://github.com/louislam/uptime-kuma/pull/2693/commits/645fd94bba62990e3031c9571c92fcb4e1cd481e
feat: add ability to group monitors in dashboard
97bd306
https://github.com/louislam/uptime-kuma/pull/2693/commits/97bd306a09e3d1e677f9a3e954d9f8a482d781bc
Merge branch 'louislam:master' into group-monitors
aba515e
https://github.com/louislam/uptime-kuma/pull/2693/commits/aba515e17245a743362380d118a12a73c6113f08
feat: disable childs if parent is disabled
f3ac351
https://github.com/louislam/uptime-kuma/pull/2693/commits/f3ac351d755ffa6ba9195a45db8bf7eea261c492
feat: set childs under maintenance if parent is too
faf3488
https://github.com/louislam/uptime-kuma/pull/2693/commits/faf3488b1ea00d97386477017248430c7c020dab
fix: unfold tree if monitor is accessed directly
9a46b50
https://github.com/louislam/uptime-kuma/pull/2693/commits/9a46b50989619251fbb022ead0481e5f51cd09c5
docs: add comments
aee4c22
https://github.com/louislam/uptime-kuma/pull/2693/commits/aee4c22deef2c90c595e79c3dea44cf6f77133cb
perf: only do one filter instead of 3 in editMonitor
d48eb24
https://github.com/louislam/uptime-kuma/pull/2693/commits/d48eb24046ed7f11350b3e85a9e4731486fa837a
docs: more comments
661fa87
https://github.com/louislam/uptime-kuma/pull/2693/commits/661fa87134cb84a3674ca68263a1eae5c5e9e9b5
feat: make parent link clickable

File Changes
(13 files https://github.com/louislam/uptime-kuma/pull/2693/files)

A db/patch-add-parent-monitor.sql
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-51b16eaf34f735fad97ba6b0fb3f1efa4853f0c0e6a9572e9bd8ca17d453ac64
(6)
M server/database.js
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-75a796c4b3bcbb9621bd3801586525dc485ae31d666392f0786bdc2357ce0378
(1)
M server/model/monitor.js
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-0c4874f149ad01732ad730f36c95c99e37f5c3f13e9b52f628e2e284daa72d27
(145)
M server/server.js
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-8cf1ffe3788af127768748703e2f15dcfb3a05ffd20b4c2375223637e3260938
(5)
M server/socket-handlers/maintenance-socket-handler.js
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-6fa0af0c1d743c08010abc2890cf0f4cac2e1b54e76caf9668c9d4a4d03b43fb
(2)
M src/components/MonitorList.vue
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-75b1c5113c67448810c24d51f6de9a31640e30eb02ebb7a2b7c4669c4882b720
(56)
A src/components/MonitorListItem.vue
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-4d4136813a643c67972dbc1aaf7ce1e6e67788f70c5b4164cf9df60f0d8c4259
(208)
M src/lang/de-DE.json
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-0670c82b6965569442fb4c17bca84ca7bfcf25a9b0cb14f7c7db2359748c3a32
(4)
M src/lang/en.json
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-f420468eb712a68d53557a4f5272b25a83938166b073c7ce47e0de8bd32064d7
(4)
M src/pages/Details.vue
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-14b79667ab4cf0e704d870f66e3ba69f5638d61ad463355387b4cc76cbfcfdcb
(20)
M src/pages/EditMaintenance.vue
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-e6b53a45732558c636155fdba04535744c31f89891393100109c4ce574790df6
(34)
M src/pages/EditMonitor.vue
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-f672603317047f3e6f27b0d7a44f6f244b7dbb5d0d0a85f1059a6b0bc2cb9aa0
(55)
M src/pages/StatusPage.vue
https://github.com/louislam/uptime-kuma/pull/2693/files#diff-8c464f12c308bf19516adb323a1ff12a6e6857b0899750ff7c4560d263d8b5f9
(31)

Patch Links:

https://github.com/louislam/uptime-kuma/pull/2693.patch
https://github.com/louislam/uptime-kuma/pull/2693.diff

—
Reply to this email directly, view it on GitHub
https://github.com/louislam/uptime-kuma/pull/2693, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQHN6EUVRIQK5RCRZMVAY23WUUSKTANCNFSM6AAAAAAUJVKWFU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Finally, some order is brought into my life.
Hopefully the feature is approved soon!

Awesome .. can you please also share if there is an API which we can use to script existing objects into groups..

Awesome .. can you please also share if there is an API which we can use to script existing objects into groups..

If I'm seeing it correctly kuma only has a small amount of api endpoints. Most of them for generating badge svgs.
So currently there isn't a endpoint for creating or editing monitors.
I don't want to speek for the project maintainers but i think this is a feature which we can expect somewhere in the future but currently there isn't a api for this type of automation.
In difference to most projects the kuma frontend gets all data through websockets. If you really wan't to script your task maybe you can connect to this websocket and send commands to it like the frontend client does... But this would be a very tricky task i think. Or create a python script which clicks for you in the browser.
But currently i find no official api support for automation.

I am using this API => https://pypi.org/project/uptime-kuma-api/0.9.0/

I am using this API => https://pypi.org/project/uptime-kuma-api/0.9.0/

Exactly what I said. The creator of this API build its own client which connects to the web socket. So technically it's possible to use groups with this API. But the Maintainer of the API project first has to implement this feature.

Quickly reviewed most of it and it looks quite reasonable! I don't expect this to be merged anytime soon but:

to prevent infinite recursion, should there be a server-side check for loops? Or the parenting should be limited to a reasonable number?
Using isActive in the backup code probably won't work since it needs to set the actual active column in DB? But I'm also refactoring this in #2667 so maybe we can ignore it for now.

I don't expect this to be merged anytime soon

Why do you think that? I mean, it's a bigger feature, but also a long expected from many users.

to prevent infinite recursion, should there be a server-side check for loops?

Currently, there is a client-side check to stop loops. But maybe a server-side check should be added too, to be secure.

Or the parenting should be limited to a reasonable number?

I also thought about this. It would stop users from creating their complete own structure, but maybe. The question is how deep should it max go. And should this value be editable using the Settings Page, or an env?

Using isActive in the backup code probably won't work since it needs to set the actual active column in DB?

isActive only is a getter for checking if the monitor itself and all ancestors are active. To set the value, you still use this.active. The same goes for getting the active state of the monitor itself.
The backup logic should still be working.

I've been waiting for this feature! I'm very excited for this to get merged!!

to prevent infinite recursion, should there be a server-side check for loops?

Thanks to server- and client-side checks, endless loops should now be impossible.
Thanks for the hint @chakflying

It sounds like it is duplicate of #1236?

It sounds like it is duplicate of #1236?
@louislam

Partially. Dependence is just one function of this PR. Only a side functionality. The main feature is the grouping. That the group displays the state of the children only got added because I wanted something to be shown in the group state. But it's very useful.

It is good to read the pull request rules first next time. A large pull request should have a discussion first.
I originally planned to merge #1236 in 1.21.0.
Since I believe that I can merge either one pr only. It may be hard for me to make a decision. Usually I will reject the later one. But in this case, I think this pr's UI/UX is better and the solution is straightforward.
While for #1236, it provides many-to-many relation, which means a monitor belong to more than one group. But I don't like the new status DEGRADED is being added.
So it is hard for me to choose.

It is good to read the pull request rules first next time. A large pull request should have a discussion first.

I saw the part about discussion, but I thought you just mean that there should exist an issue for this.
Because there are several, I thought this would fulfill the requirement. My mistake.

So it is hard for me to choose.

In theory, it would also be possible to implement both options. Because they are not exactly the same.
Like I said, my feature is primary for grouping the monitors for better visibility in the dashboard and status page.
The main question for the other pr is. Do users need many-to-many relations? I don't see a use case for this. But maybe others do.

Please merge this.
Grouping functionality is superior in every way to dependent monitors.
I was surprised this project hadn't already gotten groups implemented.
Lots of backend stuff happening, but from a functionality having 100+ monitors....
Need groups, man.

+1 for this PR for grouping

@julian-piehl
Hello.
The grouping is very well done. Thanks to you, this tool has become much more convenient to use (especially if you need to monitor more than 50 programs :)).
Can You adjust the backup function in Your repository?
It would be cool if the monitors remembered which group they were in when restoring from backup. At the moment, need to choose a group again after backup restoring.

Depending on the answer I think that the status degraded is to be studied

Personally, I quite like the degraded status for the follow use case:
Say I have a website hosted in multiple locations. I created a monitor for all of these locations. Logically I put the into a group because they are all related. If one of the monitors goes down, the entire group is marked as down. This doesn't really fit because the site will still be accessible from the other locations. In this case, I think degraded works because if a location is down, the site will still respond but may be slower than normal, hence it is degraded.

Depending on the answer I think that the status degraded is to be studied

Personally, I quite like the degraded status for the follow use case:
Say I have a website hosted in multiple locations. I created a monitor for all of these locations. Logically I put the into a group because they are all related. If one of the monitors goes down, the entire group is marked as down. This doesn't really fit because the site will still be accessible from the other locations. In this case, I think degraded works because if a location is down, the site will still respond but may be slower than normal, hence it is degraded.

This is a very good example and we can find many of them.
We need the group functionality but it should not influence or try to solve the notification problem

As the comparison with 1236 has already been opened here, I'd like to add another 2 cents to it:
For me this addition does not solve an annoyance I'm currently having which would be solved by 1236. Lets take this scenario:

Have a database server
Have 5+ applications using the same database server
Database server makes an oopsie and is down

I'm getting pinged for the database server
I'm getting pinged 5+ more times for each service

This solution would work if I add each of the 5+ services as a new "group" and add their web monitors to each and - given each monitor can only belong to one group - then add 5+ copies of the database monitor to all the groups. So now I'm still getting pinged 5+ times by each group as the database server still is failing, marking 5+ checks red and also the web monitors are failing as the database is gone.
With the solution of 1236 I could have one database monitor and 5+ web monitors watching my applications. Adding the database as a dependency of those 5+ monitors they are automatically "silenced" by the database going down and I'm pinged one time. Also I only need to monitor the database once and reduce stress on the database server.
(Given the applications are using more shared services this gets only more complex and messy but I think the point is already quite clear with the database only.)

Hi i need some asist. i cant fing groups in monito type.

Hi i need some asist. i cant find groups in monitor type.

@Chubi01 This PR has not been merged yet. As such it has not been released.

This feature is very interesting, do you need more hands to approve this PR ?
I think if this PR is approved and added to the next release is going to add much value to the project.

This feature is very interesting, do you need more hands to approve this PR ? I think if this PR is approved and added to the next release is going to add much value to the project.

Yes, it will be in next release hopefully. I don't have time to review pull requests recently, but will review later. Feel free to test it by this special docker image.
docker run --rm -it -p 3000:3000 -p 3001:3001 --pull always -e 'UPTIME_KUMA_GH_REPO=julian-piehl:group-monitors' louislam/uptime-kuma:pr-test

Details: https://github.com/louislam/uptime-kuma/wiki/Test-Pull-Requests

This feature is very interesting, do you need more hands to approve this PR ? I think if this PR is approved and added to the next release is going to add much value to the project.

Yes, it will be in next release hopefully. I don't have time to review pull requests recently, but will review later. Feel free to test it by this special docker image.
docker run --rm -it -p 3000:3000 -p 3001:3001 --pull always -e 'UPTIME_KUMA_GH_REPO=julian-piehl:group-monitors' louislam/uptime-kuma:pr-test

Details: https://github.com/louislam/uptime-kuma/wiki/Test-Pull-Requests

This docker do not work. But I find other way to test it.
Do you have a ETA for move this to the new branch? any way to more people can help on the process, just following this guidelines to pull requests?

Earlier we discussed about the status being displayed for groups. And the other solution using Dependencies (#1236).
I initially just added this feature to the groups because I didn't know what icon to display else.
Maybe it would be a cool idea to remove this feature from this pr again and put the expand icon in the status position instead.
Then the pr #1236 could be merged too, allowing to create better dependency monitors like some people requested.

Testing this PR, I haven't found any major issues, but I do have some suggestions:

If they are always empty and N/A, they could be hidden.

Prefixing the group name seems to be too long in my opinion.

If a monitor has been added to a group, should it be hidden in the "root" list?

In terms of UX, it would be great if users could add a list of child monitors in the edit page. However, considering it is just the first PR for the group, it is ok to not include this in this stage. We could do it in another PR.
An example from the maintenance edit page:

If they are always empty and N/A, they could be hidden.

Ping and avg ping can be hidden, because there is no use.
The graph shows the same stats the other monitors show. For example, if one children is red, the group is also red, which gets displayed in the graph.

Prefixing the group name seems to be too long in my opinion.
If a monitor has been added to a group, should it be hidden in the "root" list?

This prefix style is only applied in two cases:

Select Boxes
When you are using the search bar

That's also the reason you are seeing the element 2 times (point 3). This happens when using the search, because both, the parent and the child are matching your search key.
I have no idea how I could visualize this in a better way. Ideas are appreciated.
When using no search key, the groups are displayed like expected.

In terms of UX, it would be great if users could add a list of child monitors in the edit page. However, considering it is just the first PR for the group, it is ok to not include this in this stage. We could do it in another PR.

A great idea. I will look into it.

If they are always empty and N/A, they could be hidden.

Ping and avg ping can be hidden, because there is no use. I will change this. The graph shows the same stats the other monitors show. For example, if one children is red, the group is also red, which gets displayed in the graph.

Prefixing the group name seems to be too long in my opinion.
If a monitor has been added to a group, should it be hidden in the "root" list?

This prefix style is only applied in two cases:
1. Select Boxes

2. When you are using the search bar

That's also the reason you are seeing the element 2 times (point 3). This happens when using the search, because both, the parent and the child are matching your search key. I have no idea how I could visualize this in a better way. Ideas are appreciated.
When using no search key, the groups are displayed like expected.

In terms of UX, it would be great if users could add a list of child monitors in the edit page. However, considering it is just the first PR for the group, it is ok to not include this in this stage. We could do it in another PR.

A great idea. I will look into it.

No problem, I got your point. I merge it first.

Heck yeah it's coming gents and ladies!

It would be possible to add the function of when I add a group or subgroup on the status pages, the same to let the group open or close it in the customer view, as there is in the dashboard the possibility to open the group or when it is open it can be closed ?
Dashboard

Page Status

Fantastic! Thank you!

Hello, I upgrade to 1.22.0-beta.0 from 1.21.3, but Monitor Group only has none and can not clickable. How to add group?

It would be possible to add the function of when I add a group or subgroup on the status pages, the same to let the group open or close it in the customer view, as there is in the dashboard the possibility to open the group or when it is open it can be closed ?
Dashboard
Page Status

Is someone working on this feature, a dropdown on monitor group in the status page?
Its easy to identify what happen under the group if we have this feature.

No, nobody is working on this as far as I know.
If you/someone else would like to contribute such a change, here is the Contribution guide:
https://github.com/louislam/uptime-kuma/blob/master/CONTRIBUTING.md
Necroposting on closed PRs is also really not scalable from a project maintanance perspective
=> See https://github.com/louislam/uptime-kuma/issues/3324 for the correct issue ^^

authors: CommanderStorm Computroniks DagothDeer HorseLuke ItsCryptic Luzifer Uruknara a1git chakflying david7867z faverbuj fr4nc1stein julian-piehl keskad kleper ldev-coder lightingman117 louislam mabed-fr sullivan1337


text:
chore: fixed a typo for internal, unused part of the file upload icon for status page

⚠⚠⚠ Since we do not accept all types of pull requests and do not want to waste your time. Please be sure that you have read pull request rules:
https://github.com/louislam/uptime-kuma/blob/master/CONTRIBUTING.md#can-i-create-a-pull-request-for-uptime-kuma
Tick the checkbox if you understand [x]:

[x] I have read and understand the pull request rules.

Description
Fixes https://github.com/louislam/uptime-kuma/issues/4749
Type of change
Please delete any options that are not relevant.

Bug fix (non-breaking change which fixes an issue)

Checklist

[x] My code follows the style guidelines of this project
[x] I ran ESLint and other linters for modified files
[x] I have performed a self-review of my own code and tested it
[x] I have commented my code, particularly in hard-to-understand areas (including JSDoc for methods)
[x] My changes generates no new warnings
[ ] My code needed automated testing. I have added them (this is optional task)

I consider this as an uncontroversial bugfix
=> I would like this to be in 1.23.X instead
Could you rebase this to said branch via running
npm run rebase-pr-to-1.23.X

That's a lot of changes??

I think somebody changed the target to 1.23.x; it was originally based on the main branch.

Needs to be rebased. Frank asked me to do that, but I've been too busy with my job

For context: my original PR was 1 commit, +/- 1 line

I guess then this is going into v2.0.
Think this can also be considered a "feature" as this strictly speaking adds functionality to the internal api and not to the user interface.
=> merging with junior maintainer approval.

@CommanderStorm I made a backport to 1.23.X here: https://github.com/louislam/uptime-kuma/pull/4757


text:
麻烦填加一下，阜阳师范大学校徽

好的，下次一起更新

authors: lovefc zhangxipo


text:
Semi-transparent outlines on raw, premultiplied input

Thanks for utility, great work.
I use the utility inside the map-tile-server-gl node-app. Server configuration when rendering raster tiles allows you to set the color of the lines in rgba. But as a result, PNG-images enclose a gray or black substrate under the translucent white color. Result on the attached screenshot.
Images on the server are created as follows:
var image = sharp(data, {
          raw: {
            width: params.width * scale,
            height: params.height * scale,
            channels: 4
          }
        })
...
image.png()

I tried using .embed (), .ensureAlpha (), passing the create parameter to the constructor, pointing to background with transparent white, but it didn’t work.
How do I get a white backing in white translucent color, or do I get a 24-bit PNG supporting translucent colors?

Hello, what is the source of data? Could it have been premultiplied?
What does ... refer to in this code sample? Are you able to provide a standalone code sample and input image data that exhibits this behaviour?

https://github.com/klokantech/tileserver-gl/blob/master/src/serve_rendered.js
lines from 417 to 451

https://github.com/mapbox/mapbox-gl-native/issues/9124

"The Node binding however, returns the premultiplied image buffer as is"

Sorry, I didn't quite understand what it was about in this post.
The point is that in any case, the image will be with a gray stroke under the translucent pixels?

It looks like tileserver-gl uses mapbox-gl-native to render, which returns RGBA data that has been premultiplied. This is what's causing the artifacts you're seeing on (semi-)transparent pixels.
The quick fix would probably be to modify tileserver-gl to unpremultiply pixel values before passing them to sharp - there's an example of what's required to do this in https://github.com/consbio/mbgl-renderer/issues/25

Thanks for help, I've add the issue to server's author, because haven't found the solution myself. The are no using mbgl-renderer as javascript-library, I see only mbgl/renderer c-sources and can't move futher, unfortunately

authors: justewg lovell

repo: lovell/sharp


text:
Composite with gravity, right and bottom params

What are you trying to achieve?
Put watermark using gravity in composite with an offset from left and bottom edge.
Have you searched for similar feature requests?
I didn't find it. I found just left and top params, but I have to place watermark manually.
What would you expect the API to look like?
Create offset param in object.
await sharp('image.png')
  .composite([{
    input: 'watermark.png',
    gravity: 'southwest',
    offset: {
      left: 10,
      bottom: 10
    }
  }]).toFile('image.png');

What alternatives have you considered?
If I want to achieve this, I have to calculate image size, watermark size and make some calculation.
Is there a sample image that helps explain?
At now I have:

I would like to achieve this:

Hi, perhaps you could extend the watermark image by 10 transparent pixels along the left and bottom edges?

You're right. I haven't thought about this 🤦♂

I know this is an old one, but it'd be really great if you could add "right" and "bottom" options. It's not always possible to add this kind of margin to the image itself. Also it'd be a lot better to be able to fully control the position programmatically.

authors: SixBytesUnder dszymczuk lovell


text:
0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and musl libs

Possible install-time or require-time problem

[x] I have read the documentation relating to installation.

Are you using the latest version of sharp?

[x] I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

If you cannot confirm this, please upgrade to the latest version and try again before opening an issue.
If you are using another package which depends on a version of sharp that is not the latest, please open an issue against that package instead.
Are you using a supported runtime?

[ ] I am using Node.js 18 with a version >= 18.17.0
[x] I am using Node.js 20 with a version >= 20.3.0
[ ] I am using Node.js 21 or later

If you cannot confirm any of these, please upgrade to the latest version and try again before opening an issue.
Are you using a supported package manager?

[x] I am using npm >= 9.6.5
[ ] I am using yarn >= 3.2.0
[ ] I am using pnpm >= 7.1.0

If you cannot confirm any of these, please upgrade to the latest version and try again before opening an issue.
What is the complete output of running npm install --verbose --foreground-scripts sharp in an empty directory?
npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using npm@10.2.3
npm info using node@v20.10.0
npm verb title npm install sharp
npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp"
npm verb logfile logs-max:10 dir:/Users/tmcgannon/.npm/_logs/2023-11-29T22_24_45_285Z-
npm verb logfile /Users/tmcgannon/.npm/_logs/2023-11-29T22_24_45_285Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/sharp 182ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/semver 47ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/detect-libc 126ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color 134ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-darwin-x64 354ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linuxmusl-x64 369ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-arm 450ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-wasm32 465ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-darwin-arm64 495ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-s390x 609ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-x64 611ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linuxmusl-arm64 611ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linuxmusl-x64 610ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-x64 610ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-darwin-x64 613ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-arm 613ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-ia32 653ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linuxmusl-arm64 662ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-darwin-arm64 666ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-s390x 667ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-x64 669ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-arm64 708ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-arm64 765ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color-convert 50ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@emnapi%2fruntime 266ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color-name 44ms (cache revalidated)
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-win32-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-win32-ia32
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-wasm32
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@emnapi/runtime
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linuxmusl-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linuxmusl-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-s390x
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-arm
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linuxmusl-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linuxmusl-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-s390x
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-arm
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-darwin-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-darwin-x64
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 89ms
npm info run sharp@0.33.0 install node_modules/sharp node install/check

> sharp@0.33.0 install
> node install/check

npm http fetch GET 200 https://registry.npmjs.org/xml2js 48ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/axios 59ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/tough-cookie 60ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/jsonwebtoken 62ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/dottie 106ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@azure%2fms-rest-js 67ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/azurite 91ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/sequelize 168ms (cache revalidated)
npm info run sharp@0.33.0 install { code: 0, signal: null }

added 8 packages, and audited 297 packages in 4s

58 packages are looking for funding
  run `npm fund` for details

8 vulnerabilities (7 moderate, 1 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?
➜  test npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp

  System:
    OS: macOS 14.1
    CPU: (8) arm64 Apple M1
    Memory: 83.52 MB / 8.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.10.0 - /usr/local/bin/node
    npm: 10.2.3 - /usr/local/bin/npm
    bun: 1.0.14 - ~/.bun/bin/bun

Problem description
When I run npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc, it installs both glibc and musl versioned libraries:

Expected behavior
Only sharp-libvips-linux-x64 and sharp-linux-x64 should be installed.

Please see https://github.com/npm/cli/pull/6914

I'm probably missing something, but I'm seeing the same thing happening when using either pnpm or yarn which do allow to set the supported architectures.
Example package.json file
{
  "name": "sharp-binaries",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "4.18.2",
    "sharp": "0.33.0"
  },
  "os": ["linux"],
  "cpu": ["x64"],
  "libc": ["glibc"],
  "pnpm": {
    "supportedArchitectures": {
      "os": ["linux"],
      "cpu": ["x64"],
      "libc": ["glibc"]
    }
  }
}

PNPM
Uses the pnpm property in package.json to setup supported architectures as defined here.
Dockerfile
FROM node:18.18-alpine3.18
RUN corepack enable && corepack prepare pnpm@latest --activate
COPY package*.json ./
RUN pnpm i
COPY index.js ./
CMD ["node", "index.js"]

Build targetting linux/amd64
docker build . --no-cache --platform=linux/amd64 --tag 'sharp-binaries:latest'

Results
I'm seeing 4 packages (using Docker desktop) where I'd expect to see 2 (sharp-libvips-linux-x64 and sharp-linux-x64):

Yarn
Uses os, cpu and libc properties in package.json as described here.
Dockerfile
FROM node:18.18-alpine3.18
COPY package*.json ./
RUN yarn install
COPY index.js ./
CMD ["node", "index.js"]

Build targetting linux/amd64
docker build . --no-cache --platform=linux/amd64 --tag 'sharp-binaries:latest'

Results
I'm seeing 19 packages where I'd expect to see 2 (sharp-libvips-linux-x64 and sharp-linux-x64)

@adriaanmeuris For pnpm, please see https://github.com/pnpm/pnpm/pull/7214 - if the latest version of pnpm is installing the wrong dependencies, then please ask at the pnpm repo.
For yarn, it looks like you're using version 1.
$ docker run --rm -it node:18.18-alpine3.18 yarn --version
1.22.19

Please use at least version 3 of yarn, but avoid its "plug'n'play" linker.
Version 1 of yarn is no longer actively developed and is missing so many useful newer features. See also https://github.com/evanw/esbuild/issues/2949

Thanks, I will track the npm issue.

For whoever ends up here - the --libc npm feature is now merged npm/cli/pull/6914

authors: adriaanmeuris klausbadelt lovell tmcgannon


text:
Problem comparing images: Buffer length of each extract piece

Goal: Split an image and get only the non repetitive parts
I am doing a lib / utility which goal is to insert an image, split into pieces and get non repetitive ones.  I made with Jimp while I am migrating to sharp (performance is way better).
I have use this to code:

Dev enviroment is:

OS: Elementary OS 8 x86_64 (derivated from Ubuntu 24.04)
Node version: 22.11.0
sharp version: 0.33.5
sharp-phash version: 2.2.0

Image of reference

How Jimp compare images?
Jimp has changed its docs but in the past they suggest to use two metrics:

Distance (%) -> hamming distance of two images based on their perceptual hash
Difference (%) -> diffs two images

These metrics are between 0-1, so to say images are equal Jimp proposed to do something like this
import { Jimp, diff, distance as dist } from 'jimp'

const MAX_DISTANCE = 0.01 // up to you
const MAX_DIFFERENCE = 0.01 // up to you

const areEquals = (image1: Jimp, image2: Jimp): boolean => {
    const distance = dist(image1, image2)
    const difference = diff(image1, image2)
    return (distance < MAX_DISTANCE) && (difference < MAX_DIFFERENCE)
}

Getting Hamming distance with Sharp
In order to get the hamming distance based on perceptual hash I am using the dependency sharp-phash. So to get the distance is something like this
import { Sharp } from 'sharp'
import phash from 'sharp-phash'
import dist from 'sharp-phash/distance'

const getDistance = async (image1: Sharp, image2: Sharp): Promise<number> => {
    const [buffer1, buffer2]: [Buffer, Buffer] = await Promise.all([image1.toBuffer(), image2.toBuffer()])
    const [phash1, phash2]: [string, string] = await Promise.all([ buffer1, buffer2])
    // denominator is in order to get as a %
    return dist(phash1, phash2) / Math,max(phash1.length, phash2.length)
}

It works perfect... BUT. When I try to compare some similar images like these two images, I get 0 as distance

Getting Difference with Sharp
Checking how Jimp calculates the difference, they use the dependency pixelmatch. This dep compare two Buffer and get the number of pixels which differs. So to get the difference is something like this
import { Sharp } from 'sharp'
import pixelmatch from 'pixelmatch'

// suppose image1 & image2 have the same size (width & height)
const getDifference = async (image1: Sharp, image2: Sharp): Promise<number> => {
    const [buffer1, buffer2] = await Promise.all([image1.toBuffer(), image2.toBuffer()])
    const { width, height } = await image1.metadata()
    const numDifferentPixels = pixelmatch(buffer1, buffer2, null, width, height)
    return numDifferentPixels / (width * height)
}

The problem I'm having here is when I split the hole chess, each part has different Buffer length. Following the example with the hamming distance, the next images have 1268 and 1261 of length in its Buffer respectively

Buffer.length = 1268

Buffer.length = 1261
pixelmatch cannot compare Buffers of different length.
Doubts

How is possible that each part, which has the same size, has different Buffer length?
When I extract / crop each part (next is the code I use to do it), if I check the metada, each part has the same width and height than the original image (720px x 720px) but when I save as a file, each part is correct (90px x 90px), how could it be?import { Sharp } from 'sharp'
const getPart = (image: Sharp, x: number, y: number, width: number, height: number): Sharp => (
    image.clone().extract({ left: x, top: y, width, height})
)

It looks like pixelmatch accepts raw pixel data, whereas it appears you may be passing it compressed PNG image data.
https://sharp.pixelplumbing.com/api-output#raw
- const [buffer1, buffer2] = await Promise.all([image1.toBuffer(), image2.toBuffer()])
+ const [buffer1, buffer2] = await Promise.all([image1.raw().toBuffer(), image2.raw().toBuffer()])

Hi @lovell ,
Yes, the example images are png.
If I try that approach I have the error Error: Image data size does not match width/height.. It comes from this line in pixelmatch.
It looks like the obtained buffer (with raw().toBuffer() does not have the length width * height * 4. I don't know why is 4 (probably the channels R-G-B-Aplha?).
I'm a bit lost here. How do you compare images with sharp?

@lovell there is another effect I don't understand. After extract / crop an image, if I take a look of its metada, it has the same width and height of the original image. I apply a clone() function before using extract() and I do not apply any resize() before or after (do I have to?)

I don't know why is multiplied 4 (probably the channels R-G-B-Aplha?).

Yes, try https://sharp.pixelplumbing.com/api-channel#ensurealpha

If I take a look of its metada, it has the same width and height of the original image.

https://sharp.pixelplumbing.com/api-input
"It does not take into consideration any operations to be applied to the output image, such as resize or rotate."
This statement includes the extract operation.
The use of clone is typically used for Stream-based input, so probably isn't required.

Now it works, thanks a lot @lovell !!!
This can be a nice feature in sharp -> Compare Images. I leave there as suggestion XD
Thanks again @lovell

authors: crisconru lovell


text:
[dv/pwrmgr] Check pwrmgr handling of rom inputs

The transition to active state will be blocked depending on rom_ctrl_i and lc inputs. Consider adding a covergroup for this logic.
The logic is described at ROM Integrity Checks.

yeah i think this one can be deferred. It isn't related to any of the reset domain change stuff.
Let's go ahead with it.

authors: matutem tjaychen

repo: lowRISC/opentitan


text:
[RFC] Add support for AES CFB-128 and OFB-128 modes

This issue proposes the addition of Cipher Feedback (CFB) and Output Feedback (OFB) cipher block modes to AES to meet the TPM2.0 specification requirements.
Note that CFB mode is the only cipher required by the TPM2.0 specification. We request support for OFB mode as well since it is very similar to CFB and may be potentially implemented via incremental changes.
Reference: TPM Library Part 1 Architecture Family: 2.0 Level 00 Revision 01.59 Section 11.4.7 Symmetric Encryption.
Filed as RFC since this is a feature request.
Adding for visibility: @vogelpi @alphan @cdgori @gkelly @sjgitty @tjaychen

Thanks @moidx for opening this issue and adding a reference to the relevant TPM spec.
On top of the current architecture, these modes can be added at low hardware cost:

CFB-128 meaning CFB with a feedback length or segment size of 128 bits can be added at zero hardware cost. (Theoretically, the feedback length can be any number between 1 and 128. But according to the referenced document TPM 2.0 only requires 128 - the most simple variant.)
OFB requires one more 128-bit mux input.

By adding one more 128-bit mux input, block operation for both modes can be made 1 cycle faster assuming the input data can be provided fast enough by the processor (12, 14, 16 instead of 13, 15, 17 depending on the key length).
In terms of verification, I think cost is also reasonable. @rasmus-madsen already added support for CBC and CTR modes in DV. Testing also CFB-128 or OFB is mainly a matter of adding support in the DPI model which I would do.

One more note on the feedback length or segment size: Common values are 1, 8 and 128. If we need support for values other than 128, the implementation is going to be much more costly as this requires a lot of additional muxing resources. Low-power devices I have seen so far usually only support 128 bits (what is needed for TPM 2.0). I thus suggest only to implement this version, i.e., CFB-128.

I think it's fair to refine the RFC to be only for CFB-128.  If we find another application in the future that needs say, CFB-8, we can make another RFC.
Given the small resources for CFB-128 (and the need for it in TPM) this is a pretty simple one to justify, I hope.  OFB basically comes "for free" with this, other than the DV cost.

The Technical Committee has reviewed this RFC and requested modifications.
The modifications requested include to clarify the mode sizes requested. If as @vogelpi has suggested, only CFB-128 and OFB-128 are requested, then the necessary changes seem minor enough to have had sufficient review. If other mode sizes are needed, the TC has requested to estimate the cost so that other reviewers can weigh in. Also, if other sizes are requested, the TC would like clarity on the role of satisfying TPM as a requirement to weigh against the increased cost.
The TC will re-review in two weeks after the modifications have been made.

Thanks @sjgitty for reporting back.
According to the TPM 2.0 specification (referenced by @moidx ) the CFB is used with following parameters:

Size of feedback buffer, variable and plaintext variable are all equal to n (size block cipher = 128 for AES). I don't have access to the cited ISO/IEC document to verify the meaning of these parameters but the corresponding NIST document just mentions one single parameter for CFB which is the segment size/feedback length.
OFB does not have a corresponding parameter. Always all 128 bits are fed back.

Updated title to reflect the feedback configuration constrained to the block size.  I agree with @cdgori and @vogelpi to restrict this RFC to CFB-128 and OFB-128 as this fulfills the TPM requirement.

I also don't have access to the cited ISO standard but just wanted to point out that this page says that it has been revised by ISO/IEC 10116:2017.

The Technical Committee has reviewed this RFC and approved the RFC as stated. Thank you for the clarifying updates to the proposal.

The discussed modes have been implemented a while ago already and they are both supported in DV. I consider all the required work to be done and will close this issue now.
Please feel free to reopen if necessary.

authors: alphan cdgori moidx sjgitty vogelpi