text:
[BUG] ESP32 Unable to connect to ASUS ZenWiFi 8 Router

Reproduction steps
Using chip-tool as Admin on macOS. Building ESP32C3 using lighting-app, I am unable to commission the device into the WiFi network. It constantly loops between Connecting -> Connecting_Failed -> NotConnected -> Connecting.
I have supplied the device with the correct credentials.
I (71616) chip[DL]: Confirm received for CHIPoBLE TX characteristic indication (con 1) status= 0
E (71626) chip[DL]: Long dispatch time: 417 ms, for event type 7
I (71696) chip[DL]: Confirm received for CHIPoBLE TX characteristic indication (con 1) status= 14
I (71696) CHIP[DL]: Write request received for CHIPoBLE RX characteristic con 1 12
I (71796) CHIP[DL]: Write request received for CHIPoBLE RX characteristic con 1 12
I (71886) CHIP[DL]: Write request received for CHIPoBLE RX characteristic con 1 12
I (71896) chip[EM]: Received message of type 0x8 with protocolId (0, 1) and MessageCounter:167262557 on exchange 14136r
I (71906) chip[ZCL]: OpCreds: Received an AddNOC command
I (71906) chip[FP]: Validating NOC chain
I (72606) chip[FP]: NOC chain validation successful
I (72606) chip[FP]: Added new fabric at index: 0x1
I (72606) chip[FP]: Assigned compressed fabric ID: 0xFA620EFF58F3EC13, node ID: 0x00FC26B56E2979B0
I (72616) chip[TS]: Last Known Good Time: 2022-08-16T09:46:40
I (72626) chip[TS]: New proposed Last Known Good Time: 2023-03-15T19:26:42
I (72636) chip[TS]: Updating pending Last Known Good Time to 2023-03-15T19:26:42
I (72646) chip[ZCL]: OpCreds: ACL entry created for Fabric index 0x1 CASE Admin Subject 0xFFFFFFFD0011002A
I (72656) chip[DIS]: Advertise operational node FA620EFF58F3EC13-00FC26B56E2979B0
I (72666) chip[DIS]: CHIP minimal mDNS configured as 'Operational device'.
I (72676) chip[DIS]: mDNS service published: _matter._tcp
I (72676) chip[ZCL]: OpCreds: successfully created fabric index 0x1 via AddNOC
I (72686) chip[IN]: Prepared secure message 0x3fcb83d8 to 0xFFFFFFFB00000000 (1)  of type 0x9 and protocolId (0, 1) on exchange 14136r with MessageCounter:69429872.
I (72706) chip[IN]: Sending encrypted msg 0x3fcb83d8 with MessageCounter:69429872 to 0xFFFFFFFB00000000 (1) at monotonic time: 0000000000011BFB msec
I (72716) NimBLE: GATT procedure initiated: indicate;
I (72716) NimBLE: att_handle=14

I (72726) chip[DL]: Confirm received for CHIPoBLE TX characteristic indication (con 1) status= 0
E (72736) chip[DL]: Long dispatch time: 849 ms, for event type 7
I (72816) chip[DL]: Confirm received for CHIPoBLE TX characteristic indication (con 1) status= 14
I (72816) CHIP[DL]: Write request received for CHIPoBLE RX characteristic con 1 12
I (72826) chip[EM]: Received message of type 0x8 with protocolId (0, 1) and MessageCounter:167262558 on exchange 14137r
I (72846) app-devicecallbacks: PostAttributeChangeCallback - Cluster ID: '0x0030', EndPoint ID: '0x00', Attribute ID: '0x0000'
I (72846) app-devicecallbacks: Unhandled cluster ID: 48
I (72856) app-devicecallbacks: Current free heap: 103052

I (72856) chip[IN]: Prepared secure message 0x3fcb83d8 to 0xFFFFFFFB00000000 (1)  of type 0x9 and protocolId (0, 1) on exchange 14137r with MessageCounter:69429873.
I (72886) chip[IN]: Sending encrypted msg 0x3fcb83d8 with MessageCounter:69429873 to 0xFFFFFFFB00000000 (1) at monotonic time: 0000000000011CAE msec
I (72896) NimBLE: GATT procedure initiated: indicate;
I (72896) NimBLE: att_handle=14

I (72906) chip[DL]: Confirm received for CHIPoBLE TX characteristic indication (con 1) status= 0
I (72956) chip[DL]: Confirm received for CHIPoBLE TX characteristic indication (con 1) status= 14
I (72956) CHIP[DL]: Write request received for CHIPoBLE RX characteristic con 1 12
I (72976) chip[EM]: Received message of type 0x8 with protocolId (0, 1) and MessageCounter:167262559 on exchange 14138r
I (72986) chip[NP]: ESP NetworkCommissioningDelegate: SSID: Pretty Fly for a WiFi
I (72986) chip[DL]: WiFi station mode change: Enabled -> Disabled
I (73016) chip[DL]: Changing ESP WiFi mode: STA -> STA+AP
I (73016) wifi:mode : sta (84:f7:03:61:10:c0) + softAP (3a:53:5a:98:b4:a5)
I (73016) wifi:Total power save buffer number: 16
I (73036) wifi:Init max length of beacon: 752/752
I (73036) phy: chan:1,max_power:80
I (73036) phy: chan:2,max_power:80
I (73046) phy: chan:3,max_power:80
I (73046) phy: chan:4,max_power:80
I (73056) phy: chan:5,max_power:80
I (73056) phy: chan:6,max_power:80
I (73056) phy: chan:7,max_power:80
I (73066) phy: chan:8,max_power:80
I (73066) phy: chan:9,max_power:80
I (73066) phy: chan:10,max_power:80
I (73076) phy: chan:11,max_power:80
I (73076) phy: chan:12,max_power:80
I (73076) phy: chan:13,max_power:80
I (73086) phy: chan:14,max_power:80
I (73096) wifi:Init max length of beacon: 752/752
I (73106) chip[DL]: WiFi station mode change: Disabled -> Enabled
E (73106) chip[DL]: Long dispatch time: 132 ms, for event type 7
I (73106) phy: chan:1,max_power:80
I (73116) phy: chan:2,max_power:80
I (73116) phy: chan:3,max_power:80
I (73126) phy: chan:4,max_power:80
I (73126) phy: chan:5,max_power:80
I (73126) phy: chan:6,max_power:80
I (73136) phy: chan:7,max_power:80
I (73136) phy: chan:8,max_power:80
I (73136) phy: chan:9,max_power:80
I (73146) phy: chan:10,max_power:80
I (73156) phy: chan:11,max_power:80
I (73156) phy: chan:12,max_power:80
I (73166) phy: chan:13,max_power:80
I (73166) phy: chan:14,max_power:80
W (73166) wifi:Haven't to connect to a suitable AP now!
I (73176) chip[DL]: Attempting to connect WiFi station interface
I (73176) chip[DL]: WiFi station state change: NotConnected -> Connecting
I (73186) chip[DL]: Done driving station state, nothing else to do...
I (73196) phy: chan:1,max_power:80
I (73196) phy: chan:2,max_power:80
I (73196) phy: chan:3,max_power:80
I (73216) phy: chan:4,max_power:80
I (73216) phy: chan:5,max_power:80
I (73216) phy: chan:6,max_power:80
I (73226) phy: chan:7,max_power:80
I (73226) phy: chan:8,max_power:80
I (73226) phy: chan:9,max_power:80
I (73236) phy: chan:10,max_power:80
I (73236) phy: chan:11,max_power:80
I (73236) phy: chan:12,max_power:80
I (73246) phy: chan:13,max_power:80
I (73246) phy: chan:14,max_power:80
I (73236) wifi:new:<1,1>, old:<1,1>, ap:<1,1>, sta:<1,0>, prof:1
I (73856) wifi:state: init -> auth (b0)
I (73856) chip[DL]: Done driving station state, nothing else to do...
E (73856) chip[DL]: Long dispatch time: 658 ms, for event type 3
I (73866) chip[DL]: WIFI_EVENT_AP_START
I (73866) chip[DL]: WiFi AP state change: NotActive -> Active
I (73876) phy: chan:1,max_power:80
I (73876) phy: chan:2,max_power:80
I (73876) phy: chan:3,max_power:80
I (73896) phy: chan:4,max_power:80
I (73896) phy: chan:5,max_power:80
I (73896) phy: chan:6,max_power:80
I (73906) phy: chan:7,max_power:80
I (73906) phy: chan:8,max_power:80
I (73906) phy: chan:9,max_power:80
I (73916) phy: chan:10,max_power:80
I (73916) phy: chan:11,max_power:80
I (73926) phy: chan:12,max_power:80
I (73926) phy: chan:13,max_power:80
I (73926) phy: chan:14,max_power:80
I (73936) chip[DL]: Changing ESP WiFi mode: STA+AP -> STA
I (73936) wifi:mode : sta (84:f7:03:61:10:c0)
I (73946) chip[DL]: WiFi AP state change: Active -> Deactivating
I (73956) app-devicecallbacks: Current free heap: 103624

I (73956) phy: chan:1,max_power:80
I (73966) phy: chan:2,max_power:80
I (73966) phy: chan:3,max_power:80
I (73966) phy: chan:4,max_power:80
I (73976) phy: chan:5,max_power:80
I (73976) phy: chan:6,max_power:80
I (73976) phy: chan:7,max_power:80
I (73986) phy: chan:8,max_power:80
I (73986) phy: chan:9,max_power:80
I (73986) phy: chan:10,max_power:80
I (73996) phy: chan:11,max_power:80
I (73996) phy: chan:12,max_power:80
I (73996) phy: chan:13,max_power:80
I (74016) phy: chan:14,max_power:80
I (74016) chip[DL]: Done driving station state, nothing else to do...
I (74026) chip[DL]: WIFI_EVENT_AP_STOP
I (74026) chip[DL]: WiFi AP state change: Deactivating -> NotActive
I (74036) phy: chan:1,max_power:80
I (74036) phy: chan:2,max_power:80
I (74036) phy: chan:3,max_power:80
I (74046) phy: chan:4,max_power:80
I (74046) phy: chan:5,max_power:80
I (74046) phy: chan:6,max_power:80
I (74056) phy: chan:7,max_power:80
I (74056) phy: chan:8,max_power:80
I (74066) phy: chan:9,max_power:80
I (74076) phy: chan:10,max_power:80
I (74076) phy: chan:11,max_power:80
I (74086) phy: chan:12,max_power:80
I (74086) phy: chan:13,max_power:80
I (74086) phy: chan:14,max_power:80
I (74096) app-devicecallbacks: Current free heap: 103624

I (74856) wifi:state: auth -> init (200)
I (74856) wifi:new:<1,0>, old:<1,1>, ap:<255,255>, sta:<1,0>, prof:1
I (74856) chip[DL]: WIFI_EVENT_STA_DISCONNECTED
I (74866) chip[DL]: WiFi station state change: Connecting -> Connecting_Failed
I (74866) phy: chan:1,max_power:80
I (74876) phy: chan:2,max_power:80
I (74886) phy: chan:3,max_power:80
I (74886) phy: chan:4,max_power:80
I (74896) phy: chan:5,max_power:80
I (74896) phy: chan:6,max_power:80
I (74896) phy: chan:7,max_power:80
I (74906) phy: chan:8,max_power:80
I (74906) phy: chan:9,max_power:80
I (74906) phy: chan:10,max_power:80
I (74916) phy: chan:11,max_power:80
I (74916) phy: chan:12,max_power:80
I (74916) phy: chan:13,max_power:80
I (74926) phy: chan:14,max_power:80
W (74926) wifi:Haven't to connect to a suitable AP now!
I (74936) chip[DL]: WiFi station state change: Connecting_Failed -> NotConnected
I (74936) chip[DL]: Next WiFi station reconnect in 5000 ms
I (74956) chip[DL]: Done driving station state, nothing else to do...
I (74956) app-devicecallbacks: Current free heap: 104028

E (74966) chip[DL]: Long dispatch time: 106 ms, for event type 49152
W (74966) wifi:Haven't to connect to a suitable AP now!
E (74976) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
W (74986) wifi:Haven't to connect to a suitable AP now!
E (74986) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
I (75476) NimBLE: GATT procedure initiated: indicate;
I (75476) NimBLE: att_handle=14

I (75476) chip[DL]: Confirm received for CHIPoBLE TX characteristic indication (con 1) status= 0
I (75546) chip[DL]: Confirm received for CHIPoBLE TX characteristic indication (con 1) status= 14
I (79956) phy: chan:1,max_power:80
I (79956) phy: chan:2,max_power:80
I (79956) phy: chan:3,max_power:80
I (79956) phy: chan:4,max_power:80
I (79966) phy: chan:5,max_power:80
I (79966) phy: chan:6,max_power:80
I (79976) phy: chan:7,max_power:80
I (79976) phy: chan:8,max_power:80
I (79976) phy: chan:9,max_power:80
I (79986) phy: chan:10,max_power:80
I (79986) phy: chan:11,max_power:80
I (79986) phy: chan:12,max_power:80
I (79996) phy: chan:13,max_power:80
I (79996) phy: chan:14,max_power:80
W (79996) wifi:Haven't to connect to a suitable AP now!
I (80006) chip[DL]: Attempting to connect WiFi station interface
I (80016) chip[DL]: WiFi station state change: NotConnected -> Connecting
I (80026) chip[DL]: Done driving station state, nothing else to do...
W (80026) wifi:Haven't to connect to a suitable AP now!
I (80036) wifi:new:<1,0>, old:<1,0>, ap:<255,255>, sta:<1,0>, prof:1
I (80036) wifi:state: init -> auth (b0)
E (80036) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
I (81036) wifi:state: auth -> init (200)
I (81036) wifi:new:<1,0>, old:<1,0>, ap:<255,255>, sta:<1,0>, prof:1
I (81046) chip[DL]: WIFI_EVENT_STA_DISCONNECTED
I (81056) chip[DL]: WiFi station state change: Connecting -> Connecting_Failed
I (81056) phy: chan:1,max_power:80
I (81066) phy: chan:2,max_power:80
I (81066) phy: chan:3,max_power:80
I (81066) phy: chan:4,max_power:80
I (81076) phy: chan:5,max_power:80
I (81076) phy: chan:6,max_power:80
I (81076) phy: chan:7,max_power:80
I (81086) phy: chan:8,max_power:80
I (81086) phy: chan:9,max_power:80
I (81086) phy: chan:10,max_power:80
I (81096) phy: chan:11,max_power:80
I (81096) phy: chan:12,max_power:80
I (81106) phy: chan:13,max_power:80
I (81116) phy: chan:14,max_power:80
W (81116) wifi:Haven't to connect to a suitable AP now!
I (81126) chip[DL]: WiFi station state change: Connecting_Failed -> NotConnected
I (81126) chip[DL]: Next WiFi station reconnect in 5000 ms
I (81136) chip[DL]: Done driving station state, nothing else to do...
I (81136) app-devicecallbacks: Current free heap: 104144

E (81146) chip[DL]: Long dispatch time: 106 ms, for event type 49152
W (81146) wifi:Haven't to connect to a suitable AP now!
E (81156) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
W (81176) wifi:Haven't to connect to a suitable AP now!
E (81176) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
I (86136) phy: chan:1,max_power:80
I (86136) phy: chan:2,max_power:80
I (86136) phy: chan:3,max_power:80
I (86136) phy: chan:4,max_power:80
I (86146) phy: chan:5,max_power:80
I (86146) phy: chan:6,max_power:80
I (86156) phy: chan:7,max_power:80
I (86156) phy: chan:8,max_power:80
I (86156) phy: chan:9,max_power:80
I (86166) phy: chan:10,max_power:80
I (86176) phy: chan:11,max_power:80
I (86176) phy: chan:12,max_power:80
I (86186) phy: chan:13,max_power:80
I (86186) phy: chan:14,max_power:80
W (86186) wifi:Haven't to connect to a suitable AP now!
I (86196) chip[DL]: Attempting to connect WiFi station interface
I (86196) chip[DL]: WiFi station state change: NotConnected -> Connecting
I (86206) chip[DL]: Done driving station state, nothing else to do...
W (86206) wifi:Haven't to connect to a suitable AP now!
E (86216) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
I (89326) chip[DL]: WIFI_EVENT_STA_DISCONNECTED
I (89326) chip[DL]: WiFi station state change: Connecting -> Connecting_Failed
I (89336) phy: chan:1,max_power:80
I (89336) phy: chan:2,max_power:80
I (89336) phy: chan:3,max_power:80
I (89346) phy: chan:4,max_power:80
I (89346) phy: chan:5,max_power:80
I (89346) phy: chan:6,max_power:80
I (89366) phy: chan:7,max_power:80
I (89366) phy: chan:8,max_power:80
I (89366) phy: chan:9,max_power:80
I (89376) phy: chan:10,max_power:80
I (89376) phy: chan:11,max_power:80
I (89376) phy: chan:12,max_power:80
I (89386) phy: chan:13,max_power:80
I (89386) phy: chan:14,max_power:80
W (89386) wifi:Haven't to connect to a suitable AP now!
I (89396) chip[DL]: WiFi station state change: Connecting_Failed -> NotConnected
I (89406) chip[DL]: Next WiFi station reconnect in 5000 ms
I (89406) chip[DL]: Done driving station state, nothing else to do...
I (89416) app-devicecallbacks: Current free heap: 104304

W (89426) wifi:Haven't to connect to a suitable AP now!
E (89436) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
W (89436) wifi:Haven't to connect to a suitable AP now!
E (89446) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
E (90486) chip[BLE]: ack recv timeout, closing ep 0x3fc9f530
I (90486) chip[BLE]: Releasing end point's BLE connection back to application.
I (94406) phy: chan:1,max_power:80
I (94406) phy: chan:2,max_power:80
I (94406) phy: chan:3,max_power:80
I (94416) phy: chan:4,max_power:80
I (94426) phy: chan:5,max_power:80
I (94426) phy: chan:6,max_power:80
I (94436) phy: chan:7,max_power:80
I (94436) phy: chan:8,max_power:80
I (94436) phy: chan:9,max_power:80
I (94446) phy: chan:10,max_power:80
I (94446) phy: chan:11,max_power:80
I (94446) phy: chan:12,max_power:80
I (94456) phy: chan:13,max_power:80
I (94456) phy: chan:14,max_power:80
W (94456) wifi:Haven't to connect to a suitable AP now!
I (94466) chip[DL]: Attempting to connect WiFi station interface
I (94466) chip[DL]: WiFi station state change: NotConnected -> Connecting
I (94486) chip[DL]: Done driving station state, nothing else to do...
W (94486) wifi:Haven't to connect to a suitable AP now!
E (94496) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
I (94536) wifi:new:<1,0>, old:<1,0>, ap:<255,255>, sta:<1,0>, prof:1
I (94536) wifi:state: init -> auth (b0)
I (95536) wifi:state: auth -> init (200)
I (95536) wifi:new:<1,0>, old:<1,0>, ap:<255,255>, sta:<1,0>, prof:1
I (95536) chip[DL]: WIFI_EVENT_STA_DISCONNECTED
I (95546) chip[DL]: WiFi station state change: Connecting -> Connecting_Failed
I (95546) phy: chan:1,max_power:80
I (95556) phy: chan:2,max_power:80
I (95556) phy: chan:3,max_power:80
I (95566) phy: chan:4,max_power:80
I (95576) phy: chan:5,max_power:80
I (95576) phy: chan:6,max_power:80
I (95576) phy: chan:7,max_power:80
I (95586) phy: chan:8,max_power:80
I (95586) phy: chan:9,max_power:80
I (95586) phy: chan:10,max_power:80
I (95596) phy: chan:11,max_power:80
I (95596) phy: chan:12,max_power:80
I (95596) phy: chan:13,max_power:80
I (95606) phy: chan:14,max_power:80
W (95606) wifi:Haven't to connect to a suitable AP now!
I (95616) chip[DL]: WiFi station state change: Connecting_Failed -> NotConnected
I (95616) chip[DL]: Next WiFi station reconnect in 5000 ms
I (95636) chip[DL]: Done driving station state, nothing else to do...
I (95636) app-devicecallbacks: Current free heap: 104156

E (95646) chip[DL]: Long dispatch time: 106 ms, for event type 49152
W (95646) wifi:Haven't to connect to a suitable AP now!
E (95656) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
W (95666) wifi:Haven't to connect to a suitable AP now!
E (95666) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F

> I (100636) phy: chan:1,max_power:80
I (100636) phy: chan:2,max_power:80
I (100636) phy: chan:3,max_power:80
I (100636) phy: chan:4,max_power:80
I (100646) phy: chan:5,max_power:80
I (100646) phy: chan:6,max_power:80
I (100656) phy: chan:7,max_power:80
I (100656) phy: chan:8,max_power:80
I (100656) phy: chan:9,max_power:80
I (100666) phy: chan:10,max_power:80
I (100666) phy: chan:11,max_power:80
I (100666) phy: chan:12,max_power:80
I (100676) phy: chan:13,max_power:80
I (100676) phy: chan:14,max_power:80
W (100676) wifi:Haven't to connect to a suitable AP now!
I (100686) chip[DL]: Attempting to connect WiFi station interface
I (100696) chip[DL]: WiFi station state change: NotConnected -> Connecting
I (100706) chip[DL]: Done driving station state, nothing else to do...
I (100716) wifi:new:<1,0>, old:<1,0>, ap:<255,255>, sta:<1,0>, prof:1
I (100716) wifi:state: init -> auth (b0)
I (101716) wifi:state: auth -> init (200)
I (101716) wifi:new:<1,0>, old:<1,0>, ap:<255,255>, sta:<1,0>, prof:1
I (101716) chip[DL]: WIFI_EVENT_STA_DISCONNECTED
I (101736) chip[DL]: WiFi station state change: Connecting -> Connecting_Failed
I (101736) phy: chan:1,max_power:80
I (101746) phy: chan:2,max_power:80
I (101746) phy: chan:3,max_power:80
I (101746) phy: chan:4,max_power:80
I (101756) phy: chan:5,max_power:80
I (101756) phy: chan:6,max_power:80
I (101756) phy: chan:7,max_power:80
I (101766) phy: chan:8,max_power:80
I (101766) phy: chan:9,max_power:80
I (101766) phy: chan:10,max_power:80
I (101776) phy: chan:11,max_power:80
I (101776) phy: chan:12,max_power:80
I (101786) phy: chan:13,max_power:80
I (101786) phy: chan:14,max_power:80
W (101786) wifi:Haven't to connect to a suitable AP now!
I (101806) chip[DL]: WiFi station state change: Connecting_Failed -> NotConnected
I (101806) chip[DL]: Next WiFi station reconnect in 5000 ms
I (101816) chip[DL]: Done driving station state, nothing else to do...
I (101826) app-devicecallbacks: Current free heap: 104144

E (101826) chip[DL]: Long dispatch time: 106 ms, for event type 49152
W (101836) wifi:Haven't to connect to a suitable AP now!
E (101836) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
W (101846) wifi:Haven't to connect to a suitable AP now!
E (101846) chip[DL]: Failed to get configured network when updating network status: Error ESP32:0x0500300F
I (103116) chip[IN]: Prepared secure message 0x3fcb8688 to 0xFFFFFFFB00000000 (1)  of type 0x9 and protocolId (0, 1) on exchange 14138r with MessageCounter:69429874.
I (103136) chip[IN]: Sending encrypted msg 0x3fcb8688 with MessageCounter:69429874 to 0xFFFFFFFB00000000 (1) at monotonic time: 00000000000192DA msec
I (106816) phy: chan:1,max_power:80
I (106816) phy: chan:2,max_power:80
I (106816) phy: chan:3,max_power:80
I (106816) phy: chan:4,max_power:80
I (106826) phy: chan:5,max_power:80
I (106826) phy: chan:6,max_power:80
I (106836) phy: chan:7,max_power:80
I (106836) phy: chan:8,max_power:80
I (106836) phy: chan:9,max_power:80
I (106846) phy: chan:10,max_power:80
I (106846) phy: chan:11,max_power:80
I (106846) phy: chan:12,max_power:80
I (106856) phy: chan:13,max_power:80
I (106866) phy: chan:14,max_power:80
W (106866) wifi:Haven't to connect to a suitable AP now!
I (106876) chip[DL]: Done driving station state, nothing else to do...

Bug prevalence
Whenever I do this
GitHub hash of the SDK that was being used
561d23d
Platform
esp32
Platform Version(s)
1.0.0
Anything else?
No response

@s-mcclain Just to make sure: this is a 2.4GHz WiFi network, right?
@shubhamdp @dhrishi

@s-mcclain ESP32 support 2.4GHz. Please confirm that the Wi-Fi network is not 5GHz. Else the Wi-Fi connection will fail.

@bzbarsky-apple / @dhrishi yes that's correct. The network which I've provided is 2.4 GHz.
The WiFi network SSIDs are also separated based on the frequency (i.e. There exists WiFi A for 2.4G, WiFi B for 5G, and WiFi C for 6G.

Another note... Seems like if I use a different ESP32C3 then it appears to be connecting (for now at least). Same commands and everything.
From glancing through the WiFi Router logs, there is nothing which indicates it has blocked the device from associating into the network. But even after waiting several days, it still has not been able to join into the network successfully.

As it worked with a different ESP32C3, it should not be related to the software. May be it is some issue with the devkit.

authors: bzbarsky-apple dhrishi s-mcclain


text:
ZCL Cleanup Plugin Macro

Problem
Since plugins macros are not generated anymore by ZAP they were moved to /src/app/util/plugin-config.h However no cleanup was done to see which one we should keep.
Proposed Solution
Removed unused macro from /src/app/util/plugin-config.h

I have gone thought all the macros. Here is the list of macros that are currently used from the set of macros from plugin-config.h from #3464 :
#define EMBER_BINDING_TABLE_SIZE 10
#define EMBER_AF_PLUGIN_COLOR_CONTROL_SERVER_XY
#define EMBER_AF_PLUGIN_COLOR_CONTROL_SERVER_TEMP
#define EMBER_AF_PLUGIN_COLOR_CONTROL_SERVER_HSV
#define EMBER_AF_PLUGIN_GROUPS_SERVER
#define EMBER_AF_PLUGIN_IAS_ZONE_CLIENT
#define EMBER_AF_PLUGIN_IAS_ZONE_CLIENT_MAX_DEVICES 10
#define EMBER_AF_PLUGIN_IAS_ZONE_SERVER_ZONE_TYPE 541
#define EMBER_AF_PLUGIN_LEVEL_CONTROL
#define EMBER_AF_PLUGIN_LEVEL_CONTROL_MAXIMUM_LEVEL 255
#define EMBER_AF_PLUGIN_LEVEL_CONTROL_MINIMUM_LEVEL 0
#define EMBER_AF_PLUGIN_LEVEL_CONTROL_RATE 0
#define EMBER_AF_PLUGIN_REPORTING
#define EMBER_AF_PLUGIN_REPORTING_TABLE_SIZE 5
#define EMBER_AF_PLUGIN_REPORTING_ENABLE_GROUP_BOUND_REPORTS
#define EMBER_AF_PLUGIN_SCENES
#define EMBER_AF_PLUGIN_SCENES_TABLE_SIZE 3

That said I don't think adding src/app/util/plugin-config.h is the way to go.
I would have expect ZAP to have generated something  in gen_config.h.
For example, if the groups server is enabled, it could have generated:
#define EMBER_AF_PLUGIN_GROUPS_SERVER

Of if any of the attributes is tagged as reportable, it could have generated:
#define EMBER_AF_PLUGIN_REPORTING

Or if the Level Control server has been selected, it could have generated:
#define EMBER_AF_PLUGIN_LEVEL_CONTROL
#define EMBER_AF_PLUGIN_LEVEL_CONTROL_MAXIMUM_LEVEL 255
#define EMBER_AF_PLUGIN_LEVEL_CONTROL_MINIMUM_LEVEL 0
#define EMBER_AF_PLUGIN_LEVEL_CONTROL_RATE 0

Or depending on which commands have been selected for the Color Control cluster it may have generated:
#define EMBER_AF_PLUGIN_COLOR_CONTROL_SERVER_XY

but not
#define EMBER_AF_PLUGIN_COLOR_CONTROL_SERVER_TEMP

I understand that the generated templates may differs for a different framework than CHIP, but since in this case we are speaking of templates targeting CHIP itself I would have expected those to match the current code architecture.
So I think that's something we need to fix in the current templates that will land in #3638 since having those #define be part of src/app/ have some side effects on the other clusters.

source: gharchive/issue

authors: jepenven-silabs vivien-apple

repo: project-chip/connectedhomeip

license_type: permissive

license_source: github-api


text:
Pull cirque repo

Problem
ToT master seems to fail cirque checks. Suspect due to cirque repo version.
Change overview
cd third_party/cirque/repo
git checkout master
git pull origin

Testing
Expect CI to pass on cirque.

Cirque CI passes (the rest are flakes). Marking as hotfix.

source: gharchive/pull-request


text:
Update code PairDevice calls to work with network params.

Problem
The setup code PairDevice functions had no way to pass in commissioning parameters, which meant it wasn't possible to commission BLE devices by code.
Change overview

Adds a function to the auto commissioner to allow adding commissioning parameters incrementally
Add commissioning parameters as a function parameter to the pair device calls for setup codes
Adds pairing options to chip tool to allow this path

Testing
Chip tool commissioning M5 over BLE

This PR aligns all high-level commissioning pathways to use the auto commissioning state machine. In the case of a cli based human driven command this might make sense but it also forces the commissioner using the payload string APIs to gather network credentials for all possible networks prior to commissioning.
It remains desirable for some use cases to only have PASE setup and to trigger commissioning at some point later.
For example, what if you don't have Thread credentials available? What if you want to present an error when you realize that someone/something is trying to commission an accessory that your network cannot support? How do you even query what network technologies the accessory has without attempting to provision it with something.
At least for now, we should allow the caller to specify auto commissioning when using the payload APIs and update SetupCodePairer to also just perform PASE if requested by the caller.
As an aside, pretty sure this breaks CHIPTool on iOS but that thing gets no love these days.
@msandstedt thoughts?

I would be open to the idea that the setup code pairer triggers a pase connection only in all cases (BLE and on-network), and commission is called separately. But the accessor functions for that should be renamed to make that clear.

Proposal:

I fix Michael's thing and remove the "add" command in the auto commissioner
PairDevice commands using the setup code pairer trigger commissioning using network credentials provided apriori. Once https://github.com/project-chip/connectedhomeip/pull/13829 goes in, you can just provide thread and wifi and the commissioner will select the appropriate one.
We add additional EstablishPASEConnection functions based off the setup code that trigger PASE only for folks that want that. That way we have all the combos.

Was the intent to have set up code pairer do pase only? Because that's not how it was written originally. In the original instance, setup code pairer triggered commissioning and it is named as such.

You're completely right about this. The meaning of Pairing got separated from Commissioning and CASE at some point and now they're being aligned again so that explains the weirdness with naming all over the place.
It used to mean "PASE is ready and node is Commissioned" after which it was up to the commissioner to actually trigger the CASE (via get Connected Device) after providing network credentials. But that's old stuff now so let's ignore it.
Your proposal for the most part sounds acceptable.

PairDevice commands using the setup code pairer trigger commissioning using network credentials provided apriori. Once Check feature map for network technology #13829 goes in, you can just provide thread and wifi and the commissioner will select the appropriate one.

I think the problem here is that it's a little bit backwards to handover everything before even starting to commission.

We add additional EstablishPASEConnection functions based off the setup code that trigger PASE only for folks that want that. That way we have all the combos.

This sounds good. Unfortunately, I'm not sure how to exercise this from the cli tooling. So that's something we'll have to think about.

I would be open to the idea that the setup code pairer triggers a pase connection only in all cases (BLE and on-network), and commission is called separately. But the accessor functions for that should be renamed to make that clear.

In my mind, the primary reason to include PASE with discovery (the SetUpCodePairer's current primary role seems to be discovery) is the ambiguity around the discriminator.  Location of a node with a matching discriminator does not guarantee you've found the node you're looking for.
But if the code that discovers also establishes PASE, then it can hand back a session to the commissioner and commissioner will have very high confidence that it is talking to the node it intends, because successful establishment of PASE means both nodes have the same shared secret.  Furthermore, we always have to establish PASE, so this doesn't introduce more branching in the discoverer.
However, I find it odd that the SetUpCodePairer would have to do any other commissioning steps.  It seems like instead of the SetUpCodePairer consuming a commissioner, it should provide a delegate interface so that it can signal the commissioner when its done.  This would leave the commissioner to make any further commissioning decisions.

This PR forces getting network credentials before doing discovery and finding out whether they are even needed.
That's not acceptable, unfortunately. We have to allow commissioners using SetUpCodePairer to do discovery and set up PASE and only then deal with getting network credentials as needed.

Yes, it will not be possible to realize many necessary product flows unless network credentials can be provided dynamically, as needed.  And in many cases this will be after discovery.
However, sorting that out seems like a heavy lift for our feature cut deadline.  If this is a critical feature (in my view it is), can't we bring this through the necessary gatekeeping to potentially continue development and really solve the problem?

OK, I updated this to be a bit more specific about what each part of the API does. I don't really love the "PairDevice" nomenclature we have going on, but we're kind of past the point of fixing that - it's deeply embedded into the stack.
I added an option to establish a pase connection though the setup code parier. That gets hooked up thorough chip tool by not specifiying a network technology. I really don't love that, but I also don't really want to rework chip tool to provide an entirely separate command for establishing a pase connection only. I would argue that chip tool is example code anyway, and that can probably be done after TE8.

If you want to get information about the device, you can establish a pase connection, read whatever, fill in the required parameters and then commission. This is the same as before, just that the device isn't hanging out in a half-commissioned state with no failsafe running.

authors: cecille msandstedt sagar-apple


text:
Reduce number of replies on MinMds query processing.

UDP packets are received on every listening interface, resulting in single queries being multipled by the interface count and this causes more packets than expected being generated.
This change will only process queries that are originating from the same interface that the mdns endpoint is bound to.

Marking as ready for review as this reduces number of replies in minmdns.
I am still wondering if maybe there is an update potential for minmdns to use less udp sockets and listening ports as I would expect addr_any to cover listening over more interfaces (the existing behavior seems to cover that).

obsolete after #27219

Re-opened as the other PR seems to have issues on OpenIOT. The error of Failed to reply to query that I get there is not helpful, so I am unsure on how long this will take to debug. Maybe once we have better tracing/logging piped through we can actually see what queries and answers are seen.

can we cherry pick this PR on v1.1-branch ?


text:
[v1.2] [ESP32] Platform Event when ble is deinitialized (#33186)

Cherry-picking 040e5bfd0f6096750633d5cdce204f96a58cdaec (MR #33186) on v1.2-branch

Merging the platform specific change to v1.2

authors: dhrishi shubhamdp


text:
TC-OPCREDS-3.1 The script does not display information for specific steps.

Please follow the below template to file issues/concerns/feature enhancement against Test Scripts:
Summary Title:
[SVE Matter 1.3] TC-OPCREDS-3.1 The script does not display information for specific steps.

Description:
When running the Python script for TC-OPCREDS_3.1, the script does not provide a response for step 16, step 31, step 69, and step 70. Note that the script does seem to run the steps i.e. Step 17 checks the output of Step 16 and passes.
Steps to reproduce:
Run TC-OPCREDS_3.1 using the TH UI.
Logs:
TC-OPCREDS_3-1_2024_03_06_19_26_18.log
Additional Info:

Step 16, please see https://github.com/project-chip/connectedhomeip/issues/30798
Step 31 - there is no response expected here - there is no step, just a further check of the fabrics table read in step 30
Step 69 - please see https://github.com/project-chip/connectedhomeip/issues/31265
Step 70 - depends on step 69

Hi @CoekaertsJasper can you give it a shot with the modified script that cecille has provided above and confirm if the test is passing ?

Those aren't script modifications, they're links to sdk bugs

Need to wait for these SDK bugs to get fixed
Step 16, please see https://github.com/project-chip/connectedhomeip/issues/30798
Step 31 - there is no response expected here - there is no step, just a further check of the fabrics table read in step 30
Step 69 - please see https://github.com/project-chip/connectedhomeip/issues/31265
Step 70 - depends on step 69

authors: CoekaertsJasper cecille raju-apple

repo: project-chip/matter-test-scripts


text:
there are errors in endpoint_config if added multiple eps

Dear all,
there are errors in endpoint_config.h if I added ep0(root node), ep1(humidity sensor) and ep2(temperature sensor), is there anything should to be noticed? or is there ways to updated my zap application?
The brief error as below, I also attched the endpoint_config.h file, please kindly take a look, thank you.
../../../examples/H1_EF4_Z22/efr32/third_party/connectedhomeip/zzz_generated/H1_EF4_Z22/zap-generated/endpoint_config.h:671:29: error: exponent has no digits
  671 | #define ATTRIBUTE_MAX_SIZE (337ERROR: DeviceTypeList, invalid size, arrayERROR: ServerList, invalid size, arrayERROR: ClientList, invalid size, arrayERROR: PartsList, invalid size, array4211142)
      |                             ^~~~~~~~
../../../examples/H1_EF4_Z22/efr32/third_party/connectedhomeip/src/app/util/attribute-storage.cpp:65:6: note: in expansion of macro 'ATTRIBUTE_MAX_SIZE'
   65 | #if (ATTRIBUTE_MAX_SIZE == 0)
      |      ^~~~~~~~~~~~~~~~~~
../../../examples/H1_EF4_Z22/efr32/third_party/connectedhomeip/zzz_generated/H1_EF4_Z22/zap-generated/endpoint_config.h:671:37: error:  ':' without preceding '?'
  671 | #define ATTRIBUTE_MAX_SIZE (337ERROR: DeviceTypeList, invalid size, arrayERROR: ServerList, invalid size, arrayERROR: ClientList, invalid size, arrayERROR: PartsList, invalid size, array4211142)
      |                                     ^
../../../examples/H1_EF4_Z22/efr32/third_party/connectedhomeip/src/app/util/attribute-storage.cpp:65:6: note: in expansion of macro 'ATTRIBUTE_MAX_SIZE'
   65 | #if (ATTRIBUTE_MAX_SIZE == 0)
      |      ^~~~~~~~~~~~~~~~~~
../../../examples/H1_EF4_Z22/efr32/third_party/connectedhomeip/zzz_generated/H1_EF4_Z22/zap-generated/endpoint_config.h:671:29: error: exponent has no digits
  671 | #define ATTRIBUTE_MAX_SIZE (337ERROR: DeviceTypeList, invalid size, arrayERROR: ServerList, invalid size, arrayERROR: ClientList, invalid size, arrayERROR: PartsList, invalid size, array4211142)
      |                             ^~~~~~~~
../../../examples/H1_EF4_Z22/efr32/third_party/connectedhomeip/src/app/util/attribute-storage.cpp:68:31: note: in expansion of macro 'ATTRIBUTE_MAX_SIZE'
   68 | #define ACTUAL_ATTRIBUTE_SIZE ATTRIBUTE_MAX_SIZE
      |                               ^~~~~~~~~~~~~~~~~~
../../../examples/H1_EF4_Z22/efr32/third_party/connectedhomeip/src/app/util/attribute-storage.cpp:71:23: note: in expansion of macro 'ACTUAL_ATTRIBUTE_SIZE'
   71 | uint8_t attributeData[ACTUAL_ATTRIBUTE_SIZE];

Adding + @bzbarsky-apple and @tecimovic to the conversation. Any reason we still have not shifted to zap-config.h template for Matter?

@brdandu Does zap-config generate output that can be consumed by the Silicon Labs code that was contributed to Matter?
But for the above: what version of ZAP is being used? That looks like something that was not handling non-external list-typed attributes correctly, which has been fixed for a long time.....

I do not know if zap-config has everything that endpoint_config has since there may have been a divergence that I do not know. @jmartinez-silabs , @jepenven-silabs it might be good to check the difference between the two an add whatever is needed to zap-config.
@bzbarsky-apple, @Leo2442926161  can I close this issue in that case?

Well, it would be good to understand the exact steps that were followed that led to the broken behavior...

Closing since there was no follow up with exact steps. Please reopen if you would like to follow up.

Can we reopen this issue?
I've got the below build errors:
                 from /home/lopeznegrete/ncs/modules/lib/matter/src/app/util/attribute-storage.cpp:18: /home/lopeznegrete/Projects/Matter/tempsensor/src/zap-generated/endpoint_config.h:632:29: error: exponent has no digits   632 | #define ATTRIBUTE_MAX_SIZE (125ERROR: DeviceTypeList, invalid size, arrayERROR: ServerList, invalid size, arrayERROR: ClientList, invalid size, arrayERROR: PartsList, invalid size, array4222242)       |                             ^~~~~~~~ /home/lopeznegrete/ncs/modules/lib/matter/src/app/util/attribute-storage.cpp:50:6: note: in expansion of macro 'ATTRIBUTE_MAX_SIZE'    50 | #if (ATTRIBUTE_MAX_SIZE == 0)       |      ^~~~~~~~~~~~~~~~~~ /home/lopeznegrete/Projects/Matter/tempsensor/src/zap-generated/endpoint_config.h:632:37: error:  ':' without preceding '?'   632 | #define ATTRIBUTE_MAX_SIZE (125ERROR: DeviceTypeList, invalid size, arrayERROR: ServerList, invalid size, arrayERROR: ClientList, invalid size, arrayERROR: PartsList, invalid size, array4222242)       |                                     ^ /home/lopeznegrete/ncs/modules/lib/matter/src/app/util/attribute-storage.cpp:50:6: note: in expansion of macro 'ATTRIBUTE_MAX_SIZE'    50 | #if (ATTRIBUTE_MAX_SIZE == 0)       |      ^~~~~~~~~~~~~~~~~~ /home/lopeznegrete/Projects/Matter/tempsensor/src/zap-generated/endpoint_config.h:632:29: error: exponent has no digits   632 | #define ATTRIBUTE_MAX_SIZE (125ERROR: DeviceTypeList, invalid size, arrayERROR: ServerList, invalid size, arrayERROR: ClientList, invalid size, arrayERROR: PartsList, invalid size, array4222242)       |                             ^~~~~~~~ /home/lopeznegrete/ncs/modules/lib/matter/src/app/util/attribute-storage.cpp:53:31: note: in expansion of macro 'ATTRIBUTE_MAX_SIZE'    53 | #define ACTUAL_ATTRIBUTE_SIZE ATTRIBUTE_MAX_SIZE       |                               ^~~~~~~~~~~~~~~~~~ /home/lopeznegrete/ncs/modules/lib/matter/src/app/util/attribute-storage.cpp:56:23: note: in expansion of macro 'ACTUAL_ATTRIBUTE_SIZE'    56 | uint8_t attributeData[ACTUAL_ATTRIBUTE_SIZE];       |                       ^~~~~~~~~~~~~~~~~~~~~ /home/lopeznegrete/Projects/Matter/tempsensor/src/zap-generated/endpoint_config.h:587:22: error: exponent has no digits   587 |       .clusterSize = 0ERROR: DeviceTypeList, invalid size, arrayERROR: ServerList, invalid size, arrayERROR: ClientList, invalid size, arrayERROR: PartsList, invalid size, array42, \       |                      ^~~~~~ /home/lopeznegrete/ncs/modules/lib/matter/src/app/util/attribute-storage.cpp:104:54: note: in expansion of macro 'GENERATED_CLUSTERS'   104 | constexpr const EmberAfCluster generatedClusters[] = GENERATED_CLUSTERS;       |                                                      ^~~~~~~~~~~~~~~~~~ /home/lopeznegrete/Projects/Matter/tempsensor/src/zap-generated/endpoint_config.h:617:31: error: exponent has no digits   617 |   { ZAP_CLUSTER_INDEX(14), 3, 90ERROR: DeviceTypeList, invalid size, arrayERROR: ServerList, invalid size, arrayERROR: ClientList, invalid size, arrayERROR: PartsList, invalid size, array4212 }, \
I have completed the following steps:

Starting from the template sample under the NordicSemi ncs v2.4.0.
Remove zap-generated folder
Open src/template.zap file with zap tool.
Then. Select '+ ADD NEW ENDPOINT' > Select 'Matter Temperature Sensor (0x0302)' > Press Create
Then, generate files to src/zap-generated folder
Execute west build

This line is extracted from endpoint_config.h. From other zap-generated files, this define is a number (e.g (170)).
#define ATTRIBUTE_MAX_SIZE (125ERROR: DeviceTypeList, invalid size, arrayERROR: ServerList, invalid size, arrayERROR: ClientList, invalid size, arrayERROR: PartsList, invalid size, array4222242)

Solved the error by defining DeviceTypeList, ServerList, ClientList, and PartsList to 0 instead of leaving it blank

Solved the error by defining DeviceTypeList, ServerList, ClientList, and PartsList to 0 instead of leaving it blank

Defining it to 0 where exactly, @lopeznegrete ?

That really should not matter for these attributes...  It sounds like something is very confused in the way you are generating these things, so the "these must be external" logic is not getting picked up.  Setting default to 0 might "fix" the compile issue, but I expect the generated configuration is still wrong.

@bzbarsky-apple
I am using the zap tool out of the box and following the above steps. No changes whatsoever.  What could be confused in the way that files are generated? What "external" logic is not picked up?
Zap and system version

Exactly what are the contents of "template sample under the NordicSemi ncs v2.4.0"?
Exactly how do you do the "generate files to src/zap-generated folder" step?

Exactly what are the contents of "template sample under the NordicSemi ncs v2.4.0"?

Template sample under Nordic NCS v2.4.0 branch
https://github.com/nrfconnect/sdk-nrf/tree/v2.4-branch/samples/matter/template
No much to add here.

Exactly how do you do the "generate files to src/zap-generated folder" step?

I click on the generate option in the zap tool and selected the src/zap-generated folder as target folder. Please see image below.

No much else to add here neither.

I click on the generate option in the zap tool

@lopeznegrete There is your problem.  That's almost certainly not passing the right config to the generation to take into account SDK settings.  You want to use the Matter generation scripts...

I click on the generate option in the zap tool

@lopeznegrete There is your problem. That's almost certainly not passing the right config to the generation to take into account SDK settings. You want to use the Matter generation scripts...

How do you do that?

https://github.com/project-chip/connectedhomeip/blob/master/docs/code_generation.md#running-codegen

I click on the generate option in the zap tool

@lopeznegrete There is your problem. That's almost certainly not passing the right config to the generation to take into account SDK settings. You want to use the Matter generation scripts...

I don't find it appropriate to quote it as 'your problem'. From a user perspective, if there is a button that states "Generate ..." take for granted that most people will understand this button is to generate the files. Regardless if it is a Matter generation. To be constructive, that button or info around should indicate under what circumstances it will generate valid files.

Just to be clear:

"your problem" here just meant "the problem you are running into".
I completely agree that the fact that "Generate..." is (a) enabled and (b) does the wrong thing is wrong.

That said, filing a new clear issue about that problem, rather than using this one (which was initially about something else and has a bunch of noise) is probably the best approach here.

repo: project-chip/zap

license_source: bigquery


text:
Inaccurate accounting of GPUs available to high-priority AppWrappers

High-priority AppWrappers sometimes are not dispatched despite sufficient resource availability if properly discounting resources in use by lower-priority AppWrappers.

@metalcycling could you flesh out this issue?

@metalcycling @asm582 can we close this issue given that we have disabled histograms for now?


text:
[QUESTION] Getting error when running copa patch

What is your question?
I would like to know what am I missing in my patch command below.  I get error when running copa patch ....  Some help is very much appreciated.  Thank you.
Copa Version
copa version 0.5.1

Docker Version
Client:
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:28:49 2023
 OS/Arch:           darwin/amd64
 Context:           default

Server: Docker Desktop 4.25.2 (129061)
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:32:16 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Buildx Version
github.com/docker/buildx v0.11.2-desktop.5 f20ec1393426619870066baba9618cf999063886

Execution Error
copa patch --debug -i docker.io/bitnami/python:3.11.5 -r /tmp/py.json -t 3.11.5-patch
DEBU[0000] updates to apply: &{{{debian 11.7} {amd64}} [{curl 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u10 CVE-2023-38545} {curl 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27533} {curl 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27534} {curl 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27535} {curl 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27536} {curl 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27538} {curl 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u9 CVE-2023-28321} {curl 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u9 CVE-2023-28322} {curl 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u10 CVE-2023-38546} {libc-bin 2.31-13+deb11u6 2.31-13+deb11u7 CVE-2023-4911} {libc-dev-bin 2.31-13+deb11u6 2.31-13+deb11u7 CVE-2023-4911} {libc6 2.31-13+deb11u6 2.31-13+deb11u7 CVE-2023-4911} {libc6-dev 2.31-13+deb11u6 2.31-13+deb11u7 CVE-2023-4911} {libcurl3-gnutls 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u10 CVE-2023-38545} {libcurl3-gnutls 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27533} {libcurl3-gnutls 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27534} {libcurl3-gnutls 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27535} {libcurl3-gnutls 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27536} {libcurl3-gnutls 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27538} {libcurl3-gnutls 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u9 CVE-2023-28321} {libcurl3-gnutls 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u9 CVE-2023-28322} {libcurl3-gnutls 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u10 CVE-2023-38546} {libcurl4 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u10 CVE-2023-38545} {libcurl4 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27533} {libcurl4 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27534} {libcurl4 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27535} {libcurl4 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27536} {libcurl4 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u8 CVE-2023-27538} {libcurl4 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u9 CVE-2023-28321} {libcurl4 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u9 CVE-2023-28322} {libcurl4 7.74.0-1.3+deb11u7 7.74.0-1.3+deb11u10 CVE-2023-38546} {libgssapi-krb5-2 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libk5crypto3 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libkrb5-3 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libkrb5support0 1.18.3-6+deb11u3 1.18.3-6+deb11u4 CVE-2023-36054} {libncurses6 6.2+20201114-2+deb11u1 6.2+20201114-2+deb11u2 CVE-2023-29491} {libncursesw6 6.2+20201114-2+deb11u1 6.2+20201114-2+deb11u2 CVE-2023-29491} {libnghttp2-14 1.43.0-1 1.43.0-1+deb11u1 CVE-2023-44487} {libssl-dev 1.1.1n-0+deb11u5 1.1.1v-0~deb11u1 CVE-2023-3446} {libssl-dev 1.1.1n-0+deb11u5 1.1.1v-0~deb11u1 CVE-2023-3817} {libssl1.1 1.1.1n-0+deb11u5 1.1.1v-0~deb11u1 CVE-2023-3446} {libssl1.1 1.1.1n-0+deb11u5 1.1.1v-0~deb11u1 CVE-2023-3817} {libtinfo6 6.2+20201114-2+deb11u1 6.2+20201114-2+deb11u2 CVE-2023-29491} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-45871} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-1989} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-4244} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-42753} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-4622} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-4623} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-4921} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-6176} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-20588} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-37453} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-3772} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-3773} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-39189} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-39192} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-39193} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-39194} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-42754} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-42755} {linux-libc-dev 5.10.191-1 5.10.197-1 CVE-2023-42756} {ncurses-base 6.2+20201114-2+deb11u1 6.2+20201114-2+deb11u2 CVE-2023-29491} {openssl 1.1.1n-0+deb11u5 1.1.1v-0~deb11u1 CVE-2023-3446} {openssl 1.1.1n-0+deb11u5 1.1.1v-0~deb11u1 CVE-2023-3817}]}
DEBU[0000] Trying docker driver
DEBU[0000] serving grpc connection
DEBU[0000] stopping session
DEBU[0000] Could not use docker driver                   error="failed to solve: requested experimental feature mergeop  has been disabled on the build server: only enabled with containerd image store backend\nrequested experimental feature diffop  has been disabled on the build server: only enabled with containerd image store backend\nmissing required buildkit functionality"
DEBU[0000] Trying buildx driver
DEBU[0000] Could not use buildx driver                   error="listing workers for Build: failed to list workers: Unavailable: connection error: desc = \"transport: Error while dialing: read /Users/jlim/.docker/buildx/instances: is a directory\""
DEBU[0000] Trying default buildkit addr
DEBU[0000] Could not use buildkitd driver                error="<nil>"
WARN[0000] --debug specified, working folder at /var/folders/yy/k0b7rymj0235v51zmhd5qwt40000gp/T/copa-1804197243 needs to be manually cleaned up
Error: could not use docker driver: failed to solve: requested experimental feature mergeop  has been disabled on the build server: only enabled with containerd image store backend
requested experimental feature diffop  has been disabled on the build server: only enabled with containerd image store backend
missing required buildkit functionality
could not use buildx driver: listing workers for Build: failed to list workers: Unavailable: connection error: desc = "transport: Error while dialing: read /Users/jlim/.docker/buildx/instances: is a directory"
could not use buildkitd driver: %!w(<nil>)

@josephlim75 do you have a buildx builder running already? you can list with docker buildx ls
since you are already on docker v24, alternatively you can also enable containerd image store (containerd snapshotter)
please see https://project-copacetic.github.io/copacetic/website/quick-start#buildkit-connection-examples for usage

@sozercan , thanks.  It works now after specifying the --addr endpoint to point to the buildx instance.

repo: project-copacetic/copacetic


text:
fix(ci): Do not initialize image trust store on every convey

Should miticage failures such as https://github.com/project-zot/zot/actions/runs/6148316168/job/16681757801?pr=1788
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Closing this as nobody has experienced that failure since Andreea's fix.

repo: project-zot/zot


text:
How can an externel router peer with calico node route reflector

Hey guys, I'm glad to see calico node can be configured to act as a route reflector for other calico nodes in calico 3.3.
But when I set external router peer with calico node rr, I got error:
bird: BGP: Unexpected connect from unknown address xxxx (port xxxx)
My calico version is v3.3.2.

@caseydavenport I met the same problem, How did you resolve?

@shenshouer I closed this one due to inactivity and no clear reproduction scenario.
Could you please raise a new issue describing what you are seeing?

authors: adamswanglin caseydavenport shenshouer

repo: projectcalico/calico


text:
Manifests directory is 2 years old :)

Expected behaviour
modulo ocp... We probably should add notes to the manifests/ directory to clarify that they're not published anymore ?  or maybe just delete if from head.

Are you judging by the fact that some of the files in the repo under manifests/ocp haven't changed in the last 2 years?
Those files are linked to other files that do get updated and when the docs are published, those file are being generated with updated content.
If I'm misunderstanding please clarify what files you believe aren't being updated.


text:
calico.yaml cannot be downloaded

calico.yaml cannot be downloaded using URL https://docs.projectcalico.org/manifests/calico.yaml
Request URL: https://docs.projectcalico.org/manifests/calico.yaml
Request Method: GET
Status Code: 404
Remote Address: 35.198.80.163:443
Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

Context

Your Environment

Calico version
Orchestrator version (e.g. kubernetes, mesos, rkt):
Operating System and version:
Link to your project (optional):

In Calico v3.24, manifest links in the docs are now just raw links to the manifests in the Calico repo.
For the manifest you want, try: https://raw.githubusercontent.com/projectcalico/calico/v3.24.0/manifests/calico.yaml
That said, I'd still expect https://docs.projectcalico.org/manifests/calico.yaml to point to the latest release version of that manifest.

thx for link, yeah, I've also just found that I can use something like that https://raw.githubusercontent.com/projectcalico/calico/master/manifests/calico.yaml, but it will be better to have old link in working state, because a lot of forums and some training materials are pointing to exactly that link. JFYI yesterday https://docs.projectcalico.org/manifests/calico.yaml worked fine.

Perfect! It works now! Thanks!

Hey @olekszhel , sorry for the disruption. We had some docs site refactoring and it looks like we missed a couple things during those changes. I just pushed out a fix and I think that this should be working now.

authors: lmm mgleung olekszhel


text:
unknown field "spec.allowedEgressDomains"

I deployed tigera-operator using ArgoCD:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: tigera-operator
  namespace: argocd
spec:
  project: third-party
  source:
    repoURL: 'https://docs.projectcalico.org/charts/'
    targetRevision: 3.26.3
    helm:
      values: |
        installation:
          # Configures Calico networking.
          calicoNetwork:
            # Note: The ipPools section cannot be modified post-install.
            ipPools:
            - blockSize: 24
              cidr: 100.64.0.0/16
              disableBGPExport : false
              encapsulation: VXLANCrossSubnet
              natOutgoing: Enabled
              nodeSelector: all()
    chart: tigera-operator
  destination:
    server: 'https://kubernetes.default.svc'
    namespace: tigera-operator
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - PruneLast=true
      - CreateNamespace=true
      - ApplyOutOfSyncOnly=true
      - ServerSideApply=true

I want to create NetworkSet rule:
kind: NetworkSet
apiVersion: projectcalico.org/v3
metadata:
  name: google
  labels:
    destinations: google
  namespace: hipstershop
spec:
  nets: []
  allowedEgressDomains:
    - clouddebugger.googleapis.com
    - cloudtrace.googleapis.com
    - metadata.google.internal
    - monitoring.googleapis.com

Got this error:
Error from server (BadRequest): error when creating "GlobalNetworkSet.yaml": NetworkSet in version "v3" cannot be handled as a NetworkSet: strict decoding error: unknown field "spec.allowedEgressDomains"

also got similar error
Error from server (BadRequest): error when creating "GlobalNetworkSet.yaml": GlobalNetworkPolicy in version "v3" cannot be handled as a GlobalNetworkPolicy: strict decoding error: unknown field "spec.tier"

for this manifest
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: application.allow-egress-internet
spec:
  tier: application
  order: 0
  selector: all()
  egress:
    - action: Allow
      source: {}
      destination:
        selector: destinations == "internet"
  types:
    - Egress

same with the calico binary
kubectl calico create -f GlobalNetworkSet.yaml
Failed to execute command: error parsing document: field in document is not recognized or is in the wrong location: allowedEgressDomains
Failed to execute command: error parsing document: field in document is not recognized or is in the wrong location: tier

Hi @medoedoff - those fields are only supported by Calico Enterprise or Calico Cloud - if you look at docs.tigera.io and go to the Calico Open Source section you can see the available fields: https://docs.tigera.io/calico/latest/reference/resources/networkset - you can also look at the Cloud / Enterprise docs to review the features available in those products.

authors: matthewdupre medoedoff


text:
Add icon to links

Add blue shield icon to Kubernetes network policy and Calico network policy.

Approved by Alex P.
Can merge now
No back porting required

Deploy preview for calico ready!
Built with commit 222b43257322b76942695ff5062f3b41f4a9aa94
https://deploy-preview-3078--calico.netlify.com


text:
Move info on contributing to docs out of toplevel readme

I remove the part on "finding docs-archive". Figured enough time has passed that we can get rid of that.

ping @caseydavenport , I rebased this and applied your suggestions.


text:
Ensure Ports in DesiredEnvoyService() Func are as Expected

Ensure the ports provided to the DesiredEnvoyService() are as expected. In Gateway test cases where Contour is updated with a bad port name, the operator panics:
2021-03-11T10:57:42.289-0800	INFO	gateway_controller	ensured contour service for contour	{"namespace": "contour-operator", "name": "contour-gateway-sample"}
E0311 10:57:42.289141   90885 runtime.go:78] Observed a panic: runtime.boundsError{x:1, y:1, signed:true, code:0x0} (runtime error: index out of range [1] with length 1)
goroutine 183 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x239efc0, 0xc000bae7e0)
	/Users/daneyonhansen/code/go/src/github.com/projectcontour/contour-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0x95
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/Users/daneyonhansen/code/go/src/github.com/projectcontour/contour-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x89
panic(0x239efc0, 0xc000bae7e0)
	/usr/local/Cellar/go/1.15.4/libexec/src/runtime/panic.go:969 +0x1b9

This is due to a race condition where the gatewayclass is considered valid (status not updated yet), so the gateway controller ensures the Contour's child resources.

Closing since Gateway API support is being migrated to contour.

repo: projectcontour/contour-operator


text:
Fix deprecation warning

http://guides.rubyonrails.org/upgrading_ruby_on_rails.html#error-handling-in-transaction-callbacks
https://github.com/projectjellyfish/api/issues/521

:+1:

authors: chriskacerguis masonforest

repo: projectjellyfish/api


text:
feat: add ipmi_sel_events_time

Add type in ipmi_sel_events_count_by_state - so it can be distinguish between sel events
ipmi_sel_events_count_by_state{type="Power Supply",state="Critical"} 18

Add new metric about more detailed resent time occurrence event in log, so it can be stored in prometheus even after sel log was cleared.
It's not 1-1 sel log event copy, so it won't be with big cardinality. 18 real event are presented as 4 metrics with resent time occurrence
ipmi_sel_events_time{event="Power Supply Failure detected",name="Power Supply 2 Status",state="Critical",type="Power Supply"} 1.731064451e+09
ipmi_sel_events_time{event="Power Supply Failure detected ; Fan Fault",name="Power Supply 2 Status",state="Critical",type="Power Supply"} 1.731064449e+09
ipmi_sel_events_time{event="Power Supply input lost (AC/DC)",name="Power Supply 2 Status",state="Critical",type="Power Supply"} 1.727789819e+09
ipmi_sel_events_time{event="Redundancy Lost",name="System Board PS Redundancy",state="Critical",type="Power Supply"} 1.731064452e+09

Original sel-log for example above:
4   | Sep-10-2024 | 16:14:38 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected ; Fan Fault
5   | Sep-10-2024 | 16:14:38 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected
7   | Sep-10-2024 | 16:14:42 | System Board PS Redundancy       | Power Supply                | Critical | Redundancy Lost
8   | Oct-01-2024 | 09:26:41 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply input lost (AC/DC)
9   | Oct-01-2024 | 09:26:50 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected ; Fan Fault
10  | Oct-01-2024 | 09:26:50 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected
14  | Oct-01-2024 | 11:51:45 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected ; Fan Fault
15  | Oct-01-2024 | 11:51:47 | System Board PS Redundancy       | Power Supply                | Critical | Redundancy Lost
16  | Oct-01-2024 | 11:51:47 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected
18  | Oct-01-2024 | 13:36:59 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply input lost (AC/DC)
21  | Oct-01-2024 | 13:37:58 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected ; Fan Fault
23  | Oct-01-2024 | 13:37:59 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected
26  | Oct-07-2024 | 19:24:08 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected ; Fan Fault
28  | Oct-07-2024 | 19:24:11 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected
29  | Oct-07-2024 | 19:24:12 | System Board PS Redundancy       | Power Supply                | Critical | Redundancy Lost
31  | Nov-08-2024 | 11:14:09 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected ; Fan Fault
32  | Nov-08-2024 | 11:14:11 | Power Supply 2 Status            | Power Supply                | Critical | Power Supply Failure detected
33  | Nov-08-2024 | 11:14:12 | System Board PS Redundancy       | Power Supply                | Critical | Redundancy Lost

Hello, can you check the PR?
@bitfehler @SuperQ

@RichiH maybe you can check this PR?

license: MIT


text:
Sync up with upstream dependencies

Many of the dependencies are out of date. Ran a jb update and build to pull in the latest versions.

Nice! Thank you!

authors: brancz jessebye

repo: prometheus-operator/kube-prometheus


text:
Update prometheus-rules.yaml

fix HPAMaxedOut alert. If max and min hpa values are equal, it shouldn't fire alert or warning

thx @paulfantom, I will fix it in that project.

thx @paulfantom, I will fix it in that project.

Closing as this is already included.

authors: paulfantom utkusonmez


text:
How to change the default url link from fired alert from hostname to IP address

Hello,
I have setup alertmanager & able to receive slack messages for alert conditions:
It's not consistent on how alerts are received. I'm not getting alerts most of the times.
`receivers:

name: 'alertmanager_alerts'
slack_configs:

send_resolved: true
api_url: 'hook_url'
channel: '#alerts'
text: "<!channel> \ndescription: {{ .CommonAnnotations.description }}\nsummary: {{ .CommonAnnotations.summary }}"
`
I want to specify the slack message url from :9093#/alerts?receiver=alertmanager_alerts to IP address. Since I don't have DNS enable so where & how to specify IP address instead of hostname.

Also in alert manager UI the source link for an alert points to hostname:9090 prometheus instance. Is it possible to change that to IPAddress:9090?

@sushovan23 I don't understand your questions. It would be helpful if you could add your full Alertmanager configuration and describe your issue in more depth.

I want to specify the slack message url from :9093#/alerts?receiver=alertmanager_alerts to IP address. Since I don't have DNS enable so where & how to specify IP address instead of hostname.

Where do you want to specify the slack message url? In the UI to filter by receiver?

Also in alert manager UI the source link for an alert points to hostname:9090 prometheus instance. Is it possible to change that to IPAddress:9090?

The Alertmanager UI uses whatever is written in the generatorURL to redirect to the source of an alert. Changing from hostname to IP would need to be done in Prometheus.

Hello,
Thanks for the response. I read more on generatorURL & figured out that I needed to use -web.external-url option while starting prometheus which would provide accessible url in slack notifications & alert manager UI.

Hello, Thanks for the response. I read more on generatorURL & figured out that I needed to use -web.external-url option while starting prometheus which would provide accessible url in slack notifications & alert manager UI.

Here is an example docker-compose for alertmanager-
https://stackoverflow.com/questions/47885588/how-to-set-prometheus-alertmanager-external-url-via-configuration
Similar command item can used for prom as well

authors: mannharleen mxinden sushovan23

repo: prometheus/alertmanager


text:
Fix NOTE block rendering

Fixes: https://github.com/prometheus/docs/issues/1986
Signed-off-by: Richard Hartmann richih@richih.org

@juliusv I was uncertain if I should split out .alert into its own section or not. Preferences?

:+1: Though there may be more block-style elements that could cause a similar problem if they occur at the top of the page.

authors: RichiH juliusv

repo: prometheus/docs


text:
performance improvement idea: drop metric names later

Currently, many simple functions such as abs() call DropMetricName() on every sample.
https://github.com/prometheus/prometheus/blob/0906f2eafa3506d411f2053ecba160a6c5ca0387/promql/functions.go#L536-L539
It would be more efficient to leave till the end and do once per series.  DropMetricName is hashed, so the additional work on every step of a range query is not huge, but we could eliminate that work and the hash table.
This is not a trivial change: there are about four different ways that functions get called and it took me a while just to follow all the logic.  I do think it should be possible to refactor this area more simply and understandably.
For functions that take a vector, any metric name in the output is ignored!
https://github.com/prometheus/prometheus/blob/0906f2eafa3506d411f2053ecba160a6c5ca0387/promql/engine.go#L1400-L1405

Some discussion about not dropping the metric name: #11397

With #14477 in, one might argue we are now doing what's being asked for here.
However, the new behavior is behind a feature flag, and if it is not set, we do the same thing as before (if I have read the code correctly).
It might very well be that the delayed name removal turns out to be a mistake, and then we are back to square one.
I guess we could still do the name removal late (as done in #14477) but make sure that logically things still work as if the name is not removed later (e.g. error out if in between there are duplicate label sets not considering the name; make it impossible to label-replace the name label in between, …).
I hope that #14477 will be the way to go for the future (although it creates some problems when dissecting a query and explore it in parts, may it be manually or with Promlens), but hope is not a strategy…
@bboreham your call if you want to leave this open.

Hello from the bug scrub.
@bboreham we decided to close this for now, but please re-open if you want to keep this in the loop.

repo: prometheus/prometheus


text:
Enable handling of overlapping blocks by default

Prometheus has had the support for handling overlapping blocks (both during compaction and querying) for a very long time, and it has been used by Thanos, Cortex, Mimir at large scale.
I propose we enable them by default now and not hard fail on finding overlapping blocks. I also propose we still have the switch to disable compaction of overlapping blocks because it is useful and required by downstream consumers with out-of-order feature (what that I mean Mimir, but will also be required by Cortex when they support out-of-order ingestion).
cc @bwplotka

Yes, I like it. It proven to work pretty well for us too.

In which case are overlapping blocks not a bug in regular Prometheus?

Also I think it helps in ease of use when working with out of order and backfilling, you don't have to do an extra step of enabling overlapping blocks (while also having to remember that overlapping blocks need to be enabled).

We log a warning when overlapping blocks are found, so if we come across it, we can see what was the reason then instead of stopping Prometheus from functioning.

If its that much rare and the codebase can already handle querying from multiple overlapping blocks I think thats another reason to simply enable overlapping blocks by default :+1:

authors: bwplotka codesome jesusvazquez roidelapluie


text:
Recording Rules take more time in Prometheus 2.3.0 after upgrade

Proposal
Use case. Why is this important?
Prometheus recording rules should not take more time
Bug Report
What did you do?
Upgraded Prometheus 2.2.1 to 2.3.0. The same set of recording rules were taking less time with previous version.
What did you expect to see?
Alarms based on metrics (recording rule evaluation) should not have gotten fired.
What did you see instead? Under which circumstances?
We are observing alarms firing.
Environment

System information:
Linux 4.1.12-124.19.7.el7uek.x86_64 x86_64

Prometheus version:
prometheus, version 2.3.0 (branch: heads/v2.3.0, revision:
290d71791a507a5057b9a099c9d48703d86dc941)
build user:       teamcity@tca-hv05-agent018-default
build date:       20181030-04:34:03
go version:       go1.11.1

Can you share the recording rules and the definition of the firing alerts?

We've looked at this as part of our bug scrub, and lacking information we're presuming this is no longer an issue as we're up to 2.17.1. If you've more information, please let us know.

authors: brian-brazil dhbarman simonpasquier


text:
Flush Head chunks to disk

The idea is to flush the head chunks to disk as soon as it is created and use it for the following purposes

mmap the flushed chunks in the Head instead of holding it in the memory, hence reducing the memory footprint of Head
Consider those chunks as checkpoints and process WAL accordingly, hence reducing the WAL replay time heavily

I had attempted checkpointing the chunks and replaying it here, but the idea has evolved since then (as described above).
I will come up with a design doc to comment on in the coming weeks.
/cc @brancz @krasi-georgiev

The completed chunks aren't sufficient as a checkpoint, you'd also need to checkpoint all non-completed chunks. You can then replay the WAL from only that checkpoint - after loading in all the completed chunks.

Yes, in my previous attempt I was writing all the chunks (complete and incomplete). But in this case (initial idea), where we write out the chunks in real-time when they are complete (and not like a checkpoint exactly), writing incomplete chunks would be rather expensive to mmap (but can be done only for the replay purpose). I will keep this in mind when working on the design doc, thanks!

And we'd still have the complete chunks via mmap hopefully, which would make them off-heap and save RAM.

Yes, that's the plan

I have jotted down some high-level ideas in this doc https://docs.google.com/document/d/184urkLQnM7rqLmGvS66I15jU2pyPk_qwd8v_qDlXczo/edit?usp=sharing

authors: brian-brazil codesome


text:
Read remote influxdb empty data,No datapoints found. But start by manual  command no problem

use command start prometheus work on centos 7 , I can see data in UI . command:
 /opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml --storage.tsdb.retention=60d --web.console.libraries=/opt/prometheus/console_libraries --web.console.templates=/opt/prometheus/consoles --web.enable-lifecycle --web.enable-admin-api

But use systemctl start, can not see data in UI , systemctl config  :
[root@HZ-11859-6:system]11:30:18# cat /etc/systemd/system/prometheus.service

[Unit]
Description=Prometheus Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network-online.target

[Service]
User=root
#Restart=on-failure
Type=simple
ExecStart= /opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml --storage.tsdb.retention=60d --web.console.libraries=/opt/prometheus/console_libraries --web.console.templates=/opt/prometheus/consoles --web.enable-lifecycle --web.enable-admin-api

[Install]
WantedBy=multi-user.target

prometheus config:
# my global config
global:
  # scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  - /opt/prometheus/rules/*.yaml

remote_write:
  - url: "http://localhost:9201/write"

remote_read:
  - url: "http://localhost:9201/read"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['localhost:9090']

remote_storage_adapter config:
[root@xxxx:system]11:03:49# cat remote_storage_adapter.service
[Unit]
Description=remote_storage_adapter
After=network-online.target

[Service]
Type=simple
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/opt/prometheus-adapt/remote_storage_adapter  --influxdb-url=http://172.21.219.98:8086/ --influxdb.database=aitutor_aggregation --influxdb.retention-policy=autogen
LimitNOFILE=65000

Restart=always

[Install]
WantedBy=multi-user.target

prometheus version:
[root@xxxx:system]11:05:39# /opt/prometheus/prometheus --version
prometheus, version 2.22.1 (branch: HEAD, revision: 00f16d1ac3a4c94561e5133b821d8e4d9ef78ec2)
  build user:       root@516b109b1732
  build date:       20201105-14:02:25
  go version:       go1.15.3
  platform:         linux/amd64

Effect picture， start by manual command ：

start by systemctl：

The same data source, the same configuration file，why different effects ？

add config `` in systemctl ,Can work ：
--storage.tsdb.path=/data/prometheus

why ?

It makes more sense to ask questions like this on the prometheus-users mailing list rather than in a GitHub issue. On the mailing list, more people are available to potentially respond to your question, and the whole community can benefit from the answers provided.


text:
kuma_sd does not report all targets

What did you do?
Use the new kuma_sd to discover scrape targets.
What did you expect to see?
5 targets.
What did you see instead? Under which circumstances?
Only 1 target, which changes when the discovery reports new targets.
It looks like they are just being overwritten, likely because of the "Source" of all the groups is "kuma", which is used as a key for targets in the manager.
We could solve this by namespacing group sources, i.e.: instead of "kuma", `"kuma/{mesh}/{dataplane}" which is unique.

Prometheus version: 2.29-rc.0

Prometheus configuration file:

global:
  evaluation_interval: 1m
  scrape_interval: 10s
  scrape_timeout: 10s
scrape_configs:
  - honor_labels: true
    job_name: 'kuma-dataplanes'
    kuma_sd_configs:
      - server: "http://localhost:5676"

cc @roidelapluie
I'll have a fix PR in shortly.

You will also need to send empty targetgroups for disappearing groups.

You will also need to send empty targetgroups for disappearing groups.

Sounds good, I'll keep a cache of target group source names and diff on update.

How do we get notified about deleted resources?

How do we get notified about deleted resources?

The xDS server sends the entire "state of the world" with every response, so missing resources indicate their deletion. When they are deleted, the xDS server also invalidates the cache and its version, so subsequent requests will be sent the full update.

authors: austince roidelapluie


text:
slog: various fixes

Contains a few fixes for things related to the slog-ification. Please see individual commits for details.
TL;DR -- fixes a regex in a test, fixes a data race, and removes some unused methods on the dedupe logger.

cc: @SuperQ

I need to play with this more, still seeing data races again now where I wasn't before :thinking:

Ok, this is ready for review :+1:


text:
tlsconfig: support for nextProtos

Builds successfully with local go.mod replace pointing to https://github.com/prometheus/common/pull/713
Unsure if there are additional e2e tests to run / update or consumers of the change in common above, this was the only one I could find.

Converting to draft as it depends on a custom revision of prometheus/common.


text:
Hide HTTP auth parts from instance label

This  introduces an extra function in the Target interface (PublicURL)
which is used to populate the instance field in metrics, instead of using the raw URL from the config.

Hey thanks! Btw., the problem of having to put username+password into the URL would already go away with the HTTPS+basic-auth change I was working on in https://github.com/prometheus/prometheus/issues/474 on this branch: https://github.com/prometheus/prometheus/commits/scheme-and-auth. That will allow configuring the credentials in the config once per job, outside of the URL. This will also enable basic auth for SD-discovered targets.
Anyways, I think this change looks good, but would make one small change to it: we already decided that we want the instance label just to contain the <host>:<port> in the future (because we assume the likelihood to be low to have two different instances on the same host/port combination with different schemes or endpoints), so how about changing your PublicURL method to only return <host>:<port>? /cc @brian-brazil Should be fine to introduce that change now, right?
We're also deciding how to integrate different types of service discovery now, which includes coming up with a generalized method for deriving instance/job/other labels (both from static configuration and from potential meta-information provided by the SD mechanism itself). That will change this code again, but I'm pretty sure that in the default case, it will still map <host>:<port> into the instance label.

Yes, changing to <host>:<port> sound good to me as well. I can also already introduce a way to override the instance label to anything the user might want, via the job config - or do you want to omit this for now and introduce this when SD naming generalization comes into play?

Yep, we definitely want to think about that latter part a bit more before introducing configuration surface for it (which we might need to break again later). Thanks!

Updated to use host:port. The function is called InstanceIdentifier(), as I didn't like how the proposed InstanceLabel() collided with the InstanceLabel const ("label").

Woot, thanks! Merging.


text:
Backfill from OpenMetrics format

Related PR : https://github.com/prometheus/prometheus/pull/7586
Design Document here

Expanding on ideas about the command structure we just brainstormed during the sync meeting:
As discussed, the command should not be import (and neither backfill, as in #7675) but something that is both more appropriate and applies to both. Our best idea so far is create-blocks-from, with the next sub-command telling what (e.g. openmetrics, rules). I think we should also try to stay as faithful as possible to existing flags and args conventions in the promtool commands. For example, promtool tsdb (dump|list|analyze) all use an argument (called "db path") to define where the TSDB files are. Here and in #7675, we use a flag.
Backfilling from rules: promtool tsdb create-blocks-from rules [<flags>] [<db path>]
Backfilling from OM file: promtool tsdb create-blocks-from openmetrics [<flags>] [<OM file>] [<db path>]
As for the other tsdb commands, the default value for <db path> is defaultDBPath, i.e. data/`.
The flag common to both sub-commands is block-length (hidden, default 2h).
The rules sub-command has the additional flags url (server to evaluate against), start, end, step, rule-file (ExistingFiles(), i.e. repeatable).
Since we want to allow multiple rule files for rule backfilling, but only one input file for backfilling from a file, we use a flag for the former but an argument for the latter.

I have resolved all the comments. Am currently writing unit tests.

@beorn7 @bwplotka @codesome I have made changes according to the older review and written a unit test. The unit test is just for one "valid" Openmetric string for now, am working on adding more. Please feel free to take another look/review at the PR.

Note: My review didn't include the last two commits (when I said "more tests needed").
Will take another look after rebasing.

@beorn7 @codesome @bwplotka Please feel free to review again.

@beorn7 @bwplotka @codesome Please feel free to review again.

I tried this out in practice.
Good news: it works.
Slightly bad news: It spams a lot of logs to stderr. I haven't dug down into it, but it appears that it scans all the pre-existing block metadata in the output directory each time a block writer is opened. That's in principle a bit wasteful, but probably not a big deal in practice. More annoying is that it dumps the block metadata to stderr. So for each block created, you get an increasingly longer list. In this example, I have created three blocks in an empty output dir:
$▶ ./promtool tsdb create-blocks-from openmetrics ~/tmp/test.om tmp
BLOCK ULID  MIN TIME  MAX TIME  DURATION  NUM SAMPLES  NUM CHUNKS  NUM SERIES  SIZE
BLOCK ULID                  MIN TIME                       MAX TIME                       DURATION  NUM SAMPLES  NUM CHUNKS  NUM SERIES  SIZE
01EQGM74YG8QG2FKEFSF068Z25  2020-11-19 07:39:17 +0000 UTC  2020-11-19 07:44:17 +0000 UTC  5m0.001s  4            1           1           521B
BLOCK ULID                  MIN TIME                       MAX TIME                       DURATION  NUM SAMPLES  NUM CHUNKS  NUM SERIES  SIZE
01EQGM74YG8QG2FKEFSF068Z25  2020-11-19 07:39:17 +0000 UTC  2020-11-19 07:44:17 +0000 UTC  5m0.001s  4            1           1           521B
BLOCK ULID                  MIN TIME                       MAX TIME                       DURATION  NUM SAMPLES  NUM CHUNKS  NUM SERIES  SIZE
01EQGM74YG8QG2FKEFSF068Z25  2020-11-19 07:39:17 +0000 UTC  2020-11-19 07:44:17 +0000 UTC  5m0.001s  4            1           1           521B
01EQGM7522CCBVCNYVM9XQ3E99  2020-11-19 10:25:57 +0000 UTC  2020-11-19 10:30:57 +0000 UTC  5m0.001s  4            1           1           521B

So you get just the headline, then one block data logged, then two.
If you create blocks in a pre-existing output dir, like into a live Prometheus, things get really confusing:
$▶ ./promtool tsdb create-blocks-from openmetrics ~/tmp/test.om data
BLOCK ULID                  MIN TIME                       MAX TIME                       DURATION  NUM SAMPLES  NUM CHUNKS  NUM SERIES  SIZE
01ENZS849NW0S3FVWS4VPQPYZF  2020-10-30 16:00:00 +0000 UTC  2020-10-31 02:00:00 +0000 UTC  10h0m0s   6390412      54279       489         5MiB570KiB556B
01EP254ZV71MH6CS6931Z0KS4K  2020-10-31 10:00:00 +0000 UTC  2020-11-01 00:00:00 +0000 UTC  14h0m0s   16426967     139854      489         14MiB54KiB424B
01EP30KWPG1NA6DKE2N8C894SR  2020-11-01 10:00:00 +0000 UTC  2020-11-01 18:00:00 +0000 UTC  8h0m0s    8606888      72861       489         11MiB826KiB45B
01EP4QHPM12FCWJB9SVMPB2PQ8  2020-11-01 18:00:00 +0000 UTC  2020-11-02 12:00:00 +0000 UTC  18h0m0s   12436232     105135      489         10MiB764KiB213B
01EP9CWDKHTH2F7R27HQ3Y88PW  2020-11-02 12:00:00 +0000 UTC  2020-11-02 20:00:00 +0000 UTC  8h0m0s    10903238     92421       489         11MiB184KiB869B
01EP9CWCZ9WJ6ZR0DA1ZAJQ532  2020-11-03 22:00:00 +0000 UTC  2020-11-04 00:00:00 +0000 UTC  2h0m0s    3032346      25910       489         2MiB701KiB638B
01EPAGYAWNBXNBQG0C2A21735M  2020-11-04 00:00:00 +0000 UTC  2020-11-04 18:00:00 +0000 UTC  18h0m0s   10448452     88998       489         9MiB47KiB714B
01EPCEQVZE0CCEAG12MAB2MXZ8  2020-11-04 18:00:00 +0000 UTC  2020-11-05 12:00:00 +0000 UTC  18h0m0s   15242611     129585      489         13MiB558KiB269B
01EPCEQVSTDK9DG8F9YZ821ACD  2020-11-05 12:00:00 +0000 UTC  2020-11-05 14:00:00 +0000 UTC  2h0m0s    3520800      29829       489         3MiB14KiB826B
01EQGM7R59W5A70D5R96P8B628  2020-11-05 14:00:00 +0000 UTC  2020-11-05 16:00:00 +0000 UTC  2h0m0s    2817134      23961       489         2MiB416KiB86B
BLOCK ULID                  MIN TIME                       MAX TIME                       DURATION  NUM SAMPLES  NUM CHUNKS  NUM SERIES  SIZE
01ENZS849NW0S3FVWS4VPQPYZF  2020-10-30 16:00:00 +0000 UTC  2020-10-31 02:00:00 +0000 UTC  10h0m0s   6390412      54279       489         5MiB570KiB556B
01EP254ZV71MH6CS6931Z0KS4K  2020-10-31 10:00:00 +0000 UTC  2020-11-01 00:00:00 +0000 UTC  14h0m0s   16426967     139854      489         14MiB54KiB424B
01EP30KWPG1NA6DKE2N8C894SR  2020-11-01 10:00:00 +0000 UTC  2020-11-01 18:00:00 +0000 UTC  8h0m0s    8606888      72861       489         11MiB826KiB45B
01EP4QHPM12FCWJB9SVMPB2PQ8  2020-11-01 18:00:00 +0000 UTC  2020-11-02 12:00:00 +0000 UTC  18h0m0s   12436232     105135      489         10MiB764KiB213B
01EP9CWDKHTH2F7R27HQ3Y88PW  2020-11-02 12:00:00 +0000 UTC  2020-11-02 20:00:00 +0000 UTC  8h0m0s    10903238     92421       489         11MiB184KiB869B
01EP9CWCZ9WJ6ZR0DA1ZAJQ532  2020-11-03 22:00:00 +0000 UTC  2020-11-04 00:00:00 +0000 UTC  2h0m0s    3032346      25910       489         2MiB701KiB638B
01EPAGYAWNBXNBQG0C2A21735M  2020-11-04 00:00:00 +0000 UTC  2020-11-04 18:00:00 +0000 UTC  18h0m0s   10448452     88998       489         9MiB47KiB714B
01EPCEQVZE0CCEAG12MAB2MXZ8  2020-11-04 18:00:00 +0000 UTC  2020-11-05 12:00:00 +0000 UTC  18h0m0s   15242611     129585      489         13MiB558KiB269B
01EPCEQVSTDK9DG8F9YZ821ACD  2020-11-05 12:00:00 +0000 UTC  2020-11-05 14:00:00 +0000 UTC  2h0m0s    3520800      29829       489         3MiB14KiB826B
01EQGM7R59W5A70D5R96P8B628  2020-11-05 14:00:00 +0000 UTC  2020-11-05 16:00:00 +0000 UTC  2h0m0s    2817134      23961       489         2MiB416KiB86B
01EQGM7XQCA49K4CJ2KCRP72Y4  2020-11-19 07:39:17 +0000 UTC  2020-11-19 07:44:17 +0000 UTC  5m0.001s  4            1           1           521B
BLOCK ULID                  MIN TIME                       MAX TIME                       DURATION  NUM SAMPLES  NUM CHUNKS  NUM SERIES  SIZE
01ENZS849NW0S3FVWS4VPQPYZF  2020-10-30 16:00:00 +0000 UTC  2020-10-31 02:00:00 +0000 UTC  10h0m0s   6390412      54279       489         5MiB570KiB556B
01EP30KWPG1NA6DKE2N8C894SR  2020-11-01 10:00:00 +0000 UTC  2020-11-01 18:00:00 +0000 UTC  8h0m0s    8606888      72861       489         11MiB826KiB45B
01EP4QHPM12FCWJB9SVMPB2PQ8  2020-11-01 18:00:00 +0000 UTC  2020-11-02 12:00:00 +0000 UTC  18h0m0s   12436232     105135      489         10MiB764KiB213B
01EP9CWCZ9WJ6ZR0DA1ZAJQ532  2020-11-03 22:00:00 +0000 UTC  2020-11-04 00:00:00 +0000 UTC  2h0m0s    3032346      25910       489         2MiB701KiB638B
01EPAGYAWNBXNBQG0C2A21735M  2020-11-04 00:00:00 +0000 UTC  2020-11-04 18:00:00 +0000 UTC  18h0m0s   10448452     88998       489         9MiB47KiB714B
01EPCEQVZE0CCEAG12MAB2MXZ8  2020-11-04 18:00:00 +0000 UTC  2020-11-05 12:00:00 +0000 UTC  18h0m0s   15242611     129585      489         13MiB558KiB269B
01EPCEQVSTDK9DG8F9YZ821ACD  2020-11-05 12:00:00 +0000 UTC  2020-11-05 14:00:00 +0000 UTC  2h0m0s    3520800      29829       489         3MiB14KiB826B
01EQGM7R59W5A70D5R96P8B628  2020-11-05 14:00:00 +0000 UTC  2020-11-05 16:00:00 +0000 UTC  2h0m0s    2817134      23961       489         2MiB416KiB86B
01EQGM7XQCA49K4CJ2KCRP72Y4  2020-11-19 07:39:17 +0000 UTC  2020-11-19 07:44:17 +0000 UTC  5m0.001s  4            1           1           521B
BLOCK ULID                  MIN TIME                       MAX TIME                       DURATION  NUM SAMPLES  NUM CHUNKS  NUM SERIES  SIZE
01EPAGYAWNBXNBQG0C2A21735M  2020-11-04 00:00:00 +0000 UTC  2020-11-04 18:00:00 +0000 UTC  18h0m0s   10448452     88998       489         9MiB47KiB714B
01EPCEQVZE0CCEAG12MAB2MXZ8  2020-11-04 18:00:00 +0000 UTC  2020-11-05 12:00:00 +0000 UTC  18h0m0s   15242611     129585      489         13MiB558KiB269B
01EPCEQVSTDK9DG8F9YZ821ACD  2020-11-05 12:00:00 +0000 UTC  2020-11-05 14:00:00 +0000 UTC  2h0m0s    3520800      29829       489         3MiB14KiB826B
01EQGM7R59W5A70D5R96P8B628  2020-11-05 14:00:00 +0000 UTC  2020-11-05 16:00:00 +0000 UTC  2h0m0s    2817134      23961       489         2MiB416KiB86B
01EQGM7XQCA49K4CJ2KCRP72Y4  2020-11-19 07:39:17 +0000 UTC  2020-11-19 07:44:17 +0000 UTC  5m0.001s  4            1           1           521B
01EQGM7XV46K6QVQMG9T8V2PBE  2020-11-19 10:25:57 +0000 UTC  2020-11-19 10:30:57 +0000 UTC  5m0.001s  4            1           1           521B

The simplest would be to somehow intercept this logging.
On the other hand, getting information about the blocks created would be neat. But please exactly once, and only for newly created blocks. (Right now, we get no logging at all about the last block created, and all the other blocks in increasing number of times.)

@beorn7 @codesome Printing the block metadata is left todo, will not be able to work on it today. Have resolved other comments except that and this https://github.com/prometheus/prometheus/pull/8084#discussion_r528346019 . Please feel free to review again.

@bwplotka hey there! Im trying to use what you described here: https://grafana.com/blog/2020/09/02/how-were-improving-backfill-methods-to-get-older-data-into-prometheus/ however I wonder what would be the way to import csv nowadays given all these pull request merged. I did noticed this: https://github.com/prometheus/prometheus/blob/main/docs/storage.md but can't find an explicit example of importing a csv file as it was described in your talk/demo session.
Thanks a lot !!

There is no direct way of importing metrics data from CSV yet. As it turned out, CSV is really not very well suited for metrics data, which makes it hard to come up with a good generally useful approach. Having said that, it's still on the agenda, we just didn't do it as the first thing.
The linked design doc has a bunch of concerns about CSV in the discussion. Contributions are welcome, but best to be discussed via the community channels first. (Questions and discussions on a closed GH PR easily get lost.)


text:
Allow specifying content for new files via url query params

I'd like to create a bookmarklet or Chrome extensions to make 1-click publishing for Jekyll blogs easy. If I could just point a request to prose.io/#{account}/{repo}/new/master/_posts?content={content}&metadata={metadata} , this would be awesome. Digging around, I'm not sure it's even possible without support in app, as it would require injecting code that interfaces with CodeMirror.
Thoughts?

Since #437 was close as wontfix, recommending to close this issue too

authors: agius fulldecent

repo: prose/prose