text:
Error: Warning stuck buffer_pool buffer with IO workload

Environment info

NooBaa Version: VERSION
Platform: Kubernetes 1.14.1 | minikube 1.1.1 | OpenShift 4.1 | other: specify

noobaa-20241104 (5.17.1) - standalone noobaa
Actual behavior

Ran the Warp IO workload of "get"

./warp get --insecure --duration 60m --host .com:6443 --access-key KCxP4AN9937kVqoCrNIs --secret-key bIdwF/5nJtSnrHWXrhPOhkv1WqGjtayMk6D+aU/U --tls --obj.size 256M --bucket warp-get-bucket-reg  2>&1| tee /tmp/warp-get-11nov2024.log
observed following in the log  (system running concurrently long versioning test as well in other directory)
Nov 11 03:54:17 node-gui0 [3532906]: [nsfs/3532906] [ERROR] CONSOLE:: Error: Warning stuck buffer_pool buffer    at BuffersPool.get_buffer (/usr/local/noobaa-core/src/util/buffer_utils.js:218:25)    at async NamespaceFS.read_object_stream (/usr/local/noobaa-core/src/sdk/namespace_fs.js:1080:46)    at async NsfsObjectSDK._call_op_and_update_stats (/usr/local/noobaa-core/src/sdk/object_sdk.js:543:27)    at async Object.get_object [as handler] (/usr/local/noobaa-core/src/endpoint/s3/ops/s3_get_object.js:116:25)    at async handle_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:161:19)    at async Object.s3_rest [as handler] (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:66:9)
grep "Error: Warning stuck buffer_pool buffer" noobaa.log | grep "03:54" |wc -l
30

No errors on the client node is observed. I am saying around 03:54 , because the GPFS daemon has started back on the 1 node (out of 2 node protocol node) , where the RR-DNS is configured the IO continued to run when the HA happened previously. So this above message is nothing related to HA (will attach the logs)

Warp analyze  --> ignore 10 errors of "read tcp" as this is expected since the HA is happening (i.e gpfs is starting)

./warp analyze --analyze.op GET --analyze.v  warp-get-2024-11-11[030028]-5qrv.csv.zst --debug
7550 operations loaded... Done!

----------------------------------------
Operation: GET (5050). Ran 1h0m4s. Size: 256000000 bytes. Concurrency: 20.
Errors: 15
First Errors:
 * https://<cesip>:6443/, 2024-11-11 03:52:28 -0500 EST: read tcp 9.42.124.113:59180->9.42.93.99:6443: read: connection reset by peer
 * https://<cesip>:6443/, 2024-11-11 03:52:28 -0500 EST: read tcp 9.42.124.113:59164->9.42.93.99:6443: read: connection reset by peer
 * https://<cesip>:6443/, 2024-11-11 03:52:28 -0500 EST: read tcp 9.42.124.113:59270->9.42.93.99:6443: read: connection reset by peer
 * https://<cesip>:6443/, 2024-11-11 03:52:28 -0500 EST: read tcp 9.42.124.113:59244->9.42.93.99:6443: read: connection reset by peer
 * https://<cesip>:6443/, 2024-11-11 03:52:28 -0500 EST: read tcp 9.42.124.113:59256->9.42.93.99:6443: read: connection reset by peer
 * https://<cesip>:6443/, 2024-11-11 03:52:28 -0500 EST: read tcp 9.42.124.113:59258->9.42.93.99:6443: read: connection reset by peer
 * https://<cesip>:6443/, 2024-11-11 03:52:28 -0500 EST: read tcp 9.42.124.113:59178->9.42.93.99:6443: read: connection reset by peer
 * https://<cesip>:6443/, 2024-11-11 03:52:28 -0500 EST: read tcp 9.42.124.113:59166->9.42.93.99:6443: read: connection reset by peer
 * https://<cesip>:6443/, 2024-11-11 03:52:28 -0500 EST: read tcp 9.42.124.113:59222->9.42.93.99:6443: read: connection reset by peer
 * https://<cesip>:6443/, 2024-11-11 03:52:28 -0500 EST: read tcp 9.42.124.113:59234->9.42.93.99:6443: read: connection reset by peer

Requests considered: 5014:
 * Avg: 14.302s, 50%: 17.271s, 90%: 21.121s, 99%: 23.583s, Fastest: 1.486s, Slowest: 1m54.959s, StdDev: 9.26s
 * TTFB: Avg: 141ms, Best: 11ms, 25th: 55ms, Median: 73ms, 75th: 93ms, 90th: 118ms, 99th: 216ms, Worst: 1m35.183s StdDev: 2.096s
 * First Access: Avg: 17.059s, 50%: 19.136s, 90%: 21.355s, 99%: 23.466s, Fastest: 1.486s, Slowest: 1m52.859s, StdDev: 8.407s
 * First Access TTFB: Avg: 152ms, Best: 21ms, 25th: 59ms, Median: 78ms, 75th: 101ms, 90th: 130ms, 99th: 335ms, Worst: 1m33.717s StdDev: 2.18s
 * Last Access: Avg: 10.997s, 50%: 7.742s, 90%: 20.381s, 99%: 22.633s, Fastest: 1.486s, Slowest: 1m52.228s, StdDev: 6.788s
 * Last Access TTFB: Avg: 91ms, Best: 16ms, 25th: 52ms, Median: 69ms, 75th: 86ms, 90th: 108ms, 99th: 192ms, Worst: 36.283s StdDev: 782ms

Throughput:
* Average: 340.77 MiB/s, 1.40 obj/s

Throughput, split into 239 x 15s:
 * Fastest: 678.2MiB/s, 2.78 obj/s (15s, starting 03:57:01 EST)
 * 50% Median: 248.9MiB/s, 1.02 obj/s (15s, starting 03:26:16 EST)
 * Slowest: 43.6MiB/s, 0.18 obj/s (15s, starting 03:14:16 EST)

Default endpoint forks in the system with 2 CES S3 nodes having 1 CES IP each assigned
Expected behavior

Are we expected to get these ERRORS , as posted above ?
CONSOLE:: Error: Warning stuck buffer_pool buffer    at BuffersPool.get_buffer
Steps to reproduce

Run warp as shown below and it occurred on a system where it had medium workload i can say
More information - Screenshots / Logs / Other output
will update once the logs are uploaded https://ibm.ent.box.com/folder/293508364523

@shirady can you please take a look ?

Hi @rkomandu ,
Could you please reproduce and provide logs with a higher debug level?
Additional Details
At this moment I'm looking for a level-1 printing from here:
https://github.com/noobaa/noobaa-core/blob/e1bf29e82128a9e0d359b262978d208ccff228e5/src/util/buffer_utils.js#L205
As you can see from the error stuck:

[ERROR] CONSOLE:: Error: Warning stuck buffer_pool buffer    at BuffersPool.get_buffer (/usr/local/noobaa-core/src/util/buffer_utils.js:218:25)    at async NamespaceFS.read_object_stream (/usr/local/noobaa-core/src/sdk/namespace_fs.js:1080:46)    at async NsfsObjectSDK._call_op_and_update_stats (/usr/local/noobaa-core/src/sdk/object_sdk.js:543:27)    at async Object.get_object [as handler] (/usr/local/noobaa-core/src/endpoint/s3/ops/s3_get_object.js:116:25)    at async handle_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:161:19)    at async Object.s3_rest [as handler] (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:66:9)

It comes on GET request, in NSFS it is read_object_stream after we tried to get_buffer (a buffer from the memory to execute the read operation).
We want to see the values of the sem._waiting_value and want to see the trend: Does it keep raising? Does it increase and decrease during the test? etc.
We know that it started with 0:

[3532906]: [nsfs/3532906]    [L0] core.sdk.namespace_fs:: NamespaceFS: buffers_pool  [ BufferPool.get_buffer: sem value: 2097152 waiting_value: 0 buffers length: 0, BufferPool.get_buffer: sem value: 33554432 waiting_value: 0 buffers length: 0, BufferPool.get_buffer: sem value: 425721856 waiting_value: 0 buffers length: 0, BufferPool.get_buffer: sem value: 3825205248 waiting_value: 0 buffers length: 0 ]

This printing comes from here:
https://github.com/noobaa/noobaa-core/blob/e1bf29e82128a9e0d359b262978d208ccff228e5/src/util/buffer_utils.js#L217-L223
This warning printing might be in 2 minutes waiting - as configure here:
https://github.com/noobaa/noobaa-core/blob/e1bf29e82128a9e0d359b262978d208ccff228e5/config.js#L789
BTW, Although it is a warning, I'm not sure why it is printed as with console.error and not console.warn, I can try and suggest a code change.

@shirady

I had issues with the GPFS machine and couldn't reproduce a whole test run.
lets see if we can get your cluster working for the run.
For now kickstarted the run for 90min , debuglevel set to all for now

DEBUGLEVEL : all
ENABLEMD5 : true
Will update once run complete
Note: High level thought, this bug might be recreated with the load on the system. Am saying this because the issue 8524 with versioning PUT method didn't run into this error when ran for 3hrs (ran on Wed)

@shirady
Ran for the 90 min Warp get op run and didn't run into the buffer_pool error
./warp get --insecure --duration 90m --host :6443 --access-key KCxP4AN9937kVqoCrNIs --secret-key bIdwF/5nJtSnrHWXrhPOhkv1WqGjtayMk6D+aU/U --tls --obj.size 256M --bucket newbucket-warp-get-8521-15nov  2>&1| tee /tmp/newbucket-warp-get-255M-8521-15nov.log
[root@gui0 log]# zgrep "stuck" noobaa.log-20241115.gz
[root@gui0 log]# grep "stuck" noobaa.log
[root@gui1 log]# zgrep "stuck" noobaa.log-20241115.gz
[root@gui1 log]# grep "stuck" noobaa.log

Please try to check from the code flow perspective , as mentioned it could also be w/r/t load on the system

Hi,
I will share that I ran the warp twice and didn't reproduce it.
I didn't find the printing "Error: Warning stuck buffer_pool buffer" (or any "stuck" in the logs).
I ran it with a high debug level and didn't find a place where the waiting_value is not 0, from this output:
https://github.com/noobaa/noobaa-core/blob/e1bf29e82128a9e0d359b262978d208ccff228e5/src/util/buffer_utils.js#L205
@rkomandu, I'm planning to check other things I will update you here about it.
Additional information:
I run:
./warp get --host=<IP-address-of-1-node> --access-key=<> --secret-key=<> --obj.size=256M --duration=60m --bucket=<bucket-name> --objects 1500 --insecure --tls (after creating the bucket).
I had to set the --objects due to space limits on the machine.
The outputs:

Without changing the NSFS_CALCULATE_MD5

----------------------------------------
Operation: PUT. Concurrency: 20
* Average: 367.05 MiB/s, 1.50 obj/s

Throughput, split into 192 x 5s:
 * Fastest: 389.2MiB/s, 1.59 obj/s
 * 50% Median: 368.5MiB/s, 1.51 obj/s
 * Slowest: 325.8MiB/s, 1.33 obj/s

----------------------------------------
Operation: GET. Concurrency: 20
* Average: 849.39 MiB/s, 3.48 obj/s

Throughput, split into 239 x 15s:
 * Fastest: 896.0MiB/s, 3.67 obj/s
 * 50% Median: 851.6MiB/s, 3.49 obj/s
 * Slowest: 774.0MiB/s, 3.17 obj/s

Changing the NSFS_CALCULATE_MD5 to true

----------------------------------------
Operation: PUT. Concurrency: 20
* Average: 161.74 MiB/s, 0.66 obj/s

Throughput, split into 141 x 15s:
 * Fastest: 171.4MiB/s, 0.70 obj/s
 * 50% Median: 162.1MiB/s, 0.66 obj/s
 * Slowest: 148.0MiB/s, 0.61 obj/s

----------------------------------------
Operation: GET. Concurrency: 20
* Average: 821.38 MiB/s, 3.36 obj/s

Throughput, split into 238 x 15s:
 * Fastest: 853.9MiB/s, 3.50 obj/s
 * 50% Median: 823.2MiB/s, 3.37 obj/s
 * Slowest: 741.7MiB/s, 3.04 obj/s

Hi,
I would add that I tried to reproduce the error occurrence by adding code changes (to force the warning to be printed):
I set a shorter timeout (30 milliseconds instead of 2 minutes).
In config.js:
- config.NSFS_BUF_POOL_WARNING_TIMEOUT = 2 * 60 * 1000;
+ config.NSFS_BUF_POOL_WARNING_TIMEOUT = 30; //SDSD

I reduced the sem_size (the size represents 16 MiB, twice the size of NSFS_BUF_SIZE_L = 8388608  = 8 * 1024 * 1024, 8 MiB, the size of NSFS_BUF_POOL_MEM_LIMIT_L is 3825205248 = 3648MiB, when running on 258MiB in the steps below).
In namespace_fs.js:
-  sem_size: config.NSFS_BUF_POOL_MEM_LIMIT_L,
+ sem_size: 16777216, //SDSD

I added a printing to see when a buffer is allocated:
In buffer_utils.js:
        } else {
+            console.log('SDSD in buffer allocation');
            buffer = this.buffer_alloc(this.buf_size);
        }

Steps:
Before the code changes:

Create an account with the CLI: sudo node src/cmd/manage_nsfs account add --name <account-name> --new_buckets_path /tmp/nsfs_root1 --access_key <access-key> --secret_key <secret-key> --uid <uid> --gid <gid>
Note: before creating the account need to give permission to the new_buckets_path: chmod 777 /tmp/nsfs_root1, chmod 777 /tmp/nsfs_root1.
Start the NSFS server with: sudo node src/cmd/nsfs --debug 5
Notes:

I Change the config.NSFS_CHECK_BUCKET_BOUNDARIES = false; //SDSD because I’m using the /tmp/ and not /private/tmp/.

Create the alias for S3 service:alias nc-user-1-s3=‘AWS_ACCESS_KEY_ID=<access-key> AWS_SECRET_ACCESS_KEY=<secret-key> aws --no-verify-ssl --endpoint-url https://localhost:6443’.
Check the connection to the endpoint and try to list the buckets (should be empty): nc-user-1-s3 s3 ls; echo $?
Add bucket to the account using AWS CLI: nc-user-1-s3 s3 mb s3://bucket-buf (bucket-buf is the bucket name in this example)
Create the content for the body in size 256MB: dd if=/dev/urandom of=256MB_file bs=1M count=256
Put object: nc-user-1-s3 s3api put-object --bucket bucket-buf --key 256MB_file --body ./256MB_file.

After the code changes mentioned above:
8. Restart the server (by ctrl + c and run sudo node src/cmd/nsfs --debug 5, if you want you can redirect to a file: sudo node src/cmd/nsfs --debug 5 > logs_get_object_stream_4_code_changes.txt 2>&1)
9. Get the object: nc-user-1-s3 s3api get-object --bucket bucket-buf --key 256MB_file output_256MB_file
The operation completed:

Dec-4 16:30:53.142 [Upgrade/37314]    [L1] core.util.http_utils:: HTTP REPLY RAW GET /bucket-buf/256MB_file

I could see the request timeout error logs twice:

Dec-4 16:30:52.875 [Upgrade/37314] [ERROR] CONSOLE:: Error: Warning stuck buffer_pool buffer
Dec-4 16:30:52.880 [Upgrade/37314] [ERROR] CONSOLE:: Error: Warning stuck buffer_pool buffer

Before the mentioned printings I saw that those cases were when a buffer was allocated.

Dec-4 16:30:52.844 [Upgrade/37314]   [LOG] CONSOLE:: SDSD in buffer allocation
Dec-4 16:30:52.849 [Upgrade/37314]   [LOG] CONSOLE:: SDSD in buffer allocation

This issue was not reproduced (we both tried), and it is probably something specific to a run on a specific node state.
If, in the future, we will find a certain pattern, we can change the values in the config.js file.
As suggested, I can suggest to change the console log printing from ERROR to WARN.

@shirady What is the next step to do with this issue?

@romayalon, I would need to check if we can improve the current behavior of buffer pools.
I asked in the original issue of buffer pool a question about it, and if we see that we have ideas we might have a discussion about it.

@shirady , ran into this problem (occurrence in noobaa.log) when running Warp and trying to perform suspend/resume (HA functionality)
noobaa stage rpm that Romy has generated for other issue (8577) noobaa-core-5.17.1-20241211.el9.ppc64le is used
rpm location : https://noobaa-core-rpms.s3.us-east-1.amazonaws.com/noobaa-core-5.17.1-20241211-stage_5_17_2.el9.ppc64le.rpm
./warp mixed --insecure --duration 120m --host gpfs-p10-s3-ces.rtp.raleigh.ibm.com:6443 --access-key X0oJKaTmq5I2hGUGhqzd --secret-key uyYY9OA5syqZ+LsIAiv4f+Dn2mcqlr4kLzpvJ/sQ --tls --obj.size 128M --disable-multipart --bucket newbucket-ha-di-efix-warp-sus-res-16dec   2>&1| tee /tmp/newbucket-ha-di-efix-warp-sus-res-16dec.log
warp: <ERROR> download error: read tcp 9.42.124.184:58094->9.42.93.100:6443: read: connection reset by peer
warp: <ERROR> download error: read tcp 9.42.124.184:58100->9.42.93.100:6443: read: connection reset by peer                                                   Benchmarking:                09m38s / 2h00m00s ┃▓▓▓▓▓▓▓▓█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░┃   8.03%

Dec 16 00:14:34 node-gui1 [3201480]: [nsfs/3201480]    [L1] core.server.system_services.stats_aggregator:: standalon_update_nsfs_stats. nsfs_stats = { fs_workers_stats: { fileread: { count: 8, error_count: 0, min_time: 682, max_time: 936, sum_time: 6137 } }, io_stats: { read_count: 0, read_bytes: 67108864 } }
Dec 16 00:14:38 node-gui1 [3201481]: [nsfs/3201481]    [L0] core.sdk.endpoint_stats_collector:: bucket stats - newbucket-ha-di-efix-warp-sus-res-16dec application/octet-stream : { read_count: 2 }
Dec 16 00:14:38 node-gui1 [3201481]: [nsfs/3201481]    [L0] core.sdk.endpoint_stats_collector:: namespace stats - undefined : { read_count: 2, read_bytes: 256000000 }
Dec 16 00:14:40 node-gui1 [3201480]: [nsfs/3201480]    [L0] core.sdk.endpoint_stats_collector:: bucket stats - newbucket-ha-di-efix-warp-sus-res-16dec application/octet-stream : { read_count: 2 }
Dec 16 00:14:40 node-gui1 [3201480]: [nsfs/3201480]    [L0] core.sdk.endpoint_stats_collector:: namespace stats - undefined : { read_count: 2, read_bytes: 256000000 }

Dec 16 00:14:38 node-gui1 [3201481]: [nsfs/3201481]    [L0] core.sdk.endpoint_stats_collector:: namespace stats - undefined : { read_count: 2, read_bytes: 256000000 }
Dec 16 00:14:40 node-gui1 [3201480]: [nsfs/3201480]    [L0] core.sdk.endpoint_stats_collector:: bucket stats - newbucket-ha-di-efix-warp-sus-res-16dec application/octet-stream : { read_count: 2 }
Dec 16 00:14:40 node-gui1 [3201480]: [nsfs/3201480]    [L0] core.sdk.endpoint_stats_collector:: namespace stats - undefined : { read_count: 2, read_bytes: 256000000 }
Dec 16 00:16:27 node-gui1 [3201481]: [nsfs/3201481] [ERROR] CONSOLE:: Error: Warning stuck buffer_pool buffer    at BuffersPool.get_buffer (/usr/local/noobaa-core/src/util/buffer_utils.js:218:25)    at async NamespaceFS.read_object_stream (/usr/local/noobaa-core/src/sdk/namespace_fs.js:1080:46)    at async NsfsObjectSDK._call_op_and_update_stats (/usr/local/noobaa-core/src/sdk/object_sdk.js:543:27)    at async Object.get_object [as handler] (/usr/local/noobaa-core/src/endpoint/s3/ops/s3_get_object.js:116:25)    at async handle_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:161:19)    at async Object.s3_rest [as handler] (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:66:9)
Dec 16 00:16:28 gpfs-p10-gui1 [3201480]: [nsfs/3201480] [ERROR] CONSOLE:: Error: Warning stuck buffer_pool buffer    at BuffersPool.get_buffer (/usr/local/noobaa-core/src/util/buffer_utils.js:218:25)    at async NamespaceFS.read_object_stream (/usr/local/noobaa-core/src/sdk/namespace_fs.js:1080:46)    at async NsfsObjectSDK._call_op_and_update_stats (/usr/local/noobaa-core/src/sdk/object_sdk.js:543:27)    at async Object.get_object [as handler] (/usr/local/noobaa-core/src/endpoint/s3/ops/s3_get_object.js:116:25)    at async handle_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:161:19)    at async Object.s3_rest [as handler] (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:66:9)
Dec 16 00:16:28 node-gui1 [3201481]: [nsfs/3201481] [ERROR] CONSOLE:: Error: Warning stuck buffer_pool buffer    at BuffersPool.get_buffer (/usr/local/noobaa-core/src/util/buffer_utils.js:218:25)    at async NamespaceFS.read_object_stream (/usr/local/noobaa-core/src/sdk/namespace_fs.js:1080:46)    at async NsfsObjectSDK._call_op_and_update_stats (/usr/local/noobaa-core/src/sdk/object_sdk.js:543:27)    at async Object.get_object [as handler] (/usr/local/noobaa-core/src/endpoint/s3/ops/s3_get_object.js:116:25)    at async handle_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:161:19)    at async Object.s3_rest [as handler] (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:66:9)
Dec 16 00:16:28 node-gui1 [3201480]: [nsfs/3201480] [ERROR] CONSOLE:: Error: Warning stuck buffer_pool buffer    at BuffersPool.get_buffer (/usr/local/noobaa-core/src/util/buffer_utils.js:218:25)    at async NamespaceFS.read_object_stream (/usr/local/noobaa-core/src/sdk/namespace_fs.js:1080:46)    at async NsfsObjectSDK._call_op_and_update_stats (/usr/local/noobaa-core/src/sdk/object_sdk.js:543:27)    at async Object.get_object [as handler] (/usr/local/noobaa-core/src/endpoint/s3/ops/s3_get_object.js:116:25)    at async handle_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:161:19)    at async Object.s3_rest [as handler] (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:66:9)

Hi,
@rkomandu thanks for the information.
I noticed in the logs that we have this printings:
Dec 16 00:51:02 gpfs-p10-gui1 [3201481]: [nsfs/3201481]    [L1] core.util.buffer_utils:: BufferPool.get_buffer: sem value 3674210304 waiting_value 0 buffers length 30

The max I noticed of buffers length is 30.
we know that for 128MiB it reads every iteration reads 8 MiB and uses the buffer size of NSFS_BUF_SIZE_L, which means that at this point it uses 8 * 30 = 240 MiB.
Could you please share what are the resources for the noobaa service on this node? (systemctl status noobaa)
I want to see if it might be related in someway.

@shirady
have restarted multiple times for testing other defect (HA feature, suspend/resume), so the timestamp of dec 16th 00:51 is not possible , the noobaa resources that are used is as shown below

2024-12-16_03:05:39.371-0500: [I] mmcesop: isServiceRunning: rc=3, response=Redirecting to /bin/systemctl status noobaa.service
○ noobaa.service - The NooBaa service.
     Loaded: loaded (/usr/lib/systemd/system/noobaa.service; enabled; preset: disabled)
     Active: inactive (dead) since Mon 2024-12-16 03:05:35 EST; 3s ago
   Duration: 2d 21h 7min 25.210s
    Process: 3200676 ExecStart=/usr/local/noobaa-core/bin/node /usr/local/noobaa-core/src/cmd/nsfs.js (code=killed, signal=TERM)
    Process: 3980362 ExecStop=/bin/kill $MAINPID (code=exited, status=0/SUCCESS)
   Main PID: 3200676 (code=killed, signal=TERM)
        CPU: 6h 58min 23.820s

This indicates the Noobaa service is running for the 2d 21h 7min etc
In noobaa.log
Dec 16 00:50:58 node-gui1 node[3769297]: [/3769297]    [L2] core.util.os_utils:: promise exec systemctl status noobaa | grep Memory  false
..
Dec 16 00:51:11 node-gui1 node[3769733]: [/3769733]    [L2] core.util.os_utils:: promise exec systemctl status noobaa | grep Memory  false

@rkomandu,

Could you edit your comment and add the "Memory" printing as well?
I tried to search inside the maximum number of of the buffer length in the logs that I received. If there is s timestamp that you suggest to focus in, please let me know.

I will update you that I plan to add logging printing related to duration of actions that are in around the flow of the timeout message.
My current assumptions:

It might be related to the buffer allocation - In the example above  we can see that there was the printing of

Dec-4 16:30:52.844 [Upgrade/37314] [LOG] CONSOLE:: SDSD in buffer allocation

and after 30 milliseconds

config.NSFS_BUF_POOL_WARNING_TIMEOUT = 30; //SDSD

there is a printing of

Dec-4 16:30:52.875 [Upgrade/37314] [ERROR] CONSOLE:: Error: Warning stuck buffer_pool buffer

It is something that was slow during the GET (reading the object) - although I tried to search for the string "took too long" (grep "took too long") and the maximum size I noticed was "687.859 ms" in "FileFsync" (probably related to PUT) and not higher than 200 ms for "FileRead".

Reminder the timeout is  2 minutes, so currently, I don't have evidence in the logs for that assumption.
https://github.com/noobaa/noobaa-core/blob/970e67b9b5fe505f1c6fde864b41412773aa9922/config.js#L793

@shirady , in the noobaa log the Memory grep was performed, no other way could get the memory at that timestamp when the value was at 30 for the length.

authors: rkomandu romayalon shirady

repo: noobaa/noobaa-core


text:
text-security doesn't work with exotic characters

example 'テ' or 'Б' will appear and not be replaced

This seems like a duplicate of #4.
Have you tried version 2 (alpha, not yet working in IE), as mentioned in this comment?

I had the same problem yesterday.
@noppa i did not se the version 2 branch :(
I made a fork of your project using typescript and addes the possibility to generate fonts with multiple unicode ranges. For example
npm run build -- -s disc square circle -u U+0-780 U+781-5555 U+5556-aaaa U+aaab-ffff

Will build 12 different fonts ( 3 shapes with 4 ranges ) and generate a css that combines them.
A modern browser will then only load the needed fonts.
https://github.com/harterSe/text-security

Solved by using version 2.0.0-alpha.2

@edewit @harterSe
Just published version 3 has support for all unicode characters and for modern browsers the file size is actually smaller than it was in version 1. Also supports IE with a larger compatibility version of the font.

source: gharchive/issue

authors: edewit harterSe noppa

repo: noppa/text-security

license: mit

license_type: permissive

license_source: bigquery


text:
Optimize the matching of pbs jobs to ARC jobs

The original nested loop with linear search was expensive;
by sorting the arrays beforehand only a single traversal
of both structures simultaneously is required.
Additional info in the writeup here: https://www.nikhef.nl/~dennisvd/arc-infosys-speedup/arc-infosys-speedup.html
This runs in production at Nikhef.

Regarding the code ran for multiple queues, maybe you can apply this hack suggested by some LSF.pm users in this GGUS discussion
https://ggus.eu/index.php?mode=ticket_info&ticket_id=154333#update#32
I would not like this hack to be part of the ARC codebase because it encourages developers to continue writing LRMS modules using the old interface and I do not want to have hardcoded LRMS names in ARC0mod.pm code.
Best would be to rewrite PBS.pm as SLURMmod.pm where jobs and queues are collected separately and then matched later, and include this optimization you suggest there.

source: gharchive/pull-request

authors: dvandok floridop

repo: nordugrid/arc

license_source: github-api


text:
Creating new calendars

Hi,
how difficult it'll be to allow creating new calendars and not passing calendar_id?
What are the cases I should handle to allow that? and not passing calendar_id to Google::Calendar.new
Any technical reason its not already supported?

For some time now I've wanted to refactor the code so there would be a base object and a calendar factory object so it would be easier to interact with multiple calendars and work with calendar attributes.  But I haven't had the time to do it properly.  So yes, it is possible, but it will require a modest investment in time.

@szich Thanks for the reply.
Could you give me some pointers to where I should look into just so I could estimate how long it'll take me?

If all you want to do is be able to create an instance of the Google::Calendar class without passing in a calendar_id that would be easy to accomplish.  If you want to refactor the library, as I discussed above that will take a bit more time.
Either way a close look at the underling API is the best place to start:
https://developers.google.com/google-apps/calendar/v3/reference/

It's difficult to give you a specific answer because I'm not sure exactly what you have in mind, so  it depends on what you want to accomplish.  As I mentioned above if all you want to do is modify the existing code so you can create an instance of the Google::Calendar class that involves minimal changes that will take a couple of hours at most.  If you want to refactor the library, then you will be making more changes.  The ones I have in mind (which may or may not meet your needs) involve adding a base class and updating the google calendar class to remove all of the connection plumbing and add calendar specific attributes.  Thinking about it now, it doesn't seem like to much of a heavy lift, but when refactoring code it's easy to get sucked in.

Ok, thanks!

authors: szich udiudi


text:
readonly state issue

i sir it is possible to make read only state + activate state. because sir i donot see any differenct in read state and disabled state. What i want is the state of checkbox should be active , either on or off but user should not able to change it.

angular.element('[type="checkbox"]').bootstrapSwitch('state',true, true);
angular.element('[type="checkbox"]').bootstrapSwitch('disabled', true);

repo: nostalgiaz/bootstrap-switch


text:
Add description of key revocation and rescinding signatures

Add these scenarios to the TUF design document.

Moved to notaryproject/tuf#1

repo: notaryproject/notation


text:
Sync settings between devices

What makes the feature necessary?
People who want to use the same settings on multiple devices and/or platforms, i.e. multiple computers and browsers could have their settings in sync. They could also change a setting on one and have it synced to the other devices or browsers.
How would the feature work?
A settings file (JSON) is being synchronised through a web service, like browser sync, Google Drive, Dropbox, or possibly even a place inside one's Notion account. Upon loading of NE, or change of a setting, the JSON file is being synced.

Tremendous work needs a server or a wifi connection to successfully implement it. The other option would be having a notion document with code in your notion workspace. This could actually work. Let me know what u think.

@Xytronix I don't think there would be any need for a server, since you can store files in notion. Just a simple JSON settings file stored somewhere in Notion, which gets read from on startup and gets written to everytime I change some settings would be lovely :)!

authors: Xytronix enoversum

repo: notion-enhancer/repo

license: MIT


text:
Command Updates and Minor Bug Fixes

trying to integreate get_ip into playing

integreate ip into playing

#18 #25 #24 #23

repo: notoriouslogank/mcswitch


text:
Change the way people are associated with repos

We still add the "sender" as a reviewer by default. After that, it's up to the sender to push more people in as reviewers.

bors r+

bors r+

repo: notriddle/bors-ng


text:
add discord and twitter links

added link to the discord server I just made and the twitter thread

The Discord YOU made ?

The Discord YOU made ?

yes, i asked him to :)
https://twitter.com/nousr_/status/1566221567540965376?s=21&t=NbTENKOQo6YKqG4o3CGdYQ

authors: Lewington-pitsos nousr tocram1

repo: nousr/koi


text:
Optimized azks

See Issue #186.
With the removal of previous states of HistoryTreeNode, we don't have a need for keeping states for multiple epochs. In result, HistoryTreeNode, HistoryNodeState and HistoryChildState can simply be replaced with TreeNode.
This diff starts this conversion by adding the labels of its (1) left and right children, and (2) its hash to the HistoryTreeNode (not yet re-named). Actual hash updates will be added in the next diffs. Rather than keeping the node hashes at the parent, I will keep them at the node itself.
NOTE: The PR builds with cargo build but not ready for testing. The purpose of the PR is to "get it out there" to get feedback rather than merging it.

Summary
This PR so far has removed the need for storing states-per-epoch and revised the HistoryTreeNode (referred to as node below) to contain the only (current) state (aka hash), significantly reducing the storage requirements.
Important Changes

Each node keeps a label of their left and right children, and its hash.
For leaf nodes the hash is over the initial digest and the label.
For non-leaf nodes, the hash is over the left and right child, and the label.
For the root, we use the empty node hash: over the empty value and empty label.
The last_epoch has been kept to be used in append-only proofs whereas birth_epoch is not needed anymore.
For each call, set_child and update_node_hash writes the updated nodes to the storage for consistency. See below for optimizations over this.

Next Steps:

[ ] Clean up: We should remove the dead-code, debug output, unused HistoryChildState and HistoryNodeState, along with the storage layer code for these nodes.
[ ] Updating proofs:: Since some hash calculations have been updated, the proofs need to be updated accordingly.
[ ] Clippy: Make sure Clippy passes.
[ ] Merge with main (?): At this point, we should be able to merge this in (possibly to a separate branch).
[ ] Addition of epoch to the leaf: Leaves should include the latest epoch information in their initial hash. This will allow us to track when a leaf was inserted into the tree.
[ ] Enabling concurrent read/write operations: During a sequencing operation a node might get updated. This means if a client requests a proof during sequencing, we might return an inconsistent proof (a mix of states from the current and next epochs). To circumvent this, we should keep two hashes in the node and update the second one with the new state while moving the current second state to the first (i.e., shift the state to the left).
[ ] Remove birth epoch: Not needed anymore
[ ] Renaming HistoryTreeNode as TreeNode: No history, only current.

Improvements

Tests: We should add more tests for small trees and nodes with non-uniform distribution to increase test coverage for tree creation.
Minimizing storage writes: If a node is not written to the storage as soon as it is updated, a dirty-read may occur for consecutive operations. We might be able to minimize these writes by allowing short-term inconsistencies by restructuring the code. Another question to look into is how expensive are these instant-writes? (@afterdusk mentioned we use transactions to minimize the impact)
Tree Visualization: Noticed it is hard to track the current state of the tree through logs. Maybe a visual representation could assist us in this?

Discussed in-person with @eozturk1 and @Jasleen1 , let's disable the currently-failing tests and fix clippy, so that we can land and @Jasleen1 can work on top of the changes here (filling in the rest of the checkboxes).

I have disabled the failing tests and updated the next steps to list those tests.

Not sure what the integration test issue is exactly, but perhaps we can merge into a separate branch so that @Jasleen1 can start working on top of it. cc: @slawlor

repo: novifinancial/akd


text:
In development while changing static.config.js: TypeError: global.reloadAll is not a function

I found this line here:
https://github.com/nozzle/react-static/blob/57ecc92a63182aab91112be5c3e787c338308b09/packages/react-static/src/browser/components/RouteData.js#L19
This should be uncommented I guess. It is being called here:
https://github.com/nozzle/react-static/blob/57ecc92a63182aab91112be5c3e787c338308b09/packages/react-static/src/browser/index.js

I'll get that taken care of.

authors: bkniffler tannerlinsley

repo: nozzle/react-static


text:
Fix path for imports on Windows

Is this a bug report?
Fixes https://github.com/nozzle/react-static/issues/184
Environment

react-static -v: 4.3.2
node -v: 9.2.0
yarn --version or npm -v: yarn 1.3.2

Then, specify:

Operating system: Windows 10
Browser and version (if relevant):

Steps to Reproduce

(Write your steps here:)

Run example on Windows (I used the styled-components one)

Expected Behavior
Run without error.
Cause of issue
path.resolve(config.paths.ROOT, template), is returning OS specific paths. On Windows they contain backslashes instead of slashes. To fix this, all backslashes need to be replaced with slashes, because even on Windows the import paths have forward slashes.
`

Nice work!

Thanks. Would you consider to make a 4.3.3 soon, so that we are back to a working version on Windows? That would be rather great....

I'll make a release in the next few hours :)
On Wed, Nov 22, 2017 at 11:54 AM Karl Anders notifications@github.com
wrote:

Thanks. Would you consider to make a 4.3.3 soon, so that we are back to a
working version on Windows? That would be rather great....
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/nozzle/react-static/pull/191#issuecomment-346442683,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFUmCSPs14l-sDtgNv40H3joyPSZzf3Kks5s5G3rgaJpZM4QnSAx
.

authors: karland tannerlinsley


text:
🛑 NpComplete is down

In 157d049, NpComplete (https://npcomplete.dev/) was down:

HTTP code: 502
Response time: 3513 ms

Resolved: NpComplete is back up in 96afbf2.

repo: npcomplete-dev/podcast-site-upptime


text:
Saving re-attached entities configured with ForNpgsqlUseXminAsConcurrencyToken() throws DbUpdateConcurrencyException

Scenario:
When detached entity is attached to new context and set to Modified state in order for SaveChanges() to pick-up updates the DbUpdateConcurrencyException is thrown if the entity is also configured with ForNpgsqlUseXminAsConcurrencyToken().
minimal example

I should probably add that in the example there are no concurrent updates going on. In fact there's single write requested, so it's clearly a bug.

license: PostgreSQL


text:
Weird reformatting on TLS closure

Pre:
            SCRIPT_TASK_ROOT.with(|root| {
                 *root.borrow_mut() = Some(&script_task as *const _);
             });

Post:
                                                  SCRIPT_TASK_ROOT.with(|root| {
                 *root.borrow_mut() = Some(&script_task as *const _);
             });

This was part of a much-larger reformatting:
Pre:
        spawn_named_with_send_on_failure(format!("ScriptTask {:?}", id), task_state::SCRIPT, move || {
            let roots = RootCollection::new();
            let _stack_roots_tls = StackRootTLS::new(&roots);
            let chan = MainThreadScriptChan(script_chan);
            let channel_for_reporter = chan.clone();
            let script_task = ScriptTask::new(compositor,
                                              script_port,
                                              chan,
                                              control_chan,
                                              control_port,
                                              constellation_chan,
                                              Arc::new(resource_task),
                                              storage_task,
                                              image_cache_task,
                                              mem_profiler_chan.clone(),
                                              devtools_chan);

            SCRIPT_TASK_ROOT.with(|root| {

Post:
        spawn_named_with_send_on_failure(format!("ScriptTask {:?}", id),
                                         task_state::SCRIPT,
                                         move || {
                                                  let roots = RootCollection::new();
                                                  let _stack_roots_tls = StackRootTLS::new(&roots);
                                                  let chan = MainThreadScriptChan(script_chan);
                                                  let channel_for_reporter = chan.clone();
                                                  let script_task = ScriptTask::new(compositor,
                                                                                script_port,
                                                                                chan,
                                                                                control_chan,
                                                                                control_port,
                                                                                constellation_chan,
                                                                                Arc::new(resource_task),
                                                                                storage_task,
                                                                                image_cache_task,
                                                                                mem_profiler_chan.clone(),
                                                                                devtools_chan);

                                                  SCRIPT_TASK_ROOT.with(|root| {

Can you post a bit more context? This is the part of rustfmt that is currently undergoing many changes, so these examples can help a lot!

Ah, thanks ;-)

Fixed!

authors: jdm marcusklaas

repo: nrc/rustfmt


text:
Update bower.json

Hi nreese, thank you for the plugin, work of art.
In my case, compilation in docker fails with your bower.json settings.

thanks for the PR

authors: michimau nreese


text:
Basic mobile support

Touch support for controls (stopPropagation for touch events?), see reported issues:
brouter web: Pulldown menu do not work

Profile and Alternative option lists not selectable

Is there any internet browser for android devices which works with http://brouter.de/brouter-web/ (altitude profile, 4 possibilities, GPX download)?

not able to insert (copy and paste) Profile code
not able to click/touch on GPX download

I know that you did some work regarding this issue @nrenner (and maybe #35), can you share the current status please? I'd again gladly help to make it (and #35 too) happens!

Was going to ask and share what I have. I would be happy if you would take over the mobile support!
Actually, there isn't much yet, just some first experiments, see the mobile-test branch. I also have some notes, I will clean these up and make a wiki page tomorrow.
What are your ideas? What would be your approach?

@bagage See wiki pages Mobile and Mobile code snippets. So I guess that's all I have right now.

Thanks, awesome! Your wiki seems complete enough of existing issues; I have yet no solution as a developer, but I'll dig into some code & your references & other responsive design maps to be inspired from.
I'll update you when I have more precise ideas.

Here attempt to make BRouter-web a little bit more mobile friendly.
This is still a WIP branch and there is still some missing features but it should be enough to test and see if it is the way to go.

Wow, that's already a working mobile version!
Some thoughts:

Profile name: consider long names e.g. 'vm-forum-velomobil-schnell'
Elevation graph: What was the reason for having our own implementation? What are your plans for that?
Options: while this is Ok for mobile, on the desktop I would still like being able to switch through profiles/alternatives (e.g. up/down keys) and see the result without a modal blocking the screen.
No left pane / sidebar: not really needed for mobile, but I think we can't do without on desktop, esp. for data tab (missing?) and future turn instructions list #42 and tag graph #45, that all should be next to the map and synchronize highlight on hover.

I will continue the work started by @RoPP, but here are some replies:

long names won't fit on small screens, maybe we should avoid having too long names? In my opinion this is not a very important information anyway once you selected it.
i'll fix up/down shortcuts on desktop.
data tab should indeed be readded on desktop. Considering #42 and #45 let's see when they happen!

I will continue the work started by @RoPP, but here are some replies:

long names won't fit on small screens, maybe we should avoid having too long names? In my opinion this is not a very important information anyway once you selected it.
i'll fix up/down shortcuts on desktop.
data tab should indeed be readded on desktop. Considering #42 and #45 let's see when they happen!

In the mean time is there any ETA for next release (to use #49 and #50 as soon as possible)? Thanks.

Longnames can be used quite often, referring to values of various flags or tunables. Especially with my  profile generator
See e.g. "Car-FastEco-TollFree-SecondaryRoads.brf"

long names won't fit on small screens, maybe we should avoid having too long names? In my opinion this is not a very important information anyway once you selected it.

Not sure about avoiding long names, maybe switching to an alternative short name or shorten display with "..." in the middle on small screens. But agree that as a simple solution we can also just not show it.

i'll fix up/down shortcuts on desktop.

One idea would be to directly integrate the two dropdowns into the top bar.

data tab should indeed be readded on desktop. Considering #42 and #45 let's see when they happen!

Forgot to mention the Itinerary control that is shown as tab in the transit demo (see config.js).
Generally, the mobile version already looks great, just a bit too focused on mobile for my taste right now. I don't insist on keeping the current left pane, some other collapsible Sidebar will also do for the data table and for editing the profile.
My original motivation for brouter-web was to build some kind of IDE for profile development and a planning tool with a focus on desktop. So mobile support would be great, it just shouldn't affect the desktop experience too much, e.g. by requiring more clicks than before or slowing down feedback loops.

In the mean time is there any ETA for next release (to use #49 and #50 as soon as possible)? Thanks.

Well, the plan was to have version 0.6.0 ready for the start of the bike season - so much for ETAs. Already was thinking about splitting the milestone, but still wanted to look for icon set alternatives and do some other minor stuff.
But now that we have exceeded the Bing usage limit, I really should add a replacement and get some release out soon.

authors: RoPP bagage nrenner poutnikl

repo: nrenner/brouter-web


text:
fix(Timeline): infinites are rendered incorrectly when followed by another Piece

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

Bug fix

What is the current behavior? (You can also link to an open issue here)

If a user-stopped Infinite Piece is followed by any other Piece, it's renderedDuration will be set to the difference between it's inPoint and next Pieces inPoint, even if it ended before the next Piece.

What is the new behavior (if this is a feature change)?

The user-stopped Infinite Piece maintains it's duration in the timeline.

Other information:

Status

[x] Code documentation for the relevant parts in the code have been added/updated by the PR author
[x] The functionality has been tested by the PR author
[ ] The functionality has been tested by NRK

Codecov Report
Base: 66.95% // Head: 75.27% // Increases project coverage by +8.32% :tada:

Coverage data is based on head (c272e31) compared to base (8f542cd).
Patch has no changes to coverable lines.

Additional details and impacted files
@@              Coverage Diff              @@
##           release44     #771      +/-   ##
=============================================
+ Coverage      66.95%   75.27%   +8.32%
=============================================
  Files            254       25     -229
  Lines          28741     5016   -23725
  Branches        4010      502    -3508
=============================================
- Hits           19243     3776   -15467
+ Misses          9039     1238    -7801
+ Partials         459        2     -457

Impacted Files
Coverage Δ

meteor/client/lib/rundown.ts

packages/job-worker/src/rundownPlaylists.ts

meteor/lib/collections/CoreSystem.ts

meteor/server/lib.ts

meteor/lib/systemTime.ts

...kages/job-worker/src/blueprints/context/context.ts

...b-worker/src/playout/__tests__/helpers/rundowns.ts

meteor/server/publications/lib.ts

meteor/server/api/serviceMessages/postHandler.ts

meteor/server/api/rest/rest.ts

... and 269 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

authors: codecov-commenter jstarpl

repo: nrkno/sofie-core


text:
A [GET] on /users respond with 'NameError (uninitialized constant Knock)'

I use rails 5.0.0.beta3 and rbenv and used this guide
With rbenv i created a new rails api project rbenv exec rails _5.0.0.beta3_ new levented --api
After that i scaffold my user bin/rails g scaffold user name:string family_name:string nick_name:string email:string
routes.rb
Rails.application.routes.draw do
  mount Knock::Engine => '/knock'
  resources :users
end

application_controller.rb
class ApplicationController < ActionController::API
  include Knock::Authenticable
end

users_controller.rb
class UsersController < ApplicationController
  before_action :set_user, only: [:show, :update, :destroy]

  # GET /users
  def index
    @users = User.all

    render json: @users
  end
end

A [GET] to /users prints the following error
ActionController::RoutingError (uninitialized constant ApplicationController::Knock):

app/controllers/application_controller.rb:2:in `<class:ApplicationController>'
app/controllers/application_controller.rb:1:in `<top (required)>'
app/controllers/users_controller.rb:1:in `<top (required)>'
Started GET "/users" for 127.0.0.1 at 2016-03-15 20:57:53 +0100

What did i wrong?

I haven't tested knock with rails 5 yet so I'm not sure. I'll let you know when I do.

I tested Knock (master) with Rails 5 beta4. It seems to work fine without any modifications :)
Closing this for now, but feel free to reopen if you still have an issue with the latest beta. If you do, please provide a failing test as well.
Thank you 👍

I have the same problem with knock 2.0 when I want to login.
routes.rb
Rails.application.routes.draw do
  namespace :api, defaults: {format: :json} do
    mount Knock::Engine => '/login'
    resources :users, :only => [:create]
  end
end

application_controller.rb
class ApplicationController < ActionController::API
  include ActionController::Serialization
  include Knock::Authenticable
end

error
Started POST "/api/login/auth_token" for ::1 at 2016-10-26 12:01:30 +0200
Processing by Knock::AuthTokenController#create as JSON
  Parameters: {"auth"=>{"email"=>"email@test.com", "password"=>"[FILTERED]"}, "auth_token"=>{"auth"=>{"email"=>"email@test.com", "password"=>"[FILTERED]"}}}
Completed 500 Internal Server Error in 1ms (ActiveRecord: 0.0ms)
NameError (uninitialized constant Knock::Auth)

@YMonnier you have probably fixed this but for anyone else having this issue, you need to change the endpoint that you're posting to from auth_token to whatever entity you're authing against - probably user_token. I think this is a change brought in by Knock 2.0.

I have the same issue in rails 'rails', '~> 6.0.2', '>= 6.0.2.1' following the Readme steps does prompt the same issue referring to the line in application controller include Knock::Authenticable. Any help ?

I second @hackerghost93 current using rails 6 and getting the error uninitialized constant Knock::Authenticable on any route inheriting from ApplicationController
currently added

gem 'knock' to gemfile + ran bundle install
ran the generator rails generate knock:install
ran rails generate knock:token_controller app (app being model I am authenticating)
In my ApplicationController I currently have

class ApplicationController < ActionController::Base
  include Knock::Authenticable

  def authenticate_app
    app = authenticate_for(App)
    if(app.nil?)
      render json: {"errors": ["resource not found"]}, status: 404
    else
      app
    end
  end
end

currently, any request made to any controller inheriting from ApplicationController (through postman or rspec) results in the error
Failure/Error: include Knock::Authenticable

NameError:
  uninitialized constant Knock::Authenticable

@nsarno any steps I can take to debug this?

I second @hackerghost93 current using rails 6 and getting the error uninitialized constant Knock::Authenticable on any route inheriting from ApplicationController
currently added

gem 'knock' to gemfile + ran bundle install
ran the generator rails generate knock:install
ran rails generate knock:token_controller app (app being model I am authenticating)
In my ApplicationController I currently have

class ApplicationController < ActionController::Base
  include Knock::Authenticable

  def authenticate_app
    app = authenticate_for(App)
    if(app.nil?)
      render json: {"errors": ["resource not found"]}, status: 404
    else
      app
    end
  end
end

currently, any request made to any controller inheriting from ApplicationController (through postman or rspec) results in the error
Failure/Error: include Knock::Authenticable

NameError:
  uninitialized constant Knock::Authenticable

EDIT
Doing some digging, I found the solution to this for rails 6 thanks to this comment thread. In your application.rb add this line
require_relative 'boot'

require 'rails/all'

# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)

module YOURRAILSAPPNAME
  class Application < Rails::Application
    # Initialize configuration defaults for originally generated Rails version.
    config.load_defaults 6.0 and config.autoloader = :classic   <=== this line

    # Settings in config/environments/* take precedence over those specified here.
    # Application configuration can go into files in config/initializers
    # -- all .rb files in that directory are automatically loaded after loading
    # the framework and any gems in your application.
  end
end

thank you, helped me a lot!

authors: Galileuu YMonnier colinramsay hackerghost93 jnewman12 maluramichael nsarno

repo: nsarno/knock


text:
Fix of the vtu polyhedron filter

Background
The current vtu filter for polyhedron cells is not compatible with the specification of the vtu format. The error was introduced (by me) in a5f59bb.
Changes

The vtu polyhedron filter requires, for each cell, the nodes of each of the cell's faces. This is implemented as a list of numpy arrays.
Reading vtu files with general and varying number of faces per cell and nodes per face, is complex compared to more standardized cases. The extensions are (mainly) confined to specific blocks of code, which should not need to be touched for changes to other grid types etc. At a few points, this localization led to slightly more complex code, but I think it is worth it.

Main problem with the presented solution
The list of cell-faces has, from what I can see, no natural placement in the Mesh data structure. I tried to put it as cell_data, but this breaks the convention that cell_data is numpy arrays. This lead to all kind of trouble - for a glaring example see the changes to helpers.py, function write_read().
My preferred solution would be to introduce a new dictionary on the top level of the Mesh class (name: cell_faces?), and move the information there. This should have no impact on the rest of the code base (except that the ugly hack in the test function can be moved to a subfunction), but it will significantly reduce the entropy of the affected parts of the vtu filter. On a wider level, a dedicated field may be useful for other grid formats as well. However, I thought it best to first present a solution with less impacts on the fundamentals - I assume you try to be restrictive with changes to the Mesh class.
Should you be willing to introduce such a new top-level field, I can take care of implementation (should be easy), and include it in this PR. I do not have oversight of all other grid formats supported by meshio, but from what I can tell, no changes outside Mesh and vtu are needed.
Other comments

The naming convention for polyhedron cell blocks is now explicitly set as polyhedron{number_of_nodes_per_cell} (it could have been faces per cell).
The filter can handle polyhedra with different number of nodes, but not combinations of polyhedra with other types of cells. It should not be too difficult to implement, but the preferred solution in this case is to represent all cells as polyhedra.

My preferred solution would be to introduce a new dictionary on the top level of the Mesh class (name: cell_faces?), and move the information there. This should have no impact on the rest of the code base (except that the ugly hack in the test function can be moved to a subfunction), but it will significantly reduce the entropy of the affected parts of the vtu filter. On a wider level, a dedicated field may be useful for other grid formats as well.

Yes, e.g. polyMesh #660.

My preferred solution would be to introduce a new dictionary on the top level of the Mesh class (name: cell_faces?)

This sounds good. cell_faces or polyhedron_faces are a reasonable names. Best go ahead and add it to the mesh class.

Great. I will take care of it within a few days.
Wrt name, I like polyhedron_faces better - it makes it clearer that this is necessary for certain cell types only. So I will use this unless I hear otherwise.

polyhedron_faces are now added to the Mesh, and the vtu reader uses it.

The list of cell-faces has, from what I can see, no natural placement in the Mesh data structure

Why do we keep a separate list polyhedron_faces instead of treating the faces as polygon cells?

This sounds good. cell_faces or polyhedron_faces are a reasonable names. Best go ahead and add it to the mesh class.

Quoting myself here. Question: Why don't we include polyhedra into the cells list? We do that for polygons, right?

Not sure I follow you. You want to add the polyhedron faces (which by themselves are polygons) as separate cells? I guess that can be done, but you would then need to identify which of these new cells go together to form a 3d cell? Or am I missing something?

Not sure I follow you.

It's a question. If I see things correctly, a VTU file contains polyhedra as a list of polygonal faces. Is this correct. Why do we need a separate member in Mesh for it? Can't we just store the polyhedra as they come in a CellBlock like the other cells?

So, each cell must have each of its faces specified by its nodes (with an ordering of the nodes). Since the polyhedron can have any number of faces, which again have any number of nodes, you need more than a specific ordering of the nodes in a cell-node relation. As an example, in a pyramid the information needed to define the square at the base is different than the other 4 sides (and it would change again if the base was split into two triangles).
The polygon is different since it is 2d, thus the faces can be assumed to be defined by subsequent nodes in the cell-node relation.
I hope that was an answer to the question.

Alright, yes. Still: Why don't we put this exact infromation in a CellBlock?

If that does not break with what you would assume about the content in a CellBlock, I agree it is much better. I kind of assumed that the CellBlock content should be a standard numpy array.
Will this require modification of export filters where polyhedrons are invalid?

If that does not break with what you would assume about the content in a CellBlock, I agree it is much better. I kind of assumed that the CellBlock content should be a standard numpy array.

I'm not too sure about it myself but I think it should be alright. Try it out.

Will this require modification of export filters where polyhedrons are invalid?

What do you mean when you say "filter"?

Pushed a version with storage in CellBlocks. It works. The price for not having a separate variable for the face-nodes is special treatment of mesh.cells whenever there is the possibility that a polyhedron is present in the mesh. For instance in test.helpers.read_write():
https://github.com/keileg/meshio/blob/55501aceddf2bae7ab5ae7aef826166342bf778f/test/helpers.py#L349-L358
Similarly, if-elses are needed at a few places in _vtu.py, and will likely be needed as other mesh formats are extended to cover polyhedra.
I'm not sure myself which version I would prefer, but this decision clearly requires wider perspectives than mine.

Can you append a file that contains polyhedral data as described in this PR?

A splitting of the unit cube is attached (same as that generated by helpers.polyhedron_mesh). The file extension is changed to .txt to be compatible with what GitHub supports.
polyhedron_example.txt

authors: gdmcbain keileg nschloe

repo: nschloe/meshio


text:
项目首次渲染，cesium.js过大导致页面加载过慢

如题，这种大文件首次加载怎么在vite里边处理呢，或者说怎么让它懒加载？

我觉得没啥办法，想用他类库就要先加载它吧

哦，应该没有考虑过这个，首页默认会有一个《script src = cesium.js》 ，不知道加一个 type="module" 会不会延迟加载？你可以先手动改一下试试

目前（<2023-02-04 Sat 15:43:33 UTC+08:00>）我是这样操作的来避免首页加载Cesium.js的：

将首页和子页面都使用动态导入的方式进行懒加载 https://router.vuejs.org/zh/guide/advanced/lazy-loading.html
设置本插件的rebuildCesium为true并重启vite

但所有页面仍都会加载cesium/Widgets/widgets.css，需求点是让Cesium库只在相应页面加载。
src/main.js:
import { createRouter, createWebHashHistory } from "vue-router";
const Home = function () {
  return import("./Home.vue");
};
const Map = function () {
  return import("./Map.vue");
};
const routes = [
  { path: "/", component: Home },
  { path: "/map", component: Map },
];
const router = createRouter({ history: createWebHashHistory(), routes });
app.use(router);

vite.config.js:
import cesium from "vite-plugin-cesium";
plugins.push(cesium({ rebuildCesium: true }));

大佬，可以写个回调进去吗？在cesiumjs加载完时触发这个回调函数？

repo: nshen/vite-plugin-cesium


text:
CI: enable builds

Based on https://github.com/NSLS-II/conda-pack-template.

The only major change here compared to https://github.com/NSLS-II/conda-pack-template is the updated GHA CI workflow, the configs/ and envs/ dirs were copied from that repo's main branch.


text:
Test invocation always fails

neo-go version 0.93.0-pre-189-g2ee755e0
Pretty new issue, haven't seen it in 0.93.0-pre-158-g71494e6a
All test invocations from CLI or client fails with error: json: cannot unmarshal number 0.0099891 into Go struct field invokeAux.gasconsumed of type int64

#1723, certainly.

Tried to use chain dump created with 0.93.0-pre-189-g2ee755e0 in old neo-go single node environment. That caused the issue.

authors: alexvanin roman-khimov

repo: nspcc-dev/neo-go


text:
NeoFSBlockFetcher may seem hanging

with NeoFSBlockFetcher address set to ssl neogo silentlty doesn't work
steps to reproduce:
NeoFSBlockFetcher:
  Addresses:
    - st1.storage.fs.neo.org:443
    - st2.storage.fs.neo.org:443
    - st3.storage.fs.neo.org:443
    - st4.storage.fs.neo.org:443

log:
INFO        starting NeoFS BlockFetcher service

neogo silentlty doesn't work

It works. BlockFetcher is trying to deal, and we have a relatively large deal timeout, so the error will be returned after timeout.

deal timeout is not an answer, as reply happens instantly:
% curl -v http://st1.storage.fs.neo.org:443
* Host st1.storage.fs.neo.org:443 was resolved.
* IPv6: (none)
* IPv4: 157.90.176.145
*   Trying 157.90.176.145:443...
* Connected to st1.storage.fs.neo.org (157.90.176.145) port 443
* using HTTP/1.x
> GET / HTTP/1.1
> Host: st1.storage.fs.neo.org:443
> User-Agent: curl/8.11.0
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 400 Bad Request
< Server: nginx
< Date: Wed, 04 Dec 2024 14:50:10 GMT
< Content-Type: text/html
< Content-Length: 248
< Connection: close
< Strict-Transport-Security: max-age=63072000; includeSubDomains
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
<

<title>400 The plain HTTP request was sent to HTTPS port</title>

400 Bad Request

The plain HTTP request was sent to HTTPS port

nginx

* shutting down connection #0

@cthulhu-rider probably knows something about it, I don't think it can be fixed with current SDK.

The problem is not on the NeoGo side. pool.Dial returns only after n*DealTimeout seconds where n is the number of nodes in the pool, so the problem is related to the pool itself.  The following code is responsible for initial pool dealing on the NeoGo side:
https://github.com/nspcc-dev/neo-go/blob/8952922c4d4f4474e390cf3afa4abcdf20c7960d/pkg/services/blockfetcher/blockfetcher.go#L202-L204
The result is (with smaller dial timeout):
2024-12-04T16:22:52.244+0300	INFO	starting NeoFS BlockFetcher service
2024-12-04T16:22:53.255+0300	INFO	persisted to disk	{"blocks": 0, "keys": 116, "headerHeight": 0, "blockHeight": 0, "took": "12.85244ms"}
2024-12-04T16:25:52.248+0300	ERROR	skipping NeoFS BlockFetcher	{"error": "failed to dial NeoFS pool: at least one node must be healthy"}

So @532910 you may submit an issue to the NeoFS SDK repo.

@cthulhu-rider probably knows something about it, I don't think it can be fixed with current (as in version used by NeoGo) SDK.

the behavior is the same in the latest SDK. The error we receive from gRPC layer is
open gRPC connection: gRPC dial: context deadline exceeded: connection error: desc = "error reading server preface: http2: frame too large"

and gRPC dialer considers reading server preface as a temporary error: it tries again until the deadline arrives
in opposite, NeoFS connection fails immediately on non-temporary failures. For example, both single client and pool return immediately if we set ports to 444:
gRPC dial: connection error: desc = "transport: error while dialing: dial tcp 157.90.176.145:444: connect: connection refused"

@AnnaShaleva is right that pool's dial timeout sums all the client ones. And yeah, everything that can be improved in this area is located in the NeoFS SDK

% journalctl -n1 -u neogo-morph-rpc
Dec 06 14:36:14 tenes4 neogo-morph-rpc[446809]: INFO        starting NeoFS BlockFetcher service
% date
Fri  6 Dec 17:09:18 UTC 2024

authors: 532910 AnnaShaleva cthulhu-rider roman-khimov


text:
Native NEO and GAS contracts

Following #778 we need to implement native NEO and GAS contracts (they're tightly related, so it makes no sense splitting this into two tasks). These contracts should also be used in the genesis block.

I think we have enough of them now, some minor corrections can be done in separate tasks.


text:
Run-and-move-down doesn't display result on last line of file

Activating run-and-move-down while on the last line in the file (no trailing newline) causes the result bubble to not displayed. In addition to not showing the output, this creates the false impression that nothing was executed.
I suspect the cause is that the move-down operation has to create a new line where one previously did not exist, which triggers the output bubble to be hidden.

I can't reproduce this issue. Could you post the file you're using to reproduce this issue?

Steps to reproduce (in parentheses are action names, and the keyboard shortcuts on my system):

Open a new file (Application: New File; Cmd-N)
Switch to Python syntax (Grammar Selector: Show; Cmd-Shift-L; also available via toolbar on bottom)
Type "1"
Do run-and-move-down (Hydrogen: Run and Move Down)

Expected behavior: output "1" is shown
Actual behavior: no output is shown
It helps to have line numbers enabled in the sidebar to see that the line you're typing on is actually the last (i.e. there is no line number greater than it).
On my system this issue actually can't be reproduced by opening an existing file and not modifying it, because Atom enforces the presence of terminating newlines on every save/load.

@nikitakit I couldn't reproduce this issue. Could you take a screenshot of what you see in the dev console (shift-ctrl-i)?

I'm surprised you can't reproduce. The dev console output is not useful, so I went and debugged it myself.
Video of the issue: http://imgur.com/a/Ux2zW
And a fix (not very good abstraction, but I can't come up with a better alternative offhand)
diff --git a/lib/main.coffee b/lib/main.coffee
index 74984ca..b30ca0d 100644
--- a/lib/main.coffee
+++ b/lib/main.coffee
@@ -264,14 +264,20 @@ module.exports = Hydrogen =
     runAndMoveDown: () ->
         codeBlock = @findCodeBlock()
         unless codeBlock?
             return

         [code, row] = codeBlock
         if code? and row?
+            if row >= @editor.getLastBufferRow()
+                @editor.moveToBottom()
+                @editor.insertNewline()
+                @editor.moveLineUp()
+                @editor.moveLineDown()
+
             @createResultBubble code, row
             @moveDown row

     runCell: () ->
         [startRow, endRow] = CellManager.getCurrentCell()
         if endRow > startRow
             for i in [startRow .. endRow - 1] when @blank(endRow)

The issue is the bubble is created, then moveDown calls @editor.insertNewline(), which immediately invalidates the bubble.

Apparently I can't follow instructions to the letter :disappointed: !
Anyway, I can now reproduce it and to trigger this issue is important that the file is untitled.

I think we could fix this, if we change the way the marker for the result bubble is set (https://github.com/nteract/hydrogen/blob/master/lib/main.coffee#L163). So it isn't invalidated if a new line is inserted.

@lgeiger I'm still trying to understand why it doesn't happen with named files.

I can reproduce this issue now with named files as well. The important thing is that no newline is inserted by atom at the bottom.

On 06/07/16 10:57, Lukas Geiger wrote:

I think we could fix this, if we change the way the marker for the result bubble is set (https://github.com/nteract/hydrogen/blob/master/lib/main.coffee#L163). So it isn't invalidated if a new line is inserted.

Do you mean that we wouldn't ever invalidate markers?

I haven't thought about this in detail but I think marking the beginning of the line instead of the end would prevent this issue.

@lgeiger It works! Shall I submit a PR?

@n-riesco Great go ahead!

@lgeiger I spoke too fast. After further testing, having the marker located at the beginning of the line only invalidates the marker if the beginning of the line is changed.

@n-riesco Yes thats true. We could also mark the whole line and use an different invalidate option

@lgeiger I've tried:
        marker = @editor.markBufferRange [[row, 0], [row, lineLength]],
            invalidate: 'inside'

but it fails to invalidate when changing the end of a line.
I've also tried:
        marker = @editor.markBufferRange [[row, 0], [row, lineLength]],
            invalidate: 'touch'

but it displays the same problem as markBufferPosition.
I think we could use markBufferPosition to improve how the result bubbles are invalidated, but I'd leave this for another PR.
I'm getting more convinced that, to solve this issue, we need to treat files not ending with an EOL as a special case.

@n-riesco Thanks a lot for investigating!

I'm getting more convinced that, to solve this issue, we need to treat files not ending with an EOL as a special case.

I agree. We could simply don't call @moveDown if the cursor is at the last line.

authors: lgeiger n-riesco nikitakit

repo: nteract/hydrogen


text:
Docker proposal

Follow up to https://github.com/nteract/hydrogen/pull/524
I think the simplest way is a dockerfile:
FROM python:2
ARG TINI_VERSION=v0.10.0
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN chmod +x /tini
RUN pip install jupyter_kernel_gateway
ENV KG_LIST_KERNELS=True

A docker-compose:
version: '2'
services:
  jupyter:
    context: .
    entrypoint: ["/tini", "--"]
    command: ["jupyter", "kernelgateway", "--ip=0.0.0.0", "--port=8888"]

And a kernel settings default:
[{"name": "docker", "options": {"baseUrl": "http://localhost:8888"}}]

I'd propose this replaces basically all the README from L250-580. But probably I'm missing something and there's useful stuff there?
Questions:

Do we need a docker-compose file, or should we just the commands in the dockerfile?
Should we map the current path by default? This would allow you to run the code you're working on
What else are we missing?

Thanks for opening the issue. Unfortunately I don't know much about Docker.
@tiangolo May be able to answer your questions.

Thanks for the response @tiangolo !

...But in that case, wouldn't it be easier to use a Jupyter Docker stack?

Yes, good idea.

Or the equivalent docker-compose.yml, probably both options.

Agreed

The kernel settings wouldn't necessarily be like you specified above, as many users won't run that from a Linux machine but rather from Windows or Mac, and in those cases it will mostly be a Virtual Machine with a local IP (as 192.168.99.100).

I use Docker for Mac and localhost worked for me. What makes you think you need the IP?

So it should probably be kept for backward compatibility.

Your call, but there is a lot there! Might be simpler as a link to a 'Further detail' / 'Previous implementation' page. For a first time user, there's a lot of detail that's not really needed, and makes it look arduous to set this up when it's surprisingly simple. I'm sure there are several good approaches for making it simpler, though.

I think I can easily add that to the README.md (it's almost already written in this issue 😆 ). Or do you want to take a stab at it and create a PR?

If you want to add it I would be honored! If you prefer me to, I'm happy to take a stab in the near future.

Excellent @MaximilianR !
Let's try with a Jupyter Docker stack.

I use Docker for Mac and localhost worked for me. What makes you think you need the IP?

Interesting that it just works in Mac! I didn't know. In Docker for Windows, and in Docker Toolbox (at least in Windows) it doesn't work like that, the VM gets an IP in the local (VM) network, so you have to use that IP (it is not bound to the localhost).
On the other side, if the user is connecting to a Docker running in a different machine in the same network, the IP would also be different.

Your call, but there is a lot there! Might be simpler as a link to a 'Further detail' ...

You're right. It is a lot indeed. Maybe we can split it in a separate file KERNEL_CONNECTION.md and link to it from the main README.md as with the plug-ins.

So, the steps to do would be:

Move everything from the section Custom Kernel Connection to a file KERNEL_CONNECTION.md and link to it from that section with a short note explaining what it is and that the easier method would probably be the Kernel Gateway "documented above".

In the Example Kernel Gateway section add a sub-section with instructions on how to set it up with a Jupyter Docker Stack. Including:

A Dockerfile
How to build and run it with docker commands
How to build and run it with docker-compose
A note with a link to jump to the Connect Atom section as those instructions would also apply unmodified.

The section Testing it gets a note to generalize it to both Dockerfiles.

In the section Terminate the connection and container add a note explaining how to do it with the Jupyter Docker Stack (that would be a lot simpler).

Is that right @MaximilianR ?
Would that be OK @lgeiger ?

Beautiful, @tiangolo

In the section Terminate the connection and container add a note explaining how to do it with the Jupyter Docker Stack (that would be a lot simpler).

That's just Ctrl-C now 💃

@tiangolo @MaximilianR Thank you a lot for those excellent explanations! I'm sure that this will help a lot of users 🎉
I love the idea of a separate KERNEL_CONNECTION.md file!

Would that be OK @lgeiger ?

That sounds awesome!

Excellent! I'll test the setup and work on a PR.

As the PR is already merged, I think we can close this issue.

authors: MaximilianR lgeiger tiangolo


text:
Error when starting masakari-controller

Hello,
I downloaded the master branch of this project.
now I can vagrant up the controller and compute nodes.
(with some try and error but adding apt-get and pip cmd to vagrantfile fixed them).
deploy OpenStack in Controller/Compute and devstack started OK.
but when I try to start the masakari-controller it's not running
stack@controller:~ /devstack$ sudo service masakari-controller start
Starting masakari:  *
stack@controller:~ /devstack$ sudo service masakari-controller status
** masakari is not running
the only error I found in the log is:
stack@controller:~/masakari$ cat /var/log/masakari/masakari-controller.log
Feb 06 03:23:18  masakari(20458): CRITICAL: An error during initializing masakari controller: No option 'log_file' in section: 'log''
Feb 06 03:23:18  masakari(20458): ERROR: <type 'exceptions.OSError'>'
Feb 06 03:23:18  masakari(20458): ERROR: [Errno 17] File exists: '/var/log/masakari/''
Feb 06 03:23:18  masakari(20458): ERROR:   File "/opt/masakari/controller/masakari_controller.py", line 102, in init
os.makedirs(log_dir)
'
Feb 06 03:23:18  masakari(20458): ERROR:   File "/usr/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
'
any idea how to fix this? Thanks.

Hi Chtwerdna,
It seems that, masakari failed to init the log file directory and create the log file.
Could you please share the [log] section of your masakari.conf file.
You may retry after completely delete the "/var/log/masakari",
DO NOT FORGET to backup the previous log files before delete it.
And, if you have no strong reason to use this older version of Masakari,
I strongly recommend you to move to latest version of  Masakari in Openstack.

Hi sampathP,
Thanks for the hint.
the [log] section in my /etc/masakari/masakari-controller.conf only has one line.
[log]
log_level = info
After googling I found masakari has a sample file for this conf.
https://github.com/ntt-sic/masakari/blob/master/masakari-controller/etc/masakari-controller.conf.sample
(there is also a sample file in the directory: /etc/masakari/masakari-controller.conf.sample )
is these two file I found the missing configurations.
After adding the following two lines, the controller starts normally.
log_file = /var/log/masakari/masakari-controller.log
logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(threadName)s] %(message)s
By the way, I know that there's a latest version of masakari, but I'm using OpenStack mikata.
Also this is the first try to verify the function and behaviors so I just need a quick setup environment.
After this I may try the new masakari in Openstack.
Thanks again for both the hint and the recommendation.

Hi Chtwerdna,
As you found out, with log settings, this should work fine.
Even in openstack masakari, masakari-monitors and python-masakariclient,
you can find the samples of required config file and you can generate the full config file with tox.
Let us know if you need any help..

authors: chtwerdna sampathP

repo: ntt-sic/masakari-deploy


text:
🛑 Secret Site BIS is down

In 5fc4c09, Secret Site BIS ($SECRET_SITE) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Secret Site BIS is back up in 2938e0f after 25 minutes.


text:
🛑 Università e ricerca is down

In 1196e84, Università e ricerca (https://www.mur.gov.it/it) was down:

HTTP code: 403
Response time: 610 ms

Resolved: Università e ricerca is back up in da6fdc3 after 5 minutes.


text:
🛑 Difesa is down

In 43df89a, Difesa (http://www.difesa.it/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Difesa is back up in dd62a33.


text:
server Start to return error instead of calling log.Fatal()

In golang app, usually we handle the error at top level command. Sub function calls just return the error to the upper function for it to decide what to do with the error.
Can Numaflow UDF grpc server start method return the error instead of calling log.FatalF for error handling?
Example usage with Cobra
var myCmd = &cobra.Command{
	Use:          "my-cmd",
	RunE:         runQuery,
}

func init() {
	RootCmd.AddCommand(myCmd)
}

func handle(ctx context.Context, key string, d fn.Datum) fn.Messages {
	msg := d.Value()
	// If msg is not an integer, drop it, otherwise return it with "even" or "odd" key.
	if num, err := strconv.Atoi(string(msg)); err != nil {
		return fn.MessagesBuilder().Append(fn.MessageToDrop())
	} else if num%2 == 0 {
		return fn.MessagesBuilder().Append(fn.MessageTo("even", msg))
	} else {
		return fn.MessagesBuilder().Append(fn.MessageTo("odd", msg))
	}
}

func runQuery(cmd *cobra.Command, args []string) error {
	return server.New().RegisterMapper(fn.MapFunc(handle)).Start(context.Background())
}

If the request is just for cobra.Command, use Run instead of RunE, which does not need to return an error.
The Start function in the SDK is intended to be the top level - it is used to start a gRPC server, it does not need to have an upper level to manage it - imagine with a gRPC server start error thrown to upper level, what you can do other than printing the log and letting the container panic? The SDK is intended to simplify the UDF implementation, returning an error means the SDK users have to deal with the error.

I still think it's better to let the user handle the error by them self. They can ignore it if not care. But more robust applications have different ways of handling errors, like sending notifications and tracking errors in Sentry. If the sdk log.Fatalf then the app won't be able to handle such logic.

Another case is where the user want to abstract  numaflow away from the application code. They might be using multiple orchestration tool and only some functions will be handled by numaflow. They would create an interface and wrap around the numaflow-sdk. Not returning error won't let them handle the errors nicely

If you think it's better to not have user handling the error, I can create a separated function, like StartE and that one will return error. Does it sound good?

Makes sense, thanks!

authors: sandangel whynowy

repo: numaproj/numaflow-go


text:
NotImplementedError followed by LoweringError

I got this error message:
 LoweringError: Failed at object (object mode backend)
 make_function(name=$const0.31, code=<code object <listcomp> at 0x7f8da9c439c0, file "<ipython-
 input-13-d32eb8727d1e>", line 103>, closure=$0.29, defaults=None)

 File "<ipython-input-13-d32eb8727d1e>", line 103:
     def make_results(self, Xtest_vect, ytest):
         <source elided>

         relativ = pd.DataFrame([proba.loc[row].sort_values()[-2:][1]/proba.loc[row].sort_values()[-2:][0] for row in proba.index], index= ytest.index, columns=["Relativ"])
    ^
 [1] During: lowering "$0.32 = make_function(name=$const0.31, code=<code object <listcomp> at 0x7f8da9c439c0, file "<ipython-input-13-d32eb8727d1e>", line 103>, closure=$0.29, defaults=None)" at <ipython-input-13-d32eb8727d1e> (103)
 -------------------------------------------------------------------------------
 This should not have happened, a problem has occurred in Numba's internals._

The code I am running (within a class):
 @jit
 def degree_error(self, preds, ytest):
         error_deg = []
         for pred, real in zip(preds.values, ytest.values):
             if pred != real:
                 for ind, val in enumerate(pred[0]):
                     if val != real[0][ind]:
                         error_deg.append(8-ind)
                         break
                     else:
                         continue
             else:
                 error_deg.append(0)
                 continue

         return pd.DataFrame(error_deg, index=ytest.index, columns=['Grad av feil'])

 @jit
 def make_results(self, preds, virkelig):
      error_deg = self.degree_error(preds, virkelig)

Are there issues because of my code or is this something you can fix?
numba version: 0.39.0

Thanks for the report. Without a functioning reproducer it is hard to ascertain precisely what went wrong. However, speculating... It seems like the code is returning or creating a function in a manner that Numba cannot handle (make_function op code in use). Numba 0.40 has some diagnostics added that catch cases it cannot handle and report to the user as such. Perhaps upgrade your Numba installation to obtain better reporting, also there were also some really important fixes in liveness analysis for @jit in 0.40, without them those bugs might also be contributing to the problem reported.
For your specific code, it looks like there are sections of pure algorithmic code and then things like Pandas DataFrame creation. Numba will do a whole lot better performance wise if you can compile the algorithmic part under @jit(nopython=True) and then just pass the result to e.g. Pandas. The 5 minute guide to Numba is a good place to start for working out how to design your code so Numba can really help, then perhaps take a look at performance tips.
Hope this helps.

Closing this issue as it seems to be resolved. If this is not the case please re-open with a comment about any item that appears to be unresolved. Many thanks.

authors: christinekaush stuartarchibald

repo: numba/numba


text:
Copy prange loop header to after the parfor.

Resolves #7630
Numba can insert statements with side-effects into the prange loop header and expect those to be executed after the last loop iteration for correctness.  We copy the loop header after the parfor here then for correctness.

/azp run


text:
Goverance Model: HoloViz

Name: Qudirah Alimi
Project Assigned: HoloViz
Links: https://holoviz.org/community.html
https://holoviz.org/contributing.html
https://holoviz.org/
https://github.com/holoviz/holoviz/issues
HoloViz
The HoloViz project (formally known as PyViz) is managed using a shared governance model, which means that the project is open to contributions from anyone, regardless of background or experience. The governance model for HoloViz is based on transparency, inclusivity, and shared responsibility, with all members of the community having an equal say in the direction of the project. Contributions are evaluated based on their technical merit and alignment with the project's goals. Decisions are made through consensus, with all members of the community having an equal say in the direction of the project.  HoloViz’s tools include Panel, HvPlot, HoloViews, GeoViews, DataShader, and more.
The contributing page outlines the steps involved in contributing to HoloViz, including how to set up a development environment, how to submit a contribution, and how to get involved in the community. The governance model for HoloViz is based on transparency, inclusivity, and shared responsibility, with all members of the community having an equal say in the direction of the project. Contribution ranges from as low as making a simple edit in documentation, writing about the project in blog posts, or starting a discussion to forking the repository and writing codes. The benefits go as far as even landing a job. There is a discussion forum where everyone can ask and answer questions and communication is also carried out on GitHub in pull requests. A contributor is encouraged to open new issues and link PRs for contributions unless it is something as easy as a little documentation.
The HoloViz organization consists of two teams. The team that is in charge of contents and those in charge of attending to issues opened on the HoloViz project. They are referred to as HoloViz dev and Triaging team respectively. The data storage buckets are managed by Jean-Luc Steven(@jlstevens) and Philipp Rudiger (@philippjfr) and are both accessible on their GitHub pages using those handles. Phillip is also in charge of the HoloViz Discourse forum. Communication can be done via issues and PRs on GitHub, through PyViz Gitters, or through a meeting held openly and welcome to every contributor. A link to the google calendar can be easily found on the community page.
Finding and understanding the Governance Model was easy for me. It just took reading and digging a little deep. It was easy to find and understand the governance model.

@arliss-NF I am done. Please take a look and let me know what you think. Thank you <3


text:
NUnit3 result format output as nunit2 reports test dll as ignored if one or more child tests marked as ignored

@CharliePoole commented on Fri Sep 02 2016
@harrisonmeister commented on Fri Sep 02 2016
Hi,
I'm not sure if this is by design or an issue, so please feel free to close if by design.
I am running a test project using the NUNit3 console.exe, and I receive the following output:
<?xml version="1.0" encoding="utf-8" standalone="no"?>
<!--This file represents the results of running a test suite-->
<test-results name="D:\PATH_TO_LIBRARY\Some.Library.dll" total="2" errors="0" failures="0" not-run="1" inconclusive="0" ignored="1" skipped="0" invalid="0" date="2016-09-02" time="13:20:50">
  <environment nunit-version="3.4.1.0" clr-version="4.0.30319.42000" os-version="Microsoft Windows NT 10.0.10586.0" platform="Win32NT" cwd="D:\Git\Source" machine-name="REDACTEED" user="REDACTED" user-domain="REDACTED" />
  <culture-info current-culture="en-GB" current-uiculture="en-GB" />
  <test-suite type="Assembly" name="D:\PATH_TO_LIBRARY\Some.Library.dll" executed="False" result="Ignored">
    <properties>
      <property name="_PID" value="19472" />
      <property name="_APPDOMAIN" value="test-domain-" />
    </properties>
    <reason>
      <message><![CDATA[One or more child tests were ignored]]></message>
    </reason>
    <results>
      <test-suite type="TestSuite" name="A" executed="False" result="Ignored">
        <reason>
          <message><![CDATA[One or more child tests were ignored]]></message>
        </reason>
        <results>
          <test-suite type="TestSuite" name="B" executed="False" result="Ignored">
            <reason>
              <message><![CDATA[One or more child tests were ignored]]></message>
            </reason>
            <results>
              <test-suite type="TestSuite" name="C" executed="False" result="Ignored">
                <reason>
                  <message><![CDATA[One or more child tests were ignored]]></message>
                </reason>
                <results>
                  <test-suite type="TestFixture" name="D" executed="False" result="Ignored">
                    <reason>
                      <message><![CDATA[One or more child tests were ignored]]></message>
                    </reason>
                    <results>
                      <test-case name="A.B.C.D.Ignored_Test" executed="False" result="Ignored">
                        <properties>
                          <property name="_SKIPREASON" value="Ignored for some reason" />
                        </properties>
                        <reason>
                          <message><![CDATA[Ignored for some reason]]></message>
                        </reason>
                      </test-case>
                      <test-case name="A.B.C.D.Test_That_Works" executed="True" result="Success" success="True" time="0.027" asserts="0">
                        <reason>
                          <message><![CDATA[]]></message>
                        </reason>
                      </test-case>
                    </results>
                  </test-suite>
                </results>
              </test-suite>
            </results>
          </test-suite>
        </results>
      </test-suite>
    </results>
  </test-suite>
</test-results>

This marks the library test result as Ignored and not executed, even though one of the tests has run.
Specifically I have code which merged test result summaries (based on the nunit2 format). As NUnit2 (2.6.4) always added "time" and "asserts" as attributes, I'm just trying to work out if the result I am seeing in NUnit 3 is expected or not.
[TestFixture]
    public class ApisEstaCalculatorTests
    {
        [SetUp]
        public void SetUp()
        {

        }

        [Test]
        public void Test_That_Works()
        {
            Assert.Pass();
        }

        [Test, Ignore("Ignored for some reason")]
        public void Ignored_Test()
        {

        }
    }

Above is the sample code, and here is the associated console I am running
D:\packages\NUnit.ConsoleRunner\tools\nunit3-console.exe "--noheader" "--where=!(cat == 'Integration' || cat == 'IntegrationTest' || cat == 'IntegrationsTest' || cat == 'IntegrationTests' || cat == 'IntegrationsTests' || cat == 'Integration Test' || cat == 'Integration Tests' || cat == 'Integrations Tests' || cat == 'Approval Tests' || cat == 'AcceptanceTest' || cat == 'PerformanceTest')" "--labels=Off" "--timeout=900000" "--stoponerror" "--result=D:\TestResult-unit-64-0.xml;format=nunit2" "D:\PATH_TO_LIBRARY\Some.Library.dll"
My main issue is that
<test-suite type="Assembly" name="D:\PATH_TO_LIBRARY\Some.Library.dll" executed="False" result="Ignored"> doesnt contain the time or asserts attributes any more. Running against 2.6.4 console runner, it does.

@CharliePoole commented on Fri Sep 02 2016
It does seem like a bug to me. I'm moving it to the nunit-console project because it's the NUnit Engine, not the Framework, that translates nunit3 output to nunit2 format.
The XML produced by this translation is not going to be identical. We ensure that the format is correct, so that existing programs that process nunit2 XML files will be able to run. The semantics can obviously vary since NUnit 3 works differently from NUnit 2.
In this case, the biggest problem I see is that your assembly is shown as not having been executed. I suspect that's why there is no time and no assertion count - those attributes don't make sense if the assembly was never executed! But of course it was executed. :-)

@harrisonmeister commented on Fri Sep 02 2016
Hi @CharliePoole, thanks - yes that makes sense. I took a quick look at the NUnit2 writer and it looked like it omitted the attributes if the Test result was Skipped.
I'm in the process of upgrading a very large (1000+ projects) from Nunit 2.6.4 to 3.4.1 so the legacy format is very useful in leaving the test reporting side alone whilst I do the main upgrade (we had lots of uses of ExpectedException!)
I have also written a workaround that I will test on Monday to our test output which should also allow me to continue the upgrade
On a slight side note, we also use your nunit-summary executable. Do you have any guidance on how to "upgrade" to something more up-to-date when I take the plunge to remove the legacy v2 format from the result spec option on the command line?
Thanks, Mark

@CharliePoole commented on Sat Sep 03 2016
@harrisonmeister Another think I just learned about our V2 output is that it uses the V2 names for properties if you ran V2 tests under V3. If you run V3 tests but ask for V2 output, you get the V3 property names. Again, the format complies with your request but the content may differ.
This primarily affects properties Description (V2 _DESCRIPTION) and Categories (V2 _CATEGORIES). I don't think it's an error per se but it is a limitation. We could, of course build in more knowledge of the names of well-known properties.
Regarding nunit-summary... I haven't looked at it in ages. I'll take a look and see how much work an update would take. Are you interested in working on it or just using it. :-)

@harrisonmeister commented on Sun Sep 04 2016
@CharliePoole - I think for my scenario different property names is ok, since I am merging the test result xml files into a single file.
Re: nunit-summary, I'd be more than happy to work on it :-)

@CharliePoole commented on Sun Sep 04 2016
I did some updates to nunit-summary, changing the license, restructuring directories and updating it to use VS2015. I'll write an issue on that repo mentioning you and we can figure out how to proceed.

@theBFG commented on Thu Aug 24 2017
Hey, are there any updates or progress on this bug?

@ChrisMaddock commented on Thu Aug 24 2017
@theBFG This is acknowlegded as a bug and on our backlog, just needs someone to pick it up an put in a fix. A pull request would be welcome, if you're willing! 😄
@CharliePoole - this probably belongs in the nunit2-result-writer repo now, right? I'd guess it was moved here before we split the extensions out.

@rprouse commented on Sat Sep 16 2017
Agreed that this is probably a result writer issue. I am going to move it.

Picking this up after a long absence and examining the output I see that the problem reflects, at least in part, the way the NUnit 3 framework records this situation. Any ignored test is given the Skipped:Ignored' result, which propagates to each higher level test, including the assembly. The only distinction with the higher level tests is that they have siteset toChild`.
We discussed, quite a long time ago, using the Warning result in these cases, but we didn't do it because it would be a breaking change. I hope this will be re-examined for NUnit 4.
For the same reason (i.e. compatibility) I think we have to keep the same values here as in V2 itself. We should show the missing elements, just as V2 does, but we should also show the  same result as V2 even if it isn't logical to consider the assembly as not run.

:tada: This issue has been resolved in version 3.7.0 :tada:
The release is available on:

GitHub Release
NuGet Package
Chocolatey Package

Hi @CharliePoole , I am also facing similar issue. v3.7.0 still doesn't resolve the issue.

Thanks @CharliePoole , I read it through. I am running tests using nunit console runner with nunit3. I am getting results as
`

 One or more child tests were ignored

 One or more child tests were ignored

 One or more child tests were ignored

 One or more child tests were ignored

`

"For the same reason (i.e. compatibility) I think we have to keep the same values here as in V2 itself. We should show the missing elements, just as V2 does, but we should also show the same result as V2 even if it isn't logical to consider the assembly as not run."
The extension is designed to give (as close as possible) the same results as was given by the V2 Console Runner. In the earlier discussion, the issue was resolved by

Making sure that we show the time and assert count in the assembly element.
Continuing to show the assembly as ignored, just as the V2 console runner would show it.

Can you explain further why that resolution doesn't work for you?

Hi @CharliePoole , If you can see the attachment, the time and assert count is not showing up at assembly level. Please do needful, what I could be missing or let me know if you need any specific of my implementation info.

Hi @rprouse , could you please tell, where you able to resolve the issue that you actually reported?

authors: CharliePoole JanibashaShaikTR janijbsstudios rprouse


text:
Avoid double negative assertions

Description
Patterns like Expect(boolVal).ShouldNot(Equal(false)) or Expect(boolVal).ShouldNot(BeFalse()), the linter will suggest Expect(boolVal).Should(BeTrue())
Type of change

[x] New feature (non-breaking change which adds functionality)

How Has This Been Tested?

[x] Added test case and related test data
[x] Update the functional test

Checklist:

[x] I have performed a self-review of my own code
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I ran golangci-lint

Pull Request Test Coverage Report for Build 4468264479

22 of 22 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.5%) to 82.981%

Totals

Change from base Build 4468004751:
0.5%

Covered Lines:
668

Relevant Lines:
805

💛  - Coveralls

authors: coveralls nunnatsa

repo: nunnatsa/ginkgolinter


text:
Keeping completion menu always open

Currently it is possible to trigger opening the completion menu by a specific key binding like tab, but my usage is very completion heavy and I want to keep the completion menu always open for more discoverability and ease of use. Is it possible to configure reedline so that it keeps the completion menu always open, or reopen it after each key or each space? If not, would you accept a PR adding it?

I don't think it's possible right now. I'm always up for cool PRs as long as they're implemented well. I know that's vague but we're a little strict on what we accept here as to keep the maintenance easier. I'd say go for it, and we might accept it.

authors: HKalbasi fdncred

repo: nushell/reedline


text:
Documentation on images location

Can't figure out a proper images location.
We have:

images
assets/images
static/images
resources/_gen/images (ok, this one we can exclude)

Images are used in both static and assets. Static is mainly for SVG and native WEBP, while assets are images processed by Hugo Pipes. So the featured_image from frontmatter is processed by hugo pipes, while the header/nav would use the  from static. Does that make sense? My apologies, but from what I could tell, that is the way Hugo is built. I would say, if you want to just use webp for your site, then just use static. If you do want to use PNG files, use them from assets, and add on the image processing to convert to webp.

Makes sense, thank you! :)

authors: nusserstudios yuriizinets

repo: nusserstudios/tailbliss


text:
Bugfix: Added key to GetAuth function

Addresses https://github.com/nuxt-community/apollo-module/issues/132

thanks @timhanlon I'll check it out soon

authors: dohomi timhanlon

repo: nuxt-community/apollo-module