text:
Plugin doessn't recognize existing cluster and creates a duplicate

When I delete the jupyterhub container for a user, and then attempt to start the server, instead of recreating the container, it tries to create a new cluster. It is not finding the existing cluster and reusing it.
[E 2017-01-30 17:47:37.596 JupyterHub web:1548] Uncaught exception GET /jupyter/hub/user/carolynvs (108.166.30.187)
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.5/site-packages/tornado/web.py", line 1469, in _execute
        result = yield result
      File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 484, in get
        yield self.spawn_single_user(current_user)
      File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 306, in spawn_single_user
        yield gen.with_timeout(timedelta(seconds=self.slow_spawn_timeout), f)
      File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 245, in spawn
        raise e
      File "/opt/conda/lib/python3.5/site-packages/jupyterhub/user.py", line 226, in spawn
        yield gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
      File "/srv/jupyterhub/src/jupyterhub-carina/jupyterhub_carina/CarinaSpawner.py", line 151, in start
        cluster = yield self.create_cluster()
      File "/srv/jupyterhub/src/jupyterhub-carina/jupyterhub_carina/CarinaSpawner.py", line 169, in create_cluster
        return (yield self.carina_client.create_cluster(self.cluster_name))
      File "/srv/jupyterhub/src/jupyterhub-carina/jupyterhub_carina/CarinaOAuthClient.py", line 120, in create_cluster
        response = yield self.execute_oauth_request(request)
      File "/srv/jupyterhub/src/jupyterhub-carina/jupyterhub_carina/CarinaOAuthClient.py", line 244, in execute_oauth_request
        return (yield self.execute_request(request, raise_error))
      File "/srv/jupyterhub/src/jupyterhub-carina/jupyterhub_carina/CarinaOAuthClient.py", line 275, in execute_request
        return (yield http_client.fetch(request, raise_error=raise_error))
    tornado.httpclient.HTTPError: HTTP 413: Payload Too Large

The Carina API in this case is returning 413 Payload Too Large, indicating that I've hit my 3 cluster quota. If I had had fewer clusters, it would have made a duplicate cluster with the same name.

Oops, I should have realized this would be a problem! The old Carina would ignore requests to create a new cluster, if one already existed with the same name. Now that you can create multiple  clusters with the same name, I need to check for it and reuse the cluster if found.

repo: jupyterhub/jupyterhub-carina


text:
refactor(connector): [Square] change error message from NotSupported to NotImplemented

Type of Change

[ ] Bugfix
[ ] New feature
[ ] Enhancement
[x] Refactoring
[ ] Dependency updates
[ ] Documentation
[ ] CI/CD

Description
Consistent error messages for not implemented payment method.
Motivation and Context
Resolves  #2861
Checklist

[x] I formatted the code cargo +nightly fmt --all
[x] I addressed lints thrown by cargo clippy
[x] I reviewed the submitted code
[ ] I added unit tests for my changes where possible
[ ] I added a CHANGELOG entry if applicable

Hey @nain-F49FF806 , Thanks for PR.
But we'd prefer that contributors comment on an issue before opening a PR for it, so that other contributors are aware that you are working on it. This would also help reduce duplicate effort towards solving the same issue. Please ensure you follow it moving forward.

@swangi-kumari Noted.
Saw no one seemed assigned, so just went ahead to have a look at the code before making a comment.
I didn't know if I was going to take it, until I had already finished. Sorry. :sweat_smile:
On a more fortunate note, I only laid eyes on this today, So hopefully not much time wasted. :crossed_fingers:
Regards.

Hey @nain-F49FF806 ,
It's fine if you missed. But can you go to respective issue for which you raised this PR and comment on that issue, so that I can assign it to you.

Hey @nain-F49FF806 ,
Please resolve the conflicts from your branch.

@swangi-kumari Have rebased onto the most recent main.
Please have a look and let me know if there's anything else needed.

source: gharchive/pull-request

repo: juspay/hyperswitch

license_type: permissive

license_source: github-api


text:
[Client] Add support for zig client library

Description
Adding support for zig client for superposition. The steps to work with the client are mentioned in the README.md file in the client.

@NishantJoshi00 can we remove the merge commit? It breaks semantic guidelines here. Please rebase if possible

@NishantJoshi00 can we remove the merge commit? It breaks semantic guidelines here. Please rebase if possible

I have done that 👍 .
But, I am curious. What merge policy do you use?

rebase and merge/squash and merge, depending on the use-case

squash and merge

In this case, my commit messages shouldn't matter, only the PR title would be of significance, right?

@NishantJoshi00 yup

@juspay/sdk-backend can we start reviewing this?

repo: juspay/superposition


text:
Face detection don't work on AWS Lambda due to node-canvas not working on node 8.10

Hi there!
I am trying to get this up and running on AWS Lambda but are having trouble as it seem the 'canvas' package is broken on node 8.10. I am trying to get some face detection going.
https://github.com/Automattic/node-canvas/issues/1252#issuecomment-437598572
It looks like its kind of a pain to try and run any other version of node on AWS Lambda, so I started trying to use it without the canvas package. As you mention in the README and I also noticed that the faceapi.locateFaces() function also takes a Tensor4D class as input. I have never used Tensorflow and I am a little confused as how to turn a ArrayBuffer from axios into a correctly shaped Tensor4D object.
I am fetching a jpeg image using axios.
I found the tf.tensor4d function but not sure what shape and dtype it should be.
Do you have any idea?
My code so far:
const { data: imageBuffer } = await axios.get(url, {
	responseType: 'arraybuffer'
})
const imageTensor = tf.tensor4d(imageBuffer, [?, ?, ? ,?])
const faces = await detector.locateFaces(imageTensor)

Error messages look similar to this one:
Error: Based on the provided shape, [1,2,3,4], and dtype float32, the tensor should have 24 values but has 68695
Any help is greatly appreciated!

Hey man, hows it going?
Ohh that's a pity, node-canvas is obviously the easiest way to use face-api.js with node.
Well you can either pass in tf.Tensor3Ds or tf.Tensor4Ds with a batch size of 1:
const imageTensor = tf.tensor3d(imageBuffer, [height, width, channels]), where channels should be 3 and in RGB order.
However, the imageBuffer you receive from axios is most likely jpeg or png encoded right? You have to create the image tensor from the raw image data, so you will have to decode the data first. There are probably npm packages, which can do that for you. For example I have seen people using ffmpeg for this.

Thanks!
Tried the get-pixels npm package and it works!
get-pixels return a 4 channel array all the time, even for jpegs 🤷 , so had to remove the alpha channel.
const pixels = await new Promise<Pixels>(resolve => {
	getPixels(url, (err, pixels: Pixels) => {
		if (err) {
			throw err
		}
		console.log('pixels:', pixels)
		resolve(pixels)
	})
})

// remove alpha channel
const RGBValues = []
pixels.data.forEach((px, i) => {
	if ((i + 1) % 4 != 0) {
		RGBValues.push(px)
	}
})
const imageTensor = tf.tensor3d(RGBValues, [pixels.shape[1], pixels.shape[0], 3]) as any
const faces = await detector.locateFaces(imageTensor)

Now to the task of getting it up on Lambda without breaking their code size limit, wish me luck!

Thanks!
Tried the get-pixels npm package and it works!
get-pixels return a 4 channel array all the time, even for jpegs 🤷 , so had to remove the alpha channel.
const pixels = await new Promise<Pixels>(resolve => {
	getPixels(url, (err, pixels: Pixels) => {
		if (err) {
			throw err
		}
		console.log('pixels:', pixels)
		resolve(pixels)
	})
})

// remove alpha channel
const RGBValues = []
pixels.data.forEach((px, i) => {
	if ((i + 1) % 4 != 0) {
		RGBValues.push(px)
	}
})
const imageTensor = tf.tensor3d(RGBValues, [pixels.shape[1], pixels.shape[0], 3]) as any
const faces = await detector.locateFaces(imageTensor)

Now to the task of getting it up on Lambda without breaking their code size limit, wish me luck!

Thanks, that helped me. But it's quite slowly (1000ms) compared to canvas ( < 10ms ).

tensorflow team is working on a solution: https://github.com/tensorflow/tfjs/issues/298#issuecomment-442263569

Nice!

It wasn't easy, but I finally got it running in a Lambda on AWS.
The tfjs-node package includes a native library and has to be built using the correct environment. I achieved this using Docker and the lambci/lambda:nodejs8.10 image from the https://github.com/lambci/docker-lambda project. This image seem to be one the best images to simulate the Lambda environment.
The built tfjs-node module includes a 110mb binary file.
@tensorflow/tfjs-node/deps/lib/libtensorflow.so
The Lambda size limit is 250mb (uncompressed).
Well, with 140mb left for other modules it should be fine right?
I tried deploying the package using serverless deploy resulting in this cryptic error message.
ENOENT: no such file or directory, open '/Users/bobmoff/Projects/picular/serverless-face-api/node_modules/@tensorflow/tfjs-node/build/Release/libtensorflow.so'
The file exists alright, but its a symlink that points to
libtensorflow.so -> /var/task/node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow.so
Ahh, the symlink was created inside the docker container and it is a not a relative link. Ok, so i delete the symlink and created a new one that is relative.
libtensorflow.so -> ../../deps/lib/libtensorflow.so
But zipping up the package, using serverless deploy results in a zip that when uncompressed is 343mb ?? Using Disk Inventory X on the uncompressed folder I saw this:

Why are there 2 large binaries? Shouldn't one of them just be a symlink ? The symlink is gone, and replaced with a copy of the binary. Hmm.
After a bit of research I learned that the default behaviour when zipping symlinks is that they get "resolved" (link is followed and original content is copied). Ok. But there is a --symlinks flag for the zip program that can modify this behaviour to keep symlinks as they are, instead of resolving them. This solved the size issue, nut now I realised that I depend on a local project that is linked that actually need to be resolved when packaged. I couldn't find any way to specify what folder/file should be resolved or not when packaging through serverless. There is a way to exclude files/folders from package though. I excluded the symlink.
package:
    exclude:
        - node_modules/@tensorflow/tfjs-node/build/Release/libtensorflow.so

So using the following commands I first package the project (excluding the symlink) and then manually include the symlink again into the zip archive. (-y is the shorthand for --symlinks)
sls package --package my-artifacts
zip -y my-artifacts/serverless-face-api.zip node_modules/@tensorflow/tfjs-node/build/Release/libtensorflow.so

Now I have a package that is 233mb! Yeah, 17mb to spare! Hehe. Ofc there are more size reduction that can be made be excluding more stuff, like excluding unused weights/models from face-api etc.. But I was just happy to get below the limit.
Happy as a unicorn on christmas I deployed my neat "little" package to AWS.
sls deploy --package my-artifacts
.. and lived happily ever after.

libtensorflow.so appears to be 183mb now! don't suppose you could share your package.json for version numbers?

Sure, here are my deps, haven upgraded in a while :)
{
    "@tensorflow/tfjs-node": "^0.2.1",
    "axios": "^0.18.0",
    "face-api.js": "^0.17.1",
    "get-pixels": "^3.3.2",
    "lodash": "^4.17.11",
    "module-alias": "^2.1.0",
    "moment": "^2.23.0",
    "monk": "^6.0.6",
    "url-join": "^4.0.0"
}

The @tensorflow package (version 1.2.3) is now 294MB!  libtensorflow.so.1.14.0 is now 216MB, and libtensorflow_framework.so.1.14.0 is another 35MB. Pretty sure there is now way to get this package into Lambda.
@MatthewAlner did you manage to get it working with a newer version? Or do I need to roll all the way back to the versions used by @bobmoff

Perhaps you could try to deploy using https://www.openfaas.com/

I didn't in the end 😞

Thanks for all the helpful info in this issue, and of course thanks again to @justadudewhohacks . I did eventually manage to get it working on Lambda, but it wasn't easy. I ran into all the same problems that @bobmoff did, and if I'd read his post more carefully at the start I would have saved myself a lot of time (in particular the libtensorflow.so linking issue). I'm using the Node 10.x runtime.
The newer versions of tfjs-node just get bigger and bigger, especially their bundled .so files. There is no way I could find to make the newer versions fit within Lambda's 250MB limit, so I had to roll way back to v0.2.1. You also need to roll face-api.js back to a similarly old version, as it seems face-api.jsis coupled to a specific version oftfjs-node`. Below is my deps:
  "dependencies": {
    "@tensorflow/tfjs-node": "^0.2.1",
    "face-api.js": "^0.17.1",
    "get-pixels": "^3.3.2"
  }

If you really wanted to use the latest version of Tensorflow and Face-api, I suspect you could use a technique like this https://github.com/lucleray/tensorflow-lambda, which basically uploads a zipped copy of the dependencies, and then unzips them into /tmp before running them. There is obviously overhead for unzipping the files, but if your lambdas instances are often reused then that initial overhead fades away.

Hey @bobmoff, just wondering if you had any issues with memory leaks in your setup? My face detection lambdas get reused most of the time because they get called so often, and I've noticed the memory used slowly grows with each invocation, until it finally breaks the Lambda memory limit and starts again. No idea if it's get-pixels, tfjs-node or face-api.js that is the issue.

Howdy @henrydawson. No, sorry. I havent experienced that, but that is probably due to the fact that I havent had the pressure like you seem to have, so my containers/lambdas have probably been recycled before that happenes.

Hi @henrydawson and all,
just some feedback on canvas running on lambda and also a question.
I've managed to make canvas run on lambda by using https://github.com/lambci/docker-lambda to make some tests. Thanks to the "build" image I had to run a bash shell on the docker build image to:
1/ run npm install
2/ copy 3 libraries from the docker build image /lib64 to the root of my project: libblkid.so.1, libmount.so.1, libuuid.so.1
I'm now trying to use https://github.com/lucleray/tensorflow-lambda for tensorflow but with no luck.
My little project is based on one of the examples from the repository. It uses the common directory as in the examples directory and I've tryed to subsitute the import '@tensorflow/tfjs-node'; from the env.ts file with import "tensorflow-lambda" but it does not work.
It looks like tfjs-node have a mechanism to supersede the tfjs-core functions but this is not happening with tensorflow-lambda. Any hint on how I could make this happen ?
Thanks in advance !

@bobmoff i am new to this, and wanted to ask if you've got an example code to get started with face-api.js and AWS lambda?

@bobmoff i am new to this, and wanted to ask if you've got an example code to get started with face-api.js and AWS lambda?

Sorry. Not using this any more. But Lambda have upped the limit to 10GB for each function so size should not be a problem any longer.

@bobmoff I really wanna move this facial recognition to lambda. Any help would be highly appreciated. I tried with this version also
"dependencies": {
    "@tensorflow/tfjs-node": "^0.2.1",
    "face-api.js": "^0.17.1",
    "get-pixels": "^3.3.2"
  }

but still limit exceeding.

Are you reaching the max limit of 10gb ?
The issue I had was getting below 250mb.

Are you reaching the max limit of 10gb ?
The issue I had was getting below 250mb.

Going above deployment package unzipped limit size (250mb)

If you switch using containers you can use 10 gb.
https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/

Hi, I did get this working on Lambda a long time ago, but that was with older versions, and I ran into a lot of issues (especially the hard to resolve memory issues mentioned above). Eventually, I gave up on Lambda and just run it on EC2 using spot instances, which was cheaper and better for my workload. It was also simpler as I didn't have mess around with the file size issues.
That said, I recently used EFS with Lambda for some image processing that required extra space, and it was really easy to use. I imagine you could use EFS to get around all the size limits.
https://aws.amazon.com/blogs/compute/using-amazon-efs-for-aws-lambda-in-your-serverless-applications/

@bobmoff @henrydawson thank you both of you for pointing me in the direction.

@bobmoff I have successfully deployed in Lambda using containers. But I'm facing one issue. I'm just doing faceapi.detectSingleFace().withFaceLandmarks().withFaceDescriptor() in lambda to detect single face and returning that output as a response via API Gateway. I can't able to return without doing JSON.stringify(result) and If I do stringify the result, that result not working in final comparison. I dont know what I'm doing wrong.

source: gharchive/issue

authors: MatthewAlner bobmoff henrydawson justadudewhohacks lpsBetty nasr18 truff77 wangsijie

license: MIT


text:
Add iubenda.com

Added iubenda.com entry to sites.json

Hi @MrHeliox  , I also made a PR request, but it seems this repo isn't maintained anymore. Luckily an active fork can be found here: https://github.com/jdm-contrib/jdm

Ok thanks !

authors: MrHeliox mheesters

license: mit

license_source: bigquery


text:
Fix analyser issues on testing framework code

Summarise the changes this Pull Request makes.
Fix various analyser issues on testing framework code
Please include a reference to a GitHub issue if appropriate.
#401

I wanted to enable most analyser rules on the code, but it seems that we aren't ready for that yet. Getting a clean bill of health from the analyser on e.g. JustSaying.csproj is when it starts analysing JustSaying.TestingFramework, and when that's clear then the 250+ messages come in from the unit tests.
It might be possible soon to enable analysers with only a few rules disabled, on the non-test projects.

There's also a level of pragmatism to apply to analyzers with regards to the test code. For example, requiring ConfigureAwait(...) just adds visual noise. I think it's fine for the test infrastructure parts, but in the tests themselves it's not necessary.
In this lib I turned off that rule, plus the ban on underscores and something else that I can't remember what it is off the top of my head.
https://github.com/justeat/httpclient-interception/blob/6756febbd23879077bea037c24a5b7014c8b4fc9/tests/HttpClientInterception.Tests/JustEat.HttpClientInterception.Tests.csproj#L4

Right, what I did here Is to use a second, more forgiving ruleset for the tests.

I find one rule set with overrides per-project via NoWarn easier to maintain than files which are 90% the same, but whichever.

Suggest running Build.ps1 locally before committing to check it actually compiles, rather than relying on the CI to do it.

authors: AnthonySteele martincostello

repo: justeat/JustSaying


text:
🛑 Misskey is down

In e7322df, Misskey (https://mkacg.social) was down:

HTTP code: 530
Response time: 156 ms

Resolved: Misskey is back up in a5e4408 after 10 minutes.


text:
Add JustList: {JustList name}

Before submission deletes this line:
THIS IS NOT A TOKEN LISTING REQUEST FORM. IF YOU DO NOT FOLLOW THE FORMAT OR MAKE A GENERIC TOKEN REQUEST YOUR ISSUE WILL BE DELETED WITHOUT COMMENT
YOUR JUSTLIST MUST FOLLOW THE JSON SPECIFICATION
https://github.com/justswaporg/justlists/example.justlists.ts
Checklist

[x] I understand that this is not the place to request a token listing.
[x] I have tested that my JustList is compatible by pasting the URL into the add a list UI at justswap.org.
[x] I understand that filing an issue or adding liquidity does not guarantee addition to the justlists website.

Please provide the following information for your token.
JustList URL must be HTTPS.
JustList URL:
JustList Name:
Link to the official homepage of the JustList manager:

Sorry, your issue will be closed as you did not submit your information in the correct format.

repo: justswaporg/justlists


text:
🛑 Main Website is down

In e115651, Main Website (https://tecnoscientifica.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Main Website is back up in ef82ab8.

repo: justudin/tsp-status


text:
initial code

Some notes

Borrowed some code from https://github.com/atom/markdown-preview because we want a similar behavior w.r.t. to opening a new tab in the right pane when running a juttle or switching to an existing tab if the juttle has already been run.
I couldn't get the styles from juttle-client-library to load properly using an import so copied them from node_modules into styles and checked it in.

@mnibecker

Before merge:
I would use 0.7.0 jcl instead, deconstruct your view object on close, and also check for inputs in a program, and also fix the annoying dropdown css things

@mnibecker addressed your feedback, take a look? (will squash afterwards)

+1

authors: go-oleg mnibecker

repo: juttle/atom-juttle-viewer


text:
authors: add initial AUTHORS.md

This includes the list of current project contributors as well as
all of the people who contributed to the project before the project
became open source.

@dmehra

+1

authors: demmer dmehra

repo: juttle/juttle


text:
QueenParseTreeVisitor.java: Finish implementing reference...

The puzzle 1964704442 originating from #49 has to be resolved:
https://github.com/jvmqueen/queen-of-java/blob/a57821a29b40406177b70a9b243fccf2a35eb56a/src/main/java/org/queenlang/transpiler/QueenParseTreeVisitor.java#L864-L864
"Finish implementing reference types with array types.".
The puzzle was created by amihaiemil at 2022-12-31 19:11:52 +0200.
Estimation is 60 minutes.
If you have any technical questions, don't ask me, I won't be able to help. Open new issues instead.

@amihaiemil this is your task now, please go ahead. Deadline (when this ticket should be closed) is 2023-01-10T17:20:34.813658.
Estimation here is 60 minutes, that's how much you will be paid.
Remember, you don't have to solve everything in this ticket - you can solve it partially and leave todo markers in the code, which will become future tasks.
If you have any questions don't ask me, I'm not a technical person. Open new tickets instead.
If you don't have time or simply don't want to work on this, you can always resign.

@amihaiemil Don't forget to close this ticket before the deadline (2023-01-10T17:20:35). You are past the first half of the allowed period.

Puzzle disappeared from the code, that's why I closed this ticket.

@amihaiemil thank you for resolving this ticket. I've just added it to your active invoice. You can always check all your invoices and more on the Contributor Dashboard.

repo: jvmqueen/queen-of-java


text:
Subdomain 502 Bad gateway

Using any subdomain gives me a 502 Bad gateway screen. Im new at using this tool, but have checked the certs folder, and it looks like all certs for my domain (and new subdomains) are the same key. Any tips on getting LetsEncrypt to work with subdomains?

I'm having the same issue. The wildcard crt and key are readable in the container, it redirects from http to https, and then gives:
nginx.1    | 2017/09/15 01:55:25 [error] 37#37: *1 SSL_do_handshake() failed (SSL: error:1408F10B:SSL routines:ssl3_get_record:wrong version number) while SSL handshaking to upstream, client: 47.180.[redacted].[redacted], server: jira.my.network, request: "GET /list HTTP/2.0", upstream: "https://172.20.0.4:8080/list", host: "jira.my.network"

The certs are included (from /etc/nginx/conf.d/default.conf):
server {
server_name jira.my.network;
listen 443 ssl http2 ;
...
ssl_certificate /etc/nginx/certs/my.network.crt;
ssl_certificate_key /etc/nginx/certs/my.network.key;
...

Look at my issue https://github.com/jwilder/nginx-proxy/issues/930 , One of your issues is likely that proxy pass is pulling from your service via https and not https.

I've been intermittently having the same problem and have found this workaround which resolves the issue, until it surfaces again:

stop nginx, nginx-gen and ngingx-letsencrypt,
delete the volume for /etc/nginx/conf.d/ (keep all other volumes including certs).
bring everything up again

Voila (at least for my setup). I wish I knew how to reproduce this issue, but have yet to figure out what triggers it.
My docker-compose.yml file is based on https://github.com/jwilder/nginx-proxy/blob/master/docker-compose-separate-containers.yml, the only addition is the letsencrypt service:
  nginx-letsencrypt:
    image: jrcs/letsencrypt-nginx-proxy-companion
    container_name: nginx-letsencrypt
    restart: always
    volumes:
      - nginx-certs:/etc/nginx/certs/:rw
      - /var/run/docker.sock:/var/run/docker.sock:ro
    volumes_from:
      - nginx
    environment:
      NGINX_DOCKER_GEN_CONTAINER: 'nginx-gen'

This bug was actually getting so annoying that I created a small script which does the above 3 steps for me:
#!/bin/bash

# name of current directory
project_name=${PWD##*/}
# remove hyphen
project_name=${project_name/[-]}
# name of the volume which holds nginx configuration files
volume_to_delete=${project_name}_nginx-conf

# Stop proxy services (nginx, nginx-gen, nginx-letsencrypt)
$(docker-compose stop| tee /dev/tty)

# Delete the `nginx-conf` volume, this should force `nginx-gen` to create a new one from scratch, and
# that seems to fix the problem, until it surfaces again...
docker volume rm $volume_to_delete

# Bring proxy back up
$(docker-compose up -d| tee /dev/tty)

Hello everyone . I need your help . I tried all the solutions proposed in the nginx-proxy docker and alternative solutions on google, nothing changes.
How to fix this problem.
i have 2 servers :

S1: Docker CE:
Nginx-proxy
Letsencryp
Emby
Ombi (I had problems to validate it)
Deluge (web access impossible functional daemon)
(I did not manage to install organizr, and/or heimdall)
OS: Ubuntu 18.04 lts
    S2: Docker CE:
    Nginx-proxy
    Letsencryp
    Subsonic
    Transmission
    (I was unable to install organizr and/or heimdall)
    OS: Ubuntu 16.04 lts

`nginx.1    | 2018/12/12 10:46:12 [error] 221#221: *8607 no live upstreams while connecting to upstream, client: 77.147.234.230, server: heimdall.nomdedomain.com, request: "GET / HTTP/2.0", upstream: "http://heimdall.nomdedomaine.com/", host: "heimdall.nomdedomaine.com"
nginx.1    | heimdall.nomdedomaine.com 77.147.234.230 - - [12/Dec/2018:10:46:12 +0000] "GET / HTTP/2.0" 502 173 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
nginx.1    | 2018/12/12 10:46:13 [error] 221#221: *8607 no live upstreams while connecting to upstream, client: 77.147.234.230, server: heimdall.nomdedomaine.com, request: "GET /favicon.ico HTTP/2.0", upstream: "http://heimdall.nomdedomaine.com/favicon.ico", host: "heimdall.nomdedomaine.com"
nginx.1    | heimdall.nomdedomaine.com 77.147.234.230 - - [12/Dec/2018:10:46:13 +0000] "GET /favicon.ico HTTP/2.0" 502 173 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
nginx.1    | ombi.nomdedomaine.com 77.147.234.230 - - [12/Dec/2018:10:46:13 +0000] "GET /favicon.ico HTTP/2.0" 404 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
nginx.1    | 2018/12/12 10:49:27 [error] 221#221: *8612 upstream prematurely closed connection while reading response header from upstream, client: 77.147.234.230, server: deluge.nomdedomaine.com, request: "GET / HTTP/2.0", upstream: "http://172.17.0.4:8112/", host: "deluge.nomdedomaine.com"
nginx.1    | deluge.nomdedomaine.com 77.147.234.230 - - [12/Dec/2018:10:49:27 +0000] "GET / HTTP/2.0" 502 173 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
nginx.1    | 2018/12/12 10:49:27 [error] 221#221: *8612 upstream prematurely closed connection while reading response header from upstream, client: 77.147.234.230, server: deluge.nomdedomaine.com, request: "GET /favicon.ico HTTP/2.0", upstream: "http://172.17.0.4:8112/favicon.ico", host: "deluge.nomdedomaine.com"

my docker
docker run -d -p 80:80 -p 443:443 \
      --name nginx-proxy\
      --restart always \
      -e DEFAULT_HOST=nomdedomain.com \
      -v /home/user/docker/nginx-proxy/certs:/etc/nginx/certs:ro \
      -v /home/user/docker/nginx-proxy/dhparam:/etc/nginx/dhparam \
      -v /etc/nginx/vhost.d \
      -v /etc/nginx/conf.d \
      -v /usr/share/nginx/html \
      -v /var/run/docker.sock:/tmp/docker.sock:ro \
      --label com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy \
       jwilder/nginx-proxy:latest

I did the max test the container are ok, the ports are ok (I think)
I tried this tutorial but I can not do that the file is read loading the container.
https://wiki.ssdt-ohio.org/display/rtd/Adjusting+nginx-proxy+Timeout+Configuration
Any help is welcome . Thank you all

authors: dasmedium geirgp mabushey renfyld

repo: jwilder/nginx-proxy


text:
Varnish Cache and SSL termination

I would like to use nginx-proxy with a Varnish cache layer in front of it. In my normal nginx configuration I listen to port 443 and simply proxy to port 80 where Varnish will listen.
Could you implement this feature?

In my normal nginx configuration I listen to port 443 and simply proxy to port 80 where Varnish will listen.

In this scenario, you can replace nginx with nginx-proxy.  When you start your Varnish container to listen on port 80, as long as nginx-proxy has the SSL certificate for the give hostname, it will listen on 443 and send the requests back to Varnish on 80 by default.

I would like to use nginx-proxy with a Varnish cache layer in front of it.
This is the opposite of the other sentence.  If you want Varnish in front if nginx-proxy, just add it as backend in VCL.

In my normal nginx configuration I listen to port 443 and simply proxy to port 80 where Varnish will listen.

In this scenario, you can replace nginx with nginx-proxy. When you start your Varnish container to listen on port 80, as long as nginx-proxy has the SSL certificate for the give hostname, it will listen on 443 and send the requests back to Varnish on 80 by default.

I would like to use nginx-proxy with a Varnish cache layer in front of it.
This is the opposite of the other sentence.  If you want Varnish in front if nginx-proxy, just add it as backend in VCL.

Varnish is the last piece component I am trying to put together in my Wordpress Stack.
I ran varnish by running:
`
nginx-proxy:
image: jwilder/nginx-proxy
container_name: nginx-proxy
restart: always
ports:
- "443:443"
volumes:
- conf:/etc/nginx/conf.d
- vhost:/etc/nginx/vhost.d
- html:/usr/share/nginx/html
- dhparam:/etc/nginx/dhparam
- ./certs:/etc/nginx/certs:ro
- /var/run/docker.sock:/tmp/docker.sock:ro
networks:
  - nginx-proxy

varnish:
image: varnish
container_name: varnish
# depends_on:
#   - nginx-proxy
restart: always
volumes:
- ./default.vcl:/etc/varnish/default.vcl
ports:
- "80:80"
expose:
- 80
- 443
networks:
- nginx-proxy
- internal
./default.vcl
vcl 4.0;
backend sentrytire {
.host = "172.19.0.3";
.port = "80";
}
sub vcl_recv {
}
sub vcl_backend_response {
set beresp.ttl = 10s;
set beresp.grace = 1h;
}
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = "cached";
} else {
set resp.http.X-Cache = "uncached";
}
}
I confirm if checking curl -IL http://example.com, I am getting varnish cache working fine.
HTTP/1.1 200 OK
Server: nginx
Date: Sat, 14 Sep 2019 00:35:52 GMT
Content-Type: text/html; charset=UTF-8
Vary: Accept-Encoding
X-Powered-By: PHP/7.3.6
Link: http://example.com/wp-json/; rel="https://api.w.org/"
Link: http://example.com/; rel=shortlink
X-Varnish: 65540 32777
Age: 1
Via: 1.1 varnish (Varnish/6.2)
X-Cache: cached
Accept-Ranges: bytes
Content-Length: 168575
Connection: keep-alive
`
Now here come the hard part setting up jwilder/nginx-proxy to reverse proxy port 443 to varnish on port 80.
By default, its nginx.tmpl will generate the /etc/nginx/conf.d/default.conf. I follow some tutorial to edit the SERVER part of example.com to
location / {     # NETWORK nginx-proxy IP of Varnish Container is 172.18.0.4     proxy_pass http://172.18.0.4:80; }
But doing service nginx restart will reset /etc/nginx/conf.d/default to its nginx.tmpl file.
So I am thinking to edit the nginx.tmpl but i have no idea where the file reside. I also would like to run multiple container with multiple domain to cache thru 1 varnish container. I still have question mark on that. If anyone has successfully setting up this stack, I really appreciate your inputs and answers on this.

following.  I'm also trying to get this scenario working where I have one varnish container with to cache all my wordpress and send back to nginx for https to end user.

authors: Bram-Zijp jfrancais kamermans kimlonglecis


text:
Placeholder not updating

I am trying to update the placeholder but it is not working. any solution please
"dependencies": { "@react-native-community/async-storage": "^1.6.1", "@react-native-community/netinfo": "^4.2.1", "axios": "^0.19.0", "jetifier": "^1.6.4", "jsonwebtoken": "^8.5.1", "jwt-decode": "^2.2.0", "native-base": "^2.13.8", "pubnub-react": "^1.3.2", "query-string": "^6.8.3", "react": "16.8.6", "react-native": "0.60.4", "react-native-calendars": "^1.212.0", "react-native-camera": "git+https://git@github.com/react-native-community/react-native-camera.git", "react-native-chart-kit": "^3.3.1", "react-native-check-box": "^2.1.7", "react-native-datepicker": "^1.7.2", "react-native-document-picker": "^3.2.4", "react-native-gesture-handler": "^1.4.1", "react-native-gifted-chat": "^0.11.3", "react-native-image-picker": "^1.1.0", "react-native-linear-gradient": "^2.5.6", "react-native-progress": "^3.6.0", "react-native-push-notification": "^3.1.9", "react-native-qrcode-svg": "^5.2.0", "react-native-select-multiple": "^2.1.0", "react-native-svg": "^9.9.3", "react-native-table-component": "^1.2.1", "react-native-tag-input": "0.0.21", "react-native-vector-icons": "^6.6.0", "react-native-video": "^5.0.2", "react-native-video-controls": "^2.2.3", "react-native-webview": "^7.0.5", "react-navigation": "^3.12.1", "react-redux": "^7.1.1", "redux": "^4.0.4", "redux-logger": "^3.0.6", "redux-promise": "^0.6.0", "redux-thunk": "^2.3.0", "socket.io-client": "^2.3.0", "uuid": "^3.3.3" },

The placeholder is set through the inputProps param. I haven't tried updating the placeholder before, but you'd need to make sure the inputProps param itself changes, since TagInput is a PureComponent. If you replace the inputProps parameter with a different one that contains a different placeholder param, and the placeholder doesn't change, it could be a bug in TextInput.
Unfortunately I won't be able to help you further unless I can reproduce the issue you're describing. If you can provide an MCVE that reproduces the issue, in the form of an Expo Snack or a react-native init'd repo hosted on GitHub, I can try and investigate it further.

I found the solution to change the placeholder thanks for the reply.

authors: Ashoat gauravsbagul


text:
Fix NoMethodError when a 2 segment token is missing 'alg' header

This PR fixes an issue that was introduced at https://github.com/jwt/ruby-jwt/pull/425
Currently if one passes a token of 2 segments without alg header the decode method raises NoMethorError because algorithm is nil but it tries to call casecmp on it here:
https://github.com/jwt/ruby-jwt/blob/master/lib/jwt/decode.rb#L116
The expected result would be to raise an exception that is inherited from JWT::DecodeError instead of a generic exception like NoMethodError.

Great catch and nice with additional tests to ensure this behaviour in the future.
Added a comment about the added condition. Would suggest making the check more generic to avoid checking for a certain method.

@anakinj updated

Looks great. Would you be so kind and add a changelog entry to the CHANGELOG.md under the fixes?

@anakinj done

Super! Thank you for the effort fixing this!❤

authors: anakinj cmrd-senya

repo: jwt/ruby-jwt


text:
how to choose cuda for compiling in pytorch-1.0 branch?

Hi, thanks for your work.
Would you please tell me how to change my CUDA_PATH for compiling in pytorch-1.0
I have two CUDA (9.0 and 9.2). But the one that I want to use is not saved in /usr/local/cuda/.
What should be modified in setup.py to change the CUDA_PATH. It always uses the CUDA in /usr/local/cuda/ by default.

`export LD_LIBRARY_PATH=/path/cuda/cuda-10.0/cuda/lib64:/path/cuda/cuda-10.0/cudnn/v7.5.0/lib64:$LD_LIBRARY_PATH
export CUDA_HOME=/path/cuda/cuda-10.0/cuda
export CUDA_TOOLKIT_ROOT_DIR=$CUDA_HOME`

export LD_LIBRARY_PATH=/path/cuda/cuda-10.0/cuda/lib64:/path/cuda/cuda-10.0/cudnn/v7.5.0/lib64:$LD_LIBRARY_PATH
export CUDA_HOME=/path/cuda/cuda-10.0/cuda
export CUDA_TOOLKIT_ROOT_DIR=$CUDA_HOME

thanks

export LD_LIBRARY_PATH=/path/cuda/cuda-10.0/cuda/lib64:/path/cuda/cuda-10.0/cudnn/v7.5.0/lib64:$LD_LIBRARY_PATH
export CUDA_HOME=/path/cuda/cuda-10.0/cuda
export CUDA_TOOLKIT_ROOT_DIR=$CUDA_HOME

could you please tell me where to set these words?

export LD_LIBRARY_PATH=/path/cuda/cuda-10.0/cuda/lib64:/path/cuda/cuda-10.0/cudnn/v7.5.0/lib64:$LD_LIBRARY_PATH
export CUDA_HOME=/path/cuda/cuda-10.0/cuda
export CUDA_TOOLKIT_ROOT_DIR=$CUDA_HOME

could you please tell me where to set these words?

in faster-rcnn.pytorch/lib/make.sh

thanku but i run pytorch1.0
There is no make.sh file here
------------------ 原始邮件 ------------------
发件人: "YunqiuXu"<notifications@github.com>;
发送时间: 2019年12月10日(星期二) 晚上7:12
收件人: "jwyang/faster-rcnn.pytorch"<faster-rcnn.pytorch@noreply.github.com>;
抄送: "jimmyfa"<1123701799@qq.com>; "Comment"<comment@noreply.github.com>;
主题: Re: [jwyang/faster-rcnn.pytorch] how to choose cuda for compiling in pytorch-1.0 branch? (#605)
export LD_LIBRARY_PATH=/path/cuda/cuda-10.0/cuda/lib64:/path/cuda/cuda-10.0/cudnn/v7.5.0/lib64:$LD_LIBRARY_PATH
export CUDA_HOME=/path/cuda/cuda-10.0/cuda
export CUDA_TOOLKIT_ROOT_DIR=$CUDA_HOME
could you please tell me where to set these words?
in faster-rcnn.pytorch/lib/make.sh
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

export LD_LIBRARY_PATH=/path/cuda/cuda-10.0/cuda/lib64:/path/cuda/cuda-10.0/cudnn/v7.5.0/lib64:$LD_LIBRARY_PATH
export CUDA_HOME=/path/cuda/cuda-10.0/cuda
export CUDA_TOOLKIT_ROOT_DIR=$CUDA_HOME

could you please tell me where to set these words?

in faster-rcnn.pytorch/lib/make.sh

i run with pytorch1.0,there is no make.sh here  >-<

Maybe you can write and run a shell script by yourself like this:
export XXXX
python setup.py build develop
or just try to export these things using command line, before run python setup.py build develop

Maybe you can write and run a shell script by yourself like this:
export XXXX
python setup.py build develop
or just try to export these things using command line, before run python setup.py build develop

emmm其实我不太会

authors: Songtingt XuYunqiu junqiangwu


text:
Activate on Enter

Most of the comment documentors activate the comment block once the user presses Enter instead of when they type ///.
For me, I like making comment blocks that look like this:
///////////////////////////////////////////////////////////////////////////
/// Comment block that sticks out!
///////////////////////////////////////////////////////////////////////////

When I do so it attempts to make a document block.
So if you could make it so when you pressed ///Enter it gets activated instead of the current way, that would be awesome!

v.0.0.6
settings.json (Code > Preferences > Workspace Settings)
{
    "docomment.activateOnEnter": true
}

authors: TheColorRed k--kato

repo: k--kato/vscode-docomment


text:
PR to Support Netbox 3.3

This is my first attempt at a contribution to this plugin. We are deploying ISIS/MPLS-SR currently, and this plugin is very helpful in documenting our network.
So here goes...
Netbox recently refactored the API Serializers. The BGP Plugin needs these updated to function.
I was able to test this in my development instance here. With a version bump in the init.py, the plugin seems to work after this change.

Hi, we have recently upgrade to netbox3.3 and BGP plugin prevent netbox from starting up.
I will be grateful if this issue can be investigated and resolved as soon as possible.

Hello all, we have the same problem. I would appreciate if you can resolve it as soon as possible

FWIW, you can take the changes in this PR and add them to your production instance, it should get it running again. YMMV.

At a glance it looks like this change should also be backwards compatible with version 3.2.x although I haven't verified yet.

Thanks for raising this PR, I was going to raise the same PR.
You have missed a couple of things;

The version number in the README: https://github.com/k01ek/netbox-bgp/blob/main/README.md?plain=1#L20
The version number in the Makefile: https://github.com/k01ek/netbox-bgp/blob/main/Makefile#L2

I have run this against NetBox 3.3.2 and 3.2.9 and both are working for me.

Made those updates, wasn't sure the next version number was for me to decide, since I am not the owner of this.

#106

authors: Omripresent fahimeh2010 jwbensley k01ek kvedder-amplex wtayyeb


text:
Panic may occur during load testing

Although not known to be reproducible, the following error occurred in some cases.
$ runn loadt --load-concurrent 3 --duration 20s tests/**/*.yml
panic: runtime error: index out of range [7] with length 7

goroutine 26 [running]:
github.com/k1LoW/runn.(*operator).recordAsMapped(...)
        /Users/k2tzumi/go/pkg/mod/github.com/k1!lo!w/runn@v0.69.1/operator.go:374
github.com/k1LoW/runn.(*operator).recordNotRun(0x140aa154780, 0x103f8fee0?)
        /Users/k2tzumi/go/pkg/mod/github.com/k1!lo!w/runn@v0.69.1/operator.go:343 +0x370
github.com/k1LoW/runn.(*operator).runInternal(0x140aa154780, {0x103fb0778, 0x14002caa000})
        /Users/k2tzumi/go/pkg/mod/github.com/k1!lo!w/runn@v0.69.1/operator.go:962 +0x53c
github.com/k1LoW/runn.(*operator).run(0x140aa154780, {0x103fb0778?, 0x14002caa000})
        /Users/k2tzumi/go/pkg/mod/github.com/k1!lo!w/runn@v0.69.1/operator.go:808 +0x144
github.com/k1LoW/runn.(*operators).runN.func1()
        /Users/k2tzumi/go/pkg/mod/github.com/k1!lo!w/runn@v0.69.1/operator.go:1283 +0x138
github.com/k1LoW/concgroup.(*Group).Go.func1()
        /Users/k2tzumi/go/pkg/mod/github.com/k1!lo!w/concgroup@v1.0.0/concgroup.go:37 +0xc8
golang.org/x/sync/errgroup.(*Group).Go.func1()
        /Users/k2tzumi/go/pkg/mod/golang.org/x/sync@v0.1.0/errgroup/errgroup.go:75 +0x5c
created by golang.org/x/sync/errgroup.(*Group).Go
        /Users/k2tzumi/go/pkg/mod/golang.org/x/sync@v0.1.0/errgroup/errgroup.go:72 +0xa4

Thank you for your report!

We are looking for reproduction codes.

If we can reproduce it in past versions, we will try to verify it in the latest version.

The following commands also caused errors in case of runn version 0.70.1
$ runn run --verbose --shard-n 4 --shard-index 0 tests/**/*.yml

.. snip..

panic: runtime error: index out of range [2] with length 2

goroutine 1245 [running]:
github.com/k1LoW/runn.(*operator).recordAsMapped(...)
	/Users/runner/work/runn/runn/operator.go:374
github.com/k1LoW/runn.(*operator).recordNotRun(0xc034f00d20, 0xc00047c7a0?)
	/Users/runner/work/runn/runn/operator.go:343 +0x325
github.com/k1LoW/runn.(*operator).runInternal(0xc034f00d20, {0x1db5e78, 0xc0352521e0})
	/Users/runner/work/runn/runn/operator.go:973 +0x74c
github.com/k1LoW/runn.(*operator).run(0xc034f00d20, {0x1db5e78?, 0xc0352521e0})
	/Users/runner/work/runn/runn/operator.go:808 +0x23e
github.com/k1LoW/runn.(*operators).runN.func1()
	/Users/runner/work/runn/runn/operator.go:1283 +0x26d
github.com/k1LoW/concgroup.(*Group).Go.func1()
	/Users/runner/go/pkg/mod/github.com/k1!lo!w/concgroup@v1.0.0/concgroup.go:37 +0x96
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/Users/runner/go/pkg/mod/golang.org/x/sync@v0.1.0/errgroup/errgroup.go:75 +0x64
created by golang.org/x/sync/errgroup.(*Group).Go
	/Users/runner/go/pkg/mod/golang.org/x/sync@v0.1.0/errgroup/errgroup.go:72 +0xa5
Error: Process completed with exit code 2.

If an error occurs during the execution of the loop portion, it appears to be an Issue error, and the original error may be hidden?


text:
[WIP] Pruned transducer stateless5 recipe for AISHELL4

I am trying to build a pruned-transducer-stateless5 recipe for AISHELL4.

Here is a problem that I want to ask for some suggestions on.
In aishell4,  the text includes the unit <sil>, such as:
呃<sil>一个惊喜吧呃对合影惊喜自己当作是留念然后
那就可以让部长申两个活动，一个是<sil>联欢会的一个加分儿的项目，还有一个就是<sil>
那由D同学做一个P P T吧，他是<sil>学计算机的，然后就让他

So, I need to process these texts which include <sil>.
The first choice for me is to remove the <sil> when generating text. And also I need to remove it in the testing text.
The second choice is to treat it as a single modeling unit, being the same as a char unit.
I am not sure which is better, or maybe there are any other better good ideas for this case.

Are there recipes in other frameworks? What are others doing about it?

A good idea.

Do we accept perl codes for processing text?

I think maybe it's better if I change it to python format.

Do we accept perl codes for processing text?

If there are only several lines of perl script that can be embedded in prepare.sh, that would be fine.
Otherwise, I recommend you to use Python.

With num_workers=0 it has to be slow, that's expected.

When I use num-workers=2 to run again based on one GPU, there are some training logs:
(k2-python) luomingshuang@de-74279-k2-train-2-0602201035-5fb6d86964-mclm7:~/codes/icefall-pruned-rnnt5-aishell4/egs/aishell4/ASR$ CUDA_VISIBLE_DEVICES='4' python pruned_transducer_stateless5/train.py --max-duration 220 --num-workers 2 --exp-dir pruned_transducer_stateless5/exp-one-gpu
2022-06-08 07:55:57,171 INFO [train.py:877] Training started
2022-06-08 07:55:57.403222: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /ceph-sh1/fangjun/software/cuda-10.2.89/lib:/ceph-sh1/fangjun/software/cuda-10.2.89/lib64:
2022-06-08 07:55:57.403275: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-06-08 07:56:00,409 INFO [train.py:887] Device: cuda:0
2022-06-08 07:56:00,476 INFO [lexicon.py:176] Loading pre-compiled data/lang_char/Linv.pt
2022-06-08 07:56:00,486 INFO [train.py:898] {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 1, 'reset_interval': 100, 'valid_interval': 200, 'feature_dim': 80, 'subsampling_factor': 4, 'model_warm_step': 50, 'env_info': {'k2-version': '1.15.1', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': 'f8d2dba06c000ffee36aab5b66f24e7c9809f116', 'k2-git-date': 'Thu Apr 21 12:20:34 2022', 'lhotse-version': '1.3.0.dev+git.4198446.dirty', 'torch-version': '1.11.0', 'torch-cuda-available': True, 'torch-cuda-version': '10.2', 'python-version': '3.8', 'icefall-git-branch': 'icefall-pruned-rnnt5-aishell4', 'icefall-git-sha1': '296303a-dirty', 'icefall-git-date': 'Tue Jun 7 22:36:20 2022', 'icefall-path': '/ceph-meixu/luomingshuang/icefall', 'k2-path': '/ceph-ms/luomingshuang/k2_latest/k2/python/k2/__init__.py', 'lhotse-path': '/ceph-meixu/luomingshuang/anaconda3/envs/k2-python/lib/python3.8/site-packages/lhotse-1.3.0.dev0+git.4198446.dirty-py3.8.egg/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-2-0602201035-5fb6d86964-mclm7', 'IP address': '10.177.74.202'}, 'world_size': 1, 'master_port': 12354, 'tensorboard': True, 'num_epochs': 30, 'start_epoch': 1, 'start_batch': 0, 'exp_dir': PosixPath('pruned_transducer_stateless5/exp-one-gpu'), 'lang_dir': 'data/lang_char', 'initial_lr': 0.003, 'lr_batches': 5000, 'lr_epochs': 6, 'context_size': 2, 'prune_range': 5, 'lm_scale': 0.25, 'am_scale': 0.0, 'simple_loss_scale': 0.5, 'seed': 42, 'print_diagnostics': False, 'save_every_n': 4000, 'keep_last_k': 30, 'average_period': 100, 'use_fp16': False, 'num_encoder_layers': 24, 'dim_feedforward': 1536, 'nhead': 8, 'encoder_dim': 384, 'decoder_dim': 512, 'joiner_dim': 512, 'manifest_dir': PosixPath('data/fbank'), 'max_duration': 220, 'bucketing_sampler': True, 'num_buckets': 300, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'drop_last': True, 'return_cuts': True, 'num_workers': 2, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'input_strategy': 'PrecomputedFeatures', 'training_subset': 'L', 'blank_id': 0, 'vocab_size': 3284}
2022-06-08 07:56:00,487 INFO [train.py:900] About to create model
2022-06-08 07:56:01,081 INFO [train.py:904] Number of model parameters: 94337552
2022-06-08 07:56:04,849 INFO [asr_datamodule.py:429] About to get train cuts
2022-06-08 07:56:04,853 INFO [asr_datamodule.py:231] About to get Musan cuts
2022-06-08 07:56:04,854 INFO [asr_datamodule.py:238] Enable MUSAN
2022-06-08 07:56:04,962 INFO [asr_datamodule.py:263] Enable SpecAugment
2022-06-08 07:56:04,962 INFO [asr_datamodule.py:264] Time warp factor: 80
2022-06-08 07:56:04,962 INFO [asr_datamodule.py:276] Num frame mask: 10
2022-06-08 07:56:04,962 INFO [asr_datamodule.py:289] About to create train dataset
2022-06-08 07:56:04,963 INFO [asr_datamodule.py:318] Using DynamicBucketingSampler.
2022-06-08 07:56:09,181 INFO [asr_datamodule.py:334] About to create train dataloader
2022-06-08 07:56:09,181 INFO [asr_datamodule.py:437] About to get dev cuts
2022-06-08 07:56:09,183 INFO [asr_datamodule.py:365] About to create dev dataset
2022-06-08 07:56:09,772 INFO [asr_datamodule.py:384] About to create dev dataloader
2022-06-08 07:56:09,773 INFO [train.py:1054] Sanity check -- see if any of the batches in epoch 1 would cause OOM.
2022-06-08 08:00:27,984 INFO [train.py:815] Epoch 1, batch 0, loss[loss=0.9558, simple_loss=1.912, pruned_loss=9.052, over 5285.00 frames.], tot_loss[loss=0.9558, simple_loss=1.912, pruned_loss=9.052, over 5285.00 frames.], batch size: 23, lr: 3.00e-03
2022-06-08 08:00:30,086 INFO [train.py:815] Epoch 1, batch 1, loss[loss=0.7711, simple_loss=1.542, pruned_loss=9.094, over 5263.00 frames.], tot_loss[loss=0.8632, simple_loss=1.726, pruned_loss=9.073, over 10495.15 frames.], batch size: 21, lr: 3.00e-03
2022-06-08 08:01:10,457 INFO [train.py:815] Epoch 1, batch 2, loss[loss=0.8648, simple_loss=1.73, pruned_loss=9.108, over 5434.00 frames.], tot_loss[loss=0.8637, simple_loss=1.727, pruned_loss=9.085, over 15824.20 frames.], batch size: 99, lr: 3.00e-03
2022-06-08 08:01:11,278 INFO [train.py:815] Epoch 1, batch 3, loss[loss=0.6862, simple_loss=1.372, pruned_loss=9.249, over 5430.00 frames.], tot_loss[loss=0.818, simple_loss=1.636, pruned_loss=9.127, over 21095.96 frames.], batch size: 33, lr: 3.00e-03
2022-06-08 08:01:21,158 INFO [train.py:815] Epoch 1, batch 4, loss[loss=0.5969, simple_loss=1.194, pruned_loss=9.26, over 5197.00 frames.], tot_loss[loss=0.774, simple_loss=1.548, pruned_loss=9.154, over 26082.00 frames.], batch size: 16, lr: 3.00e-03
2022-06-08 08:01:22,072 INFO [train.py:815] Epoch 1, batch 5, loss[loss=0.6028, simple_loss=1.206, pruned_loss=9.481, over 5392.00 frames.], tot_loss[loss=0.7444, simple_loss=1.489, pruned_loss=9.21, over 31213.18 frames.], batch size: 18, lr: 3.00e-03
2022-06-08 08:01:27,888 INFO [train.py:815] Epoch 1, batch 6, loss[loss=0.5526, simple_loss=1.105, pruned_loss=9.342, over 5306.00 frames.], tot_loss[loss=0.7163, simple_loss=1.433, pruned_loss=9.23, over 36207.05 frames.], batch size: 22, lr: 3.00e-03
2022-06-08 08:01:28,707 INFO [train.py:815] Epoch 1, batch 7, loss[loss=0.5148, simple_loss=1.03, pruned_loss=9.262, over 5371.00 frames.], tot_loss[loss=0.69, simple_loss=1.38, pruned_loss=9.234, over 41215.97 frames.], batch size: 29, lr: 3.00e-03
2022-06-08 08:01:42,786 INFO [train.py:815] Epoch 1, batch 8, loss[loss=0.4969, simple_loss=0.9939, pruned_loss=9.257, over 5310.00 frames.], tot_loss[loss=0.6678, simple_loss=1.336, pruned_loss=9.236, over 46113.82 frames.], batch size: 18, lr: 3.00e-03
2022-06-08 08:01:43,608 INFO [train.py:815] Epoch 1, batch 9, loss[loss=0.5297, simple_loss=1.059, pruned_loss=9.424, over 5346.00 frames.], tot_loss[loss=0.6533, simple_loss=1.307, pruned_loss=9.256, over 50998.68 frames.], batch size: 25, lr: 3.00e-03
2022-06-08 08:02:01,751 INFO [train.py:815] Epoch 1, batch 10, loss[loss=0.554, simple_loss=1.108, pruned_loss=9.458, over 5376.00 frames.], tot_loss[loss=0.6438, simple_loss=1.288, pruned_loss=9.275, over 55864.69 frames.], batch size: 37, lr: 3.00e-03
2022-06-08 08:02:02,639 INFO [train.py:815] Epoch 1, batch 11, loss[loss=0.4267, simple_loss=0.8533, pruned_loss=9.177, over 5476.00 frames.], tot_loss[loss=0.6242, simple_loss=1.248, pruned_loss=9.267, over 60782.04 frames.], batch size: 17, lr: 3.00e-03
2022-06-08 08:02:24,529 INFO [train.py:815] Epoch 1, batch 12, loss[loss=0.6245, simple_loss=1.249, pruned_loss=9.511, over 5390.00 frames.], tot_loss[loss=0.6242, simple_loss=1.248, pruned_loss=9.287, over 65564.22 frames.], batch size: 49, lr: 3.00e-03
2022-06-08 08:02:25,393 INFO [train.py:815] Epoch 1, batch 13, loss[loss=0.5387, simple_loss=1.077, pruned_loss=9.375, over 5440.00 frames.], tot_loss[loss=0.6176, simple_loss=1.235, pruned_loss=9.294, over 70348.58 frames.], batch size: 31, lr: 3.00e-03
2022-06-08 08:02:41,581 INFO [train.py:815] Epoch 1, batch 14, loss[loss=0.5919, simple_loss=1.184, pruned_loss=9.385, over 5390.00 frames.], tot_loss[loss=0.6158, simple_loss=1.232, pruned_loss=9.3, over 75035.09 frames.], batch size: 49, lr: 3.00e-03
2022-06-08 08:02:42,388 INFO [train.py:815] Epoch 1, batch 15, loss[loss=0.5398, simple_loss=1.08, pruned_loss=9.256, over 5431.00 frames.], tot_loss[loss=0.6106, simple_loss=1.221, pruned_loss=9.297, over 79715.74 frames.], batch size: 62, lr: 3.00e-03
2022-06-08 08:02:55,783 INFO [train.py:815] Epoch 1, batch 16, loss[loss=0.5844, simple_loss=1.169, pruned_loss=9.465, over 5343.00 frames.], tot_loss[loss=0.6089, simple_loss=1.218, pruned_loss=9.308, over 84261.59 frames.], batch size: 33, lr: 3.00e-03
2022-06-08 08:02:56,663 INFO [train.py:815] Epoch 1, batch 17, loss[loss=0.4624, simple_loss=0.9247, pruned_loss=9.312, over 5159.00 frames.], tot_loss[loss=0.6004, simple_loss=1.201, pruned_loss=9.308, over 88577.97 frames.], batch size: 14, lr: 3.00e-03
2022-06-08 08:03:14,726 INFO [train.py:815] Epoch 1, batch 18, loss[loss=0.5623, simple_loss=1.125, pruned_loss=9.351, over 5383.00 frames.], tot_loss[loss=0.5982, simple_loss=1.196, pruned_loss=9.31, over 93075.19 frames.], batch size: 47, lr: 3.00e-03
2022-06-08 08:03:15,588 INFO [train.py:815] Epoch 1, batch 19, loss[loss=0.4958, simple_loss=0.9917, pruned_loss=9.32, over 5322.00 frames.], tot_loss[loss=0.5926, simple_loss=1.185, pruned_loss=9.311, over 97466.44 frames.], batch size: 28, lr: 3.00e-03
2022-06-08 08:03:44,387 INFO [train.py:815] Epoch 1, batch 20, loss[loss=0.5633, simple_loss=1.127, pruned_loss=9.368, over 5469.00 frames.], tot_loss[loss=0.591, simple_loss=1.182, pruned_loss=9.314, over 101960.77 frames.], batch size: 61, lr: 3.00e-03
2022-06-08 08:03:45,262 INFO [train.py:815] Epoch 1, batch 21, loss[loss=0.5749, simple_loss=1.15, pruned_loss=9.378, over 5434.00 frames.], tot_loss[loss=0.5902, simple_loss=1.18, pruned_loss=9.317, over 106375.17 frames.], batch size: 72, lr: 3.00e-03
2022-06-08 08:03:51,698 INFO [train.py:815] Epoch 1, batch 22, loss[loss=0.4955, simple_loss=0.991, pruned_loss=9.392, over 5433.00 frames.], tot_loss[loss=0.5856, simple_loss=1.171, pruned_loss=9.321, over 110744.42 frames.], batch size: 27, lr: 3.00e-03
2022-06-08 08:03:52,595 INFO [train.py:815] Epoch 1, batch 23, loss[loss=0.5053, simple_loss=1.011, pruned_loss=9.519, over 5275.00 frames.], tot_loss[loss=0.5819, simple_loss=1.164, pruned_loss=9.33, over 114911.97 frames.], batch size: 18, lr: 3.00e-03
2022-06-08 08:04:01,234 INFO [train.py:815] Epoch 1, batch 24, loss[loss=0.4291, simple_loss=0.8583, pruned_loss=9.412, over 5441.00 frames.], tot_loss[loss=0.5749, simple_loss=1.15, pruned_loss=9.334, over 119203.85 frames.], batch size: 12, lr: 3.00e-03
2022-06-08 08:04:17,872 INFO [train.py:815] Epoch 1, batch 25, loss[loss=0.5338, simple_loss=1.068, pruned_loss=9.364, over 5457.00 frames.], tot_loss[loss=0.5731, simple_loss=1.146, pruned_loss=9.335, over 123468.81 frames.], batch size: 60, lr: 3.00e-03
2022-06-08 08:04:18,744 INFO [train.py:815] Epoch 1, batch 26, loss[loss=0.4739, simple_loss=0.9479, pruned_loss=9.445, over 5341.00 frames.], tot_loss[loss=0.5689, simple_loss=1.138, pruned_loss=9.34, over 127575.12 frames.], batch size: 25, lr: 3.00e-03
2022-06-08 08:04:32,789 INFO [train.py:815] Epoch 1, batch 27, loss[loss=0.4645, simple_loss=0.929, pruned_loss=9.407, over 5286.00 frames.], tot_loss[loss=0.5647, simple_loss=1.129, pruned_loss=9.342, over 131585.37 frames.], batch size: 20, lr: 3.00e-03
2022-06-08 08:04:33,656 INFO [train.py:815] Epoch 1, batch 28, loss[loss=0.4937, simple_loss=0.9875, pruned_loss=9.484, over 5393.00 frames.], tot_loss[loss=0.5619, simple_loss=1.124, pruned_loss=9.348, over 135662.52 frames.], batch size: 28, lr: 3.00e-03
2022-06-08 08:04:46,499 INFO [train.py:815] Epoch 1, batch 29, loss[loss=0.4878, simple_loss=0.9755, pruned_loss=9.323, over 5469.00 frames.], tot_loss[loss=0.559, simple_loss=1.118, pruned_loss=9.347, over 139774.89 frames.], batch size: 39, lr: 3.00e-03
2022-06-08 08:04:47,382 INFO [train.py:815] Epoch 1, batch 30, loss[loss=0.4376, simple_loss=0.8751, pruned_loss=9.239, over 5087.00 frames.], tot_loss[loss=0.5547, simple_loss=1.109, pruned_loss=9.343, over 143464.15 frames.], batch size: 12, lr: 3.00e-03

When I use num-workers=4 to run again based on one GPU, there are some training logs:
(k2-python) luomingshuang@de-74279-k2-train-2-0602201035-5fb6d86964-mclm7:~/codes/icefall-pruned-rnnt5-aishell4/egs/aishell4/ASR$ CUDA_VISIBLE_DEVICES='4' python pruned_transducer_stateless5/train.py --max-duration 220 --num-workers 4 --exp-dir pruned_transducer_stateless5/exp-one-gpu
2022-06-08 08:05:43,112 INFO [train.py:877] Training started
2022-06-08 08:05:43.287827: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /ceph-sh1/fangjun/software/cuda-10.2.89/lib:/ceph-sh1/fangjun/software/cuda-10.2.89/lib64:
2022-06-08 08:05:43.287867: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-06-08 08:05:45,859 INFO [train.py:887] Device: cuda:0
2022-06-08 08:05:45,916 INFO [lexicon.py:176] Loading pre-compiled data/lang_char/Linv.pt
2022-06-08 08:05:45,924 INFO [train.py:898] {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 1, 'reset_interval': 100, 'valid_interval': 200, 'feature_dim': 80, 'subsampling_factor': 4, 'model_warm_step': 50, 'env_info': {'k2-version': '1.15.1', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': 'f8d2dba06c000ffee36aab5b66f24e7c9809f116', 'k2-git-date': 'Thu Apr 21 12:20:34 2022', 'lhotse-version': '1.3.0.dev+git.4198446.dirty', 'torch-version': '1.11.0', 'torch-cuda-available': True, 'torch-cuda-version': '10.2', 'python-version': '3.8', 'icefall-git-branch': 'icefall-pruned-rnnt5-aishell4', 'icefall-git-sha1': '296303a-dirty', 'icefall-git-date': 'Tue Jun 7 22:36:20 2022', 'icefall-path': '/ceph-meixu/luomingshuang/icefall', 'k2-path': '/ceph-ms/luomingshuang/k2_latest/k2/python/k2/__init__.py', 'lhotse-path': '/ceph-meixu/luomingshuang/anaconda3/envs/k2-python/lib/python3.8/site-packages/lhotse-1.3.0.dev0+git.4198446.dirty-py3.8.egg/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-2-0602201035-5fb6d86964-mclm7', 'IP address': '10.177.74.202'}, 'world_size': 1, 'master_port': 12354, 'tensorboard': True, 'num_epochs': 30, 'start_epoch': 1, 'start_batch': 0, 'exp_dir': PosixPath('pruned_transducer_stateless5/exp-one-gpu'), 'lang_dir': 'data/lang_char', 'initial_lr': 0.003, 'lr_batches': 5000, 'lr_epochs': 6, 'context_size': 2, 'prune_range': 5, 'lm_scale': 0.25, 'am_scale': 0.0, 'simple_loss_scale': 0.5, 'seed': 42, 'print_diagnostics': False, 'save_every_n': 4000, 'keep_last_k': 30, 'average_period': 100, 'use_fp16': False, 'num_encoder_layers': 24, 'dim_feedforward': 1536, 'nhead': 8, 'encoder_dim': 384, 'decoder_dim': 512, 'joiner_dim': 512, 'manifest_dir': PosixPath('data/fbank'), 'max_duration': 220, 'bucketing_sampler': True, 'num_buckets': 300, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'drop_last': True, 'return_cuts': True, 'num_workers': 4, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'input_strategy': 'PrecomputedFeatures', 'training_subset': 'L', 'blank_id': 0, 'vocab_size': 3284}
2022-06-08 08:05:45,925 INFO [train.py:900] About to create model
2022-06-08 08:05:46,400 INFO [train.py:904] Number of model parameters: 94337552
2022-06-08 08:05:50,115 INFO [asr_datamodule.py:429] About to get train cuts
2022-06-08 08:05:50,118 INFO [asr_datamodule.py:231] About to get Musan cuts
2022-06-08 08:05:50,120 INFO [asr_datamodule.py:238] Enable MUSAN
2022-06-08 08:05:50,228 INFO [asr_datamodule.py:263] Enable SpecAugment
2022-06-08 08:05:50,228 INFO [asr_datamodule.py:264] Time warp factor: 80
2022-06-08 08:05:50,229 INFO [asr_datamodule.py:276] Num frame mask: 10
2022-06-08 08:05:50,229 INFO [asr_datamodule.py:289] About to create train dataset
2022-06-08 08:05:50,229 INFO [asr_datamodule.py:318] Using DynamicBucketingSampler.
2022-06-08 08:05:54,631 INFO [asr_datamodule.py:334] About to create train dataloader
2022-06-08 08:05:54,631 INFO [asr_datamodule.py:437] About to get dev cuts
2022-06-08 08:05:54,633 INFO [asr_datamodule.py:365] About to create dev dataset
2022-06-08 08:05:55,577 INFO [asr_datamodule.py:384] About to create dev dataloader
2022-06-08 08:05:55,578 INFO [train.py:1054] Sanity check -- see if any of the batches in epoch 1 would cause OOM.
2022-06-08 08:10:30,286 INFO [train.py:815] Epoch 1, batch 0, loss[loss=0.9558, simple_loss=1.912, pruned_loss=9.052, over 5285.00 frames.], tot_loss[loss=0.9558, simple_loss=1.912, pruned_loss=9.052, over 5285.00 frames.], batch size: 23, lr: 3.00e-03
2022-06-08 08:10:31,157 INFO [train.py:815] Epoch 1, batch 1, loss[loss=0.7711, simple_loss=1.542, pruned_loss=9.094, over 5263.00 frames.], tot_loss[loss=0.8632, simple_loss=1.726, pruned_loss=9.073, over 10495.15 frames.], batch size: 21, lr: 3.00e-03
2022-06-08 08:10:47,571 INFO [train.py:815] Epoch 1, batch 2, loss[loss=0.8648, simple_loss=1.73, pruned_loss=9.109, over 5434.00 frames.], tot_loss[loss=0.8637, simple_loss=1.727, pruned_loss=9.085, over 15824.20 frames.], batch size: 99, lr: 3.00e-03
2022-06-08 08:10:48,376 INFO [train.py:815] Epoch 1, batch 3, loss[loss=0.6862, simple_loss=1.372, pruned_loss=9.248, over 5430.00 frames.], tot_loss[loss=0.818, simple_loss=1.636, pruned_loss=9.127, over 21095.96 frames.], batch size: 33, lr: 3.00e-03
2022-06-08 08:10:49,258 INFO [train.py:815] Epoch 1, batch 4, loss[loss=0.5969, simple_loss=1.194, pruned_loss=9.258, over 5197.00 frames.], tot_loss[loss=0.774, simple_loss=1.548, pruned_loss=9.153, over 26082.00 frames.], batch size: 16, lr: 3.00e-03
2022-06-08 08:10:50,107 INFO [train.py:815] Epoch 1, batch 5, loss[loss=0.6028, simple_loss=1.206, pruned_loss=9.48, over 5392.00 frames.], tot_loss[loss=0.7444, simple_loss=1.489, pruned_loss=9.21, over 31213.18 frames.], batch size: 18, lr: 3.00e-03
2022-06-08 08:10:57,468 INFO [train.py:815] Epoch 1, batch 6, loss[loss=0.5526, simple_loss=1.105, pruned_loss=9.341, over 5306.00 frames.], tot_loss[loss=0.7163, simple_loss=1.433, pruned_loss=9.229, over 36207.05 frames.], batch size: 22, lr: 3.00e-03
2022-06-08 08:10:58,273 INFO [train.py:815] Epoch 1, batch 7, loss[loss=0.5148, simple_loss=1.03, pruned_loss=9.262, over 5371.00 frames.], tot_loss[loss=0.69, simple_loss=1.38, pruned_loss=9.234, over 41215.97 frames.], batch size: 29, lr: 3.00e-03
2022-06-08 08:11:01,953 INFO [train.py:815] Epoch 1, batch 8, loss[loss=0.4969, simple_loss=0.9939, pruned_loss=9.256, over 5310.00 frames.], tot_loss[loss=0.6678, simple_loss=1.336, pruned_loss=9.236, over 46113.82 frames.], batch size: 18, lr: 3.00e-03
2022-06-08 08:11:02,764 INFO [train.py:815] Epoch 1, batch 9, loss[loss=0.5297, simple_loss=1.059, pruned_loss=9.423, over 5346.00 frames.], tot_loss[loss=0.6533, simple_loss=1.307, pruned_loss=9.256, over 50998.68 frames.], batch size: 25, lr: 3.00e-03
2022-06-08 08:11:08,665 INFO [train.py:815] Epoch 1, batch 10, loss[loss=0.5538, simple_loss=1.108, pruned_loss=9.457, over 5376.00 frames.], tot_loss[loss=0.6437, simple_loss=1.287, pruned_loss=9.275, over 55864.69 frames.], batch size: 37, lr: 3.00e-03
2022-06-08 08:11:09,524 INFO [train.py:815] Epoch 1, batch 11, loss[loss=0.4266, simple_loss=0.8533, pruned_loss=9.177, over 5476.00 frames.], tot_loss[loss=0.6242, simple_loss=1.248, pruned_loss=9.266, over 60782.04 frames.], batch size: 17, lr: 3.00e-03
2022-06-08 08:11:15,845 INFO [train.py:815] Epoch 1, batch 12, loss[loss=0.6245, simple_loss=1.249, pruned_loss=9.511, over 5390.00 frames.], tot_loss[loss=0.6242, simple_loss=1.248, pruned_loss=9.286, over 65564.22 frames.], batch size: 49, lr: 3.00e-03
2022-06-08 08:11:16,639 INFO [train.py:815] Epoch 1, batch 13, loss[loss=0.5386, simple_loss=1.077, pruned_loss=9.375, over 5440.00 frames.], tot_loss[loss=0.6176, simple_loss=1.235, pruned_loss=9.293, over 70348.58 frames.], batch size: 31, lr: 3.00e-03
2022-06-08 08:11:29,614 INFO [train.py:815] Epoch 1, batch 14, loss[loss=0.5918, simple_loss=1.184, pruned_loss=9.385, over 5390.00 frames.], tot_loss[loss=0.6157, simple_loss=1.231, pruned_loss=9.3, over 75035.09 frames.], batch size: 49, lr: 3.00e-03
2022-06-08 08:11:30,393 INFO [train.py:815] Epoch 1, batch 15, loss[loss=0.5396, simple_loss=1.079, pruned_loss=9.256, over 5431.00 frames.], tot_loss[loss=0.6106, simple_loss=1.221, pruned_loss=9.297, over 79715.74 frames.], batch size: 62, lr: 3.00e-03
2022-06-08 08:11:31,175 INFO [train.py:815] Epoch 1, batch 16, loss[loss=0.5844, simple_loss=1.169, pruned_loss=9.466, over 5343.00 frames.], tot_loss[loss=0.6089, simple_loss=1.218, pruned_loss=9.308, over 84261.59 frames.], batch size: 33, lr: 3.00e-03
2022-06-08 08:11:32,022 INFO [train.py:815] Epoch 1, batch 17, loss[loss=0.4623, simple_loss=0.9246, pruned_loss=9.311, over 5159.00 frames.], tot_loss[loss=0.6004, simple_loss=1.201, pruned_loss=9.308, over 88577.97 frames.], batch size: 14, lr: 3.00e-03
2022-06-08 08:11:48,858 INFO [train.py:815] Epoch 1, batch 18, loss[loss=0.5627, simple_loss=1.125, pruned_loss=9.35, over 5383.00 frames.], tot_loss[loss=0.5982, simple_loss=1.196, pruned_loss=9.31, over 93075.19 frames.], batch size: 47, lr: 3.00e-03
2022-06-08 08:11:49,684 INFO [train.py:815] Epoch 1, batch 19, loss[loss=0.4959, simple_loss=0.9917, pruned_loss=9.319, over 5322.00 frames.], tot_loss[loss=0.5926, simple_loss=1.185, pruned_loss=9.311, over 97466.44 frames.], batch size: 28, lr: 3.00e-03
2022-06-08 08:11:50,537 INFO [train.py:815] Epoch 1, batch 20, loss[loss=0.563, simple_loss=1.126, pruned_loss=9.368, over 5469.00 frames.], tot_loss[loss=0.591, simple_loss=1.182, pruned_loss=9.314, over 101960.77 frames.], batch size: 61, lr: 3.00e-03
2022-06-08 08:11:51,418 INFO [train.py:815] Epoch 1, batch 21, loss[loss=0.575, simple_loss=1.15, pruned_loss=9.378, over 5434.00 frames.], tot_loss[loss=0.5902, simple_loss=1.18, pruned_loss=9.317, over 106375.17 frames.], batch size: 72, lr: 3.00e-03
2022-06-08 08:12:00,784 INFO [train.py:815] Epoch 1, batch 22, loss[loss=0.4955, simple_loss=0.991, pruned_loss=9.393, over 5433.00 frames.], tot_loss[loss=0.5855, simple_loss=1.171, pruned_loss=9.321, over 110744.42 frames.], batch size: 27, lr: 3.00e-03
2022-06-08 08:12:01,681 INFO [train.py:815] Epoch 1, batch 23, loss[loss=0.5052, simple_loss=1.01, pruned_loss=9.521, over 5275.00 frames.], tot_loss[loss=0.5819, simple_loss=1.164, pruned_loss=9.33, over 114911.97 frames.], batch size: 18, lr: 3.00e-03
2022-06-08 08:12:04,138 INFO [train.py:815] Epoch 1, batch 24, loss[loss=0.4291, simple_loss=0.8582, pruned_loss=9.416, over 5441.00 frames.], tot_loss[loss=0.5749, simple_loss=1.15, pruned_loss=9.334, over 119203.85 frames.], batch size: 12, lr: 3.00e-03
2022-06-08 08:12:05,003 INFO [train.py:815] Epoch 1, batch 25, loss[loss=0.5337, simple_loss=1.067, pruned_loss=9.363, over 5457.00 frames.], tot_loss[loss=0.5731, simple_loss=1.146, pruned_loss=9.335, over 123468.81 frames.], batch size: 60, lr: 3.00e-03
2022-06-08 08:12:12,781 INFO [train.py:815] Epoch 1, batch 26, loss[loss=0.4739, simple_loss=0.9478, pruned_loss=9.444, over 5341.00 frames.], tot_loss[loss=0.5689, simple_loss=1.138, pruned_loss=9.34, over 127575.12 frames.], batch size: 25, lr: 3.00e-03
2022-06-08 08:12:13,682 INFO [train.py:815] Epoch 1, batch 27, loss[loss=0.4643, simple_loss=0.9287, pruned_loss=9.404, over 5286.00 frames.], tot_loss[loss=0.5647, simple_loss=1.129, pruned_loss=9.342, over 131585.37 frames.], batch size: 20, lr: 3.00e-03
2022-06-08 08:12:17,462 INFO [train.py:815] Epoch 1, batch 28, loss[loss=0.4937, simple_loss=0.9874, pruned_loss=9.483, over 5393.00 frames.], tot_loss[loss=0.5619, simple_loss=1.124, pruned_loss=9.348, over 135662.52 frames.], batch size: 28, lr: 3.00e-03
2022-06-08 08:12:20,571 INFO [train.py:815] Epoch 1, batch 29, loss[loss=0.4876, simple_loss=0.9752, pruned_loss=9.322, over 5469.00 frames.], tot_loss[loss=0.559, simple_loss=1.118, pruned_loss=9.347, over 139774.89 frames.], batch size: 39, lr: 3.00e-03
2022-06-08 08:12:27,676 INFO [train.py:815] Epoch 1, batch 30, loss[loss=0.4398, simple_loss=0.8795, pruned_loss=9.228, over 5087.00 frames.], tot_loss[loss=0.5548, simple_loss=1.11, pruned_loss=9.343, over 143464.15 frames.], batch size: 12, lr: 3.00e-03

I also have a question when I use 4 GPUs for running this recipe, there are 3 GPUs whose utilization is 100% while another GPU's utilization is 0% most of the time. The case is shown as the following picture:

It seems 4 workers is still not enough to saturate the GPUs. You have a few seconds spike of latency every 4 batches. Try increasing it further if your system resources allow.

If 1 GPU is idle, it actually means that all GPUs are idle; the others are just waiting on a spin-lock inside some kernel, waiting to be able to sync the data via nccl.

I'm a little surprised to see (from your screenshot) that it is unzipping data.  Can you tell me what kind of archive you are dealing with here, or what format?  I didn't realize we had any setup where unzipping data is something that the data loaders would do.

Yes, it is the jsonl.gz format.

When I create the WebDataset for testing, I find that the speed of this process is very slow. (The following picture shows that it will take many hours to finish it. The following video is the process of creating webdataset that I use py-spy to look. ) Are there some methods or suggestions to improve the speed?

https://user-images.githubusercontent.com/37799481/172527657-75f0cd88-943b-40d2-9974-a196980b11f7.mp4

So it's the decoding process that is slow?
From the stack traces in the video, it appears to be decoding the audio rather than just loading the features.  Is this expected?
I think if you have just one CPU thread responsible for decoding encoded audio, it's not surprising that that would be slower than ctual decoding on a GPU.

... I am also a bit confused about what decode.py does.  Is that ASR decoding?  Why is it creating  tarball?

Based on the current decode.py in this pr (I remove the Webdataset part), it decode fastly.

In my video, it doesn't run to the decoding process, just the process of creating WebDataset.

So it's the decoding process that is slow? From the stack traces in the video, it appears to be decoding the audio rather than just loading the features. Is this expected? I think if you have just one CPU thread responsible for decoding encoded audio, it's not surprising that that would be slower than ctual decoding on a GPU.

About this pruned_transducer_stateless5 recipe trained with AISHELL4, my best results for testing set are as follows:

epoch
avg
decoding-method
CER(%)

30
25
greedy_search
30.05

30
25
modified_beam_search
29.16

30
25
fast_beam_search
29.20

A baseline result for testing set from (wenet)[https://github.com/wenet-e2e/wenet/tree/main/examples/aishell4/s0] is 32.58%(CER). Our results are better than theirs.

@luomingshuang
Could you also upload a torchscript model to https://huggingface.co/luomingshuang/icefall_asr_aishell4_pruned_transducer_stateless5/tree/main/exp
?
I will add it to https://huggingface.co/spaces/k2-fsa/automatic-speech-recognition
after you upload it.

authors: csukuangfj danpovey luomingshuang pzelasko


text:
Master failing to join cluster - TLS handshake error: bad certificate

K3s Version:
k3s version v1.17.2+k3s1 (cdab19b0)
Node(s) CPU architecture, OS, and Version:
Linux 3.10.0-1062.12.1.el7.x86_64 #1 SMP Thu Dec 12 06:44:49 EST 2019 x86_64 x86_64 x86_64 GNU/Linux
Cluster Configuration:
4 masters, 8 agents
Describe the bug:
Master node dropped from the cluster and is unable to rejoin due to certificate issues.
Steps To Reproduce:
Running airgap installation with the below service file
[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
Wants=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
Type=notify
EnvironmentFile=/etc/systemd/system/k3s.service.env
KillMode=process
Delegate=yes
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/opt/app/mdx/MdxCache1/k3s/bin/k3s \
    server \
        '--data-dir' \
        '/opt/app/mdx/MdxCache1/k3s-data' \
        '--datastore-endpoint' \
        'https://...:2379,https://...:2379,https://...:2379,https://....:2379,https://....:2379' \
        '--datastore-cafile' \
        '/opt/app/mdx/MdxCache1/k3s/etcd/ca.crt' \
        '--datastore-certfile' \
        '/opt/app/mdx/MdxCache1/k3s/etcd/etcd-server.crt' \
        '--datastore-keyfile' \
        '/opt/app/mdx/MdxCache1/k3s/etcd/etcd-server.key' \
        '--tls-san' \
        'KUBERNETES-CA' \

Installed K3s:

Expected behavior:
Master node should join the cluster
Actual behavior:
Master failed to join the cluster with http: TLS handshake error from 127.0.0.1:40404: remote error: tls: bad certificate
Additional context / logs:
Oct 06 17:06:00 lonrs13760 k3s[12866]: http: TLS handshake error from 127.0.0.1:40396: remote error: tls: bad certificate
Oct 06 17:06:00 lonrs13760 k3s[12866]: time="2020-10-06T17:06:00.382530597+01:00" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml"
Oct 06 17:06:00 lonrs13760 k3s[12866]: time="2020-10-06T17:06:00.382558967+01:00" level=info msg="Run: k3s kubectl"
Oct 06 17:06:00 lonrs13760 k3s[12866]: time="2020-10-06T17:06:00.382571906+01:00" level=info msg="k3s is up and running"
Oct 06 17:06:00 lonrs13760 systemd[1]: Started Lightweight Kubernetes.
Oct 06 17:06:00 lonrs13760 k3s[12866]: time="2020-10-06T17:06:00.382810224+01:00" level=info msg="module overlay was already loaded"
Oct 06 17:06:00 lonrs13760 k3s[12866]: time="2020-10-06T17:06:00.382836321+01:00" level=info msg="module nf_conntrack was already loaded"
Oct 06 17:06:00 lonrs13760 k3s[12866]: time="2020-10-06T17:06:00.382851414+01:00" level=info msg="module br_netfilter was already loaded"
Oct 06 17:06:00 lonrs13760 k3s[12866]: I1006 17:06:00.385210   12866 controller.go:606] quota admission added evaluator for: addons.k3s.cattle.io
Oct 06 17:06:00 lonrs13760 k3s[12866]: http: TLS handshake error from 127.0.0.1:40404: remote error: tls: bad certificate
Oct 06 17:06:00 lonrs13760 k3s[12866]: http: TLS handshake error from 127.0.0.1:40410: remote error: tls: bad certificate
Oct 06 17:06:00 lonrs13760 k3s[12866]: time="2020-10-06T17:06:00.418355513+01:00" level=info msg="Using registry config file at /etc/rancher/k3s/registries.yaml"
Oct 06 17:06:00 lonrs13760 k3s[12866]: time="2020-10-06T17:06:00.421453594+01:00" level=info msg="Logging containerd to /opt/app/mdx/MdxCache1/k3s-data/agent/containerd/containerd.log"
Oct 06 17:06:00 lonrs13760 k3s[12866]: time="2020-10-06T17:06:00.421624206+01:00" level=info msg="Running containerd -c /opt/app/mdx/MdxCache1/k3s-data/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /opt/app/mdx/MdxCache1/k3s-data/agent/containerd"
Oct 06 17:06:00 lonrs13760 k3s[12866]: time="2020-10-06T17:06:00.421784727+01:00" level=info msg="Waiting for containerd startup: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial unix /run/k3s/containerd/containerd.sock: connect: connection refused\""

Have tried removing the certs from /opt/app/mdx/MdxCache1/k3s-data/server/tls and deleting /api/v1/namespaces/kube-system/secrets/k3s-serving secret to regenerate new certificates and restarting but no improvement.

You don't appear to be supplying the --token to joining servers - are you doing this via the environment file, or some other means?

Yes, but you still need a token as it is used to encrypt/decrypt the bootstrap data, which includes the CA certificates:
https://rancher.com/docs/k3s/latest/en/installation/install-options/server-config/#cluster-options
https://github.com/rancher/k3s/blob/master/pkg/cluster/storage.go#L31

Just to clarify, this a master node which disconnected from the cluster and is attempting to rejoin. We've never used the token when installing the master nodes since they connect to the data store to get the shared secret.
Will try passing in the token at service start.
I've attempted to remove all the certificates in k3s-data/server/tls folder so it would regenerate new ones on starting the service with no luck. Have also forced regeneration of the k3s-serving secret but that didn't help either.

I tried adding the token when spinning up the master but had no luck with that.
Tied removing the node from the cluster via kubectl delete node  and reinstalling which also did allow the node to join the cluster with certificate issues.
In the end I ended up deleting all under k3s-data/server/cred/ and k3s-data/server/tls which I saw were not being cleaned up via the uninstall script.
Can I get a bit more insight on what exactly is in the cred folder, more specifically the below
ipsec.psk
node-passwd
passwd

I understand that k3s-serving secret is used to create everything in the tls folder, which then those certificates are used in the kubeconfig for each of the services.  But I'm not entirely sure how the above files fit into this.

I have same issue when trying to connect k3s server to etcd.
It's worth noting, that CA, certificate and key are correct, because I'm able connect to etcd using wget and openssl:
wget \
--ca-certificate=etcd_ca_bundle.crt \
--certificate=master1_client.crt \
--private-key=master1_client.key \
-O - https://etcd1:2379/metrics

openssl s_client -showcerts \
-connect etcd1:2379 \
-CAfile etcd_client_ca_bundle.crt \
-cert master1_client.crt \
-key master1_client.key

Both of those work fine for me.
My CA chain is like this: Root CA -> Client CA -> etcd1 & master1 certificates. The etcd_client_ca_bundle.crt consists of Root CA and Client CA.

K3s does not use the Kubernetes verbosity flags - can you try k3s --debug server or setting the K3S_DEBUG=1 environment variable?

@brandond Yes I'm sure because there are no issues when using wget and OpenSSL. I have also tried illegal certificates just to test what happens if I supply wrong CA or certificate. Then I get error from wget and OpenSSL as I should. Both of those tests confirm that certificate chain is correct. The only thing which I could currently think of is that I'm using ECC certificates and k3s fails to deal with that.
I'm already using --debugflag. Commands are as follows:
curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_START="true" sh -s -

sudo /usr/local/bin/k3s \
--debug \
server \
-v=3 \
--log=/srv/k3s/log/k3s.log \
--data-dir=/srv/k3s/data/ \
--alsologtostderr \
--node-taint CriticalAddonsOnly=true:NoExecute \
--tls-san="api.kube" \
--datastore-endpoint="https://etcd1:2379,https://etcd2:2379" \
--datastore-cafile="/srv/k3s/ssl/etcd_client_ca_bundle.crt" \
--datastore-certfile="/srv/k3s/ssl/master1_client.crt" \
--datastore-keyfile="/srv/k3s/ssl/master1_client.key"

Output from terminal:
INFO[0000] Preparing data dir /srv/k3s/data/data/688c8ca42a6cd0c042322efea271d6f3849d3de17c850739b0da2461f6c69ee8
DEBU[0002] Running /srv/k3s/data/data/688c8ca42a6cd0c042322efea271d6f3849d3de17c850739b0da2461f6c69ee8/bin/k3s-server [/usr/local/bin/k3s --debug server -v=3 --log=/srv/k3s/log/k3s.log --data-dir=/srv/k3s/data/ --alsologtostderr --node-taint CriticalAddonsOnly=true:NoExecute --tls-san=api.kube --datastore-endpoint=https://etcd1:2379,https://etcd2:2379 --datastore-cafile=/srv/k3s/ssl/etcd_client_ca_bundle.crt --datastore-certfile=/srv/k3s/ssl/master1_client.crt --datastore-keyfile=/srv/k3s/ssl/master1_client.key]
time="2020-10-15T12:10:49.439802664+03:00" level=info msg="Starting k3s v1.18.9+k3s1 (630bebf9)"

And then it just goes to etcd connection loop. From etcd side:
2020-10-15T12:11:04.470810+03:00 etcd1 etcd: {"level":"warn","ts":"2020-10-15T12:11:04.470+0300","caller":"embed/config_logging.go:279","msg":"rejected connection","remote-addr":"master1:47246","server-name":"etcd1","error":"remote error: tls: bad certificate"}
2020-10-15T12:11:13.182322+03:00 etcd1 etcd: {"level":"warn","ts":"2020-10-15T12:11:13.181+0300","caller":"embed/config_logging.go:279","msg":"rejected connection","remote-addr":"master1:47250","server-name":"etcd1","error":"remote error: tls: bad certificate"}
2020-10-15T12:11:28.817367+03:00 etcd1 etcd: {"level":"warn","ts":"2020-10-15T12:11:28.816+0300","caller":"embed/config_logging.go:279","msg":"rejected connection","remote-addr":"master1:47254","server-name":"etcd1","error":"remote error: tls: bad certificate"}
2020-10-15T12:11:59.561427+03:00 etcd1 etcd: {"level":"warn","ts":"2020-10-15T12:11:59.560+0300","caller":"embed/config_logging.go:279","msg":"rejected connection","remote-addr":"master1:47272","server-name":"etcd1","error":"remote error: tls: bad certificate"}
2020-10-15T12:12:46.740392+03:00 etcd1 etcd: {"level":"warn","ts":"2020-10-15T12:12:46.739+0300","caller":"embed/config_logging.go:279","msg":"rejected connection","remote-addr":"master1:47302","server-name":"etcd1","error":"remote error: tls: bad certificate"}
2020-10-15T12:14:03.222264+03:00 etcd1 etcd: {"level":"warn","ts":"2020-10-15T12:14:03.221+0300","caller":"embed/config_logging.go:279","msg":"rejected connection","remote-addr":"master1:47346","server-name":"etcd1","error":"remote error: tls: bad certificate"}
2020-10-15T12:16:00.722905+03:00 etcd1 etcd: {"level":"warn","ts":"2020-10-15T12:16:00.721+0300","caller":"embed/config_logging.go:279","msg":"rejected connection","remote-addr":"master1:47400","server-name":"etcd1","error":"remote error: tls: bad certificate"}

Can you share the actual output of the openssl s_client command? Does it work properly if you use rsa instead of ecdsa?

@brandond, it seems to me, that intermediate certificate (Client CA) causes the issue for k3s (full chain isn't being recursively checked?). If I change chain to Root CA > etcd1 & master1 then everything works fine with both RSA and ECC.
I also changed etcd1 configuration so it only has Client CA as client CA (not full chain/bundle like previously), etcd1 cert is issued by Client CA. Both wget and openssl are still working fine when specifying correct Root CA or Root CA & Client CA bundle. Authentication also works fine when specifying master1 cert (also issued by Client CA) and key.
Output is here thou I'd slightly had to redact it. As You can see, verification succeeds.
$ openssl s_client -showcerts \
> -connect etcd1:2379 \
> -CAfile root_ca.crt \
> -cert master1.crt \
> -key master1.key
CONNECTED(00000003)
depth=2 CN = Root CA
verify return:1
depth=1 CN = Client CA
verify return:1
depth=0 CN = etcd1
verify return:1
---
Certificate chain
 0 s:/CN=etcd1
   i:/CN=Client CA
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
 1 s:/CN=Client CA
   i:/CN=Root CA
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=etcd1
issuer=/CN=Client CA
---
Acceptable client certificate CA names
/CN=Client CA
Client Certificate Types: RSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:ECDSA+SHA256:RSA+SHA384:ECDSA+SHA384:RSA+SHA512:ECDSA+SHA512:RSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA256:ECDSA+SHA256:RSA+SHA384:ECDSA+SHA384:RSA+SHA512:ECDSA+SHA512:RSA+SHA1:ECDSA+SHA1
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2430 bytes and written 1208 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-ECDSA-AES256-GCM-SHA384
Server public key is 384 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-AES256-GCM-SHA384
    Session-ID: ...
    Session-ID-ctx:
    Master-Key: ...
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket:
    ...

    Start Time: ...
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Connection: close

400 Bad Requestclosed

If I don't specify authentication cert & key I still can connect correctly, but receive cert error, as I should, but the verification still succeeds:
$ openssl s_client -showcerts \
> -connect etcd1:2379 \
> -CAfile root_ca.crt
CONNECTED(00000003)
depth=2 CN = Root CA
verify return:1
depth=1 CN = Client CA
verify return:1
depth=0 CN = etcd1
verify return:1
139800371771280:error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:s3_pkt.c:1493:SSL alert number 42
139800371771280:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
---
Certificate chain
 0 s:/CN=etcd1
   i:/CN=Client CA
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
 1 s:/CN=Client CA
   i:/CN=Root CA
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=etcd1
issuer=/CN=Client CA
---
Acceptable client certificate CA names
/CN=Client CA
Client Certificate Types: RSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:ECDSA+SHA256:RSA+SHA384:ECDSA+SHA384:RSA+SHA512:ECDSA+SHA512:RSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA256:ECDSA+SHA256:RSA+SHA384:ECDSA+SHA384:RSA+SHA512:ECDSA+SHA512:RSA+SHA1:ECDSA+SHA1
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1585 bytes and written 138 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-ECDSA-AES256-GCM-SHA384
Server public key is 384 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-ECDSA-AES256-GCM-SHA384
    Session-ID:
    Session-ID-ctx:
    Master-Key: ...
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    Start Time: ...
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

In that case you'll probably need to concatenate the root and intermediate (or root and client) CAs into a single file, and pass that as the --datastore-cafile.

@brandond I have also tried both concatenated (bundle) and non-concatenated ways, no luck with k3s. Both ways worked fine with OpenSSL and wget.
Also, as far as I know the bundle (Root CA + Intermediate CA) needs to be only on server side, since full certificate chain is being sent from server to client anyway. Client should only have Root CA to confirm the "starting" point of the full chain, to verify the chain.

Have you tried with etcdctl? That might be a more relevant test, since it would use the same Go crypto routines as k3s.

Same with when using etcdctl from master1. I think I just figured out what's the issue. Sadly (or luckily) issue is between the keyboard and chair. I completely ignored the fact that checking CN is not mandatory when SAN field is present, so I just assumed that etcdctl crypto routines behave like "everyone else" and still check CN as last resort measure. Obviously it's not that way and I need to add DNS: etcd1 to X509v3 Subject Alternative Name also... Sad thing is that wget (and many other) completely ignore that part in RFC to maintain backwards compatibility. Also the scripts I used to generate the certificates weren't smart enough to put the domain also to subjectAltName.
So, issue solved for me. Thank You for the help @brandond . Running etcdctl from master1 really helped to figure this out for me.

Yeah, ai think that the CN fallback was removed from the golang crypto routines in 1.15.

https://golang.org/doc/go1.15#commonname

It can be temporarily re-enabled by adding the value x509ignoreCN=0 to the GODEBUG environment variable.

Closing as per https://github.com/k3s-io/k3s/issues/2369#issuecomment-713555350

maybe do not set hostname

authors: BeardyC brandond daenney gynter oppshorer


text:
insecure network interface

I have multiple network interfaces on the host. One of them is connected to an insecure network, so I have existing iptable rules to block INPUT traffic on that interface.
K3s seems to open the NodePort access on that insecure interface by default. How can I re-order and make sure the k3s iptables rules and existing/manual iptables won't step on each other?

ens192 is the insecure interface, if I manually insert the reject rule to the top, client on the insecure network will no longer be able to access the nodeport:
iptables -I FORWARD -i ens192 -j REJECT

BTW, before I install k3s on the host, there was only one reject rule in the FORWARD chain:
sudo iptables -S FORWARD
-P FORWARD ACCEPT
-A FORWARD -i ens192 -j REJECT --reject-with icmp-port-unreachable

After installing k3s, it somehow got moved to the 4th position,
-A FORWARD -m comment --comment "kubernetes forwarding rules" -j KUBE-FORWARD
-A FORWARD -m conntrack --ctstate NEW -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A FORWARD -m conntrack --ctstate NEW -m comment --comment "kubernetes externally-visible service portals" -j KUBE-EXTERNAL-SERVICES
-A FORWARD -i ens192 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -s 10.42.0.0/16 -j ACCEPT
-A FORWARD -d 10.42.0.0/16 -j ACCEPT

so the reject is no longer taking affect. KUBE-FORWARD installed by k3s let through the insecure traffic.

When you mention nodePort access, do you mean the port created by servicelb? By default it opens 443 and 80 for traefik, right?

I was talking about NodePort (those high ports on the node), but the Ingress/Traefik ports (80/443) has the same issue. My pre-existing iptable rules block inbound access to any ports on the node stopped working, those NodePorts and LB Ports are all open to the insecure interface.

The nodePort is, by default, open for all interfaces.
There is one thing I don't understand. After deploying k3s, when you run iptables -I FORWARD -i ens192 -j REJECT, you rule will be on top, so it will be the first one to be executed, doesn't that solve your problem?

Are you suggesting the issue I reported (k8s/flannel moves my custom iptable rule) only happens when deploying k3s? If I add customer rules afterwards, k3s will no longer change the iptable rules order during its lifecycle?

Are you suggesting the issue I reported (k8s/flannel moves my custom iptable rule) only happens when deploying k3s? If I add customer rules afterwards, k3s will no longer change the iptable rules order during its lifecycle?

From my experience, yes. The network policy chain (KUBE-ROUTER-FORWARD) might move before it but the rest should stay below (KUBE-FORWARD included). Can you please give it a try?

Yeah, I had my k3s cluster running (idle) for a few days and don't see the rules moving. Since I don't know exact which action triggers the move, it is hard to test.

The nodePort is, by default, open for all interfaces.

@manuelbuil Would it be possible to configure which interface NodePorts are exposed on?

The nodePort is, by default, open for all interfaces.

@manuelbuil Would it be possible to configure which interface NodePorts are exposed on?

Sorry for the late reply, I was on parental leave @laszlocph . Yes! https://kubernetes.io/docs/concepts/services-networking/service/#service-nodeport-custom-listen-address

Hope you had a good one! And thanks @manuelbuil

authors: gfrankliu laszlocph manuelbuil


text:
Subject appear twice

Expected behaviour
In Settings / Global settings / Display, i've plugged "Correspondent above subject".
Actual behaviour
Correspondent disappear, and the subject appear twice.
Steps to reproduce
Environment
K-9 Mail version:
5.010
Android version:
5.1
Account type (IMAP, POP3, WebDAV/Exchange):
Gmail, IMAP
UP: sorry for the last issue in french, i've just made it too fast, was thinking of french dev...
Thanks for anderstanding, nothing against you.

Here's a screenshot with "Correspondent above subject" enabled:

And here's one with the setting disabled:

As you can see, with the setting disabled, the subject is just repeated at the start of the preview, where I'd now expect to see the correspondent appear.

This only happens when switching the setting and returning to the message list. The list will be displayed properly if the message list screen is closed and reopened.

This should be fixed in recent beta versions (5.7xx).

authors: cketti frederiiiic madduck


text:
Play store version stuck at 5.206

Please search to check for an existing issue (including closed issues, for which the fix may not have yet been released) before opening a new issue: https://github.com/k9mail/k-9/issues?q=is%3Aissue
Expected behavior
The Google Play store version should be the latest - 5.207.
Actual behavior
The Google Play store version appears to be stuck at 5.206 for some users.
Steps to reproduce

Open https://play.google.com/store/apps/details?id=com.fsck.k9
Look in additional info section
Version is listed as 5.206. This is the same version installed on my Android devices and I am unable to force an upgrade to 5.207 through the Play Store.

Environment
K-9 Mail version: 5.206
Android version: 7.0
A quick IRC chat with Khaytsus indicates that they are seeing 5.207.  However, I am not the only person seeing the version stuck at 5.206.  See https://android.stackexchange.com/questions/174070/google-play-store-dont-show-latest-update-for-k-9-mail.
Here is a copy-and-paste from https://play.google.com/store/apps/details?id=com.fsck.k9 on my Mac:
ADDITIONAL INFORMATION
Updated
22 March 2017
Installs
5,000,000 - 10,000,000
Current Version
5.206

Thanks. 5.207 was only released for alpha testers. I pushed that version to the release channel just now.

Fix confirmed. Google Play just notified me of the update and I've upgraded to 5.207 through the Play Store.
Thanks!


text:
Feature: add option to couple images in inbox to email identities instead of contact photos

I would love to have a clear indication in my inbox to which of my 10+ identities an email was send. I have no use for these big images of each contact. I would much rather display a picture or icon of the receiving identity, rahter than the sending contact.
This could be an optional setting.

So instead of those linked to the name of the contact, I would like them linked to the name of an identity.
(Btw I am sure there are loads people who do like it as is, I am merely suggestion to have an optional alternative to it)
Environment
K-9 Mail version:
5.400
Android version:
7.0
Account type (IMAP, POP3, WebDAV/Exchange):
IMAP

pulled that image of the interwebs btw, was to lazy to screencap my own inbox


text:
A strange German translation inconsistency

Hi there,
I'm currently using K-9 Mail v5.600 (installed from F-Droid) on three devices: a bq Aquaris M4.5 smartphone (Android 5.1), a Cubot Quest smartphone (Android 9) and a Samsung Galaxy Tab A 2016 tablet (Android 8.1). The language is set to German on each device. The inbox screen shows a translation inconsistency: The case of the first letter of the German word "in" depends on the device (or the Android version?)
See the screenshots:
** Cubot smartphone and Samsung tablet:**

bq smartphone:

Strange, isn't it? The correct spelling is shown on the latter screenshot with the lower-case "i". What's going on here?
The cycling cat

We were appending the output of DateUtils.getRelativeTimeSpanString() to "Next poll", or in your case "Nächster Abruf". Looks like that output is different depending on the Android version.
But this text has been removed and is no longer part of the app. Current beta versions are not affected by this. Closing this issue.

authors: cketti cyclingcat