text:
Improve DX for ntl build + ntl deploy command sequence

Problem
Some users will run the build command followed by deploy with the expectation of it being a two-step process, in which the first command creates a set of artefacts that the second command deploys. Unfortunately, this isn't always the case.
For example, imagine a site with a plugin that modifies certain configuration properties related to functions, via a onPreBuild hook. When running build, those changes will affect the functions bundling stage and the generated zip archives will have the expected contents. But if a deploy command follows, the functions bundling stage will run again, except this time the plugin will not run, so the changes to the configuration will not take place and a new set of zip archives (with the wrong result) will be deployed instead of the ones generated previously.
Proposed solution
When functions are bundled as part of the deploy command, the zip archives are placed in a .netlify/functions directory. The deploy command could treat this directory as a bundling cache, so that if exists, it skips the bundling stage and directly uploads the archives that have been previously generated.
Caching is always difficult and we want to make sure we minimise the risk of deploying the wrong functions, so here are a few scenarios and how CLI would respond.
Scenario 1: Using ntl deploy without ntl build
A user uses ntl dev or any other development server to develop their site locally. To deploy, they run their build command (e.g. npm run build) and then deploy to Netlify using ntl deploy. They don't use ntl build as part of their workflow.
In this scenario, CLI would continue to work in the same way, since the deploy command does not create a .netlify/functions directory, so the functions would be bundled each time.
Scenario 2: Using ntl deploy --build
A user uses ntl dev or any other development server to develop their site locally. To deploy, they use the netlify deploy --build command, which will build their site and deploy it in one go.
In this case, we can safely skip the functions bundling (see https://github.com/netlify/cli/issues/2596), since we know for a fact that the build and the deployment are an atomic operation.
Scenario 3: Using ntl deploy followed by ntl build
A user uses netlify dev or any other development server to develop their site locally. When they're ready to deploy, they use the ntl build to build their site followed by ntl deploy to finalise the deployment.
In this case, we know that .netlify/functions will contain the correct functions, bundled and ready for deployment, by the time ntl deploy starts, so we can skip a second bundling stage.
It's worth noting that if the user modifies the source of a function between the ntl build and ntl deploy commands, CLI will deploy the wrong function (i.e. the cached version instead of the newly-modified one). However, in this scenario we've established that the user relies on the CLI to build their site, so I think it's reasonable to expect that a deploy needs to be preceded by the ntl build command, which in its turn would update the cache for the next ntl deploy invocation.
We could put some extra measures in place to reduce the likelihood of us deploying the wrong functions, such as:

Show a message as part of ntl deploy with the timestamp of the functions being deployed and instructions for forcing a refresh. Something like:

Deploying X functions created on 2021-07-27 at 17:49:00. To bypass the cache, run netlify deploy --build

(Note: the deploy command already has the flag it needs to force the functions to be re-bundled: --build)

Set a TTL for the cached functions. When ntl deploy finds a .netlify/functions directory, it discards it if it's older than a pre-configured interval.

When ntl deploy runs and finds a .netlify/functions directory, it compares the hash of each function in the cache against the hash of the functions it finds in the configured functions directory. If any of the hashes differ, the cached function is discarded.
⚠ Note: We don't know which files contribute to the function output until we bundle it, so it only makes sense to create a hash of the function's main file. In practice this means that we wouldn't be able to detect changes to the function's supported or included files using this mechanism, making its relevance questionable.

Any thoughts are very much appreciated! 🙇🏻

i think these are all great scenarios!
in scenario 1, does ntl deploy run the plugins in place of ntl build? right now i expect that from ntl build or ntl deploy --build
scenario 2 i probably find the most "confusing" out of the three. i've always found it a bit weird that the deploy command has a --build flag when ntl build exists, but that could just be me!
scenario 3 we discussed on slack and is the workflow i'm most used to/comfortable with. i could definitely see a case for doing away with ntl build if scenario 1 somehow does everything. i think this would mimic vercel's api.

Linking some issues for context, https://github.com/netlify/cli/issues/179, https://github.com/netlify/cli/issues/1671 and https://github.com/netlify/cli/issues/1591
Some users were/are expecting ntl deploy to run a build. As Build Plugins role in the build increases, this expectation becomes stronger.
For the specific use case of function bundling I think we can always deploy from .netlify/functions if exists.
@eduardoboucas, we'll need to consider that in some cases (e.g. Essential Next.js plugin) function changes can be generated by plugins, for example a user changes their site's code, the plugin generates a different function as a result, so we won't know which functions changes until we run a build.
I think we can experiment with TTL/Timestamps and see how it goes.
Another option is to do a breaking change in ntl deploy to always run a build and provide a flag to skip it.
Also we might provide additional warnings if the user has plugins configured (as those are the main reason to use ntl build at the moment).

Before anything else, I should clarify that the goal of the three scenarios above isn't to pick one flow in favour of the others. I was just trying to capture ways in which people might use the CLI today, and how well the proposed solution stacks up against people's expectations.
Now to @lindsaylevine's points:

in scenario 1, does ntl deploy run the plugins in place of ntl build? right now i expect that from ntl build or ntl deploy --build

No. Plugins would continue to run only on ntl build and ntl deploy --build.

scenario 2 i probably find the most "confusing" out of the three. i've always found it a bit weird that the deploy command has a --build flag when ntl build exists, but that could just be me!

I think there's place for both. If you know you want to build and deploy, I think it's useful to have a single command for that sequence of operations. But you might want to simply build the site, in which case it's important that ntl build exists on its own.
And @erezrokah's points:

For the specific use case of function bundling I think we can always deploy from .netlify/functions if exists.

I think this would solve the majority of problems we're currently seeing with the relationship between these two commands, since any functions created by ntl build (e.g. via plugins) would be respected in ntl deploy.

@eduardoboucas, we'll need to consider that in some cases (e.g. Essential Next.js plugin) function changes can be generated by plugins, for example a user changes the site's code, the plugin generates a different function as a result, so we won't know which function changes until we run a build.

Yes, I shouldn't even have mentioned point 3, despite the alert. I think any attempts to diff the functions will be problematic, so we should avoid going down that path.

Some users were/are expecting ntl deploy to run a build. As Build Plugins role in the build increases, this expectation becomes stronger.
(...)
Another option is to do a breaking change in ntl deploy to always run a build and provide a flag to skip it.
Also we might provide additional warnings if the user has plugins configured (as those are the main reason to use ntl build at the moment).

That's interesting! If ntl deploy runs a build, why would you ever use ntl build? I fear this would not help with the confusion between the two commands, and might actually contribute to it.
Also, one consequence of that breaking change would be people building their sites twice, if their workflows use ntl build + ntl deploy. This would catch people off-guard, since builds would continue to succeed but they would start taking twice as long.
I'm not saying I'm against it, I just think we need to have a deeper think about the commands we make available if we decide to go down this path, and whether some of them become redundant.

authors: eduardoboucas erezrokah lindsaylevine

repo: netlify/cli


text:
Add function like env:export to make .env files from site config

Is your feature request related to a problem? Please describe.
When moving from one dev machine to an other i want to be able to quickly setup my .env file with secrets.
Describe the solution you'd like
I'm already using the env:import command to move all the .env content into netlify, that same system could be reversed to setup the new dev environment
Describe alternatives you've considered
At this time i can use env:list and write the file manually, but having an env:export which automatically formats the output would be nice
Additional context
Add any other context or screenshots about the feature request here.
Can you submit a pull request?
No. I don't really trust my own skills, but will look into it based on the env:list when i get some time.

Pull requests are welcome! If you would like to help us add this feature, please check our
contributions guidelines.

Hi @N0K0! I agree there are times when this can be useful. Do you ever use netlify dev to run your site with env vars automatically included?

Hi! Personally i've been using an alternative to the netlify dev command, since from what i can see running nuxt makes the rest of the netlify dev command hang while waiting for it to finish! And the default command, nuxt build removes all the nice dev features like Hot Module Reload from Nuxt.
Been using the following instead:
  "scripts": {
    "build": "nuxt build",
    "generate": "nuxt generate",
    "nuxt": "nuxt",
    "serve_func": "netlify functions:serve",
    "full": "concurrently --kill-others \"npm run nuxt\" \"npm run serve_func\""
  },

The full command gives me HMR for both functions and Nuxt :muscle:

@erezrokah
We currently output environment variables as easy to read ASCII tables:
> netlify env:list

site: my-first-site
.--------------.
| Environment variables |
|--------------|
| Key  | Value |
|------|-------|
| TEST | mucho |
| KEY  | 1234  |
'--------------'

Obviously, putting these in an .env file and then cleaning them up is tedious. So I do see the need of something like env:export.
But, since the code for env:export will almost be exactly the same as env:list, it doesn't make sense create a separate command for it.
Instead, we could provide a flag like --plain which would output something like this to the command line:
> netlify env:list --plain

TEST=mucho
KEY=1234

Further on this:
We could either take a path argument to write to since people also use .env.(dev|local|staging|prod) OR we could just list these variables and leave it to the users to redirect the output to the appropriate file.
with path
> netlify env:list --plain .env

with redirection
> netlify env:list --plain > .env

I feel like we should go with the 2nd option (with redirection) since I wouldn't normally expect a list command to create a file.
What are your thoughts on this?

Just as good if not an better alternative in my eyes!
If its easy to add the flag i might be able to do it :sweat_smile:
Not familiar with the codebase really :)

I feel like we should go with the 2nd option (with redirection) since I wouldn't normally expect a list command to create a file.

👍 to this

@erezrokah Should I take this one up since @N0K0 is unfamiliar with the codebase?

@tinfoil-knight, looks like this is a good opportunity to get @N0K0 involved.
@N0K0 would you like to take this on with @tinfoil-knight guiding you?

@erezrokah Should I take this one up since @N0K0 is a bit unfamiliar with the codebase?

@erezrokah Should I take this one up since @N0K0 is a bit unfamiliar with the codebase or try to guide around @N0K0?
Edit: My previous comment got deleted due to a mis-click.

https://github.com/netlify/cli/issues/3262#issuecomment-959785013 is a response to ⬆

Just noticed that the --json flag is not exposed in the relevant help output, not directly related, but a hint would be nice :+1:
❯ netlify env --help
(Beta) Control environment variables for the current site

USAGE
  $ netlify env

OPTIONS
  --debug                                                      Print debugging information
  --httpProxy=httpProxy                                        Proxy server address to route requests through.
  --httpProxyCertificateFilename=httpProxyCertificateFilename  Certificate file to use when connecting using a proxy server

EXAMPLES
  netlify env:list
  netlify env:get VAR_NAME
  netlify env:set VAR_NAME value
  netlify env:unset VAR_NAME
  netlify env:import fileName

COMMANDS
  env:get     Get resolved value of specified environment variable (includes netlify.toml)
  env:import  Import and set environment variables from .env file
  env:list    Lists resolved environment variables for site (includes netlify.toml)
  env:set     Set value of environment variable
  env:unset   Unset an environment variable which removes it from the UI

❯ netlify env:list --help
Lists resolved environment variables for site (includes netlify.toml)

USAGE
  $ netlify env:list

OPTIONS
  --debug                                                      Print debugging information
  --httpProxy=httpProxy                                        Proxy server address to route requests through.
  --httpProxyCertificateFilename=httpProxyCertificateFilename  Certificate file to use when connecting using a proxy server

Just noticed that the --json flag is not exposed in the relevant help output, not directly related, but a hint would be nice

Yup. I saw that too while going through the codebase. We could document the --json flag in a separate PR. Nice catch nonetheless.

Looking at https://github.com/netlify/cli/pull/371 for hints :)
Hopefully I'll have some time to fix it soon. Need to read up on contrib and such too :+1:

@verythorough Do you think this issue is still relevant? If yes, I'll add the feature otherwise we can just close this issue.

@tinfoil-knight yes, I think this would still be useful! Your proposal for the --plain flag (without the path argument) sounds great, and if you can document --json at the same time, that'd be awesome.
Side note: for folks who may be looking to use env:list and env:import to copy env vars from one site to another, we now have env:clone (docs), which handles that in one step. ✨

source: gharchive/issue

authors: N0K0 erezrokah tinfoil-knight verythorough

license: MIT

license_type: permissive

license_source: github-api


text:
Using --alias with invalid characters "slugifies" the string in the shell but sends the characters unchanged to Deploy.subdomain_alias.

- Do you want to request a feature or report a bug?
A bug.
- What is the current behavior?
When using --alias to create a custom subdomain for a deploy, characters which would be invalid for a domain name are "slugified" in the shell URL but the database is still getting the unmodified string.
- If the current behavior is a bug, please provide the steps to reproduce.
Here is an example deploy:
$ netlify deploy --alias="not/working"
Deploy path:        /Users/me/data/git/overlordofmu/netlify-cli-manual-deploys/public
Configuration path: /Users/me/data/git/overlordofmu/netlify-cli-manual-deploys/netlify.toml
Deploying to draft URL...
✔ Finished hashing 8 files
✔ CDN requesting 0 files
✔ Finished uploading 0 assets
✔ Deploy is live!

Logs:              https://app.netlify.com/sites/netlify-cli-manual-deploys/deploys/5efaea211010bad6650a7350
Website Draft URL: https://not-working--netlify-cli-manual-deploys.netlify.app

If everything looks good on your draft URL, deploy it to your main site URL with the --prod flag.
netlify deploy --prod

However, if I check the actual content of Deploy.subdomain_alias it is "not/working" instead of not-working. The URL in the deploy above does not work: https://not-working--netlify-cli-manual-deploys.netlify.app
- What is the expected behavior?
That the same "slugified" string shown in the shell command output be used for the subdomain_alias value and the given URL in the shell will then work.
- Local Environment Information
$ netlify version --verbose
────────────────────┐
 Environment Info   │
────────────────────┘

  System:
    OS: macOS 10.15.5
    CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
  Binaries:
    Node: 12.18.0 - ~/.nvm/versions/node/v12.18.0/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.14.4 - ~/.nvm/versions/node/v12.18.0/bin/npm
  Browsers:
    Chrome: 83.0.4103.116
    Firefox: 77.0.1
    Safari: 13.1.1
  npmGlobalPackages:
    netlify-cli: 2.54.0

netlify-cli/2.54.0 darwin-x64 node-v12.18.0

** Follow-ups **
https://community.netlify.com/t/netlify-cli-alias-urls-not-found/17831

This is an issue with our API, I opened an issue for it internally. It looks like the CLI should not slugify the branch argument when invoking the deploy API call.

I've hit this issue now as well but regarding upper-case letters. --alias nicename works while --alias niceName doesn't. What are your suggested method to deal with this at the moment? I want to use aliases based on branch names, is there any certain transform I can apply to them to make them work? Slugify?

@HugoGranstrom, "slugify" meaning "to take a string and make it suitable to be used as a slug".
The definition of "slug" itself can be found here: https://en.wikipedia.org/wiki/Clean_URL#Slug
Our slugs (which is also true of most but not all slugs) are always lowercase. It won't be possible to use mixed case slugs. We could request a feature request for this but I'm 100% certain that our service will never be able to support mixed case slugs. In other words, I'm 100% sure the feature request will never get made.
So, it may be helpful to ask: "Why am I 100% sure we won't support mixed case slugs?"
I'm sure because, to support them, we would need to differentiate between cases at the DNS level. We would need to treat nicename--example.netlify.app and niceName--example.netlify.app as two different domain names. However, DNS itself ignores case. Because DNS itself lacks support for case-sensitivity, we would not be able to support mixed case slugs.
To summarize, a mixed case slug won't happen because the slugs get used for subdomains and DNS has no concept of uppercase or lowercase.
Also, please be warned that the --alias option is intended to be used as a single use alias. If you reuse the same alias for a future deploy those aliases are not branch deploys and branch deploy features will not work for those deploys. The only way to use branch subdomain features currently is to do the build at Netlify.
There is an open feature request for true branch subdomain support with manual deploys here: https://github.com/netlify/cli/issues/1572.

@overlordofmu Thank you sincerely for your answer :D

I'm sure because, to support them, we would need to differentiate between cases at the DNS level. We would need to treat nicename--example.netlify.app and niceName--example.netlify.app as two different domain names. However, DNS itself ignores case. Because DNS itself lacks support for case-sensitivity, we would not be able to support mixed case slugs.

If this is the case, why isn't this limitation documented anywhere? I expected Netlify CLI to at least tell me that something was wrong about my alias if it doesn't fit the template instead of just giving me a broken link, don't you agree? Some solution is to be expected to this at least, for example fixing the bug this issue was opened for so that the printed and deployed URLs actually match (both are the slugified string for example) (Especially since it has been a year now since this was found and I'm certainly not the only one who has gotten headaches trying to figure this out).

Also, please be warned that the --alias option is intended to be used as a single use alias. If you reuse the same alias for a future deploy those aliases are not branch deploys and branch deploy features will not work for those deploys. The only way to use branch subdomain features currently is to do the build at Netlify.

I'm totally fine with it overwriting the last instance and I have no need for branch deploy whatever that is, I'm just deploying a prebuilt directory anyway and I just want a human-readable URL. But because of this, I'm just using the the commit hash for now :)

authors: HugoGranstrom erezrokah overlordofmu


text:
chore: cleanup

This does some devtooling cleanups:

use fs.rm instead of fs.rmdir because fs.rmdir logs a deprecation warning.
lock file maintenance, all dependencies in lock-file are updated to latest valid version
update eslint config, to fix unsupported typescript warning when eslint runs.
Instead of FORCE_COLOR=0 environment variable use chalk.level=0 so that the tests work when not run through npm
Remove ava rule in renovate config, as we do not use ava here anymore.
Increase test timeout to 30s

I feel like you're the king of maintenance ✨

I ❤ maintenance.

source: gharchive/pull-request

repo: netlify/edge-bundler


text:
Fix button alignment on older safari

- Summary
Fix issue with misaligned button icons on older browsers. - closes #914
This issue was connected to a flex-bug, that disallowed adding display: flex to certain HTML elements such as <button>
https://bugs.webkit.org/show_bug.cgi?id=169700
- Test plan
Testing this fix requires an older version of safari, or access to browserstack 😞 (I tested it, but it would be good, for somebody else to also take a look).
- Description for the changelog
Fix issue with misaligned button icons on older browsers.
- A picture of a cute animal (not mandatory but encouraged)

Blob fish is the best

Awesome, I'm really happy it works :)

repo: netlify/netlify-cms


text:
req.headers.host is undefined on netlify deployment

I'm trying to read the host in getServerSideProps. It works in local environment with
next dev (obviously)
next build && netlify dev
However when deployed on netlify with this plugin,
req.headers.host is undefined. Any info on this? Thank you.

Hi @s-kris, thanks for posting the issue!
Hmm, you are right. I just tested with a plain Netlify Function to see what information is available:
exports.handler = function (event, context, callback) {
  callback(null, {
    statusCode: 200,
    headers: {
      "Content-Type": "application/json; charset=utf-8",
    },
    body: JSON.stringify({
      event,
      context,
    }),
  });
};

See results here: https://next-on-functions.netlify.app/.netlify/functions/print
Seems like host is available on headers, but is not available on multiValueHeaders (which is used by next-on-netlify). But we can manually add the host to the multiValueHeaders, that should be fine.
I'll let you know when it's done! :fire:

Awesome, thank you @FinnWoelm

Hi @FinnWoelm, do you have some update about it?

Hi, I open a really simple PR that solves this issue. I need to do this to my project, just in case my PR be useful to you, I opened the PR. Case not, just close it.

We are currently running into this issue and it prevents us from using next-on-netlify in production, do you have an ETA when this is going to be merged and released?

@FinnWoelm can we solve this issue quickly? Our whole business depend on it :)

Hi all, sorry for the delay in getting this out the door! We were busy refactoring a lot of the codebase last week.
@lindsaylevine just merged the fix into main. The changes should be published to npm within the next 24 hours. If you prefer not to wait, you can already use the latest changes by running npm install netlify/next-on-netlify#main --save. This will give you the cutting-edge version of next-on-netlify right away.
PS: We are super excited that @lindsaylevine has joined Netlify and the next-on-netlify team! :raised_hands:  @lindsaylevine has a ton of experience and will help push next-on-netlify to the next level. You will be seeing a lot of @lindsaylevine going forward :grin: :clap:

@FinnWoelm @lindsaylevine That's great to hear. Looking forward to new major features. (hint: pure next-inc-ssg support)
Thanks for your awesome work!

Just wanted to add here that I added the 'host' header to the multiValueHeaders object for functions on the netlify-side of things. It's currently being deployed but should be live on our regular CDN and will be live on our Enterprise CDN soon.

authors: FinnWoelm Yegair abdelhamid-attaby depadiernos leomeneguzzi s-kris

repo: netlify/next-on-netlify


text:
Update Next.js

Fix name, homepage, twitter for Next.js

Thanks! The Twitter handle will need to be specific to the project, as follower count is treated as a metric. I don't believe Next.js has one.

Correct, Next.js doesn't have its own twitter handle (yet)
I removed the twitter line

authors: erquhart styfle

repo: netlify/staticgen

license: mit

license_source: bigquery


text:
Fixes a couple of minor weaknesses

Description
Some weaknesses were detected, fixing here a bunch of them, especially inconsistent null-checks or suppressed errors
Dependencies

n/a
Checklist
If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

[ ] Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
[ ] Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
[ ] Does this PR require product documentation?

[ ] If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.

[ ] Does this PR require a product release notes entry?

[ ] If so, fill in "Release Note Text" in the JIRA.

[ ] Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.

[ ] If so, make sure it is described in the JIRA ticket.

QE requirements (check 1 from the list):

[ ] Standard QE validation, with pre-merge tests unless stated otherwise.
[ ] Regression tests only (e.g. refactoring with no user-facing change).
[x] No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

thanks @KalmanMeth
/approve

repo: netobserv/flowlogs-pipeline


text:
Animated WebP image issue using LoadTexturesAsync

using unity.webp 0.3.11
Due to our application scenario, asynchronous loading is required.
There is an issue of image output when you call the method below and use it.

WebP.Experiment.Animation.WebP.LoadTexturesAsync(byte[])

We tested 3 images.

https://colinbendell.github.io/webperf/animated-gif-decode/2.webp
https://colinbendell.github.io/webperf/animated-gif-decode/3.webp
https://mathiasbynens.be/demo/animated-webp-supported.webp

There is a problem with the pixel looking broken as shown in the screenshot below.

And the error log below is also output.
[WebPDecoderWrapper] Bytes to texture error: LoadRawTextureData: not enough data provided (will result in overread).

I also have same problem of that. There is something what I can't understand when passing decoding option. So please avoid to use Experiment that function.

check Samples/example2/animation.unity then moidfy WebpAnimation.cs logic to fit your game.

@netpyoung
As I checked, the WebPDemuxGetFrame function returns the width and height values abnormally.

@dw8869
It seems so. WebP.Experiment.Animation.WebPDecoderWrapper.CreateDecoder couldn't create proper decoder.
I also tried to modify to change decoder's behaviour via options but failed until now.

any updates?


text:
Request: luks encryption option

This may be the wrong way to go about it, but is there any option that could create a luks encryption unattended installer?

It's been a while since I ran a manual installation, but if the default installer provides an option for it (I believe it does for the home directory?), then it should be possible and a matter of getting the right questions answered. If not, you would need to use a postinstall task of some sort.

changing
partman-auto/method                                     string      lvm
to
partman-auto/method                                     string      crypto
in
netson.seed
will enable luks encryption

authors: netson rotflol slimxwhitman

repo: netson/ubuntu-unattended


text:
Query and error logs

Previously, a TomCat Java server handeled the logging but after its removal, this application has to do it.
This issue should be used to discuss the log format, how logging will be enabled, and where logs are saved (= the responsibilities of this application).

Current format for args:
queryTime,[TAB]queryText[TAB]ipAddress[TAB]url[NEWLINE]
(https://git.webis.de/code-research/arguana/args/args-framework/-/blob/master/src/main/java/me/args/server/QueryParser.java#L130)
But if we want to extend this to several services, we should rather use some key-value-based format to allow for special cases that some services have and others have not. We then need a common vocabulary for the usual fields.
Plausible would be one JSON per line (as suggested by Michael). We can first discuss here, but should then find a place for the result. Probably in the web services notes: https://webis.de/facilities.html?q=service
What could we use as a starting point?
Also pinging @mam10eks in case he wants to chime in.

I am indeed interested in that topic ;)
I think logging the important data  (query logs) using JSONL to CephFS makes sense.
But I guess that for error logs or similar we may use services that we already have running in our infrastructure: I think ChatNoir (maybe others too?) use LOKI for that.
Maybe @phoerious also has an oppinion on that?

So which fields should we standardize?

Some date-time field (various names are plausible, as the format one could use either the HTTP ones or "2020-09-24T06:29:42Z"... I have a preference for the latter as it is a bit easier to work with). "timestamp" seems to be a widespread name for this.
Some identifier so that we can match queries of the same user and remove our own test queries in later analyses. I think the IP address is the best thing we have for this, even if we then have to anonymize the logs before processing. Args will likely in the future also have a "user name" (a random string stored in the browser) that could be placed in this field instead of the IP address (we will get a few session features in args, and I want to be able to transfer my session across devices... unsure whether this option should be available to the end-user). Maybe just "user"?
The full request URL (important for me to distinguish usage of the GUI from API calls and for all the different parameters that args supports)... Maybe "url"?

It is also good to have a "query" field with the query text. This will allow us to log also other requests into the same log, where the queries can then be identified easily and one does not need to parse the URL to work with them.

I have had the same thoughts for ChatNoir already. My conclusion was that logging to Elasticsearch would be the best thing. For long-term storage, the Elasticsearch bulk format could be used. I would not log directly to CephFS, since that only creates problems when logging from multiple instances at the same time.

the HTTP ones or "2020-09-24T06:29:42Z"... I have a preference for the latter as it is a bit easier to work with). "timestamp" seems to be a widespread name for this.

That's an ISO datetime and the standard format in both Python and Elasticsearch. HTTP dates are notoriously hard to work with.

How easy is it to export the Elasticsearch logs to a text file? It is a requirement for args to easily get access to the logs in a text format. But if there is somewhere a button that you can click (accessible over VPN), I think this would be fine.
We solved logging from multiple instances for our purposes by using the hostname in the filename.
Logging to the filesystem is nice when I start a server locally for testing, but it would be ok-ish for me to have different possibilities here (for testing/production).

the HTTP ones or "2020-09-24T06:29:42Z"... I have a preference for the latter as it is a bit easier to work with). "timestamp" seems to be a widespread name for this.

Let's please use the ISO format. Virtually every language has native support for this and I don't want to use a third-party library just for parsing timestamps.

The full request URL

gRPC/gRPC-web services don't use URL params. They use Protobuf messages in the request body, so we have to log that as well (Protobuf has native support for converting messages to JSON). So gRPC services need to log a "url" (?) and "message" (?) field.
(I don't really like "message", it's way too general. Anyone has some better ideas?)

To summaries: Every log entry should look like this so far:
{
  "timestamp": string, // ISO date time
  "user": string, // identifier for the current user
  "url": string, // the full request URL
  "message"?: unknown, // the request Protobuf message (only for gRPC services)
  "query"?: string, // (optional) the query text of the request
}

Services are allowed to add more custom fields.

Looks good! Not sure if we should then make the "url" mandatory.
Should we create an own Java repo for creating/writing/parsing log records? Or just have one document at some place that describes the fields? Like in the FAQ or the services howto?

Not sure if we should then make the "url" mandatory.

I don't see why one wouldn't want to log it. Could you an example?

Should we create an own Java repo for creating/writing/parsing log records? Or just have one document at some place that describes the fields? Like in the FAQ or the services howto?

We should most definitely write this down instead of having just a reference implementation. Maybe in FAQ under "How to do logging"?
I don't know if we need a dedicated Java project for that for now. How many services will use this kind of logging?
(This is a C++ project, so a Java library isn't going to help me here)

If a service has just one URL, it does not make much sense to log it.
At the moment I'd indeed then favor a FAQ entry, but I have to think a bit more about this. However, in order to get things done, it might be good to just start the FAQ entry and then move it somewhere else if necessary (but keep a link). I agree that this has priority over a reference implementation.

I also agree that elasticsearch makes sense, and from the application-view it should not make any overhead, since probably all popular logging-libraries should have a logstash plugin.
@johanneskiesel: I think the best way to transform the elasticsearch logs into plain text files like JSONL is by using the scroll-api. (Janek recommendet this in the wstud-stustu-kolleg channel.)
@RunDevelopment Since this is a c/c++ project, do you know good logging libraries that may have out of the box support for logstash? I did just googled a bit and found that log4cplus for example does not seem to have out of the box support (e.g. here).

Ok, I will have a look at logstash. But Janek suggested that it is possible to first write to disk and then use Logstash to write Batches to Elasticsearch. I think I like this option a lot. Then the log files are already accessible as text, but we still get the uniform place and analytical powers of Elasticsearch.
I started the entry here: https://git.webis.de/code-generic/code-webis-faq/-/blob/master/README.md#how-to-code-logging
Feel free to extend/improve

Since this is a c/c++ project, do you know good logging libraries that may have out of the box support for logstash?

I thought about using boost::log since I'm using boost anyway but it's not a hard preference. It seems pretty easy to add a logstash sink for boost::log but I haven't found any C++ logging lib that supports logstash out of the box (given that one only needs a few line of code, I can understand why).

Nice!
But even when it is only a few lines, this is (maybe) complex:

What is when the remote service is (temporarily) not available or does not accept writes?

Our elasticsearch cluster shares the infrastructure with our hadoop cluster. E.g., I encountered several times the problem that elasticsearch was not able to write a single new record (solving this required some manual steps. Elasticsearch stays in this state until someone fixes this manually).

Still, logging should have no impact at the application's performance, even when the remote service is currently not available and there is a queue of pending log messages.

Of course logging does not need to have to be perfect, but I would hope that available implementations/plugins would apply some reasonable best practices fur such stuff.

Still, logging to the file-system can also fail (e.g. temporarely problems with CephFS). So I guess it would still be ok to log directly to elasticsearch, but I would feel more save when we additionally add an prometheus alert (e.g. no submitted queries within the last 48 hours) that we recognize potential problems after a few days.

authors: RunDevelopment johanneskiesel mam10eks phoerious


text:
HpackEncoder lookup performance for headers with unique values

Profiling one of our services which uses gRPC, I noticed that the call to HpackEncoder::getEntryInsensitive is cpu intensive.
The reason is that we are using tracing, which is implemented by having a header with trace id as value. So each header value is unique.
This is a worst case scenario for getEntryInsensitive, as the hash map it uses is keyed by header name. So there's a single bucket with dozens of entries, all of which have the same name but different values, and all of them are traversed each time the tracing header is being looked up. Of course if there is any other header which happens to fall to the same hash bucket, it suffers the same fate.
A possible workaround is to make the header sensitive, so that it won't be added to the hash map. But this is a bit hackish, as it's not a sensitive header. Also, HpackEncoder is not configurable using grpc-java.
A more robust approach is to have an additional hash map keyed by header name and value, instead of just the name. I implemented this approach, so if there's interest I can create a PR.
WDYT?

Please create PR with benchmark results.

Fixed by https://github.com/netty/netty/pull/12665

authors: amirhadadi hyperxpro normanmaurer

repo: netty/netty


text:
ResourceLeakDetectorTest.testConcurentUsage failure

Expected behavior
ResourceLeakDetectorTest.testConcurentUsage should always pass.
Actual behavior
ResourceLeakDetectorTest.testConcurentUsage failed while building on the CI server.
[22:46:42] :             [io.netty:netty-common] Tests in error:
[22:46:42] :             [io.netty:netty-common]   ResourceLeakDetectorTest.testConcurentUsage:89->Object.wait:-2 » TestTimedOut ...

https://garage.netty.io/teamcity/viewLog.html?buildId=14412&buildTypeId=pulls_netty&tab=buildLog
Steps to reproduce
Build on the CI server
Minimal yet complete reproducer code (or URL to code)
N/A (existing unit test).
Netty version
4.1.7-SNAPSHOT
PR https://github.com/netty/netty/pull/6106 (assumed to be unrelated)
JVM version (e.g. java -version)
Java HotSpot(TM) 64-Bit Server VM
jdk1.8.0_112
OS version (e.g. uname -a)
Linux, version 3.16.0-4-amd64
https://garage.netty.io/teamcity/agentDetails.html?id=7&agentTypeId=7&realAgentName=debian7-1

@normanmaurer - PTAL

Will check

Am 08.12.2016 um 01:24 schrieb Scott Mitchell notifications@github.com:
@normanmaurer - PTAL
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@Scottmitch from the thread dump it seems like it was just not complete yet as it still created new leaks etc. I will increase the timeout .

Fixed by https://github.com/netty/netty/pull/6120

authors: Scottmitch normanmaurer


text:
SSLHandler slow

Expected behavior
Minimal SSL overhead (max 5-10 %) with Java SSL.
OpenSSL fails with "io.netty.util.internal.OutOfDirectMemoryError", will open separate issue
Actual behavior
Pure Java SSL > 3 times slower than without SSL
Steps to reproduce
Run the provided code
Minimal yet complete reproducer code (or URL to code)
https://github.com/floragunncom/netty-memleak/tree/bigsslslow (see README)
https://github.com/floragunncom/netty-memleak/releases/tag/slow1
Logs: https://gist.github.com/floragunncom/8e52824974675d3de523ff1ef1aa478f
Netty version
4.1.9.Final
JVM version (e.g. java -version)
Java Version: 1.8.0_102 Oracle Corporation
JVM Impl.: 25.102-b14 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM
OS version (e.g. uname -a)
Mac OS X x86_64 10.10.5

maybe related to https://github.com/netty/netty/issues/6139

@floragunncom - Are you writing large amounts of data? If so please try https://github.com/netty/netty/pull/6569.

@Scottmitch #6569 ist not the problem here i think because pure Java SSL is also slow

@Scottmitch @normanmaurer What is the expected max. overhead of using SSL in terms of data throughput with recent Java 8 on modern hardware?

PR #6569 only applies to the OpenSSL based engine, which is why I suggested trying this. However it looks like the OpenSSL memory issue is being tracked in https://github.com/netty/netty/issues/6572.
The maximum overhead may depend upon a few things ... hardware support for encryption primitives, cipher used, etc... The Java SSLEngine is known to be slow and introduce relatively high amounts of overhead in certain configurations. This is why we go through the trouble of supporting a SSLEngine based upon OpenSSL.

With 4.1.10.Final-SNAPSHOT (and fixed #6572) i get now some measurements:

No ssl -> 482 ms
Java SSL -> 5000 ms
OpenSSL -> 1392 ms
OpenSSL Refcount -> 1482 ms

OpenSSL is 3 times slower than without ssl (but > 3 times faster than Java SSL)
So OpenSSL add an overhead of approx. 300% to SSL communication. Seems way to much for me?

@floragunncom I am not sure what is the expected overhead of SSL. I guess it also really depends on the cipher etc. Did you check what kind of overhead you for example see with a C app ?

I started with creating a simple java program which transfer bytes over simple plain sockets (non-ssl socket vs. ssl sockets). SSL is approx 100% slower than without. This also not really what i expected but its a lot faster compared to the netty tests.
Results:

Mac14,1
4 x 2,7 GHz Intel Core i5

OS: Mac OS X x86_64 10.10.5
Java Version: 1.8.0_131 Oracle Corporation
JVM Impl.: 25.131-b11 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
TLSv1.2
200 mbyte in 7433 ms
28 mbyte/sec
224 mbit/sec

OS: Mac OS X x86_64 10.10.5
Java Version: 1.8.0_131 Oracle Corporation
JVM Impl.: 25.131-b11 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM
no ssl mode
200 mbyte in 3518 ms
66 mbyte/sec
528 mbit/sec

https://github.com/floragunncom/netty-memleak/tree/simpleperf
https://github.com/floragunncom/netty-memleak/releases/tag/simpleperf1

@floragunncom I will have a look tomorrow.

@floragunncom please note that your different Main* classes do different things in terms of how these are echo back data. You should build your server on top of the EchoServer if you want to compare it. I hacked up something very quickly here and actually the perf is even better in netty compares to just use plain sockets (I still used java sockets for the client side tho to make it easier to compare). Also in your code you also do an extra  memory copy for each write by calling data.copy() which is also not what you do in your socket example where you not do an extra memory copy each time.
So to make it short your two tests are not really comparable.

@normanmaurer I think you got me wrong. Its not about the absolute figures, its about the ratio between SSL and nonSSL figures. So i'am not comparing netty vs. the plain socket impl, i compare the ratio between SSL/nonssl for netty and the ratio between SSL/nonssl for plain socket.
Netty: Java SSL ist about 6 times slower than without SSL
Plain Socket: Java SSL is about two times slower than without SSL

@normanmaurer any idea how we can proceed here?

@floragunncom - There are a few PRs pending which may impact the performance of the SslHandler. Please try the following PRs https://github.com/netty/netty/pull/6365 https://github.com/netty/netty/pull/6720.
I very briefly looked at the code and the server does a writeAndFlush of 5 bytes on every read to every channel [1]. Each TLS packet will have at least this much overhead (and more overhead after the handshake) for framing/encryption/etc... Perhaps try and aggregate data (which is what https://github.com/netty/netty/pull/6720 does) and simplify the test case to remove the collection of channels to remove some variability.
[1] https://github.com/floragunncom/netty-memleak/blob/bigsslslow/src/main/java/io/netty/example/securechat/SecureChatServerHandler.java#L63

Maybe related to https://github.com/netty/netty/issues/3318

@floragunncom - Please update your example and try the PRs as requested above. The smaller the size of each plaintext write the larger the overhead will be for TLS.

I will close this issue for now as there are issues as identified above and suggestions made to improve the measurement. Feel free to re-open after the suggested approaches have been implemented and please report the specific impact and direction taken.

authors: Scottmitch floragunncom normanmaurer


text:
Correctly handle ChannelInputShutdownEvent in ReplayingDecoder

Motivation:
b112673554bafc1eccfd43913a3e8605337dd7fb added ChannelInputShutdownEvent support to ByteToMessageDecoder but missed updating the code for ReplayingDecoder. This has the effect:

If a ChannelInputShutdownEvent is fired ByteToMessageDecoder (the super-class of ReplayingDecoder) will call the channelInputClosed(...) method which will pass the incorrect buffer to the decode method of ReplayingDecoder.

Modifications:
Share more code between ByteToMessageDEcoder and ReplayingDecoder and so also support ChannelInputShutdownEvent correctly in ReplayingDecoder
Result:
ChannelInputShutdownEvent is corrrectly handle in ReplayingDecoder as well.

@trustin PTAL

LGTM

Cherry-picked into 4.0 (f94edd2e92f6e978c0b911aabbd101df54d86279) and 4.1 (6108b7297b0cb8359dc2f486b01ab440692b9433)

authors: normanmaurer trustin


text:
你好参数都对。无法联网，怎么处理呢？

你好参数都对。无法联网，怎么处理呢？

Log 文件贴一下

authors: networkextension vanxv


text:
vpp-forwarder panics after long time working

Expected Behavior
vpp-forwarder never panics.
Current Behavior
vpp-forwarder panics after long time working with multiple kernel clients.
Steps to Reproduce

Use basic setup.
Start single icmp-responder NSE and multiple (20) clients requesting kernel://icmp-responder/nsm-1.
Wait for the clients to start and connect to the NSE.
Delete all NSE, client pods (not deployment and replicaset).
Repeat 3-4 several times.
vpp-forwarder panics.

Failure Logs
Panic message
unexpected fault address 0x7f034a74e610
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x7f034a74e610 pc=0xb64f91]

goroutine 383993 [running]:
runtime.throw(0xd7459f, 0x5)
	/go/src/runtime/panic.go:1117 +0x72 fp=0xc00c416128 sp=0xc00c4160f8 pc=0x437bf2
runtime.sigpanic()
	/go/src/runtime/signal_unix.go:741 +0x268 fp=0xc00c416160 sp=0xc00c416128 pc=0x44eb08
git.fd.io/govpp.git/adapter/statsclient.(*statSegmentV2).GetStatDirOnIndex(0xc0003defa0, 0x7f034a38dc40, 0x6abc, 0x7f034a74e500, 0x7f034a74e580, 0x2f, 0x80, 0x4)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/adapter/statsclient/statseg_v2.go:77 +0x31 fp=0xc00c416168 sp=0xc00c416160 pc=0xb64f91
git.fd.io/govpp.git/adapter/statsclient.(*StatsClient).listIndexesFunc(0xc00058a140, 0xc00c416250, 0xd4, 0xc00abc77c0, 0x0, 0x0, 0x7f3bf544e400)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/adapter/statsclient/statsclient.go:488 +0xea fp=0xc00c4161f0 sp=0xc00c416168 pc=0xb62b0a
git.fd.io/govpp.git/adapter/statsclient.(*StatsClient).listIndexes(0xc00058a140, 0xc0046c9480, 0x1, 0x1, 0x10, 0xc0063e08a0, 0x0, 0x10, 0x0)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/adapter/statsclient/statsclient.go:469 +0x14c fp=0xc00c416280 sp=0xc00c4161f0 pc=0xb628ac
git.fd.io/govpp.git/adapter/statsclient.(*StatsClient).PrepareDir(0xc00058a140, 0xc0046c9480, 0x1, 0x1, 0x0, 0xc00c416448, 0x40e0f8)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/adapter/statsclient/statsclient.go:202 +0xb0 fp=0xc00c4163f0 sp=0xc00c416280 pc=0xb60770
git.fd.io/govpp.git/core.(*StatsConnection).updateStats.func1(0x989680, 0xc000000005)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/core/stats.go:148 +0x185 fp=0xc00c416498 sp=0xc00c4163f0 pc=0x8fdb25
git.fd.io/govpp.git/core.(*StatsConnection).updateStats(0xc00058a190, 0xc00058a1b8, 0xc0046c9480, 0x1, 0x1, 0xc60640, 0x13c2038)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/core/stats.go:166 +0x104 fp=0xc00c416558 sp=0xc00c416498 pc=0x8f9da4
git.fd.io/govpp.git/core.(*StatsConnection).GetInterfaceStats(0xc00058a190, 0xc00c4167c8, 0x0, 0x100000033)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/core/stats.go:311 +0x8f fp=0xc00c416740 sp=0xc00c416558 pc=0x8fb20f
github.com/networkservicemesh/sdk-vpp/pkg/networkservice/stats.retrieveMetrics(0xe90320, 0xc007a56fc0, 0xc00058a190, 0xc005afbab0, 0xc00b70be00)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk-vpp@v0.0.0-20210708105057-0bce7cb4ad25/pkg/networkservice/stats/common.go:44 +0xa5 fp=0xc00c4167f0 sp=0xc00c416740 pc=0xb684a5
github.com/networkservicemesh/sdk-vpp/pkg/networkservice/stats.(*statsServer).Request(0xc00008cac0, 0xe90320, 0xc007a56fc0, 0xc00ae9c500, 0xe90320, 0xc007a56fc0, 0xc00ba7f9c8)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk-vpp@v0.0.0-20210708105057-0bce7cb4ad25/pkg/networkservice/stats/server.go:56 +0x15f fp=0xc00c416868 sp=0xc00c4167f0 pc=0xb68dbf
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc000556db0, 0xe90320, 0xc007a56fc0, 0xc00ae9c500, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00c416928 sp=0xc00c416868 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc000206870, 0xe90320, 0xc007a56b40, 0xc00ae9c500, 0xc678a0, 0xc007a56b10, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c416998 sp=0xc00c416928 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc000a23680, 0xe90320, 0xc007a56390, 0xc00ae9c500, 0xc00b97ea58, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c416a08 sp=0xc00c416998 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc007a56390, 0xc00ae9c500, 0xc678a0, 0xc000a23680, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00c416a68 sp=0xc00c416a08 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc000a236e0, 0xe90320, 0xc000a23bf0, 0xc00ae9c500, 0x16, 0xbe9401, 0xbe94e0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00c416ad8 sp=0xc00c416a68 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/mechanisms/sendfd.sendFDServer.Request(0xe90320, 0xc000a23bf0, 0xc00ae9c500, 0xe90320, 0xc000a23bf0, 0xc00080f340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/mechanisms/sendfd/server.go:40 +0x8b fp=0xc00c416c80 sp=0xc00c416ad8 pc=0xa9ad6b
github.com/networkservicemesh/sdk/pkg/networkservice/common/mechanisms/sendfd.(*sendFDServer).Request(0x13c2038, 0xe90320, 0xc000a23bf0, 0xc00ae9c500, 0xe90320, 0xc000a23bf0, 0xc00ba7f968)
	<autogenerated>:1 +0x50 fp=0xc00c416cc0 sp=0xc00c416c80 pc=0xa9b310
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc000556da0, 0xe90320, 0xc000a23bf0, 0xc00ae9c500, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00c416d80 sp=0xc00c416cc0 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc000206840, 0xe90320, 0xc000a236b0, 0xc00ae9c500, 0xc678a0, 0xc000a23680, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c416df0 sp=0xc00c416d80 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc0004e4450, 0xe90320, 0xc0004e56b0, 0xc00ae9c500, 0xc00b97eeb0, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c416e60 sp=0xc00c416df0 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc0004e56b0, 0xc00ae9c500, 0xc678a0, 0xc0004e4450, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00c416ec0 sp=0xc00c416e60 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc0004e44b0, 0xe90320, 0xc0004e4930, 0xc00ae9c500, 0x7f034ba08598, 0xc004e9d360, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00c416f30 sp=0xc00c416ec0 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/mechanisms/recvfd.(*recvFDServer).Request(0xc0001001b0, 0xe90320, 0xc0004e4930, 0xc00ae9c500, 0xe90320, 0xc0004e4930, 0xc00ba7f560)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/mechanisms/recvfd/server.go:66 +0x312 fp=0xc00c416ff0 sp=0xc00c416f30 pc=0xa98432
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc000556d90, 0xe90320, 0xc0004e4930, 0xc00ae9c500, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00c4170b0 sp=0xc00c416ff0 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc000206810, 0xe90320, 0xc0004e4480, 0xc00ae9c500, 0xc678a0, 0xc0004e4450, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c417120 sp=0xc00c4170b0 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00b5ddb30, 0xe90320, 0xc00567e810, 0xc00ae9c500, 0xc00b97f1e0, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c417190 sp=0xc00c417120 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc00567e810, 0xc00ae9c500, 0xc678a0, 0xc00b5ddb30, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00c4171f0 sp=0xc00c417190 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00b5ddb90, 0xe90320, 0xc00b5ddf20, 0xc00ae9c500, 0x0, 0x0, 0xc00080f340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00c417260 sp=0xc00c4171f0 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/updatetoken.(*updateTokenServer).Request(0xc0002180f0, 0xe90320, 0xc00b5ddf20, 0xc00ae9c500, 0xe90320, 0xc00b5ddf20, 0xc00ba7f170)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/updatetoken/server.go:50 +0xd8 fp=0xc00c4172a8 sp=0xc00c417260 pc=0xa91918
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc000556d80, 0xe90320, 0xc00b5ddf20, 0xc00ae9c500, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00c417368 sp=0xc00c4172a8 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc0002067e0, 0xe90320, 0xc00b5ddb60, 0xc00ae9c500, 0xc678a0, 0xc00b5ddb30, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c4173d8 sp=0xc00c417368 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00b5dc660, 0xe90320, 0xc00b5dd170, 0xc00ae9c500, 0xc001167498, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c417448 sp=0xc00c4173d8 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc00b5dd170, 0xc00ae9c500, 0xc678a0, 0xc00b5dc660, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00c4174a8 sp=0xc00c417448 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00b5dc6c0, 0xe90320, 0xc00b5dca50, 0xc00ae9c500, 0xc0045256e0, 0xc00b5dc6f0, 0xc00b5dc9c0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00c417518 sp=0xc00c4174a8 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/monitor.(*monitorServer).Request(0xc000071f20, 0xe90320, 0xc00b5dca50, 0xc00ae9c500, 0xe90320, 0xc00b5dca50, 0xc00ba7f110)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/monitor/server.go:81 +0x6e fp=0xc00c417580 sp=0xc00c417518 pc=0xa91dee
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc000556d70, 0xe90320, 0xc00b5dca50, 0xc00ae9c500, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00c417640 sp=0xc00c417580 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc0002067b0, 0xe90320, 0xc00b5dc690, 0xc00ae9c500, 0xc678a0, 0xc00b5dc660, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c4176b0 sp=0xc00c417640 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00b1e3380, 0xe90320, 0xc00b1e3f20, 0xc00ae9c500, 0xc001167770, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c417720 sp=0xc00c4176b0 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc00b1e3f20, 0xc00ae9c500, 0xc678a0, 0xc00b1e3380, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00c417780 sp=0xc00c417720 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00b1e33e0, 0xe90320, 0xc00b1e3ef0, 0xc00ae9c500, 0xc000206600, 0xe90320, 0xc00b1e3ef0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00c4177f0 sp=0xc00c417780 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/utils/metadata.(*metadataServer).Request(0xc000206600, 0xe90320, 0xc00b1e3ef0, 0xc00ae9c500, 0xe90320, 0xc00b1e3770, 0xc00ba7f0b0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/utils/metadata/server.go:43 +0xe2 fp=0xc00c417860 sp=0xc00c4177f0 pc=0x99ca82
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc000556d60, 0xe90320, 0xc00b1e3770, 0xc00ae9c500, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00c417920 sp=0xc00c417860 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc000206780, 0xe90320, 0xc00b1e33b0, 0xc00ae9c500, 0xc678a0, 0xc00b1e3380, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c417990 sp=0xc00c417920 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc0077b5dd0, 0xe90320, 0xc00b1e2b10, 0xc00ae9c500, 0xc001167a50, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c417a00 sp=0xc00c417990 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc00b1e2b10, 0xc00ae9c500, 0xc678a0, 0xc0077b5dd0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00c417a60 sp=0xc00c417a00 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc0077b5e30, 0xe90320, 0xc00b1e2240, 0xc00ae9c500, 0xe68e70, 0xe99cd0, 0xc00ba7f050)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00c417ad0 sp=0xc00c417a60 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/timeout.(*timeoutServer).Request(0xc0002180d8, 0xe90320, 0xc00b1e2240, 0xc00ae9c500, 0xe90320, 0xc00b1e2240, 0xc00ba7f038)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/timeout/server.go:61 +0x195 fp=0xc00c417bc0 sp=0xc00c417ad0 pc=0xa94815
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc000556d50, 0xe90320, 0xc00b1e2240, 0xc00ae9c500, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00c417c80 sp=0xc00c417bc0 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc000206750, 0xe90320, 0xc0077b5e00, 0xc00ae9c500, 0xc678a0, 0xc0077b5dd0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c417cf0 sp=0xc00c417c80 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc0013645d0, 0xe90320, 0xc0077b5680, 0xc00ae9c500, 0xc000f3adb0, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00c417d60 sp=0xc00c417cf0 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc0077b5680, 0xc00ae9c500, 0xc678a0, 0xc0013645d0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00c417dc0 sp=0xc00c417d60 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc001364630, 0xe90320, 0xc0077b5650, 0xc00ae9c500, 0xc464e0, 0xc00008d3c0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00c417e30 sp=0xc00c417dc0 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/serialize.(*serializeServer).Request.func1()
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/serialize/server.go:45 +0xf7 fp=0xc00c417eb8 sp=0xc00c417e30 pc=0x98c297
github.com/networkservicemesh/sdk/pkg/tools/multiexecutor.(*MultiExecutor).AsyncExec.func2.1()
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/tools/multiexecutor/multi_executor.go:54 +0x4e fp=0xc00c417f00 sp=0xc00c417eb8 pc=0x98158e
github.com/edwarnicke/serialize.(*Executor).process(0xc0013651d8, 0xc00ab71f68)
	/root/go/pkg/mod/github.com/edwarnicke/serialize@v1.0.7/serialize.go:68 +0x3f fp=0xc00c417fd0 sp=0xc00c417f00 pc=0x5b68df
runtime.goexit()
	/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00c417fd8 sp=0xc00c417fd0 pc=0x46e9a1
created by github.com/edwarnicke/serialize.(*Executor).AsyncExec
	/root/go/pkg/mod/github.com/edwarnicke/serialize@v1.0.7/serialize.go:56 +0x1f8

vpp-forwarder.log

@edwarnicke
From my point of view it looks like an issue in govpp, because we don't actually pass anything in there:
https://github.com/networkservicemesh/sdk-vpp/blob/f160fab4ce285a594b2c26580b52fa9c3c5ae739/pkg/networkservice/stats/common.go#L43-L47
Both statsConn and stats here are valid non-nil values.

Again with different stack trace:
Panic message
unexpected fault address 0x7fedc15d9340
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x7fedc15d9340 pc=0x402d87]

goroutine 859523 [running]:
runtime.throw(0xd7459f, 0x5)
	/go/src/runtime/panic.go:1117 +0x72 fp=0xc00d927e50 sp=0xc00d927e20 pc=0x437bf2
runtime.sigpanic()
	/go/src/runtime/signal_unix.go:741 +0x268 fp=0xc00d927e88 sp=0xc00d927e50 pc=0x44eb08
indexbody(0x7fedc15d9340, 0x34, 0x80, 0xc00c54da68, 0x4, 0x8, 0x1, 0x3, 0x8, 0x0, ...)
	/go/src/internal/bytealg/index_amd64.s:256 +0x2c7 fp=0xc00d927e90 sp=0xc00d927e88 pc=0x402d87
bytes.Index(0x7fedc15d9340, 0x34, 0x80, 0xc00c54da68, 0x4, 0x8, 0x246)
	/go/src/bytes/bytes.go:1108 +0x3c5 fp=0xc00d927f28 sp=0xc00d927e90 pc=0x554385
regexp.(*inputBytes).index(0xc00fea08a8, 0xc00fc4f180, 0x0, 0x0)
	/go/src/regexp/regexp.go:450 +0x85 fp=0xc00d927f70 sp=0xc00d927f28 pc=0x5ab525
regexp.(*Regexp).backtrack(0xc00fc4f180, 0x7fedc15d9340, 0x34, 0x80, 0x0, 0x0, 0x0, 0x0, 0x13c1d20, 0x0, ...)
	/go/src/regexp/backtrack.go:342 +0x3d8 fp=0xc00d928000 sp=0xc00d927f70 pc=0x5a5758
regexp.(*Regexp).doExecute(0xc00fc4f180, 0x0, 0x0, 0x7fedc15d9340, 0x34, 0x80, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/regexp/exec.go:535 +0x45c fp=0xc00d9280c0 sp=0xc00d928000 pc=0x5a7edc
regexp.(*Regexp).doMatch(...)
	/go/src/regexp/exec.go:514
regexp.(*Regexp).Match(...)
	/go/src/regexp/regexp.go:531
git.fd.io/govpp.git/adapter/statsclient.(*StatsClient).listIndexes.func1(0x7fedc15d9340, 0x34, 0x80, 0x7fedc15d9330)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/adapter/statsclient/statsclient.go:463 +0xcd fp=0xc00d928168 sp=0xc00d9280c0 pc=0xb676ad
git.fd.io/govpp.git/adapter/statsclient.(*StatsClient).listIndexesFunc(0xc0020bf2c0, 0xc00d928250, 0xd4, 0xc00fc4f180, 0x0, 0x0, 0x7faf17ffb400)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/adapter/statsclient/statsclient.go:490 +0x1b8 fp=0xc00d9281f0 sp=0xc00d928168 pc=0xb62bd8
git.fd.io/govpp.git/adapter/statsclient.(*StatsClient).listIndexes(0xc0020bf2c0, 0xc0101bb570, 0x1, 0x1, 0xc00f8c6ae0, 0xc60640, 0x13c2038, 0xbee420, 0xc010b1bf90)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/adapter/statsclient/statsclient.go:469 +0x14c fp=0xc00d928280 sp=0xc00d9281f0 pc=0xb628ac
git.fd.io/govpp.git/adapter/statsclient.(*StatsClient).PrepareDir(0xc0020bf2c0, 0xc0101bb570, 0x1, 0x1, 0xc010b1bf90, 0xc00d928450, 0x49e71d)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/adapter/statsclient/statsclient.go:202 +0xb0 fp=0xc00d9283f0 sp=0xc00d928280 pc=0xb60770
git.fd.io/govpp.git/core.(*StatsConnection).updateStats.func1(0x40d99b, 0xc0101bb570)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/core/stats.go:148 +0x185 fp=0xc00d928498 sp=0xc00d9283f0 pc=0x8fdb25
git.fd.io/govpp.git/core.(*StatsConnection).updateStats(0xc0020bf310, 0xc0020bf338, 0xc0101bb570, 0x1, 0x1, 0xc60640, 0x13c2038)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/core/stats.go:166 +0x104 fp=0xc00d928558 sp=0xc00d928498 pc=0x8f9da4
git.fd.io/govpp.git/core.(*StatsConnection).GetInterfaceStats(0xc0020bf310, 0xc00d9287c8, 0x0, 0x100000002)
	/root/go/pkg/mod/git.fd.io/govpp.git@v0.3.6-0.20210202134006-4c1cccf48cd1/core/stats.go:311 +0x8f fp=0xc00d928740 sp=0xc00d928558 pc=0x8fb20f
github.com/networkservicemesh/sdk-vpp/pkg/networkservice/stats.retrieveMetrics(0xe90320, 0xc00d64c810, 0xc0020bf310, 0xc00456d810, 0xc01002d700)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk-vpp@v0.0.0-20210708105057-0bce7cb4ad25/pkg/networkservice/stats/common.go:44 +0xa5 fp=0xc00d9287f0 sp=0xc00d928740 pc=0xb684a5
github.com/networkservicemesh/sdk-vpp/pkg/networkservice/stats.(*statsServer).Request(0xc001f5aec0, 0xe90320, 0xc00d64c810, 0xc010b1a870, 0xe90320, 0xc00d64c810, 0xc00d2e7c68)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk-vpp@v0.0.0-20210708105057-0bce7cb4ad25/pkg/networkservice/stats/server.go:56 +0x15f fp=0xc00d928868 sp=0xc00d9287f0 pc=0xb68dbf
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc001e8f080, 0xe90320, 0xc00d64c810, 0xc010b1a870, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00d928928 sp=0xc00d928868 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc001f72210, 0xe90320, 0xc00d64c390, 0xc010b1a870, 0xc678a0, 0xc00d64c360, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d928998 sp=0xc00d928928 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00ffceff0, 0xe90320, 0xc00ffcfbc0, 0xc010b1a870, 0xc0100b6a58, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d928a08 sp=0xc00d928998 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc00ffcfbc0, 0xc010b1a870, 0xc678a0, 0xc00ffceff0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00d928a68 sp=0xc00d928a08 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00ffcf050, 0xe90320, 0xc00ffcf440, 0xc010b1a870, 0x16, 0xbe9401, 0xbe94e0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00d928ad8 sp=0xc00d928a68 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/mechanisms/sendfd.sendFDServer.Request(0xe90320, 0xc00ffcf440, 0xc010b1a870, 0xe90320, 0xc00ffcf440, 0xc0037dc100)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/mechanisms/sendfd/server.go:40 +0x8b fp=0xc00d928c80 sp=0xc00d928ad8 pc=0xa9ad6b
github.com/networkservicemesh/sdk/pkg/networkservice/common/mechanisms/sendfd.(*sendFDServer).Request(0x13c2038, 0xe90320, 0xc00ffcf440, 0xc010b1a870, 0xe90320, 0xc00ffcf440, 0xc00d2e7c08)
	<autogenerated>:1 +0x50 fp=0xc00d928cc0 sp=0xc00d928c80 pc=0xa9b310
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc001e8f070, 0xe90320, 0xc00ffcf440, 0xc010b1a870, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00d928d80 sp=0xc00d928cc0 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc001f721e0, 0xe90320, 0xc00ffcf020, 0xc010b1a870, 0xc678a0, 0xc00ffceff0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d928df0 sp=0xc00d928d80 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00f8c6ab0, 0xe90320, 0xc00f8c7770, 0xc010b1a870, 0xc0100b6eb0, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d928e60 sp=0xc00d928df0 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc00f8c7770, 0xc010b1a870, 0xc678a0, 0xc00f8c6ab0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00d928ec0 sp=0xc00d928e60 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00f8c6b10, 0xe90320, 0xc00f8c6f30, 0xc010b1a870, 0x7fedc21701d8, 0xc00f08b180, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00d928f30 sp=0xc00d928ec0 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/mechanisms/recvfd.(*recvFDServer).Request(0xc001f653e0, 0xe90320, 0xc00f8c6f30, 0xc010b1a870, 0xe90320, 0xc00f8c6f30, 0xc00d2e78d8)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/mechanisms/recvfd/server.go:66 +0x312 fp=0xc00d928ff0 sp=0xc00d928f30 pc=0xa98432
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc001e8f060, 0xe90320, 0xc00f8c6f30, 0xc010b1a870, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00d9290b0 sp=0xc00d928ff0 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc001f721b0, 0xe90320, 0xc00f8c6ae0, 0xc010b1a870, 0xc678a0, 0xc00f8c6ab0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d929120 sp=0xc00d9290b0 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00c821ce0, 0xe90320, 0xc0093a8a50, 0xc010b1a870, 0xc0100b71e0, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d929190 sp=0xc00d929120 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc0093a8a50, 0xc010b1a870, 0xc678a0, 0xc00c821ce0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00d9291f0 sp=0xc00d929190 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00c821d40, 0xe90320, 0xc0093a8150, 0xc010b1a870, 0x0, 0x0, 0xc0037dc100)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00d929260 sp=0xc00d9291f0 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/updatetoken.(*updateTokenServer).Request(0xc0002220e0, 0xe90320, 0xc0093a8150, 0xc010b1a870, 0xe90320, 0xc0093a8150, 0xc00d2e75f0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/updatetoken/server.go:50 +0xd8 fp=0xc00d9292a8 sp=0xc00d929260 pc=0xa91918
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc001e8f050, 0xe90320, 0xc0093a8150, 0xc010b1a870, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00d929368 sp=0xc00d9292a8 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc001f72180, 0xe90320, 0xc00c821d10, 0xc010b1a870, 0xc678a0, 0xc00c821ce0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d9293d8 sp=0xc00d929368 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00c820960, 0xe90320, 0xc00c8214d0, 0xc010b1a870, 0xc0076e9498, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d929448 sp=0xc00d9293d8 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc00c8214d0, 0xc010b1a870, 0xc678a0, 0xc00c820960, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00d9294a8 sp=0xc00d929448 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00c8209c0, 0xe90320, 0xc00c820d20, 0xc010b1a870, 0xc008869fe0, 0xc00c8209f0, 0xc00c820c90)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00d929518 sp=0xc00d9294a8 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/monitor.(*monitorServer).Request(0xc001e8df20, 0xe90320, 0xc00c820d20, 0xc010b1a870, 0xe90320, 0xc00c820d20, 0xc00d2e7590)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/monitor/server.go:81 +0x6e fp=0xc00d929580 sp=0xc00d929518 pc=0xa91dee
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc001e8f040, 0xe90320, 0xc00c820d20, 0xc010b1a870, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00d929640 sp=0xc00d929580 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc001f72150, 0xe90320, 0xc00c820990, 0xc010b1a870, 0xc678a0, 0xc00c820960, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d9296b0 sp=0xc00d929640 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc008ae1110, 0xe90320, 0xc00c8200f0, 0xc010b1a870, 0xc0076e9770, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d929720 sp=0xc00d9296b0 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc00c8200f0, 0xc010b1a870, 0xc678a0, 0xc008ae1110, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00d929780 sp=0xc00d929720 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc008ae11a0, 0xe90320, 0xc00c8200c0, 0xc010b1a870, 0xc001f65f80, 0xe90320, 0xc00c8200c0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00d9297f0 sp=0xc00d929780 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/utils/metadata.(*metadataServer).Request(0xc001f65f80, 0xe90320, 0xc00c8200c0, 0xc010b1a870, 0xe90320, 0xc008ae1620, 0xc00d2e7530)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/utils/metadata/server.go:43 +0xe2 fp=0xc00d929860 sp=0xc00d9297f0 pc=0x99ca82
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc001e8f030, 0xe90320, 0xc008ae1620, 0xc010b1a870, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00d929920 sp=0xc00d929860 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc001f72120, 0xe90320, 0xc008ae1140, 0xc010b1a870, 0xc678a0, 0xc008ae1110, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d929990 sp=0xc00d929920 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00e1d31a0, 0xe90320, 0xc011af3560, 0xc010b1a870, 0xc0076e9a50, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d929a00 sp=0xc00d929990 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc011af3560, 0xc010b1a870, 0xc678a0, 0xc00e1d31a0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00d929a60 sp=0xc00d929a00 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00e1d3200, 0xe90320, 0xc011af2d50, 0xc010b1a870, 0xe68e70, 0xe99cd0, 0xc0068d9338)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00d929ad0 sp=0xc00d929a60 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/timeout.(*timeoutServer).Request(0xc0002220d8, 0xe90320, 0xc011af2d50, 0xc010b1a870, 0xe90320, 0xc011af2d50, 0xc0068d9308)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/timeout/server.go:61 +0x195 fp=0xc00d929bc0 sp=0xc00d929ad0 pc=0xa94815
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*beginTraceServer).Request(0xc001e8f020, 0xe90320, 0xc011af2d50, 0xc010b1a870, 0x0, 0x0, 0x0)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:56 +0x165 fp=0xc00d929c80 sp=0xc00d929bc0 pc=0x99aa65
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc001f720f0, 0xe90320, 0xc00e1d31d0, 0xc010b1a870, 0xc678a0, 0xc00e1d31a0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d929cf0 sp=0xc00d929c80 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00cb32e70, 0xe90320, 0xc00e1d29f0, 0xc010b1a870, 0xc0069eddb0, 0x49e306, 0xd10340)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:69 +0x25a fp=0xc00d929d60 sp=0xc00d929cf0 pc=0x97a97a
github.com/networkservicemesh/sdk/pkg/networkservice/core/trace.(*endTraceServer).Request(0x13c2038, 0xe90320, 0xc00e1d29f0, 0xc010b1a870, 0xc678a0, 0xc00cb32e70, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/trace/server.go:96 +0x99 fp=0xc00d929dc0 sp=0xc00d929d60 pc=0x99b459
github.com/networkservicemesh/sdk/pkg/networkservice/core/next.(*nextServer).Request(0xc00cb32ed0, 0xe90320, 0xc00e1d29c0, 0xc010b1a870, 0xc464e0, 0xc001f5b0c0, 0xe90320)
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/core/next/server.go:71 +0xfc fp=0xc00d929e30 sp=0xc00d929dc0 pc=0x97a81c
github.com/networkservicemesh/sdk/pkg/networkservice/common/serialize.(*serializeServer).Request.func1()
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/networkservice/common/serialize/server.go:45 +0xf7 fp=0xc00d929eb8 sp=0xc00d929e30 pc=0x98c297
github.com/networkservicemesh/sdk/pkg/tools/multiexecutor.(*MultiExecutor).AsyncExec.func2.1()
	/root/go/pkg/mod/github.com/networkservicemesh/sdk@v0.5.1-0.20210708104739-6d7c5b4e2cbc/pkg/tools/multiexecutor/multi_executor.go:54 +0x4e fp=0xc00d929f00 sp=0xc00d929eb8 pc=0x98158e
github.com/edwarnicke/serialize.(*Executor).process(0xc00cb339b8, 0xc00d2e70b0)
	/root/go/pkg/mod/github.com/edwarnicke/serialize@v1.0.7/serialize.go:68 +0x3f fp=0xc00d929fd0 sp=0xc00d929f00 pc=0x5b68df
runtime.goexit()
	/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00d929fd8 sp=0xc00d929fd0 pc=0x46e9a1
created by github.com/edwarnicke/serialize.(*Executor).AsyncExec
	/root/go/pkg/mod/github.com/edwarnicke/serialize@v1.0.7/serialize.go:56 +0x1f8

Looks like this issue was just a result of vpp-forwarder killed with a signal. Issue is no more reproducible with disabled k8s liveness check.

That's good news... but why is a liveliness failing?

That's good news... but why is a liveliness failing?

In the lack of CPU/memory resources it may take more than default liveness probe timeout to make a gRPC request to the forwarder application.
So it is still running, but it looks like dead for the probe.

repo: networkservicemesh/sdk-vpp


text:
🛑 Website is down

In 6013a0b, Website (https://netz0.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Website is back up in 4ec68d4 after 4 minutes.


text:
Deserializing messages over tcp

I'm trying to decide how to delimit messages over tcp (as a socket read may result in the entire message, and may require reading until you hit a delimiter, or read until you've read some predetermined number of bytes.
Is there some mechanism to know I've reached the beginning of a serialized message, and read the header from there (so I know how many more bytes to read) or should I just write the serialized entity and delimit messages myself?

im also looking for that, i need a way to identify packets
i'm using socket to share byte[] that's the packet, but i need a way to get the header, or at least the 1st byte without having to deserialize or to wrap it into an other byte[] because it would cause overhead
this is what i have now, but i find it more like a hack than solution
		[ZeroFormattable]
		class HelloPacket
		{
			[Index(0)] public virtual string Mail { get; set; }

			[Index(1)] public virtual string Pass { get; set; }

			public static implicit operator byte[](HelloPacket packet)
			{
				var serialized = ZeroFormatterSerializer.Serialize(packet);
				var data = new byte[serialized.Length + 1];
				data[0] = (byte) PacketId.Hello;

				Buffer.BlockCopy(serialized, 0, data, 1, data.Length);
				return data;
			}
		}

authors: RUSshy ronnyek

repo: neuecc/ZeroFormatter


text:
Clean up databases created by tests once tests are done

I ran tests using npm test and look what I have. Now I got to clean them up manually :cry:

Hum so a long time ago rethinkdb didn't like to have too many databases/tables - even removed. I think 2.1 fixed that but I haven't tried it.
That being said, the assumption is basically to just kill/delete the data directory/restart rethinkdb when developing on rethinkdbdash.

At least with 2.1 deleting is fine. After running tests a few times, I had 100+ databases left over for cleanup which I could remove using a simple query. I think all the tests should be updated to automate cleanup once the tests have run.
IMO, it's sort of unfair to assume that I would delete my rethinkdb data directory just because I ran the tests. I wouldn't want to lose out on my development dbs of another project. And this assumption inherits the assumption that I am working on a single project at any given point of time which may not be the case.

@kulbirsaini I absolutely agree with you, but I think there is just a different expectation from the maintainer's perspective. If you take a look over here you can pull out the concept of the TestFixture for the thinky tests and easily rework the tests such that tables and databases are cleaned up properly after test runs.

@mbroadst I could but if that's not going to be merged I don't think I would.
I use my own package Jamadar to cleanup databases. It's fairly straight forward.
var Jamadar = require('jamadar');
var db = new Jamadar(servers: [
        { host: 'localhost', port: 28015 }
      ]);
db.getDbList()
  .then(function(results) {
    // All db names created by rethinkdbdash are of 32 characters.
    var filtered = results.filter(function(result) {
      return result.length === 32;
    });
    console.log(filtered.length, 'databases to be dropped');
    return db.dropDbsIfExist(filtered);
  })
  .then(function() {
    console.log('done');
  });

I'm fine with a PR that clean the databases after the test. Not a PR that create one database and one table per test.

authors: kulbirsaini mbroadst neumino

repo: neumino/rethinkdbdash


text:
Document and provide example of the outputs of the query tools

Relates to the discussion from yesterday's hackathon.
Downstream tools will need to make use of the TSV outputed by the query-tool.
To make it easier it would be better if actual TSV files were stored as files in the query tool repo, so they wan be accessed by the CI of other tools.
Ideally those files should be generated by the CI of the query-tool so they are always up to date: this would allow to let downstream know of any change in the "shape" of this output because their CI would break.
Added advantage of having actual files, those examples can also be used to be shown in the official documentation and not just the wiki.
Steps for this:

[x] https://github.com/neurobagel/neurobagel_examples/issues/2
[x] https://github.com/neurobagel/bagel-cli/issues/187

[x] https://github.com/neurobagel/neurobagel_examples/issues/3
[x] https://github.com/neurobagel/documentation/issues/27

@surchs

That sounds good @Remi-Gau, just to be clear here you want there to be an up to date example .tsv file inside the query tool repo and then have a link from the docs to these files, correct?

yes pretty much
you could even render the actual example files in the docs AND have a link
but in general, have in the repo up to date TSVs that match the actual output of the query tool

@neurobagel/dev. I think we need to split this into at least two parts:

Have queryable test endpoint / on-demand API+graph inside the CI
write workflow to query the endpoint, save the results and store them publicly.

Any further splits?

@surchs since we discussed creating these examples manually for now but moving them to a central location, the issue description has been updated with sub-issues. Feel free to add anything you think is missing :)

I think this issue can be closed now as example query tool outputs have been added to https://github.com/neurobagel/neurobagel_examples and referenced in the Neurobagel documentation here.
@rmanaem do you think it'd be worth updating the query tool README to link to docs section with the example outputs?
If not, I think we can go ahead and close this.

Yea I think it'd be a good idea. I'll take care of it.

Closing as all subtasks are complete

authors: Remi-Gau alyssadai rmanaem surchs

repo: neurobagel/query-tool


text:
Consider splitting up imaging -> histology & (in-vivo) imaging

There's (nearly) no overlap in tools between the two sets of fluroescence microscopy, so it may be useful to split up the wiki sections.

Currently, it's 'imaging' and 'histology', with examples of what tools would go where

authors: adamltyson niksirbi

repo: neuroinformatics-unit/troubleshooting


text:
Change Evaluation Of Myelin Thickness And Area

Will fix #190.
Now, the thickness and area of the myelin is relative to its corresponding axon.
Error handling for equivalent diameter and area values has been added with the _check_measures_are_relatively_valid method.

Coverage decreased (-0.1%) to 81.728% when pulling afe362bc7bcc76643ea97c95a76a21aaef7bad28 on maf88/190-modify-myelin-output-metrics into 05cb434667350c84a4f0a5319d002fa6f5fb881b on master.

repo: neuropoly/axondeepseg


text:
numpy.linalg.linalg.LinAlgError: singular matrix

data: mgh-bay3_spine-generic_20171201/t2
Spinal Cord Toolbox (master/c1e4e9cefd485570916857e17f3703d728160f2a)
Running /Users/julien/code/sct/scripts/sct_process_segmentation.py -i t2_seg_manual.nii.gz -p csa -vert 2 -vertfile t1_seg_labeled_reg.nii.gz -ofolder csa

Check parameters:
.. segmentation file:             t2_seg_manual.nii.gz

Create temporary folder (/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-180613161703-PkT5TR)...

Copying input data to tmp folder and convert to nii...
sct_convert -i /Users/julien/code/spine_generic/mgh-bay3_spine-generic_20171201/t2/t2_seg_manual.nii.gz -o /var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-180613161703-PkT5TR/segmentation.nii.gz # in /Users/julien/code/spine_generic/mgh-bay3_spine-generic_20171201/t2

Change orientation to RPI...
sct_image -i segmentation.nii.gz -setorient RPI -o segmentation_RPI.nii.gz # in /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-180613161703-PkT5TR

Open segmentation volume...

Get data dimensions...
  64 x 320 x 320

Smooth centerline/segmentation...
.. Get center of mass of the centerline/segmentation...
.. Computing physical coordinates of centerline/segmentation...
.. Smoothing algo = nurbs

Fitting centerline using B-spline approximation...
Test: # of control points = 5
Error on approximation = 42.06 mm
Test: # of control points = 6
WARNING: NURBS instability -> wrong reconstruction
Test: # of control points = 7
WARNING: NURBS instability -> wrong reconstruction
Test: # of control points = 8
Error on approximation = 0.79 mm
Test: # of control points = 9
Error on approximation = 0.5 mm
Test: # of control points = 10
Traceback (most recent call last):
  File "/Users/julien/code/sct/scripts/sct_process_segmentation.py", line 1231, in <module>
    main(sys.argv[1:])
  File "/Users/julien/code/sct/scripts/sct_process_segmentation.py", line 242, in main
    compute_csa(fname_segmentation, output_folder, overwrite, verbose, remove_temp_files, step, smoothing_param, slices, vert_lev, fname_vertebral_labeling, algo_fitting=param.algo_fitting, type_window=param.type_window, window_length=param.window_length, angle_correction=angle_correction, use_phys_coord=use_phys_coord)
  File "/Users/julien/code/sct/scripts/sct_process_segmentation.py", line 647, in compute_csa
    x_centerline_fit, y_centerline_fit, z_centerline, x_centerline_deriv, y_centerline_deriv, z_centerline_deriv = smooth_centerline('segmentation_RPI.nii.gz', algo_fitting=algo_fitting, type_window=type_window, window_length=window_length, nurbs_pts_number=3000, phys_coordinates=True, verbose=verbose, all_slices=False)
  File "/Users/julien/code/sct/scripts/sct_straighten_spinalcord.py", line 157, in smooth_centerline
    z_centerline_deriv, mse = b_spline_nurbs(x_centerline, y_centerline, z_centerline, nbControl=None, path_qc=curdir, point_number=nurbs_pts_number, verbose=verbose, all_slices=all_slices)
  File "/Users/julien/code/sct/scripts/msct_smooth.py", line 298, in b_spline_nurbs
    nurbs = NURBS(degree, point_number, data, False, nbControl, verbose, all_slices=all_slices, twodim=twodim)
  File "/Users/julien/code/sct/scripts/msct_nurbs.py", line 152, in __init__
    self.pointsControle = self.reconstructGlobalApproximation(P_x, P_y, P_z, self.degre, self.nbControle, w)
  File "/Users/julien/code/sct/scripts/msct_nurbs.py", line 645, in reconstructGlobalApproximation
    P_xb = (R.T * W * R).I * Tx.T
  File "/Users/julien/code/sct/python/lib/python2.7/site-packages/numpy/matrixlib/defmatrix.py", line 972, in getI
    return asmatrix(func(self))
  File "/Users/julien/code/sct/python/lib/python2.7/site-packages/scipy/linalg/basic.py", line 687, in inv
    raise LinAlgError("singular matrix")
numpy.linalg.linalg.LinAlgError: singular matrix

workaround until fixed: use Hanning smoothing:
sct_process_segmentation -i t2_seg_manual.nii.gz -p csa -vert 2 -vertfile t1_seg_labeled_reg.nii.gz -ofolder csa -a

repo: neuropoly/spinalcordtoolbox


text:
Update librfxcodec submodule

Let's bring  neutrinolabs/librfxcodec#2 to the xrdp.

I created #435.

repo: neutrinolabs/xrdp