text:
Improve switch time complexity for constant cases

First version of an improvement to the switch statement to change the time complexity from O(n) to O(log n) (see https://github.com/PlutoLang/Pluto/issues/860).
This does currently not pass the debug tests because of the extra code generated in the main chunk.

Some notes for the current implementation.
The generated bytecode is not stable due to the use of lua tables. I do not know how important this point is.
It would also be possible to move the lookup table initialization into the chunk. This would remove like two instructions from the switch for the initialization test. However, this would iniliatize all switches, even the ones never used.

Rebased this branch because we treat Git as a tree; not a directed graph.

The switchimpl function is now incredibly long, so I'm assuming you're taking ownership and I can ping you if there's any issues with it? :P

The generated bytecode is not stable due to the use of lua tables. I do not know how important this point is.

Could you elaborate on this some more? Would this PR cause issues with the upgrade to Lua 5.5?

You can ping me when there are issues with the switch implementation.
With not stable I mean that string.dump will give out different results on different runs for functions with switch statements due to different order of the table initialization.
I use a lua table to during the compiling to gather all the constants, but a lua table traversal order is not defined, so can result in different orderings.

I see. That does sound a bit ugly, but I also can't imagine a better solution than putting a std::stack<std::map<...>> on LexState.

I could use a std::vector in the switchimpl function, but that would just be there for the ordering. So if a stable output is important, this should be simple to add.

You cannot stack-allocate such a thing because RAII is not guaranteed here.

You are right, then I need to look into case_pc too as it does this currently.

Yes, with the table lookup, check for initialization, and other overhead the binary search will only be used from around 16 cases upwards.
Try with the switch speed test:
local N = 2000
local code = "return function(val)\nswitch val do"
for i=1, N do
    code = code .. "\ncase " .. i .. ".5: return " .. i
end
code = code .. "\ndefault: return -1\nend\nend"
local f = load(code)()
-- Initialize, this might take more time
assert(f(0) == -1)
for i=1, N do
    assert(f(i+0.5) == i)
end
assert(f(0) == -1)

The problem is that I think >95% of switch blocks are gonna have <16 cases.

And are the other stack allocated std::vector in the parser then not also buggy?

That depends. If there is a chance a Lua error could be thrown in their scope, then it would not be deallocated when a longjump is used.

The problem is that I think >95% of switch blocks are gonna have <16 cases.

Yes, but they are hard to get better. When multiple case values map to the same case then it can happen earlier.
int average_time_linear = num_consts/2;
int average_time_table = luaO_ceillog2((unsigned int)case_pc.size()) + 4; /* Load and check cache, load value, check nil case, then do the binary search */
if (average_time_linear <= average_time_table) {
   /* Not worth the effort, just do a linear scan */

That depends. If there is a chance a Lua error could be thrown in their scope, then it would not be deallocated when a longjump is used.

Yes, that could happen in almost all the cases I saw, so I assumed that Pluto is using exceptions and not longjump.

I fixed the memory leak issues with std::vector in the switch statement, need to check std::function.
Will also look in the speed decrease of the example you mentioned.

I would advise just using a void* for user-data and casting it as before. Also nothing wrong with std::vector, as long as it's somewhere on LexState, because that does actually get cleaned up in all cases.

When putting the vector on LexState it would need to handle reentrancy, And the tables also use the correct allocation function from the Lua state.

I guess the speed decrease for the simple case is due to the switch always starting with a jump and not with a conditional jump like before. This results in one instruction more per loop. I will look to emit the first case in place and jump on false to the other logic.

Still a pessimisation, which really shouldn't happen since you seem to have logic to "not bother" for small switches? Maybe this patch would've been better if it focused on trying to add the new functionality without effectively rewriting the whole thing.

When just looking at
local value = 3
switch value do
  case 1:
  case 2:
  case 3:
  cc = cc + 1
  break
  case 4:
  case 5:
  break
end

I get the same code from main & this branch now. So I have no clue where the slowdown should come from.

You are able to reproduce it, tho, right?

No, I am not able to reproduce the slowdown. Over multiple runs sometimes the main branch is faster, sometimes this one.

Well, it seems to only be on MSVC, and I guess "within margin of error," but now I've tried a benchmark where your implementation should undoubtably be faster, and I can also not detect a difference:
local start = os.clock()
local cc = 0
for i = 1, 100000000 do
  local value = 3
  switch value do
    case 1: break
    case 2: break
    case 3: break
    case 4: break
    case 5: break
    case 6: break
    case 7: break
    case 8: break
    case 9: break
    case 10: break
    case 11: break
    case 12: break
    case 13: break
    case 14: break
    case 15: break
    case 16: break
    case 17: break
    case 18: break
    case 19: break
    case 20: cc = cc + 1 break
  end
end
print(os.clock() - start)
return print(cc)

With the 20 cases both average_time_linear and average_time_table are both 9, so it will still use the linear lookup. Either add a case 21 or remove the break from all cases but 19.

Okay, now we have ~3.5 on main, and ~2.4 on this branch. That is cool, but...

It's an optimisation targeted at a very small minority of switch blocks.
That is slower on MSVC in the remaining (overwhelming majority) of switch blocks.
At the cost of increasing the severely increasing the code complexity of the parser's switch functions.

I don't know why the same bytecode and only changes to the parser make the interpreter slower on MSVC. Maybe something gets aligned differenty which reduces the speed. But that could happen with any change.
As for the complexity vs benefit is something you need to decide. I have no problem with closing this. I had my fun implementing this and have no problem when I do not need to maintain in and further.

I think this is a very impressive optimization, however I share some of the concerns already raised in this thread. The most concerning point goes to maintainability of the switch statement. The complexity increase is great enough to say that the lessons learned over time with our switch statement — the bugs fixed, the overall polish applied, the time-tested factor — would be reset. While the performance increase is quite good, the scope of when that performance increase will apply is too limited to justify the aforementioned costs.

authors: Sainan XmiliaH well-in-that-case

repo: PlutoLang/Pluto


text:
Proposal: podcast:episodeType

I'm preparing an episode of The Audacity to Podcast to teach about the "full," "bonus," and "trailer" episode types and when to use them. And I thought it would be good for use to include a drop-in replacement tag for <itunes:episodeType> since it pairs very well with episode numbers and seasons.
Thus, the <podcast:episodeType> tag, but with some improvements.
Apple explains their tag as follows:

The episode type.
If an episode is a trailer or bonus content, use this tag.
Where the episodeType value can be one of the following:

Full (default). Specify full when you are submitting the complete content of your show.

Trailer. Specify trailer when you are submitting a short, promotional piece of content that represents a preview of your current show.

Bonus. Specify bonus when you are submitting extra content for your show (for example, behind the scenes information or interviews with the cast) or cross-promotional content for another show.

The rules for using trailer and bonus tags depend on whether the itunes:season and itunes:episode tags have values:
Trailer:

No season or episode number: a show trailer

A season number and no episode number: a season trailer. (Note: an episode trailer should have a different  than the actual episode)

Episode number and optionally a season number: an episode trailer/teaser, later replaced with the actual episode

Bonus:

No season or episode number: a show bonus

A season number: a season bonus

Episode number and optionally a season number: a bonus episode related to a specific episode

The resulting tag would look like <itunes:episodeType>trailer</itunes:episodeType>
The most important takeaway is that these bonus and trailer types are connected to something: the whole show, a season, or an episode.
I think our tag should function the same with this information, and add some additional uses.

Extra or special: This would be the type for any kind of extra episode that's a break from the norm, but is not a bonus for existing content. For example, if my show is a weekly solo show, but I publish a bunch of 5-minute interviews from an event, those interviews would not be the bonus type, because they're not attached to any existing content. They're standalone, but extra, and thus the "extra" type would be more appropriate than "bonus." This would give podcasters the episode type I often see them incorrectly thinking "bonus" is for.
Rerun: This would indicate that the episode is republishing old content—either in whole or multiple segments.
Sequel or miniseries: That might not be the right word, but here's what I envision. This type indicates that an episode is a part of a miniseries regardless of the whole show type. Consider some TV shows, for example. The show is an overall serial format, but then there's a part 1 and part 2 miniseries. In other words, a small series within a series. But with out bonus and trailer types are attached to an episode/season number, I think this tag would stand alone, allowing "part 1" to be episode 10 and "part 2" to be episode 11. What happens when there are multiple miniseries and how to group them? That could probably be handled with our season tag. But the following is another possible solution.

Additional label attribute?
While the Apple tag allows only a single value, I think our tag should have an optional "label" attribute, which can display to the audience as well as help group miniseries together.
For example, <podcast:episodeType label="Foundations">miniseries</podcast:episodeType>
This could be used when a particular season is already necessary, but the miniseries spreads across seasons or is independent of them.
But I'll admit this is probably an extremely rare scenario and nearly every situation I consider could be handled by <podcast:season> instead.
For example, many pastors will preach sermon series, but sometimes take a break from that series for holiday themes or summer. So a "Foundations" sermon series could be indicated in the season tag, while the episodes are assigned to it as normal episodes. But the "miniseries" episode type could be for a series too small to warrant its own season.
Support multiple types?
I can foresee the need for an episode to be multiple types. Like rerunning a past bonus episode. Or maybe something is a trailer for a miniseries. So it might be necessary to allow multiple instances of the episodeType tag.

Added a "rerun" type, thanks to inspiration from thejavakat.

It was pointed out to me that we're allowing decimals in episode numbers to allow for the things I think episode types can solve better.

I thought of an additional episode type: promo or promotion. This can be used for an episode that serves only to promote something, like a special event, another podcast, a sale, and such.
This kind of content shouldn't be a trailer or bonus because it might not be tied to the rest of the show (it could be promoting a different show or something that's not even a podcast). And it's not a "special episode" either.
Promos would be the kind of content a podcaster might want to remove after the event has happened. But they might also opt to leave the promo in the feed indefinitely, such as when crosspromoting another podcast (of their own or someone else's).
This would pair perfectly with my new proposal for a <podcast:expire> tag (#448) to make episodes (of any type) automatically dim or hide from podcast apps and directories after a time is past.

source: gharchive/issue

repo: Podcastindex-org/podcast-namespace

license_type: permissive

license_source: github-api


text:
I output gradient values during training and observed that gradient explosion has always existed, but as long as nan values do not appear, it does not seem to affect the training effect. However, more and more gradient disappearance problems have made it impossible for my network to learn useful information.

Hello, I trained on my two-category semantic segmentation task. I found that the first few pieces of data trained well, but the performance will be very poor later. I output the gradient information to view it. I found that as the network trains, More and more layers are experiencing the vanishing gradient problem, but I cannot add a bn layer to the network. Do you have any suggestions?

Maybe you can try to add a normalization in QKV project just as PTv2?

I also notice gradient explosions giving me nans and infs, I suspect some additional normalization might help, but I'm not sure where and for which layer. For now I am using gradient clipping at a value of 1.0 (torch.nn.utils.clip_grad_norm_(model.parameters(), grad_clip_val))
It seems to be related more to the 3D sparse convolutions for my use case rather than the attention stuff. I'm curious if you have tried using regular batch norm here instead of layer norm? In my mind the layer norm makes more sense, but I do see examples in spconv using 3d convolutions followed by batch norm. I'll give it a shot and report back at some point.

Maybe you can try to add a normalization in QKV project just as PTv2?

do you mean at the start or at the end of the attention layer ?

I also notice gradient explosions giving me nans and infs, I suspect some additional normalization might help, but I'm not sure where and for which layer. For now I am using gradient clipping at a value of 1.0 (torch.nn.utils.clip_grad_norm_(model.parameters(), grad_clip_val))
It seems to be related more to the 3D sparse convolutions for my use case rather than the attention stuff. I'm curious if you have tried using regular batch norm here instead of layer norm? In my mind the layer norm makes more sense, but I do see examples in spconv using 3d convolutions followed by batch norm. I'll give it a shot and report back at some point.

We used to apply BN in v1 and v2; (Check our ablation in paper) we want to enable LN as it is independent of batch size. Nice try, waiting for your response.

I also notice gradient explosions giving me nans and infs, I suspect some additional normalization might help, but I'm not sure where and for which layer. For now I am using gradient clipping at a value of 1.0 (torch.nn.utils.clip_grad_norm_(model.parameters(), grad_clip_val))
It seems to be related more to the 3D sparse convolutions for my use case rather than the attention stuff. I'm curious if you have tried using regular batch norm here instead of layer norm? In my mind the layer norm makes more sense, but I do see examples in spconv using 3d convolutions followed by batch norm. I'll give it a shot and report back at some point.

Any update on this ?
Has anyone found the origin of the gradient explosion and any way to fix it without gradient clipping ?

@AxelCole
If you look at the spconv library, they have 1D batch norm layers on the feature channels of the points, try replacing the 3D submanifold convolution layers with a sparse sequential using 3d submanifold convolution, batch norm and relu, might do the trick and add some stability.

Clip gradient is very easy to apply (I will update it in the next version), like below:
(pointcept.engine.train.py)
def run_step(self):
        input_dict = self.comm_info["input_dict"]
        for key in input_dict.keys():
            if isinstance(input_dict[key], torch.Tensor):
                input_dict[key] = input_dict[key].cuda(non_blocking=True)
        with torch.cuda.amp.autocast(enabled=self.cfg.enable_amp):
            output_dict = self.model(input_dict)
            loss = output_dict["loss"]
        self.optimizer.zero_grad()
        if self.cfg.enable_amp:
            self.scaler.scale(loss).backward()
            self.scaler.unscale_(self.optimizer)
            if self.cfg.clip_grad is not None:
                torch.nn.utils.clip_grad_norm_(
                    self.model.parameters(), self.cfg.clip_grad
                )
            self.scaler.step(self.optimizer)

            # When enable amp, optimizer.step call are skipped if the loss scaling factor is too large.
            # Fix torch warning scheduler step before optimizer step.
            scaler = self.scaler.get_scale()
            self.scaler.update()
            if scaler <= self.scaler.get_scale():
                self.scheduler.step()
        else:
            loss.backward()
            if self.cfg.clip_grad is not None:
                torch.nn.utils.clip_grad_norm_(
                    self.model.parameters(), self.cfg.clip_grad
                )
            self.optimizer.step()
            self.scheduler.step()

        if self.cfg.empty_cache:
            torch.cuda.empty_cache()
        self.comm_info["model_output_dict"] = output_dict

authors: AxelCole Gofinge JamesMcCullochDickens SpeedyGonzales949 yueyangwen

license: MIT


text:
Typo on Zamazenta's French name

Hi! It seems that the French name for Zamazenta is wrong on https://pokeapi.co/api/v2/pokemon-species/889
It should be "Zamazenta", not "Zamazent"!
Have a nice day!
(English is not my native language, sorry in advance!)

Should be fixed, can you check?

Yep, that's perfect! Thank you!

repo: PokeAPI/pokedex


text:
Allow for 3-7 decimal points for coordinates

Some exports only provide three decimal point accuracy, and sometimes 4.440000 turns into 4.44, need to adjust the regex.

:+1:

source: gharchive/pull-request

repo: PokemonGoF/PokemonGo-Bot


text:
Made docker file a whole lot smaller! (67% to be exact)

Short Description:
Rebased the docker to use alpine python build since it's a lot lighter and easier to download and handle on low connections speed

please wait for merge trying to optimize it even more

@TheSavior save to merge now :)
if someone else has an idea on how to make it even smaller let me know.

Please keep the following ENTRYPOINT code
ENTRYPOINT ["python", "pokecli.py"]

In order to execute docker image and pass parameters, such as ptc, username, password

@nivong isn't this a bit odd? why would one run setup.sh -i during docker build. why would i save my username and password in the docker image?

@mjmadsen: this creates some security concerns for me if the username and password are in the docker image.

Sorry it's not the file that is smaller it's the installation.  Now it's 800 something MB. This puts it down to 299 MB by using alpine Linux. But I'm new to docker so if there is a better way of streamlining the whole thing please do!
I used setup.sh because installation instructions might be updated in the feature to not have to edit everything I used the setup.sh
And yes you won't be promoted ;)  tested that out ofc

Good point about the entry point will change that asap when I'm back in the office (sorry about the stupid close fat fingers)

@nivong kudos for using alpine that part i totally agree and support
my 2 cents were regarding the readability and transparency of what the Dockerfile does

Thanks will clean it up a bit. Could you perhaps contribute too? You can pr to the dev branch in my fork to update it here. Like I said I'm totally new at docker and not Linux. So if you have a better idea I'm all ears and eyes!

@th3w4y updated, is this more like it?

@nivong
looks good
but check below version for the ADD directly from github
that way if user has any file modified they don't get pulled in by docker build.
FROM python:2.7.12-alpine

RUN apk add --update --no-cache alpine-sdk bash wget git

ADD https://github.com/PokemonGoF/PokemonGo-Bot/archive/dev.tar.gz .
RUN tar -zxvf dev.tar.gz && mv PokemonGo-Bot-dev /usr/src/app && rm dev.tar.gz

WORKDIR /usr/src/app
VOLUME ["/usr/src/app/configs", "/usr/src/app/web"]

ARG timezone=Etc/UTC
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

#setup the bot
RUN cd /tmp && wget http://pgoapi.com/pgoencrypt.tar.gz \
    && tar zxvf pgoencrypt.tar.gz \
    && cd pgoencrypt/src \
    && make \
    && cp libencrypt.so /usr/src/app/encrypt.so \
    && cd /tmp \
    && rm -rf /tmp/pgoencrypt*

ENV LD_LIBRARY_PATH /usr/src/app

RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install --no-cache-dir -r requirements.txt

#remove unused stuff
RUN apk del alpine-sdk\
  && apk cache clean\
  && rm -rf /var/cache/apk/*

ENTRYPOINT ["python", "pokecli.py"]

no idea what it does but you have more knowledge than me ;) added it!

@nivong check https://github.com/nivong/PokemonGo-Bot/pull/4

@nivong thanks

@nivong: before you take out wget you would have to modify the Dockerfile like this:
-RUN cd /tmp && wget http://pgoapi.com/pgoencrypt.tar.gz \
-      && tar zxvf pgoencrypt.tar.gz \
+ADD http://pgoapi.com/pgoencrypt.tar.gz /tmp/
+RUN cd /tmp && tar zxvf pgoencrypt.tar.gz \

as for git i don't  know if is used

can you do a PR to my fork :) ? but swampt today

@nivong https://github.com/nivong/PokemonGo-Bot/pull/6

@nivong please pull in also https://github.com/nivong/PokemonGo-Bot/pull/7 otherwise the suggestion i have with pulling the archive from github.com and the fact that i hardcoded dev will not be compatible with the master branch

check!

@th3w4y this comes up on testing the docker build:
Step 21 : RUN apk del alpine-sdk   && rm -rf /var/cache/apk/*
 ---> Running in 2ba527e2474e
WARNING: Ignoring APKINDEX.167438ca.tar.gz: No such file or directory
WARNING: Ignoring APKINDEX.a2e6dac0.tar.gz: No such file or directory
Removing intermediate container 2ba527e2474e
The command '/bin/sh -c apk del alpine-sdk   && rm -rf /var/cache/apk/*' returned a non-zero code: 1

Why is this a problem out of the blue? Worked before....

@nivong checking now...
docker build https://raw.githubusercontent.com/nivong/PokemonGo-Bot/691037348b7d40649f8c507e73a12334854b5526/Dockerfile
... still building

@nivong
I found the cullprit that step si ruined twice....

#remove unused stuff
RUN apk del alpine-sdk\
  && rm -rf /var/cache/apk/*

ENTRYPOINT ["python", "pokecli.py"]

#remove unused stuff
RUN apk del alpine-sdk \
  && rm -rf /var/cache/apk/*

ENTRYPOINT ["python", "pokecli.py"]

lol how did that happen, changed it thanks for checking, if you have more please do a PR i am out of the office again

@nivong sorry was my bad in commit 215832021111bd08038b337487b09d68e73787ce   i copy paste from my file over the text in the github.com editor and probably some lines remain unselected when pasting over.

@nivong now all looks good  👍
14:06 $ docker build https://raw.githubusercontent.com/nivong/PokemonGo-Bot/8832b16f610bbc7ea7cbce10b8c682450a6d05da/Dockerfile
Downloading build context from remote url: https://raw.githubusercontent.com/nivong/PokemonGo-Bot/8832b16f610bbc7ea7cbce10b8c682450a6d05da/Dockerfile  1.56 kB
Sending build context to Docker daemon 3.584 kB
Step 1 : FROM python:2.7.12-alpine
 ---> 67fa4e31547b
Step 2 : RUN apk add --update --no-cache alpine-idk git
 ---> Using cache
 ---> a4ae9dd908be
Step 3 : ARG BUILD_BRANCH
 ---> Using cache
 ---> 321d0cbbdaf6
Step 4 : ENV BUILD_BRANCH ${BUILD_BRANCH:-master}
 ---> Using cache
 ---> 2dcad1163c8d
Step 5 : ARG BUILD_REPO
 ---> Using cache
 ---> b4eaf959943c
Step 6 : ENV BUILD_REPO ${BUILD_REPO:-PokemonGoF/PokemonGo-Bot}
 ---> Using cache
 ---> 12cc3e912e5d
Step 7 : LABEL build_repo $BUILD_REPO build_branch $BUILD_BRANCH
 ---> Using cache
 ---> 06c9d4154b8c
Step 8 : ADD https://github.com/$BUILD_REPO/archive/$BUILD_BRANCH.tar.gz .
Downloading [==================================================>] 3.521 MB/3.521 MB
 ---> Using cache
 ---> ec957c9739f9
Step 9 : RUN tar -zxvf $BUILD_BRANCH.tar.gz && mv PokemonGo-Bot-* /usr/src/app && rm $BUILD_BRANCH.tar.gz
 ---> Using cache
 ---> 047661f5c49e
Step 10 : WORKDIR /usr/src/app
 ---> Using cache
 ---> c39fa7fe8bbc
Step 11 : VOLUME /usr/src/app/configs /usr/src/app/web
 ---> Using cache
 ---> 47815483d5f0
Step 12 : ARG timezone=Etc/UTC
 ---> Using cache
 ---> 9ca574da35f9
Step 13 : RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
 ---> Using cache
 ---> 2bc5d76191de
Step 14 : ADD http://pgoapi.com/pgoencrypt.tar.gz /tmp/
Downloading [==================================================>] 243.6 kB/243.6 kB
 ---> Using cache
 ---> dc84fb88af8b
Step 15 : RUN cd /tmp && tar zxvf pgoencrypt.tar.gz     && cd pgoencrypt/src     && make     && cp libencrypt.so /usr/src/app/encrypt.so     && cd /tmp     && rm -rf /tmp/pgoencrypt*
 ---> Using cache
 ---> 861c0b5a4784
Step 16 : ENV LD_LIBRARY_PATH /usr/src/app
 ---> Using cache
 ---> eb78d51f0398
Step 17 : RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
 ---> Using cache
 ---> fa24a7864b5c
Step 18 : RUN pip install --no-cache-dir -r requirements.txt
 ---> Using cache
 ---> 0e858ca5141c
Step 19 : RUN apk del alpine-sdk  && rm -rf /var/cache/apk/*
 ---> Using cache
 ---> 7f5393c52c4e
Step 20 : ENTRYPOINT python pokecli.py
 ---> Using cache
 ---> d483306518cb
Successfully built d483306518cb
(anaconda3) (root) ✔ ~
14:06 $ echo $?
0
(anaconda3) (root) ✔ ~
14:06 $

Thanks for the merge @solderzzc and thanks @th3w4y for the help :)

authors: helloj nivong th3w4y


text:
Implement callback for create_order

Will close #88

@haerdib I need your help with RPC stuff ... I mentioned your name in the comments

repo: Polkadex-Substrate/polkadexTEE-worker


text:
lit-html, lit-element and redux : _stateChanged ignore template attributes update

Also I came to this specific use-case where a LitElement updates one of its attributes from a redux state property and an attribute in a lit-html template (the template returned from the render method).
Here's the reproduction
I tried to simplify as much as I could.
Simply put, the main view (person-view) html is embedding another custom element (person-card) and set one of its attributes using the "dot notation" (.personId="${this._personId}").
When the state changes (clicking the button) the main view updates its property _personId. Naturally I was expecting the view to update the html attribute of person-card as well so when person-card _stateChanged method is called its property personId value is coherent to the application global state.
I found that was natural because person-card is one leaf of the tree which is the application and should rely on its parents' state.
I think this behavior is caused from the methods _stateChanged called in an asynchronous way or technically not awaiting upper elements dom full update (I can't really tell as my understand of how the core libraries work is limited).
As for now I am trying to find a workaround but If you think this case deserves special attention I'd be gladly waiting for an update.
Love you Polymer Team !

Just a quick addition, this is the workaround I found and I think this is the syntax some lit-element showcases are using.
However in my project I really need to do the treatment in the _stateChanged method because the computed property is coherent to the element and is used subsequently in other elements' methods.
What I mean is that the property is not used for rendering purpose only. So I maintain the state of this issue.

It looks like lit-element will rerender the template of the view asynchronously, so it turns out the personId gets updated in the child component after the _stateChanged method has been called. In your example the parent already has the correct state, but this state hasn't been propagated to the child yet.
I'm not sure what the best approach is to solve the issue (add a synchronous variant of _invalidate to LitElement?), but in any case this looks more like an issue for Polymer/lit-element than lit-html. Could you open a ticket there?

@bgotink Ok I will, thanks for your answer.

authors: bgotink vdegenne

repo: Polymer/lit-html

license_source: bigquery


text:
Remove a test.only in the lit-html tests.

I haven't looked into the cause for the failures this exposes, but I believe they're all legitimately failing tests so we shouldn't (necessarily) wait to make them pass in this PR before merging.

Sorry about this; I'm working on fixes for this, I think it's better to fix them than to have master red for some amount of time.

Ok, that was a simple merging mistake when reverting one of the code factorings late in the lit-ssr PR: https://github.com/Polymer/lit-html/commit/961e95c95ef3a2dd36fbe00d660ab927012d923e#diff-bead45bb43cda474d916704550a1b26322667aa42b814c48300b05be79b9a7ebR1181
Will push a fix to this branch.

authors: bicknellr kevinpschaaf


text:
paper-input docs not showing up in catalog

They are there when I run polyserve. Instead, I get paper-textarea and friends.

Closed by 4f6b270a8

authors: atotic mbleigh

repo: Polymer/polymer-element-catalog


text:
[0.8.0-rc.5] observers as an array

Property observers work when custom element is instantiated, but never after that.  No errors in console.  Here is the javascript from the custom element definition used in my test:
var ChatUser = Polymer({
	is: "chat-user",
	properties: {
		user: Object
	},
	constructor: function (user) {
		this.user = user;
	},
	observers: [
		"userStatusChanged(user.status)"
	],
	userStatusChanged: function (status) {
		console.log("updating status", status);
		this.notifyPath("user.status", status);
	}
});

Testing the custom element's observer:
var userData = {username: "Bob", status: "Offline"};
var user = new ChatUser(userData); // console logs "updating status Offline"
// change status
userData.status = "Online"; // observer ignores this

You need to use the setPathValue method to update the property (or update it using
data binding). See:
https://www.polymer-project.org/0.8/docs/devguide/data-binding.html#path-binding
(Note: in 0.9, setPathValue changes its name to just set).
In this case, you're updating the status from outside, I think instead of:
userData.status = "Online";

In your example, you'd do something like:
user.set("user.status", "Online");

So I have to explicitly call a setter on every custom element using that object.  I was under the impression that using the observers array would set up Object.observe.  I was hoping I could have more than one element observe a single object, so that changes to that one object would essentially be broadcast to all the observing elements, updating their views.
A use case along the lines of this example would be a chat room with a user list of ChatUser elements, and a chat window of ChatMessage elements.  If "Bob" were to change his nickname to "Robert", updating his userData object would change his nickname both in the user list, and on any chat messages he's posted in the chat window.

0.8/0.9 does not use Object.observe.  This was one of the performance changes -- As I understand it, O.o and its polyfill were both quite unpredictable from a performance standpoint.
If you use data binding to pass values, you can get many of the same benefits, but how exactly you'd do that may depend on how your app is set up. I recommend putting together a code sample and either posting a question on Stack Overflow or joining the Polymer slack channel (bit.ly/polymerslack)... There are a lot of people on there who've been working with 0.8/0.9 and may have suggestions.

Thank you very much for your help.  I'll do as you've suggested.

authors: arthurevans jasonpecor

repo: Polymer/polymer


text:
Deep property change from undefined to null is not propagated

Repro: http://jsbin.com/xatiqibofu/edit?html,output
The code changes a deep property and displays the object. Initially the object is empty. When I change the property to "x", the result becomes {property: "x"} which is expected. When I change property to null, it should become {property: null}, but instead it stays empty. The change is not even propagated at all.
I understand how undefined is handled strangely, but this is null which should work like any valid value. So hopefully this is just an oversight, ie. somewhere you're checking "if (oldvalue == newvalue)", and undefined happens to be == to null, but instead it should be ===, which would work. (I see you have a well thought out comparison here: https://github.com/Polymer/polymer/blob/ff6e884ef4f309d41491333860a8bc9c2f178696/src/standard/notify-path.html#L100)

Seems that your issue has been fixed, as the JSBin shows {"property":null} now.

authors: TimvdLippe fejesjoco


text:
Ability to configure a11ySuite rules, exceptions?

Is it possible to configure a11ySuite rules, or ignore specific instances of errors/warnings?

Yes you can pass an array of rule named to skip...
a11ySuite('default', ['duplicateId']);

The rulenames come from accessibility-developer-tools...
https://github.com/GoogleChrome/accessibility-developer-tools/tree/master/src/audits

Awesome!  Thanks Rob!

There should be a way to configure these ignore rules from wct.conf.json file, so that they can be applied to all tests globally.

authors: dbatiste jaichandra robrez

repo: Polymer/web-component-tester


text:
Upgrade to Typescript 3.5.3

Update dependencies on lint and tfjs
Disable new operator-linebreak lint rule
Fix Lint inssues in mock-slot-composer.ts and sequence.ts
Add skipLibCheck for tfjs as that is recommended in https://www.tensorflow.org/js/tutorials/setup#typescript

Okay, have approvals from two continents.
If this causes a problem I'd be happy to rollback.

repo: PolymerLabs/arcs


text:
Break cycle between Runtime and Arc.

Moves cycle count from 12 to 11.
Introduces a StoreRegistry interface for arcs to use when registering
and unregistering their stores.
Part of https://github.com/PolymerLabs/arcs/issues/1878

Closing in favor of https://github.com/PolymerLabs/arcs/pull/4351.


text:
Fix memory leaks

(Root #1)
TaskHandle (from testapp) holds an EntityHandleManager
EntityHandleManager holds StorageProxies
StorageProxy holds a crdt field
crdt field holds a RawEntity
(Root #2)
(serviceConnection -> service.registerCallback -> onProxyMessage holds ServiceStore, also held by StorageProxy)
ServiceStore holds a Channel
channel has a Flow attached
channel can hold a lambda
lambda has in it {   service.sendProxyMessage(message.toProto().toByteArray(), result) }
that lambda has in it a captured 'message' parameter from the function call
message contains ModelUpdate which holds RawEntity
Fixes to TestApp

Invoke EntityHandleManager.close() after tests finish

Fixed to Core

make StorageProxy.crdt a nullable field and make close() null it
change onLifecycleDestroyed() to invoke scope.cancel() (cancel the flow)
add channel.cancel() to ServiceStore.off()  and reinitialize channel & flow with new fresh copies

/gcbrun

/gcbrun

I'm submitting this, only the HandleManager tests are flaky, but on G3 I was able to get TAP to pass with enough runs.

I'm submitting this, only the HandleManager tests are flaky, but on G3 I was able to get TAP to pass with enough runs.

I think this change caused additional flakyness, so we are unable to import it, see eg cl/313015371

authors: cromwellian galganif


text:
Adding functions to events seems to be executed immediately

Hi! :100:
I'm trying this lib, and in first look it seems fantastic and works. Works, but almost. It's seems that when attach an event handler it is executed immediately. For example setting a function on onclick event is called on render, not when element is clicked.
const msg = 'Hello!'
const Main = () => html`

${msg}!

  <button onclick=${() => {  console.log('woohoo') }}>+1</button>
`

render(Main(), document.body)

http://jsbin.com/sogitix/28/edit?js,console,output

Because functions are used as extension points (function values are called with the Part object for an expression so that it can imperatively modify the Part), values that are intended to be functions, like event handlers must be returned from a wrapper function:
html`

${msg}!

  <button onclick=${_=> () => {  console.log('woohoo') }}>+1</button>
`

Also, make sure you're using lit-extended, because core lit-html only sets attributes, which means the function would be converted to a string.

This is talked about a little bit here, though could clearly use some more detail: https://github.com/PolymerLabs/lit-html#function-values

This is talked about a little bit here

Link don't work, the anchor :D nevermind, found it and probably make sense to show above example to clarify the things.

make sure you're using lit-extended

hearing it for the first time

values that are intended to be functions, like event handlers must be returned from a wrapper function:

Wouldn't it be better to switch the places. So if that function returns a function use it like now, so when someone don't care about parts and etc he will be able to pass normal function.
for example
html`

${msg}!

  <button onclick=${clicked}>+1</button>
`

function clicked (evt) {
  console.log('woo hoo, called when clicked'
  // if it returns a function then use it as currently
  return (part) => part.setValue((part.previousValue + 1) || 0)
}

Wouldn't it be better to switch the places. So if that function returns a function use it like now, so when someone don't care about parts and etc he will be able to pass normal function.

Yeah, and I filed #29 for it. I'll try to get it done asap

Hooo!! :tada: Awesome! :clap:
So i believe it will fix this issue too, or I must use lit-extended? And if i must, should i include and the core too?

Well, to write to properties instead of attributes you need to use lit-extended - because that's a pretty opinionated default

I'm going to close this as working-as-intended for now, but I'll change the behavior in #29. Thanks for the report!

Also, you need to import html from lit-html.js and render from lit-extended.js for now... I'll revisit re-exporting html from lit-extended soon.

Great, great, thanks for the tip. :wink:

authors: charlike justinfagnani

repo: PolymerLabs/lit-html


text:
Issues with data:image?

It appears that when running polymerCssBuild on our project, data:image css declarations seem to get formatted incorrectly.
for example...
background: url("data:image/svg+xml;base64,PD94bW....");

becomes
background: url("data:image/svg+xml;)

Does that only happen with image/svg+xml? I just checked our project, and our image/png data: URLs survived the processing properly.

authors: ankon barnomics

repo: PolymerLabs/polymer-css-build


text:
crackmapexec /usr/lib/python3/dist-packages/pywerview/requester.py:144: SyntaxWarning: "is not" with a literal. Did you mean "!="?   if result['type'] is not 'searchResEntry':

Describe the bug
I can not use crackmapexec due this issue
To Reproduce
Steps to reproduce the behavior:

Go to 'terminal'
Write crackmapexec smb IP
See error

Expected behavior
I need to use crackmapexec but I can not, i would like to fix this error
Screenshots

Crackmapexec info

OS: [e.g. Kali]
Version of CME [e.g. v5.0.2]
Installed from apt or using latest release ? Please try with latest release before openning an issue

Additional context
I tried to fix it by seeing the code, go to the file, and you have this code

I tried writing != instead "is not", and then I tried to do it, nothing happens,I have no the error, but crackmapexec do nothing now, whitout any request

Please update crackmapexec, your version is way out of date. This Syntax version is fixed for several versions

repo: Porchetta-Industries/CrackMapExec


text:
chore: allow manual initialization of the SDK

:bulb: Motivation and Context
Closes https://github.com/PostHog/posthog-flutter/issues/22
Closes https://github.com/PostHog/posthog-flutter/pull/113
Required for https://github.com/PostHog/posthog-flutter/pull/116
Automatic mode is still default so not a breaking change.
:green_heart: How did you test it?
Running auto init and automatic mode on android and ios
:pencil: Checklist

[X] I reviewed the submitted code.
[ ] I added tests to verify the changes.
[ ] I updated the docs if needed.
[X] No breaking change or entry added to the changelog.

@thisames if you'd like to review as well :)

@thisames if you'd like to review as well :)

for me, everything is ok

authors: marandaneto thisames

repo: PostHog/posthog-flutter


text:
[Bug]: Connecting to environment not working

What happened?
Connecting via username/password still not possible. After the version update the error message changed, but the problem still exists, that I can't connect to any environment.

The log output was added through creating the bug via the error message. Don't know if that helps.
What version of the tool are you using?
v2.1.1
What version of Visual Studio Code are you running?
1.73.1
Any relevant session id from VSCode?
acfaf457-e549-442d-85bf-574c00bd8e231669014766144
Relevant log output
Error occured in connectDataverse command.
Error code: undefined.
Error message: undefined.
Error stack: undefined.

I have exactly same issue. Please help

Can you check with newer versions released? Thanks.

authors: Power-Maverick neb-miletic wd21sos

repo: Power-Maverick/DataverseDevTools-VSCode


text:
ADReplicationSiteLink Add Integration tests, add setting 'options'

Pull Request (PR) description
Add integration testing ability. Adds ability to enable options such as Change Notification Replication
This Pull Request (PR) fixes the following issues

Fixes #356
Fixes #504

Task list

[ ] Added an entry under the Unreleased section in the CHANGELOG.md.
Entry should say what was changed, and how that affects users (if applicable).
[ ] Resource documentation added/updated in resource directory README.md.
[ ] Resource parameter descriptions added/updated in schema.mof
and comment-based help.
[ ] Comment-based help added/updated.
[ ] Conceptual help topic added/updated (cultureFolder\about_ResourceName.help.txt).
[ ] Localization strings added/updated in all localization files as appropriate.
[ ] Examples appropriately added/updated.
[ ] Unit tests added/updated. See DSC Resource Testing Guidelines.
[ ] Integration tests added/updated (where possible). See DSC Resource Testing Guidelines.
[ ] New/changed code adheres to DSC Resource Style Guidelines and Best Practices.

This change is

Codecov Report

Merging #509 into dev will decrease coverage by <1%.
The diff coverage is 80%.

@@         Coverage Diff         @@
##            dev   #509   +/-   ##
===================================
- Coverage    98%    98%   -1%
===================================
  Files        23     23
  Lines      3108   3159   +51
  Branches     10     10
===================================
+ Hits       3063   3098   +35
- Misses       35     51   +16
  Partials     10     10

Hi @johlju , are you able to review this and #510 at all please? I know there will be a conflict on the changelog.md when one is merged. Thank you

@1800Zeta Thank you for this! The integration tests worked perfectly! 😃

Thank you so much @johlju

authors: 1800Zeta codecov-io johlju

repo: PowerShell/ActiveDirectoryDsc


text:
Find-PSResource with multiple tags not working as expected

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.

Steps to reproduce
I can run these commands to search for resources by tag.
Find-PSResource -Tag yml -type module
Find-PSResource -Tag json-type module

The -Tag parameter will accept an array, so it is reasonable to assume I can run:
Find-PSResource -Tag yml,json -type module

Expected behavior
I would expect to see the results from the two separate commands.

Actual behavior
I get the error: Package with Tags 'yml, json' could not be found in any registered repositories

The parameter will work only when all tags are in the same module.

PS C:\> Find-PSResource -Tag demo,teaching -Type module

Name            Version Prerelease Repository Description
----            ------- ---------- ---------- -----------
PSTeachingTools 4.2.0              PSGallery  A set of commands and tools for teaching PowerShell.

Error details
Exception             :
    Type    : Microsoft.PowerShell.PSResourceGet.UtilClasses.ResourceNotFoundException
    Message : Package with Tags 'yml, json' could not be found in any registered repositories.
    HResult : -2146233088
TargetObject          : Microsoft.PowerShell.PSResourceGet.Cmdlets.FindHelper
CategoryInfo          : ObjectNotFound: (Microsoft.PowerShel….Cmdlets.FindHelper:FindHelper) [Find-PSResource], ResourceNotFoundException
FullyQualifiedErrorId : PackageWithTagsNotFound,Microsoft.PowerShell.PSResourceGet.Cmdlets.FindPSResource
InvocationInfo        :
    MyCommand        : Find-PSResource
    ScriptLineNumber : 1
    OffsetInLine     : 1
    HistoryId        : 26
    Line             : Find-PSResource -Tag yml,json -type module
    Statement        : Find-PSResource -Tag yml,json -type module
    PositionMessage  : At line:1 char:1
                       + Find-PSResource -Tag yml,json -type module
                       + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    InvocationName   : Find-PSResource
    CommandOrigin    : Internal
ScriptStackTrace      : at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo :

Environment data
PS C:\> Get-module Microsoft.PowerShell.PSResourceGet;$PSVersionTable | format-Table

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Binary     1.0.0                 Microsoft.PowerShell.PSResourceGet  {Find-PSResource, Get-InstalledPSResource, Get-P…

Name                           Value
----                           -----
PSVersion                      7.4.0
PSEdition                      Core
GitCommitId                    7.4.0
OS                             Microsoft Windows 10.0.22631
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Same behavior in Windows PowerShell.

Visuals
No response

Indeed, it looks like Find-Module's OR logic with respect to multiple tags mistakenly became AND logic in Find-PSResource.

@jdhitsolutions thanks for creating this issue! We intentionally changed our -Tag logic to reflect AND logic with Find-PSResource instead of OR, so that the search is less computationally expensive and the search result is narrower. If a user wishes to see packages with tag1ORtag2` they would need to provide 2 separate commands.
Good catch that the docs do not yet reflect this, we will update this on our end, thanks!

@jdhitsolutions in the docs for Find-PSResource for the -Tag parameter it states, "If multiple tags are specified, the cmdlet only returns resources that include all the tags provided." Just wanted to point to that, but please feel free to re-open if you feel we should add more clarity thanks :)

authors: anamnavi jdhitsolutions mklement0

repo: PowerShell/PSResourceGet


text:
Fixes #1720 - move rule docs and update tests

PR Summary

PR Checklist

Moved ./RulesDocumentation to ./docs/Rules

Moved ./docs/markdown to ./docs/Cmdlets

Update Pester test for documentation

[x] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[x] Summarized changes

[x] Change is not breaking

[ ] Make sure all .cs, .ps1 and .psm1 files have the correct copyright header

[x] Make sure you've added a new test if existing tests do not effectively test the code changed and/or updated documentation

[x] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

Looks like platyPS invocations in the build script also need to be updated

@rjmholt OK, I missed that. Will update.

authors: rjmholt sdwheeler

repo: PowerShell/PSScriptAnalyzer

license: mit


text:
Having to click twice to close

Steps to reproduce:

Open powershell in any folder
Write to powershell "code ." //open the folder with visual studio code
Click to powershell window close button one time. Window will not close.
Click a second time, windows will close.

I think it should close when I first click. This problem is just "code . " Occurs when I run the command.
I guess when powershell is unfocus when it opens another window it can't detect the first shutdown.

That's an interesting find (in the future, please provide details about your environment, as suggested in the issue template; in the case at hand, it would have been helpful to know that the problem only occurs (in regular console windows) on Windows).
However, the problem is not related to PowerShell, it is the behavior of regular console windows (conhost.exe) on Windows, as evidenced by the fact that a regular cmd.exe console window exhibits the same behavior.
Therefore, please close this issue and report it at https://github.com/microsoft/terminal/issues.
Also interesting to note that Windows Terminal is not affected.

repo: PowerShell/PowerShell


text:
Get-ChildItem does unnecessary work if path isn't found

I have tested the behaviour with PS5 and PS7.
Steps to reproduce
$files = Get-ChildItem -Path 'c:\mysepecialfolders*' -Filter '*.dll' -Recurse
foreach ($file in $files)
{
    $file.FullName
}

Expected behavior
Seach for all folder which start with mysepecialfolders and search inside this folders for all .dll files.

Actual behavior
Depends.
IF there is a folder like c:\mysepecialfolders1 or there are more folders that match,
then Get-ChildItem is fast and searches only inside this folders for all *.dll files.

If there is no folder that matches, then Get-ChildItem searches the whole drive.

Behavior request
If there is no folder that matches, then Get-ChildItem should return no files immediately without searching the whole drive.

Environment data
Name                           Value
----                           -----
PSVersion                      7.0.0
PSEdition                      Core
GitCommitId                    7.0.0
OS                             Microsoft Windows 10.0.18363
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

I have checked the behavior with Process Monitor.


text:
Powershell can't run on macos 14 beta2

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.
[X] Refer to the FAQ.
[X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce
install powershell-7.3.5-osx-arm64.pkg ok
run powershell Application PowerShell cannot be opened
Expected behavior
-

Actual behavior
-

Error details
No response
Environment data
-

Visuals
No response

Do you have Rosetta installed? There was a problem where MacOS thought that PowerShell.app required Rosetta...
https://github.com/PowerShell/PowerShell/issues/18548

authors: rhubarb-geek-nz teslachina


text:
ConvertFrom-Yaml, ConvertTo-Yaml

Would be great to support Yaml natively.
This was also mentioned by @fabiendibot on #3046
It would also be nice if the CMDLets had the goal of cleanly handling conversion of objects that came from XML as it seems like it would be a frequent use case.  Maybe some good tests around this conversion?

We had a similar discussion from DSC aspect,
allowing us to change json based configuration files, we wanted to have options for modifying xml based files, YAML based files ,INI based files supporting RegEx swaps from within Text Manipulation cmdlets.
Lack of existing support in PS means we have to work hard to get such ability.
It has been on hold pending community contribution, but if it was baked into PS, it would make it much easier for the DSC part as well.

When you say natively, do you mean like XML or JSON?
The current thinking is that YAML should not be baked into PowerShell at all, instead it should be a separate module that you can update without picking up a new version of PowerShell.
If YAML were baked into PowerShell like XML, that would be impossible (think [xml]"b")
If we went the JSON route, you'd have cmdlets to work with YAML - so not really baked into PowerShell, but you'd still have the drawbacks of needing to update PowerShell to get YAML updates.

@lzybkr I know we said we didn't want to bring in a new library, but I think this is something we might need to reassess. Ideally, we should also ship the module on the Gallery, but I think a TON of modern scenarios require YAML now.
Maybe not in 6.0 timeframe, but we should talk about it.

@ArieHein - I have some simple functions that save and retrieve a hash array to the registry.  Only handle REG_SZ - but for a simple set of settings it is sufficient - let me know if you want a copy.
I mispoke when I said "native" - I primarily meant "built-in" - it wouldn't bother me if they were shipped-in script modules that could be updated.

Our first discussion #2109

@iSazonov - ah yes I see!
I noticed the reference to AWS support of YAML on the thread - I have been converting some templates and have found this to helpful: https://github.com/awslabs/aws-cfn-template-flip

@iSazonov thanks for the pointer, I couldn't find it for some reason. Remember it well, though.
In re-reading that original thread, I think we should definitely implement the cmdlets at some point in the future, and ship them in the Gallery. Based on their quality, and people's perceived usefulness (along with some refactoring work we hope to do after 6.0.0), we can make the in-box vs. Gallery-only call.

We have Yaml in https://github.com/PowerShell/platyPS

yeah this would be awesome to have, ended up using https://github.com/awslabs/aws-cfn-template-flip to convert

@MattTunny Welcome to contribute! :-)

There's a Windows Server user voice openned to vote for it :-)
https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/11088495-out-of-the-box-support-for-yaml-like-csv-xml-j

This should definitely be part of the native PS 6.1 library. So many things these days are in YAML.

There are now psyaml and powershell-yaml modules on the PSGallery but both are not even able to round-trip a YAML file from a VSTS build definition. I don't mind if the module is baked into PowerShell or is a module from the PSGallery.

I wonder if the core problem here is the clunky way we deploy modules. Today, you have to find, trust and install a module before you can use it.  Compare this with the (apparently) slick way that Javascript does var m = require('mymodule').  Maybe we should have some way to do what DSC does but for native PowerShell. In DSC, when a module is referenced in a configuration, it's automatically downloaded and installed on the target node with no manual effort. Making critical but non-core modules available that way should eliminate the "it should be part of core" arguments. And for nodes that are disconnected from the net, we could have a tool that bundled the dependencies in a script into an archive which is then deployed to the target. This is how the Azure DSC resource extension works - there is a tool that scans a script to figure out the required modules then builds a zip file containing everything that is needed and publishes it to a blob. The Azure resource extension then pulls this blob, installs the modules and runs the script.

@bgshacklett From what I've heard from the Puppet guys, there are just no good YAML parsers :-)

Is platyPS parser good enough?
@vors Is there simple way to reuse platyPS YAML parser in PowerShell Core repo?

I prefer the idea of a separate official module in PowerShell Gallery like @lzybkr says because it would be possible to use it in older powershell versions and it could have its own releases. That would be like the sqlserver module. @BrucePay if it were a page in PowerShell Gallery with Microsoft own modules, it would be easier to find and everybody would know that they can trust them.
But I would understand if it were backed into Powershell as XML and JSON.
The important thing is that it exists ConvertFrom-YAML and ConvertFrom-YAML official functions because YAML is a widely used format for configuration files and it shouldn't be a third-party module, as @bgshacklett point out.
I made a blog entry testing and comparing the two modules I've found to work with YAML files: PSYaml and powershell-yaml.
They have different behaviours because internally they're using different objects:

module
mappings
sequences

PSYaml
OrderedDictionary
Array

powershell-yaml
Hastable
List

I think we need a standard ConvertFrom-YAML and ConvertFrom-YAML.

Actually, ConvertFrom-Yaml in powershell-yaml uses OrderedDictionary when converting with the -ordered parameter.
I've been using this module successfully for a while (in my Datum module for DSC Configuration data, and with kitchen yamls), but don't have a vsts build definition to test with.

Bear in mind that the right way to call it is: get-content -Raw MyFile.yml | ConvertFrom-Yaml -Ordered (people often miss the -Raw).

I wonder why we'd need a Microsoft official module, putting even more overhead on MSFT and reinventing the wheel... Maybe trying to contribute to an existing one first, add tests to avoid regression, open issues to make sure the owner knows the problems is a better approach...
You know what happens when you're trying to create a standard out of the 99 existing implementations...
And yes it would be better outside the language, I agree that the dependency management could be better, bundling everything in PS is not a solution though.
The broad npm issue is also a failure in process. Fork and re-publish fixed it in no time, building apps out of latest version of the internet was the reason it broke so many live apps.

I agree with @gaelcolas I think this is better with everyone working with the owners of an existing community module to raise and ensure quality.
I'll just add that tests for such a project should include working with a large number of real-world YAML files for things like AppVeyor, Travis CI, VSTS, AWS CloudFormation, etc. For my own experience with YAML deserilization, I have had little success with one solution working universally and have ultimately had to reinvent the wheel several times. In that sense, I agree with @BrucePay "there are just no good YAML parsers".

We are talking about this platyPS module because it is already actively used in PowerShell Help environment. I guess no one from MSFT can tell how good this module is because of Code of Conduct. They can either silently reject it or improve it.
And although we've been talking about this a long time ago I don't see how we could use the components of this module here in a simple way.
Maybe @adityapatwardhan and @SteveL-MSFT  will open their plans and timeline especially as the new Help RFC is already in the experiment stage.

My personal view is that I would rather see more community modules succeed and become de facto standard than requiring "official" modules from Msft.

@iSazonov It is one thing to have a solution that works for serializing/deserializing a well defined schema. It is quite another to have a solution that works in general with all schemas that are compliant YAML.

I understand the desire of MSFT to reuse community projects to cut costs. But the situation is, in fact, that MSFT may not make use of so many community projects:

many have bad code, have no trust
many projects are one person

MSFT has published Powershell specifications more 10 years ago, but nobody ported it yet until MSFT did.
The OpenSSL Project has existed for many years but nobody ported it to Windows while MSFT has not done this.
MSFT revealed many thousands of API interfaces, but how many of them were ported to Unix?
The interesting thing about why the company launched its project .Net Core rather than reuses Mono?
PowerShell is already a year and a half is an open source project, but I see that in this repository only one person from the community makes systematic contribution in the code @markekraus  and only one person makes systematic analysis @mklement0.
I don't think that if we divide the project into parts, then we got more contributions.
I don't think the situation will change tomorrow. I wouldn't count on it.

@markekraus I very hope on http://yaml.org/spec/1.2/spec.html#id2802346 :-)

@iSazonov makes important points about support, trust and maintenance of 3rd party modules. Some 3rd party module can become a success and mature like e.g. Pester.
However, one should not assume that a great YAML module will evolve on its own over the next years. The reality is that most modules are published by authors who solved a particular problem and did the good deed of publishing their generic base code. This is how we ended up with 2 modules that aim to solve the same problem. Ideally one would need to merge them to focus efforts, otherwise they are going drift apart further in the future or just become stale and soon there will be more modules published by other people.
The underlying problem of having a proper parser indicates that basic (and substantial in terms of effort) ground work is needed and required to have a good YAML module.
I am not a YAML expert, but is this just a problem of the loose language specification itself or specific interpretation by various systems like VSTS or AppVeyor or is this only the lack of a good parser?
I found it frustrating to write YAML in VSCode and only when running it in VSTS to get an error that the VSTS parser does not like it...

To me this conversation is a case in point with open source's  "code curation / architecture" problem.
Open source provides good seeding ideas and code bases - but if a serious architecture eye is not given to it when adopted as the most general solution - then it's 10 years of bug fixes for items that could have been taken care of in a decent design review.
In the true cases of @bergmeister "mature successes" it is often an active maintainer that has taken on the mission of generalizing the code base.  But that can't be guaranteed to happen.
I think some of us our saying "YAML support is like support for writing files - it's core - it should be architected in the same way => with intention to the be the gold standard for that functionality"

Very valid points on software maturity.  I haven't looked closely at the two modules listed here, nor at yamldotnet to make any opinion.  Something we can look at as we start planning for 6.2.0

Don't get me wrong, I do value the experience and systematic approach of the PowerShell team and MSFT developers, I just think it's wrong for them to try to fill all the gaps with a module of their own stamped MSFT... It does not scale (and we've seen the problem with DSC resources already).
Increasing the reliance on MSFT provided modules is fragile, and does not help grow the community, nor the diversity of the ecosystem.
I'm in favour of MSFT contributing to open source projects to share their experience and help improve practices and quality, while not creating a dependence on them (because you know, squirrels...!).
The MSFT as unique provider of approved things is an old model that they struggle already to educate on, and it is not helping the community to encourage this approach (i.e. I'll wait, or moan, at Microsoft for not solving the problem I have kind of attitude in the OSS ecosystem).
I agree YAML support is core, instead of the PS team re-writing from scratch, why not help existing maintainers of projects to improve, and give them an opportunity to merge projects and hear from them what it would take. A bit like an apprenticeship/mentorship from PS team on core functionality modules.
Just re-writing a new module sounds like an engineer's reaction to solve a problem which is not an engineering problem. Re-writing a YAML module is an easy engineering task for the PS Team, but would not (help to) fix the community maturity problem, nor give the right incentive.
Whether Yaml is the strategic item to tackle this is MSFT's call though :)

@bergmeister
I'll preface this with myself not being a YAML expert. I happened to do some research on this when I wanted to bake some AppVeyor like yaml configs into my own franken-pipeline. I looked at how a dozen or so C# projects were consuming YAML. Since the PowerShell projects use YamlDotNet, I can only assume it's no easier. Though i have at leats toyed around with both PSYaml and powershell-yaml and look less closely at a few projects which use them.

I am not a YAML expert, but is this just a problem of the loose language specification itself or specific interpretation by various systems like VSTS or AppVeyor or is this only the lack of a good parser?

I suspect it's the nature of YAML being readable by humans at the possible expense of being more easily readable by machines. This readability-first paradigm extends into the way YAML authors write their YAML files. Though the resulting YAML is compliant under YAML spec, it is parsed in such away as to be unusable in code without using the deserialized object as an intermediary to an actually useful object.
That is to say, that 90% of the time the deserilization from YAML to an Object is not the issue, but the data design/architecture is. The other 10% of the time is parsing issues for which I can only chalk up to "YAML is hard to parse, man." However, the deserialized objects are often only slightly more useful than regexing what you are looking for....
As an example, the secure strings in AppVeyor.yml
environment:
  my_var1: value1
  my_var2: value2
  my_secure_var1:
    secure: FW3tJ3fMncxvs58/ifSP7w==

powershell-yaml and YamlDotNet do convert this to an object, but good luck using it without a bunch of logic. Once you have that logic, good for this schema, but what about another?
Some of these same data design problems plague JSON, but it is (in my experience and opinion) much easier to make models that can work around those shortcomings due to the more rigid nature of JSON. Trying to make models for any of the yaml deserializers mentioned in this thread is a nightmare if and where it is possible.
Granted, models are not a feature currently available in the JSON cmdlets, though I would really like to add it. If I had a say in the "official" YAML module/cmdlets I would put it down as a "must have" feature. It is a missed opportunity especially with the addition of PowerShell classes in v5.
IMO, Just getting YAML strings into an Object isn't good enough. That appears to be easy (90% of the time at least). The trick is getting YAML strings into useful objects. That requires some flexibility from the solution. But that flexibility must also be somewhat approachable and not require @IISResetMe and @lzybkr there to give you serialization advice....
To that affect, I haven't seen anything that works on general scope. Projects adopt the solutions available, and then use their output as intermediaries for actually useful objects (leading to a bunch of wheel reinventing that probably should be baked in upstream). Or, the projects compromise YAML readability for ease of parsing from YAML to objects.

@gaelcolas

I agree YAML support is core, instead of the PS team re-writing from scratch, why not help existing maintainers of projects to improve, and give them an opportunity to merge projects and hear from them what it would take

Ask yourself why MSFT started .Net Core project instead of continuing Mono many years later.
MSFT is a community too. And as any community has the same problems of interaction with other communities.

For context, I am not implying any work be done from scratch - code could be adopted - but should then be scrutinized from a Systems Development architecture perspective before being improved.  It could even be open source after that review and re-release.
My point is to have a significant architectural review and remediation from a team that thoroughly understands the nuances of core code that will be leveraged virtually everywhere.

Another model always worth considering is acquire/contract/second.  On this basis an effort is made to reach commercial terms with one or more community members/firms to recruit their services for a MSFT-led/facilitated development cycle to re-vamp and (in some fashion) integrate/connect the product(s).  This was done successfully with Xamarin, which kicked the project to the Net Foundation, licensed it under MIT, and recruited/contracted/involved key resources such as Miguel de Icaza and Nat Friedman via Xamarin.  Some whine that this is open source treason.  But it does create positive incentives for folks and small firms to conceive and develop projects that later could be fit for widespread adoption and integration into at least one major ecosystem.  Certainly it's preferred to jumping straight to a blank slate in-house redo that copies the whole concept and functionality and many of the ideas but jettisons the creators and (ostensibly) the code.

@iSazonov sorry for a late reply, no the platyPS yaml parser is no good: it only supports key value pairs. We also use YamlDotNet to generate yaml there.

Regarding the significant sentiment towards keeping this out of the core feature set: there's a very significant difference in how PowerShell handles dependencies compared to, say, Ruby, Python or Node.js.
Each of these languages has dependency management tools (bundler, pip, npm/yarn) which make the management of external dependencies easy and, more importantly, reproducible. Having something like a Gemfile/Gemfile.lock or package.json/package-lock.json [,yarn.lock] which makes for easy installation of all required packages and ensures that you are staying at a very specific patch level is a very significant distinction which is, in my opinion, what makes third-party libraries for something this fundamental feasible.
Perhaps there's something that could be done with Nuget to solve this issue, but I've never seen any articles describing dependency management strategies/patterns for PowerShell. Having the gallery is great, but if you've got to install all required packages manually it becomes unfeasible for any significant deployment.

What I do no each new pwsh system? I do Install-Module -Name powershell-yaml.
Mongo, Kubernetes, Istio, Ansible - I use. All these are YAML and I do have YAML templates and transfomations.

@dzmitry-lahoda Issue #5681 proposed to have a rich' version of PowerShell that ships with a set of common modules such as e.g. Pester`, etc. Please post in this issue but given that there seems to be no clear winner between the currently 2 available yaml modules and they are clobbering each other, it might be a hard decision to pick a favourite.

I see only one YAML :(

Pester, yeh. Too heavy to ship BDD framework into mainline, unlike YAML reader for my pwsh container applications.

Has this thread been concluded. What is the recommended(or suggested) module to use by Microsoft?
DevOps pipelines uses yaml. All my deployment automation are build with powershell. Seems yaml and powershell do not play nice. Is powershell a bad choice for Azure DevOps automation?
Need to think carefully of my future use/innovation and would appreciate some direction.
Thanks in advance!

@dirkslab You could use https://github.com/cloudbase/powershell-yaml

Thanks @iSazonov, that is the solution I am testing with at the moment. The solution so far seems to work fine. There is probably nothing wrong using the solution.
Note that using powershell-yaml you need to ok an untrusted module. This is the part that i am battling to understand. Microsoft recommends using yaml pipelines. Microsoft (or at least this thread) suggest using 3rd party module so you can integrate yaml config with powershell, but do not endorse or recommend any. How do you explain that logically to enterprise.
My experience so far has always been that if you do not use Microsoft endorsed solutions, that would mute any support or understanding from Microsoft for your solution issues(this does not matter if the unsupported part touches anything causing problems). The mere fact that you have unsupported part usually results in no support/responsibility .
Maybe things have changed in this OpenSource era. Simple official response and guidance from Microsoft would set me at ease and help me understand.
Appreciated your response. Regards.

@dirkslab I think your MSFT account manager is right person to ask about support policy.

CoreFX team asks about use cases

Appart from obvious benefits that yaml is all around us in CI/CD today and number of configuration systems, the added benefit of ConvertTo-Yaml is representing nasted HashTable in human readable format, unlike ConvertTo-Json that we have to use now which makes output not very readable.
I use Write-HashTable in the meantime, but this would be great to have OTB.

I hate yaml, I really hate it. However there are a couple of facets worth the MS team considering:

It's become the defacto language of CI: docker-compose.yaml, ansible, kuber, k8s, github, circle, azure, ... And it seems to crawl out of CI into the projects that use it.

$config = Invoke-WebRequest https://$ciserver/api/projects/$project/config.yaml | ConvertFrom-Yaml
$config['started'] = Get-Date
$config['options'] = $options
Invoke-WebRequest -Method Post https://$ciserver/api/projects/$project/build -Body ($config | ConvertTo-Yaml)

Having this ship with powershell would be transformative in evangelising to CI groups.
"If we switch to ms powershell we can automatically" -> "Tell me more?"
vs
"If we switch to ms powershell and download some scripts from the gallery" -> "no"

Really, this is by-the-by, but yaml is a superset of json, such that json is an abbreviated form of yaml, an efficient yaml parser is an efficient json parser,

Can this be reconsidered for 7.1? I'm also having issues with using an untrusted module and something so DevsOpsy should really be native to PowerShell.

IMHO, YAML is as popular as JSON and CSV, and not having inbox converters for YAML in PowerShell is kind of sad. Having inbox YAML converters will also ensure that their behavior is on par with JSON converters, which is not the case with community modules.
Don't get me wrong - I appreciate that folks create modules for community, but in the current state of the world, YAML conversion is table stakes - we don't expect folks to download 3rd-party modules for JSON conversion.

@dirkslab You could use https://github.com/cloudbase/powershell-yaml

 GitHub**cloudbase/powershell-yaml**PowerShell CmdLets for YAML format manipulation. Contribute to cloudbase/powershell-yaml development by creating an account on GitHub.

The problem is that unlike ConvertFrom-Json, this module deserializes to Hashtable instead of a proper object (or PSCustomObject), which doesn't allow using the pipeline in the following way:
$obj | ? Categories -gt 0 | % Name

My current workaround:
'...' | ConvertFrom-Yaml | ConvertTo-Yaml -JsonCompatible | ConvertFrom-Json

Suboptimal, to say the least.

@marektoman, your example should work with a hashtable:
PS> @{ one = 1; two = 2 } | ? one -eq 1 | % two
2

Also note that you can cast hashtables to [pscustomobject] (albeit only one by one - see #13838):
PS> @{ one = 1; two = 2 }, @{ one = 10; two = 20 } | % { [pscustomobject] $_ }

one two
--- ---
  1   2
 10  20

@marektoman, your example should work with a hashtable:
PS> @{ one = 1; two = 2 } | ? one -eq 1 | % two
2

Also note that you can cast hashtables to [pscustomobject] (albeit only one by one - see #13838):
PS> @{ one = 1; two = 2 }, @{ one = 10; two = 20 } | % { [pscustomobject] $_ }

one two
--- ---
  1   2
 10  20

Good point, the behavior is unrelated to hashtable, but it is likely related to the way each module does output.
When I run
$yamlArray | ConvertFrom-Yaml | ? Name -eq 'All'

, it doesn't work. However, this does:
($yamlArray | ConvertFrom-Yaml) | ? Name -eq 'All'

Thanks @marektoman , my solution was to remove the extra yaml module overhead and settle somewhere in between yaml and json.
I did something similar as you when requiring an pscustomobject:
$obj = Get-Content -Path .\deployment-config.yml -raw | ConvertFrom-Yaml | convertto-json -Depth 100 | ConvertFrom-json -Depth 100
The direction I have chosen to go with:
I have replaced all my yaml config with .psd1 and using Import-PowerShellDataFile
Not as clean as yaml, more than sufficient and easy to read though. Also eliminated any and all module issue.
Examples:
Sharing project with other devs that are domain policy restricted on installing random and unsigned modules, not an issue anymore.
I can now . browse path before I convert it to a string
adding comments are easy
My personal goals were to need little to no 3rd party modules for my ps project. I am now back on track and it's hassle free.

@marektoman : The (...) forces enumeration, which in turn suggests that when ConvertFrom-Yaml outputs an array of hashtables that it does so as a single object, which contravenes the expected streaming (one-by-one) pipeline behavior. It is what '[ 1, 2 ]' | ConvertFrom-Json used to do  (see #3424), and still does in Windows PowerShell).

@dirkslab Good idea for that use case. My use case is a shell independent tool that produces single output, which is both human and machine readable.
@mklement0 I see, thanks for all the info.

It is absolutely CRAZY that there is not first class support for yaml in PowerShell.  I am just stunned.
The Kubernetes movement alone should be more than enough to get this prioritized.

We should admit that YAML is used practically in many/most of cloud, CI, configuration, orchestration tools that IT pros, devs and DevOps need to use. So if PowerShell want to stay (most :D) valuable in that world for each it was designed, created and tailored specifically it should support it out of the box.

can i donate 10 USD for the feature? I know there are services where donation is done until some dev implements

This page talks a lot about the powershell-yaml module as an option to get YAML support.
Does this library work on Linux with Powershell Core? or it just a Windows only thing.  (I noticed that its dependencies include .Net Framework, so I am worried.)

@Vaccano it works well on Linux.

My initial thinking is that YAML support should be standard in PowerShell 7.x. YAML is such an integral language to today's enterprises that its omission is glaring considering support for XML and JSON.  I have no problem with a separate module that can also be installed from the gallery, but it should be included by default in a new PowerShell 7.x installation. I know there are organizations that won't consider anything open source, and certainly nothing that doesn't originate or is supported by Microsoft. Given the value of PowerShell as a cloud language, native YAML support seems like the right decision.
I am also less concerned about making the module downlevel compatible. If someone needs YAML conversion support for Windows PowerShell, there appear to be a number of options in the PowerShell Gallery.

As a normal user, I've installed PowerShell Core and use it in my work.
Our projects' ps1 scripts are not compatible with PowerShell Core. I have switched to the legacy PowerShell to run them.
And it's difficult to install a new module in this constrained work environment.
I don't know how to write PowerShell scripts, and now I have to live with the old PowerShell.
I feel sad because the new shiny PowerShell Core that I'm in love, is not usable in even a simple real-world usage.
Maybe, I have to rewrite those *.ps1 files by using a serious programming language.

@PowerShell/wg-powershell-cmdlets has been discussing this issue, and despite the extremely high demand (and my own personal desire to have strong, canonical YAML cmdlets), we're unfortunately going to have to close it (for now) as Won't Fix. The reasoning is two-fold:

While the PowerShell package is already fairly large, we're doing our best not to add new dependencies to the package so that we don't continue to grow our size unfettered, and because we believe that pulling modules from the PowerShell Gallery (or an internal trusted Gallery, having been vetted appropriately by an enterprise) should be a standard part of a PS7 workflow.
As a team, we simply don't have the capacity to build a .NET YAML parser from scratch, nor do we have the capacity to vet, support, and service a 3rd party library for which the security implications are high-stakes.

If .NET were to officially support a YAML library, we (as Microsoft) could potentially look into supporting a YAML module on the Gallery. But until that point, the existing community modules will have to suffice.

Thanks, Joey. I'd like to file a Feature Request in dotnet but don't understand it well enough to know which repo to add the issue. Can you please suggest which repo would be most appropriate within https://github.com/dotnet ?

(or could a Microsoft person/PowerShell team member place to the request as it may get a higher priority?)

See https://github.com/dotnet/runtime/issues/28414

Thank you @iSazonov. That doesn't look promising :/ Seems that we do need an advocate from the PowerShell team to create the request, eh?

I guess Microsoft has internal conclusion that it is low priority and small area.
You can ask PowerShell team online today on Community Call https://github.com/PowerShell/PowerShell-RFC/issues/299

This makes me sad.
Powershell says ".Net has to do it for us to do it"
.Net says "It is up to Powershell if they want to do it"
I wonder if some of the higher ups at Microsoft realize the runaround that is going on here.
YAML is important.  Having it from a signed Trusted Source is critical for most companies.  Powershell has built in restrictions making it hard to use things not signed by trusted sources.

I give up.  It seems that the enough dust has been kicked up to obscure this issue.  I imagine at some point someone high up at Microsoft will say "What?  No YAML support in PowerShell?"  Fingers will be pointed, blame laid, excuses made (etc).  Then it will get done.  But no amount of requests by the community will get this through.

Why has this been closed as WONTFIX? Seriously, YAML is so common these days, all the cloud-native stuff uses it. One can do so much with just vanilla PowerShell before they need to import a module, and not being to deal with YAML shouldn't be one of them. The built-in JSON cmdlets is one of the reasons why JSON is to popular in PowerShell: it's there, it's standardized, and it works. I am sure I am not alone defaulting to JSON just for that reason, and thinking twice about using YAML if I don't really need it, as it makes my scripts less portable or standalone.

@awakecoding joey's explanation is here: https://github.com/PowerShell/PowerShell/issues/3607#issuecomment-901288121
The Json library, even though it was Newtonsoft, was "blessed" by Microsoft as the official JSON library till system.text.json came around.
It's a bummer for sure, I would love native YAML support as well, but I personally think the reasoning is sound. I think energies should be focused on more easily pulling down dependencies quickly for a script/module.

Seriously surprised this isn't already included. It is sorely needed in the community and would seem like a obvious inclusion.

It's a shame.
We pay many billions in royalties and still have to beg M$ for many years so that not Amazon or Google don't step in again to solve such "problems".
Yes, it's a shame.

This makes me sad.
Powershell says ".Net has to do it for us to do it" .Net says "It is up to Powershell if they want to do it"
I wonder if some of the higher ups at Microsoft realize the runaround that is going on here.
YAML is important. Having it from a signed Trusted Source is critical for most companies. Powershell has built in restrictions making it hard to use things not signed by trusted sources.
I give up. It seems that the enough dust has been kicked up to obscure this issue. I imagine at some point someone high up at Microsoft will say "What? No YAML support in PowerShell?" Fingers will be pointed, blame laid, excuses made (etc). Then it will get done. But no amount of requests by the community will get this through.

After 5 years, it's time to ask M$$ again to do its homework, especially because we need YAML on azure.
Please vote:
https://github.com/PowerShell/PowerShell/issues/16785

@mubed Probably won't happen until MS releases a first party YAML library:
https://github.com/PowerShell/PowerShell/issues/3607#issuecomment-901288121
Several YAML modules available on the gallery though:
https://www.powershellgallery.com/packages?q=yaml

Native YAML support in PowerShell is extremely necessary nowadays. We have to remember that PowerShell was primary designed for IT professionals and system administrators which most of cases have to deploy vanilla PowerShell scripts over the production environment because they cannot deploy modules (no internet connection, security constraints, not signed binaries etc). Also the module footprint is a concern. A single 2KB script that connects to SQL Server requires a 50 MB Sql Server module to work. If an script requires more than one module, I cannot imagine how much disk space would be required. NPM packages feeling? I really does not understand this concern of PowerShell team of not changing the standard library. Are PowerShell going to the same Javascript path?

@ronascentes it's pretty simple. There is no first-party Microsoft YAML library (there is a JSON one and hence why its there), so having a first-party YAML support brings all kinds of licensing and whatnot issues until that happens.
Fortunately there's lots of great community modules, my favorite being YaYaML.
https://github.com/jborean93/PowerShell-Yayaml
Just run install-module yayaml and you're good to go.

I really does not understand this concern of PowerShell team of not changing the standard library. Are PowerShell going to the same Javascript path?

@ronascentes - It is two parts of Microsoft pointing the finger at the other.  The PowerShell team says that they need a YAML parser built into .NET before they can add native YAML support to PowerShell.  The PowerShell team says it does not have the manpower to build and maintain a .NET YAML parser, and thus cannot add native YAML support PowerShell.  (They are unwilling add native YAML support based on a community YAML parser.  This is due to licensing and support issues.)
The .NET team says there is not a need for a native .NET YAML parser.  They site that there is a fine community YAML parser already out there.  They say that if PowerShell needs a YAML parser, they need to build and maintain their own parser.
So basically, it is one company (Microsoft) acting as two companies (PowerShell and .NET).  The two parts of Microsoft are unwilling to get together and make a plan to fix this situation.

The .NET team says there is not a need for a native .NET YAML parser. They site that there is a fine community YAML parser already out there. Because of this, they are unwilling to build a YAML parser into .NET. They say that if PowerShell needs a YAML parser, they need to build and maintain their own parser. (Which the PowerShell team says they cannot do.)

Same thing was said about json, yet we have System.Text.Json today 🤷

@StephenHodgson agreed, and as soon as the dotnet team recognizes that and ships System.Text.Yaml in Dotnet 11 (or whatever), I'm sure a first party PS module will be soon to follow :)
It will take someone thinking that the existing libraries are not "good enough" in the same way it was determined for NewtonSoft. Honestly surprised as much of a Not-Invented-Here culture that exists over there that it hasn't happened sooner.
Related Discussion FYI: https://github.com/dotnet/runtime/issues/83313

The .NET team says there is not a need for a native .NET YAML parser. They site that there is a fine community YAML parser already out there. Because of this, they are unwilling to build a YAML parser into .NET. They say that if PowerShell needs a YAML parser, they need to build and maintain their own parser. (Which the PowerShell team says they cannot do.)

Same thing was said about json, yet we have System.Text.Json today 🤷

Without JSON patch support...

You could make a module for that using this library.
https://github.com/weichch/system-text-json-jsondiffpatch

authors: ArieHein BrucePay DarwinJS DavidChennells JustinGrote MattTunny Satak StephenHodgson SteveL-MSFT Vaccano autosvet awakecoding bergmeister bgshacklett dirkslab dzmitry-lahoda gaelcolas iSazonov jdhitsolutions joeyaiello josepmv kfsone lzybkr m3nax majkinetor marektoman markekraus mklement0 potatoqualitee redstrike ronascentes schittli vors wasker


text:
Need a way to bring back legacy Windows aliases for consistent experience

To address https://github.com/PowerShell/PowerShell/issues/929 we removed aliases that collided with native tools.  However, some customers may want the legacy Windows PowerShell aliases because they are consistent with what they are used to.  Proposal that has been discussed is to have a module that only contains aliases that users can import to get back that previous experience.

Where should this new module go?

It's not about "what we're used to." I'm a Mac user that has barely used PowerShell at all on windows. If I switched to powershell, it would be for the object-oriented model. My expectation is that ls gives me a stream of objects, because that is powershell's killer feature. Without that, PS is nothing.

I still remember curl and Invoke-WebRequest quarrel.
Powershell thought about introducing this grammar?
{Module name}. Command name
or
(Module name). Command name
or others
In this case, add a linux local command module, the inside of the ls command, link to / bin / ls can.
It is simple and does not change the old ones.
You can even make modules based on some linux release.eg:
(Centos).ls    <----- module name.command name
{Mac}.ls
[Ubuntu1604].ls

@Gaelan for your use case, I would suggest using Get-ChildItem or gci as the short form, meant for interactive usage. I understand that it can be difficult to retrain muscle memory (I'm having trouble myself), but being explicit with what your asking the shell to give you is almost always going to give you a better experience than relying on aliases which were not originally intended for your platform. The decision to remove these aliases has not removed any functionality in PowerShell, but only changed the way the commands need to be invoked.
ls certainly falls into a grey area, as it's a built-in in both PowerShell and Bash, but in the case of commands such as curl, wget, etc. which mask functionality that already exists on the system, I don't think there was really any question of what the "correct" behavior is, only the least disruptive. There was no solution that was going to be ideal for everyone. At least with this solution you will have an easy option to re-enable the aliases you wish to have in your environment.

@dchristian3188 If your question was about a module place I believe it is https://github.com/PowerShell/PowerShell/tree/master/src/Modules/Shared

Putting them in a module is only one proposal. The point is that there needs to be an interactive (and possibly also parse-time) mechanism for turning GNU/POSIX colliding aliases on and off. (Side note: it's not just *nix users who may want to turn them on, Windows users may also want to turn these aliases off.)

Also, current thinking is that this might be blocked on PowerShell/PowerShell-RFC#48, which is why I'm removing up-for-grabs and personally prioritizing driving that RFC to completion (it's blocking other usability issues as well).

@chuanjiao10 for cmdlet name collision, we already support that today:
Microsoft.PowerShell.Management\Get-ChildItem

3q @SteveL-MSFT .
Hello everyone:
I found this linux local command return error in powershell, but normal in BASH:
PS /root> /usr/bin/ls -l /root/*.xml
#return /usr/bin/ls: can not access /root/*.xml: no that file or directory

PS /root> ls -l /root/*.xml
#return /usr/bin/ls: can not access /root/*.xml: no that file or directory

So I suggest: linux local command in THE module name,THE module link to linux path. Powershell explain linux local command, different treatment.
Linux\ls
Or
Ubuntu1604\ls

@chuanjiao10: That might be the behavior you want, but you received far too much feedback on day one that people want things like ls -la to just work out of the box. (Side note: the full path to the binary will always work, no matter what path we take here.)
The fact that you have a preference is why we're proposing that there be an opt-in mechanism for exactly the behavior you're describing.
Sound reasonable?
Also, the reason your example is failing is because we need to implement globbing with #954

Is here a way to explicitly start external (OS) command?
This is start cmdlet:
&"where"

Can we do somethink like:
$nativecommand:where

@iSazonov - I believe that can be done with Get-Command:
$where = Get-Command -Name where -CommandType Application
& $where

@chuanjiao10 the globbing issue you have above should be fixed in beta.1

yes i saw. thanks @daxian-dbw goodjob!

authors: Gaelan SteveL-MSFT bgshacklett bielawb chuanjiao10 dchristian3188 iSazonov joeyaiello


text:
Windows PowerShell Event Provider GUID still in use

The Windows PowerShell guid is used for tracing here:
https://github.com/PowerShell/PowerShell/blob/c1c5344a8897262433141ecbc13bb06ac2c4bbef/src/System.Management.Automation/utils/tracing/TracingGen.cs#L38
It should use the PowerShell Guid from here:
https://github.com/PowerShell/PowerShell/blob/master/src/PowerShell.Core.Instrumentation/PowerShell.Core.Instrumentation.man#L34

Actually, I think this is WorkFlow code and can be removed.


text:
Thread-safe HttpKnownHeaderNames initialization

PR Summary
Fix #15513
Move HttpKnownHeaderNames initialization to static constructor.
PR Context
See #15513.
PR Checklist

[x] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[x] Summarized changes
[x] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
[x] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.

Breaking changes

[x] None
OR
[ ] Experimental feature(s) needed

[ ] Experimental feature name(s):

User-facing changes

[x] Not Applicable
OR
[ ] Documentation needed

[ ] Issue filed:

Testing - New and feature

[x] N/A or can only be tested interactively
OR
[ ] Make sure you've added a new test if existing tests do not effectively test the code changed

Tooling

[ ] I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
OR
[ ] I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:

[ ] Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode
(which runs in a different PS Host).

[ ] Issue filed:

[ ] Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.

[ ] Issue filed:

[ ] Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).

[ ] Issue filed:

[ ] Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).

[ ] Issue filed:

Even with this, the hashset will still be accessed from multiple threads concurrently in the cases given in #15513, won't it? We should probably add a test using the code samples in the issue to verify that this is sufficient to resolve it. Since the user reporting it mentioned that it was a bit intermittent, we should repeat their test a few times in our own tests.

Even with this, the hashset will still be accessed from multiple threads concurrently in the cases given in #15513, won't it?

A root of the issue was that second thread read the HashSet while first thread still initializes the HashSet.
The HashSet is used only in one place for reading so no need to concern more about race conditions.

We should probably add a test

It's not worth it. We will not be able to make such tests stable and defensible. Please use compiled artifact to check locally.

Even with this, the hashset will still be accessed from multiple threads concurrently in the cases given in #15513, won't it?

A root of the issue was that second thread read the HashSet while first thread still initializes the HashSet.
The HashSet is used only in one place for reading so no need to concern more about race conditions.

We should probably add a test

It's not worth it. We will not be able to make such tests stable and defensible. Please use compiled artifact to check locally.

If the only writes are the ones when the object is generated :
s_contentHeaderSet.Add(HttpKnownHeaderNames.Allow);
s_contentHeaderSet.Add(HttpKnownHeaderNames.LastModified);
I think the write race condition is eliminated with this change.
I am a C# dev, but I have zero knowledge on the Powershell source code. Help I could help you somehow.

I am a C# dev, but I have zero knowledge on the Powershell source code. Help I could help you somehow.

PowerShell is great project to improve skills. It's too big to learn it well in a short time but you can pick an area you like best and work there with some workgroup https://devblogs.microsoft.com/powershell/powershell-working-groups/
After monitoring issues and PRs for about a month you will be able to make good contributions.

@daxian-dbw Could you please review?

@SteveL-MSFT Could you please approve? It is impossible to merge without green flag :-)

I see that there's a pending review from @daxian-dbw
Is there anything else you need from me?
Thank you

@TravisEz13 Please review/approve.