text:
fix(lint): lint now waits until exiting the loop to write to file, handles multiple validation results

Description

[x] lint now waits until exiting the loop to write to file
[x] handles multiple validation results

Related Issue
#481

https://github.com/defenseunicorns/lula/assets/86802655/f23e4eeb-47e2-4b8e-a05f-ed5517bcdf79

repo: defenseunicorns/lula


text:
Add Argument option to set vagrant provider and box image

As we look to utilize zarf to begin packaging up bigbang and the desired environment in support of multiple air gapped information systems, I think it would extremely useful to enable developers to select the vagrant provider and also the box image to be used (in particular my team is looking at RHEL or Rocky which we plan on STIG'ing to make our our SCA/ISSP(s)and AO(s).)
I am currently running into issues with running make all in the examples and trying to get zarf ready to test in our Tanzu dev and AWS. VBox is not allowed by our IA team even for dev, so we must use vmWare Workstation or Fusion, which makes for an enjoyable time for trying to build a dev version with a connected pc or mac.
The work everyone has done so far is amazing, thank you for your contributions on Zarf!

Thinking out loud:
I want to start looking at the Vagrantfile situation, I'm thinking #85 goes first and is meant to be the simplification of the vagrant-based code by combining the stuff that is in the root folder with the stuff that is in the examples folder, and maybe adding some kind of framework that makes it easier to add additional dimensions of support such as other distros or hypervisors.
This issue seems more tricky to me, since none of the core Zarf team uses vmware. @jeff-mccoy and I use VirtualBox and @btlghrants uses libvert. We would need to identify someone who does to be the maintainer of that additional capability. @jbone81 is that something you would be interested in supporting?

The other option is we add documentation on how to more easily extend the existing codebase to support whichever alternative technology any particular user is using, such as @btlghrants's work to do a "VM-within-a-VM so that he can do VBox based virtualization inside his libvirt environment.

With #84, #85, and #123 (this issue) all tied in some way to being able to run/demo Zarf in 1) disparate OSes, and 2) on different hypervisors, it seems like a good place to start would be, like you say:  "some kind of framework that makes it easier to add additional dimensions of support such as other distros or hypervisors"
The first part of that would need to be figuring out / making a statement about what we do & don't want to support for dev vs. demo.  For example, perhaps:

development is vbox-only -- our recommendation for others is to run vbox-in-something-else (like I do) but we don't support it (or maybe just not initially, I dunno) and

demo environments are available for other OS / hypervisor combos -- check the support matrix for notes (etc.).

A policy like that (at least) puts out of scope the requirement for us to support ALL the crazy dev-required, Vagrant edge cases that occur across different providers (like how they all support different synced_folder mechanisms) while still allowing us to show off & test Zarf in other OSes--it's just a matter of copying Zarf in on box build (or by simple download or whatever) and needs almost none of the tricky Vagrant-provider-specific stuff in this case.
A framework for running a pre-compiled version of Zarf in a bunch of different OSes is much lighter to make/maintain than one to develop Zarf across the same skew, after all.

Or we could just really change how we think about the whole thing--I'd be super interested in a K3d (or pick your distro) + kubevirt to do this.  We could then also get e2e doing that as discussed before.  Then the issue of vm providers goes away completely.  This would also unify testing/examples much better + enable seamless e2e local testing.

Love it. Want to team up on something like that for the next Dash Days?

I mean... if that's in-scope, I'm game--seems hella interesting to me, tbh.

This would be great for dash days--also we (Defense Unicorns) need to prove out kubevirt automation anyone for other teams, so this would be super valuable.

Okay. I'll put this back on the shelf for now till after the next dash days. I'll look at continuing to improve the E2E testing stuff.

sounds great, also some of this pain goes away too once native apply is done, because then you won't really need a VM to just test how zarf works

Totally agree, I am still having issues with vmware-desktop playing nicely as a provider, but I was able to swap the vm out for a rocky8 box and that works fine, except I blew up the fs a few times once I applied the stig profile to the vm... currently going to figuring out some lessons learned about which stig items make for "fun" times when rolling out a fresh airgap deploy.
I still think it would be a very useful thing for devs to be able to use flags to force a vagrant provider for testing, but it should be a later low hanging fruit things because there are so many other very cool things in the roadmap.

I propose we close this, since we are working to move away from Vagrant as a recommended/supported option as we move toward just specifying Docker + KinD/K3d. Vagrant is still a valid way to run, especially if wanting to do a linux-based K3s "Applicance Mode" deployment on a dev laptop, but I'm thinking we will want to stop officially supporting that setup and just let the user use whatever they want, maybe with some suggestions/recommendations.

source: gharchive/issue

authors: RothAndrew btlghrants jbone81 jeff-mccoy

repo: defenseunicorns/zarf

license_type: permissive

license_source: github-api


text:
Add Docstrings to Exported Functions and Variables

Add (somewhat basic) docstrings to all exportable functions, consts, and package global variables. This is one small step to getting us a cleaner and more standardized codebase.

Boo I thought you had already merged this in 😭

Those conflicting files are gross

yeah this something the intern said they were going to merge into the other PR, but it never made it in......

So ummm, maybe we close this for now until we have time to look at it again @YrrepNoj ?

source: gharchive/pull-request

authors: Racer159 YrrepNoj jeff-mccoy


text:
Update Spring framework

We're a bit behind the latest release

we update it all at the same time using #215

repo: defold/extender

license: MIT


text:
FR: malwr Support

https://malwr.com/

Curious: what specifically do you want to see from the support?

Now that hashes are possible as indicators, pulling back associated malwr reports would be handy.  However, Brian and I talked about this some and I think we concluded that the API isn't currently functional for anything but submissions.  ThreatCrowd, though, is pulling from Malwr somehow.

ThreatCrowd could be scraping it but also could be in that fortunate group who already has additional endpoints. From the API page in the profile section:

We started enabling some API functions to all registered users, more will eventually come in the future. What you see is what you can have, we do NOT enable any private APIs to anybody.

Makes sense.  We're also sort of bummed that ThreatCrowd's API isn't a little more robust at this time.  Could pull a bunch from them in a single integration, but it's not there yet.

Not sure if you guys are interested, but I'm working on importing
indicators from Cuckoo (which is what Malwr is based on.)
On Sun, Nov 8, 2015 at 9:13 PM, John D. Swanson notifications@github.com
wrote:

Makes sense. We're also sort of bummed that ThreatCrowd's API isn't a
little more robust at this time. Could pull a bunch from them in a single
integration, but it's not there yet.
—
Reply to this email directly or view it on GitHub
https://github.com/defpoint/threat_note/issues/51#issuecomment-154908059
.

I don't think any reasonable integration is unwelcome, but I can't make that call.  I think that the more flexible threat_note becomes, the wider its adoption is likely to be.

So Cuckoo support was added in #64 so given malwar.com doesn't have an API I think this is the closest this will get. Good to close?

authors: alxhrck krmaxwell sroberts swannysec

license_source: bigquery


text:
ADD: DEFUND Durango PD

Mayor: DeanBrookie@durangogov.org
Council: ChrisBettin@durangogov.org
Council: barbaraNoseworthy@durangogov.org
Council: MelissaYoussef@durangogov.org
Hello,
My name is [YOUR NAME HERE], and I am a resident of Durango, Colorado. We are experiencing a revolution across this country. For too long black lives and BIPOC in general have lived in fear of police departments who act violently with little or no consequence. Attempts at reform and extra training has done little to alter this systemic problem.
As a result I call for immediate defunding of Durango PD. At well over 8 million dollars, their budget accounts for 16% of the general fund for the city. In contrast, the funds allotted  for community services like aide for the homeless,  low income assistance, and youth services are less than 1 million. That’s over 8 times the amount of funding for police versus community services. This is absurd, we need to invest in the wellbeing of the community NOT the police.
Police are not trained to properly deal with most of the calls they answer and often make situations worse, especially when BIPOC are involved. What we need are more mental health professionals, more youth counselors, and more resources for homeless folks and those struggling to make ends meet. Investing so much money in the police shows a priority for maintaining a inequitable status quo, not a commitment toward community enhancement and racial justice.
Americans are crying out for justice. And you all are in a position to make a lasting change in our city. This can not wait, I urge you to address this problem head on and reallocate police funds towards programs that will make a safer, healthier, and more thriving community.
Please don’t wait, act now.
[YOUR NAME]
[YOUR ADDRESS]
[YOUR EMAIL]
[YOUR PHONE NUMBER]

Hi [USER]! Thanks for your interest in adding [CITY] to the site. Can you provide more detail about the city budget to make a more compelling argument, showing the disparities in numbers?
Please see our email style guide for some suggestions.
Thank you!

Updated

Reviewing now

Some minor edits:

Mayor: DeanBrookie@durangogov.org
Council: ChrisBettin@durangogov.org
Council: barbaraNoseworthy@durangogov.org
Council: MelissaYoussef@durangogov.org
Hello,
My name is [YOUR NAME HERE], and I am a resident of Durango, Colorado. We are experiencing a revolution across this country. For too long Black people, Indigenous people, and People of Color  have lived in fear of police departments who act violently with little or no consequence. Attempts at reform and extra training have done little to alter this systemic problem.
As a result, I call for immediate defunding of Durango PD. At well over 8 million dollars ($8,417,771), their budget accounts for 16% of the general fund for the city. In contrast, the funds allotted for community services like aide for the homeless, low income assistance, and youth services are less than 1 million ($921,102). Police have over 9 times the amount of funding of community services. This is absurd. We need to invest in the well-being of the community NOT the police.
Police are not trained to properly deal with most of the calls they answer and often make situations worse, especially when Black people, Indigenous people, and People of Color are involved. What we need are more mental health professionals, more youth counselors, and more resources for homeless folks and those struggling to make ends meet. Investing so much money in the police shows a priority for maintaining a inequitable status quo, not a commitment toward community enhancement and racial justice.
Americans are crying out for justice. And you all are in a position to make a lasting change in our city. This can not wait. I urge you to address this problem head on and reallocate police funds towards programs that will make a safer, healthier, and more thriving community.
Please don’t wait, act now.
[YOUR NAME]
[YOUR ADDRESS]
[YOUR EMAIL]
[YOUR PHONE NUMBER]

👍🏻👍🏻

on it

authors: adultman-cooley mahrer swallan todd-m


text:
Proposal feat(controller): execute initial task in the background

On start the controller execute some tasks that delays the "readiness" of the pod.
Is possible to execute the tasks in background?

What tasks do you want to delay? The only real tasks I can think off is load_db_state_to_k8s so I am assuming that's what you are referring to since we can't background the DB migration and such :)
We most def could put those into the background very easily as it is one command but off hand the things I'd be worried about are:

People running operations against the API before the management command is done which could leave us in a fragile-state
Bubbling up errors from the management command, https://github.com/deis/controller/issues/741 is an example of something we'd somehow have to get set

I think we should seriously look at pulling this off but I wanted to put my initial thoughts in here and see what others (and you @aledbf) think.

@helgi this https://github.com/deis/controller/blob/master/rootfs/api/management/commands/load_db_state_to_k8s.py#L11 is taking almost 25 seconds (I had to comment the App.objects.all() loop)

authors: aledbf helgi

repo: deis/controller


text:
fix(healthcheck): check if the healthchecks are failing on a new deploy

fixes #989
Testing Instructions:

deploy an app(example-go) and make sure its working
now set an readiness healthcheck such that it fails deis healthchecks:set readiness httpGet 2000 --type web so that the new deploy is failed
3)without this pr changes the command should execute fine after this PR changes the deploy should rollback to previous release.

Don't deployments handle exactly this? They are supposed to not put a pod live unless liveness and readiness pass, by default if either of those is missing the respective type is true (as in passed)

The change looks good, just am baffled that other checks didn't get it

yes...deployments catch this and the pod will be in 0/1 state which means it is running but not ready and we don't consider this as failure because of which the secrets and env of the previous release gets deleted and hence there will be 2 releases but nothing active.

Ah I see, so it goes through the full timeout and bails out that way?
I did make it so that we keep as many secrets as available RS, so that code may be broken, as well. Let's talk about tomorrow, if I don't get to it first

Current coverage is 86.64% (diff: 10.52%)

No coverage report found for master at e1c6e71.

Powered by Codecov. Last update e1c6e71...b71eaae

authors: codecov-io helgi kmala


text:
Understanding new architecture

Hello.
I have been reading documentation and architecture documents and I cannot find any reference to persistent volume claims (to use storage quotas and storage plugins) and location inside domain ruting (ex. redirect /secondapp to another pod).
Can someone clarify this?
Thanks in advice.

Persistent volume claims are not currently applicable for applications deployed to k8s via Deis Workflow, as Workflow focuses on managing and deploying 12factor applications. (See http://12factor.net) Such applications do not store any state by writing it directly to a file system; rather, they depend on "backing services" such as object stores and databases to store state.

Ok. Thanks

authors: amon-ra krancour

repo: deis/deis


text:
feat(client): add --git-hostname option

--git-hostname= option for register/login purposes. If used
'git_hostname' will be added to client.json and used as git server
hostname, when app is created or git:remote is used. It's useful when
controler is behind Cloudflare's Flexible SSL, which will block any
non-HTTP/HTTPS traffic and git server behind different hostname.
It doesn't break any functionality.
Closes #3948.

Thanks for the contribution! Please ensure your commits follow our style guide. This code will be tested once a Deis maintainer reviews it.

Jenkins, add to whitelist.

looks like this PR failed integration tests:

Creating application...  ...  o.. done, created appssample
Creating config...  ... done, v2

=== appssample
BUILDPACK_URL: https://example.com
Traceback (most recent call last):
  File "<string>", line 2566, in <module>
  File "<string>", line 2562, in main
  File "<string>", line 2495, in _dispatch_cmd
  File "<string>", line 534, in apps_create
  File "<string>", line 1625, in _git_remote_create
KeyError: 'git_hostname'

Sorry, haven't noticed earlier :/. I'll fix it asap.

This failed in Deis' integration tests:
=== appssample
BUILDPACK_URL: https://example.com
Traceback (most recent call last):
  File "<string>", line 2566, in <module>
  File "<string>", line 2562, in main
  File "<string>", line 2495, in _dispatch_cmd
  File "<string>", line 534, in apps_create
  File "<string>", line 1625, in _git_remote_create
KeyError: 'git_hostname'

Since the rewrite of deis CLI to go is nearing completion, it's become counterproductive to add new features to the soon-to-be-deprecated python client. But allowing for a custom git hostname as suggested in #3948 is still a valid use case, so a similar change (with tests!) in the go code under client-go/ would be welcome. Thanks @szymonpk!

authors: bacongobbler deis-admin mboersma szymonpk


text:
external minio option

an option to set to external minio?

ok, closing it in favour of https://github.com/deis/workflow/issues/481

repo: deis/workflow

license: mit


text:
docs(components): document the monitoring stack

closes #591

@bacongobbler the doc should mention optional persistent storage for influxdb/grafana

done. That should reflect those changes.

authors: bacongobbler rimusz


text:
fix: switch reference normalization to use docker parsing

Description
What this PR does / why we need it:
New testing revealed that the subject reference parsing logic was not comprehensive and didn't account for non library docker.io repositories. Upon further investigation, I found that Ratify is using the wrong reference parsing method from the upstream distribution repo. We should rely entirely on upstream's normalization methods to normalize subject refs into fully qualified references (adding latest if not tag/digest not present, use docker.io/library or docker.io/ when domain not specified, etc.) This PR updates the reference parsing method to use the upstream normalization function. It also updates the tests to add more coverage.
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Please also list any relevant details for your test configuration

[ ] Unit Tests pass
[ ] e2e tests pass

Checklist:

[ ] Does the affected code have corresponding tests?
[ ] Are the changes documented, not just with inline documentation, but also with conceptual documentation such as an overview of a new feature, or task-based documentation like a tutorial? Consider if this change should be announced on your project blog.
[ ] Does this introduce breaking changes that would require an announcement or bumping the major version?
[ ] Do all new files have appropriate license header?

One other finding is that this affects the request cache that was implemented at the http server handler level. Currently the cache key for that is the raw subject ref passed in from the External Data request. As a result, if someone specifies hello-world for one workload and docker.io/library/hello-world for another workload, they would be 2 different cache keys; however they resolve to the same normalized docker ref --> docker.io/library/hello-world:latest

repo: deislabs/ratify


text:
Uncontrolled inputs.

Currently uncontrolled inputs (One's that aren't linked to state) are re-rendered on every update.
Below when the "Toggle" button is clicked it will also clear the text input. React doesn't act this way and I'm wondering if there is a way around it without explicitly controlling the inputs state?
Home = {
  getInitialState: function() {
    return {};
  },
  handleClick: function(e, component, setState) {
    var state = component.state;
    return setState({
      clicked: !state.clicked
    });
  },
  render: function(arg) {
    var props, state;
    props = arg.props, state = arg.state;

        { state.clicked ?
Clicked
 : undefined }
        <button onClick={ Home.handleClick }>Toggle</button>
        <input type="text"/>

  }
};

Seems to happen when the order of elements gets moved around (Maybe internally the child list is flattened?)

Yeah this was something we were messing around with recently and something we deliberately did different. The only reason we didn't make it similar to React is that we've never run into a case where this didn't work for us.
In this case, the way the diffing works, is that it ignores undefined or null elements. So it's like it's not even there, meaning that when the div exists, the indexes all shift and they're replacing the HTML elements. Because the DOM keeps state you lose the value the user added.
To get around this you can either:

Not use undefined, and instead use a  or similar.
Just hide the div instead of removing it
Make the input a component and use the key attribute so the state is kept around (we don't yet support keys on elements, only on components).

Hope that helps!

That helps, that's what I thought was happening. Works fine without nullish elements. I am curious though, what is the benefit of ignoring null items as children? Seems like it would be convenient to keep them simply for child indexes.

That's a good point. I mainly use it so that I can just easily show and hide elements conditionally:
let sheet = null;
if (thereShouldBeASheet) {
  sheet = <Sheet />
}

return
{sheet}

But that could easily be done another way. What would you expect the falsy values to look like? Just empty strings?

Not 100% on the internals of deku, but I would guess that an nullish value when diffed would simply act as a removeChild operation.

Not 100% on the internals of deku, but I would guess that an nullish value when diffed would simply act as a removeChild operation.

Yeah this is how it works. If a virtual node is seen on the left, but on the right the value is undefined, it's treated as a removal. However, when rendering virtual elements, any null children are removed before the diff. This is creating the weird behaviour you're seeing (and others have brought up).
We could just keep null elements in there and render them as empty strings potentially. We'd need to do this because the diff looks for the real DOM elements via the index. So if the children looked like this:
[
  null,

]
```

But the real children looked like this:

```html

```

Then when using the indexes it would be wrong. We could either be tricky with this and shift the index, or we could just render a `<noscript/>` for `null` values so nothing renders.

@anthonyshort was this resolved?

authors: DylanPiercey anthonyshort

repo: dekujs/deku


text:
ERROR (Yajl::ParseError): lexical error: invalid char in json text.

when use redis-dump, something went wrong.
os：windows
the error value is \x03\x01000AAABfgSYbkk+7S9B94lIjJDrTQhJDSW\xB9,when export a json file ,is \u0003\u0001000AAABfgSYblkDQSRDVLxJv7DxWrXA3B+? .so it is wrong when load.
ERROR (Yajl::ParseError): lexical error: invalid char in json text.
gSYbkk+7S9B94lIjJDrTQhJDSW?,"\u0003\u0001000AAABfgSYblkDQSRD
(right here) ------^

@delano

repo: delano/redis-dump


text:
Multi platform support

The OAuth Authentication process nowadays can be triggered from many devices equipped with different os: watch OS, iOS, tvOS.
Unfortunately, not all of this operating system can rely on a safe way to implement the OAuth 2.0 authentication flow:

tvOS does not support SafariServices or Webkit.
watchOS does not support SafariServices or Webkit.
Today extension can not present webpages.

My idea to address this issue would be to implement for SwiftyOAuth following goals:

[ ] Make the library use app extension API only.
[ ] Add support for multiple platform: iOS, watchOS, tvOS.
[ ]  Improve -authorize method depending on current platform and capabilities.

What do you think?

Multi-platform support is definitely on the roadmap!
But I first want to make it very good on iOS before thinking about all the compatibility issues.
Here is my roadmap:

[ ] Store the token in the KeyChain (it will be the default instead of the current NSUserDefaults)
[ ] Add an option to present a WKWebView instead of SFSafariViewController / open Safari mobile

Some providers (like LinkedIn) don't accept app links and the only way to use OAuth is to handle the incoming URLs in webView(_:decidePolicyForNavigationAction:decisionHandler:)

(I'm started working on the WKWebView stuff btw)

Great,
I'm working on a separate branch on my repo for the multi platform support since I really think this is a required feature. Maybe later on you can give a look to see how it looks.
As I see from your roadmap I'd be more than happy to work on the keychain support.
Do you agree to use as dependency one of the library already available? I think this can help the development process and the library would have everything set to install dependencies for testing purposes too.
Sorry we are definitely going out of topic here. I can start a new issue to discuss this.

As I see from your roadmap I'd be more than happy to work on the keychain support.

You would be a life savior!!!

Do you agree to use as dependency one of the library already available to access the keychain?

No :/
We only need to set/get a dictionary in the KeyChain. It's really a pain to work with it, I know, but adding a dependency just for that is a bit too much...
I'm gonna keep a eye on your fork! Thanks a lot !! 🙏

I'm started working on the WKWebView stuff btw.

To pave the way for the multi platform support it would be great to remove call to openURL method from UIApplication Class

I pushed the next branch with the WebView thing (WIP) if you want to take a stab :)

💯
Next thing is to remove the reference to UIApplication to present the WKWebview or the SafariViewController.
Maybe the authorize can take as parameter an hostViewController? 🤔

Next thing is to remove the reference to UIApplication to present the WKWebview or the SafariViewController.

Why?

Maybe the authorize can take as parameter an hostViewController?

No, all the configuration (useWebView, additionalParams, etc) should be in only one place

By having in mind to support multi platform one day this task  UIApplication.sharedApplication() is not an allowed API.

We'll see when this day comes

Le 11 juin 2016 à 22:49, Fabio notifications@github.com a écrit :
By having in mind to support multi platform one day this task UIApplication.sharedApplication() is not an allowed API.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

authors: delba fabiomassimo

repo: delba/SwiftyOAuth


text:
Knex update

Updates knex to 2.1.0 and fixes some issues

Anyone?

I was about to make a PR for this also, we are going to have to fork in the meantime.

Updated to knex 2.3.0

Updated to knex 2.4.2

Can somebody please review my pull request?

@spilliton ?

Could anyone have a look at this? We are also running our tests against this branch for a couple of monthes.
@emurphy ?

authors: HaTiIn inform-aschneid schmetzyannick spilliton

repo: deliverr/data-knex-snowflake-dialect


text:
Source for images

Hi,
I'm trying to find the source code for the image 940776968316.dkr.ecr.eu-west-1.amazonaws.com/deliveryhero/kubecost-reports-exporter.
Is that open sourced anywhere?
Thanks!

Hey @nyambati can you either update the chart to use a public image, make our image public or add a Dockerfile to this chart?

@max-rocket-internet @hamza-tumturk I have updated the chart to include the source code and the reference to public repository. changes can be found in this PR #183

Resolved in https://github.com/deliveryhero/helm-charts/pull/183

repo: deliveryhero/helm-charts


text:
Get-OMEProfile | Delete-OMEProfile | Invoke-OMETemplateDeploy

Do you have a function to support grabbing the existing OME Profiles and deleting. I see you have functions to Detach and Unassign profiles. Although not sure on the difference as the descriptions are the same. Would love to do the following;

Identify if device has a profile
Unassign/detach profile from device
Delete leftover profile
Either create profile, or permit Invoke-OMETemplateDeploy to name the new profile it creates.

Sorry if this is duplicated and already exists

Detach and Unassign are the same thing. OME and OME-M have different API endpoints that do the same thing.
I will create commandlets for Get-OMEProfile and Remove-OMEProfile. I'll look into Profile naming as well.
I'm not a fan of the 1-to-1 mapping of Templates to Devices. This was a change that came with the MX platform. We used to be able to deploy as many Templates to devices as we wanted. Another method to do this is to deploy Templates to iDRACs directly, outside of OME. Tex has some commandlets that communicate with the iDRAC directly without this limitation. Then you can deploy as many Templates as you want without worrying about Profiles. Something to consider...
https://www.powershellgallery.com/packages/IdracRedfishSupport

Detach and Unassign are the same thing. OME and OME-M have different API endpoints for this.
I will work on creating commandlets for Get-OMEProfile and Remove-OMEProfile. I'll look into Profile naming as well.
I'm not a fan of the 1-to-1 mapping of Templates to Devices. This was a change that came with the MX platform. We used to be able to deploy as many Templates to devices as we wanted. Another method to do this is to deploy Templates to iDRACs directly, outside of OME. Tex has some commandlets that communicate with the iDRAC directly without this limitation. You could store your Templates in a git repo instead of OME since they are just XML/JSON files. Then you can deploy as many Templates as you want without worrying about Profiles. Something to consider...
https://www.powershellgallery.com/packages/IdracRedfishSupport

Going direct to iDRAC is something i do right now, but was looking to simpilfy the workflow for unattended iso installs through OME as the broker. Problem I have is templates to devices is something I have to work with as I have MX platforms, so need to use that target. But also have standalone rackmounts, so need OME also.

Would you mind sending me an email directly? I have some additional questions.
Trevor_Squillario@Dell.com

I think I covered everything in https://github.com/dell/OpenManage-PowerShell-Modules/releases/tag/3.7.0

Identify if device has a profile

# Get Profile Deployed Directly to Device
$Profile = Get-OMEProfile -Verbose | Where-Object { $_.ProfileName -eq "Profile from template 'MX740c_FC VLAN w IP' 00001" }
$DeviceId = $Profile.TargetId
$DeviceId
$Device = $DeviceId | Get-OMEDevice -FilterBy "Id"
$Device

# Get Profile Deployed to Chassis Slot
$Profile = Get-OMEProfile -Verbose | Where-Object { $_.ProfileName -eq "Profile from template 'MX740c_FC VLAN w IP' 00001" }
$DeviceSlotId = $Profile.DeviceIdInSlot
$Device = $DeviceSlotId | Get-OMEDevice -FilterBy "Id"
$Device

Unassign/detach profile from device

# Unassign Profile by device
Invoke-OMEProfileUnassign -Device $("37KP0ZZ" | Get-OMEDevice) -Wait -Verbose

# Unassign Profile on multiple devices
$("37KP0ZZ", "37KT0ZZ" | Get-OMEDevice) | Invoke-OMEProfileUnassign -Wait -Verbose

# Unassign Profile by template
Invoke-OMEProfileUnassign -Template $("TestTemplate01" | Get-OMETemplate) -Wait -Verbose

# Unassign Profile by profile name
Invoke-OMEProfileUnassign -ProfileName "Profile from template 'TestTemplate01' 00001" -Wait -Verbose

Delete leftover profile

"TestProfile01" | Get-OMEProfile | Remove-OMEProfile

Either create profile, or permit Invoke-OMETemplateDeploy to name the new profile it creates.

So you can't dictate the Profile name when Deployed from a Template. However you can create a new Profile and specify the name.
# Create a new Profile from a Template
"Test Template 01" | Get-OMETemplate | New-OMEProfile -NamePrefix "Test Profile" -NumberOfProfilesToCreate 3

# Create a new Profile from a Template and mount ISO from NFS share to Virtual Media
"Test Template 01" | Get-OMETemplate | New-OMEProfile -NamePrefix "Test Profile" -NumberOfProfilesToCreate 3 -NetworkBootShareType "NFS" -NetworkBootShareIpAddress "192.168.1.100" -NetworkBootIsoPath "/mnt/data/iso/OS.iso" -Verbose

# Assign Profile to Device
$Device = "933NCZZ" | Get-OMEDevice
"Test Profile 00001" | Get-OMEProfile | Invoke-OMEProfileAssign -TargetId $Device.Id -Verbose
Invoke-OMEProfileUnassign -ProfileName "TSTest 00001" -Wait -Verbose

# Assign Profile to Chassis Slot 1 and Apply Immediately ***This will force a reseat of the sled***
*See https://www.dell.com/support/kbdoc/en-us/000214041/mx7000-deploying-a-slot-based-template-shows-the-profile-as-assigned-and-virtual-identities-as-reserved

$Chassis = "933MCZZ" | Get-OMEDevice
$ChassisSlots = $Chassis | Get-OMEDeviceDetail -InventoryType "chassisSlotsList" | Select-Object -ExpandProperty InventoryInfo
$SlotId = $ChassisSlots | Where-Object { $_.Number -eq "1" -and $_.DeviceType -eq 1000 } | Select-Object -ExpandProperty Id
"Test Profile 00001" | Get-OMEProfile | Invoke-OMEProfileAssign -TargetId $SlotId -AttachAndApply -Wait -Verbose

repo: dell/OpenManage-PowerShell-Modules


text:
[BUG]: Authorization fails to install policies if selinux is enabled

Bug Description
During RPM install for Authorization, the policy-install.sh script fails if selinux is enabled.
Logs
Logs during RPM install:
Installing karavi-authorization-1.3.0
Using config file: /root/.karavi/config.json
Extracting files...Done!
Installing karavictl into /usr/local/bin...Done!
Installing Karavi-Authorization...Done!
error: failed to install policies (see /tmp/policy-install-for-karavi124838659): exit status 1
Installation Complete

Log output snippet from /tmp/policy-install-for-karavi124838659:
/tmp/karavi-installer-3494155233/policy-install.sh: line 8: /usr/local/bin/k3s: Permission denied

Screenshots
No response
Additional Environment Information
No response
Steps to Reproduce

Enable selinux
Install Authorization RPM

Expected Behavior

RPM install should be successful even if selinux is enabled

CSM Driver(s)
n/a
Installation Type
No response
Container Storage Modules Enabled
Authorization
Container Orchestrator
n/a
Operating System
RHEL 7.9

Reopening because this issue still exists when SELinux is enabled.

repo: dell/csm


text:
Predicate filtering does not apply column mapping

Not sure if I'm just doing something wrong but modifying the read-table-single-threaded example to apply a predicate, e.g.
let predicate = Expression::binary(BinaryOperator::GreaterThan, Expression::column("column_name"), 123);
let scan= snapshot
        .into_scan_builder()
        .with_schema_opt(read_schema_opt)
        .with_predicate(Arc::new(predicate))
        .build()?;

This only works if the column name I pass in is the physical name, not the logical name

I think there is more to this, I'm seeing some unexpected results even for physical fields:

Matching on physical String field

let predicate = Expression::binary(
        BinaryOperator::Equal,
        Expression::Column(ColumnName::new(["id"])),
        Expression::literal("A"),
    );

(cd kernel/examples/read-table-single-threaded; cargo run -- ../../tests/data/app-txn-checkpoint)

returns all the records without any filtering

Matching on physical Integer field

let predicate = Expression::binary(
        BinaryOperator::GreaterThan,
        Expression::Column(ColumnName::new(["value"])),
        6,
    );

(cd kernel/examples/read-table-single-threaded; cargo run -- ../../tests/data/app-txn-checkpoint)

filters out records with value being 1, 2 and 3 but still shows 4, 5, 6, 7 and greater

Matching on physical Long field

let predicate = Expression::binary(
        BinaryOperator::GreaterThan,
        Expression::Column(ColumnName::new(["number"])),
        6 as i64,
    );

(cd kernel/examples/read-table-single-threaded; cargo run -- ../../../acceptance/tests/dat/out/reader_tests/generated/multi_partitioned/delta/)

this seems to work fine

Matching on logical (partition) String field

let predicate = Expression::binary(
        BinaryOperator::GreaterThan,
        Expression::Column(ColumnName::new(["letter"])),
        Expression::literal("a"),
    );

(cd kernel/examples/read-table-single-threaded; cargo run -- ../../../kernel/tests/data/basic_partitioned/)

returns 0 records even though there is a partition with letter=a with valid records

Lastly, using a predicate with non-existing column name does not throw and just returns no records

A bit of further digging confirms that the predicate is only used for physical fields so your initial guess was right @jtanx
As for the other issues I listed above, they can probably be explained by examining how predicates work inparquet::arrow::async_reader::ParquetRecordBatchStreamBuilder which is used by the default engine

repo: delta-incubator/delta-kernel-rs


text:
xxx.dfs.core.windows.net not found in running [BUG]

Delta lake uses data lake storage gen2 for storage and generates an error of xxx.dfs.core.windows.net not found. The error occurs after the program runs for a few days, and it suddenly appears during normal operation. question.
------------------->
Configuration property xxx.dfs.core.windows.net not found.
at org.apache.hadoop.fs.azurebfs.AbfsConfiguration.getStorageAccountKey(AbfsConfiguration.java:342) ~[?:?]
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.initializeClient(AzureBlobFileSystemStore.java:812) ~[?:?]
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.(AzureBlobFileSystemStore.java:149) ~[?:?]
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:108) ~[?:?]
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361) ~[hadoop-common-3.2.0.jar:?]
at org.apache.parquet.hadoop.util.HadoopOutputFile.fromPath(HadoopOutputFile.java:58) ~[parquet-hadoop-1.10.1.jar:1.10.1]
at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:389) ~[parquet-hadoop-1.10.1.jar:1.10.1]
at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:349) ~[parquet-hadoop-1.10.1.jar:1.10.1]
at org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.(ParquetOutputWriter.scala:36) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat$$anon$1.newInstance(ParquetFileFormat.scala:149) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.sql.delta.Checkpoints$.$anonfun$writeCheckpoint$2(Checkpoints.scala:286) ~[?:?]
at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:863) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:863) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) ~[spark-core_2.12-3.1.1.ja
<-------------------
According to the documentation of hadoop-azure, the problem is caused by the wrong url, but there is no problem with my url, because the program can run normally, and it has been running for a few days, and the log can be queried that the problem will occur frequently Generated, but sometimes the executor will retry successfully, and sometimes the executor will fail to retry and cause the entire task to fail
According to the official documentation of delta lake connecting to data lake storage gen2, the connection is using oAuth, but my connection method is using shareKey
        String accountName = conf.get("spark.sensor.storage.azure.accountName", "");
        String accountKey = conf.get("spark.sensor.storage.azure.accountKey", "");
        String system = conf.get("spark.sensor.storage.azure.system", "");
        conf.set("spark.delta.logStore.class", "org.apache.spark.sql.delta.storage.AzureLogStore");
        conf.set("fs.azure.account.auth.type." + accountName + ".dfs.core.windows.net", "SharedKey");
        conf.set("fs.azure.account.key." + accountName + ".dfs.core.windows.net", accountKey);
        conf.set("fs.azure.enable.flush", "true");
        conf.set("fs.azure.enable.check.access", "true");

Does it have anything to do with this?
Environment information
Azure data lake storage gen2
Delta Lake version: 1.0.0
Spark version: 3.1.2
Hadoop version: 3.2.0
Scala version: 2.12

@kongluo do you have concurrent queries in the same Spark applications and each query itself may modify conf?

@kongluo do you have concurrent queries in the same Spark applications and each query itself may modify conf?

No, there is only one spark task to consume kafka data, and then write to data lake storage gen2, no other tasks

@kongluo do you have concurrent queries in the same Spark applications and each query itself may modify conf?

Does it have anything to do with the connection method with the data lake storage gen2? This problem was not found when the amount of data was small, and it would appear every few days when the amount of data was large

Does it have anything to do with the connection method with the data lake storage gen2?

I'm not familiar with the code inside hadoop-azure. But will try to help go through the Spark and Delta Lake part to see if it's a bug in Spark or Delta Lake.
Could you provide the entire stack trace? When Spark throws an exception like this, it should also have a stack trace in the driver.

Does it have anything to do with the connection method with the data lake storage gen2?

I'm not familiar with the code inside hadoop-azure. But will try to help go through the Spark and Delta Lake part to see if it's a bug in Spark or Delta Lake.
Could you provide the entire stack trace? When Spark throws an exception like this, it should also have a stack trace in the driver.

I am using structured streaming to process delta lake data to data
lake storage gen2, the driver does not print some error messages, only the above error messages. After the above errors occur, the driver and executor still exist, but the task status of the Streaming query will change from running to finishing

I'm wondering if you are able to share the logs around the error message. In addition, could you provide your code if possible?

I'm wondering if you are able to share the logs around the error message. In addition, could you provide your code if possible?

--------------------->
val conf: SparkConf = new SparkConf()
.set("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog")
.set("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.setAppName(SENSOR_STREAMING)
//          .setMaster("local[*]")
conf.set("spark.delta.logStore.class", "org.apache.spark.sql.delta.storage.AzureLogStore")
conf.set("fs.azure.account.auth.type." + accountName + ".dfs.core.windows.net", "SharedKey");
conf.set("fs.azure.account.key." + accountName + ".dfs.core.windows.net", xxx);
conf.set("fs.azure.enable.flush", "true");
conf.set("fs.azure.enable.check.access", "true");

val spark = SparkSession
  .builder()
  .config(conf)
  .getOrCreate()

spark.sparkContext.setLogLevel("INFO")

val TABLE_LOCATION: String = "abfss://xxx@xxx.dfs.core.windows.net/xxx"
val checkpointLocation: String = "abfss://xxx@xxx.dfs.core.windows.net/xxx"

import spark.implicits._
val sourceDF: DataFrame = spark.readStream
  .format("kafka")
  .option("kafka.bootstrap.servers", KafkaUtil.getKafkaServers(spark))
  .option("subscribe", "xxx")
  .option("maxOffsetsPerTrigger","20000")
  .load()
  .mapPartitions(par => {
    par.map(data => {
      DeltaUtil.transformData(data) //change data
    })
  })
  .filter(_._2 != null)
  .toDF(ID, VALUE, TIMESTAMP)
  .withColumn(S_TIME, get_json_object(col(VALUE), PATH_STIME).cast(DataTypes.LongType))
  .withColumn(PRODUCT_NAME, get_json_object(col(VALUE), PATH_PRODUCTNAME))
  .withColumn(PARTITION_DAY, from_unixtime(get_json_object(col(VALUE), PATH_STIME) / 1000, YYYY_MM_DD))

val query: StreamingQuery = sourceDF.writeStream
  .format("delta")
  .trigger(Trigger.ProcessingTime("40000"))
  .outputMode(OutputMode.Append())
  .option("mergeSchema", "true")
  .partitionBy(PRODUCT_NAME, PARTITION_DAY)
  .option("checkpointLocation", checkpointLocation)
  .start(TABLE_LOCATION)

query.awaitTermination()
spark.close()

<------------------------------
I filtered out some sensitive information, the overall operation is like this, that is, structured streaming consumes the data in kafka, after some transformation of the data, it is directly stored in the data lake storage gen2 through the format of delta lake

I'm wondering if you are able to share the logs around the error message. In addition, could you provide your code if possible?

the logs is :
--------------->
{"instant":{"epochSecond":1650520466,"nanoOfSecond":189222000},"thread":"Executor task launch worker for task 0.0 in stage 88.0 (TID 1327)","level":"INFO","loggerName":"org.apache.parquet.hadoop.ParquetOutputFormat","message":"Parquet block size to 134217728","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
{"instant":{"epochSecond":1650520466,"nanoOfSecond":189536000},"thread":"Executor task launch worker for task 0.0 in stage 88.0 (TID 1327)","level":"INFO","loggerName":"org.apache.parquet.hadoop.ParquetOutputFormat","message":"Max row count for page size check is: 10000","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
{"instant":{"epochSecond":1650520466,"nanoOfSecond":189469000},"thread":"Executor task launch worker for task 0.0 in stage 88.0 (TID 1327)","level":"INFO","loggerName":"org.apache.parquet.hadoop.ParquetOutputFormat","message":"Page size checking is: estimated","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
{"instant":{"epochSecond":1650520466,"nanoOfSecond":231847000},"thread":"Executor task launch worker for task 0.0 in stage 88.0 (TID 1327)","level":"INFO","loggerName":"org.apache.spark.sql.execution.datasources.parquet.ParquetWriteSupport","message":"Initialized Parquet WriteSupport with Catalyst schema:\n{\n  "type" : "struct",\n  "fields" : [ {\n    "name" : "txn",\n    "type" : {\n      "type" : "struct",\n      "fields" : [ {\n        "name" : "appId",\n        "type" : "string",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "version",\n        "type" : "long",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "lastUpdated",\n        "type" : "long",\n        "nullable" : true,\n        "metadata" : { }\n      } ]\n    },\n    "nullable" : true,\n    "metadata" : { }\n  }, {\n    "name" : "add",\n    "type" : {\n      "type" : "struct",\n      "fields" : [ {\n        "name" : "path",\n        "type" : "string",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "partitionValues",\n        "type" : {\n          "type" : "map",\n          "keyType" : "string",\n          "valueType" : "string",\n          "valueContainsNull" : true\n        },\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "size",\n        "type" : "long",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "modificationTime",\n        "type" : "long",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "dataChange",\n        "type" : "boolean",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "tags",\n        "type" : {\n          "type" : "map",\n          "keyType" : "string",\n          "valueType" : "string",\n          "valueContainsNull" : true\n        },\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "stats",\n        "type" : "string",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "partitionValues_parsed",\n        "type" : {\n          "type" : "struct",\n          "fields" : [ {\n            "name" : "product_name",\n            "type" : "string",\n            "nullable" : true,\n            "metadata" : { }\n          }, {\n            "name" : "partition_day",\n            "type" : "string",\n            "nullable" : true,\n            "metadata" : { }\n          } ]\n        },\n        "nullable" : true,\n        "metadata" : { }\n      } ]\n    },\n    "nullable" : true,\n    "metadata" : { }\n  }, {\n    "name" : "remove",\n    "type" : {\n      "type" : "struct",\n      "fields" : [ {\n        "name" : "path",\n        "type" : "string",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "deletionTimestamp",\n        "type" : "long",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "dataChange",\n        "type" : "boolean",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "extendedFileMetadata",\n        "type" : "boolean",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "partitionValues",\n        "type" : {\n          "type" : "map",\n          "keyType" : "string",\n          "valueType" : "string",\n          "valueContainsNull" : true\n        },\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "size",\n        "type" : "long",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "tags",\n        "type" : {\n          "type" : "map",\n          "keyType" : "string",\n          "valueType" : "string",\n          "valueContainsNull" : true\n        },\n        "nullable" : true,\n        "metadata" : { }\n      } ]\n    },\n    "nullable" : true,\n    "metadata" : { }\n  }, {\n    "name" : "metaData",\n    "type" : {\n      "type" : "struct",\n      "fields" : [ {\n        "name" : "id",\n        "type" : "string",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "name",\n        "type" : "string",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "description",\n        "type" : "string",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "format",\n        "type" : {\n          "type" : "struct",\n          "fields" : [ {\n            "name" : "provider",\n            "type" : "string",\n            "nullable" : true,\n            "metadata" : { }\n          }, {\n            "name" : "options",\n            "type" : {\n              "type" : "map",\n              "keyType" : "string",\n              "valueType" : "string",\n              "valueContainsNull" : true\n            },\n            "nullable" : true,\n            "metadata" : { }\n          } ]\n        },\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "schemaString",\n        "type" : "string",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "partitionColumns",\n        "type" : {\n          "type" : "array",\n          "elementType" : "string",\n          "containsNull" : true\n        },\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "configuration",\n        "type" : {\n          "type" : "map",\n          "keyType" : "string",\n          "valueType" : "string",\n          "valueContainsNull" : true\n        },\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "createdTime",\n        "type" : "long",\n        "nullable" : true,\n        "metadata" : { }\n      } ]\n    },\n    "nullable" : true,\n    "metadata" : { }\n  }, {\n    "name" : "protocol",\n    "type" : {\n      "type" : "struct",\n      "fields" : [ {\n        "name" : "minReaderVersion",\n        "type" : "integer",\n        "nullable" : true,\n        "metadata" : { }\n      }, {\n        "name" : "minWriterVersion",\n        "type" : "integer",\n        "nullable" : true,\n        "metadata" : { }\n      } ]\n    },\n    "nullable" : true,\n    "metadata" : { }\n  } ]\n}\nand corresponding Parquet message type:\nmessage spark_schema {\n  optional group txn {\n    optional binary appId (UTF8);\n    optional int64 version;\n    optional int64 lastUpdated;\n  }\n  optional group add {\n    optional binary path (UTF8);\n    optional group partitionValues (MAP) {\n      repeated group key_value {\n        required binary key (UTF8);\n        optional binary value (UTF8);\n      }\n    }\n    optional int64 size;\n    optional int64 modificationTime;\n    optional boolean dataChange;\n    optional group tags (MAP) {\n      repeated group key_value {\n        required binary key (UTF8);\n        optional binary value (UTF8);\n      }\n    }\n    optional binary stats (UTF8);\n    optional group partitionValues_parsed {\n      optional binary product_name (UTF8);\n      optional binary partition_day (UTF8);\n    }\n  }\n  optional group remove {\n    optional binary path (UTF8);\n    optional int64 deletionTimestamp;\n    optional boolean dataChange;\n    optional boolean extendedFileMetadata;\n    optional group partitionValues (MAP) {\n      repeated group key_value {\n        required binary key (UTF8);\n        optional binary value (UTF8);\n      }\n    }\n    optional int64 size;\n    optional group tags (MAP) {\n      repeated group key_value {\n        required binary key (UTF8);\n        optional binary value (UTF8);\n      }\n    }\n  }\n  optional group metaData {\n    optional binary id (UTF8);\n    optional binary name (UTF8);\n    optional binary description (UTF8);\n    optional group format {\n      optional binary provider (UTF8);\n      optional group options (MAP) {\n        repeated group key_value {\n          required binary key (UTF8);\n          optional binary value (UTF8);\n        }\n      }\n    }\n    optional binary schemaString (UTF8);\n    optional group partitionColumns (LIST) {\n      repeated group list {\n        optional binary element (UTF8);\n      }\n    }\n    optional group configuration (MAP) {\n      repeated group key_value {\n        required binary key (UTF8);\n        optional binary value (UTF8);\n      }\n    }\n    optional int64 createdTime;\n  }\n  optional group protocol {\n    optional int32 minReaderVersion;\n    optional int32 minWriterVersion;\n  }\n}\n\n       ","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
{"instant":{"epochSecond":1650520466,"nanoOfSecond":335283000},"thread":"Executor task launch worker for task 0.1 in stage 88.0 (TID 1328)","level":"INFO","loggerName":"org.apache.spark.executor.Executor","message":"Running task 0.1 in stage 88.0 (TID 1328)","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
{"instant":{"epochSecond":1650520466,"nanoOfSecond":334983000},"thread":"dispatcher-Executor","level":"INFO","loggerName":"org.apache.spark.executor.CoarseGrainedExecutorBackend","message":"Got assigned task 1328","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
{"instant":{"epochSecond":1650520466,"nanoOfSecond":336527000},"thread":"Executor task launch worker for task 0.1 in stage 88.0 (TID 1328)","level":"INFO","loggerName":"org.apache.spark.broadcast.TorrentBroadcast","message":"Started reading broadcast variable 154 with 1 pieces (estimated total size 4.0 MiB)","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
{"instant":{"epochSecond":1650520466,"nanoOfSecond":335899000},"thread":"Executor task launch worker for task 0.1 in stage 88.0 (TID 1328)","level":"INFO","loggerName":"org.apache.spark.MapOutputTrackerWorker","message":"Updating epoch to 54 and clearing cache","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
org.apache.hadoop.fs.azurebfs.contracts.exceptions.ConfigurationPropertyNotFoundException: Configuration property xxx.dfs.core.windows.net not found.
at org.apache.hadoop.fs.azurebfs.AbfsConfiguration.getStorageAccountKey(AbfsConfiguration.java:342) ~[?:?]
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.initializeClient(AzureBlobFileSystemStore.java:812) ~[?:?]
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.(AzureBlobFileSystemStore.java:149) ~[?:?]
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:108) ~[?:?]
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361) ~[hadoop-common-3.2.0.jar:?]
at org.apache.spark.sql.delta.util.DeltaFileOperations$.$anonfun$registerTempFileDeletionTaskFailureListener$1(DeltaFileOperations.scala:331) ~[?:?]
at org.apache.spark.sql.delta.util.DeltaFileOperations$.$anonfun$registerTempFileDeletionTaskFailureListener$1$adapted(DeltaFileOperations.scala:328) ~[?:?]
at org.apache.spark.TaskContext$$anon$2.onTaskFailure(TaskContext.scala:141) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.TaskContextImpl.$anonfun$markTaskFailed$1(TaskContextImpl.scala:115) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.TaskContextImpl.$anonfun$markTaskFailed$1$adapted(TaskContextImpl.scala:115) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.TaskContextImpl.$anonfun$invokeListeners$1(TaskContextImpl.scala:137) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.TaskContextImpl.$anonfun$invokeListeners$1$adapted(TaskContextImpl.scala:135) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) ~[scala-library-2.12.10.jar:?]
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) ~[scala-library-2.12.10.jar:?]
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) ~[scala-library-2.12.10.jar:?]
at org.apache.spark.TaskContextImpl.invokeListeners(TaskContextImpl.scala:135) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.TaskContextImpl.markTaskFailed(TaskContextImpl.scala:115) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.scheduler.Task.run(Task.scala:136) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
at java.lang.Thread.run(Unknown Source) [?:?]
{"instant":{"epochSecond":1650520466,"nanoOfSecond":338821000},"thread":"Executor task launch worker for task 0.1 in stage 88.0 (TID 1328)","level":"INFO","loggerName":"org.apache.spark.storage.memory.MemoryStore","message":"Block broadcast_154_piece0 stored as bytes in memory (estimated size 82.3 KiB, free 3.7 GiB)","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
{"instant":{"epochSecond":1650520466,"nanoOfSecond":339448000},"thread":"Executor task launch worker for task 0.1 in stage 88.0 (TID 1328)","level":"INFO","loggerName":"org.apache.spark.broadcast.TorrentBroadcast","message":"Reading broadcast variable 154 took 2 ms","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
org.apache.hadoop.fs.azurebfs.contracts.exceptions.ConfigurationPropertyNotFoundException: Configuration property xxx.dfs.core.windows.net not found.
at org.apache.hadoop.fs.azurebfs.AbfsConfiguration.getStorageAccountKey(AbfsConfiguration.java:342) ~[?:?]
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.initializeClient(AzureBlobFileSystemStore.java:812) ~[?:?]
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.(AzureBlobFileSystemStore.java:149) ~[?:?]
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:108) ~[?:?]
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) ~[hadoop-common-3.2.0.jar:?]
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361) ~[hadoop-common-3.2.0.jar:?]
at org.apache.parquet.hadoop.util.HadoopOutputFile.fromPath(HadoopOutputFile.java:58) ~[parquet-hadoop-1.10.1.jar:1.10.1]
at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:389) ~[parquet-hadoop-1.10.1.jar:1.10.1]
at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:349) ~[parquet-hadoop-1.10.1.jar:1.10.1]
at org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.(ParquetOutputWriter.scala:36) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat$$anon$1.newInstance(ParquetFileFormat.scala:149) ~[spark-sql_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.sql.delta.Checkpoints$.$anonfun$writeCheckpoint$2(Checkpoints.scala:286) ~[?:?]
at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:863) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:863) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.rdd.RDD.iterator(RDD.scala:337) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.scheduler.Task.run(Task.scala:131) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500) ~[spark-core_2.12-3.1.1.jar:3.1.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
at java.lang.Thread.run(Unknown Source) [?:?]
{"instant":{"epochSecond":1650520466,"nanoOfSecond":340008000},"thread":"Executor task launch worker for task 0.1 in stage 88.0 (TID 1328)","level":"INFO","loggerName":"org.apache.spark.storage.memory.MemoryStore","message":"Block broadcast_154 stored as values in memory (estimated size 323.2 KiB, free 3.7 GiB)","endOfBatch":true,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":13,"threadPriority":5}
<-------------
I also desensitized the log, and our log is converted into json format. I shared the log before and after the error report.

@kongluo thanks for the stack trace. I think you are hitting a issue that has been fixed in 1.0.1: https://github.com/delta-io/delta/commit/61763fe9d404532b534ac1e16998a0d6e905bbba Could you try 1.0.1 and see if the issue still happens?

Closing this as the issue has been fixed in 1.0.1. Feel free to re-open this issue if you still see the issue after upgrading to 1.0.1.

@kongluo thanks for the stack trace. I think you are hitting a issue that has been fixed in 1.0.1: 61763fe Could you try 1.0.1 and see if the issue still happens?

After I upgraded to 1.0.1, I observed it for a few days, and no errors were reported so far. Thank you very much for your reply.

authors: kongluo zsxwing

repo: delta-io/delta


text:
TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/84015

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/110821


text:
wish : remove from playlist

It seems impossible to remove a podcast from the playlist without  marking it read or deleting it.
I wish it were possible to remove a podcast from the playlist but leave it downloaded in the library

It looks like the full view (>1280 Width) has a dedicated button for that. I added the same button to the smaller views. I checked and the Podcast was still in the downloaded folder as well as market as unplayed. Can @deltakosh confirm that that button just deletes the entry from the play list?

The Trashcan:

Yep this is correct :D

Hello
I made a few comments, added a few items to the wishlist over the past few
days because I have decided to use exclusively my lumia 950 to listen to my
daily podcasts ( before I used Downcast on iOS), and your app is so close
to being perfect (for me)
Therefore, all my comments and bug reports are for mobile....I am not
completely used to the universal app concept... :)
for example, time remaining (or total time?) does appear on the PC app but
not on the mobile app..which woul be very usefull !
best regard
2017-06-14 22:43 GMT+02:00 David Catuhe notifications@github.com:

Yep this is correct :D
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/deltakosh/Podcasts/issues/48#issuecomment-308552407,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AcELuB2yVHidsylI6no-motfMdKwcjTeks5sEEYCgaJpZM4N5v4q
.

I swipe the item on the playlist to the left. It shows me the hidden button to delete it.

authors: MMEnter deltakosh glehenry kceiw

repo: deltakosh/Podcasts


text:
Error reading Postgis - Geometry column

I am getting following error when reading Geometry column from table.
System.InvalidOperationException: 'Expected column "geom" of type "System.Object" at position 6 (0-based indexing) but received column "geom" of type "NetTopologySuite.Geometries.Geometry".'

The column in database table is declared as
geom geometry(Point,4326),

Using Npgsql works correctly.
I have following set but it looks like the type provider is not respecting it.
NpgsqlConnection.GlobalTypeMapper.UseNetTopologySuite() |> ignore

following is the test app code
// Learn more about F# at http://fsharp.org

open System
open Npgsql

NpgsqlConnection.GlobalTypeMapper.UseNetTopologySuite() |> ignore

open FSharp.Data.Npgsql
open System.Runtime.InteropServices

module DbConnection =
    [<Literal>]
    let ConnectionStringName = "ConnectionStringName"

    let private IsWindows = System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(OSPlatform.Windows)
    let private ConnectionStringSeparator = if IsWindows then ":" else "__"
    let AppConnectionString = lazy Environment.GetEnvironmentVariable("ConnectionStrings" + ConnectionStringSeparator + ConnectionStringName)

type AppProvider = NpgsqlConnection<DbConnection.ConnectionStringName, ConfigType = ConfigType.Environment>

let openConnection() =
    let con = new Npgsql.NpgsqlConnection(DbConnection.AppConnectionString.Value)
    con.Open()
    con.TypeMapper.UseNetTopologySuite() |> ignore
    con

let testNpgsql() =
    let conStr = @"mycon";
    use con = new Npgsql.NpgsqlConnection(conStr)
    con.Open()
    use cmd = new Npgsql.NpgsqlCommand("select geom from addresses", con)
    use reader = cmd.ExecuteReader()
    reader.Read() |> ignore
    printfn "%A" reader.[0]

let testTypeProvider() =
    use con = openConnection()
    use trn = con.BeginTransaction()
    use cmd =
        AppProvider.CreateCommand<"select * from addresses", XCtor = true>(con, trn)
    let address = cmd.Execute() |> List.head
    printfn "%A" (address.geom)

[<EntryPoint>]
let main argv =
    //testNpgsql()
    testTypeProvider()
    0 // return an integer exit code

Anyone any clues as to why this is broken?

Fixed in 0.2.10


text:
content build: add pipenv and pyenv python installs of 3.6.8 and 2.7.15

Add pipenv and pyenv installs of python 2 and 3.
Plan to use pipenv as part of dev dependency installs for unit tests

Docker Image Ready - Dev
Docker automatic build at CircleCI has deployed your docker image: devdemisto/content-build:1.0.0.157
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/content-build/tags
Get started by pulling the image:
docker pull devdemisto/content-build:1.0.0.157

Docker Metadata

Image ID: sha256:d225db9c036c0f3f9327ccfb4862dbc29b9714b41670c7f2582d9ec78efe8e94
Created: 2019-03-02T22:17:49.011360529Z
Arch: linux/amd64
Command: ["/bin/sh"]
Environment:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
NODE_VERSION=9.11.2
YARN_VERSION=1.5.1
DEBIAN_FRONTEND=noninteractive
LANG=C.UTF-8

Labels:

org.opencontainers.image.authors:Demisto <containers@demisto.com>
org.opencontainers.image.revision:12a573510c141f9f3179295adae59808138325b9
org.opencontainers.image.version:1.0.0.157

Docker Image Ready - Dev
Docker automatic build at CircleCI has deployed your docker image: devdemisto/content-build:1.0.0.158
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/content-build/tags
Get started by pulling the image:
docker pull devdemisto/content-build:1.0.0.158

Docker Metadata

Image ID: sha256:2aa7035303045df219c337af68661e821481327d285ec37d23feb13e8841ff63
Created: 2019-03-02T22:27:27.383204818Z
Arch: linux/amd64
Command: ["/bin/sh"]
Environment:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
NODE_VERSION=9.11.2
YARN_VERSION=1.5.1
DEBIAN_FRONTEND=noninteractive
LANG=C.UTF-8

Labels:

org.opencontainers.image.authors:Demisto <containers@demisto.com>
org.opencontainers.image.revision:454b2e9c4560b2073febf441bf27e585d2fc171b
org.opencontainers.image.version:1.0.0.158

Docker Image Ready - Dev
Docker automatic build at CircleCI has deployed your docker image: devdemisto/content-build:1.0.0.159
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/content-build/tags
Get started by pulling the image:
docker pull devdemisto/content-build:1.0.0.159

Docker Metadata

Image ID: sha256:f1e123ab1c52ef4a03905c9c9bc940c0241b75d1220dcd5190a1cef503c4a7ce
Created: 2019-03-03T09:01:33.631164645Z
Arch: linux/amd64
Command: ["/bin/sh"]
Environment:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
NODE_VERSION=9.11.2
YARN_VERSION=1.5.1
DEBIAN_FRONTEND=noninteractive
LANG=C.UTF-8

Labels:

org.opencontainers.image.authors:Demisto <containers@demisto.com>
org.opencontainers.image.revision:04538f42d2a7150bba419cfc7c736244de6df8aa
org.opencontainers.image.version:1.0.0.159

authors: dc-builder glicht

repo: demisto/dockerfiles


text:
Support sockets

I think socket support is a must have for any ambitious framework, since it would probably be the deciding factor to use Denali for anyone needing them or thinking that they would in the future.
Also, it's an opportunity to shine, since socket code is usually more cluttered then router code, so if we can have a convention around sockets as well, all the better.

💯
Yea, we'll definitely need to support sockets at some point. A quick brain dump of some of my thoughts on this so far:
Don't "unify" transports by default
I'm generally not a fan of the Sails.js style approach where controller actions can be reached by HTTP or socket connections, and the framework abstracts that away.
The framework should certainly allow for that, but the idea that most of your socket code is identical to HTTP handling code seems like a bad assumption to me. Sockets have fundamental differences from the standard HTTP req/res paradigm, and abstracting away those differences just results in a lowest common denominator of HTTP & sockets, without any of the strengths of either transport.
We need to find the right primitives
Denali's Actions are a fundamental primitive that enables building extensible and robust HTTP endpoints. In the same fashion, we need to think through what the fundamental primitives of a socket server would be.
Unfortunately, I don't have a ton of experience with production apps that use sockets extensively. From what I can see looking at other frameworks, a Channel might be a candidate for that low level primitive, but I'm not sure.
Ideally, we could find a primitive that enables building more complex / opinionated abstractions on top. In the same way that Actions can be composed into more complex addons (i.e. denali-auth), I'd like to see socket primitives composable into more complex structures.
Multiple primitives?
Perhaps a good solution might involve multiple primitives for different use cases? I.e. we could take the "unified transport" approach mentioned above (i.e. the client can reach any action via HTTP or sockets), but then offer another primitive (i.e. Channels) for use cases that lean of the unique bi-directional / realtime nature of sockets?

So, I think our next step here is to identify what some common use cases are for sockets. From there, we can hopeful come up with a good primitive, with stories for how the use cases are satisfied. Then we can get into the weeds of specific implementation details.
I put up a wiki page detailing this - feel free to add any of your own thoughts too. I figure that can be a home base for the current state of our planning.

Closing in favor of http://discuss.denalijs.org/t/sockets-realtime-support/29

authors: davewasmer knownasilya

repo: denali-js/denali


text:
Markdown definition lists are not rendered in site preview

Please select the area your Bug applys to. (Multiple selections are Possible)

[ ] UI
[ ] Workbench
[x] Publishing
[ ] Pods
[ ] Markdown
[ ] Hierarchy
[ ] Graph
[ ] Misc (Choose this if your not sure)

Describe the bug
Markdown definition lists are not rendered correctly in site preview. See screenshot below.
To Reproduce
Steps to reproduce the behaviour:

Add a definition list to a note:Item 1
: Definition for Item 1

Run "Dendron: Site Preview" command
Navigate to the note's preview page
The definition list is rendered as a single line: Item 1 : Definition for Item 1

Expected behaviour
Markdown definition lists are properly rendered:

Screenshots

Desktop (please complete the following information)

OS: Win 10x64
Dendron version: 0.36.2

Thanks for submitting. this is a feature parity issue between the local preview or the site preview. adding to our backlog

Definition lists are not rendered in V2 Preview/Publish as well.

authors: aleksey-rowan kevinslin

repo: dendronhq/dendron


text:
Web admin root path + http auth

Hi !
I'm using a broker with a modified webadmin path + a protected http access :

URL http://my-broker.local:8161/activemq-console/xml/queues.jsp is working after a http auth step.

But this collectd module can't handle auth nor webadmin path changes :
> ./activemq_info.py my-broker.local 8161
DEBUG:__main__:Using host my-broker.local
DEBUG:__main__:Using port 8161
Enter username for ActiveMQRealm at my-broker.local:8161: <handtyped_user>
Enter password for iprint in ActiveMQRealm at my-broker.local:8161: <handtyped_password>
INFO:__main__:activemq_info plugin [verbose]: activemq_info plugin: No info received, offline node or turned off ActiveMQ
DEBUG:__main__:Metrics to send:
None

urllib seems waiting user console input for credentials, which is not really great for an automated process
To make it work in any situation, I've made multiple changes here :

switching from urllib to requests : requests handles http auth way easier.
adding -w / --webadmin option that allow to change default webadmin root path

default is still .../admin/...

adding -l / --login and -p / --password to handle http auth.

authentication is tried only if login option is specified.

Result :
> ./activemq_info.py my-broker.local 8161 -w activemq-console -l my_user -p my_pass
DEBUG:__main__:Using host my-broker.local
DEBUG:__main__:Using port 8161
DEBUG:__main__:Authenticating with my_user
INFO:urllib3.connectionpool:Starting new HTTP connection (1): my-broker.local
DEBUG:urllib3.connectionpool:"GET /activemq-console/xml/queues.jsp HTTP/1.1" 200 3250
DEBUG:__main__:Metrics to send:
{ 'counters': [ (u'ORDER_IN', 'enqueueCount', u'1'),
                (u'ORDER_IN', 'dequeueCount', u'0'),
              [...]
              ],
  'gauges': [ (u'ORDER_IN', 'size', u'1'),
              (u'ORDER_IN', 'consumerCount', u'0'),
              [...]
              ]}

I think I failed my rebase, let me fix that...

And I missed collectd config either ...

Will fix typo and add note about requests later.
Thanks a lot!

authors: BarthV deniszh

repo: deniszh/collectd-activemq-python


text:
Consider providing the ability to color objects such as "InterfacePropertyDef" externally

At present, we can only use DocPrinter to decide if it should colorize or not. But when deno_doc is used as a crate, we may not always want to print an entire DocNode object. But if we don't call colors::enable, no colors will be displayed.
related: https://github.com/denoland/deno_doc/pull/38

@zhmushan that would require completely rewrite of printer so it can format any struct available in the crate. I'm happy to take contributions but it's something we won't be working on in the foreseeable future.

I'm gonna close this issue for now as there's not much interest in it and it would require significant engineering effort.

authors: bartlomieju zhmushan


text:
Add: Add a string type to know that our function will return a string

Description
Add a type for the function to know that it will return a string
Issue: #1369

Do we actually need this? It seems like typescript can always infer that this returns a string, so I'm not sure what this does. That being said, I'm still new to typescript, so maybe I'm missing something.
If anything, it seems like we should remove the static typing from the 2nd method in this file, assetSrcSet. It's currently declared like this:
export function assetSrcSet(srcset: string): string {

and returns like this:
  return constructed.join(",");

so we know (and so does typescript) it always returns a string.

@deer It's more of a cosmetic change, since like you said TS can infer those types. It would help in cases where changes suddenly change the inferred return type though. But that said in all the years working with TS my experience is that this is more of a theoretical risk that I haven't encountered in practice. But I don't mind if somebody wants to add these return types to ramp up on the code base. So I'm happy to merge such PRs when CI passes.

It's just like @marvinhagemeister said. It's more about a theoretical risk rather than an actual risk and also for me is to continue with the typing just like i've seen in other functions just like the next one which has a :string as a return.
Thank you both!

authors: deer lausuarez02 marvinhagemeister

repo: denoland/fresh


text:
Plugin tries to push through Obsidian Mobile

Hey,
Obsidian Git plugins seems to try syncing via the mobile app, but is unsuccessful, I cannot specify the path on mobile because the path does not exist on mobile, but on my desktop.

You can't specify a path, because there is no way to install git on a mobile device. The plugin ships it's own git implementation.

Actually, there was an option to "disable on this device" as the very last option in settings which fixed the issue.

You can't specify a path, because there is no way to install git on a mobile device. The plugin ships it's own git implementation.

What I mean by that, by the way, is that the notification implied I need to specify the path. I forgot to include the prompt screenshot so it might have been unclear!

repo: denolehov/obsidian-git


text:
add wait method

Adds a method to EventEmitter called wait that awaits the completion of each listener. Note: awaiting this method does not work the same way as awaiting emit.

I dont see much need for this; maybe this should be emit's behaviour? it just seems confusing to have such similar methods

authors: SadiePi crowlKats

repo: denosaurs/event


text:
Is there a way to include multiple numbers?

Hi All,
I'm sorry this is a very very basic question but today is the first time I have used Raw! I am loving it but having a few problems with my data (which is very simple)
I have 2 strings and each strings has 2 numbers but when I go to visualise I am not able to show both strings, with both of their values at the same time.
I can only show one string with both values. Is there any way to rectify this? On the charts I have been looking at I am only numbers are singular.
Any help would be much appreciated.
Many thanks

Hello,
can you provide a sample of the data? I'm not sure to have understood the problem

Probably you have multiple values in the same cell. In this case you must work on the dataset splitting them or in different columns, or in different rows, then reuploading it in RAWGraphs. OpenRefine could be a good solution:
https://github.com/OpenRefine/OpenRefine/wiki/Cell-Editing#splitting-multiple-values-within-cells-to-produce-records

repo: densitydesign/raw


text:
fixed small typo error

I only fixed a small typo error

Thanks @0xflotus

repo: denysdovhan/spaceship-prompt


text:
Docker image with multiple architectures is updated to the same hash

Is there an existing issue for this?

[X] I have searched the existing issues

Package ecosystem
docker
Package manager version
No response
Language version
No response
Manifest location and content before the Dependabot update

# https://github.com/zulu-openjdk/zulu-openjdk/blob/7e5fcfad3a22416db51719e4d80ca07004ac2f91/17-jre-headless-latest/Dockerfile
# https://hub.docker.com/r/azul/zulu-openjdk/tags?page=1&name=17-jre-headless

FROM azul/zulu-openjdk@sha256:59fe77a0432aa1393d3b8c8d0cf39b551c2516e899508fa809346b1fe9966d13 as zulu-amd64
FROM azul/zulu-openjdk@sha256:6292231e5fb042c01caa573f6ffeeb5da1b9c82586bff6086770b8ea42575cc7 as zulu-arm64

dependabot.yml content

version: 2
updates:
  - package-ecosystem: "maven"
    directory: "/"
    schedule:
      interval: "daily"
  - package-ecosystem: "npm"
    directory: "/"
    schedule:
      interval: "daily"
  - package-ecosystem: "gradle"
    directory: "/"
    schedule:
      interval: "daily"
  - package-ecosystem: "docker"
    directory: "/"
    schedule:
      interval: "daily"

Updated dependency
azul/zulu-openjdk
What you expected to see, versus what you actually saw
Expected:
I was hoping that dependabot would be able to filter the updates based on the architecture
What I saw:
A single pull request updating both architectures to the same version e63e4ba2e7d3fdc47bcc67c64a854914887f9c5fc3b3208ebc03d5c4f2d955bc

diff --git a/Dockerfile b/Dockerfile
index df1fcae..fdd24ab 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,5 +1,5 @@
 # https://github.com/zulu-openjdk/zulu-openjdk/blob/7e5fcfad3a22416db51719e4d80ca07004ac2f91/17-jre-headless-latest/Dockerfile
 # https://hub.docker.com/r/azul/zulu-openjdk/tags?page=1&name=17-jre-headless

-FROM azul/zulu-openjdk@sha256:59fe77a0432aa1393d3b8c8d0cf39b551c2516e899508fa809346b1fe9966d13 as zulu-amd64
-FROM azul/zulu-openjdk@sha256:6292231e5fb042c01caa573f6ffeeb5da1b9c82586bff6086770b8ea42575cc7 as zulu-arm64
+FROM azul/zulu-openjdk@sha256:e63e4ba2e7d3fdc47bcc67c64a854914887f9c5fc3b3208ebc03d5c4f2d955bc as zulu-amd64
+FROM azul/zulu-openjdk@sha256:e63e4ba2e7d3fdc47bcc67c64a854914887f9c5fc3b3208ebc03d5c4f2d955bc as zulu-arm64

Native package manager behavior
No response
Images of the diff or a link to the PR, issue, or logs
Pull request
https://github.com/yeikel/dependabot-playground/pull/66/files
Smallest manifest that reproduces the issue

# https://github.com/zulu-openjdk/zulu-openjdk/blob/7e5fcfad3a22416db51719e4d80ca07004ac2f91/17-jre-headless-latest/Dockerfile
# https://hub.docker.com/r/azul/zulu-openjdk/tags?page=1&name=17-jre-headless

FROM azul/zulu-openjdk@sha256:59fe77a0432aa1393d3b8c8d0cf39b551c2516e899508fa809346b1fe9966d13 as zulu-amd64
FROM azul/zulu-openjdk@sha256:6292231e5fb042c01caa573f6ffeeb5da1b9c82586bff6086770b8ea42575cc7 as zulu-arm64

This seems related to https://github.com/dependabot/dependabot-core/issues/6700

repo: dependabot/dependabot-core


text:
EditorConfig should use 'space' not 'spaces'

The valid option for indent_style is space not spaces per the documentation :-)

I have no idea why the build is failing

I'll handle the failure - it's unrelated. Thanks for the PR!

authors: evenh greysteil


text:
Fixed index error on deps parsing

Added additional check that list is not empty
Close #6

Perfect! Thank you

authors: MrMrRobat orsinium


text:
uploading via PhpSecLib with PEM file doesn't work

Deployer version: 3.0.10
PHP version: 5.6.9
Deployment target(s) OS - Ubuntu 12.04
The upload function gives some pretty cryptic errors and fails when authenticating with PEM and using PhpSecLib for uploading. I get:
  [RuntimeException]
  NET_SFTP_STATUS_FAILURE: Failure
  NET_SFTP_STATUS_FAILURE: Failure
  NET_SFTP_STATUS_FAILURE: Failure

While the command line scp works just fine:
scp -i aws.pem file.php ubuntu@somehost.amazonaws.com:/var/www/location

Bug in PhpSecLib?

It appears that way. It seems to be messing packets up in the SFTP class

Is somebody have a work around ? that will realy help me.
Thx

The isssue still exists. I get this while using upload:
NET_SFTP_STATUS_FAILURE: Failure
...
NET_SFTP_STATUS_FAILURE: Failure
NET_SFTP_STATUS_FAILURE: Failure

Exception trace:
 () at phar://D:/bin/deployer/deployer.phar/src/Server/Remote/PhpSecLib.php:136
 Deployer\Server\Remote\PhpSecLib->upload() at phar://D:/bin/deployer/deployer.phar/src/functions.php:373
 upload() at D:\xampp\htdocs\domizeli.ru\deploy.php:37
 {closure}() at n/a:n/a
 call_user_func() at phar://D:/bin/deployer/deployer.phar/src/Task/Task.php:79
 Deployer\Task\Task->run() at phar://D:/bin/deployer/deployer.phar/src/Executor/SeriesExecutor.php:43
 Deployer\Executor\SeriesExecutor->run() at phar://D:/bin/deployer/deployer.phar/src/Console/TaskCommand.php:82
 Deployer\Console\TaskCommand->execute() at phar://D:/bin/deployer/deployer.phar/vendor/symfony/console/Command/Command.php:259
 Symfony\Component\Console\Command\Command->run() at phar://D:/bin/deployer/deployer.phar/vendor/symfony/console/Application.php:840
 Symfony\Component\Console\Application->doRunCommand() at phar://D:/bin/deployer/deployer.phar/vendor/symfony/console/Application.php:192
 Symfony\Component\Console\Application->doRun() at phar://D:/bin/deployer/deployer.phar/vendor/symfony/console/Application.php:123
 Symfony\Component\Console\Application->run() at phar://D:/bin/deployer/deployer.phar/src/Deployer.php:101
 Deployer\Deployer->run() at phar://D:/bin/deployer/deployer.phar/bin/dep:62
 require() at D:\bin\deployer\deployer.phar:4

I would love to do whatever I can to fix this issue, but that is mainly testing. I am on Windows. Is it not broken on other OSs. This bug seems to be rather important. Is there any chance it will be worked on some time soon?

I'm getting the same error when I pass only directory path and not full file path for the second parameter.
For example this throws the mentioned error:
upload('test.txt', '/some/upload/dir');
but this doesn't
upload('test.txt', '/some/upload/dir/test.txt');

authors: Kleinast brianium elfet ivangretsky levani

repo: deployphp/deployer


text:
🛑 Derailed Home is down

In d344bbb, Derailed Home (https://derailed.one) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Derailed Home is back up in b0ba7ee.

repo: derailed-org/derailed-oss-status


text:
First clip doesn't load

Hi,
I'm using the Frigate Lovelace card. Since the latest update the Lovelace card doesn't show the 1st (latest) video anymore.
I'm using the card in the "most recent clip" view.

I need to
Android HA app:
Swipe to the previous video and then swipe back to the newest video to be able to watch the newest video.
Firefox browser:
Navigate to another dashboard tab and go back to my "security" tab to be able to watch the newest video.
or
Choose to edit the HA dashboard > Then it showes the video correctly
Chrome browser:
Navigate to another dashboard tab and go back to my "security" tab to be able to watch the newest video.
or
Choose to edit the HA dashboard > Then it showes the video correctly
I did think about the browser cache and history wipe.
No error(s) logs in the HA log
No error(s) in the Frigate log
Yesterday I rebuild all of the Frigate installation:
Deleted HA add-on
Deleted HA integration > Wich the Lovelace card is comming from
Deleted all Frigate related files via the file editor
Deleted all Media
Media > Frigate (Integration folder)
Media > My Media > frigate (Add-On folder)
Deleted the Lovelace card and builded it back up via the GUI
  - type: custom:frigate-card
    cameras:
      - camera_entity: camera.frigate_cam1
      - camera_entity: camera.frigate_cam2
      - camera_entity: camera.frigate_cam3
    view:
      default: clip
      update_force: true
      scan:
        show_trigger_status: true
    menu:
      style: outside
      buttons:
        media_player:
          enabled: false
        timeline:
          enabled: false
        live:
          enabled: false
        snapshots:
          enabled: false
    media_viewer:
      controls:
        title:
          mode: none
        thumbnails:
          mode: none
        timeline:
          mode: below
          window_seconds: 28800
    media_gallery:
      controls:
        thumbnails:
          show_favorite_control: false
          show_timeline_control: false
          show_download_control: false
        filter:
          mode: none

Just to make sure I understand, the most recent clip doesn't work, but all the others do? Is this the case no matter how recently the clip was? i.e. even if it was hours ago, the first clip will never work?

Hi,
That is correct.
I have 3 camera's configured in the card. Only the first (default) camera clip acts this way.

I have 3 camera's configured in the card. Only the first (default) camera clip acts this way.

And can you view the same clip that won't load via the HA media browser? (i.e. outside the card, but inside the HomeAssistant Media sidebar tab)

In the media browser I see the same issue. The first clip doesn't load

I have same problem.
In browser console, I have an error, when click on first clip:
96996.w7vKRby48d0.js:1 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'KeySystemFormats')
    at 96996 (96996.w7vKRby48d0.js:1:283711)
    at Function.c (bootstrap:19:32)
    at async p.value (ha-hls-player.ts:112:11)
96996 @ 96996.w7vKRby48d0.js:1
c @ bootstrap:19
log-message.ts:37

GET https://xxx/unknown/webpack/bootstrap 404 (Not Found)

Hi dermotduffy.
I did report it upstream first ;)
https://github.com/blakeblackshear/frigate/issues/9827
I think it is not a Frigate Add-on media browser problem because in the media browser of the add-on I do not have issues.
I have 2 media browser directories:

From the Frigate Add-on: Media > My Media > frigate
From this Frigate integration: Media > Frigate

Media directory number 1 does not show this behaviour. Some clips are split in 2 peaces inside the directory.
The most recent clip (recording) of Media directory number 2 is showing the same behaviour as the Lovelace card.

Frigate Add-on 0.13.1

I'm sorry you are being bounced around! On the other issue you mentioned the card and not that this ALSO doesn't work in the stock HA media browser without the card being involved.

I think it is not a Frigate Add-on media browser problem because in the media browser of the add-on I do not have issues.

I think the issue can only be with one of these (none of which are the Frigate add-on, and none of which are the card):

Frigate itself ... which passes the video to ...
... the Frigate integration ... which plays through ...
... the video player that resides in the HASS frontend to play videos

Put simply: You could uninstall the card entirely, and it still wouldn't play in the HA media browser. Can't be the card ;-)

Hi,
Correct me if I am wrong but your integration provides a media directory.
In that directory I can reproduce the issue. In the Frigate Add-on directory I can't

Correct me if I am wrong but your integration is creating a media directory. In this directory I can reproduce the issue.

That's the integration, this repo is the card.

Swipe to the previous video and then swipe back to the newest video to be able to watch the newest video.

This indicates that it is not a backend issue right? Because the same request works after updating the ui?

This indicates that it is not a backend issue right? Because the same request works after updating the ui?

I missed that, great point! However, there's must be more than just the card going on here since the issue happens in the stock media browser without the card at all.
@woempiej  I wonder if you try to open it in the HA media browser (and it fails), but then you try to open it a second time, does it succeed? [i.e. are we seeing the same symptoms in the media browser as in the card].
Both the card and HA share the same video player code, so it could be that. Or I guess it could be that somehow something on the backend is returning different data on the first and subsequent requests?

Hi,
Here a small tutorial of my actions to try to reproduce

@dermotduffy Yes the second time in the media browser there is no issue.
Because of this, I thought it was a problem just with the default camera in the card but it is not!
When I log out of HA en close the browser > Login again ant follow the above tutorial to another camera > It is showing the same behaviour > Latest recording isnt't playing

How strange. Thanks for the walk-thru screenshot. For the record, I can do the same, go to the media browser, choose Recordings, choose a camera, choose the latest available hour -- it will play first time every time. I'm on HA 2024.1.3 (slightly  older than you), I can try to upgrade later to see if perhaps something changed in HA itself?
Suggestion: In one window, navigate to "Logs" in your Frigate instance, and choose "Nginx" on the top-right. In another window, repeat your steps above (with the first request than fails, and the second that succeeds), then paste in what new log lines are created. Interesting ones likely contain something like "GET /vod/2024-02/..." [these are the requests HA is making to your backend, I'm curious if both requests are being received at all].

Looks like both requests are served okay from the backends perspective (both http status 200), although the response sizes look small compared to my testing, but that could be a red herring.
@woempiej Do you get any messages in your Javascript console (press F12 in Chrome), when you attempt to play the video from the media browser?

Anyone have any working solutions? I am experiencing the exact same issue as described above

Hi all,
Issue is fixed in HA 2024.3.0

Can confirm that 2024.3 has fixed this issue

authors: NickM-27 PablaV dermotduffy wizu woempiej

repo: dermotduffy/frigate-hass-card