text:
Document Advanced Use Cases for Apply instead of just string transformations

Problem description
As I was developing I needed to access Output/Input variables to be used in other .NET code such as templating a file. Currently the documentation only talks about using Tuple/Apply for changing/accessing variables to then convert to another simple string. My use case required me to run more complex logic inside the Apply method that would read a file, template it using Input/Outputs, then return this new "file" as a new Output string to be used when VM is created. I was finally able to figure this out after talking to people on Slack but it would be really helpful to add to the documentation more about this use case of logic needing to be inside the apply wrapper to access variables. It would also be beneficial to go more into detail on why you have to put logic inside these wrappers and cannot use regular variables in the language, where the reasons are they are not created yet at preview, or just not available yet, etc. It took me a bit to wrap my head around the Input/Output variables and why I couldn't just ToString them to it's raw form.
Here is an example code that I used which shows I needed to pass in outputs using the tuple and running the logic inside the apply
// Template Userdata file using Input/Output Pulumi Variables
var userDatafileGzipBase64 = Output.Tuple(args.Name.ToOutput(), args.DomainName.ToOutput(), tlsPrivateKey.PublicKeyOpenssh).Apply(x =>
{
    var (hostName, domainName, sshPubKey) = x;
    // Using liquid template engine to render file
    var renderedTemplate = Template.Parse(userDataFileRaw).Render(Hash.FromAnonymousObject(new
    {
        vmHostName = hostName,
        vmDomainName = domainName,
        vmSshPubKey = sshPubKey,
    }));
    return StringHelper.EncodeStringToGzipBase64(renderedTemplate);
});

Great point @mitchellmaler - I think we could definitely add another more complex example like the one you share above to https://www.pulumi.com/docs/intro/concepts/programming-model/#all.

authors: lukehoban mitchellmaler

repo: pulumi/docs


text:
Eliminate hyperlinks to TF docs

It looks like we are emitting hyperlinks to TF docs. For instance, see this page:

The docker_image hyperlink goes to https://www.terraform.io/docs/providers/docker/r/image.html. Shouldn't it link to https://www.pulumi.com/docs/reference/pkg/docker/remoteimage/?
Interestingly, that resource page also links to TF docs!

@joeduffy I am working through this right now - I have managed this on a few resources so far - there is an issue in the home repo tracking this - ping me for more information if you need it

Ah got it, I'll close this as a dupe -- didn't know there was an existing issue.

source: gharchive/issue

license_type: permissive

license_source: github-api


text:
Incorrect link

File: docs/reference/pkg/aws/s3/bucketobject.md
source
pulumi.asset.Asset | pulumi.asset.Archive => 404

I just ran into this as well. This seems to be the case across all languages and we're attempting to link to the language API doc for Archives rather than our pulumi implementation.
e.g.

https://www.pulumi.com/docs/reference/pkg/aws/s3/bucketobject/#source_nodejs links to the Mozilla docs
https://www.pulumi.com/docs/reference/pkg/aws/s3/bucketobject/#source_csharp links to Microsoft's docs

i do not see the links referenced in this issue, @clstokes is this still an issue?

Looks like the changes to move docs into the Pulumi Registry made this a non-issue. Closing.

authors: clstokes susanev thrixton


text:
FSx -> EFS Documentation Typo

I noticed in most of the EFS types, there's a reference to FSx for the fileSystemId property. (I specifically noticed this in the TypeScript types.)
https://github.com/pulumi/pulumi-aws/blob/86e3d6fc051753ae12890c0c775775a5d5dbe6c6/sdk/dotnet/Ecs/Outputs/TaskDefinitionVolumeEfsVolumeConfiguration.cs#L21

This is another instance of the issue tracked in https://github.com/pulumi/pulumi-terraform-bridge/issues/529.

Going to close this out as a duplicate of https://github.com/pulumi/pulumi-terraform-bridge/issues/529 to track the required work centrally.

authors: garrett-hopper lukehoban

repo: pulumi/pulumi-aws


text:
Repeating sentence in Azure 'Create a New Project' page

File: themes/default/content/docs/get-started/azure/create-project.md
At the end, the sentence "After the command completes, the project and stack will be ready." is repeated three times. It seems all three can be removed, since the sentence "After some dependency installations from npm, the project and stack will be ready." seems to explain what happens.

Ah, nevermind, seems one of my Chrome plugins was causing this.

repo: pulumi/pulumi-hugo


text:
Add reference architecture examples that mimic real projects

We should help people understand how to build more sophisticated solutions out of the building blocks.
This section probably needs much more thought and should be based on real customer projects.
Suggestions:

[ ] Here is how you take your React app, link it to ASP.NET backend with a Cosmos DB, and here's why you need Pulumi for that
[ ] Here is how you make a static web site with a serverless backend
[ ] Lots of pages for Kubernetes strugglers
[ ] Microservices, CQRS, Event Sourcing - less mess with Pulumi
[ ] Data-intensive and Machine learning scenarios

Going to close this out as a duplicate of several things that are being tracked elsewhere:

Architecture Templates
Examples subsite
Best Practices Architecture Guides

authors: bermudezmt lukehoban


text:
Add methods to the Reference / Query class to match Javascript docs

List of operations here:
https://firebase.google.com/docs/reference/js/firebase.database.Reference
If I'm putting together a wishlist: orderByChild, startAt / endAt / equalTo, isEqual, limitToFirst / limitToLast, update.

Thanks for taking time to submit this issue! I should actually have some time today to work on this. Stay tuned.

1.0.0-dev.6.0 is now on Pub:
https://pub.dartlang.org/packages/firebase_admin_interop/versions/1.0.0-dev.6.0#-changelog-tab-
Should have all methods you mentioned.
Only Query.off, Query.on and Reference.transaction are missing, I think. Feel free to submit a separate issue if any of those become critical (and I don't get to them sooner :-) ).

authors: TonyDowney pulyaevskiy

repo: pulyaevskiy/firebase-admin-interop


text:
URL checking for content types

A primary hurdle to the refactor of our code is having a common way to check that urls have what is needed to move forward to show a page to the user. Content-type management pages have a specific URL path pattern which can be tested.
Feature: URL checks before page load
  As a core developer
  I want content management urls to be confirmed for correctness
  So that a route's code can be only code required to create the route's user interface.

  Scenario: URL
    Given that routes in Punchcard must follow specific patterns
      And that those routes can be broken up into testable parts-of-the-path
     When setting up routes in Punchcard
     Then I need to have middleware that checks each part of a path for correctness
      And it should check a content path's parts start with an existing content type
      And it should check a content id is in UUID format
      And it should check a content revision is a number
      And it should check the url's action exists among available actions

  Scenario: Workflow
    Given that content management pages require a workflow
    When testing routes for content in Punchcard
    Then I need to have the system check that a content-type's workflow exists

  Scenario: Error
    Given that a user needs to know what has gone wrong with their attempt to visit a url
    When a content route path test fails
    Then the error should be captured
      And the error should trigger the next step to show the user what has gone wrong.

URL Checks

[x] URL
[x] Workflow
[x] Errors

completed by #309

repo: punchcard-cms/punchcard

license_source: bigquery


text:
[Bug]: Freezing when querying frames on certain websites

Minimal, reproducible example
const puppeteer = require("puppeteer");

async function main () {
    console.log("Operative");

    const browser = await puppeteer.launch();
    const page = await browser.newPage();

    await page.goto('http://fonologomed.it/');

    const allIframeLinks = [];
    const frames = page?.frames() ?? [];

    for (const frame of frames) {
        try {
            if (!frame?.url() || frame.detached) {
                continue;
            }

            const links = await frame.$$eval('a', (anchors) => anchors.map((anchor) => ({
            href: anchor.href,
                text: anchor.text,
    	    })));

            allIframeLinks.push(...links);
        }
        catch (e) {
            console.log(e);
        }
    }

    console.log(allIframeLinks);
    console.log("Success");
}

main();

Error string
no error
Bug behavior

[ ] Flaky
[ ] PDF

Background
We want to retrieve the  links from the iframes within a website. This works for the majority of visited websites, but we noticed on certain websites this operation completely is freezing (the Promise is pending forever).
Running the reproducible example on http://fonologomed.it/ will freeze
For a contrarian example, running the reproducible example on https://www.aranzulla.it will NOT freeze
Expectation
The promise to resolve or at least get an error
Reality
The operation gets stuck as the Promise never gets resolved
Puppeteer configuration file (if used)
No response
Puppeteer version
22.6.5
Node version
20.3.0
Package manager
npm
Package manager version
9.6.7
Operating system
macOS

I guess it cannot be reproduced by the workflow because it gets stuck?

The promise frame.$$eval is never resolved.

I think it is another example of https://github.com/puppeteer/puppeteer/issues/10696 basically the browser built-in PDF viewer creates empty iframes which never gets an execution context created for them as regular iframe. So the eval calls wait for the context as expected from the Chrome DevTools Protocol, but never gets them for those frames. Maybe it is because it is treated as part of the browser UI although the technical implementation is an iframe.

A workaround could be like
      let parent = frame.parentFrame();
      let skip = false;
      while (parent) {
        if (parent.url().endsWith('.pdf')) {
          skip = true;
          break;
        }
        parent = parent.parentFrame();
      }
      if (!frame?.url() || frame.detached || skip) {
        continue;
      }

or more general since it is reasonable to not expect any context on about:blank pages
      if (!frame?.url() || frame.detached || frame.url('about:blank')) {
        continue;
      }

Hi @OrKoN,
thank you for the answer
We thought it might be a PDF issue in fact we tried filtering by PDF extension as you proposed in the first workaround. Although we didn't traverse the parent frames so this is why probably our solution didn't work. We will try your first workaround
The second workaround is problematic. In fact in our solution we want to read frames with "about:blank". From our tests it results that calling frame.url() on iframes without a src set, returns "about:blank". We have several cases of frames without a src set, they are automatically generated by plugins and widgets, and since they have content we need, we want to read them

You can also disable PDF viewer via OOPIFs for now via
args: [
      '--disable-features=PdfOopif'
    ]

You can also disable PDF viewer via OOPIFs for now via
args: [
      '--disable-features=PdfOopif'
    ]

Thank you,
is there any documentation about this option?

There is an issue for that https://issues.chromium.org/issues/40268279

After further investigation, it looks like the feature might not be the reason for hanging (it seems to reproduce now with the feature disabled). It appears that the iframes from the pdf viewer extensions get reported into the page sessions via CDP (although they probably have to be contained to the extension target).

I am experiencing freezing problem for http URLs but not https. Could that be the main issue here?

authors: AntonioFavero OrKoN Vasile-Peste przhkv

repo: puppeteer/puppeteer


text:
page.evaluate querySelectorAll is always empty

I have a node project using puppeteer.
I run the following code in console and I get 170 results back
circlenod = window.document.querySelectorAll('area[templateid="ucChart_pnlip"]');
    for (var i = 0; i < circlenod.length; i++) {
        console.log('circlenod --> : ', circlenod[i]);
    }

However when I try to use page evaluate in puppet or $$ method, I get no results coming back
let test;
let list = await page.$$('area[templateid="ucChart_pnlip"]');
console.log('list ===> ', list); ===> this is empty
await page.evaluate(() => {
    console.log('coming in ??'); ==> never see this
    test = Array.from(document.querySelectorAll('area[templateid="ucChart_pnlip"]'));
    for (var i = 0; i < test.length; i++) {
        console.log('circlenod --> : ', test[i]); ==> never see this
    }
 })
 console.log('test', test); ==> undefined

This is an example of the element. How can I extract this information from the attribute after looping through all the <area /> fields="date|5/1/2020|14"
<area shape="poly" coords="802,235,807,233,807,241,802,243" fields="date|5/1/2020|14" templateid="ucChart_pnlip" href="../../../../../../UserControls/History/Single/#" onclick="charttip_Show(this, event);return false" alt="">

@aslushnikov hey bud I am tagging you since I have seen many responds you been giving out

I use something like this-
const elementArray = await page.$$('area[templateid=ucChart_pnlip]')
for (const element of elementArray) {
const text = await element.evaluate(e => e.innerText)
console.log('TEXT: ' + text)
}

i'm experiencing same issue

authors: SeongwoonHong bezba sfelice777


text:
Does waitForFunction support async function

Steps to reproduce
Tell us about your environment:

Puppeteer version: 0.10.2
Platform / OS version: MacOS
URLs (if applicable):

What steps will reproduce the problem?
I want to wait an async code, for example
await page.waitForFunction(async () => {
  await new Promise(resolve => setTimeout(resolve, 1000));
});

What is the expected result?
It should wait until the function return true;
What happens instead?
it doesn't.

It shouldn't. You can use page.evaluate for this.
await page.waitForFunction(async () => {
  await new Promise(resolve => setTimeout(resolve, 1000));
});

Both evaluate and waitForFunction throw the same error when using async:
 Evaluation failed: ReferenceError: tslib_1 is not defined

      at __puppeteer_evaluation_script__:1:16
      at ExecutionContext._evaluateInternal (node_modules/puppeteer/src/common/Executi
onContext.ts:273:13)
      at ExecutionContext.evaluate (node_modules/puppeteer/src/common/ExecutionContext
.ts:138:12)

authors: aslushnikov cloydlau popomore


text:
fix: defaultViewport can be null in BrowserConnectOptions

Steps to reproduce
Description:
As per API defaultViewport field in launch options ( puppeteer.lauch( { defaultViewport: ... } ) link ) can have the three below values:

Viewport object - The specified viewport will be applied.
undefined (i.e. defaultViewport field is absent) - Default viewport will be applied.
null - No viewport will be applied. Useful in headful mode, because page content will accomodate to browser extent.

Tell us about your environment:

Puppeteer version: 8.0.0
Platform / OS version: N/A
URLs (if applicable): https://github.com/puppeteer/puppeteer/blob/0b5969dea9048dc34a4d2e109e96c42e8928182c/src/common/BrowserConnector.ts#L40
Node.js version: N/A

What steps will reproduce the problem?

Create a TypeScript project that launches a Chromium instance using puppeteer.launch(options) function.
Among options specify defaultViewport field as null.

Please include code that reproduces the issue.
puppeteer.lauch({
  headless: false,
  defaultViewport: null,
});

What is the expected result?
Code is accepted as valid, no type error raises.
What happens instead?
Type error raises, which tells that null is not valid value for defaultViewport.

@googlebot I signed it!

Could you please add more details? Ideally - open an issue justifying the need of the change.
What is the use-case when the defaultViewport is null?

@sadym-chromium, thank you for the advice.
I added details.

@sadym-chromium Another open pull request with linked issues https://github.com/puppeteer/puppeteer/pull/6942
Also, according to spec defaultViewport <?Object> Sets a consistent viewport for each page. Defaults to an 800x600 viewport. null disables the default viewport. null disables the default viewport.

@sadym-chromium thank you for your review and approval.
I realized that #6942 is the proper fix of the same issue.
I would close this issue in favour of #6942. Could you review that one, please?

Closed in favour of #6942

source: gharchive/pull-request

authors: L0tso sadym-chromium zoltan-boros


text:
Add 'config' parameter to with_project spec helper

Use Case
The project spec_helper function with_project currently reads a variable project_config from the rspec test it's used in, which can be opaque when reading the tests. We should pass config to with_project explicitly.

Closed by #2466

authors: beechtom lucywyman

repo: puppetlabs/bolt


text:
Add LXD Transport

LXD, via the lxc command line tool, provides the primitives for building a full-featured
bolt transport: remote execution, pushing and pulling files and directories, setting UID,
GID, and working directories.

I am opening this as a draft because I will need some help getting this over the finish line. Some methods are unimplemented, there aren't tests, there aren't docs, etc. So there's quite a bit of work to do,
Feel free to ping me on Slack with suggestions or questions.

Waiting for CLA signature by @dontlaugh
@dontlaugh - We require a Contributor License Agreement (CLA) for people who contribute to Puppet, but we have an easy click-through license with instructions, which is available at https://cla.puppet.com/
Note: if your contribution is trivial and you think it may be exempt from the CLA, please post a short reply to this comment with details. http://docs.puppet.com/community/trivial_patch_exemption.html

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.2 out of 3 committers have signed the CLA.:white_check_mark: beechtom:white_check_mark: lucywyman:x: dontlaughYou have signed the CLA already but the status is still pending? Let us recheck it.

Closing in favor of https://github.com/puppetlabs/bolt/pull/2648

authors: CLAassistant dontlaugh puppetcla


text:
(FACT-596) Update selinux_config_policy fact

The newer version of selinux use the term "Loaded policy name"
in lieu of "Policy from config file".
This updated version matches on both for backard compatibility
reasons.
This should fix FACT-756 as well.

CLA signed by all contributors.

@peterhuene do you know if cfacter needs touchup to match this?

@frconil Thank you for your contribution!
PR Triage Notes:

@frconil can you add a unit test for this change?
@frconil can you retarget at the stable branch?
We will also need to verify if this is an issue for cfacter, and file a ticket if needed /cc @peterhuene @MikaelSmith

@joshcooper I've added the changes and test on the stable branch, should I open a new PR and refer this one?

@frconil yes, can you close in favor of a pull on stable (also including the spec changes)?
Thanks!

Oops! I see you did as #848. Thanks!
Closing this one in favor of that.

authors: frconil joshcooper kylog puppetcla

repo: puppetlabs/facter


text:
(PUP-6257) support --allow-authorization-extensions

This commit updates puppet cert sign to accept
--allow-authorization-extensions, which permits the signing of CSRs with
authorization extensions.
This closes the loop for utilizing authorization extensions in PE as we
now have a way of signing auth extensions in a trusted way.
This commit comes with a minor refactoring of ca.sign and
ca.check_internal_signing_policies to take an options hash.
important my puppet abilities are too weak to manually test this. I would love if someone could do that (and maybe show me how). And it'd be great to have acceptance tests for this.

CLA signed by all contributors.

@adrienthebo updated.
i'll point out that tests started failing for me on master with this error: https://gist.github.com/nathanielksmith/d7ad121f02c32652af400b6995784180

I've set up a functional review env here at xuw8g83nboton5b.delivery.puppetlabs.net (note I've manually signed pp17.delivery.puppetlabs.net).
If you want to replicate that environment you should be able to by placing these files in puppet's acceptance directory and executing the appropriate command in command.sh.
If you want to deploy using new code you'll need to make sure they are pushed to your remote and update this option, you can also do a git pull in /opt/puppet-git-repos/puppet to update new code if you're iterating on a previous environment. Feel free to iterate in the above env.

@justinstoller updated; and thanks for the functional testing!

The code looks fine and I've validated that this is working as expected!
👍

authors: justinstoller nathanielksmith puppetcla

repo: puppetlabs/puppet


text:
(maint) Iterable performance improvements

commit message from 4a137aa8e1902f10fe999659dd079272d444b538:
In Puppet's functions involving iteration we call
Iterable.asserted_iterable() to ensure whatever we are given is
properly coerced into an iterator. However, we also do a recursive type
inference on any elements of an Array, or keys and values if passed a
Hash. This is unnecessary for nearly every workflow and can be
prohibitively costly when working with large collections.
The one usage that could use an Iterator with typed elements is if a
user called step or reverse_each and attempted to save that
value or pass it into a 4x Puppet function defined w/in a .pp file. In
those cases a user could assert that they received an
Iterator[SomeElementType].
To account for this we add a new optional parameter to
Iterable.asserted_iterable and Iterable.on that allows us to control
whether collection elements are recursively typed. We now only request
elements of a collection to be recursively typed if the user has called
the specific arities of reverse_each or step.

from commit c27195cc58bf8ebe9768554090049148bc159480:
Within Iterable we register a method_missing hook to delegate any
methods called on it that have not been defined to the wrapped
enumeration. However, we call several methods within our functions that
we can easily track and know will end up being handled by
method_missing.
Instead of leaving the delegation of methods we need for core Puppet
functions to the method_missing handler this patch directly delegates
for explicitness.

CLA signed by all contributors.

It dawned on me after I wrote up that commit message that I could add a param for infer_elements to asserted_iterable and pass that as true in the 1-arity versions of reverse_each and step. I'm all for removing Iterator/Iterable as part of the public API in 7, but that would let us side step the issue in 5.x & 6.x, I think.

Update the code and PR message(s) based on the above observation.

jenkins please test this with servertests

For the first commit, just checking my understanding, only step and reverse_each are need the inference because those are the only two usages that can return the Iterable itself (if not passed a block)? It looks like all of the other Puppet functions that call asserted_iterable return a simple value of some kind, which I assume is type-checked separately.

I think these changes make sense to me. Happy to merge once the docs comments are updated.

authors: Magisus justinstoller puppetcla


text:
(maint) Update to latest ezbake

Ezbake 0.3.21 was just released, so this updates to use that version.

Refer to this link for build results (access rights to CI server needed):
http://jenkins-enterprise.delivery.puppetlabs.net/job/enterprise_puppetdb_init-multijob_githubpr-stable/60/
Test PASSed.

authors: puppetlabs-jenkins underscorgan

repo: puppetlabs/puppetdb


text:
(maint) improve dotestseq macro

Before this commit the dotestseq macro didn't show the values of the bindings per iteration in the test name, only the names of them.
Testing case ([version endpoint] method)
Testing case ([version endpoint] method)
Testing case ([version endpoint] method)
Testing case ([version endpoint] method)

This commit imoproves the macro such that it now produces better annotated test names like so:
Testing case: [version endpoint]: [:v4 "/v4/catalogs/foo/edges"], method: :get
Testing case: [version endpoint]: [:v4 "/v4/catalogs/foo/edges"], method: :post
Testing case: [version endpoint]: [:v4 "/v4/catalogs/foo/resources"], method: :get
Testing case: [version endpoint]: [:v4 "/v4/catalogs/foo/resources"], method: :post

Note that we've applied a similar fix in pcp-broker some time ago already: https://github.com/puppetlabs/pcp-broker/commit/839a77204ca043d25b5b3ae7f21c7cd42dc83a8e

Refer to this link for build results (access rights to CI server needed):
http://jenkins-enterprise.delivery.puppetlabs.net/job/enterprise_puppetdb_init-multijob_githubpr-master/608/
Test PASSed.

Refer to this link for build results (access rights to CI server needed):
http://jenkins-enterprise.delivery.puppetlabs.net/job/enterprise_puppetdb_init-multijob_githubpr-master/609/
Test PASSed.

@mruzicka I think there are pending comments on this PR, can you take a look?

@mruzicka ping

This has been pending for quite a while with no update, I'm going to close it. Please re-open when ready.

authors: kbarber mruzicka puppetlabs-jenkins


text:
(MODULES-2286) Create Tests for Basic DSC Resource "File"

Add tests for "File" DSC resource type.

Note that some of the negative tests that match Puppet error output are going to need to be modified as a result of MODULES-2292 and the new error output.
See https://github.com/puppetlabs/puppetlabs-dsc/commit/217b994e00325c95a29c626a399a830608a99093  for a couple of changes that were made to the suite to address the error message changes.

@Iristyle @zreichert Fixed, ready for review again.

LGTM +1

authors: Iristyle cowofevil zreichert

repo: puppetlabs/puppetlabs-dsc


text:
S0007 increase required free disk space

Use Case
PE 2021.6 ships the pe_databases module by default I think. PE recommends using it for older versions as well. It configures 3 timers that run pg_repack on the pe-puppetdb. pg_repack requires free diskspace that's at least as big as the (biggest table+index)*2
from https://reorg.github.io/pg_repack/:

Performing a full-table repack requires free disk space about twice as large as the target table(s) and its indexes. For example, if the total size of the tables and indexes to be reorganized is 1GB, an additional 2GB of disk space is required.

Describe the Solution You Would Like
It would be helpful if the fact could identify the biggest table and multiply it by two and use that as lower limit
Describe Alternatives You've Considered
if another implementation is too tricky, maybe the message for S0007 should be adjusted like:

more than 20% free, but check you table size. maybe you need more free disk space.

Additional Context
Indeed pg_repack doesn't work reliable in my environment when I've less than the recommended free storage.

Hi @MartyEwings, can you take a look at the issue please?

@bastelfreak Let me have a think how best to handle this, at the moment 20% was arbitrary, as it was really to indicate gradual linear growth.
Ideally yes finding those tables we repack, adding them up and multiplying by 2 would be perfect, id need to check the calculation times on that one, as I'm trying to make sure this module doesn't blow out faster resolution times.
Trying to pick a static value that covers nearly all cases is probably best here, Obviously 50% free would account for repacking EVERY table in the postgres instance, but this is too high in practice, and probably would always be triggered making the indicator meaningless for most. So the ideal number is probably somewhere between 20% and 50%.
I think,and id like to hear your thoughts, doing some sort of napkin math on the most likely static percentage for this scenario, AND putting the additional considerations in the warning message may be the way to go?

I've some stats from a customer:
SELECT nspname || '.' || relname AS "relation",
    pg_size_pretty(pg_total_relation_size(C.oid)) AS "total_size"
  FROM pg_class C
  LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
  WHERE nspname NOT IN ('pg_catalog', 'information_schema')
    AND C.relkind <> 'i'
    AND nspname !~ '^pg_toast'
  ORDER BY pg_total_relation_size(C.oid) DESC
  LIMIT 20;

(thanks to the awesome postgresql wiki where I copied this from ages ago)
which produces:
             relation             | total_size
----------------------------------+------------
 public.resource_params           | 181 GB
 public.resource_params_cache     | 55 GB
 public.catalog_resources         | 10 GB
 public.reports_20220914z         | 2173 MB
 public.reports_20220915z         | 2170 MB
 public.reports_20220913z         | 2043 MB
 public.edges                     | 1942 MB
 public.reports_20220912z         | 1780 MB
 public.reports_20220911z         | 1741 MB
 public.reports_20220916z         | 954 MB
 public.factsets                  | 267 MB
 public.resource_events_20220914z | 193 MB
 public.resource_events_20220913z | 128 MB
 public.resource_events_20220916z | 106 MB
 public.resource_events_20220915z | 82 MB
 public.resource_events_20220912z | 68 MB
 public.resource_events_20220911z | 58 MB
 public.fact_paths                | 19 MB
 public.catalogs                  | 8352 kB
 public.certnames                 | 4784 kB
(20 rows)

pe-puppetdb=#

total DB size is around 260GB, but resource_params eats up the majority of it, roughly 70%. If that ratio is the same for all users, it would mean we need free disk space that's at least 70% of the DB size. so we could do $free_diskspace - ($used_diskspace * 0,7). if the result is negative we need more diskspace. Does that make sense? (under the assumption that there is mostly postgresql on the partition and nothing else that eats a lot of diskspace).

Increasing the threshold for available space would cause instances of this alert to rise.
Closing this out.

@bastelfreak It's an expensive set of functions call, that wouldn't be performant in a fact. What we can do however is a status check for if repacks are succeeding or not, that would be less expensive and would output the same result.

authors: Aaronoftheages MartyEwings bastelfreak


text:
Allow usage of 'role PUBLIC' in GRANT/REVOKE:

To generate this kind of query:
REVOKE CONNECT ON DATABASE database1 FROM PUBLIC;

I created following puppet code:
postgresql::server::database_grant { 'revoke connect on database1':
    ensure => 'absent',
    db => 'database1',
    privilege => 'CONNECT',
    role => 'PUBLIC',
}

This generate following error:
Error: /Stage[main]/Profile::Postgresql_gis/Postgresql::Server::Grant[revoke ALL on schema public to public:]/Postgresql_psql[grant:revoke ALL on schema public to public:]: Could not evaluate: Error evaluating 'unless' clause, returned pid 7360 exit 1: 'ERROR:  role "PUBLIC" does not exist

because PUBLIC is a "implicit" role and is not listed in pg_roles table.
To fix this I modify query that test if role exists.

Puppet call the SQL function : has_database_privilege() to check privileges. This function works with 'PUBLIC' role but you need to specify 'PUBLIC' as lowercase : 'public'. See https://www.postgresql.org/docs/9.1/static/functions-info.html

@david22swan Can you take a look at this PR ? Is there anything I can do to help merging this PR ?

@Vampouille Sorry for your PR being left to lie for so long.
In order to get this merged we would require test coverage to be added to ensure that your change works correctly and allow us to catch any error's that occur to it in the future, and for the documentation to be updated to include a reference to it.

@david22swan Can you reconsider this PR ? I don't known how to write units tests and even don't known what to test is this case. Do you have some link to other PR that add tests for the database_grant resource ?
Additionally, this PR does not add a new feature, it just fix a bug in this puppet module:
With postgres this works :
REVOKE CONNECT ON DATABASE application_db FROM PUBLIC;

With this puppet module this does not work:
postgresql::server::database_grant { 'revoke connect on application_db':
  ensure    => 'absent',
  db        => 'application_db',
  privilege => 'CONNECT',
  role      => 'PUBLIC',
}

repo: puppetlabs/puppetlabs-postgresql


text:
(SERVER-2265) Refactor utils, configs, actions into separate dirs

This commit separates the utils, configs, and actions into their own directories.
It also renames files to be more consistant.

@justinstoller is this still broken? When you look at individual commits, you can see that each file was renamed, and then changed. But when we look at the total changes, it's showing deleted files again... Will we still retain the git history?

It looks great, though we should have (SERVER-2265) at the start of each commit

@Magisus @justinstoller fixed rebase issue, and tested locally. Looks like it works!

authors: justinstoller lazartea

repo: puppetlabs/puppetserver-ca-cli


text:
error "Package `eieio-1.4' is unavailable"

Installed ntemacs24-bin-20120825 on Windows 8.1
Cloned and copied master to c:/Users/Mark/AppData/Roaming/.emacs.d/
Result of launching with --debug-init
Debugger entered--Lisp error: (error "Package `eieio-1.4' is unavailable")
  signal(error ("Package `eieio-1.4' is unavailable"))
  error("Package `%s-%s' is unavailable" "eieio" "1.4")
  package-compute-transaction((gist) ((eieio (1 4)) (gh (0 8 1)) (tabulated-list (0))))
  package-install(gist)
  (if (package-installed-p package) nil (package-install package))
  (unless (package-installed-p package) (package-install package))
  prelude-require-package(gist)
  mapc(prelude-require-package (ace-jump-mode ace-jump-buffer ace-window anzu browse-kill-ring dash discover-my-major diff-hl diminish easy-kill epl expand-region flycheck gist git-timemachine gitconfig-mode gitignore-mode god-mode grizzl guru-mode ov projectile magit move-text operate-on-number smartparens smartrep undo-tree volatile-highlights zenburn-theme zop-to-char))
  prelude-require-packages((ace-jump-mode ace-jump-buffer ace-window anzu browse-kill-ring dash discover-my-major diff-hl diminish easy-kill epl expand-region flycheck gist git-timemachine gitconfig-mode gitignore-mode god-mode grizzl guru-mode ov projectile magit move-text operate-on-number smartparens smartrep undo-tree volatile-highlights zenburn-theme zop-to-char))
  (if (prelude-packages-installed-p) nil (message "%s" "Emacs Prelude is now refreshing its package database...") (package-refresh-contents) (message "%s" " done.") (prelude-require-packages prelude-packages))
  (unless (prelude-packages-installed-p) (message "%s" "Emacs Prelude is now refreshing its package database...") (package-refresh-contents) (message "%s" " done.") (prelude-require-packages prelude-packages))
  prelude-install-packages()
  eval-buffer(#<buffer  *load*-1098> nil "c:/Users/Mark/AppData/Roaming/.emacs.d/core/prelude-packages.el" nil t)  ; Reading at buffer position 3110
  load-with-code-conversion("c:/Users/Mark/AppData/Roaming/.emacs.d/core/prelude-packages.el" "c:/Users/Mark/AppData/Roaming/.emacs.d/core/prelude-packages.el" nil t)
  require(prelude-packages)
  eval-buffer(#<buffer  *load*> nil "c:/Users/Mark/AppData/Roaming/.emacs.d/init.el" nil t)  ; Reading at buffer position 4156
  load-with-code-conversion("c:/Users/Mark/AppData/Roaming/.emacs.d/init.el" "c:/Users/Mark/AppData/Roaming/.emacs.d/init.el" t t)
  load("c:/Users/Mark/AppData/Roaming/.emacs.d/init" t t)

Pending upstream fix in https://github.com/defunkt/gist.el/pull/67.

But in any case, you don't appear to be using my config: the backtrace you pasted says "prelude"...

(My config uses yagist instead, which is probably why none of my users had reported this problem earlier...)


text:
Add some new types to the API to help clarify

People have a really hard time figuring this out, and I understand why, it didn't click for me either for a while, mainly due to the way \recv -> works. I talked a bit to Christoph about it, and in that conversation it occurred to me we can probably help a great deal just by throwing some types in the mix to hide some of the "low-level complexity" that the existing types expose.
Also nobody knows what the Left and Right mean for the emit, even though it does say in the comment, so similar to the Step we have in MonadRec I guess an either-iso is fine to use here to help too.
This is just an idea, I'm not precious about the names I've used here or anything, just thought I'd throw it out there to see what the reception would be. If we want to do this I'll add comments to the new stuff too.
/cc @paf31 @natefaubion @kRITZCREEK

👍 I'd probably rename Emission to Step or something, with the constructor renamed to Emit, but I like it!

That alternative type & constructor name sound good to me. 👍 Although it will mean that it ends up as emit emitter $ Emit a :smile:
May as well hold off on actually merging this until we IOify etc. since that'll help with the point about rows anyway. Unless anyone really wants to get their hands on it now!

How about Finish instead of Result? I think another important bit will be to just have an example of a setInterval emitter that fires every second or so.

How about these names and things? I figured providing emit/close functions for Emitter rather than having emit emitter $ Emit "foo" / emit emitter $ Finish "foo" kinda thing seemed nice.

authors: garyb kRITZCREEK paf31

license: MIT


text:
🛑 Jurnal is down

In cca66b4, Jurnal (https://jurnal.unmer.ac.id) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Jurnal is back up in 2c4392f after 16 minutes.

repo: pusimgit/upptime


text:
Pulsa Grastis

git clone https://github.com/termux/termux-packages

Add a meaningful entry to the changelog in the README.md.
You can add yourself to the list of contributors in the package.json if you
like. If you don't do it, you won't be mentioned.

Thanks!

git clone https://github.com/termux/termux-packages

repo: pvorb/clone


text:
Need to migrate autoconfiguration files to springboot 3.x.x style

Hi,
For now, this repo using META-INF/spring-factories as autoconfiguration files, but after spring-boot 3.0.0, we need to migrate it to META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports
See ref here: https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-3.0-Migration-Guide#auto-configuration-files
I got no permission to create branch nor raise PR, so I attached a patch here:
Subject: [PATCH] Fix autoconfiguration for spring-boot 3.x.x
---
Index: pom.xml
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/pom.xml b/pom.xml
--- a/pom.xml	(revision 2515c472c7f0872a93f28da7e16a53adc3c635e2)
+++ b/pom.xml	(revision bd73fa8fe2e9b4b5e7d64ebfad9e4a114e38ca65)
@@ -4,7 +4,7 @@
   <modelVersion>4.0.0</modelVersion>

   <artifactId>kjson-spring3</artifactId>
-  <version>7.1.2</version>
+  <version>7.1.3</version>
   <name>Spring Boot 3 JSON message converter for kjson</name>
   <description>Message converter which simplifies use of kjson by Spring Boot 3 applications</description>
   <packaging>jar</packaging>
Index: src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports
new file mode 100644
--- /dev/null	(revision bd73fa8fe2e9b4b5e7d64ebfad9e4a114e38ca65)
+++ b/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports	(revision bd73fa8fe2e9b4b5e7d64ebfad9e4a114e38ca65)
@@ -0,0 +1,1 @@
+io.kjson.spring.JSONSpringAutoConfiguration
\ No newline at end of file

Hope that would help

Thank you very much for that Yixiong – your help with Spring configuration is very much appreciated.
I have uploaded version 7.1.3 incorporating your patch; it may take a few minutes to appear in Maven Central.
Thanks again,
Peter


text:
Error occurred while remounting root filesystem

Describe the bug
Always get stuck in 19/31 and cannot restore or jailbreak. I installed the tvOS13 file to disable OTA, deleted all the users tweaks in cydia and restore in Unc0ver, I replaced the battery at Apple this morning then when I try to jailbreak, the issue happened.
To Reproduce
Steps to reproduce the behavior:

Go to '....'
Click on '....'
Scroll down to '....'
See error

Screenshots

Device (please complete the following information):

iOS Version: 13.5
iOS Device: Xs
unc0ver Version: 5.2

Place an "x" between the brackets if true:

[ ] this is a bug others will be able to reproduce
[x] this issue is present with all tweaks uninstalled(except for default packages) or disabled
[x] this issue is present after a rootfs restore
[x] this issue is present on the latest version of unc0ver

Logs
[] unc0ver is not for sale
[] If you purchased unc0ver, please report the seller
[] Get unc0ver for free at https://unc0ver.dev
[] Configured to share anonymous OS crash logs
[] Machine Name: iPhone11,2
[] Model Name: D321AP
[] Kernel Version: Darwin Kernel Version 19.5.0: Tue Apr 28 22:24:50 PDT 2020; root:xnu-6153.122.1~1/RELEASE_ARM64_T8020
[] Kernel Page Size: 0x4000
[] System Version: iOS 13.5 (Stable) (Build: 17F75)
[] Stage (1): Ensuring resources
[+] Ensured resources
[] Stage (2): Exploiting kernel
[haxx] We in quarantine but still out here!
[+] Exploited kernel
[] Stage (3): Initializing
[] Stage (4): Finding kernel slide
[+] Found kernel slide: 0x1678c000
[] Stage (5): Finding kernel offsets
[+] Found kernel offsets
[] Stage (6): Finding data structures
[+] Found data structures
[] Stage (7): Obtaining root privileges
[+] Obtained root privileges
[] Stage (8): Disabling sandbox
[+] Disabled sandbox
[] Stage (9): Updating host port
[+] Updated host port
[] Stage (10): Enabling dynamic codesign
[+] Enabled dynamic codesigning
[] Stage (12): Saving kernel task port
[+] Saved kernel task port
[] Stage (13): Saving kernel task info
[+] Saved kernel task info
[] Stage (14): Disabling codesigning
[+] Disabled codesigning
[] Stage (15): Obtaining entitlements
[+] Obtained entitlements
[] Stage (16): Connecting AppleMobileFileIntegrity
[+] Connected to AppleMobileFileIntegrity
[] Stage (17): Purging software updates
: Performing command: PurgeSuspendedUpdate
: Performing se command: PurgeSuspendedUpdate
[+] Purged software updates
[] Stage (18): Setting boot-nonce generator
[+] Set boot-nonce generator
[*] Stage (19): Remounting root filesystem
fs_snapshot_list: Invalid argument

Go back into settings, general, iphone storage and delete the OTA update for 3.5.1

I knew that should be work, but there’s no OTA update file in storage and my phone is unable to check the OTA now, The OTA seemed has been disabled by Unc0ver. At the beginning I thought I could re-download the OTA then delete it to fix the issue...
Eleot77

在 2020年6月22日，上午4:42，Hamlock-maneuverr notifications@github.com 写道：

Go back into settings, general, iphone storage and delete the OTA update for 3.5.1
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Ok try this. Run unc0ver with "disable updates" off. restart the phone and then click the "software update" in Settings, general. you may need to wait for it to download by itself.  when it does, delete the update in the "iphone storage" section in general. Then rerun unc0ver with "disable updates" turned back on.  This should fix the problem.

Same problem. Can’t run uncover with disable updates off. It just causes the same error to be thrown in step 19. Is this the real fix? Do we need an update that allows just toggling the software update disable/enable feature that doesn’t attempt to run the standard 31 step process?

what version of unc0ver are you trying tom use?

5.2.1
[*] unc0ver is not for sale [*] If you purchased unc0ver, please report the seller [*] Get unc0ver for free at https://unc0ver.dev [*] Configured to share anonymous OS crash logs [*] Machine Name: iPhone11,2 [*] Model Name: D321AP [*] Kernel Version: Darwin Kernel Version 19.5.0: Tue Apr 28 22:24:50 PDT 2020; root:xnu-6153.122.1~1/RELEASE_ARM64_T8020 [+] Kernel Page Size: 0x4000 [*] System Version: iOS 13.5 (Stable) (Build: 17F75) [*] Stage (1): Ensuring resources [+] Ensured resources [*] Stage (2): Exploiting kernel [haxx] We in quarantine but still out here! [+] Exploited kernel [*] Stage (3): Initializing [*] Stage (4): Finding kernel slide [+] Found kernel slide: 0x6210000 [*] Stage (5): Finding kernel offsets [+] Found kernel offsets [*] Stage (6): Finding data structures [+] Found data structures [*] Stage (7): Obtaining root privileges [+] Obtained root privileges [*] Stage (8): Disabling sandbox [+] Disabled sandbox [*] Stage (9): Updating host port [+] Updated host port [*] Stage (10): Enabling dynamic codesign [+] Enabled dynamic codesigning [*] Stage (12): Saving kernel task port [+] Saved kernel task port [*] Stage (13): Saving kernel task info [+] Saved kernel task info [*] Stage (14): Disabling codesigning [+] Disabled codesigning [*] Stage (15): Obtaining entitlements [+] Obtained entitlements [*] Stage (16): Connecting AppleMobileFileIntegrity [+] Connected to AppleMobileFileIntegrity [*] Stage (17): Purging software updates : Performing command: PurgeSuspendedUpdate : Performing se command: PurgeSuspendedUpdate [+] Purged software updates [*] Stage (18): Setting boot-nonce generator [+] Set boot-nonce generator [*] Stage (19): Remounting root filesystem fs_snapshot_list: Invalid argument

you have an unfinished IOS update download. You need to allow system updates in unc0ver and let it download.  then delete it.  this is the problem you are having.

Performing the reset all settings should allow the IOS update to actually show up in the iphone storage section. But this will take time to download and will almost never let you know its downloading. It's basically a waiting game. Just make sure you are on wifi so it will download. NOW I want to be clear here.  You are NOT letting it actually update your phone.  You are just letting it download. Once the update appears and you delete it.  Start unc0ver and disable auto updates in unc0ver. The jailbreak should work fully now.

I installed a tweak in Cydia to disable ota updates as well. I wonder if that could still be affecting it. On WiFi all night should have downloaded and finished if it was going to work. Whenever I try to manually search for updates it says an error has occurred

yes its still affecting it.  try to uninstall the tweak

have you tried an older version of unc0ver?
4.3.1?

yes its still affecting it. try to uninstall the tweak

I can’t uninstall it (or don’t know how). Cydia is removed and can’t ssh in either.

you try resetting?

Reset/erase all content and settings. Then restored from iCloud backup.

ok don't do the icloud restore yet.  jailbreak first.

ok don't do the icloud restore yet. jailbreak first.

Didn't work :(

have you tried the new unc0ver?

Hello guys im joining in too, i have the same exactly error, nothing to do with the ota update i guess? the error still persist even on wiped iphone (setting > restore data and setting) and even thought different itunes backup, the problem appeared when i tryed to switch to odyssey to give it a try and everything was good when i installed some tweak and it frooze, i decided to go back to u0 so i rootfs and after when i clicked jailbreak it goes to reboot userspace but it crash and it boot in non jb state

i have same issue on 5.3.1 with iP6 iOS 12.4.7. But it shows "jailbreak process completed without any errors"

I have the same issue after  I replaced the battery at Apple with my iPhone XS MAX on 13.5 . I tried everything I could do with it, but it showed "Error occurred while remounting root filesystem" everytime. I tried to use every version of  Unc0ver and  Odyssey but none of them was worked. Is it the bug of Unc0ver tool ?  Can it be fixed ?

Well, about 4 months later I already updated my phone to 13.7 waiting for the jailbreak. Dude, just update to 14.1 as fast as you can, it can’t be fixed.
发自我的iPhone

在 2020年10月13日，上午11:37，heyforresthzl notifications@github.com 写道：

I have the same issue after I replaced the battery at Apple with my iPhone XS MAX on 13.5 . I tried everything I could do with it, but it showed "Error occurred while remounting root filesystem" everytime. I tried to use every version of Unc0ver and Odyssey but none of them was worked. Is it the bug of Unc0ver tool ? Can it be fixed ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hi
I have the same issue after I also replaced today the battery at AppleStore with my iPhone XS on 13.5
Unc0ver versions tried are 5.2.1 & 5.3.1

Hi
I have the same issue after I also replaced today the battery at AppleStore with my iPhone XS on 13.5
Unc0ver versions tried are 5.2.1 & 5.3.1

救救我的11pm，13.5，我是中国用户也这个问题

救救我的11pm，13.5，我是中国用户也这个问题

没救了兄弟，这个问题无解
发自我的iPhone

在 2021年1月12日，17:04，unc0ver46 notifications@github.com 写道：

救救我的11pm，13.5，我是中国用户也这个问题
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

没救了兄弟，这个问题无解
发自我的iPhone

在 2021年1月12日，17:04，unc0ver46 notifications@github.com 写道：

救救我的11pm，13.5，我是中国用户也这个问题
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Help me, I am a Chinese user and many people have encountered this problem

Help me, I am a Chinese user and many people have encountered this problem

没救了兄弟，这个问题无解发自我的iPhone
…
在2021年1月12日，17：04，unc0ver46 @.***>主题：救救我的11 pm，13.5，我是中国用户也这个问题—您编写了线程。直接回复此电子邮件，在GitHub上查看或取消订阅。

你是中国大神吗？

没救了兄弟，这个问题无解发自我的iPhone
…
在2021年1月12日，17：04，unc0ver46 @.***>主题：救救我的11 pm，13.5，我是中国用户也这个问题—您编写了线程。直接回复此电子邮件，在GitHub上查看或取消订阅。

你是中国大神吗？

Help me, I am a Chinese user, I do not understand English, many people have encountered this problem, help us, I love you

Help me, I am a Chinese user, I do not understand English, many people have encountered this problem, help us, I love you

Help me, I am a Chinese user, I do not understand English, many people have encountered this problem, help us, I love you

Help me, I am a Chinese user, I do not understand English, many people have encountered this problem, help us, I love you

Can't your English be so good?

Can't your English be so good?

@unc0ver-bot

@unc0ver-bot

升级到iOS14等待越狱，这个问题从去年五月份就出现了，无解
发自我的iPhone

在 2021年1月12日，17:14，unc0ver46 notifications@github.com 写道：

@unc0ver-bot
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

升级到iOS14等待越狱，这个问题从去年五月份就出现了，无解
发自我的iPhone

在 2021年1月12日，17:14，unc0ver46 notifications@github.com 写道：

@unc0ver-bot
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

升级到iOS14等待越狱，这个问题从去年五月份就出现了，无解发自我的iPhone
…
在2021年1月12日，17：14，unc0ver46 @.***>写道：  @ unc0ver博特 -您收到这是因为你撰写的线程。直接回复此电子邮件，在GitHub上查看或取消订阅。

等geofilza怎么样？

升级到iOS14等待越狱，这个问题从去年五月份就出现了，无解发自我的iPhone
…
在2021年1月12日，17：14，unc0ver46 @.***>写道：  @ unc0ver博特 -您收到这是因为你撰写的线程。直接回复此电子邮件，在GitHub上查看或取消订阅。

等geofilza怎么样？

Also go to the official to change the battery, I want to know if this will affect my phone jailbreak, model 8p

先在Unc0ver中Restore至未越狱状态，进入设置抹掉手机后再更换电池，否则会导致无法再进入越狱状态

在 2021年2月26日，19:13，xu378947986 notifications@github.com 写道：

Also go to the official to change the battery, I want to know if this will affect my phone jailbreak, model 8p
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

did anyone had success with the 6.0.0 version? ios13.5 IP 11 PRO


text:
fix(pickers/telescope): handle nil selection in open

Describe what this PR does / why we need it
It may happen that selection is nil, which results in an error being logged.
Does this pull request fix one issue?
Describe how you did it
This silently skips the failing function in such cases.
Describe how to verify it
Run Octo pr search and fiddle around with filtering and navigation
Special notes for reviews
Checklist

[x] Passing tests and linting standards
[x] Documentation updates in README.md and doc/octo.txt

Thanks!

authors: legobeat pwntester


text:
v0.3 - CI - Add Next.js Build Cache Persistence

Add build caching according to docs below
https://nextjs.org/docs/advanced-features/ci-build-caching

uses: actions/cache@v2
with:
  # See here for caching with `yarn` https://github.com/actions/cache/blob/main/examples.md#node---yarn or you can leverage caching with actions/setup-node https://github.com/actions/setup-node
  path: |
    ~/.npm
    ${{ github.workspace }}/.next/cache
  # Generate a new cache whenever packages or source files change.
  key: ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}-${{ hashFiles('**.[jt]s', '**.[jt]sx') }}
  # If source files changed but packages didn't, rebuild from a prior cache.
  restore-keys: |
    ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}-

Build cache was added and the message no longer prints during builds

repo: pwrdrvr/microapps-core


text:
FR (Versioning): Improve NuGet Version Range Support

Should fix the following comment from #49:- https://github.com/pwsh-cs-tools/core/issues/49#issuecomment-1898185715

+1, sharing the temp path causes errors if two packages contain the same file eg large depencency trees like Azure.Monitor.OpenTelemetry.Exporter

Part of the issue is that Azure.Monitor.OpenTelemetry.Explorer is likely using an older version, and that the Bootstrapper is detecting an unbounded version range (which is currently not supported).
However, a bit of work has been already done to improve Import-Package's versioning control when the -CachePath parameter was introduced, so logic to compare semantic versions may already be available.

Improving the NuGet Version Range handling should also help with 2 other problems:

What to do when 2 dependencies in the dependency tree require 2 different ranges of the same package
Should help improve the $loaded logic.

To make this work this would require a change to all of the versioning and version control logic.
Thankfully, we could likely get away with using $bootstrapper.GetAllVersions()

I think a starting point would be to add a field to PackageData that caches a call to GetAllVersions().
In theory, this would do 2 things:

Would optimize and limit calls to the NuGet API
Could possibly allow range and version selection to be moved outside of Build-PackageData

This could also allow for a lot more advanced version control logic, since PackageData is currently limited to GetStable() and GetPrerelease()

This would likely warrant recursion when selecting dependencies for the best range of a package.
To optimize looping, I think the best option is to only recurse when the upper bounds of a range has changed.

The upper and lower bounds should include 3 versions:

the boundary itself
the stable version in range closest to the boundary
the prerelease version in range closest to the boundary

calculating this for the lower boundary will be interesting, because typically the stable version would be closest to the lower bound
an interesting edge case would be how to handle boundaries that are prereleases.

Another problem that would warrant a complete refactor is that currently due to Import-Package's simplistic version control, Import-Package is able to load dependencies at the same time that it builds the dependency tree.
While I might be able to keep this design in mind:

To optimize looping, I think the best option is to only recurse when the upper bounds of a range has changed.

I think it might be best to separate loading and tree parsing into 2 separate steps. This would slow Import-Package down, but would allow me to add more features down the road.

An optimization that could help with dependency loading is using the v3-flatcontainer API to download just the nuspec, instead of the entire .nupkg when parsing the dependency tree.

https://api.nuget.org/v3-flatcontainer/{package-id}/{package-version}/{package-id}.nuspec

An optimization that could help with dependency loading is using the v3-flatcontainer API to download just the nuspec, instead of the entire .nupkg when parsing the dependency tree.

https://api.nuget.org/v3-flatcontainer/{package-id}/{package-version}/{package-id}.nuspec

Example implementation:
# NuGet Package ID and Version
$packageId = "YOUR_PACKAGE_ID"
$packageVersion = "YOUR_PACKAGE_VERSION"

# NuGet API URL
$url = "https://api.nuget.org/v3-flatcontainer/$packageId/$packageVersion/$packageId.nuspec"

# Fetch and parse the .nuspec XML data
try {
    $response = Invoke-WebRequest -Uri $url
    $nuspecXml = [xml]$response.Content  # Parses the content as XML

    # The XML data is now stored in the variable $nuspecXml
    Write-Host "Successfully fetched and parsed .nuspec data for package $packageId version $packageVersion"
    # Display the XML content
    $nuspecXml.OuterXml
} catch {
    Write-Host "Error fetching or parsing .nuspec data: $_"
}

# You can now use the variable $nuspecXml to work with the XML data

Adding such support would also likely warrant refactoring any filesystem modifications done by Resolve-CachedPackages, as this would cause a performance hit any time the ranges are updated.

EDIT: PartialZip is available in C# and could be transpiled to PowerShell (or installed during the bootstrapper process)

I'm leaning on transpiling as partialzip libraries tend to not be well maintained.
https://github.com/Jan-Kruse/PartialZip

It doesn't look like NuGet supports byte ranges. It also looks like they don't have an API for exposing the content of the .nupkgs:

see: https://github.com/NuGet/NuGetGallery/issues/7751

Feature implementation started. See: https://github.com/pwsh-cs-tools/Import-Package/commit/28550db1e6588af8eab31f13e43784f2aed8fde9

repo: pwsh-cs-tools/core


text:
numpy version

In KCI.py
num_eig = np.int(np.floor(T / 2))
raised Attribute Error
AttributeError: module 'numpy' has no attribute 'int'. np.intwas a deprecated alias for the builtinint. To avoid this error in existing code, use intby itself. Doing this will not modify any behavior and is safe. When replacingnp.int, you may wish to use e.g. np.int64ornp.int32to specify the precision. If you wish to review your current use, check the release note link for additional information. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:

Hi, thanks for reporting. It should have been fixed in PR #156. Could you please try using the latest version and see if that issue remains?

authors: kunwuz quanganhct

repo: py-why/causal-learn


text:
Tailings ponds connections are weird

Mod source
PyAE Beta
Which mod are you having an issue with?

[ ] pyalienlife
[ ] pyalternativeenergy
[X] pycoalprocessing
[ ] pyfusionenergy
[ ] pyhightech
[ ] pyindustry
[ ] pypetroleumhandling
[ ] pypostprocessing
[ ] pyrawores

Operating system

=Windows 10

What kind of issue is this?

[X] Compatibility
[ ] Locale (names, descriptions, unknown keys)
[ ] Graphical
[ ] Crash
[ ] Progression
[ ] Balance
[ ] Pypostprocessing failure
[ ] Other

What is the problem?

The tailings pond only has connections on the bottom and right. There are no connections on the top or left, no matter how you rotate it.
Additionally, when you rotate the pond twice, it breaks the existing connection

Steps to reproduce

Place tailings pond. Rotate. There are no connections on left or top
Place a tailings pond, connect a pipe to it. Rotate the pond twice and you'll see the pipe no longer connects with the pond. This happens if you connect at the right or bottom and with underground and overground pipes

Additional context
Working pipe

Broken pipe

Log file
No response

Fixed. It's no longer rotatable and has pipe connections on all sides.

authors: notnotmelon wesdotcool

repo: pyanodon/pybugreports

license: Unlicense


text:
Check if the solution exists before creating a GIF (Simulation.create_gif and BatchStudy.create_gif)

I liked how #3723 checks if the simulations have been solved before proceeding ahead to the plotting functionality. It would be nice to have similar checks in -
https://github.com/pybamm-team/PyBaMM/blob/c3def31a04df163769851733657d9b49b3700bca/pybamm/batch_study.py#L184-L187
and
https://github.com/pybamm-team/PyBaMM/blob/c3def31a04df163769851733657d9b49b3700bca/pybamm/simulation.py#L1047-L1050

Would like to take this on.

Hi @prady0t are you still working on this issue?

yes

@Saransh-cpp BatchStudy class doesn't have an attribute solution unlike Simulation. How to check for empty solutions there?

You can check if the self.sims list is empty and throw an error if it is -
https://github.com/pybamm-team/PyBaMM/blob/c3def31a04df163769851733657d9b49b3700bca/pybamm/batch_study.py#L116
The simulations are being initialised and solved in the solve() method, so this would be a good indication of if the user has called the solve() method before calling the create_gif() method.

why is
https://github.com/pybamm-team/PyBaMM/blob/063a3830afd5547f7a78e283ae81058465cbdd3e/pybamm/batch_study.py#L116
declared outside __init__ should'nt it be inside it?

authors: R-Yash Saransh-cpp prady0t

repo: pybamm-team/PyBaMM


text:
Only check the changed lines

Sometimes, some changes sneak into master that don't pass the checks.
If afterwards there a pull request is made in a completely different section of the code - the same checks will fail! with no fault of the contributor.
A possible fix for this is to only run checks on the lines that are changed by the PR.

This should be what is currently being done - beefore filters it's responses to lines that match the range of the diff of the patch being examined.
Can you provide an example where this has happened?

repo: pybee/beefore


text:
Allow use of Setuptools launchers

Currently the application startup script is hard coded in the per-platform template.
This is quite restrictive in that it only works if you have a module named 'app' with a function 'main' which returns an object with a function main_loop (on windows at least, other platforms can be different).
Also, on windows this means that in task manager you see pythonw.exe rather than the name of the distributed application.
To make briefcase more flexible I propose using the built in mechanism for defining entry points / applications.
On windows this already creates exe launchers that have the desired application names.
This also allows having multiple gui and console applications in the one installer package.
In this PR the windows launcher creation has been modified slightly to create relocatable exe applications in the contents/app folder.
Start menu shortcuts will be created for each of these rather than the static launcher currently defined in the template.
These executables will show  in task manager with the user specified names.
On linux, executable launch scripts will be created for each entrypoint defined.
On MacOS, if an entrypoint is defined with the same name as the formal_name specified it will be the main application. There are launch scripts for any others also defined in the bundle/Contents/MacOS the an end user could run from the command line etc.
Mobile platforms have not been adapted to this mechanism as there's no way to produce multiple executables.
If desired however they could also be updated in the same way to allow defining the custom entry point which would be used in preference to the templated method.
If no entry_points are defined it falls back to the launching mechanism as defined in the platform templates.

Hi, could I get some feedback on this feature?
I feel it should be ready to go and won't impact existing users who don't change their setup.py to use the feature.

repo: pybee/briefcase


text:
Troubleshooting Windows Crash Tut1

Provide some information related to wrapped handler and windows loop as expected current behavior.

Executing a wrapped handler in windows causes the app to crash.

It provides some additional details in the troubleshooting section.

PR Checklist:

[x ] All new features have been tested
[x ] All new features have been documented
[x ] I have read the CONTRIBUTING.md file
[ x] I will abide by the code of conduct

Updated with changes.

repo: pybee/toga


text:
Adds bayesian sampling

This PR adds Monte Carlo sampling methods; completed by extending Pyro, and integrating the pybop.problem class.
To Do:

[ ] Update sigma definition within BayesianSampler.define_sampling_model class
[ ] Integrate parameter_prior variable with pybop.parameter
[ ] Add support for adding pyro dependencies after pip installation (similar to plotly)
[ ] Add tests and notebook examples
[ ] Update docs

Closes #6.

out of interest, why pyro? numpyro is faster and uses jax, so integrates well with pybamm. and PINTS has a lot of MCMC samplers (and slice samplers, SMC samplers....)
Nothing wrong with pyro of course!

Hi @martinjrobins, great question - it comes down to the alignment between a Jax and a non-Jax implementation. From the Jax side, I agree that numpyro should be next on the list to be added, but it requires the completion of #3395, which I'm hoping to complete over the holidays.
At the top level, I think we should provide a Jax and non-Jax version of MCMC, as there are functional constraints on the Jax solver (e.g. I don't think it supports the experiment class, correct?). In this sense, using pyro and numpyro should allow us to reuse the design or at least minimise the structural changes. While using Pints' MCMC implementation would reduce the number of dependencies the project has, I think it would require two different approaches to MCMC between the JAX and non-JAX classes. I'm keen to hear your thoughts on this, while I have this initial Pyro example, I'm not too far into the implementation to pivot if we agree on a better design.

I'm not opposed to providing access to the pyro mcmc samplers, my main concern is that its not always possible to get gradients, or accurate gradients, and as far as I can see the only samplers in pyro are gradient-based ones (hmc and nuts). using pints would give you access to many more sampling methods, but its probably less optimised than pyro.
Ideally, you would have three sampling backends, pyro, numpyro and pints.

authors: BradyPlanden martinjrobins

repo: pybop-team/PyBOP


text:
Travis builds failing with E402 module level import not at top of file

Travis builds have been failing due to an import in:
https://github.com/pydanny/cookiecutter-django/blob/master/{{cookiecutter.repo_name}}/{{cookiecutter.repo_name}}/wsgi.py#L28
What's the best way to deal with this? Ignore it or move the import to the top of the line?
All Pull Requests are failing due to this.

Fixed in 5fefde8a66af02ae0948e5fe668831b90db905a4

authors: kevgathuku luzfcb

repo: pydanny/cookiecutter-django


text:
Automate the makefile

This blog post document explains what I want to do. https://daniel.feldroy.com/posts/autodocumenting-makefiles

Hi, I am new here I came here after seeing the "good first issue" lable, I kind of get what listo means . I would like to do the changes But I didnt understand what makefile is can you help me with that

Please read the blog article to see what I want added to the

I have done the PR  please review

Completed by @dysai1234. Thanks for all your hard work!

repo: pydanny/listo


text:
Parse big ints using PyLong_FromString instead of BigInt

Seems to be a little bit slower, I thought this might be faster as it's what ujson does.
@davidhewitt any thoughts? Most of the time seems to be in PyLong_FromString, flamegraph here.

closing as this wasn't faster for strings, we'll use #65 instead.

repo: pydantic/jiter


text:
Pydantic Settings env_prefix is applying to secrets filenames

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
The documentation for Pydantic Settings states the env_prefix applies to environment variables.  However, it is also applying to secret filenames.  See https://docs.pydantic.dev/latest/concepts/pydantic_settings/#validation-of-default-values
I think it makes more sense to have env_prefix only applies to environment vars, as is reflected in the documentation, so I would think this is a code bug, but if the behavior is intended, the documentation should reflect that.
Example Code
>>> from pydantic_settings import BaseSettings, SettingsConfigDict
>>> class Settings(BaseSettings):
...     model_config = SettingsConfigDict(secrets_dir="/run/secrets")
...     secret_key: str
>>> a = Settings()
>>> a.secret_key
'mysupersecretkeyshhhh'

>>> class Settings(BaseSettings):
...     model_config = SettingsConfigDict(secrets_dir="/run/secrets", env_prefix="PB_")
...     secret_key: str
...
>>> a = Settings()
Traceback (most recent call last):
  File "<python-input-8>", line 1, in <module>
    a = Settings()
  File "/home/workspace/.venv/lib/python3.13/site-packages/pydantic_settings/main.py", line 171, in __init__
    super().__init__(
    ~~~~~~~~~~~~~~~~^
        **__pydantic_self__._settings_build_values(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<25 lines>...
        )
        ^
    )
    ^
  File "/home/workspace/.venv/lib/python3.13/site-packages/pydantic/main.py", line 212, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Settings
secret_key
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.9/v/missing

Python, Pydantic & OS Version
Python 3.13.1, Debian 12 (Docker Container), Pydantic 2.23.4, Pydantic Settings 2.7

 python -c "import pydantic.version; print(pydantic.version.version_info())"
             pydantic version: 2.9.2
        pydantic-core version: 2.23.4
          pydantic-core build: profile=release pgo=false
                 install path: /home/workspace/.venv/lib/python3.13/site-packages/pydantic
               python version: 3.13.1 (main, Dec  4 2024, 20:40:24) [GCC 12.2.0]
                     platform: Linux-6.1.0-22-amd64-x86_64-with-glibc2.36
             related packages: typing_extensions-4.12.2 pydantic-settings-2.7.0
                       commit: unknown

This is an intended behavior. nev_prefix will be applied to all settings sources. So, this is not a bug.
I agree that the env_prefix name is not a good name for a config used by all the setting sources. file_prefix or prefix are probably better names. but this is something that we can't change in this version because it will introduce a breaking change.

Cool, for now then can I suggest updating the https://docs.pydantic.dev/latest/concepts/pydantic_settings/#environment-variable-names section in the docs to something along the lines of "Setting Name Prefixes" and update the text?
Then, consider if it would be worth it to create a new name and alias the old one to it, then deprecate the old one?
The key for me is that I read the documentation and was surprised by the actual behavior, so if the documentation is updated, I don't think it is hard to work around it.  I just wasn't sure it was intentional or not :)  Oh, and thank you @hramezani for your contributions to Django and Pydantic, I use both of these tools extensively!  Your contributions are very appreciated!

Cool, for now then can I suggest updating the https://docs.pydantic.dev/latest/concepts/pydantic_settings/#environment-variable-names section in the docs to something along the lines of "Setting Name Prefixes" and update the text?

Agree, we need to update the document here.

Then, consider if it would be worth it to create a new name and alias the old one to it, then deprecate the old one?

Would be a good idea. probably I work on it later.

The key for me is that I read the documentation and was surprised by the actual behavior, so if the documentation is updated, I don't think it is hard to work around it. I just wasn't sure it was intentional or not :)
Oh, and thank you @hramezani for your contributions to Django and Pydantic, I use both of these tools extensively! Your contributions are very appreciated!

Thanks for your message. it worth a lot to me ❤

authors: ensignavenger hramezani

repo: pydantic/pydantic-settings


text:
feat: Make SecretStr hashable

Change Summary
Makes SecretStr hashable, cf. https://github.com/pydantic/pydantic/issues/4387
Related issue number
fix #4387
Checklist

[x] Unit tests for the changes exist
[x] Tests pass on CI and coverage remains at 100%
[x] Documentation reflects the changes where applicable
[x] changes/<pull request or issue id>-<github username>.md file added describing change
(see changes/README.md for details)
[x] My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

please review

please update

Do I need another please review here? :)

Please review

Great, thanks so much.

authors: chbndrhnns hramezani samuelcolvin

repo: pydantic/pydantic


text:
ENH: query price presentation (uses pyfiglet fonts)

Hi,
to keep up the momentum I'm sharing my fancy query price print outs.
Feel free to abandon this if you do not like it.
At RTBHOUSE we are using it with pleasure :)
Feature:
Query price is printed with ascii art font. Font gets larger as price increases.
Sample outputs

below $0,1 (plain text)

Processed: 10.3 Gb
Price: 0.05 $
Retrieving results...

between $0,1 - $0,5 (first level of ascii font)

Processed: 31.3 GB
Price:
 ||_    __         __
(||    /  \    /| |_
_||)   \__/.    | __)
 ||

Retrieving results...

between $0.5 - $1

     __,--,_.     ___      ______   _____
    /       |    / _ \    |____  | | ____|
   |   (----`   | | | |       / /  | |__
    \   \       | | | |      / /   |___ \
.----)   |      | |_| |  __ / /     ___) |
|_    __/        \___/  (__)_/     |____/
  '--'

between $1 - $2

Processed: 219.2 Gb
Price:
 .----------------.  .----------------.  .----------------.  .----------------.
| .--------------. || .--------------. || .--------------. || .--------------. |
| |     __       | || |              | || |     ____     | || |   _______    | |
| |    /  |      | || |              | || |   .'    '.   | || |  |  ___  |   | |
| |    `| |      | || |              | || |  |  .--.  |  | || |  |_/  / /    | |
| |     | |      | || |              | || |  | |    | |  | || |      / /     | |
| |    _| |_     | || |      _       | || |  |  `--'  |  | || |     / /      | |
| |   |_____|    | || |     (_)      | || |   '.____.'   | || |    /_/       | |
| |              | || |              | || |              | || |              | |
| '--------------' || '--------------' || '--------------' || '--------------' |
 '----------------'  '----------------'  '----------------'  '----------------'

Retrieving results...

between $2 - $5

Processed: 501.9 Gb
Price:

       .            .--~*teu.                   xeee      cuuu....uK
       E           dF     988Nx                d888R      888888888
    .x+E:..       d888b   `8888>              d8888R      8*888**"
  u8~  E  `b.     ?8888>  98888F             @ 8888R      >  .....
 t8E   E d888>     "**"  x88888~           .P  8888R      Lz"  ^888Nu
 88N.  E'8888~          d8888*`           :F   8888R      F     '8888k
 88888b&.`""`         z8**"`   :         x"    8888R      ..     88888>
 '88888888e.        :?.....  ..F    .   d8eeeee88888eer  @888L   88888
   "*8888888N      <""888888888~  .@8c         8888R    '8888F   8888F
  uu. ^8*8888E     8:  "888888*  '%888"        8888R     %8F"   d888"
 @888L E `"88E     ""    "**"`     ^*       "*%%%%%%**~   ^"===*%"`
'8888~ E   98~
 `*.   E  .*"
   `~==R=~`

Retrieving results...

above $5

       $$$$$
       $:::$
   $$$$$:::$$$$$$              66666666                000000000     77777777777777777777
 $$::::::::::::::$            6::::::6               00:::::::::00   7::::::::::::::::::7
$:::::$$$$$$$::::$           6::::::6              00:::::::::::::00 7::::::::::::::::::7
$::::$       $$$$$          6::::::6              0:::::::000:::::::0777777777777:::::::7
$::::$                     6::::::6               0::::::0   0::::::0           7::::::7
$::::$                    6::::::6                0:::::0     0:::::0          7::::::7
$:::::$$$$$$$$$          6::::::6                 0:::::0     0:::::0         7::::::7
 $$::::::::::::$$       6::::::::66666            0:::::0 000 0:::::0        7::::::7
   $$$$$$$$$:::::$     6::::::::::::::66          0:::::0 000 0:::::0       7::::::7
            $::::$     6::::::66666:::::6         0:::::0     0:::::0      7::::::7
            $::::$     6:::::6     6:::::6        0:::::0     0:::::0     7::::::7
$$$$$       $::::$     6:::::6     6:::::6        0::::::0   0::::::0    7::::::7
$::::$$$$$$$:::::$     6::::::66666::::::6        0:::::::000:::::::0   7::::::7
$::::::::::::::$$       66:::::::::::::66  ......  00:::::::::::::00   7::::::7
 $$$$$$:::$$$$$           66:::::::::66    .::::.    00:::::::::00    7::::::7
      $:::$                 666666666      ......      000000000     77777777
      $$$$$

Codecov Report

Merging #71 into master will decrease coverage by 72.73%.
The diff coverage is 6.66%.

@@            Coverage Diff             @@
##           master     #71       +/-   ##
==========================================
- Coverage   73.37%   0.64%   -72.74%
==========================================
  Files           4       4
  Lines        1540    1554       +14
==========================================
- Hits         1130      10     -1120
- Misses        410    1544     +1134

Impacted Files
Coverage Δ

pandas_gbq/gbq.py
1.66% <6.66%> (-73.48%)
:arrow_down:

pandas_gbq/__init__.py
16.66% <0%> (-83.34%)
:arrow_down:

pandas_gbq/tests/test_gbq.py
0% <0%> (-82.88%)
:arrow_down:

pandas_gbq/_version.py
0% <0%> (-44.41%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd53b5c...66ecb06. Read the comment docs.

now it's clean and easy! please revie

@parthea, good point, thx
now price is well formatted with '$' prefix and USD suffix

Sample output is:
Requesting query... ok.
Job ID: job_XXXxXxXX0xxXxX00xxx00X_XxXx
Query running...
Query done.
Processed: 18.9 GB
Standard price: $0.09 USD

Retrieving results...
Got 1 rows.

Total time taken 4.08 s.
Finished at 2017-07-12 15:17:38.

generally wait for comments on PRs before merging pls

Generally, ok.
But in this specific case, there were two approval comments, and since my further changes does only changed string literal, which pass my local manual checks and Travis builds as well, I've decided to go forward and merge them.
Is there any problem with merged code?

@tworec not a problem.

ok :)

authors: codecov-io jreback tworec

repo: pydata/pandas-gbq


text:
ENH: Accept list and range as levels for groupby in non-MultiIndexed objects

[x] closes #13901
[x] tests added / passed
[x] passes git diff upstream/master | flake8 --diff
[x] whatsnew entry

Current coverage is 85.29% (diff: 100%)

Merging #13907 into master will decrease coverage by <.01%

@@             master     #13907   diff @@
==========================================
  Files           139        139
  Lines         50087      50091     +4
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits          42723      42726     +3
- Misses         7364       7365     +1
  Partials          0          0

Powered by Codecov. Last update 0be0d67...b41bfc1

@sinhrks Made all changes and additions.

@jreback @sinhrks Check out the latest amendment. How does it look to you?

@jreback There is a test that fails, but it has nothing to do with groupby and I cannot reproduce it on my machine. Thoughts?

@jreback

can you rebase / update?

Rebased and all green.

ok lgtm. @jorisvandenbossche

thanks!

authors: agraboso codecov-io jreback

repo: pydata/pandas


text:
Consider how to deal with the proliferation of decoder options on open_dataset

There are already lots of keyword arguments, and users want even more! (#843)
Maybe we should use some sort of object to encapsulate desired options?

We already have the dictionary. Users can make a decode_options dictionary, and then call what they want to with **decode_options.

One advantage to creating classes to encapsulate these options is that it's easier to do error checking on field names and values than a dictionary. Though a dictionary is nice and lightweight.
The simplest thing to do would be to move these options out of open_dataset into a single other argument, e.g., open_dataset(filename, **kwargs) -> open_dataset(filename, decode_options=kwargs).

I would disagree with the form open_dataset(filename, decode_options=kwargs) over open_dataset(filename, **kwargs), because the former breaks normal Python style. It would make the documentation for the arguments somewhat awkward ("decode_options is a dictionary which can have any of the following keys [...]"). It also forces the user to use a dictionary instead of having the option to use a dictionary or the regular style of entering kwargs.
What do you mean when you say it's easier to do error checking on field names and values? The xarray implementation can still use fields instead of a dictionary, with the user saying open_dataset(filename, **kwargs) if they feel like it. I think I'm not understanding something here.

My concern is that open_dataset has 1 required and 12 optional arguments. This is too many to easily understand, and is generally considered poor software design. The standard solution is to group related options into immutable objects, e.g., DecoderOptions or just Decoder (if we want to bundle related methods on the object).
pandas.read_csv is a more extreme example of the same issue.

It is considered poor software design to have 13 arguments in Java and other languages which do not have optional arguments. The same isn't necessarily true of Python, but I haven't seen much discussion or writing on this.
I'd much rather have pandas.read_csv the way it is right now than to have a ReadOptions object that would need to contain exactly the same documentation and be just as hard to understand as read_csv. That object would serve only to separate the documentation of the settings for read_csv from the docstring for read_csv. If you really want to cut down on arguments, open_dataset should be separated into multiple functions. I wouldn't necessarily encourage these, but some possibilities are:

Have a function which takes in an undecoded dataset and returns a CF-decoded dataset, instead of a decode_cf kwarg
Have a function which takes in an unmasked/unscaled dataset and returns a masked/scaled dataset, instead of mask_and_scale
Have a function which takes in a dataset with undecoded times and returns a decoded dataset, instead of decode_times
similarly for decode_coords, chunks, and drop_variables. Should chunks and drop_variables even exist as kwargs, given that the functions to do these to a dataset already exist?

All of that aside, the DecoderOptions object already exists if that's what you want - it's the dict.

I'd much rather have pandas.read_csv the way it is right now than to have a ReadOptions object that would need to contain exactly the same documentation and be just as hard to understand as read_csv.

Certainly I agree here. The alternative would be separating out related functionality into related groups, e.g., NameOptions, ParserOptions, MissingValueOptions, DatetimeOptions, etc., basically exactly the groupings you see in the pandas docs.

I agree that having too many keyword arguments is poor design; it's representative of either failing to abstract anything away or having the object/function just do too much. For a specific example, this jumps out to me as a problem:
        ds = conventions.decode_cf(
            store, mask_and_scale=mask_and_scale, decode_times=decode_times,
            concat_characters=concat_characters, decode_coords=decode_coords,
            drop_variables=drop_variables)

Already open_dataset takes 5 parameters just to pass on directly to another function. This means to add a 6th to decode_cf, you have to update the code and doctstring there, and then make those same changes to open_dataset. Now, you could argue that they're used again within the function within open_dataset
            token = tokenize(file_arg, group, decode_cf, mask_and_scale,
                             decode_times, concat_characters, decode_coords,
                             engine, chunks, drop_variables)

but again you're using all of these parameters together. If all of these variable values are needed to define the state, you already have an implicit object in your code; you're just not using the language syntax to help you by encapsulating it.
I'd be in favor of having lightweight classes (essentially mutable named tuples) vs. dictionaries. The former allows more discoverability to the interface (i.e. tab completion in IPython) as well as better up-front error checking (you could use __slots__ to permit only certain attributes). My experience with assembling dictionaries for options is a world of typo-prone pain; trying to prevent that is especially important when teaching new users. You could still give this class the right hooks (e.g. __iter__, asdict) to allow it to be passed as **kwargs to decode_cf.

See #4490 for a concrete API proposal. I think we can move the discussion there.

authors: dcherian dopplershift mcgibbon shoyer

repo: pydata/xarray


text:
Combine by point coords

This PR was based off of #3926, though it probably doesn't need to be and could be rebased if we wanted to merge this first.

[x] Closes #3774
[x] Tests added
[x] Passes isort -rc . && black . && mypy . && flake8
[x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Closing, see https://github.com/pydata/xarray/issues/3774#issuecomment-866963499


text:
Py3 tutorial

Make tutorial-code py3 compatible.

Fix print-statement.
Drop {}.iterkeys().
STILL left one iteritems() (doc/tutorial/cproject.py)

Coverage remained the same at 99.76% when pulling 7adf594bfe6bba86ee9b1a08248e1823e639fa15 on ankostis:py3_tutorial into 61d2a2eef6b46da65efb0afb8d919dfe4959c87d on pydoit:master.

Take 2: Used future modernize with hand-removing needless changes.
Left from __future import print statement in case the tutorial-code gets test-cased ever.

Coverage remained the same at 99.73% when pulling 103d0af75c188f564c6e4bc6f48fb9b13492680c on ankostis:py3_tutorial into 545e4e67c93496d75440fe9d98db06dd51b60b09 on pydoit:master.

Coverage remained the same at 99.73% when pulling 4feaa8894f0e8872dd04a982c5ffb6df1d406370 on ankostis:py3_tutorial into 545e4e67c93496d75440fe9d98db06dd51b60b09 on pydoit:master.

thanks!

In most cases, you could leave the future import out, provided that you don’t use commas.
>>> #py2
>>> print("hello")
hello
>>> print("hello", "world")
("hello", "world")

>>> #py3
>>> print("hello")
hello
>>> print("hello", "world")
hello world

authors: Kwpolska ankostis coveralls schettino72

repo: pydoit/doit

license: mit


text:
new lexer request: zsh

(Original issue 1128 created by anntzer on 2015-07-17T21:31:11.546453+00:00)
zsh's builtins set is a bit different from bash's, e.g. setopt is (correctfully) not listed as a builtin by BashLexer (which currently also handles zsh files) but a separate ZshLexer would be nice to cover the differences.

(Original issue was assigned to tshatch)

(Original comment by tshatch on 2015-10-17T02:17:13.457416+00:00)
Issue #1107 was marked as a duplicate of this issue.

(Original comment by tshatch on 2015-10-17T02:18:00.901223+00:00)
Do you happen to have a doc link that lists the builtins?

(Original comment by zmwangx on 2015-10-17T02:23:25.171427+00:00)
http://zsh.sourceforge.net/Doc/Release/Shell-Builtin-Commands.html

(Original comment by zmwangx on 2015-10-17T02:25:08.365748+00:00)
Actually there are more, scattered around the docs. For instance, there are a few in http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Zle-Builtins, and a few in http://zsh.sourceforge.net/Doc/Release/Completion-System.html.

(Original comment by zmwangx on 2015-10-17T02:26:42.458876+00:00)
There's also a function index that might help: http://zsh.sourceforge.net/Doc/Release/Functions-Index.html. But some are contribs, not builtins.

repo: pygments/pygments-migration-test


text:
Offline mode - expiration time

com.google.android.libraries.youtube.offline.model.OfflineVideoPolicy
isExpired void & isOfflineStatePlayable should interests us :D

I have some spare time today, i'll check what can be done later ;)

authors: esgie pylerSM

repo: pylerSM/YouTubeBackgroundPlayback


text:
pipenv check fails on 11.10.1.dev1 version

pipenv check fails on 11.10.1.dev1 version

Expected result
No exceptions in output.
Actual result
$ pipenv check
Checking PEP 508 requirements…
Passed!
Checking installed package safety…
An error occurred:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/safety.zip/__main__.py", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/safety.zip/safety/cli.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/safety.zip/safety/util.py", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/safety.zip/dparse/__init__.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/safety.zip/dparse/parser.py", line 5, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/safety.zip/yaml/__init__.py", line 3, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/safety.zip/yaml/yaml3/__init__.py", line 13, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/usr/local/lib/python3.6/site-packages/pipenv/patched/safety.zip/yaml/yaml3/cyaml.py", line 5, in <module>
  File "ext/_yaml.pyx", line 18, in init _yaml (ext/_yaml.c:23185)
AttributeError: module 'yaml' has no attribute 'error'

Steps to replicate
Install latest dev version 11.10.1.dev1 and run pipenv check on project.

Awesome. Thanks for testing!

(Not awesome that there’s a bug but awesome that you caught it!)

@Gr1N I probably didn't solve the problem but the error message may be more helpful if you pip install --upgrade --pre pipenv now

@techalchemy problem looks solved:
$ pipenv --version
pipenv, version 11.10.1.dev2
$ pipenv check
Checking PEP 508 requirements…
Passed!
Checking installed package safety…
All good!

repo: pypa/pipenv


text:
feat: adding PEP 639 support

This adds PEP 639 support. Details for backends mentioned in readme.
I left normalization and validation to packaging (https://github.com/pypa/packaging/pull/828), otherwise we'd want to basically vendor that code and keep a license list up to date.

[x] Need to add tests for the various error and warning conditions.
[x] And I forgot the classifier error and warning bit.

Close https://github.com/pypa/pyproject-metadata/issues/130.

I scrolled through PEP639 and I've noticed a couple more things: this PR does not implement support for custom licenses (the LicenseRef-[idstring] metadata fields) and the validation of the glob patters is not as strict as specified in the PEP.

This PR doesn't do any validation of the license string at all, so LicenseRef-[idstring] should be fine? I'd assume the packaging PR would handle those?

Could I get a quick set of eyes on #133 and #134? I'll rebase this once those are in.

Sorry, continuing my quest for better error messages...

Thanks @henryiii Everything looks good to me.
I can have a try at integrating this into meson-python and see how it goes (we need to make sure that the license files are included in the wheel, otherwise I don't see anything else that needs to be done in the build backend) and report back. Or you can merge it and we can fix any shortcoming later. I'll probably will not have much time to work on this in the next couple of days...

There will be one more set of warnings or errors (if License :: is present), and a few more tests.
Will ping both you and Frost when it's ready.

@frostming and @dnicolodi, this is ready for review (@dnicolodi, the only thing changed since you've looked last is more tests and some new License related warnings and an error. And I followed your suggestion of putting the warning in validate, added a param to turn the warnings off so they only show up once).

I've released 0.9.0b1 so backend authors can try these changes out before we release them fully.
@dnicolodi for meson-python
@frostming for pdm
@henryiii for scikit-build-core
@pradyunsg for sphinx_theme_builder

authors: dnicolodi henryiii

repo: pypa/pyproject-metadata