text:
[Bug]: Memory leak in NetworkEventManager causing crashes

Bug description
Using 18.0.5 but reproduced on a variety of versions with no resolution. It appears that navigating away from a page before it has finished loading causes request maps, such as #requestWillBeSentMap, to consume memory until the heap is exhausted.
const browser = await puppeteer.launch(options);
const page = await browser.newPage();

await page.setRequestInterception(true);

page.on(
    'request',
    request => {
        const resourceType = request.resourceType();

        // Simple hook to remove any images or fonts
        if (resourceType === 'image' || resourceType === 'font') {
            request.abort();
        } else {
            request.continue();
        }
    },
);

// Navigate to and from URLs that trigger larger network requests

This is especially prevalent with larger payloads and often leads to a hard crash within a few hours. You may need to load a page that sends many large payloads to reproduce effectively.
Puppeteer version
18.0.5
Node.js version
16.18.0
npm version
1.22.19 (Yarn)
What operating system are you seeing the problem on?
Linux, Windows

@Crunkle it'd be great if you could provide a test script that demonstrates the memory growth? The NetworkManager code indeed relies heavily on backend events and if events arrive in unexpected order that might lead to a memory leak.

Sure, I will try and get around to it this weekend. A simple page navigate loop should suffice.

@Crunkle thanks for the script! Unfortunately, I don't seem to be able to repro: SHOULD_MEMORY_LEAK = true my memory usage gets to the peak of 43.1 MB and stays the same (I ran the script for a few minutes). I am on Mac and Node 18.

@Crunkle do you run Puppeteer@19.2.2?

@OrKoN I was originally on 18 but am able to reproduce on the latest 19 release under Windows. I have not tested recently but can confirm this was an issue under Linux as well, but perhaps Chromium has updated since my last test. You may need to marginally increase the timeout to ensure the fetch is executed before navigate.
See this video for a Windows demo.

Still not able to repro (tried different time outs up to 2000ms): in my case, if I look into the CDP log I get Network.loadingFailed events upon navigation that are handled by the network event manager and clears all local data. I wonder if you see those events in the CDP log too? ('env DEBUG=puppeteer:*')

I see one Network.loadingFailed per request being sent in the log, which I am guessing is the correct behavior. This event does not seem to trigger any clearing of the #requestWillBeSentMap though, at least for me, and the data is retained within. If you are still unsure then I can have a dig at the Puppeteer source myself to try and pinpoint the issue. I am pretty confident after testing in an isolated container that it is on the Puppeteer side, and I have a hunch #9283 is the same issue.

@Crunkle I wonder if Network.loadingFailed events are being only sometimes missed, causing the leak? It'd be hard to isolate the issue then. In general, copying the Network events in the order they arrive from the log into a test https://github.com/puppeteer/puppeteer/blob/main/test/src/NetworkManager.spec.ts#L39 should help to reproduce the issue.

Is there any update on this or any workarounds? I am also facing this issue.
I use the request interception feature for scope management in my project https://github.com/karthikuj/sasori
In a couple websites too many requests get stored in the heap and then it crashes with JavaScript heap out of memory error.
I am not sure why it only happens in a couple websites and not all of them :/

@karthikuj do you have an example to reproduce the leak? would you be able to add a test case to https://github.com/puppeteer/puppeteer/blob/9ec5f25ea6f9e60d250c7413e122a5c32faeb3f9/packages/puppeteer-core/src/cdp/NetworkManager.test.ts#L37 with the requests that you find leaking?

@OrKoN I don't have a clear understanding of why this is happening, and the website I found this on is a bit confidential, but I will try reproducing it with some testcase whenever I get the time.

Still not able to reproduce. If anyone is able to reproduce with a minimal example, please post it here or feel free to file a new issue.

authors: Crunkle OrKoN karthikuj


text:
feat: add fromSurface option

What kind of change does this PR introduce?
Feature.
Did you add tests for your changes?
No, I am unsure how to do that. Also, it seems like there is no test for the captureBeyondViewport option.
If relevant, did you update the documentation?
Yes.
Summary
This PR adds the fromSurface option to the Page.screenshot method.
This option can e.g. be used to properly screenshot WebGPU (see https://github.com/mrdoob/three.js/pull/24109 for a discussion).
Does this PR introduce a breaking change?
No.

This is great.

Thanks for the PR. Please provides tests for this.

Added the test.
As you can see, the screenshot is not 500x500 - this is because fromSurface: false mode ignores viewport, but not browser window's bounds, but they are not set by the Puppeteer - this is exactly https://github.com/puppeteer/puppeteer/issues/1910.
I can make a PR solving it after this PR is merged (my workaround requires capturing fromSurface: false screenshot) or can include it in this PR.

Added the test.
As you can see, the screenshot is not 500x500 - this is because fromSurface: false mode ignores viewport, but not browser window's bounds, but they are not set by the Puppeteer - this is exactly https://github.com/puppeteer/puppeteer/issues/1910.
I can make a PR solving it after this PR is merged (my workaround requires capturing fromSurface: false screenshot) or can include it in this PR.

@jrandolf Should I include the fix in this PR or in a separate PR?
I'm afraid that because my workaround requires getting width and height from screenshot, an image-parsing library should be added - e.g. either jpeg-js or pngjs should be moved from devDependencies to dependencies... Would that be OK?

I think we would not want to bring an image parsing library into the list of our prod dependencies. Could you please describe the latest problem you are facing? IIUC, the screenshot is bigger than the current viewport because setting the viewport does not change the window bounds?
P.S. also if you could rebase once more, I would run the test on the bots

Could you please describe the latest problem you are facing? IIUC, the screenshot is bigger than the current viewport because setting the viewport does not change the window bounds?

The problem is that capturing Chromium screenshot in fromSurface: false mode ignores the page viewport, but not window bounds. And the Puppeteer does not change window bounds in the headful mode because of https://github.com/puppeteer/puppeteer/issues/1910.

P.S. also if you could rebase once more, I would run the test on the bots

Done.

The problem is that capturing Chromium screenshot in fromSurface: false mode ignores the page viewport, but does not ignore window bounds. And the Puppeteer does not change window bounds in the headful mode because of https://github.com/puppeteer/puppeteer/issues/1910.

(just for me to understand it) so once we capture a screenshot with fromSurface=false we could crop it to the current viewport dimensions and that would give us a correct screenshot of the viewport?

Checked the following methods of getting true bounds:

Getting width and height of fromSurface: true-screenshot - works,
Getting width and height of fromSurface: false-screenshot - does not work,
Getting document.body.clientWidth and document.body.clientHeight - does not work,
Getting document.body.getBoundingClientRect() - does not work.

So the capturing of the screenshot appears to be the only option.

It also returns full window bounds (i.e. with UI).

It seems that the test does not even work because of different UI sizes on different platforms, I think? Not sure...

Yes, exactly - it works perfectly for me on Windows.

Also not sure about chrome-headless mode (does the options work in it or only in true headful), will investigate...

It does not support the option. So everything in this PR seems to be correct :-)

@OrKoN So is this PR going to be merged? I can temporary disable the test until the decision of what to do with https://github.com/puppeteer/puppeteer/issues/1910 will be made.

@LeviPesin let's land the PR but update the test to just verify that a screenshot is taken. The dimensions of the screenshot are not really under control of Puppeteer so I think the solution to that can be found later.

Done.

let's also remove the golden image file!

source: gharchive/pull-request

authors: LeviPesin OrKoN jrandolf munrocket

repo: puppeteer/puppeteer

license_type: permissive

license_source: github-api


text:
URL parameter requires scheme

npx puppeteer-recorder v8.dev does not work, but npx puppeteer-recorder http://v8.dev does. Should we have a way to insert http:// automatically if it's not there?

That's a great idea. I'll add that.

Do you think it should default to https: or http:?

https sgtm

source: gharchive/issue

authors: hashseed jschfflr

repo: puppeteer/recorder

license_source: bigquery


text:
(GH-1292) Migrate Vault ruby plugin to be in a module

This moves the Vault plugin from being a ruby plugin that lives in Bolt
to being a ruby task which lives in the puppetlabs-vault module.

CLA signed by all contributors.

Ahh, boo merge conflict in puppetfile.

authors: donoghuc lucywyman puppetcla

repo: puppetlabs/bolt


text:
(PUP-5879) Use FileSystem.read where applicable to read JSON, settings and other files as UTF-8

Adds additional UTF-8 support to Puppet when reading files from disk, including:

Settings like puppet.conf
PMT ignore files - .gitignore and .pmtignore
Hiera YAML and JSON files
Modules metadata.json
Route auth files
MsgPack

CLA signed by all contributors.

Peculiar unrelated registry spec error on Windows:
Failures:

  1) Puppet::Util::Windows::Registry#values when reading values should return REG_MULTI_SZ values
     Failure/Error: expect(written).to eq(pair[:value])

       expected: ["string1", "string2"]
            got: ["string1", "string2", "\u6F00"]

       (compared using ==)
     # ./spec/integration/util/windows/registry_spec.rb:222:in `block (6 levels) in <top (required)>'
     # ./spec/integration/util/windows/registry_spec.rb:213:in `block (5 levels) in <top (required)>'

Finished in 21 minutes 2 seconds (files took 16.53 seconds to load)
20166 examples, 1 failure, 62 pending

Failed examples:

rspec './spec/integration/util/windows/registry_spec.rb[1:3:5:3]' # Puppet::Util::Windows::Registry#values when reading values should return REG_MULTI_SZ values

@Iristyle We picked that up in PUP-5993

#4898 is the next PR in this series

authors: Iristyle glennsarti puppetcla

repo: puppetlabs/puppet


text:
(PUP-6827) fix test to meet password complexity reqs

Prior to this commit, the test '#6857: redact password hashes when applying in
noop mode' would fail on a host that had 'Password must meet complexity
requirements' set to true, because puppet was applying user management
manifests containing passwords that did not meet said requirements. This commit
updates the test to meet the complexity requirements (documented here:
https://technet.microsoft.com/en-us/library/hh994562(v=ws.11).aspx), allowing tests to pass.
Assumedly prior hosts in the acceptance testing systems were either
pre-configured with this requirement disabled or it was disabled at test time
for other tests (the latter of which seems unlikely).
Signed-off-by: Moses Mendoza moses@puppet.com

This previously failed on the internal windows 2016 64-bit test host.
Passing run: https://gist.github.com/MosesMendoza/2a7f178661ad0bf8c24f49bad254165f

For consistency's sake, do you think we ought to look into whether or not the other hosts had different settings, and update this new one to match (or vice versa, now that the test is fixed)?

CLA signed by all contributors.

@Magisus seems reasonable. You mean update check the templates in the vm pool for inconsistencies? @jcoconnor thoughts?

@Magisus I think that probably shouldn't hold up this PR though?

No, it doesn't need to hold up the PR. I'm 👍 on this change.

authors: Magisus MosesMendoza puppetcla


text:
(PUP-9077) Fix problems with 3.x functions returning :undef

Since the calling API for 3.x functions requires nil values to be
transformed to Ruby Symbol :undef such values could be returned from 3.x
functions thus causing 4.x functions to also get such values.
This is bad because :undef is a 3.x construct that should not be
exposed in Puppet's modern APIs.
This commit fixes this by rewriting all returned values from 3.x
functions - transforming any nested :undef found in Array or Hash
values. The rewrite does not replace :undef in other types of values.
In this commit, the ToDataConverter is also updated to handle
an :undef value in the rare event that it escaped from a 3.x runtime.

AppVeyor failed with slow operation detection... rerunning jobs

CLA signed by all contributors.

The same problem is addressed as part of #7019 which is targeting 5.5.x - and thus will fix the same thing but in a different way.
Marked this PR as blocked as it should really be closed in favour of merging #7019 and then merging it to master for Puppet 6.

Since #7019 is now merged this PR is no longer meaningful. Closing.

authors: hlindberg puppetcla


text:
(PDB-3193) Fix beaker AWS duplicate keys

This is just a branch for me to use while we fix the beaker issue on beaker master branch.

Test FAILed

CLA signed by all contributors.

@puppetlabs-jenkins retest this please

Test FAILed

@puppetlabs-jenkins retest this please

Test FAILed

Test FAILed

@puppetlabs-jenkins retest this please

Test FAILed

@puppetlabs-jenkins retest this please

Test PASSed

@puppetlabs-jenkins retest this please

Test PASSed

@puppetlabs-jenkins retest this please

Test PASSed

@puppetlabs-jenkins retest this please

Test PASSed

@puppetlabs-jenkins retest this please

Test PASSed

@puppetlabs-jenkins retest this please

Test PASSed

@puppetlabs-jenkins retest this please

Test PASSed

@puppetlabs-jenkins retest this please

Test PASSed

@puppetlabs-jenkins retest this platform

@kbarber what's the status on this? We're still getting a lot of failures on this

Test PASSed

Test PASSed

authors: kbarber puppetcla puppetlabs-jenkins senior

repo: puppetlabs/puppetdb


text:
Has this been abandoned?

.....

@earsdown not at all. We don't have a large team internally and we're split between multiple things however. Prioritisation is always the hard part. We'll likely have another .z release out shortly with some of the RDS fixes and any other low hanging fruit.

authors: earsdown garethr

repo: puppetlabs/puppetlabs-aws


text:
Instances for HashMap

I noticed that many of the optics in this package have instances for Map but not for HashMap. I would find it useful to have access to Ix and At (among others) for HashMaps. Would you be willing to accept a PR for this?

I'd prefer if we added the instances there instead perhaps, since this library is in contrib, and we tend not to add dependencies on things outside of core/contrib. That ensures we can keep the libraries up to date - when we have external dependencies, it's possible that dependencies go stale, or the owner is too busy to review/merge updates when we're trying to update things here, etc.

Makes sense, thanks.

authors: drewolson garyb

license: mit


text:
Indentation level after type declaration (#51)

If we're on the line following a type annotation,
and our line doesn't start with dots or arrows,
and the previous line isn't an instance declaration,
then indentation level is 0.

Thanks @redxaxder

authors: coot redxaxder

repo: purescript-contrib/purescript-vim


text:
Foldable1.intercalate

Relaxes Foldable.interacalate's Monoid constraint to Semigroup.

Sadly, this has to be breaking due to the added import of purescript-partial. If anyone sees a way to get around that, I'd be very happy.

Another note: Haskell calls this intercalate1, but I'm really not sure that's necessary. People can do qualified imports: https://hackage.haskell.org/package/semigroupoids-5.2.2/docs/Data-Semigroup-Foldable.html

I've rewritten this to use techniques from Haskell's semigroupoids, such that it no longer needs unsafePartial. It should now be non-breaking.

This looks good! Would you mind resolving the conflict?

Would you mind resolving the conflict?

done.

authors: garyb matthewleon


text:
Changed return types of ceil, floor, round and (%) to return Int.

Seems obvious to me.

This has been raised a few times, but there are a few problems doing this:

(%) intentionally operates on Numbers, as it's the only way to get modulo behaviour for non-Ints. You can already do this with mod for Int.
Using Int for the others restricts the acceptable values they can operate on to the range of Int32, significantly smaller than the range of integers that Number can safely represent.

Also 0.5 % 0.7 does not produce an Int, so the type isn't right for that either :smile:

We should probably document those functions to explain the types :)

Maybe these belong in -integers?

Yeah, that would make sense.

We have floor, ceil, and round with the types @jutaro wanted in integers now, so I think this can be closed.

authors: garyb hdgarrood jutaro paf31

repo: purescript/purescript-math


text:
Add support for Heading tags

Heading. tags are getting stripped out.
'h1','h2','h3','h4','h5','h6' seem to be missing from the htmlEls array in purifycss.js

Many thanks

authors: kennyt phreezer

repo: purifycss/purifycss


text:
🛑 Puro.earth Registry Services API (Staging) is down

In 6965815, Puro.earth Registry Services API (Staging) (https://rs.api-preview.purostaging.com/$API_SERVICES_HEALT_ENDPOINT) was down:

HTTP code: 503
Response time: 306 ms

Resolved: Puro.earth Registry Services API (Staging) is back up in 870c934 after 4 hours, 48 minutes.

repo: puro-earth/status

license: MIT


text:
Repetitive crashes from Firebase Crashlytics dashboard for live app in app store

I have this repetitive error in iOS version
I used pusher_channels_flutter: ^2.2.0
$s23pusher_channels_flutter32SwiftPusherChannelsFlutterPluginC14fetchAuthValue8socketID11channelName17completionHandlerySS_SSy0eD00eJ0CSgctFyypSgcfU_TA
0  libsystem_kernel.dylib         0x7198 __pthread_kill + 8
1  libsystem_pthread.dylib        0xd5f8 pthread_kill + 208
2  libsystem_c.dylib              0x1c4b8 abort + 124
3  libswiftCore.dylib             0x2f2998 swift::fatalError(unsigned int, char const*, ...) + 130
4  libswiftCore.dylib             0x2f29b4 swift::warningv(unsigned int, char const*, char*) + 26
5  libswiftCore.dylib             0x2ea6ec swift::swift_dynamicCastFailure(swift::TargetMetadata<swift::InProcess> const*, swift::TargetMetadata<swift::InProcess> const*, char const*) + 74
6  libswiftCore.dylib             0x2ea764 swift::_conformsToProtocol(swift::OpaqueValue const*, swift::TargetMetadata<swift::InProcess> const*, swift::TargetProtocolDescriptorRef<swift::InProcess>, swift::TargetWitnessTable<swift::InProcess> const**) + 118
7  libswiftCore.dylib             0x2ee6ac swift_dynamicCastImpl(swift::OpaqueValue*, swift::OpaqueValue*, swift::TargetMetadata<swift::InProcess> const*, swift::TargetMetadata<swift::InProcess> const*, swift::DynamicCastFlags) + 278
8  pusher_channels_flutter        0x9d88 $s23pusher_channels_flutter32SwiftPusherChannelsFlutterPluginC14fetchAuthValue8socketID11channelName17completionHandlerySS_SSy0eD00eJ0CSgctFyypSgcfU_TA + 156
9  pusher_channels_flutter        0x5bec $sypSgIegn_yXlSgIeyBy_TR + 84
10 Flutter                        0x5e1dd0 InternalFlutterGpuTestClass_Method + 6860
11 Flutter                        0x3e598 (Missing UUID 4c4c44da55553144a1f7c3e66c77eeff)
12 Flutter                        0x2e5d68 (Missing UUID 4c4c44da55553144a1f7c3e66c77eeff)
13 Flutter                        0x2e94fc (Missing UUID 4c4c44da55553144a1f7c3e66c77eeff)
14 CoreFoundation                 0xc1bb0 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 28
15 CoreFoundation                 0x82de4 __CFRunLoopDoTimer + 884
16 CoreFoundation                 0x2c0fc __CFRunLoopDoTimers + 284
17 CoreFoundation                 0x755bc __CFRunLoopRun + 1940
18 CoreFoundation                 0x79d20 CFRunLoopRunSpecific + 584
19 GraphicsServices               0x1998 GSEventRunModal + 160
20 UIKitCore                      0x37180c -[UIApplication _run] + 868
21 UIKitCore                      0x371484 UIApplicationMain + 312
22 Runner                         0x8d54 main + 6 (AppDelegate.swift:6)
23 ???                            0x1bc6f4344 (Missing)```

Could you share the code in use for this, along with an overview of when this error is encountered? For example, does it happen when connecting to Channels / receiving events?

I don't know when and how it happens, I have this repetitive crashes daily in firebase crashlytics

Hi @benw-pusher. I am facing this issue of app crashing as well on a test flight build on my client's iphone. But I have figured the app crashes only when the iphone is running on latest ios version 17, as the app works fine on my iphone 13 pro max with ios 16.
I have been using the same code for the app provided in the example app and its working as well. So I don't need to provide you any example code because app crashes with same code on ios 17 but works fine on earlier versions.
I am using latest available version 2,2,0 of the package.
Kindly help us resolve the issue as our team is stuck for release this app to the app store.
Thanks is advance.

I have tested the example app on an iOS 17 device and no crashes are observed.
Do you know when this issue occurs? For example, subscribing to a particular channel type / receiving an event?

Hi @benw-pusher. I am facing this issue of app crashing as well on a test flight build on my client's iphone. But I have figured the app crashes only when the iphone is running on latest ios version 17, as the app works fine on my iphone 13 pro max with ios 16. I have been using the same code for the app provided in the example app and its working as well. So I don't need to provide you any example code because app crashes with same code on ios 17 but works fine on earlier versions. I am using latest available version 2,2,0 of the package. Kindly help us resolve the issue as our team is stuck for release this app to the app store. Thanks is advance.

I am facing the same issue. did. u find any solution?

the crash was caused by the onAuthorizer passed when initiating pusher. it was not returning expected type of response that was casted in the library code

authors: benw-pusher kadersaka mohamedsalah414 saurabhunify

repo: pusher/pusher-channels-flutter


text:
Channel name is invalid error

When  i am subscribing new private encrypted channel facing below error.
Error
I/PusherChannelsFlutter(14182): Start com.pusher.client.Pusher@4140054
E/MethodChannel#pusher_channels_flutter(14182): Failed to handle method call
E/MethodChannel#pusher_channels_flutter(14182): java.lang.IllegalArgumentException: Channel name private-encrypted-my-channel is invalid. Private channel names must start with "private-" and presence channel names must start with "presence-"
E/MethodChannel#pusher_channels_flutter(14182): 	at com.pusher.client.channel.impl.ChannelImpl.<init>(ChannelImpl.java:37)
E/MethodChannel#pusher_channels_flutter(14182): 	at com.pusher.client.channel.impl.PrivateChannelImpl.<init>(PrivateChannelImpl.java:30)
E/MethodChannel#pusher_channels_flutter(14182): 	at com.pusher.client.util.Factory.newPrivateChannel(Factory.java:88)
E/MethodChannel#pusher_channels_flutter(14182): 	at com.pusher.client.Pusher.subscribePrivate(Pusher.java:283)
E/MethodChannel#pusher_channels_flutter(14182): 	at com.pusher.channels_flutter.PusherChannelsFlutterPlugin.subscribe(PusherChannelsFlutterPlugin.kt:143)
E/MethodChannel#pusher_channels_flutter(14182): 	at com.pusher.channels_flutter.PusherChannelsFlutterPlugin.onMethodCall(PusherChannelsFlutterPlugin.kt:68)
E/MethodChannel#pusher_channels_flutter(14182): 	at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:262)
E/MethodChannel#pusher_channels_flutter(14182): 	at io.flutter.embedding.engine.dart.DartMessenger.invokeHandler(DartMessenger.java:295)
E/MethodChannel#pusher_channels_flutter(14182): 	at io.flutter.embedding.engine.dart.DartMessenger.lambda$dispatchMessageToQueue$0$DartMessenger(DartMessenger.java:319)
E/MethodChannel#pusher_channels_flutter(14182): 	at io.flutter.embedding.engine.dart.-$$Lambda$DartMessenger$TsixYUB5E6FpKhMtCSQVHKE89gQ.run(lambda)
E/MethodChannel#pusher_channels_flutter(14182): 	at android.os.Handler.handleCallback(Handler.java:751)
E/MethodChannel#pusher_channels_flutter(14182): 	at android.os.Handler.dispatchMessage(Handler.java:95)
E/MethodChannel#pusher_channels_flutter(14182): 	at android.os.Looper.loop(Looper.java:154)
E/MethodChannel#pusher_channels_flutter(14182): 	at android.app.ActivityThread.main(ActivityThread.java:6119)
E/MethodChannel#pusher_channels_flutter(14182): 	at java.lang.reflect.Method.invoke(Native Method)
E/MethodChannel#pusher_channels_flutter(14182): 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:886)
E/MethodChannel#pusher_channels_flutter(14182): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:776)
E/flutter (14182): [ERROR:flutter/lib/ui/ui_dart_state.cc(198)] Unhandled Exception: PlatformException(error, Channel name private-encrypted-my-channel is invalid. Private channel names must start with "private-" and presence channel names must start with "presence-", null, java.lang.IllegalArgumentException: Channel name private-encrypted-my-channel is invalid. Private channel names must start with "private-" and presence channel names must start with "presence-"
E/flutter (14182): 	at com.pusher.client.channel.impl.ChannelImpl.<init>(ChannelImpl.java:37)
E/flutter (14182): 	at com.pusher.client.channel.impl.PrivateChannelImpl.<init>(PrivateChannelImpl.java:30)
E/flutter (14182): 	at com.pusher.client.util.Factory.newPrivateChannel(Factory.java:88)
E/flutter (14182): 	at com.pusher.client.Pusher.subscribePrivate(Pusher.java:283)
E/flutter (14182): 	at com.pusher.channels_flutter.PusherChannelsFlutterPlugin.subscribe(PusherChannelsFlutterPlugin.kt:143)
E/flutter (14182): 	at com.pusher.channels_flutter.PusherChannelsFlutterPlugin.onMethodCall(PusherChannelsFlutterPlugin.kt:68)
E/flutter (14182): 	at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:262)
E/flutter (14182): 	at io.flutter.embedding.engine.dart.DartMessenger.invokeHandler(DartMessenger.java:295)
E/flutter (14182): 	at io.flutter.embedding.engine.dart.DartMessenger.lambda$dispatchMessageToQueue$0$DartMessenger(DartMessenger.java:319)
E/flutter (14182): 	at io.flutter.embedding.engine.dart.-$$Lambda$DartMessenger$TsixYUB5E6FpKhMtCSQVHKE89gQ.run(lambda)
E/flutter (14182): 	at android.os.Handler.handleCallback(Handler.java:751)
E/flutter (14182): 	at android.os.Handler.dispatchMessage(Handler.java:95)
E/flutter (14182): 	at android.os.Looper.loop(Looper.java:154)
E/flutter (14182): 	at android.app.ActivityThread.main(ActivityThread.java:6119)
E/flutter (14182): 	at java.lang.reflect.Method.invoke(Native Method)
E/flutter (14182): 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:886)
E/flutter (14182): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:776)
E/flutter (14182): )
E/flutter (14182): #0      StandardMethodCodec.decodeEnvelope (package:flutter/src/services/message_codecs.dart:607:7)
E/flutter (14182): #1      MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:167:18)
E/flutter (14182): <asynchronous suspension>
E/flutter (14182): #2      PusherChannelsFlutter.subscribe (package:pusher_channels_flutter/pusher_channels_flutter.dart:237:5)
E/flutter (14182): <asynchronous suspension>
E/flutter (14182): #3      Pusher._init (package:zifhy_app/pusher/pusher_file.dart:58:5)
E/flutter (14182): <asynchronous suspension>
E/flutter (14182): `

```

For Private channel it is working fine. Facing the above issue with private-encrypted-channel

I've been able to replicate this issue and have raised an issue with our Engineering team to investigate. I will let you know when we have an update on this.

Just have to swap the switch case. Refer this,
https://github.com/macpraveen/pusher-channels-flutter/commit/9a07d94d00a9764cffad6671efe61b1ad19afb07
I can give a PR if needed.

authors: benw-pusher devansvd macpraveen samclam


text:
🛑 Payment Gateway is down

In 60e401c, Payment Gateway (https://payment.unmer.ac.id/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Payment Gateway is back up in f6bff27 after 1 hour, 14 minutes.

repo: pusimgit/upptime


text:
Is it recommended to use @babel/parse or @putout/babel?

Version (npm -v): 9.6.4
Node Version node -v: 20.1.0
OS (uname -a on Linux): Mac
Browser name/version: N/A
Used Command Line Parameters: N/A

Hey, in the Example section you recommend using @babel/parser however I faced some issues, for example when printing code like this:
type test = {
fn: () => void
}

Output:
type test = {
fn: () =>
}

Note: No void keyword printed.
It took me some time to figure out what's going on, only after looking at your tests I realized that you are using @putout/babel.  Not sure whether it's a bug or just README has to be tweaked.

Recommended way is using @putout/babel, this is Babel 8 bundled to CommonJS

Is it works for you?

@coderaiser Yes, it worked out 👍

repo: putoutjs/printer


text:
Not handling destructured args

Just came across this:
$ cat mini.coffee
a = 1
b = 1
f = ({a,b}) ->
  a = 0

$ ./node_modules/.bin/coffeelint ./mini.coffee
  ✗ ./mini.coffee
     ✗ #1-4: Outer scope variable overwrite. a.

✗ Lint! » 1 error and 0 warnings in 1 file

Similarly, destructuring array assignment does this:
[a, ...] = thing()

$ node_modules/.bin/coffeelint -f coffeelint.json test/splatty-destruct.coffee
/home/ben/code/coffeelint-bundle/node_modules/coffeelint-variable-scope/index.js:113
              name = v.variable ? v.value.base.value : v.base.value;
                                                             ^

TypeError: Cannot read property 'value' of undefined
  at /home/ben/code/coffeelint-bundle/node_modules/coffeelint-variable-scope/index.js:113:62
  at /home/ben/code/coffeelint-bundle/node_modules/coffeelint/node_modules/coffee-script/lib/coffee-script/nodes.js:211:17
  at Block.exports.Base.Base.eachChild (/home/ben/code/coffeelint-bundle/node_modules/coffeelint/node_modules/coffee-script/lib/coffee-script/nodes.js:199:17)
  at Block.exports.Base.Base.traverseChildren (/home/ben/code/coffeelint-bundle/node_modules/coffeelint/node_modules/coffee-script/lib/coffee-script/nodes.js:209:19)
  at VariableScopeRule.module.exports.VariableScopeRule.nodeAssigns (/home/ben/code/coffeelint-bundle/node_modules/coffeelint-variable-scope/index.js:96:12)
  at VariableScopeRule.module.exports.VariableScopeRule.lintNode (/home/ben/code/coffeelint-bundle/node_modules/coffeelint-variable-scope/index.js:46:22)
  at VariableScopeRule.module.exports.VariableScopeRule.lintAST (/home/ben/code/coffeelint-bundle/node_modules/coffeelint-variable-scope/index.js:18:21)
  at ASTLinter.module.exports.ASTLinter.lint (/home/ben/code/coffeelint-bundle/node_modules/coffeelint/lib/coffeelint.js:569:43)
  at Object.coffeelint.lint (/home/ben/code/coffeelint-bundle/node_modules/coffeelint/lib/coffeelint.js:364:67)
  at ErrorReport.module.exports.ErrorReport.lint (/home/ben/code/coffeelint-bundle/node_modules/coffeelint/lib/coffeelint.js:714:51)
  at lintFiles (/home/ben/code/coffeelint-bundle/node_modules/coffeelint/lib/commandline.js:96:19)
  at Object.<anonymous> (/home/ben/code/coffeelint-bundle/node_modules/coffeelint/lib/commandline.js:249:21)
  at Object.<anonymous> (/home/ben/code/coffeelint-bundle/node_modules/coffeelint/lib/commandline.js:254:4)
  at Module._compile (module.js:434:26)
  at Object.Module._extensions..js (module.js:452:10)
  at Module.load (module.js:355:32)
  at Function.Module._load (module.js:310:12)
  at Module.require (module.js:365:17)
  at require (module.js:384:17)
  at Object.<anonymous> (/home/ben/code/coffeelint-bundle/node_modules/coffeelint/bin/coffeelint:34:5)
  at Module._compile (module.js:434:26)
  at Object.Module._extensions..js (module.js:452:10)
  at Module.load (module.js:355:32)
  at Function.Module._load (module.js:310:12)
  at Function.Module.runMain (module.js:475:10)
  at startup (node.js:117:18)
  at node.js:951:3

authors: bchociej yang


text:
Preparation for jailbreak 13.3

Ok people looks like everyone is having problems so I’ll try to help I do have programming knowledge so I’ll try to help best I can.
First of all you MUST delete the 13.3.1 update package Hopefully providing that you didn’t install it so go to settings and storage Scroll down you will see it there should be 2 delete both.
Install iOS ninja AltStore is fucking garbage so uninstall if you were conned into installing it.
Next install unc0ver 4.0 ONLY if you have a A12 or A13 device on 13.0 to 13.3 anything and I mean anything even lower won’t work.
After install turn on airplane mode then open unc0ver wait 30 seconds then press jailbreak it should install and reboot not respring then when you get back to the home screen turn off airplane mode then go back to unc0ver wait 30 seconds and hit jailbreak Cydia should be installed and will respring that should be it

Don’t come on here and say “I have a iPhone 11 on 12.4.2 waiting for unc0ver and it doesn’t work” well of course not obb-timestamp exploit Didnt exist yet in those firmwares please understand all of you were told 13 to 13.3 THATS IT

I can also help getting those tweaks to work as well

Why would pwn remove the support for 12.4 and not say anything about it? If the latest u0 is for 13-13.3 ONLY, why would he not make a separate jailbreak.app for that?
Doesnt make sense for him to not just add support to an existing jailbreak and let the app choose the best exploit for the device.
Im not questioning you so dont @ me

I can help getting those tweaks to work as well

Why would pwn remove the support for 12.4 and not say anything about it? If the latest u0 is for 13-13.3 ONLY, why would he not make a separate jailbreak.app for that?
Doesnt make sense for him to not just add support to an existing jailbreak and let the app choose the best exploit for the device.
Im not questioning you so dont @ me

Because you need to download the older releases

He didn’t put any other exploit in this release if you go to unc0ver settings you will see the obb-time stamp exploit but that’s it if he included all of the other 3 exploits then it would be ok but you would need to know which one to use

Oh and the tweak developers need to update there apps for kppless jailbreak like this one and is one of the reasons people are having problems the other reasons are substrate and version compatibility

16 hours ago pwn tweeted this:
"Successfully verified that the #unc0ver jailbreak still works on iOS 11.0-12.4.1.
I will now compile the release build, make sure that it jailbreaks successfully with the internal beta testers and release it!"
And you still think pwn removed support?
I havent looked at the files for 4.0 yet but im pretty sure this is just a bug where u0 chose to use 13-13.3 substitute and not the one for 12.4. pwn did rewrite the substitute support package so might be something there

Don’t know how when the exploit you were using isn’t there

You don’t need me to tell u go to unc0ver go to settings then all the way down select the exploit hmm there seems there is only one

nah i know that

Well I’m sure that’s why your having problems ether that or your going to have to restore rootfs and lose everything and start over with the new exploit

I have a feeling that both exploits are having a conflict since roofFS is expecting the old exploit and now is getting a new one

Thats my thought as well. Looking into that as we speak


text:
“Back to registry” button could just trigger a page reload

Would be simpler to just window.location.reload(), rather than storing and replacing the DOM as we do at present.

Fixed in 5db7fb6.

repo: pwyf/iati-org-viz


text:
Configure RN project to use SVGs

Describe the desired behavior
When starting a new react native project using the RNC CLI, the project should be ready to go with SVG icons out of the box.
Describe the current behavior
Currently, the default project is not configured to allow the use of SVG icons, so copying and pasting code snippets that includes icons will give you the red screen of death.
Is this request related to a current issue?
Discovered when testing the new version of the RN component library.
Additional Context
We need to include packages for:

react-native-svg
react-native-svg-transformer
@types/react-native-vector-icons (dev dependency for typescript projects)

Then we need to follow the steps for configuring react-native-svg-transformer (https://github.com/kristerkari/react-native-svg-transformer#step-3-configure-the-react-native-packager)

Maybe also add in react-native-safe-area-context while working on this issue...

Maybe also add in react-native-safe-area-context while working on this issue...

Let's add any peer dependencies that are required by any of the packages we install by default.

When doing the SVG setup, also remember to update the android gradle file to properly link the icons (this is why sometimes the icons don't load on android).

If this is quick, we might be able to lump this in with the work we're doing to connect up the new react templates. Otherwise, we can definitely get it working when we do the RN templates.

authors: huayunh joebochill

repo: pxblue/pxblue-cli


text:
Doesn't seem to work on the Canadian Uber Eats

When I use the script on the Canadian Uber Eats https://www.ubereats.com/ca/feed
It doesn't seem to filter out non BOGO offers or hide blacklisted restaurants
The script has a little red icon on TamperMonkey indicating the script it loaded
And it shows the injected fields to configure the script settings on the page
But if I click any uber eats filters or apply the Better Eats filter it hides the script entirely and doesn't show anything filtered out
Is there anything I could do to assist if you need any troubleshooting

@GabrielSerealia hey for now the script only works on the pre-filtered "Offers" page
make sure you have that option checked off.

authors: GabrielSerealia pxue

repo: pxue/better-eats


text:
Does my Gardena lawn mower think it's a vacuum cleaner?

My Gardena Smart Sileno 750 tells Home Assistant that is "cleaning" (= "städar" in Swedish) when its cutting the grass...

Is it possible to change these status messages?

"Städar" = Cleaning isnt the same as OK_CUTTING

Until very recently (last release) there was no entity class for lawn mower in HA, so the Vaccuum one was used instead (as they are fairly similar). Lots of nice automatic UI functions (Tile cards etc) derive from this. You can of course use the entities and actions "raw" and create any UI you want. Eventually the maintainers will likely port the integration to use the new entity class (although it's still pretty limited, but the Automower integration already uses it).

authors: klercker tomasenskede

repo: py-smart-gardena/py-smart-gardena


text:
MLGST log-likelihood _objective_func_probs functions take vectorGS …

Pretty sure someone must have split up these functions without realizing they needed vectorGS in scope.
Was causing test.test_packages.algorithms.testCore.TestCoreMethods.test_MLGST to fail

(I wanted this reviewed because I'm not 100% sure this was the intended functionality)

Made redundant by a8992a10585e4752d6fcc803777e8159fb9da15b -- ignore me!

repo: pyGSTio/pyGSTi


text:
Documentation for covariance estimators

Classes Covariance, ERPCovariances, XdawnCovariances, HankelCovariances have a parameter estimator, with the same description "For a complete list of estimator, see utils.covariance." But module utils.covariance does not provide a documentation about supported estimators.
This PR solves #97, adding the documentation of function utils.covariance.covariances, and making cross-references to it.

@agramfort , can you merge?

done
I gave you write access to repo congrats :)

authors: agramfort qbarthelemy

repo: pyRiemann/pyRiemann


text:
TimeoutError: Server did not start in 10 seconds

Hi,
I am trying to read the rst file with following commands:
from ansys.dpf import core as dpf
my_data_sources= dpf.DataSources()
my_data_sources.set_result_file_path("C:/Users/amd8kor/Desktop/ANSYS_DPF/WS031_file.rst")
I have installed the pip packages but not able to read the rst file as I am getting the following error :
File "C:\Users\amd8kor.conda\envs\py37\lib\site-packages\ansys\dpf\core\server.py", line 538, in launch_dpf
raise TimeoutError(f"Server did not start in {timeout} seconds")
TimeoutError: Server did not start in 10 seconds
I even tried to connect to dpf server with following command but I am getting the same error:

from ansys.dpf import core as dpf
dpf.connect_to_server(ip='10.0.0.22', port=50054)

Is there a work around for this problem? Thanks in advance
Regards,
Akshay

hi @settyakshay , please could you tell me what is the version of pyDPF that you are using (pip show ansys.dpf.core) and what is the version of the Ansys workbench that you installed?
Thanks
Ramdane

Hi..
Name: ansys-dpf-core
Version: 0.3.4
I have v2021R1 ANSYS installed. Thanks

@settyakshay is this possible for you to upgrade to v2021R2?

@rlagha I can but it might take sometime to download and install. I will do it in parallel. Is there any other work around to get this running for v2021R1?

then you will need to install previous version of ansys.dpf.core but I'd recommend you to install 2021R2

@rlagha I installed ANSYSv2021R2 and the issue is resolved. Thanks

Great thanks for the update @settyakshay.
Ramdane

authors: rlagha settyakshay

repo: pyansys/pydpf-core


text:
Fix upload_file_in_tmp_folder

Pipelines have been seen failing due to ansys.dpf.core.core.upload_file_in_tmp_folder returning an empty string, making for subsequent failed calls to load_library and such.
This PR aims at investigating why the returned string is sometimes empty by adding a test on the obtained result to the call to self._api.data_processing_upload_file.
This has also been a source of random failing examples during the documentation generation.
Edit: the idea of adding a check for empty paths returned by upload_file_to_tmp is good. Yet the source of the problem might be:

a lack of TMP env variable (which should be set and does not explain the randomness of the failure)
the server having not finished to properly start, hence the request would go wrong? That would at least explain the randomness.

Isn't this related to #466  ??

@PProfizi isn't this related to #466 ??
By the way, I re-run your tests, it seems the problem was a 503 error. Now it works.

Hi @germa89, the part about the empty string seems to be the same thing, yes. Yet the DPFServerException: a unimplemented error ocurred: error is not the same, so that might be specific to Docker.

Closing as stale. The errors mentioned seems to have been solved in the meantime.


text:
Documenting differences between MAPDL and PyMAPDL

Adding documentation about differences between MAPDL and PyMAPDL.
Close #1723

@pvargasm please your part (about plotting) in the corresponding section.
@mikerife and @pmaroneh please do check the text regarding the listing. I presume there are many things to fix.
@PipKat if you can have a look (maybe not yet, just wait for the comments)
@koubaa and @akaszynski for further input/guidance

Hi @germa89 , nice addition! LGTM.

https://mapdl.docs.pyansys.com/dev/getting_started/faq.html#why-pymapdl-results-are-different-than-the-ones-shown-in-the-mapdl-gui

repo: pyansys/pymapdl


text:
add BatchStudy class

Description
Add class to compare different models and conditions
Fixes #906
Type of change
Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #) - note reverse order of PR #s. If necessary, also add to the list of breaking changes.

[ ] New feature (non-breaking change which adds functionality)
[ ] Optimization (back-end change that speeds up the code)
[ ] Bug fix (non-breaking change which fixes an issue)

Key checklist:

[ ] No style issues: $ flake8
[ ] All tests pass: $ python run-tests.py --unit
[ ] The documentation builds: $ cd docs and then $ make clean; make html

You can run all three at once, using $ python run-tests.py --quick.
Further checks:

[ ] Code is commented, particularly in hard-to-understand areas
[ ] Tests added that prove fix is effective or that feature works

What does solver_stats and export_data do?
Could you please check it? You can run this code here
https://colab.research.google.com/drive/109KJSumPxzxi3OToN6N8g6wjuD5Yb2oe?usp=sharing

Thanks @tinosulzer
I am working on the unit tests, will add them soon.

Thanks @tinosulzer
I am working on the unit tests, will add them soon. I have also fixed few other things I found and pushed them in the same PR rather than making a new one, hope you don't mind 😃

@tinosulzer The macOS and windows checks are getting cancelled as they are running for more than 6 hours, I don't know why. I ran the batch_study tests locally and I successfully got this
Add -v for more debug output
..
----------------------------------------------------------------------
Ran 2 tests in 72.316s

OK

repo: pybamm-team/PyBaMM


text:
No module named 'toga.style'

Expected Behavior

when I try to run "A slightly less toy example",   find err:
ModuleNotFoundError: No module named 'toga.style'
Current Behavior

Steps to reproduce

Your Environment

Python Version (list the specific version number)
py 3.6

Operating System and Version (select from the following and list the specific version number; if your OS is not listed, list that as well)

[ ] macOS - version:
[ ] Linux - distro:   - version:
[y ] Windows - version: win10  64bit
[ ] Other - name:  - version:

Toga Target (the type of app you are trying to generate)

[ ] android
[ ] cocoa
[ ] django
[ ] gtk
[ ] iOS
[ ] tvOS
[ ] watchOS
[ ] winforms
[ ] win32
[ ] Other (please specify)

You've almost certainly run the tutorial for Toga 0.3.0 (the current development version), but installed 0.2.15 (the current stable version).

repo: pybee/toga


text:
Postponed validation support

Feature Request
What I'm talking about is not actual postponed validation itself, but the ability to create model with invalid data and then manually check for errors and decide what to do next.
Despite the fact that validate_model func actually has argument raise_exc, which could control this behavior, currently pydantic throws error on any invalid data and doesn't allow user to have any control on it. Moreover didn't find any place where this arg used.
import datetime

from anystrenum import StrEnum
from pydantic import BaseModel

class Gender(StrEnum):
    male: str
    female: str
    other: str
    not_given: str

class User(BaseModel):
    id: int
    name: str
    gender: Gender
    registration_date: datetime.datetime

    class Config:
        postponed_validation = True

user = User(id='123', name='John Doe', gender='other than other', registration_date='broken')
if user.errors:
    print({error.vaule: error for error in user.errors})
    # {'other than other': 'value is not a valid enumeration member (type=type_error.enum)',
    #  'broken': 'invalid datetime format (type=type_error.datetime)'}

print(user.gender)
# other than other

user.raise_for_errors()

'''
Traceback (most recent call last):
...

pydantic.error_wrappers.ValidationError: 2 validation errors
gender
  value is not a valid enumeration member (type=type_error.enum)
registration_date
  invalid datetime format (type=type_error.datetime)
'''

You could do a try-catch around model.parse_obj with an exception handler for ValidationError that looks at the generated errors. I’ll update with an example shortly, but there are many like this in the pydantic tests.

Imagine we creating a models in different places and want to handle all errors in one place. In this place we don't know what was the initial data and can't access it, but need to figure out what value caused error.
For example imagine we have decorator which checking all models returned by some function:
import functools
import logging

log = logging.getLogger('Bad data handler')

def handle_bad_data(raw_data_on_errors=True):
    def decorator(func):
        @functools.wraps(func)
        def inner(*args, **kwargs):
            model = func(*args, **kwargs)
            if model.errors:
                log.error('Model %s has bad values:\n%s' % (
                    model, ',\n'.join((f'    {error.value}: {error}' for error in model.errors))
                    )
                )
                if raw_data_on_errors:
                    return model.data
                model.raise_for_errors()
            return model
        return inner
    return decorator

Not to necessarily discourage the development of this feature, but for what it's worth I think that pattern could be handled in a relatively clean way by passing around (model_type, raw_data) pairs prior to successful parsing, and writing handler functions / decorators / etc that expect inputs of that shape. You could then use the ValidationError to determine which fields had problems and handle them however you like, with access to the raw data.
Also, you may be able to accomplish this in a reusable way through the use of generics:
from typing import Any, Generic, TypeVar, Dict, Optional, Type
import datetime
from enum import Enum

from pydantic.generics import GenericModel
from pydantic import BaseModel, ValidationError, validator

T = TypeVar("T")

class Gender(Enum):
    male = 'male'
    female = 'female'
    other = 'other'
    not_given = 'not_given'

class User(BaseModel):
    id: int
    name: str
    gender: Gender
    registration_date: Optional[datetime.datetime]

class ErrorModel(GenericModel, Generic[T]):
    raw_data: dict
    parsed: Optional[T]
    errors: list

    @validator("parsed", pre=True, always=True)
    def parse_raw_data(cls, v: Optional[T], values: Dict[str, Any]) -> Optional[T]:
        type: Type[BaseModel] = cls.__fields__["parsed"].type_.__args__[0]
        try:
            v = type.parse_obj(values["raw_data"])
            values["errors"] = []
        except ValidationError as exc:
            v = None
            values["errors"] = exc.errors()
        return v

    @validator("errors", always=True, pre=True)
    def handle_errors(cls, v, values):
        return values["errors"]

user_data_1 = dict(id='123', name='John Doe', gender='other than other', registration_date='broken')
user_data_2 = dict(id='123', name='John Doe', gender='male')

error_model_1 = ErrorModel[User](raw_data=user_data_1)
error_model_2 = ErrorModel[User](raw_data=user_data_2)
print(error_model_1)
print(error_model_1.parsed)
print(error_model_1.errors)
'''
ErrorModel[User] raw_data={'id': '123', 'name': 'John Doe', 'gender': 'other than other', 'registration_d… errors=[{'loc': ('gender',), 'msg': 'value is not a valid enumeration member', 'type':… parsed=None
None
[{'loc': ('gender',), 'msg': 'value is not a valid enumeration member', 'type': 'type_error.enum'}, {'loc': ('registration_date',), 'msg': 'invalid datetime format', 'type': 'type_error.datetime'}, {'loc': ('registration_date',), 'msg': 'value is not none', 'type': 'type_error.none.allowed'}]
'''
print(error_model_2)
print(error_model_2.parsed)
print(error_model_2.errors)
'''
ErrorModel[User] raw_data={'id': '123', 'name': 'John Doe', 'gender': 'male'} errors=[] parsed=<User id=123 name='John Doe' gender=<Gender.male: 'male'> registration_date=Non…
User id=123 name='John Doe' gender=<Gender.male: 'male'> registration_date=None
[]
'''

Personally when I want to do this I've used validate_model(model, raw_data, raise_exc=False) directly.
This is why I added the raise_exc argument I think.
Personally I'm not that keen on this, however I agree it would be very simple to add a raise_exc property to config.
Let's leave this open and see if anyone else wants it.

Linked to #1864

Linked to #1864

Even if you validate your object in its creation you could easily end up with an invalid object and
before you do something important as writing to a database/file you would want to verify what the
fields of the objects you are writing are still valid.
Example:

class User(BaseModel):
  email: StrictStr

  @validator('email')
  def validate_email(cls, email):
    # Some regular expression checks for a valid email here
    return email

  def change_email(self, new_email: StrictStr):
    self.email = new_email

class RepositoryManager(BaseModel):
  repositoryUsers: List[User]
  writer: object

  def write_users_to_file(self):

    for user in self.repositoryUsers:

      # Validate that user objects have valid fields before writing even if their email has changed.
      user.check()

    self.writer.write_to_file(self.repositoryUsers)

My personal opinion is that user.validate() sounds like a more appropriate name, but that's not so important.

Is there any follow up on this? This feature would be very useful instead of relying on the undocumented methods.

I've been doing some thinking about what an elegant solution to this problem might look like.  If I understand correctly, the essence of the issue is that we're looking for a way to create models with potentially invalid data, describing a resolution mechanism to decide what to do when validation fails.
Building upon @dmontagu's approach of passing around a (model_type, raw_data) pair, I've proposed a reusable construct called a DynamicGenerator in #2866 that takes some raw data and a validator, and can use the validator to decide what to do with the raw data.
I define a DynamicGenerator called FirstSuccessfulOf, which evaluates to its first argument that successfully validates.  We can use this to solve the first problem in this thread in a way that respects @MVrachev's note: that we must eventually end up with an object that succeeds in validation, or raise a ValidationError.
class Gender(StrEnum):
    male: str
    female: str
    other: str
    not_given: str

class User(BaseModel):
    id: int
    name: str
    gender: Gender
    registration_date: Optional[datetime.datetime]

user = User(
        id='123',
        name='John Doe',
        gender=FirstSuccessfulOf('other than other', 'not_given', log_non_fatal = True),
        registration_date=FirstSuccessfulOf('broken', None, log_non_fatal = True),
        __handle_non_fatal__ = print)

For both gender and registration_date, FirstSuccessfulOf will try each of its possible values in turn.  If none of the possible values validate successfully, then a ValidationError is raised.  We can inject behaviour to handle non-fatal errors in __handle_non_fatal__ - in this case, we're simply printing them.
Other things I like about this approach include that is that it's nestable, and works with validators.  Consider the following example:
class A(BaseModel):
    ints: conlist(int, min_items=1)

a = A(ints = FirstSuccessfulOf(
        AllSuccessfulOf(None, lambda: None),
        AllSuccessfulOf(1, None, 3)
    ))

print(a)
# ints=[1, 3]

This expresses that the list ints should be generated through one of two mechanisms: either AllSuccessfulOf(None, lambda: None) or AllSuccessfulOf(1, None, 3). The first, when evaluated, essentially returns the empty list: none of its arguments successfully validated as ints. Then FirstSuccessfulOf validates the result against the validators of A.ints, which fails because the conlist has min_items=1. Then the next mechanism is tried.

Anyone willing to give some feedback on my feature request? :slightly_smiling_face:

I don't know if this is relevant, but this feature would be greatly appreciated when working with SQLModel, which, when creating a model with table=true, disables data verification. It would be great to have a clean way to check the object data before trying to do a write to the database for example.

It's been a while, any updated thoughts on an API that you'd like for this? I think it could be done through something like a generator or weak attributes or similar, but now that we've had a few more years to play with pydantic and work around these sorts of issues, I'm thinking anyone who still cares might have more fleshed out ideas. (I haven't closely reviewed the comments above, maybe the suggestions are reasonable.)

I'm going to close this because it's stale; if anyone cares about this and wants this capability, please open a new feature-request issue with a proposed API

authors: Bobronium HelloEdit MVrachev MrMrRobat PrettyWood adam-discover dmontagu eddsalkield samuelcolvin

repo: pydantic/pydantic


text:
Use khash for Series.value_counts, add raw function to algorithms.py

Get better performance that way. Also optimize for case of known range (a la groupby)

punting on the known range issue for now

repo: pydata/pandas