text:
Pack structs in C bindings

This creates a separate issue out of the latter part of this comment and the one that follows.
When bindings are generated from the C headers in cryptoki-sys, they do so using whatever structure alignment and packing is assumed for the target platform unless packing is made explicit in the headers for bindgen to read. Currently, packing is only specified for Windows and left implied everywhere else.
You can confirm that forcing a 1-byte alignment for structs on other platforms that it  does indeed produce bindings that differ in terms of size and field offsets, and is not the implicit default.
Meanwhile, the PKCS#11 standard (both 2.x and 3.x, Section 2.1) are very clear that

Cryptoki structures are packed to occupy as little space as is possible.  Cryptoki structures SHALL be packed with 1-byte alignment.

This would seem to imply that packing be explicit for all target bindings. But when this is done, several problems arise:

Rust assumes a >1-byte minimum alignment for struct members, which makes referencing anything beyond the 0th item undefined behavior. Each such instance of this (hundreds in the auto-generated tests) produces a valid, unsuppressible compiler warning. This is a known issue for bindgen that doesn't appear to be nearing a solution any time soon.
Tests written for this crate using SoftHSM seg fault. Whether this is at the rust level (dereferencing unaligned addresses) or at the C level (mismatch with struct packing internal to SoftHSM) is unclear. The latter doesn't seem like it should be the case, but the fact that tests are currently passing would seem to be an endorsement of implicit, non-compact alignment.

So, something is incorrect here, but what exactly that is needs to be investigated. Even if it turns out the way the bindings are currently generated is correct, that fact should still be documented conspicuously to avoid further misconception.

Well, this is frustrating.
Long story short: Seg faults I was seeing working on #66 are indeed a packing issue. It just took me a while to get around to confirming it.
I now have a clear picture of why following the spec to use 1-byte structure packing results in seg faults: libsofthsm2 doesn't follow it!
When C_GetFunctionList is called, the returned list struct is the 2-byte library version followed by all the function pointers. With default packing (on presumably all 64-bit Linux) this pads the version out to the width of another pointer. This is contrary to the spec and requires an explicit override. However, libsofthsm2 is using default packing everywhere except Windows, returning this (wrong!) platform-default padding. So if a client sets up their structs correctly, as soon as they try to call into a function from the list, they get a value that's part of the correct address and part an adjacent one, seg faulting on deref.
According to this issue, it seems to have become such a deeply ingrained assumption (originating in pk11-kit) it's now a de facto standard they refuse to fix. :roll_eyes:
So I guess there's a bit of a philosophical question to be answered here: follow the standard or the mob?

Changing the packing for a struct is just a matter of adding an attribute to it at build time, right? Could we add a feature to control that, so that users are free to choose whichever suits them?

That's right. However, I don't think there's a way to test the bindings since there's currently only one way to simulate the device. It would require matching compilation(s) of libsofthsm2. There's a preprocessor hook for that, but the current CI setup uses the image default instead of doing the compilation itself.

Copying this here from the original PR:
It seems that structure packing should be limited to Windows.  This is a known inaccuracy of the spec, and there is a proposal to change this in v3.2.  This was accepted without objection according to these meeting minutes.
Even in older versions of the spec, it seems nobody was actually packing structures on Unix.  So following the spec literally will cause ABI incompatibility with every existing PKCS#11 implementation.

According to this issue, it seems to have become such a deeply ingrained assumption (originating in pk11-kit) it's now a de facto standard they refuse to fix. roll_eyes

AFAIK, it goes back much further than that.  See this thread between TC members for some background.

authors: ionut-arm jhagborgftx vkkoskie


text:
Create contextually appropriate file system access on papi

Need to make a way for extensions to access papi in a controlled way. They need to be force-restricted to accessing only their own project folder and their own user folder and such. Maybe we can make a proxy for each extension that restricts certain functionality based on "attributes" attached to the properties of the services on the papi or something. We also need to consider how to do this for web views as well.
Make a note to extension developers that they can use this in place of fs.
It would be wise to investigate VSCode's context object they pass into the activate function and get an idea how that might be helpful to understand for our design.
Depends on #30, #56

Access their own extension installation folder R
Write to something user-specific - user appdata/local - extension-specific settings R/W
Access their own project-specific file in some way R/W

source: gharchive/issue

repo: paranext/paranext-core

license: MIT

license_type: permissive

license_source: github-api


text:
Design the UI properly

The UI is currently thrown together quickly. it should have a real design

New design under way and being implemented

repo: pareeohnos/ktrade


text:
Implement Access Controls

Implement access controls for page viewing/editing/creation.

Added rudimentary access control in c6d6b1e16e5daa1cb736c98a1a25adf0dd515483

repo: pariahsoft/PyWW

license: mit

license_source: bigquery


text:
Trying to Log with ERROR level and this bug:

Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/handlers.py", line 1478, in _monitor
    self.handle(record)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/handlers.py", line 1461, in handle
    handler.handle(record)
  File "/usr/local/lib/python3.7/site-packages/python_telegram_logger/handlers.py", line 109, in handle
    self.emit(record)
  File "/usr/local/lib/python3.7/site-packages/python_telegram_logger/handlers.py", line 116, in emit
    mode=self.formatter.MODE,
AttributeError: 'Formatter' object has no attribute 'MODE'```

Problem was in my config file: your library doesn't support the formatter:
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        'simple': {
            'format': '%(asctime)s %(name)-12s %(levelname)-8s %(message)s'
        }
    },
    "handlers": {
        "telegram": {
            "class": "python_telegram_logger.Handler",
            "token": Config.TALEGRAM_TOKEN,
            "chat_ids": [Config.TELEGRAM_MONITORING_CHAT_ID],
            # "formatter": "simple",
            "level": "WARNING",
        },
         "console": {
            "class": "logging.StreamHandler",
            "formatter": "simple",
            "level": "INFO",
        },
    },
    "root": {
        "handlers": ["telegram", "console"],
        "level": "INFO",
    }
}```

repo: parikls/python-telegram-logger


text:
Add section "Common Misconceptions"

We should add a section "Common Misconceptions" or "Common (Beginner?) Mistakes" with the things we see in real-world contracts. Such as:

A misconception that we often see is that developers think the smart contract is the actual interface for the user.
That's why we often see mnemonic String types returned (in the worst case verbatim error messages instead of error codes).

We should explain why contract size is important.
We sometimes get questions like "My 100K contract can't be deployed". This is usually due to some third party crate dependency with default features enabled. Oftentimes the size can be reduced by tuning the features of that dep or using the built-in ink! API instead (oftentimes it's crypto/hashing dependencies).

We should explain better why String should be avoided and especially what the alternatives are.
We already have a section How do I use String in my contract? in our FAQ, but it goes in a bit of a different direction. Using String pulls in logic needed for e.g. UTF-8 handling, which one might not need, but it blows up the contract file size. Also, for String dynamic allocation is needed, increasing the file size of the contract as well and being expensive in terms of performance. A use-case I have now seen a couple times is to use String to generate some uuid, we should look into an alternative for doing that.

Can be documented as "Best practices in ink!"
We should have this soon.

authors: SkymanOne cmichi

repo: paritytech/ink-docs


text:
Potential grandpa performance issue

3 validators setup. Due some libp2p issue the connections between nodes are very unstable.
As the result of the unstable connections, the chain stop finalizing. This is fine as I know people are investigating on the libp2p issue.
Now my chain have best block 36722 and finalized 18681.
I copied the db folder of a validator node (synced up to date) and started a new node. It start using 100% CPU on single core and stops responding (no syncing, no RPC, no Ctrl+C to kill).
I did some quick investigation and found out the only thread ImportQueue is busy (using 100% CPU) and all the work are inside this
https://github.com/paritytech/substrate/blob/82744fbb6f4d677f2edfe9d88737c237622c97a4/core/finality-grandpa/src/import.rs#L90
and most of time are in this loop
https://github.com/paritytech/substrate/blob/82744fbb6f4d677f2edfe9d88737c237622c97a4/core/client/src/client.rs#L1173
I am not familiar how to do performance analysis on rust but just few random pause shows most of time are accessing rocks db for block headers to traverse in the chain.
This probably causes libp2p timeout issue because it takes more than 5mins in my cause to finish best_containing call.
I am not familiar with this part of the code, it will be good if someone do #1444 and make sure best_containing can always be completed in a reasonable amount of time.

I believe the issue is that the leaf set was not being pruned when leaves were finalized and therefore it just grew unbound. https://github.com/paritytech/substrate/pull/1871 fixed this.

Can you check if the version you're using contains that PR?

Cool. I did not have that change. I will update substrate.

authors: andresilva xlc

repo: paritytech/substrate


text:
Remove keystore when doing calls and producing a proof

Follow up on #4145
Removes keystore parameter from calls that might be invoked in untrusted environment. I believe we were passing None to all of them anyway.

Needs some fixes :)

source: gharchive/pull-request

authors: bkchr tomusdrw


text:
PyMCon

Hi!
As you may have already seen on Twitter or on PyMC Discourse, we are planning a virtual conference for the PyMC community. All the information is available in the Discourse post.
We are currently looking for conference chairs and volunteers and would be very grateful if you could share the word! We also want to encourage you to, if you are interested and available, apply to be a conference chair.

We have also put together a presentation about chair and volunteer tasks, in case it helps. We will consider chair applications until July 24th, after which I will close the issue, unless you prefer to close it before.

Hi @OriolAbril, thanks for letting me know about this. I'm already volunteering with PyData Global so I don't have capacity to help another conference right now. Good luck though!

authors: OriolAbril parsing-science


text:
🛑 Parti Townhall is down

In f1595d5, Parti Townhall (https://townhall.kr/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Parti Townhall is back up in e411200.

repo: parti-coop/parti-mx-upptime


text:
[Performance] Expand PyQComposedBlock to 2 qubit gates

Modify https://github.com/pasqal-io/qadence/blob/main/qadence/backends/pyqtorch/convert_ops.py#L198 so that its able to merge multi qubit gates acting on the same qubits into a single torch.tensordot operation
Some remarks:

Identify the correct conditions in which multi quibit gates can be merged. Feel free to take inspiration from https://github.com/pasqal-io/horqrux/blob/main/horqrux/apply.py#L73
Add tests which check that your solution gives the same result as the un-merged version
Nice to have: Add some benchmarks which timeit to check how much speedup you achieved for a certain problem

Hi @ducthanh1991, thanks for your interest in this issue. To start it off, it would be easier to consider only merging gates acting on the same qubit support, including the same control qubit.
One option that I am not sure if @dominikandreasseitz had in mind or not is to also merge single-qubit gates together with two-qubit gates if the support of the single-qubit one is included in the two-qubit merger.

Hi @jpmoutinho, @dominikandreasseitz ,
Thanks for the recommendations! I was wondering why it was necessary to defined a parameter qubits for PyQComposedBlock? From my understanding, it takes a list of Primitives to instantiate the class. One could use the method target of the Primitives to define the list of qubits. Or is there something else that I am missing?

Hi @jpmoutinho, @dominikandreasseitz ,
Thanks for the recommendations! I was wondering why it was necessary to defined a parameter qubits for PyQComposedBlock? From my understanding, it takes a list of Primitives to instantiate the class. One could use the method target of the Primitives to define the list of qubits. Or is there something else that I am missing?

@ducthanh1991 sure thats also possible. its just the way we chose to do it at the time. by the way, are you officially taking over the issue?

Hi @dominikandreasseitz, yes I would like to be assigned to this issue.

@dominikandreasseitz , @jpmoutinho, in wich file can I put the test?

Hello @dominikandreasseitz, @jpmoutinho ,
Could you review my work PR 447?

repo: pasqal-io/qadence


text:
Add Objective C implementation

Please, pretty please.
I know there's a way to use swift code inside objective C project, but it usually gets pretty ugly.

Haha. I thought about it. But man, I thought that Swift was the way forward. I don't think I will be doing a swift implementation. Maybe if someone does it for me and does a pull request. But as for now, I do not have the time since this is just my little blow-off-steam-from-work project :)

Ok, I'll go ahead and try to find the least disgusting of Objective C implementations then :) Thanks anyway - for the reply and for apparently great work.

Thanks. Its all good.

any updates?

on ObjectiveC?
Not really.
Managing this is already a lot for me.
Maybe someone can do a pull request.

i wish somebody's me, but i myself got stuck. anyways appreciate your time Sir @patchthecode

authors: dreamBegin patchthecode voodoo-mobile voodoo-rocks

repo: patchthecode/JTAppleCalendar


text:
when i scroll in my calendar days start missing why is this

Hi see below  but when my calendar loads first time its 100% but as i scroll days keep going missing why is this ?

maybe your calendar is high?
¯_(ツ)_/¯

haha.
just kidding.
Can you shoe me your code for your cellForItem function ?

I believe that you are not resetting your cells in that function.
Keep in mind the calendar is a UICOllectionView subclass.
Your cells are being re-used. Just like anyother UICollectionView / UITableView in swift, you have to reset your cells.
closing this issue. let me know if this resolved it.

func calendar(_ calendar: JTAppleCalendarView, cellForItemAt date: Date, cellState: CellState , indexPath: IndexPath) -> JTAppleCell {
let cell = calendar.dequeueReusableJTAppleCell(withReuseIdentifier: "CalendarCell", for: indexPath) as! CustomCalendarCell
cell.dateLabel.text = cellState.text
handleCellSelected(view: cell, cellSate: cellState)
handleCellColor(view:cell,cellSate: cellState)
return cell
}

what's in these 2 functions?
       handleCellSelected(view: cell, cellSate: cellState)
       handleCellColor(view:cell,cellSate: cellState)

Here is the sample code of the examples by the way ->
https://github.com/patchthecode/CalendarTutorial

func handleCellColor(view:JTAppleCell?,cellSate:CellState)
{

    guard let validCell = view as? CustomCalendarCell else {return}

    if cellSate.isSelected{
      validCell.dateLabel.textColor = UIColor.blue
    }
    else {
        if(cellSate.dateBelongsTo == .thisMonth){
            validCell.dateLabel.textColor = UIColor.black
        }else {
            validCell.dateLabel.isHidden = true

        }

}
}
func handleBookedColor(view:UITableViewCell?,status:String)
{
    guard let validCell = view as? SlotTableViewCell else {return}

    if status == "available" {
       validCell.bookingColor.backgroundColor = UIColor.green
    }
    else {
      validCell.bookingColor.backgroundColor = UIColor.red
        }
    }
}

sorry these are what you are missing

do i just do a simple reload of my collection view to rest the collection view ?  as you mentioned cant see that in code you provided . did some digging and found this method
// For instance, it is not enough to only have the first if clause:
if date == someDate {
configureCellViewInThisWay()
}
// You may also need the else clause to fully reset the cell because cells are being reused.
else {
configureCellViewInThatOtherWay()
}
https://patchthecode.github.io/DateCellConfiguration/

validCell.dateLabel.isHidden = true

Where do you have validCell.dateLabel.isHidden = false ?

i dont have a false . should it be in the else
like this
if cellSate.isSelected{
validCell.dateLabel.textColor = UIColor.blue
}
else {
if(cellSate.dateBelongsTo == .thisMonth){
validCell.dateLabel.textColor = UIColor.black
}else {
validCell.dateLabel.isHidden = true
    }

else{
validCell.dateLabel.isHidden = false
}

that false looked like it solved it thanks
any idea how i would go about making past dates in the calendar not selectable but future and current date selectable .
sorry about all the question still new

authors: patchthecode ruwaldl


text:
unchecked or unsafe operations

Please help me. Thanks in advance.
Launching lib\main.dart on sdk gphone64 x86 64 in debug mode...
main.dart:1
√  Built build\app\outputs\flutter-apk\app-debug.apk.
Error connecting to the service protocol: failed to connect to http://127.0.0.1:49875/ONzENH4FU_g=/
Exited (sigterm)

Closing my emulator and restarting fixed it.

repo: patoliavishal/flutter-FCM


text:
default a select option is checked

Hey there,
So by default one option within the select list is checked..How do I load the tree so that none of the items are checked?
Thanks in advance

That should already happen. Do you have an example?

I found my issue.. apparently there is a  'multiple="multiple" ' attribute that needs to be set otherwise it seems to check the first option entry.

authors: netbandit patosai


text:
Create a database based on a diagram

Hello Reviewer,
thanks for taking the time to go through my projects, comments and remarks are welcomed. Here are some actions and results we followed in order to complete the exercise:

created the queries for the database and tables inside the schema_based_on_diagram.sql file; the tables were created according to the diagram provided in the exercise
added indexes for the different foreign keys in each table
Happy review 😉 .

Hi there, we worked on the project together but only one of us wrote the file. But now I also updated the file that says we worked together. Thank you for accepting our apology.

authors: Al-Amin-Khan-Shakil patriciachrysy

repo: patriciachrysy/clinic-database


text:
Adaptive solver ignores step_ts for impulse discontinuous ODE

I'm trying to integrate a jump discontinuous ODE using an adaptive solver (i.e. discontinuous at a specific timepoint).
However, the integration seems to ignore my step_ts:
import diffrax

t0, t1 = 0, 3
jump = 0.5  #  let's jump at this point
step_ts = [t0, jump, t1]

def vector_field(t, y, args):
    return -1.0 * y + 100.0 * (t == jump)  # also tried jnp.allclose but doesn't work either

term = diffrax.ODETerm(vector_field)
solver = diffrax.Tsit5()
saveat = diffrax.SaveAt(ts=step_ts)
stepsize_controller = diffrax.PIDController(
    pcoeff=0.0,
    icoeff=1.0,
    dcoeff=0.0,
    rtol=1e-4,
    atol=1e-6,
    step_ts=step_ts,
)

sol = diffrax.diffeqsolve(
    term,
    solver,
    t0=t0,
    t1=t1,
    dt0=None,
    y0=1,
    saveat=saveat,
    stepsize_controller=stepsize_controller,
)

print(sol.ts)
assert jump in sol.ts
print(sol.ys)

Output:
[0.  0.5 3. ]
[1.         0.6068427  0.04981759]  # should be something much larger since we have a large jump at t=0.5

I'm avoiding jump_ts based on https://github.com/patrick-kidger/diffrax/issues/58, which addressed the case where you have piecewise constant vector fields. However I'm not quite sure what to do when there's a delta-function in the vector field (impulse external control)
Is there a way to do this in diffrax?

Hi, I think what you are trying to do here is not really well-defined. A delta distribution is not something you'll be able to handle directly in any numerical code.
Of course, you can do the standard engineering trick and integrate the differential equation in an epsilon environment around the jump $t_0$ which gives
$$\int_{t_0 - \epsilon}^{t_0 + \epsilon} \dot y  dt = y(t_0 + \epsilon) - y(t_0 - \epsilon) = - \int_{t_0 - \epsilon}^{t_0 + \epsilon} y dt + 100 \int_{t_0 - \epsilon}^{t_0 + \epsilon} \delta(t - t_0) dt \xrightarrow{\epsilon \rightarrow 0} 100 $$
and tells you that $y$ will jump by 100 after passing $t_0$. So what you could do is to integrate the differential equation without the delta distribution numerically until $t_0$ and then continue the integration from $t_0$ onwards with the initial condition given by whatever the first integration produced + 100.

As @jaschau suggests, probably the simplest approach is to just split your problem into two separate diffeqsolves. (Actually a lax.scan over diffeqsolves would be slightly better, as that'll reduce compilation time.) And then apply your desired impulse between the two solves. It's expected that using step_ts wouldn't result in the delta function being noticed, I'm afraid -- what you're trying to do isn't within the normal remit of differential equation solvers.
Other notes:

If your impulse time isn't known statically then you could detect the time using an event, which halts the solve once it occurs.
I would quite like to extend the event interface to support this kind of thing directly. What you're after here is a continuous (i.e. potentially-within-step) non-terminating event. If you're comfortable with the numerics then I'd welcome PRs on this.

[Also, wait, we can use LaTeX on GitHub now? Hurrah!]

That makes sense! Thank you to both of you :) Would be great for diffrax to have non-terminating event handling, especially for the simple case where the timestamps are known. Unfortunately won't be able to code up the feature now but maybe in the future!

authors: KeAWang jaschau patrick-kidger

repo: patrick-kidger/diffrax


text:
Expression transfer

How can I make an expression transfer from a realtime mesh (made from eos) to a loaded mesh (loaded before, but also made from eos)?

Have a look at our recent paper, equation (4). You can transfer the psi's from one face to another.

Thanks, it worked!
Now, another question:
How I improve the expression animations? I want to be able to see the cheeks of the model moving when transfering expressions.

Cool. As to your new question, I guess you need some sort of real-time window to display the model and then linearly interpolate the blendshape weights from the original value to the new value? Not sure really what you mean but this goes beyond this GitHub issue - I'll therefore close it, but feel free to reply nevertheless with your findings!

repo: patrikhuber/eos


text:
Running as sudo in Linux means that the home directory is going to be /root

Running as sudo in Linux shouldn't be required.

Yes, I agree. Will make this configurable for each provider and only require sudo if a used provider needs it.

Should be fixed in the next preview version. Now, either a resource provider can say that it needs administrative permissions, or you can opt-in for it when declaring the resource.
context.Resource<PowerShellScript>("Install Chocolatey")
    .Script("~/install-chocolatey.ps1")
    .RequireAdministrator()
    .Unless("if (Test-Path \"$($env:ProgramData)/chocolatey/choco.exe\") { exit 1 }")
    .After<RegistryKey>("Set execution policy")
    .After<Download>("https://chocolatey.org/install.ps1");

authors: itsme-alan patriksvensson

repo: patriksvensson/cupboard


text:
Pagination with 0 items allows clickable next page button

We'd like to update the pagination component so that when you have 0 items, you cannot click the next page. It causes a new render and will get caught in an infinite loading state in our app.

Hey @johnsonm325, there should be no changes needed in the PatternFly 4 CSS to support this. Would you mind filing an issue in the patternfly-react repo, and closing this one?
https://github.com/patternfly/patternfly-react/issues/new

Yes, sorry about that. Thanks! There is already an issue for it: https://github.com/patternfly/patternfly-react/issues/1949

repo: patternfly/patternfly-next


text:
feat(wizard): add in-page variation

fixes https://github.com/patternfly/patternfly-next/issues/2073

PatternFly-Next preview: https://patternfly-next-pr-2186.surge.sh

@mcoker I think it looks pretty good. My questions have been asked by @matthewcarleton. I would rather see the changes apply to .pf-c-wizard rather than .pf-m-in-page, but as long as we revisit during breaking change period, it looks good to me.

@christiemolloy @matthewcarleton  decreased the font size of the first step name in the toggle.

:tada: This PR is included in version 2.28.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

authors: mattnolting mcoker patternfly-build redallen


text:
Samples Issue

In the samples of piano, reggae and techno samples are sometimes inverted and sometimes even replace each other. Sometimes, I don't even think that this is the music of techno.
I hope this can get fixed as soon as possible.

Ah, not only reggae and techno, all the samples in the piano category is a mess.

I'll look into it.
Maybe I'll get to fix it by the end of the week. Thanks for pointing it out.

Fixed in v1.1.1-beta.

authors: Humanoid-X pattlebass

repo: pattlebass/Music-DJ


text:
Phone Input Extension

Expected Behavior
You can configure the input to accept extensions.
Actual Behavior
It does not allow the extension to be entered in any format.

Hello, can you please give a documentation about what phone extensions are? Do you if this is supported by awesome-phonenumber?
Thanks.

After looking for other issues about extension in other projects (vue-tel-input, react-phone-number-input, awesome-phonenumber), it looks like extensions are complicated to handle, so I won't add this feature sorry.
Maybe the best solution for you is to add another dedicated input for extensions next to the phone input.

authors: jssuttles paul-thebaud

repo: paul-thebaud/v-phone-input


text:
🛑 Taqueria Bonjour (frontend) is down

In 5f48aa2, Taqueria Bonjour (frontend) (https://bonjour.paulynomial.com) was down:

HTTP code: 403
Response time: 63 ms

Resolved: Taqueria Bonjour (frontend) is back up in 169732b.

repo: paulkiernan/paulynomial-uptime


text:
"added" event

Not sure if it's missing in the documentation or just doesn't exist but any chance there's an event we can hook into when a user adds a range?

yes, there is an event 'add'.

authors: jacobkossman paveltyavin

repo: paveltyavin/multirangeslider


text:
Searchbar gets stuck on page reload

Link to reproduction
/
Describe the Bug
Tiny bug...
When you refresh the page after searching through a collection via the searchbar, it gets stuck on that search query. See video below:
https://github.com/payloadcms/payload/assets/52583398/09b95039-803a-408f-8ebb-27b18212e9f4
Like I said, tiny bug, not a big deal. Can be a bit annoying during dev though, since we refresh a lot.
To Reproduce

go into a collection
search via de searchbar
refresh
The input is gone but the collection is still filtered

Payload Version
2.0.8
Adapters and Plugins
No response

Hi @silveltman If this is beginner-friendly I am interested in working on it else, I wouldn't mind shadowing the assignee while working on this, please. Thank you

authors: Nwakaego-Ego silveltman

repo: payloadcms/payload


text:
Payload with getPayloadHMR don't close or spams db connections during Static Site Generation

Link to reproduction
No response
Payload Version
3.0.0-beta.68
Node Version
v20.11.1
Next.js Version
next@15.0.0-rc.0
Describe the Bug
I'm using postgres 12, and static generation with getStatisProps combined with local api of getPayloadHMR
Having 32 statically generated pages I encounter a problem that the last few pages are not generated right away, worker for generation terminates and restarts, finishing page generation. On second screenshot you can see a peak of  postgres connections - max 100 for my system - in idle state, after peak drops, that's when workers are terminated, restarted and finished. This works after some optimizations I've made, because before that, SSG were failing with too many clients from postgres.
Is this an intended behavior and I should scale connections as count of my pages growth or that's going something wrong with how connections are managed in local api?

Reproduction Steps
When I will have time maybe I will try to create an example repo, but rough reproduction:

Get a postgress db - set max_connection to 100 or less.
Create a bunch of pages
Try generate with next build

Adapters and Plugins
db-postgres

Hey @Dovchik — let us check this out on our end and get back to you

OK so here's some more info.
I just did a test with over a hundred pages to statically generate:
import type { Page } from 'payload-types'
import { getPayloadHMR } from '@payloadcms/next/utilities'
import config from '@payload-config'

const Page: React.FC<{ params: { id: string } }> = async ({ params }) => {
  const payload = await getPayloadHMR({ config })

  const page = await payload.findByID({
    collection: 'pages',
    id: params.id,
  })

  console.log(`page ${page.title} rendering`)

  return page.title
}

export default Page

export const generateStaticParams = async () => {
  const payload = await getPayloadHMR({ config })

  const query = await payload.find({
    collection: 'pages',
    limit: 200,
  })

  return query.docs.map((doc) => ({
    id: doc.id,
  }))
}

I've also added a quick console.log('INITIALIZED') to my Payload config's onInit which will log each time that Payload is freshly initialized (thus creating a new database connection).
On build time, I see all of the pages generated (and successfully console logging the page title), but I only get 8 INITIALIZED logs.
Why is there more than 1 INITIALIZED message? Because Next.js tries to run different build steps in different Node.js threads, so that work can be done in parallel. That means that things like getStaticParams will be run in a separate thread, needing to initialize Payload again.
But the INITIALIZED messages don't scale linearly (or at all) related to how many pages I build. I could do more, and the INITIALIZED messages do not go up. That means to me that our internal Payload init memoization layer is working as intended, so something else must be happening with your config / setup.
Can you create a minimally reproducible example so that I can test further?
On a separate note, in production, we're going to try and look into properly disconnecting the database once the process is terminated. It might just be that the connections are not being properly "cleaned up" and staying open, which might add up over time. They should idle and timeout, but we can probably do something about that. Hang tight for that. In the meantime, I'd love to see if you are actually seeing a Payload initialization per static page you're generating. That would be helpful!

@jmikrut I think I had found a culprit
I've created a repo where I got the same behavior I described before, here is the link to the repo: https://github.com/Dovchik/reproduce-connection-overflow/tree/main
All was building smoothly, but then I added generateMetadata, which also fetches payloadHMR, the error happened: https://github.com/Dovchik/reproduce-connection-overflow/blob/3c61431638384aa3dff5a75c2af243c2a607f0c7/src/app/[locale]/pages/[slug]/page.tsx#L11

@Dovchik nice, ok, we're on it

Hey @Dovchik , thanks for the repo and helping get to the bottom of this.
I pulled it down and ran it. On build, it peaked at ~45 connections - still not great but not 100 like you are seeing.
What version of PG are you running?
We're looking into some connection optimization now.

Hey @denolfe , it's postgres v12.18
Now, the build is going good, peaking in 45 for a different project where I initially encountered the problem, and just 5 connections for repo I had posted 😕
For background, I'm running hyper-v VM linux with 30 virtual thread as my machine has 32 available.
I just did an experiment where I reduced virtual threads count available for VM from 30 to 8 and build came through okay
Now, setting it back to 30 and build is also okay ❓ 😕
Also, @jmikrut for repo I posted I get 27 INITIALIZED log in async onInit(payload) for 30 threads, and just 7 when thread count is 8 - which is obvious, meaning node utilized thread to generated in parallel - more thread more parallel processes - more connections.
Anyway, my workaround for local build is reducing thread count, but that's far from ideal, as I need to restart the VM each time when I need multi-thread performance for my other stuff.
Also, If I encounter the problem again I will try to reproduce again, I'll let you know if I will got any success.

Anyway, the bug persist, monitoring my Vercel deployments, gives around 50 active connections, which are not dropped.

Hey @Dovchik , how are you monitoring connections?
I went ahead and deployed the repo to Vercel and connected it up to Vercel Postgres. However, I was only able to see the connections top out at ~20. I was monitoring by running this query every second:
SELECT pid, usename, application_name, client_addr, backend_start
FROM pg_stat_activity;

Anything I might be missing that you're doing differently?

hey @denolfe I use Dashboard in DBeaver for that
Sorry for the confusion, the vercel's connection count is for my other project, not the repo I posted.
When I open page connection count rising, but after some time dropping again, depends on what I do, go to dashboard or other places it may raise to around 50.
If I try to deploy in that moment, I will get failing build as too many connections are floating around. As well as site will start throwing 500

Hi, we are experiencing a similar issue with the DB connection increasing, I wanted to not that we don't use getPayloadHMR we also not using version 3 but 2 with db-postgres:
"payload": "2.14.1",
"@payloadcms/db-postgres": "0.8.5",

With the adapter:
db: postgresAdapter({
    // Ref: https://payloadcms.com/docs/database/postgres#options
    pool: {
      connectionString: env.DATABASE_URI,
      max: env.POOL_MAX_CONNECTIONS, // Default 100
      idleTimeoutMillis: 10000,
    },
  }),

@laurent-boundary Is this during development, when building, or deployed? If deployed, what platform?

We have containerized with docker Payload and Postgres for development, on local environment we can see 17 connections with one user. On prod we quickly see the number of connections ramp up to 100.
Payload is running on node:20.12-alpine, we build it like so:
"build:payload": "yarn payload build",
"build:server": "tsc",

On prod we deploy to AWS this image on a EC2 Fargate.
DB wise,  for dev we spin up this docker image postgres:16-alpine on AWS we have an RDS with postgres

We have a similar issue and it is a critical issue that prevents us from using payload. We are using Payload v3, PostgreSQL, and Vercel. The issue occurs when we send about 20 parallel requests to Payload. We see "too many clients already" errors in the logs. Yo can reproduce the issue by sending different number of requests. Even a media collection page that sends many requests for images can produce this issue.

For everyone in this thread - we are continuing to look into this. One thing to try in the meantime is to see if the excess connections are caused by transactions.
Can anyone here try to disable transactions by passing transactionOptions: false to your postgresAdapter? We will also try this on our end today.

I will try the transactionOptions: false on my site to see if it resolves anything. I've also been running into this issue on a static blog page setup where it creates 400+ connections after navigating to a statically generated blog page. The build process does not give errors.
Note I'm using the standard MongoDB connection setup and fetching blog metadata from payload during the generateMetadata step.

@jmikrut transactionOptions do not exist for postgres adapter it seems, at least typescript do not allow.

@laurent-boundary you will likely need to update your @payloadcms/db-postgres version in order to get access to the transactionOptions property

We recently updated to the latest, that seems to still be the latest, that was our first idea to fix the problem :

I just updated Payload to the latest 2.25.0 version as well:

While I still couldn't have typescript accepting, I tried to force it with ts-ignore

I then tried to refresh the page a few time and navigate through collections (not even adding anything to the DB) and still have a spike of connections (locally with one user, myself)

Ah @laurent-boundary if TS says that property isn't there then it definitely isn't gonna work. I think I led you astray earlier - transactionOptions was released solely for the 3.x Postgres adapter.

All of that happen while running, not building, (we use version 2 of Payload not v3) that's why I'm not sure if it's fully related.

I think this is correct. It is likely not related to any v2 -> v3 changes and is likely more linked to transactions in general. Let us backport transactionOptions to 2.x so that you can test with that. Would be good to know if that fixes it for you!

Amazing, I'll try that when ready :) Thanks 🙏

@jmikrut in the meantime I created a repro for payload v2, I used npx create-payload-app@latest with blank and postgres options.
Multiple thing to note:

postgres option do not work out of the box, it comes with mongo in the docker compose
I was going to add a media collection because it seems to increase the issue but just refreshing the page create 7 connections

Hey @laurent-boundary , thank you for the reproduction repo.
Running your docker-compose file without payload at all, produces a baseline connection count of 6.
When adding payload back into the mix and clicking around, I was only able to bring the connection count up to 9. This makes me think this is a non-issue (for v2 at least) and just a misunderstanding of the baseline connection count.
That being said, the pool options of the db adapter can accept a max value if you also want to pass that. This is what it would look like. You'll need at least 2 from my testing.
  db: postgresAdapter({
    pool: {
      connectionString: process.env.DATABASE_URI,
      max: 2,
    },
  }),

On production in AWS we use postgres through RDS AWS managed DB (not docker), we spike at 100 connections, and that's definitely linked to Payload, I'll check that and complete the repro until I see something similar.
We tried the max with one unsuccessfully, but we will try 2 and see.

I tried the max: 2 it works for a bit and then when trying to access the admin page, we get an infinite loading.

any update here?
Seems like it's related: I am using s3 bucket. If disable images on chrome level - level of connections is ~5 all the time. But once images are enabled, simple opening the website causes 300+ connections to mongoDB. In my page I fetch globals with few objects and ~4-5 images + 2 collections with 6+8 images.
here is my config:
export default buildConfig({
  admin: {
    user: Users.slug
  },
  collections: [
    Users,
    Media,
    Category,
    Products,
    Orders,
  ],
  globals: [Settings],
  editor: lexicalEditor(),
  localization: {
    locales: ['en', 'uk'],
    defaultLocale: 'uk',
    fallback: true
  },
  i18n: {
    supportedLanguages: { en, uk },
    defaultLanguage: 'uk'
  },
  secret: process.env.PAYLOAD_SECRET || '',
  typescript: {
    outputFile: path.resolve(dirname, 'payload-types.ts')
  },
  db: mongooseAdapter({
    url: process.env.DATABASE_URI || '',
    connectOptions: {},
    transactionOptions: false
  }),
  maxDepth: 2,
  sharp,
  plugins: [
    s3Storage({
      collections: {
        media: true
      },
      bucket: process.env.S3_BUCKET,
      config: {
        credentials: {
          accessKeyId: process.env.S3_ACCESS_KEY_ID,
          secretAccessKey: process.env.S3_SECRET_ACCESS_KEY
        },
        region: process.env.S3_REGION
      }
    })
  ]
})

Hey @dmytro-lishchenko — this is interesting. Here are some of my findings.
When we go to stream images located in S3 from our Payload upload file handler, we do indeed need to make a query to the database in order to do things like run access control, etc.
https://github.com/payloadcms/payload/blob/main/packages/plugin-cloud-storage/src/utilities/getFilePrefix.ts#L11
As written, this was creating a new transaction for each image loaded. Depending on configuration, I believe a new transaction has the potential to create a new connection.
We've recently updated our transaction logic on 3.x beta to no longer initiate transactions on read (we only do them on create / update / delete now). Are you on v2 or v3? It might be worth updating to v3 to see if this problem improves for you or if it stays the same.
Where are you deploying Payload?

OK so we've done a LOT of digging on this and have come up with the following learnings.
The number one most important thing to get to the bottom of is to find out if Payload is connecting to the database multiple times.
You should be able to determine this by adding a console.log to your config.onInit method, indicating "INITIALIZED" or similar. Everything that we've tried thus far shows that Payload is, as expected, only connecting to the database when it needs to and is already properly caching database connections. You might see that your  process does indeed connect multiple times, but that's probably due to multiple CPU threads or Next.js inner-workings as I described here.
Outside of what Payload is already doing, I really don't think that there's much we can do further in order to cache the connection / ensure that it is re-used accordingly.
That being said, we still want to provide solutions to everyone here, and our solutions will depend on where and how you are hosting your project.
Vercel / Serverless hosts
On Vercel or any serverless host, with traditional database connections, it's relatively normal to see a bunch of connections that scale linearly based on how many serverless functions are started for your application. Each serverless function will spawn its own connection, and as your site has more and more lambda functions spun up to serve traffic, you'll see more database connections. This is what I think is the case with @dmytro-lishchenko and anyone else in this thread experiencing this issue while deploying to Vercel.
Some database vendors like Supabase and I believe Vercel Postgres will indeed create mechanisms to handle this such as pgBouncer and similar.
What we plan to do for serverless hosts is to create serverless-friendly HTTP adapters, which will mitigate this by not opening a traditional database connection and solely fetching data from HTTP rather than going through TCP / similar.
Then, if you deploy on Vercel, you should use a serverless-friendly DB, or use one of our HTTP adapters.
Long-running servers still experiencing this problem
Some of you in this thread appear to be experiencing these connection counts increasing while deploying on traditional services like AWS Fargate (@laurent-boundary) and honestly this completely baffles me. I would LOVE to get to the bottom of this. We can't recreate any of these scenarios though and will need a solid reproduction in order to help here, as all our tests show a stable connection count which does not rise. Note that it's normal to have more than 1 connection in a long-running server but the connection number should not rise exponentially.
@laurent-boundary do you have a way of knowing how many Fargate containers are spun up to handle your traffic? If you have lots of Fargate containers, your issue may be similar to the Vercel / serverless information I just described above. Connections will scale linearly with containers and if you have many, you may need to convert to an HTTP adapter like I described above as well.
Tips to mitigate this
As @dmytro-lishchenko pointed out, you can largely limit the amount of connections that are created by fine-tuning your connection options (like mentioned here). I would suggest that anyone experiencing this tries to dial in their connection options to see if that helps put a lid on the connection count until we can further diagnose.
Conclusion
What we are most interested in at this point is a consistent reproduction of connection counts spiking on long running servers - i.e. not Vercel or serverless hosts because that is relatively expected. We will keep this issue open until we can release HTTP database adapters for serverless environments.
More from us shortly!

@jmikrut  Hi, I am in the same team with Laurent and I'll follow this issue.
First of all, thank you for the reply.
We are only use a single container at the moment on AWS, our setup for this application is a single cluster with a single service running a single task, and we don't have much traffic in our web application as we only had a single client in PROD using it when we experienced the issue.
The highest amount of concurrent users we had was less then 20 when everyone at the office was using it in STG.
We tweaked the connections configurations a few times but every time it still reached the max number of the pool size and connections we had set, it only took more time to get to that point but in the end it got to that point, and when that happens we get the behavior that Laurent said where the applications stops responding cause the DB connections reached the max.

Hey @bm-spencer that sounds bad. Absolutely no clue what would be causing that.
Can you try to set this property on your pool config options to see if that resolves the issue?
idleTimeoutMillis: 30000

Going to continue looking into this. Are you deploying only Payload 2.0, without Next.js running on the same server? Or are you deploying Next.js as well?

@jmikrut Hey, we've already set that to idleTimeoutMillis: 10000.
This is our config.
  db: postgresAdapter({
    // Ref: https://payloadcms.com/docs/database/postgres#options
    pool: {
      connectionString: env.DATABASE_URI,
      max: env.POOL_MAX_CONNECTIONS, // 500;
      idleTimeoutMillis: 10000,
    },
  }),

We set POOL_MAX_CONNECTIONS to 500 as lower values make the application crash earlier when they reach the max pool size and the connections are not closed in time.
Our server.js looks something like this.
// server.js
const app = express();

app.use(/* middlewares */)

const start = async (): Promise<void> => {
  await payload.init({
    secret: env.PAYLOAD_SECRET,
    express: app,
    onInit: () => {
      payload.logger.info(`Payload Admin URL: ${payload.getAdminURL()}`);
    }
  });

  app.listen(env.PORT);
};

start();

We are using
// package.json
  "@payloadcms/bundler-webpack": "1.0.7",
  "@payloadcms/db-postgres": "0.8.5",
  "@payloadcms/plugin-sentry": "0.0.6",
  "@payloadcms/richtext-slate": "^1.0.0",
  "drizzle-kit": "0.22.7",
  "drizzle-orm": "0.31.2",
  "express": "^4.17.1",
  "payload": "2.14.1",

and we run the server with this command yarn serve
// package.json
"serve": "cross-env PAYLOAD_CONFIG_PATH=dist/payload.config.js NODE_ENV=production node dist/server.js"

So I don't think it would restart the server if it crashes.

Hey @jmikrut we're having similar issues in production to @bm-spencer on a very similar setup both infrastructure (ECS + AWS RDS) and version wise
While trying to get to the bottom of our problems we added a log of the pool waiting size payload.db.pool.waitingCount to our healthcheck endpoint (which is performing a simple count of user records).
This sits at 0, but once or twice a day inexplicably skyrockets to thousands, before the checks fails and a new instance is deployed

Alongside seeing db connection spam (I work with Danielfrenda), we have also seen some issues with unclosed transactions / idle in transactions that could be contributing to the pool freezing up and our instance becoming unresponsive.

Could also be related to whatever is happening here?

To add to the above, the default idle transaction timeout on RDS is 24hrs. When this is reached and the transaction is terminated the resulting error throw is not caught by payload and will cause the process to exit. This is also the case if those transactions are killed manually

We also encountered the memory leak issue when connecting to a MongoDB ReplicaSet but not when connected to a non-ReplicaSet. What fixed our issue was to downgrade the MongoDB server (self-hosted) from v7.0.0 to v6.0.9
Before we arrived to the fix:

We tried disabling transactions by setting transactionOptions: false,, did not fix the memory leak but it did fix the "Too much open connections" issue.
Our onInit logs were only printing once, so the DB adapter did not create multiple connections.
We tried shortening the timeouts like above but this only increased the rate of memory churn, so we reverted back.
Saw this compatibility table from Mongoose and noticed that @payloadcms/db-mongodb: 1.7.1 is using "mongoose": "6.12.3",, which is only compatible up to MongoDB server 6.x

Here's our current working setup:
DB Server: MongoDB v6.0.9
  // package.json
  "dependencies": {
    "@payloadcms/db-mongodb": "^1.7.1",
    "payload": "^2.25.0",
  }

// payload.config.ts
export default buildConfig({
  // ...
  collections: [Users, Posts],
  db: mongooseAdapter({
    url: 'mongodb://host1.mydomain.com:60200,host2.mydomain.com:60200,host3.mydomain.com:60200/myDbName',
    connectOptions: {
      auth: {
        username: 'myUsername',
        password: 'myPassword',
      },
      authSource: 'myAuthSource',
      tls: true,
      tlsCertificateFile: process.env.TLS_CERTIFICATE_FILE,
      tlsCertificateKeyFile: process.env.TLS_CERTIFICATE_KEY_FILE,
      tlsCAFile: process.env.TLS_CA_FILE,
      replicaSet: process.env.REPLICA_SET_NAME,
    },
    transactionOptions: false,
  }),
});

Here's a graph of the memory usage (encircled is when the MongoDB version downgrade was applied)

After 1 hour, the application used up 1GB of memory (from 180mb) and would crash due to the limits in our Openshift cluster.
After applying the downgrade, the memory usage stabilizes at 180mb

Payload is using mongoose 6.12.3
Mongoose 6.12.4 has release notes that upgrade mongodb driver to 4.17.2
Release notes: https://github.com/Automattic/mongoose/blob/53d382b0f9df9590fcb1c1468de38dbeb56b7f03/CHANGELOG.md#6124--2023-12-27
Mongodb driver 4.17.2 has release notes about a connection leak issue
Release notes: https://github.com/mongodb/node-mongodb-native/releases/tag/v4.17.2
Not sure if it's relevant, but wanted to mention it.
We also see high connection usages with our mongodb instances at mongodb.com, but have not narrowed it down.

Same problem here, the number of connections keep increasing with every request and does not seem to go down. If I terminate them (manually or set idle timeout), payload exits with the following error.
error: terminating connection due to administrator command
    at u.parseErrorMessage (/.next/server/chunks/416.js:139:49786)
    at u.handlePacket /.next/server/chunks/416.js:139:46444)
    at u.parse /.next/server/chunks/416.js:139:45090)
    at Socket.<anonymous> /.next/server/chunks/416.js:139:41858)
    at Socket.emit (node:events:519:28)
    at addChunk (node:internal/streams/readable:559:12)
    at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)
    at Readable.push (node:internal/streams/readable:390:5)
    at TCP.onStreamRead (node:internal/stream_base_commons:191:23)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
  length: 116,
  severity: 'FATAL',
  code: '57P01',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'postgres.c',
  line: '3304',
  routine: 'ProcessInterrupts'
}
Error: Connection terminated unexpectedly
    at h.<anonymous> (/.next/server/chunks/416.js:203:15615)
    at Object.onceWrapper (node:events:633:28)
    at h.emit (node:events:519:28)
    at Socket.<anonymous> (/.next/server/chunks/416.js:203:27108)
    at Socket.emit (node:events:519:28)
    at TCP.<anonymous> (node:net:338:12)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17)

I'm running the program with pnpm build && pnpm start on a bare metal server using pm2:
After a little digging I realized that I had a collection hook that creates it's own transaction (separate from the request's transaction) and I forget to commit it in all cases (there were premature returns in the hook and I did not commit those transaction if it early returns).
const separateTransaction= async ({
  doc, // full document data
  payload,
}: {
  doc: AppUserLog;
  payload: Payload;
}) => {
  const req = {
    payload,
    transactionID: await payload.db.beginTransaction(),
  } as PayloadRequest;
  try {
    if (true){
        // premature return without committing
        return;
    }
    if (
      req.transactionID &&
      typeof req.payload.db.commitTransaction === "function"
    ) {
      await req.payload.db.commitTransaction(req.transactionID);
    }
  } catch (e) {
    req.payload.logger.error(e);
    if (
      req.transactionID &&
      typeof req.payload.db.rollbackTransaction === "function"
    )
      await req.payload.db.rollbackTransaction(req.transactionID);
  }
};

@coagmano I'm dealing with the exact same issue you described above with the idle transactions and the unresponsive CMS.
Did you ever found a way to prevent/mitigate this from happening?

@tiago-meireles
We worked around the issue by setting query timeouts in the config for the postgres adaptor and then handling the error when the connection is killed.
db: postgresAdapter({
    migrationDir: path.resolve(__dirname, 'migrations'),
    pool: {
      application_name: 'payload-cms',
      connectionString: process.env.PAYLOAD_POSTGRES_URI,
      idleTimeoutMillis: 20000,
      idle_in_transaction_session_timeout: 20000,
      query_timeout: 20000,
      // @ts-expect-error - lock is not defined in the types - however supported in pg library
      lock_timeout: 10000,
      max: 20,
    },
}),

The timeout that actually resolved this specific issue was idle_in_transaction_session_timeout.
But it was still throwing an unhandled error from within pg which would kill the server, so we also needed to add a global unhandled exception handler that didn't kill the process for these exceptions.
process.on('uncaughtException', (error) => {
  if (error.message && error.message.includes('Connection terminated unexpectedly')) {
    // Exception when Postgres terminates the connection due to idle_in_transaction_session_timeout
    console.error('PostgreSQL connection error:', error);
  } else if (error.message && error.message.includes('idle-in-transaction')) {
    // Exception when pg's idle_in_transaction_session_timeout expires
    console.error('PostgreSQL idle-in-transaction error:', error);
  } else {
    console.error('Uncaught Exception thrown', error);
    process.exit(1);
  }
});

We also did some optimization of db queries for RichText / LexicalHTML fields by skipping the HTML serialization hooks in GraphQL queries unless the HTML field was specifically requested. This allowed the sitemap to render in 280 - 360ms instead of 1.1 - 4.4+ seconds - requiring an order of magnitude fewer db connections.
Here's a snippet from our util function for defining CollectionConfigs
const serializedHTMLField = lexicalHTML(field.name, { name: `${field.name}_html` });
// ...
        {
          ...serializedHTMLField,
          required: field.required,
          defaultValue: 'placeholder',
          validate: () => true,
        },
          hooks: {
            afterRead: [
              async (args) => {
                // Prevent serializing HTML field in GraphQL API unless explicitly requested
                if (
                  args.req.payloadAPI === 'GraphQL' &&
                  !args.req.body?.query.includes(serializedHTMLField.name)
                ) {
                  return null;
				}
                // Directly call the afterRead hook to serialize the HTML field
                return serializedHTMLField.hooks?.afterRead?.[0]?.(args);
              },
            ],
          },
        } as TextField,

And finally, we now run 2 instances of payload on ECS with the minimum allowed health check duration, so if it does die the failover happens faster.

Hi, we have the same issue in production (deployed in Vercel, latest beta, using a free MongoDB Atlas cluster). The connections in the screenshot below happen when I just load our landing page and browse a couple of other content pages. This obviously impacts site responsiveness negatively, not to mention hitting the connection limits in MongoDB Atlas.

Has any MonboDB user figured out a workaround for this issue? Or maybe any news on the HTTP adapters mentioned in @jmikrut's post?
If we deploy the software in Google Cloud Run, we have no issues, but we'd like to use Vercel instead.

any update?

We found a potential issue with Virtual field usage in our case. We have nested relationships that use virtual fields to look up the relations between the document.
We implemented based on this document: https://payloadcms.com/blog/learn-how-virtual-fields-can-help-solve-common-cms-challenges
Every time an API request is made, the virtual fields look ups are triggered which initiates the recursive DB queries instantly and hits max connection count. Once we disable the virtual fields. The connection seems stable.

If anyone still has this issue, can they try to play around with these options https://nextjs.org/docs/app/api-reference/next-config-js/staticGeneration? I believe they should help here.
const nextConfig: NextConfig = {
  experimental: {
    staticGenerationRetryCount: 1,
    staticGenerationMaxConcurrency: 8,
    staticGenerationMinPagesPerWorker: 25,
  },
}

@r1tsuu I was running into the too many clients... issue, and setting staticGenerationMaxConcurrency to 2 resolved the issue for me.
These setting worked:
    // ...
    experimental: {
      staticGenerationRetryCount: 1,
      staticGenerationMaxConcurrency: 2,
      staticGenerationMinPagesPerWorker: 25
    },

Thank you!

@Dovchik,
We don't know the cause of this still. If anyone has more information they'd like to share as to what they've seen and done to fix it, that could be helpful.
I'm trying to decide what to do with this issue. Close it? Move it to a discussion? Keep it?

authors: DanRibbens Dovchik WilsonLe abid-salahi andershermansen arime-pareto bm-spencer coagmano danielfrenda denolfe dmytro-lishchenko erdidoqan greenlover1991 jmikrut laurent-boundary lynndylanhurley r1tsuu tiago-meireles


text:
Using env variables in prefix string comes out as "undefined"

Update
This apparently is also an issue in other plugins. Using environment variables as part of the plugin configuration is not possible.

Original post
Following will return undefined/media as prefix in API calls, however images will be uploaded to the directory path in S3 using the prefix value from .env. So you will have to manually generate the file URL to inject the env variable (which works for some reason).
export default buildConfig({
  serverURL: process.env.PAYLOAD_PUBLIC_SERVER_URL,
  collections: [Page, Media],
  typescript: {
    outputFile: path.resolve(__dirname, 'payload-types.ts'),
  },
  graphQL: {
    schemaOutputFile: path.resolve(__dirname, 'generated-schema.graphql'),
  },
  plugins: [
    cloudStorage({
      collections: {
        media: {
          adapter, // see docs for the adapter you want to use
          disablePayloadAccessControl: true,
          prefix: `${process.env.S3_PREFIX}/media`,
          // generateFileURL: (args) => `https://s3.${process.env.S3_REGION}.amazonaws.com/${process.env.S3_BUCKET}/${process.env.S3_PREFIX}/media/${args.filename}`,
          disableLocalStorage: true,
        },
      },
    }),
  ],
});

Adapter for reference:
const adapter = s3Adapter({
  config: {
    forcePathStyle: true,
    region: process.env.S3_REGION,
    endpoint: `https://s3.${process.env.S3_REGION}.amazonaws.com`,
    credentials: {
      accessKeyId: process.env.S3_ACCESS_KEY_ID,
      secretAccessKey: process.env.S3_SECRET_ACCESS_KEY,
    },
    // ... Other S3 configuration
  },
  bucket: process.env.S3_BUCKET,
});

Because it returns undefined/media but the file is actually stored in my-prefix/media, then the media files aren't deleted automatically from the bucket when media items are deleted in cms.

Closing this issue as I found the previous thread discussing this matter: https://github.com/payloadcms/payload/issues/1654#issuecomment-1356809053

repo: payloadcms/plugin-cloud-storage


text:
IPN Response Invalid

Hello,
I keep receiving an Invalid IPN: cmd=_notify-validate error and can't find any documentation on how to fix this. It seems like none of my $_POST values are actually being sent to my IPN listener.
Here's my form:
<script async="async" src="https://www.paypalobjects.com/js/external/paypal-button.min.js?merchant=z129043-facilitator-1@niu.edu"
data-button="buynow"
data-name="ezLeague credits"
data-quantity="<?php echo $purchase_amount; ?>"
data-amount="1"
data-currency="USD"
data-callback="http://www.ezleaguegaming.com/demo/payment-complete.php"
data-env="sandbox"
data-type="form"
></script>

and then i'm using the PHP IPN Listener example from PayPal with no changes made:
https://github.com/paypal/ipn-code-samples/blob/master/paypal_ipn.php
I don't understand why none of my fields are being sent to the listener. Help is greatly appreciated. Thanks.
Michael

spoke to PayPal and they were having issues with their Sandbox Environment the past 48hours. there was no issue with my code, the problem was on their end.

repo: paypal/JavaScriptButtons


text:
Addition to file reading section

Add catalog of common file formats to read/parse

Also add example of map functions to iterate and read in multiple files in one folder

Lesson 10 uses map_dfr() so it would be good to explain those functions before getting to the final project.

I have it covered in lesson 3.
-Patrick

From: rkjulian notifications@github.com
Sent: Saturday, September 8, 2018 5:53:41 PM
To: pcmathias/MSACL-intermediate-R-course
Cc: Patrick C Mathias; Assign
Subject: Re: [pcmathias/MSACL-intermediate-R-course] Addition to file reading section (#19)
Lesson 10 uses map_dfr() so it would be good to explain those functions before getting to the final project.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHubhttps://github.com/pcmathias/MSACL-intermediate-R-course/issues/19#issuecomment-419653087, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACvHKRiZPr72Lz-SlItSVg5VKeWqgiciks5uY-gFgaJpZM4WT4YU.

I merged my branch into master with no conflicts, so your merge should now include all my new stuff.
Randy
From: Patrick Mathias notifications@github.com
Sent: Saturday, September 8, 2018 11:56 AM
To: pcmathias/MSACL-intermediate-R-course MSACL-intermediate-R-course@noreply.github.com
Cc: Randall Julian rkjulian@indigobio.com; Comment comment@noreply.github.com
Subject: Re: [pcmathias/MSACL-intermediate-R-course] Addition to file reading section (#19)
I have it covered in lesson 3.
-Patrick

From: rkjulian <notifications@github.commailto:notifications@github.com>
Sent: Saturday, September 8, 2018 5:53:41 PM
To: pcmathias/MSACL-intermediate-R-course
Cc: Patrick C Mathias; Assign
Subject: Re: [pcmathias/MSACL-intermediate-R-course] Addition to file reading section (#19)
Lesson 10 uses map_dfr() so it would be good to explain those functions before getting to the final project.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHubhttps://github.com/pcmathias/MSACL-intermediate-R-course/issues/19#issuecomment-419653087, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACvHKRiZPr72Lz-SlItSVg5VKeWqgiciks5uY-gFgaJpZM4WT4YU.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/pcmathias/MSACL-intermediate-R-course/issues/19#issuecomment-419653235, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AArk23bU5mw7_dmMAy5dD9djBypylPblks5uY-h8gaJpZM4WT4YU.

authors: pcmathias rkjulian

repo: pcmathias/MSACL-intermediate-R-course


text:
GetAll feature doesn't use plural

When scaffolding en entity with an GetAll-Feature, the feature name doesn't use the plural. Is this intended or a bug?
For example when having an Orders Entity with an GetAll Feature, I would expect that the Feature would be called GetAllOrders instead of GetAllOrder

Steps to reproduce

Further technical details
Craftsman version (dotnet tool list -g): 0.23.0

Nice catch thanks. I'll get this in a patch or the next release


text:
[tool.pdm.version] source = "scm" breaks when package is installed from git

pyproject.toml:
[project]
name = "test"
version = "0.0.0"
requires-python = "~=3.8"
dependencies = [
  "amaranth @ git+https://github.com/whitequark/amaranth@pdm-build"
]

After installing:
$ pdm install; pdm list
All packages are synced to date, nothing to do.
Installing the project as an editable package...
  ✔ Update test 0.0.0 -> 0.0.0 successful

🎉 All complete!

╭────────────┬──────────────────────┬─────────────────────────╮
│ name       │ version              │ location                │
├────────────┼──────────────────────┼─────────────────────────┤
│ MarkupSafe │ 2.1.3                │                         │
│ pyvcd      │ 0.4.0                │                         │
│ amaranth   │ 0.1.dev1323+ga0a3c07 │                         │
│ test       │ 0.0.0                │ -e /home/whitequark/tmp │
│ Jinja2     │ 3.1.2                │                         │
╰────────────┴──────────────────────┴─────────────────────────╯

The version is 0.1.dev1323+ga0a3c07 but it should be 0.4.devSomething.
Note that setuptools_scm handles this correctly.

It coudn't find any git tags on the repo https://github.com/whitequark/amaranth
How can it be 0.4.devSomething?

Oh, you're completely right. I extracted this from a larger example that used a pull request, but the exact same thing happened there. Sorry for the false alarm!

authors: frostming whitequark

repo: pdm-project/pdm-backend


text:
🛑 idpsalientup is down

In 3d01a86, idpsalientup (https://indirectpro.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: idpsalientup is back up in 1d0fc4a.

repo: pearanalytics/uptime