text:
DateTime is typed as Date but actual value is string

Bug description
DateTime is typed as Date but actual value is string
DateTime gets typed as Date in generated prisma client while return value is string
How to reproduce

Go to https://github.com/savaparoski/prisma-wrongly-typed
Clone repo
Install dependencies
Run docker compose up
Generate prisma client
Run ts-node index.ts

Expected behavior
Expected that DateTime maps to string type.
Prisma information
https://github.com/savaparoski/prisma-wrongly-typed/blob/master/schema.prisma
Environment & setup
https://github.com/savaparoski/prisma-wrongly-typed
Prisma Version
4.6.1

The actual value isn't a string. Not sure what your example repo is supposed to prove. Are you saying because console.log is converting the Date to a string when logging it's of the string type?
console.log will stringify objects. It does not mean the objects you log are of type string.
To see the actual type perhaps try :
  console.log(typeof(findRes?.createdAt))
  console.log(findRes?.createdAt instanceof Date)
  console.log(findRes?.createdAt.constructor.toString())

True


text:
Prisma CLI command returns "0" as exit code

Describe the bug
Running prisma CLI command like "prisma deploy", always return "0" as exit code.

To Reproduce

Do not start prisma container
Run prisma deploy --force and check we get Could not connect to server at...
Check exitCode with echo $?... it is 0 instead of >=1

Expected behavior
Returned exitCode should be >=1
Screenshots
If applicable, add screenshots to help explain your problem.
Versions (please complete the following information):

OS: Ubuntu 16.04
prisma CLI: prisma/1.15.3 (linux-x64) node-v8.11.4
Prisma Server: 1.15

Additional context
This bug prevent running script like:
set -eux

docker-compose up --detach
timeout --preserve-status 5s sh -c 'until prisma deploy --force; do sleep 1; done'

I am unable to reproduce this one. @brikou Can you please share a sample project (datamodel + primsa.yml) where this is happening? Hooks or something else might be causing this.
Here are the logs from my console.
divyendusingh [prisma-3045]$ prisma deploy
 ▸    Could not connect to server at http://localhost:4467. Please check if your server is running.

Get in touch if you need help: https://www.prisma.io/forum/
To get more detailed output, run $ export DEBUG="*"
divyendusingh [prisma-3045]$ echo $?
1

divyendusingh [prisma-3045]$ prisma deploy --force
 ▸    Could not connect to server at http://localhost:4467. Please check if your server is running.

Get in touch if you need help: https://www.prisma.io/forum/
To get more detailed output, run $ export DEBUG="*"
divyendusingh [prisma-3045]$ echo $?
1

divyendusingh [prisma-3045]$ prisma --version
prisma/1.17.0-beta.24 (darwin-x64) node-v10.4.0

You can close the issue as I cannot reproduce it (I get the right 1 status code). I'll reopen with more reproduction steps if encounter again this. Thanks @divyenduz

I am facing the same issue using different prisma versions (included latest). I am running Prisma inside a docker container.
root@09848c2dd1b4:/app# prisma deploy; echo $?
 ▸    Could not connect to server at http://prisma:4466.
 ▸    Please check if your server is running.

Get in touch if you need help: https://www.prisma.io/forum/
To get more detailed output, run $ export DEBUG="*"
0
root@09848c2dd1b4:/app#

FYI @divyenduz @brikou

source: gharchive/issue

authors: brikou brunocascio divyenduz

repo: prisma/prisma

license_type: permissive

license_source: github-api


text:
Migrate renames my indexes & throws Drift detected error even when nothing changes in DB/Schema

Bug description
I am trying to move an existing Postgres DB with ~190 tables to Prisma. My prisma schema was created successfully by introspecting and I also created an initial migration by taking sql dump of my DB schema. But the problem is that the 2nd time I run npx prisma migrate dev --preview-feature, it creates a new migration which renames my existing unique indexes. Also on each run, I keep getting Drift detected error & need to reset my DB everytime. So the migration sequence is something like this.
Migration 1: Full SQL dump of my existing schema created manually
Migration 2: Auto created by Prisma migrate after 1st run of migrate dev as below (only few sample lines)
-- AlterIndex
ALTER INDEX "django_celery_beat_periodictask_name_key" RENAME TO "django_celery_beat_periodictask.name_unique";

-- AlterIndex
ALTER INDEX "django_celery_results_taskresult_task_id_key" RENAME TO "django_celery_results_taskresult.task_id_unique";

-- AlterIndex
ALTER INDEX "django_rest_passwordreset_resetpasswordtoken_key_f1b65873_uniq" RENAME TO "django_rest_passwordreset_resetpasswordtoken.key_unique";

-- AlterIndex
ALTER INDEX "django_site_domain_a2e37b91_uniq" RENAME TO "django_site.domain_unique";

-- AlterIndex
ALTER INDEX "emailtemplate_emaillayout_name_key" RENAME TO "emailtemplate_emaillayout.name_unique";

-- AlterIndex
ALTER INDEX "emailtemplate_emailtemplate_name_key" RENAME TO "emailtemplate_emailtemplate.name_unique";

On each migrate run - Keep getting Drift detected error. Logs below.
Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma
Datasource "db": PostgreSQL database "masterdb", schema "public" at "nest-postgres:5432"

warn "nativeTypes" is a preview feature. There may be bugs and it's not recommended to use it in production environments.
  MigrateEngine:rpc starting migration engine with binary: /usr/src/app/node_modules/@prisma/engines/migration-engine-linux-musl +0ms
  MigrateEngine:rpc SENDING RPC CALL {"id":1,"jsonrpc":"2.0","method":"devDiagnostic","params":{"migrationsDirectoryPath":"/usr/src/app/prisma/migrations"}} +8ms
  MigrateEngine:stderr Jan 26 06:52:21.738  INFO migration_engine: Starting migration engine RPC server git_hash="e51dc3b5a9ee790a07104bec1c9477d51740fe54" +0ms
  MigrateEngine:stderr Jan 26 06:52:21.792  INFO quaint::single: Starting a postgresql connection. +52ms
  MigrateEngine:stderr Jan 26 06:52:24.259  INFO DevDiagnostic:calculate_drift:sql_schema_from_migration_history: quaint::single: Starting a postgresql connection. +2s
  MigrateEngine:stderr Jan 26 06:52:48.719  INFO DevDiagnostic:validate_migrations:sql_schema_from_migration_history: quaint::single: Starting a postgresql connection. +24s
  migrate:dev {
  migrate:dev   devDiagnostic: '{\n' +
  migrate:dev     '  "action": {\n' +
  migrate:dev     '    "tag": "reset",\n' +
  migrate:dev     '    "reason": "Drift detected: Your database schema is not in sync with your migration history."\n' +
  migrate:dev     '  }\n' +
  migrate:dev     '}'
  migrate:dev } +0ms
? Drift detected: Your database schema is not in sync with your migration history.

We need to reset the PostgreSQL database "masterdb" at "nest-postgres:5432"

How to reproduce
Not sure about this. I can share my complete schema file privately if needed.

Expected behavior
Once the schema is in sync with DB after 1st migration, subsequent runs of migrate should not create any migration and no drift detected error should occur if anything has not changed.

Prisma information
Logs already shared above. Sample schema below.
generator client {
  provider        = "prisma-client-js"
  previewFeatures = ["nativeTypes"]
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

model django_celery_beat_periodictask {
  id                                  Int                                  @id @default(autoincrement()) @db.Integer
  name                                String                               @unique @db.VarChar(200)
  task                                String                               @db.VarChar(200)
  args                                String                               @db.Text
  kwargs                              String                               @db.Text
  queue                               String?                              @db.VarChar(200)
  exchange                            String?                              @db.VarChar(200)
  routing_key                         String?                              @db.VarChar(200)
  expires                             DateTime?                            @db.Timestamptz(6)
  enabled                             Boolean                              @db.Boolean
  last_run_at                         DateTime?                            @db.Timestamptz(6)
  total_run_count                     Int                                  @db.Integer
  date_changed                        DateTime                             @db.Timestamptz(6)
  description                         String                               @db.Text
  crontab_id                          Int?                                 @db.Integer
  interval_id                         Int?                                 @db.Integer
  solar_id                            Int?                                 @db.Integer
  one_off                             Boolean                              @db.Boolean
  start_time                          DateTime?                            @db.Timestamptz(6)
  priority                            Int?                                 @db.Integer
  headers                             String                               @db.Text
  clocked_id                          Int?                                 @db.Integer
  django_celery_beat_clockedschedule  django_celery_beat_clockedschedule?  @relation(fields: [clocked_id], references: [id])
  django_celery_beat_crontabschedule  django_celery_beat_crontabschedule?  @relation(fields: [crontab_id], references: [id])
  django_celery_beat_intervalschedule django_celery_beat_intervalschedule? @relation(fields: [interval_id], references: [id])
  django_celery_beat_solarschedule    django_celery_beat_solarschedule?    @relation(fields: [solar_id], references: [id])

  @@index([clocked_id], name: "django_celery_beat_periodictask_clocked_id_47a69f82")
  @@index([crontab_id], name: "django_celery_beat_periodictask_crontab_id_d3cba168")
  @@index([interval_id], name: "django_celery_beat_periodictask_interval_id_a8ca27da")
  @@index([solar_id], name: "django_celery_beat_periodictask_solar_id_a87ce72c")
}

model django_celery_results_taskresult {
  id               Int      @id @default(autoincrement()) @db.Integer
  task_id          String   @unique @db.VarChar(255)
  status           String   @db.VarChar(50)
  content_type     String   @db.VarChar(128)
  content_encoding String   @db.VarChar(64)
  result           String?  @db.Text
  date_done        DateTime @db.Timestamptz(6)
  traceback        String?  @db.Text
  hidden           Boolean  @db.Boolean
  meta             String?  @db.Text
  task_args        String?  @db.Text
  task_kwargs      String?  @db.Text
  task_name        String?  @db.VarChar(255)

  @@index([date_done], name: "django_celery_results_taskresult_date_done_49edada6")
  @@index([hidden], name: "django_celery_results_taskresult_hidden_cd77412f")
  @@index([status], name: "django_celery_results_taskresult_status_cbbed23a")
  @@index([task_name], name: "django_celery_results_taskresult_task_name_90987df3")
}

Environment & setup

OS: MacOS Catalina 10.15.1, Running inside docker node:14.15.4-alpine
Database: PostgreSQL
Node.js version: v14.15.4
Prisma version:

@prisma/cli          : 2.15.0
@prisma/client       : 2.15.0
Current platform     : linux-musl
Query Engine         : query-engine e51dc3b5a9ee790a07104bec1c9477d51740fe54 (at node_modules/@prisma/engines/query-engine-linux-musl)
Migration Engine     : migration-engine-cli e51dc3b5a9ee790a07104bec1c9477d51740fe54 (at node_modules/@prisma/engines/migration-engine-linux-musl)
Introspection Engine : introspection-core e51dc3b5a9ee790a07104bec1c9477d51740fe54 (at node_modules/@prisma/engines/introspection-engine-linux-musl)
Format Binary        : prisma-fmt e51dc3b5a9ee790a07104bec1c9477d51740fe54 (at node_modules/@prisma/engines/prisma-fmt-linux-musl)
Studio               : 0.340.0
Preview Features     : nativeTypes

Thanks for reporting this! We'll investigate the problem

Thanks for reporting this! We'll investigate the problem

To replicate this issue, below works fine:
datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

generator client {
  provider        = "prisma-client-js"
  previewFeatures = ["nativeTypes"]
}

model ShopLocation {
  id        Int       @id @default(autoincrement())
  name      String    @unique @db.VarChar(32)

  @@map("shop_locations")
}

But when adding
model ShopLocation {
  id        Int       @id @default(autoincrement())
  name      String    @unique @db.VarChar(32)
+ createdAt DateTime  @default(now()) @map("created_at") @db.Timestamptz
+ updatedAt DateTime  @updatedAt @map("updated_at") @db.Timestamptz
+ deletedAt DateTime? @map("deleted_at") @db.Timestamptz

  @@map("shop_locations")
}

And running
npx migrate dev --preview-feature

It asks to create a new migration which generates
-- AlterTable
ALTER TABLE "shop_locations" ALTER COLUMN "created_at" SET DATA TYPE TIMESTAMPTZ,
ALTER COLUMN "updated_at" SET DATA TYPE TIMESTAMPTZ,
ALTER COLUMN "deleted_at" SET DATA TYPE TIMESTAMPTZ;

Version: "@prisma/cli": "^2.15.0"

@pandaniell this is a separate issue. It should already be fixed in prisma@dev, and in today's (to-be-published) release.

@codeslayer1 So, I'm trying to help you out here. I quickly found out that it has way too many moving parts. If you could share your prisma directory with the schema.prisma and the initial SQL dump, it would be easier for me to replicate this bug. You can find me from our public slack or through email bruijn@prisma.io.

Have DMed you on Slack @pimeys

The PR I just made fixes the foreign key re-creation bug. We should never do that.
The other issue is index renaming. That one will not be fixed any time soon. There's a way to not generate these migrations though, take a look into your data model, when you find a column:
foo Int @unique

and you see this specific unique constraint renamed, change the model into different form:
foo Int

@@unique([foo], name: "the_original_name_of_your_constraint")

to prevent index renaming. Or, if you want, just let Prisma rename them. I'll take this into discussion again with our product next week, and I understand this is a bit annoying how it works currently.

Hey @pimeys. I just updated to 2.17 and the rename issue is fixed but the drift detected issue still remains (Prisma says drift detected every time I run migrate dev even if I don't change anything).
Can you please let me know if that was fixed in 2.17 since this issue is closed? If not, Should I reopen this issue or create a fresh issue for drift detected issue?

I suspect there is drift for different reasons (there are a number of changes in 2.17, detailed in the release notes). Please open a new issue with as much detail as possible if you suspect the drift detection is mistaken. Thanks!

@pimeys I just ran into the issue with unique index renaming here: https://github.com/prisma/prisma/issues/5282#issuecomment-774217101
Just wondering if there's an update on getting that fixed, I understand I have a workaround (haven't tried yet, but I assume it will work)

I think the constraint work @do4gr is doing should get this one more streamlined too...

authors: codeslayer1 lastmjs pandaniell pimeys tomhoule


text:
API calls failing with Error querying the database: db error: FATAL: remaining connection slots are reserved for non-replication superuser connections

Bug description
I have been running my app on GCP that connects to a postgres instance also running in GCP, for the past few months. However, since yesterday, my REST API calls are only working intermittently.
I have been getting this error about 50% of the time. Other times, it seems to work fine:
Error querying the database: db error: FATAL: remaining connection slots are reserved for non-replication superuser connections
I have been initializing one prisma instance per endpoint (total 8 endpoints) and till date have had zero issues using connection_limit=80&pool_timeout=50 on the DB string.
I am not sure if it is from have idle open connections that aren't closing or if it is from my prisma queries or some other issue. Can you kindly advise me on how to fix this issue? Any help would be much appreciated. I am unable to reproduce this issue as it is quite sporadic.
Thank you
Default
2021-05-17 17:10:00.614 GMTInvalid `prisma.instance.findMany()` invocation:
Default
2021-05-17 17:10:00.614 GMT
Default
2021-05-17 17:10:00.614 GMT
Default
2021-05-17 17:10:00.614 GMT Error querying the database: db error: FATAL: remaining connection slots are reserved for non-replication superuser connections
Default
2021-05-17 17:10:00.614 GMT at cb (/server/models/prisma-client/runtime/index.js:79166:17)
Default
2021-05-17 17:10:00.614 GMT at processTicksAndRejections (internal/process/task_queues.js:97:5)
Default
2021-05-17 17:10:00.614 GMT(node:14) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 4)
Default
2021-05-17 17:12:00.721 GMT(node:14) UnhandledPromiseRejectionWarning: Error:
Default
2021-05-17 17:12:00.721 GMTInvalid `prisma.events.findMany()` invocation:
Default
2021-05-17 17:12:00.721 GMT

Here is an example of the an endpoint where prisma instance gets initiated:
require('../../config/config');
const express = require('express');
const router = express.Router();

const prisma = require('../models/prismaClient.js');
const sendEmailSummary = require('../functions/sendEmailSummary');

router.post('/', async (req, res) => {
    await sendEmailSummary(prisma);
    res.sendStatus(200);
});

module.exports = router;

And here is the prisma client:
require('../../config/config');
const { PrismaClient } = require('./prisma-client');

const url = process.env.DATABASE_URL;
let prisma;

if (process.env.NODE_ENV === "production") {
  prisma = new PrismaClient({
    datasources: { db: { url } },
  })
} else {
  if (!global.prisma) {
    global.prisma = new PrismaClient({
      datasources: { db: { url } },
    })
  }
  prisma = global.prisma
}

module.exports = prisma;

How to reproduce

Expected behavior
Should be returning values from database
Prisma information

Environment & setup

OS:  MacOS
Database: PSQL
Node.js version:  14
Prisma version: 2.18.0

Prisma Version
2.18.0

Hi @shwethashyam1, the error message you mention in the title indeed means that all connections are in use and no new ones can be opened. Do you maybe have monitoring on that database that can show you the open and idle or active connections, optimally in relation to your database connection limit?
The connection_limit=80 you mentioned, times 8 instances/endpoints would mean that in theory your applications can in parallel open 640 connections. Is that lower than the number of maximum supported connections of your database instance?
Your 8 instances are all running constantly I assume?

Hi janpio, apologies for the late reply! I completely missed this comment. You are right in (the implication) that the # connections may be higher than the number of maximum supported connections of your database instance.
That is exactly the case. My cloudSQL database supports up to 250 connections.
As for the 8 instances, about 5 of them are cron jobs. The remaining three are REST APIs.
Does this mean I will need to reduce the connection limit?

Probably yes, or add some retry middleware that catches the error message, waits a bit and just tries again (this assumes that e.g. the cronjobs make sure to free the connections again after they have been used and do not just keep them open for an hour until the next run). The math should work out in a way that you will not go over the number available.
(Often the cheaper and easier solution is to just increase the limit by paying a bit more though.)

So I figured out what the issue was. Sometimes (maybe 1 in 10 times or less often), when the prisma query is incorrect (for e.g. if I have incorrectly specified a field or a field doesn't exist), prisma is not throwing the error itself.
Instead it opened a lot of prisma connections and doesn't seem to close it -- so subsequent prisma queries ended up throwing the "fatal db.." error. I haven't enforced typescript so that could a way I can prevent incorrect queries to begin with. But perhaps could there be an issue also with prisma's error handling messages?

Absolutely, if you have a reproduction for this it would be amazing. We should really try to fix this of course.

Hi @janpio
struggling to repro this error even though it still happens sometimes (especially now when running my jest tests). Should I close this for now, or keep it open?

Let's keep it open a bit and you just come back to it if you find a reproduction or get more information - otherwise we will close it in a few weeks or months.

Hi @janpio an update: this also seems to happen more when I am running my app on localhost and then deploying it to google cloud after. There is a lot of query-engine-darwin processes that I then manually have to kill to prevent this error from happening in google cloud. Should I try adding ?pgbouncer=true in my connection string?

If you are not using PgBouncer for Postgres, no.
Are you talking to the same database locally as on your deployment on Google Cloud? If you have many local query engine processes running, this would make sense that it blocks a lot of connections. The question then becomes: Why do you have these many local connections open?
If I understand your code snippet above correctly, it should make sure you only have 1 PrismaClient instance running usually that gets reused across the app, not instantiate new ones all over the place. Can you observer how the number of open Query Engine Processes increases?
We currently have a preview feature nApi which uses a different architecture to run the Engine, which should not be in danger of having the same behavior. Maybe you can try that out: https://github.com/prisma/prisma/issues/6301

Yes, the database url is exactly the same and you are right, technically, with the above prisma client in the code snippet, it shouldn't spawn new instances so rapidly.
Running app on localhost w/o pgAdmin 4:  It usually goes up to ~50 query-engine-darwin processes on my activity monitor when the error happens.
Deploying as a container on google cloud: Sometimes I have about 6-8 query-engine-darwin processes open (even after killing/quitting pgAdmin 4), when this error pops up.
I will look into nApi, thank you!

Running app on localhost w/o pgAdmin 4: It usually goes up to ~50 query-engine-darwin processes on my activity monitor when the error happens.

Does it go up first, and then the error or the other way around? I would expect it going up over time to cause the error sooner or later.

I usually check the processes after encountering the error and it is around 50+. Then I terminate all of them locally and wait for the error to happen again - by then the # processes would have increased to the same.

Ok then this is the reason for your problem.
Maybe observe the process list to see how it goes from 1 to 2 (or whatever starts that increase) and see if you can identify the action that does that. Something you are doing - or your code is - has that effect and should be fixed.

Ok I will look into it :)

Connection pool not killing idle connections is expected, I am working on it on https://github.com/importcjj/mobc/pull/60
In localhost make sure that if you are running ts-node or npx or other that it properly kills children. i had instances where the engine was not properly killed if the parent didnt call sigterm or if the application catched the signal and didnt call close on the prisma client.

I have experienced this twice in a production app. The situation resolves itself after 5-10 minutes usually, but all Prisma queries return this error during that time.

Hi, I am facing the same issue, I am using cloud SQL and running a containerized nestjs app on cloud run.

@garrensmith This is due to the reaper not doing its job after a load spike (and being bad in general). My PR on original mobc should help/point in the right direction. https://github.com/importcjj/mobc/pull/60

@callumlocke can you confirm what version of Prisma you are using?
@Sytten thanks. Once I have a moment I'll take a look.

I'm having this issue with "@prisma/client": "^4.10.1" :(

To fix this issue ,Actually
💁

I added  prisma.$disconnect(); after each query.

    const degreeData = await prisma.degree.findUnique({
      where: where,
    }).finally(() => {
      prisma.$disconnect();
    });

Then we added connection_limit=20&pool_timeout=50
Based on the maximum pool connection we have
To see the maximum connection in postgres
SHOW max_connections;
OR
SELECT name, setting FROM pg_settings WHERE name = 'max_connections' OR name = 'max_worker_processes';

And also to terminate all the  connected connection
SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = 'db_name'  AND pid <> pg_backend_pid();

Then , I removed await Promise.all([query1, query2]) and make that as seperate query.

Now it was fixed.
Prisma Team , kindly confirm this changes , I have made are correct and about each points.
**Thank you **  😃😍

Hi @shwethashyam1, @Sytten, and @Venkat03003, we are investigating how to reproduce this issue.
From my understanding, this issue presents itself when a long-running server has a Prisma Client connected to a Postgres database, as some connections from the pool are leaking. What's not clear is whether this only happens with the GCP Postgres instances, or with any Postgres instance. Would you consider this a faithful reproduction of this issue? If so, it looks like the problem is solved as of mobc 0.8.1 (https://github.com/importcjj/mobc/issues/63), which we have pulled in as of prisma@4.14.0 (https://github.com/prisma/prisma-engines/pull/3905). Just in case, I encourage you all to try again with the most recent Prisma version (5.0.0) to assess whether this issue is still present. Thanks!

i had instances where the engine was not properly killed if the parent didn't call sigterm or if the application catched the signal and didnt call close on the prisma client.

As far as I know, we've fixed a bug related to the Prisma Query Engine not terminating properly several months ago, but that doesn't exclude some other bug still exists.

Prisma Team, kindly confirm these changes

@Venkat03003, which Prisma version did you use in this comment above?

Hello!
Unfortunetely, we are also experiencing this issue. Same scenarioas everyone else has discussed: Everything seems to be working fine then suddenly our connection spikes and the app becomes inactive for ~5 min until it eventually fixs itself.
We have a web app that uses Prisma to interact with a PostgreSQL DB.
Originally we were using Prisma version 3.14, but after reading this we updated our packages to version 5.0. Sadly, the issue still comes up.

Here is a graph from AWS showing the connection spikes where we are encountering the issues.
I have not been able to reproduce the issue. The spikes seem to be completely random.
Reading through this tread. I saw that @Venkat03003 manually disconnected his connection. This is something I have not tried yet. However, my assumption is that this is not necessary with Prism’s connection pooling. Also, reading this connection guide from Prisma (https://www.prisma.io/docs/guides/performance-and-optimization/connection-management) it explicitly says: “You do not need to explicitly $disconnect().”
I am not sure if this is an issue with Prisma or if this is some kind of issue on our end. We are continuing to investigate the situation. Just thought I would contribute to the matter to keep the discussion alive.
Thanks

I got this, too!
We are building things on Vercel with Prisma + Supabase DB.

Hi Jacky!
Sorry, it has been a few weeks since you commented. I figured I would share some updates we had though.
Originally, I do not think we had the pooling set up properly. We were just connecting directly to our db: "postgresql//...." Then I was made aware of Prisma's Data Proxy service. THIS was the connection pooling service we were looking for. Now we would connect to the pooling service: "prisma//..." So, we had set that up. Unfortunately, we were still having problems. However this time it was different. Instead of it being the connection limit, we were getting 504 errors because our requests were timing out. After reaching out to Prisma, in turned out it was a bug on their end. Nurul worked with me on this and his service was fantastic.
Eventually I was informed it was fixed, and we were invited to use Prisma Accelerate which contains their Data Proxy service and things seem to working great. We even noticed significant improvements in our load times. Here is what our connection chart looks like now for the past 4 weeks:

Hope this helps!
-Joey

Does anyone have an idea on how to reproduce this behavior of the in-use connection exploding?
Does this go beyond your configured Pool connection_limit when it happens or does it just use all of them?

@janpio its been a while since I touched that code, but here is a typical usecase:

Smaller postgres DB with connection limit of 100
Service with connection pool of max 20 x 3 instances (60 connections total max)
You do a blue/green type update of services, that spawns 3 new instances
If you are under load the new services will try to grab new connections as soon as they receive traffic
GCP Cloud Run doesnt have guarantees and I have seen some containers take up to 10m to get killed
Usually to counter that you set an aggressive idle timeout, but for a long time the connection pool of prisma didnt enforce those (in the background at least). I think this is fixed?
That results in connection limit errors that was originally posted 2y ago

So my take away before closing would be:

Write documentation around this issue and the connection limit of postgres
Suggest users use some kind of pooler or prisma proxy to fix it
if not possible, suggest they increase the connection limit of postgres if they can (usually I set the limit to 2x the expected usage to have my rolling update buffer)
Also suggest and explain how to set the connection idle timeout and how this will impact (pros and cons) the application

Also double check that the idle timeout is actually respected before all that 😅
I think it can be a strong argument to use prisma proxy for many potential customers since technical and non technical will be able to appreciate the non negligible effort that needs to be done to avoid hitting a DB connection limit when running many instances of an application.

@Sytten @aliedp @garrensmith @janpio @callumlocke @jackypan1989 @Jolg42  could you guys give detailed view about pool management in NodeJS with Prisma  and Postgres db.
Why we need to append in this postgres db url with connection_limit=20&pool_timeout=50
what is connection_limit and pool_timeout
About Ram Size , and buffer Size (For Every Query , for operations it will use some memory) in Postgres reated to Postgres.
while managing pool , we have to consider Ram , buffer size , pool timeout , and connection_limit for each instance.
Anyone kindly explain this about above points in detailed view and share your points and thoughts regarding that.

@venkateshb-icanio It is not appropriate to mention a list of users in a comment that is not about the original issue. Please don't do this (I will edit teh @ mentioned out of your message) and open a new discussion instead (without at-mentioning anyone). Thank you.

Service with connection pool of max 20 x 3 instances (60 connections total max)
You do a blue/green type update of services, that spawns 3 new instances
If you are under load the new services will try to grab new connections as soon as they receive traffic
GCP Cloud Run doesnt have guarantees to kill old containers and I have seen some containers take up to 10m to get killed

So, the situation you describe it is more or less expected to run into this error, correct?

Usually to avoid that problem that you set an aggressive idle timeout (10-60s), but for a long time the connection pool of prisma didnt enforce those (in the background at least). I think this is fixed?

Isn't an idle timeout enforced on the database side (considering that Prisma does not offer an option to set one for the connection pool, it has to be) so the connections should be closed from the server side and hence not take up connections any more? (Only Prisma will still potentially think the connection is still open, but as that is on the old deployment it will not try to run a query any more).
Am I missing something?
Your other points are of course correct.

So, the situation you describe it is more or less expected to run into this error, because the deployment can indeed end up using more connections than the database can provide, correct?

Correct but the pool and the engine can help mitigate this. It can retry when it sees this error and it can kill idle connections.

Isn't an idle timeout enforced on the database side (considering that Prisma does not offer an option to set one for the connection pool, it has to be) so the connections should be closed from the server side and hence not take up connections any more? (Only Prisma will still potentially think the connection is still open, but as that is on the old deployment it will not try to run a query any more).

Usually the idle timeout is handled by the client pool. I know the setting postgres idle_in_transaction_session_timeout exists but usually its locked by the cloud provider and AFAIK prisma doesn't have a way to run code on connection open so you cant easily set it on each connection.
I believe I added the support for idle timeout in quaint more than a year ago, but it never was documented? But also it didnt work in the past because you need a reaper background thread or tokio task to kill the connections. I think that was fixed in mobc by someone at Prisma a couple months ago.
So like I said it kinda "works as intended" but prisma could help devs handle the error better and reduce the occurrence of it (and its a good selling point for your managed service).

any update on this? I'm currently having this issue using Prisma, Postgres and Digital Ocean.

This problem also exists on supabase and github actions images. Any updates on this please?

any update here, every time i need to restart the db server.

any update on that ?

There's a similar problem where hot reloading causes too many connections to be open. If you're one of those, please use the following to resolve this:
https://www.prisma.io/docs/orm/prisma-client/setup-and-configuration/databases-connections#prevent-hot-reloading-from-creating-new-instances-of-prismaclient
More specifically, just create a client.ts file to ensure a single instance of PrismaClient stays. Hope this helps!

I solved it with this code. I hope it can be helpful to someone.
`import { INestApplication, Injectable, OnModuleInit, Inject } from '@nestjs/common';
import { PrismaClient, Prisma } from '@prisma/client';
import { REQUEST } from '@nestjs/core';
import { ApmHelper } from '...';
import apm from 'elastic-apm-node';
import { transformSqlShort, transformSqlDetailed } from './instrumentationUtil';
type Event = { timestamp: Date; query: string; params: any; duration: number };
const MAX_LABEL_LENGTH = 1000;
@Injectable()
export class PrismaService extends PrismaClient implements OnModuleInit {
constructor(@Inject(REQUEST) private request: Request) {
super({
log: [
{
level: 'query',
emit: 'event',
},
{
level: 'info',
emit: 'event',
},
{
level: 'warn',
emit: 'event',
},
{
level: 'error',
emit: 'event',
},
],
});
this.$use(async (params, next) => {
  const span = ApmHelper.Instance.getApmAgent().startSpan(
    `${params.model}.${params.action}`,
    `prisma`,
    params.dataPath.join('.'),
  ) as apm.Span;
  ApmHelper.Instance.getApmAgent().currentSpan?.setLabel('prisma:model', params.model);
  try {
    return await next(params);
  } catch (error) {
    throw error;
  } finally {
    if (span) span.end();
  }
});
// This will in fact log the queries but they will not be correctly shown in the span tree.
this.$on('query' as never, async (e) => {
  const e2 = e as Event;
  const agent = ApmHelper.Instance.getApmAgent();
  const isTransactionSampled = (agent.currentTransaction as any)?.sampled || false;
  if (agent.currentTransaction && isTransactionSampled) {
    const span = agent.currentSpan;
    if (span) {
      assignLabels(span, e2);
    } else {
      const endTime = e2.timestamp.getTime();
      const startTime = new Date(endTime - e2.duration);
      const options: apm.SpanOptions = { startTime: startTime.getTime() };
      const span2 = agent.currentTransaction?.startSpan(transformSqlShort(e2.query), 'db', options);
      if (span2) {
        assignLabels(span2, e2);
        span2?.end(endTime);
      }
    }
  }

  function assignLabels(span: apm.Span | null, _e: Event) {
    span?.setLabel('db_statement', transformSqlDetailed(_e.query));
    span?.setLabel('db_statement_raw', getStr(_e.query));
    span?.setLabel('db_params', getStr(_e.params));
  }
});

}
async onModuleInit() {
await this.$connect();
}
async enableShutdownHooks(app: INestApplication) {
process.on('beforeExit', async () => {
await app.close();
});
}
function getStr(params: any): apm.LabelValue {
try {
return JSON.stringify(params).substring(0, MAX_LABEL_LENGTH);
} catch (error: any) {
return 'Error parsing params: ' + typeof params + ' ' + error?.message;
}
}
`

authors: Sytten Venkat03003 ananay f4i2u1 frogeth garrensmith irunner-toptal jackypan1989 janpio jkomyno joshsny lanfersieckj nikolaschou shwethashyam1 siddjellali sonipranjal venkateshb-icanio yasseralsaidi


text:
Seed data causing intermittent engine crashes and SQL errors in node 15

Bug description
There are intermittent issues when trying to seed the data in node 15. Node 14 works fine and causes no errors.
The errors can vary:
Unique constraint failed on the fields: (`email`)

or

Unique constraint failed on the fields: (`slug`,`teamId`)

Error logs with debug enabled and the last 5 SQL statements executed.
https://gist.github.com/viperfx/39390c9f891aa7c4bcd82c86791763a0
@janpio and I have been in contact via Slack and he has everything needed to run the code and reproduce the conditions.
How to reproduce
We use redwoodjs internally and run the seed/reset command with DEBUG=* yarn rw prisma migrate reset --force > debug.log 2>&1. However, I am unsure if redwood is a factor here - just thought I would mention.

Use the seed.js and schema.prisma file
Create a firebase project, enabled authentication module with username/password auth.
Store the firebase serviceAccountKey JSON blob as the env variable FIREBASE_CREDENTIALS
Setup a local docker postgres instance with version 12.6 or a hosted version such as supabase and set credentials to DATABASE_URL. (I have been able to reproduce this locally and in supabase - their postgres version is 12.6).
Use node15 and run the seed script.
Script may not exit at the end, so use ctrl-c after output has ended.

Expected behavior

Runs successfully in node 15 similar to node 14.

Prisma information
@janpio has the seed.js and prisma.schema file to reproduce the issue.
Environment & setup

OS: Windows WSL2 using Ubuntu (have also reproduced with OSX)
Database: Postgres 12.6
Node.js version:  v15.14.0

Prisma Version
2.21.2

Hey @viperfx
Sorry for late reply here. Can you still reproduce this with the latest version? Node also released v16 so please try again with that.

🗒 We heard the feedback and we are planning a redesign of db seed (currently planned to be released early September)
How it will work is that a prisma.seed property in the package.json of your project will be required if you want to use prisma db seed.
This property is where you will be able to put any command you want to execute your seed script.
More info/details & how to try it out now in:
https://github.com/prisma/prisma/issues/8732

The redesign of db seed was released with Prisma 3. Please try it, this problem should be gone.


text:
Prisma crashes when I use the same table for counting and findUnique

Bug description
When I try to count followers of a User model, which itself is an array of users, like this:
 const users = await ctx.prisma.user.findUnique({
  where: { id: source.id },
  include: {
    _count: {
      select: {
        followers: true
      }
    }
  }
})

I get this error:
ApolloError:
Invalid `prisma.user.findUnique()` invocation:

  Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState("42712"), message: "table name \"users\" specified more than once", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("parse_relation.c"), line: Some(442), routine: Some("checkNameSpaceConflicts") }) }) })
    at new ApolloError (/tmp/prisma-count-repro/node_modules/.pnpm/@apollo+client@3.3.20_graphql@15.5.1+react@17.0.2/node_modules/@apollo/client/errors/errors.cjs.js:31:28)
    at /tmp/prisma-count-repro/node_modules/.pnpm/@apollo+client@3.3.20_graphql@15.5.1+react@17.0.2/node_modules/@apollo/client/core/core.cjs.js:1474:47
    at both (/tmp/prisma-count-repro/node_modules/.pnpm/@apollo+client@3.3.20_graphql@15.5.1+react@17.0.2/node_modules/@apollo/client/utilities/utilities.cjs.js:963:53)
    at /tmp/prisma-count-repro/node_modules/.pnpm/@apollo+client@3.3.20_graphql@15.5.1+react@17.0.2/node_modules/@apollo/client/utilities/utilities.cjs.js:956:72
    at new Promise (<anonymous>)
    at Object.then (/tmp/prisma-count-repro/node_modules/.pnpm/@apollo+client@3.3.20_graphql@15.5.1+react@17.0.2/node_modules/@apollo/client/utilities/utilities.cjs.js:956:24)
    at Object.next (/tmp/prisma-count-repro/node_modules/.pnpm/@apollo+client@3.3.20_graphql@15.5.1+react@17.0.2/node_modules/@apollo/client/utilities/utilities.cjs.js:964:49)
    at notifySubscription (/tmp/prisma-count-repro/node_modules/.pnpm/zen-observable@0.8.15/node_modules/zen-observable/lib/Observable.js:135:18)
    at onNotify (/tmp/prisma-count-repro/node_modules/.pnpm/zen-observable@0.8.15/node_modules/zen-observable/lib/Observable.js:179:3)
    at SubscriptionObserver.next (/tmp/prisma-count-repro/node_modules/.pnpm/zen-observable@0.8.15/node_modules/zen-observable/lib/Observable.js:235:7) {
  graphQLErrors: [
    Error:
    Invalid `prisma.user.findUnique()` invocation:

      Error occurred during query execution:
    ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState("42712"), message: "table name \"users\" specified more than once", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("parse_relation.c"), line: Some(442), routine: Some("checkNameSpaceConflicts") }) }) })
        at cb (/tmp/prisma-count-repro/node_modules/.pnpm/@prisma+client@2.23.0_prisma@2.23.0/node_modules/@prisma/client/runtime/index.js:35105:17)
        at runMicrotasks (<anonymous>)
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
        at async resolve (webpack-internal:///./graphql/resolvers.ts:47:23)
        at async Promise.all (index 1)
        at async Promise.all (index 0) {
      locations: [],
      path: [Array]
    }
  ],
  networkError: null,
  extraInfo: undefined
}

How to reproduce

Clone https://github.com/talentlessguy/prisma-count-repro
Add these to .env:

BASE_URL=http://localhost:3000
NEXTAUTH_URL=http://localhost:3000

Go to github.com > Settings > Developer > OAuth Apps and save GITHUB_ID and GITHUB_SECRET to an .env file

Run

# db startup
pnpm generate
pnpm dev

Open /api/auth/signin. Sign In with GitHub.
Go to /your-github-nickname. Nickname is a slug so spaces are converted to dashes.
See this error

Expected behavior
I would expect it to not crash and return the followers count
Prisma information
generator client {
  provider        = "prisma-client-js"
  previewFeatures = ["selectRelationCount"]
}

datasource db {
  provider = "postgresql"
  url      = env("DB_URL")
}

model Account {
  id                 Int       @id @default(autoincrement())
  compoundId         String    @unique @map("compound_id")
  userId             Int       @map("user_id")
  providerType       String    @map("provider_type")
  providerId         String    @map("provider_id")
  providerAccountId  String    @map("provider_account_id")
  refreshToken       String?   @map("refresh_token")
  accessToken        String?   @map("access_token")
  accessTokenExpires DateTime? @map("access_token_expires")
  createdAt          DateTime  @default(now()) @map("created_at")
  updatedAt          DateTime  @default(now()) @map("updated_at")

  @@index([providerAccountId], name: "providerAccountId")
  @@index([providerId], name: "providerId")
  @@index([userId], name: "userId")
  @@map("accounts")
}

model Session {
  id           Int      @id @default(autoincrement())
  userId       Int      @map("user_id")
  expires      DateTime
  sessionToken String   @unique @map("session_token")
  accessToken  String   @unique @map("access_token")
  createdAt    DateTime @default(now()) @map("created_at")
  updatedAt    DateTime @default(now()) @map("updated_at")

  @@map("sessions")
}

model User {
  id            Int       @id @default(autoincrement())
  name          String
  email         String?   @unique
  emailVerified DateTime? @map("email_verified")
  image         String?
  createdAt     DateTime  @default(now()) @map("created_at")
  updatedAt     DateTime  @default(now()) @map("updated_at")
  posts         Post[]
  views         Int       @default(0)
  slug          String    @unique @default(uuid())

  following User[] @relation(name: "UserFollows", references: [id])
  followers User[] @relation(name: "UserFollows", references: [id])
  userId    Int?
  @@map("users")
}

model VerificationRequest {
  id         Int      @id @default(autoincrement())
  identifier String
  token      String   @unique
  expires    DateTime
  createdAt  DateTime @default(now()) @map("created_at")
  updatedAt  DateTime @default(now()) @map("updated_at")

  @@map("verification_requests")
}

model Image {
  id               Int      @id @default(autoincrement())
  publicId         String   @unique @map("public_id")
  version          Int
  signature        String
  width            Int
  height           Int
  format           String
  resourceType     String   @map("resource_type")
  createdAt        DateTime @map("created_at")
  bytes            Int
  type             String
  etag             String
  url              String   @unique
  secureUrl        String   @unique @map("secure_url")
  originalFilename String   @map("original_filename")

  Post Post[]

  @@map("image")
}

model Post {
  id        Int      @id @default(autoincrement())
  createdAt DateTime @default(now()) @map("created_at")
  caption   String?
  image     Image    @relation(fields: [imagePublicId], references: [publicId])
  published Boolean  @default(false)
  authorId  Int      @map("author_id")
  author    User     @relation(fields: [authorId], references: [id])
  views     Int      @default(0)

  imagePublicId String
  @@map("post")
}

Environment & setup

OS: Manjaro
Database: PostgreSQL 13.3
Node.js version: 16.4

Prisma Version
prisma               : 2.23.0
@prisma/client       : 2.23.0
Current platform     : debian-openssl-1.1.x
Query Engine         : query-engine adf5e8cba3daf12d456d911d72b6e9418681b28b (at node_modules/.pnpm/@prisma+engines@2.23.0-36.adf5e8cba3daf12d456d911d72b6e9418681b28b/node_modules/@prisma/engines/query-engine-debian-openssl-1.1.x)
Migration Engine     : migration-engine-cli adf5e8cba3daf12d456d911d72b6e9418681b28b (at node_modules/.pnpm/@prisma+engines@2.23.0-36.adf5e8cba3daf12d456d911d72b6e9418681b28b/node_modules/@prisma/engines/migration-engine-debian-openssl-1.1.x)
Introspection Engine : introspection-core adf5e8cba3daf12d456d911d72b6e9418681b28b (at node_modules/.pnpm/@prisma+engines@2.23.0-36.adf5e8cba3daf12d456d911d72b6e9418681b28b/node_modules/@prisma/engines/introspection-engine-debian-openssl-1.1.x)
Format Binary        : prisma-fmt adf5e8cba3daf12d456d911d72b6e9418681b28b (at node_modules/.pnpm/@prisma+engines@2.23.0-36.adf5e8cba3daf12d456d911d72b6e9418681b28b/node_modules/@prisma/engines/prisma-fmt-debian-openssl-1.1.x)
Default Engines Hash : adf5e8cba3daf12d456d911d72b6e9418681b28b
Studio               : 0.393.0
Preview Features     : selectRelationCount

Thanks for reporting this, I have isolated the reproduction in a small script which can use in further debugging. I can confirm this issue.
Minimal reproduction: https://github.com/harshit-test-org/prisma-issue-7893


text:
No response from lambda function when hitting database.

Context is this occurs when deployed through netlify. It works locally (sigh)
I have two different queries:
t.field('test', {
      type: 'Bool',
      nullable: true,
      resolve: (parent, args, ctx) => {
        return {
          value: true,
        };
      },
    });

t.crud.coupons();

The top one returns but the bottom one times out. I extended the timeout but still nothing. Any thoughts?
Initially I thought it was a db connection issue but mutations work; they affect the db (checked via pgadmin) but they also do not return responses. Also placed console.logs everywhere so I know that db queries are successful but again if the function requires a db connection it ultimately times out on client side. Just to double-check I put console.log right before the return and I see the log message pretty much right after calling the function. But receive a timeout after 30 seconds.
I have no idea where to start debugging. Maybe it's an OS issue since it works locally?

Here are the logs I get from turning on debugging. Maybe they could help? What I think is weird compared to local is that it looks like the server gets started up every call as opposed to once in local.
20:36:42  START RequestId: 230bc096-5dd4-41f9-bf4f-04676f85013f Version: $LATEST
20:36:42  2019-10-29T20:36:42.796Z getos { version: 'OpenSSL 1.0.2k-fips 26 Jan 2017\n' }
20:36:42  2019-10-29T20:36:42.797Z getos { ls: 'lrwxrwxrwx 1 root root 29 Oct 17 17:51 libssl.so.10 -> ../usr/lib64/libssl.so.1.0.2k\nlrwxrwxrwx 1 root root 29 Oct 17 17:51 libssl.so.1.0.2k -> ../usr/lib64/libssl.so.1.0.2k\n-rwxr-xr-x 1 root root 332384 Oct 23 2018 libssl3.so\nlrwxrwxrwx 1 root root 16 Oct 17 17:51 libssl.so.10 -> libssl.so.1.0.2k\n-rwxr-xr-x 1 root root 461344 Apr
20:36:42  2019-10-29T20:36:42.836Z photon Request:
20:36:42  2019-10-29T20:36:42.836Z photon query {
20:36:42  findManyCoupon {
20:36:42  code
20:36:42  }
20:36:42  }
20:36:42  2019-10-29T20:36:42.839Z engine { PRISMA_DML: 'REDACTED'}
20:36:42  PORT: '32775',
20:36:42  RUST_BACKTRACE: '1',
20:36:42  RUST_LOG: 'info',
20:36:42  OVERWRITE_DATASOURCES: '[]' }
20:36:42  2019-10-29T20:36:42.839Z engine { cwd: '/var/task' }
20:36:42  2019-10-29T20:36:42.840Z getos { platform: 'linux', libssl: '1.0.2' }
20:36:43  2019-10-29T20:36:43.174Z engine:log { message: 'Starting 2 workers',
20:36:43  level: 'info',
20:36:43  application: 'prisma',
20:36:43  date: 2019-10-29T20:36:43.173Z }
20:36:43  2019-10-29T20:36:43.175Z engine:log { message: 'Starting server on 0.0.0.0:32775',
20:36:43  level: 'info',
20:36:43  application: 'prisma',
20:36:43  date: 2019-10-29T20:36:43.174Z }
20:36:43  2019-10-29T20:36:43.175Z engine:log { message: 'Started http server on 0.0.0.0:32775',
20:36:43  level: 'info',
20:36:43  application: 'prisma',
20:36:43  date: 2019-10-29T20:36:43.174Z }
20:36:43  2019-10-29T20:36:43.954Z engine Ready after try number 0
20:36:44  2019-10-29T20:36:44.010Z photon Response:
20:36:44  2019-10-29T20:36:44.010Z photon { findManyCoupon:
20:36:44  [ { code: 'aaaaaa' },
20:36:44  { code: 'asdf' },
20:36:44  { code: 'asdfasdf' },
20:36:44  { code: 'gewec' } ] }
20:36:52  END RequestId: 230bc096-5dd4-41f9-bf4f-04676f85013f
20:36:52  REPORT RequestId: 230bc096-5dd4-41f9-bf4f-04676f85013f Duration: 10010.16 ms Billed Duration: 10000 ms Memory Size: 1024 MB Max Memory Used: 112 MB
20:36:52  2019-10-29T20:36:52.801Z 230bc096-5dd4-41f9-bf4f-04676f85013f Task timed out after 10.01 seconds

I'm not sure about netlify's limits -- but I've had some issues with serverless on Zeit because of the large size of the query-engine-darwin that gets packaged in photon's runtime.


text:
Fix caching spec to use cache

The prior version of this spec did not actually use the cache, because when the :cache argument is not supplied, the default is not to use the cache.
In working on this I discovered a couple of problems:

The documentation does not clearly state that when you don't supply the :cache argument, the default cache won't be used. I think this is reinforced by this test, which was written with the assumption that things will be cached.
The documentation states that to implement your own cache, you should implement the interface of Prismic::BasicCache ("You can pass any object implementing the same methods as the BasicCache"). However, the Prismic::Api#caching method does not use this interface -- it uses the interface defined by Prismic::LruCache. #get does not take a block in BasicCache and #get_or_set does not exist in LruCache. This means that if you instantiate the Prismic API like this, it does not work:

api = Prismic.api('https://swiftype.prismic.io/api', :access_token => "prismic_access_token", :cache => Prismic::DefaultApiCache)

I think this should be fixed, but I'm not sure how you would like to do that. So this PR simply fixes the test so it actually uses the LruCache.

Hi,
Thank for pointing that out - We want the DefaultCache to be used if you specify anything, to ensure people get the best perfs, so that's the fix we will do.

Hi,
Thank you very much for your PR.
Actually having no cache by default wasn't the wanted behaviour, we want to have a default cache by default so ensure the best performances (but still allow the user to deactivate the cache if he really wants to). I fixed that recently.

source: gharchive/pull-request

authors: erwan look

repo: prismicio/ruby-kit


text:
General Clarifications and Questions

Hey fwends, getting caught up and have a few clarifications and questions:
Seller Ad Service
Trying to understand what restrictions might be envisioned here. I saw "The seller's ad service is the existing stack of the seller that facilitates real-time bidding auctions" so want to dig in on that more:

Is that literally the case, that this is existing code on existing servers running in existing datacenters (caveat that code changes to accommodate SellerFrontEnd flow)? Or are there restrictions such as, this is expected to run in the cloud in a Nitro Enclave, or the types of communication out of the server (log processing, requests in hot path to other systems, creative serving)?
Would Chrome browser attempt to restrict the request/response structure to/from Seller Ad Service, other than the encrypted Protected Audience Data portion as described here.
Would there be any audit expectation?

Event Level Reporting (for now)
I'd like to understand the "for now" part better. Is the expectation that some level of Event Level Reporting will continue just with more restricted data (i.e. no or less Interest Group data, more noise, etc), or that there will be no Event Level Reporting at all? I ask for a few reasons:

Sellers and buyers have a strong claim to event level data that exists now that is not specific to the user who the impression was served to, and that data is used for many purposes outside of targeting.
Perhaps not the Seller, but the Buyer who is creating and using the Interest Groups has a less-strong-but-still-reasonable claim on wanting to analyze the usefulness of their Interest Groups at a detailed level, and to see "landscape" data showing them how they might adjust their buying structures and settings.
Ad Techs, at the least the one I've been working for for 11 years, often need Event Level Data for operational purposes such as debugging.

Also, even in the "for now" times, will SellerFrontEnd and BuyerFrontEnd service be able to log event level data and phone home?
TEEs
I am no security expert, so if there's something very obvious here than apologies, but has any consideration been given to TEEs running in non-public-cloud environments but providing the necessary constraints, attestations, etc, through some combination of technical and audit requirements? I ask because one of the twix-inesses I see here is that Ad Techs (at least one I work for) will likely continue to have to support some set of existing use cases outside of a Fledge/Parakeet context, and those use cases are of size anywhere between non-trivial and quite substantial. Having to setup a TEE inside of a non-public-dc with some set of even relatively "intrusive" requirements could be preferable to forcing network and system topologies.

Closing as after more study I have some of these answers and think it makes sense to discuss others in separate threads.

repo: privacysandbox/fledge-docs

license_source: bigquery


text:
RFC: Generalized class members

This RCF PR change the way class members are parsed.
Before, the order was strict: class annotations, then extern code block, then super clauses, then properties.
After, all these elements can appear in any order.
The need is to enable class annotations inside the class (cf #1158) I did not find another simple way to do this.
Now, one can write
class B
   super A
   serializable
   var i: Int
   autoinit x, i
end

Currently, I expect a lot of tests to fail, I just PR this to collect comments and suggestions before going further.

60 fails, not that bad, I mainly broke nitpretty (make sense since it is strongly coupled with the AST) and the serialization (make sense since it injected superclasses).

I like the fact that you can now declare super, annotations and extern code block in a "random" fashion, however it buzzes me to see properties declared in the middle of it all, ideally I'd like to force the properties to be declared below stuff that concern the class definition itself.
Aside from that, +1 !

@R4PaSs, acutally it can be usefull to group properties by concerns.
class Dwarf
    is "something about all the class"

    # here we build a property group for `Race` related methods
    super Race

    redef var strength = 10
    redef var stamina = 10

    # here another group for the `Weapon` concern.
    super Weapon # dwarves can be used as a weapon

    redef fun attack...
    redef fun pary...
end

+1

@Morriar : Oh, good call ! I really didn't think of that !
Well in this case it seems my point falls in the not_relevant category :)
+1 then !

Miniupdate

improve new grammar
update nitpretty, @Morriar you may want to look at the changes
still some work to do, especially serialization

reroll: clean the commits and fix auso-serialization

Enfin les tests passent. Youpi

+1
The fact that super clauses and class annotations are considered as properties bother me from a semantical point of view. But actually, the implem is quite cool so...

+1

Rhaaa. Vous imaginez pas à quel point ca me coute en santé mentale de faire du sablecc3. Je vais quand même essayer de voir ce que je peux faire pour préserver le oneliner de classe.

@Morriar yes, properties are a bad name but the important part in APropdef is the def. In fact APropdef is a trivial subclass of ADefinition (no new property). Thus the naming is mainly historical and constrained by the rules of sablecc3

reroll presque complet: j'ai récrit la grammaire pour la partie modifiée et changé l'AST.
Le code est plus simple, le découpage en commit plus lisible et, cerise sur le gâteau, le oneliner de classes class B super A end restauré.
Par contre, nitpretty fait des trucs bizarres : le oneliner de classe marche pas (je l'ai donc désactivé) et les commentaires devant le super sont dupliqués (d'où des nouveaux fixme). @Morriar je te laisse regarder ça, j'ai essayé de sauver les meubles mais ces deux trucs restent louche pour moi.

authors: Morriar R4PaSs privat xymus

repo: privat/nit


text:
IE11 issues

By default when I load a page using this directive in IE11 I get the error
Error in directive frag inserted hook: "TypeError: Unable to get property 'apply' of undefined or null reference"
This is pretty simple to fix by adding a polyfill for ParentElement.append and one for ChildElement.replaceWith.
The problem is that after I add the polyfill, I run into another issue altogether:
Error in directive frag inserted hook: "Error: Out of stack space"
It's pretty difficult to debug for me as the stack trace just looks like

Any ideas on what the issue is or what the solution is?

Sorry, I haven't had access to IE11 but I can look into testing it later.
Can you provide reproduction steps?
From your report, I don't know what code you used to produce the error or the polyfill you used.

The polyfills I tried are from here https://www.npmjs.com/package/element-polyfill
And the code is fairly simple. In the template, I have a div with v-frag on it, and inside that div I have multiple elements like

I think it should be pretty easy to get access to IE11 if you get one of these vms
https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/

Thought about this some more, and I'm not sure if I can justify spending my time to add IE11 support. IE11 support is being deprecated everywhere.
PR welcome if you want to give it a shot.
Happy to reconsider if this package gains more popularity & adoption (GH stars/npm downloads).

Damn. That's really unfortunate. Thanks for the honesty, though.

authors: Moghul privatenumber

repo: privatenumber/vue-frag

license: MIT


text:
Allow for manual application of masks to saturated SDSS areas

Fixes #465.

Before masking:
{20: {'precision': tensor(0.5646),
  'recall': tensor(0.9801),
  'f1': tensor(0.7164),
  'class_acc': tensor(0.9820),
  'conf_matrix': tensor([[79,  3],
          [ 0, 85]])},
 24: {'precision': tensor(0.4576),
  'recall': tensor(0.7024),
  'f1': tensor(0.5542),
  'class_acc': tensor(0.7824),
  'conf_matrix': tensor([[266,   7],
          [107, 144]])}}

After masking:
{20: {'precision': tensor(0.6395),
  'recall': tensor(0.9751),
  'f1': tensor(0.7724),
  'class_acc': tensor(0.9821),
  'conf_matrix': tensor([[80,  3],
          [ 0, 85]])},
 24: {'precision': tensor(0.4711),
  'recall': tensor(0.6997),
  'f1': tensor(0.5631),
  'class_acc': tensor(0.7816),
  'conf_matrix': tensor([[264,   7],
          [107, 144]])}}

repo: prob-ml/bliss


text:
Many HTML Tags after asterisk/underscore crashes the browser

Hi there.
I noticed that when I have many html tags after asterisk (*) or underscore (_) character it crashes the browser.
For example, try this text on the playground https://probablyup.com/markdown-to-jsx/:
Nove Controles Programáveis

Sob Seu Comando!
Obtenha excelentes resultados com as configurações padrão incluídas no produto ou configure gatilhos de um único botão para ações que normalmente requerem vasculhar menus. Ponha comunicações por bate-papo a um fácil alcance. Reduza DPI temporariamente. Retribua qualquer comando de jogo ou macro multicomando a qualquer um dos nove botões programáveis com o Logitech Gaming Software (LGS) opcional.
*Algumas configurações de perfil requerem o Logitech Gaming Software, disponível em gaming.logitech.com/software. G300s, G9 DPI Loop and G8 profile switch buttons close up Left angled mouse emphasizing compact shape, comfort and durability.

Formato Ambidestro Esculpido
Conforto é rei!
Design versátil em uma forma compacta, este mouse foi construído para conforto e durabilidade.

Iluminação Configurável Pelo Usuário
Conhecimento é poder!
Atribua uma cor e uma identidade exclusivas a cada perfil de usuário com até sete variações de de cores para saber quais configurações estão ativas durante um jogo.

Perfis na Memória de Bordo
O mouse com cérebro!
Configure o mouse uma vez e faça-o memorizar. Cada jogo requer uma configuração exclusiva, seja World of Warcraft ou Guild Wars 2. Programe todos os nove botões para poder levar as configurações para outros computadores.

Alternância de Dpi Imediata
Esteja pronto num instante!
Faça os movimentos certos em qualquer situação de jogo. Alterne múltiplas configurações de DPI para rapidamente selecionar opções na tela ou examinar o mapa do jogo (até 2500 DPI), ou ainda pormenorizar miras de alvos com precisão de pixel ou selecionar unidades (250 DPI) usando botões localizados de forma conveniente.

Taxa de Transmissão de um Milissegundo
A ciência da velocidade!
Tenha a certeza de que cada comando será executado com a maior rapidez possível. Até oito vezes mais rápido do que um mouse USB padrão, o G300 captura suas jogadas com a mesma rapidez com que você as faz.

Construção Durável
Construído para sobreviver!
Os testes de engenharia classificaram os botões principais a dez milhões de cliques e os pés de percurso a 250 km. Capaz de sobreviver às sessões de jogo mais intensas, o G300 pode ir tão longe quanto você.

Pés Elegantes
Uma vantagem material!
Os pés de politetrafluoroetileno de baixa fricção eliminam a fricção em quase todas as superfícies para proporcionar movimentos mais suaves, movimentos precisos do cursor e exatidão aprimorada.

Software de Configuração Fácil de Usar
Torne-o pessoal - facilmente!
Use o Logitech Gaming Software opcional para corresponder comandos perfeitamente aos jogos preferidos. Configurações de arrastar-e-soltar simples permitem que você personalize perfis de botão e rastreamento para qualquer jogo. Ou use personalizações pré-configuradas com detecção de jogo automática.

Imagem meramente ilustrativa.
Todas as informações divulgadas são de responsabilidade do fabricante/fornecedor.
Verifique com os fabricantes do produto e de seus componentes eventuais limitações à utilização de todos os recursos e funcionalidades.

Anyone knows how to deal with it?

I've seen similar bugs a few times and my guess is it's probably the italics/em rule doing too much backtracking (here's an issue that had a potentially similar problem in simple-markdown: https://github.com/Khan/simple-markdown/pull/73 )
I'll take a look at this this week and check whether that's the cause, and try to find a fix.

authors: ariabuckles arturhaddad

repo: probablyup/markdown-to-jsx

license: mit


text:
Error when building Discord appimage (copying files)

Trying to build Discord with this .yml-file

app: Discord

ingeredients:
  script:
    - wget https://discordapp.com/api/download?platform=linux&format=tar.gz
    - tar zxvf discord*.tar.gz

script:
  - cp -r ../Discord/* usr/bin/
  - find . -name discord.png -exec cp {} discord.png \;
  - cat > Discord.desktop <<EOF
  - [Desktop Entry]
  - Name=Discord
  - StartupWMClass=discord
  - Comment=All-in-one voice and text chat for gamers that's free, secure, and works on both your desktop and phone.
  - GenericName=Internet Messenger
  - Exec=Discord %u
  - Icon=discord.png
  - Type=Application
  - StartupNotify=true
  - Categories=Network;InstantMessaging;
  - EOF

Gives me this error:

[...]
+ sed s/_=/+=/g
+ sed -i -e 's|^_script+=("||g' /tmp/recipe_script
+ sed -i -e 's|")$||g' /tmp/recipe_script
+ bash -ex /tmp/recipe_script
+ cp -r '../Discord/*' usr/bin/
cp: Aufruf von stat für '../Discord/*' nicht möglich: Datei oder Verzeichnis nicht gefunden

How should the copy mechanism be aligned for it to work?

Please ask the makers of Discord to provide an official AppImage. I am happy to support them at that.

authors: Serkan-devel probonopd

repo: probonopd/AppImages


text:
Found part of u-boot within DJI FW code

So - I found the GPL code in monolithic firmware module. I assume this means we have right for the whole module?
What next?
Should all findings be public, or better to keep details private?

For now I would keep it private ;-)

Since DJI did not responded to my e-mail, time to share the report:
https://gist.github.com/mefistotelis/ee88b8c5a213e78f6f0467573d28b970

I wonder if one could somehow automate this type of analysis based on statistical methods...

What I did in first step is just copy strings to Google. That can be automated. Then matching specific functions is harder..
When I started, I assumed DJI used SDK code from Atmel. So I first found Atmel SAM sources they freely (not under GPL) distribute. I expected everything to match Atmel libs. But around SD and MMC functions, there were substantial differences - code didn't matched. So I started googling various strings I found in the code; I was usually getting a few results, but one which was there for every string was u-boot. I downloaded it, reverted GIT repo to around the time when the firmware was published - and then noticed some functions are just identical.
So automated system would need to have at least strings from many GPL projects, best with dates when each line was added to the project. Then it would have to do strings of FW binary, and start matching.
The fact that firmwares are encrypted would have to be handled manually though. I mean, it's not hard to automate if you have a decryptor, like we do with drones up to Mavic. But then you only support the firmwares for which you made automatic decryption procedure.

turnitin.com for source code ;-)

authors: MrBurnsAT mefistotelis probonopd

repo: probonopd/phantom-licensecheck


text:
added code coverage

added code coverage.
To run use npm run coverage

@itaditya - you don't need to commit the coverage directory to GitHub, as its generated automatically every time.

@JasonEtco yes I found out later, deleted the coverage directory in #25

authors: JasonEtco itaditya

repo: probot/background-check


text:
Make feature_request label configurable

I'm already using "Enhancement" as label for feature requests plus don't personally like the look of the _ in the name as it does not match other label's style. Searching the codebase for feature_request didn't return any results- would be great to understand where this comes from plus make it configurable.

@andig are you sure you're asking this in the correct repo?  I don't think this belongs here.

Thank you.. migrated to https://github.com/machine-learning-apps/Issue-Label-Bot/issues/75

Get Outlook for iOShttps://aka.ms/o0ukef

From: andig notifications@github.com
Sent: Thursday, July 16, 2020 2:56:42 PM
To: probot/stale stale@noreply.github.com
Cc: Subscribed subscribed@noreply.github.com
Subject: Re: [probot/stale] Make feature_request label configurable (#290)
Thank you.. migrated to machine-learning-apps/Issue-Label-Bot#75https://github.com/machine-learning-apps/Issue-Label-Bot/issues/75
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/probot/stale/issues/290#issuecomment-659605588, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQHMCHMFUCFOCAVY2YWTX5TR35EOVANCNFSM4OVJG7TQ.

repo: probot/stale

license: isc


text:
[INFRA] Reviewing Dependabot updates is time consuming - Find alternatives

This project declares a lot of dependencies (most of them are dev dependencies), reviewing PR created by Dependabot takes too much time.
For instance today, here is the list of PR dependabot would like to create (we allow Dependabot to open 2 PR at a given time).
The preview is not deployed on dependabot PR, so manual tests must be done.
updater | +----------------------------------------------------------------------+
updater | |                 Changes to Dependabot Pull Requests                  |
updater | +---------+------------------------------------------------------------+
updater | | created | gatsby ( from 4.21.1 to 4.22.0 )                           |
updater | | created | gatsby-plugin-manifest ( from 4.21.0 to 4.22.0 )           |
updater | | created | gatsby-transformer-remark ( from 5.21.0 to 5.22.0 )        |
updater | | created | gatsby-plugin-typescript ( from 4.21.0 to 4.22.0 )         |
updater | | created | @emotion/react ( from 11.10.0 to 11.10.4 )                 |
updater | | created | @typescript-eslint/eslint-plugin ( from 5.36.0 to 5.36.1 ) |
updater | | created | @typescript-eslint/parser ( from 5.36.0 to 5.36.1 )        |
updater | | created | @emotion/styled ( from 11.10.0 to 11.10.4 )                |
updater | | created | @fortawesome/fontawesome-svg-core ( from 6.1.2 to 6.2.0 )  |
updater | | created | gatsby-plugin-sitemap ( from 5.21.0 to 5.22.0 )            |
updater | | created | @mui/material ( from 5.10.1 to 5.10.3 )                    |
updater | | created | gatsby-remark-responsive-iframe ( from 5.21.0 to 5.22.0 )  |
updater | | created | @fortawesome/free-solid-svg-icons ( from 6.1.2 to 6.2.0 )  |
updater | | created | @fortawesome/free-brands-svg-icons ( from 6.1.2 to 6.2.0 ) |
updater | | created | gatsby-plugin-google-gtag ( from 4.21.0 to 4.22.0 )        |
updater | | created | gatsby-plugin-mdx ( from 3.20.0 to 4.1.0 )                 |
updater | | created | @mui/icons-material ( from 5.8.4 to 5.10.3 )               |
updater | | created | gatsby-plugin-styled-components ( from 5.21.0 to 5.22.0 )  |
updater | | created | @types/react ( from 18.0.17 to 18.0.18 )                   |
updater | +---------+------------------------------------------------------------+

Proposal
Use tool like Renovate that are able to update several dependencies at the same time.

Let's try to use it 😉

authors: csouchet tbouffard


text:
Restore CollapseServicesOnStart checkbox in Settings

From https://github.com/processhacker/processhacker/commit/a37876f614cb7ecc4d19d11802aac2f305fca7f8:
At least

If you don't want to loose the checkbox, add a popup explaining where to find the configuration option

or, "fix properly" by:

I'll add the option back (implemented as a wrapper around the Misc menu) if someone creates a bug ticket.

I've added this to the Settings > Usernotes options

authors: dmex stdedos

repo: processhacker/processhacker


text:
chore: group dependabot updates when minor/patch

[x] change dependabot config to group dependencies

leave major dependency updates to their own PR so they stand out and are tested correctly
prefix the PRs with chore(deps) to adhere to conventional commits

[x] change GitHub Actions to use SHAs instead of tags

used frizbee

ran frizbee ghactions -d .github/workflows locally
also fixes formatting

Checklist:

[x] I have updated the necessary documentation
[x] I have signed off all my commits as required by DCO
[x] My build is green

@jmeridth LGTM but can you explain the benefit of using checksums for Actions? Is it to thwart a supply chain attack, or just better specificity in case of a breaking change?

@jmeridth LGTM but can you explain the benefit of using checksums for Actions? Is it to thwart a supply chain attack, or just better specificity in case of a breaking change?

Exactly. Prevent supply chain attack. Tags can move. They are mutable. SHAs are not.

@njbbaer @jhollinger can one of y'all merge? I am unable.

authors: jmeridth njbbaer

repo: procore-oss/blueprinter


text:
Create feature component

Landr has a "highlight" component, but there no way to list features, or it's not obvious re: how to do so.

@andrewnhem the highlights are similar to features, as on the site we don't mention them as highlights. We list them in a separate section. Is there any other difference between highlights and features that I am missing?

authors: andrewnhem iamsolankiamit

repo: product-os/landr


text:
Name doesn't fit patter

We typically call them t-FROM-to-TARGET. Mind that when you rename the repo you need to

create a PR after that
delete all old Transformer version OR
remove the "owns" link between the loop and the old Transformer repo
create "owns" link from loop to new Transformer repo

@Hades32 I think to make things easy I will jsut copy the source code over to https://github.com/product-os/t-node-module-source-to-node-module

since that already exists

Sure. Just make sure to do the cleanup anyway, so don't get confused later

Alright time for me to totally mess up all my hard work on this thing by changing the links... pray for me

Old TF deleted in JF, made pr to new one https://github.com/product-os/t-node-module-source-to-node-module/pull/2

New TF linked to loop in JF

Closing since TF code moved

repo: product-os/t-npm-builder


text:
NameError: name 'z' is not defined

运行7-10程序：
name_prompt = "\nWhat's your name? "
place_prompt = "If you could visit one place in the world, where would it be? "
continue_prompt = "\nWould you like to let someone else respond? (yes/no) "

responses = {}

while True:
    name = input(name_prompt)
    place = input(place_prompt)

    responses[name] = place

    repeat = input(continue_prompt)
    if repeat != 'yes':
        break

print("\n--- Results ---")
for name, place in responses.items():
    print(name.title() + " would like to visit " + place.title() + ".")

报错信息
$ python -u 7-10.py

What's your name? z
Traceback (most recent call last):
  File "7-10.py", line 8, in <module>
    name = input(name_prompt)
  File "<string>", line 1, in <module>
NameError: name 'z' is not defined

难道是因为输入的时候类型是"z"导致的，而不是z？

把input换成raw_input就可以了。
书上是python3版本，而我本机是python2版本。
网上教程简易python程序NameError: name 'xxx' is not defined？


text:
Move from .CODEOWNERS back to pull approve, and support optional reviewers

Problem
In order to make our approver requirements a little more dynamic, let's use pull approve to do so :)
Summary of Changes
Set up reviewer groups for parts of the tree, applied them as optional

With this landed, pullaprove seems to be consistently failing with:

pullapprove — codeowners-android team was not found on this repository

yeah, @woody-apple will look as soon as he has time.  this wasn't intended to land quite yet

repo: project-chip/connectedhomeip


text:
Updating format strings per feedback in Darwin

Per feedback, generally replacing format strings

@bzbarsky-apple fixed a bunch of them we had as well.


text:
MTRDevice XPC Interface

work in progress - goal is to get a simple test running first with a method not on MTRDevice, then start implementing MTRDevice methods

work ongoing in other branches


text:
Make build_python work on mac (tested on a BigSur)

Problem
scripts/build_python.sh does not work on mac.
Summary of Changes
Update the script to work on mac:

document requiring brew install coreutils
add BigSur define to avoid whl install errors
add validation that virtualenv is available as a module and pip-install it if it is not (on mac virtualenv does not seem available, even though pigweed is already virtualenv)

Here's a version without needing coreutils:
https://github.com/project-chip/connectedhomeip/pull/5194

closing after #5194


text:
chore: bump trivy and buildkit

Describe the changes in this pull request using active verbs such as Add, Remove, Replace ...

Update trivy to 0.37.2
Update buildkit to 0.11.0

Closes #<issue_ID>

Codecov Report
Base: 34.45% // Head: 34.45% // No change to project coverage :thumbsup:

Coverage data is based on head (6ff5fc4) compared to base (ac17bab).
Patch has no changes to coverable lines.

:exclamation: Current head 6ff5fc4 differs from pull request most recent head f737e26. Consider uploading reports for the commit f737e26 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #48   +/-   ##
=======================================
  Coverage   34.45%   34.45%
=======================================
  Files          12       12
  Lines        1126     1126
=======================================
  Hits          388      388
  Misses        717      717
  Partials       21       21

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

authors: codecov-commenter sozercan

repo: project-copacetic/copacetic


text:
Professional activities no longer display on People & Group records.

Problem Description:
Professional Activities does not render on People and Group pages.
Expected Behavior/Solution:
The Professional Activity data should be rendered.
Requirements:

[x] The getCarriedOut function should return the nested classified_as/id if the object does not contain a /equivalent property

Needed for promotion:
If an item on the list is not needed, it should be crossed off but not removed.

[x] Wireframe/Mockup - Kam
[x] ~Committee discussions - Sarah~
[x] ~Feasibility/Team discussion - Sarah~
[x] ~Backend requirements - TBD~
[x] Frontend requirements - Kam
[ ] Are new regression tests required for QA - Amy
[x] ~Questions~

UAT/LUX Examples:

https://lux-front-tst.collections.yale.edu/view/person/85ab18e6-b208-4658-8ea4-abaca7a60812 (Should have higher education as professional activity)
PRD: https://lux.collections.yale.edu/view/person/85ab18e6-b208-4658-8ea4-abaca7a60812

Dependencies/Blocks:
N/A
Related Github Issues:
N/A
Related links:

Bugherd: https://www.bugherd.com/projects/284041/tasks/2469

Wireframe/Mockup:
Expected:

Moved this to the 10-07 milestone as search results for "cats and dogs" did not show on the FE of some records as it belongs under professional activities. Unsure of the priority of this as research is TBD but wanted to leave it on the milestone so we don't lose track of it.

From Teams Conversation on 10/3:

This issue is okay to move from the prioritization review to backlog and should be labeled as important.

Approved by UAT

Looks good in PROD. Closing

repo: project-lux/lux-frontend


text:
Error code 3003 (Failed to deserialize the account.) should specify an account

current behavior
Here are example logs:
  logs: [
    'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS invoke [1]',
    'Program log: Instruction: CreateMarket',
    'Program 11111111111111111111111111111111 invoke [2]',
    'Program 11111111111111111111111111111111 success',
    'Program log: AnchorError occurred. Error Code: AccountDidNotDeserialize. Error Number: 3003. Error Message: Failed to deserialize the account.',
    'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS consumed 28416 of 200000 compute units',
    'Program Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS failed: custom program error: 0xbbb'
  ],

desired behavior
You would expect the error to tell me which account is the issue, as has been the case with some other errors since 0.22.

does this happen during an init?

@paul-schaaf I could be misremembering, but I believe that this was indeed during an init and i'd miscalculated space to be smaller than the struct actually needed.

there's indeed sth missing in the init code so we can merge that and see whether this issue comes up again.
Thanks for reporting it!

authors: asktree paul-schaaf

repo: project-serum/anchor


text:
Update clap to v3.1.6: clap version used in dex has unstable API

Currently used version of clap is 3.0.0-beta.1, see
https://github.com/project-serum/serum-dex/blob/a3b4cf71193611fd96af9b277db93a505d581648/dex/crank/Cargo.toml#L14
This leads to compilation error when building downstream projects in solana: build log
Why it happens: because we trying to update clap to version 3.1.5 and version resolution works such a ways that it picks the latest version from Cargo.lock with the same major release. It is assumed that all the crates with the same major have compatible API which is not the case for clap's beta. (please, correct me if I'm wrong)

resolved

repo: project-serum/serum-dex


text:
wip: open in vscode

Description
Fixes https://github.com/projectatomic/studio-extension/issues/27
Depends on https://github.com/containers/podman-desktop/issues/5672
Screenshots
todo
Tests
todo

Closing as too old and too many conflicts

repo: projectatomic/ai-studio


text:
Update containers/storage to fix locking bug

Update to commit hash 1e5ce40cdb84ab66e26186435b1273e04b879fef
Fixes #445

@rh-atomic-bot r+

authors: mheon rhatdan

repo: projectatomic/libpod


text:
calico drops pod packet in cali-from-wl-dispatch chain under iptables v1.8.4 (nf_tables)

Expected Behavior

calico accepts pod packet in cali-from-wl-dispatch chain
Current Behavior

calico drops pod packet in cali-from-wl-dispatch chain
Possible Solution

Steps to Reproduce (for bugs)

1.install rhel 8.2
2.install docker 19.03
3.install k8s 1.20.5
4.install calico 3.15.1
calico-node env config:
            - name: CLUSTER_TYPE
              value: "k8s,bgp"
            # The BGP IP address.
            - name: IP
              value: ""
            # Enable IPIP
            - name: CALICO_IPV4POOL_IPIP
              value: "Off"
            - name: CALICO_IPV4POOL_VXLAN
              value: "Off"
            # Set MTU for tunnel device used if ipip is enabled
            - name: FELIX_IPINIPMTU
              valueFrom:
                configMapKeyRef:
                  name: calico-config
                  key: veth_mtu
            # The default IPv4 pool to create on startup if none exists. Pod IPs will be
            # chosen from this range. Changing this value after installation will have
            # no effect. This should fall within `--cluster-cidr`.
            - name: CALICO_IPV4POOL_CIDR
              value: "177.177.0.0/16"
            # Disable file logging so `kubectl logs` works.
            - name: CALICO_DISABLE_FILE_LOGGING
              value: "false"
            # Set Felix endpoint to host default action to ACCEPT.
            - name: FELIX_DEFAULTENDPOINTTOHOSTACTION
              value: "ACCEPT"
            # Disable IPv6 on Kubernetes.
            - name: FELIX_IPV6SUPPORT
              value: "false"
            # Set Felix logging to "info"
            - name: FELIX_LOGSEVERITYSCREEN
              value: "info"
            - name: FELIX_HEALTHENABLED
              value: "true"
            - name: FELIX_IPTABLESBACKEND
              value: "nft"

Context

pod on one node can not visit k8s svc and endpoint on other node.
node ip:
[root@node32 ~]# kubectl get no -owide
NAME     STATUS   ROLES                  AGE   VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                               KERNEL-VERSION          CONTAINER-RUNTIME
node30   Ready    control-plane,master   47h   v1.20.5   192.168.205.30   <none>        Red Hat Enterprise Linux 8.2 (Ootpa)   4.18.0-193.el8.x86_64   docker://19.3.15
node31   Ready    control-plane,master   47h   v1.20.5   192.168.205.31   <none>        Red Hat Enterprise Linux 8.2 (Ootpa)   4.18.0-193.el8.x86_64   docker://19.3.15
node32   Ready    control-plane,master   47h   v1.20.5   192.168.205.32   <none>        Red Hat Enterprise Linux 8.2 (Ootpa)   4.18.0-193.el8.x86_64   docker://19.3.15

trace port and ip in raw table:
[root@node32 ~]# iptables -t raw -nvL PREROUTING
Chain PREROUTING (policy ACCEPT 23M packets, 7134M bytes)
 pkts bytes target     prot opt in     out     source               destination
  32M 6286M cali-PREROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:6gwbT8clXdHdC1b1 */
  32M 6294M MATRIX-SECURITY-POLICY  all  --  *      *       0.0.0.0/0            0.0.0.0/0
   14   840 TRACE      tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp spt:22334
    9   540 TRACE      tcp  --  *      *       0.0.0.0/0            192.168.205.31       tcp dpt:8443
# Warning: iptables-legacy tables present, use iptables-legacy to see them

curl node 192.168.205.31:8443 in pod:
postgres [ / ]$  curl -k https://192.168.205.31:8443
curl: (7) Failed to connect to 192.168.205.31 port 8443: Connection timed out

trace log show the packet has been dropped in cali-from-wl-dispatch
...
Apr  9 17:02:08 node32 kernel: TRACE: filter:FORWARD:rule:1 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307) MARK=0x40000
Apr  9 17:02:08 node32 kernel: TRACE: filter:KUBE-FORWARD:return:5 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307) MARK=0x40000
Apr  9 17:02:08 node32 kernel: TRACE: filter:FORWARD:rule:2 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307) MARK=0x40000
Apr  9 17:02:08 node32 kernel: TRACE: filter:KUBE-SERVICES:return:4 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307) MARK=0x40000
Apr  9 17:02:08 node32 kernel: TRACE: filter:FORWARD:rule:3 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307) MARK=0x40000
Apr  9 17:02:08 node32 kernel: TRACE: filter:KUBE-EXTERNAL-SERVICES:return:1 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307) MARK=0x40000
Apr  9 17:02:08 node32 kernel: TRACE: filter:FORWARD:rule:4 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307) MARK=0x40000
Apr  9 17:02:08 node32 kernel: TRACE: filter:cali-FORWARD:rule:1 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307) MARK=0x40000
Apr  9 17:02:08 node32 kernel: TRACE: filter:cali-FORWARD:rule:2 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307)
Apr  9 17:02:08 node32 kernel: TRACE: filter:cali-from-hep-forward:return:1 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307)
Apr  9 17:02:08 node32 kernel: TRACE: filter:cali-FORWARD:rule:3 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307)
Apr  9 17:02:08 node32 kernel: TRACE: filter:cali-from-wl-dispatch:rule:3 IN=calieaf0cb8d5bc OUT=ens3 MAC=ee:ee:ee:ee:ee:ee:6a:1c:76:89:28:6e:08:00 SRC=177.177.51.193 DST=192.168.205.31 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=51237 DF PROTO=TCP SPT=58976 DPT=8443 SEQ=3000208233 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A8842580B0000000001030307)

# the end

The last tace log shows the packet from pod matches the 3th rule in chain cali-from-wl-dispatch.And that rule is DROP.
[root@node32 ~]# iptables -t filter -nvL cali-from-wl-dispatch
Chain cali-from-wl-dispatch (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 cali-fw-calieaf0cb8d5bc  all  --  calieaf0cb8d5bc *       0.0.0.0/0            0.0.0.0/0           [goto]  /* cali:SUkR3oOc-KCAknVM */
    0     0 cali-fw-calied6cc3f6d38  all  --  calied6cc3f6d38 *       0.0.0.0/0            0.0.0.0/0           [goto]  /* cali:HrvQMfRA4wyAIhzm */
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:anng98CFsFoSkr47 */ /* Unknown interface */
# Warning: iptables-legacy tables present, use iptables-legacy to see them

But the packet input interface is calieaf0cb8d5bc and should match the 1th rule in cali-from-wl-dispatch.
So confused!
Your Environment

Calico version 3.15.1
Orchestrator version (e.g. kubernetes, mesos, rkt):k8s 1.20.5
Operating System and version:rhel 8.2/iptables v1.8.4 (nf_tables)
Link to your project (optional):

https://github.com/kubernetes/kubernetes/issues/100767

repo: projectcalico/calico


text:
policy/v1beta1 PodDisruptionBudget Deprecated in v1.21+, unavailable in v1.25+

When applying the Calico manifest on a native Kubernetes cluster running v1.21.0, the following warning is shown:
Warning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
The PodDisruptionBudget is created as expected for the time being.  This is a new warning as of v1.21 and is not displayed on earlier releases of Kubernetes.
Expected Behavior
The poddisruptionbudget.policy/calico-kube-controllers should be created without a warning.  Ideal would be if you could check Kubernetes version before declaring the Policy.
Current Behavior
On versions earlier than 1.21.0, no warning is generated.  Versions 1.21+ will generate a warning.  Versions 1.25+ will fail to create the object.
Possible Solution
Update Installation documentation to have users using version 1.21+ edit calico.yaml line 3850 prior to applying the configuration.  They need to change:
apiVersion: policy/v1beta1
to:
apiVersion: policy/v1
Steps to Reproduce (for bugs)
N/A.  The system is working as expected given the prevalence of Kubernetes deployments earlier than 1.21.
Context
This will become an issue as more users upgrade to Kubernetes 1.21 and beyond.  It will need to be corrected once 1.25 is released.
Your Environment

Calico version: 3.17.4
Orchestrator version (e.g. kubernetes, mesos, rkt): Kubernetes 1.21.0
Operating System and version: Ubuntu 20.04
Link to your project (optional): N/A

@darkhonor yep, we'll definitely need to update those manfiests.
It will still work fine to use the v1beta1 API for now. We'll want to switch the default in a release or so in order to make sure we're compatible with +/- a Kubernetes release.

Any news on this yet?

See https://github.com/projectcalico/calico/pull/4926

We probably want to merge this PR when v1.23 comes out, since then we'll support v1.21, v1.22, and v1.23.

Hello, And I'm super new of k8s. I follow the k8s install guide, And I choose calico as network implements, I see this warning
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+

This make me little bit anxiety. Then I check my version, I'm using:

debian 11
k8s 1.22.3 c92036820499fedefec0f847e2054d824aea6cd1
docker 20.10.10 b485636
containerd.io 1.4.11
calico's website install command "kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml"

So I found this issue, Can I use this current version calico guide safely?

@Alceatraz you can use the guide safely - we'll update to remove the PSP objects from Calico in v3.23, at which point the warning will go away.

Update here - PodDisruptionBudgets have been updated to v1 for the upcoming v3.23 release. PodSecurityPolicy removal is still in progress.

Made a new issue to track PSPs, PDB will be done in v3.23

authors: Alceatraz caseydavenport darkhonor pacoxu vzuevsky


text:
Allow tee access metadata files

Signed-off-by: jizhenlo zhenlong.z.ji@intel.com

Expected syntax:
Commit message one-line summary
  Extra paragraphs can be added here.

  Longer description that explains the problem / solution
  or whatever that needs to be said

  It can contain
    multiple lines
    multiple paragraphs
    blank lines
    references

  But each paragraph must be followed by a blank line

  However, you need to keep the 'Tracked-On', 'Change-Id',
  'Signed-off-by', ..., fields in the **lastest** paragraph of
  the commit message.

  Change-Id: I9e58f79c7faf7f0edb9792e8b5322ab161331141
  Tracked-On: https://jira.server.ext/browse/PROJECTA-123
  Signed-off-by: Developer Name <developer.name@intel.com>

Expected syntax:
Commit message one-line summary
  Extra paragraphs can be added here.

  Longer description that explains the problem / solution
  or whatever that needs to be said

  It can contain
    multiple lines
    multiple paragraphs
    blank lines
    references

  But each paragraph must be followed by a blank line

  However, you need to keep the 'Tracked-On', 'Change-Id',
  'Signed-off-by', ..., fields in the **lastest** paragraph of
  the commit message.

  Change-Id: I9e58f79c7faf7f0edb9792e8b5322ab161331141
  Tracked-On: https://jira.server.ext/browse/PROJECTA-123
  Signed-off-by: Developer Name <developer.name@intel.com>

Expected syntax:
Commit message one-line summary
  Extra paragraphs can be added here.

  Longer description that explains the problem / solution
  or whatever that needs to be said

  It can contain
    multiple lines
    multiple paragraphs
    blank lines
    references

  But each paragraph must be followed by a blank line

  However, you need to keep the 'Tracked-On', 'Change-Id',
  'Signed-off-by', ..., fields in the **lastest** paragraph of
  the commit message.

  Change-Id: I9e58f79c7faf7f0edb9792e8b5322ab161331141
  Tracked-On: https://jira.server.ext/browse/PROJECTA-123
  Signed-off-by: Developer Name <developer.name@intel.com>

repo: projectceladon/device-intel-sepolicy