text:
MongoDB: Upsert fails on update with relation when notablescan: 1

Bug description
Using MongoDB, the following error occurs when running an upsert and the document already exists:
PrismaClientUnknownRequestError:
Invalid `prisma.user.upsert()` invocation:

  Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: RawDatabaseError { code: "unknown", message: "Command failed (NoQueryExecutionPlans): error processing query: ns=ninja-diary.projectsTree: $expr {$and: [{$in: [\"$ownerId\", {$const: [\"qGdWa0VNzHfW8GRaDxz5nfjPDRt1\"]}]}, {$or: [{$ne: [{$ifNull: [\"$ownerId\", {$const: null}]}, {$const: null}]}, {$eq: [\"$ownerId\", {$const: null}]}]}]}Sort: {}\nProj: { _id: true, ownerId: true }\nCollation: { locale: \"simple\" }\n planner returned error :: caused by :: No indexed plans available, and running with 'notablescan')" } })
    at Object.request (/usr/app/node_modules/@prisma/client/runtime/index.js:45582:15)
    at async PrismaClient._request (/usr/app/node_modules/@prisma/client/runtime/index.js:46405:18)
    at async setSyncTimeForUser (/usr/app/src/features/sync/utils.ts:29:18)
    at async /usr/app/src/features/sync/push.ts:19:5
    at async Proxy._transactionWithCallback (/usr/app/node_modules/@prisma/client/runtime/index.js:46366:18)
    at async push (/usr/app/src/features/sync/push.ts:15:3)
    at async /usr/app/src/app.ts:442:7 {
  clientVersion: '3.12.0'
}

How to reproduce
model User {
  id       String    @id @map("_id") @db.String
  name String
  lastSync DateTime  @updatedAt
  projects Project[]

  @@map("users")
}

model Project {
  id        String   @id @map("_id") @db.String
  name      String
  owner     User     @relation(fields: [ownerId], references: [id])
  ownerId   String   @db.String
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
  isDeleted Boolean  @default(false)

  @@index([ownerId])
  @@index([createdAt])
  @@index([updatedAt])
  @@index([isDeleted])
  @@map("projects")
}

Run this once:
  await prisma.user.upsert({
    where: { id: userId },
    create: { id: userId, name: "foo" },
    update: { name: "foo2" },
  });

This will create the document, which works.
Then run it a second time, which will execute the "update" part since the document exists. It now fails with the error above.
If I remove the relation from the schema (and only have a "dum" ownerId field) the opertation works again.
Expected behavior
It should not fail
Prisma information
Prisma v3.12.0
mongoDB 4.4
model User {
  id       String    @id @map("_id") @db.String
  name String
  lastSync DateTime  @updatedAt
  projects Project[]

  @@map("users")
}

model Project {
  id        String   @id @map("_id") @db.String
  name      String
  owner     User     @relation(fields: [ownerId], references: [id])
  ownerId   String   @db.String
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
  isDeleted Boolean  @default(false)

  @@index([ownerId])
  @@index([createdAt])
  @@index([updatedAt])
  @@index([isDeleted])
  @@map("projects")
}

Run this once:
  await prisma.user.upsert({
    where: { id: userId },
    create: { id: userId, name: "foo" },
    update: { name: "foo2" },
  });

Environment & setup

OS: debian
Database: MongoDB Atlas v4.4
Node.js version: 16.13

Prisma Version
prisma                  : 3.12.0
@prisma/client          : 3.12.0
Current platform        : debian-openssl-1.1.x
Query Engine (Node-API) : libquery-engine 22b822189f46ef0dc5c5b503368d1bee01213980 (at node_modules/@prisma/engines/libquery_engine-debian-openssl-1.1.x.so.node)
Migration Engine        : migration-engine-cli 22b822189f46ef0dc5c5b503368d1bee01213980 (at node_modules/@prisma/engines/migration-engine-debian-openssl-1.1.x)
Introspection Engine    : introspection-core 22b822189f46ef0dc5c5b503368d1bee01213980 (at node_modules/@prisma/engines/introspection-engine-debian-openssl-1.1.x)
Format Binary           : prisma-fmt 22b822189f46ef0dc5c5b503368d1bee01213980 (at node_modules/@prisma/engines/prisma-fmt-debian-openssl-1.1.x)
Default Engines Hash    : 22b822189f46ef0dc5c5b503368d1bee01213980
Studio                  : 0.459.0
Preview Features        : interactiveTransactions, extendedIndexes

Hm, I can not reproduce the problem here.
I can run this script many times with the schema you provided:
const { PrismaClient } = require('@prisma/client')

const prisma = new PrismaClient()

async function main() {
  let userId = "3"
  await prisma.user.upsert({
    where: { id: userId },
    create: { id: userId, name: "foo" },
    update: { name: "foo2" },
  });
}

main()
  .catch((e) => {
    console.error(e)
    process.exit(1)
  })
  .finally(async () => {
    await prisma.$disconnect()
  })

This is tested with MongoDB Atlas Serverless, which runs 5.3.1.
(Waiting for 4.4 to be provisioned now)

I can "unfortunately" also not reproduce this with a MongoDB Atlas running 4.4.
Did you try the reproduction you provided in a new database? Maybe the existing data or stat of the server influences what you are seeing.

I am also unable to reproduce the issue.

@janpio @dpetrick Was notablescan enabled on the cluster you tried ? (Require Indexes for All Queries option in advanced cluster configuration)

It seems the operation fails at this step:
db.Proj.aggregate([ { $match: { $expr: { $and: [ { }, { $and: [ { $in: [ "$ownerId", [ "x", ], ], }, { $or: [ { $ne: [ { $ifNull: [ "$ownerId", null, ], }, null, ], }, { $eq: [ "$ownerId", null, ], }, ], }, ], }, ], }, }, }, { $project: { _id: 1, ownerId: 1, }, }, ])

But I cannot figure out which index is missing...

@janpio yes please 😄
You can also enable with mongo client / shell
db.admin().command({ setParameter: 1, notablescan: 1 });

Sorry for not specifying this earlier, I was unaware of the existence of this configuration setting until my followup investigation yesterday.

I've found a lot more cases where this happens. It seems the generated aggregate commands fails to find an index when performing operations only on the _id field.
For example, this fails:
txn.project.updateMany({
  where: {
    id: { in: projectIds }, // array of project ids
  },
  data: {
    updatedAt: timestamp,
    isDeleted: true,
  },
});

But the following succeeds:
txn.project.updateMany({
  where: {
    id: { in: projectIds }, // array of project ids
    isDeleted: false
  },
  data: {
    updatedAt: timestamp,
    isDeleted: true,
  },
});

At least the warning is explicit:

Will test when configuration is applied.
(If you have the time and means, figuring out if this also happens with newer versions would be interesting as well)

@janpio I am able to reproduce locally with mongo v5.0.6 and prisma v3.13.0

I did some more investigating and the issue seems broader, as a lot of operations seem to cause a collection scan instead of using an index.
Specifically, I have found that using the in operator always causes an aggregation command that fails to use an index. For example
prisma.project.findMany({
  where: {
    id: { in: [uuid()] },
  },
});

causes a collection scan. I have confirmed this by running explain on the aggregate command prisma ends up running (using query logging).
I added a couple million objects to the collection, and running the findMany command takes over 300ms locally vs <5ms when using mongoDB driver

Temporary conclusion then if we can reproduce any of this: Prisma MongoDB is currently not compatible with notablescan: 1 - correct?

@janpio yes correct.

I can unfortunately confirm this:
PS C:\Users\Jan\Documents\throwaway\12793> node .\script.js
PrismaClientUnknownRequestError:
Invalid `prisma.user.upsert()` invocation:

  Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: RawDatabaseError { code: "unknown", message: "Command failed (NoQueryExecutionPlans): error processing query: ns=12793.projectsTree: $expr {$and: [{$in: [\"$ownerId\", {$const: [\"3\"]}]}, {$or: [{$ne: [{$ifNull: [\"$ownerId\", {$const: null}]}, {$const: null}]}, {$eq: [\"$ownerId\", {$const: null}]}]}]}Sort: {}\nProj: { _id: true, ownerId: true }\nCollation: { locale: \"simple\" }\n planner returned error :: caused by :: No indexed plans available, and running with 'notablescan')" } })
    at Object.request (C:\Users\Jan\Documents\throwaway\12793\node_modules\@prisma\client\runtime\index.js:45633:15)
    at async PrismaClient._request (C:\Users\Jan\Documents\throwaway\12793\node_modules\@prisma\client\runtime\index.js:46456:18)
    at async main (C:\Users\Jan\Documents\throwaway\12793\script.js:8:5) {
  clientVersion: '3.13.0'
}

(Tested with MongoDB 4.4 on MongoDB Atlas, via setting Configuration => Additional Setings => More Configuration Options => Require Indexes for All Queries => "On")
Now that we know that, would you expect this query to work (= necessary index being present) with your current schema?

Simplified schema for further reproduction:
generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "mongodb"
  url      = env("DATABASE_URL")
}

model User {
  id       String    @id @map("_id") @db.String
  name String
  projects Project[]
}

model Project {
  id        String   @id @map("_id") @db.String
  owner     User     @relation(fields: [ownerId], references: [id])
  ownerId   String   @db.String
}

and script with query logging:
const { PrismaClient } = require('@prisma/client')

const prisma = new PrismaClient({ log: ["query"]})

// A `main` function so that we can use async/await
async function main() {
    let userId = "3"
    await prisma.user.upsert({
        where: { id: userId },
        create: { id: userId, name: "foo" },
        update: { name: "foo2" },
      });
}

main()
  .catch((e) => {
    console.error(e)
    process.exit(1)
  })
  .finally(async () => {
    await prisma.$disconnect()
  })

Internal discussion about the query and index usage: https://prisma-company.slack.com/archives/C0239SYUZV1/p1652120382980849
TLDR: The current query structure makes it really hard / impossible for the query to be supported by indexes, because of the $or (which makes sure it's not null although we know we are searching for a value).

@janpio To answer your question, I do not think it is a missing index on my end. I have spent a very long time investigating this and tested all kinds of indexes on all collections.
FYI: Much simpler operations also fail (without relations)

prisma.project.findMany({
  where: {
    id: { in: [uuid()] },
  },
});

As written, the way the query above is generated internally by Prisma it is really hard / impossible for an index to be used.
The same might be true for the other one you just posted. Optimally enable query logging and post the prisma:query output for that Prisma Client query as well.

I've created a benchmark against mongoose, it is described in more details here: https://github.com/prisma/prisma/discussions/14811

Is there an ETA on when the performance issues outlined in this issue and https://github.com/prisma/prisma/issues/14871 will be addressed ?
I was hoping for some improvement before I had to go to production, but that is looking less and less likely, so I am planning to replace prisma with mongoose in all our mongo-backed services.
Is there any chance for performance improvement within the next 1-2 months ?

The 'problem' is in your client config - search for
mongoRsResource.Client.DisableTableScan();
This prevents to query to fields without index. If you enable TableScan, you can shoot your performance into nirvana if you have a large database.

@AndreasKarz the issue is that the prisma client generates overly complex queries, which force a table scan. My db has disabled table scans.

As written, the way the query above is generated internally by Prisma it is really hard / impossible for an index to be used.
The same might be true for the other one you just posted. Optimally enable query logging and post the prisma:query output for that Prisma Client query as well.

Is performance problem mentioned in Prisma documentation? Because this is a huge factor while choosing an ORM.

This prevents to query to fields without index. If you enable TableScan, you can shoot your performance into nirvana if you have a large database.

@AndreasKarz the intention isn't to run with notablescan in production, but to run with it in development to catch potential performance issues in development and add appropriate indexes before moving to production to ensure the fast running of queries.

any updates on this issue ? seems like indexes are not used either with include parameter.

@AmineHouari98 I encourage you to add your case to https://github.com/prisma/prisma/issues/14871 there are many queries with performance issues that are not documented

I am also having issues where trying to just find documents greater than a certain date
const start = dayjs(Date.now()).subtract(8, 'hours').toDate();

return this.prismaService.tarp.findMany({
  where: {
    timestamp: {
       gt: start,
     },
  },
});

Ends up in a query like
"db.TARPDocuments.aggregate([ { $match: { $expr: { $and: [ { $gt: [ \"$timestamp\", { $literal: DateTime(\"2023-06-02 6:59:24.852 +00:00:00\"), }, ], }, { $ne: [ \"$timestamp\", \"$$REMOVE\", ], }, ], }, }, }, { $project: { _id: 1, installationId: 1, vehicleId: 1, timestamp: 1, tyreId: 1, hidden: 1, overridden: 1, injuryTemperature: 1, injuryHeight: 1, injuryWidth: 1, tyreAbsoluteTemp: 1, tyreDifferentialTemp: 1, injuryDifferentialTemp: 1, }, }, ])"

Due to the $expr this doesn't seem to be taking advantage of indexes on the database resulting in a query time of 2s as it tries to search through hundreds of thousands of documents.

any updates on this issue ? seems like indexes are not used either with include parameter.

Same here, just found our atlas have so much collscan for in queries but the fields are all created unique index on

any updates on this issue ? seems like indexes are not used either with include parameter.

We're encountering the same issue with 'include' after going to production. We were able to hack around it for now. However, we will likely migrate off of Prisma if a fix doesn't come sometime soon which stinks since the DX has been amazing outside of the performance issues w/ mongodb.

This (using in) has just caused us an enormous ($$$) bill on mongodb atlas (serverless bills by usage), due to prisma running full table scans and ignoring indexes. I was shocked and disappointed to learn that this is known issue without a workaround. There should be a warning for everyone to not use prisma with mongodb currently.

Yep, if I know before hand that bad performance means not using index and costing me 100x more then I wouldn't have gone with Prisma. I would have used something like Mongoose instead.

something like Mongoose instead

My recommendation is https://mikro-orm.io/. Haven't used it with MongoDB, but with PG it was amazing. And of course migrating off MongoDB mid-term in most use-cases.

This is extremely disappointing and a bit deceiving to hide from Prisma Mongo users. the fact a simple where _id in [] won't use an index is shocking. We will be migrating off Prisma. and to think we were considering accelerate 😞

Speaking of, the original problem with upsert() when it goes into update has also been fixed in 5.14.0.
Here the original reproduction with 3.12.0 again:
PS C:\Users\Jan\Documents\throwaway\12793> node .\script.js
prisma:query db.User.aggregate([ { $match: { $expr: { $and: [ { $and: [ { $eq: [ "$_id", "4", ], }, { $or: [ { $ne: [ { $ifNull: [ "$_id", null, ], }, null, ], }, { $eq: [ "$_id", null, ], }, ], }, ], }, ], }, }, }, { $project: { _id: 1, }, }, ])
prisma:query db.Project.aggregate([ { $match: { $expr: { $and: [ { }, { $and: [ { $in: [ "$ownerId", [ "4", ], ], }, { $or: [ { $ne: [ { $ifNull: [ "$ownerId", null, ], }, null, ], }, { $eq: [ "$ownerId", null, ], }, ], }, ], }, ], }, }, }, { $project: { _id: 1, ownerId: 1, }, }, ])
PrismaClientUnknownRequestError:
Invalid `prisma.user.upsert()` invocation:

  Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: RawDatabaseError { code: "unknown", message: "Command failed (NoQueryExecutionPlans): error processing query: ns=12793repro2.ProjectTree: $expr {$and: [{$in: [\"$ownerId\", {$const: [\"4\"]}]}, {$or: [{$ne: [{$ifNull: [\"$ownerId\", {$const: null}]}, {$const: null}]}, {$eq: [\"$ownerId\", {$const: null}]}]}]}\nSort: {}\nProj: { _id: true, ownerId: true }\nCollation: { locale: \"simple\" }\n planner returned error :: caused by :: No indexed plans available, and running with 'notablescan')" } })
    at Object.request (C:\Users\Jan\Documents\throwaway\12793\node_modules\@prisma\client\runtime\index.js:45582:15)
    at async PrismaClient._request (C:\Users\Jan\Documents\throwaway\12793\node_modules\@prisma\client\runtime\index.js:46405:18)
    at async main (C:\Users\Jan\Documents\throwaway\12793\script.js:8:5) {
  clientVersion: '3.12.0'
}

And the same script run with 5.14.0:
PS C:\Users\Jan\Documents\throwaway\12793> node .\script.js
prisma:query db.User.aggregate([ { $match: { $expr: { $and: [ { $and: [ { $and: [ { $eq: [ "$_id", { $literal: "4", }, ], }, { $ne: [ "$_id", "$$REMOVE", ], }, ], }, ], }, { }, ], }, }, }, { $project: { _id: 1, }, }, ])
prisma:query db.User.updateMany({ _id: { $in: [ "4", ], }, },{ $set: { name: { $literal: "foo2", }, }, })
prisma:query db.User.aggregate([ { $match: { $expr: { $and: [ { $and: [ { $eq: [ "$_id", { $literal: "4", }, ], }, { $ne: [ "$_id", "$$REMOVE", ], }, ], }, ], }, }, }, { $project: { _id: 1, name: 1, }, }, ])

Yay!

authors: AmineHouari98 AndreasKarz DoubleJ-G boredland domharrington dpetrick janpio jvgaeta off99555 paulrostorp peymil shaikegross yufangOPUS

repo: prisma/prisma


text:
Does Prisma offer Transactions across multiple files?

Discussed in https://github.com/prisma/prisma/discussions/12974

Originally posted by jaequery April 23, 2022
Coming from TypeORM, I am surprised to see that Prisma does not have any way to perform transactions in a similar way TypeORM does.
With TypeORM and the transactional-cls-hooked decorator, we can do such as:
@Transactional()
async updateManyThings() {
   await this.productService.updateStuff();
   await this.categoryService.updateAnotherStuff();
   await this.customerService.changeOtherStuff();
}

This allows so that if there is an error on any of the service calls, it'd safely rollback.
Does Prisma offer anything close to that? Only thing that I've come across is to perform the operation all inside a single method, which is not all that useful for us, as we rely heavily on using service files to perform mutable operations respectively.

https://www.prisma.io/docs/concepts/components/prisma-client/transactions
I think this is what you need.

https://www.prisma.io/docs/concepts/components/prisma-client/transactions
I think this is what you need.

I am not sure if any of them works across different files but do correct me if I'm wrong.

Any thought on doing this @jaequery, @salvajavier: Returning $transaction to the user and pass it to the API interface as a argument. Something like this:
const transactionObj = prisma.createTransaction();
const user = await prisma.user.create({
  data: {
    email: 'elsa@prisma.io',
    name: 'Elsa Prisma',
  },
  { transactions: transactionObj }
})
const temp = prisma.temp.update({ where: { id: 1}, data: {f1: 1},  { transactions: transactionObj }})

@jaequery, @kasir-barati I think the point is that you cannot return a transaction built in another function because if you do it just run. So as the prisma.transaction$ functions receives an array as argument I do this:
async function mainFunction(){

    //This function can be anywhere an returns an array of transactions.
    let prismaExternalOperations = await getPrismaOperation()

    let prismaLocalOperation= prisma.product.update()

    // Use the spread operator to pass to prisma.transactions$ function all the arguments
    // returned from getPrismaOperation
    prisma.transactions$([prismaLocalOperation, ...prismaExternalOperations])
}

function prismaOperation (data) {
    let transactions$ = []

    let firstTransaction = prisma.customer.create({data})
    let secondTransaction = prisma.local.create({data})

    transactions$.push(firstTransaction, secondTransaction)

    // Instead of returning the prisma operation you wrap them into an array so they can be returned without executing.
    return transactions$
}

I'm facing this same problem. I created repositories for each of my database entities, to be able to isolate the prism of my use cases. But in services where I need to interact with different repositories, I get stuck in not being able to control the transaction of the repositories' methods, in a centralized way.
Is there any existing way to resolve this?

Hi, I'm from the Issues https://github.com/prisma/prisma/issues/12458 @millsp said I should look there for other issues.
I am working in a NestJs project with the CleanArchitecture. That means, we have our business logic in a domain layer and call single accessors from there, which then call prisma functions around a model (e.g. user).
Since we have business logic that needs to access multiple accessors, we can't use the callback transaction at all.
Here is a super simplified example:
@Injectable()
export class UserAccessor {
  constructor(private prismaClient: PrismaClient) {
  }

  async create(...): Promise<User> {
    return this.prismaClient.user.create(...)
  }
}

@Injectable()
export class SettingsAccessor {
  constructor(private prismaClient: PrismaClient) {
  }

  async create(user: User): Promise<Settings> {
    return this.prismaClient.userSettings.create(...)
  }
}

@Injectable()
export class UserDomain {
  constructor(private readonly userAccessor: UserAccessor,
              private readonly settingsAccessor: SettingsAccessor,
  ) {
  }

  async createUser(...) {
    // start Transaction
    const user = await this.userAccessor.create(...)
    // maybe more stuff done between this accessor calls
    const settings = await this.settingsAccessor.create(user)
    // end Transaction
  }
}

As @sijakubo pointed this out in the other Issue, Java/Spring/Hibernate uses this too in form of the @Transactional() Annotation around a service or function and also in tests. Spring Transaction Management: @Transactional In-Depth
This Topic is also Discussed in https://github.com/prisma/prisma/issues/13004

I have the same problem. I use CleanArchitecture and I can't work with transactions.

I don't have the time to understand the internals of Prisma but I believe such feature can work by utilizing a series of hooks and event emitters. Has anyone from Prisma attempted to replicate the @Transactional() decorator? I'm curious if there is some kind of a technological blocker that makes it impossible atm.

I'll add onto this. Utilizing NestJS and cannot utilize transactions across multiple services.

Also looking for a solution to this. No way, it seems, to create a transaction which contains Prisma client invocations inside other classes. Also using NestJS

What i ended up doing was for each function passing in an optional PrismaClient parameter. If the parameter exists, it uses that context instead of the one native to the controller/service.
It's tedious for sure, had a couple hundred -_-.

I am using NestJS and have considered replacing it with TransactionClient for the inability to share transactions between services. Doing this would eliminate the need to pass PrismaClient as a method argument. (This is useful for processing outside of the application, e.g., batch processing.)
An example is below.
import { Test } from '@nestjs/testing';
import { Prisma } from '@prisma/client';
import { AppModule } from 'src/app.module';
import { PrismaService } from 'src/prisma/prisma.service';

export const getModuleWithTransactionClient = async (
  prisma: Prisma.TransactionClient,
) => {
  return await Test.createTestingModule({
    imports: [AppModule],
  })
    .overrideProvider(PrismaService)
    .useValue(prisma)
    .compile();
};

use this
    await prisma.$transaction(async (prisma) => {
      const module = await getModuleWithTransactionClient(prisma);

      const service = module.get<StudentAdminService>(StudentAdminService);
      const service2 = module.get<ClassMemberAdminService>(
        ClassMemberAdminService,
      );

However, I don't think this is the right approach and hope a better way is implemented.

I wanted to use Prisma, migrating from TypeORM, until I researched this, and it's basically a deal breaker. We require a way to keep all our resource DB logic in their respective services while being able to get executed in single transactions.

Yes, dealbreaker here as well.

Related: https://github.com/prisma/prisma/issues/12458

@janpio this is important feature for us and would be great if this is prioritized.

Here is a workaround with prisma.$extends
Creating request context
import { RequestContext } from 'nestjs-request-context';

export type PrismaTransactionClient = Omit<PrismaClient, ITXClientDenyList>;

export class AppRequestContext extends RequestContext {
  public prismaTransactionClient: PrismaTransactionClient;
}

export class RequestContextService {
  public static getContext(): AppRequestContext {
    const ctx: AppRequestContext = RequestContext.currentContext.req;
    return ctx;
  }

  public static setPrismaTransaction(
    prismaTransactionClient: PrismaTransactionClient,
  ): void {
    const ctx = this.getContext();
    ctx.prismaTransactionClient = prismaTransactionClient;
  }

  public static getPrismaTransaction(): PrismaTransactionClient {
    const ctx = this.getContext();
    return ctx.prismaTransactionClient;
  }

  public static cleanPrismaTransaction(): void {
    const ctx = this.getContext();
    ctx.prismaTransactionClient = undefined;
  }
}

Creating extended prima client
const extendedPrismaClient = (client: PrismaClient) =>
  client.$extends({
    client: {
      // async onModuleInit() {
      // Uncomment this to establish a connection on startup, this is generally not necessary
      // https://www.prisma.io/docs/concepts/components/prisma-client/working-with-prismaclient/connection-management#connect
      // await Prisma.getExtensionContext(this).$connect();
      // },
      enableShutdownHooks(app: INestApplication) {
        Prisma.getExtensionContext(client).$on('beforeExit', async () => {
          await app.close();
        });
      },
      async $transaction<T>(handler: () => Promise<T>) {
        return Prisma.getExtensionContext(client).$transaction(async (tx) => {
          if (!RequestContextService.getPrismaTransaction()) {
            RequestContextService.setPrismaTransaction(tx);
          }

          try {
            const result = await handler();
            console.debug(`[${new Date()}] transaction committed`);
            return result;
          } catch (e) {
            console.debug(`[${new Date()}] transaction aborted`);
            throw e;
          } finally {
            RequestContextService.cleanPrismaTransaction();
          }
        });
      },
    },
    query: {
      $allModels: {
        $allOperations({ model, operation, args, query }) {
          const tx = RequestContextService.getPrismaTransaction();

          if (tx) {
            // eslint-disable-next-line @typescript-eslint/no-unsafe-call
            return tx[model][operation](args);
          }

          return query(args);
        },
      },
    },
  });

export type IExtendedPrismaClient = new (
  options?: Prisma.PrismaClientOptions,
) => ReturnType<typeof extendedPrismaClient>;

export const ExtendedPrismaClient = <IExtendedPrismaClient> class {
  constructor(options?: Prisma.PrismaClientOptions) {
    const client = new PrismaClient(options);

    // eslint-disable-next-line no-constructor-return
    return extendedPrismaClient(client);
  }
};

Extending in prisma service
@Injectable()
export class PrismaService extends ExtendedPrismaClient {
  constructor() {
    super({
      datasources: {
        db: { url: DatabaseConfig.databaseUrl },
      },
    });
  }
}

Using transaction
await this.prisma
      .$transaction(async () => {
        const user = await this.prisma.user.create({
          data: {
            firstName: 'fsdf',
            email: 'huil',
          },
        });
      })

+1
I'm passing the prisma instance as the function parameter, my code soon looks ugly for sure

I'm developing a clean architecture application in Nest.js and I am tired of using Prisma's TransactionClient between modules and I came across this problem looking for a good good solution.
I implemented the PrismaClientManager by AsyncLocalStorage with nestjs-cls that makes it easier to use transactions, although it is implemented using Nest.js, so please take a look if you have the same problem.
https://github.com/uki1014/nestjs-prisma-client-manager

@uki1014 i have implemented similar solution. And works as expected until now. Thanks for sharing. I believe this solution can be used whoever is seeking to solve this problem.

@yigitahmetsahin is it publicly available?
@uki1014 Just a quick feedback after just a glance (😅): It would be much much better if you could implement it using decorators. Not sure how though. But in Typeorm you only need to pass the entity's ref to the transaction. And as such you can just create a module and pass them to that module in order to know which module you should have in order to add that decorator.

@kasir-barati no i don't have it available publicly, but my solution is very similar to @uki1014. I believe much better to bound in functions so that you will have more control over where transaction start and end. Decorator wraps whole method which might not be ideal for all cases.

@kasir-barati @yigitahmetsahin
yes, I do not think decorators are a good solution to this problem.
For example, if you annotate @Transaction for a service method, it would be easy to exceed the timeout limit of the transaction if the in-memory logic or read SQL is taking a long time.
It would encourage stretching the timeout in that case, but I don't think that is a good idea because it would stretch the database locks.
Better would be to be able to initiate a TRANSACTION when you finally write with the read data and parameters.
keeping the scope of the transaction to a minimum is an important option.
@Transaction
class SomeService {
  async create() {
    // slow query
    const someData = await prisma.some.findMany(...)

    // Slow queries increase the probability of timeout.
    await prisma.other.create(...);
  }
}

class SomeService {
  async create() {
    // slow query
    const someData = await prisma.some.findMany(...)

    await this.prismaClientManager.transaction(async () => {
      await prisma.other.create(...);
    })
  }
}

@uki1014 Acknowledged, But still I believe that you need to decorate your method with @transaction and not your class. Besides if you only wanna only wrap some part of your login in a transaction you can alway opt to have a new method called someSelfDescriptiveName and decorate it with @transaction. This way you are illuminating cognitive complexity.
Also I guess we could have both 🤔, right?
TBF IDK much about how easy each one of these two would be in unit test too. So frankly speaking I guess I am just too much influenced by decorators 😂

@uki1014 hey, I have implemented similar above and we both have issue with background jobs. This solution only works in context of a request. I have implemented it to work anywhere, might share it later if interested

@Kainar01 would be nice if you can also share your solution so that we can get benefit of the alternative approach to this problem rather than limiting it to request context.

@Kainar01 any update from your solution which can work without context of a request?

Happy new year all!
@yigitahmetsahin sorry for taking long. Here is the snippet I use.
I created context manager with async_hooks
import { AsyncLocalStorage } from 'async_hooks';
import { PrismaClient } from '@prisma/client';
import { ITXClientDenyList } from '@prisma/client/runtime/library';

export type PrismaTransactionClient = Omit<PrismaClient, ITXClientDenyList>;

type Context = {
  transaction: PrismaTransactionClient;
};

export class TransactionContextService {
  private static readonly cls = new AsyncLocalStorage<Context>();

  static getTransaction() {
    const context = this.getContext();

    return context?.transaction;
  }

  private static getContext(): Context | undefined {
    return this.cls.getStore();
  }

  static async runInContext<T>(tx: PrismaTransactionClient, handler: () => Promise<T>): Promise<T> {
    return this.cls.run({ transaction: tx }, handler);
  }
}

And created extended prisma client
import { Prisma, PrismaClient } from '@prisma/client';

import { TransactionContextService } from './transaction-context.service';

/**
 * This function creates a new transaction if there is none in the current context.
 * It will commit the transaction if it completes successfully, or abort if an error is encountered.
 * @param {PrismaClient} client - The PrismaClient instance.
 * @param {Function} handler - The function that will be performed within the transaction.
 * @return {Promise<T>} - The result of the transaction.
 */
async function performTransaction<T>(client: PrismaClient, handler: () => Promise<T>) {
  if (TransactionContextService.getTransaction()) {
    return handler();
  }

  return client.$transaction(async (tx) => TransactionContextService.runInContext(tx, handler));
}

function createExtendedPrismaClient(options?: Prisma.PrismaClientOptions) {
  const client = new PrismaClient(options);

  return client.$extends({
    client: {
      async $transaction<T>(handler: () => Promise<T>) {
        return performTransaction(client, handler);
      },
    },
    query: {
      $allModels: {
        $allOperations({ model, operation, args, query }) {
          const tx = TransactionContextService.getTransaction();

          if (tx) {
            // eslint-disable-next-line @typescript-eslint/no-unsafe-call
            return tx[model][operation](args);
          }

          return query(args);
        },
      },
    },
  });
}

export type IExtendedPrismaClient = new (options?: Prisma.PrismaClientOptions) => PrismaClient & {
  $transaction: <T>(handler: () => Promise<T>) => Promise<T>;
};

export const ExtendedPrismaClient = class {
  constructor(options?: Prisma.PrismaClientOptions) {
    // eslint-disable-next-line no-constructor-return
    return createExtendedPrismaClient(options);
  }
} as IExtendedPrismaClient;

Initiate as regular prisma client and use it in code as this:
 this.prisma.$transaction(async () => {
  await this.prisma.account.create(); // transaction taken from context
})

@Kainar01 thanks for sharing this detailed answer. I have one question, you are using AsyncLocalStorage for storing the transaction context. How this can work without request? Because as far as i know async local storage is only present if there is a request.

@yigitahmetsahin from the nestjs docs

It is similar to a thread-local storage in other languages.

It is not necessarily bound to request

I just wanted to give me 2 cents on this topic because the absence of @Transactional decorator is also being very damaging to my clean architecture application here.
The reality is that every single solution presented here "solves" one or another use case, the ideal world is where we could control the transactions on method level by passing properties like Spring Framework does, it's the gold standard.
TypeORM folks have developed a literal copy of this functionality here:
https://www.npmjs.com/package/typeorm-transactional
The idea is to simply copy their idea and do this via AsyncLocalStorage or nest-cls package, considering all the options.
Unfortunately I'm running against time else I'd spend some days trying to implement this myself.

@renatoaraujoc can you share some link to the way that "Spring Framework" handles transaction in the way you have explained it. Btw I don't remember that typeorm-transactional decorator gives you the ability to pass properties to it.
I also didn't get what you meant by properties, do you mean like passing entities as prop to indicate which tables should be included in the transaction. 🤔

by looking at the source code of the TypeORM transactional repo, it looks like they are using EventEmitters to signal start and end of transaction.
I just used ChatGPT to help write something up here but i think something like this can work:
import { PrismaClient } from '@prisma/client';
const prisma = new PrismaClient();

// Transactional decorator
function Transactional() {
  return function (target: any, propertyName: string, descriptor: PropertyDescriptor) {
    const originalMethod = descriptor.value;

    descriptor.value = async function (...args: any[]) {
      return await prisma.$transaction(async (transaction) => {
        // Add transaction to the Prisma context for this execution
        const contextWithTransaction = { ...this, prisma: transaction };

        // Call the original method with the transaction context
        return await originalMethod.apply(contextWithTransaction, args);
      });
    };

    return descriptor;
  };
}

class MyService {
  // The service methods don't need to be aware of transactions
  @Transactional()
  async myMethod() {
    // Perform database operations using 'this.prisma'
    const data1 = await this.otherService.method();
    const data2 = await this.otherService2.method();
    const data3 = await this.otherService3.method();

    return this.findById(x);
  }
}

@renatoaraujoc can you share some link to the way that "Spring Framework" handles transaction in the way you have explained it. Btw I don't remember that typeorm-transactional decorator gives you the ability to pass properties to it.
I also didn't get what you meant by properties, do you mean like passing entities as prop to indicate which tables should be included in the transaction. 🤔

Their @Transactional decorator is very much like Springs, they accept the same isolation methods, etc, see:
https://github.com/Aliheym/typeorm-transactional/blob/master/src/decorators/transactional.ts

by looking at the source code of the TypeORM transactional repo, it looks like they are using EventEmitters to signal start and end of transaction.
I just used ChatGPT to help write something up here but i think something like this can work:
import { PrismaClient } from '@prisma/client';
const prisma = new PrismaClient();

// Transactional decorator
function Transactional() {
  return function (target: any, propertyName: string, descriptor: PropertyDescriptor) {
    const originalMethod = descriptor.value;

    descriptor.value = async function (...args: any[]) {
      return await prisma.$transaction(async (transaction) => {
        // Add transaction to the Prisma context for this execution
        const contextWithTransaction = { ...this, prisma: transaction };

        // Call the original method with the transaction context
        return await originalMethod.apply(contextWithTransaction, args);
      });
    };

    return descriptor;
  };
}

class MyService {
  // The service methods don't need to be aware of transactions
  @Transactional()
  async myMethod() {
    const data1 = await this.otherService.method();
    const data2 = await this.otherService2.method();
    const data3 = await this.otherService3.method();

    return this.findById(x);
  }
}

@jaequery it's a good start but it's not as simple as that since we can have @Transactional methods that call other @Transactional annotated methods, that's where the propagation kicks in.
In reality, you have to work with a proxied prisma object that will return the proper prisma object or the transaction object given if we're in a transaction and it's propagation.

any update on this issue guys?? i am facing the same problem with clean architecture.

source: gharchive/issue

authors: Kainar01 Rykuno Uki884 YahiaBadr adrian-goe bombillazo callumthomson jaequery janpio kasir-barati luatnd ramiresdallago renatoaraujoc roberto-souza salvajavier uki1014 yigitahmetsahin

license_type: permissive

license_source: github-api


text:
Citus extension support, create_distributed_table

Bug description
Under the hood Citus connects to the coordinator node (database server) that routes queries to the relevant worker node (another database server). Prisma on the other hand, connects to postgres as the default database if no database is specified.
❯ npx prisma migrate dev
Prisma schema loaded from prisma/schema.prisma
Datasource "db": PostgreSQL database "postgres", schema "public" at "localhost:5433"

When distributing tables through a migration, Prisma can't find the tables in the database:
Error:
❯ npx prisma migrate dev
Prisma schema loaded from prisma/schema.prisma
Datasource "db": PostgreSQL database "postgres", schema "public" at "localhost:5433"

Error: P3006

Migration `20220521072549_distribute_tables` failed to apply cleanly to the shadow database.
Error code: P1014
Error:
The underlying table for model `company` does not exist.

How to reproduce

git clone git@github.com:ruheni/citus-prisma-reproduction.git
cd citus-prisma-reproduction
npm install
npx prisma migrate dev
docker-compose up

Create a new migration to distribute the tables
npx prisma migrate dev --create-only --name distribute-tables

Edit new migration with Citus syntax to distribute tables:
SELECT create_distributed_table('Company','id');
SELECT create_distributed_table('Campaign','company_id');
SELECT create_distributed_table('Ad','company_id');
SELECT create_distributed_table('Click','company_id');
SELECT create_distributed_table('Impression', 'company_id');

Apply migration
npx prisma migrate dev

The second error in the bug description pops up
❯ npx prisma migrate dev
Prisma schema loaded from prisma/schema.prisma
Datasource "db": PostgreSQL database "postgres", schema "public" at "localhost:5433"

Error: P3006

Migration `20220521072549_distribute_tables` failed to apply cleanly to the shadow database.
Error code: P1014
Error:
The underlying table for model `company` does not exist.

Expected behavior

Connect to the right database server/ Let citus handle the database connection
Successfully distribute tables in the cluster

Prisma information
datasource db {
  provider = "postgres"
  url      = "postgresql://postgres:@localhost:5433"
}

generator client {
  provider = "prisma-client-js"
}

model Company {
  id BigInt @id @default(autoincrement()) @db.BigInt

  name      String   @db.Text
  image_url String?  @db.Text
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt

  campaign    Campaign[]
  ads         Ad[]
  clicks      Click[]
  impressions Impression[]
}

model Campaign {
  id BigInt @default(autoincrement()) @db.BigInt

  name           String @db.Text
  cost_model     String @db.Text
  state          String @db.Text
  monthly_budget BigInt @db.BigInt

  blacklisted_site_urls String[] @db.Text
  createdAt             DateTime @default(now())
  updatedAt             DateTime @updatedAt

  company    Company @relation(fields: [company_id], references: [id], onDelete: NoAction, onUpdate: NoAction)
  company_id BigInt

  ads Ad[]
  @@id([company_id, id])
}

model Ad {
  id                BigInt   @default(autoincrement()) @db.BigInt
  name              String   @db.Text
  image_url         String?  @db.Text
  target_url        String?  @db.Text
  impressions_count BigInt   @default(0)
  click_count       BigInt   @default(0)
  createdAt         DateTime @default(now())
  updatedAt         DateTime @updatedAt

  clicks      Click[]
  impressions Impression[]

  company    Company @relation(fields: [company_id], references: [id])
  company_id BigInt

  campaign            Campaign? @relation(fields: [campaign_company_id, campaign_id], references: [company_id, id], onDelete: NoAction, onUpdate: NoAction)
  campaign_company_id BigInt?
  campaign_id         BigInt?

  @@id([company_id, id])
}

model Click {
  id                 BigInt   @default(autoincrement()) @db.BigInt
  clicked_at         DateTime @db.Timestamp()
  site_url           String   @db.Text
  cost_per_click_usd Decimal  @db.Decimal(20, 10)
  user_ip            String   @db.Inet
  user_data          Json     @db.JsonB

  company    Company @relation(fields: [company_id], references: [id])
  company_id BigInt

  ad            Ad?     @relation(fields: [ad_company_id, ad_id], references: [company_id, id], onDelete: NoAction, onUpdate: NoAction)
  ad_company_id BigInt?
  ad_id         BigInt?

  @@id([company_id, id])
}

model Impression {
  id                  BigInt   @default(autoincrement()) @db.BigInt
  seen_at             DateTime @db.Timestamp()
  site_url            String   @db.Text
  cost_per_impression Decimal  @db.Decimal(20, 10)
  user_ip             String   @db.Inet
  user_data           Json     @db.JsonB

  company    Company @relation(fields: [company_id], references: [id])
  company_id BigInt

  ad            Ad?     @relation(fields: [ad_company_id, ad_id], references: [company_id, id], onDelete: NoAction, onUpdate: NoAction)
  ad_company_id BigInt?
  ad_id         BigInt?

  @@id(fields: [company_id, id])
}

Environment & setup

OS: Mac OS
Database: PostgreSQL
Node.js version: 16.13.1

Prisma Version
prisma                  : 3.14.0
@prisma/client          : 3.14.0
Current platform        : darwin
Query Engine (Node-API) : libquery-engine 2b0c12756921c891fec4f68d9444e18c7d5d4a6a (at node_modules/@prisma/engines/libquery_engine-darwin.dylib.node)
Migration Engine        : migration-engine-cli 2b0c12756921c891fec4f68d9444e18c7d5d4a6a (at node_modules/@prisma/engines/migration-engine-darwin)
Introspection Engine    : introspection-core 2b0c12756921c891fec4f68d9444e18c7d5d4a6a (at node_modules/@prisma/engines/introspection-engine-darwin)
Format Binary           : prisma-fmt 2b0c12756921c891fec4f68d9444e18c7d5d4a6a (at node_modules/@prisma/engines/prisma-fmt-darwin)
Default Engines Hash    : 2b0c12756921c891fec4f68d9444e18c7d5d4a6a
Studio                  : 0.460.0

Prisma only connects to the server defined in the connection string, which in your case I guess was the "coordinator node (database server)". The error message indicates that this goes wrong when migrate devs shadow database comes into play. I guess our logic to just create an additional database does not play well with how Citus then tries to store the tables via create_distributed_table.
We'll have to investigate further what exactly is going on.
Initial approach would be to create a second Citus instance manually and use that explicitly as the shadow database, and see what happens then: https://www.prisma.io/docs/concepts/components/prisma-migrate/shadow-database#cloud-hosted-shadow-databases-must-be-created-manually

any updates on this?

@janpio has the team had a chance to look at this? Are there any workarounds?

I believe I have discovered the issue: citus doesn't seem play well with Prisma's naming convention of using PascalCase for table names. I got this to work with the following schema, which maps to snake case for each table and column.
generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

model User {
  id              Int
  distributionKey Int    @map("distribution_key")
  name            String @map("name")
  favoriteColor   String @map("favorite_color")

  @@id([id, distributionKey])
  @@unique([distributionKey, name])
  @@map("user")
}

Here is the migration that worked:
CREATE EXTENSION IF NOT EXISTS citus;

SELECT create_distributed_table('user', 'distribution_key');

When trying to use PascalCase for the table name, the migration fails similar to above. Creating table names in snake case seems to be necessary for citus to work properly.

@janpio Is it possible to automatically map camel case in the prisma schema to snake case in the database? Needing to map every table and every field  would be error prone and basically a non-starter. I would imagine this would make working with Postgres much easier, as there is a strong preference to just use snake case there.

authors: derek-w-ds janpio ruheni simibruda


text:
Support @unique on nullable fields for databases that support it

It would be great if we could take advantage of Postgres's capability to express uniqueness on not null values.
Currently if I add the @unique constraint on a nullable field I can't have more than 1 record with a null assigned.
Further reading on how this can be expressed:
https://matjaz.it/tutorial-unique-constraint-on-null-values-in-postgresql/

Thanks @ctrlplusb for raising this issue. I can definitely see the value in this. I think the reason we don't support this yet is because Postgres is the only one of the databases that behaves nicely, the others reject duplicate NULLs. We'll look into this one.

Possibly related in SQLite: https://github.com/prisma/prisma/issues/3130

I would love to see this as I think it would solve the following use case.
In my app, a User has many Email like so:
model User {
  id                 String              @id @default(cuid())
  createdAt          DateTime            @default(now())
  updatedAt          DateTime            @default(now()) @updatedAt
  emails             Email[]
  username           String              @unique
}

model Email {
  id                 String    @id @default(cuid())
  createdAt          DateTime  @default(now())
  updatedAt          DateTime  @default(now()) @updatedAt
  email              String    @unique
  public             Boolean   @default(false)
  primary            Boolean   @default(false)
  owner              User      @relation(fields: [ownerId], references: [id])
  ownerId            String

  @@unique([ownerId, primary])
}

While the above works fine, it only allows you to add 2 emails. The one that's primary, and the one that's not.

Any news on this? Would be a nice feature to have.

Any news on this? Would be a nice feature to have.

Currently, we can't soft delete any records with a timestamp field (as per the docs) as @unique won't work (not actually deleted) and @unique(FieldGoesHere, deleted) doesn't recognise unique nulls.

In case anyone comes across my comment and is in a similar situation, we've worked around this by having two distinct
deleted: Boolean and deletedAt: Date fields in our prisma schema, and have manually added a partial unique postgres constraint to our SQL migrations (prisma doesn't support this but will ignore it) like so:
CREATE UNIQUE INDEX "User_unique_email" ON "User"("email", "deleted") WHERE ("deletedAt" is NULL);

@tomhoule I see you removed the process/candidate flag. Does this mean we shouldn't expect this in the near future?

@beeman Can't remember what the discussion was 12 days ago — the issue is more complicated than it seems, because unique constraint on nullable columns behave very differently between databases, and we'd have to reflect that in the client (unique constraints change the API we generate). Implementing this may require unpacking "index expressions" in the schema language, that would make the interaction between nulls and unique constraints clearer. In any case, this is work that has to be prioritized relatively to other important work, so we can't guarantee we'll work on it very soon. It's a feature people are interested in though, so it's definitely being taken into account.

In general, the candidate label only means someone thinks we should consider working on it in the next two weeks, not whether it's prioritized on a team level at any larger timescale. The product roadmap is where to find the big picture.

Thanks a lot for your detailed and quick reply @tomhoule, makes total sense..
Going to keep an eye on this and hope to see it land at some point on the roadmap.

For anyone stumbles on this, or related, issue, it is working in prisma ^2.20.1" in my case.
I have a simple user model, with a phone field, which can be optional, but must be unique when filled in.
I managed to create a user with this:
this.prisma.user.create({
      data: {
        email,
        name,
        phone: phone != null ? phone : undefined,
      },
    });

Now, the phone number can be undefined, or a unique input.
Cheers!

For anyone stumbles on this, or related, issue, it is working in prisma ^2.20.1" in my case.
I have a simple user model, with a phone field, which can be optional, but must be unique when filled in.
I managed to create a user with this:
this.prisma.user.create({
      data: {
        email,
        name,
        phone: phone != null ? phone : undefined,
      },
    });

Now, the phone number can be undefined, or a unique input.
Cheers!

I think the issue might appear when you create more users that don't have a phone number.
In my case, if I recall correctly, there could only be 1 item that has an undefined value.

Hi, any update about this?

I'd really love this feature too.

Also came across this issue when trying to implement soft deletes via a deleted_at DateTime? timestamp.
I want to make a query like:
await prisma.user.findUnique({
  where: {id_deleted_at: {id: '...', deleted_at: null}}
})
``

To do this I tried adding the a `@@unique([id, deleted_at])` index. However, Prisma requires all fields in a compound unique index to be non-nullable. As a result, I'm forced to change my query from a `findUnique` to `findFirst`, which results in losing the nice guarantees of uniqueness.

I also ran into this issue in the `primary Boolean` case described above. Would be great to have this feature natively.

I have been modify Soft delete middleware as the doc for support findUnique with alias:
import { assign, forOwn, isObject } from 'lodash';

...
      if (params.action == 'findUnique') {
        forOwn(params.args?.where, (value, key) => {
          if (isObject(value)) {
            delete params.args.where[key];
            params.args.where = assign(params.args.where, { ...value });
          }
        });
        // Change to findFirst - you cannot filter
        // by anything except ID / unique with findUnique
        params.action = 'findFirst';
        // Add 'deletedAt' filter
        // ID filter maintained
        params.args.where['deletedAt'] = null;
      }
...

Although it losing the nice guarantees of uniqueness, but working fine !
Sorry because I live lodash, so I use some utility of lodash 😂

I have been modify Soft delete middleware as the doc for support findUnique with alias:
import { assign, forOwn, isObject } from 'lodash';

...
      if (params.action == 'findUnique') {
        forOwn(params.args?.where, (value, key) => {
          // If the field is a compound unique index
          if (isObject(value)) {
            delete params.args.where[key];
            params.args.where = assign(params.args.where, { ...value });
          }
        });
        // Change to findFirst - you cannot filter
        // by anything except ID / unique with findUnique
        params.action = 'findFirst';
        // Add 'deletedAt' filter
        // ID filter maintained
        params.args.where['deletedAt'] = null;
      }
...

Although it losing the nice guarantees of uniqueness, but working fine !
Sorry because I live lodash, so I use some utilities of lodash alternative native javascript 😂

My use case I have a table that has self-referenced "parents" of which some rows have a parent and some do not. I need to update this table to add rows with self-parent, or null if no parent. I cannot seem to figure out how to do this across unique constraints...
eg,
@@unique([id, something, something2, parent])

parent: {
connect: {
table_id_something_something2_parent: {
id: id,
something: 'somevalue'
something2: 'somevalue'
parent: null // this doesnt work b/c it isnt ok with this being null, b/c the "parent" that i'm trying to find itself has no parent, hence null
}
}

This is a late reply, but I ran into this as well. My solution was a bit simpler (though less efficient, and not workable directly for the deleted_at case). I moved the nullable unique column to a related table. If the related record is added it must be unique (using the @unique) keyword. Not writing the record is equivalent to the null case.

any update on this, this is a real needed fearture, for example Mongodb has supporting for partial index through partialFilterExpression, this will help the nullable field to be unique only and only if the condition will met
eg:
database.collection('User').createIndex({ email: 1 }, {
      unique: true, partialFilterExpression: { userId: { '$exists': true, '$type': 7 } }
}),

the amail address will be checked for uniquness only if it exist and its type is a Mongo UUID
I think this is a must feature @lucasjohnston

I am also facing this issue but with SQL Server. I have a FKEY with NULL allowed as prisma schema validation tells me to add a null or convert it to a n-m relationship. SQL Server allows for a constraint to be added to Indexes as well such as:
CREATE UNIQUE NONCLUSTERED INDEX idx_yourcolumn_notnull ON YourTable(yourcolumn) WHERE yourcolumn IS NOT NULL;.
Ps. Is there any workaround I can use in the meantime?

@janpio Jumping in here to learn more about that status into this, clarify the problem, and hopefully get a workaround. @petrbela You made a comment that this isn't an issue, referencing non-prisma documentation. If you are able, the best solution will be to describe how this is a non-issue using PSL and the Prisma client to demonstrate.
Example problem: A contact entity has a one-to-one relationship with an address entity, but it's optional. This means, there exists an addressId in the contact table, holding null as a default wherever the end user has decided not to supply an address.
The desired schema:
model Contact {
  id              String            @id @default(cuid())
  firstName       String
  lastName        String
  email           String
  address         Address?         @relation(fields: [addressId], references: [id])
  addressId       String?          @unique     //GOAL: one-to-one relationship with nullable addressId

  @@map("contacts")
}

model Address {
  id          String       @id @default(cuid())
  addressOne  String
  addressTwo  String?
  city        String
  state       String
  zip         String
  Contact  Contact?

  @@map("addresses")
}

By not providing @unique to addressId, prisma throws an error on address within Contact stating that this relationship should be one-to-many if it is not @unique

Error parsing attribute "@relation": A one-to-one relation must use unique fields on the defining side. Either add an @unique attribute to the field addressId, or change the relation to one-to-many.

By providing @unique to addressId, the error everyone is talking about crops up, where only one Contact may have a null addressId, and all future instances result in an error.
Therefore, the ask is to allow a one-to-one relationship with nullable values. @petrbela since you are saying the postrgres treats null values as "not equal", do you have a PSL example that can help to achieve this?

@nickfreemandesign See here https://github.com/petrbela/prismatest. Uses sqlite but postgres and mysql treat nulls as distinct, too.

@petrbela can confirm that this example works in postgres! a thank you. which also has me scratching my head on where the other error came from... @beeman @ctrlplusb does the above repo demonstrate a solution to your use case?

I fixed mine my creating a custom migration
`-- This is an empty migration.
ALTER TABLE [dbo].[User] DROP CONSTRAINT [User_email_key];
CREATE UNIQUE NONCLUSTERED INDEX User_Email_Unique_AllowNull ON [dbo].[User] (email)
WHERE email IS NOT NULL;
`

Any update on this? I need this feature, please

+1 Vote for me, I have found I need this quite often.

1 any update for mongodb

1 Need this feature as well

+1 for mongodb

+1 for mongodb

Postgres15 added the NULLS NOT DISTINCT clause for this exact use-case, see E.2.3.1.2. Indexes  or in the documentation

How do you do this for mongodb?

+1

plus 1 for feature for Postgres

+1

+1

+1

Seems to be connected with more general:
https://github.com/prisma/prisma/issues/6974

+1

+1
This should be able to be supported with COALESCE(nullable_field, 'NULL')

+1
Have a self-related table to keep a tree of categories.
Top-level categories have NULL parent
model Category {
	id       			Int			@id @default(autoincrement())
	name				String
	parentCategoryId	Int?
	parentCategory		Category?	@relation("parentCategory", fields: [parentCategoryId], references:[id])
	childCategories		Category[]	@relation("parentCategory")

	@@unique([name, parentCategoryId])
}

and I cannot call findUnique with name and null parent

I think this is supported in mssql as well

+1

+1 for Postgres

+1 for postgres

+1 for postgres, please implement

This is probably the last feature deficit that pushes us to Drizzle. Not being able to perform an .update with a where: { } that uses a nullable column, and having to fall back to $queryRaw is just plain silly.

@shellscape where in update/find queries has been available using the extendedWhereUnique feature flag (out of preview since 5.0)

@petrbela thanks for the heads up. it's still lacking in that it requires an id (or whatever the primary key column may be named) along side any other additional column names and doesn't allow arbitrary column use.

@shellscape Yeah that's not available currently, though you can findFirst and then call update with the id.

This is very useful. Manual migrations to solve this generates another problem when squashing migrations.
Prisma just says this: "All fields that make up the unique constraint must be mandatory fields. The following model is not valid because id could be null".
https://www.prisma.io/docs/reference/api-reference/prisma-schema-reference#general-2
I hope that someday we'll have this feature. It's been long years since the creation of this issue.

I think the reason we don't support this yet is because Postgres is the only database we support that behaves this way, the others reject duplicate NULLs.

I don't believe this is true.
From the ANSI standard for SQL (SQL-92) https://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt:

A unique constraint is satisfied if and only if no two rows in a table have the same non-null values in the unique columns. (emphasis mine)

I'm having trouble finding a clearer definition of exactly how to treat null values, but this stack overflow answer interprets it and subsequent SQL standards to mean that duplicate nulls should be accepted: https://stackoverflow.com/a/767702/3248882
A cursory search on the internet shows that the majority of databases accept duplicate null values by default, including Postgres, MySQL, and SQLite. This site in particular clearly shows that the following databases allow duplicate null values:

Apache Derby
H2
MariaDB
MySQL
PostgreSQL
SQLite

https://modern-sql.com/caniuse/unique-nulls-distinct
As far as I can see, only MSSQL and SQL Server disallow duplicate null values by default
tl;dr prisma's default behaviour does not match the default behaviour of the majority of modern RDBMS products.

Using MS SQL and trying to create a 1:1 optional relation fails as:

Prisma requires a @unique constraint on the foreign key
Prisma won't allow multiple NULL values for that foreign key

Why.

Its been a year since i started tracking it and yet nothing has changed. I am going to look into it myself

+1 for postgres

@ruheni do relation joins make this any easier? i know we added that recently

+1 for mongoDB

So?

Hmmm..?

Bumping this as it's a major issue for me as well. I have two models that are one-to-one related:
model Player {
  id        String    @id @default(auto()) @map("_id") @db.ObjectId
  steam_id  String    @unique
  ...
  user      User?     @relation(fields: [user_id], references: [id])
  user_id   String?    @db.ObjectId @unique
  ...
  @@map("players")
}```

and

model User {
id            String    @id @default(auto()) @map("_id") @db.ObjectId
name          String?
email         String?   @unique
...
player        Player?
...
@@map("users")
}

As you can see, user_id is an optional field. However the relation REQUIRES it to be marked as unique. In doing so, I get the following error: duplicate key { user_id: null }

I imagine this will be an issue for excluded emails too, though I haven't run into it yet

I'm pretty sure that I've solved this here.
#23870

pending since 2019?

As a workaround ,  we can define a default value in the optional field.

This is pretty major as the lack of this feature will require a significant rethink of our database model or dropping prisma in favor of something less attractive.

As a workaround , we can define a default value in the optional field.

do you have example? i dont get it

As a workaround , we can define a default value in the optional field.

do you have example? i dont get it

@u007
You can do something like this -
vendorTransactionId  String?               @default("XXXX")
If i do not get data , instead of null , i am inserting a default value , which will give me unique constraint error .
works in my case.

but it is supposed to be unique, having default means it could duplicate?

Even if you default to a random UUID, it still doesn't solve some cases. Like optional unique 1-1 relationships described above

but it is supposed to be unique, having default means it could duplicate?

@u007 You are correct ,
but my case was something like this -
Initially my model was like -
vendorTransactionId  String?
name   String
@@unique[vendorTransactionId, name]

Due to this value like ,Multiple same name row entries were being created if vendorTransactionId was absent(with NULL vendorTransactionId).
By defining default on vendorTransactionId , helps me in handling unique constraint.

so does setting vendorTransaction to null still works?
best regards,
James Tan
Merc Studio
eMail: @.***
www: www.mercstudio.com
On Sat, Jul 13, 2024 at 1:42 PM vivek @.***> wrote:

but it is supposed to be unique, having default means it could duplicate?
@u007 https://github.com/u007 You are correct ,
but my case was something like this -
Initially my model was like -
vendorTransactionId  String?
name   String
@@unique[vendorTransactionId, name]
Due to this value like ,Multiple same name row entries were being created
if vendorTransactionId was absent(with NULL vendorTransactionId).
By defining default on vendorTransactionId , helps me in handling unique
constraint.
—
Reply to this email directly, view it on GitHub
https://github.com/prisma/prisma/issues/3387#issuecomment-2226783915,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAC65TIIDN4KPIPDPYQPA4DZMC437AVCNFSM4QIDDEN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMRSGY3TQMZZGE2Q
.
You are receiving this because you were mentioned.Message ID:
@.***>

any update?

Almost 5 years, and no update on this?

Just discovered this issue after 3 months of a running project. Maybe we should stop using Prisma on our side. Which is sad, and terryfing since we would need to rewrite a lot of code. Jeez.
The workaround changing the 1-1 to an 1-N made our DBA have nightmares. We are searching for other -valid- workarounds.

If this is going to be of any help, here is my problem (simplified) and the workaround...
As an example, let's imagine we have access cards handed out to teachers and hallway monitors (among students).
So we have teachers and students databases. Not all students have an access card and there can only be ONE access card given to anyone. Problem is that some students won't get an access card. There is no need for a separate table for hallway monitors. I can't also create a mandatory relationship for access cards, since it can be given to a teacher OR a student (2 different tables).
There may even be access cards not associated to anyone. My workaround was to create 2 tables: TeacherAccessCardLink and StudentAccessCardLink. Then I can create the typical one-to-one relationship, where the "mandatory"/"non-optional" side is always on the side of "...Link" tables. The relationships on the Students, Taechers or Cards are always optional.
Sort of it is one-to-one-to-one relationship, where...
Student/Teacher     Link       Access Card
optional                   unique    optional
Link?                       _id Int    Link?
I hope I made it as simple as possible. Until this is supported officially, this seems to be working for me.

authors: Geebrox Gherdty Hatko Lucasmm016 Nedi11 Nik-Novak NodeAdam PatoGordo RoyBkker SarenT Vandivier aalises athyuttamre beeman bilalesi birla cbreezier chadmott clearly-outsane ctrlplusb euberdeveloper euclid1990 eugene-korobko feliperugai gustawdaniel hadyrashwan iaomar janpio jeffrafter jeremiah-olisa jomaora jonalport karljv kyuhakyuk lucasjohnston lucy2329 luizfelipelaviola luluhoc lumenwrites matthewmueller michealroberts microsoftly nickfreemandesign ntaraujo omerman petrbela raiadi2008 rayyan808 rcbevans real2two rgcl shellscape slinden2 stevenpinker tomhoule u007 v29neil vuongnv-0085 zachblume


text:
Introspect result has the invalid name for @relation

Describe the bug
@relation name args value is generated against the shape [A-Z][a-zA-Z0-9]* after prisma introspect. It shows the error during deploy after introspect.
To Reproduce

compose docker along with postgres DB setting and run docker container
prisma introspect in cli
prisma deploy
shows error
ERROR: The provided name: account_messageboxes_receivers is not a valid name for a relation. It can only have up to 54 characters and must have the shape [A-Z][a-zA-Z0-9]*

Expected behavior
I firstly set up DB with Django ORM. Some tables have a relation column each other with option 'related_name='. I think this is generated to @relation(name: something) into datamodel.prisma. But the problem is the generated result is something like this @relation(name: account_messageboxes_receivers). It is missing the double quote, the first Uppercase letter, and the underscores. I changed it to @relation(name: "AccountMessageboxesReceivers") then it is deployed successfully.
Versions (please complete the following information):

OS: macOS Mojave
prisma CLI: prisma/1.23.1 (darwin-x64) node-v11.0.0
Prisma Server: I don't know how to check Prisma Server version.

Any workaround for this? It's happening to me after introspecting a Postgres database.

@matthewpick
Can you please try this again with the latest version? We have done some introspection improvements there

@mattdamon108 @pantharshit00 ^^

This is still an issue

@serranoarevalo Can you please provide a postgres schema that I can use to reproduce this issue.


text:
Breaking change: Error classes are exported as types

Bug description
I used to be able to do:
import { PrismaClientKnownRequestError } from '@prisma/client';

if (!(err instanceof PrismaClientKnownRequestError)) {
    ...
  }

But now the classes are exported as types so we can't use them in instanceof.
Expected behavior
They should be exported as values.
Environment & setup

Prisma version: 2.12.0

Problem lies here: https://github.com/prisma/prisma/blob/2.12.0/src/packages/client/src/generation/TSClient/common.ts#L77
Should have been import instead of type.

import { PrismaClientKnownRequestError } from '@prisma/client/runtime';
Bumped into the same issue, using that for now.

This is released in 2.12.1.

authors: DregondRahl Sytten timsuchanek


text:
ENOENT error when running prisma as /api in next.js

I want to run prisma inside next.js as a /api route. When I was setting it up at first only with the graphql-auth example everything was working fine (beside some problems with typegen with photon, but there's already an issue). Then I changed the schema and added some resolvers and objectTypes and when I wanted to start it again, when going to the route I get the output:
[Error: ENOENT: no such file or directory, mkdir] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'mkdir'
}

and I cannot query the server: "error": "JSON.parse: unexpected character at line 1 column 1 of the JSON data"
When running the same files as a standalone server(with what graphql-auth example provides), it's running fine.
Not sure if it's related to next.js or not, I tried turning on debug on photon and apollo-server, but the only thing I got is:
getos { platform: 'darwin', libssl: undefined } +0ms
Happy to provide more info if needed

Hi,
I am able to run a simple next js app with Prisma without any problems. You can find the code here: https://github.com/harshit-test-org/prisma-next-routes

So can you please provide me with a minimal reproduction of this? You can also try the example that I shared and see whether that works for you.

@pantharshit00 okay, can I add you as a collaborator to my github project for you to see?

Thanks, I got the invite. I will check it out and report back here.

I was able to reproduce it in your repository. I am not really sure what is causing it. Can you please invite @timsuchanek to that repository so that he can take a look as he is currently working on photon internals.

@pantharshit00 sure

I am hitting a similar error with
[Error: ENOENT: no such file or directory, mkdir] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'mkdir'
}

I was able to get it working with this handler:
import { permissions } from "./permissions";
import { schema } from "./schema";
import { createContext } from "./context";
import { ApolloServer } from "apollo-server-micro";
import { applyMiddleware } from "graphql-middleware";

// Good reference: https://github.com/maticzav/graphql-shield/tree/master/examples/with-apollo-server-lambda

const apolloServer = new ApolloServer({
  // schema input replaces typeDefs and resolvers
  schema: applyMiddleware(schema, permissions),
  context: createContext,
  introspection: true,
});

export const config = {
  api: {
    bodyParser: false,
  },
};
const handler = apolloServer.createHandler({ path: "/api/graphql2" });

export default handler;

@darknoon mine looks pretty much the same:
import { ApolloServer } from "apollo-server-micro";
import { applyMiddleware } from "graphql-middleware";
import { createContext } from "./src/context";
import { permissions } from "./src/permissions";
import { schema } from "./src/schema";

const apolloServer = new ApolloServer({
  schema: applyMiddleware(schema, permissions),
  context: createContext
});

export const config = {
  api: {
    bodyParser: false
  }
};

export default apolloServer.createHandler({ path: "/api/graphql" });

introspection doesn't make any differences

Thanks, everyone for posting the information!
After looking into your reproduction @BjoernRave , I found, that the problem here has nothing to do with prisma2, but it's the combination of next.js and nexus, which doesn't play well.
The problem is, that next.js replaces __dirname with /. This will inevitably make tools fail that rely on it. There were efforts on the Next.js side to fix that, but they got deprioritized.
Concretely, the line https://github.com/prisma-labs/nexus/blob/develop/examples/nexus-prisma/src/index.ts#L144
    schema: path.join(__dirname, './generated/schema.graphql'),

will be rewritten to path.join('/', './generated/schema.graphql') which resolves to //generated/schema.graphql.
Nexus then wants to write there in https://github.com/prisma-labs/nexus/blob/develop/src/typegenMetadata.ts#L101
How can you find that out without the stack trace?
This small hack helps getting the stack trace back for mkdir:
import fs from 'fs'

const oldMkDir: any = fs.mkdir
;(fs as any).mkdir = (...args: any[]) => {
  const err = new Error()
  console.error(`Calling mkdir: ${err.stack}`)
  console.error(`dirname ${__dirname}`)
  return oldMkDir(...args)
}

So a separate issue is, that the stack trace is eaten, which is very likely also a Next.js problem, as nexus just acts a library in this situation.
TL;DR How to get around this problem?
For now, you can e.g. remove the outputs configuration for the makeSchema function of nexus:
export const schema = makeSchema({
  types,
  plugins: [nexusPrismaPlugin()],
  // outputs: {
  //   schema: __dirname + '/generated/schema.graphql',
  //   typegen: __dirname + '/generated/nexus.ts',
  // },
  typegenAutoConfig: {
    sources: [
      {
        source: '@prisma/photon',
        alias: 'photon',
      },
      {
        source: require.resolve('./context'),
        alias: 'Context',
      },
    ],
    contextType: 'Context.Context',
  },
})

You can also try the workaround posted here: https://github.com/zeit/next.js/issues/8251#issuecomment-547702744https://github.com/zeit/next.js/issues/8251#issuecomment-547702744

@timsuchanek thanks for putting the detail in your writeup. Aren't the outputs required to make typescript work, though? (they weren't in the sample project and I had to add them)

@darknoon if the outputs are not provided, the nexus related parts are typed any. To work with that you probably have to disable the typescript strict mode in your tsconfig.json.

Hey everyone 👋,
@huv1k gave me a workaround. You can create a next.config.js file, and set the following env variable:
// next.config.js
module.exports = {
  env: {
    PROJECT_DIRNAME: __dirname,
  },
}

Then, use that env variable instead of __dirname
import { makeSchema } from 'nexus-prisma'
import { join } from 'path'

const schema = makeSchema({
  outputs: {
    typegen: join(process.env.PROJECT_DIRNAME, 'typegen.ts'),
    schema: join(process.env.PROJECT_DIRNAME, 'schema.ts'),
  }
})

It's not great, but at least it's a working workaround!

I'm not able to get that to work on deployment, is there something I missed?

@allpwrfulroot where are you deploying it to? Optimally you could even post a reproduction repo that we can look into.

I found a different approach to this:
const getPath = (fileName: string) =>
  path.join(process.cwd(), "generated", fileName);

// then use it to genetrate the output
    nexusPrismaPlugin({
      outputs: {
        typegen: getPath("nexusPrisma.ts")
      }
    })

Closing due to inactivity, as this seems to be fixed. If you still face this problem, please create a new issue.

authors: Albert-Gao BjoernRave Weakky allpwrfulroot darknoon pantharshit00 timsuchanek


text:
Document our current understanding of GPC on Android

This should include alternative tracking methods, the limitations of the current tracking limitation approach, our review of Google's policies (in this repository), the relevance of the AdID etc.
What replacements do apps use instead of AAID?

It would probably be good to set up an Overleaf document for this, and use the PETS template.

It would probably be good to set up an Overleaf document for this, and use the PETS template: https://petsymposium.org/authors.php#submission-guidelines

Good point! I will do that.

Good point! I will do that.

OK, I have done that. :)

I've been using this document to do my documentation along with a possible timetable outline: https://docs.google.com/document/d/1Be53srG1lVe-XujykA0AOfLY-Epdf9extInS_-vZNNE/edit?usp=sharing. On the matter of alternative AdID, this paper was particularly interesting https://arxiv.org/pdf/2212.03615.pdf

Will be working further on documentation as the project moves forward.

authors: SebastianZimmeck kasnder wesley-tan

repo: privacy-tech-lab/gpc-android

license: MIT


text:
🆕 Software Suggestion | openSUSE Leap

Basic Information
Name: openSUSE Leap
Category: Operating System
URL: opensuse.org
Description
Beside Fedora and Ubuntu, openSUSE Leap could be third choice for beginners. It is one of the major distros, and its stability is remarkable.
Why I am making the suggestion

openSUSE contains several desktop environments for users, and it's easy to use as Fedora and Ubuntu.
openSUSE uses systemd as init like Fedora and Ubuntu.
openSUSE Leap is stable just like Debian. Fedora and Ubuntu are close to beta channel (I even got a non-bootable kernel after the update of Fedora once).
Ubuntu contains Amazon ads and data leaks by default, which is not privacy friendly. Debian is more preferable.
SELinux is enabled on Fedora by default, which confuses a beginner a lot.

My connection with the software
I am a daily user of openSUSE Leap and a former daily user of Fedora.
I personally prefer stable distros such as openSUSE Leap and Debian.

[x] I will keep the issue up-to-date if something I have said changes or I remember a connection with the software.

As both an Ubuntu and openSUSE user on desktop and servers, I need to point out a few things here, but I do agree that openSUSE should be added to the list. I would also add the Tumbleweed version to the advanced operating systems section, since it could be seen as a more stable version of Arch Linux, but that is more of a personal preference.

Ubuntu contains Amazon ads and data leaks by default, which is not privacy friendly.

The Amazon problem went away years ago, and Ubuntu does not leak data by default on the desktop—it has opt-in telemetry, with the keyword being opt-in.

SELinux is enabled on Fedora by default, which confuses a beginner a lot.

The equivalent to SELinux in the Ubuntu and openSUSE world is AppArmor. Being enabled and restrictive by default should be the norm, not the exception, and Fedora does this mostly right in my opinion. Those are Linux kernel security modules which allow you to force cURL, for example, to never overwrite your .bashrc file—they provide strong mandatory access controls.

As both an Ubuntu and openSUSE user on desktop and servers, I need to point out a few things here, but I do agree that openSUSE should be added to the list. I would also add the Tumbleweed version to the advanced operating systems section, since it could be seen as a more stable version of Arch Linux, but that is more of a personal preference.

Ubuntu contains Amazon ads and data leaks by default, which is not privacy friendly.

The Amazon problem went away years ago, and Ubuntu does not leak data by default on the desktop—it has opt-in telemetry, with the keyword being opt-in.

SELinux is enabled on Fedora by default, which confuses a beginner a lot.

The equivalent to SELinux in the Ubuntu and openSUSE world is AppArmor. Being enabled and restrictive by default should be the norm, not the exception, and Fedora does this mostly right in my opinion. Those are Linux kernel security modules which allow you to force cURL, for example, to never overwrite your .bashrc file—they provide strong mandatory access controls.

Glad to know Ubuntu has gotten rid of Amazon. On second thought, who SELinux actually annoys are admins, especially when deploying a web server, not common users.

It is also worth noting that several useful privacy-enhancing tools are available in the distribution:

tor
torbrowser-launcher
various VPNs (openvpn, wireguard, anyconnect)

@bmwiedemann in which distro aren't they available?

@bmwiedemann in which distro aren't they available?

I did a quick grep and found
alpine and Slackware seem to have no tor.
Additionally, CentOS, Gentoo, Guix, Mageia seem to lack torbrowser.

One of openSUSE's biggest benefits is that it installs with BTRFS and Snapper configured by default.
With this combination, every time you update your system or install a new package, it automatically makes a snapshot. If an update or install leads to problems, you can easily roll back to the snapshot before the update.
If an update causes problems that prevents you from booting into the OS, you can select the snapshot before the update at the GRUB menu to boot into. After doing that, you can use Snapper to roll back into that snapshot (when you boot into a snapshot at GRUB, it's read-only).
This combination makes recovering from problems extremely easy.

At privacytools.io website, mainstream general-purpose distributions like Ubuntu, Debian, Fedora and Arch are recommended but not openSUSE. This omission strikes me as very odd because, technical considerations aside, it's definitely more user-friendly and failproof than (plain) Arch, does not have the record of privacy-invading issues of Ubuntu and is community-driven while sponsored by a parent company just like Fedora (if having ties to a corporation were an issue).
openSUSE is a very mature and technically competent distribution while being easy to install and easy to use as an Introductory Operating System, having a decades-old reputation and commitment to free and open source software.
GNU's endorsement of openSUSE is only prevented by the inclusion a nonfree repo, which is added to the system by default but no packages from it are pre-installed (and can be easily disabled). Any telemetry measures that I'm aware of have to be expressely implemented by the user.
openSUSE's documentation provides a complete guide for system security and hardening.

authors: HouraisanNEET N-Prime bmwiedemann cbnavigator ghost jamesponddotco rusty-snake


text:
feat: support .d.ts from .mts

closes https://github.com/privatenumber/pkgroll/issues/23

:tada: This PR is included in version 1.6.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

source: gharchive/pull-request

repo: privatenumber/pkgroll


text:
ISSUE#1: Handle null dates

Creating new PR targeting your upstream repo instead of my forked repo - review changes applied. Will close my original local PR

Codecov Report

Merging #7 (1164753) into master (bec526b) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master       #7      +/-   ##
==========================================
+ Coverage   98.03%   98.05%   +0.01%
==========================================
  Files           4        4
  Lines         153      154       +1
  Branches       41       42       +1
==========================================
+ Hits          150      151       +1
  Misses          3        3

Impacted Files
Coverage Δ

src/lib/date-utils.ts
100.00% <100.00%> (ø)

src/lib/parse.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bec526b...1164753. Read the comment docs.

Found a couple bugs:

If a date is selected, and you delete all text at once, invalid stays as false. However, if you delete it one character at a time, invalid is true.

When the datepcker is focused and the value is null, using the arrow keys does not select a new value (you can focus the datepicker by clicking somewhere there is no button)

When the datepicker is focused and the value is null, using the arrow keys does not select a new value

^-- If we decide to hide the default day in the picker per the earlier suggestion this issue goes away (I think). Right now I don't have a strong opinion about it. For this PR I will see if we can get the arrow keys to select the value.
Will try to address both of these shortly. Also feel free to put up piggy back prs with your suggested changes and I can just merge them into mine.

Also feel free to put up piggy back prs with your suggested changes and I can just merge them into mine.

Pushed them all into here. We should be pretty much good to go.

If we decide to hide the default day in the picker per the earlier suggestion this issue goes away (I think)

So right now with my change to not highlight any date, using the arrow keys will select a date relative to shownDate (even when no date is highlighted). I think it's nice for them to select something so that you can still select dates via keyboard. Maybe it would be better if it selects from an edge (e.g LeftArrow selects top-right value), or relative to the current date, but I'm fine with how it is now either way.

Resolved conflicts and updated the PR - we should be good to go but feel free to poke at it more.

repo: probablykasper/date-picker-svelte


text:
signal math

re: https://github.com/processing/p5.js-website/issues/189

.mult and .add may not be necessary. Either make order of operations clear, or just replace them with .scale.
test that scale is scaling signals as expected
make example to show what is happening, and how scaling signals is useful for modulation. For example, connect to a filter frequency.

One thing that might be helpful could be to also output a waveform visualization of the modulator. This is what the author of Programming Sound with Pure Data does when he teaches FM and I found it very helpful. If you want, I could try submitting a PR that adds it. No worries if not!

authors: therewasaguy toolness

license: mit

license_source: bigquery


text:
Fix example for 3D: materials

Deleted basicMaterial() function.

thank you!


text:
patch: Remove packageManager property with wrangler-action 3.3.0

After discussing https://github.com/cloudflare/wrangler-action/issues/180, it turns out packageManager property is supposed to be optional.
With the new wrangler-action, we no longer need this property.
Signed-off-by: Vipul Gupta (@vipulgupta2048) vipulgupta2048@gmail.com

@balena-ci I self-certify!

repo: product-os/flowzone


text:
🛑 Tarot Ana is down

In a959fc4, Tarot Ana (https://www.tarot-ana.fr) was down:

HTTP code: 500
Response time: 11579 ms

Resolved: Tarot Ana is back up in a97369d.

repo: proger-su/upptime


text:
Do no longer force-install a default nginx.conf

As far as I can tell, a default nginx.conf does no longer need to be force-installed. Let me know if I forgot something.

The force install was introduced here.

Seems like tests fail?

Fixed.

Going to close this. A real fix needs to take into account the fact that we don't use the PPA for nginx with the debian solution :( I'll take care of this in #1467.

Scratch that. Seems this can go in after all :)

Closing in favor of #1468. Thanks for your work @SonicHedgehog!

Whoa! We want both I think, yea?

Right I'm a nub. Reopened.

repo: progrium/dokku


text:
Need factory reset command for SDK

Summary :  Once we send the factory reset command, the device should delete all previous provisioning information.
Based on LocalConfigDisabled Attribute
The LocalConfigDisabled attribute SHALL allow a local Node configuration to be disabled. When
this attribute is set to True the Node SHALL disable the ability to configure the Node through an onNode user interface. The value of the LocalConfigDisabled attribute SHALL NOT in any way modify,
disable, or otherwise affect the user’s ability to trigger a factory reset on the Node.
Refer : 11.1.6.1. Attributes https://github.com/CHIP-Specifications/connectedhomeip-spec/blob/build-sample/pdf/main.pdf

Summary : Once we send the factory reset command, the device should delete all previous provisioning information.

What factory reset command?  Matter has no such command, @Survensa.

The value of the LocalConfigDisabled attribute SHALL NOT in any way modify, disable, or otherwise affect the user’s ability to trigger a factory reset on the Node.

What this means is this.  If the device has a "press and hold this to factory reset" button, that button will continue to function as it should even if LocalConfigDisabled is set to false.

@Survensa
Are you talking about device specific factory reset command? We don't have such command specified in the cluster.

@bzbarsky-apple @yufengwangca What @Survensa meant here,

Make sure SDK has the implementation that spec recommended for factory reset condition , Ref attached
Need a way to send that command to the example apps, Either by means of some signal or something,  so that we can test device is performing all the actions that spec is recommending when factory reset command sent

@cjandhyala Just to be clear: that section of the spec is not about a "command" in the Matter sense, right?
The SDK itself does not provide a factory reset mechanism, which involves wiping various storage that the application, not the SDK, knows about.  That mechanism needs to be provided by the SDK consumer.
Some of the in-tree example apps that run on actual devices do have such mechanisms.  That said, the mention of "some signal or something" sounds like you're talking about the linux/darwin apps, and for those "factory reset" corrresponds to killing the app, erasing the KVS on disk, and restarting the app...

@bzbarsky-apple Correct, that section of the spec is not about the command.
I agree that the actual implementation of factory reset function should come from the consumer of the SDK.Just like the OTA feature has reference implementation, we were expecting there may be some reference implementation/API  that each platform owner adopts. So we wanted to make sure SDK has the spec required things implemented for factory reset condition.
if "factory reset" corrresponds to killing the app, erasing the KVS on disk, and restarting the app, we are good with it.

Closing this, as there's no requirement here from the SDK.

authors: Survensa bzbarsky-apple cjandhyala woody-apple yufengwangca

repo: project-chip/connectedhomeip


text:
Build is broken on macOS

Problem
Seeing these build errors while building on macOS
../src/lib/support/CHIPMem.h:67:37: error: redefinition of default argument
extern CHIP_ERROR MemoryInit(void * buf = nullptr, size_t bufSize = 0);
                                    ^     ~~~~~~~
gen/include/support/CHIPMem.h:67:37: note: previous definition is here
extern CHIP_ERROR MemoryInit(void * buf = nullptr, size_t bufSize = 0);
                                    ^     ~~~~~~~
In file included from ../src/lib/support/CHIPArgParser.cpp:49:
In file included from gen/include/support/CHIPMemString.h:29:
In file included from ../src/lib/support/ScopedBuffer.h:27:
../src/lib/support/CHIPMem.h:67:59: error: redefinition of default argument
extern CHIP_ERROR MemoryInit(void * buf = nullptr, size_t bufSize = 0);
                                                          ^         ~
gen/include/support/CHIPMem.h:67:59: note: previous definition is here
extern CHIP_ERROR MemoryInit(void * buf = nullptr, size_t bufSize = 0);
                                                          ^         ~
In file included from ../src/lib/support/CHIPArgParser.cpp:49:
In file included from gen/include/support/CHIPMemString.h:29:
In file included from ../src/lib/support/ScopedBuffer.h:27:
../src/lib/support/CHIPMem.h:166:12: error: redefinition of 'New'
inline T * New(Args &&... args)
           ^
gen/include/support/CHIPMem.h:166:12: note: previous definition is here
inline T * New(Args &&... args)
           ^
In file included from ../src/lib/support/CHIPArgParser.cpp:49:
In file included from gen/include/support/CHIPMemString.h:29:
In file included from ../src/lib/support/ScopedBuffer.h:27:
../src/lib/support/CHIPMem.h:179:13: error: redefinition of 'Delete'
inline void Delete(T * p)
            ^
gen/include/support/CHIPMem.h:179:13: note: previous definition is here
inline void Delete(T * p)

Proposed Solution
Fix the build issues.

Seeing this on a clean workspace:
In file included from ../src/platform/Darwin/PlatformManagerImpl.cpp:28:
In file included from ../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.cpp:33:
../src/platform/Linux/MdnsImpl.h:29:10: fatal error: 'avahi-client/client.h' file not found
#include <avahi-client/client.h>

Not sure if the errors in the original issue comment are still occurring.

Tag: @gjc13

While trying to repro this, I see the same error...
In file included from ../src/platform/Darwin/PlatformManagerImpl.cpp:28:
In file included from ../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.cpp:33:
../src/platform/Linux/MdnsImpl.h:29:10: fatal error: 'avahi-client/client.h' file not found
#include <avahi-client/client.h>

I think. both of these are valid errors.

Also, tag: @mspang

authors: pan-apple sagar-apple


text:
[ICD] kICDMonitoringBufferSize is too small to handle full "full width" node ID

Reproduction steps
When using "full width" node ID, for example, 0xffff000000000000, We cannot registerClient successfully using the lit example app, it would complain the buffer is too small,
https://github.com/project-chip/connectedhomeip/blob/17f25a1772f7ba7535d57cd4dfa43ac1fc295843/src/app/icd/server/ICDMonitoringTable.h#L37C25-L37C49
[1727120175.806454][15188:15188] CHIP:SVR: debug 63
0x08, tag[Fully Qualified (6 Bytes)]: 0xd973::0xffa3::0xff, type: Octet String (0x10), length: 0, value: hex:
[1727120175.806476][15188:15188] CHIP:SPT: debug>>>152701000000000000FFFF270201000100FDFFFFFF300310D0E9EB3B00000000
[1727120175.806486][15188:15188] CHIP:SPT: debug>>>0000000000000000300410D0E9EB3B00000000000000000000000024050018
We may do TLV::EstimateStructOverhead(sizeof(uint64_t), sizeof(uint64_t), sizeof(xxxx), etc) * 3 / 2
Bug prevalence
always
GitHub hash of the SDK that was being used
8750c55
Platform
core
Platform Version(s)
1.4
Anything else?
No response

Why * 3 / 2?

we need slack of at least 50%  to make sure we can grow and then if there is a firmware upgrade/downgrade, that it doesn't brick

Where is the "50%" coming from?  What if the new firmware grows by more than that in terms of what it's storing (since presumbly it's the "downgrade" case that needs the extra space)?

authors: bzbarsky-apple yunhanw-google


text:
Remove suppression of stack-size warning in cluster-objects.

Problem
We're suppressing a compiler warning for no good reason.
Change overview
Stop suppressing it.
Testing
Tree still compiles.

/rebase


text:
[EFR32]Feature wifinetworkdiagnostics implmentation for RS9116

Problem
Missing implementation for wifinetworkdiagnostics cluster for RS9116
Change overview
Added implementation for wifinetworkdiagnostics for EFR32+RS9116 combo
Testing
Tested using EFR32+RS9116.
TC-DGWIFI-2.1 tested

This depends upon #20106


text:
Fix storage location of Docker images and containers for Mac or linux

This will fix information about used memory space when build image of docker on osx host machine. Docker has another directory on linux and osx.
Problem
What is being fixed?  Examples:

Fix crash on startup
Fixes #12345 Frobnozzle is leaky (exactly like that, so GitHub will auto-close the issue).

Change overview
What's in this PR
Testing
How was this tested? (at least one bullet point required)

If unit tests were added, how do they cover this issue?
If unit tests existed, how were they fixed/modified to prevent this in future?
If new unit tests are not added, why not?
If integration tests were added, how do they verify this change?
If new integration tests are not added, why not?
If manually tested, what platforms controller and device platforms were manually tested, and how?
If no testing is required, why not?

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@DamMicSzm Thank you for the fix!

authors: CLAassistant DamMicSzm bzbarsky-apple