text:
No content is written in PDFs uploaded using the instructions for multipart/form-data

I have an objective to send a PDF file from one server to a REST API which handles some archiving. I am using .NET Core 3.1 and RestEase
I have a simple console app that runs at a certain time everyday. The relevant code is as follows:
using System;
using System.Linq;
using System.Threading.Tasks;
using RestEase;

namespace CandidateUploadFile
{
    class Program
    {
        static async Task Main(string[] args)
        {
            try
            {
                var apiClientBuilder = new ApiClientBuilder<ITestApi>();
                var api = apiClientBuilder.GetApi("https://localhost:5001");
                var candidate = await api.GetCandidateByEmailAddress("tester@aol.com");
                var fileName = "tester.pdf";
                var fileBytesToUpload = await FileHelper.GetBytesFromFile($@"./{fileName}");
                var result = await api.UploadCandidateFileAsync(fileBytesToUpload, candidate.Data.First().Id, fileName);
            }
            catch (Exception e)
            {
                System.Console.WriteLine(e);
            }
        }
    }
}

apiClientBuilder does some auth-header adding, and that's really it. I'm certain that bit isn't relevant.
ITestApi looks like this:
using System.Collections.Generic;
using System.Net.Http;
using System.Threading.Tasks;
using Models;
using RestEase;

namespace CandidateUploadFile
{
    public interface ITestApi : IApi
    {
        [Get("v1/candidates/{candidateId}")]
        Task<Models.Response<Candidate>> GetCandidate([Path] string candidateId);

        [Get("v1/candidates")]
        Task<Models.Response<IEnumerable<Candidate>>> GetCandidateByEmailAddress([Query] string email);

        [Get("v1/candidates")]
        Task<Models.Response<IEnumerable<Candidate>>> GetCandidates();

        [Post("v1/candidates/{candidateId}/files?perform_as=327d4d21-5cb0-4bc7-95f5-ae43aabc2db7")]
        Task<string> UploadFileAsync([Path] string candidateId, [Body] HttpContent content);

        [Get("v1/users")]
        Task<Models.Response<IEnumerable<User>>> GetUsers();
    }
}

Per the docs, I'm using an extension method to handle the multipart/form-data piece, along with headers that are required by my API. That extension method looks like this:
using System.Collections.Generic;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;

namespace CandidateUploadFile
{
    public static class ApiExtension
    {
        public static async Task<string> UploadCandidateFileAsync(this ITestApi api, byte[] data, string candidateId, string fileName)
        {
            var content = new MultipartFormDataContent();

            var fileContent = new ByteArrayContent(data);
            fileContent.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
            fileContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data")
            {
                Name = "file",
                FileName = fileName
            };

            content.Add(fileContent);

            return await api.UploadFileAsync(candidateId, content);
        }
    }
}

So what will happen when my console app executes is: I will get a successful response from the upload endpoint, and the file on the archive server gets created, but it's blank.
It may be important to know that this does not happen when I send, say, a .txt file. The .txt file will save with the expected content.
Not sure where to start on this one; it's not clear as to if this is a library issue or something else. Any insight would be helpful. Thank you!

I'm sure this helps or is comparable to your use-case, but here's how I did use it:
RestEaseApi interface method
/// <summary>
/// uploads an image
/// </summary>
/// <param name="petId">ID of pet to update</param>
/// <param name="content">An extension method is generated to support the exact parameters.</param>
/// <param name="additionalMetadata">Additional Metadata</param>
[Post("/pet/{petId}/uploadImage")]
[Header("Content-Type", "application/octet-stream")]
Task<ApiResponse> UploadFileAsync([Path] long petId, [Body] HttpContent content, [Query] string additionalMetadata);

And my extension method:
/// <summary>
/// uploads an image
/// </summary>
/// <param name="api">The Api</param>
/// <param name="petId">ID of pet to update</param>
/// <param name="additionalMetadata">Additional Metadata</param>
/// <param name="file">The content.</param>
public static Task<ApiResponse> UploadFileAsync(this IPetStoreOpenApi3Api api, long petId, string additionalMetadata, byte[] file)
{
    var content = new MultipartFormDataContent();

    var fileContent = new ByteArrayContent(file);
    content.Add(fileContent);

    return api.UploadFileAsync(petId, content, additionalMetadata);
}

Full examples can be found at https://github.com/StefH/RestEase-Client-Generator (https://github.com/canton7/RestEase/issues/143)

https://github.com/StefH/RestEase-Client-Generator/blob/master/examples/RestEaseClientGeneratorConsoleApp/Examples/PetStoreOpenApi302/Api/IPetStoreOpenApi3Api.cs
https://github.com/StefH/RestEase-Client-Generator/blob/master/examples/RestEaseClientGeneratorConsoleApp/Examples/PetStoreOpenApi302/Api/PetStoreOpenApi3ApiExtensions.cs

The starting point is to grab Fiddler and see what data's actually being uploaded.
My other thought is, are you sure that a multipart upload is correct? You don't necessarily need that, see these examples.

I should clarify: the API I'm calling is not my own; I'm restricted to follow their contract and expectations. They do require a multipart/form-data request per their documentation
@StefH Seems like the only differences between what you shared and mine is that you included a Content-Type header of application/octet-stream, and you did not include content-level headers like ContentDisposition. Without the content-disposition headers, I get a 400 from the API:
"{\"code\":\"BadRequestError\",\"message\":\"File is not included in the request\"}"
Also, I tried leaving my content-level headers, but adding application/octet-stream as a request-level header. I get the same 400 error when trying that

I think the next step is Fiddler, then. See what RestEase is putting out, and see what a successful upload looks like. Then play spot the difference.
Note that this is probably nothing to do with RestEase -- you're talking directly to HttpClient at this point.

Sure. So since the file I send via Postman results in a file with content, I'll compare the raw request gathered from Fiddler for Postman with the request submitted from my application, which results in a blank file. Body shortened for brevity:
Postman (good result):
POST https://api.lever.co/v1/candidates/<<removed>>/files?perform_as=<<removed>> HTTP/1.1
Authorization: Basic  << removed >>
User-Agent: PostmanRuntime/7.24.1
Accept: */*
Cache-Control: no-cache
Postman-Token: c4b2acb0-1255-4664-a8ba-26005f74a146
Host: api.lever.co
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Type: multipart/form-data; boundary=--------------------------065702623202247563183338
Content-Length: 83219

----------------------------065702623202247563183338
Content-Disposition: form-data; name="file"; filename="tester.pdf"
Content-Type: application/pdf

%PDF-1.4
%����
501 0 obj
<</Linearized 1/L 83006/O 503/E 63657/N 1/T 72865/H [ 1076 225]>>
endobj

xref
501 39
0000000016 00000 n
0000001490 00000 n
0000001646 00000 n
 . . . .
 . . . .
trailer
<</Size 501/ID[<D99D73436DB5804C8C3D777F3763EE1A><9961447053C7064DAB15A51D2FE6A510>]>>
startxref
116
%%EOF

----------------------------065702623202247563183338--

Request fired from my code, using the extension method approach (blank file submitted; bad result):
POST https://api.lever.co/v1/candidates/<<removed>>/files?perform_as=<<removed>> HTTP/1.1
Host: api.lever.co
Authorization: Basic <<removed>>
Content-Type: multipart/form-data; boundary="b5c023cd-c936-4575-aac9-e1ab9821c03d"
Content-Length: 134392

--b5c023cd-c936-4575-aac9-e1ab9821c03d
Content-Type: application/pdf
Content-Disposition: form-data; name="file"; filename="tester.pdf"

%PDF-1.4
%����
501 0 obj
<</Linearized 1/L 83006/O 503/E 63657/N 1/T 72865/H [ 1076 225]>>
endobj

xref
501 39
0000000016 00000 n
0000001490 00000 n
0000001646 00000 n
 . . . .
 . . . .
trailer
<</Size 501/ID[<D99D73436DB5804C8C3D777F3763EE1A><9961447053C7064DAB15A51D2FE6A510>]>>
startxref
116
%%EOF

--b5c023cd-c936-4575-aac9-e1ab9821c03d--

It's easy to see the request-level headers are quite different, but I'm not sure any of those are relevant to having the file content successfully written, no? Would you have any recommendations on what to try in the code that would possibly influence the writing of the data to the file itself?

I see that the Content-Length is totally different?

Yes, I noticed that as well. I'm not sure what that's all about. I've shared my code to be transparent; it should be plain to see that I'm not doing anything more than adding the content the same way you demonstrated in your example

I'm very suspicious of that different content length as well.
If you use a text file, which you said works, do you get the same content length?
Is the actual uploaded contents different? I'd be surprised if there wasn't a difference somewhere. A hex editor should be able to do a binary diff.
How does your FileHelper.GetBytesFromFile work?

Hm, yes I am seeing discrepancies in VBinDiff.. I'm not sure what Postman is doing that my code isn't, that's the trouble.
The FileHelper code looks like this:
using System.IO;
using System.Text;
using System.Threading.Tasks;

namespace CandidateUploadFile
{
    public static class FileHelper
    {
        public static async Task<byte[]> GetBytesFromFile(string filePath)
        {
            var fileContents = "";
            using (var sr = new StreamReader(filePath))
            {
                fileContents = await sr.ReadToEndAsync();
            }

            var fileAsBytes = Encoding.Default.GetBytes(fileContents);

            return fileAsBytes;
        }
    }
}

Replace that code with just ‘File.ReadAllBytes(fileName);‘

Success! Thank you so much. I was at wit's end on this one..
Are you thinking it was due to me encoding with UTF-8?

You interpreted the bytes as utf-8 text, then turned them into utf-16 bytes (Encoding.Default is utf-16)

I guess the docs suggests otherwise:
https://docs.microsoft.com/en-us/dotnet/api/system.text.encoding.default?view=netcore-3.1#the-default-property-on-net-core

But, pdfs are binary and shouldn't be interpreted as text anyway.
Yes sorry, misremembered. System code page, which could be anything

I see.. That explains why .txt files were working and not pdfs.. Yikes, my bad. Foolish of me. Thank you for the help @StefH @canton7

(also, reading a file async does nothing unless it was opened as async. You'd also be better off grabbing a FileStream and using that as a SteamContent: then the whole thing doesn't have to be loaded into memory at once).
Glad you solved it!

authors: StefH canton7 sdg-sjb


text:
multiple databases support

As far as i see, the plugin only supports a single database, since it stores the database handle internally after opening the db. What about returning the handle and/or some kind of db instance with the query methods?

@chriswep Look closely to the readme and the application provided as example ie https://github.com/jepiqueau/capacitor-apps/tree/master/IonicAngular/jeep-test-app and you will see that you can use multiple databases at once.  If the comment is to make query with join to multiple databases, you right so it is something that i could have a look at. Hope it's clarify

@jepiqueau hmm, unless you mean that you can open a database, do a query, then open another db and do a query on that one, i don't see it. as far as i understand the README, opening a database does not return a handle or connection object but just sets the database internally. i can't see how multiple databases can be open at the same time this way

@chriswep the databases are not open at the same time and you do not need it as the plugin open the db with the curent name before each command and close it after command completion. So i includes an example of how it works
  async testSQLitePluginMulti() {
    // Open First Database
    let result:any = await this._sqlite.open({database:"firstdb"});
    console.log("Open database : " + result.result.toString());
    let sqlcmd: string = `
    BEGIN TRANSACTION;
    CREATE TABLE IF NOT EXISTS users (
        id INTEGER PRIMARY KEY NOT NULL,
        email TEXT UNIQUE NOT NULL,
        name TEXT,
        age INTEGER
    );
    CREATE TABLE IF NOT EXISTS messages (
        id INTEGER PRIMARY KEY NOT NULL,
        title TEXT NOT NULL,
        body TEXT NOT NULL
    );
    PRAGMA user_version = 1;
    COMMIT TRANSACTION;
    `;
    var retExe: any = await this._sqlite.execute({statements:sqlcmd});
    console.log("retExe ",retExe)
    // Insert some Users
    sqlcmd = `
    BEGIN TRANSACTION;
    DELETE FROM users;
    INSERT INTO users (name,email,age) VALUES ("Whiteley","Whiteley;Bill.com",30);
    INSERT INTO users (name,email,age) VALUES ("Jones","Jones.com",44);
    COMMIT TRANSACTION;
    `;
    retExe = await this._sqlite.execute({statements:sqlcmd});
    // Select all Users
    sqlcmd = "SELECT * FROM users";
    var retSelect = await this._sqlite.query({statement:sqlcmd,values:[]});
    console.log('retSelect ',retSelect.values.length)
     for (let i:number =0; i< retSelect.values.length; i++) {
      console.log("results users: ",i,retSelect.values[i]);
    }
    // Open Second Database
    result = await this._sqlite.open({database:"seconddb"});
    console.log("Open database : " + result.result.toString());
    sqlcmd = `
    BEGIN TRANSACTION;
    CREATE TABLE IF NOT EXISTS contacts (
        id INTEGER PRIMARY KEY NOT NULL,
        email TEXT UNIQUE NOT NULL,
        name TEXT
    );
    PRAGMA user_version = 1;
    COMMIT TRANSACTION;
    `;
    var retExe: any = await this._sqlite.execute({statements:sqlcmd});
    // Insert some Contacts
    sqlcmd = `
    BEGIN TRANSACTION;
    DELETE FROM contacts;
    INSERT INTO contacts (name,email) VALUES ("Watson","Watson.com");
    INSERT INTO contacts (name,email) VALUES ("Churchill","Churchill.com");
    INSERT INTO contacts (name,email) VALUES ("Ferguson","Ferguson.com");
    COMMIT TRANSACTION;
    `;
    retExe = await this._sqlite.execute({statements:sqlcmd});
    // Select all Contact
    sqlcmd = "SELECT * FROM contacts";
    var retSelect = await this._sqlite.query({statement:sqlcmd,values:[]});
    console.log('retSelect ',retSelect.values.length)
     for (let i:number =0; i< retSelect.values.length; i++) {
      console.log("results contacts : ",i,retSelect.values[i]);
    }
    // Reopen the first store
    result = await this._sqlite.open({database:"firstdb"});
    // Insert a new User with SQL and Values
    sqlcmd = "INSERT INTO users (name,email,age) VALUES (?,?,?)";
    let values: Array<any>  = ["Simpson","Simpson@example.com",69];
    var retRun: any = await this._sqlite.run({statement:sqlcmd,values:values});
    // Select all Users
    sqlcmd = "SELECT * FROM users";
    var retSelect = await this._sqlite.query({statement:sqlcmd,values:[]});
    console.log('retSelect ',retSelect.values.length)
     for (let i:number =0; i< retSelect.values.length; i++) {
      console.log("results users: ",i,retSelect.values[i]);
    }
    // Insert some Messages
    const messages: Array<any> = [
      {title:"test post 1",content: "content test post 1"},
      {title:"test post 2",content: "content test post 2"}
    ]
    sqlcmd = `
    BEGIN TRANSACTION;
    DELETE FROM messages;
    INSERT INTO messages (title,body) VALUES ("${messages[0].title}","${messages[0].content}");
    INSERT INTO messages (title,body) VALUES ("${messages[1].title}","${messages[1].content}");
    COMMIT TRANSACTION;
    `;
    retExe = await this._sqlite.execute({statements:sqlcmd});
    // Select all Messages
    sqlcmd = "SELECT * FROM messages";
    var retSelect = await this._sqlite.query({statement:sqlcmd,values:[]});
    console.log('retSelect ',retSelect.values.length)
     for (let i:number =0; i< retSelect.values.length; i++) {
      console.log("results messages: ",i,retSelect.values[i]);
    }

  }

hope this clarify

@jepiqueau
I think what @chriswep means is that it would be nicer if you would return a Typesricpt DB handle where you can call all other calls expect open.
->
   let db: CapDatabase = await this._sqlite.open({database:"firstdb"});
   let db2: CapDatabase = await this._sqlite.open({database:"seconddb"});
   db.execute(...)
   db2.execute(...)
   db2.query(...)
   db.query(...)

THis would make it nicer top work with.

@digaus i understand your point and the one of @chriswep, but if you do that the plugin is just reduce to open a database and return a DB handle and that is it.

@digaus i understand your point and the one of @chriswep, but if you do that the plugin is just reduce to open a database and return a DB handle and that is it.

Yes thats it but why is this bad? The functions would still be there but called with the DB handle?

@jepiqueau @digaus maybe i'm missing something here, but isn't opening a sqlite database doing more than setting a pointer to a file in the underlying native sqlite implementation (like initialising indices, caches, metadata, etc.)? In most use cases those databases will need to be open at the same time, with a lot of parallel access going on. Even with no initialisation cost, this will likely fail due to parallelism - unless there is some kind of blocking mechanism in place (which in turn will slow everything down).

if you do that the plugin is just reduce to open a database and return a DB handle and that is it.

again, maybe i'm missing something. but we still need all the db methods bridged to javascript on the handle (since we can't call the native methods). So we need a javascript db object/handle/connection that will allow us to do stuff with this db. Thats at least how the Cordova-sqlite plugin works.

@jepiqueau @digaus maybe i'm missing something here, but isn't opening a sqlite database doing more than setting a pointer to a file in the underlying native sqlite implementation (like initialising indices, caches, metadata, etc.)? In most use cases those databases will need to be open at the same time, with a lot of parallel access going on. Even with no initialisation cost, this will likely fail due to parallelism - unless there is some kind of blocking mechanism in place (which in turn will slow everything down).

if you do that the plugin is just reduce to open a database and return a DB handle and that is it.

again, maybe i'm missing something. but we still need all the db methods bridged to javascript on the handle (since we can't call the native methods). So we need a javascript db object/handle/connection that will allow us to do stuff with this db. Thats at least how the Cordova-sqlite plugin works.

At least for electron it should work by default:
https://github.com/kriasoft/node-sqlite#opening-multiple-databases

@chriswep @digaus We could discuss this for ever. I didn't intent to replicate what  the Cordova plugin is doing. People can still work with the Cordova plugin as far as i know. To stay on the safe side, i decided to have only one database open at a time and even more the database is opened and closed after each sql statement execution completion. So you can have an application accessing multiple databases but only one after an other to avoid the parallelism issues and the complexity of blocking mechanism.

i decided to have only one database open at a time and even more the database is opened and closed after each sql statement execution completion.

thanks for the clarification!

@jepiqueau seeing this now is the kind of "official" sqlite community capacitor plugin i want to reopen this discussion to avoid potentially creating a second plugin for mostly the same purpose. i'm currently still using the Cordova plugin in capacitor however i want to get rid of it at some time.
I looked into the code a bit and it seems to me that this plugin indeed creates a new connection for each request/query/transaction. Can someone shed some light on the performance overhead for this? And apart from performance this also doesn't seem to be how a database (and also the underlying sqlite implementations) are supposed to work?
If we want to use multiple databases there comes the issue of parallelism into play. If i see it correctly the plugin currently saves the database name and options in its internal state. So we would need to use open and close before and after each call to execute on this plugin. And if i don't overlook something this still wouldn't make 100% sure that parallel queries go to the correct database (or break another running operation) since they all operate on the same global CapacitorSQLPlugin object and its state.
I suggest to refactor this to return some kind of db-handle object when opening a database. I suppose this would involve separating some methods into a new class that handles the connection/db-handle in its state. Opening a database would then return an instance to this class. This way apps could have multiple database handles open that can be safely used in parallel. Also the plugin wouldn't need to open and close the database for each query/transaction.
For backwards compatibility, an instance of this db-handle class could be used internally by the plugin on default and only returned requested specifically.

@chriswep I understand now that if we want to make ORM support with typeorm which seems to have the consensus of the community, we need  to have the db connection available and so the multiple support of databases. This requires a complete refactoring of the plugin which is quite a task but feasible. Would you be kind and some others to participate to this refactoring but before to start refactoring it would be better to have a complete specification of the refactoring process, would you like to write it as with your expertise in using the cordova plugin you know precisely what to do. You didn't convince me few months ago but since i got the request of ORM support i understood you were right. Tell me what are your thoughts

if one of the goals is to have a TypeORM driver at the end we could start from there. There is already an abstract sqlite driver in TypeORM that all sqlite-based drivers extend: https://github.com/typeorm/typeorm/tree/master/src/driver/sqlite-abstract. It might make sense to take the Cordova driver as a template for a capacitor driver: https://github.com/typeorm/typeorm/tree/master/src/driver/cordova
For this to work the capacitor plugin needs to have some kind of openDatabase method that returns a connection object with the query method(s) and a close method.
If the latter is done, implementing the typeorm-driver is basically duplicating an existing sqlite driver and making a few minor modifications. As far as i see, TypeORM uses a single query method (which expects query and parameters). The other sqlite pass this to a single driver-method and look if the query starts with INSERT to determine what to return (insertId, array of rows, etc.). In a similar way the capacitor plugin can use its different query methods run/query/execute.

@chriswep i start looking at this for Android but i didn't find a way to return a  kind of db-handle object from a createConnection method.
so what i could may be do is something like this
await this.sqlite.createConnection({database: "testNew", encrypted: false,
                                   mode: "no-encryption", version: 1});
await this.sqlite.open({database: "testNew"});
await this.sqlite.execute({database: "testNew", statements: statements});
await this.sqlite.runSQL({database: "testNew", statement: statement, values: values});
await this.sqlite.createConnection({database: "testEncrypted", encrypted: true,
                                   mode: "secret", version: 1});
await this.sqlite.open({database: "testEncrypted"});
await this.sqlite.execute({database: "testEncrypted", statements: statements});
await this.sqlite.selectSQL({database: "testNew", statement: statement, values: values});
await this.sqlite.runSQL({database: "testNew", statement: statement, values: values});
await this.sqlite.runSQL({database: "testEncrypted", statement: statement, values: values});
...

in the plugin, a dictionary handling the pair dbName, dbHandler
this will allow for multiple database connections
so the connection will be done once and not anymore for each request/execute/query/transaction
i am making the test in using androidx.sqlite.db.xxx so hopefully get less problems with database parallelism
What do you think about this or do you have any idea on passing kind of db-handle object, will you be ready to test a preliminary release on Android

i'm not familiar enough with the system sqlite on android to speak into the db-handle issue. in the end it just comes down to what you save in the internal state to identify/access the correct database. if android-sqlite does not have db-handles and you just pass the same dbName to requests (i.e. it manages the open databases internally) then you just keep track of string.
I think it would be a nice touch if the plugin would do some abstraction and return an instance like this after connecting:
export interface ISQLiteDatabaseConnection {
    runSQL(statement: string, values?: any[]) => Promise<ISQLiteResult>;
}

class SQLiteDatabaseConnection implements ISQLiteDatabaseConnection {
    constructor(private dbName: string; private sqlite: SQLiteDriver) {}
    runSQL(statement: string, values?: any[]) {
        return this.sqlite({ database: this.dbName, statement, values});
    }
}

class CapacitorSQLite {
    createConnection({dbName, ...) {
         await this.sqlite.createConnection(...);
         return new SQLiteDatabaseConnection(dbName, this.sqlite);
    }
}

Then you could do:
const db = await CapacitorSQLite.createConnection({ dbName: 'test', ...})
const response = await db.runSQL('SELECT ...');

@chriswep
const db = await CapacitorSQLite.createConnection({ dbName: 'test', ...})
const response = await db.runSQL('SELECT ...');

this is what i was trying to do and test but it doesn't work . i asked the question to the capacitor team of how to pass a Native Object pointer to the typescript code but i did not get the answer yet.

@chriswep I will see if i can implement what you suggest in a typescript wrapper service to the CapacitorSQLite plugin

@chriswep i have created a wrapper plugin for Angular that way (see below) and it is working well. Do you thing that will answer your request?
so i create a class sqliteConnection.ts
export abstract class ISQLiteConnection {
    abstract open(): Promise<any>;
    abstract close(): Promise<any>;
    abstract execute(statements: string): Promise<any>;
}
export class SQLiteConnection implements ISQLiteConnection {
    constructor(private dbName:string, private sqlite:any) {}
    async open(): Promise<any> {
        const res: any = await this.sqlite.open(
                                            {database: this.dbName});
        return res;
    }
    async close(): Promise<any> {
        const res: any = await this.sqlite.close(
                                            {database: this.dbName});
        return res;
    }
    async execute(statements: string): Promise<any> {
        const res: any = await this.sqlite.execute(
                                            {database: this.dbName,
                                             statements: statements});
        return res;
    }
    // TODO other commands to run, query, ...
}

i then create a service sqlite.service.ts
import { Injectable } from '@angular/core';
import { SQLiteConnection } from './/sqliteConnection';

import { Plugins, Capacitor } from '@capacitor/core';
import 'capacitor-plugin-sqlite';
const { CapacitorSQLite } = Plugins;

@Injectable({
    providedIn: 'root'
})

export class SQLiteService {
    handlerPermissions: any;
    sqlite: any;
    isService: boolean = false;
    platform: string;

    constructor() {
    }
      /**
   * Plugin Initialization
   */
    initializePlugin(): Promise<boolean> {
        return new Promise (resolve => {
            this.platform = Capacitor.platform;
            console.log("*** platform " + this.platform)
            this.sqlite = CapacitorSQLite;
            if(this.platform === "android") {
                this.handlerPermissions = this.sqlite.addListener(
                    'androidPermissionsRequest', async (data:any) => {
                    if (data.permissionGranted === 1) {
                      resolve(true);
                    } else {
                      console.log("Permission not granted");
                      resolve(false);
                    }
                });
                try {
                    this.sqlite.requestPermissions();
                } catch (e) {
                    console.log("Error requesting permissions " + JSON.stringify(e));
                    resolve(false);
                }
            } else {
                resolve(true);
            }
        });
    }
    async echo(value: string): Promise<any> {
        return await this.sqlite.echo({value: value});
    }
    async createConnection(database:string, encrypted: boolean,
                           mode: string,
                           version: number): Promise<SQLiteConnection> {
        const res: any = await this.sqlite.createConnection({
                                database, encrypted, mode, version});
        if (res.result) {
            return new SQLiteConnection(database,this.sqlite);
        } else {
            return null
        }
    }
    async closeConnection(database:string): Promise<any> {
        return await this.sqlite.createConnection({database});
    }
}

and now call the service from a component
import { Component, AfterViewInit } from '@angular/core';
import { SQLiteService } from '../services/sqlite.service';

@Component({
  selector: 'app-home',
  templateUrl: 'home.page.html',
  styleUrls: ['home.page.scss'],
})
export class HomePage implements AfterViewInit {
  sqlite: any;
  platform: string;
  handlerPermissions: any;
  initPlugin: boolean = false;

  constructor(private _sqlite: SQLiteService) {}

  async ngAfterViewInit() {
    // Initialize the CapacitorSQLite plugin
    this.initPlugin = await this._sqlite.initializePlugin();
    const result: boolean = await this.runTest();
    if(result) {
      console.log("$$$ runTest was successful");
    } else {
      console.log("$$$ runTest failed");
    }
  }

  async runTest(): Promise<boolean> {
    let result: any = await this._sqlite.echo("Hello World");
    console.log(" from Echo " + result.value);
    // initialize the connection
    const db = await this._sqlite
                .createConnection("testNew", false, "no-encryption", 1);
    let retB: any = await db.open();
    if (!retB.result) {
      return false;
    }
    // create tables
    const statements: string = `
    CREATE TABLE IF NOT EXISTS users (
        id INTEGER PRIMARY KEY NOT NULL,
        email TEXT UNIQUE NOT NULL,
        name TEXT,
        company TEXT,
        size FLOAT,
        age INTEGER,
        last_modified INTEGER DEFAULT (strftime('%s', 'now'))
    );
    CREATE INDEX IF NOT EXISTS users_index_name ON users (name);
    CREATE INDEX IF NOT EXISTS users_index_last_modified ON users (last_modified);
    CREATE TRIGGER IF NOT EXISTS users_trigger_last_modified
    AFTER UPDATE ON users
    FOR EACH ROW WHEN NEW.last_modified <= OLD.last_modified
    BEGIN
        UPDATE users SET last_modified= (strftime('%s', 'now')) WHERE id=OLD.id;
    END;
    PRAGMA user_version = 1;
    `;
    retB = await db.execute(statements);
    if (retB.changes < 0) {
      return false;
    }
    retB = await this._sqlite.closeConnection("testNew");
    if(!retB.result) {
      return false;
    } else {
      return true;
    }

  }
}

@chriswep , @digaus I have published a first alpha release of the refactor npm i @capacitor-community/sqlite@refactor which basically do what @chriswep has described. It is only developed for Android platform and doesn't contain all the methods. But i think is enough for understanding the principle. i also published a Ionic/angular app demonstrating the use of the new plugin  https://github.com/jepiqueau/angular-sqlite-app-refactor If you could have a look and comment

thanks @jepiqueau for pushing this forward. i wonder if SQLiteConnection is part of the plugin interface or if the user has to implement it themselves? if possible i think it would make sense to have that included in / returned from the plugin directly. don't know however how flexible you are with Capacitor plugins on the javascript side.

@chriswep SQLiteConnection and SQLiteDBConnection are part of the javascript side of the plugin
import { Plugins, Capacitor } from '@capacitor/core';
import '@capacitor-community/sqlite';
import { SQLiteDBConnection, SQLiteConnection } from '@capacitor-community/sqlite';
const { CapacitorSQLite } = Plugins;
...
const sqlitePlugin: any = CapacitorSQLite;
this.sqlite = new SQLiteConnection(sqlitePlugin);
...
const db: SQLiteDBConnection = await this.sqlite.createConnection(
                                "testDB", true, "secret", 1);
...
    // open db testNew
    ret = await db.open();
    if (!ret.result) {
      return false;
    }

    // create tables in db
    ret = await db.execute(createSchema);
    if (ret.changes.changes < 0) {
      return false;
    }
    // add one user with statement and values
    let sqlcmd: string =
                "INSERT INTO users (name,email,age) VALUES (?,?,?)";
    let values: Array<any>  = ["Simpson","Simpson@example.com",69];
    ret = await db.run(sqlcmd,values);
    console.log()
    if(ret.changes.lastId !== 1) {
      return false;
    }
    // add one user with statement
    sqlcmd = `INSERT INTO users (name,email,age) VALUES ` +
                              `("Brown","Brown@example.com",15)`;
    ret = await db.run(sqlcmd);
    if(ret.changes.lastId !== 2) {
      return false;
    }
    // select all users in db
    ret = await db.query("SELECT * FROM users;");
    if(ret.values.length !== 2 || ret.values[0].name !== "Simpson" ||
                                  ret.values[1].name !== "Brown") {
...

Voilà, that simple as you requested

@chriswep , @digaus  The refactor process is now ended with the new release 2.9.0-beta.1 of the plugin. if you could have a look and revert to me .  It has been a quite heavy process, but all the tests pass. you could see the usage in a Ionic/Angular app https://github.com/jepiqueau/angular-sqlite-app-refactor Hope you will find what you were looking for. After testing, you could close the issue.

@chriswep as you did not comment i close the issue which is solved by the refactor.

@chriswep as you did not comment i close the issue which is solved by the refactor.

thanks @jepiqueau - i'll get back to this when i tackle the migration form cordova-sqlite

thanks @jepiqueau - i'll get back to this when i tackle the migration form cordova-sqlite

thanks @chriswep

thanks @chriswep

source: gharchive/issue

authors: chriswep digaus jepiqueau

repo: capacitor-community/sqlite

license: MIT

license_type: permissive

license_source: github-api


text:
bug: Barcode Scanning plugin failing to build on simulator for fresh Ionic project

Plugin(s)

[X] Barcode Scanning
[ ] Face Detection
[ ] Face Mesh Detection
[ ] Selfie Segmentation
[ ] Translation

Version
6.0.0
Platform(s)

[ ] Android
[X] iOS
[ ] Web

Current behavior
When adding the barcode scanning plugin, version 6.0.0, the capacitor ceases to build the app via the command line. Error: SwiftEmitModule normal x86_64 Emitting\ module\ for\ App (in target 'App' from project 'App'). Just before adding this package it all worked just fine.
Expected behavior
To be able to build my Ionic app
Reproduction
https://github.com/JaapWeijland/barcode-scanning-repro
Steps to reproduce

Install @ionic/cli (7.2.0)
Run ionic start and create a blank React app.
Run ionic cap add ios
Run ionic cap run ios
Choose iPhone 15 simulator
Verified the build works!
Run npm i @capacitor-mlkit/barcode-scanning
Run ionic cap sync
Run ionc cap run ios
Choose iPhone 15 simulator
Failed with aforementioned error.

Other information
Apple M1 Pro, Sonoma 14.4.1
Xcode 15.3
Simulator iPhone 15
Cocoapods 1.15.2, which pod: /opt/homebrew/bin/pod (so its installed with homebrew)
Using capacitor v6
Note: when I run the app via Xcode choosing a simulator with rosetta active, it works. However, I can't choose these rosetta simulators in the list ionic cap run ios provides me.
Capacitor doctor
[success] iOS looking great! 👌
Before submitting

[X] I have read and followed the bug report guidelines.
[X] I have attached links to possibly related issues and discussions.
[X] I understand that incomplete issues (e.g. without reproduction) are closed.

So I found how I can fix this by altering the ionic cap ios run command of capacitor. In node_modules, in the file @capacitor/cli/dist/ios/run.js, change the line that says id=\${target.id} to id=\${target.id},arch=x86_64. This will force capacitor to build the project using Rosetta, which is needed to build a project using this plugin.
I'm not sure if this is a problem with this plugin, or a missing feature of capacitor to allow for building using Rosetta, especially now that this option to do this as a configuration within Xcode has been removed by Apple.

This should fix your issue: https://capawesome.io/blog/troubleshooting-capacitor-ios-issues/#could-not-find-module-capacitor-for-target-x86_64-apple-ios-simulator
No need to use Rosetta. Feel free to take a look at the official demo app for reference: https://github.com/robingenz/capacitor-mlkit-plugin-demo/blob/main/ios/App/Podfile

Thanks for your response. I already tried this, and unfortunately, it does not work. I added the lines to the reproduction repo as well, so you can try it out if you want.

Okay, now I was able to reproduce your problem. But only in the simulator, on a device it works without problems.
This is how i fixed it: https://github.com/robingenz/capacitor-mlkit-plugin-demo/commit/febf32339702917889d65451a0a62bdd22836e2b

Works! Thanks alot. Since from now on all newcomers will pretty much experience what I've experienced when they want to run their app on a simulator, it might be handy to add this to the documentation or FAQ or something similar. Anyway, thank you very much for helping out!

I'm glad that it is now working. I have updated the blog post.

authors: JaapWeijland robingenz

repo: capawesome-team/capacitor-mlkit


text:
allowing only a reverse proxy

hello,  im curently running a reverse proxy setup with multiple containers in docker compose. I have a domain  that the reverse proxy uses. I am planning on using authentication via the reverse proxy  for all my apps.
However the reverse proxy can be bypassed by entering domain.com:port of specific container this is because my reverse proxy is on the same machine as my other services and i cant shut any ports via the router.
can i use whalewall to restrict the incoming request of all the container to the reverse proxy and would doing this prevent this kind of bypass?
im new to containerization and networking so sorry if its already stated elsewhere
one way that has worked before was to have the reverse proxy on a separate device and ip address  but i cant do that anymore and i don't know how i can do that with docker

Yes, you should be able to accomplish this by setting whalewall rules for the reverse proxy container and all containers behind the reverse proxy. You would only set mapped port rules and expose ports for the reverse proxy container and then all other containers wouldn't be able to be accessed directly.

repo: capnspacehook/whalewall


text:
added a form element to stop console.error from running every time yo…

…u click in the homepage search div; additional tweaks to address console errors

also changed the event handler for the form to be "submit" instead of "click" with the form element, the "submit" handler now works and won't generate a console error every time you click in the search div

source: gharchive/pull-request

repo: capydays/drinks-for-all


text:
Add additional title/notes/copyright message

Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
Add some title as a window title maybe or copyright message or some other message at the bottom. Like:
Title:

Copyright or additional message:

Describe alternatives you've considered
Editing, lol.
Additional context
Maybe I can develop this feature.

Please use an addition image editors to add copyright information. If you want to add the Carbon copyright/watermark, you can do so in the settings men

repo: carbon-app/carbon


text:
fix(stateful-table): update pagination properly on deletes

Closes #1906
Summary

If all items on the last page of a Stateful table were deleted the page would show empty and the pagination bar would disappear. This PR uses the TABLE_APPLY_ACTION action to also update the pagination to reflect the changes in data on delete actions.

Change List (commits, features, bugs, etc)

update pagination.totalItems on delete
check if the current page is greater than the expected total number of page after the deletion and update the current page if necessary
pass the updated totalItems to the view.pagination prop instead of the initial totalItems passed to the table.

Acceptance Test (how to verify the PR)

On the simple stateful table story, turn on pagination, go to page 10, and delete all items on that page. Those 10 items should be deleted and then you'll be moved to page 9 where the pagination bar will show 81-90 of 90 items.

Regression Test (how to make sure this PR doesn't break old functionality)

Page should not change if deleting items (event full pages) when not on the last page of the table.

Things to look for during review

[ ] Make sure all references to iot or bx class prefix is using the prefix variable
[ ] (React) All major areas have a data-testid attribute. New test ids should have test written to ensure they are not changed or removed.
[ ] UI should be checked in RTL mode to ensure the proper handling of layout and text.
[ ] All strings should be translatable.
[ ] The code should pass a11y scans (The storybook a11y knob should show no violations). New components should have a11y test written.
[ ] Unit test should be written and should have a coverage of 90% or higher in all areas.
[ ] All components should be passing visual regression test. For new styles or components either a visual regression test should be written for all permutations or the base image updated.
[ ] Changes or new components should either write new or update existing documentation.
[ ] PR should link and close out an existing issue

Looks ok. Will check once deploy preview is udpated.

authors: davidicus kevinsperrine


text:
Modal shouldSubmitOnEnter should dispatch "click:button--primary"

Refs: #310
Currently, Modal shouldSubmitOnEnter only dispatches the "submit" event. It should also fire "click:button--primary."

Fixed in v0.52.0
CC @sallaben


text:
[Enhancement] Support secondary or multi-column sorting on tables

Requesting an enhancement on the Carbon tables to support secondary sorting or more.  We have an issue open on ICP where the user is having UX issues on one of our tables.
The data being generated for one specific table is all being created at ICP installation; so they all share the same timestamp.  All of our tables default to sorting by time of creation.  We are doing API calls about every 5 seconds, so the data that is getting returned starts shifting in random order each call because the timestamps are all the same.
We may default this table to sort by the name field for now, but we prefer to have the newest information always available to the user at the top by default.  It would be great to be able to sort by time first and name second.  The user who opened the issue on ICP provided the screenshots below:
First View:

After API call:

Hi @showeimer! 👋 Thanks for taking the time to suggest this enhancement 🙂
Definitely agreed that multi sort would be valuable! Unfortunately this isn’t something immediately on our roadmap, as we are working on table edits currently, but we would gladly accept PRs that add in this behavior!

authors: joshblack showeimer

repo: carbon-design-system/carbon-components

license_source: bigquery


text:
chore(components): introduce @HostListener

@HostListener(type, options?) decorator attaches an event listener of the host element of a custom element.
The custom element class using @HostListener() has to extend HostListenerMixin that does the actual event listener work. Otherwise, @HostListener() throws an error.

Sorry, not quite what I was meaning ... the old signature for the HostListener was amazing, I was thinking more for the class mixins - HostListenerMixin -> @HostListenerMixin

authors: asudoh cal-smith


text:
Website content crafting and publishing: Components/ Masthead L0 grouped typeahead results enhancement

Objective
Provide Carbon for IBM.com adopter the best practices and guidance they need to adopt Carbon for IBM.com.
General publishing process
Please make sure you always start the content crafting process by viewing the 2020 website content publishing process box document first.
Publishing for Components section website page

Content crafting starts as soon as the design is finalized.
Components section includes UI components and Layout components.
The content can be at the MVP level. MVP means the content meets the basic needs of an adopter for learning. It does not mean the content crafting will or should stop at this stage. Future enhancement should be working on with a separate task issue.
MVP content publishing template and requirement can be found in the Publishing template and best practices/_UI and Layout Components folder.
Please use the visual and design spec files inside of the Carbon for IBM.com Box folder as a reference.
Content publishing to the live website can be code or markdowns. If you need guidance on how to start with markdowns, please see the step by step instruction in the Editing and publishing in Github Box document.
When the page is published, please also make sure the components table list on the Overview page is updated as part of the publishing task.

File naming and storage

Name the Sketch file with the following convention: Name of the category + components/pattern/guideline + version. ex: Components-Feature card block v1.0, Guidelines-Expressive theme v1.0, Help-Overview v1.0 and etc.
Please save all content publishing work files (box, markdowns link, or Sketch file) in the proper sub-folder under the ✍ Content + Design folder.

Publishing acceptance criteria

[ ] Content crafting is done.
[ ] Asset image creation is done.
[ ] Page design is done (as needed).
[ ] Publishing with markdown is done.
[ ] Update the Overview page component table list with the new component name, link to the newly published component page, description and status when the markdown is done.
[ ] Create a thumbnail of the new component.   Thumbnail examples can be viewed here.  The Sketch file that will help you get started can be found here.  If you are not familiar with component thumbnail creation, please ask another designer or Jenny for assistance.  Once the thumbnail is created, please review with Wonil for approval, then export the file as an SVG for implementation.
[ ] Add the new component thumbnail to the Overview page/Gallery view by adding the image file to the components.json markdown file. Please follow the file naming format so it can be linked to the website page automatically (ex: thumbnail svg file for "Card section – Carousel" should be "cardsection-carousel.svg").
[ ] QA'd by Jenny.
[ ] Web page preview is approved by Linda, Jeff, and Wonil.
[ ] PR merged by Jenny.
[ ] Web page is published and live on the website.

Based on Kenny's and Jenny's participation in the KC design and engineering meetings: The features needed are customized to the KC platform and are not something that we would ever add to the carbon for ibm.com code base.


text:
[Video player] Additional states:  hover, active, focus

Our Video player have no hover state.
Expected: video still to darken 8%
Similar to image card hover seen https://ibmdotcom-web-components-canary.mybluemix.net/?path=/story/components-card-in-card--default
Based on the comments below, this epic includes the work for all of the potential states for the video player.
Additional Information

A bug for the focus state has been created and scheduled in the April release.  issue (#5660 )

All states need to be looked at here I believe.
:hover
:active
:focus

Discussion:  The designs are out of the box Kaltura so no designs are created.  Changed to Feature request
Dev research:  1)  Investigate if the out of the box player can be customized
Tasks: Convert to epic, Design & Dev research and investigation, design & specs, BA, dev issues (WCC & stand alone react), Visual QA & Prod QA, web site

The designs are out of the box Kaltura so no designs are created.

Just to be clear I don't think this is out of the box Kaltura. We are using an thumbnail image and play icon, and then replacing those with Kaltura out of the box.

Related issue Dragos recently looked at https://app.zenhub.com/workspaces/carbon-for-ibmcom-5d449f3642eb1962336cbe52/issues/carbon-design-system/carbon-for-ibm-dotcom/5642 If we think this is a good feature to add, we need to consider the pros and cons listed. Personally I'm on the side of NOT allowing custom cover image.

Collecting more info. Hover should use:
Lighten 8% on dark themes
Darken 8% on light themes
Is this possible?
Based on card spec:

Interactive prototype I build awhile ago in Figma:
https://www.figma.com/proto/pVH385cIPauyJgF7o1k2CK/Card-hover?node-id=36%3A0&scaling=min-zoom

authors: RobertaJHahn photodow shixiedesign


text:
[Masthead v2]: Setup a Storybook preview of the L0/Masthead v2 and wrap under a feature flag

Engineering info:

Innovation Team JIRA Ticket: https://jsw.ibm.com/browse/ADCMS-2481
DPO consulting engineer: N/A

Description

Add a storybook preview of the L0/Masthead v2 for reviewing and testing purposes.
This preview should be wrapped behind a feature flag.

Additional information

Overview and scope of the engineering work should be included in the linked epic (#9208)

Acceptance criteria

[ ] Masthead v2 should be visible in the SB's left column

The work for this issue will be accounted for in this PR - https://github.com/carbon-design-system/carbon-for-ibm-dotcom/pull/9275


text:
Create add-on package for animated icons

I have created animations of some of the Carbon Icons, to be used in the API Connect landing page. We would like to start an animation library to make the existing animations available for others to use, and to enable others to design and contribute additional animations.
Prototype of animations: https://pages.github.ibm.com/Krista-Starr/motion/#/landing-page
(hover on the tiles)
Animations are created using CSS and are of the form:
.trigger-selector:hover .icon-element-selector {
  ... some CSS
}

The user of the package could either use a pre-canned trigger selector e.g. .carbon-animate-icon:hover, or use an SCSS mix-in to apply the animation to a trigger-selector of their choice.
We would need to have classes added on svg elements, perhaps optionally based on a component switch, and ideally added by the icon builder code.
Credit to @lee-chase for the above implementation ideas.

Re-opening issue in Carbon monorepo.

repo: carbon-design-system/carbon-icons


text:
feat: card refactor

Contributes to #525
This PR addresses refactoring the Card component to use this base & themed component pattern. Now the card component has been broken up into three separate ones. Card, ExpressiveCard, and ProductiveCard where the themed components are just wrappers for card.
import { Card }
...
export const ExpressiveCard = ({ props specific to the expressive card }) => <Card {...props} />

So the main idea here is to split up the components and their dependencies based on their specific use case. I removed the card from storybook because I think if we went this route we would encourage users to use the themed component and not the base, which is also why I removed the prop comments from card. There is some duplication of props and prop comments between the two themed components, but since we're relying on storybook for documentation I think it's just something we have to live with even though code duplication isn't ideal, but now when you look at the storybook for each component they now display the props that they actually depend on. previously when everything was in a single place we had received some feedback that looked like

why is X a control in expressive when X is only applicable to productive

Now that's no longer a problem.
Using this pattern now all the shared code is found in the Card folder and anything specifically related to the themed components is found in their respective folder, which is mainly css. There is also some general cleanup going on here.

@dcwarwick looks like you did not complete the review on this PR.
Sorry @davidmenendez I think we said we'd submitted a review.

@dcwarwick

We don't need to explicitly destructure and rebuild props, because we can just collect the entire set of props and pass them through.

I was thinking about this one a lot. my thinking is that this is another step to ensure the consumer is unable to pass an undesirable prop (accidentally or otherwise). since they're just wrappers for the card component it would still be possible to send productive only props to expressive and vice versa if we simply pass it as ...props. it does mean more lines on our end, but i think going about it this way helps ensure less bugs and unwanted behavior for users.

@davidmenendez It's a good thought. The thing is, one of our guidelines calls for us to pass all 'extra' props through as HTML attributes onto the main HTML element, which the Carbon components also do. That means we need to include a spread ...rest at the end of the props destructuring, and then add ...rest when passing them through to the wrapped component. In fact, that was the review comment I originally started to make -- then we realised that if we're doing that then we might as well drop all the explicit destructuring, since any props passed in that we don't explicitly destructure will get passed across anyway.

that's a good point. i hadn't considered that. what do you think about creating a utility that strips undesired props? something like
/**
 * props should be an object of properties
 * blacklist should be an array of undesired property names
 */
const stripProps = (props, blacklist) => // object not containing blacklisted props

i think doing something like this will help cut down on bugs and consumer bug reports because ultimately with this kind of pattern there is a certain level of prop sharing between the base component and the wrapper components

@dcwarwick while I was able to add this new utility function to solve the prop problem I don't think we can remove the proptypes since ultimately these changes are here to benefit storybook and our documentation. so we will still have some redundancy there, but better safe than sorry, right? 😂

@lee-chase @dcwarwick i've made a small additional update to move each card to a shared Cards folder in storybook to improve organization.

These storybook changes have been approved by design and they are very happy with the improved UX.

authors: davidmenendez dcwarwick lee-chase


text:
Mega menu mobile – overlapping menu items or double menus.

Detailed description

Describe in detail the issue you're having.

When viewing the mobile menu on my iPhone in Safari, if the content within a category is a shorter length than the previous high level category I can see both sets of menu text. In some cases the text is overlapping, and other cases not.
In the screen shots below "Security, Supply chain, Payment plans for products, View all" are a part of the Products & Solutions menu, however I am viewing Products & Solutions > Top products or Supply chain sub categories.

Products & Solutions > Top products
Products & Solutions > Supply chain

Is this a feature request (new component, new icon), a bug, or a general
issue?

Bug

Is this issue related to a specific component?

Mega menu mobile

What did you expect to happen? What happened instead? What would you like to
see changed?

User should only see the content from a single category or sub category they are viewing.

What browser are you working in?

Safari iOS 13.6 on iPhone 7

What version of the IBM.com Library are you using?

Latest canary

What offering/product do you work on? Any pressing ship or release dates we
should be aware of?

August
Steps to reproduce the issue

Open mega menu mobile from latest canary
Products & Solutions > Top products
Scroll down in menu
etc.

Additional information

Screenshots or code
Notes

Adding another issue to above list,
Issue: On Mobile - Hamburger menu - while clicking on "Data & Analytics, IT infrastructure and security" menu options, user is navigated to end of the list options and user needs to scroll up to see start of the list.

authors: Praveen-sr oliviaflory

repo: carbon-design-system/ibm-dotcom-library


text:
test(Selenium): adding selenium e2e tests

Related Ticket(s)
{{https://app.zenhub.com/workspaces/ibmcom-library-5d449f3642eb1962336cbe52/issues/carbon-design-system/ibm-dotcom-library/2455}}
Description
Added following test scripts
Linklist
Footer
Masthead
Contentblocksimple
Leadspceblock
Pictogram
CalloutQuote
Contentblocksimple
Imagewithcaption
TOC
Added one folder 'URL_List'

Removed chromedriver and init file.

@chetankan @Praveen-sr fyi, I pushed an initial change in order to let it run at least in a linux environment. However, to get this to work on linux, we would have to get selenium (which does exist) in the Jenkins agent, but would have to include dotenv and chromedriver, so we will have to do a customization there.
One thought, I'm wondering if we should be using WebDriverJS instead? It aligns fairly nicely with the rest of our repo, and we can pull those in and run agnostically across any operating system? If we're ok with that, I can take some time to convert the existing tests for WebdriverJS.

So we now seem to have a different error than the original one, basically need to get the offline mirror back in sync. I have the missing file in my local, please let me know if yarn install without offline mode doesn't download it automatically somehow.

So we now seem to have a different error than the original one, basically need to get the offline mirror back in sync. I have the missing file in my local, please let me know if yarn install without offline mode doesn't download it automatically somehow.

You're probably right. I tried running a yarn install after yarn clean, still getting the error:

@asudoh I'm also debating about ditching this approach entirely in favor of programmatically creating e2e tests like how you have set up with the karma accessibility test runner. How easy would it be to adapt component level tests similar to what's in here and get harnessed into what you are running with karma?

@jeffchew Sounds interesting, does it mean automatic walk-through of each stories and running tests against them? If so, wondering how test cases for each stories are same/different. For karma-accessibility-checker all the test cases are the same because the only thing the test case does is letting DAP engine scan the DOM.

@jeffchew Sounds interesting, does it mean automatic walk-through of each stories and running tests against them? If so, wondering how test cases for each stories are same/different. For karma-accessibility-checker all the test cases are the same because the only thing the test case does is letting DAP engine scan the DOM.

Hey @asudoh , what I meant was using the same approach where for each component, you set the ReactDOM.render of the story component with the corresponding props for that particular e2e test. Each test could be set for whatever variations you need to test for each component. For this PR, currently what is being rendered is the actual URL via selenium then using WebdriverIO API to traverse the DOM and manipulating the page for each e2e test.

@jeffchew I see, thank you for clarifying. Given Selenium nature where test cases are defined in out-of-browser process (e.g. node.js or Java), we need to create a test harness UI (pseudo app) to do the React rendering for each test cases, but it'll definitely be a good way to do E2E testing given doing so allows us to isolate out the Storybook-specific stuffs.

You're probably right. I tried running a yarn install after yarn clean, still getting the error:

I have seen this several times in my other efforts. For all of those cases I solved it be editing yarn.lock to remove all @babel/core entries and re-running yarn install.

Closing this PR out. I'm going to investigate another way of approaching this per discussion above, but will keep the branch alive for now.

authors: asudoh chetankan jeffchew


text:
Add 4 new sizes for user avatar

### Tasks
- [ ] Use type tokens from Figma for each text sizes

From Cate:
Carbon 10 tokens used for user profile image

Code: Text & and icon colors need to be converted to tokens

Icon: Dark theme ($icon-inverse), Light theme ($icon-03)
Text: Dark theme ($text-inverse-01), Light theme ($text-04)

Code: Initials — type scale convert to text tokens

XL 64 ($product-heading-04), L 32 ($body-short-01), M 24 ($label-01)

Missing sizes for the 48px size.

repo: carbon-design-system/ibm-products


text:
feat(input): adds support for input debounce

Ref: https://autoricardo.atlassian.net/browse/CAR-6929
This PR fixes the input debounce problems.
To use the debounce functionality for an input, just pass a prop debounce with a number as value (delay).

If you two get into a call to discuss this, I'd love to understand this better as well. There are some issues I remember that might be vaguely related

@gzimh @lkappeler what's the staus here? Can we move forward?

@Averethel I need to find some time to wrap my head around, hopefully, tomorrow if you have capacity feel free to take over :) just let me know.

@lkappeler all good thanks for taking care 👍🏻

So I had some time to look into the topic...
In general, I think it goes in the right direction with what you wanna achieve with updating the input state and calling the callback after the denounce.
Regarding the event.persist() I am a bit worried that this is a rather short term solution as they say with v17 event pooling will be dropped...
I looked a bit in the usage of our input callbacks and I think we can update it towards debouncing values or like a defined object as the primary use case is to update something based on the new value (found this artcle) It will allow us to avoid the problems with the event target that is gone while still providing the needed values to update the app state. something like
{ name, cleared, value } = event.target

the cleared, we might be able to factor out
Before you start I would like to get @Averethel or @masone opinion on the value-based debounce approach so don't head in the wrong direction. (Also from my perspective at this point consider updating the bug to a tech task)
Please DM me if you have questions or feel like it's simpler to have a chat on it

@lkappeler Could we try to go with this solution and look into the alternative (react 17 compatible) approach in a follow-up? What do you think?

From my point of view, I would avoid implementing a fix that will create more work when we start tackling the update for the following reasons:

The functionality is broken atm and it has been worked around in the components package and the concrete projects. We kinda already created a follow up with this bug to properly tackle the case when implementing the feature the bug was created from. By going with an incompatible approach with the current react version I fear that we are creating an overhead of work by creating an in-between version that will increase the refactoring efforts when tackling the update.
The projects are already using custom debounce implementations in wrappers etc (components lib as well that probably need to be updated towards using the functionality from the base input so I would rather define a future proof approach
We are more than 6 months behind with updating react so it is something we wanna do in the future for sure and I don't wanna create more effort than already needed

What I would do is break the interface and return the new value as a string instead of an event, along with keeping the interface for the delay and replace the usage across the projects as a fix and form there create follow-ups to refactor out the custom implementations withing the components lib and the consuming projects so we can get to a state where we rely on the implementation in a single place to keep it maintainable.
That being said I don't have a strong opinion on how to organize it in the form of tickets and sprint planning, we should stay on the topic and find a way forward and keep an eye on not creating overhead by seeking a fast way to close the task.

I'm with Lorenzo here. however, feel free to approach it your preferred way if you commit to tackling it immediately. We can split tickets and PRs sensibly, but I'd like to follow through on Lorenzos proposal in coming days and weeks. As you'll be the one working on it, feel free to approach it the way you think brings you to success

Had a discussion with Gzim:
He pointed out that he expects the implementation to work fine with react 17 as their breaking change was to remove a backwards compatibility implementation on top of the browsers events.
https://reactjs.org/docs/legacy-event-pooling.html

This is because React reused the event objects between different events for performance in old browsers, and set all event fields to null in between them. With React 16 and earlier, you have to call e.persist() to properly use the event, or read the property you need earlier.

In React 17, this code works as you would expect. The old event pooling optimization has been fully removed, so you can read the event fields whenever you need them.

This is a behavior change, which is why we’re marking it as breaking, but in practice we haven’t seen it break anything at Facebook. (Maybe it even fixed a few bugs!) Note that e.persist() is still available on the React event object, but now it doesn’t do anything

We are not fully sure if I will just work or the value will be empty. He will create a PoC (locally with build react app with react 17) to see if it works or not.
If it works we tackle the unit test and go with the implementation otherwise we will discuss how to continue.

I've created a PoC with react: ^17.0.2 and I can confirm that this solution works well 🎉
As the docs mention, calling event.persist() in react 17 doesn't have any effect. But we need to explicitlly call it on 16.x.x.

What actions will we need to plan to bring this change to the projects? Do you plan to do them with your current task or would we create additional ones?

we're still relying on persisting the event. if I remember correctly, Lorenzo pointed out that this is deprecated in the current React version. can you elaborate on the plan again?

We are clear in that regard, react 17 wont have any breaking changes. Basically they will implement the e.presist() behaviour by default and calling e.presist() will make no difference. But in the current version it's a must that we call it.

this sounds very odd. I'd prefer to understand what is going on exactly before merging with this workaround
Yes indeed. It's needed for tests only. In real-world executions, the e is passed without losing the target.value as is expected.
I assume something is going on with tests that causes the e.target.value to be empty, so, as a workaround I copied the object state to a new object and passed that instead.

What actions will we need to plan to bring this change to the projects? Do you plan to do them with your current task or would we create additional ones?

https://github.com/carforyou/carforyou-dealerhub-web/pull/1326 I did only one for now, its for the search input on Contact Requests (DH)
Basically all we would need to do is to pass in the debounce prop to Input component

Update: Colin added  the changes discussed with Pascal in a stacked PR: https://github.com/carforyou/carforyou-listings-web/pull/3257

After another discussion with @lkappeler  we decided that the workaround (object copying) is not worth it just so the test can pass. So, we decided to skip the test completely for now.

authors: Averethel disufi gzimh lkappeler masone

repo: carforyou/carforyou-components-pkg


text:
Add placeholder {{cargo_embedded}}.

Suddenly saw a need for this kind of feature - should have created an issue first, but alas....
If not liked, lets use it as a discussion on how we might support this kind of functionality.
It creates a new builtin placeholder: {{cargo_embedded}}. The placeholder can then be used to e.g ignore certain files if the user is expanding the template inside an existing cargo project.
In other words: it makes it easy to create templates that can be used to create sub-projects 😄

Nice idea.
So with sub project you mean something that is not a full cargo project but like a component or piece that will go into an existing project?
As for example an end to end test module? Or in general a module of a project?

In my case, in my bevy-template-rs templates (it's a multi template repository), all templates can be used as stand-alone projects. Very often though, I find myself using the crate-plugin template for minor crates (workspace members) inside a bigger one.
In that case, the template should NOT contain all the CHANGELOG.md, LICENSE-*, cargo/rustc configuration etc. files, and I like to print a little message informing the user to remeber to add the new project to the workspace.
Also, some lines in the expanded Cargo.toml is not needed (and may even cause problems) if the current template is used in this way (for instance setting up optimization levels).

@sassman any new thoughts?

I did oversee the comments, but I guess we have a winner name, right?

authors: sassman taurr

repo: cargo-generate/cargo-generate


text:
Procedural export of map

Hello Carla team,
I would like to know what is the best way to have access to every object in the world and every vertex of that object.
I wish to procedurally export the 3D representation (bbox or vertices) of the object in the world, so that I can render them myself.
The reason I want to do this is because I need a top down view of the environment like in this paper (ChauffeurNet) https://arxiv.org/pdf/1812.03079.pdf
Of course, I could use semantic segmentation and translate the camera above the car looking down, but I need precise information about where in the world is something like a traffic light or a marking lane.
I saw in Python API reference that for the class world there is a method get_actors(), but it only returned the traffic lights objects (since I had no other agent). I also thought of writing some C++ code for that, but I don't have that much experience with Unreal. If the C++ code is the best way to go, do you have any suggestion like what kind of class to inherit, or how to execute that code? I compiled last weekend the source code so many times that I began to become familiar with the project structure.
Thank you for your help.

Hello @Iftimie
I would like to know what is the best way to have access to every object in the world and every vertex of that object. I wish to procedurally export the 3D representation (bbox or vertices) of the object in the world, so that I can render them myself.
Now I have a problem. There is a triangular road sign model in the scene, and I want to get its vertices coordinates . Do you have a new solution?
Thank you.

repo: carla-simulator/carla


text:
New vehicle import issue

I want to develop a new vehicle and import in CARLA,
As per CARLA guidelines https://carla.readthedocs.io/en/latest/tuto_A_add_vehicle/
I have downloaded the skeleton of the Vehicle and when I try to import the fbx file in Blender,
I am getting the following error: ASCII fbx files are not supported
What I have tried :

I opened the file seperately in VSCode and saved it as a new file

Please guide me on how to proceed further. If I am doing something wrong, please let me know

It will not import because Blender does not import ASCII FBX files...
But you can try this:
https://www.reddit.com/r/blender/comments/csw9as/ascii_fbx_not_importing_try_this_method_28/

Yes this is an issue with blender, you should try what @BAROBA-3d recommended, saving the fbx in a format that blender can open will work.

Thank you @gsmario , @BAROBA-3d
I converted the file from ASCII fbx to Binary fbx using Autodesk fbx converter and it worked after that.
Here are the steps I followed: https://www.artstation.com/mattmurch/blog/ZA8l/quickly-fix-blender-ascii-fbx-import-error

authors: BAROBA-3d gsmario kanakab


text:
Atmospheric attenuation rate value meaning is asked

Hello,
what does Atmospheric Attenuation Rate: AtmospAttenRate.RecommendedValues = { TEXT("0.004") } mean?
Carla version: 0.9.11
Windows 10
UE 4.24
Thanks and have a nice weekends!

This question has been addressed in the discord channel.


text:
Building CarlaUnrealEditor failed under WSL

I tried to build Carla's dev-ue5 branch under WSL but it failed.
I followed the UE5_BUILD_README file but it failed in Building CarlaUnrealEditor

Hi,
this is from the CarlaUE5 doc.

I can see you use "/mnt/e/", which is the default path of an external drive.

I somehow missed this doc, but I faced similar issue trying to run Carla from WSL... and docker on windows didn't go well either :(

Hi, this is from the CarlaUE5 doc.
I can see you use "/mnt/e/", which is the default path of an external drive.

Thanks, I'll try again

Don't use sudo.
The output is telling you "Refusing to run with the root privileges."
If you have a permission problem, you set them to 777 to make sure it works (755 would be better)
chmod -R 777 path/to/carla

or

sudo chmod -R 777 path/to/carla

I used this commands, because I had a problem with the ros installation
git clone -b ue5-dev https://github.com/carla-simulator/carla.git CarlaUE5

cd CarlaUE5

env GIT_LOCAL_CREDENTIALS=your_git_user@your_git_token bash -x CarlaSetup.sh

cmake -G Ninja -S . -B Build --toolchain=$PWD/CMake/LinuxToolchain.cmake -DLAUNCH_ARGS="-prefernvidia" -DCMAKE_BUILD_TYPE=Release -DBUILD_CARLA_UNREAL=ON -DCARLA_UNREAL_ENGINE_PATH=$CARLA_UNREAL_ENGINE_PATH

cmake --build Build --target launch

不要使用 sudo。输出告诉您 “Refusing to run with the root privileges”。如果你有权限问题，你可以将它们设置为 777 以确保它有效（755 会更好）
chmod -R 777 path/to/carla

or

sudo chmod -R 777 path/to/carla

我使用了这个命令，因为我在 ros 安装时遇到了问题
git clone -b ue5-dev https://github.com/carla-simulator/carla.git CarlaUE5

cd CarlaUE5

env GIT_LOCAL_CREDENTIALS=your_git_user@your_git_token bash -x CarlaSetup.sh

cmake -G Ninja -S . -B Build --toolchain=$PWD/CMake/LinuxToolchain.cmake -DLAUNCH_ARGS="-prefernvidia" -DCMAKE_BUILD_TYPE=Release -DBUILD_CARLA_UNREAL=ON -DCARLA_UNREAL_ENGINE_PATH=$CARLA_UNREAL_ENGINE_PATH

cmake --build Build --target launch

I tried this command and it seems to be useful. But there was an error when running UE5Editor. And I ensure that the GPU driver and Vulkan have been installed, and information can be queried using nvidia-smi and vulkaninfo

Hi,
What Patrick said should indeed work. Could you share the output of both nvidia-smi and vulkaninfo? Btw, please attach logs using text files instead of images if possible.
Marcel

Do you have an CPU with graphics card?
Maybe your computer is using the CPU instead of the Nvidia graphic card (a guess)
sudo nvidia-settings

authors: MarcelPiNacy-CVC PatrickPromitzer merryHunter qq1183713118


text:
Adding 360 Lidars support (similar to Velodyne HDL-32E or VLP-16)

You may add any number of lidars to the car. Lidars have following settings with specified default values:
# Number of lasers
Channels = 32
# Measure distance
Range = 5000
# Points generated by all lasers per second
PointsPerSecond = 100000
# Lidar rotation frequency
RotationFrequency = 10
# Upper laser angle, counts from horizontal,
# positive values means above horizontal line
UpperFovLimit = 10
# Lower laser angle, counts from horizontal,
# negative values means under horizontal line
LowerFovLimit = -30
# wether to show debug points of laser hits in simulator
ShowDebugPoints = False
# Position relative to the player.
LidarPositionX = 0
LidarPositionY = 0
LidarPositionZ = 250
# Rotation relative to the player.
LidarRotationPitch = 0
LidarRotationRoll = 0
LidarRotationYaw = 0
Default values choosed close to Velodyne HDL-32E.
Also one lidar added to ./manual_control.py. You may see bird eye view of lidar points in upper left corner.
Lidar was made using UWorld::LineTraceSingleByChannel method. Serialization of lidar measurement made similarly to camera images.

Lidar measurements send in form of packet. Packet is points captured by lidar during last tick. So it is points from segment covered in last tick. On low FPS packets may include points from 360 degrees.

Hi @parilo,
The code looks great! seems to work as expected. Just noticed one thing, when a laser shot doesn't hit anything you leave the point at {0,0,0}, (you can see the white dot at the centre of the image). Do you anticipate any problem by not adding the misses to the list of points? At ALidar::ReadPoints we can skip the points if the ray-trace failed.

Hi @nsubiron,
If misses will not be added we will lose organized point cloud (I mean structure of whole points packet). For some algorithms it may be reasonable to have organized one. Also serialization and deserialization code will need to be rewritten because it assumes equals point counts in each channels. I can rewrite code to unorganized point cloud (remove misses) if it is needed. I leaved misses with {0, 0, 0} mainly for possible future use of organized point cloud.

authors: nsubiron parilo


text:
Check whether the send side is not idle, not the recv side

We think this fixes #312 -- for window updates, we should be checking against the client-initiated send streams, not the server-initiated recv streams.

@carllerche I wrote a test for this PR based on the description you posted in https://github.com/carllerche/h2/pull/313#pullrequestreview-158202788, but the test fails on both this branch and on master (with a PROTOCOL_ERROR).
It's possible I wrote the test incorrectly. Here's the code:
#[test]
fn srv_window_update_on_lower_stream_id() {
    // See https://github.com/carllerche/h2/issues/208
    let _ = ::env_logger::try_init();
    let (io, srv) = mock::new();

    let srv = srv.assert_client_handshake()
        .unwrap()
        .recv_settings()
        .send_frame(frames::headers(3).eos())
        .send_frame(frames::window_update(2, 66666));

    let client = client::Builder::new()
        .initial_stream_id(3)
        .handshake::<_, Bytes>(io)
        .unwrap()
        .and_then(|(mut client, h2)| {
            let request = Request::builder()
                .method("POST")
                .uri("https://http2.akamai.com/")
                .body(()).unwrap();
            let _ = client.send_request(request, false).unwrap();
            h2.then(move |result| {
                assert!(result.is_ok(), "result: {:?}", result);
                Ok::<_, ()>(())
            })
        });

    srv.join(client).wait().unwrap();
}

@hawkw I think the mock has to wait until it receives the headers frame? Also, I don't see where the stream is being closed.

@carllerche

I think the mock has to wait until it receives the headers frame?

Ah, you're right, there's a missing recv_frame in the mock --- can't believe I overlooked that. :sweat:

Also, I don't see where the stream is being closed.

I thought the client sending an EOS and then dropping the response would close the stream?

@hawkw it closes half... i think you might need to send a reset to force close.

Oops, sorry -- working on this dropped off of my queue. @hawkw thanks for adding in the tests :)

I am working on this, I have a test that fails now.

authors: carllerche hawkw rbtying


text:
Yet another MIO + coroutine asynchronous I/O library

Hi, charllerche,
I just release my first MIO + coroutine asynchronous I/O library simplesched:
https://github.com/zonyitoo/simplesched
Just implement it with the most simple techniques, I may release another library with improved scheduling algorithm in the future.
The performance test looks good. :)

Out of curiosity compared it with my implementation (https://github.com/tailhook/rotor-http/blob/master/examples/hello_world_server.rs):
> wrk -c 400 -t 2 http://localhost:8888
Running 10s test @ http://localhost:8888
  2 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9.22ms    1.13ms  23.08ms   90.99%
    Req/Sec    21.75k     1.60k   24.06k    86.00%
  432820 requests in 10.01s, 36.32MB read
Requests/sec:  43219.72
Transfer/sec:      3.63MB

And your implementation looks like this on my laptop:
> wrk -c 400 -t 2 http://localhost:8889
Running 10s test @ http://localhost:8889
  2 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    13.71ms    1.33ms  28.99ms   90.34%
    Req/Sec    14.64k     0.95k   15.91k    83.50%
  291449 requests in 10.01s, 25.02MB read
Requests/sec:  29103.17
Transfer/sec:      2.50MB

(both run single threaded, because I don't have too much cores, and my example doesn't support threads)
Unfortunately my implementation is much more incomplete, so comparison is unfair.

@tailhook Your implementation have better performance! Let me see what can I do...

@zonyitoo it's to early to compare, because I don't read any headers for determining request length, expect header, whatever... so it may be slower in the final implementation.

Looks cool. I'm going to close this issue for now. You can open a PR if you want to add a link to the project in the Readme.
Cheers.

authors: carllerche tailhook zonyitoo

repo: carllerche/mio

license: mit


text:
kqueue: not close when dropped

Hi, I notice that kqueue won't be close when dropped, which will cause our bench tests failed due to "too many open files".
I also notice that epoll is closed when dropped. So why kqueue is different?

It doesn't look intentional. Drop was added to epoll way before kqueue ever existed. Seems simple enough to remedy.


text:
fix read_callback user provided input case (in)sensitivity

Description
This PR is about to handle the situation when user provides input for contains or not_contains with capital letters while case_insensitive attribute only lowers device input's case. Previously, input is not matched though the difference is only letter case. This fix basically lowers user input strings if case_insensitive is True and string in operator will work as intended.
Type of change
Please delete options that are not relevant.

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
pytest case added in line with the spirit of the project owner.
Checklist:

[X] My code follows the style guidelines of this project (no GitHub actions complaints! run make lint before
committing!)
[X] I have commented my code, pydocstyle and darglint are happy, docstrings are in google docstring format, and all
docstrings include a summary, args, returns and raises fields (even if N/A)
[X] I have added tests that prove my fix is effective or that my feature works, if adding "functional" tests please
note if there are any considerations for the vrnetlab setup
[X] New and existing unit tests pass locally with my changes

@realvitya thank you as always!! LGTM!

authors: carlmontanari realvitya

repo: carlmontanari/scrapli


text:
Fixes bug for WLSS/WLSV revision methods and changes test_revision dataset

Fixes #22: bug where WLSV and WLSS revision methods are treated as OLS. Changes the dataset used in test_revision.py to be the vis_nights dataset to avoid singular matrix error.

You need to check what is hapenning with your PR @ryanvolpi. Apparently some problem on python setup command line. Click on details so you can check and fix that, PR needs to pass all CI checking before merging.

@ryanvolpi @vtoliveira sorry for being so late to this, but CI is fixed. Thanks both for contributing!

authors: carlomazzaferro ryanvolpi vtoliveira

repo: carlomazzaferro/scikit-hts


text:
Improper Tag Array in Unrelated Notes Breaking RPG Manager

I finally located this issue.
In one of my older notes, specifically for a "timeline" style note. I had a template with YAML tags that had a tag array in the form of:

tags: key 1, key 2, key 3
Obsidian itself works despite it not being a proper array, as shown in the following screenshot.

The array should be.

tags: [key 1, key 2, key 3]
Once corrected, then Obsidian changes to look like so.

I checked other variations of tags in YAML, with the exception of tags: #tag as it is hopefully well known to not use hashtags in YAML, but it may be worth to investigate that as well.
Attached also is the console log that is created when the error occurs.
obsidian.md-1662854035885.log

@sigrunixia writing the tags as
tags: #tag1, #tag2, #tag3

in the frontmatter is not a correct yaml. you should alway write them as
tags: [#tag1, #tag2, #tag3]

by fixing that, the issue will sort itself.
I will push a fix to catch this edge-case in a minute

Small fix released in v1.0.6
However, if the tags are not written in a correct yaml, RPG Manager cannot parse them :S

authors: carlonicora sigrunixia

repo: carlonicora/obsidian-rpg-manager


text:
Bitfinex support

added needed files. Some functions work. I hope for community help to finish it faster.

Completed.
Test:
[bfxbot@gene zenbot]$ ./zenbot.sh list-selectors

bitfinex:
  bitfinex.BCC-BTC   (BCC/BTC)
  bitfinex.BCC-USD   (BCC/USD)
  bitfinex.BCU-BTC   (BCU/BTC)
  bitfinex.BCU-USD   (BCU/USD)
  bitfinex.BTC-USD   (BTC/USD)
  bitfinex.DSH-BTC   (DSH/BTC)
  bitfinex.DSH-USD   (DSH/USD)
  bitfinex.ETC-BTC   (ETC/BTC)
  bitfinex.ETC-USD   (ETC/USD)
  bitfinex.ETH-BTC   (ETH/BTC)
  bitfinex.ETH-USD   (ETH/USD)
  bitfinex.LTC-BTC   (LTC/BTC)
  bitfinex.LTC-USD   (LTC/USD)
  bitfinex.RRT-BTC   (RRT/BTC)
  bitfinex.RRT-USD   (RRT/USD)
  bitfinex.XMR-BTC   (XMR/BTC)
  bitfinex.XMR-USD   (XMR/USD)
  bitfinex.XRP-BTC   (XRP/BTC)
  bitfinex.XRP-USD   (XRP/USD)
  bitfinex.ZEC-BTC   (ZEC/BTC)
  bitfinex.ZEC-USD   (ZEC/USD)

[bfxbot@gene zenbot]$ ./zenbot.sh backfill bitfinex.btc-usd
cannot backfill bitfinex.BTC-USD: exchange does not offer historical data

[bfxbot@gene zenbot]$ ./zenbot.sh sim bitfinex.btc-usd
2017-05-21 08:00:00    2069.20 USD              647                               0.00000 BTC  1000.00 USD    +0.0%   -1.1%
{ strategy: 'trend_ema',
  sell_stop_pct: 0,
  buy_stop_pct: 0,
  profit_stop_enable_pct: 0,
  profit_stop_pct: 1,
  max_slippage_pct: 5,
  buy_pct: 99,
  sell_pct: 99,
  order_adjust_time: 30000,
  max_sell_loss_pct: 25,
  markup_pct: 0,
  days: 90,
  currency_capital: 1000,
  asset_capital: 0,
  rsi_periods: 14,
  start: 1487548800000,
  stats: false,
  selector: 'bitfinex.BTC-USD',
  mode: 'sim',
  period: '1h',
  min_periods: 36,
  trend_ema: 34,
  buy_rate: 0,
  sell_rate: 0,
  max_buy_duration: 1,
  max_sell_duration: 1,
  oversold_rsi_periods: 14,
  oversold_rsi: 0 }
end balance 1000.00000000 (0.00%)
buy hold 1010.36168133 (1.04%)
vs. buy hold -1.03%
0 trades over 0 days (avg NaN trades/day)
wrote sim_result.html

[bfxbot@gene zenbot]$ ./zenbot.sh trade --paper  bitfinex.btc-usd
fetching pre-roll data:
cannot backfill bitfinex.BTC-USD: exchange does not offer historical data
               DATE            PRICE     DIFF            VOL         RSI                         ACTIONS                      BAL                PROFIT
2017-05-21 08:00:00    2069.20 USD              647
---------------------------- STARTING PAPER TRADING ----------------------------
2017-05-21 09:00:00    2067.10 USD   -0.1%      667                               0.00000 BTC  1000.00 USD    +0.0%   +0.1%
2017-05-21 09:01:17    2067.10 USD   +0.0%        2                               0.00000 BTC  1000.00 USD    +0.0%   +0.1%

[bfxbot@gene zenbot]$ ./zenbot.sh trade  bitfinex.btc-usd
fetching pre-roll data:
cannot backfill bitfinex.BTC-USD: exchange does not offer historical data
               DATE            PRICE     DIFF            VOL         RSI                         ACTIONS                      BAL                PROFIT
2017-05-21 08:00:00    2069.20 USD              647
2017-05-21 09:00:00    2067.10 USD   -0.1%      334
---------------------------- STARTING LIVE TRADING ----------------------------
BTC-USD BTCUSD
2017-05-21 09:03:25    2067.10 USD   +0.0%        5                               0.00000 BTC  72.96 USD    +0.0%   +0.0%

[bfxbot@gene zenbot]$ ./zenbot.sh buy --size 0.1 bitfinex.btc-usd

buy delayed: +99.3% of funds (72.44 USD) on hold
placing order...

nice :D I will try to test it soon. Note that recently I moved the extension confs into the main conf, thus the conflict.
For backfilling, I wonder if we can use https://cryptowat.ch/ for that.

I also moved conf to main, so is the write access conflict. Updated everything to the last version.

Now may buy and sell, though we have order status error:
/home/rebootfin/zenbot/commands/buy.js:48
              console.log('order status: '.grey + s.api_order.status.green + ', bid: '.grey + n(s.api_order.price).format('0.00000000').yellow + ', '.grey + n(quote.bid).subtract(s.api_order.pric
e).format('0.00000000').red + ' below best bid, '.grey + n(s.api_order.filled_size).divide(s.api_order.size).format('0.0%').green + ' filled'.grey)
                                                                    ^
TypeError: Cannot read property 'green' of undefined
    at /home/rebootfin/zenbot/commands/buy.js:48:69
    at /home/rebootfin/zenbot/extensions/bitfinex/exchange.js:106:9

moving to separate extension repo


text:
"Open in Editor" button .md file is missing response text at the end

What happened?
Hello, the generated .md file does not have all the response from the original response.

I'm using this because of #225
Relevant log output or stack trace
No response
Steps to reproduce

Start a new chat.
Ask for a long text 600 lines long.
Click on the "Open in Editor" button.
See the end of the .md

CodeGPT version
2.2.6
Operating System
macOS

Hmm, this issue might be related to the concurrency issues introduced following the chat tool window UI update.
The completion events that triggered set of UI updates were not running on the Event Dispatch Thread (EDT).
Please upgrade the plugin to v2.2.8, this version should be more stable.

Sorry mate, v2.2.8 does not fix it 😢

authors: carlrobertoh delmicio

repo: carlrobertoh/CodeGPT


text:
Fix resonant wavenumber definition

Aside from what seems like a simple typo ($\Omega v_\parallel$ instead of $\Omega / v_\parallel$), I think there's an issue with expression of $k_{res}$ in terms of $r_L$. In the text it was simply $r_L^{-1}$, which can't be the case since there's also a pitch angle dependence. But also in the lectures we wrote $\frac{1}{r_L \mu}$ (as per my and Luis's notes), which also seems to miss the $\sin \theta$ from the definition of Larmor radius.
So, here I tried to write down the correct definition, with $\tan$ originating from $v_\perp / v_\parallel$

Also, the tangent makes sense in the limit of small pitch angles -- it produces $k_{res} \to 0$, implying $\lambda \to \infty$, which corresponds to straight line particle trajectory.


text:
Extend AskForWebsite with supporting instructor role

Applied both for condition check and for recipients of the email.

@fmichonneau this PR makes two corrections in how AskForWebsite worked:

supporting instructors are now taken into account when checking if AskForWebsite can work (at least one instructor or supporting instructor must be assigned, to have someone to send the email to)
supporting instructors are now added to the context (under instructors like normal instructors). With this approach, you will not be able to distinguish normal instructors and supporting instructors in the email body.

Please confirm you agree with both points, or provide corrections otherwise.

Thanks Piotr, this is good!
I think it's not a problem to not be able to distinguish between instructors and supporting instructors for this particular email.

authors: fmichonneau pbanaszkiewicz

repo: carpentries/amy


text:
Schemas may not be named after primitives

Exception in thread "main" org.apache.avro.AvroTypeException: Schemas may not be named after primitives: string

The FHIR schema defines it's own types for some values that conflict with AVRO's primitives.
We need to sanitize record names.

boolean
int
long
float
double
bytes
string

https://avro.apache.org/docs/1.10.1/spec.html#schema_primitive

There are two parts to this problem.

Sanitize the record name itself.
Also up date any AvroRef so it continues to point to the correct thing.

repo: carreragroup/json-to-avro-schema


text:
Add TLB implementation to log off-chain and follow implementation on-chain

At the moment, the TLB is implemented as an off-chain optimization to machine-emulator, but not on-chain in machine-solidity-step. This works for conforming software that executes a fence.vma instruction whenever the virtual memory map is changed. This is the way to ensure deterministic behavior according to the RISC-V specification. However, an ill-formed or malicious program can mess with the virtual memory map and then fail to execute fence.vma, either by accident or on purpose. This would cause the TLB to be outdated relative to the virtual memory map, and thus lead to mismatching behavior on- and off-chain.
To be on the safe side and prevent this potential source of non-determinism, we have decided to have matching TLB implementations on- and off-chain. This will require changes to the machine-emulator implementation, so it becomes possible to efficiently implement the TLB on-chain. Then, the state accesses related to the TLB will have to be logged. Once these changes have been made, a matching TLB implementation will be added to the on-chain code in machine-solidity-step.
This issue will track the progress of these changes.
See cartesi/machine-emulator#21.

Since the microarchitecture, the TLB is implemented in the big emulator and there is no need to be implemented on-chain, therefore this not an issue anymore.

authors: diegonehab edubart

repo: cartesi/machine-solidity-step


text:
Check and block sensitive words in the topic and reply contents

The admin portal should support to add some sensitive words to the blacklist. If anyone wants to post a topic or reply, in which some words are in the blacklist, we should block them from posting.

Working on this.

authors: Kininaru hsluoyz

repo: casbin/casbin-forum


text:
Improve competition view page

From: https://conf.casbin.com/

To: https://www.kaggle.com/competitions/nfl-big-data-bowl-2023/overview/description

Could you please assign this to me since I would like to work on it?

I find an issue where we can improve, which is that we can add English content to this page.
Currently, the content is shown in Chinese, even though I have switched to English mode.
The following picture describes this:

@hsluoyz Do you think we need this feature? If so, I am available to work on this.

@ZhengLin-Li  what you said is already done in: https://github.com/casbin/confita/pull/41 , in order not to diverse the original issue, these two comments will be deleted. For new issue please create new issue

authors: ZhengLin-Li hsluoyz raghavpatidar

repo: casbin/confita