text:
Ryuk container name already in use

The error message from docker is:

Conflict. The container name "/testcontainers-ryuk-181a7bc5-71b4-41c5-89ee-b5f0b0f378ed" is already in use by container "7b89e6644ce929b67b9934dd1caf2b414f98e7143490b20d2cdcc36645c60830". You have to remove (or rename) that container to be able to reuse that name.

Based on looking through the code, you are creating a static SESSION_ID in the DockerClientFactory but you are creating a Ryuk container per docker client. There isn't a check for an existing container with that id.
I'm not sure what the correct solution is. The 2 options I see are:

have a per-client session id
handle the container already existing by checking and returning the id

We are using version 1.12.5
This happens during CI builds using docker-in-docker on BuildKite. EC2 hosts running Amazon Linux. Does not happen locally on macOS or on linux laptops.

Also affected by this.

Affected by this using 1.13.0

My tests worked fine on 1.12.4, but start to fail 1.13.0 with this error.
Also running docker-in-docker.

For information, we've got the same issue with an ElasticSearch container. Worked great with version 1.10.6 but ran into this bug when we upgraded to 1.13.0

The issue seems to be fixed in 1.14.0

I am facing the same issue on version 1.14.0.
Tests work fine on local machine, it just fails on Jenkins with
Caused by: com.github.dockerjava.api.exception.ConflictException: {"message":"Conflict. The container name "/testcontainers-ryuk-ea491299-e6f7-432e-b5f2-cb4cda1c38f5" is already in use by container "99297d75a66c387adc1f29e2e40f0224bbf75587e6fcb72fff4c2e0b94027c94". You have to remove (or rename) that container to be able to reuse that name."}

For everyone who is getting the issue:
Please report your test setup. What containers are you using, how do you start them, etc etc

@bsideup
In my tests, I use my 2 custom containers which are built using fabric8 docker-maven-plugin at maven package phase. Tests run at maven verify
Test Configuration

Single test class
Using junit5 for running tests: v5.6.2
testcontainers version : 1.14.0
I have my custom 2 images, lets call imageA and imageB. Lifecycle of these containers is managed by testcontainers by using @Container annotation
The whole cascading exception is about 100 lines or so, and is availble at testcontainersException

Please let me know if anything more is required

Can others confirm that they use JUnit 5 and/or Startables.deepStart?

i'm using junit5 as well. not the extension that ships with the library, we wrote out own, but it is junit5.

Update:
I was using testcontainers inside Jenkins container
Changing the Jenkins container network mode from bridge to host and mapping the jnekins directory in same location as it was inside by following testcontainers:Docker in docker solved this issue

@bsideup I recently hit this error and we are using JUnit 4 and are not using Startables.deepStart.
Here is how the container is defined:
    @ClassRule
    public static CouchDBContainer couchdb = new CouchDBContainer(new ImageFromDockerfile()
                    .withDockerfileFromBuilder(builder -> builder.from("couchdb:1.7")
                                    .copy("/opt/couchdb/etc/local.d/testcontainers_config.ini", "/opt/couchdb/etc/local.d/testcontainers_config.ini")
                                    .copy("/etc/couchdb/cert/couchdb.pem", "/etc/couchdb/cert/couchdb.pem")
                                    .copy("/etc/couchdb/cert/privkey.pem", "/etc/couchdb/cert/privkey.pem")
                                    .build())
                    .withFileFromFile("/opt/couchdb/etc/local.d/testcontainers_config.ini", new File("lib/LibertyFATTestFiles/couchdb-config/testcontainers_config.ini"), 644)
                    .withFileFromFile("/etc/couchdb/cert/couchdb.pem", new File("lib/LibertyFATTestFiles/ssl-certs/couchdb.pem"), 644)
                    .withFileFromFile("/etc/couchdb/cert/privkey.pem", new File("lib/LibertyFATTestFiles/ssl-certs/privkey.pem"), 644))
                                    .withLogConsumer(FATSuite::log);

Here is a link to the src with additional context:
https://github.com/OpenLiberty/open-liberty/blob/master/dev/com.ibm.ws.couchdb_fat/fat/src/com/ibm/ws/couchdb/fat/FATSuite.java#L37
And here is the complete stack trace (notice that it is all stemming from a logging statement in GenericContainer.doStart())
org.testcontainers.containers.ContainerLaunchException: Container startup failed
at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:320)
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:300)
at org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:1011)
at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29)
Caused by: com.github.dockerjava.api.exception.ConflictException: {"message":"Conflict. The container name \"/testcontainers-ryuk-efaa6546-d512-42f6-aaea-912af1ec2d81\" is already in use by container \"80e8a01c729781819e8a072fc6b810fe829fb3c33eb91b891e7727ed2145fd23\". You have to remove (or rename) that container to be able to reuse that name."}
at com.github.dockerjava.okhttp.OkHttpInvocationBuilder.execute(OkHttpInvocationBuilder.java:291)
at com.github.dockerjava.okhttp.OkHttpInvocationBuilder.execute(OkHttpInvocationBuilder.java:271)
at com.github.dockerjava.okhttp.OkHttpInvocationBuilder.post(OkHttpInvocationBuilder.java:129)
at com.github.dockerjava.core.exec.CreateContainerCmdExec.execute(CreateContainerCmdExec.java:33)
at com.github.dockerjava.core.exec.CreateContainerCmdExec.execute(CreateContainerCmdExec.java:13)
at com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35)
at com.github.dockerjava.core.command.CreateContainerCmdImpl.exec(CreateContainerCmdImpl.java:595)
at org.testcontainers.utility.ResourceReaper.start(ResourceReaper.java:91)
at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:155)
at org.testcontainers.images.builder.ImageFromDockerfile.resolve(ImageFromDockerfile.java:84)
at org.testcontainers.images.builder.ImageFromDockerfile.resolve(ImageFromDockerfile.java:37)
at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:20)
at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:27)
at org.testcontainers.shaded.com.google.common.util.concurrent.Futures$3.get(Futures.java:1332)
at org.testcontainers.images.RemoteDockerImage.getImageName(RemoteDockerImage.java:97)
at org.testcontainers.images.RemoteDockerImage.imageNameToString(RemoteDockerImage.java:107)
at org.testcontainers.images.RemoteDockerImage.toString(RemoteDockerImage.java:26)
at java.base/java.lang.String.valueOf(String.java:2951)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1268)
at org.testcontainers.containers.GenericContainer.logger(GenericContainer.java:603)
at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:309)

@bsideup so I guess my findings are sort of along the same lines as Startables.deepStart, because we have 2 things trying to potentially issue the docker image name at the same time?
I think that if we simply removed that log statement it would fix the issue, but maybe the code needs a more thorough review for getDockerImageName() kicking off a significant amount of work for the ImageFromDockerfile usage pattern, or perhaps we could offer an alternative to getDockerImageName() that can be easily used to identify the docker image name without flowing any docker commands?

@aguibert FYI I just submitted https://github.com/testcontainers/testcontainers-java/pull/2930 that should fix eager resolve of LazyFuture from RemoteDockerImage#toString (that is triggered from GenericContainer#toString, for example).
It is unrelated to the Ryuk problem but seems to be a miss from #2558
BTW from the stacktrace, it looks like you're using a version that is older that 1.14.x, consider updating.

hmm, we're using 1.14.0, not the latest, but it is 1.14.X

Okay, after looking at the code, I noticed that we were not caching Ryuk's failure: #2935
So, the actual problem was that ResourceReaper was failing to connect to started Ryuk container, but another attempt at getting a client would not know that and attempt to start it again.
This logs confirms that:
https://pastebin.com/pZHXKzRF

Caused by: java.lang.IllegalStateException: Can not connect to Ryuk

great find @bsideup, I think this issue can be closed now that #2935 is in right? Or do you want to wait for users to verify?

Desperately awaiting a release version with these fixes included (Jenkins here).

@enote-kane well, what was fixed won't fix your Jenkins issue. There is still an issue with "failed to connect to Ryuk" on your environment, just now it will be clear (before, it was hidden behind "container name already in use" because it was trying to make another attempt at starting Ryuk but it was already running (but failed to connect to it)

@bsideup Oh, that's bad. I was wondering why this is the case only on our EC2 Jenkins slaves running Ubuntu whereas we also have other Ubuntu clients where we where unable to reproduce this issue at all.
Is there another bug for the connect problem?

@enote-kane there are no known bugs with Ryuk. Please check your network settings, firewall, debug connectivity, etc etc.

@bsideup Hm, I am confused as I wasn't able to spot any connectivity issues in my log output, but will try to have more debug logging.

OK, after looking again, I think you mean this, right?
21:48:55.030 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd: c08828298fdc19f416eada33b0d6a3b844127853f12cdfca7e2e6667cb00ee24
21:48:55.532 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd: c08828298fdc19f416eada33b0d6a3b844127853f12cdfca7e2e6667cb00ee24,false
21:48:55.533 [main] DEBUG com.github.dockerjava.core.exec.InspectContainerCmdExec - GET: com.github.dockerjava.okhttp.OkHttpWebTarget@13006998
21:49:25.629 [main] ERROR org.testcontainers.utility.ResourceReaper - Timeout out waiting for Ryuk. Ryuk's log: <<<
2020/07/09 21:48:55 Pinging Docker...
2020/07/09 21:48:55 Docker daemon is available!
2020/07/09 21:48:55 Starting on port 8080...
2020/07/09 21:48:55 Started!

But I don't get why this should be different when run by a Jenkins pipeline than any such other environment (docker-in-docker).
The actual command that is being executed is like this:
docker run -t -d -u 1001:1001 --network=bridge \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -u 1001:999 # jenkins user ID + docker group ID \
    -w /opt/jenkins/workspace/... \
    -v /opt/jenkins/workspace/...:/opt/jenkins/workspace/...:rw,z \
    -v /opt/jenkins/workspace/...@tmp:/opt/jenkins/workspace/...@tmp:rw,z \
    -e ... \
    maven:3-openjdk-14 \
    mvn --batch-mode --strict-checksums test -DtrimStackTrace=false

So the only thing I could think of is the standard bridge network that get's in the way here, right?

Please reopen. I am having this issue on a very uncomplicated "hello world" type of project, rendering the TestContainers library unusable at the moment.

Apr 26, 2021 3:06:41 PM org.junit.jupiter.engine.execution.JupiterEngineExecutionContext close
SEVERE: Caught exception while closing extension context: org.junit.jupiter.engine.descriptor.ClassExtensionContext@d72284c
com.github.dockerjava.api.exception.ConflictException: Status 409: {"message":"Conflict. The container name "/testcontainers-ryuk-f3039b1f-d175-4c4f-981b-aca78f782347" is already in use by container "882f294ea3178cf5d254e78f782e73ee8bd53430d4388b5ecaffab3a9e81ae5b". You have to remove (or rename) that container to be able to reuse that name."}

My Gradle build file:

largeTestImplementation 'org.testcontainers:junit-jupiter:1.15.3'
largeTestImplementation 'org.testcontainers:postgresql:1.15.3'

My machine is a Hyper-V virtual machine (Windows 10 x64) running Docker (20.10.5) on WSL2 (which technically becomes a nested VM). WSL2's distro is Ubuntu 20.04.

authors: Leonti aguibert bsideup enote-kane kmanco martinandersson r-sniper stephanebastian toadzky vikrant82

repo: testcontainers/testcontainers-java


text:
Test cases are being skipped when running in parallel a factory class containing dependent methods

TestNG Version
7.0.0
Expected behavior
All the test case cases should be executed.
Actual behavior
Only the first test case is executed by both threads and then another test class setup is done.
Is the issue reproducible on runner?

[ ] Shell
[ ] Maven
[ ] Gradle
[ ] Ant
[x] Eclipse
[ ] IntelliJ
[ ] NetBeans

Note:
In our case, it is important to keep setGroupByInstances(true) and setPreserveOrder(false)
Test case sample
package tests.testCases;

import java.util.ArrayList;

import org.testng.TestNG;
import org.testng.xml.XmlClass;
import org.testng.xml.XmlSuite;
import org.testng.xml.XmlTest;

public class main
{
    public static void main(String[] args) throws ClassNotFoundException
    {
        XmlSuite suite;
        XmlTest test;
        TestNG testNG;
        XmlClass xmlClass;
        ArrayList<XmlTest> tests;
        ArrayList<XmlClass> testClasses;
        ArrayList<XmlSuite> testSuites;

        testNG = new TestNG();
        testNG.setVerbose(0);

        testSuites = new ArrayList<XmlSuite>();
        suite = new XmlSuite();
        suite.setName("Suite");

        test = new XmlTest();
        test.setName("testCases");
        test.setSuite(suite);
        test.setGroupByInstances(true);
        test.setPreserveOrder(false);

        testClasses = new ArrayList<XmlClass>();
        xmlClass = new XmlClass(Class.forName("tests.testCases.TestCaseB"));
        testClasses.add(xmlClass);
        test.setClasses(testClasses);

        tests = new ArrayList<XmlTest>();
        tests.add(test);

        suite.setTests(tests);
        suite.setGroupByInstances(true);
        suite.setPreserveOrder(false);
        suite.setParallel(XmlSuite.ParallelMode.INSTANCES);
        suite.setThreadCount(2);

        testSuites.add(suite);

        testNG.setXmlSuites(testSuites);

        testNG.run();
    }
}

Test case sample
package tests.testCases;

import org.testng.ITestContext;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;

public class TestCaseA
{
    @BeforeClass (alwaysRun=true)
    protected void setUpTestClass(ITestContext context)
    {
        System.out.println("setUpTestClass with thread id " + Thread.currentThread().getId());
    }

    @AfterClass (alwaysRun=true)
    protected void tearDownTestClass()
    {
        System.out.println("tearDownTestClass with thread id " + Thread.currentThread().getId());
    }
}

Test case sample
package tests.testCases;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;

public class TestCaseB extends TestCaseA
{
    @DataProvider(name = "data")
    private static Object[] createDataProvider()
    {
        return new Object[][] {{"b"},
        {"a"},
        {"b"},
        {"c"}};
    }

    @Factory(dataProvider = "data")
    public TestCaseB(String a)
    {
        System.out.println("Inside constructor with " + a + " parmeter and thread id "  + Thread.currentThread().getId());
    }

    @Test
    public void testA1() throws Exception
    {
        System.out.println("testA1 " + Thread.currentThread().getId());
    }

    @Test(dependsOnMethods = "testA1")
    public void testA2() throws Exception
    {
        System.out.println("testA2  with thread id " + Thread.currentThread().getId());
    }

    @Test(dependsOnMethods = "testA1")
    public void testB1() throws Exception
    {
        System.out.println("testB1  with thread id" + Thread.currentThread().getId());
    }
}

@galinaff - I retried this with TestNG 7.9.0 with a bit of tweaks to your sample so that the output is readable. I am not able to reproduce the problem. It works fine. I will be closing this. If this is still a problem with 7.9.0 please comment and we can re-open the issue.
Here's how the altered code looks like:
import org.testng.ITestContext;
import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;

import java.util.LinkedHashSet;
import java.util.Set;

public class TestCaseA {

    @BeforeClass(alwaysRun = true)
    protected void setUpTestClass(ITestContext context) {
        log();
    }

    @AfterClass(alwaysRun = true)
    protected void tearDownTestClass() {
        log();
    }

    protected void log() {
        ITestResult itr = Reporter.getCurrentTestResult();
        String instance = toString();
        long threadId = Thread.currentThread().getId();
        String text = String.format("Executing method [%s] on instance [%s] with thread id [%d]",
                itr.getMethod().getMethodName(), instance, threadId);
        Set<String> set = TestMain.logs.computeIfAbsent(instance, key -> new LinkedHashSet<>());
        synchronized (set) {
            set.add(text);
        }
    }
}

import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;

public class TestCaseB extends TestCaseA {

    private final String text;

    @DataProvider(name = "data")
    private static Object[] createDataProvider() {
        return new Object[][]{{"b"},
                {"a"},
                {"b"},
                {"c"}};
    }

    @Factory(dataProvider = "data")
    public TestCaseB(String a) {
        this.text = a;
    }

    @Test
    public void testA1() {
        log();
    }

    @Test(dependsOnMethods = "testA1")
    public void testA2() {
        log();
    }

    @Test(dependsOnMethods = "testA1")
    public void testB1() {
        log();
    }

    @Override
    public String toString() {
        return "Instance[" + this.text + "]";
    }
}

import java.util.Collections;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;

import org.testng.TestNG;
import org.testng.xml.XmlClass;
import org.testng.xml.XmlSuite;
import org.testng.xml.XmlTest;

public class TestMain {
    public static final Map<String, Set<String>> logs = new ConcurrentHashMap<>();

    public static void main(String[] args) throws ClassNotFoundException {

        TestNG testNG = new TestNG();
        testNG.setVerbose(2);
        XmlSuite suite = newSuite();
        testNG.setXmlSuites(Collections.singletonList(suite));
        testNG.run();
        for (Map.Entry<String, Set<String>> each : logs.entrySet()) {
            System.err.println("Printing logs for instance " + each.getKey());
            each.getValue().forEach(System.err::println);
        }
    }

    private static XmlSuite newSuite() {
        XmlSuite suite = new XmlSuite();
        suite.setName("Suite");
        suite.setGroupByInstances(true);
        suite.setPreserveOrder(false);
        suite.setParallel(XmlSuite.ParallelMode.INSTANCES);
        suite.setThreadCount(2);
        addTestToSuite(suite);
        return suite;
    }

    private static void addTestToSuite(XmlSuite suite) {
        XmlTest test = new XmlTest();
        test.setName("testCases");
        test.setSuite(suite);
        test.setGroupByInstances(true);
        test.setPreserveOrder(false);
        test.setClasses(Collections.singletonList(new XmlClass(TestCaseB.class)));
        suite.setTests(Collections.singletonList(test));
    }
}

Execution output:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testA1
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testA1
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testB1
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testB1
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testA1
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testA2
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testA2
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testB1
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testA2
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testA2
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testA1
PASSED: com.rationaleemotions.github.issue3105.TestCaseB.testB1

===============================================
    testCases
    Tests run: 12, Failures: 0, Skips: 0
===============================================

===============================================
Suite
Total tests run: 12, Passes: 12, Failures: 0, Skips: 0
===============================================

Printing logs for instance Instance[a]
Executing method [setUpTestClass] on instance [Instance[a]] with thread id [26]
Executing method [testA1] on instance [Instance[a]] with thread id [26]
Executing method [testA2] on instance [Instance[a]] with thread id [26]
Executing method [testB1] on instance [Instance[a]] with thread id [26]
Executing method [tearDownTestClass] on instance [Instance[a]] with thread id [26]
Printing logs for instance Instance[b]
Executing method [setUpTestClass] on instance [Instance[b]] with thread id [27]
Executing method [testA1] on instance [Instance[b]] with thread id [27]
Executing method [setUpTestClass] on instance [Instance[b]] with thread id [26]
Executing method [testA1] on instance [Instance[b]] with thread id [26]
Executing method [testA2] on instance [Instance[b]] with thread id [27]
Executing method [testB1] on instance [Instance[b]] with thread id [27]
Executing method [tearDownTestClass] on instance [Instance[b]] with thread id [27]
Executing method [testA2] on instance [Instance[b]] with thread id [26]
Executing method [testB1] on instance [Instance[b]] with thread id [26]
Executing method [tearDownTestClass] on instance [Instance[b]] with thread id [26]
Printing logs for instance Instance[c]
Executing method [setUpTestClass] on instance [Instance[c]] with thread id [27]
Executing method [testA1] on instance [Instance[c]] with thread id [27]
Executing method [testA2] on instance [Instance[c]] with thread id [27]
Executing method [testB1] on instance [Instance[c]] with thread id [27]
Executing method [tearDownTestClass] on instance [Instance[c]] with thread id [27]

Process finished with exit code 0

Thank you very much for your help!
Your example is working in the last version but still my example with System.out.println() still doesn't work. We still see the that thread continues to another class instance setup without finishing running the all the dependent methods and the after class.

Thank you!
We refactored the code in the following way, using System.out.println() or Reporter.log(), receiving the same result as you can see in the output. The main is the same for both examples.
Main
package tests.testCases;
import java.util.ArrayList;
import org.testng.TestNG;
import org.testng.xml.XmlClass;
import org.testng.xml.XmlSuite;
import org.testng.xml.XmlTest;
public class main
{
public static void main(String[] args) throws ClassNotFoundException
{
XmlSuite suite;
XmlTest test;
TestNG testNG;
XmlClass xmlClass;
ArrayList tests;
ArrayList testClasses;
ArrayList testSuites;
    testNG = new TestNG();
    testNG.setVerbose(0);

    testSuites = new ArrayList<XmlSuite>();
    suite = new XmlSuite();
    suite.setName("Suite");

    test = new XmlTest();
    test.setName("testCases");
    test.setSuite(suite);
    test.setGroupByInstances(true);
    test.setPreserveOrder(false);

    testClasses = new ArrayList<XmlClass>();
    xmlClass = new XmlClass(Class.forName("tests.testCases.TestCaseB"));
    testClasses.add(xmlClass);
    test.setClasses(testClasses);

    tests = new ArrayList<XmlTest>();
    tests.add(test);

    suite.setTests(tests);
   suite.setGroupByInstances(true);
    suite.setPreserveOrder(false);
    suite.setParallel(XmlSuite.ParallelMode.INSTANCES);
    suite.setThreadCount(2);

    testSuites.add(suite);

    testNG.setXmlSuites(testSuites);

    testNG.run();
}

}
1st test case sample
package tests.testCases;
import org.testng.ITestContext;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
public class TestCaseA
{
protected String text;
@BeforeClass (alwaysRun = true)
public void setUpTestClass(ITestContext context)
{
    System.out.println(String.format("Executing method [setUpTestClass] on " +
                                      "instance [%s] with thread id [%d]",
                                      this.text,
                                      Thread.currentThread().getId()));
}

@AfterClass (alwaysRun = true)
public void tearDownTestClass()
{
    System.out.println(String.format("Executing method [tearDownTestClass] on " +
                                     "instance [%s] with thread id [%d]",
                                     this.text,
                                     Thread.currentThread().getId()));
}

}
package tests.testCases;
import org.testng.ITestContext;
import org.testng.Reporter;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
public class TestCaseA
{
protected String text;
@BeforeClass (alwaysRun = true)
public void setUpTestClass(ITestContext context)
{
    Reporter.log(String.format("Executing method [setUpTestClass] on instance " +
                               "[%s] with thread id [%d]",
                               this.text,
                               Thread.currentThread().getId()),
                 true);
}

@AfterClass (alwaysRun = true)
public void tearDownTestClass()
{
    Reporter.log(String.format("Executing method [tearDownTestClass] on instance " +
                               "[%s] with thread id [%d]",
                               this.text,
                               Thread.currentThread().getId()),
                 true);
}

}
2nd test case example
package tests.testCases;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;
public class TestCaseB extends TestCaseA
{
@DataProvider(name = "data")
private static Object[] createDataProvider()
{
return new Object[][] {
{"a"},
{"b"},
{"c"},
{"d"}};
}
@Factory(dataProvider = "data")
public TestCaseB(String a)
{
    this.text = a;
}

@Test
public void testA1() throws Exception
{
    System.out.println(String.format("Executing method [testA1] on instance " +
                                     "[%s] with thread id [%d]",
                                     this.text,
                                     Thread.currentThread().getId()));
}

@Test(dependsOnMethods = "testA1")
public void testA2() throws Exception
{

    System.out.println(String.format("Executing method [testA2] on instance " +
                                     "[%s] with thread id [%d]",
                                     this.text,
                                     Thread.currentThread().getId()));
}

@Test(dependsOnMethods = "testA1")
public void testB1() throws Exception
{

    System.out.println(String.format("Executing method [testB1] on instance " +
                                     "[%s] with thread id [%d]",
                                     this.text,
                                     Thread.currentThread().getId()));
}

}
package tests.testCases;
import org.testng.Reporter;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;
public class TestCaseB extends TestCaseA
{
@DataProvider(name = "data")
private static Object[] createDataProvider()
{
return new Object[][] {
{"a"},
{"b"},
{"c"},
{"d"}};
}
@Factory(dataProvider = "data")
public TestCaseB(String a)
{
    this.text = a;
}

@Test
public void testA1() throws Exception
{
    Reporter.log(String.format("Executing method [testA1] on instance " +
                                     "[%s] with thread id [%d]",
                                     this.text,
                                     Thread.currentThread().getId()),
                 true);
}

@Test(dependsOnMethods = "testA1")
public void testA2() throws Exception
{

    Reporter.log(String.format("Executing method [testA2] on instance [%s] with " +
                               "thread id [%d]",
                               this.text,
                               Thread.currentThread().getId()),
                 true);
}

@Test(dependsOnMethods = "testA1")
public void testB1() throws Exception
{

    Reporter.log(String.format("Executing method [testB1] on instance [%s] with " +
                               "thread id [%d]",
                               this.text,
                               Thread.currentThread().getId()) ,
                 true);
}

}
The output for both cases is similar:
[12:23] Valeri Miller
Executing method [setUpTestClass] on instance [d] with thread id [18]
Executing method [setUpTestClass] on instance [a] with thread id [19]
Executing method [testA1] on instance [a] with thread id [19]
Executing method [testA1] on instance [d] with thread id [18]
Executing method [setUpTestClass] on instance [b] with thread id [19]
Executing method [testA1] on instance [b] with thread id [19]
Executing method [setUpTestClass] on instance [c] with thread id [18]
Executing method [testA1] on instance [c] with thread id [18]
Executing method [testA2] on instance [a] with thread id [19]
Executing method [testB1] on instance [a] with thread id [19]
Executing method [testA2] on instance [d] with thread id [18]
Executing method [testB1] on instance [d] with thread id [18]
Executing method [tearDownTestClass] on instance [d] with thread id [18]
Executing method [testA2] on instance [c] with thread id [18]
Executing method [testB1] on instance [c] with thread id [18]
Executing method [tearDownTestClass] on instance [c] with thread id [18]
Executing method [tearDownTestClass] on instance [a] with thread id [19]
Executing method [testA2] on instance [b] with thread id [19]
Executing method [testB1] on instance [b] with thread id [19]
Executing method [tearDownTestClass] on instance [b] with thread id [19]

@galinaff - I am kind of confused. So does that mean that the issue no longer exists or am I missing something?

@galinaff

From the output we can see that the threads continue to another setup test class instance without finishing to run all class's methods (A2, B1 and tearDownTestClass).

You have set suite.setParallel(XmlSuite.ParallelMode.INSTANCES); So the instances will run in parallel. AFAIK, that's the behaviour.
Are you expecting that TestNG should do the following?

Create an instance of the test class via the factory
run its configuration and test
proceed with the next instance

If that's what you are expecting then you should be removing the parallel attribute and just stick to group-by-instances=true
Your test class also has depends which means that TestNG will segregate your execution into independent and dependent methods. TestNG will proceed with running your independent methods first, before it comes back to running the dependent methods.
Also can you please help with the following:

Point me to the documentation wherein this expectation is being called out. I ask because I dont remember this mentioned in the documentation.
Has this ever worked for you in any of the previous released testng versions?
Please help refactor the test case, to include assertions to match with what you have as expectations. Print statements don't help a lot to be honest and it becomes a bit more confusing in terms of ensuring that the desired behaviour is achieved.

@galinaff

Create an instance of the test class via the factory
Run its configuration and tests
Proceed with the next instance

In that case, you should be running with a thread pool size of 1 because the moment you say parallel, TestNG will try to run things in parallel. In this case, your parallelism strategy is mentioned as instances, so TestNG will try to run the testng methods (configuration and test both) for each of the instances in parallel.

Do you mean that these segregated methods will be executed in different class instances/different threads, no matter that they are contained in one class?

No. TestNG will always associate the corresponding instance to the test/configuration method to which it belongs to. I believe that my earlier output should re-iterate this. But thread affinity is NOT guaranteed because, TestNG internally works with creating a Directed acyclic graph wherein, the independent methods are first found followed by the dependent ones (This is the same technique that even build tools also use, when it comes to figuring out what to run first and what to run later).
TestNG merely takes the responsibility of submitting task execution to a thread pool executor. After that the thread pool executor decides what to run on which thread based on the tasks that arrive via its linked blocking queue.

Because my expectation is that the independent methods will be executed first, then the dependent ones but all this in the scope of one class instance execution.

Yes, within the scope of class instantiation is true, but if you look at it, you have 2 threads, and you have 4 instances wherein there are configuration and also test methods that can be run.
So first the configuration methods will be run on each of the 2 threads (2 before class methods)
after that, we have 4 * 1 independent test methods (tests with no depends on) and so they will be run.
After that, the dependent methods in the context of its upstream dependency based on instance will get executed.
If you strictly want to go instance by instance execution, then you should reduce your thread pool size to 1 (or) disable parallel execution and just use group-by-instances to true (Oh btw, group-by-instances and parallel execution negate each other. So TestNG will NOT honour group-by-instances if you have turned on parallel execution)
I hope that clarifies how TestNG works.

I understand, some changes in the code will have to be done then. Thank you very much for your explanation!

source: gharchive/issue

authors: galinaff krmahadevan

repo: testng-team/testng

license_type: permissive

license_source: github-api


text:
dwarf: ignores tombstone entries in inlined information

We deleted the DWARF test case from TinyGo in https://github.com/tetratelabs/wazero/pull/1120#discussion_r1103983115,
but I found there's actually a bug in our dwarf handling which can be covered by the deleted case.
This revives the TinyGo test case, and adds handling of tombstone addresses so that we could
emit the correct inlined information.
After
wazero run main.wasm
panic: NOOOOOOOOOOOOOOO
error instantiating wasm binary: module[] function[_start] failed: wasm error: unreachable
wasm stack trace:
        .runtime._panic(i32)
                0x18f3: /Users/mathetake/Downloads/tinygo/src/runtime/runtime_tinygowasm.go:70:6
        .main.c()
                0x2ff9: /Users/mathetake/Downloads/tinygo/tmo/main.go:16:7
        .main.b()
                0x2f97: /Users/mathetake/Downloads/tinygo/tmo/main.go:12:3
        .main.a()
                0x2f39: /Users/mathetake/Downloads/tinygo/tmo/main.go:8:3
        .main.main()
                0x2149: /Users/mathetake/Downloads/tinygo/tmo/main.go:4:3
        .runtime.run$1()
                0x1fcb: /Users/mathetake/Downloads/tinygo/src/runtime/scheduler_any.go:25:11
        .runtime.run$1$gowrapper(i32)
                0x6f0: /Users/mathetake/Downloads/tinygo/src/runtime/scheduler_any.go:23:2
        .tinygo_launch(i32)
                0x23: /Users/mathetake/Downloads/tinygo/src/internal/task/task_asyncify_wasm.S:59
        .runtime.scheduler()
                0x1ec4: /Users/mathetake/Downloads/tinygo/src/internal/task/task_asyncify.go:109:17 (inlined)
                        /Users/mathetake/Downloads/tinygo/src/runtime/scheduler.go:236:11
        .runtime.run()
                0x1d92: /Users/mathetake/Downloads/tinygo/src/runtime/scheduler_any.go:28:11
        ._start()
                0x1d12: /Users/mathetake/Downloads/tinygo/src/runtime/runtime_wasm_wasi.go:21:5

Before
This incorrectly includes the tombstone addresses multiple times.

panic: NOOOOOOOOOOOOOOO
error instantiating wasm binary: module[] function[_start] failed: wasm error: unreachable
wasm stack trace:
        .runtime._panic(i32)
                0x18f3: /Users/mathetake/Downloads/tinygo/src/runtime/runtime_tinygowasm.go:70:6 (inlined)
                        /Users/mathetake/Downloads/tinygo/src/runtime/panic.go:71:7
        .main.c()
                0x2ff9: /Users/mathetake/Downloads/tinygo/tmo/main.go:16:7 (inlined)
        .main.b()
                0x2f97: /Users/mathetake/Downloads/tinygo/tmo/main.go:12:3 (inlined)
        .main.a()
                0x2f39: /Users/mathetake/Downloads/tinygo/tmo/main.go:8:3 (inlined)
        .main.main()
                0x2149: /Users/mathetake/Downloads/tinygo/tmo/main.go:4:3 (inlined)
        .runtime.run$1()
                0x1fcb: /Users/mathetake/Downloads/tinygo/src/runtime/scheduler_any.go:25:11 (inlined)
                        /Users/mathetake/Downloads/tinygo/src/runtime/panic.go:71:7
        .runtime.run$1$gowrapper(i32)
                0x6f0: /Users/mathetake/Downloads/tinygo/src/runtime/scheduler_any.go:23:2 (inlined)
                       /Users/mathetake/Downloads/tinygo/src/runtime/gc_stack_portable.go:46:21
        .tinygo_launch(i32)
                0x23: /Users/mathetake/Downloads/tinygo/src/internal/task/task_asyncify_wasm.S:59
        .runtime.scheduler()
                0x1ec4: /Users/mathetake/Downloads/tinygo/src/internal/task/task_asyncify.go:109:17 (inlined)
                        /Users/mathetake/Downloads/tinygo/src/runtime/panic.go:71:7
        .runtime.run()
                0x1d92: /Users/mathetake/Downloads/tinygo/src/runtime/scheduler_any.go:28:11 (inlined)
                        /Users/mathetake/Downloads/tinygo/src/runtime/panic.go:71:7
        ._start()
                0x1d12: /Users/mathetake/Downloads/tinygo/src/runtime/runtime_wasm_wasi.go:21:5 (inlined)
                        /Users/mathetake/Downloads/tinygo/src/runtime/panic.go:71:7

JFYI  @pelletier maybe this might be interesting to you

source: gharchive/pull-request

repo: tetratelabs/wazero


text:
STLINk in error state does not abort the program.

After changing a few things I suddenly decided to try to test "no device attached". I detached the target ground pin and I think this sent the STLINK into an error state.
When the STLINK is in error state, starting up as normal is not the right thing to do.
assurancetourix:~> ~/stm32/stlink.git2/build/Release/src/gdbserver/st-util -m
st-util 1.6.0-33-gb8813fa-dirty
[!] send_recv send request failed: LIBUSB_ERROR_IO
[!] send_recv STLINK_GET_VERSION
[!] send_recv send request failed: LIBUSB_ERROR_IO
[!] send_recv STLINK_GET_CURRENT_MODE
[!] send_recv send request failed: LIBUSB_ERROR_IO
[!] send_recv STLINK_GET_CURRENT_MODE
[!] send_recv send request failed: LIBUSB_ERROR_IO
[!] send_recv STLINK_DEBUG_ENTER
[!] send_recv send request failed: LIBUSB_ERROR_IO
[!] send_recv STLINK_JTAG_DRIVE_NRST
[!] send_recv send request failed: LIBUSB_ERROR_IO
[!] send_recv STLINK_DEBUG_RESETSYS
[!] send_recv send request failed: LIBUSB_ERROR_IO
[!] send_recv STLINK_DEBUG_READCOREID
2020-03-21T17:10:40 ERROR common.c: Failed to read core_id
[!] send_recv send request failed: LIBUSB_ERROR_IO
[!] send_recv STLINK_JTAG_READDEBUG_32BIT
2020-03-21T17:10:40 WARN common.c: Invalid flash type, please check device declaration
[!] send_recv send request failed: LIBUSB_ERROR_IO
[!] send_recv STLINK_DEBUG_RESETSYS
2020-03-21T17:10:40 INFO gdb-server.c: Listening at *:4242...

Fixed by pull #890

Hey Nightwalker, I might seem to be able to fix a thing or two, but this one is out of my league. I can program Unix-programs just fine, and I can write embedded stuff for AVR and STM, but interacting with an STLINK is not my specialty.
The previous things I fixed were simple "just search for where the message comes from" and change that. But this one would be nice-to-fix if someone who knows the STLINK hardware and could take a look.
Between the errors of an older version and the latest git... it looks as if any libusb interaction with the STLINK is met with an error: The kernel knows the device is there, but the device is no longer responding through USB. That would make it impossible to fix.  That's what I think is the case, but it would be a good idea not to trust me and confirm with someone who knows more than me. (about this).

repo: texane/stlink


text:
Refactor: Support a Local-First API as a first-class feature

This is an outcome of the "great refactor" (#414). This Issues is to flag the required documentation updates, external API changes, and outreach (blog posts) plans that will be required to make this new and exciting change more visible.

See work on-going here: https://github.com/textileio/js-threaddb to be merged into this repo when out of alpha release phase.

repo: textileio/js-threads

license: MIT


text:
Provide CID output in plain text, not JSON as pow ffs stage does

In an attempt to automate pow we capture the output of the stage command:
tfile=$(mktemp /tmp/foo.XXXXXXXXX)
pow ffs stage ${action_dir}/${book_title} > $tfile

Then we use basic cut/sed to pull the CID value out (notice we use sed to cleanup a leading space, and the ascii color tags that are in the output):
cid=$(cat ${tfile} | sed 's/\x1b\[[0-9;]*m//g' | sed 's/ //g' | cut -d":" -f2)

However when we run the script, we get a JSON error, even though we weren't passing any JSON back in (we assume the snippet that contains the CID must have something attached to it):
- writing title and cid to db: philosophicaltra7117roya,bafybeiceeb3zon2ffc4l5byjafrnhdjuc4v6zshfgryxp7sv2bjfpr57ru
pow ffs config push bafybeiceeb3zon2ffc4l5byjafrnhdjuc4v6zshfgryxp7sv2bjfpr57ru -t b776fdda-3495-4270-bbc4-5ea7af209b46
> Error! Unexpected end of JSON input

Now the curious thing, even though that failed, if we just cut/paste that line from the screen output, it works!
pow ffs config push bafybeiceeb3zon2ffc4l5byjafrnhdjuc4v6zshfgryxp7sv2bjfpr57ru -t b776fdda-3495-4270-bbc4-5ea7af209b46
> Success! Pushed cid storage config for bafybeiceeb3zon2ffc4l5byjafrnhdjuc4v6zshfgryxp7sv2bjfpr57ru to FFS with job id: 730dd808-48c6-4e28-a11d-43e78b3f0dce

After a lot of work, we feel the output from pow ffs stage contains non ascii characters that we can't (or aren't yet!) cleaning up, so would request a 'plain text' or 'raw' output from the command so that we can capture the CID to allow us to automate the the pushing of staged items.

same erros when running a batch script cid caught in stage.

In an attempt to automate pow we capture the output of the stage command:
tfile=$(mktemp /tmp/foo.XXXXXXXXX)
pow ffs stage ${action_dir}/${book_title} > $tfile

Then we use basic cut/sed to pull the CID value out (notice we use sed to cleanup a leading space, and the ascii color tags that are in the output):
cid=$(cat ${tfile} | sed 's/\x1b\[[0-9;]*m//g' | sed 's/ //g' | cut -d":" -f2)

However when we run the script, we get a JSON error, even though we weren't passing any JSON back in (we assume the snippet that contains the CID must have something attached to it):
- writing title and cid to db: philosophicaltra7117roya,bafybeiceeb3zon2ffc4l5byjafrnhdjuc4v6zshfgryxp7sv2bjfpr57ru
pow ffs config push bafybeiceeb3zon2ffc4l5byjafrnhdjuc4v6zshfgryxp7sv2bjfpr57ru -t b776fdda-3495-4270-bbc4-5ea7af209b46
> Error! Unexpected end of JSON input

Now the curious thing, even though that failed, if we just cut/paste that line from the screen output, it works!
pow ffs config push bafybeiceeb3zon2ffc4l5byjafrnhdjuc4v6zshfgryxp7sv2bjfpr57ru -t b776fdda-3495-4270-bbc4-5ea7af209b46
> Success! Pushed cid storage config for bafybeiceeb3zon2ffc4l5byjafrnhdjuc4v6zshfgryxp7sv2bjfpr57ru to FFS with job id: 730dd808-48c6-4e28-a11d-43e78b3f0dce

After a lot of work, we feel the output from pow ffs stage contains non ascii characters that we can't (or aren't yet!) cleaning up, so would request a 'plain text' or 'raw' output from the command so that we can capture the CID to allow us to automate the the pushing of staged items.

After some investigation, I found that adding -c /path/to/config_file solved my problem.

authors: ilzc philcryer

repo: textileio/powergate


text:
fix(fixer): fix to work commands that accept Node

Correct these method

insertTextAfter(node, text)
insertTextBefore(node, text)
replaceText(node, text)
remove(node)

fix  #229

closed by https://github.com/textlint/textlint/issues/229#issuecomment-245310565
We will remove methods that accept TxtNode.

repo: textlint/textlint


text:
add airdrop note

add airdrop note

https://discord.com/channels/357957786904166400/817766991799648286/910819951047229470

Hello @DougAnderson444 , this is not really accurate. Wallet addresses return a balance of 0 if you don't set any balance before. It is true that you can mint an unlimited amount, but it's not true that the balances are infinite.
Thanks for the PR!

Ok, can we add how to mint the unlimited amount then? I didn't see that
anywhere, although I haven't had any issues deploying a contract from a
random keypair
On Thu., Nov. 18, 2021, 2:13 p.m. Cedrik Boudreau, @.***>
wrote:

Hello @DougAnderson444 https://github.com/DougAnderson444 , this is not
really accurate. Wallet addresses return a balance of 0 if you don't set
any balance before. It is true that you can mint an unlimited amount, but
it's not true that the balances are infinite.
Thanks for the PR!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/textury/arlocal/pull/47#issuecomment-973130948, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABTDW7KUKIH4BKDPSBF5CXLUMU66LANCNFSM5IJDMCTA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Added the details on the readme. Also if you're using an older version it will work as expected. This was a WIP and changes started to appear this week :)

repo: textury/arlocal


text:
send not working

I have tried 2 times to send tezos, each time it said it was ok. first time i tried with 0 fees (cause it is written that is is currently allowed, second time I tried with 1420micro tezos which is normally the minumum. Is is because of this that is does not work ? Is 2500micro tezos the minimum on tezbox ? Does tezbox take fees on top of blockchain fees ?

i found a fix to this isusse go to Tezbox wallet with Chrome or Firefox browser:
https://wallet.tezbox.com
and restore your wallet then go to setting and find better node

Hello, have you been able to resolve your issue ?

I have tried 2 times to send tezos, each time it said it was ok. first time i tried with 0 fees (cause it is written that is is currently allowed, second time I tried with 1420micro tezos which is normally the minumum. Is is because of this that is does not work ? Is 2500micro tezos the minimum on tezbox ? Does tezbox take fees on top of blockchain fees ?

Hello,
Have you been able to resolve your issue ?
Have you been able to send out your xtz ?

@optimum-dulopin Hello, Have you been able to resolve this issue ?

@optimum-dulopin Hello, Have you been able to resolve this issue ?

@optimum-dulopin Hello, Have you been able to resolve this issue ?

@optimum-dulopin Has your issue been resolved ?

@optimum-dulopin Has your issue been resolved ?

@optimum-dulopin Has your issue been resolved ?

repo: tezbox/desktop-wallet


text:
Add support for GHC 9.4

Will fix #14. Currently doesn't build.

I don't think that this is going to be worth the effort.

repo: tfausak/splint

license: ISC


text:
Ensure Default Service account is not used at a project level

Possible check to target the following resource

[ ] google_project_iam_member

closing, tracking in spreadsheet

repo: tfsec/tfsec


text:
pip

Hi, thanks for your work, I find it superuseful.
Have you think in packaging it for pip? It would be nice to install from there.

Yes.. I've been meaning to.
Sorry about the delay with that.
I will try to get it up before I leave on vacation this week!

authors: malkab tgoodlet

repo: tgoodlet/pytest-interactive

license: mit

license_source: bigquery


text:
Overall training process

Thank you for your excellent work.
I am a little confused about the overall training process of CORA.
Could you please describe the overall training process?
Thank you very much!

Thank you for your excellent work. I am a little confused about the overall training process of CORA. Could you please describe the overall training process? Thank you very much!

Hello, you know how to train now, could you share your training experience (including the environment configuration of the operation), thank you very much!

Hi! Has anyone reproduced the results?

Hi! Has anyone reproduced the results?

I tried to reproduce it, but limited to the memory of gpu(maybe CORA needs 8 x 80G), the result of novel ov-coco is slightly lower than 35.1 with RN50 (about 34.3

@eternaldolphin Hi, which GPU devices do you use to reproduce the results?

@eternaldolphin Hi, which GPU devices do you use to reproduce the results?

4*40G

Hi! Has anyone reproduced the results?

I tried to reproduce it, but limited to the memory of gpu(maybe CORA needs 8 x 80G), the result of novel ov-coco is slightly lower than 35.1 with RN50 (about 34.3
Welcome to more discussion with my wechat Maytherebe

That's wonderful! I would like to ask if it is possible for u to kindly open-source your implementation on github so that other people can learn from it and reproduce the results of CORA. Hope I'm not being rude. Thanks.

Excuse me, what is your batchsize setting? I used 4 × 48G, batchsize is set to 4, but CUDA out of memory. @eternaldolphin

Hi! Has anyone reproduced the results?

I tried to reproduce it, but limited to the memory of gpu(maybe CORA needs 8 x 80G), the result of novel ov-coco is slightly lower than 35.1 with RN50 (about 34.3
Welcome to more discussion with my wechat Maytherebe

That's wonderful! I would like to ask if it is possible for u to kindly open-source your implementation on github so that other people can learn from it and reproduce the results of CORA. I hope I'm not being rude. Thanks.

sorry for missing the information, you can refer to this https://github.com/eternaldolphin/cora-dev. And welcome to use LaMI-DETR as baseline, which can train ov-lvis in one day with 840G A100 or in two days with 832G V100

authors: QHCV eternaldolphin kinredon shaniaos wusize ysysys666 yuki1ssad


text:
Onboarding task9

Description
dependencie injection with Hilt
Related Issues
task09-onboarding
Visual reference
https://user-images.githubusercontent.com/53837410/193362335-084bd4c3-6279-4b46-9622-d14e2add2d71.mp4

For documenting purposes, if we use this codelab in the future, the following changes are required to make it work on Apple M1:

authors: joaobertacchi thalysmcarrara

repo: thalysmcarrara/android-hilt


text:
Thanos: Remote Read API

Since streamed chunked remote read implementation is merged in prometheus project (https://github.com/prometheus/prometheus/pull/5703), do we have any plan to expose remote read API in thanos?
FYI: original request for this feature: https://github.com/thanos-io/thanos/issues/1128

Our Store gRPC API is exactly like remote read (but gRPC) (:
Looks like @accent-atul from the issue you mentioned want to build proxy layer for StoreAPI.  I think I personally would be fine to maintain remote read API on querier since it's literally similar API to StoreAPI so the shim would be lightweight.
What's your use case?

@bwplotka Our use case is we'd like to expose our thanos cluster to customers which is using prometheus native ruler (not thanos ruler), prometheus native ruler only supports remote read APIs but not Store gRPC API.
From my understanding, in prometheus world, remote read API should be best for us especially chunked remote read API is supported now :-)
Any thoughts, please advise.

What Prometheus ruler not Thanos ruler?
The benefit of Thanos ruler is that it supports partial response strategies (but you can work around those with Querier with certain partial response strategy)

@bwplotka

What Prometheus ruler not Thanos ruler?

We have one use case which run prometheus ruler with a rule orchestration engine (manage prometheus ruler cluster, rule distribution,  resume/failover, provide CRUD APIs, etc. We implement it since we have 50,000+ rules and cannot be supported by node and rule CRUD is frequent), this prometheus ruler doesn't have local tsdb (just for rule evaluation), it points to our thanos cluster for remote read. This is why I'd like to say it will be great that thanos can support remote read API, then it will be easy to be used by other prometheus eco other products.
BTW: Partial response strategies is a good feature, however, I think it might not be the good choice for us since it loose accuracy.

Rule evaluations in Prometheus have exactly the same characteristics when done with remote data. Also the rule engine in Thanos is literally the same as Prometheus (we use Prometheus’ rule engine as a library). There is no difference in the setups that you’re describing, just the Thanos one works today without Thanos having a remote read endpoint.

@brancz yes, you are correct, our use case can replace prometheus ruler with thanos ruler to use thanos Storage APIs. However, I still think remote read API is the common API in prometheus eco system, it might be better to support it in thanos.

So using Thanos Ruler has many benefits:

First of all Ruler does not point to StoreAPI, but to PrometheusAPI - so Prometheus, or Qurier for example. If you use Prometheus you would user Remote Read -> StoreAPI. This has a few issues: You don't use our Qurier deduplication and downsampling (!) features.
Secondly Ruler exposes StoreAPI so Thanos system has access to the ruler recording rules data as a well as ALERTS metric. You can achieve similar with Prometheus + sidecar though.
You have more control on partial responses.

I agree with @brancz and I am not opposed on remote read generally, but I just don't want users to use something that would be anti-pattern and misleading as you lose so many benefits Thanos gives. (:
What would be the benefit of remote read API in Thanos then?
Maybe easier migration process? :thinking:

FYI, we dropped the idea of building remote read endpoint in thanos. Instead, we attempted to have separate remote read adapter which internally calls gRPC and can scale independently. For simplicity and to avoid dealing with chunk format changes,  we changed the adapter to call thanos http query range api . So Prometheus ---remote read ---> Adapter ----thanos http query range api --> Thanos.
Our use case is similar, we want to upfront Prometheus for query and running rules as our Prometheus connects to multiple Thanos, and other internal telemetry platform and provides unified query and rules execution.

Thanks @bwplotka for what benefits Thanos Ruler could provide, I definitely agree that our use case can use Thanos Ruler to replace prometheus Ruler.
However, I still think sometimes thanos remote read API could be useful. Assume that we have 2 data sources, one is thanos cluster and another is native prometheus cluster with third-party databases (such as ClickHouse, Couchbase, etc.). Assume we'd like to set up an unified Ruler for these 2 data sources, it might be hard to do that if thanos cluster doesn't expose Prometheus remote read API. What do you think?

@accent-atul Why not use thanos Storage API (gRPC) which query raw metrics? Thanos http query range API doesn't return raw metrics and inefficient.

Yea @accent-atul it looks like this Adapter would be almost seamless if you would just implement it against StoreAPI - essentially going through PromQL engine twice is not perfect. (:
Looks like remote read adapter might be nice then indeed (?)

@garrettlish The first attempt was with gRPC. It worked, but to avoid the chunk format and encoding which could change in future for efficiency, we switched to simple query range API. It's giving decent performance. I agree, it may not return raw metrics and apply step resolution (required by thanos and Prometheus).
Is there any good example/utility available to decode the data as returned by storage API. I wasn't very keen on the application maintaining than decoder.
As of now, most of our telemetry is stored in internal telemetry platform which we read through remote read. We are still evaluating options, and will revisit this adapter shortly.

We had a need for a remote read adapter as discussed in this issue, we've now made it available at: https://github.com/G-Research/thanos-remote-read using the gRPC Store API.

Amazing!
Added docs change to both Prometheus: https://github.com/prometheus/docs/pull/1616 and Thanos: https://github.com/thanos-io/thanos/pull/2566

This is issue is done :+1:

authors: accent-atul brancz bwplotka dgl garrettlish

repo: thanos-io/thanos


text:
fetcher: Made metaFetcher go routine safe; Fixed multiple bucket UI +fetcher issues.

Fixed https://github.com/thanos-io/thanos/issues/2349
Fixed races (we were reusing fetcher by both bucket UI and compaction syncs...
Fixed logging
Added singleflight to ensure we don't synchronize too often.
Signed-off-by: Bartlomiej Plotka bwplotka@gmail.com
cc @squat Would be awesome to merge this before release as it's a major race :/ That might interact with what compact sees.

Anyone @squat @povilasv ? I already have follow up PR (:

Thanks!

Follow up: https://github.com/thanos-io/thanos/pull/2354

:p that's this issue. Follow up: #2357

:man_facepalming:

authors: bwplotka squat


text:
added definiton of chunk and referenced it in the docs

Fixes https://github.com/thanos-io/thanos/issues/3627

[ ] I added CHANGELOG entry for this change.
[ ] Change is not relevant to the end user.

Changes

Update store.md
Update design.md
Update storage.md
Update troubleshooting.md

Verification

cc @GiedriusS
Please review the changes 😃


text:
Merge release v0.32.5 to main

[x] I added CHANGELOG entry for this change.
[ ] Change is not relevant to the end user.

Changes

Verification

Oops, forgot that we shouldn't squash :facepalm:

authors: GiedriusS saswatamcode


text:
Topic Generation

Generate the Topic for each formed cluster.
Update the infrastructure (ssm parameters for openai api, other updates)

done

repo: the-deep-nlp/core-server-infra


text:
added Vue CLI doc

NOTE:

Please fill out the template below

Did you read the instructions in the README?

Y

Have you read the instructions in CONTRIBUTING?

Y

Have you written this article by yourself? Have you taken inspiration from anywhere and then written it in your words?

I originally learned from the VueJS website, but have used the commands so much that I didn't have to pull up anything in order to write the document. The commands basically give you all of the information. I did have to practice this and re-write a bit because I was missing installing vue cli before the vue cli service, but that was troubleshot by myself without reference to documentation.

We already have an article on Vue

authors: asbourlotos the-ethan-hunt

repo: the-ethan-hunt/first-timers-guide


text:
Fixed weird crashing loading errors because the == operator was retur…

…ning a boolean 'false' rather than nil, which then spat out errors from register_craft.

This seems to solve #4

Thank you for providing a fix for #4.
I have found an easier solution in 8279436cde4cd2cdf007340634c51868017adef1 where I replaced the type by
type(r[2]) == "string" and r[2] or nil. This way minetest will use it's own default recipe type.
Nonetheless I appreciate your effort. Sorry for taking so much time until I fix it myself.


text:
Fixed Main Menu update

Update to fix main menu on scroll. https://thebiggive.atlassian.net/browse/COM-35

Desktop view looks good to me now, but on mobile there's this empty grey bar between the menu and the top of the content. Can we lose that?

Desktop view looks good to me now, but on mobile there's this empty grey bar between the menu and the top of the content. Can we lose that?

Can you tell me if this view is as a result of you loading in desktop view and reducing the size to mobile in your brwoser?

authors: bdsl tomwpaterson-shape

repo: thebiggive/components


text:
imgscalr.com respond 404 Not Found

Hello!
Just want see imgscalr in action but http://imgscalr.com/ is not work properly.
Please fix it.
Or it only rest-based without any interface's interactions?

Yes, and maven.thebuzzmedia.com is down. What is the alternative?

I took imgscalr.com down because the daily traffic was very low and I don't
have time to polish it up to really expose all the functionality in
imgscalr (the library) - it was misleading and more of a tech showcase for
an article I wrote 5 years ago.
maven.thebuzzmedia.com is down because imgscalr moved into Maven central
repo - no additional "repository" tags needed, it's in the official Maven
repo! :)

Riyad
http://thebuzzmedia.com
On Tue, Oct 27, 2015 at 6:40 PM, Carlos Rico Avendaño <
notifications@github.com> wrote:

Yes, and maven.thebuzzmedia.com is down. What is the alternative?
—
Reply to this email directly or view it on GitHub
https://github.com/thebuzzmedia/imgscalr/issues/115#issuecomment-151665982
.

So where can we find the APIs documentation?

Same as @daboom, Where can we find the API documentation?

Not sure about the documentation but the library can be found here: http://mvnrepository.com/artifact/org.imgscalr/imgscalr-lib/4.2

Documentation can be found at: http://javadoc.io/doc/org.imgscalr/imgscalr-lib

authors: H-Amarado Makishima crico daboom iarriola ksclarke thebuzzmedia

repo: thebuzzmedia/imgscalr


text:
Added reference to schema files

Now all language and config files have a link to a proper schema.json file.

@Kolonsky don't forget to mention in PR description that it closes #8 for automatically close related issue.

authors: Kolonsky koloml

repo: thecorecity/cognitum


text:
[BUG] Code of conduct not found

Describe the bug
When you open a new pr you are suggested to go to the code of conduct, currently its not found
To Reproduce
Open new pr, you will see this :

Expected Behavior
The link should redirect to the project code of conduct
Screenshot/ Video
Instead, this is what we see :

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I would like to take this issue, is this https://github.com/thecyberworld/TheCyberHUB/blob/dev/CODE_OF_CONDUCT.md the place where we would get redirected instead?

@Tamir198 Yes, need to change from "main" to "dev" in all the templates. Thanks for raising this issue 😁

repo: thecyberworld/TheCyberHUB


text:
sc.pl.violin doesn't work with groupby and ax passed together

[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of scanpy.
[ ] (optional) I have confirmed this bug exists on the master branch of scanpy.

Minimal code sample (that we can copy&paste without having any data)
Option 1: group with two keys, passing two axes
import scanpy as sc
import matplotlib.pyplot as plt
adata = sc.datasets.pbmc3k()
adata.obs['group'] = adata.obs.index.to_series().str.startswith("A").astype(str)
fig, axes = plt.subplots(1, 2)
sc.pl.violin(adata2, keys=['CD8A', 'CD8B'], groupby="group", ax=axes)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [51], in <module>
      4 adata2.obs['group'] = adata2.obs.index.to_series().str.startswith("A").astype(str)
      5 fig, axes = plt.subplots(1, 2)
----> 6 sc.pl.violin(adata2, keys=['CD8A', 'CD8B'], groupby="group", ax=axes)

File /opt/conda/envs/analysis/lib/python3.8/site-packages/scanpy/plotting/_anndata.py:835, in violin(adata, keys, groupby, log, use_raw, stripplot, jitter, size, layer, scale, order, multi_panel, xlabel, ylabel, rotation, show, save, ax, **kwds)
    833     axs = [ax]
    834 for ax, y, ylab in zip(axs, ys, ylabel):
--> 835     ax = sns.violinplot(
    836         x=x,
    837         y=y,
    838         data=obs_tidy,
    839         order=order,
    840         orient='vertical',
    841         scale=scale,
    842         ax=ax,
    843         **kwds,
    844     )
    845     if stripplot:
    846         ax = sns.stripplot(
    847             x=x,
    848             y=y,
   (...)
    854             ax=ax,
    855         )

File /opt/conda/envs/analysis/lib/python3.8/site-packages/seaborn/_decorators.py:46, in _deprecate_positional_args.<locals>.inner_f(*args, **kwargs)
     36     warnings.warn(
     37         "Pass the following variable{} as {}keyword arg{}: {}. "
     38         "From version 0.12, the only valid positional argument "
   (...)
     43         FutureWarning
     44     )
     45 kwargs.update({k: arg for k, arg in zip(sig.parameters, args)})
---> 46 return f(**kwargs)

File /opt/conda/envs/analysis/lib/python3.8/site-packages/seaborn/categorical.py:2408, in violinplot(x, y, hue, data, order, hue_order, bw, cut, scale, scale_hue, gridsize, width, inner, split, dodge, orient, linewidth, color, palette, saturation, ax, **kwargs)
   2405 if ax is None:
   2406     ax = plt.gca()
-> 2408 plotter.plot(ax)
   2409 return ax

File /opt/conda/envs/analysis/lib/python3.8/site-packages/seaborn/categorical.py:1043, in _ViolinPlotter.plot(self, ax)
   1041 def plot(self, ax):
   1042     """Make the violin plot."""
-> 1043     self.draw_violins(ax)
   1044     self.annotate_axes(ax)
   1045     if self.orient == "h":

File /opt/conda/envs/analysis/lib/python3.8/site-packages/seaborn/categorical.py:761, in _ViolinPlotter.draw_violins(self, ax)
    759 def draw_violins(self, ax):
    760     """Draw the violins onto `ax`."""
--> 761     fill_func = ax.fill_betweenx if self.orient == "v" else ax.fill_between
    762     for i, group_data in enumerate(self.plot_data):
    764         kws = dict(edgecolor=self.gray, linewidth=self.linewidth)

AttributeError: 'numpy.ndarray' object has no attribute 'fill_betweenx'

Option 2: group with two keys, passing first of two axes
import scanpy as sc
import matplotlib.pyplot as plt
adata = sc.datasets.pbmc3k()
adata.obs['group'] = adata.obs.index.to_series().str.startswith("A").astype(str)
fig, axes = plt.subplots(1, 2)
sc.pl.violin(adata2, keys=['CD8A', 'CD8B'], groupby="group", ax=axes[0])

No traceback, but the second axis is simply not plotted.

Option 3: group with two keys, passing one axis
import scanpy as sc
import matplotlib.pyplot as plt
adata = sc.datasets.pbmc3k()
adata.obs['group'] = adata.obs.index.to_series().str.startswith("A").astype(str)
fig, ax = plt.subplots()
sc.pl.violin(adata, keys=['CD8A', 'CD8B'], groupby="group", ax=ax)

No traceback, even though this should error. Plots just the first of the two keys.

Versions

WARNING: If you miss a compact list, please try `print_header`!
-----
anndata     0.7.8
scanpy      1.8.2
sinfo       0.3.1
-----
PIL                         8.4.0
anndata                     0.7.8
asttokens                   NA
attr                        21.2.0
backcall                    0.2.0
cffi                        1.15.0
colorama                    0.4.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.0
debugpy                     1.5.1
decorator                   5.1.1
defusedxml                  0.7.1
django                      4.0
executing                   0.8.2
google                      NA
h5py                        2.10.0
idna                        3.1
igraph                      0.9.8
importlib_resources         NA
ipykernel                   6.7.0
ipython_genutils            0.2.0
ipywidgets                  7.6.5
jedi                        0.18.1
jinja2                      3.0.3
joblib                      1.1.0
jsonschema                  4.4.0
jupyter_server              1.13.3
kiwisolver                  1.3.2
leidenalg                   0.8.8
llvmlite                    0.37.0
markupsafe                  2.0.1
matplotlib                  3.5.1
matplotlib_inline           NA
mpl_toolkits                NA
natsort                     8.0.2
nbformat                    5.1.3
numba                       0.54.1
numexpr                     2.8.0
numpy                       1.19.5
packaging                   21.3
pandas                      1.1.5
parso                       0.8.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
prometheus_client           NA
prompt_toolkit              3.0.24
psutil                      5.8.0
ptyprocess                  0.7.0
pure_eval                   0.2.1
pvectorc                    NA
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.6.0
pydevd_concurrency_analyser NA
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.10.0
pyparsing                   3.0.6
pyrsistent                  NA
pytz                        2021.3
scanpy                      1.8.2
scipy                       1.5.3
scprep                      1.1.0
seaborn                     0.11.2
send2trash                  NA
setuptools                  58.0.4
setuptools_scm              NA
sinfo                       0.3.1
six                         1.16.0
sklearn                     0.24.2
sphinxcontrib               NA
stack_data                  0.1.4
statsmodels                 0.13.1
tables                      3.6.1
terminado                   0.12.1
texttable                   1.6.4
tornado                     6.1
tqdm                        4.62.3
traitlets                   5.1.1
typing_extensions           NA
wcwidth                     0.2.5
yaml                        6.0
zipp                        NA
zmq                         22.3.0
-----
IPython             8.0.0
jupyter_client      6.1.12
jupyter_core        4.9.1
jupyterlab          3.2.8
notebook            6.4.7
-----
Python 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:59:51) [GCC 9.4.0]
Linux-5.4.0-1064-gcp-x86_64-with-glibc2.10
16 logical CPU cores
-----
Session information updated at 2022-02-10 16:38

Thanks for the report. This is essentially a documentation issue, as the ax argument cannot work with groupby in this context. This is a holdout from implementations before seaborn made the distinction between Figure and Axes level plots.

Thanks @ivirshup ! In addition to changing the documentation, it would probably make sense to throw an error if these two arguments are passed together.

authors: ivirshup scottgigante-immunai

repo: theislab/scanpy


text:
Testing issues

some random comment

Finished task

repo: thelizri/PotBot


text:
radarr anime logo - text & mobile

radarr-anime-logo
I hope to find the .psd file used as the source, so I can upload it someday.

radarr-anime.png contains 900x292 .png file.
The version on a white background isn't great, but neither is the one for 4K.

radarr-anime-mobile.png contains 900x900 .png file.
Even here, the version on a white background isn't great, but mine is a bit better than the 4K one. 😅.

Please rebase to develop

authors: GilbN andreirotariu


text:
Some of the classes are from Bootstrap 4

Hi. Thanks for the job done, but you should consider updating your demo. I spent hours investigating, why your template didn't look like it had to, and some JS code didn't work at all. You use ml-* classes for margins, while in bootstrap 5 they use ms-*, and there are many things like that.
Take a look at this: https://help.blocsapp.com/knowledge-base/migrate-from-bootstrap-4-to-bootstrap-5/

Hi. Thanks for the job done, but you should consider updating your demo. I spent hours investigating, why your template didn't look like it had to, and some JS code didn't work at all. You use ml-* classes for margins, while in bootstrap 5 they use ms-*, and there are many things like that.
Take a look at this: https://help.blocsapp.com/knowledge-base/migrate-from-bootstrap-4-to-bootstrap-5/


text:
Limit & Skip issue

Hello
When using LIMIT and SKIP in the same query (findMany for instance) it gives me an issue because it generates something like :
MATCH (Node) RETURN Node ASC LIMIT $limit SKIP $skip

But it should be like
MATCH (Node) RETURN Node ASC SKIP $skip LIMIT $limit

to work properly.
SKIP should appears before  LIMIT
Thank you for your incredible work

My bad I wasn't up to date

Hey, thanks for your kind words :) Indeed it was an issue (#43) that was fixed on version v.1.10.3

authors: BOISSARD themetalfleece

repo: themetalfleece/neogma


text:
When friend is removed information for friend remains as it is

Bug Description
When friend is removed information for friend remains as it is
Steps to Reproduce

Go to friend overview window
Click on three dots at top right corner
Click on Remove friend
See result

Actual Result
When friend is removed information for friend remains as it is
Expected Result
Information should be removed
OS version
Windows
Screenshot/Recording
http://recordit.co/XlwstQYl4f

This has been moved to the Design Repository

authors: kristykjlee webdeveloperswj

repo: thenewboston-developers/Account-Manager


text:
Transaction fee

Regarding https://github.com/thenewboston-developers/Core/issues/11#issuecomment-1111074022 we can require a minimum fee to send each message. This is configurable by the node maintainer. It will be similar to these alpha network fields.

Instead of account_number we can have owner to represent the node maintainer. We can replace default_transaction_fee with transaction_fee and set the default (and minimum value) to 1.
So the amount still represents the amount being sent to the recipient - this can still be 0. There will however be an additional fee to send each message which is configurable by the node owner. The minimum fee is 1. This way the owner can raise the fee to throttle traffic if needed. A transaction_fee field can be included in the request body so that the backend can ensure the user is agreeing to the fee (incase it's recently been change). We do not need to include any owner or transaction_fee_account or anything though. All transaction fees can just be be deducted from the senders account (basically burnt).
Sample request body:
{
  "sender": "eb01f474a637e402b44407f3c1044a0c4b59261515d50be9abd4ee34fcb9075b",
  "recipient": "6fac7f7e2b90173bfc6ef8ee34f9c92438b5eb8f579ef8d84464b820bbfecfc1",
  "amount": 0,
  "transaction_fee": 1,
  "payload": {
    "message": "Hey"
  },
  "signature": "df64c0689b2f203c345269aea47d90576c6c37ce168d6084016772f79b2dfb1228de053ec753a2b87e4b404929a8bc6123acf56e7acd178716077e4ddb9b3208"
}

I think transaction_default_fee is a correct name rather than default_transaction_fee (since it ambiguous to mean some sort of default transaction that has a fee).
@buckyroberts do you want owner and transaction_default_fee be configured via Django Admin or settings.py (env vars) is good enough?
Also why not to name the setting just transaction_fee since there is no such thing as non-default transaction fee and also we still force clients to include transaction_fee into block body?

Bucky — Today at 9:10 PM
Can we keep is as is? So there will be a super user with access to Django admin but that's more just for development. The owner should never need to go into Django admin. The owner as of now is really just the account that can update the transaction_fee via API request.
Or more simply, the owner is the one who can make PATCH requests to /config to change either the owner or transaction_fee
Then GET /config can be public so that the clients can know what the transaction_fee is. We can include the owner in the response just for completeness but it won't really be used for anything.
dmugtasimov — Today at 9:13 PM
How should owner acquire an API token for an API request? Or we do signature-based authentication?
Bucky — Today at 9:13 PM
They can just send in a signed request. We can match the account_number with the owner.
dmugtasimov — Today at 9:14 PM
ok

The most promising was https://github.com/jazzband/django-constance . But although there is activity in github on PyPI it was update like 1.5 years ago. Therefore I decide to implement own updateable setting. It should be simple to do.

https://github.com/thenewboston-developers/Core/pull/69

https://github.com/thenewboston-developers/Core/pull/70

https://github.com/thenewboston-developers/Core/pull/71

https://github.com/thenewboston-developers/Core/pull/72

authors: buckyroberts dmugtasimov

repo: thenewboston-developers/Core


text:
2 Updates to the Analytics

we don't want the FE to know what various categories of analytics there are, nor which graphs should be shown for which analytics category.

Therefore, we should have a endpoint that gives back the list of categories there are:

GET /analytics_categories

[
 {title: 'Community', id: 'community', endpoint: '/community_analytics'},
 {title: 'Economy', id: 'economy', endpoint: '/economy_analytics'},
 ...
 {title: 'Other Social', id: 'other', endpoint: '/other_social_analytics'},
 {title: 'Website', id: 'website', endpoint: '/website'}
]

- title will be used for the Label.
- id will be used for FE links (eg 'other social' analytics is at the route '/analytics/other'
- endpoint is the BE endpoint
- data shape is an array to dictate which order the analytics should be displayed in

Also, for specific analytic category endpoint, it should look something like:

 GET /economy_endpoints with optional timeframe as params:

 [
  {
    "title": "Total Coins Distributed",
    "data": {
       "2021-06-27T00:00:00Z": 54,
       "2021-06-20T00:00:00Z": 432,
       ...
    }
  },
  {
    "title": "Total Coins Distributed to Core",
    "data": { ... }
  },
  {
    "title": "Total Coins Distributed to Faucet",
    "data": { ... }
  },
  {
    "title": "Total Coins Distributed to Projects
",
    "data": { ... }
  },
]

This way, the FE doesn't have to specifically know which key it should parse, nor does it have to have the title for each graph hardcoded.

These two changes will make it easier for us moving forward. If we decide to add/remove a particular analytics or category, we can simply do so by updating the BE.
For example, we are going to add Pinterest as one of our social media sections. So we'll probably need an analytics endpoint for that.

here's my second (i think better) option (things changed in bold. First data shape is changed too):
we don't want the FE to know what various categories of analytics there are, nor which graphs should be shown for which analytics category. We also don't want the BE to have hardcoded the various categories, since new categories can be formed in the future (such as pinterest)
Therefore, we should have a endpoint that gives back the list of categories there are:
GET /analytics_categories

[
 {title: 'Community', id: 'community'},
 {title: 'Economy', id: 'economy'},
 ...
 {title: 'Other Social', id: 'other'},
 {title: 'Website', id: 'website'}
]

title will be used for the Label.
id will be used for FE links (eg 'other social' analytics is at the route '/analytics/other'
The ID will also be used to grab the BE endpoint for that particular category (example below)
data shape is an array to dictate which order the analytics should be displayed in

Also, for specific analytic category endpoint, it should look something like:
GET /analytics/economy with optional timeframe as params:
 [
  {
    "title": "Total Coins Distributed",
    "data": {
       "2021-06-27T00:00:00Z": 54,
       "2021-06-20T00:00:00Z": 432,
       ...
    }
  },
  {
    "title": "Total Coins Distributed to Core",
    "data": { ... }
  },
  {
    "title": "Total Coins Distributed to Faucet",
    "data": { ... }
  },
  {
    "title": "Total Coins Distributed to Projects
",
    "data": { ... }
  },
]

heres a model example i shared:
Class AnalyticsCategory:
  title: string;             // "Other Social Media"
  id: string;                // "other"
  analytics: Analytics[];

Class Analytics:
  uuid: string;              // Example, lets say this was 'test123'
  title: string;             // 'Total Coins Distributed to Core'
  dataset: AnalyticsDataset[];

Class AnalyticsDataset:
  analytics: string;        // ie 'test123'
  date: Timestamp;
  value: number;

GET /analytics-categories

[
 {title: 'Community', id: 'community', analytics: ['123', '124', ...]},   (these should come back as nested data instead)
 {title: 'Economy', id: 'economy', analytics: ['111', '112', ...]},
 ...
 {title: 'Other Social', id: 'other', analytics: []},
 {title: 'Website', id: 'website', analytics: []}
]

GET /analytics/123123 (uuid for 'Community')

{
  uuid: '123123';
  title: 'Community'
}

GET /analytics-dataset?analytics=123123&from=monday&to=friday     (obviously monday/friday should be proper dates)

[
 {date: 'monday', value: 56},
 {date: 'tuesday', value: 128},
 ...
]

@angle943 I'm not sure I get exactly the requirement and how the use case will be, mostly the last part on the models. But it seems you want to be able to fetch all the analytics categories available then use the id of those categories to fetch the respective analytics. Also, have all the analytics in one model instead of the different analytics models we had?
If that's the case we don't need 3 Models, we can simply have AnalyticsCategory ( /analytics-categories) and Analytics(/analytics) which has category as foreign key.
Class AnalyticsCategory:
  title: string;
  id: string;

  Class Analytics:
    uuid: string;
    category: "website12344s"; // (query specific analytics using this id) /analytics?category=123123&from=monday&to=friday
    value: number;
    date: Timestamp

@ThaDeveloper

With the model you presented, how will you distinguish getween Total Coins Distributed analytics and Total Coins Distributed to Core analytics? Each AnalyticsCategory can have more than one graph

Isn't that the purpose of Analytics Category  to different  the different analytics? Total coins distributed is the total of core,faucet and projects?

Sorry @ThaDeveloper didn't see your message until now. This was what i was thinking

@ThaDeveloper here's the link to the mocks fyi: https://www.figma.com/file/wWisZsaUf9xzkCCqlzvG3G/Analytics?node-id=1%3A4

btw i don't think AnalyticsDataset is the right word, as dataset refers to a collection of data points. maybe AnalyticsData or AnalyticsDatapoint is a better word for that diagram above

Having to fetch from 3 different models to get one type of analytics will be a performance bottleneck, 2 models work as described above

@ThaDeveloper okay let me know when it is ready

repo: thenewboston-developers/Website-API


text:
feature: add on_open and on_close function hook.

hi, theniceboy. i need remove some keymaps in maps.t table.
can you add on_open and on_close function in default setup?

Can you elaborate? I'm not sure what the maps.t table is. Thanks

sorry, i means like toggleterm. such as below png.

authors: chaozwn theniceboy


text:
Revalidate uploaded CSV terminates on successive 'revalidate' requests

Upon uploading CSV for validation (e.g. fixtures/revalidate.csv)

run server via foreman or csvlint.io
upload fixtures/csvs/revalidate
if "LF\n" is selected from 'Line terminator' drop down and Revalidate clicked 2 * 'Structural problem warnings' are returned on a refreshed validation screen
if Line Terminator "\r\n" selected subsequent to above step and revalidate clicked rails terminates with a Bad URI exception

Stack trace reveals that on the three steps pass a different argument to the CSVlint Gem on each pass
1.
*Csvlint::Validator.parse_extension(source#StringIO)
*Csvlint::Validator.initialize(source#StringIO, dialect#NilClass, schema#NilClass, options#Hash)
2.
*Csvlint::Validator.parse_extension(source#Tempfile)
*Csvlint::Validator.initialize(source#Tempfile, dialect#Hash, schema#NilClass, options#Hash)
3.
*Csvlint::Validator.initialize(source#NilClass, dialect#Hash, schema#NilClass, options#Hash) *Csvlint::Validator.parse_extension(source#NilClass)`
likely cause of fault is
validation = Validation.validate(self.url || self.csv, schema.try(:url), loaded_schema, dialect)

returning nil for both self.url and self.csv and the subsequent Validation.validate method having no rescue for it's io parameter equallying nil
def self.validate(io, schema_url = nil, schema = nil, dialect = nil)

The error is caused by nil being passed to self.validate for $arg io
This results in
validator = Csvlint::Validator.new( io, dialect, schema && schema.fields.empty? ? nil : schema )

receiving io as nil, which results in the@source instance variable within CSVLint Gem > validate.rb, causing the method validate to return the errors which are at the root of this issue

Comparison backtraces
#0  Csvlint::Validator.parse_extension(source#Tempfile) at /Users/stephenfortune/Documents/csvlint.rb/lib/csvlint/validate.rb:252
#1  Csvlint::Validator.initialize(source#Tempfile, dialect#Hash, schema#NilClass, options#Hash) at /Users/stephenfortune/Documents/csvlint.rb/lib/csvlint/validate.rb:34
#2  Class.new(*args) at /Users/stephenfortune/Documents/csvlint/app/models/validation.rb:35
#3  #<Class:Validation>.validate(io#Tempfile, schema_url#NilClass, schema#NilClass, dialect#Hash) at /Users/stephenfortune/Documents/csvlint/app/models/validation.rb:35
#4  Validation.update_validation(dialect#Hash) at /Users/stephenfortune/Documents/csvlint/app/models/validation.rb:158
 #5  ValidationController.update at /Users/stephenfortune/Documents/csvlint/app/controllers/validation_controller.rb:37

at this point self has a self.csv parameter
press revalidate button again crash
#0  Csvlint::Validator.parse_extension(source#NilClass) at /Users/stephenfortune/Documents/csvlint.rb/lib/csvlint/validate.rb:252
#1  Csvlint::Validator.initialize(source#NilClass, dialect#Hash, schema#NilClass, options#Hash) at /Users/stephenfortune/Documents/csvlint.rb/lib/csvlint/validate.rb:34
#2  Class.new(*args) at /Users/stephenfortune/Documents/csvlint/app/models/validation.rb:35
#3  #<Class:Validation>.validate(io#NilClass, schema_url#NilClass, schema#NilClass, dialect#Hash) at /Users/stephenfortune/Documents/csvlint/app/models/validation.rb:35
#4  Validation.update_validation(dialect#Hash) at /Users/stephenfortune/Documents/csvlint/app/models/validation.rb:158
#5  ValidationController.update at /Users/stephenfortune/Documents/csvlint/app/controllers/validation_controller.rb:37

Frame 5 is in validation controller, and in this state the csv_id has been set to nil either because it doesn't retreive the Mongo record correctly or the Validation object has been shorn of its csv_id parameter

repo: theodi/csvlint


text:
Get Function Return Value

Hey, i have yet another question xD
So i have been playing around with the library, working great but im wondering about the function calling.
Lets say i have this function in my class TestFunction:
public function getWeather()
{
    $weather = 'Sunny';
   echo $weather;
   return $weather;
}

In the above code, everything works fine and the bot will trigger the echo statement.
But lets say i want the bot to use the returned value, or atleast tell the user it has used the function.
Right now it will run the function, and the echo statement will show 'Sunny'. What approach would i take to make the bot reply with the returned value? Which is 'Sunny' in this case.  So for example bot reply would be 'Today its sunny outside!'. But the bot would choose the words to use.
Im using QuestionAnswer class to test this btw. I did notice the Message class has a static function called functionResult, where would this be stored?
I have been looking into the functions that make the function run, but im stuck on this atm.
Note: im testing on simple php website, nothing dynamic etc no javascript, i simply load the page to interact with the chatbot

Hey @samuelgjekic , you can use something like :
You can use generateTextOrReturnFunctionCalled so you can call manually the function yourself
$stringOrFunctionInfo = $this->openAIChat->generateTextOrReturnFunctionCalled($prompt);
if ($stringOrFunctionInfo instanceof FunctionInfo) {
        // $toolResponse can be a very long string
        $toolResponse = FunctionRunner::run($stringOrFunctionInfo);

You can parse the FunctionInfo yourself and decide to do whatever you want with it before calling other functions.
You can have a look in AutoPHP folder, I use it a lot.
Did it help you?

Hey @MaximeThoonsen thanks for your fast reply! I found the generateTextOrReturnFunction yesterday, but i did not know i had to run the function when checking the instanceof, thank you it works perfect.

authors: MaximeThoonsen samuelgjekic

repo: theodo-group/LLPhant


text:
Asynchronous Maybe

Hi i`m a beginner. i have a little problem (currently solved)
I am not good at English, so I am not sure if it will be delivered well.
anyway
my code
cell[0].value = '=spreadSheet funtion';
cell[1].value = '=spreadSheet funtion';

await cell.[0].save();
await cell.[1].save();

The spreadsheet has been updated with the code above.
I'll call you back to the next code.
// i expect
await getCells() // => result : ['1','2','3','4']
// result
await getCells()  // => result : ['loding..', 'loding'.., '1', '2'];

Even if I checked that the spreadsheet was updated, it was the same.
The same is true of executing the code in five seconds.
after a long thought
cell[0].value = '=spreadSheet funtion';
cell[1].value = '=spreadSheet funtion';
sheet.bulkUpdateCells(cells);

As I wrote it in the above code, I got the expected result.
so, Did I write the wrong code?
or Is it related to sync of save function?
thank you

You didn't do anything wrong. The old version of this module was callback based, so async/await did not work.
But, I've just a published a new version of this module which is a complete rewrite using Google's V4 Sheets API (v3 is being deprecated on March 3, 2020).
Docs available here - https://theoephraim.github.io/node-google-spreadsheet
Please test it out and I think you'll find it easier to use.
Thanks! 😄

authors: NohSeungWon theoephraim

repo: theoephraim/node-google-spreadsheet

license: unlicense


text:
Error when upgrading from 2.7.0 to 2.8.0: Token retrieval failure

Hi All,
We encountered an error after upgrading the league/oauth2-client library from version 2.7.0 to version 2.8.0. The error occurs when running a command that interacts with a third-party service to retrieve a token.
Steps to reproduce:

Use version 2.8.0 of the library.
Execute a request to a third-party service to retrieve a token using the following method:

With version 2.7.0, the command works correctly and retrieves a valid token.
Actual behavior:
Starting from version 2.8.0, the described error is triggered.

Discussion to be continued in https://github.com/thephpleague/oauth2-client/pull/1030 please.

Can you try if  https://github.com/thephpleague/oauth2-client/pull/1053 fixes your issue?

authors: amine-betari barryvdh liayn


text:
Authorization and OAuth2 compliance

When dealing with an OAuth-Server which is not supporting the authorization with credentials inside the request body the Abstract- or GenericProvider is not directly working.
So you have to implement a new provider which is overriding the getAccessTokenOptions method.
It would be better if the AbstractProvider would stick to the recommended behavior from RFC 6749 of sending the client credentials using the basic auth (authorization header). As you can see this section also tells that sending credentials inside the body is not recommended and should be limited to clients unable to use basic auth.
The getAccessTokenOptions could look like this:
    /**
     * Builds request options used for requesting an access token.
     *
     * @param  array $params
     * @return array
     */
    protected function getAccessTokenOptions(array $params)
    {
        $options = [
            'headers' => [
                'content-type' => 'application/x-www-form-urlencoded',
                'authorization' => 'Basic ' . base64_encode($params['client_id'] . ':' . $params['client_secret'])
            ]
        ];
        if ($this->getAccessTokenMethod() === self::METHOD_POST) {
            $options['body'] = $this->getAccessTokenBody($params);
        }

        return $options;
    }

As the client_id and client_secret is still inside the params array, they are also still inside the request body, so they could be removed from the params if necessary.

In practice, most of the OAuth 2 servers we've encountered do not support basic authentication. Instead, they support only passing these parameters in the body. As such, our generic base package will support the most common use case. If you encounter a provider that supports basic authentication, our recommendation is to extend the AbstractProvider and create a new concrete provider class to support that provider.
Thanks!

@ramsey basic authorization should be supported or this library is not fully spec compliant and does not work with some oauth2 provides, e.g.: https://github.com/ory/hydra
See https://tools.ietf.org/html/rfc6749#section-2.3.1

I was actually able to find the point in the RFC which specifically talks about not including those in the body:

Including the client credentials in the request-body using the two
parameters is NOT RECOMMENDED and SHOULD be limited to clients unable
to directly utilize the HTTP Basic authentication scheme (or other
password-based HTTP authentication schemes).  The parameters can only
be transmitted in the request-body and MUST NOT be included in the
request URI

Source

@ramsey  Definitely agree @arekkas comment on this. I hope the library can support basic authorization

Please see my previous comment regarding our recommendation on how to use Basic authentication with a provider:

Our recommendation is to extend the AbstractProvider and create a new concrete provider class to support that provider.

While that is sufficient, wouldn't it be nice to have that provider in the standard library, as the other option is officially discouraged in the RFC? After all, this is a library in the area of cyber security, so why not adhere to the recommendations of the people who wrote it? :)

I'm reopening the issue. I misunderstood the request.
If introduced, this will break BC in the GenericProvider, since any implementation using or extending the base library to use the client_credentials grant type to authorize with a provider that doesn't support Basic authentication will break. I think this is a pretty big BC break, and as noted earlier, when we released the library, we had found few providers in the wild that used Basic authentication for the client_credentials grant.
As an aside, there's an even greater discussion about why this is a result of the "spec" not being a true specification at all but, rather, a framework and it even acknowledges in section 1.8 that it is likely to produce a wide range of non-interoperable components…but I digress. :smile:
I'm open to thoughts and ideas on how we can use Basic authentication for the client_credentials grant type, without breaking BC.

The snippet below supports both Basic Auth as well for backwards compability the credentials as part of the body:
    /**
     * Builds request options used for requesting an access token.
     *
     * @param  array $params
     * @return array
     */
    protected function getAccessTokenOptions(array $params)
    {
        $options = [
            'headers' => [
                'content-type' => 'application/x-www-form-urlencoded',
                'authorization' => 'Basic ' . base64_encode($params['client_id'] . ':' . $params['client_secret'])
            ]
        ];
        if ($this->getAccessTokenMethod() === self::METHOD_POST) {
            $options['body'] = $this->getAccessTokenBody($params);
        }

        return $options;
    }

Please make sure to urlencode both client_id and client_secret. Not sure what the PHP equivalent is, but roughly:
base64_encode(urlencode($params['client_id']) . ':' . urlencode($params['client_secret']))

Adjusted the example with your remark @arekkas

About the URL encoding of passwort and secret I found this section:

The client identifier is encoded using the "application/x-www-form-urlencoded" encoding algorithm per Appendix B, and the encoded value is used as the username; the client password is encoded using the same algorithm and used as the password

See rfc6749 section 2.3.1

Yes, the idea behind it is that the id is urlencoded when included as a query parameter as well as in the x-www-form-urlencoded body, so it also should be encoded in the authorization header

Thanks. Would anyone care to open a pull request with tests for this?

I decided to go with a configuration option for the provider, where the default value is still the auth via form params.

@ramsey Just a heads up, Reddit's API is an example of a fairly major API that requires basic authentication when requesting an access token.

The oauth2-client implementation for Reddit provides both as well.

Also the implementation doesn't know what the provider accepts, at least on configuring the provider you should know it. So it's sufficient if both options can be configured. It's not necessary to send both at the same time. Btw with #653 it would be easy to implement also an OptionProvider with supports basic auth and post params

Has been resolved with #653

@jenschude Umm, how do I apply this with a GenericProvider? Is there documentation that shows how to pass the client_credentials grant with basic auth headers rather than in body? My code is like:
        $provider = new \League\OAuth2\Client\Provider\GenericProvider([
            'clientId'                => $this->clientId,
            'clientSecret'            => $this->clientSecret,
            'redirectUri'             => '',
            'urlAuthorize'            => 'https://mysite.com/oauth/authorize',
            'urlAccessToken'          => 'https://mysite.com/oauth/token',
            'urlResourceOwnerDetails' => 'https://mysite.com/oauth/resource'
        ]);
$accessToken = $provider->getAccessToken('client_credentials');

There is a https://github.com/thephpleague/oauth2-client/blob/master/src/OptionProvider/HttpBasicAuthOptionProvider.php to authenticate using Basic Auth

Ok, thanks. I was looking through the Reddit provider for a sample.

And it would be something like:
GenericProvider($options, new HttpBasicAuthOptionProvider()) when i remember it right

See also https://github.com/thephpleague/oauth2-client/blob/master/src/Provider/AbstractProvider.php#L137

@jenschude Yep, that works too. Thanks.

authors: ChrisThompsonTLDR SaifurRahmanMohsin Wassasin arekkas jayS-de jenschude ramsey zhenbo