text:
OKCoin Intl streaming service NPE (3.0.1-SNAPSHOT)

The streaming service for OKCoin Intl throws a NPE exception:
            (...)
            ExchangeSpecification exSpec = new ExchangeSpecification(exchangeName);

            if(exchangeName.equals("com.xeiam.xchange.okcoin.OkCoinExchange")){
               exSpec.setApiKey("XXXXX");
               exSpec.setSecretKey("YYYYY");
               exSpec.setExchangeSpecificParametersItem("Use_Intl", true);
            }

            //Get XChange interface
            Exchange exchange = ExchangeFactory.INSTANCE.createExchange(exSpec);
            ca.omnibus.Trading.Exchange.Exchange ex = exchanges.bootstrapExchange(exchange);

            final StreamingExchangeService service = exchange.getStreamingExchangeService(new OkCoinExchangeStreamingConfiguration());

            service.connect();

            while(true){
                try {
                  ExchangeEvent event = service.getNextEvent();

                  if(event != null) {
                    System.out.println("---> "  + event.getEventType());

                    if(event.getEventType().equals(ExchangeEventType.TICKER)) {
                      Ticker ticker = (Ticker) event.getPayload();

                      //long x = (new Date()).getTime() - ticker.getTimestamp().getTime();
                      //System.out.println("Delay " + x);
                    }

                  }
                }
                catch (InterruptedException e) {
                    //interruption handling code here
                }
            }

Stacktraces:
Grave:   java.lang.NullPointerException
	at com.xeiam.xchange.okcoin.service.streaming.OkCoinWebSocketService.parseDepth(OkCoinWebSocketService.java:84)
	at com.xeiam.xchange.okcoin.service.streaming.OkCoinWebSocketService.parseMarketData(OkCoinWebSocketService.java:66)
	at com.xeiam.xchange.okcoin.service.streaming.OkCoinWebSocketService.onReceive(OkCoinWebSocketService.java:47)
	at com.xeiam.xchange.okcoin.service.streaming.WebSocketClientHandler.channelRead0(WebSocketClientHandler.java:75)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
	at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:182)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
	at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
	at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:182)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)
	at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
	at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:182)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:971)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:854)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:249)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
	at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
	at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:182)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:886)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:126)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:510)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:467)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:381)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:834)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
	at java.lang.Thread.run(Thread.java:745)

Grave:   java.lang.NullPointerException
	at com.xeiam.xchange.okcoin.service.streaming.OkCoinWebSocketService.parseTicker(OkCoinWebSocketService.java:78)
	at com.xeiam.xchange.okcoin.service.streaming.OkCoinWebSocketService.parseMarketData(OkCoinWebSocketService.java:72)
	at com.xeiam.xchange.okcoin.service.streaming.OkCoinWebSocketService.onReceive(OkCoinWebSocketService.java:47)
	at com.xeiam.xchange.okcoin.service.streaming.WebSocketClientHandler.channelRead0(WebSocketClientHandler.java:75)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
	at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:182)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
	at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
	at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:182)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)
	at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
	at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:182)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:971)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:854)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:249)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
	at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
	at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:182)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:886)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:126)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:510)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:467)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:381)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:834)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
	at java.lang.Thread.run(Thread.java:745)

Background: In my app, a thread is started to open a streaming socket to every exchange (that supports it). The aforementioned code is inside the run() method.

@glabmoris Is this resolved now with the fix from yesterday or separate issue?

Seperate issue completely. This seems related to the parser for WS data. It does receive the data but throws NPEs when parsing it.

This only happens in the 3.0.1-SNAPSHOT branch. The develop branch's streaming demo runs fine

Ok great, then latest fix seems to fix this (strangely no parsing code has been changed in 3.0.1-SS and develop)

Can the fact that the consumer and the service reside on the same thread within a JavaEE container be the cause of the issue ?

No unlikely, the switching to international mode did not work in the version before the fix, which then probably tripped up the parsing. If you tested this in develop and it works you can close the issue.

My app depends on 3.0.1-SNAPSHOT. Maybe changing some dependencies around or updating the snapshot might solve things.

I forgot to mention that the working demo was for the CNY site. The Intl site doesn't work at all if we tweak the demo to use it:
package com.xeiam.xchange.examples.okcoin.streaming;

import java.util.Date;
import java.util.concurrent.TimeUnit;

import com.xeiam.xchange.Exchange;
import com.xeiam.xchange.ExchangeFactory;
import com.xeiam.xchange.ExchangeSpecification;
import com.xeiam.xchange.dto.marketdata.Ticker;
import com.xeiam.xchange.okcoin.OkCoinExchange;
import com.xeiam.xchange.okcoin.service.streaming.OkCoinExchangeStreamingConfiguration;
import com.xeiam.xchange.service.streaming.ExchangeEvent;
import com.xeiam.xchange.service.streaming.ExchangeEventType;
import com.xeiam.xchange.service.streaming.StreamingExchangeService;

public class OkCoinSocketIODemo {

  public static void main(String[] args) throws InterruptedException {

    ExchangeSpecification exSpec = new ExchangeSpecification(OkCoinExchange.class);
    exSpec.setSecretKey("aa");
    exSpec.setApiKey("bb");
    exSpec.setExchangeSpecificParametersItem("Use_Intl", true);

    Exchange exchange = ExchangeFactory.INSTANCE.createExchange(exSpec);
    final StreamingExchangeService service = exchange.getStreamingExchangeService(new OkCoinExchangeStreamingConfiguration());

    Thread consumer = new Thread("consumer") {

      @Override
      public void run() {

        while (!isInterrupted()) {
          try {
            ExchangeEvent event = service.getNextEvent();

            if(event != null) {
              System.out.println("---> "  + event.getPayload() + " " + event.getEventType());

              if(event.getEventType().equals(ExchangeEventType.TICKER)) {
                Ticker ticker = (Ticker) event.getPayload();

                long x = (new Date()).getTime() - ticker.getTimestamp().getTime();
                System.out.println("Delay " + x);
              }

            }

          } catch (InterruptedException e) {
            this.interrupt();
          }
        }
      }

    };

    // Start consumer.
    consumer.start();

    // Start streaming service.
    service.connect();

    // Demonstrate for 30 seconds.
    TimeUnit.SECONDS.sleep(30);

    // Disconnect streaming service.
    service.disconnect();

    // Interrupt consumer to exit.
    consumer.interrupt();
  }

}

are you using develop? Anything but develop will not work with switching to international

Yes, I'm using develop. I've added Intl demos as use cases if you're interested in documenting both sides more easily (or maybe you'd prefer to have them as unit tests. Your pick)

Ok, I got the problem:
  public OkCoinExchangeStreamingConfiguration() {
    marketDataCurrencyPairs = new CurrencyPair[]{ CurrencyPair.BTC_CNY };
  }

  public OkCoinExchangeStreamingConfiguration(CurrencyPair[] marketDataCurrencyPairs) {
    this.marketDataCurrencyPairs = marketDataCurrencyPairs;
  }

By default, and regardless of the ExchangeSpecification, the CurrencyPair defined for the ExchangeStreamingService is BTC_CNY. In order to get around this, one can pass the currency pair to use. This is ugly, but works:
package com.xeiam.xchange.examples.okcoin.streaming;

import java.util.Date;
import java.util.concurrent.TimeUnit;

import com.xeiam.xchange.Exchange;
import com.xeiam.xchange.ExchangeFactory;
import com.xeiam.xchange.ExchangeSpecification;
import com.xeiam.xchange.currency.CurrencyPair;
import com.xeiam.xchange.dto.marketdata.Ticker;
import com.xeiam.xchange.okcoin.OkCoinExchange;
import com.xeiam.xchange.okcoin.service.streaming.OkCoinExchangeStreamingConfiguration;
import com.xeiam.xchange.service.streaming.ExchangeEvent;
import com.xeiam.xchange.service.streaming.ExchangeEventType;
import com.xeiam.xchange.service.streaming.StreamingExchangeService;

public class OkCoinIntlSocketIODemo {

  public static void main(String[] args) throws InterruptedException {

    ExchangeSpecification exSpec = new ExchangeSpecification(OkCoinExchange.class);
    exSpec.setSecretKey("aa");
    exSpec.setApiKey("bb");

    //Force intl
    exSpec.setExchangeSpecificParametersItem("Use_Intl", true);

    Exchange exchange = ExchangeFactory.INSTANCE.createExchange(exSpec);

    //Forcing the currency pair is required for Intl
    final StreamingExchangeService service = exchange.getStreamingExchangeService(new OkCoinExchangeStreamingConfiguration(new CurrencyPair[]{ CurrencyPair.BTC_USD }));

    Thread consumer = new Thread("consumer") {

      @Override
      public void run() {

        while (!isInterrupted()) {
          try {
            ExchangeEvent event = service.getNextEvent();

            if(event != null) {
              System.out.println("---> "  + event.getPayload() + " " + event.getEventType());

              if(event.getEventType().equals(ExchangeEventType.TICKER)) {
                Ticker ticker = (Ticker) event.getPayload();

                long x = (new Date()).getTime() - ticker.getTimestamp().getTime();
                System.out.println("Delay " + x);
              }

            }

          } catch (InterruptedException e) {
            this.interrupt();
          }
        }
      }

    };

    // Start consumer.
    consumer.start();

    // Start streaming service.
    service.connect();

    // Demonstrate for 30 seconds.
    TimeUnit.SECONDS.sleep(30);

    // Disconnect streaming service.
    service.disconnect();

    // Interrupt consumer to exit.
    consumer.interrupt();
  }

}

I'm submitted the additional unit tests and an appropriate demo in a PR

authors: glabmoris jheusser

repo: timmolter/XChange


text:
ERROR - 422 Unprocessable Entity

Both in my own config AND in the one provided at "inovex blog" / Foreman Provision: A Comfortable Resource Management Tool for Foreman, I receive an error like this:
# ./bin/foreman-provision --auth_credentials=conf/foreman.yaml --config_file=conf/config.yaml --verbose
I, [2016-11-06T15:54:18.160777 #26382]  INFO -- : Creating ForemanProvision::Subnet "default"
I, [2016-11-06T15:54:18.819490 #26382]  INFO -- : Creating ForemanProvision::Domain "vagrant.inovex.de"
I, [2016-11-06T15:54:19.143171 #26382]  INFO -- : Creating ForemanProvision::Hostgroup "Test-Hosts"
I, [2016-11-06T15:54:20.550855 #26382]  INFO -- : Creating ForemanProvision::Host "test1.vagrant.inovex.de"
E, [2016-11-06T15:54:20.898614 #26382] ERROR -- : Error while processing provisoning: "#<RestClient::UnprocessableEntity: 422 Unprocessable Entity>"!
/usr/local/share/gems/gems/rest-client-2.0.0/lib/restclient/abstract_response.rb:223:in `exception_with_response'
/usr/local/share/gems/gems/rest-client-2.0.0/lib/restclient/abstract_response.rb:103:in `return!'
/usr/local/share/gems/gems/rest-client-2.0.0/lib/restclient/request.rb:860:in `process_result'
/usr/local/share/gems/gems/rest-client-2.0.0/lib/restclient/request.rb:776:in `block in transmit'
/usr/share/ruby/net/http.rb:852:in `start'
/usr/local/share/gems/gems/rest-client-2.0.0/lib/restclient/request.rb:766:in `transmit'
/usr/local/share/gems/gems/rest-client-2.0.0/lib/restclient/request.rb:215:in `execute'
/usr/local/share/gems/gems/rest-client-2.0.0/lib/restclient/request.rb:52:in `execute'
/usr/local/share/gems/gems/rest-client-2.0.0/lib/restclient/resource.rb:67:in `post'
/usr/local/share/gems/gems/foreman_api-0.1.11/lib/foreman_api/base.rb:78:in `http_call'
/usr/local/share/gems/gems/foreman_api-0.1.11/lib/foreman_api/base.rb:61:in `perform_call'
/usr/local/share/gems/gems/foreman_api-0.1.11/lib/foreman_api/resources/host.rb:63:in `create'
/root/develop/foreman_provision/lib/foreman_provision/base_resource.rb:13:in `create'
/root/develop/foreman_provision/lib/foreman_provision/host.rb:90:in `create'
/root/develop/foreman_provision/lib/foreman_provision/provision.rb:52:in `manage_resource'
/root/develop/foreman_provision/lib/foreman_provision/provision.rb:268:in `block in run'
/root/develop/foreman_provision/lib/foreman_provision/provision.rb:230:in `each'
/root/develop/foreman_provision/lib/foreman_provision/provision.rb:230:in `run'
./bin/foreman-provision:55:in `<main>'
Error while processing provisoning: "#<RestClient::UnprocessableEntity: 422 Unprocessable Entity>"!

Running in test-mode shows no possible issue(s):
I, [2016-11-06T15:58:30.345649 #26727]  INFO -- : Running provision in test mode
I, [2016-11-06T15:58:30.470354 #26727]  INFO -- : Skipping - ForemanProvision::Subnet "default" already exists!
I, [2016-11-06T15:58:30.551276 #26727]  INFO -- : Skipping - ForemanProvision::Domain "vagrant.inovex.de" already exists!
I, [2016-11-06T15:58:30.651724 #26727]  INFO -- : Skipping - ForemanProvision::Hostgroup "Test-Hosts" already exists!
I, [2016-11-06T15:58:30.745355 #26727]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a1", :name=>"test1.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-06T15:58:30.838915 #26727]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a2", :name=>"test2.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-06T15:58:30.932525 #26727]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a3", :name=>"test3.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-06T15:58:31.056828 #26727]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a4", :name=>"test4.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-06T15:58:31.149408 #26727]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a5", :name=>"test5.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-06T15:58:31.247472 #26727]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a6", :name=>"test6.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-06T15:58:31.343319 #26727]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a7", :name=>"test7.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-06T15:58:31.436212 #26727]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a8", :name=>"test8.vagrant.inovex.de", :ensure=>"present"}

Any clues?
Tested with commit 5944904 from Oct. 27th 2015 on Foreman 1.13.

Hey there, foreman_provision is over a year old and uses api version v1. I also think, there were serveral breaking changes in the foreman api between Oct 27th 2015 and Foreman 1.13

@hbokh : At least host creation should still work with Foreman 1.13. The error might be, that Foreman cannot save the host because it is invalid for whatever reason. I just pushed a commit, which improves the error handling. Can you try it with this code? It should show, what's going on.

Pulled the latest commit (commit 765b33a) and first ran in test-mode:
# ./bin/foreman-provision --auth_credentials=conf/foreman.yaml --config_file=conf/config.yaml --verbose --test
I, [2016-11-11T09:49:47.424929 #13545]  INFO -- : Running provision in test mode
I, [2016-11-11T09:49:47.545000 #13545]  INFO -- : Skipping - ForemanProvision::Subnet "default" already exists!
I, [2016-11-11T09:49:47.665067 #13545]  INFO -- : Skipping - ForemanProvision::Domain "vagrant.inovex.de" already exists!
I, [2016-11-11T09:49:47.790837 #13545]  INFO -- : Skipping - ForemanProvision::Hostgroup "Test-Hosts" already exists!
I, [2016-11-11T09:49:47.890719 #13545]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a1", :name=>"test1.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-11T09:49:47.989834 #13545]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a2", :name=>"test2.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-11T09:49:48.090838 #13545]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a3", :name=>"test3.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-11T09:49:48.213271 #13545]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a4", :name=>"test4.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-11T09:49:48.309284 #13545]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a5", :name=>"test5.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-11T09:49:48.402268 #13545]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a6", :name=>"test6.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-11T09:49:48.493537 #13545]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a7", :name=>"test7.vagrant.inovex.de", :ensure=>"present"}
I, [2016-11-11T09:49:48.587898 #13545]  INFO -- : Would have created ForemanProvision::Host {:build=>true, :hostgroup=>"Test-Hosts", :mac=>"06:10:9f:dc:aa:a8", :name=>"test8.vagrant.inovex.de", :ensure=>"present"}

Now the non-test at least does not show the ruby-errors but is still failing on the inovex example-config:
# ./bin/foreman-provision --auth_credentials=conf/foreman.yaml --config_file=conf/config.yaml --verbose
I, [2016-11-11T09:49:57.515356 #13552]  INFO -- : Skipping - ForemanProvision::Subnet "default" already exists!
I, [2016-11-11T09:49:57.597794 #13552]  INFO -- : Skipping - ForemanProvision::Domain "vagrant.inovex.de" already exists!
I, [2016-11-11T09:49:57.703916 #13552]  INFO -- : Skipping - ForemanProvision::Hostgroup "Test-Hosts" already exists!
I, [2016-11-11T09:49:57.797885 #13552]  INFO -- : Creating ForemanProvision::Host "test1.vagrant.inovex.de"
E, [2016-11-11T09:49:58.160819 #13552] ERROR -- : HTTP Error: {
  "error": {"id":null,"errors":{"operatingsystem_id":["can't be blank"]},"full_messages":["Operatingsystem can't be blank"]}
}

@hbokh : Please check, that the Host actually has an operating system set. I don't know the example, but judging from a brief look you might want to check if the OS is correctly set on the Hostgroup "Test-Hosts".

Danke Timo. You can check the mentioned example over here: Foreman Provision: A Comfortable Resource Management Tool for Foreman.
As you can see operatingsystem is defined in :hostgroups: as CentOS 7.0 and the hostgroup is included for every entry under :hosts:.

Yes, but that doesn't necessarily mean that the OS actually exists. Please check in the Foreman UI that the hostgroup really has the OS set.

Never thought about that in the first place...
Working fine now:
# ./bin/foreman-provision --auth_credentials=conf/foreman.yaml --config_file=conf/config.yaml --verbose
I, [2016-11-11T13:02:54.554932 #20761]  INFO -- : Skipping - ForemanProvision::Subnet "default" already exists!
I, [2016-11-11T13:02:54.667180 #20761]  INFO -- : Skipping - ForemanProvision::Domain "vagrant.inovex.de" already exists!
I, [2016-11-11T13:02:54.807218 #20761]  INFO -- : Skipping - ForemanProvision::Hostgroup "Test-Hosts" already exists!
I, [2016-11-11T13:02:54.932710 #20761]  INFO -- : Creating ForemanProvision::Host "test1.vagrant.inovex.de"
I, [2016-11-11T13:02:55.620799 #20761]  INFO -- : Creating ForemanProvision::Host "test2.vagrant.inovex.de"
I, [2016-11-11T13:02:56.249920 #20761]  INFO -- : Creating ForemanProvision::Host "test3.vagrant.inovex.de"
I, [2016-11-11T13:02:56.824534 #20761]  INFO -- : Creating ForemanProvision::Host "test4.vagrant.inovex.de"
I, [2016-11-11T13:02:57.412967 #20761]  INFO -- : Creating ForemanProvision::Host "test5.vagrant.inovex.de"
I, [2016-11-11T13:02:58.057071 #20761]  INFO -- : Creating ForemanProvision::Host "test6.vagrant.inovex.de"
I, [2016-11-11T13:02:58.641013 #20761]  INFO -- : Creating ForemanProvision::Host "test7.vagrant.inovex.de"
I, [2016-11-11T13:02:59.211364 #20761]  INFO -- : Creating ForemanProvision::Host "test8.vagrant.inovex.de"

I'll try with my own config next.
Vielen dank!

source: gharchive/issue

authors: hbokh marcohelmerich timogoebel

license: MIT

license_type: permissive

license_source: github-api


text:
Fix For React Navigation Beta 20+

This seems to fix the issue people are having with React Navigation Beta.20+. Just need to pass in previousScene.

Thank you very much! I'll check this out at the weekend

@timomeh sorry man, should have caught that. I am equally as picky in my own code reviews.

Will release it soon

source: gharchive/pull-request

authors: keeleycarrigan timomeh

license: mit

license_source: bigquery


text:
Replaced the --enforce-whitespace option with --ignore-whitespace

Restores original behavior of enforcing whitespace by default. The
behavior was broken in a backwards incompatible way starting with commit
5f6d4bd45e0fa8e3b6b2bea81d3eb7368ee312c3. Ignoring whitespace is now
opt-in instead of opt-out.
Ignoring whitespace by default has lead to a number of confused users,
myself included. CI would seemingly pass but isort --apply would still
make changes. Some users have asked for the original behavior be
restored and the new feature be behind a CLI argument.
Refs #378
Fixes #423
Fixes #481

Rebased to resolve merge conflicts

Looks good! Will merge as soon as travis finishes.
Thanks!

authors: jdufresne timothycrosley

repo: timothycrosley/isort


text:
Rename zh_CN.lang to zh_cn.lang

The file zh_CN.lang must be lowercase, otherwise it will lead to English

Thanks for the help!

authors: HaeDon timrwood

repo: timrwood/roost


text:
Unable to share folders under Parallels

Box is based on 10.11.5 (El Capitan) (I followed https://spin.atomicobject.com/2015/11/17/vagrant-osx/ to the letter)
Tried using nfs, as well as the default
Vagrant messages seem to indicate the folders were mounted successfully...

    ==> default: Mounting shared folders...
    default: /vagrant => /Users/darrenw/Projects/####/#####
    default: /var/cache/apt/archives => /Users/darrenw/Projects/###/####/cache/apt/archives

The following interesting log entries are in /var/log/system.log:
com.apple.xpc.launchd[1] (com.parallels.vm.prl_fsd): Service only ran for 0 seconds. Pushing respawn out by 10 seconds.

...however, the process did start up by the time I logged in...
local:~ vagrant$ ps -ef | grep prl
    0    79     1   0  9:04PM ??         0:00.07 /Library/Parallels Guest Tools/prltoolsd
    0   181     1   0  9:04PM ??         0:00.06 /Library/Parallels Guest Tools/prl_fsd /Volumes/SharedFolders --share

If I open the GUI and try sharing a folder, it appears to work but, again, nothing actually gets shared. This might mean it's a Parallels issue, so I'll try installing directly (without osx-vm-templates) to see if that makes a difference.

Sharing worked fine a regular Parallels/OSX guest. The problem is that when the VM is created, the 'Isolate Mac from virtual machine' option is turned on. Turning this off immediately exposes the shared folders.
Can this option be turned off when packer creates the image?

Closing as this was fixed in #66.

authors: dazwin timsutton

repo: timsutton/osx-vm-templates


text:
Eleventy integration guide?

Howdy!
TinaCMS looks super rad and it looks like it would scratch a current itch as a git-backed CMS for various Eleventy (11ty) projects I have been working on. While I haven't looked into it deeply enough at this point, is this integration possible?
Also, I would be willing to help document that process with a tiny helping hand from the community where needed.
Thanks again for your hard work!

Doh! Of course it does! It says react-based right on the tin.
I have used alternatives in the past, but while they're awesome, they just weren't suitable for the arrangement that my team and I had envisioned, namely not requiring external accounts/management for small internal projects.
I suppose it'd make more sense to close this task and open a new ticket with a more generalized direction, e.g., Add Support for Non-React Based Applications, or whatever :stuck_out_tongue:

I have some ideas on how we can make it work with other static site generators. So that ticket would be most welcomed.

Tina relies on React High Order Components

It says react-based right on the tin.

We're going to have to update our docs a bit. Neither of these statements are correct.
TinaCMS is not React based, we just focused on React integration first. Sure, the sidebar is React, but the core of the CMS doesn't know anything about React. As far as the core of Tina is concerned the sidebar is just another consumer, like the rest of your site.
react-tinacms is just provides some helpers for accessing the CMS and using it in React components, but React is by no means required to interact with the CMS on your site.
gatsby-tinacms-remark provides the remarkForm higher order components because Gatsby is React based and that's the API that makes senses.
The way Tina is built, it's already totally possible to add other adapter libraries (e.g. vue-tinacms) for interacting with the cms .

Thank you for the clarification @ncphillips . I'll take a look at the implementations and report back with progress/issues.

@ncphillips I see you closed two issues related to this.
Do you mean TinaCMS currently supports Eleventy?

It means we only support React integration for now, if you can use React with Eleventy, then you might be able to use Tina, but if you are to add React to Eleventy, you might as well use a React framework like NextJS then.

It means we only support React integration for now, if you can use React with Eleventy, then you might be able to use Tina, but if you are to add React to Eleventy, you might as well use a React framework like NextJS then.

This is exactly what I'm doing. I'm using NextJS with Eleventy, but now I'm considering use Tina. Any idea how to start?

authors: DirtyF jpatters matbrgz ncphillips surjithctly trst

repo: tinacms/tinacms


text:
GTK + QT templates

I've been using @Misterio77's version of the flatcolor GTK theme, and did a bit of work on extending it to use the full range of base16 colours.
As I understand, GNOME is now moving towards GTK4/Libadwaita, which has a nice small template for colour definitions that matches up fairly nicely with base16's definitions.
I haven't found any colour template for QT themes though. I did just find Oomox, and it seems to be able to manage something similar, although I haven't tried it out yet.
I'd like to consolidate methods of defining colours for these toolkits/frameworks, and then match them up to base16's definitions. Hopefully this would let the major DEs use base16 without much effort. Does anyone have any information/thoughts on this?

yup themix-base16-plugin got base16-like templates for both gtk4 and kde but they using extra variables not defined in base16, and need to be replaced to standard ones, other than that they working nowadays, but might break as soon as smth in gtk/kde changes
oomox theme itself have extras for qt5/qt6 theme compat but those templates are not base16-compatible at all, so rewriting them would be same as writing them from scratch

There's a couple of themes (one of them being materia) that support having their colors configured at compile time. Here's how we do it with nix: https://github.com/Misterio77/nix-colors/blob/main/lib/contrib/gtk-theme.nix#L37

There's a couple of themes

list of the themes would be more useful in this context rather than your nix-code for generating oomox-file format config for building Materia

I don't know which one exactly. Just sharing that it's possible without oomox - just bash is enough.

you just generating themix/oomox config file, idk if you're clueless about it or purposely undermining the effort of our project there

idk if you're clueless about it or purposely undermining the effort of our project there
That's very rude. As I said, I was just sharing that it's possible to replace variables & rebuild themes. Themes that are replaceble without recompiling (such as flatcolor) are very limited.

but any theme could be recolored by a simple sed command.

sure, do it
but while you do that was done by themix/oomox project - don't pretend like it's your work - that's what rude, not pointing that out

There's a couple of themes

so what are those mysterious "couple of themes" which are not related to Oomox, already mentioned in the first message?

I've put together template for GTKSourceView for gedit and stuffs: https://github.com/RobLoach/base16-gtksourceview
Should I just put a link to it in the readme? Happy to transfer it to this org too.

@RobLoach nice work 👍 add it also to https://raw.githubusercontent.com/chriskempson/base16-templates-source/master/list.yaml so it would be accessible for Classic Base16 users as well

repo: tinted-theming/home


text:
LLaMA fails since 0fd44259 on M2

With the commit 0fd44259 (#2698) the LLaMA example broke.
command: python examples/llama.py --prompt "Hello." --count 10 --temperature 0 --timing
output:
using METAL backend
using LLaMA-7B model
ram used:  0.00 GB, layers.0.attention.wq.weight                      :   0%|                                                                                | 0/292 [00:00<?, ?it/s]
loaded weights in 48.52 ms, 0.03 GB loaded at 0.69 GB/s
Traceback (most recent call last):
  File "/.../tinygrad/examples/llama.py", line 373, in <module>
    llama = LLaMa.build(MODEL_PATH, TOKENIZER_PATH, model_gen=args.gen, model_size=args.size, quantize=args.quantize)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../tinygrad/examples/llama.py", line 174, in build
    load_state_dict(model, weights, strict=False)
  File "/.../tinygrad/tinygrad/nn/state.py", line 68, in load_state_dict
    v.assign(state_dict[k].to(v.device)).realize()
  File "/.../tinygrad/tinygrad/tensor.py", line 105, in realize
    run_schedule(self.lazydata.schedule())
  File "/.../tinygrad/tinygrad/realize.py", line 28, in run_schedule
    prg = lower_schedule_item(si)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../tinygrad/tinygrad/realize.py", line 19, in lower_schedule_item
    return Device[si.out.device].get_runner(si.ast)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../tinygrad/tinygrad/device.py", line 314, in get_runner
    def get_runner(self, ast:LazyOp) -> CompiledASTRunner: return self.to_program(self.get_linearizer(ast))
                                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../tinygrad/tinygrad/device.py", line 286, in to_program
    return CompiledASTRunner(k.ast, k.name, src, k.global_size, k.local_size, runtime_args).build(self.compiler, self.runtime)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../tinygrad/tinygrad/device.py", line 254, in build
    self.lib = compiler.__wrapped__(self.prg) if getenv("DISABLE_COMPILER_CACHE") else compiler(self.prg)
                                                                                ^^^^^^^^^^^^^^^^^^
  File "/.../tinygrad/tinygrad/helpers.py", line 260, in wrapper
    return diskcache_put(table, key, func(*args, **kwargs))
                                     ^^^^^^^^^^^^^^^^^^^^^
  File "/.../tinygrad/tinygrad/runtime/ops_metal.py", line 18, in compile_metal
    library = unwrap2(MetalDevice.compiler_device.newLibraryWithSource_options_error_(prg, options, None))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../tinygrad/tinygrad/helpers.py", line 45, in unwrap2
    assert err is None, str(err)
AssertionError: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:3:55: error: invalid type 'const device __bf16 *' for buffer declaration
kernel void E_131072_32_4(device unsigned int* data0, const device __bf16* data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) {
                                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:3:68: note: type '__bf16' cannot be used in buffer pointee type
kernel void E_131072_32_4(device unsigned int* data0, const device __bf16* data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) {
                                                                   ^
" UserInfo={NSLocalizedDescription=program_source:3:55: error: invalid type 'const device __bf16 *' for buffer declaration
kernel void E_131072_32_4(device unsigned int* data0, const device __bf16* data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) {
                                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:3:68: note: type '__bf16' cannot be used in buffer pointee type
kernel void E_131072_32_4(device unsigned int* data0, const device __bf16* data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) {
                                                                   ^
}

i think this has been fixed on master. can you verify?

Not fixed unfortunately.

what's the error you are getting? is it the same as https://github.com/tinygrad/tinygrad/issues/2226? or still something related to bf16?

+1. M2 broken for coder.py starting 0fd44259
looking at the diff, I guess M2 doesn't support bfloat16.  Over my ski's here.  Hope this helps.
create model: 145.43 ms
download weights: 75.65 ms
ram used:  0.00 GB, layers.0.attention.wq.weight                      :   0%| |
loaded weights in 75.73 ms, 0.03 GB loaded at 0.44 GB/s
weights -> model: 111.40 ms
Traceback (most recent call last):
File "/Users/martin/working/tinygrad/examples/coder.py", line 37, in
nn.state.load_state_dict(model, fix_bf16(convert_from_huggingface(part1, model, 32, 8)), strict=False)
File "/Users/martin/working/tinygrad/tinygrad/nn/state.py", line 60, in load_state_dict
v.assign(state_dict[k].to(v.device)).realize()
File "/Users/martin/working/tinygrad/tinygrad/tensor.py", line 105, in realize
run_schedule(self.lazydata.schedule())
File "/Users/martin/working/tinygrad/tinygrad/realize.py", line 28, in run_schedule
prg = lower_schedule_item(si)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/martin/working/tinygrad/tinygrad/realize.py", line 19, in lower_schedule_item
return Device[si.out.device].get_runner(si.ast)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/martin/working/tinygrad/tinygrad/device.py", line 308, in get_runner
def get_runner(self, ast:LazyOp) -> CompiledASTRunner: return self.to_program(self.get_linearizer(ast))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/martin/working/tinygrad/tinygrad/device.py", line 280, in to_program
return CompiledASTRunner(k.ast, k.name, src, k.global_size, k.local_size, runtime_args).build(self.compiler, self.runtime)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/martin/working/tinygrad/tinygrad/device.py", line 249, in build
self.lib = compiler.wrapped(self.prg) if getenv("DISABLE_COMPILER_CACHE") else compiler(self.prg)
^^^^^^^^^^^^^^^^^^
File "/Users/martin/working/tinygrad/tinygrad/helpers.py", line 240, in wrapper
return diskcache_put(table, key, func(args, kwargs))
^^^^^^^^^^^^^^^^^^^^^
File "/Users/martin/working/tinygrad/tinygrad/runtime/ops_metal.py", line 18, in compile_metal
library = unwrap2(MetalDevice.compiler_device.newLibraryWithSource_options_error_(prg, options, None))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/martin/working/tinygrad/tinygrad/helpers.py", line 45, in unwrap2
assert err is None, str(err)
^^^^^^^^^^^
AssertionError: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:3:57: error: invalid type 'const device __bf16 ' for buffer declaration
kernel void E_512_64_8_16_4(device unsigned int data0, const device __bf16 data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:3:70: note: type '__bf16' cannot be used in buffer pointee type
kernel void E_512_64_8_16_4(device unsigned int data0, const device __bf16 data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) {
^
" UserInfo={NSLocalizedDescription=program_source:3:57: error: invalid type 'const device __bf16 ' for buffer declaration
kernel void E_512_64_8_16_4(device unsigned int data0, const device __bf16* data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
program_source:3:70: note: type '__bf16' cannot be used in buffer pointee type
kernel void E_512_64_8_16_4(device unsigned int* data0, const device __bf16* data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) {
^
}

what's the error you are getting? is it the same as #2226? or still something related to bf16?

no its not the same issue, it seems to be the issue the bf16-cast as @soitgoes mentioned.

please reopen if #3774 does not fix the issue for you

@chenyuxyz seems not to be fixed. Is some of the C-code cached before compilation? Happy to check again

oh i only fixed coder in my earlier, missed that llama has the treatment. should be fixed after #3789. If your device does not support bfloat16 buffer, you need to add SUPPORT_BF16=0 in the command

Indeed fixed on my Mac using the command: SUPPORT_BF16=0 python examples/llama.py --prompt "Hello."

authors: chenyuxyz eichenroth soitgoes

repo: tinygrad/tinygrad


text:
fix: fix browser support

closes #171

Alternative could be to feature detect using something like this:
  if ((globalThis as any).process?.hrtime?.bigint) {
    //
  }

Thanks for the quick fix!

Alternative could be to feature detect using something like this:

I plan to add the JS runtime as a Bench property later on. And also add it to the results object.

With v3.0.1 the error is:

Uncaught TypeError: Cannot read properties of undefined (reading 'versions')

With v3.0.1 the error is:

Uncaught TypeError: Cannot read properties of undefined (reading 'versions')

Fixed in 3.0.3. The linter removed overzealously null condition checks.

authors: dasa jerome-benoit

repo: tinylibs/tinybench


text:
Swipe Card in single Direction

i need swipe Card in down direction only. How can i achieve that?
Also it is crashing when swipe up direction: error: fatal error: Can't form Range with upperBound < lowerBound

let newCenterX = initialCellCenter!.x Chnages and worked

@Pitsko : Please Help

I will try to look in this issue in a few next days

Crash is fixed in 1.0.1
So you need drag and drop + swift just to down direction
@AbhishekNFX please confirm if you still need it ?, sorry for big delay

authors: AbhishekNFX Pitsko

repo: tispr/tispr-card-stack


text:
Nib automatically imports 'text.styl' files in a directory

Steps to reproduce
Create two files in the same directory, a.styl and text.styl:
// a.styl
@import 'nib'

// text.styl
*
  color red

On the command line, enter
stylus -u nib -p a.styl

This prints
* {
  color: #f00;
}

rather than nothing (which I would expect).
Cause
lib/nib/index.styl imports several other files, among them @import 'text'. However, there is no text.styl file, instead there is a text/index.styl file. Presumably the import algorithm looks for more $name.styl files before looking for $name/index.styl files.

I had a similar issue with a lib having a normalize.styl which is inappropriately imported by nib.
Fixed it for my project at F4-Group/nib@9cfa2fa24685259c8004d90a5e9d5565cdf8af7a

repo: tj/nib


text:
🛑 cloud.tjo.space is down

In b45ae4e, cloud.tjo.space (https://cloud.tjo.space/status.php) was down:

HTTP code: 0
Response time: 0 ms

Resolved: cloud.tjo.space is back up in dd760cd after 8 days, 20 minutes.

repo: tjo-space/status


text:
💥 Update data types names and use single code

Breaking changes.

DataSeries has been renamed to TimeSeries
DataTable has been renamed to Table
Both TimeSeries and Table types now contain a single code string

Codecov Report

Merging #1 into master will increase coverage by 1.38%.
The diff coverage is 83.33%.

@@            Coverage Diff             @@
##           master       #1      +/-   ##
==========================================
+ Coverage   77.77%   79.16%   +1.38%
==========================================
  Files           3        3
  Lines          45       48       +3
==========================================
+ Hits           35       38       +3
  Misses         10       10

Impacted Files
Coverage Δ

src/QuandlAccess.jl
100.00% <ø> (ø)

src/api.jl
78.57% <75.00%> (+1.64%)
:arrow_up:

src/types.jl
80.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1d8b5a9...7513049. Read the comment docs.


text:
fix: base64

Web tkey encode in  url safe no padding base64 .
fix the rust to support both standard and url safe no padding format

https://github.com/torusresearch/tkey-rust/pull/171
correlated pr

We need to find the flakiness in the following test:
integrationTests.test_TssModule_multi_tag()

This can be merged.

authors: ieow metalurgical

repo: tkey/tkey-mpc-swift


text:
Jordan/import export react components

WIP Import/Export React Components + iFrame SDK Client Change

I also saw that R was M coz I like wood

repo: tkhq/sdk


text:
feat: implement commit-msg hook (#45)

This pull request implements a commit-msg hook to verify that commit messages are following the conventional commit best practices. It is calling a new target git-verify that is also capable of validate a full commit log as well as the list of new commits provided by a branch.
The pull request also restructures the organization and availability of some default variables in Makefile.base to improve maintainability, customizabillity, and testability.

Pull Request Test Coverage Report for Build 7733812834

0 of 9 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 7693314872:
0.0%

Covered Lines:
294

Relevant Lines:
294

💛  - Coveralls

authors: coveralls tkrop

repo: tkrop/go-make


text:
没看到源码

请问下这个是开源的吗？

还在规划 @windyrainy

请问下这个是开源的吗？

建议你去看看这个项目 gojue/ecapture

authors: editso ufnslab windyrainy

repo: tlsdump/tlsdump


text:
[1.4] Add macroable trait to service classes for easy extension

This allows quick extension of the base Service classes using macros in your AppServiceProvider. Instead of having to wire it all up yourself in multiple places or submit a PR to get the method added, you can easily add your own methods to the Service classes.
This would also utilize the awesome auth that is already in place meaning you don't have to recreate all of that.
Usage example:
Service::macro('addUetTags', function($tags){
    $serviceCall = $this->call(ServiceClientType::CampaignManagementVersion13);

    try {
        $requestTags = [];
        foreach($tags as $tag) {
            $tag = new UetTag();
            foreach($tag as $key => $value) {
                $tag->{$key}  = $value;
            }

            array_push($requestTags,$tag);
        }

        $request = new AddUetTagsRequest();
        $request->UetTags = $requestTags;

        $serverResponse = $serviceCall->GetService()->AddUetTags($request);

        return $serverResponse;
    } catch (\Exception $e) {
        print $serviceCall->GetService()->__getLastRequest()."\n";
        print $serviceCall->GetService()->__getLastResponse()."\n";
    }
});

This is great! Thanks so much. Everything looks good so far. I'm going to push it this weekend.

authors: modernben timothymarois

repo: tmarois/laravel-ads-sdk


text:
--maxcolseps 0 has no effect; ocropus-gpageseg still performs whitespace column segmentation

Expected Behavior

I'm expecting ocropus-gpageseg to split the below image into rows without doing any column segmentation. To this effect I'm specifying the arguments --maxcolseps 0 --maxseps 0 (of course the latter is default.)
The command even registers that I have specified 0 as the value of maxcolseps:
root@acf1f5c7b5aa:/tmp# ocropus-gpageseg -d --maxcolseps=0 --maxseps=0 5823821_0.png
INFO:
INFO:  ########## /usr/local/bin/ocropus-gpageseg -d --maxcolseps=0 --maxseps=
INFO:
INFO:  5823821_0.png
INFO:  scale 21.977261
INFO:  computing segmentation
INFO:  computing column separators
INFO:  considering at most 0 whitespace column separators
INFO:  debug _1thresh.png
INFO:  debug _2grad.png
INFO:  debug _3seps.png
INFO:  debug _4seps.png
INFO:  debug _colwsseps.png
INFO:  computing lines
INFO:  debug _cleaned.png
INFO:  debug _lineseeds.png
INFO:  debug _seeds.png
INFO:  propagating labels
INFO:  spreading labels
INFO:  number of lines 200
INFO:  finding reading order
INFO:  writing lines
INFO:     195  5823821_0.png 22.0 196

Current Behavior

However, ocropus-gpageseg is doing column separation.
Here are a few examples of rows which should be full bleed, but instead they have been broken into whitespace-separated columns first, then into rows:

Possible Solution

Unsure!
Steps to Reproduce

Image: 5823821_0.png

ocropus-nlbin 5823821_0.png
ocropus-gpageseg -d --maxcolseps 0 5823821_0.png
Or: ocropus-gpageseg -d --maxcolseps 0 --maxseps 0 5823821_0.png

Your Environment

Python version: Python 2.7.9
Python dependency versions:
In case it's helpful, I've also included all docker build output, which indicates the versions of the various dependencies in my Docker container: https://gist.github.com/freen/bae76e580237c9f2ec4cf4bdd022e4bd
Git revision of ocropy:

root@acf1f5c7b5aa:/app/ocropy# git log -n1
commit ae84a8edaf0b76135f749ba66fc30c272d0726d0
Merge: 3b843f5 7eac431
Author: Tom Breuel <tmbdev@gmail.com>
Date:   Fri Aug 11 12:31:33 2017 -0700

    Merge branch 'master' of github.com:tmbdev/ocropy

Operating System and version:
Debian GNU/Linux 8.6 (jessie)

As an update: it looks like --hscale=100 does the trick. That being said the documentation for --maxcolseps 0 definitely seems to indicate that additional parameters are unneeded for this effect.

The --maxcolseps parameter works here as expected and some of the lines are already taken the full width:

However, that other lines are not taking the full width, that is connected to the internals of the compute_line_seeds-function. AFAIK it kind of smears over the text and will allow some space before and after. Thus, if two text blocks are horizontally not too far away, then they will be connected. Otherwise they will be saved in two separate images.
I see that this is not the desired output in your example. I don't know the reason or details for the implementation of function. Your observation with --hscale is interesting. I have to admit that I don't know what this parameter is doing overall. Do you understand what this parameter is doing exactly? More or improved documentation would always be welcome here.

Thanks for your reply, @zuphilip.

Your observation with --hscale is interesting. I have to admit that I don't know what this parameter is doing overall. Do you understand what this parameter is doing exactly? More or improved documentation would always be welcome here.

I don't know what the actual algorithmic effect of the parameter is, I just arrived at this outcome by experimenting with the various command line options. Sorry I can't provide more info.

authors: freen zuphilip

repo: tmbdev/ocropy


text:
🛑 Homepage is down

In 6dcf6c9, Homepage (https://hcmus.edu.vn/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Homepage is back up in 05b9dfa.

repo: tmkhiem/hcmus-web-uptime


text:
Correct the logic as to uppercase if NOT xhtml AND html.

Note that currently this breaks many, many tests.
From MDN:
In XML (and XML-based languages such as XHTML), tagName preserves case.
On HTML elements in DOM trees flagged as HTML documents, tagName returns
the element name in the uppercase form. The value of tagName is the same
as that of nodeName.

Hey @tmpvar , I came across this issue because using htmlparser2 means this.namespaceURI is always null, and hence document.body which depends upon the case of tagName being upper case breaks.

This is not correct. MDN is not a spec, and I think you are reading it wrong anyway. Regardless, the spec is at https://dom.spec.whatwg.org/#dom-element-tagname and step 2 does what the current code does (which is the opposite of what your change does). That is why you are breaking all the tests.

OK @domenic thanks for pointing me to that. I guess the confusion comes that the HTML namespace is http://www.w3.org/1999/xhtml which is somewhat confusing :)
The bug here then would seem to be that when using the htmlparser2 parser the namespace is not being set, i.e. it is null. This then causes the tagName to be lowercase, which breaks all the methods that depend on the value being upper case such as document.body.

Well, in general we only support parse5, because of issues exactly like this; we are working on removing htmlparser2 support entirely for the next release, so in the meantime I guess it's deprecated.

OK, thanks for the info. Is there a ticket for that? The performance of parser5 seems to be pretty poor, I guess if pluggable parsers are being removed I would be keen to understand if there's another extension mechanism.

#1316. In general we find parsing performance to be the least important part of jsdom performance, and the correctness improvements are drastic (just look at the massive number of issues closed by introducing parse5: https://github.com/tmpvar/jsdom/issues?q=is%3Aissue+label%3Aparsing+is%3Aclosed). But parse5 always accepts pull requests.
The alternative in the future will be for other parsers to create shim interfaces that behave exactly like parse5's, so they can pretend to be parse5. That's a ways off though.

@mjgp2

The performance of parser5 seems to be pretty poor,

Nah, it's just fine (should be compared somehow with htmlparser2-dom):
gumbo-parser   : 34.9208 ms/file ± 21.4238
html-parser    : 24.8224 ms/file ± 15.8703
html5          : 419.597 ms/file ± 264.265
htmlparser     : 60.0722 ms/file ± 384.844
htmlparser2-dom: 12.0749 ms/file ± 6.49474
htmlparser2    : 7.49130 ms/file ± 5.74368
hubbub         : 30.4980 ms/file ± 16.4682
libxmljs       : 14.1338 ms/file ± 18.6541
parse5         : 22.0439 ms/file ± 15.3743
sax            : 49.6513 ms/file ± 26.6032

@inikulin thanks for this, I think I was conflating html5 vs parse5 :)

authors: domenic inikulin mjgp2

repo: tmpvar/jsdom


text:
Fix copyline for tmux 2.4

With the current implementation, if your shell is :
$ echo 'foo'
your cursor is at the end of the line, it will copy $ echo 'foo' on
tmux 2.4, and echo 'foo' on tmux 2.3 and below.
This normalizes the behavior on both versions so that only echo 'foo'
is copied

Ooh. I misunderstood what that was supposed to do.
Thanks.

repo: tmux-plugins/tmux-yank


text:
Please clarify license

In setup.py, it says that the source code is licensed under MIT.
In LICENSE, license is GPL-3.0 and does not specify if it is GPL-3.0 only or if it also accepts newer versions.

AFAICT what's in the code counts, so this would be MIT.
@tobetter Please change the licence tab on the main GitHub page, Next to README.

authors: dennizzzz erdnaxe

repo: tobetter/odroid-homecloud


text:
Update the FPS counter to show a min-to-average FPS range

This makes dropped frames much more visible. In the graph 59fps
looks very similar to 60fps, but in VR it's likely to mean
a very degraded experience.

This is a great tweak, and makes the FPS counter far more useful. Thanks!

authors: klausw toji


text:
feat(console): add missing configuration

This changes supports target_addr, env_filter, and retain_for in configuration file.
❯ env -u LANG -u COLORTERM -u RUST_LOG cargo run -- gen-config
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/tokio-console gen-config`
target_addr = 'http://127.0.0.1:6669/'
env_filter = 'info'
retain_for = '6s'

[charset]
lang = 'en_us.UTF8'
ascii_only = false

[colors]
enabled = true
truecolor = true
palette = 'all'

[colors.enable]
durations = true
terminated = true

Related to #331
Note

should I use define the table in the config file, like [charset]?
is it better a field name log instead of env_filter?
what is the default value for env_filter?

I used info now.

these changes may conflict with #323

@hawkw
Thanks reviewing. I have made corrections to the areas you pointed out. Please review again.

Sorry I didn't review this earlier, I must have missed it!

Don't worry about it:v:

repo: tokio-rs/console


text:
Add PartialOrd, Ord, and Hash to EasyBuf, generalize PartialEq.

This PR adds the derivable traits to EasyBuf so that types containing it can have the usual #[derive()]s.
It also generalizes PartialEq to all T: AsRef<[u8]>.

Thanks!

authors: alexcrichton partim

repo: tokio-rs/tokio-core


text:
Facebook, Twitter のOGP画像のキャッシュを一括で削除するスクリプトを組む

改善詳細 / Details of Improvement

#774 の実装により、各グラフの個別のページができ、シェアができるようになりました。
しかし、URL は同一なため、現行仕様でいくならば、グラフが更新されたタイミングでOGP画像も更新すべきです。
現在は、facebook debugger 及び Twitter Card Validator で手動で更新をしているのですが、全てのカードを毎日更新するのは大変です。
何らかの方法で自動化させたいと考えています。

期待する見せ方・挙動 / Expected behavior

理想は、GitHub Actions に組み込めること
手動実行バッチ（シェルスクリプトなど）でも、無いよりはまし

ざっと調べた限り、Facebook は cURL で更新することが可能。ただし、アクセストークンは延長しても3ヶ月で切れるので、3ヶ月毎に更新が必要？
curl -i -X GET \
 "https://graph.facebook.com/v6.0/https://stopcovid19.metro.tokyo.lg.jp/cards/number-of-confirmed-cases/?access_token=${ACCESS_TOKEN}"

Twitter は不明

Facebook についてはこちらでも一括消去できるみたい。
https://developers.facebook.com/tools/debug/sharing/batch/

Facebook についてはこちらでも一括消去できるみたい。
https://developers.facebook.com/tools/debug/sharing/batch/

やってみたけど、キャッシュが消去されるわけではなかった。

Twitter の方は、twitter:image が更新されていれば大丈夫っぽい。
https://developer.twitter.com/en/docs/tweets/optimize-with-cards/guides/troubleshooting-cards

Facebook についてはこちらでも一括消去できるみたい。
https://developers.facebook.com/tools/debug/sharing/batch/

やってみたけど、キャッシュが消去されるわけではなかった。

#1681 の問題かも

関連して、#2002 のエラーも出ています。
APIとかを使うよりも、scrapy とか selenium とか使ってFacebook Debugger や Twitter Validator を操作するほうが早い気がしてきました。

@halsk
はじめまして。
ogp画像が更新されていることを確認する方法ってどうしたらいいでしょうか？

@halsk はじめまして。

Twitter の方は、twitter:image が更新されていれば大丈夫っぽい。

ある程度時間が立たないと、クロールし直してくれないようなので、Twitter Card Validatorなりでキャッシュを削除する手順は必要なのではないでしょうか？
https://developer.twitter.com/en/docs/tweets/optimize-with-cards/guides/troubleshooting-cards#refreshing

@t-tajiri
Facebookであれば Facebook Debugger で確認でき、Twitter では実際にtweetしようとすると分かります。

@cgetc
はい。このチケットは、その作業をいちいち手作業でやるのが面倒なので、自動化したいという Issue です。

https://github.com/tokyo-metropolitan-gov/covid19/issues/1497#issuecomment-599815823
Facebookに関しては、OGPのBugfixがマージされており、私の方で試した感じではBatch Invalidatorが正常に動作していそうです。

Twitter Card ValidatorにはAPIはないので、seleniumなどで対応するほうがいいかもしれません。
https://twittercommunity.com/t/how-to-clear-twitter-card-cache-with-api/62974
短縮URLを発行して、ツイートする方法もありますが、どちらにしても、工数がかかりそうです。

短縮URLを発行して、ツイートする方法を一旦試してみましたが、期待した動作ではありませんでした。Twitter Card Validatorは正常に動作するので、seleniumなどで対応する方法が無難そうです。
@halsk 仕事の関係もありますし、半日以上かかってしまってもいいのでしたら、私の方で対応します。

ありがとうございます！急ぎませんので、大丈夫です。

@halsk まずはTwitterの方だけでいいですか？

はい。大丈夫です

GitHub Actionsにはまだ組み込めていませんが、Mac環境では一発でまとめて手動実行できるぐらいにはできています。
https://github.com/tokyo-metropolitan-gov/covid19/pull/2240
実行するたびにログインのメールが届いてしまうので、捨てメアドでTwitterのアカウントを新規作成するのをお勧めします。

@halsk Facebookは以下のスクリプトをGithub Actionに組み込めば自動化できると思うのですが、その方向で進めたほうがいいですか？
https://github.com/tokyo-metropolitan-gov/covid19/blob/development/ui-test/delete_fbcache.sh

@cgetc このスクリプトには2つ問題がありまして、1つには、Access Token が短期間でしか使えないこと、もう一つは、API経由でページキャッシュを消そうとしたときに以下のエラーが出ることです。
{
  "error": {
    "message": "Invalid parameter",
    "type": "OAuthException",
    "code": 100,
    "error_subcode": 1611016,
    "is_transient": false,
    "error_user_title": "オブジェクトの値が無効です",
    "error_user_msg": "プロパティ「og:locale:locale」に指定された値「en」はタイプ「enum」としてパースできないため、URL「https://stopcovid19.metro.tokyo.lg.jp/en/cards/details-of-confirmed-cases/」のオブジェクト(タイプ「website」)は無効です。",
    "fbtrace_id": "Am7ZTnacPiXjpHEIUWpH5lT"
  }
}
```

Access Token は、今は下記の画面で取得できるものを使っていますが、永久には使えないようで、すぐに無効になってしまいます。

![image](https://user-images.githubusercontent.com/16509/77842864-4309ad80-71d2-11ea-83bc-0c128e53fc1f.png)

まだ試してはいないですが、無期限のアクセストークンは発行できるらしいです。
https://gist.github.com/xl1/fe779a817a9d4938193d

まだ試してはいないですが、無期限のアクセストークンは発行できるらしいです。
https://gist.github.com/xl1/fe779a817a9d4938193d
おおお、求めていたものが。。
locale エラーが直れば自動化できそう！

試してみましたが、 scope に manage_pages を含んだトークンを取得する という部分が問題ですね。
この scope を持ったアプリを私が管理していないので、このトークンが取得できません。

Facebookページの管理者にトークンを発行してもらわないと難しいですね。

https://developers.facebook.com/docs/graph-api/reference/v6.0/url#update のAPIは更新しようとしているサイトとアクセストークンは無関係のように思えます。
関係ないFacebookページをもとに発行したアクセストークンを用いて試したところ、同様のog:localeのエラーが発生するので、更新はサイト管理者に限定していないのでは？

アプリ作って、テストしてみているのですが、なぜかまだlocale のエラーが出てますね。

ここで指定している http://dev-covid19-tokyo.netlify.com/ の og:locale が ja_JP になっていることは確認できています。
エラーには、og:locale:locale とか書いてあっておかしい？

私の方ではエラーになりませんでした。

エラーには、og:locale:locale とか書いてあっておかしい？
これは以前と同じエラーなので関係ないです。

一度エラーが発生したら、FacebookDebuggerでやる必要があったりするのか？
とはいえ、一度エラーが解消していれば、サイト更新をしても正しく反映されるようにはなりそうな気がします。

すでにLocaleの変更は本番に反映済みのようで、delete_fbcache.sh を手元で動かしたら、正常に動いていそうです。
以下ログの抜粋です。
%
HTTP/2 200
vary: Accept-Encoding
x-app-usage: {"call_count":2,"total_cputime":0,"total_time":0}
content-type: application/json; charset=UTF-8
facebook-api-version: v6.0
strict-transport-security: max-age=15552000; preload
pragma: no-cache
x-fb-rev: 1001916064
access-control-allow-origin: *
cache-control: private, no-cache, no-store, must-revalidate
x-fb-trace-id: Atlnly9LPk0
x-fb-request-id: ADVbOwHT2PmZGsx2yAfTT_v
expires: Sat, 01 Jan 2000 00:00:00 GMT
x-fb-debug: 7X3GlE+MuM77S/6s2rAgJ8P9bDrANzHo+Xg0YNL4tGghFRRNFOmDiwSa0H2ShHsXG6Vx8wgakjTy5XZbrUFBAg==
date: Mon, 30 Mar 2020 09:16:10 GMT
alt-svc: h3-27=":443"; ma=3600
date: Mon, 30 Mar 2020 09:16:11 GMT

{"url":"https:\/\/stopcovid19.metro.tokyo.lg.jp\/","type":"website","title":"\u6771\u4eac\u90fd \u65b0\u578b\u30b3\u30ed\u30ca\u30a6\u30a4\u30eb\u30b9\u611f\u67d3\u75c7 \u5bfe\u7b56\u30b5\u30a4\u30c8","locale":{"locale":"ja_jp"},"image":[{"url":"https:\/\/stopcovid19.metro.tokyo.lg.jp\/ogp.png"}],"description":"\u5f53\u30b5\u30a4\u30c8\u306f\u65b0\u578b\u30b3\u30ed\u30ca\u30a6\u30a4\u30eb\u30b9\u611f\u67d3\u75c7 (COVID-19) \u306b\u95a2\u3059\u308b\u6700\u65b0\u60c5\u5831\u3092\u63d0\u4f9b\u3059\u308b\u305f\u3081\u306b\u3001\u6771\u4eac\u90fd\u304c\u958b\u8a2d\u3057\u305f\u3082\u306e\u3067\u3059\u3002","site_name":"\u6771\u4eac\u90fd \u65b0\u578b\u30b3\u30ed\u30ca\u30a6\u30a4\u30eb\u30b9\u611f\u67d3\u75c7 \u5bfe\u7b56\u30b5\u30a4\u30c8","updated_time":"2020-03-30T09:16:09+0000","id":"2935876866463721","application":{"id":"2879625188795443","name":"\u6771\u4eac\u90fd\u65b0\u578b\u30b3\u30ed\u30ca\u611f\u67d3\u75c7\u5bfe\u7b56\u30b5\u30a4\u30c8","url":"http:\/\/stopcovid19.metro.tokyo.lg.jp\/"}}

長期トークンはこちらのやり方のほうが最新のようでした。
https://developers.facebook.com/docs/facebook-login/access-tokens/refreshing
この方法で関係ないFacebookアプリのapp_idとapp_secretをもとに作成したトークンで、同様に実行して、同様の結果になりました。

@cgetc @halsk 最終的に利用するトークンですが、「長期ページアクセストークン」という理解で良いでしょうか？「長期ユーザーアクセストークン」の取得まではうまくいったのですが、「長期ページアクセストークンの取得で空データが戻ってくる現象に悩んでいます。何かお気づきの点があれば、教えてください。
{"data":[]}

@MasaGon 長期ページアクセストークンのほうが正しいと思いますが、長期ユーザーアクセストークンでも動作したと記憶しています。

@MasaGon 長期ページアクセストークンを取得するにはmanage_pagesのスコープが必要のようです。以下の手順で長期ページアクセストークンが取得できました。
1. Graph API Exploler でスコープにmanage_pagesを追加してトークンを発行する
2. 1.のトークンをもとに長期ユーザアクセストークンを取得する
3. 2.のトークンをもとに長期ページアクセストークンを取得する

@cgetc ありがとうございます！
どうやら具体的にpages_show_listでリンクしているページが一つもないと、/me/accounts には何もデータが出力されないという問題のようでした。ダミーページを作って対応しました。

authors: MasaGon cgetc halsk t-tajiri


text:
「心配なときに」多言語 追記

「新型コロナウイルス感染症が心配なときに」多言語ページに、「ひまわり」の電話番号の追加をお願いします。
参考）「ひまわり」https://www.himawari.metro.tokyo.jp/qq13/qqport/tomintop/other/fks230.php
・作業対象URL
英語　https://stopcovid19.metro.tokyo.lg.jp/en/flow
簡体字　https://stopcovid19.metro.tokyo.lg.jp/zh-cn/flow
繁体字　https://stopcovid19.metro.tokyo.lg.jp/zh-tw/flow
韓国語　https://stopcovid19.metro.tokyo.lg.jp/ko/flow
・追記していただきたい文言
For support in English, Chinese, Korean, Spanish and Thai, please call the Tokyo Health Care Information Center (Himawari)
０３－５２８５－８１８１(Hours: 9:00 a.m. to 8:00 p.m.  includes weekends and holidays.)

なお、簡体字・繁体字・韓国語のページにも上記英語の文言の掲載をお願いします。
それぞれの言語への訳出は現段階では不要です。
・追記していただきたい箇所
-「受診相談窓口」
　平日（夜間）
　～
　03-5320-4592
　の下
もしくは

別添のキャプチャ画像にある、画面上部

どちらが実装しやすいでしょうか。

@halsk こちら #1876 でマージされたようなのでクローズしてもいいですか？

クローズします！

authors: NaoyaShimizzz halsk kaizumaki


text:
grub bootloader password missing in hiera

The hiera data for adding grub password is missing from several distributions.

Therefore in all distros except SLES this in configured:
cis_security_hardening::rules::grub_password::enforce: false
So having no password in the config should not harm.

authors: parryb tom-krieger


text:
Delete configmap-xxx from ./config

Fix #5
Still hub.docker.com cannot deliver its event.

It was port conflict. We have to seek a way to enable metrics stuff from around eventing.

What exactly is the causing conflict?
Conflict between proxy sidecar for RA ksvc and webhook metrics?

I am not 100% sure for this as I have not understood the whole mechanism of the metrics yet:
installed ./config on the master, applied example.yaml.
from kubectl describe deployments.apps <dockerhubsource generated name>
Containers:
   user-container:
    Port:       8080/TCP
    Host Port:  0/TCP
    Environment:
      EVENT_SOURCE:              default/dockerhub-source
      METRICS_DOMAIN:            knative.dev/sources
      NAMESPACE:                 default
      K_LOGGING_CONFIG:          {"zap-logger-config":..."}
      K_METRICS_CONFIG:          {"Domain":"knative.dev/sources","Component":"dockerhub-source","PrometheusPort":0,"ConfigMap":{}}
      K_TRACING_CONFIG:          {"backend":"","debug":"false","sample-rate":"0"}
      K_SINK:                    http://event-display.default.svc.cluster.local
      K_CE_OVERRIDES:
      PORT:                      8080
...
   queue-proxy:
    Image:       gcr.io/knative...
    Ports:       8022/TCP, 9090/TCP, 9091/TCP, 8012/TCP
    Host Ports:  0/TCP, 0/TCP, 0/TCP, 0/TCP
...
    Environment:
      SERVING_NAMESPACE:                      default
      SERVING_SERVICE:                        dockerhub-source-489gd
      SERVING_CONFIGURATION:                  dockerhub-source-489gd
      SERVING_REVISION:                       dockerhub-source-489gd-fchpx
      QUEUE_SERVING_PORT:                     8012
      CONTAINER_CONCURRENCY:                  0
...
      USER_PORT:                              8080
      SYSTEM_NAMESPACE:                       knative-serving
      METRICS_DOMAIN:                         knative.dev/internal/serving
      DOWNWARD_API_LABELS_PATH:               /etc/podinfo/labels
      SERVING_READINESS_PROBE:                {"tcpSocket":{"port":8080,"host":"127.0.0.1"},"successThreshold":1}
...

Today, on the one hand, queue-proxy container created by ksvc controller apparently tries to  acquire 9090 of the Pod for serving component.
On the other hand, Receive Adapter v2 metrics server on user-container on ksvc tries to acquire 9090 of the Pod if Prometheus config is set to 0.
We have config-xxx.yaml in the ./config, so the K_METRICS_CONFIG is injected via this part. You can see PrometheusPort: 0, so metrics in the Adapter should try to acquire 9090(reason). If we do not have config-xxx.yaml, then it is not injected so the metrics gets no-op and the Pod gets up successfully.
This was expected in the knative.dev/pkg#1005 so far, so we can choose another port.
GitLabSource's tracing/logging/metrics was enabled in eventing-contrib#1215, but I assume that the error does not surface yet today since eventing-contrib/gitlab/config does not includes config-xxx.yaml. It is no-op today(See eventing-contrib#1215 Release note).
Anyway, this patch does not really fix the issue. I have to look deeper for metrics.
p.s. Event exists everywhere. Maybe, in the future, the knative user has to make their custom event source whereas they do not care about metrics stuff as RA is offered from eventing. Should I raise an issue for that? 🤔 I am not confident though...
What do you think, @aliok ?

IMO, metrics stuff should not block you from doing the actual DockerHub integration stuff. If you like, just disable (or leave it disabled) the metrics stuff.

Eventing does not support the metrics feature completely today. Let's leave it😀
Thanks!


text:
Underrated

I think that's some good work presented in this plugin.
If it's not to bothering for you i'd submit it to awsome neovim for you, to expose it to a larger user base.

Of course you are free ， but  sometimes  i don't have much time to deal with issues

It's added 🙂👍

authors: tobealive tomasky


text:
controls is undefined

This is similar issue as #348 (and 449).
When I use production version of code (minimize/uglify), then it stops work with error message:
 Error: controls is undefined .link/<@http://localhost:9000/bower_components/angular-leaflet-directive/dist/angular-leaflet-directive.js:1081:1

In html I use:

The problem is that even if I correctly set $scope.controls = ..whatevet... in my controller the controls variable is still undefined. I simply put console.log to angular-leaflet-directive.js line above. The other setting of leaflet scope is set correctly.
Personally I do not thing that problem is in minification (I use standard yeoman-generator generated grunt file).

Hi @lksv , I haven't been able to replicate this issue, could you provide a jsffiddle, or a stand-alone example where we can see it?

I am going to close this issue. I found this is duplicate of #348.
Unfortunately the workaround described in #348 do not work for me. The only way (I know) make it working is change the controls name directly in source.

authors: lksv tombatossals

repo: tombatossals/angular-leaflet-directive


text:
Using the Yandex layer

Hello,
I'm trying to add the Yandex Layer but i get this error : ReferenceError: ymaps is not defined
here is the code i use :
layers: {
        baselayers: {
          yandex: {
            name: 'Yandex',
            type: 'yandex',
            layerOptions: {
              layerType: 'map',
            }
          },
          yandexTraffic: {
            name: 'Yandex Traffic',
            type: 'yandex',
            layerOptions: {
              layerType: 'map',
              traffic: true,
            }
          }
        }
      }

Is there something i do wrong ?
Thank you !

Hi, I think you forgot to include the Yandex library in your html head section:
    <script src="http://api-maps.yandex.ru/2.0/?load=package.map&lang=ru-RU" type="text/javascript"></script>

I have added a Yandex example (it was missing), briefly it will be included in the example browser.

authors: eljazouli tombatossals


text:
Question: Is there any way to integrate with redux-act?

Is there a way to integrate with redux-act, the action creator lib for redux.

Integrate in what way?  Can you clarify what you want to do?
For what it's worth, you can certainly write reducer functions that use Redux-ORM to do the work internally.  I've got some examples of doing that in my blog posts Practical Redux, Part 1: Redux-ORM Basics and
Practical Redux, Part 2: Redux-ORM Concepts and Techniques.

@markerikson thanks for you quick response.
I would like to use redux-act to create the actions so that I don't need to write a lot of switch statement inside a reducer.
 static reducer(action, Model, session) {
        switch (action.type) {
                ....
        }
}

to this for less boilerplate:
[action] : (state) => state + 1;

@markerikson Thank you all the same. I have implemented it.
The new reducer creator:
import { createAction, createReducer } from 'redux-act';

import orm from '../orm';

export default function createOrmReducer() {
    const session = orm.session(orm.getEmptyState());
    let actionHandler = {};
    session.sessionBoundModels.forEach(modelClass => {
        if (typeof modelClass.actionHandler === 'function') {
            Object.assign(actionHandler, modelClass.actionHandler());
        }
    });
    return createReducer(actionHandler, session.state);
}

In the model class:
static actionHandler() {
        return {
            [someAction]: (state, payload) => {
                const session = orm.session(state || state.getEmptyState());
                const { Model } = session;
                Model.delete();
                return session.state;
            }
        }
}

authors: RealDeanZhao markerikson

repo: tommikaikkonen/redux-orm


text:
Amount of tenants supported for this multitenant approach

As the title stands, i want to know how much tenants you guess (or already know based on your experience) django-tenants can handle and if you know practices that can increase this number.
I'm sorry for posting this as an issue but i didn't knew where else to post it.
Thanks.

Personal I haven't tried that many this is because my SAAS project isn't live yet. I have heard people using it with hundreds
Django Tenants uses PostgeSQL Schema which people have tested with thousands see http://stackoverflow.com/questions/14893964/how-many-schemas-can-be-created-in-postgres
However I can see issue when it come to migrating database even with parallel migrations I can see it taking hours
Hope this helps a little.

Hi, from my side, we are using it as final product already. For now we have about 150 tenants, and it's number after cleaning db (we deleted inactive clients that weren't paying for a while) so it could be around hundred more. Only disadvantage is time of migrations like Tom said, as it's normal time multiplied by 150... sometimes it's about 20min, and not always running it in parallel really works because of deadlocks. Thought till now we had just 2 migrations that couldn't be run that way.
This can be speed up by squashing migrations, really everything runs much faster after that.
I don't think thousands is limit here, it's more your db, and if you would split it in shards it can support more and more schemas. The only thing would be keeping public schema slim as much possible, because then in theory schemas could exist even in different db's, at different servers.

Thanks for your answers, i had used django tenants on an application with 63 Tenants, working fine but there is not so much concurrency, on the other hand we had used on a system with 500~ Tenants, working fine but slow when looping through tenants with connection object, the real pain was the migrations like you guys said, we started the project on Django 1.8, our migrations take at least 4 hours, i've read that on Django 1.9 the rendering models process is so fast than 1.8 so upgrading must speed up this task.
@kosz85 your 150 tenants are using the app at same time? (concurrency).
Supose we have an application that need thousands of tenants with a big ammount of information stored on each one, did you guys know about strategies that can scale (or autoscale in the best scenario) without affecting the application side (Customizing django or django-tenants)
Thank you

Yes, each tenant have from 3 to 20 users, but there are few tenants with 100 users. All are used in parallel mostly at same time (working hours).
Why would you loop through tenants? That's rather ridiculous, and we never do it at request time. And in fact in celery processes we just spawn new jobs. This are rather not critical things, so we have special queue for slower tasks - but in general we have already 4 different queues (fastest, normal, slower, and very slow ;) )
If your migrations takes 4h... then smth is wrong with them. If you add 1 migration it shouldn't take more then few minutes. I thought already how to solve that, and it's possible to have almost downtime less upgrades, but after squashing migrations I just didn't need to implement it. But, you can try develop Nginx conf, that would ask django/postgres about schema version and use different uwsgi app (old for old schemas, and new for new schemas) that way migration would take same amount of time, but each tenant downtime would be only few seconds. Only constriction here are migrations of public schema, but with little tricks it can be also migrated like that.
I also use Django 1.8 and my migrations are even slower then standard ones as we are using python metaclasses, which aren't supported by Django so we use custom migration core to support them, which happen to be slower (but only for Models with multiple metaclasses). And we don't have problems with 4h migrations, the longest one was maybe near 40min. Check squashing and parallel migrations.
About scaling, you can always just scale your postgres, and it's already naturally divided on schema level so it should be really easy. In fact with good replication of public schema, you could in theory have your schemas in different postgres db's and with good made db-router it won't make any difference, as they are separated anyway. That's why I told that it's important to have only small amount of data in public schema.

Supose we have an application that need thousands of tenants with a big ammount of information stored on each one, did you guys know about strategies that can scale

We had our app working a few years before we knew about multitenancy on django, so our example may be far from optimal; we have about 200 tenants, we use one server to serve everything (webserver, app, database) for  X number of tenants depending on tenants usage, we use another app for "central management (automation tools, support tickets, announcements, and other readonly information)". We don't use django migrations, we execute SQL queries directly into each schema with which executes on every tenant of each server.

I am wondering if publishing possible strategies for 'sharding' would fall into the scope of this project's documentation. Multi tenancy is a big thing these days and people need to know how to make the most of wonderful multi tenant approaches such like this. Thanks :)

authors: MiltonLn juanefren kosz85 lorinkoz tomturner

repo: tomturner/django-tenants


text:
keys * command - "*" taken as a key to search on the slot map

Hi,
I have noticed that if I submit ( keys * ), * is taken as key and then get the hashed value to map the slot map (get the slot map). Then that command only submits (keys) to the preselected cluster node.
Even if we want to get all the keys starts with "A*" (like Aaa, Aab, Abc), client considers "A*" as a single key and follow the same procedure to execute the command only on a particular cluster node.
My Expectation is to get all the keys in the cluster using "*" as a wild card.

Hi @Tiroshan,
because most of the redis command have the key in the second position, the library will determine the slot based on the second argument for a normal query. For the command "keys", the behavior should be different, the command should be run on each node of the cluster. To do so you can always do:
eredis_cluster:qa(["KEYS", "*"])

it is true than one common interface eredis_cluster:q() that accepts any argument would be great. That can be an improvement of the library...

Thanks,
This worked for me.

authors: Tiroshan adrienmo


text:
Updating handling of sample priority extraction

This change updates the sample priority extraction logic to match what is provided in Datadog's SDKs.  I.e. any positive inter value is treated as a true value, any 0 or negative values is treated as a false value. Non-integer values cause context extraction to fail with an error.
This partially addresses #17 in that extraction will now work correctly for values > 1; it does not propagate the non 0/1 values to any outgoing trace context.

For reference, here's out Datadog's SDK in go handles extraction of the sampling priority header


text:
Fix Travis CI

Fixes #304
Also upgrades Travis CI build from precise -> trusty

Codecov Report

Merging #348 into master will decrease coverage by 7.2%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #348      +/-   ##
==========================================
- Coverage   82.49%   75.28%   -7.21%
==========================================
  Files           5        5
  Lines         777      777
  Branches        0      226     +226
==========================================
- Hits          641      585      -56
- Misses        136      137       +1
- Partials        0       55      +55

Impacted Files
Coverage Δ

tmuxp/config.py
85.36% <0%> (-8.95%)
:arrow_down:

tmuxp/cli.py
57.05% <0%> (-7.06%)
:arrow_down:

tmuxp/workspacebuilder.py
89.09% <0%> (-6.07%)
:arrow_down:

tmuxp/util.py
90.9% <0%> (-6.07%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4159cb1...71c68a9. Read the comment docs.

Codecov Report

Merging #348 into master will decrease coverage by 6.82%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #348      +/-   ##
==========================================
- Coverage   82.49%   75.67%   -6.83%
==========================================
  Files           5        5
  Lines         777      777
  Branches        0      226     +226
==========================================
- Hits          641      588      -53
  Misses        136      136
- Partials        0       53      +53

Impacted Files
Coverage Δ

tmuxp/config.py
85.36% <0%> (-8.95%)
:arrow_down:

tmuxp/cli.py
57.05% <0%> (-7.06%)
:arrow_down:

tmuxp/util.py
90.9% <0%> (-6.07%)
:arrow_down:

tmuxp/workspacebuilder.py
90.9% <0%> (-4.25%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4159cb1...6641368. Read the comment docs.

Codecov Report

Merging #348 into master will decrease coverage by 6.82%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #348      +/-   ##
==========================================
- Coverage   82.49%   75.67%   -6.83%
==========================================
  Files           5        5
  Lines         777      777
  Branches        0      226     +226
==========================================
- Hits          641      588      -53
  Misses        136      136
- Partials        0       53      +53

Impacted Files
Coverage Δ

tmuxp/config.py
85.36% <0%> (-8.95%)
:arrow_down:

tmuxp/cli.py
57.05% <0%> (-7.06%)
:arrow_down:

tmuxp/util.py
90.9% <0%> (-6.07%)
:arrow_down:

tmuxp/workspacebuilder.py
90.9% <0%> (-4.25%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4159cb1...6641368. Read the comment docs.

authors: codecov-io tony

repo: tony/tmuxp


text:
About tag

hi Why is the associated download tag defined as the base type？I need an Object type tag What should I do

@jacksPeak The tag is set as a String so that It can be saved to the database.  If you need a more structured dataset, the request and download objects have an extras property. Hope this helps.

authors: jacksPeak tonyofrancis

repo: tonyofrancis/Fetch


text:
download error by unexpected end of stream

i'm got error when download file on samsung j2
java.net.ProtocolException: unexpected end of stream
before it stop down because error sometime throw log
FileDownloader downloads slice FileSlice(id=1506190530, position=4, startBytes=63909978, endBytes=85213304, downloaded=18539174)
java.net.ProtocolException: unexpected end of stream
at okhttp3.internal.http1.Http1Codec$FixedLengthSource.read(Http1Codec.java:4
my config
FetchConfiguration fetchConfiguration = new FetchConfiguration.Builder(context)
.enableRetryOnNetworkGain(true)
.setDownloadConcurrentLimit(10)
.setHttpDownloader(new HttpUrlConnectionDownloader(Downloader.FileDownloaderType.PARALLEL))
.build();
Fetch.Impl.setDefaultInstanceConfiguration(fetchConfiguration);

@huongnv251291 are you able to reproduce the error more than once.

Hello @tonyofrancis, Great job on this library. It has been smooth up until now. I keep getting "Unexpected End of Stream" error

authors: BlackVick huongnv251291 tonyofrancis


text:
[Feature request] Query download(s) by tag

Hey, thanks for the awesome library.
Currently I'm using tag to determine the download/request. (Because Download id is type of Int, but in my case we are using String for primary key, that is why I'm using tag).
So I need a method/fun to query download(s) by tag.
How about if we make Download id and group_id are type of String instead of Int?
Thanks

@nthtrung09it What we can do is add a method to get the downloads by tag.
fetch.getDownloadsByTag(tag, callback).

@nthtrung09it This issue is now fixed in new Fetch version 3.0.10/Androidx 3.1.4.
  fetch.getDownloadsByTag("Test", new Func<List<Download>>() {
            @Override
            public void call(@NotNull List<Download> result) {
                //get all downloads with the tag
            }
        });

Thanks @tonyofrancis, I've just checked and it works as expected.


text:
Resource Not Found Error for Offline Videos as Input

Thanks for the great work!!
Instead of RTSP stream, I used an offline locally stored video as an input.
changed the config. to
type=2
uri=file:///home/test.mp4
Both Yolo3 and CustomVision.ai models couldn't be able to pick up this path and shows "Resource NOT found error"
The same video runs on Resnet10 model of DeepStream samples.
Do I need to add any more changes to the config file to run videos as input.

It sounds like the video file is not accessible to the container.  Have you properly mounted the path to the video file to make it accessible to for playback in the DeepStream module?  The default configuration bind mounts the /data/misc/storage path, you may have luck by referencing the file from within that path.

authors: santle toolboc


text:
Compact income vs. expense doesn't work

Name of Defective Feature

Compact Income vs. Expense

System Settings (Required)

ToolKit Version: 2.7.2
Operating System (Mac/Windows/Etc): Windows
Browser (Chrome/Firefox/Safari): Chrome
Browser Version: 67
Other Extensions: Lots

Export of Toolkit Settings (Bottom Left of Settings Page)

Settings Export
[{"key":"AccountsDisplayDensity","value":"0"},{"key":"AccountsEmphasizedOutflows","value":true},{"key":"AccountsStripedRows","value":false},{"key":"AdditionalColumns","value":true},{"key":"AdjustableColumnWidths","value":false},{"key":"AutoDistributeSplits","value":true},{"key":"BetterScrollbars","value":"0"},{"key":"BudgetCategoryFeatures","value":true},{"key":"BudgetProgressBars","value":"0"},{"key":"BudgetQuickSwitch","value":false},{"key":"CalendarFirstDay","value":"0"},{"key":"CategoryActivityCopy","value":true},{"key":"CategoryActivityPopupWidth","value":"0"},{"key":"ChangeEnterBehavior","value":true},{"key":"CheckCreditBalances","value":true},{"key":"CheckNumbers","value":false},{"key":"ClearSelection","value":true},{"key":"CollapseSideMenu","value":true},{"key":"ColourBlindMode","value":false},{"key":"CompactIncomeVsExpense","value":true},{"key":"CreditCardEmoji","value":false},{"key":"CurrentMonthIndicator","value":true},{"key":"CustomFlagNames","value":false},{"key":"DaysOfBuffering","value":true},{"key":"DaysOfBufferingHistoryLookup","value":"0"},{"key":"DisableToolkit","value":false},{"key":"DisplayTargetGoalAmount","value":"2"},{"key":"DisplayTotalMonthlyGoals","value":false},{"key":"DisplayUpcomingAmount","value":false},{"key":"EasyTransactionApproval","value":true},{"key":"EditAccountButton","value":"0"},{"key":"EnableRetroCalculator","value":false},{"key":"EnlargeCategoriesDropdown","value":true},{"key":"EnterToMove","value":true},{"key":"GoalIndicator","value":false},{"key":"GoalWarningColor","value":true},{"key":"GoogleFontsSelector","value":"0"},{"key":"HideAccountBalancesType","value":"0"},{"key":"HideAgeOfMoney","value":false},{"key":"HideHelp","value":true},{"key":"HideReferralBanner","value":true},{"key":"HighlightNegatives","value":false},{"key":"ImportNotification","value":"2"},{"key":"IncomeFromLastMonth","value":"0"},{"key":"IncomeVsExpenseHoverHighlight","value":true},{"key":"LargerClickableIcons","value":true},{"key":"MonthlyNotesPopupWidth","value":"0"},{"key":"NavDisplayDensity","value":"0"},{"key":"Pacing","value":"0"},{"key":"PrintingImprovements","value":true},{"key":"PrivacyMode","value":"0"},{"key":"QuickBudgetWarning","value":false},{"key":"ReconciledTextColor","value":"4"},{"key":"RemovePositiveHighlight","value":true},{"key":"RemoveZeroCategories","value":true},{"key":"ResizeInspector","value":false},{"key":"RightClickToEdit","value":true},{"key":"RowHeight","value":"0"},{"key":"RowsHeight","value":"0"},{"key":"RunningBalance","value":"1"},{"key":"SeamlessBudgetHeader","value":false},{"key":"ShowCategoryBalance","value":true},{"key":"ShowIntercom","value":true},{"key":"SpareChange","value":false},{"key":"SplitKeyboardShortcut","value":true},{"key":"SplitTransactionAutoAdjust","value":true},{"key":"SquareNegativeMode","value":false},{"key":"StealingFromFuture","value":true},{"key":"SwapClearedFlagged","value":false},{"key":"TargetBalanceWarning","value":false},{"key":"ToBeBudgetedWarning","value":false},{"key":"ToggleMasterCategories","value":false},{"key":"ToggleSplits","value":true},{"key":"ToggleTransactionFilters","value":"1"},{"key":"TransactionGridFeatures","value":true},{"key":"ViewZeroAsEmpty","value":false},{"key":"goalIndicator","value":false},{"key":"highlightNegativesNegative","value":false},{"key":"l10n","value":"0"},{"key":"pacing","value":"0"},{"key":"popupCalculator","value":false},{"key":"reports","value":true}]

Steps to Reproduce the Behavior

Turn on the feature
Go to the report page

Expected Behavior

Compacted report
Actual Behavior
No change (still uncompacted). No console errors.

This may be due to 7e8d4df5c566d1b3257c2191ae6de10530cad391 changing the shouldInvoke() to return isCurrentRouteBudgetPage() rather than getCurrentRouteName().indexOf('reports.income-expense') > -1. The former seems like the wrong check - unless somehow the function has a really nondescriptive name.

Hey @tcanens the fix for this should be in our current release! If you notice the issue still exists and you're definitely running the newest version of the toolkit, please reopen! Thanks!
Please note, all browser can take up to an hour to provide updates to all users. You can force an update if it's available for you by going to chrome://extensions and clicking the "Update" button. Firefox users can go to about:addons, click the setting cog, and click "Check for updates...".

authors: joshmadewell tcanens

repo: toolkit-for-ynab/toolkit-for-ynab