text:
Meta: Warehouse's handling and validation of distribution filenames

This is a meta-issue, filed to track multiple independent problems and potential solutions to Warehouse's handling of distribution filenames (i.e., sdist and wheel filenames). I'm going to attempt to index all of them, but I'll almost certainly miss one or more.
Background material
Key PEPs and PyPA standards:

PEP 427 defines the wheel distribution format, including the wheel filename format. PEP 427 is unfortunately internally inconsistent about distribution name normalization, as mentioned in this comment.
PEP 625 is the most recent sdist filename PEP. It punts to PEP 427 for distribution name normalization, meaning that it carries some of the same ambiguity.
PyPA's Binary Distribution Format Spec is the living standard copy of PEP 427. It eliminates the ambiguity in the original PEP, making it clear that the normalization only applies to the distribution name and is strictly equivalent to PEP 503 normalization, followed by replacing - with _.

Key discussions:

Amending PEP 427 (and PEP 625) on package normalization rules contains an extensive discussion on amending PEP 427 and PEP 625, but hasn't seen active conversation in a couple of months.

Outstanding issues and PRs:

https://github.com/pypi/warehouse/issues/10030: Warehouse does not currently except distribution filenames that have been normalized from . to _.
https://github.com/pypi/warehouse/pull/10072: @uranusjr's fix for the above, deferred due to a lack of specification clarity.

Outstanding issues
Warehouse does not support normalized namespace package names
Per both the discuss thread and #10030: namespace packages are commonly denoted as package.foo, which gets normalized to package-foo (PEP 503) and package_foo (wheel-style distribution name).
As such, Warehouse should accept wheels and sdists that start with package_foo for the package.foo package. But it currently doesn't, and complains about a mismatched prefix instead.
The relevant code:
https://github.com/pypi/warehouse/blob/68d12169e2f683e5bca839a6bc3eaa0a0828da56/warehouse/forklift/legacy.py#L1133-L1140
Warehouse accepts invalid wheel filenames
Separately, Warehouse's current wheel filename validation is probably overly permissive.
This happens in a few different places:

_is_valid_dist_file fails open rather than closed. In particular, anything that ends with .whl and contains a WHEEL file is treated as valid, even if it does not have all of the PyPA/PEP 427 required filename components.

Extended wheel filename validation uses a regular expression, but doesn't actually check all parts of the resulting match:
https://github.com/pypi/warehouse/blob/68d12169e2f683e5bca839a6bc3eaa0a0828da56/warehouse/forklift/legacy.py#L1265-L1275
In particular, the build, pyver, and abi components are never checked, meaning that they might be missing entirely.
As a result, there is at least one invalid wheel filename (pyffmpeg-2.0.5-cp35.cp36.cp37.cp38.cp39-macosx_10_14_x86_64.whl) already present on PyPI, with correspondingly invalid metadata available via the JSON API (note the incorrect python_version field):
    {
      "comment_text": "",
      "digests": {
        "md5": "d8a9fddd534dc56bfad1343c0f4d0cec",
        "sha256": "962c2d87ee264cfedace8cd1186efe6d898095b74783e9bdba356d15ccd91f64"
      },
      "downloads": -1,
      "filename": "pyffmpeg-2.0.5-cp35.cp36.cp37.cp38.cp39-macosx_10_14_x86_64.whl",
      "has_sig": false,
      "md5_digest": "d8a9fddd534dc56bfad1343c0f4d0cec",
      "packagetype": "bdist_wheel",
      "python_version": "2.0.5",
      "requires_python": null,
      "size": 11052093,
      "upload_time": "2021-05-15T16:29:06",
      "upload_time_iso_8601": "2021-05-15T16:29:06.646801Z",
      "url": "https://files.pythonhosted.org/packages/56/2c/e25e4322c12a75e9f478106b8919c0a011b28edb32171fa21ebe14513022/pyffmpeg-2.0.5-cp35.cp36.cp37.cp38.cp39-macosx_10_14_x86_64.whl",
      "yanked": false,
      "yanked_reason": null
    },

So, to summarize: there are some distribution filenames that PyPI incorrectly accepts, and some other filenames that PyPI incorrectly rejects, all modulo the current PEPs and living PyPA specifications.
Separately, there's a whole rats' nest of presentation issues and ambiguity between package names, distribution names, etc. I think these are mostly separate from the question of distribution filename acceptance and validation, but they'll be important to consider as well.

xref #12245 as the PEP 625 tracking issue.

repo: pypi/warehouse


text:
Cannot render metrics in graphana

Hi guys,
I just made a logging setup composed of

graphana (2.1.3)
graphite-api (latest)
cyanite (latest)
cassandra (latest)

Unfortunately after spending hours and hours trying to graph some metrics in graphana, it's simple impossible to make it work.
It seems that cyanite is crashing at some point and refusing to communicate with graphite-api after some point in time.
I'm starting cyanite using:
java -cp ./target/cyanite-0.5.1-standalone.jar io.cyanite
INFO [2015-09-21 09:22:21,290] main - io.cyanite installing signal handlers
INFO [2015-09-21 09:22:21,298] main - io.cyanite ready to start system
DEBUG [2015-09-21 09:22:21,352] main - com.datastax.driver.core.Cluster Starting new cluster with contact points [localhost/127.0.0.1:9042]
DEBUG [2015-09-21 09:22:21,377] main - com.datastax.shaded.netty.util.internal.logging.InternalLoggerFactory Using SLF4J as the default logging framework
DEBUG [2015-09-21 09:22:21,390] main - com.datastax.shaded.netty.util.internal.PlatformDependent0 java.nio.Buffer.address: available
DEBUG [2015-09-21 09:22:21,390] main - com.datastax.shaded.netty.util.internal.PlatformDependent0 sun.misc.Unsafe.theUnsafe: available
DEBUG [2015-09-21 09:22:21,390] main - com.datastax.shaded.netty.util.internal.PlatformDependent0 sun.misc.Unsafe.copyMemory: available
DEBUG [2015-09-21 09:22:21,390] main - com.datastax.shaded.netty.util.internal.PlatformDependent0 java.nio.Bits.unaligned: true
DEBUG [2015-09-21 09:22:21,393] main - com.datastax.shaded.netty.util.internal.PlatformDependent Java version: 8
DEBUG [2015-09-21 09:22:21,395] main - com.datastax.shaded.netty.util.internal.PlatformDependent -Dio.netty.noUnsafe: false
DEBUG [2015-09-21 09:22:21,395] main - com.datastax.shaded.netty.util.internal.PlatformDependent sun.misc.Unsafe: available
DEBUG [2015-09-21 09:22:21,395] main - com.datastax.shaded.netty.util.internal.PlatformDependent -Dio.netty.noJavassist: false
DEBUG [2015-09-21 09:22:21,396] main - com.datastax.shaded.netty.util.internal.PlatformDependent Javassist: unavailable
DEBUG [2015-09-21 09:22:21,396] main - com.datastax.shaded.netty.util.internal.PlatformDependent You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes.  Please check the conf
iguration for better performance.
DEBUG [2015-09-21 09:22:21,398] main - com.datastax.shaded.netty.util.internal.PlatformDependent -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
DEBUG [2015-09-21 09:22:21,398] main - com.datastax.shaded.netty.util.internal.PlatformDependent -Dio.netty.bitMode: 64 (sun.arch.data.model)
DEBUG [2015-09-21 09:22:21,398] main - com.datastax.shaded.netty.util.internal.PlatformDependent -Dio.netty.noPreferDirect: false
DEBUG [2015-09-21 09:22:21,410] main - com.datastax.driver.core.SystemProperties com.datastax.driver.FORCE_NIO is undefined, using default value false
INFO [2015-09-21 09:22:21,412] main - com.datastax.driver.core.NettyUtil Did not find Netty's native epoll transport in the classpath, defaulting to NIO.
DEBUG [2015-09-21 09:22:21,417] main - com.datastax.shaded.netty.channel.MultithreadEventLoopGroup -Dio.netty.eventLoopThreads: 2
DEBUG [2015-09-21 09:22:21,427] main - com.datastax.shaded.netty.channel.nio.NioEventLoop -Dio.netty.noKeySetOptimization: false
DEBUG [2015-09-21 09:22:21,428] main - com.datastax.shaded.netty.channel.nio.NioEventLoop -Dio.netty.selectorAutoRebuildThreshold: 512
DEBUG [2015-09-21 09:22:21,451] main - com.datastax.shaded.netty.util.ResourceLeakDetector -Dcom.datastax.shaded.netty.leakDetectionLevel: simple
DEBUG [2015-09-21 09:22:21,692] main - com.datastax.driver.core.SystemProperties com.datastax.driver.DISABLE_COALESCING is undefined, using default value false
DEBUG [2015-09-21 09:22:21,729] main - com.datastax.shaded.netty.buffer.PooledByteBufAllocator -Dio.netty.allocator.numHeapArenas: 1
DEBUG [2015-09-21 09:22:21,730] main - com.datastax.shaded.netty.buffer.PooledByteBufAllocator -Dio.netty.allocator.numDirectArenas: 1
DEBUG [2015-09-21 09:22:21,730] main - com.datastax.shaded.netty.buffer.PooledByteBufAllocator -Dio.netty.allocator.pageSize: 8192
DEBUG [2015-09-21 09:22:21,730] main - com.datastax.shaded.netty.buffer.PooledByteBufAllocator -Dio.netty.allocator.maxOrder: 11
DEBUG [2015-09-21 09:22:21,730] main - com.datastax.shaded.netty.buffer.PooledByteBufAllocator -Dio.netty.allocator.chunkSize: 16777216
DEBUG [2015-09-21 09:22:21,730] main - com.datastax.shaded.netty.buffer.PooledByteBufAllocator -Dio.netty.allocator.tinyCacheSize: 512
DEBUG [2015-09-21 09:22:21,730] main - com.datastax.shaded.netty.buffer.PooledByteBufAllocator -Dio.netty.allocator.smallCacheSize: 256
DEBUG [2015-09-21 09:22:21,730] main - com.datastax.shaded.netty.buffer.PooledByteBufAllocator -Dio.netty.allocator.normalCacheSize: 64
DEBUG [2015-09-21 09:22:21,730] main - com.datastax.shaded.netty.buffer.PooledByteBufAllocator -Dio.netty.allocator.maxCachedBufferCapacity: 32768
DEBUG [2015-09-21 09:22:21,730] main - com.datastax.shaded.netty.buffer.PooledByteBufAllocator -Dio.netty.allocator.cacheTrimInterval: 8192
WARN [2015-09-21 09:22:24,776] main - com.datastax.shaded.netty.util.internal.ThreadLocalRandom Failed to generate a seed from SecureRandom within 3 seconds. Not enough entrophy?
DEBUG [2015-09-21 09:22:24,777] main - com.datastax.shaded.netty.util.internal.ThreadLocalRandom -Dio.netty.initialSeedUniquifier: 0x73c1d6d6c618c119 (took 3003 ms)
DEBUG [2015-09-21 09:22:24,794] main - com.datastax.shaded.netty.buffer.ByteBufUtil -Dio.netty.allocator.type: unpooled
DEBUG [2015-09-21 09:22:24,794] main - com.datastax.shaded.netty.buffer.ByteBufUtil -Dio.netty.threadLocalDirectBufferSize: 65536
DEBUG [2015-09-21 09:22:25,015] cluster1-nio-worker-0 - com.datastax.driver.core.Connection Connection[localhost/127.0.0.1:9042-1, inFlight=0, closed=false] Connection opened successfully
DEBUG [2015-09-21 09:22:25,064] cluster1-nio-worker-0 - com.datastax.shaded.netty.util.Recycler -Dio.netty.recycler.maxCapacity.default: 262144
DEBUG [2015-09-21 09:22:25,147] main - com.datastax.driver.core.ControlConnection [Control connection] Refreshing node list and token map
DEBUG [2015-09-21 09:22:25,350] main - com.datastax.driver.core.ControlConnection [Control connection] Refreshing schema
DEBUG [2015-09-21 09:22:25,465] main - com.datastax.driver.core.ControlConnection [Control connection] Refreshing node list and token map
DEBUG [2015-09-21 09:22:25,543] main - com.datastax.driver.core.ControlConnection [Control connection] Successfully connected to localhost/127.0.0.1:9042
INFO [2015-09-21 09:22:25,543] main - com.datastax.driver.core.policies.DCAwareRoundRobinPolicy Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwar
eRoundRobinPolicy constructor)
INFO [2015-09-21 09:22:25,543] main - com.datastax.driver.core.Cluster New Cassandra host localhost/127.0.0.1:9042 added
DEBUG [2015-09-21 09:22:25,566] cluster1-nio-worker-1 - com.datastax.driver.core.Connection Connection[localhost/127.0.0.1:9042-2, inFlight=0, closed=false] Connection opened successfully
DEBUG [2015-09-21 09:22:25,586] cluster1-nio-worker-1 - com.datastax.driver.core.Session Added connection pool for localhost/127.0.0.1:9042
INFO [2015-09-21 09:22:25,669] main - io.cyanite.engine starting engine
DEBUG [2015-09-21 09:22:25,719] main - io.netty.channel.MultithreadEventLoopGroup -Dio.netty.eventLoopThreads: 2
DEBUG [2015-09-21 09:22:25,744] main - io.netty.channel.nio.NioEventLoop -Dio.netty.noKeySetOptimization: false
DEBUG [2015-09-21 09:22:25,745] main - io.netty.channel.nio.NioEventLoop -Dio.netty.selectorAutoRebuildThreshold: 512
WARN [2015-09-21 09:22:28,763] main - io.netty.util.internal.ThreadLocalRandom Failed to generate a seed from SecureRandom within 3 seconds. Not enough entrophy?
DEBUG [2015-09-21 09:22:28,763] main - io.netty.util.internal.ThreadLocalRandom -Dio.netty.initialSeedUniquifier: 0x87d233603e18c119 (took 3003 ms)
DEBUG [2015-09-21 09:22:28,803] main - io.netty.buffer.ByteBufUtil -Dio.netty.allocator.type: unpooled
DEBUG [2015-09-21 09:22:28,806] main - io.netty.buffer.ByteBufUtil -Dio.netty.threadLocalDirectBufferSize: 65536
DEBUG [2015-09-21 09:22:28,809] main - io.netty.util.NetUtil Loopback interface: lo (lo, 0:0:0:0:0:0:0:1%lo)
DEBUG [2015-09-21 09:22:28,810] main - io.netty.util.NetUtil /proc/sys/net/core/somaxconn: 128
DEBUG [2015-09-21 09:22:28,854] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle org.eclipse.jetty.server.Server@781acaa8 added {qtp657244232{STOPPED,8<=0<=50,i=0,q=0},AUTO}
DEBUG [2015-09-21 09:22:28,856] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle org.eclipse.jetty.server.Server@781acaa8 added {org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@3c0faed4,AUTO}
DEBUG [2015-09-21 09:22:28,872] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle HttpConnectionFactory@51eb5e67[HTTP/1.1] added {HttpConfiguration@56952bbe{32768/8192,8192/8192,https://:443,[]},POJO}
DEBUG [2015-09-21 09:22:28,879] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle ServerConnector@58fd37dd{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.server.Server@781acaa8,UNMANAGED}
DEBUG [2015-09-21 09:22:28,880] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle ServerConnector@58fd37dd{null,[]}{0.0.0.0:0} added {qtp657244232{STOPPED,8<=0<=50,i=0,q=0},AUTO}
DEBUG [2015-09-21 09:22:28,881] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle ServerConnector@58fd37dd{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@3c0faed4,AUTO}
DEBUG [2015-09-21 09:22:28,881] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle ServerConnector@58fd37dd{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.io.ArrayByteBufferPool@f0d7950,POJO}
DEBUG [2015-09-21 09:22:28,883] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle ServerConnector@58fd37dd{null,[http/1.1]}{0.0.0.0:0} added {HttpConnectionFactory@51eb5e67[HTTP/1.1],AUTO}
DEBUG [2015-09-21 09:22:28,885] claypoole-2-0 - org.eclipse.jetty.server.ServerConnector ServerConnector@58fd37dd{HTTP/1.1,[http/1.1]}{0.0.0.0:0} added HttpConnectionFactory@51eb5e67[HTTP/1.1]
DEBUG [2015-09-21 09:22:28,888] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle ServerConnector@58fd37dd{HTTP/1.1,[http/1.1]}{0.0.0.0:0} added {org.eclipse.jetty.server.ServerConnector$ServerConnectorManager@6ba0d6e7,M
ANAGED}
DEBUG [2015-09-21 09:22:28,888] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle org.eclipse.jetty.server.Server@781acaa8 added {ServerConnector@58fd37dd{HTTP/1.1,[http/1.1]}{0.0.0.0:8090},AUTO}
DEBUG [2015-09-21 09:22:28,894] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle org.eclipse.jetty.server.handler.HandlerList@63379410[qbits.jet.server.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a@675
8df33] added {qbits.jet.server.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a@6758df33,AUTO}
DEBUG [2015-09-21 09:22:28,895] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle org.eclipse.jetty.server.Server@781acaa8 added {org.eclipse.jetty.server.handler.HandlerList@63379410[qbits.jet.server.proxy$org.eclipse.j
etty.server.handler.AbstractHandler$ff19274a@6758df33],AUTO}
DEBUG [2015-09-21 09:22:28,895] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle starting org.eclipse.jetty.server.Server@781acaa8
INFO [2015-09-21 09:22:28,901] claypoole-2-0 - org.eclipse.jetty.server.Server jetty-9.3.z-SNAPSHOT
DEBUG [2015-09-21 09:22:28,921] claypoole-2-0 - org.eclipse.jetty.server.handler.AbstractHandler starting org.eclipse.jetty.server.Server@781acaa8
DEBUG [2015-09-21 09:22:28,922] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle starting qtp657244232{STOPPED,8<=0<=50,i=0,q=0}
DEBUG [2015-09-21 09:22:28,924] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle STARTED @12717ms qtp657244232{STARTED,8<=8<=50,i=7,q=0}
DEBUG [2015-09-21 09:22:28,924] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle starting org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@3c0faed4
DEBUG [2015-09-21 09:22:28,924] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle STARTED @12718ms org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@3c0faed4
DEBUG [2015-09-21 09:22:28,924] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle starting org.eclipse.jetty.server.handler.HandlerList@63379410[qbits.jet.server.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19
274a@6758df33]
DEBUG [2015-09-21 09:22:28,924] claypoole-2-0 - org.eclipse.jetty.server.handler.AbstractHandler starting org.eclipse.jetty.server.handler.HandlerList@63379410[qbits.jet.server.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff1927
4a@6758df33]
DEBUG [2015-09-21 09:22:28,924] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle starting qbits.jet.server.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a@6758df33
DEBUG [2015-09-21 09:22:28,925] claypoole-2-0 - org.eclipse.jetty.server.handler.AbstractHandler starting qbits.jet.server.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a@6758df33
DEBUG [2015-09-21 09:22:28,925] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle STARTED @12718ms qbits.jet.server.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a@6758df33
DEBUG [2015-09-21 09:22:28,925] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle STARTED @12718ms org.eclipse.jetty.server.handler.HandlerList@63379410[qbits.jet.server.proxy$org.eclipse.jetty.server.handler.AbstractHand
ler$ff19274a@6758df33]
DEBUG [2015-09-21 09:22:28,925] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle starting ServerConnector@58fd37dd{HTTP/1.1,[http/1.1]}{0.0.0.0:8090}
DEBUG [2015-09-21 09:22:28,928] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle ServerConnector@58fd37dd{HTTP/1.1,[http/1.1]}{0.0.0.0:8090} added {sun.nio.ch.ServerSocketChannelImpl[/0:0:0:0:0:0:0:0:8090],POJO}
DEBUG [2015-09-21 09:22:28,929] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle starting HttpConnectionFactory@51eb5e67[HTTP/1.1]
DEBUG [2015-09-21 09:22:28,929] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle STARTED @12722ms HttpConnectionFactory@51eb5e67[HTTP/1.1]
DEBUG [2015-09-21 09:22:28,929] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle starting org.eclipse.jetty.server.ServerConnector$ServerConnectorManager@6ba0d6e7
DEBUG [2015-09-21 09:22:28,932] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle starting org.eclipse.jetty.io.ManagedSelector@3d9ddbbf id=0 keys=-1 selected=-1
DEBUG [2015-09-21 09:22:28,933] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle STARTED @12726ms org.eclipse.jetty.io.ManagedSelector@3d9ddbbf id=0 keys=0 selected=0
DEBUG [2015-09-21 09:22:28,933] claypoole-2-0 - org.eclipse.jetty.util.thread.QueuedThreadPool queue org.eclipse.jetty.io.ManagedSelector@3d9ddbbf id=0 keys=0 selected=0
DEBUG [2015-09-21 09:22:28,934] qtp657244232-34 - org.eclipse.jetty.util.thread.QueuedThreadPool run org.eclipse.jetty.io.ManagedSelector@3d9ddbbf id=0 keys=0 selected=0
DEBUG [2015-09-21 09:22:28,934] qtp657244232-34 - org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume EPR Idle/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@dfde8ed execute
DEBUG [2015-09-21 09:22:28,934] qtp657244232-34 - org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume EPR Prod/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@dfde8ed produce enter
DEBUG [2015-09-21 09:22:28,934] qtp657244232-34 - org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume EPR Prod/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@dfde8ed producing
DEBUG [2015-09-21 09:22:28,934] qtp657244232-34 - org.eclipse.jetty.io.ManagedSelector Selector loop waiting on select
DEBUG [2015-09-21 09:22:28,935] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle STARTED @12728ms org.eclipse.jetty.server.ServerConnector$ServerConnectorManager@6ba0d6e7
DEBUG [2015-09-21 09:22:28,935] claypoole-2-0 - org.eclipse.jetty.util.component.ContainerLifeCycle ServerConnector@58fd37dd{HTTP/1.1,[http/1.1]}{0.0.0.0:8090} added {acceptor-0@14888435,POJO}
DEBUG [2015-09-21 09:22:28,936] claypoole-2-0 - org.eclipse.jetty.util.thread.QueuedThreadPool queue acceptor-0@14888435
DEBUG [2015-09-21 09:22:28,936] qtp657244232-35 - org.eclipse.jetty.util.thread.QueuedThreadPool run acceptor-0@14888435
INFO [2015-09-21 09:22:28,938] claypoole-2-0 - org.eclipse.jetty.server.ServerConnector Started ServerConnector@58fd37dd{HTTP/1.1,[http/1.1]}{0.0.0.0:8090}
DEBUG [2015-09-21 09:22:28,939] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle STARTED @12732ms ServerConnector@58fd37dd{HTTP/1.1,[http/1.1]}{0.0.0.0:8090}
INFO [2015-09-21 09:22:28,939] claypoole-2-0 - org.eclipse.jetty.server.Server Started @12732ms
DEBUG [2015-09-21 09:22:28,940] claypoole-2-0 - org.eclipse.jetty.util.component.AbstractLifeCycle STARTED @12733ms org.eclipse.jetty.server.Server@781acaa8
DEBUG [2015-09-21 09:22:37,659] cluster1-timeouter-0 - com.datastax.driver.core.RequestHandler onTimeout triggered but the response was completed by another thread, cancelling (retryCount = 0, queryState = QueryState(count=0, inProgress=f
alse, cancelled=false), queryStateRef = QueryState(count=0, inProgress=false, cancelled=false))
DEBUG [2015-09-21 09:22:55,473] cluster1-nio-worker-0 - com.datastax.driver.core.Connection Connection[localhost/127.0.0.1:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
DEBUG [2015-09-21 09:22:55,477] cluster1-nio-worker-0 - com.datastax.driver.core.Connection Connection[localhost/127.0.0.1:9042-1, inFlight=0, closed=false] heartbeat query succeeded
DEBUG [2015-09-21 09:22:55,647] cluster1-nio-worker-1 - com.datastax.driver.core.Connection Connection[localhost/127.0.0.1:9042-2, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
DEBUG [2015-09-21 09:22:55,651] cluster1-nio-worker-1 - com.datastax.driver.core.Connection Connection[localhost/127.0.0.1:9042-2, inFlight=0, closed=false] heartbeat query succeeded

Inserting values over 2003/nc works .
DEBUG [2015-09-21 09:24:41,058] claypoole-1-0 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "local.random.diceroll", :time 1442827475, :mean 4.0, :max 4.0, :min 4.0, :sum 4328.0, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-21 09:24:46,011] claypoole-1-1 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "local.random.diceroll", :time 1442827480, :mean 4.0, :max 4.0, :min 4.0, :sum 7744.0, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-21 09:24:51,008] claypoole-1-2 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "local.random.diceroll", :time 1442827485, :mean 4.0, :max 4.0, :min 4.0, :sum 8564.0, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-21 09:24:55,488] cluster1-nio-worker-0 - com.datastax.driver.core.Connection Connection[localhost/127.0.0.1:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
DEBUG [2015-09-21 09:24:55,491] cluster1-nio-worker-0 - com.datastax.driver.core.Connection Connection[localhost/127.0.0.1:9042-1, inFlight=0, closed=false] heartbeat query succeeded
DEBUG [2015-09-21 09:24:56,007] claypoole-1-3 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "local.random.diceroll", :time 1442827490, :mean 4.0, :max 4.0, :min 4.0, :sum 7696.0, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-21 09:24:58,060] cluster1-timeouter-0 - com.datastax.driver.core.RequestHandler onTimeout triggered but the response was completed by another thread, cancelling (retryCount = 0, queryState = QueryState(count=0, inProgress=false, cancelled=false), queryStateRef = QueryState(count=0, inProgress=false, cancelled=false))
DEBUG [2015-09-21 09:25:01,011] claypoole-1-0 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "local.random.diceroll", :time 1442827495, :mean 4.0, :max 4.0, :min 4.0, :sum 8580.0, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-21 09:25:03,059] cluster1-timeouter-0 - com.datastax.driver.core.RequestHandler onTimeout triggered but the response was completed by another thread, cancelling (retryCount = 0, queryState = QueryState(count=0, inProgress=false, cancelled=false), queryStateRef = QueryState(count=0, inProgress=false, cancelled=false))
DEBUG [2015-09-21 09:25:06,007] claypoole-1-1 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "local.random.diceroll", :time 1442827500, :mean 4.0, :max 4.0, :min 4.0, :sum 8032.0, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-21 09:25:08,059] cluster1-timeouter-0 - com.datastax.driver.core.RequestHandler onTimeout triggered but the response was completed by another thread, cancelling (retryCount = 0, queryState = QueryState(count=0, inProgress=false, cancelled=false), queryStateRef = QueryState(count=0, inProgress=false, cancelled=false))

The data in cassandra looks good.
cqlsh:metric> SELECT * FROM metric ;

 id                                                                        | time       | point
---------------------------------------------------------------------------+------------+--------------------------------------
 {path: 'local.random.diceroll', resolution: {precision: 5, period: 3600}} | 1442827475 | {max: 4, mean: 4, min: 4, sum: 4328}
 {path: 'local.random.diceroll', resolution: {precision: 5, period: 3600}} | 1442827480 | {max: 4, mean: 4, min: 4, sum: 7744}
 {path: 'local.random.diceroll', resolution: {precision: 5, period: 3600}} | 1442827485 | {max: 4, mean: 4, min: 4, sum: 8564}

(3 rows)

After inserting some values into cyanite/cassandra, graphana is unable to query cyanite anymore and it the following errors start to appear
[pid: 15533|app: 0|req: 456/465] 1.2.3.5 () {40 vars in 686 bytes} [Mon Sep 21 09:28:28 2015] GET /metrics/find?query=* => generated 291 bytes in 22 msecs (HTTP/1.1 500) 2 headers in 84 bytes (1 switches on core 0)

So, my question is, why is the pulling of the metrics failing ?

Hey @ valentinul,
how did you install your setup? I'm finished mine today and already gathering metrics and I'm also able to use them in Grafana via Graphite-API:
SELECT * FROM metric.metric ;
 tenant | period | rollup | path                                                              | time       | data
--------+--------+--------+-------------------------------------------------------------------+------------+------------
        |  60480 |     10 | collectddj-cassandra_unifycollectd.interface-venet0.if_packets.rx | 1443431150 |      [0.1]
        |  60480 |     10 | collectddj-cassandra_unifycollectd.interface-venet0.if_packets.rx | 1443431160 |      [0.2]

Maybe I can help you out somehow.

Cheers,
Szop

Ok I've managed it to install Cyanite 0.5.1. Please provide your steps so I can help you out and compare with mine.
Cheers,
Szop

It seem's like I've run into a annoying bug. Graphite-API and Cyanite are playing well until collectd/diamonds sends it's first metrics to port 2003 (carbon on Cyanite):
tail -f /var/log/cyanite/cyanite.log
INFO [2015-09-28 15:04:04,551] main - com.datastax.driver.core.Cluster New Cassandra host /127.0.0.1:9042 added
INFO [2015-09-28 15:04:04,615] main - io.cyanite.engine starting engine
INFO [2015-09-28 15:04:04,707] claypoole-2-0 - org.eclipse.jetty.server.Server jetty-9.3.z-SNAPSHOT
INFO [2015-09-28 15:04:04,724] claypoole-2-0 - org.eclipse.jetty.server.ServerConnector Started ServerConnector@36c01317{HTTP/1.1,[http/1.1]}{10.6.232.1:8080}
INFO [2015-09-28 15:04:04,724] claypoole-2-0 - org.eclipse.jetty.server.Server Started @4192ms
DEBUG [2015-09-28 15:04:17,252] claypoole-1-3 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "dj-cassandra2_unify.df-root.df_complex-reserved", :time 1443445445, :mean 0.0, :max 0.0, :min 0.0, :sum 0.0, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-28 15:04:17,252] claypoole-1-1 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "dj-cassandra2_unify.interface-lo.if_errors.tx", :time 1443445445, :mean 0.0, :max 0.0, :min 0.0, :sum 0.0, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-28 15:04:17,253] claypoole-1-0 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "dj-cassandra2_unify.df-root.df_complex-used", :time 1443445445, :mean 1.77610752E9, :max 1.77610752E9, :min 1.77610752E9, :sum 1.77610752E9, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-28 15:04:17,266] claypoole-1-2 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "dj-cassandra2_unify.df-root.df_complex-free", :time 1443445445, :mean 5.191098368E10, :max 5.191098368E10, :min 5.191098368E10, :sum 5.191098368E10, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-28 15:04:17,272] claypoole-1-0 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "dj-cassandra2_unify.interface-lo.if_errors.rx", :time 1443445445, :mean 0.0, :max 0.0, :min 0.0, :sum 0.0, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}
DEBUG [2015-09-28 15:04:17,278] claypoole-1-3 - io.cyanite.engine in write callback:  #io.cyanite.engine.buckets.MetricSnapshot{:path "dj-cassandra2_unify.interface-venet0.if_packets.rx", :time 1443445445, :mean 0.0, :max 0.0, :min 0.0, :sum 0.0, :resolution #io.cyanite.engine.rule.Resolution{:precision 5, :period 3600}}

tail -f /var/log/apache2/*log
dj-cassandra2.unify:8013 10.6.232.1 - - [28/Sep/2015:15:04:14 +0200] "GET /metrics/find?query=* HTTP/1.1" 200 140 "-" "curl/7.35.0"
dj-cassandra2.unify:8013 10.6.232.1 - - [28/Sep/2015:15:04:15 +0200] "GET /metrics/find?query=* HTTP/1.1" 200 140 "-" "curl/7.35.0"
dj-cassandra2.unify:8013 10.6.232.1 - - [28/Sep/2015:15:04:16 +0200] "GET /metrics/find?query=* HTTP/1.1" 200 140 "-" "curl/7.35.0"
dj-cassandra2.unify:8013 10.6.232.1 - - [28/Sep/2015:15:04:17 +0200] "GET /metrics/find?query=* HTTP/1.1" 500 462 "-" "curl/7.35.0"

curl http://10.6.232.1:8013/metrics/find?query=*
[][][]
!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request.  Either the server is overloaded or there is an error in the application.</p>

After that I've to restart Cyanite :/. It was working without a problem with version 0.1

I've found out,
my Graphite-API is not "crashing" anymore with commit: 5433d57, but I'm still not able to receive Metrics in Grafana.
Cheers,
Szop

Hi guys,
Yes, the issue is present in the latest code but I'm going to try the #14 patch and see if that works for me. Thank you @szop85 and @brutasse for the time and effort invested in this issue :)
All the best,
Valentin

source: gharchive/issue

repo: pyr/cyanite

license: isc

license_type: permissive

license_source: bigquery


text:
Prototype new scheduling system

The biggest problems with event processing we faced so far is
that the consumer queues get overwhelmed and start pushing
the writer queue way behind.
This patch implements a micro-scheduling system (currently
extremely primitive), that's using queue-per-core that polls
and processes writer flushes from every worker process and "regular"
digested events (currently in non-numa-aware fashion & without
thread affinity) via single-producer-single-consumer queues.
For example, 4 workers are started, each worker has an affine
single producer (netty event loop) single consumer (worker thread)
queue. It polls events from it and ingests them.
Every worker, prior to polling it's queue, checks if there are
any pending flushes / database writes. If there are any, it
submits them. Results are collected asynchronously.
This will help us to move at least part of cyanite that's related
to writes to more predictable / controllable env. Also, we can
further improve it, make it numa-aware and make scheduling and
back-pressure smarter.

My configuration is a bit weird but i think it highlights a problem. I have cyanite seperated into "insert" and "query" pids
The query and insert pids are both churning on CPU with this merge.
top - 16:38:53 up 2 days, 19:15,  1 user,  load average: 21.37, 19.91, 10.72
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
Cpu(s): 95.6%us,  0.9%sy,  0.0%ni,  3.4%id,  0.0%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:  30871276k total,  7473708k used, 23397568k free,   156720k buffers
Swap:        0k total,        0k used,        0k free,   482628k cached

   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 35871 root      20   0 6654m 1.0g  17m S 297.9  3.4  11:30.82 java

Ill try and get jmx connected and take a look

Cyanite-query:

cyanite-insert:

@tehlers320 they should churn more idle CPU. I could insert micro-sleeps for better CPU consumption, but that'd also mean that most (or all) benefits of avoiding context-switches would be negative.
The general idea was to have a spinning scheduler, which means that we off-thread all jobs, but don't interrupt them in-between. General idea is to get 1 step closer to thread-per-core, but for that we'd need to do more work.
My measurements show that we're getting better performance generally, at cost of spinning over each queue.
I'll dig into reflector problems though. Not sure why we're getting so much related CPU spin on that...

I'm going to fix the tests and merge that one if nobody's against.

sorry, didn't react fast enough.

:) no problem! I assumed nobody was against it so merged it. Hope it was ok.

source: gharchive/pull-request

authors: ifesdjeen pyr tehlers320


text:
Ensure burn rate factors are accurate, and add exhaustion label

This PR does the following,

Updates the burn rate factors in Windows() function from 14, 7, 2, 1 to 14, 5.6, 2.8, 1 to more accurately represent 28 day burn rate in the following cases (as per conversation offline with @metalmatze),

50% error budget depleted in a day / 100% in 2 days
20% error budget depleted in a day / 100% in 5 days
10% error budget depleted in a day / 100% in 10 days
100% error budget depleted in 28 days

It also adds goDoc comments to Windows which describe its functionality for posterity.
Adds a new exhaustion label to all budget burn alerts, which represents time till the error budget is fully exhausted.
Adds new test cases to Test_windows to account for various common SLO windows like 90d, 84d, 30d, 28d, 14d & 7d

Most of this large diff is to account for changes in tests.  Let me know what you think! 🙂

Long time no see.
Would you still like to get this in?

authors: metalmatze saswatamcode

repo: pyrra-dev/pyrra

license_source: github-api


text:
Re: #979

Sorry, should have checked other tests too when filing #970; Geary also seems to suffer:
library(reticulate)
pysal <- import("pysal")
np <- import("numpy")
pth <- pysal$examples$get_path("stl_hom.txt")
f <- pysal$open(pth)
col <- f$by_col("HR8893")
y <- np$array(col)
w <- pysal$open(pysal$examples$get_path("stl.gal"))$read()
gc <- pysal$Geary(y, w)
df <- read.csv(pth, skip=1)
library(spdep)
nb <- read.gal(pysal$examples$get_path("stl.gal"))
gcR <- geary.test(df$HR8893, nb2listw(nb, style="W"))
all.equal(gc$C, unname(gcR$estimate[1]))
all.equal(gc$EC, unname(gcR$estimate[2]))
all.equal(gc$VC_rand, unname(gcR$estimate[3]))
all.equal(-gc$z_rand, unname(gcR$statistic))
gcN <- geary.test(df$HR8893, nb2listw(nb, style="W"), randomisation=FALSE)
all.equal(gc$VC_norm, unname(gcN$estimate[3]))
all.equal(-gc$z_norm, unname(gcN$statistic))

closed with #975

authors: rsbivand sjsrey

repo: pysal/pysal


text:
🛑 Pysio Home Files is down

In 892d24f, Pysio Home Files (https://files.pysio.online/93AtHome/centerStatistics) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Pysio Home Files is back up in e0e0182 after 1 hour, 10 minutes.

license: MIT


text:
Add new MIB file: ARUBAWIRED-SYSTEMINFO-MIB.mib

Need to add new MIB for HPE Aruba switches for CPU, Memory and Storage metrics.

need to remove .mib file extension.

repo: pysnmp/mibs


text:
plot_spectrum() returns a matplotlib Axes object - is that the right thing to return?

currently plot_spectrum() returns a matplotlib Axes object, which in principle can be manipulated ala:
ax = plot_spectra('combspec1-8.fits',colors='k',y="flux and uncertainty",order_numbers=False,display=False)
ax.set_xlabel('Wavelength (micron)', fontsize=15)
ax.set_ylabel('F$_{\lambda}$', fontsize=15)
ax.legend(['Flux','Uncertainty'],fontsize=14)

however, not clear how to then "show" the Axes object. If the purpose is to return a plot object for user manipulation, I think we need to return ax.plot() from the code instead (see example in https://matplotlib.org/3.5.0/tutorials/introductory/usage.html#the-object-oriented-interface-and-the-pyplot-interface)

When I push the latest one, we can figure this one out.

I have updated this program (as we talked about on email).  The default is now to display to the screen and only do the pdf if the pdf info is passed.  It is, however, currently displayed using pl.ion().  See email that I will send in a few.
My goal with this program was not necessarily to let the user muck around with it but rather it be a quick display for our QA purposes.  The titles are set by the FITS file info so that isn't something that would need to be changed (I could make the font size an input parameter).

authors: aburgasser michaelcushing

repo: pyspextool/pyspextool


text:
Custom blocks missing

Custom blocks are not yet implemented.

Duplicate of #108

repo: pystage/pystage


text:
Document how to get help

As discussed in #247, this PR introduces the official python-attrs tag on StackOverflow and encourages people to help out with questions.
It’s very much overdue to have an official way.

As for live support: I really cannot bootstrap a new IRC channel and provide live support there.  There's a bunch of attrs users on #pylib so if they’d be so gracious to host us, I wouldn’t mind adding it the docs as a secondary help channel but so far, this is all I can commit to right now
fixes #247

Someone with >= 1500 reputation needs to create the tag in Stack Overflow.

Ugh, didn’t know you need that.  Anyone?

I bet @glyph has enough SO-fu.

Could someone also explain all the implications to me?  I kinda hoped we can just do it freestyle.

Basically, I saw this so I thought I'd try tagging some SO threads with python-attrs and I'm not allowed to because it's a new tag and I don't have enough SO-fu.

hmmm https://stackoverflow.com/help/privileges/create-tags

OK I got reaperhulk to create the tag and I added it to some questions, but that needs "peer review", and I maxed out the number of questions I can do that to for now.  But it's starting to happen…

Ahhh, missed by a minute. But I was going to introduce you to this guy I know, @moshez, who definitely has the reputation. :)

Holy shit משה!
Anyhow, I guess this is ready to merge now, right?!

In the meantime, can someone with enough SO-fu take care of

please?

Looks like I can get it done, with a review.  Stand by.

OK, pending review.  Once it's in, if we want changes, I think that should be easy.

It went through; thanks Wilfredo!

authors: hynek mahmoud wsanchez

repo: python-attrs/attrs

license: mit


text:
Workarounds for broken caldav servers

Zimbra may apparently create events with both DTSTART, DTEND and DURATION, which is a breach of the standard.  A simple workaround has been implemented, just throwing away redundant information.  No check done on weather the DURATION matches DTEND-DTSTART (it did in the ical data I got from Zimbra), so this one may potentially throw away information.  (I just wrote a message to the organizer asking if the original ical data was actually produced internally Zimbra or by some other client).
I've been fighting a bit with some proprietary system in #201 that have some partial and slightly non-compliant support for caldav - and made some partial workaround for loading content from it.
One extra unrelated commit enhancing the tests code has been piggy-backed on this pull request.

I shouldn't do such things in the late evenings ... commit 22011e49d95b8db59f385af49149e8a5499e582a and 5ad53ce28efb49df0f344f4b297e5e0d142c9a0a should have been included in this pull request :-)

repo: python-caldav/caldav


text:
remove with_roles decorator and replace with has_any_role

Initially, when we wrote the with_roles decorator, discord.py's own has_any_role did not support IDs as input. Around a month ago (in this commit), this was changed.
That means, there's no longer any point in us having our own decorator for this. Our decorator should be replaced with has_any_role. See https://discordpy.readthedocs.io/en/rewrite/ext/commands/api.html#discord.ext.commands.has_any_role for documentation.

Okay, so, I no longer think we should do this.
The discord.py framework does not provide a negated version of this. We have both with_role and without_role, and we use them both a lot. Changing our with_role only would mean that we'd have has_any_role and without_role decorators that no longer matched, and would be imported from different places. This is uglier than what we've currently got.
Closing this ticket, since it hasn't seen any discussion anyway.

Would suggesting the addition of without_role functionality to discord.py be worthwhile?

I don't see why not.

From #236:
Some cogs use the cog_check method to add a check to every command in the cog. The cog_check method is the predicate itself, which poses a problem if we were to use has_any_role, which is decorator check.
One solution would be to nab has_any_role from discordpy, place it in decorators.py, and split the predicate and check.
That way we get the goodness and the usefulness of having both a decorator and a predicate for cog_checks. I can work on this if we're moving forward with this idea.

@kosayoda I think that's a good solution. If you get a chance to do it, that'll be great. I've assigned you to the ticket.

No progress for 4 months; unassigning. This is now open for anyone to work on.

authors: GhostofGoes MarkKoz heavysaturn kosayoda lemonsaurus scragly

repo: python-discord/bot


text:
Further doc improvements

Sort of a continuation to #546 to get feedback before getting to it.
Before (trying to) implementing #447, duplicate items will be needed to be removed from the inventories dict to get only the relevant results. These were mentioned in comments of the aforementioned issues.
That will drastically decrease the amount of items and their used space so I thought we can also up the cache max_size in here
https://github.com/python-discord/bot/blob/3d25c790a69421e0ed9c7c7a29ca1d5833322169/bot/cogs/doc.py#L283-L286
as we got some additional space and can save requests on the few additional items that may be there; although the chance of hitting the cache decreases with the size.

Another thing to look into is the markdown converter that's used on the contents
Discord handles embeds a bit differently on desktop and mobile (android)

Where the internal references don't show up as inline codeblock

And then the last one I goton my mind; we collect all the signatures of functions but some have aliases and others group them by category.
How should this be handled if at all?
As examples here for the first one, everything under the fetched signature is included but also relevant to how the contents are written, but in the second one we just get an alias that's mostly unneeded.

Where the internal references don't show up as inline codeblock

Do inline codeblocks within embeds not work on mobile at all or what's the deal? I agree this should be fixed if possible.

we collect all the signatures of functions but some have aliases and others group them by category. How should this be handled if at all?

There isn't a reliable way to distinguish aliases, is there? I don't think having aliases shown is that bad  of a thing anyway.

Do inline codeblocks within embeds not work on mobile at all or what's the deal? I agree this should be fixed if possible.

inline codeblocks alone are shown and work properly but the internal references are translated to (invalid) links through the converter and that is handled differently on the mobile client.
The secrets embed has this description
description: New in version 3.6.

             **Source code:**
             [Lib/secrets.py](https://github.com/python/cpython/tree/3.8/Lib/secrets.py)

             The
             [`secrets`](#module-secrets "secrets: Generate secure random numbers for managing secrets.") module
             is used for generating cryptographically strong random numbers suitable for managing data such as
             passwords, account authentication, security tokens, and related secrets.

             In particularly,
             [`secrets`](#module-secrets "secrets: Generate secure random numbers for managing secrets.") should
             be used in preference to the default pseudo-random number generator in the
             [`random`](random.html#module-random "random: Generate pseudo-random numbers with various common
             distributions.") module, which is designed for modelling and simulation, not security or
             cryptography.

There isn't a reliable way to distinguish aliases, is there? I don't think having aliases shown is that bad of a thing anyway.

Guess there isn't really; but on a related note, in case of the screenshot with the dunders, if __le__ was fetched do we want to grab all of the signatures or only the ones following __le__?

My idea for fixing the links is to parse them as markdown to find links and then check if they start with a valid schema (just http or https in this case). It could be taken a step further by trying to resolve the relative links and making them absolute. I don't know feasible this parsing is in practice.
For the dunders, in that specific case, I think they should all be included since they are all relevant to the documentation text, as you stated yourself.

Resolving the links shouldn't be much of an issue in itself, just need the url of the page it's from. But don't know how viable its is to plug that into the markdown converter instead of doing that on the already formatted string, which would be more welcome

It doesn't necessarily have to be parsing Markdown. That was just my naïve suggestion, being unfamiliar with the internals of the cog.

The cog uses a subclassed MarkdownConverter from markdownify to parse the html into markdown. Should be able to do it there but haven't look at the source yet to know how easy it'll be with the modularity of the super class

Alright to start working on the discussed features?

Yes, I think I've answered everything.

I'll begin then

After resolving the urls the end up looking like this on mobile

On desktop it's a normal codeblock with a link.
Is this behaviour fine?
Stripping of the backticks would have to be done after they were added in because of how docs are structured if we went with links without codeblock, or the urls can just return text with no linking involved

One one hand, the back ticks don't look too bad on mobile. On the other hand, if they're already links, they're distinguished enough to not need to be in code blocks too. I could go either way but leaning towards keeping the code block.

Thought about the aliases/grouped signatures, we only fetch 3 to keep it short-er so which ones do you think should be included if more than 3 are present?
Going with the previous object.__lt__ example, we can't get them all because 6 separate codeblocks would be huge.
Collecting all the signatures under the fetched ones, and then searching up if there's still space left, or picking the fetched one and then going up from the bottom came to mind for how to do it

For each direction, there are probably cases where that direction is preferable, but that cannot be determined by the code. Therefore, I don't think it matters which ones you choose. Might be best to stick to displaying whatever is adjacent to the queried signature.

Stumbled upon a few more things.
Do we need the default listing of inventories behind get?

This is implemented as a check in the get subcommand when the symbol is empty, but to me the nicer behaviour would be that output only for me !doc and nothing for an empty get.
The set shadows the set symbol from the inventories, any opinions on handling it and other subcommands as symbols when the role check doesn't pass? Currently there's just no output from the bot.
In the previous doc PR I've implemented auto deletion of the not found embed, in retrospect this would probably be better off with the trash can emoji reaction handling, any thoughts on this? I believe it'd be best to only include it on the failed gets as the chance of getting the wrong doc page is minimal.
And the last, are there any possible complications from an escaped codeblock in an embed? Currently it's converted with clean_content and then simply wrapped in backticks when not found which is simple to get out of.
https://github.com/python-discord/bot/blob/e37041622aebc8fef75d83fcd8970dc527c056bc/bot/cogs/doc.py#L384-L388

Does the url when adding an inventory through doc set need to be verified like here? https://github.com/python-discord/bot/blob/e37041622aebc8fef75d83fcd8970dc527c056bc/bot/cogs/doc.py#L126-L132 a mistake should be noticeable immediately after the command and while it's a fairly uncommon command to use it'll also reduce the needless requests.
I think this leads to worse ux or code; currently when the set_command is invoked it'll do 2 web requests out of 3 total before it sends out a typing event causing a noticeable delay during which the user could be thinking that something went wrong.
Moving it to the top of the command method still leaves out the verifying request since it's a part of the converter, and I don't really like triggering the typing event there

Do we need the default listing of inventories behind get? ... the nicer behaviour would be that output only for me !doc and nothing for an empty get.

I kind of like it. I don't think it should be empty; it should at least show the help command, but showing the available inventories feels more useful since the get command is not complex enough for people to really need to see the help.

The set shadows the set symbol from the inventories, any opinions on handling it and other subcommands as symbols when the role check doesn't pass? Currently there's just no output from the bot.

Putting it in quotes works, so I'm not concerned about it: !d "set"

In the previous doc PR I've implemented auto deletion of the not found embed, in retrospect this would probably be better off with the trash can emoji reaction handling, any thoughts on this?

Yeah, I agree. It would still autodelete it but allow for a pre-emptive delete via the reaction, right?

And the last, are there any possible complications from an escaped codeblock in an embed?

Escaping any markdown should be OK to do; I don't see any problems with doing that just to be safe.

a mistake should be noticeable immediately after the command

How? There'd be nothing preventing the post to our API, meaning the user would have to manually delete the incorrect inventory afterwards. Adding inventories is not a frequent occurrence so I don't mind extra wait time from requests. How much time does it really add anyway?

I don't really like triggering the typing event there

I don't see anything wrong with triggering it in the converter.

The set shadows the set symbol from the inventories, any opinions on handling it and other subcommands as symbols when the role check doesn't pass? Currently there's just no output from the bot.

Putting it in quotes works, so I'm not concerned about it: !d "set"

There haven't been that many, but searching through the history there were a few instances where people seemingly though it just didn't work after a !doc set, !doc get set is usually the next step but I doubt someone would try the quotes.
get set works perfectly fine so this would be just a small QoL improvement

In the previous doc PR I've implemented auto deletion of the not found embed, in retrospect this would probably be better off with the trash can emoji reaction handling, any thoughts on this?

Yeah, I agree. It would still autodelete it but allow for a pre-emptive delete via the reaction, right?

I was thinking about just adding the reaction and leaving it up to the user, but doing it automatically along with the reaction solves the problem I had with it where the messages just disappeared without any hint
I've also thought about a deeper caching system, for whole pages instead of only caching symbol embed because of the structure of some docs where all the symbols are in one page.
Do we care more about the response time and doing additional requests or the RAM space (or drive if it's stored there)? I believe only discordpy has the one page layout out of the commonly fetched packages so it'd probably top out somewhere around 3-5MB most of the time, with the other docs being separated by modules etc..

I doubt someone would try the quotes

Fair enough

get set works perfectly fine so this would be just a small QoL improvement

Isn't get shadowed too? Would you propose renaming both commands? And to what? I don't really mind them being renamed since set is seldom used and non-public and get is usually used as !d g or just !d.

Do we care more about the response time and doing additional requests or the RAM space (or drive if it's stored there)?

I think response time is important. I am not keen on the current response times for docs. I think we have enough RAM to handle it, but @j03b is more familiar with that end of things. I think presenting actual RAM usage figures would help a lot in determining if it will be an issue.

Isn't get shadowed too? Would you propose renaming both commands? And to what? I don't really mind them being renamed since set is seldom used and non-public and get is usually used as !d g or just !d.

get would be shadowed but the symbol doesn't exist in the current inventories. There shouldn't be many problems if it assumes the command was intended to be executed when the permissions are there; but renaming doesn't sound like a bad idea, although I don't know what to.

getdoc and setdoc? I don't think it really matters to pick a nice name since most of the time the aliases will be used. Just needs to be sensible and unlikely to shadow anything.

We could go with invalid identifiers like get-doc and set-doc to be completely safe but getdoc and setdoc sound good and should be future proof since there's almost no normal non python symbols that don't have the lib prefix

Do you think the doc suffix should also be added to the delete/remove and the refresh subcommands?

Yeah, wouldn't hurt. The commands are staff-only and seldom used. They'll still have convenient aliases anyway. Other options are a separate command group (possibly unintuitive) or a nested group (cumbersome?)

We have more than enough RAM for this (and we have alerts set up in case we don't (but we do)).
We average at around 4-5GB of RAM usage but the server has a total of 8GB (and 1GB swap as well).

How do we feel about removing the link from read more in truncated doc responses?
The link is already the embed's url, although it does not highlight on mobile so I've thought about removing the inline codeblocks from the title, which I think also ends up looking a bit more pleasing.

I'm rewriting the module and abstracting some things, so this would prove useful as one less thing to need to pass around.

So it highlights the title link on mobile if the code block is removed? If that's the case, I'm totally on board with removing the title code block and relying solely on the title link.

While there was not much use for the link itself, I think the highlight showed a bit better that it wasn't a part of the text. I think just the text read more blends in a bit too much, but haven't found an alternative I was entirely satisfied with

Do you think it's fine as it - but without the link, or some of the alternatives I fit into the image?

I think a simple ellipses would be fine.

I feel like it blends in too much in the first embed here,

Any other opinions on this?

Might look nicer if you strip punctuation before adding the ellipses.

authors: MarkKoz Numerlor j03b


text:
Numbers station

We want to set up a numbers station - we can probably stream this via nginx.
The station should serve some audio 24/7, but allow us to intersperse messages
into the output every once in a while. It should be possible to automatically
generate messages, e.g. via espeak or fixed recordings of DevOps team members
reading out numbers and letters.
Since Joe has radio experience, he can probably help a large part in
implementing this.
Goals

[ ] Continuously endpoint most likely via nginx

[ ] Command to intersperse output (mp3 for a start)

[ ] Command to automatically speak output (can be espeak for a start)

Scheduling can be performed via cron or at.

Python Discord, it's time.

Suggested: We have decided to infringe on Lemon's copyright (with attribution). We could use Beardfist for the static music playout.

For reference: https://discordier.github.io/sam/

repo: python-discord/infra


text:
Add jokes command

Relevant Issues
Closes #1023
Description
I added a joke command which retrieves jokes based on a user's input from the pyjokes API.
Did you:

[x] Join the Python Discord Community?
[x] Read all the comments in this template?
[x] Ensure there is an issue open, or link relevant discord discussions?
[x] Read and agree to the contributing guidelines?

To whoever merged this, please squash & merge.

repo: python-discord/sir-lancebot


text:
Backports some patches to Python 2.7 branch

PS : I had to adapt #331 due Python 2.7 support drop by upstream black and underlying click dependency...

Overall LGTM. I admit that I'm not following up on each and every change, though.

I've "only" cherry-picked commits that could easily be applied, the other ones are left for those who would need them.
Thanks for reviewing this :bow:

repo: python-distro/distro


text:
Youtube video downloader and mp4 to mp3 converter

Script Title - Automatic YouTube video downloader and mp4 to mp3 converter
Brief  -  For someone who would love to hear famous dialogue's in movies or would love to hear their favorite musical notes found only on YouTube !  It downloads the video and further converts it to mp3 format automatically. All one has to do is only paste the video link!
Instructions

Create a new folder for your script and file/folder name should be appropriate.
Create a README.md in your folder for program Instructions
add requirements.txt if needed

Programming Language

[x] Python

Happy Coding

I would like to work on this issue. Please assign me this.

@pawangeek Can I get assigned to this issue?

Hey @MMM-9 we have a similar script here #159

Hi @pawangeek , the script here is also converting it to MP3 after download, so its level up from #159, specially used for people who want songs/speech or as audio books as mp3 in their systems.

Cool than for it, but make sure, it don't have option for downloading videos, always download a audio format

@pawangeek how can I add my code in this repo as when I followed the steps, a permission denied error was coming.

@pawangeek I tried to resolve the flake8 error, so should I just edit my script in the pull request #451 or should I create a new pull request?

repo: python-geeks/Automation-scripts


text:
Doc fixes

Just a collection of minor fixes that I found when building the docs with sphinx 3.2

Huh, anything missing in this PR that makes you close this? Can I improve it in some way?

No, that got auto-closed due to the branch deletion. Sorry, will look at it now.

already in current main branch

repo: python-trio/asyncclick


text:
ClientStubStreamStream uses yield inside a task group

See https://github.com/python-trio/purerpc/blob/9b70109b02807b75346f8f6d331316963dd363c8/src/purerpc/wrappers.py#L110-L115
This causes internal corruption in trio.
I think there are two potential solutions:

Don't run the sending stream side of the client when yielding. One way to accomplish this is to cancel the sending side whenever a message is received from the server. This does require some form of cancel safety of the sending stream (meaning that if send_message is canceled, the message must not be sent). An implementation would then look something like this:

NO_MESSAGE_WAITING = object()

class ClientStubStreamStream(ClientStub):
    async def call_aiter(self, message_aiter, metadata):
        # If send_multiple is cancelled while trying to send a message, that message will be saved in this variable
        current_message = NO_MESSAGE_WAITING

        async def send_multiple():
            nonlocal current_message

            while True:
                try:
                    # Check if we got cancelled while trying to send a message
                    if current_message is NO_MESSAGE_WAITING:
                        current_message = await message_aiter.__anext__()
                except StopAsyncIteration:
                    return
                await stream.send_message(current_message)  # This must be cancel safe
                current_message = NO_MESSAGE_WAITING

        stream = await self._stream_fn(metadata=metadata)
        async with aclosing(message_aiter) as message_aiter:
            try:
                while True:
                    async with anyio.create_task_group() as task_group:
                        task_group.start_soon(send_multiple)
                        try:
                            value = await stream_to_async_iterator(stream).__anext__()
                        except StopAsyncIteration:
                            return
                        task_group.cancel_scope.cancel()
                    # Outside the task group
                    yield value
            finally:
                stream.close()

Wrap the generator in a context manager, like trio-utils provides. This would be a significant API change.

I prefer the second option as it is significantly cleaner and has less sharp edges regarding cancel safety.

Thank you.  I wonder why my project hasn't hit this.  Do you have an example that demonstrates it?

Adding the following to tests/test_errors.py triggers the bug for me:
@purerpc_channel("port")
async def test_errors_purerpc_in_client(greeter_pb2, greeter_grpc, channel):
    async def generator():
        for _ in range(7):
            yield greeter_pb2.HelloRequest()

    stub = greeter_grpc.GreeterStub(channel)

    async for resp in stub.SayHelloToMany(generator()):
        if resp.message == "2":
            raise ValueError("oops")

Wrap the generator in a context manager, like trio-utils provides. This would be a significant API change.

purerpc has few users and, by way of no bug reports about calling the stub with an async generator in 5 years, it doesn't seem anyone is using the feature.  Therefore, I'm not so concerned about an API change.
It's not trivial to use trio-util, because purerpc is built on anyio.  (I don't have plans for an anyio variation of trio-util due to lack of sane test support-- namely, missing autojump_clock.)
It seems like the async generator support is only a convenience of the API.  Isn't it possible for the API user to achieve the equivalent, given only the standard stub call?  If so, I propose removing the feature from the API.

Wrap the generator in a context manager, like trio-utils provides. This would be a significant API change.

purerpc has few users and, by way of no bug reports about calling the stub with an async generator in 5 years, it doesn't seem anyone is using the feature. Therefore, I'm not so concerned about an API change.
However, It's not trivial to use trio-util, because purerpc is built on anyio. (I don't have plans for an anyio variation of trio-util due to lack of sane test support-- namely, missing autojump_clock.)

I don't think this needs a full wrapper like trio-util provides. I can't test it now, but the following should work:
async def stream_stream_sender(stream, message_aiter):
    async with aclosing(message_aiter) as message_aiter:
        async for message in message_aiter:
            await stream.send_message(message)

async def stream_stream_receiver(stream):
    stream = stream_to_async_iterator(stream)

    async for message in stream:
        yield message

@asynccontextmanager
async def stream_stream(self, message_aiter, metadata):
    stream = await self._stream_fn(metadata=metadata)

    async with anyio.create_task_group() as task_group:
        task_group.start_soon(stream_stream_sender, stream, message_aiter)
        yield stream_stream_receiver(stream)

You would then need to use it as
async with some_stub.StreamStreamCall(some_generator()) as resp_gen:
    async for resp in resp_gen:
        ...

It seems like the async generator support is only a convenience of the API. Isn't it possible for the API user to achieve the equivalent, given only the standard stub call? If so, I propose removing the feature from the API, considering that this project is in "maintenance mode".

If I understand it correctly, the generators are the only way to use stream-stream calls in the current purerpc api. You'd have to replicate the logic in https://github.com/python-trio/purerpc/blob/9e96b7d6d5436d4b196c1cece208f907fee9c3a9/src/purerpc/client.py#L44-L66 with a custom ClientStubStreamStream implementation.
About the maintenance mode, I was planning to submit some PRs with some improvements (adding typing information to the stubs, unix socket support, proper handling of connection failures). I could let those changes live in a fork, but that seems needless fragmentation to me. Unless I missed something in my search, I don't think there is an alternative to using grpc with trio? I'm happy to help maintain this project a bit, but would still need someone to review the PRs (given the don't merge your own PR rules).

the generators are the only way to use stream-stream calls in the current purerpc api

I don't think this needs a full wrapper like trio-util provides. I can't test it now, but the following should work:

I see.  If you have time to work on a PR, I'm happy to review.  It would be important to add unit test coverage for the new stub method.
About other improvements, I'm happy to review, but I'd like to consider carefully whether the changes will add to the maintenance burden.  In other projects, I've found even things like adding type annotations to be a burden, because of added dependencies on projects with alpha stability, and having to battle the type system with every future PR.

repo: python-trio/purerpc


text:
exceptions are generally uninformative

For example, when you get a NoNodeError, all we see is NoNodeError((), {}), when NoNodeError("/path/to/node") would be much more useful.
This is because exceptions are handled in a completely generic way by the protocol-level code; instead of the create function checking a response code and creating a NoNodeError(path), we have, in serialization.py, response = EXCEPTIONS[response](), and that part of the protocol code doesn't have access to any useful information to provide to the exceptions.
Related, the EXCEPTIONS dict for some reason is not made up of the classes directly, but rather little wrapper functions that look like this:
        def create(*args, **kwargs):
            return klass(args, kwargs)

I have no idea why; it almost seems like a mistake that it was defined to return klass(args, kwargs) instead of klass(*args, **kwargs), or of course, more simply foregoing the create function entirely and just putting the klass directly into the EXCEPTIONS dict.

Seems like a good change to have to me, want to make a PR with these?

authors: harlowja radix

repo: python-zk/kazoo


text:
Graceful loop shutdown with running executor tasks

When I send a long task to tread-pool executor and immediately close the loop without waiting future returned by .run_in_executor() call I get exception report in logs:
ERROR:concurrent.futures:exception calling callback for <Future at 0x7ff1ae393630 state=finished returned NoneType>
Traceback (most recent call last):
  File "/usr/lib/python3.4/concurrent/futures/_base.py", line 297, in _invoke_callbacks
    callback(self)
  File "/home/andrew/projects/asyncio/asyncio/futures.py", line 410, in <lambda>
    new_future._copy_state, future))
  File "/home/andrew/projects/asyncio/asyncio/base_events.py", line 487, in call_soon_threadsafe
    handle = self._call_soon(callback, args)
  File "/home/andrew/projects/asyncio/asyncio/base_events.py", line 461, in _call_soon
    self._check_closed()
  File "/home/andrew/projects/asyncio/asyncio/base_events.py", line 288, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

That's because task executed in thread has finished after loop closing. The task calls call_soon_threadsafe() (see futures.wrap_future() for details) and it fails on check for loop health.
Due threaded nature it's hard to avoid situations like this: threads cannot be cancelled, we need cope with these even after loop closing. That's pretty common situation, especially for client-side jobs like web page fetching: DNS resolver is executed in thread pool also.
I see two options:

Ignore loop closing in wrap_future call, skip call_soon_threadsafe() for closed loops
Do the same but only if special boolean flag passed into run_in_executor() (while I don't see situations when I don't like the behavior, maybe flag should be True by default).

I think the first option is fine. But I'd like wrap_future to still log somehow that there was a concurrent.future that resolved after the loop was closed.

See http://stackoverflow.com/questions/32598231/asyncio-runtimeerror-event-loop-is-closed/32615276 for yet another example for the problem

Thank you @asvetlov for your AsyncIO support on StackOverflow, a coworker told me he has found a solution from one of your answer ;-)

For the record, I am running into this same issue with run_in_executor() when I close the event loop. @asvetlov's solution here works for me.

I think throwing away the callback when the loop is already closed is fine (i.e., option 1). When closing a loop we throw away all callbacks that haven't run yet, so this would just extend that behavior.
Now, arguably it is actually inconsistent that calling call_soon() on a closed loop raises an error at all -- why not just ignore the call? However, given that no further callbacks are run, we don't expect any further callbacks to be scheduled either, since only callbacks -- or coroutines, which is the same in this context -- should schedule callbacks. So it arguably doesn't matter much, and it does occasionally catch errors (I presume).
But for call_soon_threadsafe() calls made from the executor it's a different story -- we explicitly don't wait for the workers to finish (since who knows when that might be) so we should expect such calls -- but we should treat them the same regardless of whether the thread finishe just before or just after the loop is closed. So option 1 is the right one.

Pull Request would be welcome!

Can I be helpful with that? :) I'd probably be able to implement the first option if I get some initial key points.
So, for call_soon_threadsafe() we need to check self.is_closed()? Is it (probably) enough?
As for test case for this, I think it's clear.

When I use my implementation of ThreadPool (uses threading.pool under the hood) an exception not raises.

loop = asyncio.get_event_loop()
pool = ThreadPool(loop=loop)
loop.set_default_executor(pool)

# do something

pool.shutdown(wait=True)
loop.close()

Also when I call pool.shutdown(wait=True) for futures.ThresdPoolExecutor an exception raises.
exception calling callback for <Future at 0x1064c1f28 state=finished returned NoneType>
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/concurrent/futures/_base.py", line 297, in _invoke_callbacks
    callback(self)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/futures.py", line 442, in _call_set_state
    dest_loop.call_soon_threadsafe(_set_state, destination, source)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py", line 532, in call_soon_threadsafe
    handle = self._call_soon(callback, args)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py", line 506, in _call_soon
    self._check_closed()
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py", line 334, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

For uvloop the same.
exception calling callback for <Future at 0x11019cfd0 state=finished returned NoneType>
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/concurrent/futures/_base.py", line 297, in _invoke_callbacks
    callback(self)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/futures.py", line 442, in _call_set_state
    dest_loop.call_soon_threadsafe(_set_state, destination, source)
  File "uvloop/loop.pyx", line 1023, in uvloop.loop.Loop.call_soon_threadsafe (uvloop/loop.c:23215)
  File "uvloop/loop.pyx", line 500, in uvloop.loop.Loop._call_soon (uvloop/loop.c:13237)
  File "uvloop/loop.pyx", line 504, in uvloop.loop.Loop._call_soon_handle (uvloop/loop.c:13296)
  File "uvloop/loop.pyx", line 532, in uvloop.loop.Loop._check_closed (uvloop/loop.c:13856)
RuntimeError: Event loop is closed

This looks like special behaviour of futures.ThresdPoolExecutor.

Some tests from our stack fail because of this bug. We could mock the executor but since they are end to end tests ideally we shouldn't mock anything.
What's the current status of this? Is anyone working on it? Do we have a reference in https://bugs.python.org/?

I'm on my phone but a pr is open on python search for my login
Le mar. 24 oct. 2017 à 10:07, Manuel Miranda notifications@github.com a
écrit :

Some tests from our stack fail because of this bug. We could mock the
executor but since they are end to end tests ideally we shouldn't mock
anything.
What's the current status of this? Is anyone working on it? Do we have a
reference in https://bugs.python.org/?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/python/asyncio/issues/258#issuecomment-338909048, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAVFXdbgPzjwuLVAhwRp3ILT1At4y7Bbks5svZrRgaJpZM4Ff4WE
.

That PR is https://github.com/python/cpython/pull/431.
The corresponding bug is https://bugs.python.org/issue29711.
Closing this one since we're no longer developing asyncio as a separate process.

Thanks for the references!

authors: 1st1 GMLudo argaen asvetlov gvanrossum mosquito nchammas noplay rudyryk

repo: python/asyncio


text:
🛑 Dev Forum is down

In 36b8739, Dev Forum (https://devforum.roblox.com/) was down:

HTTP code: 403
Response time: 203 ms

Resolved: Dev Forum is back up in fc5ab09 after 2 hours, 44 minutes.

repo: pythoniaweb/roblox-status


text:
Epoch 51: early stopping

Hi!
Model ended training with Epoch 51: early stopping
. I am training with dataset 1040 images? I am training with my 1040 labled images given below but when I try to predict text it shows empty string.
Code ::
image = cv2.imread('./6wf4ef.jpg')
prediction_text = model.predict(image)    // return empty string
print(f"Predicted Text: {prediction_text}")

Prediction is Empty like this :

your code doesn't match image, not clear what is model

This trained model is not predicting my images sir.

This trained model is not predicting my images sir.

show your code, how you do this prediction, because I can't tell why without code...

when I run train.py, it stopped training after sometime and give this message Epoch 51: early stopping and after that when I try to predict image. it did not predict. Below 3 lines I added, it worked with different data set nut not with this.
import cv2
import typing
import numpy as np
from mltu.inferenceModel import OnnxInferenceModel
from mltu.utils.text_utils import ctc_decoder, get_cer
class ImageToWordModel(OnnxInferenceModel):
def init(self, char_list: typing.Union[str, list], *args, **kwargs):
super().init(*args, **kwargs)
self.char_list = char_list
def predict(self, image: np.ndarray):
image = cv2.resize(image, self.input_shape[:2][::-1])
image_pred = np.expand_dims(image, axis=0).astype(np.float32)

preds = self.model.run(None, {self.input_name: image_pred})[0]

text = ctc_decoder(preds, self.char_list)[0]

return text

if name == "main":
import pandas as pd
from tqdm import tqdm
from mltu.configs import BaseModelConfigs
configs = BaseModelConfigs.load("./Models/02_captcha_to_text/202308060003/configs.yaml")
model = ImageToWordModel(model_path=configs.model_path, char_list=configs.vocab)
df = pd.read_csv("./Models/02_captcha_to_text/202308060003/val.csv").values.tolist()
accum_cer = []
for image_path, label in tqdm(df):
image = cv2.imread(image_path)
prediction_text = model.predict(image)
cer = get_cer(prediction_text, label)
print(f"Image: {image_path}, Label: {label}, Prediction: {prediction_text}, CER: {cer}")
accum_cer.append(cer)
image = cv2.imread('./6wf4ef.jpg')
prediction_text = model.predict(image)
print(f"Predicted Text: {prediction_text}")

what was your CER while training model on validation data, I think your model was not trained

Yes, I can train you a model for $$. Otherwise, I can't train a model for everyone. If you are interested write me an email to pythonlessons0@gmail.com

I can give you $$ but how much? I need to crack all images of this type.

hello, were u able to eventually fix this?

authors: Zain-ul-abdin0 gwachatkozah pythonlessons zainulabdinsystems

repo: pythonlessons/mltu


text:
Add AffineQuantizedObserver

Summary:
In our static_quant flow tutorial we were still using observers from torch.ao which we plan to deprecate, this PR adds a more general observer for AffineQuantizedTensor, and has shown that we can replace the old observers (min max observer), there could be futhre work to improve perf, add new types of observation, e.g. tracking stats other than just min/max, moving average observer, histogram observer.
Test Plan:
python test/quantization/test_observer.py
python tutorials/calibration_flow/static_quant.py
Reviewers:
Subscribers:
Tasks:
Tags:

Out of curiosity (and ignorance), but why do we need to couple "Observer" to a particular dtype? Doesn't it just track statistics of a particular Tensor over time?
Or put differently, is there an example of / plans for a Bfloat16Observer or FP8Observer?

@cpuhrsch please let me know if you want me to separate the affine quantized tensor arg list and minmax observer now. it will be something like this:
# has the min_val/max_val logic
class MinMaxObserver(...):
   ...
   def forward(...):
       ...

# has the list of args
class AffineQuantizedObserver(...):
   def __init__(...):

class AffineQuantizedMinMaxObserver(AffineQuantizedObserver, MinMaxObserver):
   def calculate_qparams(...):

will land for now, we can follow up to split the logic when there are more observers I think

repo: pytorch/ao


text:
Add Waveforms for Testing Purposes section to test/README.md

This PR adds instructions around generating waveforms in tests from files and on the fly, as well as contributing new assets.

Thanks for working on this!

(rebasing should fix unittest errors)

authors: astaff mthrok vincentqb

repo: pytorch/audio


text:
Incorrect cache-info output with nosmt linux kernel command line

Hello,
I'm using the main branch of today (https://github.com/pytorch/cpuinfo/commit/3c8b1533ac03dd6531ab6e7b9245d488f13a82a5) and I've got incorrect output in cache-info on a machine booted with the "nosmt" kernel command line parameter:
WITH the "nosmt" parameter (note the weird "shared by XX processors" at the end of each line):
[NOSMT]$ ./cache-info
Max cache size (upper bound): 16777216 bytes
L1 instruction cache: 96 x 32 KB, 8-way set associative (64 sets), 64 byte lines, shared by 97 processors
L1 data cache: 96 x 32 KB, 8-way set associative (64 sets), 64 byte lines, shared by 97 processors
L2 unified cache: 96 x 512 KB (inclusive), 8-way set associative (1024 sets), 64 byte lines, shared by 97 processors
L3 unified cache: 24 x 16 MB (exclusive), 16-way set associative (16384 sets), 64 byte lines, shared by 100 processors

WITHOUT the "nosmt" parameter:
[SMT]$ ./cache-info
Max cache size (upper bound): 16777216 bytes
L1 instruction cache: 96 x 32 KB, 8-way set associative (64 sets), 64 byte lines, shared by 2 processors
L1 data cache: 96 x 32 KB, 8-way set associative (64 sets), 64 byte lines, shared by 2 processors
L2 unified cache: 96 x 512 KB (inclusive), 8-way set associative (1024 sets), 64 byte lines, shared by 2 processors
L3 unified cache: 24 x 16 MB (exclusive), 16-way set associative (16384 sets), 64 byte lines, shared by 8 processors

lscpu output WITH the "nosmt" parameter:
Architecture:         x86_64
CPU op-mode(s):       32-bit, 64-bit
Byte Order:           Little Endian
CPU(s):               192
On-line CPU(s) list:  0-95
Off-line CPU(s) list: 96-191
Thread(s) per core:   1
Core(s) per socket:   48
Socket(s):            2
NUMA node(s):         2
Vendor ID:            AuthenticAMD
BIOS Vendor ID:       Advanced Micro Devices, Inc.
CPU family:           23
Model:                49
Model name:           AMD EPYC 7552 48-Core Processor
BIOS Model name:      AMD EPYC 7552 48-Core Processor
Stepping:             0
CPU MHz:              2851.220
CPU max MHz:          2200.0000
CPU min MHz:          1500.0000
BogoMIPS:             4400.33
Virtualization:       AMD-V
L1d cache:            32K
L1i cache:            32K
L2 cache:             512K
L3 cache:             16384K
NUMA node0 CPU(s):    0-47
NUMA node1 CPU(s):    48-95
Flags:                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sme sev sev_es

lscpu output WITHOUT the "nosmt" parameter:
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              192
On-line CPU(s) list: 0-191
Thread(s) per core:  2
Core(s) per socket:  48
Socket(s):           2
NUMA node(s):        2
Vendor ID:           AuthenticAMD
BIOS Vendor ID:      Advanced Micro Devices, Inc.
CPU family:          23
Model:               49
Model name:          AMD EPYC 7552 48-Core Processor
BIOS Model name:     AMD EPYC 7552 48-Core Processor
Stepping:            0
CPU MHz:             2200.000
CPU max MHz:         2200.0000
CPU min MHz:         1500.0000
BogoMIPS:            4400.15
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0-47,96-143
NUMA node1 CPU(s):   48-95,144-191
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sme sev sev_es

The difference is are on the On-line/Off-line CPU(s) lists.
It caught me by surprise with a SIGFPE when using the NNPACK project (https://github.com/Maratyszcza/NNPACK/issues/218).
Thank you for your help.

Looking at the x86 logic (I'm horribly unfamiliar with it, so bear with me here :)), it looks like we'd only accidentally generate N caches if we assigned them each different L1I IDs. Digging through the code paths here, I don't see anywhere we would accidentally mark the same ID as different IDs, so that leads me to believe that our l1i_id calculation fails somehow.
# How we calculate the apic_bits for each cache in `cpuinfo_x86_decode_cache_properties`.
const uint32_t cores = 1 + ((regs.eax >> 14) & UINT32_C(0x00000FFF));
const uint32_t apic_bits = bit_length(cores);
...
# How we later mask to acquire the L1I ID.
const uint32_t apic_id = linux_processors[i].apic_id;
const uint32_t l1i_id = apic_id & ~bit_mask(x86_processor.cache.l1i.apic_bits);

I'd hazard a guess that this means either:

The apic_id is wrong for all processors with 'nosmt' enabled?
The apic_bits is wrong for all caches with 'nosmt' enabled?

authors: jbd prashanthswami

repo: pytorch/cpuinfo


text:
what is left_pad ?

Hello, I recently recreated fairseq.When I see learned_postitional_embedding.py,I do not particularly understand the function of self.left_pad.What do the input's data operations?
By the way, what is masked_scatter?

The learned positional embedding module replaces tokens with their position numbers, starting from padding_idx+1. The module also requires knowledge of whether the padding symbols will appear on the left or right side of the input. For example:
>>> from fairseq.modules import LearnedPositionalEmbedding
>>> pe = LearnedPositionalEmbedding(num_embeddings=8, embedding_dim=5, padding_idx=1, left_pad=True)
>>> x = torch.LongTensor([[1, 1, 5, 3, 6, 7]])
>>> x
 1  1  5  3  6  7
[torch.LongTensor of size (1,6)]
>>> pe.make_positions(x)
 1  1  2  3  4  5
[torch.LongTensor of size (1,6)]

>>> pe = LearnedPositionalEmbedding(8, 5, 1, left_pad=False)
>>> x = torch.LongTensor([[5, 3, 6, 7, 1, 1, 1]])
>>> x
    5     3     6     7     1     1     1
[torch.LongTensor of size (1,7)]
>>> pe.make_positions(x)
    2     3     4     5     1     1     1
[torch.LongTensor of size (1,7)]

Here are the PyTorch docs for masked_scatter: http://pytorch.org/docs/master/tensors.html?highlight=masked_scatter#torch.Tensor.masked_scatter_

Thank you for your reply,it really helped me a lot

Hi @myleott -- I was debugging an out of index Embedding issue I was having with running the eval_lm script for the Fan Hierarchical Model on the WritingPrompts data, and after parsing through PDB saw that some of the positional indices that were being passed into the positional embedding lookup were negative (specifically when the positions are created in LearnedPositionalEmbedding.forward() and then passed to Embedding.forward()).
I pulled a fresh copy of master and this issue resolved itself, and I noticed that the flags for left_pad seem to be removed from LearnedPositionalEmbedding. Could you explain what the change was in terms of how the model handles padding? Thanks!

This is the commit: #625. Basically the old code required knowing whether to expect padding on the left or right when generating positional ids. The new version is much simpler and works regardless of where the padding is.

Great, thank you for answering my question! Very helpful.

authors: apappu97 myleott travel-go

repo: pytorch/fairseq


text:
[Benchmark] full training benchmarks

Description
Describe your changes in detail.
Motivation and Context
Why is this change required? What problem does it solve?
If it fixes an open issue, please link to the issue here.
You can use the syntax close #15213 if this solves the issue #15213

[ ] I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes
What types of changes does your code introduce? Remove all that do not apply:

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds core functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation (update in the documentation)
[ ] Example (update in the folder of examples)

Checklist
Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

[ ] I have read the CONTRIBUTION guide (required)
[ ] My change requires a change to the documentation.
[ ] I have updated the tests accordingly (required for a bug fix or a new feature).
[ ] I have updated the documentation accordingly.

As discussed, I would rather change the examples than put this in benchmark for the following reasons:

Benchmarks are to measure speed of low-level components of the lib. Maybe in the future we'll also test full training pipelines but this seems to be oriented toward performance benchmarks in terms of return (not speed)
We already have examples and it will be confusing for the users to have 2 examples of a single algo within the lib.

At the end of the day, our examples have the same scope as these benchmarks: reproducing SOTA results. We can rename examples into "implementations" or anything else if that makes it clearer.
Also, IMO for DQN we should have 1 script for Rainbow (that we need) and 1 config for Rainbow + 1 config for DQN (original paper -- ablation study).
Images should not be hosted in torchrl but on our S3 bucket

Another point: I think each script should be an independent PR :)
We should also clearly define what metrics we must log (in order for us to make sure that all algos log the same stuff).
Here's a few to get started:

[ ] Returns
[ ] Gradient norm
[ ] Loss values (if they make sense?)
[ ] Episode length (if it makes sense)

These are just ideas obv
Each script should be accompanied by a benchmark_algoXX.yaml that we run on the RC branch before the release to make sure that the algo runs fine. This yaml file is a SOTA implementation: it must be close to the paper results / public results on another lib. Runtime isn't really an issue for this file (we can have another one that is called fast_blabla).
@BY571 for context

repo: pytorch/rl


text:
Skip some warning test on 3.10 to avoid MKL-related warning (and failure)

CC @malfet @atalman following up on our offline conversation, feel free to merge this PR or to submit a quick fix by pinning the numpy version as discussed.

It seems the issue is gone after #7189:
https://github.com/pytorch/vision/actions/runs/4308089284/jobs/7513938565#step:10:2643
I'll send a separate PR for the lint fixes. But otherwise, we can probably close this one?

Thanks all. The [surfacing of the] issue is gone on main as @pmeier described just above so I'll close this one and open the same PR against release/0.15 instead. (we could also cherry-pick https://github.com/pytorch/vision/pull/7189 into the release branch but that sounds riskier)

authors: NicolasHug pmeier

repo: pytorch/vision


text:
An exit error PyQt5 app with import pywinauto 0.6.8

Expected Behavior
Properly exit:
Process finished with exit code 0
Actual Behavior
Non-proper exit:
Process finishen with exit code -1073740771 (0xC000041D)
Steps to Reproduce the Problem

Run simple PyQt5 app with button that connected to exit function
Press exit button
Get an error if pywinauto was imported

Short Example of Code to Demonstrate the Problem
import sys
from PyQt5.QtWidgets import *
import pywinauto
class MainWindow(QWidget):
    def __init__(self):
        super(__class__, self).__init__()
        self.exit = QPushButton(self)
        self.exit.clicked.connect(self.pressed_exit_button)
    def pressed_exit_button(self):
        app.quit()
if __name__ == '__main__':
    app = QApplication(sys.argv)
    window = MainWindow()
    window.show()
    sys.exit(app.exec_())

Specifications

Pywinauto version: 0.6.8
Python version and bitness: 3.8.0.0 x64
Platform and OS: Windows 10

Some answers on Stack Overflow link this error to "bad integration" between 32-bit and 64-bit code. It might worth checking if your PyQT5 setup is fully 64-bit compatible.
Another option, that has a small chance to affect you is COM threading model that Pywinauto sets up on import: https://pywinauto.readthedocs.io/en/atspi/HowTo.html#com-threading-model
So you can try to play with sys.coinit_flags values directly.

Prolem was gone after installing 0.6.1 version. Thanks for updating!

authors: Djentmat airelil

repo: pywinauto/pywinauto


text:
Messy GUI data panel

The appearance above will occur if:

Open mp2rage
Open another model.

and will go away on the second switch:

won't be present in the third switch:

or the 4th:

Won't occur again unless you go back to the step 1 and do 2.
Labeled as requested by @mathieuboudreau

This bug also occurs for another workflow, which is how I discovered it.

Open MP2RAGE
Load a saved fit result.

Then the GUI formatting problem will happen.

:wave: Hello - I was searching around to see what MRI-related open source projects exist, and generally when I'm looking at any project I tend to see what bugs are open and whether there's anything I can do to contribute.
In this case I don't have access to Matlab, but I did search/browse around the code a bit and have a guess that the relevant code may be: https://github.com/qMRLab/qMRLab/blob/8b8fb39e2cbe1ebbe545589983a2bf07fc8fd7bc/qMRLab.m#L332
That compares to a couple of lines later in the same file: https://github.com/qMRLab/qMRLab/blob/8b8fb39e2cbe1ebbe545589983a2bf07fc8fd7bc/qMRLab.m#L336
(basically it looks like the scrolling functionality can be attached to a different handle dynamically -- maybe that's something to do with the problem here?)

authors: agahkarakuzu jayaddison mathieuboudreau

repo: qMRLab/qMRLab


text:
🛑 nitter.fediflix.org is down

In 91d46ad, nitter.fediflix.org (https://nitter.fediflix.org) was down:

HTTP code: 403
Response time: 729 ms

Resolved: nitter.fediflix.org is back up in 882f1a9 after 5 minutes.


text:
🛑 nitter.tiekoetter.com is down

In 66009aa, nitter.tiekoetter.com (https://nitter.tiekoetter.com) was down:

HTTP code: 502
Response time: 661 ms

Resolved: nitter.tiekoetter.com is back up in 6ad23bf after 46 minutes.


text:
🛑 notabird.site is down

In 94710ff, notabird.site (https://notabird.site) was down:

HTTP code: 520
Response time: 85 ms

Resolved: notabird.site is back up in 08aaf51.


text:
🛑 nitter.poast.org is down

In 296b940, nitter.poast.org (https://nitter.poast.org) was down:

HTTP code: 0
Response time: 0 ms

Resolved: nitter.poast.org is back up in df992e9 after 5 minutes.


text:
🛑 nitter.tiekoetter.com is down

In 4302a83, nitter.tiekoetter.com (https://nitter.tiekoetter.com) was down:

HTTP code: 502
Response time: 867 ms

Resolved: nitter.tiekoetter.com is back up in 32b98d8 after 21 minutes.


text:
🛑 nitter.net is down

In f3b6a4c, nitter.net (https://nitter.net) was down:

HTTP code: 0
Response time: 0 ms

Resolved: nitter.net is back up in d3f6b55.


text:
🛑 nitter.tiekoetter.com is down

In cc2883f, nitter.tiekoetter.com (https://nitter.tiekoetter.com) was down:

HTTP code: 502
Response time: 869 ms

Resolved: nitter.tiekoetter.com is back up in 4d63ea1 after 16 minutes.


text:
🛑 nitter.projectsegfau.lt is down

In 63d6fd2, nitter.projectsegfau.lt (https://nitter.projectsegfau.lt) was down:

HTTP code: 503
Response time: 223 ms

Resolved: nitter.projectsegfau.lt is back up in 14ad415 after 32 minutes.


text:
Can't assign issue

Precondition:
User has run with failed tests
Steps for reproduce:

Open run
Click "More options" for failed tests
Click "Assign issue" button
Type ticket number "Jira ID" field
Wait until the summary field is filled
Click "Assign" button

Expected results: bug is linked
Actual results: Error message. Error 500 in console
{"error":{"message":"Unexpected error has occurred. Please try again later.","code":500}}

fixed in 1.438-SNAPSHOT, will be available in the upcoming maintenance release

authors: dzmitry-pinchuk tsvrko

repo: qaprosoft/zafira