text:
Crash when trying to exit machine

Compact Machines version: 3.0.7 B197
Forge version: 14.23.2.2654
I am not using Optifine: False, but still
Link to Crashlog: here
Description of the problem including expected versus actual behavior: Logged out in a compact machine. Logged back in, and when I tried to leave I crashed
I know that I have optifine installed, BUT, I still report these bugs just in case OptiFine isn't involved and you see something in the code you may have missed.

I'm not able to reproduce this. Please try this without OptiFine and see whether the issue persists.

It stopped happening after I restarted the game. I guess it's just another mystery that will never be solved in this wonderful game we call Minceraft

repo: thraaawn/CompactMachines


text:
Timeouts (short dns timeout value?)

I'm repeatedly getting the following errors running the downloader:
Process Process-11:
Traceback (most recent call last):
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/anyio/_core/_sockets.py", line 190, in connect_tcp
    addr_obj = ip_address(remote_host)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ipaddress.py", line 54, in ip_address
    raise ValueError(f'{address!r} does not appear to be an IPv4 or IPv6 address')
ValueError: 'api.pwnedpasswords.com' does not appear to be an IPv4 or IPv6 address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/anyio/_core/_tasks.py", line 115, in fail_after
    yield cancel_scope
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 114, in connect_tcp
    stream: anyio.abc.ByteStream = await anyio.connect_tcp(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/anyio/_core/_sockets.py", line 193, in connect_tcp
    gai_res = await getaddrinfo(
              ^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError: Cancelled by cancel scope 7fbb37227890

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
    yield
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 113, in connect_tcp
    with anyio.fail_after(timeout):
  File "/usr/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/anyio/_core/_tasks.py", line 118, in fail_after
    raise TimeoutError
TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpx/_transports/default.py", line 66, in map_httpcore_exceptions
    yield
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpx/_transports/default.py", line 366, in handle_async_request
    resp = await self._pool.handle_async_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 268, in handle_async_request
    raise exc
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 251, in handle_async_request
    response = await connection.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request
    raise exc
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request
    stream = await self._connect(request)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_async/connection.py", line 124, in _connect
    stream = await self._network_backend.connect_tcp(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_backends/auto.py", line 30, in connect_tcp
    return await self._backend.connect_tcp(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 112, in connect_tcp
    with map_exceptions(exc_map):
  File "/usr/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 178, in queue_worker_process
    asyncio.run(pwnedpasswords_get_store_gather(result_queue, hash_prefixes, worker_args))
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 182, in pwnedpasswords_get_store_gather
    metadata_results = await asyncio.gather(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 228, in pwnedpasswords_get_and_store_async
    binary, metadata_latest = await pwnedpasswords_get(prefix, hash_type=hash_type, etag=etag, encoding=encoding_type)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 269, in pwnedpasswords_get
    response = await httpx_binary_response(url=url, etag=etag, encoding=encoding, debug=httpx_debug)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/hibp_downloader/lib/http.py", line 53, in httpx_binary_response
    response = await client.send(request=request, stream=True)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpx/_client.py", line 1617, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpx/_client.py", line 1645, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects
    response = await self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpx/_client.py", line 1719, in _send_single_request
    response = await transport.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpx/_transports/default.py", line 365, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/dam/.virtualenvs/pwneddb/lib/python3.11/site-packages/httpx/_transports/default.py", line 83, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectTimeout

The thread managing also does not handle the threads crashing, making those threads stall. Eventually every thread crashes and the main process gets stuck waiting for the now crashed threads to close.

Thanks for the report(s) @gregsadetsky and @wildegnux
I've not had a chance to look into this yet, however I've been able to compare Python 3.10 vs 3.11 and somewhere in between something has broken :(
My current limited check is that I've just used hibp-downloader under Python 3.10 to update my own local --data-path to see if the latest reported Naz.API dataset is included yet - the tool works fine under 3.10
Not entirely sure the root cause is a DNS lookup issue yet, as I get slightly different messages in my 3.11 test environment
2024-01-19T01:17:44+1000 | INFO | hibp-downloader | HIBP Downloader: v0.2.1
2024-01-19T01:17:44+1000 | INFO | hibp-downloader | data-path '/home/ndejong/test/data'
2024-01-19T01:17:44+1000 | INFO | hibp-downloader | metadata-path '/home/ndejong/test/data'
2024-01-19T01:17:45+1000 | INFO | hibp-downloader | Created 4 worker processes to consume a queue of prefix-hash values.
2024-01-19T01:17:52+1000 | INFO | hibp-downloader | prefix=000ef source=[lc:0 et:0 rc:249 ro:0 xx:1] processed=[4.4MB ~24824H/s] api=[55req/s 4.4MB] runtime=0.2min
2024-01-19T01:18:01+1000 | INFO | hibp-downloader | prefix=001f3 source=[lc:0 et:0 rc:499 ro:0 xx:1] processed=[8.8MB ~26053H/s] api=[58req/s 8.8MB] runtime=0.3min
Process Process-4:
Traceback (most recent call last):
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/anyio/streams/tls.py", line 140, in _call_sslobject_method
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 921, in read
    v = self._sslobj.read(len)
        ^^^^^^^^^^^^^^^^^^^^^^
ssl.SSLWantReadError: The operation did not complete (read) (_ssl.c:2580)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/anyio/_core/_tasks.py", line 115, in fail_after
    yield cancel_scope
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 34, in read
    return await self._stream.receive(max_bytes=max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/anyio/streams/tls.py", line 205, in receive
    data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/anyio/streams/tls.py", line 147, in _call_sslobject_method
    data = await self.transport_stream.receive()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1123, in receive
    await self._protocol.read_event.wait()
  File "/usr/lib/python3.11/asyncio/locks.py", line 213, in wait
    await fut
asyncio.exceptions.CancelledError: Cancelled by cancel scope 7905d01b4790

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
    yield
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 32, in read
    with anyio.fail_after(timeout):
  File "/usr/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/anyio/_core/_tasks.py", line 118, in fail_after
    raise TimeoutError
TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 66, in map_httpcore_exceptions
    yield
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 366, in handle_async_request
    resp = await self._pool.handle_async_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 268, in handle_async_request
    raise exc
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 251, in handle_async_request
    response = await connection.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 103, in handle_async_request
    return await self._connection.handle_async_request(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_async/http2.py", line 185, in handle_async_request
    raise exc
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_async/http2.py", line 148, in handle_async_request
    status, headers = await self._receive_response(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_async/http2.py", line 292, in _receive_response
    event = await self._receive_stream_event(request, stream_id)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_async/http2.py", line 333, in _receive_stream_event
    await self._receive_events(request, stream_id)
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_async/http2.py", line 361, in _receive_events
    events = await self._read_incoming_data(request)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_async/http2.py", line 452, in _read_incoming_data
    raise exc
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_async/http2.py", line 438, in _read_incoming_data
    data = await self._network_stream.read(self.READ_NUM_BYTES, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 31, in read
    with map_exceptions(exc_map):
  File "/usr/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ReadTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 178, in queue_worker_process
    asyncio.run(pwnedpasswords_get_store_gather(result_queue, hash_prefixes, worker_args))
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 182, in pwnedpasswords_get_store_gather
    metadata_results = await asyncio.gather(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 228, in pwnedpasswords_get_and_store_async
    binary, metadata_latest = await pwnedpasswords_get(prefix, hash_type=hash_type, etag=etag, encoding=encoding_type)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 269, in pwnedpasswords_get
    response = await httpx_binary_response(url=url, etag=etag, encoding=encoding, debug=httpx_debug)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/hibp_downloader/lib/http.py", line 53, in httpx_binary_response
    response = await client.send(request=request, stream=True)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpx/_client.py", line 1617, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpx/_client.py", line 1645, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects
    response = await self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpx/_client.py", line 1719, in _send_single_request
    response = await transport.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 365, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 83, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ReadTimeout
^CSIGINT received, exiting.
SIGINT received, exiting.
SIGINT received, exiting.
SIGINT received, exiting.
^CSIGINT received, exiting.
Exception ignored in atexit callback: <function _exit_function at 0x7905d137ce00>
Traceback (most recent call last):
  File "/usr/lib/python3.11/multiprocessing/util.py", line 360, in _exit_function
    _run_finalizers()
  File "/usr/lib/python3.11/multiprocessing/util.py", line 300, in _run_finalizers
    finalizer()
  File "/usr/lib/python3.11/multiprocessing/util.py", line 224, in __call__
    res = self._callback(*self._args, **self._kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/multiprocessing/queues.py", line 199, in _finalize_join
    thread.join()
  File "/usr/lib/python3.11/threading.py", line 1119, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.11/threading.py", line 1139, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ndejong/test/.venv/lib/python3.11/site-packages/hibp_downloader/main.py", line 20, in sigint_handler
    exit()
  File "<frozen _sitebuiltins>", line 26, in __call__
SystemExit: None

Note to self

write tests to catch this, we really don't have much in the way of tests at the moment  :(
figure out why the project badges are claiming 3.12 since we don't even test for 3.12
handle threads that blow up so we don't fall into an infinite wait condition
the issue here appears to be caused by httpx
tickets that looks related - https://github.com/encode/httpx/discussions/2726

New release v0.3.0 that might resolve this issue, unfortunately it's been really hard to replicate the errors on this to chase it down - https://github.com/threatpatrols/hibp-downloader/releases/tag/0.3.0
There are two key changes that I'm expecting should move the needle on this issue though -

the loosening of the dependencies to allow upgrades beyond patch releases because there is "something" funky in and around anyio where previously we were probably too opinionated about httpx dependency version that imports anyio
the increase in HTTPX request timeout from 5 seconds to 30 seconds; this seems the more likely culprit

The code is more robust around handling worker threads when they have HTTPX exceptions so we should see less infinite loop scenarios, however clean thread exiting (eg SIGINT with ^C) is still not a thing :(
Please do send feedback so we can gauge if this resolves.

Occasional errors still occur on Python 3.11.6 (on EndeavourOS):
2024-01-24T14:50:16+0100 | DEBUG | hibp-downloader | worker_index=14 prefix='0413a' hash_type='sha1' encoding_type='gzip' ignore_etag=False local_cache_ttl=43200 start_timestamp=2024-01-24 14:50:16.201230+01:00
2024-01-24T14:50:16+0100 | DEBUG | hibp-downloader | No existing metadata file 'data2/sha1/04/13/0413a.meta' using an empty entry
2024-01-24T14:50:46+0100 | ERROR | hibp-downloader | Unable to establish connection https://api.pwnedpasswords.com/range/0413a

Debug log doesn't seem to tell me anything else about those threads that might be useful.
But the failing threads do exit cleanly now so there's no more infinite wait. And to workaround the errors once could just run it in a loop until it stops giving errors.

I can also reproduce the same errors on a different machine with Python 3.10.12 on Ubuntu 22.04.3 LTS (running under WSL)

Okay, that helps - are you able to provide a list of the installed Python packages in each virtual environment you have there?

Python 3.11.6:
aiofiles==23.2.1
anyio==4.2.0
certifi==2023.11.17
click==8.1.7
colorama==0.4.6
h11==0.14.0
h2==4.1.0
hibp-downloader==0.3.0
hpack==4.0.0
httpcore==1.0.2
httpx==0.26.0
hyperframe==6.0.1
idna==3.6
markdown-it-py==3.0.0
mdurl==0.1.2
Pygments==2.17.2
rich==13.7.0
shellingham==1.5.4
sniffio==1.3.0
typer==0.9.0
typing_extensions==4.9.0

Python 3.10.12:
aiofiles==23.2.1
anyio==4.2.0
certifi==2023.11.17
click==8.1.7
colorama==0.4.6
exceptiongroup==1.2.0
h11==0.14.0
h2==4.1.0
hibp-downloader==0.3.0
hpack==4.0.0
httpcore==1.0.2
httpx==0.26.0
hyperframe==6.0.1
idna==3.6
markdown-it-py==3.0.0
mdurl==0.1.2
Pygments==2.17.2
rich==13.7.0
shellingham==1.5.4
sniffio==1.3.0
typer==0.9.0
typing_extensions==4.9.0

I am also experiencing the Unable to establish connection errors - and, from reading the logs, I assume they are not retried? 🤔

Yes, retries are not a thing in the current code - would surely help
Current thinking is that the timeouts would make sense if you consider this thing creates a lot connections which may in-turn cause an upstream network component to grok - for example a small home router may not have enough NAT state table and then intermittently show connection problems.
This line of thinking might explain why I've rarely been able to replicate this most of the time.
Could you give it another spin and reduce the --processes and --chunk-size values - if this is the underlaying issue then a feature to dynamically scale back to suit the local network conditions might be the thing needed here

New release 0.3.1 implements retries and provides CLI options to adjust the default values
https://pypi.org/project/hibp-downloader/0.3.1/
pip install --upgrade hibp-downloader

Feedback would be great

Sorry for the lack of feedback! Will run some test now!

I still run into some threads crashing:
Traceback (most recent call last):
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/anyio/streams/tls.py", line 140, in _call_sslobject_method
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 921, in read
    v = self._sslobj.read(len)
        ^^^^^^^^^^^^^^^^^^^^^^
ssl.SSLWantReadError: The operation did not complete (read) (_ssl.c:2580)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/anyio/_core/_tasks.py", line 115, in fail_after
    yield cancel_scope
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 34, in read
    return await self._stream.receive(max_bytes=max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/anyio/streams/tls.py", line 205, in receive
    data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/anyio/streams/tls.py", line 147, in _call_sslobject_method
    data = await self.transport_stream.receive()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1123, in receive
    await self._protocol.read_event.wait()
  File "/usr/lib/python3.11/asyncio/locks.py", line 213, in wait
    await fut
asyncio.exceptions.CancelledError: Cancelled by cancel scope 72b64bf7c550

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
    yield
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 32, in read
    with anyio.fail_after(timeout):
  File "/usr/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/anyio/_core/_tasks.py", line 118, in fail_after
    raise TimeoutError
TimeoutError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpx/_transports/default.py", line 67, in map_httpcore_exceptions
    yield
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpx/_transports/default.py", line 252, in __aiter__
    async for part in self._httpcore_stream:
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 361, in __aiter__
    async for part in self._stream:
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_async/http2.py", line 582, in __aiter__
    raise exc
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_async/http2.py", line 572, in __aiter__
    async for chunk in self._connection._receive_response_body(
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_async/http2.py", line 313, in _receive_response_body
    event = await self._receive_stream_event(request, stream_id)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_async/http2.py", line 333, in _receive_stream_event
    await self._receive_events(request, stream_id)
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_async/http2.py", line 361, in _receive_events
    events = await self._read_incoming_data(request)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_async/http2.py", line 452, in _read_incoming_data
    raise exc
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_async/http2.py", line 438, in _read_incoming_data
    data = await self._network_stream.read(self.READ_NUM_BYTES, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 31, in read
    with map_exceptions(exc_map):
  File "/usr/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ReadTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 193, in queue_worker_process
    asyncio.run(pwnedpasswords_get_store_gather(result_queue, hash_prefixes, worker_args))
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
2024-03-08T14:55:04+0100 | ESC[1;37mDEBUGESC[0m | hibp-downloader | Request attempt 1 of 5 for 'https://api.pwnedpasswords.com/range/0ec91'
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 197, in pwnedpasswords_get_store_gather
    metadata_results = await asyncio.gather(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 254, in pwnedpasswords_get_and_store_async
    binary, metadata_latest = await pwnedpasswords_get(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/hibp_downloader/commands/hibp_download.py", line 306, in pwnedpasswords_get
    response = await httpx_binary_response(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/hibp_downloader/lib/http.py", line 74, in httpx_binary_response
    response.binary = b"".join([part async for part in response.aiter_raw()])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/hibp_downloader/lib/http.py", line 74, in <listcomp>
    response.binary = b"".join([part async for part in response.aiter_raw()])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpx/_models.py", line 991, in aiter_raw
    async for raw_stream_bytes in self.stream:
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpx/_client.py", line 147, in __aiter__
    async for chunk in self._stream:
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpx/_transports/default.py", line 251, in __aiter__
    with map_httpcore_exceptions():
  File "/usr/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/dam/.virtualenvs/pwn/lib/python3.11/site-packages/httpx/_transports/default.py", line 84, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ReadTimeout

And now some retries (nice :+1: )
2024-03-08T14:53:30+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/05cbc'
2024-03-08T14:53:30+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/05cbf'
2024-03-08T14:53:51+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/075ea'
2024-03-08T14:54:14+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/0922a'
2024-03-08T14:55:04+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/0ccec'
2024-03-08T14:55:04+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/0cced'
2024-03-08T14:55:04+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/0ccf2'
2024-03-08T15:11:25+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/4e3ac'
2024-03-08T15:12:56+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/540c3'
2024-03-08T15:19:08+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/6bb9d'
2024-03-08T15:19:08+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/6bba1'
2024-03-08T15:39:30+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/b9611'
2024-03-08T15:39:30+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/b9615'
2024-03-08T15:40:41+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/bdb53'
2024-03-08T15:47:58+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/da102'

Unclear to me if that crashed thread is one of the retries...
That test was run on my laptop in my office (wifi) network using the default options. Running the same right now on my home network. Will also run a test on my home desktop (WSL).

Test from home network: No crashed threads, but more of these failed downloads:
2024-03-09T18:54:56+0100 | DEBUG | hibp-downloader | worker_index=6 prefix='d3215' hash_type='sha1' encoding_type='gzip' http_timeout=30 http_max_retries=5 http_debug=Falseignore_etag=False local_cache_ttl=43200 start_timestamp=2024-03-09 18:54:56.486975+01:00
2024-03-09T18:54:56+0100 | DEBUG | hibp-downloader | No existing metadata file 'test_home/sha1/d3/21/d3215.meta' using an empty entry
2024-03-09T18:54:56+0100 | DEBUG | hibp-downloader | Request attempt 1 of 5 for 'https://api.pwnedpasswords.com/range/d3215'
2024-03-09T18:55:06+0100 | WARNING | hibp-downloader | Request [1 of 5] failed for 'GET' 'https://api.pwnedpasswords.com/range/d3215'
2024-03-09T18:55:06+0100 | DEBUG | hibp-downloader | Request attempt 2 of 5 for 'https://api.pwnedpasswords.com/range/d3215'

But all failures succeeded on attempt 2. :+1:

Test in WSL (home network): With default options (which on this machine means 32 processes) it runs full boar for ~30-60 sec, then completely stops for ~30 sec (matching the http-timeout value). Then all the active download fails and it continues going at full speed again for another minute and then stops again and so on. Reducing processes / chunks size does improve things, but I had to go down to 8 processes and 1 chunk for getting it to not occasionally lock up. Might be some funky stuff in the WSL network stack I guess, since it worked mostly fine one the linux laptop in the same network.

source: gharchive/issue

authors: FlorianLoch ndejong wildegnux

repo: threatpatrols/hibp-downloader

license_type: permissive

license_source: github-api


text:
windows/tun: change wintun dependency to iwanbk/wintun

Because the original wintun crate doesn't set the MTU correctly
Fixes #474

@LeeSmet
i pointed out the dependency to my branch because i consider it as temporary, the end goal would be one of these:

put the code in threefoldtech/wintun master branch
contribute the code to the upstream

My plan is going to number two first, number one is the backup

will create github issue based on our agreement

The current fix works for us but would break users of wintun who rely on ipv4, so a commit upstream would need some more refinement. For now, I think the best way forward is to copy  the fixed function over to the windows tun module here in mcyelium. We already have some other code which basically does the same after all

The current fix works for us but would break users of wintun who rely on ipv4, so a commit upstream would need some more refinement

yes, of course.

For now, I think the best way forward is to copy the fixed function over to the windows tun module here in mcyelium

you meant copying this code (and maybe other deps or modify it)
https://github.com/iwanbk/wintun/blob/f2e5353c54cc6a8486b55112dc0de6e016491a8a/src/util.rs#L383-L413
Looks like that is also possible

We already have some other code which basically does the same after all

Can you elaborate more about this?

We currently already rely on netsh being available for windows as it is used to set the ipv6 address: https://github.com/threefoldtech/mycelium/blob/master/mycelium/src/tun/windows.rs#L119, so just copying the function and putting it here should be fine (i.e. running the netsh command to set the ipv6 mtu)

okay, looks better

OK, i close it in favor of #487

source: gharchive/pull-request

authors: LeeSmet iwanbk

repo: threefoldtech/mycelium


text:
Extra namespaces in QSFS zdb, doesn't decommission on destroy

I've deployed the latest QSFS example with no changes to the default values, on testnet. I noticed when inspecting the created zdb that there are additional namespaces beyond those used in my deployment. After I destroyed the deployment, the zdb is still available. I've queried contracts associated with my twin and don't see any that could correspond to the zdb deployment. Is it possible that I'm somehow using a zdb deployed by another user?
Before destroying:
> redis-cli -h 300:9fab:2cda:749d:aa8e:43c:aeee:dfef -p 9900

> NSLIST
 1) "default"
 2) "647-10987-meta1"
 3) "647-10987-meta2"
 4) "647-10987-meta3"
 5) "647-10987-meta4"
 6) "647-10987-data1"
 7) "647-10987-data2"
 8) "647-10987-data3"
 9) "647-10987-data4"
10) "647-10987-data5"
11) "647-10987-data6"
12) "647-10987-data7"
13) "647-10987-data8"
14) "647-10987-data9"
15) "647-10987-data10"
16) "647-10987-data11"
17) "647-10987-data12"
18) "647-10987-data13"
19) "647-10987-data14"
20) "647-10987-data15"
21) "647-10987-data16"
22) "647-10987-data17"
23) "647-10987-data18"
24) "647-10987-data19"
25) "647-10987-data20"
26) "81-11081-meta1"
27) "81-11081-meta2"
28) "81-11081-meta3"
29) "81-11081-meta4"
30) "81-11081-data1"
31) "81-11081-data2"
32) "81-11081-data3"
33) "81-11081-data4"

After destroying:
> NSLIST

 1) "default"
 2) "647-10987-meta1"
 3) "647-10987-meta2"
 4) "647-10987-meta3"
 5) "647-10987-meta4"
 6) "647-10987-data1"
 7) "647-10987-data2"
 8) "647-10987-data3"
 9) "647-10987-data4"
10) "647-10987-data5"
11) "647-10987-data6"
12) "647-10987-data7"
13) "647-10987-data8"
14) "647-10987-data9"
15) "647-10987-data10"
16) "647-10987-data11"
17) "647-10987-data12"
18) "647-10987-data13"
19) "647-10987-data14"
20) "647-10987-data15"
21) "647-10987-data16"
22) "647-10987-data17"
23) "647-10987-data18"
24) "647-10987-data19"
25) "647-10987-data20"

every node has one zdb instance only and when you create a zdb workload, it will just add a namespace for you on this zdb instance, and deletion will remove this namespace

Thanks for the explanation @AhmedHanafy725. Feels to me like this should be more isolated, but that's an issue for Zos.

repo: threefoldtech/terraform-provider-grid


text:
Add version endpoint

this could be used also as a health check from the dashboard and the app

let's update this one to include the public visible configs e.g min balances, number of trials, .. etc

Update:
Done
Just made separate endpoints for the configs and the version

/api/v1/configs
/api/v1/version

authors: AhmedHanafy725 sameh-farouk xmonader

repo: threefoldtech/tf-kyc-verifier


text: [Glass] breaks on smaller screens

authors: JorisBassle ken-de-moor


text:
Indentation after newline when appropriate.

For example:
if foo:
|

This is a travesty. I have to press one whole extra button before I can type code again! A tab, even! Way out of reach.
This is the current behavior because there is effectively no built in integration at the moment. Even something dumb like "after a :\n indent 4 spaces seems like it'd be better than the current state of things. :cry:
What I'd just :heart_eyes: to have is:
if foo:
    |

amirite?
I'd do it myself but it's midnight on Sunday here in ol' Los Angeles and my coworkers likely expect me to show up for work in the morning and extensions seem fairly complicated, just getting into them. I'd love to help though. Gimme your thoughts, I'm gonna go set 3 alarms for 8AM and watch Supernatural until I die of cringy plotlines and :pray: for the best.

I agree it is definitely an annoyance right now. I think a regex should work but might fail on edge cases, but hopefully users like yourself will report when this occurs.
I'll work to add this in the next release :smile:

I'd vote for "indent user-defined number of spaces", just to avoid unnecessary bloodshed :+1:

For consistency, I agree. On principle, screw anyone not using 4 space Python indents. :P

Vscode already has a preference setting for indentation size and spaces-or-tabs so using these would be the prefered option. Eliminate extra work for the user :+1:
But i do agree forcing 4 spaces is tempting.... :wink:

Fixed in 1902f8a.
As promised it defaults to 4 spaces if not set in the editor.tabSize setting :smile:

Thank you goodest of sirs.

authors: Raynes bgse tht13

license: mit

license_source: bigquery


text:
Fix RecipientLayoutCreator to retain contact name colors

When Settings → General settings → Colorize contacts is enabled, recipient names passed to RecipientLayoutCreator might be instances of Spanned. We therefore can't use String.format(), because that would convert the arguments to String and lose the spans.
This fixes a regression introduced in #7205.

Using "status: needs work" to remind myself to rewrite the branch history before merging. Should we (re)create a separate label for this?

Using "status: needs work" to remind myself to rewrite the branch history before merging. Should we (re)create a separate label for this?

I don't mind, but personally I don't have a need for this.

authors: cketti wmontwe

repo: thunderbird/thunderbird-android


text:
Unable to install FastAPI on Windows 10 using Python 3.8.1

I create a virtualenv then ran the command
pip install fastapi[all] but I keep going this error message
Collecting orjson<4.0.0,>=3.2.1; extra == "all"
Using cached orjson-3.3.0.tar.gz (654 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
Caused by:
process didn't exit successfully: rustc - --crate-name ___ --print=file-names -Z mutable-noalias --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg (exit code: 1)
--- stderr
See screenhot.

This is a orjson issue. Might be related to https://github.com/ijl/orjson/issues/33. Please follow the issue template.

This is a orjson issue. Might be related to ijl/orjson#33. Please follow the issue template.

This fixed it. Thanks very much.

Thanks for the help here @phy25 ! :clap: :bow:
Thanks for reporting back and closing the issue :+1:

authors: codemasta phy25 tiangolo

repo: tiangolo/fastapi

license: MIT


text:
🌐 Add German translation for docs/de/docs/advanced/response-change-status-code.md

← advanced/response-headers.md (#10628)
→ advanced/advanced-dependencies.md (#10633)
German translation progress

📝 Docs preview for commit 7f31770f61e82503e3211b6f7722d40e13c222c2 at: https://51881e84.fastapitiangolo.pages.dev

📝 Docs preview for commit f558b1183166d0d8d7042bd759c1faad433f723f at: https://19998c6a.fastapitiangolo.pages.dev

📝 Docs preview for commit f11fde100d6816e8cd1b512fa4c08ce8529b3ef3 at: https://53ab05b2.fastapitiangolo.pages.dev

📝 Docs preview for commit 9111d22fe1ce2fe8710ac5a4e5f4525e7f85940f at: https://a258b696.fastapitiangolo.pages.dev

Awesome @nilslindemann! :rocket:
As you are the champion of German translations and you have done so many, and with so much attention to detail, I'm gonna go ahead and merge this. :tada:
If others see things that could be improved later, we can check that in new PRs. :nerd_face:

authors: nilslindemann tiangolo


text:
🧪 Pydantic v2 migration, investigation, debug

🧪 Pydantic v2 migration, investigation, debug

I'll continue this in https://github.com/tiangolo/fastapi/pull/9500, with a branch on this repo to be able to push to it.


text:
build.sh and deploy.sh not copying latest code

I run these two scripts below, it looks like the build.sh copies the app code. When I run the deploy command and then go inside (exec /bin/bash) the container, I can't see the code changes.
TAG=prod FRONTEND_ENV=production bash ./scripts/build.sh
DOMAIN=xxx.com \ TRAEFIK_TAG=xxx.com \ STACK_NAME=xxx-com \ TAG=prod \ bash ./scripts/deploy.sh
If I do docker stack rm xxx-com then rerun the build.sh and deploy.sh it picks up the latest code changes in app. I thought I would be able to build and deploy without removing them from the stack, since this would cause some minor downtime if I need to docker stack rm each time.
Sorry if this is just a basic misunderstanding of my docker knowledge, I'm very new to it.

I also get this when I run DOMAIN=xxx.com \ TRAEFIK_TAG=xxx.com \ STACK_NAME=xxx-com \ TAG=prod \ bash ./scripts/deploy.sh
image backend:prod could not be accessed on a registry to record its digest. Each node will access backend:prod independently, possibly leading to different nodes running different versions of the image.
I don't get that for the other images except from celeryworker:prod

I also get this when I run DOMAIN=xxx.com \ TRAEFIK_TAG=xxx.com \ STACK_NAME=xxx-com \ TAG=prod \ bash ./scripts/deploy.sh
image backend:prod could not be accessed on a registry to record its digest. Each node will access backend:prod independently, possibly leading to different nodes running different versions of the image.
I don't get that for the other images except from celeryworker:prod
Also, the image ID shown on docker ps after running deploy does not match the backend:prod image ID when I run docker images, unless I take down the stack and rebuild then deploy again.
I guess this is some kind of private registry issue.

Regarding the error, this just indicates, that the image can't be found in a registry but your local. This means, that you built your image locally and other nodes could not access this image. This would result in two nodes with different images. If you only have one node, you can ignore this but if you have two or more you should definitely use a registry.
Why is this only shown for celeryworker ? - > The message is only printed for updated containers, this means only your celeryworker changed and the other containers not. Change the frontend or the backend and the error will be shown for them too.
You should not run docker stack rm as this would result in a downtime and loss of the logs and other stuff, always update with the deploy script. If your code isn't updated you can add - - no-cache to the build script. This will always include all changes. I did this by adding $@ to the end of the build script and provide it to build if needed.
I hope this helped, I'm on mobile and can't be as informative as I'd like to :)

@Biskit1943 thank you for your help, I'm still struggling to get this working, any more help would be appreciated.
I did the build with --no-cache flag which took a few mins, then deployed, then I exec bin/bash into the backend container and did an ls -la. It still hasn't updated the code in the app directory.

If I do ls -la outside the backend container in the host machines backend/app directory it has thisisatest.md

If I run docker images I get the following output, with the latest Image ID

My backend image that is running has the Image of 50a8920aeaa6 rather than backend:prod or the ID shown on the screenshot above

Could you try to run docker service update --force <name> not sure right now but this should do a force update of the container, which should in theory deploy the latest image.

Some more testing:
Build, deploy, then docker images

docker ps

Change code, build,  then docker images The previous backend and celeryworker images are still their, they have lost their repository and tag and are now

docker ps shows the following. It is now running the Image ID with the repository  and tag , as the build has made a new image with backend:prod and I assume two images can't have the same repo:tag:

Then finally I deploy and run 'docker ps' and I'm still stuck on the image tagged with   and latest backend:prod image isn't deployed.
docker ps

@Biskit1943 I will try the command you sent

Ok, let's recap.
You run TAG=prod FRONTEND_ENV=production bash ./scripts/build.sh to build all containers and tag them with prod
After that you run:
DOMAIN=example.com \
TRAEFIK_TAG=example.com \
STACK_NAME=example-com \
TAG=prod \
bash ./scripts/deploy.sh

You should see your STACK_NAME with docker stack ls and can see all running containers with docker stack ps STACK_NAME
Please make sure that you don't have the stack running with docker-compose up. I think you got things messed up by using docker-compose up to deploy the stack otherwise it won't be possible that the container is deployed with tag <none>...

I think this is the same issue that is described here https://github.com/moby/moby/issues/31357, regarding docker stack deploy not updating to the latest image. This does not seem solved.
In the end I worked around it by adding these lines to the end of scripts/deploy.sh (add any other services you need).
docker service update --force ${STACK_NAME}_backend
docker service update --force ${STACK_NAME}_frontend

A cleaner solution would be to tag the images incrementally and deploy a different tag on every update.

authors: Biskit1943 NMisko OddBlueDog

repo: tiangolo/full-stack-fastapi-postgresql


text:
DOM 编程艺术

DOM 基础
DOM 事件体系
事件的防抖与节流

DOM 基础
什么是 DOM
DOM（Document Object Model，文档对象模型）是 JavaScript 操作 HTML 的接口（这里只讨论属于前端范畴的 HTML DOM），属于前端的入门知识，同样也是核心内容，因为大部分前端功能都需要借助 DOM 来实现，比如：

动态渲染列表、表格表单数据；
监听点击、提交事件；
懒加载一些脚本或样式文件；

实现动态展开树组件，表单组件级联等这类复杂的操作。
如果你查看过 DOM V3 标准，会发现包含多个内容，但归纳起来常用的主要由 3 个部分组成：

DOM 节点
DOM 事件
选择区域

选择区域的使用场景有限，一般用于富文本编辑类业务，我们不做深入讨论；DOM 事件有一定的关联性，将在下一课时中详细讨论；对于 DOM 节点，需与另外两个概念标签和元素进行区分：

标签是 HTML 的基本单位，比如 p、div、input；
节点是 DOM 树的基本单位，有多种类型，比如注释节点、文本节点；
元素是节点中的一种，与 HTML 标签相对应，比如 p 标签会对应 p 元素。

举例说明，在下面的代码中，“p” 是标签， 生成 DOM 树的时候会产生两个节点，一个是元素节点 p，另一个是字符串为“Terry”的文本节点。

Terry

DOM 常用操作
查：DOM 节点获取
- getElementById // 按照 id 查询
- getElementsByTagName // 按照标签名查询
- getElementsByClassName // 按照类名查询
- querySelectorAll // 按照 css 选择器查询

增：DOM 节点的创建
请你创建一个新节点，并把它添加到指定节点的后面。

    <title>DEMO</title>

我是标题

删： DOM节点的删除
删除指定的 DOM 节点
// 获取目标元素的父元素
var container = document.getElementById('container')
// 获取目标元素
var targetNode = document.getElementById('title')
// 删除目标元素
container.removeChild(targetNode)

改：修改DOM元素
修改 DOM 元素这个动作可以分很多维度，比如说移动 DOM 元素的位置，修改 DOM 元素的属性等。
将指定的两个 DOM 元素交换位置

    <title>DEMO</title>

我是标题

我是内容

现在需要你调换 title 和 content 的位置，我们可以考虑 insertBefore 或者 appendChild。这里我们给出 insertBefore 的操作示范：
// 获取父元素
var container = document.getElementById('container')

// 获取两个需要被交换的元素
var title = document.getElementById('title')
var content = document.getElementById('content')

// 交换两个元素，把 content 置于 title 前面
container.insertBefore(content, title)

DOM 元素属性的获取和修改
仍然是上面这个 HTML 结构，现在假如需要你获取并修改 title 元素的 id 名，我们可以让 getAttribute 和 setAttribute 来帮忙：
var title = document.getElementById('title')

// 获取 id 属性
var titleId = title.getAttribute('id')

// 修改 id 属性
title.setAttribute('id', 'anothorTitle')

DOM 事件
自定义事件
<!DOCTYPE html>

  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta http-equiv="X-UA-Compatible" content="ie=edge">
  <title>Document</title>

我是A

我是B

我是C

如果我们仅仅想监听 divA 这个元素上的点击行为，我们可以用 addEventListener 来安装监听函数：

这是大家非常熟悉的操作。但是，如果我现在想实现这样一种效果：
在点击A之后，B 和 C 都能感知到 A 被点击了，并且做出相应的行为——就像这个点击事件是点在 B 和 C 上一样。
是不是觉得有点意思了？我们知道，借助时间捕获和冒泡的特性，我们是可以实现父子元素之间的行为联动的。但是此处，A、B、C三者位于同一层级，他们怎么相互感知对方身上发生了什么事情呢？
看看自定义事件怎么解决
首先大家需要了解的是，自定义事件的创建。比如说咱们要创建一个本来不存在的"clickA"事件，来表示 A 被点击了，咱们可以这么写：
var clickAEvent = new Event('clickA');

OK，现在事件有了，我们来完成事件的监听和派发：
// 获取 divB 元素
var divB = document.getElementById('divB')
// divB 监听 clickA 事件
divB.addEventListener('clickA',function(e){
  console.log('我是小B，我感觉到了小A')
  console.log(e.target)
})

// 获取 divC 元素
var divC = document.getElementById('divC')
// divC 监听 clickA 事件
divC.addEventListener('clickA',function(e){
  console.log('我是小C，我感觉到了小A')
  console.log(e.target)
})

// A 元素的监听函数也得改造下
divA.addEventListener('click',function(){
  console.log('我是小A')
  // 注意这里 dispatch 这个动作，就是我们自己派发事件了
  divB.dispatchEvent(clickAEvent)
  divC.dispatchEvent(clickAEvent)
})

可以看到被正确响应了。
大家可能也注意到了，这里我们安装和派发事件必须得拿到 divC、divA 这样确切的元素才行。有的时候，为了进一步解耦，我们也会考虑把所有的监听函数都塞到 document 这个元素上去，由它来根据不同类型的自定义事件采取不同的动作。这种思路，就和事件代理非常相似了。
事件代理
事件代理，又叫事件委托。
在如下的 HTML 里，我希望做到点击每一个 li 元素，都能输出它内在的文本内容。你会怎么做？
<!DOCTYPE html>

  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta http-equiv="X-UA-Compatible" content="ie=edge">
  <title>Document</title>

鹅鹅鹅

曲项向天歌

白毛浮绿水

红掌拨清波

锄禾日当午

汗滴禾下土

谁知盘中餐

粒粒皆辛苦

背不动了

我背不动了

按照事件的代理的思路我们可以把事件绑定到父元素 ul 上
var ul = document.getElementById('poem')
ul.addEventListener('click', function(e){
  console.log(e.target.innerHTML)
})

e.target 就是指触发事件的具体目标，它记录着事件的源头。所以说，不管咱们的监听函数在哪一层执行，只要我拿到这个 e.target，就相当于拿到了真正触发事件的那个元素。拿到这个元素后，我们完全可以模拟出它的行为，实现无差别的监听效果。
像这样利用事件的冒泡特性，把多个子元素的同一类型的监听逻辑，合并到父元素上通过一个监听函数来管理的行为，就是事件代理。通过事件代理，我们可以减少内存开销、简化注册步骤，大大提高开发效率。

防抖与节流

repo: tiantingrui/daily-harvest


text:
Add token access repositories scoping

Add token access repositories scoping. Reference: https://octokit.github.io/rest.js/v19#apps-create-installation-access-token
closes #73

+1

+1

authors: dorsegal mclavel vbyrd

repo: tibdex/github-app-token


text:
🛑 TICKETS.PH is down

In fe047a4, TICKETS.PH (https://tickets.ph) was down:

HTTP code: 0
Response time: 0 ms

Resolved: TICKETS.PH is back up in af76dfc after 68 days, 21 hours, 59 minutes.

repo: tickets/upptime


text:
🛑 TICKETS.COM.SG is down

In a9e11d1, TICKETS.COM.SG (https://tickets.com.sg) was down:

HTTP code: 502
Response time: 4128 ms

Resolved: TICKETS.COM.SG is back up in afa97a2 after 17 minutes.


text:
Skordle Scores App Removal

I'm the original author of the Skordle Scores app (#1935). I have recently realized that a misunderstanding of the cache function has resulted in the application being broken. After careful consideration, I've decided to remove the original application. I will replace it with a series of apps targeted at specific sports. I'm sorry for the trouble.

I uploaded some fixes for the application.

repo: tidbyt/community


text:
Hardcode NJ Transit station list

Description
NJ Transit updated their site and the stations are now hydrated by JS that doesn't seem easily extractable.
Instead, hardcode the list of stations as this should rarely change and can easily be extracted using a browser.
I also tried using their GraphQL API, but it requires authentication.
Copilot

🤖 Generated by Copilot at 661b279
Summary
🚉🛠💬

This pull request updates the njtransit app to use a hardcoded list of stations instead of scraping the website, which was causing errors. It also cleans up some obsolete code related to the scraping method.

STATIONS hard-coded
No more scraping the website
Winter of old code

Walkthrough

Replace scraping method for stations list with hardcoded list STATIONS to fix app functionality (link, link)
Update CSS selector for stations list in scraping function to match website changes (link)
Comment out unused scraping function fetch_stations_from_website and cache usage, but keep them for possible future use (link, link)

I have read the CLA Document and I hereby sign the CLA

I don't think I can get this under 500ms, NJ Transit's website is slow.


text:
New App: Tankerkoenig

Release follows later


text:
Add Christian Hauf to list of AUTHORIZED_CONTRIBUTORS.

This PR is to add Christian Hauf to list of AUTHORIZED_CONTRIBUTORS. We have a signed docusign CLA from them now.

@cb1kenobi @caspahouzer Review and merge please so we can re-pull in the PR that Hans reversed.

repo: tidev/organization-docs


text:
Request for good first issue

Hi @tiendc ,
I hope you are doing well. I found this project fascinating and would like to contribute more.
Can you raise some potential good first issue so I can start contributing towards them?
Thanks! ☺

@RahulDubey391 thanks a lot for your offer. I don't have a plan for new features as this lib is made based on the requirements of the projects I've been working on. If you have use case and need more functions, feel free to create issue and pull request. Adding new UT for complicated functions can also be an option.

repo: tiendc/gofn


text:
fix(planning_module): minimum prepare distance to zero

Signed-off-by: Muhammad Zulfaqar Azmi zulfaqar.azmi@tier4.jp
PR Type

Bug Fix

Description
Departure from private lane failed due to lane change parameters and slight bug.
https://github.com/autowarefoundation/autoware.universe/pull/1778
Related links
Tests performed
UC-F-10-00002_PSim_1_case1_cmn:1 scenario file is needed.
Review Procedure
UC-F-10-00002_PSim_1_case1_cmn:1 scenario file is needed.
Remarks

Pre-Review Checklist for the PR Author
PR Author should check the checkboxes below when creating the PR.

[ ] Code follows coding guidelines
[ ] Assign PR to reviewer

Checklist for the PR Reviewer
Reviewers should check the checkboxes below before approval.

[ ] Commits are properly organized and messages are according to the guideline
[ ] Code follows coding guidelines
[ ] (Optional) Unit tests have been written for new behavior
[ ] PR title describes the changes

Post-Review Checklist for the PR Author
PR Author should check the checkboxes below before merging.

[ ] All open points are addressed and tracked via issues or tickets
[ ] Write release notes

CI Checks

Build and test for PR: Required to pass before the merge.
Check spelling: NOT required to pass before the merge. It is up to the reviewer(s). See here if you want to add some words to the spell check dictionary.

@zulfaqar-azmi-t4
Should this PR be merged? Or, Is it okay to close it?

@zulfaqar-azmi-t4
ping

@tkimura4
Sorry for the late respond. I'll close the pr.


text:
Can't find FusionSolar in "add Integration"

I have set up HACS over SSH terminal and i follow your installation description.
As you see in my screenshots, but I'cant find "FusionSolar" in the list. Did I miss something?

Fount it - was in the wrong page.

repo: tijsverkoyen/HomeAssistant-FusionSolar


text:
Test 1

Testing this as a way to develop the app wireframes

Idea
Idea
Idea

Does this image show?

My thoughts are XYZ

What about this?

@BOC111 its a website bud. if you can see it, I can see it ;)

repo: tiki-bar/app


text:
Summary of email activity

As a user, I want to see a summary of my email activity, so I have a clear view who's emailing me the most and what my email habits are.
Acceptance criteria: As a user who's linked their Gmail account, I can see at a glance valuable information about my email activity:

Headline: "In the past 1 week, you got more than 200 emails that you didn’t open"
technical considerations: Notify user if they have more than 50 emails that they haven't opened. Round it to the closest  00 ie. "more than 100 emails" "more than 1200 emails". If they have less than 50 unopened emails for that week, tey "In the past month..."

Body: "Adweek, +55 others email you
more than once a week"
technical considerations: present the logos of the top 3 companies that email them, and list how many email them more than once a week

Needs definition of which data will be displayed and the origin of this data.

closing. feature not needed

authors: annastoilova mike-audi ricardobrg

repo: tiki/app


text:
Update build process for Android plugin

Without the tiki_sdk_flutter_plugin, the build process for android native aar is broken. The native implementation of tiki_sdk_android works with old binary built with tiki_sdk_flutter and tiki_sdk_flutter_plugin, but without tiki_sdk_flutter_plugin, it does not work.
Update the build process including a lean tiki_sdk_fluter_plugin that will just handle the platform channel calls. This dependency should not be included in tiki_sdk_flutter default pubspec, to avoid the inclusion of this code in pure dart implementations.
Acceptance criteria:
1 - A build script for generating the android aar and push to maven central
2 - tiki_sdk_android tests passes using the new maven dependency

Publish to maven central will be done in https://github.com/tiki/tiki-sdk-android/issues/27

repo: tiki/tiki-sdk-flutter


text:
WASM target support improvements

As a followup to https://github.com/tikv/minitrace-rust/pull/201 , I noticed an adjustment we could make to make minitrace more compatible with WASM runtime.
The main issue around HTTP calls with wasm32-unknown-unknown is that the JS runtime (fetch) makes the actual HTTP call.
This means that the exporting flow of collected traces needs to use a non-blocking, async HTTP client. So HTTP calls will be handled in "background" (by the JS runtime that loads the WASM), and returns a Promise that's being converted to Future.
To address such change, I think minitrace can introduce an option (maybe as a feature?) to have async reporters (and async flush function).
Also, most reporters today are using reqwest::blocking client, which is not available on WASM targets.
@andylokandy  I'll be happy to make a PR to add such a feature if that makes sense.

You can wrap an async function in the sync report function, by simply discarding the Future or storing the Future in a static mutex.

Yeah, but in this case, an async reporter might even be more useful 🤔 I can hook that into the wait_until (https://developers.cloudflare.com/workers/runtime-apis/handlers/fetch/#contextwaituntil) and just await it there without blocking anything.
I can understand the reason for a simple, sync report in minitrace, but in this specific case it feels like a lot of workarounds are needed.
Doing async (runtime) -> sync (Reporter) -> async (actual reporting) feels like a workaround, that can be simplified if report function was async 🤔

You can do: manually flushing -> save spans to be report in static/Arc -> report the spans in async context

You can do: manually flushing -> save spans to be report to static/Arc -> report the spans in a async context

That's a great idea! I will give it a try :)

Ok, I managed to make it work!
So basically I created a new special struct that implements Reporter. The report function only collects the span in memory, and I added an async function to do the actual reporting. I can now connect that to the rest of the setup we have.
Thank you for the help @andylokandy , I really appreciate it!

authors: andylokandy dotansimha

repo: tikv/minitrace-rust


text:
fix docker build

Signed-off-by: tabokie xy.tao@outlook.com
What is changed and how it works?
Issue Number: Ref #11312
Fix a series of errors during make docker:

failed to parse manifest
Could not find protoc installation
the linking modifiers +bundle and +whole-archive are not compatible with each other when generating rlibs
error while loading shared libraries: libstdc++.so.6

Fix `make docker`.

Related changes
Check List
Tests

Unit test

Release note
None

I checked out this PR and "make docker" still failed:
   Compiling server v0.0.1 (/tikv/components/server)
error: the linking modifiers `+bundle` and `+whole-archive` are not compatible with each other when generating rlibs

error: could not compile `tikv-ctl` due to previous error
warning: build failed, waiting for other jobs to finish...
error: could not compile `tikv-server` due to previous error
+ r=101
+ set +x
make[1]: *** [Makefile:426: x-build-dist] Error 101
make[1]: Leaving directory '/tikv'
make: *** [Makefile:257: build_dist_release] Error 2
The command '/bin/sh -c source /opt/rh/devtoolset-8/enable && make build_dist_release' returned a non-zero code: 2
make: *** [Makefile:392: docker] Error 2

/merge

/run-test

authors: kolbe tabokie

repo: tikv/tikv


text:
tidb_query: fix convert integer to duration

Signed-off-by: Lonng heng@lonng.org

What have you changed?

This PR fixes convert an integer to the duration while the integer greater than 10000000000, see: https://github.com/pingcap/tidb/blob/master/types/convert.go#L338:L338
This PR delete some duplicated test cases.

What is the type of the changes?

Bugfix (a change which fixes an issue)

How is the PR tested?

Unit test

/run-all-tests

/merge


text:
"Error: Unclosed element div" for template that worked with Handlebars

I have the following template that worked with Handlebars:
{{#if disableInteraction}}

{{/if}}
  Lots of stuff here
{{#if disableInteraction}}

{{/if}}

This worked great with Handlebars for Ember CLI. However, When trying to compile this using HTMLBars for Ember CLI, I get the following error:
Unclosed element `div` (on line 2)
Error: Unclosed element `div` (on line 2).
    at HTMLProcessor.nodeHandlers.Program (/path/to/app/bower_components/ember/ember-template-compiler.js:3385:17)
    at HTMLProcessor.acceptNode (/path/to/app/bower_components/ember/ember-template-compiler.js:3579:43)
    at HTMLProcessor.nodeHandlers.BlockStatement (/path/to/app/bower_components/ember/ember-template-compiler.js:3405:44)
    at HTMLProcessor.acceptNode (/path/to/app/bower_components/ember/ember-template-compiler.js:3579:43)
    at HTMLProcessor.nodeHandlers.Program (/path/to/app/bower_components/ember/ember-template-compiler.js:3375:16)
    at HTMLProcessor.acceptNode (/path/to/app/bower_components/ember/ember-template-compiler.js:3579:43)
    at preprocess (/path/to/app/bower_components/ember/ember-template-compiler.js:3551:55)
    at compileSpec (/path/to/app/bower_components/ember/ember-template-compiler.js:741:17)
    at TemplateCompiler.__exports__.default [as precompile] (/path/to/app/bower_components/ember/ember-template-compiler.js:643:14)
    at TemplateCompiler.processString (/path/to/app/node_modules/ember-cli-htmlbars/index.js:58:38)

Is this sort of template no longer supported, or is this a bug in HTMLBars?

This worked great with HTMLBars for Ember CLI. However, When trying to compile this using HTMLBars for Ember CLI, I get the following error:

That confuses me.  Do you mean that it worked fine with Handlebars, but not HTMLBars?

Even in Handlebars, this would not have worked properly (with bindings and whatnot).  See this JSBin using 1.9.0 that shows what I mean.  When you check the isEnabled checkbox, it adds the div's, but they are not updated properly (meaning Other Stuff is not in the right place).

@Kerrick Correct, it is not supported. You can think of block helpers like the new <template> tag.

That confuses me. Do you mean that it worked fine with Handlebars, but not HTMLBars?

Sorry, I did mean that.

Even in Handlebars, this would not have worked properly (with bindings and whatnot). See this JSBin using 1.9.0 that shows what I mean. When you check the isEnabled checkbox, it adds the div's, but they are not updated properly (meaning Other Stuff is not in the right place).

Interesting, it looks like this never actually worked and was always silently failing. Thanks for clarifying!

I understand that this is a feature, but does anybody know how we can render nested divs using HTMLbars, or is there any other tempating engine compatible with Ember?

Any news on how this could work without having to duplicate code ? Great work on Ember by the way :smile:

authors: Kerrick TheRedDevil mmun pvchaumier rwjblue

repo: tildeio/htmlbars


text:
Retained topics sometimes removed, found solution

I noticed when sometimes data is missing in de VW app (don't know why, but see screenshot attached; available km range and available % fuel is not displayed in the app when this happens), it is also directly missing in the raw jason, and then WeConnect-mqtt removes the topic from the broker. (My home-automation sets the values then to zero, causing problems in my sensors/scripts, I'd rather prefer retaining the old topics and values, and wait a few hours for data available again at the VW's server.)
A retained topic is removed from a broker by publishing the topic with the retain-flag and a null ('') payload. WeConnect-mqtt does this on line 544, by sending this command to the broker. I've commented out this command:
544c544
<             self.publish(topic=f'{self.prefix}{element.getGlobalAddress()}', qos=1, retain=True, payload='')
---
>             #self.publish(topic=f'{self.prefix}{element.getGlobalAddress()}', qos=1, retain=True, payload='')

Now topics are always retained. Perhaps you could make this configurable with a parameter? (--alwaysretain)?
I've tested a few days and the number of topics stay the same on my broker. Now I can confirm it working for the VW California 6.1 camper, with mosquitto version 2.0.11, Domoticz 2024.7 using domoticz-mqttmapper-plugin

HI!
I guess I have the same Issue since some days. WeConnect-MQTT does not update Topics anymore.
I am running an Ubuntu Server with PIP installed WeConnect-MQTT. In which Folder i can find the main file of WeConnect-MQTT  to change line 544 as you suggested?

HI! I guess I have the same Issue since some days. WeConnect-MQTT does not update Topics anymore. I am running an Ubuntu Server with PIP installed WeConnect-MQTT. In which Folder i can find the main file of WeConnect-MQTT to change line 544 as you suggested?

Hi,
Your problem is unclear to me. Are topics removed or not updated?
If:

Topics are removed from broker, you could use the solution above.
Or:
Topics are still at broker but not updated, then you should accept the terms from VW at the site. (See https://github.com/tillsteinbach/WeConnect-mqtt/issues/376 )

I'm running WeConnect-MQTT in docker, so I'm unsure where the file is stored in your Ubuntu. You could do a search (as root) for weconnect_mqtt_base.py ?
Regards, Hans

repo: tillsteinbach/WeConnect-mqtt


text:
Fix hoverRect bug

related #45
issue
hoverRect is unexpectedly not removed, when the element is un-hovered during you are dragging it.
cause
In snapsvg.js, Unhover events associated with hoverRect are not fires as long as s.elementDragging = true
This pull request...

add a bool property unhoveringDuringDragging
on dragEnd, check unhoveringDuringDragging and remove hoverRect if necessary

Thanks for reading!

Ah brilliant, thanks so much for the PR!
I'll check this over this week and get it merged in 👍

This issue persists in 1.1.1, for me at least, when running Firefox 48.0.1 on OSX 10.11.6.

authors: UNFPAmaldives chrishutchinson satotake

repo: times/cardkit


text:
feat: use commands for scoping the cli

Instead of the cli running directly the vectorizer worker, the worker has been moved to its own command. The entry point has been rename from vectorizer to pgai.
This will allow us to add other commands seamlessly in the future, without having to introduce breaking changes or multiple scripts.
The workflow will look like this:
$ pip install pgai
$ pgai --help
Usage: pgai [OPTIONS] COMMAND [ARGS]...

Options:
  --version  Show the version and exit.
  --help  Show this message and exit.

Commands:
  vectorizer

$ python -m pgai vectorizer --help
Usage: python -m pgai vectorizer [OPTIONS] COMMAND [ARGS]...

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  worker

$ pgai vectorizer worker --help
Usage: pgai vectorizer worker [OPTIONS]

Options:
  --version                       Show the version and exit.
  -d, --db-url TEXT               The database URL to connect to  [default:
                                  postgres://postgres@localhost:5432/postgres]
  -i, --vectorizer-id INTEGER     Only fetch work from the given vectorizer
                                  ids. If not provided, all vectorizers will
                                  be fetched.
  -c, --concurrency INTEGER RANGE
                                  [default: 1; x>=1]
  --log-level [DEBUG|INFO|WARN|ERROR|FATAL|CRITICAL]
  --poll-interval TIME DURATION   The interval, in duration string or integer
                                  (seconds), to wait before checking for new
                                  work after processing all available work in
                                  the queue.  [default: 5m]
  --once BOOLEAN                  Exit after processing all available work.
                                  [default: False]
  --help                          Show this message and exit.

Nit: From a UX point of view, I find pgai vectorizer-worker command sequence not pretty descriptive nor natural. Additionally, from the technical point of view, if you want to add more vectorizer subcommand , you are tied to use vectorizer-<subcommand> name pattern.
Can we add a proper subcommand level such as pgai vectorizer run or pgai vectorizer worker? Still pgai vectorizer could run a default behaviour. That let's user understand there can be more subcommand for vectorizer.

@smoya I went with pgai vectorizer worker.

@alejandrodnm can you ping me when this is live and I will update the docs to match. I'd like to do it in a separate PR as there are a couple of other doc moving parts at the moment.

authors: alejandrodnm billy-the-fish smoya

repo: timescale/pgai

license: PostgreSQL


text:
WIP: better handling of timescaleDB secrets

This PR is a work in progress.

Superseded by #305

repo: timescale/tobs


text:
Update remote depedency hdrhistogram

This PR updates the remote repo of hdrhistogram from https://github.com/filipecosta90/hdrhistogram to the now official https://github.com/HdrHistogram/hdrhistogram-go/ repo.
The PercentilesPrint usage has been adjusted accordingly.

Codecov Report

Merging #136 into master will decrease coverage by 0.11%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #136      +/-   ##
==========================================
- Coverage   69.73%   69.62%   -0.12%
==========================================
  Files         110      110
  Lines        5984     5158     -826
==========================================
- Hits         4173     3591     -582
+ Misses       1722     1479     -243
+ Partials       89       88       -1

Impacted Files
Coverage Δ

query/stat_processor.go
10.22% <0.00%> (-5.91%)
:arrow_down:

query/stats.go
100.00% <ø> (ø)

internal/inputs/utils.go
72.22% <0.00%> (-6.04%)
:arrow_down:

cmd/tsbs_load_victoriametrics/processor.go
68.42% <0.00%> (-5.50%)
:arrow_down:

cmd/tsbs_load_victoriametrics/scan.go
70.00% <0.00%> (-5.00%)
:arrow_down:

cmd/tsbs_generate_data/serialize/MongoTag.go
63.15% <0.00%> (-4.70%)
:arrow_down:

cmd/tsbs_load_siridb/scan.go
22.00% <0.00%> (-3.87%)
:arrow_down:

...d/tsbs_generate_queries/databases/siridb/common.go
83.33% <0.00%> (-3.34%)
:arrow_down:

...s_generate_queries/databases/timescaledb/common.go
80.00% <0.00%> (-3.34%)
:arrow_down:

... and 101 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7508b34...1560e53. Read the comment docs.

@RobAtticus can you check this one? The used dependency has now a proper remote org and we should move it

Looks good!
Just fix the conflicts with master and we can merge

thank you @blagojts , just resolved the conflicts. can you please revise?

repo: timescale/tsbs


text:
Why is grayscale image defined as Gray{UfixedBase{Uint8,8}?

I've read in a grayscale image which has the following attributes:
julia> cimg
Gray Image with:
data: 1920x1080 Array{Gray{UfixedBase{Uint8,8}},2}
properties:
IMcs: Gray
spatialorder:  x y
pixelspacing:  1 1
It's an array of Ufixed8 meaning values between 0 and 1. for example:
julia> cimg[1,1]

Gray{Ufixed8}(0.541)
I'd like my grayscale values to range from 0 to 255 (so I could do a histogram on the image, for example). Is there a way to convert to an array of values between 0 and 255?

Hello Tim, linspace should not be?¿
 linspace(FixedPointNumbers.Ufixed8(0),FixedPointNumbers.Ufixed8(1),256)

I Tried the following function for the histogram:
function histogram3(img)
    vals = linspace(FixedPointNumbers.Ufixed8(0),FixedPointNumbers.Ufixed8(1),256)
    h = Array(Float64,256)
    m,n = size(img)
    N = m * n
    for l = 1:length(vals)
        counter = 0.0
        @nloops 2 i img begin
            if((@nref 2 img i) == vals[l])
                counter += 1.0
            end
            h[l] = counter
        end
    end
    return h ./ N
end

You should be able to do that as
_, counts = hist(img[:], -1/256:1/256:1)

and it will be much faster.

authors: elsuizo philtomson timholy


text:
System Bridge 3.0.0

Description
⚠ Major breaking change. This is a  completely different application and will replace v2.x.x ⚠
Replace the backend with a python application. Remove NodeJS as backend. Revamp Frontend and GUI to work with the new system.
TODO

[x] Authentication (api-key or something else)
[x] Limit data API to implemented modules
[x] Update on timer (after setup)
[x] Bridges (api)
[x] Notifications
[x] #1604
[x] #1578
[x] Open Path/URL
[x] Keyboard (https://github.com/asweigart/pyautogui ?)
[x] Media/Filesystem (read, write, media browser integration - https://github.com/ActiveState/appdirs)
[x] Convert numbers to numbers and None to null in JSON API output
[x] Logging to log file
[x] Settings
[x] WebSocket (https://github.com/timmo001/system-bridge/blob/master/src/components/api/events/events.gateway.ts)
[x] Settings from WebSocket
[x] Frontend rework
[x] GUI rework
[x] Launch GUI (Launch from backend if GUI avaliable/enabled)
[x] #1555
[x] #1556
[x] #1566
[x] #1581
[x] #1599
[x] #1561
[x] #1598
[x] https://github.com/timmo001/system-bridge-site/issues/367
[x] #1562
#1557 (Removed)

Optional/Future

[ ] #1554
[ ] #1553
[ ] #1563
[ ] #1619
[ ] #1620
[ ] #1618
[ ] #1621
[ ] #1615
[ ] #1614
[ ] #1616
[ ] #1617
[ ] #1611
[ ] #1612
[ ] #1613
[ ] #1552
[ ] #1570
MacOS (Removed due to lack of knowledge)

Packages to look at:

https://github.com/kivy/plyer
https://github.com/pyusb/pyusb
https://github.com/AndreMiras/pycaw
https://github.com/asweigart/pyautogui
https://github.com/boppreh/keyboard
https://github.com/boppreh/mouse
https://github.com/gladykov/rebooter
https://github.com/pybluez/pybluez
https://github.com/yunstanford/pytest-sanic
https://github.com/ActiveState/appdirs

Moved from #1529

Future issues opened. Closing issue and milestone. Ready to release


text:
ReqwestLoader improvements

This PR adds a client option to the ReqwestLoader allowing one to use its own reqwest configuration and middleware using the reqwest-middleware library, fixing #68.
The client is now assumed to be dealing with redirections, fixing #69.

This also replaces #63

repo: timothee-haudebourg/json-ld


text:
Add 'Tetris Metadata Subtitles Extractor', modules, & CLI menu.

In the spirit of implementing dubious features that no one specifically requested but I personally find interesting 😅, here's a PR that implements #8, begins adding modules (single monolithic generate.py function, be gone!! 👋), and also adds a basic CLI menu for easier navigation.
In Summary:
https://user-images.githubusercontent.com/45280066/111083475-3619c300-84e4-11eb-9bc3-24cdc0441df9.mp4

Notes:

To run:  Navigate to your root and run python -m main.runner.
All files you're interested in processing need to posses the .mp4 extension and be housed in .\input\.
All generated .srt files will be placed in .\output\
If you're processing a large file and want to exit early, hitting ctrl-c on your keyboard will exit the main processing loop but not the app.
Still need to do a ton of stuff like handle bad CLI menu inputs, etc.  It's on the docket.  Currently, the app works fine if you behave... 😄

Pictures:
So, for example-- put your input video (*.mp4) files here that you wish to process:

And after running the subtitles generator, your .srt will appear here:

Your generated .srt should look like this:

Using an app like VLC, add the subtitle; it should look like this:

I was just gonna do something similar but saw this PR which is anyway way better than I could've done

Hi @JupeOwl !  Welcome!  👋😄  So great to see a new face here in Yobi World! 😀
Haha, thanks for the kind words!  I know some Python, but tbh, there's probably 90% of the standard library that I don't know.  Python is so incredibly powerful and can do so much!  I'd chatted with Yobi about a month ago in his Discord and in the end we didn't pursue a CLI menu (for good reasons!) but this was still a fun side-project that I'd enjoyed hacking together.  I didn't go farther with this at the time, but my thinking was that:

Since many newbies might be using the tool, it'd be cool to offer two modes:

A CLI menu version for total beginners who are just interested in the Tetris-side (extracting stats, etc).
A command line version for old-school, advanced users who are interested in embedding Yobi's tool in their own scripts.

My thinking was that once you extracted stats in a plaintext format out of a video, then you could process that metadata in Excel/Sheets/pandas/a Jupytr notebook to do Nate Silver-style/538 analysis.  The potential is endless!! 😁

(Btw, if you're curious: I actually ended up writing about my PR/'ideas-for-next-steps' here:  https://community.codenewbie.org/r002/2-python-opencv-classic-tetris-monthly-summary-tool-mar-12-2021-nep)
In general, please contribute and make a PR though!  Yobi is awesome and Yobi's CTM Summary Tool is awesome!  🎉🌟✨  Seriously though, the world needs more open-source!  Open-source is the way!  🚀🚀🚀


text:
Add myself to list of code contributors

For a list of past contributions, see:
https://github.com/timothycrosley/isort/commits?author=jdufresne

Long overdue! Thanks again for all your great contributions!

authors: jdufresne timothycrosley

repo: timothycrosley/isort


text:
Feature Request: Specify custom PayloadType

macOS 10.13.2 beta 2 introduces a new MDM PayloadType to manage whitelisted KEXTs settings only thru UAMDM.
The new KEXT whitelist policy now has a PayloadType of com.apple.syspolicy.kernel-extension-policy instead of the default com.apple.ManagedClient.preferences.  I'm sure there will be more PayloadTypes coming down the line as UAMDM seems to be the path for new security features so having an option to override the default Type would be handy in the future.
-Eric

mcxToProfile is for turning MCX or other macOS preference files into
profiles - I don't think that it'd ever take another payload type as
com.apple.ManagedClient.preferences is exactly what it is for.
How would you propose this to work?  Taking a generic plist and shoving it
into the payloadtype you've specified?
On Fri, Nov 10, 2017 at 7:01 AM, eholtam notifications@github.com wrote:

macOS 10.13.2 beta 2 introduces a new MDM PayloadType to manage
whitelisted KEXTs settings only thru UAMDM.
The new KEXT whitelist policy now has a PayloadType of
com.apple.syspolicy.kernel-extension-policy instead of the default
com.apple.ManagedClient.preferences. I'm sure there will be more
PayloadTypes coming down the line as UAMDM seems to be the path for new
security features so having an option to override the default Type would be
handy in the future.
-Eric
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/timsutton/mcxToProfile/issues/18, or mute the thread
https://github.com/notifications/unsubscribe-auth/AB_BM_zSxIXInv4IiHc8PZMDXccmmBzlks5s1GVXgaJpZM4QZrMa
.

--
Samuel Keeley

Yes, that's how I've been using this tool for some time now. I create a plist of the specific keys that I want in a profile and feed it thru mcxtoprofile. The output is a mobileconfig with the payload of the keys that I provided. I have a PR that appears works for my one use case but I'm very newbish in Python so more eyes on it would be welcomed.

An example:
I have a plist of
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>AllowUserOverrides</key>
	<false/>
	<key>AllowedTeamIdentifiers</key>
	<array>
        <string>293UQF7R4S</string>
        <string>3G47DU9NXM</string>
        <string>3T5GSNBU6W</string>
        <string>4C6364ACXT</string>
        <string>54GTJ2AU36</string>
        <string>6KR3T733EC</string>
        <string>6LS4845VS3</string>
        <string>73YQY62QM3</string>
        <string>7J8K6Q87RJ</string>
        <string>8J7TAMPT4P</string>
        <string>9PTGMPNXZ2</string>
        <string>9VB857J7CD</string>
        <string>9ZGFBWLSYP</string>
        <string>CPQQ3AW49Y</string>
        <string>DE8Y96K9QP</string>
        <string>E56RCXEQ93</string>
        <string>EG27766DY7</string>
        <string>EG7KH642X6</string>
        <string>G43BCU2T37</string>
        <string>G7HH3F8CAK</string>
        <string>J6L96W8A86</string>
        <string>PLT8TQHCAJ</string>
        <string>QED4VVPZWA</string>
        <string>U42NNPDKG7</string>
        <string>VB5E2TV963</string>
        <string>WSG985FR47</string>
        <string>YZC2T44ZDX</string>
        <string>Z3L495V9L4</string>
	</array>
</dict>
</plist>

I feed that thru mcxtoprofile via mcxToProfile.py --plist com.apple.syspolicy.kernel-extension-policy.plist --identifier com.apple.syspolicy.kernel-extension-policy -g "Company Name" --displayname=com.apple.syspolicy.kernel-extension-policy --payloadtype com.apple.syspolicy.kernel-extension-policy -o ./com.apple.syspolicy.kernel-extension-policy.mobileconfig
That outputs:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>PayloadContent</key>
	<array>
		<dict>
			<key>PayloadContent</key>
			<dict>
				<key>com.apple.syspolicy.kernel-extension-policy</key>
				<dict>
					<key>Forced</key>
					<array>
						<dict>
							<key>mcx_preference_settings</key>
							<dict>
								<key>AllowUserOverrides</key>
								<false/>
								<key>AllowedTeamIdentifiers</key>
								<array>
									<string>293UQF7R4S</string>
									<string>3G47DU9NXM</string>
									<string>3T5GSNBU6W</string>
									<string>4C6364ACXT</string>
									<string>54GTJ2AU36</string>
									<string>6KR3T733EC</string>
									<string>6LS4845VS3</string>
									<string>73YQY62QM3</string>
									<string>7J8K6Q87RJ</string>
									<string>8J7TAMPT4P</string>
									<string>9PTGMPNXZ2</string>
									<string>9VB857J7CD</string>
									<string>9ZGFBWLSYP</string>
									<string>CPQQ3AW49Y</string>
									<string>DE8Y96K9QP</string>
									<string>E56RCXEQ93</string>
									<string>EG27766DY7</string>
									<string>EG7KH642X6</string>
									<string>G43BCU2T37</string>
									<string>G7HH3F8CAK</string>
									<string>J6L96W8A86</string>
									<string>PLT8TQHCAJ</string>
									<string>QED4VVPZWA</string>
									<string>U42NNPDKG7</string>
									<string>VB5E2TV963</string>
									<string>WSG985FR47</string>
									<string>YZC2T44ZDX</string>
									<string>Z3L495V9L4</string>
								</array>
							</dict>
						</dict>
					</array>
				</dict>
			</dict>
			<key>PayloadEnabled</key>
			<true/>
			<key>PayloadIdentifier</key>
			<string>MCXToProfile.fcb6630c-f0c4-4363-b333-3f473a2bde64.alacarte.customsettings.6032b9d8-8e0a-4181-b7a4-5744b91522f0</string>
			<key>PayloadType</key>
			<string>com.apple.syspolicy.kernel-extension-policy</string>
			<key>PayloadUUID</key>
			<string>6032b9d8-8e0a-4181-b7a4-5744b91522f0</string>
			<key>PayloadVersion</key>
			<integer>1</integer>
		</dict>
	</array>
	<key>PayloadDescription</key>
	<string>Included custom settings:
com.apple.syspolicy.kernel-extension-policy

Git revision: a0e04c60fc</string>
	<key>PayloadDisplayName</key>
	<string>com.apple.syspolicy.kernel-extension-policy</string>
	<key>PayloadIdentifier</key>
	<string>com.apple.syspolicy.kernel-extension-policy</string>
	<key>PayloadOrganization</key>
	<string>Company Name</string>
	<key>PayloadRemovalDisallowed</key>
	<true/>
	<key>PayloadScope</key>
	<string>System</string>
	<key>PayloadType</key>
	<string>Configuration</string>
	<key>PayloadUUID</key>
	<string>fcb6630c-f0c4-4363-b333-3f473a2bde64</string>
	<key>PayloadVersion</key>
	<integer>1</integer>
</dict>
</plist>

I can see why you'd want to be able to do this, although we probably could have made an argument for a while to include any of the many other PayloadTypes which Profile Manager has been able to generate, going on 7 years now. mcxToProfile was originally just meant as a mechanism to do with profiles what we were doing before with MCX, using Apple's managed client support.
I'm not necessarily opposed to the idea of allowing arbitrary payload types as a flag. However, because this seems (to me, at least) to be one of Apple's ways to extend its management capabilities via config profiles, how can we know that the profiles it would generate using other Payload Types would match other tools from Apple which are also able to generate profiles? One option could be that we state this is an experimental feature and it's just going to set the value to what you want without any other changes to the structure of the data.

This is definitely experimental.  My typical use case for this tool is for non-Apple payloads. I use Apple’s tools (Profile Manager or Configurator) for Apple specific domains to make sure the necessary bits are in the profile. Since I haven’t seen this new kext payload type available in their tools it got me thinking on how to systematically create test profiles.
-Eric

On Nov 10, 2017, at 3:23 PM, Timothy Sutton notifications@github.com wrote:
I can see why you'd want to be able to do this, although we probably could have made an argument for a while to include any of the many other PayloadTypes which Profile Manager has been able to generate, going on 7 years now. mcxToProfile was originally just meant as a mechanism to do with profiles what we were doing before with MCX, using Apple's managed client support.
I'm not necessarily opposed to the idea of allowing arbitrary payload types as a flag. However, because this seems (to me, at least) to be one of Apple's ways to extend its management capabilities via config profiles, how can we know that the profiles it would generate using other Payload Types would match other tools from Apple which are also able to generate profiles? One option could be that we state this is an experimental feature and it's just going to set the value to what you want without any other changes to the structure of the data.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

authors: keeleysam poundbangbash timsutton

repo: timsutton/mcxToProfile


text:
How to config?

Please help me

I have only the token tg bot

Remove getenv("token") from the token=getenv("token") part and direcyly assign your token value(str)
e.g. token = "telegramToken"
After, type /start on telegram to start bot

And games the ids must be how :
id,id ? id, id ? id , id ?

authors: Magic-Hole tingirifistik

repo: tingirifistik/Steam-HourBooster


text:
Undefinded behaviour of DataFrame.loc[:, pd.IndexSlice[:, ["a", "b"]]] between 1.1.* and >= 1.2

IMPORTANT: Please do not create a Pull Request without creating an issue first.
Before submitting (must do checklist)

[x] Did you read the contribution guide?
[x] Did you update the docs? We use Numpy format for all the methods and classes.
[x] Did you write any new necessary tests?
[ ] Did you update the CHANGELOG?

Type of Change

[ ] Examples / docs / tutorials / contributors update
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Improvement (non-breaking change which improves an existing feature)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Proposed Changes

https://github.com/pandas-dev/pandas/pull/37992/files
Related Issue

#162
Closing issues

Codecov Report

Merging #188 (f3ea3f7) into master (9c37a03) will decrease coverage by 29.89%.
The diff coverage is 0.00%.

:exclamation: Current head f3ea3f7 differs from pull request most recent head 31414c6. Consider uploading reports for the commit 31414c6 to get more accurate results

@@             Coverage Diff             @@
##           master     #188       +/-   ##
===========================================
- Coverage   87.78%   57.88%   -29.90%
===========================================
  Files          75       75
  Lines        3438     3443        +5
===========================================
- Hits         3018     1993     -1025
- Misses        420     1450     +1030

Impacted Files
Coverage Δ

etna/transforms/sklearn.py
34.24% <0.00%> (-61.41%)
:arrow_down:

etna/models/sarimax.py
16.78% <0.00%> (-78.11%)
:arrow_down:

etna/analysis/outliers/density_outliers.py
22.72% <0.00%> (-75.00%)
:arrow_down:

etna/analysis/outliers/median_outliers.py
25.00% <0.00%> (-70.84%)
:arrow_down:

etna/datasets/datasets_generation.py
26.47% <0.00%> (-70.59%)
:arrow_down:

etna/transforms/special_days.py
28.75% <0.00%> (-70.00%)
:arrow_down:

etna/transforms/change_points_trend.py
31.25% <0.00%> (-68.75%)
:arrow_down:

etna/transforms/imputation.py
35.71% <0.00%> (-64.29%)
:arrow_down:

etna/analysis/outliers/hist_outliers.py
10.82% <0.00%> (-60.51%)
:arrow_down:

etna/clustering/distances/distance_matrix.py
34.48% <0.00%> (-60.35%)
:arrow_down:

... and 38 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c37a03...31414c6. Read the comment docs.

repo: tinkoff-ai/etna-ts


text:
Details about the file structure

It is unclear what the directory is suppose to contain in the README document.
Suppose you have some instance segmentation results of Cityscapes dataset, as the following format:
maskrcnn_val

frankfurt_000001_064130_leftImg8bit_pred.txt
frankfurt_000001_064305_leftImg8bit_0_person.png
frankfurt_000001_064305_leftImg8bit_10_motorcycle.png
...

what is the .txt file suppose to contain?

Same with the standard format required by the cityscape script , e.g.
frankfurt_000000_000294_leftImg8bit_0_person.png 24 0.9990299940109253
frankfurt_000000_000294_leftImg8bit_1_person.png 24 0.9810258746147156
frankfurt_000000_000294_leftImg8bit_2_person.png 24 0.3322693705558777
....

If you are using mmdetection, the output format meets the requirements automatically

repo: tinyalpha/BPR


text:
Embedding/argmax are O(n^2)

This is making GPT-2 slow

This comes down to cumsum being O(n^2), a fix could be having the special case of cumsum used in arange for the indexes of just +1 each time be fast.

I'll dive into this, I want to find the most generic way to fix this. I think there will be something clever.

authors: geohot wozeparrot

repo: tinygrad/tinygrad


text:
perf: cached_property on early reject is slow

PROFILE=0 python -O test/external/external_benchmark_schedule.py best out of 5
Master
***** model tensor in     17.41 ms
***** model schedule in    7.25 ms
***** model opts(53) in   29.08 ms
***** model lower in       6.35 ms
***** model rewrite in   477.46 ms
***** model linearize in 127.28 ms
14964
all 668.97 ms

Branch
***** model tensor in     17.48 ms
***** model schedule in    7.28 ms
***** model opts(53) in   28.40 ms
***** model lower in       6.30 ms
***** model rewrite in   454.29 ms
***** model linearize in 121.87 ms
14964
all 639.66 ms

Linter failure must be unrelated?

Hmm, interesting. Why is cached_property so terrible?

Oh this should just be refactored to precompute it

The best part is no part. #6327

authors: geohot roelofvandijk