text:
Flaky cross-polluting test

From https://github.com/mitmproxy/mitmproxy/runs/957862340:
=================================== FAILURES ===================================
__________ TestTermLog.test_output[None-expected_out0-expected_err0] ___________

self = <test.mitmproxy.addons.test_termlog.TestTermLog object at 0x7fc2cb7bd100>
outfile = None, expected_out = ['one', 'three'], expected_err = ['four']
capfd = <_pytest.capture.CaptureFixture object at 0x7fc2d9e7f9d0>

    @pytest.mark.usefixtures('capfd')
    @pytest.mark.parametrize('outfile, expected_out, expected_err', [
        (None, ['one', 'three'], ['four']),
        (sys.stdout, ['one', 'three', 'four'], []),
        (sys.stderr, [], ['one', 'three', 'four']),
    ])
    def test_output(self, outfile, expected_out, expected_err, capfd):
        t = termlog.TermLog(outfile=outfile)
        with taddons.context(t) as tctx:
            tctx.options.termlog_verbosity = "info"
            tctx.configure(t)
            t.log(log.LogEntry("one", "info"))
            t.log(log.LogEntry("two", "debug"))
            t.log(log.LogEntry("three", "warn"))
            t.log(log.LogEntry("four", "error"))
        out, err = capfd.readouterr()
        assert out.strip().splitlines() == expected_out
>       assert err.strip().splitlines() == expected_err
E       assert ['Traceback (most recent call last):',\n '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/server.py", '\n 'line 121, in handle',\n '    root_layer()',\n '  File '\n '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/modes/http_proxy.py", '\n 'line 9, in __call__',\n '    layer()',\n '  File '\n '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/protocol/tls.py", '\n 'line 285, in __call__',\n '    layer()',\n '  File '\n '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/protocol/http1.py", '\n 'line 103, in __call__',\n '    layer()',\n '  File '\n '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/protocol/http.py", '\n 'line 203, in __call__',\n '    if not self._process_flow(flow):',\n '  File '\n '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/protocol/http.py", '\n 'line 451, in _process_flow',\n '    if self.check_close_connection(f):',\n '  File '\n '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/protocol/http1.py", '\n 'line 88, in check_close_connection',\n '    flow.response.http_version,',\n "AttributeError: 'NoneType' object has no attribute 'http_version'",\n '',\n 'mitmproxy has crashed!',\n 'Please lodge a bug report at: https://github.com/mitmproxy/mitmproxy',\n 'four'] == ['four']
E         At index 0 diff: 'Traceback (most recent call last):' != 'four'
E         Left contains 19 more items, first extra item: '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/server.py", line 121, in handle'
E         Full diff:
E           [
E         +  'Traceback (most recent call last):',
E         +  '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/server.py", '
E         +  'line 121, in handle',
E         +  '    root_layer()',
E         +  '  File '
E         +  '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/modes/http_proxy.py", '
E         +  'line 9, in __call__',
E         +  '    layer()',
E         +  '  File '
E         +  '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/protocol/tls.py", '
E         +  'line 285, in __call__',
E         +  '    layer()',
E         +  '  File '
E         +  '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/protocol/http1.py", '
E         +  'line 103, in __call__',
E         +  '    layer()',
E         +  '  File '
E         +  '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/protocol/http.py", '
E         +  'line 203, in __call__',
E         +  '    if not self._process_flow(flow):',
E         +  '  File '
E         +  '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/protocol/http.py", '
E         +  'line 451, in _process_flow',
E         +  '    if self.check_close_connection(f):',
E         +  '  File '
E         +  '"/home/runner/work/mitmproxy/mitmproxy/mitmproxy/proxy/protocol/http1.py", '
E         +  'line 88, in check_close_connection',
E         +  '    flow.response.http_version,',
E         +  "AttributeError: 'NoneType' object has no attribute 'http_version'",
E         +  '',
E         +  'mitmproxy has crashed!',
E         +  'Please lodge a bug report at: https://github.com/mitmproxy/mitmproxy',
E            'four',
E           ]

test/mitmproxy/addons/test_termlog.py:27: AssertionError

This one is really ugly:

It's not caused by the TermLog test
I'm unable to repro it locally
The stacktrace doesn't make much sense. Some concurrent code unsetting flow.response?

🤷

This one could be more helpful:
__________ TestTermLog.test_output[None-expected_out0-expected_err0] ___________

self = <test.mitmproxy.addons.test_termlog.TestTermLog object at 0x7fb5f7e93588>
outfile = None, expected_out = ['one', 'three'], expected_err = ['four']
capfd = <_pytest.capture.CaptureFixture object at 0x7fb5f7e93390>

    @pytest.mark.usefixtures('capfd')
    @pytest.mark.parametrize('outfile, expected_out, expected_err', [
        (None, ['one', 'three'], ['four']),
        (sys.stdout, ['one', 'three', 'four'], []),
        (sys.stderr, [], ['one', 'three', 'four']),
    ])
    def test_output(self, outfile, expected_out, expected_err, capfd):
        t = termlog.TermLog(outfile=outfile)
        with taddons.context(t) as tctx:
            tctx.options.termlog_verbosity = "info"
            tctx.configure(t)
            t.log(log.LogEntry("one", "info"))
            t.log(log.LogEntry("two", "debug"))
            t.log(log.LogEntry("three", "warn"))
            t.log(log.LogEntry("four", "error"))
        out, err = capfd.readouterr()
        assert out.strip().splitlines() == expected_out
>       assert err.strip().splitlines() == expected_err
E       assert ['Exception in thread ProxyThread (*:40649):',\n 'Traceback (most recent call last):',\n '  File "/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/threading.py", '\n 'line 916, in _bootstrap_inner',\n '    self.run()',\n '  File "/home/runner/work/mitmproxy/mitmproxy/test/mitmproxy/tservers.py", '\n 'line 116, in run',\n '    self.tmaster.run()',\n '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/master.py", line '\n '108, in run',\n '    self.run_loop(loop.run_forever)',\n '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/master.py", line 92, '\n 'in run_loop',\n '    loop = asyncio.get_event_loop()',\n '  File '\n '"/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/asyncio/events.py", '\n 'line 694, in get_event_loop',\n '    return get_event_loop_policy().get_event_loop()',\n '  File '\n '"/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/asyncio/events.py", '\n 'line 602, in get_event_loop',\n '    % threading.current_thread().name)',\n "RuntimeError: There is no current event loop in thread 'ProxyThread "\n "(*:40649)'.",\n '',\n 'Exception in thread ProxyThread (*:38931):',\n 'Traceback (most recent call last):',\n '  File "/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/threading.py", '\n 'line 916, in _bootstrap_inner',\n '    self.run()',\n '  File "/home/runner/work/mitmproxy/mitmproxy/test/mitmproxy/tservers.py", '\n 'line 116, in run',\n '    self.tmaster.run()',\n '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/master.py", line '\n '108, in run',\n '    self.run_loop(loop.run_forever)',\n '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/master.py", line 92, '\n 'in run_loop',\n '    loop = asyncio.get_event_loop()',\n '  File '\n '"/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/asyncio/events.py", '\n 'line 694, in get_event_loop',\n '    return get_event_loop_policy().get_event_loop()',\n '  File '\n '"/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/asyncio/events.py", '\n 'line 602, in get_event_loop',\n '    % threading.current_thread().name)',\n "RuntimeError: There is no current event loop in thread 'ProxyThread "\n "(*:38931)'.",\n '',\n 'four'] == ['four']
E         At index 0 diff: 'Exception in thread ProxyThread (*:40649):' != 'four'
E         Left contains 32 more items, first extra item: 'Traceback (most recent call last):'
E         Full diff:
E           [
E         +  'Exception in thread ProxyThread (*:40649):',
E         +  'Traceback (most recent call last):',
E         +  '  File "/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/threading.py", '
E         +  'line 916, in _bootstrap_inner',
E         +  '    self.run()',
E         +  '  File "/home/runner/work/mitmproxy/mitmproxy/test/mitmproxy/tservers.py", '
E         +  'line 116, in run',
E         +  '    self.tmaster.run()',
E         +  '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/master.py", line '
E         +  '108, in run',
E         +  '    self.run_loop(loop.run_forever)',
E         +  '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/master.py", line 92, '
E         +  'in run_loop',
E         +  '    loop = asyncio.get_event_loop()',
E         +  '  File '
E         +  '"/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/asyncio/events.py", '
E         +  'line 694, in get_event_loop',
E         +  '    return get_event_loop_policy().get_event_loop()',
E         +  '  File '
E         +  '"/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/asyncio/events.py", '
E         +  'line 602, in get_event_loop',
E         +  '    % threading.current_thread().name)',
E         +  "RuntimeError: There is no current event loop in thread 'ProxyThread "
E         +  "(*:40649)'.",
E         +  '',
E         +  '',
E         +  'Exception in thread ProxyThread (*:38931):',
E         +  'Traceback (most recent call last):',
E         +  '  File "/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/threading.py", '
E         +  'line 916, in _bootstrap_inner',
E         +  '    self.run()',
E         +  '  File "/home/runner/work/mitmproxy/mitmproxy/test/mitmproxy/tservers.py", '
E         +  'line 116, in run',
E         +  '    self.tmaster.run()',
E         +  '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/master.py", line '
E         +  '108, in run',
E         +  '    self.run_loop(loop.run_forever)',
E         +  '  File "/home/runner/work/mitmproxy/mitmproxy/mitmproxy/master.py", line 92, '
E         +  'in run_loop',
E         +  '    loop = asyncio.get_event_loop()',
E         +  '  File '
E         +  '"/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/asyncio/events.py", '
E         +  'line 694, in get_event_loop',
E         +  '    return get_event_loop_policy().get_event_loop()',
E         +  '  File '
E         +  '"/opt/hostedtoolcache/Python/3.6.11/x64/lib/python3.6/asyncio/events.py", '
E         +  'line 602, in get_event_loop',
E         +  '    % threading.current_thread().name)',
E         +  "RuntimeError: There is no current event loop in thread 'ProxyThread "
E         +  "(*:38931)'.",
E         +  '',
E            'four',
E           ]

test/mitmproxy/addons/test_termlog.py:27: AssertionError

repo: mitmproxy/mitmproxy


text:
socket / connection leak

Problem Description
The mitmproxy keeps the server connection open/established while it is supposed to close it after the client (and especially the server) is manually disconnected.
The problem is particularly noticeable in the case of large file downloads.
Steps to reproduce the behavior:

start the mitmproxy : mitmdump --set confdir=/root/MITMPYTHON/configDir/ --listen-port 85
download a video with curl : curl --proxy localhost:85 https://www.centrepompidou.fr/_elastic_filemount/media/video/77/8c/778cedf5e246fb0e0a9f0a11f2f8b114/normal.mp4  --insecure -vvv -O /dev/null
monitor the status of the server connection opened by the mitmproxy with the netstat command : netstat -anp | grep "server.sockname" ( server.sockname is the local (ip, port) tuple for the server connection)
interrupt/close the client
again: monitor the status of the server connection netstat -anp | grep "server.sockname"

Here is my code (test.py) :
from mitmproxy import http

class testVideo:

  def request(self, flow: http.HTTPFlow):
    print('---------------------------- HTTPEvents : request', flow.request.url)

  def response(self, flow):
    print('---------------------------- HTTPEvents : response', flow.request.url)

  def client_connected(self, client):
    print("------ client_connected - client:(%s, %s)" % (client.peername, client.sockname))

  def client_disconnected(self, client):
    print("------ client_disconnected - client:(%s, %s)" % (client.peername, client.sockname))

  def server_connect(self, data):
    print("------ server_connect - client:(%s, %s) server:(%s, %s)" % (data.client.peername, data.client.sockname, data.server.sockname, data.server.peername))

  def server_connected(self, data):
    print("------ server_connected - client:(%s, %s) server:(%s, %s)" % (data.client.peername, data.client.sockname, data.server.sockname, data.server.peername))

  def server_disconnected(self, data):
    print("------ server_disconnected - client:(%s, %s) server:(%s, %s)" % (data.client.peername, data.client.sockname, data.server.sockname, data.server.peername))

  def http_connect(self, flow):
    print('---------------------------- HTTPEvents : http_connect', flow.request.url)

addons = [
  testVideo()
]

Config.yaml :
scripts:
  - "./test.py"
connection_strategy: lazy
stream_large_bodies: 30m
http2: False
websocket: False
upstream_cert: False

Current behaviour :
The server connection remains established despite the disconnection of the client and the server.
A step by step example:
step1: start the mitmproxy and start downloading the video with curl (video size: 3240M) :
[root@... ~]# curl --proxy localhost:85 https://www.centrepompidou.fr/_elastic_filemount/media/video/77/8c/778cedf5e246fb0e0a9f0a11f2f8b114/normal.mp4  --insecure -vvv -O /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to proxy localhost port 85 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 85 (#0)
* Establish HTTP proxy tunnel to www.centrepompidou.fr:443
> CONNECT www.centrepompidou.fr:443 HTTP/1.1
> Host: www.centrepompidou.fr:443
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=www.centrepompidou.fr
*       start date: Feb 13 08:54:24 2022 GMT
*       expire date: Feb 15 08:54:24 2023 GMT
*       common name: www.centrepompidou.fr
*       issuer: O=mitmproxy,CN=mitmproxy
> GET /_elastic_filemount/media/video/77/8c/778cedf5e246fb0e0a9f0a11f2f8b114/normal.mp4 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.centrepompidou.fr
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx
< Date: Tue, 15 Feb 2022 07:54:24 GMT
< Content-Type: video/mp4
< Content-Length: 3397758564
< Connection: keep-alive
< Last-Modified: Sun, 18 Jul 2021 03:33:34 GMT
< ETag: "ca85ae64-5c75d7c4e56a0"
< Accept-Ranges: bytes
< Cache-Control: max-age=604800
< Expires: Tue, 22 Feb 2022 07:54:24 GMT
<
{ [data not shown]
 42 3240M   42 1379M    0     0  63.6M      0  0:00:50  0:00:21  0:00:29 68.5M

mitmproxy log :
[root@... MITMPYTHON]# mitmdump --set confdir=/root/MITMPYTHON/configDir/ --listen-port 85
Loading script ./redirect_test.py
Proxy server listening at http://*:85
------ client_connected - client:(('172.26.14.113', 52248), ('127.0.0.1', 85))
172.26.14.113:52248: client connect
---------------------------- HTTPEvents : http_connect www.centrepompidou.fr:443
---------------------------- HTTPEvents : request https://www.centrepompidou.fr/_elastic_filemount/media/video/77/8c/778cedf5e246fb0e0a9f0a11f2f8b114/normal.mp4
------ server_connect - client:(('172.26.14.113', 52248), ('127.0.0.1', 85)) server:(None, None)
172.26.14.113:52248: server connect www.centrepompidou.fr:443 (152.228.186.147:443)
------ server_connected - client:(('172.26.14.113', 52248), ('127.0.0.1', 85)) server:(('172.26.14.113', 59490), ('152.228.186.147', 443))
172.26.14.113:52248: Streaming response from www.centrepompidou.fr.

result of netstat :
[root@dlw-squid01 ~]# netstat -anp | grep 59490
tcp   3211880      0 172.26.14.113:59490     152.228.186.147:443     ESTABLISHED 15414/python

step2: interrupt/close the client download (ctrl+c)
mitmproxy log :
.............
172.26.14.113:52248: GET https://www.centrepompidou.fr/_elastic_filemount/media/video/77/8c/778cedf5e246fb0e0a9f0a1…
                  << 200 OK (content missing)
 << Client disconnected.
------ server_disconnected - client:(('172.26.14.113', 52248), ('127.0.0.1', 85)) server:(('172.26.14.113', 59490), ('152.228.186.147', 443))
------ client_disconnected - client:(('172.26.14.113', 52248), ('127.0.0.1', 85))
172.26.14.113:52248: server disconnect www.centrepompidou.fr:443 (152.228.186.147:443)
172.26.14.113:52248: client disconnect

=> The mitmproxy shows that the client and the server connections have been closed (server disconnect and client disconnect)
result of netstat :
[root@dlw-squid01 ~]# netstat -anp | grep 59490
tcp   3211880      0 172.26.14.113:59490     152.228.186.147:443     ESTABLISHED 15414/python

=> The mitmproxy keeps the server connexion established
NB:
The server connection is correctly closed when I let the curl command finish downloading the whole file.
The server connection remains infinitely established when I manually interrupt the download (the connection is established as long as mitmproxy is running).
System Information
Mitmproxy: 7.0.4
Python:    3.9.5
OpenSSL:   OpenSSL 1.1.1l  24 Aug 2021
Platform:  Linux-3.10.0-1127.el7.x86_64-x86_64-with-glibc2.17

Thanks for the detailed report, but I can't reproduce this locally. Could you please run mitmproxy with -vvv --set proxy_debug and post the final lines starting from the first ConnectionClosed?

Here the log from the first occurrence of ConnectionClosed
To reproduce the problem, I suggest to let the video advance in the download (exceed 512M) before stopping it
172.26.14.113:47150:   >> ConnectionClosed(connection=Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.centrepompidou.fr', 'alpn_offers': [], 'timestamp_tls_setup': 1644920222.2452333, 'alpn': b'', 'certificate_list': [], 'cipher': 'ECDHE-RSA-AES128-GCM-SHA256', 'tls_version': 'TLSv1.2'}))
172.26.14.113:47150:     >> ConnectionClosed(connection=Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.centrep…
172.26.14.113:47150:       >> ConnectionClosed(connection=Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.centrep…
172.26.14.113:47150:       << Receive(RequestEndOfMessage(stream_id=1))
172.26.14.113:47150:       >> RequestEndOfMessage(stream_id=1)
172.26.14.113:47150:         >> ConnectionClosed(connection=Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.centrepompidou.fr', 'alpn_offers': [], 'timestamp_tls_setup': 1644920222.2452333, 'alpn': b'', 'certificate_list': [], 'cipher': 'ECDHE-RSA-AES128-GCM-SHA256', 'tls_version': 'TLSv1.2'}))
172.26.14.113:47150:           >> ConnectionClosed(connection=Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.centrep…
172.26.14.113:47150:             >> ConnectionClosed(connection=Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.centrep…
172.26.14.113:47150:               >> ConnectionClosed(connection=Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.centrep…
172.26.14.113:47150:               << CloseConnection({'connection': Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.centrepompidou.fr', 'alpn_offers': [], 'timestamp_tls_setup': 1644920222.2452333, 'alpn': b'', 'certificate_list': [], 'cipher': 'ECDHE-RSA-AES128-GCM-SHA256', 'tls_version': 'TLSv1.2'}), 'half_close': False})
172.26.14.113:47150:             << CloseConnection({'connection': Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.cent…
172.26.14.113:47150:           << CloseConnection({'connection': Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.cent…
172.26.14.113:47150:         << CloseConnection({'connection': Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.cent…
172.26.14.113:47150:       << Send(ResponseProtocolError(stream_id=1, message='EOF', code=502))
172.26.14.113:47150:       >> ResponseProtocolError(stream_id=1, message='EOF', code=502)
172.26.14.113:47150:       << CloseConnection({'connection': Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.centrepompidou.fr', 'alpn_offers': [], 'timestamp_tls_setup': 1644920222.2452333, 'alpn': b'', 'certificate_list': [], 'cipher': 'ECDHE-RSA-AES128-GCM-SHA256', 'tls_version': 'TLSv1.2'}), 'half_close': False})
172.26.14.113:47150:     << CloseConnection({'connection': Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.cent…
172.26.14.113:47150:   << CloseConnection({'connection': Client({'id': '…160fa9', 'peername': ('172.26.14.113', 47150), 'sockname': ('172.26.14.113', 85), 'timestamp_start': 1644920222.0173118, 'state': <ConnectionState.CAN_WRITE: 2>, 'reply': None, 'tls': True, 'sni': 'www.cent…
172.26.14.113:47150:               << Receive(RequestProtocolError(stream_id=1, message='Client disconnected.', code=499))
172.26.14.113:47150:               >> RequestProtocolError(stream_id=1, message='Client disconnected.', code=499)
172.26.14.113:47150:               << Send(RequestProtocolError(stream_id=1, message='Client disconnected.', code=499))
172.26.14.113:47150:               >> RequestProtocolError(stream_id=1, message='Client disconnected.', code=499)
172.26.14.113:47150:                 >> RequestProtocolError(stream_id=1, message='Client disconnected.', code=499)
172.26.14.113:47150:                   >> RequestProtocolError(stream_id=1, message='Client disconnected.', code=499)
172.26.14.113:47150:                   << CloseConnection({'connection': Server({'id': '…3088c6', 'address': ('www.centrepompidou.fr', 443), 'state': <ConnectionState.OPEN: 3>, 'sni': 'www.centrepompidou.fr', 'tls': True, 'timestamp_start': 1644920222.2886999, 'timestamp_tcp_setup': 1644920222.2969909, 'peername': ('152.228.186.147', 443), 'sockname': ('172.26.14.113', 59510), 'alpn_offers': [], 'timestamp_tls_setup': 1644920222.3338516, 'alpn': b'', 'certificate_list': [<Cert(cn='*.centrepompidou.fr', altnames=['*.centrepompidou.fr', 'centrepom…
172.26.14.113:47150:                 << CloseConnection({'connection': Server({'id': '…3088c6', 'address': ('www.centrepompidou.fr', 443), 'state': <ConnectionState.OPEN: 3>, 'sni': 'www.centrepompidou.fr', 'tls': True, 'timestamp_start': 1644920222.2886999, 'timestamp_tcp_setup': 1644920222.…
172.26.14.113:47150:               << CloseConnection({'connection': Server({'id': '…3088c6', 'address': ('www.centrepompidou.fr', 443), 'state': <ConnectionState.OPEN: 3>, 'sni': 'www.centrepompidou.fr', 'tls': True, 'timestamp_start': 1644920222.2886999, 'timestamp_tcp_setup': 1644920222.…
172.26.14.113:47150:             << CloseConnection({'connection': Server({'id': '…3088c6', 'address': ('www.centrepompidou.fr', 443), 'state': <ConnectionState.OPEN: 3>, 'sni': 'www.centrepompidou.fr', 'tls': True, 'timestamp_start': 1644920222.2886999, 'timestamp_tcp_setup': 1644920222.…
172.26.14.113:47150:           << CloseConnection({'connection': Server({'id': '…3088c6', 'address': ('www.centrepompidou.fr', 443), 'state': <ConnectionState.OPEN: 3>, 'sni': 'www.centrepompidou.fr', 'tls': True, 'timestamp_start': 1644920222.2886999, 'timestamp_tcp_setup': 1644920222.…
172.26.14.113:47150:         << CloseConnection({'connection': Server({'id': '…3088c6', 'address': ('www.centrepompidou.fr', 443), 'state': <ConnectionState.OPEN: 3>, 'sni': 'www.centrepompidou.fr', 'tls': True, 'timestamp_start': 1644920222.2886999, 'timestamp_tcp_setup': 1644920222.…
172.26.14.113:47150:       << CloseConnection({'connection': Server({'id': '…3088c6', 'address': ('www.centrepompidou.fr', 443), 'state': <ConnectionState.OPEN: 3>, 'sni': 'www.centrepompidou.fr', 'tls': True, 'timestamp_start': 1644920222.2886999, 'timestamp_tcp_setup': 1644920222.…
172.26.14.113:47150:     << CloseConnection({'connection': Server({'id': '…3088c6', 'address': ('www.centrepompidou.fr', 443), 'state': <ConnectionState.OPEN: 3>, 'sni': 'www.centrepompidou.fr', 'tls': True, 'timestamp_start': 1644920222.2886999, 'timestamp_tcp_setup': 1644920222.…
172.26.14.113:47150:   << CloseConnection({'connection': Server({'id': '…3088c6', 'address': ('www.centrepompidou.fr', 443), 'state': <ConnectionState.OPEN: 3>, 'sni': 'www.centrepompidou.fr', 'tls': True, 'timestamp_start': 1644920222.2886999, 'timestamp_tcp_setup': 1644920222.…
172.26.14.113:47150:               << HttpErrorHook(flow=<HTTPFlow
                request = Request(GET www.centrepompidou.fr:443/_elastic_filemount/media/video/77/8c/778cedf5e246fb0e0a9f0a11f2f8b114/normal.mp4)
                response = Response(200, no content)
                error = Client disconnected.
                client_conn = Client(172.26.14.113:47150, state=closed, tls)
                server_conn = Server(www.centrepompidou.fr:443, state=closed, tls, src_port=59510)>)
172.26.14.113:47150:             << HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150:           << HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150:         << HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150:       << HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150:     << HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150:   << HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150: GET https://www.centrepompidou.fr/_elastic_filemount/media/video/77/8c/778cedf5e246fb0e0a9f0a11f2f8b114/normal.mp4
    User-Agent: curl/7.29.0
    Host: www.centrepompidou.fr
    Accept: */*
 << 200 OK (content missing)
    Server: nginx
    Date: Tue, 15 Feb 2022 10:17:02 GMT
    Content-Type: video/mp4
    Content-Length: 3397758564
    Connection: keep-alive
    Last-Modified: Sun, 18 Jul 2021 03:33:34 GMT
    ETag: "ca85ae64-5c75d7c4e56a0"
    Accept-Ranges: bytes
    Cache-Control: max-age=604800
    Expires: Tue, 22 Feb 2022 10:17:02 GMT
 << Client disconnected.
------ server_disconnected - client:(('172.26.14.113', 47150), ('172.26.14.113', 85)) server:(('172.26.14.113', 59510), ('152.228.186.147', 443))
------ client_disconnected - client:(('172.26.14.113', 47150), ('172.26.14.113', 85))
172.26.14.113:47150: server disconnect www.centrepompidou.fr:443 (152.228.186.147:443)
172.26.14.113:47150: client disconnect
172.26.14.113:47150:   >> Reply(HttpErrorHook(flow=<HTTPFlow
    request = Request(GET www.centrepompidou.fr:443/_elastic_filemount/media/video/77/8c/778cedf5e246fb0e0a9f0a11f2f8b114/normal.mp4)
    response = Response(200, no content)
    error = Client disconnected.
    client_conn = Client(172.26.14.113:47150, state=closed, tls)
    server_conn = Server(www.centrepompidou.fr:443, state=closed, tls, src_port=59510)>), None)
172.26.14.113:47150:     >> Reply(HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150:       >> Reply(HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150:         >> Reply(HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150:           >> Reply(HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150:             >> Reply(HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150:               >> Reply(HttpErrorHook(flow=<HTTPFlow
172.26.14.113:47150: closing transports...
172.26.14.113:47150: transports closed!

Thanks. I think this might be fixed with the latest development snapshots, could you check that? At least I'm seeing correct behavior here. You can find precompiled binaries at https://mitmproxy.org/downloads/#branches/main/ if that helps.

Thanks you...
The issue is fixed with Mitmproxy: 8.0.0.dev
The server connection is immediately closed when I interrupt the client, no more established server connection after the client and server disconnect.
Would you have an idea about the date of the new release ?

Thanks a lot

source: gharchive/issue

authors: Devhercule mhils

license: mit

license_type: permissive

license_source: bigquery


text:
3 videos in MAS.531 were not uploaded to YouTube

Steps to Reproduce

Uploaded videos to the videos_final folder for MAS.531 F09 (https://drive.google.com/drive/folders/1HwVcWxSjdjrW1We6kxwz4Rg-OXPwpwfy)
Hit sync in OCW Studio for https://ocw-studio.odl.mit.edu/sites/mas-531-computational-camera-and-photography-fall-2009/
Sync says it is complete but all videos are not processed.

Expected Behavior
all videos uploaded get uploaded to YouTube and 3Play
Actual Behavior
video lectures 5-3, 6-3, 8-3 did not upload to YouTube and get submitted to 3Play
Related Issues
Possibly related issues in Sentry: https://sentry.io/organizations/mit-office-of-digital-learning/issues/?project=5739976&query=is%3Aunresolved+youtube&statsPeriod=14d

@pt2302 what's the status of this? Have you started on it?

Resources for these videos were never created in OCW Studio. Looking into what could be the cause.

There was an uploading issue to S3 that prevented creating the resources in OCW Studio. After deleting the relevant DriveFile objects in Django admin and re-syncing with Google Drive, all of the resources have been created. The videos have been uploaded to YouTube and submitted to 3Play for captioning.

repo: mitodl/ocw-studio

license_source: github-api


text:
Growing worker filesystem size for edxapp

Growing worker filesystem size on all edx workers to account for course export processes.

Description
Sets the default worker disk size to 50GB rather than 25GB
Motivation and Context
Stop paging me.
How Has This Been Tested?
Will apply the stack to the unhappy environments
Types of changes

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Enhancement (improves on existing behavior)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[X] My code follows the code style of this project. (Did you install and run the pre-commit hooks?)
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

:+1:

source: gharchive/pull-request

authors: Ardiea blarghmatey

repo: mitodl/ol-infrastructure


text:
hc cache delete -s all seems broken

Ran on my local machine, doesn't prompt users on what to delete nor does it delete anything.

Resolved by #591

repo: mitre/hipcheck


text:
Technical Support Domain Model

Technical support request and technical support answer

Lines Added: 90
Lines Deleted: 0

repo: miw-upm/betca-tpv

license: MIT


text:
'Object reference ...' error on creating/editing post

Hi,
I just cloned the source code and ran it. I did 'Init steps' and went to portal panel.
but when I want to create a "Post" in '/portal/post/create', there is a "Object reference not set to an instance of an object" error in "\src\Mix.Cms.Lib\ViewModels\MixPosts\UpdateViewModel.cs" file line 1076:

and if I fix it by checking null value, another error appear on line 1036:

even if I select  a "Parent page"
whats the issue, and how can I fix it??

@moslem-hady could you help let me know which branch you are using right now? is it develop or master.

@moslem-hady could you help try the following steps then let us know the result? Also, please always use the latest code from develop branch as currently we have migrated to .net core v3 + MySQL and there are a lot of changes and migration.

Check if pages & modules has been installed successful during installation step
Turn off then on the system cache via "Portal App Setting" > General > Site Data Cache

I'm using master branch.
pages and modules is installed.
I turned off "Site data cache", but same errors still exist.
I'll to try "develop" branch and let you know what happend.
thanks

Thanks @moslem-hady please try again with develop branch then feel free let us know if you are still facing any issues.

I tried "develop" branch. same error. but there may be an issue with initializing database. I dont know.
I opened another issue about init steps.

Update/Create Page is throwing an exception on Save as well. Just open any page (Home for example) and click on Save button.

Hi @moslem-hady , @mtxxx , our team have fixed many issues since then. Please try again and feel free let us know if you facing any other issues.

authors: Smilefounder moslem-hady mtxxx


text:
In-app messages cause: IllegalArgumentException: Software rendering doesn't support hardware bitmaps

Same as #711, which was apparently fixed in v5.8.7, but this is still occurring in v5.9.1:

Fatal Exception: java.lang.IllegalArgumentException: Software rendering doesn't support hardware bitmaps
at android.graphics.BaseCanvas.onHwBitmapInSwMode(BaseCanvas.java:673)
at android.graphics.BaseCanvas.throwIfHwBitmapInSwMode(BaseCanvas.java:680)
at android.graphics.BaseCanvas.throwIfCannotDraw(BaseCanvas.java:84)
at android.graphics.BaseCanvas.drawBitmap(BaseCanvas.java:142)
at android.graphics.Canvas.drawBitmap(Canvas.java:1648)
at android.graphics.drawable.BitmapDrawable.draw(BitmapDrawable.java:548)
at android.widget.ImageView.onDraw(ImageView.java:1453)
at android.view.View.draw(View.java:23901)
at android.view.View.draw(View.java:23773)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at androidx.constraintlayout.widget.ConstraintLayout.dispatchDraw(ConstraintLayout.java:1975)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at android.view.View.draw(View.java:23904)
at android.view.View.draw(View.java:23773)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at androidx.recyclerview.widget.RecyclerView.drawChild(RecyclerView.java:5030)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at android.view.View.draw(View.java:23904)
at androidx.recyclerview.widget.RecyclerView.draw(RecyclerView.java:4429)
at android.view.View.draw(View.java:23773)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at androidx.constraintlayout.widget.ConstraintLayout.dispatchDraw(ConstraintLayout.java:1975)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at androidx.fragment.app.FragmentContainerView.drawChild(FragmentContainerView.java:261)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at androidx.fragment.app.FragmentContainerView.dispatchDraw(FragmentContainerView.java:249)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at androidx.constraintlayout.widget.ConstraintLayout.dispatchDraw(ConstraintLayout.java:1975)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at androidx.coordinatorlayout.widget.CoordinatorLayout.drawChild(CoordinatorLayout.java:1277)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at android.view.View.draw(View.java:23771)
at android.view.ViewGroup.drawChild(ViewGroup.java:5336)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:5093)
at android.view.View.draw(View.java:23904)
at com.android.internal.policy.DecorView.draw(DecorView.java:1282)
at com.mixpanel.android.util.ActivityImageUtils.getScaledScreenshot(ActivityImageUtils.java:31)
at com.mixpanel.android.util.ActivityImageUtils.getHighlightColorFromBackground(ActivityImageUtils.java:52)
at com.mixpanel.android.mpmetrics.MixpanelAPI$PeopleImpl$2.run(MixpanelAPI.java:2541)
at android.app.Activity.runOnUiThread(Activity.java:7154)
at com.mixpanel.android.mpmetrics.MixpanelAPI$PeopleImpl.showGivenOrAvailableNotification(MixpanelAPI.java:2514)
at com.mixpanel.android.mpmetrics.MixpanelAPI$PeopleImpl.showGivenNotification(MixpanelAPI.java:2333)
at com.mixpanel.android.mpmetrics.MixpanelAPI.track(MixpanelAPI.java:2928)
at com.mixpanel.android.mpmetrics.MixpanelAPI.track(MixpanelAPI.java:900)
at com.mixpanel.android.mpmetrics.MixpanelAPI.trackMap(MixpanelAPI.java:842)

hi @ gosr , the error is from the Messaging. Are you using it right now?  We are deprecating Messaging and Mobile A/B testing features(more details). It will be removed on Jan 1, 2022.

If you are not using Messaging and Mobile A/B testing , we'd highly suggest you opt into our beta version implementation "com.mixpanel.android:mixpanel-android:6.0.0-beta+" It has the current release minus A/B Testing and Messaging and it has our full support.
If you are using Messaging and Mobile A/B testing  and do need this fix, please contact our support if you need to continue the conversation. Sorry for the inconvenience.

authors: gosr zihejia

repo: mixpanel/mixpanel-android


text:
Local storage persistence across http to https broken

If a user starts on an unsecure site, we put a record in local storage to this URL. As an example, the record will be written to http://www.gooogle.com and all Mixpanel information will be stored under this name.
Later on the site, if the user moves from unsecure to secure browsing, the library goes and looks for the information in local storage under https://www.google.com. Since the prior record was saved with http and not https, the library sees no Mixpanel information and instead creates an entirely new record.
Is it possible to save the Mixpanel information into one local storage record, regardless of the http versus https status of the webpage? Currently reporting will be broken if you use this model and switch protocols mid session.

This limitation is imposed by the browser as part of same-origin policy. There's not really anything Mixpanel can do here.

authors: nderscore ryanseams

repo: mixpanel/mixpanel-js


text:
added debug logging for the implementation error message

This will give greater clarity into what the implementation error is when debug mode is turned on.

This patch has been published in v2.14.0.

authors: jtmcmc tdumitrescu


text:
need refactoring and fix codes

I did :put_litter_in_its_place: for now.
However, this creates many garbages, so, I want to fix them.

I will release version 0.1 after this issue fixed

Related #1

add failFlag on with reporter method

[ ] fix parse flow

Now, there are some parse methods.  However, this is not useful.  I will provide middle result structure and some result converters, and fix parse methods to alias middle converters.

repo: mizunashi-mana/node-sonparser


text:
Coding style

In GitLab by @ColmTalbot on Mar 29, 2018, 23:12
It would be nice to have a coherent convention to follow, enumerated somewhere useful, maybe a repo wiki.
If we decide on the definitive ones here we can move it somewhere more permanent.
We'll generally follow PEP8 conventions.
You can set up most IDEs to check if you follow this (even emacs).
We may decide we want a few differences though. Below is my list, please comment with suggestions.
Suggested deviations from PEP8:

Increase the maximum line length from 80 -> 120 characters.

@moritz.huebner @paul-lasky

In GitLab by @moritz.huebner on Apr 9, 2018, 08:40
https://stackoverflow.com/questions/88942/why-does-pep-8-specify-a-maximum-line-length-of-79-characters

In GitLab by @moritz.huebner on Apr 9, 2018, 08:54
I would like to mention that it is important to get class and function name conventions correct right away. Changing it later in the development stage will cause a lot of code to break that people have written outside of the project.
Right now I hesitate to 'fix' any of the issues I see, because I know that it will break stuff as soon as people pull.

In GitLab by @moritz.huebner on Apr 26, 2018, 04:37
closed

In GitLab by @moritz.huebner on Apr 26, 2018, 04:37
We won't enforce character limit under 120 characters. Otherwise, we will stick to PEP8.

repo: mj-will/bilby


text:
unpkg down?

Multiple imports return 520 errors.

Confirm!

+1

+1

+1

+1

Consider switching to jsDelivr https://www.jsdelivr.com/unpkg
It's actively being developed and has been for the last 10 years

It seems that requests with specific versions are OK while unversioned requests are not - i.e.
Works - https://unpkg.com/cloudinary-video-player@1.9.0/dist/cld-video-player.min.js
Doesn't - https://unpkg.com/cloudinary-video-player/dist/cld-video-player.min.js
Is that what you guys are seeing also?

For us bulma@0.7 and bulma-extensions@6 are randomly KO

doesn't work links with parameter  @latest : https://unpkg.com/just-validate**@latest**/dist/just-validate.production.min.js
or without explicitly specifying the version : https://unpkg.com/just-validate/dist/just-validate.production.min.js
However, this is worked https://unpkg.com/just-validate@3.8.1/dist/just-validate.production.min.js

+1

+1 same here for : https://unpkg.com/dropzone@5/dist/min/dropzone.min.js

Same hre for : https://unpkg.com/@googlemaps/markerclusterer/dist/index.min.js?ver=6.0
Our production is down because of this.

Don't if directly related, but on Cloudfare status page ( https://www.cloudflarestatus.com/ ), some maintenance are planned (and in progress) on 2 datacenter.
I reproduce also the 520 error with google-chart components who load Lit related stuff.

Same here. Error code 520: https://unpkg.com/lit-html@^1.0.0?module

Also doesn't work.
https://unpkg.com/swiper@8/swiper-bundle.min.css
https://unpkg.com/swiper@8/swiper-bundle.min.js

+1

Also doensn't work the following in UK.
https://unpkg.com/react-dom@16/umd/react-dom.development.js
https://unpkg.com/react@16/umd/react.development.js

It seems that requests with specific versions are OK while unversioned requests are not - i.e. Works - https://unpkg.com/cloudinary-video-player@1.9.0/dist/cld-video-player.min.js Doesn't - https://unpkg.com/cloudinary-video-player/dist/cld-video-player.min.js
Is that what you guys are seeing also?

its not consistent
https://unpkg.com/react@latest/umd/react.production.min.js  --not working
https://unpkg.com/react/umd/react.production.min.js  --sometimes working (redirects to @18.2.0)

Vue has the same problem (but another perple see vue correctly):
https://unpkg.com/vue@3/dist/vue.global.js

It seems that requests with specific versions are OK while unversioned requests are not - i.e. Works - https://unpkg.com/cloudinary-video-player@1.9.0/dist/cld-video-player.min.js Doesn't - https://unpkg.com/cloudinary-video-player/dist/cld-video-player.min.js
Is that what you guys are seeing also?

its not consistent
https://unpkg.com/react@latest/umd/react.production.min.js --not working https://unpkg.com/react/umd/react.production.min.js --sometimes working (redirects to @18.2.0)

I'm getting the opposite result - the first one (a derivative of the version) is consistently working while the second consistently fails. Maybe the description of the result was swapped in your message for the two URLs?

It seems that requests with specific versions are OK while unversioned requests are not - i.e. Works - https://unpkg.com/cloudinary-video-player@1.9.0/dist/cld-video-player.min.js Doesn't - https://unpkg.com/cloudinary-video-player/dist/cld-video-player.min.js
Is that what you guys are seeing also?

its not consistent
https://unpkg.com/react@latest/umd/react.production.min.js --not working https://unpkg.com/react/umd/react.production.min.js --sometimes working (redirects to @18.2.0)

@Applesauce314  I'm getting the opposite result - the first one (a derivative of the version) is consistently working while the second consistently fails. Maybe the description of the result was swapped in your message for the two URLs?

+1

+1
https://unpkg.com/tone

@aleksandar-cloudinary, same here.

The CDN for everything on npm, currently serving 4B+ requests per day

Hoping that unpkg will get up quickly.

I commented in the similar issue. Happens to us just now: https://github.com/mjackson/unpkg/issues/330#issuecomment-1294990723

+1. It also fails for https://unpkg.com/typescript.

It seems that requests with specific versions are OK while unversioned requests are not - i.e. Works - https://unpkg.com/cloudinary-video-player@1.9.0/dist/cld-video-player.min.js Doesn't - https://unpkg.com/cloudinary-video-player/dist/cld-video-player.min.js
Is that what you guys are seeing also?

its not consistent
https://unpkg.com/react@latest/umd/react.production.min.js --not working https://unpkg.com/react/umd/react.production.min.js --sometimes working (redirects to @18.2.0)

@Applesauce314 I'm getting the opposite result - the first one (a derivative of the version) is consistently working while the second consistently fails. Maybe the description of the result was swapped in your message for the two URLs?
I put it correctly at the time, but apparently the behavior is inconsistent, and has swapped.

worth noting that https://status.flyio.net/ is reporting issues, but at a glance seemingly unrelated.

+1 https://unpkg.com/@stoplight/elements@7.7.3/web-components.min.js

+1 https://unpkg.com/axios/dist/axios.min.js

Consider switching to jsDelivr https://www.jsdelivr.com/unpkg
It's actively being developed and has been for the last 10 years

Thank You @jimaek we just migrate our index.html and all it's ok now...

+1 https://unpkg.com/zone.js

+1
https://unpkg.com/@popperjs/core@2
https://unpkg.com/tippy.js@6

+1
https://unpkg.com/react-dom@17/umd/react-dom.production.min.js

1

all the latest posts on fly.io's forums are people experiencing issues, likely that is the source of this issue as well.
https://community.fly.io/
https://status.flyio.net/

I can't seem to find any sort of pattern between the ones that work and the ones that don't. We're having issues with alpine on our site:
https://unpkg.com/alpinejs@3.x.x/dist/cdn.min.js
https://unpkg.com/@alpinejs/intersect@3.x.x/dist/cdn.min.js

+1

+1

I can't seem to find any sort of pattern between the ones that work and the ones that don't. We're having issues with alpine on our site:
https://unpkg.com/alpinejs@3.x.x/dist/cdn.min.js https://unpkg.com/@alpinejs/intersect@3.x.x/dist/cdn.min.js

https://cdn.jsdelivr.net/npm/alpinejs@3.10.5/dist/cdn.min.js

+1
https://unpkg.com/react-dom@17/umd/react-dom.production.min.js

https://cdn.jsdelivr.net/npm/react-dom@17/umd/react-dom.production.min.js

production is down :(

Could be related to https://status.flyio.net/ with the main site listing this as the origin server.

+1

+1 for https://unpkg.com/primevue/core/core.js and all the components and themes that come with vue

+1 for https://unpkg.com/primevue/core/core.js and all the components and themes that come with vue

https://cdn.jsdelivr.net/npm/primevue/core/core.min.js

been watching the thread to see if any updates got posted... but our dependency just started loading finally!  Hope yours do as well.

Ours just came back too. Thanks all!

+1

Ours came back too!
https://unpkg.com/react@16.14.0/umd/react.production.min.js

We've seen this consistently. This isn't the first where Cloudflare has been doing some maintenance, and unpkg gets hit hard.
Could just be coincidental, but Cloudflare maintenance just ended as well per https://www.cloudflarestatus.com/. Only two regions were impacted. We saw these errors in unrelated regions.

Still seeing errors like below
Access to script at 'https://unpkg.com/@freshworks/crayons@v4/dist/crayons/crayons.esm.js' from origin 'http://127.0.0.1:5500' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource

Now it seems to be stuck in a loop on the Cloudflare JS challenge page.

Seems to now be stuck in a loop on the Cloudflare JS challenge page - https://unpkg.com/cdn-cgi/challenge-platform.../ is called but returns 401s and it's back and forth.

Seems to now be stuck in a loop on the Cloudflare JS challenge page. The JS calls https://unpkg.com/cdn-cgi/challenge-platform/... to complete the challenge but that returns 401s and it's going back and forth.

Took me a bit to find out why are production site was done.  But same problem for us.

+1 for https://unpkg.com/leaflet@1.9.2/

+1

Looks like it's back again.

Consider switching to jsDelivr https://www.jsdelivr.com/unpkg
It's actively being developed and has been for the last 10 years

This saved my a**.
Unfortunately some of the plugins from 3rd party libraries still use unpkg in the background. They should change their CDNs too.

Not sure if it's related but getting 503 errors for me from webpagetest.org tests

HTTP/1.1 503
date: Fri, 28 Oct 2022 15:09:53 GMT
content-type: text/html; charset=UTF-8
x-frame-options: SAMEORIGIN
cross-origin-embedder-policy: require-corp
cross-origin-opener-policy: same-origin
cross-origin-resource-policy: same-origin
referrer-policy: same-origin
permissions-policy: accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()
cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
expires: Thu, 01 Jan 1970 00:00:01 GMT
vary: Accept-Encoding
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-content-type-options: nosniff
server: cloudflare
cf-ray: 76149857adbb5e12-IAD

direct access to the URL gives me Cloudflare security screen page redirects

Users can't access VPN cus Open VPN Cloud is depending on this for its SAML SSO integration portal

Consider switching to jsDelivr https://www.jsdelivr.com/unpkg
It's actively being developed and has been for the last 10 years

The only downside of this CDN is that it doesn't have the redirect behavior that unpkg has, where unpinned versions (i.e. @latest) have cache expiration of minutes but redirect to assets that have significantly higher cache expiration, tapping well into browser cache mechanisms.

has this started working for anyone..
we are seeing  it work now

re-running webpagetest.org on my site with unpkg.com called requests seems to work without 503 right now
response header
access-control-allow-origin: *
age: 74383
cache-control: public, max-age=31536000
cf-cache-status: HIT
cf-ray: 7614b0773d5e0736-IAD
content-type: text/plain; charset=utf-8
date: Fri, 28 Oct 2022 15:26:21 GMT
fly-request-id: 01GGDC92CKC8VAPHKC9TW6Z33J-iad
location: /web-vitals@2.0.1/dist/web-vitals.umd.js
server: cloudflare
strict-transport-security: max-age=31536000; includeSubDomains; preload
vary: Accept, Accept-Encoding
via: 1.1 fly.io
x-content-type-options: nosniff

On my side everything is fine now, but I get the same error of CORS policies

has this started working for anyone.. we are seeing it work now

It is intermittently working for us, looks to be region specific.

Working for us in UK now

re-running webpagetest.org on my site with unpkg.com called requests seems to work without 503 right now
response header
access-control-allow-origin: *
age: 74383
cache-control: public, max-age=31536000
cf-cache-status: HIT
cf-ray: 7614b0773d5e0736-IAD
content-type: text/plain; charset=utf-8
date: Fri, 28 Oct 2022 15:26:21 GMT
fly-request-id: 01GGDC92CKC8VAPHKC9TW6Z33J-iad
location: /web-vitals@2.0.1/dist/web-vitals.umd.js
server: cloudflare
strict-transport-security: max-age=31536000; includeSubDomains; preload
vary: Accept, Accept-Encoding
via: 1.1 fly.io
x-content-type-options: nosniff

Try this :)
https://cdn.jsdelivr.net/npm/web-vitals@2.0.1/dist/web-vitals.umd.js

Working now  for Ireland aswell

working fine in Germany again
https://twitter.com/mjackson/status/1586015926444429312

I'm getting a 500 for this: https://unpkg.com/@graphql-yoga/graphiql@3.0.3/dist/yoga-graphiql.es.js
It was working 18 hours ago.

Seeing intermittent issues today too as well.

seeing this too today
Failed to execute 'importScripts' on 'WorkerGlobalScope': The script at 'https://unpkg.com/typescript@^4.4.4/lib/typescript.js' failed to load.

I'm also getting 500s for https://unpkg.com/graphiql@1.11.5/graphiql.min.js

I'm also getting 500s for https://unpkg.com/graphiql@1.11.5/graphiql.min.js

Consider switching for better stability https://cdn.jsdelivr.net/npm/graphiql@1.11.5/graphiql.min.js

I'll actually update the upstream project I'm working with to bundle graphiql rather than loading it from the CDN and eliminate an external point of failure altogether.

https://unpkg.com/tippy.js@6 down

React is back up, so I assume all other packages are.

https://unpkg.com/react@17/umd/react.production.min.js - 520 Error

Is unpkg down ?
Getting the above error

I'm currently seeing this for https://unpkg.com/flatpickr/ as well:

Multiple imports intermittently down 4/4/23 12:54 pm PST

There are those relying on this for production when we know very well that Michael Jackson died back in 2009?

Just like Elvis.. he never really dies

Down again intermittently this morning (GMT). The jsdelivr alternative suggested by @jimaek works great. 🖖

UNPKG is down for us in the UK this morning! :(

Weird. I can open "https://unpkg.com/lit-element@2.0.1/lit-element.js?module" but link inside the document wich is "https://unpkg.com/lit-html@^1.0.0?module" cannot be open wich breaks some HomeAssistant Cards.

Same problem here

authors: AdamArutyunov AlbertoBoem AlessandroSalvetti Alexplose Alexr03 Applesauce314 ArinSantry Bahromiddin-0256 ChrystianDeMatos Domain314 J2Kbr LuckyCoders NachiketS-CanadaHelps NicolayDV OliverMaerz PITT72 Pheasey RavikumarTulugu Skalolazka123 UsAndRufus WebMasterSam WouterDoornbos YRELIS Yoshuwa aleksandar-cloudinary anthea-wu arvindanta azherebtsov bogdanpetru centminmod damianesteban dc-shackle diessica dopry dpitois emorin ettore-galli-dt gmiizi gzuidhof henalbrod iRonJ iamsyh jimaek jordaneliastam joshualeond karaaa006 kaushal-freshworks khalib lbianc markb-wa mddifilippo89 mikeldn mkroutikov mnyble mrlennartsson nazarii-semeniuk p-ob pcbowers perssonrichard robbiedhickey sathyalog sbussink-covadis sean-mystyle shhhmel smalcat78 toketas vgeirnaert-emakina vineethdas vonec ysm-dev yurijmikhalevich

repo: mjackson/unpkg


text:
Latest jquery broke YUI compressor

Looks like yui doesn't support ecmascript5+. Issue reported almost a year ago with no traction: https://github.com/yui/yuicompressor/issues/234
Switch to using CLOSURE compiler for js which seems to work better.
        <jsEngine>CLOSURE</jsEngine>
        <closureLanguage>ECMASCRIPT5</closureLanguage>

Closure compiler leaves line breaks every 500 or so chars (https://developers.google.com/closure/compiler/faq?csw=1#linefeeds) so also need to update the minification test.

Closure compiler leaves line breaks every 500 or so chars (https://developers.google.com/closure/compiler/faq?csw=1#linefeeds) so also need to update the minification test.


text:
🛑 FarmPi Mirror is down

In c4d57e0, FarmPi Mirror (https://farmpi.octofarm.net/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: FarmPi Mirror is back up in 2703d22 after 13 minutes.


text:
Uploading multiple files

It would be great, if the widget could support upload of multiple files, as the HTML input field supports this as well: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/file#multiple

See #236

authors: DirkR mkocansey

repo: mkocansey/bladewind


text:
Hpple file not found in bridging header

I get hpple file not found inside of bridging header.

Hi @EVPilot. Sorry for the late response. I'll close this issue for now. If you're still having trouble integrating WKZombie, please open a new issue with a detailed description of your setup. Thank you.

I gave up on it.

authors: EVPilot mkoehnke

repo: mkoehnke/WKZombie


text:
Sdi12 measurement #11

実装内容

SDI-12 に準拠した土壌水分センサ (以下，SDI-12センサ) を用いて，土壌水分量を測定するメソッドを実装する。
使用するセンサは，デジタルTDTセンサ (Digital Time Domain Transmissometer ) とする。

確認手順

M5Stampにプログラムを書き込み，シリアルモニタに1分ごとに測定値が出力されることを確認

スクリーンショット

測定結果

確認した環境

M1 MacBook Pro
macOS Monterey version 12.4
Arduino IDE version 1.8.16

resolve #11

確認したので，マージします。

repo: mktkhr/stamp-iot


text:
/v1/models endpoint not working ?

$ curl localhost:8080/v1/models -H "Content-Type: application/json"

2024-10-21 19:25:41,367 - INFO - Starting httpd at 127.0.0.1 on port 8080...
127.0.0.1 - - [21/Oct/2024 19:25:54] code 501, message Unsupported method ('GET')
127.0.0.1 - - [21/Oct/2024 19:25:54] "GET /v1/models HTTP/1.1" 501 -
----

<!DOCTYPE HTML>

        <meta charset="utf-8">
        <title>Error response</title>

Error response

Error code: 501

Message: Unsupported method ('GET').

Error code explanation: 501 - Server does not support this operation.

Are you using an outdated MLX LM? It works fine for me..
You can check the version:
python -c "import mlx_lm; print(mlx_lm.__version__)"

It should be something recent like 0.19.
pip install -U mlx-lm if it's old.

thanks, you are right, i had to update.

repo: ml-explore/mlx-examples


text:
Example of an async data-loader that automatically pre-fetches batches for training

Our code is setup as follows:
self.model.train(mode=True)

batch = next(train_iterator)

state = [self.model.state, self.optimizer.state]

@partial(mx.compile, inputs=state, outputs=state)
def step(input_ids: mx.array, targets: mx.array, lengths: mx.array):
    calculate_loss_and_grad_fn = nn.value_and_grad(self.model, loss_fn)

    (loss, number_tokens), grads = calculate_loss_and_grad_fn(
        self.model, input_ids, targets, lengths
    )

    clipped_grads, total_norm = optim.clip_grad_norm(
        grads, max_norm=self.args.grad_norm
    )

    self.optimizer.update(self.model, grads)

    return loss, number_tokens, total_norm

for train_step in range(self.args.train_steps):
    self.optimizer.learning_rate = self.get_learning_rate(step=train_step)

    loss, number_tokens, total_norm = step(
        batch["input_ids"], batch["targets"], batch["lengths"]
    )

    mx.eval(self.model.parameters(), self.optimizer.state, loss, total_norm)
    batch = next(train_iterator)

Unfortunately we are not saturating the GPU; I see 90-95% GPU usage from mactop.
We are building batches on the fly (as we do with pytorch) using an iterable-style dataset.
Is there a robust data-loader that pre-fetches batches and queues them up efficiently.
Looking for something that's equivalent to pytorch. Thanks!

You could try something pretty simple like the following which prefetches data given an iterator in a separate process and puts it in a queue.
import multiprocessing as mp

def prefetch(iterator):
    def f(iterator, queue):
        for sample in iterator:
            queue.put(sample)
    queue = mp.Queue(10)
    process = mp.Process(target=f, args=(iterator, queue))
    process.start()
    while True:
        sample = queue.get()
        yield sample
    process.join()

dataset = prefetch(dataset)

But I would recommend you perf first to see why you are not saturating the GPU. It may not have to do with data loading... a simple hack you can try is to use a dummy data batch that is fixed and see if that resolves the perf issues. That would point to data loading as worth fixing. Also 90-95% is not that bad.. I'm not sure how much headroom you have there.
I'm closing this as it's a bit outside the scope of MLX. But if you get evidence which suggests a performance bug or enhancement needed in MLX let us know.

@awni Thanks for the response!
We did end up writing a multi-threaded data-loader (presumably MLX drops the GIL) - we now hit a constant 98% on the GPU, which is pretty cool given that we are streaming over the files... the only drop is when we save checkpoints, which are not being saved async...

authors: awni vgoklani


text:
Scan ops do not support complex types yet

Scan ops do not support complex types yet

I get this error when trying to optimise with a loss containing .product() operation.
.sum() works fine.

Hmm, that's the gradient of product which I believe uses cumprod.  This is basically a duplicate of https://github.com/ml-explore/mlx/issues/673 which is that scan ops don't currently work on 64-bit types. We should fix that.

authors: awni gadirom

repo: ml-explore/mlx-swift


text:
[BUG] Can't install axpby in python env from examples/extensions.

Describe the bug
I want to write Metal kernel in mlx to do some compute. So first step, i try to run the axpby example follow https://ml-explore.github.io/mlx/build/html/dev/extensions.html. Every thing is ok until do "python -m pip install . ", the error information look like this:
~/Code/mlx/examples/extensions main                                                                                52s 🐍 base 16:51:33
❯ pip install .
Processing /Users/xzr/Code/mlx/examples/extensions
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      Collecting mlx@ git+https://github.com/mlx-explore/mlx@main
        Cloning https://github.com/mlx-explore/mlx (to revision main) to /private/var/folders/db/_s8gw_px34z675607w1z4vlm0000gn/T/pip-install-58eifkhu/mlx_72d2b55c006142c98f6bfedefc2a4600
        Running command git clone --filter=blob:none --quiet https://github.com/mlx-explore/mlx /private/var/folders/db/_s8gw_px34z675607w1z4vlm0000gn/T/pip-install-58eifkhu/mlx_72d2b55c006142c98f6bfedefc2a4600
        remote: Repository not found.
        fatal: repository 'https://github.com/mlx-explore/mlx/' not found
        error: subprocess-exited-with-error

        × git clone --filter=blob:none --quiet https://github.com/mlx-explore/mlx /private/var/folders/db/_s8gw_px34z675607w1z4vlm0000gn/T/pip-install-58eifkhu/mlx_72d2b55c006142c98f6bfedefc2a4600 did not run successfully.
        │ exit code: 128
        ╰─> See above for output.

        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: subprocess-exited-with-error

      × git clone --filter=blob:none --quiet https://github.com/mlx-explore/mlx /private/var/folders/db/_s8gw_px34z675607w1z4vlm0000gn/T/pip-install-58eifkhu/mlx_72d2b55c006142c98f6bfedefc2a4600 did not run successfully.
      │ exit code: 128
      ╰─> See above for output.

      note: This error originates from a subprocess, and is likely not a problem with pip.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Expected behavior
I hope can fix the question that i can do next step.
Desktop (please complete the following information):

OS Version: [MacOS 14.1]
Version [0.1.0]

Additional context
No

fatal: repository 'https://github.com/mlx-explore/mlx/' not found.           (mlx-explore)
shouldn't that be ....  repository 'https://github.com/ml-explore/mlx/.  ... (ml-explore)

I'm not sure where incorrect github url is coming from?  Did you change something in the extension?

I'm closing this for now. If you are having issues after fixing the Github URL, happy to reopen.

fatal: repository 'https://github.com/mlx-explore/mlx/' not found. (mlx-explore)
shouldn't that be .... repository 'https://github.com/ml-explore/mlx/. ... (ml-explore)

Thanks!

authors: BoilerToad Xu-Zirui awni

repo: ml-explore/mlx


text:
Few-shot object detector to TFLite

@khanhlvg the TFOD API few-shot training example uses this checkpoint ssd_resnet50_v1_fpn_640x640_coco17_tpu-8.
Would be nice to see if that fine-tuned model is convertible to TFLite using the recommended instructions. WDYT?

Sure, we can try with that model.

@khanhlvg the only issue I found there is a lack of guidance to serialize a model trained using the eager mode. I have filed an issue for that. I will keep this thread updated once I have something.

I see. TFLite team also have a plan to work on a few shot training e2e notebook including the conversion to TFLite. Therefore you can prioritize working on the MobileDet models as it's unlikely that anyone will work on it soon.

Closing the issue for now then. Feel free to open it up.

authors: khanhlvg sayakpaul


text:
Feature/build 2 ecr

I :

created a public repo called fndnt ont the correct AWS account (the name reservation is pending)
altered the build_components.sh script. So now It will create repo's if they do not exist yet and add a tag with the public ECR uri. (if you want to run it locally you will need to be logged in in Docker with AWS credentials)
I ran the script locally for one component and without building to docker hub (only ECR) succesfully
setup OIDC on the Public AWS account for github actions (using 1 2)
updated the github actions to include fetching credentials via OIDC and authenticating to the public ecr. (every AWS account can only have 1 public ECR so all aws commands using aws ecr-public ... will be ran against it.

I still need to test everything together in github itself.

I will need to alter the other github action scripts aswel but want to test this first


text:
Prepare release v0.6

upgrade to Arche v0.10.0
upgrade to Go toolchain 1.21.6

Pull Request Test Coverage Report for Build 7603565538

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 98.208%

Totals

Change from base Build 7577744204:
0.0%

Covered Lines:
548

Relevant Lines:
558

💛  - Coveralls


text:
Request: release a library

The idea is a Java (or Kotlin) library (jar) that is decoupled from the Android gui. This way we could write our own gui in Java or Kotlin for Android. The library would take prompts as input (String, JSON or what ever) and would return the answer after computing (as Strings or better as Streams).

+1

Moving to #361

authors: GameOverFlowChart li2109 tqchen

repo: mlc-ai/mlc-llm


text:
[Bug] Phi-3 model compile error

🐛 Bug
Here is the error when I try to compile the phi-3 model:

(mlcllm) [oan1pi@rng-dl01-w018:/fs/scratch/rb_bd_dlp_rng-dl01_cr_AIM_employees/oan1pi] $  cd /fs/scratch/rb_bd_dlp_rng-dl01_cr_AIM_employees/oan1pi ; /usr/bin/env /home/oan1pi/.conda/envs/mlcllm/bin/python /home/oan1pi/.vscode-server/extensions/ms-python.debugpy-2024.14.0/bundled/libs/debugpy/adapter/../../debugpy/launcher 35655 -- /fs/scratch/rb_bd_dlp_rng-dl01_cr_AIM_employees/oan1pi/mlcllm_conversion/compile.py mlcllm_conversion/Phi-3-mini-4k-instruct/config.json -o compiled --quantization q4f16_1
flashinfer is not supported on CUDA arch < 80
Compiling with arguments:
--config          Phi3Config(model_type='phi3', hidden_size=3072, vocab_size=32064, num_hidden_layers=32, num_attention_heads=32, intermediate_size=8192, rms_norm_eps=1e-05, num_key_value_heads=32, max_position_embeddings=4096, position_embedding_base=10000.0, rope_scaling=None, original_max_position_embeddings=4096, context_window_size=4096, prefill_chunk_size=4096, head_dim=96, tensor_parallel_shards=1, max_batch_size=1, kwargs={})
--quantization    GroupQuantize(name='q4f16_1', kind='group-quant', group_size=32, quantize_dtype='int4', storage_dtype='uint32', model_dtype='float16', linear_weight_layout='NK', quantize_embedding=True, quantize_final_fc=True, num_elem_per_storage=8, num_storage_per_group=4, max_int_value=7, tensor_parallel_shards=0)
--model-type      phi3
--target          {"thread_warp_size": runtime.BoxInt(32), "host": {"mtriple": "x86_64-redhat-linux-gnu", "tag": "", "kind": "llvm", "mcpu": "skylake-avx512", "keys": ["cpu"]}, "arch": "sm_70", "max_threads_per_block": runtime.BoxInt(1024), "libs": ["thrust"], "max_num_threads": runtime.BoxInt(1024), "kind": "cuda", "max_shared_memory_per_block": runtime.BoxInt(49152), "tag": "", "keys": ["cuda", "gpu"]}
--opt             flashinfer=0;cublas_gemm=0;faster_transformer=0;cudagraph=1;cutlass=1;ipc_allreduce_strategy=AUTO
--system-lib-prefix ""
--output          compiled
--overrides       context_window_size=None;sliding_window_size=None;prefill_chunk_size=None;attention_sink_size=None;max_batch_size=None;tensor_parallel_shards=None;pipeline_parallel_stages=None;disaggregation=None
Unknown output suffix: . Assuming shared library.
Traceback (most recent call last):
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/runpy.py", line 198, in _run_module_as_main
return _run_code(code, main_globals, None,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/runpy.py", line 88, in _run_code
exec(code, run_globals)
File "/home/oan1pi/.vscode-server/extensions/ms-python.debugpy-2024.14.0/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/main.py", line 71, in
cli.main()
File "/home/oan1pi/.vscode-server/extensions/ms-python.debugpy-2024.14.0/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 501, in main
run()
File "/home/oan1pi/.vscode-server/extensions/ms-python.debugpy-2024.14.0/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 351, in run_file
runpy.run_path(target, run_name="main")
File "/home/oan1pi/.vscode-server/extensions/ms-python.debugpy-2024.14.0/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 310, in run_path
return _run_module_code(code, init_globals, run_name, pkg_name=pkg_name, script_name=fname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oan1pi/.vscode-server/extensions/ms-python.debugpy-2024.14.0/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 127, in _run_module_code
_run_code(code, mod_globals, init_globals, mod_name, mod_spec, pkg_name, script_name)
File "/home/oan1pi/.vscode-server/extensions/ms-python.debugpy-2024.14.0/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 118, in _run_code
exec(code, run_globals)
File "/fs/scratch/rb_bd_dlp_rng-dl01_cr_AIM_employees/oan1pi/mlcllm_conversion/compile.py", line 142, in
main()
File "/fs/scratch/rb_bd_dlp_rng-dl01_cr_AIM_employees/oan1pi/mlcllm_conversion/compile.py", line 129, in main
compile(
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/mlc_llm/interface/compile.py", line 244, in compile
_compile(args, model_config)
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/mlc_llm/interface/compile.py", line 189, in _compile
args.build_func(
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/mlc_llm/support/auto_target.py", line 301, in build
relax.build(
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/tvm/relax/vm_build.py", line 347, in build
mod = pipeline(mod)
^^^^^^^^^^^^^
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/tvm/ir/transform.py", line 238, in call
return _ffi_transform_api.RunPass(self, mod)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "tvm/_ffi/_cython/./packed_func.pxi", line 339, in tvm._ffi._cy3.core.PackedFuncBase.call
File "tvm/_ffi/_cython/./packed_func.pxi", line 270, in tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 259, in tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 185, in tvm._ffi._cy3.core.CHECK_CALL
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/tvm/_ffi/base.py", line 481, in raise_last_ffi_error
raise py_err
File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/mlc_llm/compiler_pass/pipeline.py", line 188, in _pipeline
mod = seq(mod)
^^^^^^^^
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/tvm/ir/transform.py", line 238, in call
return _ffi_transform_api.RunPass(self, mod)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "tvm/_ffi/_cython/./packed_func.pxi", line 339, in tvm._ffi._cy3.core.PackedFuncBase.call
File "tvm/_ffi/_cython/./packed_func.pxi", line 270, in tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 259, in tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 185, in tvm._ffi._cy3.core.CHECK_CALL
File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/tvm/ir/transform.py", line 307, in _pass_func
return inst.transform_module(mod, ctx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/mlc_llm/compiler_pass/dispatch_kv_cache_creation.py", line 108, in transform_module
self.create_tir_paged_kv_cache(bb, kwargs)
File "/home/oan1pi/.conda/envs/mlcllm/lib/python3.11/site-packages/mlc_llm/compiler_pass/dispatch_kv_cache_creation.py", line 147, in create_tir_paged_kv_cache
cache = kv_cache.TIRPagedKVCache(target=self.target, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: TIRPagedKVCache.init() got an unexpected keyword argument 'enable_disaggregation'
[00:43:10] /workspace/tvm/src/relax/ir/block_builder.cc:65: Warning: BlockBuilder destroyed with remaining blocks!

To Reproduce
Steps to reproduce the behavior:

I downloaded the phi-3 model from https://huggingface.co/microsoft/Phi-3-mini-4k-instruct
2- I ran mlc_llm compile Phi-3-mini-4k-instruct -o compiled --quantization q4f16_1

Expected behavior
As far as I know, mlc_llm supports phi-3 model and the model I downloaded is the standard phi-3 model.
So, I expect the above instruction works without any issue.
Environment

Platform (e.g. WebGPU/Vulkan/IOS/Android/CUDA): CUDA
Operating system (e.g. Ubuntu/Windows/MacOS/...): Linux Fedora
Device (e.g. iPhone 12 Pro, PC+RTX 3090, ...): CPU
How you installed MLC-LLM (conda, source): pip
How you installed TVM-Unity (pip, source):
Python version (e.g. 3.10): 3.11.11
GPU driver version (if applicable): 560.35.03
CUDA/cuDNN version (if applicable): 12.4.0
TVM Unity Hash Tag (python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))", applicable if you compile models):
-USE_NVTX: OFF
USE_GTEST: AUTO
SUMMARIZE: OFF
TVM_DEBUG_WITH_ABI_CHANGE: OFF
USE_IOS_RPC: OFF
USE_MSC: OFF
USE_ETHOSU:
CUDA_VERSION: 12.3
USE_LIBBACKTRACE: AUTO
DLPACK_PATH: 3rdparty/dlpack/include
USE_TENSORRT_CODEGEN: OFF
USE_OPENCL_EXTN_QCOM: NOT-FOUND
USE_TARGET_ONNX: OFF
USE_AOT_EXECUTOR: ON
BUILD_DUMMY_LIBTVM: OFF
USE_CUDNN: OFF
USE_TENSORRT_RUNTIME: OFF
USE_ARM_COMPUTE_LIB_GRAPH_EXECUTOR: OFF
USE_THRUST: ON
USE_CCACHE: AUTO
USE_ARM_COMPUTE_LIB: OFF
USE_CPP_RTVM:
USE_OPENCL_GTEST: /path/to/opencl/gtest
TVM_LOG_BEFORE_THROW: OFF
USE_MKL: OFF
USE_PT_TVMDSOOP: OFF
MLIR_VERSION: NOT-FOUND
USE_CLML: OFF
USE_STACKVM_RUNTIME: OFF
USE_GRAPH_EXECUTOR_CUDA_GRAPH: OFF
ROCM_PATH: /opt/rocm
USE_DNNL: OFF
USE_MSCCL: OFF
USE_NNAPI_RUNTIME: OFF
USE_VITIS_AI: OFF
USE_MLIR: OFF
USE_RCCL: OFF
USE_LLVM: llvm-config --ignore-libllvm --link-static
USE_VERILATOR: OFF
USE_TF_TVMDSOOP: OFF
USE_THREADS: ON
USE_MSVC_MT: OFF
BACKTRACE_ON_SEGFAULT: OFF
USE_GRAPH_EXECUTOR: ON
USE_NCCL: ON
USE_ROCBLAS: OFF
GIT_COMMIT_HASH: fb765b633bc0d208d406a134df32c957760a5241
USE_VULKAN: ON
USE_RUST_EXT: OFF
USE_CUTLASS: ON
USE_CPP_RPC: OFF
USE_HEXAGON: OFF
USE_CUSTOM_LOGGING: OFF
USE_UMA: OFF
USE_FALLBACK_STL_MAP: OFF
USE_SORT: ON
USE_RTTI: ON
GIT_COMMIT_TIME: 2024-12-15 07:25:01 -0500
USE_HIPBLAS: OFF
USE_HEXAGON_SDK: /path/to/sdk
USE_BLAS: none
USE_ETHOSN: OFF
USE_LIBTORCH: OFF
USE_RANDOM: ON
USE_CUDA: ON
USE_COREML: OFF
USE_AMX: OFF
BUILD_STATIC_RUNTIME: OFF
USE_CMSISNN: OFF
USE_KHRONOS_SPIRV: OFF
USE_CLML_GRAPH_EXECUTOR: OFF
USE_TFLITE: OFF
USE_HEXAGON_GTEST: /path/to/hexagon/gtest
PICOJSON_PATH: 3rdparty/picojson
USE_OPENCL_ENABLE_HOST_PTR: OFF
INSTALL_DEV: OFF
USE_PROFILER: ON
USE_NNPACK: OFF
LLVM_VERSION: 18.1.8
USE_MRVL: OFF
USE_OPENCL: OFF
COMPILER_RT_PATH: 3rdparty/compiler-rt
USE_NNAPI_CODEGEN: OFF
RANG_PATH: 3rdparty/rang/include
USE_SPIRV_KHR_INTEGER_DOT_PRODUCT: OFF
USE_OPENMP: OFF
USE_BNNS: OFF
USE_FLASHINFER: ON
USE_CUBLAS: ON
USE_METAL: OFF
USE_MICRO_STANDALONE_RUNTIME: OFF
USE_HEXAGON_EXTERNAL_LIBS: OFF
USE_ALTERNATIVE_LINKER: AUTO
USE_BYODT_POSIT: OFF
USE_NVSHMEM: /workspace/nvshmem
USE_HEXAGON_RPC: OFF
USE_MICRO: OFF
DMLC_PATH: 3rdparty/dmlc-core/include
INDEX_DEFAULT_I64: ON
USE_RELAY_DEBUG: OFF
USE_RPC: ON
USE_TENSORFLOW_PATH: none
TVM_CLML_VERSION:
USE_MIOPEN: OFF
USE_ROCM: OFF
USE_PAPI: OFF
USE_CURAND: OFF
TVM_CXX_COMPILER_PATH: /opt/rh/gcc-toolset-11/root/usr/bin/c++
HIDE_PRIVATE_SYMBOLS: ON
Any other relevant information:

Additional context

The issue occurred because I was using the nightly version of MLC LLM while having the stable version of the MLC-AI package installed. This caused the problem. After reinstalling the nightly version of the MLC-AI package, everything works fine now.


text:
Datasets on reproducing FID results

    Thank you for your contribution!
    I am unable to reproduce the FID results.
    Can you provide 10000 image-caption pairs sampled from COCO validation set (C) and 1000 image-caption pairs from Flickr30K entities validation set (F)?
    Thanks!

你好。感谢您的关注。对于COCO FID，我们遵循huggingface Stable Diffuision1.4使用的评估方法。您可以在此链接 https://huggingface.co/CompVis/stable-diffusion-v1-4 查看。也就是说，他们使用来自 COCO 2017 验证集的 10k 随机提示。

Thank you for your reply！
Perhaps I was wrong, but why did I find out that Coco 2017 val set only has 5k prompts.

There are 5k images in coco 17 val set but around 25k captions. Prompts are defined at the caption-level, not the image-level.
val2017_rand_index.json
Feel free to download the above indices file if you want to ensure a fair comparison, though the variance should be very small even if you don't use exactly these indices. We center crop the images to ensure that they all have a square shape and use this codebase to calculate the FID.
The same pipeline also applies for the Flickr30K entities (val set).
Let us know if you have any other questions -- thanks!

coco 17 val 套装中有 5k 图像，但大约有 25k 字幕。提示是在标题级别定义的，而不是在图像级别定义的。
val2017_rand_index.json
如果您想确保公平的比较，请随时下载上述索引文件，尽管即使您不完全使用这些索引，方差也应该非常小。我们将图像居中裁剪，以确保它们都具有正方形，并使用此代码库来计算 FID。
同样的管道也适用于 Flickr30K 实体（val 集）。
如果您有任何其他问题，请告诉我们 - 谢谢！

Thank you for your reply, it has been of great help to me.
I have another question, how the efficiency indicator was obtained, especially how the standard deviation was obtained.
Look forward to your reply!

You can generate a number of images (say 100) with the same set of prompts for all your baselines using the same GPU. Then you time it and take the avg/std for efficiency. We strongly recommend that you manually do this for all your baselines (instead of taking our numbers) for this column because latency can differ even for the same GPU due to different system configs, hardware brand, software, etc.

You can generate a number of images (say 100) with the same set of prompts for all your baselines using the same GPU. Then you time it and take the avg/std for efficiency metrics. We strongly recommend that you manually do this for all your baselines (instead of taking our numbers) for this column because latency can differ even for the same GPU due to different system configs, hardware brand, software, etc.

Thank you for your reply, it has been of great help to me.
Wishing you a happy life.

I assume that all your questions have been resolved as of now, so I'm closing this issue. If you encounter any other questions, feel free to re-open it or submit a new issue. Have a great rest of your day!

authors: senguet zwcolin

repo: mlpc-ucsd/TokenCompose


text:
how to define a HW platform?

Would you consider the system to be the same if the main training chip is the same?
It matters for internal logistical reasons. It is easier to reserve machines with same processors but not the identical chassis/cluster.

SWG: This is complicated. We should write clear rules, but it will take work. For now, review committee will decide if two systems are sufficiently identical for MLPerf purposes.


text:
exit command doesn't exit

The instructions I've seen seem to indicate I can exit the running chatgpt wrapper by typing exit and hitting return.
However, that doesn't seem to be working for me:
$ chatgpt
Provide a prompt for ChatGPT, or type !help or ? to list commands.
1> exit

Sure, feel free to type "exit" or "quit" to end the session. Let me know if you have any other questions before you go.

2>

the command is actually !exit and not exit. I will update the message

authors: mmabrouk thehunmonkgroup

repo: mmabrouk/chatgpt-wrapper


text:
Support Countdown Mode

Overview
In Countdown mode, there is a bug in the underlying library that causes the Confirm button of this component to not work when a value is first selected by the picker. You have to move the wheel at least twice. This PR implements the workaround suggested in that issue.
Although this fixes the problem for the main use case, it unfortunately does not fix it completely. There is a use case (hopefully rare) where, if the user spins the picker to 0 hours, 0 minutes, iOS will immediately advance the picker to 0 hours 1 minute and the Confirm button won't work the next time you move the picker.... it has to be moved at least twice. I haven't been able to figure out a workaround for this.
Test Plan
I modified the example component, adding a Duration (countdown) button for iOS only (the underlying component does not support countdown on Android.) I ran the example on an iOS device and it works fine. I also ran on an Android device to make sure I didn't break anything there.

@chetstone Just wanted to say thank you for adding this workaround. I've been stuck on this problem for a couple days now and now my app is working as it should! Thank you!

authors: AManNamedGaurav chetstone


text:
wrap-nested-params only includes the last value in the vector

As of Ring 0.3.11, nested parameters are no longer being vectorized correctly. The following code will contain only the last value:
(defn nested-test [_]
  (html
     [:head [:title "test"]]
     [:body
        [:form {:action "/submit" :method "POST"}
        [:input {:type :text :name "mything[]" :value "10"}]
        [:input {:type :text :name "mything[]" :value "20"}]
        [:input {:type :submit}]]]))

(defn submit [request]
   (println (request :params)))

Expected:
{:mything ["10" "20"]}
Actual:
{:mything ["20"]}
For complete code and more context see this thread:
https://groups.google.com/d/msg/clj-noir/Hub0ica7X6c/-f22jkEEr0sJ

This is actually a problem with wrap-multipart-params, not wrap-nested-params.
The multipart middleware is not storing parameter values with the same key as a vector.

Ah whoops, updated the issue to reflect that.

authors: ibdknox weavejester

repo: mmcgrana/ring


text:
Feature Request: support case insensitivity

I think it would be nice if we had an option to specify case insensitivity for search

I would accept a contribution (PR) that found an elegant way to add this.
It could potentially be added as an additional option matchCase: false or something similar.

authors: jloveland mmckegg

repo: mmckegg/json-query


text:
hook不成功 墨墨版本5.0.30

一直弹正在寻找hook函数

框架、安卓版本、xposed api版本是？

你是不是花了上限，花完不到600了

authors: dunxuan mmmlllnnn zwj6bao


text:
Network plot for top-N features

Is it possible to create a network plot only for top-N features in the dataset? When there are a lot of features (ex. 200), the plot starts being noisy. Thank you!

Yes this should be possible! However, @AsyaOrlova, could you be a bit more specific what should be selected and on what grounds?As I understand it, you would like to select a lower number of features to put onto the circle, right? Would it be a solution to create a „test“ feature that will include all other interactions? Or would you rather remove all features that make up only a fraction (let’s say < 5%) of all the interactions?
Thanks for your interest! 😊

authors: AsyaOrlova mmschlk

repo: mmschlk/shapiq


text:
Allow user to specify region when using bref init

When I use vendor/bin/bref init, I'll get a template with us-east-1 region by default.
It was quite difficult for me to find out I could modify it in the Layers string in template.yaml.
Maybe Bref should give an opportunity to define it as a command option ?
Something like:
vendor/bin/bref init --region=eu-west-1
I tried to do something like that:
    if ($region !== 'us-east-1') {
        file_put_contents('template.yaml', implode('',
            array_map(function($data) use ($region) {
                return stristr($data,"                - 'arn:aws:lambda:us-east-1:209497400698:layer:php-73-fpm:1'") ? "                - 'arn:aws:lambda:".$region.":209497400698:layer:php-73-fpm:1'\n" : $data;
            }, file('template.yaml'))
        ));
    }

It works for HTTP, but would not for Console for example.
I'm not great at modifying files :stuck_out_tongue:

👍
I think it can be OK to blindly replace any us-east-1 string with the target region (it's a very specific string, I don't see a problem happening soon).
For example: str_replace('us-east-1', $region, file_get_contents(...))

But I think also the region could be asked by bref init (with the goal that bref init is a complete "assistant"), for example:
$ vendor/bin/bref init
In which AWS region do you want to deploy your application? [us-east-1]

We could even repeat the note added in #280 to encourage users to use us-east-1 for their first project.

authors: mnapoli ngdo-pro

repo: mnapoli/bref


text:
BUG: Missing epochs with RtEpochs

The RtEpochs misses epochs if they start at the boundary of two buffers. The trigger channel is now read only from the current buffer. If the triggers are split between two buffers it's possible to miss them since they are not long enough.
Example:
[... 0 0 0 0 1 1][1 1 1 1 0 0 0 ... ]
Trigger channel that's split to two buffers. If we want a minimum of 5 samples neither of these buffers alone triggers an epoch.
I have written a small fix and can create a pull request for it.

this is fixed in #3738

authors: Odingod agramfort

repo: mne-tools/mne-python


text:
Partial topomaps (masking?)

In my effort to use MNE Python with fNIRS data I find that a major issue is to visualize topomaps. In most fNIRS systems, channels come in limited number and only cover part of the scalp, while in EEG and MEG the whole scalp is usually covered.
It would be usuful to have some sort of masking support in plot_topomap, for example using a convex hull approach (quoting @agramfort). I'm not sure whether this can be achieved using other parameters such as outlines and image_mask and, if so, how.
p.s. Probably a new name for this feature should be introduced, because the parameter name 'mask' is taken (by mask and mask_params) and is used for a different purpose.

could you share a small -ave.fif file to test?

Sorry I had to wait for a vacation day to get back to this.
This is an evoked file with my montage. In the layout plot it does not render properly because it normalizes the coordinates though.
evoked-ave.zip

thanks for sharing
we'll look into it

Closing this as a dup of #3987, @mcaffini feel free to subscribe to that issue for updates

authors: Eric89GXL agramfort mcaffini


text:
raw.set_montage should return self

Currently raw.set_montage(montage)  does not return anything so I get None.
It would be nice if it returned self so I could chain it with other operations:
raw = mne.io.read_raw_eeglab(fname, preload=True).filter(1, None).set_montage(montage)

Agreed, +1 for PR

I will like to work on this issue for start. Thanks

this is done


text:
Aseg2 mni

fixed bug in color of aseg label
- color = np.array(all_labels_aseg[1][ind]) / 255
+ color = tuple(np.array(all_labels_aseg[1][ind]) / 255.)

add function to convert to MNI space the coo of aseg labels

@annapasca travis is not happy
can you also update a test?

Hi @agramfort !
I made some little changes and added test_aseg_vertex_to_mni but I don't know if this test makes sense.
About the travis error, it seems to me that the only error is that I have to add aseg_vertex_to_mni in doc/python_reference.rst. Is it? and if yes where?
Thks!

In the "Source Space Data" section, you can put it in as source_space.aseg_vertex_to_mni

you have travis errors due to bad docstring formatting.

authors: agramfort annapasca larsoner


text:
[MRG] Fix threshold must be monotonic condition

Fix #4724

test?

I'm going to code up a Gramfort-CI that looks at the diff and if there are no test_ files changed, writes:
test?

🤣
BTW, the test I've just added does not test for the reported issue, but only for the already existing checks:
https://github.com/cbrnr/mne-python/blob/230d097f3528f83b96d4447482d8867a91aff7a7/mne/stats/cluster_level.py#L301-L306
How do I add a test that checks if no error is raised? The following call previously caused an error:
permutation_cluster_test(data, tail=-1, threshold=dict(start=0, step=-0.1))

But now works with this fix.

You can call the function but don't need to check any of its outputs (smoke test)

OK good, so I'll add this test as well. Furthermore, I have no idea how to trigger the "Thresholds must be monotonically increasing" errors I just changed - ideas?

tail=1, threshold=dict(start=1, step=-0.5) and tail=-1, threshold=dict(start=-1, step=0.5) should do it

Never mind, this is just a warning... OK to use a test that generates a warning?

We don't want un-caught warnings, you can catch it with warnings.catch_warnings(record=True): context manager or just use randn instead of rand

(and remove the subtraction, or multiply your existing values by 10, etc.)

I can't get it to work by increasing numbers etc., these are the max values of the permutation statistics and not the input data. The above tests catch the warnings as you suggest, so I'll do the same.

tail=-1, threshold=dict(start=-1, step=0.5) triggers the first error (because step must be negative for tail==-1).

Also, how can I catch a specific ValueError with assert_raises?

We haven't so far -- there is assert_raises_regex in numpy I think (but might be too new NumPy for us), pytest might also provide something

OK, then let's better not use it for now. Can you think of a way to trigger the monotonic error for tail=-1?

No, it sounds like the checks might be redundant, then

Cool, let's see if Travis, and more importantly @agramfort is happy now :smile:

If you want, fine with me either way

After playing around with this for a while I give up. No matter how I change the data I always get the warning. For this test, I'd really like to avoid this warning though.

Maybe there is something wrong with the warning, then?

Yes, hmmmm. The problem is that it's using the F statistic, which is always positive. Then it constructs the thresholds array as follows:
thresholds = np.arange(threshold['start'], stop, threshold['step'], float)

Here, threshold['step'] is negative (because tail=-1) and stop is positive. Therefore, thresholds is empty and hence the warning.

So for this function using negative values actually doesn't make any sense. You can use the 1-sample tests (which use t) to avoid this

don't forget to update what's new

Alright, this should be it. Ready to merge if CIs return happy.

I don't know how you normally handle PRs, but in this case it would be good to squash all commits into one...

it gets squashed on merge.

Except for the OSX jobs all CIs are happy.

Thanks @cbrnr

authors: agramfort cbrnr larsoner


text:
Unified reading

This PR improves the handling of UTF-* encoded files for all formatters and fixes #137.

Gnaah, thanks. Fixed.

Awesome! Thanks.

authors: scelis sebastianludwig

repo: mobiata/twine


text:
Maestro UI automation test is working very hard

It freezes the computer when running YAML 3-4 times in a row, this is a very bad result (computer config: Macbook Pro, i7, 8th gen, 16GB RAM) it shouldn't work so hard with this configuration

Please check for other running processes on your device. Maestro itself is not too intensive on resources, but your Emulator/Simulator or IDEs might be.

ok i will check thanks

authors: Yuldoshev dmitry-zaitsev

repo: mobile-dev-inc/maestro


text:
🛑 PizzaRiing Web is down

In b76fba0, PizzaRiing Web (https://pizzariing.uy) was down:

HTTP code: 0
Response time: 0 ms

Resolved: PizzaRiing Web is back up in 4d6b9d7.

repo: mobilitysol/monitorweb


text:
Memory leaks in version 1.0.x

Describe the bug
The FirebaseApp causes the memory leaks when authentication task started and the async task was removed from the queue but it is remaining in the memory.
To Reproduce
Set the wrong end point of the Google APIs for auth task request.  The request will fail and the memory leaks will be occurred in the next auth request.
Expected behavior
The auth task's async data should be deconstructed.

This issue is now fixed in v1.1.0.

repo: mobizt/FirebaseClient


text:
Cannot deploy to github or netlify

@mrozbarry How do you you deploy mobti.me?  Ideally, I would like to deploy  a modified version of mobtime (see discussion on mobtime plus) to Netlify, github, heroku or other.  When I try  yarn deploy it fails with husky install error.

FYI, I am on vacation until Feb 9 so may not respond until then.

I deploy to digital ocean over ssh. I don't think netlify or github would work, considering they don't support persistent backends (as far as I know). With heroku, I think they support some sort of docker deployment, and since we merged #221 , I think that might be viable.
One thing to keep in mind is that the public directory is static web content, and src is a backend node application. Deploying public is easy in the sense that there is no compile steps, just load up a web server and serve that directory (which the node server does, too). The main issue is most of the free-tiered web hosting platforms won't let you run a custom nodejs server + redis or a docker image.
That said, if you want to stick to something free, heroku is likely your best bet, otherwise, get/rent a cheap ubuntu server machine.
In the future, the mobtime server might be centralized, meanwhile clients would be free to host on static sites, but I think that's a bit of a distant future.

Thanks, I will see how heroku works.  I was having trouble with the yarn deploy command with husky but your new fixes may address that.

authors: ethanstrominger mrozbarry

repo: mobtimeapp/mobtime