text:
WIP: simple PyLmod implementation using the StellarGradeBook class almost unmodified

This PR is response to @Ferdi's request to do the minimal version for 0.1.  It is a WIP because there are still some changes to bring it to the same level as PR#1.  This can be tested by running the exercise scripts in https://github.com/mitocw/grades2stellar/tree/features/pylmod_refactoring

It would be nice if you had a paired branch of grades2stellar you could point me too to go along with this

None of this works for me.  I am getting stack traces from the LMOD service regardless of what call I make:

Internal error

org.springframework.beans.TypeMismatchException: Failed to convert value of type 'java.lang.String' to required type 'java.lang.Long'; nested exception is org.springframework.core.convert.ConversionFailedException: Unable to convert value "None" from type 'java.lang.String' to type 'java.lang.Long'; nested exception is java.lang.NumberFormatException: For input string: "None"
	at org.springframework.beans.SimpleTypeConverter.convertIfNecessary(SimpleTypeConverter.java:55)
	at org.springframework.validation.DataBinder.convertIfNecessary(DataBinder.java:534)
	at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.resolvePathVariable(HandlerMethodInvoker.java:705)
	at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.resolveHandlerArguments(HandlerMethodInvoker.java:352)
	at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:171)
	at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:426)
	at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:414)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:368)
	at org.springframework.security.web.authentication.switchuser.SwitchUserFilter.doFilter(SwitchUserFilter.java:178)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:97)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:100)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:78)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:35)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
	at edu.mit.gradebookng.services.security.TouchstoneAuthenticationFilter.doFilter(TouchstoneAuthenticationFilter.java:215)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:79)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:169)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:200)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
Caused by: org.springframework.core.convert.ConversionFailedException: Unable to convert value "None" from type 'java.lang.String' to type 'java.lang.Long'; nested exception is java.lang.NumberFormatException: For input string: "None"
	at org.springframework.core.convert.support.ConversionUtils.invokeConverter(ConversionUtils.java:42)
	at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:184)
	at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:141)
	at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:93)
	at org.springframework.beans.SimpleTypeConverter.convertIfNecessary(SimpleTypeConverter.java:49)
	... 53 more
Caused by: java.lang.NumberFormatException: For input string: "None"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
	at java.lang.Long.parseLong(Long.java:410)
	at java.lang.Long.valueOf(Long.java:525)
	at org.springframework.util.NumberUtils.parseNumber(NumberUtils.java:158)
	at org.springframework.core.convert.support.StringToNumberConverterFactory$StringToNumber.convert(StringToNumberConverterFactory.java:59)
	at org.springframework.core.convert.support.StringToNumberConverterFactory$StringToNumber.convert(StringToNumberConverterFactory.java:1)
	at org.springframework.core.convert.support.GenericConversionService$ConverterFactoryAdapter.convert(GenericConversionService.java:528)
	at org.springframework.core.convert.support.ConversionUtils.invokeConverter(ConversionUtils.java:36)
	... 57 more

Cookies:

I have updated the description to include the Grades2Stellar branch to use for testing.

Be sure to clean up pep8 and pylint issues:
pep8 pylmod
pylmod/Client.py:9:1: E302 expected 2 blank lines, found 1
pylmod/__init__.py:5:18: E261 at least two spaces before inline comment
pylmod/stellargradebook.py:46:9: E124 closing bracket does not match visual indentation
pylmod/stellargradebook.py:108:24: E702 multiple statements on one line (semicolon)
pylmod/stellargradebook.py:341:9: E124 closing bracket does not match visual indentation
pylmod/stellargradebook.py:470:37: E131 continuation line unaligned for hanging indent
pylmod/stellargradebook.py:472:80: E501 line too long (80 > 79 characters)

pylint pylmod
pylint pylmod
No config file found, using default configuration
************* Module pylmod
E:  7,21: Undefined variable name 'StellarGradeBook' in __all__ (undefined-all-variable)
************* Module pylmod.Client
W:  5, 0: Relative import 'stellargradebook', should be 'pylmod.stellargradebook' (relative-import)
C:  7, 0: Invalid constant name "log" (invalid-name)
************* Module pylmod.stellargradebook
C: 46, 0: Wrong continued indentation.
        }
        ^  | (bad-continuation)
C:283, 0: Wrong hanging indentation before block.
        self, name, shortname, weight,
        ^   | (bad-continuation)
C:284, 0: Wrong hanging indentation before block.
        maxpoints, duedatestr, gradebookid='',
        ^   | (bad-continuation)
C:285, 0: Wrong hanging indentation before block.
        **kwargs
        ^   | (bad-continuation)
C:326, 0: Wrong hanging indentation before block.
        self, assignmentid, studentid, gradeval, gradebookid='', **kwargs
        ^   | (bad-continuation)
C:341, 0: Wrong continued indentation.
        }
        ^           | (bad-continuation)
C:380, 0: Wrong hanging indentation before block.
        self, datafn, create_assignments=True,
        ^   | (bad-continuation)
C:381, 0: Wrong hanging indentation before block.
        email_field=None, single=False
        ^   | (bad-continuation)
C:434, 0: Wrong hanging indentation before block.
        self, creader, create_assignments, email_field, NonAssignmentFields
        ^   | (bad-continuation)
C:470, 0: Wrong hanging indentation before block.
                                    'assignmentId' not in r.get('data')
                            |   |   ^ (bad-continuation)
C:515, 0: Wrong hanging indentation before block.
        self, creader, create_assignments, email_field, NonAssignmentFields
        ^   | (bad-continuation)
C:  1, 0: Missing module docstring (missing-docstring)
C: 15, 0: Invalid constant name "log" (invalid-name)
C: 68,12: Invalid attribute name "URLBASE" (invalid-name)
W: 76, 8: Specify string format arguments as logging function parameters (logging-not-lazy)
C: 78, 4: Invalid argument name "fn" (invalid-name)
W: 86,16: Specify string format arguments as logging function parameters (logging-not-lazy)
W: 92,16: Specify string format arguments as logging function parameters (logging-not-lazy)
C:101, 4: Invalid argument name "fn" (invalid-name)
W:103, 8: Specify string format arguments as logging function parameters (logging-not-lazy)
C:104, 8: Invalid variable name "r" (invalid-name)
F:108,12: Unable to import 'ipdb' (import-error)
C:108,25: More than one statement on a single line (multiple-statements)
E:110,20: Instance of 'StellarGradeBook' has no 'do_touchstone' member (no-member)
E:114,16: Instance of 'StellarGradeBook' has no 'init_touchstone' member (no-member)
W:107,26: Unused variable 'err' (unused-variable)
C:160, 8: Invalid variable name "gb" (invalid-name)
W:214,12: Redefining built-in 'map' (redefined-builtin)
C:168, 4: Invalid argument name "sectionName" (invalid-name)
C:220,12: Invalid argument name "x" (invalid-name)
C:220,12: Missing function docstring (missing-docstring)
W:196,21: Unused variable 'sec' (unused-variable)
C:230, 4: Invalid argument name "sectionName" (invalid-name)
C:230, 4: Missing method docstring (missing-docstring)
R:282, 4: Too many arguments (7/5) (too-many-arguments)
W:299, 8: Specify string format arguments as logging function parameters (logging-not-lazy)
W:301, 8: Specify string format arguments as logging function parameters (logging-not-lazy)
C:320,12: Invalid variable name "a" (invalid-name)
W:343, 8: Specify string format arguments as logging function parameters (logging-not-lazy)
C:374,12: Invalid variable name "s" (invalid-name)
C:406, 8: Invalid variable name "NonAssignmentFields" (invalid-name)
C:416,12: Invalid variable name "fp" (invalid-name)
C:418,12: Invalid variable name "fp" (invalid-name)
C:425,12: Invalid variable name "r" (invalid-name)
C:427,12: Invalid variable name "r" (invalid-name)
C:433, 4: Invalid argument name "NonAssignmentFields" (invalid-name)
R:433, 4: Too many local variables (25/15) (too-many-locals)
W:450,16: Specify string format arguments as logging function parameters (logging-not-lazy)
C:465,24: Invalid variable name "r" (invalid-name)
W:472,28: Specify string format arguments as logging function parameters (logging-not-lazy)
W:480,20: Specify string format arguments as logging function parameters (logging-not-lazy)
C:484,16: Invalid variable name "ok" (invalid-name)
W:487,23: Catching too general exception Exception (broad-except)
W:488,20: Specify string format arguments as logging function parameters (logging-not-lazy)
C:492,20: Invalid variable name "ok" (invalid-name)
W:501, 8: Specify string format arguments as logging function parameters (logging-not-lazy)
C:506, 8: Invalid variable name "r" (invalid-name)
C:507, 8: Invalid variable name "dt" (invalid-name)
W:508, 8: Specify string format arguments as logging function parameters (logging-not-lazy)
W:458,25: Unused variable 'assignment' (unused-variable)
W:434,23: Unused argument 'create_assignments' (unused-argument)
W:448,17: Unused variable 'student' (unused-variable)
C:514, 4: Invalid argument name "NonAssignmentFields" (invalid-name)
R:514, 4: Too many local variables (20/15) (too-many-locals)
C:536,24: Invalid variable name "r" (invalid-name)
W:547,24: Specify string format arguments as logging function parameters (logging-not-lazy)
W:552,16: Specify string format arguments as logging function parameters (logging-not-lazy)
C:557,16: Invalid variable name "r" (invalid-name)
W:530,25: Unused variable 'assignment' (unused-variable)
W:515,23: Unused argument 'create_assignments' (unused-argument)
W:525,17: Unused variable 'student' (unused-variable)

Global evaluation
-----------------
Your code has been rated at 6.49/10

Verified against edx-platform with the changes suggested above, let me know when you are ready for round whatever we are at.
Also, to point platform at the g2xserver you can run it locally with just python grades2stellar/g2s_xserver.py and then in your devstack add:
from .devstack import * # pylint: disable=wildcard-import, unused-wildcard-import

FEATURES['REMOTE_GRADEBOOK_URL'] = 'http://<your_ip_or_hostname>:8104'

to lms/envs/private.py in the edx-platform directory.  Then you can load the devops course in via studio (or command line, or sysadmin dash) since it is configured to use the demo gradebook.

Just to write down what we discussed yesterday, all those "Invalid variable name" pylint errors come from the names being too short. I don't know what all of pylint's requirements are, but one- or two-letter names are too short.
It looks like the easiest way to find out the specific meaning of an error is to do something like:
pylint --help-msg=invalid-name

Although I have to say the standard message for invalid-name leaves something to be desired.
Carson, do you have an opinion about the other errors? I would think the missing docstrings are the biggest issue.

The names don't really bother me that much since they just make it hard to read the code.  I'm generally more concerned with the programming runtime issues like unused variables that take up memory, useless strings, and overly broad exceptions which make debugging issues hard.  After those big ones, doc strings are really important (following the google style guide via a sphinx plugin: https://pypi.python.org/pypi/sphinxcontrib-napoleon), and then variable names should be more descriptive than a single letter or two will allow.

Alright, other than that last little comment, and the small fix on the other side, I'm :bullettrain_side:

Oh boy, merge time

authors: carsongee pdpinch pwilkins

repo: mitodl/PyLmod


text:
fix user profiles in MicroMaster program certificates models

What are the relevant tickets?

https://github.com/mitodl/hq/issues/3435
Description (What does it do?)

Fixing a few warnings generated https://pipelines.odl.mit.edu/runs/6c4e62bf-1d18-472d-9f20-48d7c5fd48ba
from int__micromasters__program_certificates.
The warns are due to we were pulling users email and profiles from MicroMasters, some new DEDP certificate holders' social auth profile names don't match with MITx Online, which then cause their email and profile data are blank.
For DEDP certificates on MITx Online, we are now pulling their profile and email from MITx Online directly.
How can this be tested?

Run the following command, data should be fixed now with no warnings
dbt run +int__micromasters__program_certificates
dbt test int__micromasters__program_certificates
dbt build marts__micromasters_course_certificates

The builds/dbt tests ran clean. One thing I noticed while testing is that user gender is populated less often with this change (null on about 106 rows vs 70 before the change). Is that ok?

@KatelynGit Good call. Looks like these learners didn't provide profile data on MITx Online yet, so I fill in the blank with their MicroMasters profile data. user_gender , user_address_state_or_territory and user_year_of_birth are populated more than production.
I believe  user_gender has over 250 null or empty in __micromasters_program_certificates_dedp_from_mitxonline on production. With this change, there are only 2.

The builds/dbt tests ran clean. One thing I noticed while testing is that user gender is populated less often with this change (null on about 106 rows vs 70 before the change). Is that ok?

@KatelynGit Good call. Looks like these learners didn't provide profile data on MITx Online yet, so I fill in the blank with their MicroMasters profile data. user_gender , user_address_state_or_territory and user_year_of_birth are populated more than production. I believe user_gender has over 250 null or empty in __micromasters_program_certificates_dedp_from_mitxonline on production. With this change, there are only 2.

It looks good now thanks!

source: gharchive/pull-request

authors: KatelynGit rachellougee

repo: mitodl/ol-data-platform

license_type: permissive

license_source: github-api


text:
Experimental: mitogen: Remove minification of Python source code

Done on the basis that sending a few more bytes of the network is quicker than
tokenizing and untokeninzing the modules.

As a comparison the slowest jobs in https://dev.azure.com/mitogen-hq/mitogen/_build/results?buildId=382&view=results (revision this branch is based on) took around 24 minutes.

Faster, less code, less complexity... seems like a great change to me.

authors: jamesob moreati

repo: mitogen-hq/mitogen


text:
Werkzeug 0.11 does not allow debug

Running Flask, on Linux or Windows or OS X, if you run:
from flask import Flask
app = Flask(__name__)

@app.route('/')
def index():
    return "foo"

if __name__ == '__main__':
    app.run(debug=True)

...then python kills itself with an error (cant remember from top of my head...as I am not at my PC).
However, when you remove debug=True everything is fine.
Running # pip uninstall werkzeug

... # pip install "werkzeug<0.11" ... fixes this issue.
I am new to this framework...so I apologise for this horrendous bug report. I just was wondering if I could have clarification. We are studying flask at uni and almost everyone has had this issue or others gettings Flask to install or run. Once it works, it is amazing. However, installation is a right pain.
Sorry for being vague. Hope this is of use to the devs of Flask.
Ashley

I need to know the error you are getting otherwise I have to close this as cannot reproduce this.

I have fixed it by installing Werkzeug 0.10.4 instead of 0.11. I am having to do coursework with Flask so I am not going to go out of my way to mess it up. All I can say is practically everyone in my year that I know have had one issue or another.
I have stumbled across this issue: https://github.com/mitsuhiko/werkzeug/issues/798 ... it is closed yet we are still having this issue.
Sorry for not being more help. I thought it would be better to at least make you aware that there is a potential issue rather than just ignoring it.
Thanks for the reply.
Ashley
On 21 Nov 2015 18:31, Armin Ronacher notifications@github.com wrote:I need to know the error you are getting otherwise I have to close this as cannot reproduce this.
—Reply to this email directly or view it on GitHub.

All I can say is practically everyone in my year that I know have had one issue or another.

Unfortunately this reporting is not helpful. I need to know exactly which version of Werkzeug you were running as I cannot reproduce this. 0.11 is know to be broken but the latest version is 0.11.2 which has this bug to the best of our knowledge fixed.  Are you saying you have this issue with 0.11.2?

Which ever version would have been installed with pip install flask just over a week ago.
I do not want to risk messing my install up. But if you are suggesting a new release has fixed the issue then please beg my pardon. I just wanted to alert you if you had not already known. I apologise for the hassle.
Thanks
On 21 Nov 2015 22:00, Armin Ronacher notifications@github.com wrote:
All I can say is practically everyone in my year that I know have had one issue or another.
Unfortunately this reporting is not helpful. I need to know exactly which version of Werkzeug you were running as I cannot reproduce this. 0.11 is know to be broken but the latest version is 0.11.2 which has this bug to the best of our knowledge fixed.  Are you saying you have this issue with 0.11.2?
—Reply to this email directly or view it on GitHub.

source: gharchive/issue

authors: ashleyscopes mitsuhiko

repo: mitsuhiko/flask

license_source: bigquery


text:
Configure for the inline snapshot indentation

Hi , thank you for the awesome tool!
Currently,cargo insta review generates inline snapshots in the form like:
assert_snapshot!(
    test.execute("select id, value from t1"),
    @r###"
+----+-------+
| id | value |
+----+-------+
| 2  | 102   |
+----+-------+
| 3  | 103   |
+----+-------+
| 4  | 104   |
+----+-------+

(3 rows)
"###
);

I wonder if it's possbile to make the following output with project-level configuration:
assert_snapshot!(
    test.execute("select id, value from t1"),
    @r###"
    +----+-------+
    | id | value |
    +----+-------+
    | 2  | 102   |
    +----+-------+
    | 3  | 103   |
    +----+-------+
    | 4  | 104   |
    +----+-------+

    (3 rows)
    "###
);

Maybe related to https://github.com/mitsuhiko/insta/issues/457

Thanks for the issue!
It should give reasonable indentation already. Could you post a reproducible example? The source can just be a literal string.

Starting from this test:
#[test]
fn test_foo() {
    assert_snapshot!(
        "aaa\nbbb\nccc\nddd",
        @""
    );
}

After review:
#[test]
fn test_foo() {
    assert_snapshot!(
        "aaa\nbbb\nccc\nddd",
        @r###"
    aaa
    bbb
    ccc
    ddd
    "###
    );
}

If starting from oneline:
#[test]
fn test_foo() {
    assert_snapshot!("aaa\nbbb\nccc\nddd", @"");
}

After review:
#[test]
fn test_foo() {
    assert_snapshot!("aaa\nbbb\nccc\nddd", @r###"
    aaa
    bbb
    ccc
    ddd
    "###);
}

I've a tool using ast-grep to replace @ to - to make cargo fmt works (https://github.com/mitsuhiko/insta/issues/499):
#[test]
fn test_foo() {
    assert_snapshot!("aaa\nbbb\nccc\nddd", -r###"
    aaa
    bbb
    ccc
    ddd
    "###);
}

cargo fmt:
#[test]
fn test_foo() {
    assert_snapshot!(
        "aaa\nbbb\nccc\nddd",
        -r###"
    aaa
    bbb
    ccc
    ddd
    "###
    );
}

replace - back to @:
#[test]
fn test_foo() {
    assert_snapshot!(
        "aaa\nbbb\nccc\nddd",
        @r###"
    aaa
    bbb
    ccc
    ddd
    "###
    );
}

Sorry for being slow — am I understanding correctly that the current output and the desired output have the same indentation? And the difference is the opening delimiter?
(Am on phone so possibly I'm missing something...)

Thank you for quick reply!
The desired output is
#[test]
fn test_foo() {
    assert_snapshot!(
        "aaa\nbbb\nccc\nddd",
        @r###"
        aaa
        bbb
        ccc
        ddd
        "###
    );
}

while I get
#[test]
fn test_foo() {
    assert_snapshot!(
        "aaa\nbbb\nccc\nddd",
        @r###"
    aaa
    bbb
    ccc
    ddd
    "###
    );
}

I think a couple of things here:

Specifically the indentation: this is selected at the same level as the macro. Which I think generally works OK? But open to alternatives.

Though I'd be very hesitant to add a configuration, we want to have a simple interface, so we should come up with a rule that works across most cases. Is there a better rule than the current one?

Formatting: I agree the formatting is occasionally not be ideal. cargo fmt works within macros iff the macro contents is valid rust, which @ isn't.

Ideally we could have syntax valid rust syntax, and then cargo fmt would work. Currently the difficulty is that we need to distinguish between assert_snapshot!(name, expr) and assert_snapshot!(expr, inline_value), and so we need something that isn't a valid rust expression. So I can't think of a way around it
Your ast-grep approach is clever!

Indeed. These maybe project-specific preference. The better way could be keeping the default behavior and format using ast-grep in project desiring different style.

so we need something that isn't a valid rust expression. So I can't think of a way around it

rustfmt is requiring a syntactical valid expression, but not semantically valid expression. Thus there is room to find such patterns. For example, my using -r###""### utilize the fact that unary minus to a string is valid in syntax but invalid in type. Symbols like *, !, ?(postfix) are also possible.

(great, will reply in #499)

To reply to:

Maybe further, in assert_snapshot!(expr, string_lit), we can always treat it as inline snap if the second argument is string literal because it's not likely assert_snapshot!(name, expr) has a simple string literal as expr (comparing a string lit with string lit).

I would vote to stay away from any ambiguities like this; confusion there would be very expensive...

authors: andylokandy max-sixty

repo: mitsuhiko/insta


text:
BLUETOOTH permission is listed as optional in docs but seems to be required

The Android setup docs under Step 2 - Add permissions to your AndroidManifest.xml: list the BLUETOOTH permission as optional but I am seeing a consistent crash for at least a Samsung SM-925T.  I have not tested enough to determine if this is device specific but either way it should not happen.  It causes our app to go into a crash/restart cycle of sometimes more than 10 times.
02-02 10:09:11.688 13384-13405/? E/BluetoothAdapter: Application does not have bluetooth permission, registering is failed
02-02 10:09:11.688 13384-13405/? E/BluetoothAdapter: java.lang.SecurityException: Need BLUETOOTH permission: Neither user 10348 nor current process has android.permission.BLUETOOTH.
                                                         at android.os.Parcel.readException(Parcel.java:1620)
                                                         at android.os.Parcel.readException(Parcel.java:1573)
                                                         at android.bluetooth.IBluetoothManager$Stub$Proxy.registerStateChangeCallback(IBluetoothManager.java:350)
                                                         at android.bluetooth.BluetoothAdapter.<init>(BluetoothAdapter.java:594)
                                                         at android.bluetooth.BluetoothAdapter.getDefaultAdapter(BluetoothAdapter.java:576)
                                                         at com.mixpanel.android.mpmetrics.SystemInformation.isBluetoothEnabled(SystemInformation.java:141)
                                                         at com.mixpanel.android.mpmetrics.AnalyticsMessages$Worker$AnalyticsMessageHandler.getDefaultEventProperties(AnalyticsMessages.java:559)
                                                         at com.mixpanel.android.mpmetrics.AnalyticsMessages$Worker$AnalyticsMessageHandler.prepareEventObject(AnalyticsMessages.java:573)
                                                         at com.mixpanel.android.mpmetrics.AnalyticsMessages$Worker$AnalyticsMessageHandler.handleMessage(AnalyticsMessages.java:251)
                                                         at android.os.Handler.dispatchMessage(Handler.java:102)
                                                         at android.os.Looper.loop(Looper.java:158)
                                                         at android.os.HandlerThread.run(HandlerThread.java:61)

Hi @agramian ! Thanks for reporting this. What's the SDK Version that you are using? Also, have you modified the library somehow?
Thanks!

Hi @patedit no I have not modified the library at all.  I'm using it exactly as your docs instruct.  Here's the info from my gradle file.
    compileSdkVersion 25
    buildToolsVersion "25.0.2"

    defaultConfig {
        minSdkVersion 16
        targetSdkVersion 25
...
    compile "com.mixpanel.android:mixpanel-android:4.9.3"

1 same problem

What's slightly crazy about this is that there's already this in SystemInformation.isBluetoothEnabled():
        } catch (SecurityException e) {
            // do nothing since we don't have permissions
        }

I can't figure out from the stacktrace how the app would be crashing on a SecurityException from the getDefaultAdapter() call

Hehe, I know. This crash is pretty crazy (hence my interest of including the permission in the AndroidManifest.xml). I suspect it's an OEM implementation that I'd need to dig into.

+1 similar issue:
I get the same error message every time i open the app. But it does not crash at least.
Still.. it looks wierd to see error output in my logcat every time I open the app..

same here on android N samsung device s8
05-04 08:04:41.778 12607-12634/com.sample.airwatchsdk E/BluetoothAdapter: Application does not have bluetooth permission, registering is failed
05-04 08:04:41.781 12607-12634/com.sample.airwatchsdk E/BluetoothAdapter: java.lang.SecurityException: Need BLUETOOTH permission: Neither user 10201 nor current process has android.permission.BLUETOOTH.
at android.os.Parcel.readException(Parcel.java:1693)
at android.os.Parcel.readException(Parcel.java:1646)
at android.bluetooth.IBluetoothManager$Stub$Proxy.registerStateChangeCallback(IBluetoothManager.java:375)
at android.bluetooth.BluetoothAdapter.(BluetoothAdapter.java:640)
at android.bluetooth.BluetoothAdapter.getDefaultAdapter(BluetoothAdapter.java:621)
at com.mixpanel.android.mpmetrics.SystemInformation.isBluetoothEnabled(SystemInformation.java:144)
at com.mixpanel.android.mpmetrics.AnalyticsMessages$Worker$AnalyticsMessageHandler.getDefaultEventProperties(AnalyticsMessages.java:553)
at com.mixpanel.android.mpmetrics.AnalyticsMessages$Worker$AnalyticsMessageHandler.prepareEventObject(AnalyticsMessages.java:567)
at com.mixpanel.android.mpmetrics.AnalyticsMessages$Worker$AnalyticsMessageHandler.handleMessage(AnalyticsMessages.java:247)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:154)
at android.os.HandlerThread.run(HandlerThread.java:61)

I'm having this error too:
07-28 09:24:52.657 12893-13502/br.com.sltalk E/BluetoothAdapter: Application does not have bluetooth permission, registering is failed
07-28 09:24:52.657 12893-13502/br.com.sltalk E/BluetoothAdapter: java.lang.SecurityException: Need BLUETOOTH permission: Neither user 10529 nor current process has android.permission.BLUETOOTH.
at android.os.Parcel.readException(Parcel.java:1620)
at android.os.Parcel.readException(Parcel.java:1573)
at android.bluetooth.IBluetoothManager$Stub$Proxy.registerStateChangeCallback(IBluetoothManager.java:358)
at android.bluetooth.BluetoothAdapter.(BluetoothAdapter.java:598)
at android.bluetooth.BluetoothAdapter.getDefaultAdapter(BluetoothAdapter.java:580)
at com.mixpanel.android.mpmetrics.SystemInformation.isBluetoothEnabled(SystemInformation.java:144)
at com.mixpanel.android.mpmetrics.AnalyticsMessages$Worker$AnalyticsMessageHandler.getDefaultEventProperties(AnalyticsMessages.java:620)
at com.mixpanel.android.mpmetrics.AnalyticsMessages$Worker$AnalyticsMessageHandler.prepareEventObject(AnalyticsMessages.java:634)
at com.mixpanel.android.mpmetrics.AnalyticsMessages$Worker$AnalyticsMessageHandler.handleMessage(AnalyticsMessages.java:294)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:148)
at android.os.HandlerThread.run(HandlerThread.java:61)
                                                             [ 07-28 09:24:52.667 12893:12893 W/         ]
                                                             Unable to open '/system/framework/qcom.fmradio.jar': No such file or directory

Any updates on this issue?

guys, could you kindly tell me what are the devices you are seeing this issue on and their android version? thanks!

@patedit Samsung s7 Edge

same issue on S6 EDGE and S7 EDGE

what about the android version? is it only happening on N? Do you guys see something in common with all the devices? thanks!

Android API 7.0

both devices

Same here, on galaxy s7 edge

It seems like this bug is still not solved yet.
@patedit ,@Mixplanel  can you please resolve the problem ASAP...
We don't really want to have a BLUETOOTH permission in our manifest ..requesting this is really permission so bad for the UX..

I cant even download my app from play store from emulator because emulators does not have bluetooth!!! FIX IT

@dementia2029 Sorry, I am not sure I understand your problem. Whether you can or can't download your app from Google Play is unrelated to this issue.
Guys, hopefully this PR will solve your issue https://github.com/mixpanel/mixpanel-android/pull/499 I'll make a release today!

@patedit we can't download a released app (with mixpanel) from google play to our emulator because mixpanel requires bluetooth, if the device doesn't have bluetooth then the app wont show up in google play.

@adin234 because of this pull request Mixpanel no longer requires bluetooth. If you would like to remove the need from your manifest, make a release, and try installing on an emulator again, you may do so.

@levibostian i just explained to @patedit why this issue and the app not being downloadable from the store and an emulator are related

Thanks for the clarification @adin234 ! And nice profile pic 👨💻

authors: Aidin RockyLin adin234 agramian aliamohsin dementia2029 edenman eladkatz houssemzaier levibostian patedit tHollarek victor5171

repo: mixpanel/mixpanel-android


text:
Compatibility with android 13+

Hi,
Does this code even work in Android 13?
    } else {
      directory = await getExternalStorageDirectory();
    }

in Logexporter. As we know Android 13 won't allow access to external storage

Reference: https://developer.android.com/reference/android/Manifest.permission#WRITE_EXTERNAL_STORAGE

I didn't test it so I don't know if it works for Android 13+. You can check it and let me know then.

I didn't test it so I don't know if it works for Android 13+. You can check it and let me know then.

It is not working in android 13. We can use document directory instead.

authors: Harishwarrior mjablecnik

repo: mjablecnik/MyLogger

license: MIT


text:
JDK 11

Hello!
Since we would like to start using Java 11 sooner or later, I've started fixing some of the cleanups and tests that were failing with JDK 9 (#52). I would appreciate if you could take a look at my changes and tell me if what I'm doing makes sense :)
Thank you!

Tests that are still failing:

ShutdownHookCleanUpTest.cleanUpAfterTriggeringLeak ClassLoader has not been garbage collected
I don't know how I can fix this one, since it fails in the Travis build, but works in my local Eclipse. (See #88)

LdapPoolManagerInitiatorTest.firstShouldLeak
SunGCInitiatorTest.firstShouldLeak
ReplaceDOMNormalizerSerializerAbortExceptionCleanUpTest.triggerLeakWithoutCleanup
JavaUtilLoggingLevelCleanUpTest.triggerLeakWithoutCleanup
These tests fail with the message "ClassLoader has been garbage collected, while test is expected to leak", so I would assume the leaks have been fixed.

BeanELResolverCleanUpTest.cleanUpAfterTriggeringLeak
JavaServerFaces2746CleanUpTest.cleanUpAfterTriggeringLeak
These two tests still fail, because I can't get the cleanup to work with Java 11

repo: mjiderhamn/classloader-leak-prevention


text:
See also:

https://github.com/dotnet/symreader-converter/pull/235
Just in case you find it useful

Ultimately yeah, it would be nice to have Windows PDB to some other format. I just wrote these tools for some work stuff I needed today. So many tools still need Windows PDBs it is quite sad.

authors: KirillOsenkov mjsabby

repo: mjsabby/ExtractPortablePDB


text:
Missing win-32bit packages on pip (pypi) for python 3.9/3.10

Hi,
are the 32bit packages intentionally missing from the PyPI database? (https://pypi.org/project/pyodbc/#files)
As I see the appveyor config in the repository includes the 32bit version pipelines. But the latest 32bit version for windows is for python 3.8.

I didn't realize the later versions even supported 32-bit Windows.  I also didn't know that Windows 10 used to have a 32-bit version.  (Apparently they recently stopped sending it to manufacturers, so I think it is near the end.)
I'll do some research into adding these to the CI builds.  I'll also see if Github suports everything we need now and if we can drop AppVeyor completely.
We'll want to update the setup.py for 3.8:
https://cibuildwheel.readthedocs.io/en/stable/faq/#windows-importerror-dll-load-failed-the-specific-module-could-not-be-found

are the 32bit packages intentionally missing from the PyPI database?

It appears so:
https://github.com/mkleehammer/pyodbc/blob/b7e944acdeaa3233dcac045d059d93fe46a71989/.github/workflows/ubuntu_build.yml#L210-L211
As a workaround, at least for now, a limited assortment of 32-bit Windows wheels are available from
https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyodbc

Covered by #1084

authors: crash5 gordthompson mkleehammer

repo: mkleehammer/pyodbc


text:
cursor fetch long decimal raise conversion error: InvalidOperation

when connecting to teradata and run the below query:
 csr = con.execute('select 12344556778899 as idx;')
 csr.fetchall()

---------------------------------------------------------------------------
InvalidOperation                          Traceback (most recent call last)
<ipython-input-65-332750c4d735> in <module>()
----> 1 csr.fetchall()

InvalidOperation: [<class 'cdecimal.ConversionSyntax'>]

seems pyodbc has difficulty converting long decimal from teradata to python data types.
any solution?
system: redhat 7
python: Python 2.7.11 |Anaconda 4.0.0 (64-bit)| (default, Dec  6 2015, 18:08:32)
IPython 4.1.2
pyodbc: 3.0.10

Well, first I'm surprised it chose decimal instead of int.  I suppose it returns all values greater than some constant as decimal?
Because there are so many driver bugs with the binary decimal struct, pyodbc requests it as text and calls the decimal.Decimal constructor.  This of course leads down a rabbit hole - using text leads to language issues such as "," instead of "." which the decimal class does not deal with.  I replace them.
When you select this number in a teradata client, what does it show?  Is it inserting commas?  Perhaps I need to smarten it up to see if both are present and only convert if the comma is the last item.

Any progress on on how one can get a workaround for this ? I'm getting the same error...

Can you test this with pyodbc 4.0.1 and get a trace?  I'd like to see what the format of the string is.
I'm tempted to attempt reading decimals as binary structures, but some databases don't set them right leading to garbage.  I suppose I could default to doing so and use a flag to turn it off until drivers get fixed.

Actually, the issue is going to be related to Unicode.  Teradata is sending UTF-8, I think, and I'm expecting UTF-16.  This is a place in 4.0 that didn't get updated with correct Unicode handling.  Working on it now.

Is it possible for you to test with the fix-126 branch?  I think I have it fixed but I don't have a system that was failing to test against.

authors: femibyte mkleehammer zhuobao


text:
Fixes #22 - Increase number of loop iterations to fully pull in data

Increase number of loop iterations to fully pull in data from VARCHAR 65000 fields.
The current pyodbc behavior will never pull in more than 1024 + (2048 * 9) = 19456 bytes.

I was pulling data from a Vertica database, and if a field contained more than 9728 characters, it would return nothing. This patch worked for me. Thanks.

authors: JeffAMcGee geota


text:
FloatThead with jQuery custom content scroller

Hi!
We don't know how to make work well together floatThead with jQuery custom content scroller (site: http://manos.malihu.gr/jquery-custom-content-scroller/
GitHub: https://github.com/malihu/malihu-custom-scrollbar-plugin). That plugin always sets content wrapper scrollTop/scrollLeft  to 0. So your plugin can't initiallize properly with that options: scrollTop: 85.
Теперь понятным языком: нужна плавная прокрутка, и нужна прилипающая шапка таблицы. Причем также должна учитываться шапка сайта, которая как раз и составляет 85 пикселей. В обычной ситуации хватило бы вписать в floatThead scrollTop: 85. Но дело в том, что
плагин того товарища делает scrollTop и scrollLeft постоянно равными нулю, а само body всегда равно 100%, что и делает невозможным инициализацию дефолтной прокрутки, на которую рассчитан floatThead (надеюсь, объяснили правильно).

sorry, cant respond in russian, but 2 things:
can you use this: http://mkoryak.github.io/floatThead/examples/perfect-scrollbar/ ?
if not, please create a jsfiddle and I will try to help you get that one working

I can't use a "Perfect Scrollbar" because I need smooth scrolling.
JSFiddle: http://jsfiddle.net/4kojLhah/

Does not look like I can support that lib. it does not set .scrollTop(x) instead it uses css top:x, it also does not trigger and scroll events on the element being scrolled.
My plugin relies on scrollTop and scroll events. Making these 2 things work together would require writing another plugin to bridge the gap :P

How did we solve that:
var OurTable = $('.table');
//Если мы в личном кабинете и у нас есть таблица с данными
if (OurTable .length != 0){
    OurTable .floatThead({
        useAbsolutePositioning: false,
        scrollTop: 85
    });
}

    $('body').mCustomScrollbar({
        theme:"minimal-dark",
        callbacks : {
            onScroll: function(){
                ScrollerTop = this.mcs.top;
             },
            whileScrolling: function(){

                if (OurTable .length != 0){
                    if (this.mcs.top <= -220){
                        $('.floatThead-table').css('marginTop', 85);
                    }else{
                        $('.floatThead-table').css('marginTop', 0);
                    }

                    OurTable .floatThead('reflow');
                }

            }
        }

May be now you will create a patch ;)

this does look like code I can put in the main plugin, but I can add a page to the docs about it if you tell me what 220 is?

It is like 85 in JSFiddle. Just in our desidn we don't need initialize that right after nav bar.

glad you found a hack to solve your problem

@mkoryak - did you end up adding this into the plugin? Just curious. I'm having issues with mCustomScrollbar and your plugin playing nicely (particularly when there's a POST-back in my web forms app when the infinite-scroll request is made).

sorry, i think that was a typo, i meant 'cant put into main plugin'.
have you tried trigger reflow on the table whenever weird things happen?

First off - thanks for your time (and plugin).
For some context. Here's an image of the "weird" thing that happens:

Occassionally, (can't figure out the consistent cause) when the POST-back is made, and the page refreshes - the table header is slightly off - with some weird spacing as you can see in the photo. Normally it looks like this:

I can confirm that every time the page refreshes it calls this code:

(I see that console.log each time) You mention using the reflow call. I tried doing that by calling it immediately after I call floatThead() every time, but that didn't seem to fix it. Maybe I'm doing it wrong.
I've read your docs but I'm not positive - how would you suggest I go about triggering reflow/checking when weird things happen?

what you should be doing is that every time more rows are inserted into the table, you should do the reflow: $table.trigger('reflow')
if that doesnt work try doing it within a setTimeout of 0ms

By placing the $table.trigger('reflow') after $table.floatThead(), both called on page load (which happens after rows are added), it seems to resolve the slight layout issue.
It's a partial page reload (from a POST-back) that causes more rows to be loaded up into the table.
But there's another odd issue I've bumped into: If the user is scrolling within the table while the page is reloading, the header simply disappears.

The even more odd thing is that simply by scrolling the browser page (up or down) in the slightest, it comes back.
I tried hooking into the "floatThead" event to check when this happens, in order to recall-reflow (in above image), but it doesn't help since it doesn't hit this event when the header disappears. Any thoughts on this one?
Much appreciated.

This one I have no idea about, if you can put it on jsfiddle I can probably find a fix, if you do, please create a new issue.

Just wanted to follow up.
The odd behavior I described was being caused by the fact that my table was inside of a smaller, overflowing div, yet I was not initializing floatThead with
scrollContainer: function($table) { return $table.closest('.wrapper'); } -- or similar for your needs -- as I should have been.

authors: michaelmmcrae1 mkoryak sentbyravens

repo: mkoryak/floatThead

license: mit


text:
Upgrade to RN Paper v5

At time of writing RN Paper v5 is 4 month old and should become more stable. To stay future-proof, consider an upgrade some time.

https://www.npmjs.com/package/react-native-paper?activeTab=versions
https://callstack.github.io/react-native-paper/docs/guides/migration-guide-to-5.0/

Fixed in https://github.com/mkraenz/daily-questions/issues/18.
Once that's done, should create a follow up to migrate to Material Design 3 - which seems to be the recommended way of using RN paper 5

Fixed while moving to Monorepo https://github.com/mkraenz/teatime

repo: mkraenz/daily-questions


text:
Add gpu extra with dask-cuda and bump minimum Python version to 3.9

Separately resolving dask-cuda in our components leads to dependency conflicts. Since we put a lot of focus on ML components, I think it makes sense to add it as an extra.
It only supports Python 3.9+ though, so we need to drop Python 3.8, which will be end of life later this year.

Good catch. We have Python versions in a lot of places :sweat_smile:


text:
how to build mlc-llm-cli on Linux

I want to run vicuna-7b on nv gpu based on mlc-llm.  I followed the intruction and have some changes:

Install relax.
git clone https://github.com/mlc-ai/relax.git --recursive
cd relax
mkdir build
cp cmake/config.cmake build

in build/config.cmake, set USE_CUDA, USE_CUDNN, USE_CUBLAS and USE_LLVM as ON
cmake ..
make -j
export TVM_HOME=/path/to/relax
export PYTHONPATH=$PYTHONPATH:$TVM_HOME/python

Get Model Weight
I just use the following:
git lfs install
git clone https://huggingface.co/mlc-ai/demo-vicuna-v1-7b-int3 dist/vicuna-v1-7b
mkdir -p dist/models
ln -s path/to/vicuna-v1-7b dist/models/vicuna-v1-7b

But there is no config.json and some other necessary files in vicuna-v1-7b path! They are transformed by transform_params. We need vicuna-v1-7b Pytorch format.
$ tree dist/
dist/
└── models
    └── vicuna-v1-7b
        ├── float16
        │   ├── ndarray-cache.json
        │   ├── tokenizer.model
        │   ├── params_shard_0.bin
        │   ├── params_shard_100.bin
        │   ├── params_shard_101.bin
        │   ├── params_shard_102.bin
        │   ├── params_shard_103.bin
        │   ├── params_shard_104.bin
        │   ├── params_shard_105.bin
...

I found that there is a vicuna-v1-7b with huggingface format whose dtype is float32 and vocab_size is 32001. So I downloaded this one.

build model to librarygit clone https://github.com/mlc-ai/mlc-llm.git --recursive
cd mlc-llm
# change vocab_size=32001 in llma.py
python3 build.py --model vicuna-v1-7b --dtype float16 --target cuda --max-seq-len 768 --artifact-path ../dist/

The output:

...
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 14/14 [00:10<00:00,  1.39it/s]
Total param size: 3.9229860305786133 GB
Start storing to cache ../dist/vicuna-v1-7b/float16/params
[0745/0745] saving param_744
All finished, 132 total shards committed, record saved to ../dist/vicuna-v1-7b/float16/params/ndarray-cache.json
Save a cached module to ../dist/vicuna-v1-7b/float16/mod_cache_before_build_float16.pkl.
20 static functions: [I.GlobalVar("rotary_embedding1"), I.GlobalVar("fused_decode1_fused_matmul5_multiply"), I.GlobalVar("decode4"), I.GlobalVar("slice1"), I.GlobalVar("squeeze"), I.GlobalVar("fused_decode_matmul3"), I.GlobalVar("fused_decode_fused_matmul3_add"), I.GlobalVar("fused_decode2_fused_matmul6_add"), I.GlobalVar("decode6"), I.GlobalVar("fused_transpose4_reshape4"), I.GlobalVar("transpose2"), I.GlobalVar("rms_norm1"), I.GlobalVar("fused_decode1_fused_matmul5_silu"), I.GlobalVar("reshape1"), I.GlobalVar("decode5"), I.GlobalVar("reshape"), I.GlobalVar("fused_reshape2_squeeze"), I.GlobalVar("reshape2"), I.GlobalVar("take_decode"), I.GlobalVar("fused_decode3_fused_matmul7_cast2")]
26 dynamic functions: [I.GlobalVar("fused_NT_matmul1_add1"), I.GlobalVar("extend_te"), I.GlobalVar("full"), I.GlobalVar("reshape3"), I.GlobalVar("reshape5"), I.GlobalVar("rotary_embedding"), I.GlobalVar("take_decode1"), I.GlobalVar("fused_NT_matmul_divide_maximum_minimum_cast"), I.GlobalVar("NT_matmul1"), I.GlobalVar("fused_softmax1_cast4"), I.GlobalVar("fused_NT_matmul3_silu1"), I.GlobalVar("fused_NT_matmul2_divide1_maximum1_minimum1_cast3"), I.GlobalVar("matmul8"), I.GlobalVar("transpose5"), I.GlobalVar("fused_softmax_cast1"), I.GlobalVar("fused_min_max_triu_te_broadcast_to"), I.GlobalVar("reshape7"), I.GlobalVar("reshape8"), I.GlobalVar("slice"), I.GlobalVar("fused_NT_matmul3_multiply1"), I.GlobalVar("squeeze1"), I.GlobalVar("matmul4"), I.GlobalVar("transpose3"), I.GlobalVar("rms_norm"), I.GlobalVar("fused_NT_matmul4_add1"), I.GlobalVar("reshape6")]
Dump static shape TIR to ../dist/vicuna-v1-7b/float16/mod_tir_static.py
Dump dynamic shape TIR to ../dist/vicuna-v1-7b/float16/mod_tir_dynamic.py
- Dispatch to pre-scheduled op: fused_decode1_fused_matmul5_multiply
- Dispatch to pre-scheduled op: decode4
- Dispatch to pre-scheduled op: fused_NT_matmul1_add1
- Dispatch to pre-scheduled op: decode5
- Dispatch to pre-scheduled op: NT_matmul1
- Dispatch to pre-scheduled op: fused_NT_matmul_divide_maximum_minimum_cast
- Dispatch to pre-scheduled op: fused_softmax1_cast4
- Dispatch to pre-scheduled op: fused_NT_matmul3_silu1
- Dispatch to pre-scheduled op: fused_NT_matmul2_divide1_maximum1_minimum1_cast3
- Dispatch to pre-scheduled op: fused_decode_matmul3
- Dispatch to pre-scheduled op: matmul8
- Dispatch to pre-scheduled op: fused_softmax_cast1
- Dispatch to pre-scheduled op: fused_min_max_triu_te_broadcast_to
- Dispatch to pre-scheduled op: fused_decode_fused_matmul3_add
- Dispatch to pre-scheduled op: fused_decode2_fused_matmul6_add
- Dispatch to pre-scheduled op: decode6
- Dispatch to pre-scheduled op: fused_decode1_fused_matmul5_silu
- Dispatch to pre-scheduled op: fused_NT_matmul3_multiply1
- Dispatch to pre-scheduled op: matmul4
- Dispatch to pre-scheduled op: rms_norm
- Dispatch to pre-scheduled op: fused_NT_matmul4_add1
Finish exporting to ../dist/vicuna-v1-7b/float16/vicuna-v1-7b_cuda_float16.so

Prepare lib and params
There is instruction for ios as following. After Step 3, params and vicuna-v1-7b_cuda_float16.so are under path ../dist/vicuna-v1-7b/float16/params
cd ios
./prepare_libs.sh
./prepare_params.sh

Build mlc-llm-cli
Is there any introduction on how to build mlc-llm-cli? I use the following but got error. @MasterJH5574 Could you please give some advice? Thanks a lot^_^
mkdir build && cd build
cmake .. && make
[  0%] Building CXX object CMakeFiles/mlc_llm_objs.dir/cpp/llm_chat.cc.o
[  0%] Built target mlc_llm_objs
[  0%] Building CXX object CMakeFiles/mlc_cli_objs.dir/cpp/cli_main.cc.o
[  0%] Built target mlc_cli_objs
[  0%] Generating release/libtokenizers_cpp.a
No such file or directory
make[2]: *** [tokenizers/CMakeFiles/tokenizers.dir/build.make:71: tokenizers/release/libtokenizers_cpp.a] Error 1
make[1]: *** [CMakeFiles/Makefile2:674: tokenizers/CMakeFiles/tokenizers.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

Run make -n to get debug info:
make -n
...
/z/env_init/env_tvm/lib/python3.8/site-packages/cmake/data/bin/cmake -E cmake_echo_color --switch= --green --progress-dir=/z/Dev/mlc-llm/build/CMakeFiles --progress-num=100 "Building CXX object tvm/CMakeFiles/tvm_runtime_objs.dir/src/runtime/contrib/sort/sort.cc.o"
cd /z/Dev/mlc-llm/build/tvm && /usr/bin/c++ -DDMLC_USE_FOPEN64=0 -DDMLC_USE_LOGGING_LIBRARY="<tvm/runtime/logging.h>" -DNDEBUG -DNDEBUG=1 -DTVM_INDEX_DEFAULT_I64=1 -DTVM_THREADPOOL_USE_OPENMP=0 -DTVM_USE_LIBBACKTRACE=0 -DUSE_FALLBACK_STL_MAP=0 -I/z/Dev/relax/include -I/z/Dev/relax/3rdparty/libcrc/include -isystem /z/Dev/relax/3rdparty/dlpack/include -isystem /z/Dev/relax/3rdparty/dmlc-core/include -isystem /z/Dev/relax/3rdparty/rang/include -isystem /z/Dev/relax/3rdparty/compiler-rt -isystem /z/Dev/relax/3rdparty/picojson -isystem /usr/local/cuda/include -std=c++17 -faligned-new -O2 -Wall -fPIC -std=c++17  -O2 -g -DNDEBUG -fPIC -ffile-prefix-map=..=/z/Dev/relax -MD -MT tvm/CMakeFiles/tvm_runtime_objs.dir/src/runtime/contrib/sort/sort.cc.o -MF CMakeFiles/tvm_runtime_objs.dir/src/runtime/contrib/sort/sort.cc.o.d -o CMakeFiles/tvm_runtime_objs.dir/src/runtime/contrib/sort/sort.cc.o -c /z/Dev/relax/src/runtime/contrib/sort/sort.cc
/z/env_init/env_tvm/lib/python3.8/site-packages/cmake/data/bin/cmake -E cmake_echo_color --switch= --progress-dir=/z/Dev/mlc-llm/build/CMakeFiles --progress-num=95,96,97,98,99,100 "Built target tvm_runtime_objs"
make -s -f tvm/CMakeFiles/tvm_runtime.dir/build.make tvm/CMakeFiles/tvm_runtime.dir/depend
cd /z/Dev/mlc-llm/build && /z/env_init/env_tvm/lib/python3.8/site-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /z/Dev/mlc-llm /z/Dev/relax /z/Dev/mlc-llm/build /z/Dev/mlc-llm/build/tvm /z/Dev/mlc-llm/build/tvm/CMakeFiles/tvm_runtime.dir/DependInfo.cmake --color=
make -s -f tvm/CMakeFiles/tvm_runtime.dir/build.make tvm/CMakeFiles/tvm_runtime.dir/build
make[2]: *** No rule to make target 'tvm/CMakeFiles/tvm_runtime_objs.dir/src/runtime/builtin_fp16.cc.o', needed by 'tvm/libtvm_runtime.so'.  Stop.
make[1]: *** [CMakeFiles/Makefile2:438: tvm/CMakeFiles/tvm_runtime.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

mlc-llm commit id: 909f267
relax commit id: 227cacd

There are a couple of things here

Their model uses a vulkan target, so you can’t use their HF weights.
I believe the CLI app targets Vulkan as well, so you have to test with the chat python file in the tests folder.
But there is no TVM profiling info for thr CUDA path, so your built model is gonna be much slower than their vulkan release.

git submodules update --init --recursive

I am sure I have got all submodules before building. There is no file changed after running git submodule update --init --recursive under mlc-llm project.
(env_tvm) root@12800db2b9db:/z/Dev/mlc-llm/3rdparty# ls
argparse  sentencepiece-js  tokenizers-cpp

The compilation of tokenizer-cpp depends on rust, can you confirm you have installed rust?

The compilation of tokenizer-cpp depends on rust, can you confirm you have installed rust?

Yes, compilation of tokenizer-cpp depends on Rust. Rust is not installed on my device. I will install rust firstly. Thanks for your kind help.

Rust is installed now.
# which rustc
/usr/bin/rustc
# rustc --version
rustc 1.65.0
# cd build; cmake .. -DCMAKE_VERBOSE_MAKEFILE=ON; make

The same error occured.
I tried to compile tokenizers-cpp alone. Also, error happened when generating release/libtokenizers_cpp.a. How you compile the tokenizers-cpp? @yzh119
# cd 3rdparty/tokenizers-cpp/
# mkdir build; cd build
# cmake .. .. -DCMAKE_VERBOSE_MAKEFILE=ON
# make
/z/env_init/env_tvm/lib/python3.8/site-packages/cmake/data/bin/cmake -S/z/Dev/mlc-llm/3rdparty/tokenizers-cpp -B/z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build --check-build-system CMakeFiles/Makefile.cmake 0
/z/env_init/env_tvm/lib/python3.8/site-packages/cmake/data/bin/cmake -E cmake_progress_start /z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build/CMakeFiles /z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build//CMakeFiles/progress.marks
make  -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build'
make  -f CMakeFiles/tokenizers.dir/build.make CMakeFiles/tokenizers.dir/depend
make[2]: Entering directory '/z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build'
cd /z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build && /z/env_init/env_tvm/lib/python3.8/site-packages/cmake/data/bin/cmake -E cmake_depends "Unix Makefiles" /z/Dev/mlc-llm/3rdparty/tokenizers-cpp /z/Dev/mlc-llm/3rdparty/tokenizers-cpp /z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build /z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build /z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build/CMakeFiles/tokenizers.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build'
make  -f CMakeFiles/tokenizers.dir/build.make CMakeFiles/tokenizers.dir/build
make[2]: Entering directory '/z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build'
[100%] Generating release/libtokenizers_cpp.a
cd /z/Dev/mlc-llm/3rdparty/tokenizers-cpp && /z/env_init/env_tvm/lib/python3.8/site-packages/cmake/data/bin/cmake -E env CARGO_TARGET_DIR=/z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build RUSTFLAGS="" cargo build --release
No such file or directory
make[2]: *** [CMakeFiles/tokenizers.dir/build.make:74: release/libtokenizers_cpp.a] Error 1
make[2]: Leaving directory '/z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build'
make[1]: *** [CMakeFiles/Makefile2:86: CMakeFiles/tokenizers.dir/all] Error 2
make[1]: Leaving directory '/z/Dev/mlc-llm/3rdparty/tokenizers-cpp/build'
make: *** [Makefile:94: all] Error 2

@zhaoyang-star We rely on cargo: rust's package manager.

I finally build mlc_chat_cli and libmlc_llm.so after installing Rust dev environment.
It came error that libtvm_runtime.so has not compiled with cuda runtime when I ran mlc_chat_cli. I am sure USE_CUDA, USE_CUDNN, USE_CUBLAS and USE_LLVM are ON when compiling TVM. /z/Dev/relax/ is TVM_HOME env var. The unittest test_cudnn.py also passed.
@yzh119 Could you please have a look at it? Is there something I still missing? Thanks a lot.
(env_tvm) root@12800db2b9db:/z/Dev/mlc-llm/build# ls
CMakeCache.txt  CPackConfig.cmake        Makefile             cmake_install.cmake  libmlc_llm.so  sentencepiece  tvm
CMakeFiles      CPackSourceConfig.cmake  TVMBuildOptions.txt  libmlc_llm.a         mlc_chat_cli   tokenizers
(env_tvm) root@12800db2b9db:/z/Dev/mlc-llm/build# ./mlc_chat_cli --device-name=cuda --artifact-path=/z/Dev/dist/
Use lib /z/Dev/dist/vicuna-v1-7b/float16/vicuna-v1-7b_cuda_float16.so
[00:00:37] /z/Dev/relax/src/runtime/library_module.cc:126: Binary was created using {cuda} but a loader of that name is not registered. Available loaders are VMExecutable, relax.Executable, metadata, const_loader, metadata_module. Perhaps you need to recompile with this runtime enabled.
Stack trace:
  [bt] (0) /z/Dev/mlc-llm/build/tvm/libtvm_runtime.so(tvm::runtime::Backtrace[abi:cxx11]()+0x2c) [0x7fa7cbce2efc]
  [bt] (1) ./mlc_chat_cli(tvm::runtime::detail::LogFatal::Entry::Finalize()+0x45) [0x55d806d97d61]
  [bt] (2) /z/Dev/mlc-llm/build/tvm/libtvm_runtime.so(tvm::runtime::LoadModuleFromBinary(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, dmlc::Stream*)+0x3d3) [0x7fa7cbce0013]
  [bt] (3) /z/Dev/mlc-llm/build/tvm/libtvm_runtime.so(tvm::runtime::ProcessModuleBlob(char const*, tvm::runtime::ObjectPtr<tvm::runtime::Library>, std::function<tvm::runtime::PackedFunc (int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)>, tvm::runtime::Module*, tvm::runtime::ModuleNode**)+0x590) [0x7fa7cbce06a0]
  [bt] (4) /z/Dev/mlc-llm/build/tvm/libtvm_runtime.so(tvm::runtime::CreateModuleFromLibrary(tvm::runtime::ObjectPtr<tvm::runtime::Library>, std::function<tvm::runtime::PackedFunc (int (*)(TVMValue*, int*, int, TVMValue*, int*, void*), tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)>)+0x221) [0x7fa7cbce1601]
  [bt] (5) /z/Dev/mlc-llm/build/tvm/libtvm_runtime.so(+0xcd71f) [0x7fa7cbccd71f]
  [bt] (6) /z/Dev/mlc-llm/build/tvm/libtvm_runtime.so(tvm::runtime::Module::LoadFromFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x20e) [0x7fa7cbcec26e]
  [bt] (7) ./mlc_chat_cli(+0x845c) [0x55d806d9945c]
  [bt] (8) /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7fa7cba32083]

You need to set USE_CUDA=ON when compiling mlc_llm

Thank @tqchen for your kind help.
Env:

NV T4
Tune off

(env_tvm) root@12800db2b9db:/z/Dev/mlc-llm/build# ./mlc_chat_cli --device-name=cuda --artifact-path=/z/Dev/dist/ --evaluate
Use lib /z/Dev/dist/vicuna-v1-7b/float16/vicuna-v1-7b_cuda_float16.so
Initializing the chat module...
Finish loading
You can use the following special commands:
  /help    print the special commands
  /exit    quit the cli
  /stats   print out the latest stats (token/sec)
  /reset   restart a fresh chat

[18:41:39] /z/Dev/mlc-llm/cpp/llm_chat.cc:749: logits[:10] =[-7.34375, -6.17969, 5.78125, -1.62012, -3.20312, -2.6543, -0.955566, -4.88672, -4.14844, -1.96777]
[18:41:39] /z/Dev/mlc-llm/cpp/llm_chat.cc:754: encoding-time=527.079ms, decoding-time=36.8855ms.
(env_tvm) root@12800db2b9db:/z/Dev/mlc-llm/build# ./mlc_chat_cli --device-name=cuda --artifact-path=/z/Dev/dist/
Use lib /z/Dev/dist/vicuna-v1-7b/float16/vicuna-v1-7b_cuda_float16.so
Initializing the chat module...
Finish loading
You can use the following special commands:
  /help    print the special commands
  /exit    quit the cli
  /stats   print out the latest stats (token/sec)
  /reset   restart a fresh chat

USER: Who is Lionel Messi?
ASSISTANT: Lionel Messi is a professional soccer player who was born on June 24, 1987, in Rosario, Argentina. He is widely considered to be one of the greatest soccer players of all time. Messi grew up in a family of soccer players and began playing the sport at a young age. He eventually joined the youth academy of Spanish club Barcelona, where he made his professional debut at the age of 17. Since then, Messi has established himself as one of the most talented and skilled players in the world, winning numerous accolades and helping Barcelona to numerous championships. He is known for his exceptional speed, agility, and ball control, as well as his ability to score goals and create opportunities for his teammates. Off the field, Messi is known for his charitable efforts and his commitment to promoting the sport of soccer in his home country of Argentina.
USER: /stats
encode: 60.9 tok/s, decode: 21.1 tok/s

OMG  it takes a lot to build the project ok, so why there is no readme???

：（

@zhaoyang-star  hello, when I used the model you mentioned to build,  I got the problem below, did you miss it ?
Traceback (most recent call last):
File "D:\TVM-relax\mlc-llm\build.py", line 367, in
main()
File "D:\TVM-relax\mlc-llm\build.py", line 338, in main
mod, params = llama.get_model(ARGS, config)
File "D:\TVM-relax\mlc-llm\mlc_llm\relax_model\llama.py", line 695, in get_model
hf_model = AutoModelForCausalLM.from_pretrained(model_path)
File "D:\Anaconda3\lib\site-packages\transformers\models\auto\auto_factory.py", line 434, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
File "D:\Anaconda3\lib\site-packages\transformers\models\auto\configuration_auto.py", line 796, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "D:\Anaconda3\lib\site-packages\transformers\models\auto\configuration_auto.py", line 503, in getitem
raise KeyError(key)
KeyError: 'llama'

You need to set USE_CUDA=ON when compiling mlc_llm

What if I don't have a cuda device on my computer? Thanks.

Please check out this page for building mlc_chat_cli: https://mlc.ai/mlc-llm/docs/tutorials/runtime/cpp.html

Please check out this page for building mlc_chat_cli: https://mlc.ai/mlc-llm/docs/tutorials/runtime/cpp.html

Page not found

The cli is now deprecated in the new version, checkout https://llm.mlc.ai/docs/deploy/cli.html for latest instruction

authors: AlphaAtlas Kuchiriel Poordeveloper junrushao njuhang sgSillage sleepwalker2017 tqchen yzh119 zhaoyang-star

repo: mlc-ai/mlc-llm


text:
Downgrade requests from newest version

https://github.com/docker/docker-py/issues/3256
newly released requests==2.32.0 (2 days ago) breaks docker==7.0.0. Downgrading requests until docker make a fix

@VukW Let's keep a track of this as an issue as well please?

You're right, created the issue for future: https://github.com/mlcommons/GaNDLF/issues/872

repo: mlcommons/GaNDLF


text:
TypeError: mliebelt_pgn_viewer_lib_pgnv__WEBPACK_IMPORTED_MODULE_4_.pgnView is not a function

Hi,
I am trying to use your package in Ionic/Angular project to read and parse chess PGN files but I cannot get it working.
`import * as PGNV from '@mliebelt/pgn-viewer/lib/pgnv';
PGNV.pgnView('board',{ pgn: '1. e4 e5 2. Nf3 Nc6 3. Bb5', pieceStyle: 'merida' });`
throws a following error:
core.js:4197 ERROR Error: Uncaught (in promise): TypeError: mliebelt_pgn_viewer_lib_pgnv__WEBPACK_IMPORTED_MODULE_4_.pgnView is not a function
TypeError: mliebelt_pgn_viewer_lib_pgnv__WEBPACK_IMPORTED_MODULE_4_.pgnView is not a function
at new ReadpgnPage (readpgn.page.ts:127)
at NodeInjectorFactory.ReadpgnPage_Factory [as factory] (ɵfac.js? [sm]:1)
at getNodeInjectable (core.js:4029)
at instantiateRootComponent (core.js:7826)
at createRootComponent (core.js:18375)
at ComponentFactory$1.create (core.js:22191)
at ViewContainerRef.createComponent (core.js:10108)
at IonRouterOutlet.activateWith (ionic-angular.js:2924)
at ActivateRoutes.activateRoutes (router.js:2114)
at router.js:2065
at resolvePromise (zone-evergreen.js:798)
at resolvePromise (zone-evergreen.js:750)
at zone-evergreen.js:860
at ZoneDelegate.invokeTask (zone-evergreen.js:399)
at Object.onInvokeTask (core.js:27425)
at ZoneDelegate.invokeTask (zone-evergreen.js:398)
at Zone.runTask (zone-evergreen.js:167)
at drainMicroTaskQueue (zone-evergreen.js:569)
@mliebelt I would appreciate your assistance.
Many thanks!

Solved by:

removing `import * as PGNV from '@mliebelt/pgn-viewer/lib/pgnv'; from TS module

copying pgnv.js to src/assets

adding "src/assets/js/pgnv.js" to "scripts" array in angular.json

Thank you for reporting this. I may not be able to improve the situation for you. However, the pgn-parser is a package in its own, so it should be possible to use it without the viewer. This should make it much easier to integrate.
I just looked it up, the README.md gives a complete example how to have the parser integrated in pure Javascript. That may be as well not possible for you, but the problem will be different. By having the parser only, you don't need any additional assets, so the package will be much smaller. Depending on what you want to do, this may be more appropriate.
But perhaps I misunderstood your context. The last line PGNV.pgnView('board',{ pgn: '1. e4 e5 2. Nf3 Nc6 3. Bb5', pieceStyle: 'merida' }); seems to denote that you want to visualize the chess game in an HTML page.
Just for my understanding: I though Angular would be able to use new versions of Javascript, so the import command should be available?

@mliebelt , many thanks for your reply. I wasn't able to implement it by using import but mentioned solution works.
Btw, do you have any suggestion on how to use parser.parse('[White "Me"] [Black "Magnus"] 1. f4 e5 2. g4 Qh4#', {startRule: "games"}) and return array of PGN strings without tags. Something like this:
{
"games": [ {
"tags": {
"White": "Me",
"Black": "Magnus"
},
"moves": {
"turn": "w",
"moveNumber": 1,
...
},
"pgn": "1. f4 e5 2. g4 Qh4#"
}
}
Many thanks for your support!

I just created a small test case, with your input and the following result (I leave out some of the entries in JSON).
[
  {
    "tags": {
      "White": "Me",
      "Black": "Magnus"
    },
    "moves": [
      {
        "turn": "w",
        "moveNumber": 1,
        "notation": {...},
       ...
      },
      {
        "turn": "b",
        "moveNumber": null,
        "notation": {... },
       ...
      },
      {
        "turn": "w",
        "moveNumber": 2,
        "notation": {...},
        ...
      },
      {
        "turn": "b",
        "moveNumber": null,
        "notation": {...},
        ...
      }
    ]
  }
]

So what are differences:

You get immediately an array of games (here with only one game)
Moves is an array of move objects.
There is no resulting pgn

The following code computes the pgn (without move numbers), just to show how to do it.
let moves = res[0].moves
let pgn = ''
for (move of moves) {
   pgn += move.notation.notation
   pgn += ' '
}

I hope this clarifies the topic.

authors: mliebelt olsbg

repo: mliebelt/pgn-parser


text:
fixed to work

The lack of a DMA list to support data islands caused buffer overruns and memory corruption.
Several other problems were fixed.
I have confirmed that it works for now.

Then I'll merge this code.

repo: mlorenzati/PicoDVI


text:
Ctrl + D (to exit) sends EOF char to ChatGPT

I tried using Ctrl + D to exit the REPL, and was surprised to see a description of the EOF character instead:
9>
EOF stands for "end of file." It is a special character or sequence of characters that signifies the end of a file, input stream, or output stream. In programming, EOF is often used to indicate the end of a file when reading it, or the end of data when writing to a file. When an EOF character is encountered, the program may stop reading or writing, or it may take some other action, depending on how it is designed to handle EOF.

The REPL should exit when it receives EOF.

ctrl-d is currently used in read mode (where you can paste stuff and write multiline) to prompt chatgpt. I agree that it could be confusing, but I think it is not a big issue especially since you can close with ctrl-c or using the command exit. I am closing therefore the issues

authors: kylrth mmabrouk

repo: mmabrouk/chatgpt-wrapper


text:
Add art assets to AR view

What is the best way to test this for review? I have AR Markers printed and from the AR page on the list of test pages, I can see that there are flowers for the AR Markers, but I don't see the bee. Is there something I need to do? I was using the vercel preview on my phone. @floogulinc @helloworld12321

@kklamberty for the bees you will have to assign one to yourself in Firebase.

Right, should have mentioned, sorry :relaxed: the teacher side doesn't set bees yet, so you'll need to use the firebase console, like so:

(Jinx! :stuck_out_tongue:)

Merging!

Steps to get a bee on your phone:

On your phone, go to session-test and click "anon sign in". Remember the UserID you get. (You can also click "Join Session", although it's not strictly necessary.)
On a desktop, go to the Firebase console at https://console.firebase.google.com, and navigate to Buzz About's database.
In the database, go to /sessions/demo-session/rounds/demo-round/students, and add a new document with the following contents:

Document ID: Your UserID. (Sorry, it's not a lot of fun to type :upside_down_face:)
Field: beeSpecies

Type: string
Value: apis_mellifera (or another bee species ID)

Back on your phone, on the session-test page, you should see "current bee: honey bee"
On your phone, navigate to ar. You should see your bee!
…
Profit!

authors: floogulinc helloworld12321 kklamberty

repo: mn-pollinators/buzz-about


text:
Write optical density data in SNIRF format

Fixes https://github.com/mne-tools/mne-nirs/issues/456
Write optical density data in SNIRF format.
Example usage:
# %%
import os
import mne
import snirf

from mne.io import read_raw_nirx, read_raw_snirf
from mne_nirs.io import write_raw_snirf
from mne.preprocessing.nirs import optical_density
from numpy.testing import assert_allclose

# %%
# Import raw NIRS data from vendor
# --------------------------------
#
# First we import some example data recorded with a NIRX device.

fnirs_data_folder = mne.datasets.fnirs_motor.data_path()
fnirs_raw_dir = os.path.join(fnirs_data_folder, 'Participant-1')
raw_intensity = read_raw_nirx(fnirs_raw_dir).load_data()

# %%
# Convert to optical density

raw_od = optical_density(raw_intensity)

# %%
# Write data as SNIRF
# -------------------
#
# Now we can write this data back to disk in the SNIRF format.

write_raw_snirf(raw_od, 'test_raw.snirf')

# %%
# Read back SNIRF file
# --------------------
#
# Next we can read back the snirf file.

snirf_intensity = read_raw_snirf('test_raw.snirf')

# %%
# Compare files
# -------------
#
# Finally we can compare the data of the original to the SNIRF format and
# ensure that the values are the same.

assert_allclose(raw_od.get_data(), snirf_intensity.get_data())

snirf_intensity.plot(n_channels=30, duration=300, show_scrollbars=False)

# %%
# Validate SNIRF File
# -------------------
#
# To validate that a file complies with the SNIRF standard you should use the
# official SNIRF validator from the Boston University Neurophotonics Center
# called ``snirf``. Detailed instructions for this program can be found at
# https://github.com/BUNPC/pysnirf2. Below we demonstrate that the files created
# by MNE-NIRS are compliant with the specification.

result = snirf.validateSnirf('test_raw.snirf')
assert result.is_valid()
result.display()

Contributed with ❤ by AE Studio

Great PR @florin-pop. I made some suggestions that I think will simplify the code and slightly boost test coverage. Could you please check if they actually work? And add a changelog entry then ping us for a review/merge. Thanks!

@larsoner I noticed that after you merged https://github.com/mne-tools/mne-python/pull/11665 this line started failing in CI https://github.com/mne-tools/mne-nirs/blob/975b44a1888e9b8173df7a2f718035e409d79b06/mne_nirs/statistics/tests/test_glm_type.py#L114
Should we replace it with:
with pytest.raises(ValueError, match='not present in the info'):
?

I was thinking of doing something like:
    if check_version(mne.__version__, '1.4.0'):
        expected_pick_failure = ValueError
    else:
        expected_pick_failure = Warning
    with pytest.raises(expected_pick_failure, match='could not be picked'):
        assert len(res.copy().pick(picks=["S1_D1 hbr", "S1_D1 XXX"])) == 1

but I'm not sure check_version works as expected
>>> import mne
>>> from mne.utils import check_version
>>> print(mne.__version__)
1.4.0.dev136+g263114e32
>>> print(check_version(mne.__version__, '1.4.0', return_version=True))
(False, None)

I would use mne.fixes._compare_version since it behaves as you'd expect/need here, and the pattern I usually use is:
if _compare_version(mne.__version__, '>=', '1.4'):
    ctx = pytest.raises(ValueError, match='...')
else:
    ctx = pytest.warns(RuntimeWarning, match='...')
with ctx:
    ...

Tests are green. Thanks @florin-pop @larsoner

authors: florin-pop larsoner rob-luke

repo: mne-tools/mne-nirs


text:
How to create raw and then epochs from self data array

Hello, for several days I can not solve the problem. I have an array (x,y). Now I create a raw object who contain items of my array.
mne_info = mne.create_info(ch_names = raw['info'].chanels_names,
                               sfreq = raw['info'].frequency,
                               ch_types = raw['info'].ch_types,
                               montage = raw['info'].montage)

raw_mne = mne.io.RawArray(array, mne_info)

np.where(raw_mne == array[0][0]) => true

then i create events array =>
[
[0,0,2]
[4943,0,1]
[9325,0,1]
...
]

And  make epochs object
epochs = mne.Epochs(raw_train,
                                     events,
                                     event_id=labels,
                                     add_eeg_ref=False,
                                     preload = True,
                                     tmin = 0, tmax = 4.499)

There has several questions:
events contains not cutting ranges, and i don't know these ranges. I set up Tmax variable and to get epochs.
does correctly I realized that the ends of each activity are cropped until 4500 miliseconds?
Why after cutting object epochs return old events?
But now I can not find the value of my original array in an array of epochs
data_epochs = epochs.get_data()
np.where(data_epochs == array[0][0]) => False

I can not understand what I'm doing wrong

Can you provide a runnable piece of code that demonstrates this problem.

closing due to no answer

authors: agramfort jaeilepp thisroot

repo: mne-tools/mne-python


text:
DOC ENH: better setup instructions for new contributors

Currently there is some info in the contributing page and some other info in the advanced setup page of the install docs about how to set up your system as an MNE developer.  This info should only be in one place.  Also, the info on using GitHub to set up a pull request should also probably be incorporated in the same place.
@jasmainak suggests we make a CONTRIBUTING.rst file with (more detailed) info on system setup, and use the sphinx include:: directive to paste that file into one of the documentation pages.
@drammock has a bash script for system setup; parts of that could be added.
Questions:

how much content should go in CONTRIBUTING.rst?  (i.e., just the code to set up your dev environment?  basically all of the current contributor's guide? something in between?)

where in the docs should it be included?  in the contributing guidelines page, or in the advanced installation docs?

cc @massich

Apparently, we hit the issue where the documentation on how to interact with the repo itself gets duplicated between repos, outdated, etc..
cc: @emdupre

Adding a .github repository to the org might help, here !
https://github.blog/changelog/2019-02-21-organization-wide-community-health-files/

authors: drammock emdupre massich


text:
Automatic ICA component rejection?

Is there any solution for something like MARA (eeglab) for ICA components automatic rejection?
thank you

This appears to be a question about usage, not a bug report or a new feature request. Please use the MNE mailing list or the gitter channel for usage questions.


text:
Fix warnings

trying to remove all the warnings in tests

2.7 is broken:
https://travis-ci.org/mne-tools/mne-python/jobs/289502124#L2515
Flake:
https://travis-ci.org/mne-tools/mne-python/jobs/289502119#L3091

arfff I had not seen :(

Codecov Report

Merging #4679 into master will decrease coverage by 0.1%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #4679      +/-   ##
==========================================
- Coverage   87.66%   87.55%   -0.11%
==========================================
  Files         351      351
  Lines       66752    66755       +3
  Branches    11573    11571       -2
==========================================
- Hits        58518    58450      -68
- Misses       5281     5323      +42
- Partials     2953     2982      +29

authors: agramfort codecov-io larsoner


text:
Building deb fails when menuGroup is missing

This option is optional, but we're not accessing it in a null-safe way.

Fixed via ba5b3027b69f47d5af0af49419605463ae3ecf5b

repo: mobanisto/pinpit-gradle-plugin


text:
Release v6.0.0

Changes

NPM release
Remove Zepto support in favor of jQuery
Migrate all dependencies to NPM, remove Bower
Migrate JS linting to eslint
Migrate SCSS compilation to libsass
Lint SCSS with sass-lint
Update circleCI config with modern testing steps
Add some GitHub templates for contributing, issues, and PRs

How to Test

Checkout code
Review package.json
Review Changelog
npm install
grunt test or grunt test:browser
Check examples with grunt examples

Todos:

[ ] +1
[x] Add an .npmignore
[ ] Post merge: Publish to NPM

👍 Tests pass, examples work as expected.

authors: marlowpayne mikenikles

repo: mobify/bellows


text:
[FEATURE]: Use searchable select or entity selector for operation selection in groups

Feature Motivation
current dropdown won't work as the list of options would eventually become to long
Solution Description
Use a searchable select with a cap on shown results or use an entity selector.
feature_acceptance_criteria

[ ] searchable select or entity selector added to create/edit group view
[ ] operation selection with them works

Further Information
No response
Related Issues
No response

When changing the select to a searchable select in the group view also provide an empty option to deselect the operation.

implemented in 5dc9dfffa1a1940ec6d088ee5bafec7a3e1795d6.

repo: mobile-directing-system/mds-desktop


text:
Linting Improvements - pre-commit hooks & CICD Strict Test

Soundtrack of this PR: Drexciya - Sighting in the Abyss
Motivation
In an effort to improve and enforce high-quality coding standards. Linting is now part of CICD. Any linting warning will cause a test failure and must be fixed before merge.
Additionally to help keep warnings to a minimum a swiftlint autocorrect pre-commit hook will run on all changed files, and then a swiftlint lint will run on those same files which will highlight any remaining linting issues. Not all warnings are auto-correctable.
In this PR

Add pre-commit config
Update .circleci to enforce strict linting rules
Update README.md with new setup instructions

Future Work

Fix all outstanding linting issues

new PR based off master - https://github.com/mobilecoinofficial/MobileCoin-Swift/pull/147

repo: mobilecoinofficial/MobileCoin-Swift


text:
Request for Multiarch Build Support

Description:
Hello,
I’ve noticed that the current builds of gha-runner are only available for AMD-based CPUs. With the increasing use of ARM-based computing environments, it would be highly beneficial to have support for these architectures as well.
Request:
Could you please consider adding support for ARM-based architectures in addition to the existing AMD builds? Multiarch support would significantly increase the usability and accessibility of the gha-runner project for a broader range of devices.
Additional Information:
Current Architecture Support: AMD (x86_64)
Requested Additional Architectures: ARM (e.g., armv7, arm64)
Thank you for considering this enhancement. If there are any specific requirements or steps I can assist with to facilitate this addition, please let me know.

@csabca83 Thanks for the sample PR. We will consider adding an ARM container to the artifacts.

Thanks @jgreat

Support added in #24
Thanks for your contribution!

repo: mobilecoinofficial/gha-runner


text:
Can't resolve absolute path in mocha tests

I did some test using mocha and testem and found some wierd behavior of the path resolver.
The function
 function (postfix) {
     var result = process.cwd()+postfix;
     return result;
     };

that did return the absolute path with postfix didn't returned it but only postfix. And in normal environment it works as expected.
I created a supplementary repo to show the bug. It raises in 0.10, 0.12 and 4.1 node versions.
Does anybody can explain where is the problem? Because now I can't resolve absolute path to file.

My guess
var path = require('path');
function getPath(p) { return path.resolve((__dirname || process.cwd()), p); }

authors: chilicoder kaiquewdev

repo: mochajs/mocha


text:
How to pass parameter/variable to mocha.run() instance?

Hello
How are we expected to pass a parameter to the Mocha instance we are starting programatically, without using environment variables?
In my case I have a long number of environments specified by a number (30, 31, 32, etc.) and I would like to pass a number to my Mocha test, so that I can read that environments specific config file.
Any best pratices or ways of accomplishing this?
Thanks!

Could you supply more details on the setup of these environments and how they run mocha.

Mocha doesn't have anything that allows you to pass "parameters" into test files, suites, hooks, nor tests.
If you need to do this, I'd suggest pulling in something like dotenv and using Mocha's --require to require a custom test harness file, which will then load the proper config.

How are we expected to pass a parameter to the Mocha instance we are starting programatically, without using environment variables?

Why wouldn't you use environment variables?

If you need to do this, I'd suggest pulling in something like dotenv and using Mocha's --require to require a custom test harness file, which will then load the proper config.

How would I require a specific file when starting Mocha programatically using --require?

@simonsen78 yes. You can specify multiple --require files , but note priority is earliest-first.

No response of hesitation with environment variable...
Assuming:

project named "foo"
'test' contains spec files and such, and (spec filename: "%s.spec.js")
'test/conf' contains your environment-specific config files (conf filename: "%d.conf.json")

"test/setup_env.js"
const path = require('path');
const env = process.env.FOO_CONF;
if (!env) {
  throw new Error('"FOO_CONF" not found in environment');
}
const conffile = env + '.conf.json';
const confpath = path.join(__dirname, 'conf', conffile);
console.log('loading "' + confpath + '"');
require(confpath);

Run
$ FOO_CONF="35" mocha --file 'test/setup_env.js' 'test/**/*.spec.js'

@plroebuck

Why wouldn't you use environment variables?

I'm not running my test on the actual environment. I'm running them from a server and targeting the different environments. I need to use the number of the environment in my code to build target urls for API endpoints and browser URLs.
Also I'm starting the tests in parallel, so I'm not sure if it's "safe" to run a lot of tests in parallel using env variables to configure the tests. My fear (being unsure in the matter) would be that the different Mocha instances would be using the latest value of the env variable? I would feel better/safer passing a number to Mocha and being able to access that in the process, if that makes any sense?

@craigtaub I'm not starting Mocha from the command line, I'm running it programatically using mocha.run(). Not sure you can --require files when doing it like that.

@simonsen78 ah right nope you cant (typical use is to import/setup before invoking mocha) and mocha doesn't have anything that allows you to pass params.
Wonder if something like https://www.npmjs.com/package/continuation-local-storage could help you set and persist variables per test process, not sure.

@plroebuck

Why wouldn't you use environment variables?

I'm not running my test on the actual environment. I'm running them from a server and targeting
the different environments. So it feels "wrong" to use an environment variable in this case.

You realize this is how it's done on pretty much every CI environment (e.g., Travis, AppVeyor, Jenkins), right?. You use env vars to setup your jobs.

I use the number of the environment in my Mocha code to build target URLs for API endpoints
and browser URLs.

A la RAML.

Also I'm starting the tests in parallel, so I'm not sure if it's "safe" to run a lot of tests in parallel
using env variables to configure the tests. My fear (being unsure in the matter) would be that the
different Mocha instances would be using the latest value of the env variable? I would feel
better/safer passing a number to Mocha and being able to access that in the process, if that
makes any sense?

Starting the tests in parallel - you mean like this?:
#!/bin/bash
mkdir output
for i in 1 2 3 4 5
do
  echo "Launching environment job $i"
  setup="test/setup_env.js"
  outfile="output/results-${i}.xunit.xml"
  reporterOptions="--reporter-options output=${outfile}"
  reporter="--reporter xunit"
  FOO_CONF=${i} mocha ${reporter} ${reporterOptions} --file ${setup} 'test/**/*.spec.js' &
done

Every job will have its own environment with the value of FOO_CONF at the time it was launched.

You realize this is how it's done on pretty much every CI environment (e.g., Travis, AppVeyor, Jenkins), right?. You use env vars to setup your jobs.

I realize that. But I'm not a part of the CI process. Not yet anyway. Merely testing stuff from afar. API and front end mostly.

A la RAML.

Not sure how that applies here?

Starting the tests in parallel - you mean like this?:
...
Every job will have its own environment with the value of FOO_CONF at the time it was launched.

Should mention that I have integrated Mocha in a FeathersJS + VueJS setup, and tests are run from the server code. That's what I mean when I say "programatically", like it's shown in the wiki using the API/library and mocha.run. So it's reading some "jobs" from a Mongo database, and then looping over them and starting a process for each job, like say max four at a time.
Just accomplished what I was trying to do with Jest actually. Their library has a runCLI method that takes a JSON object with settings, and there you can specify some globals that are passed on to the test process.
So something like this (sorry, writing this from memory):
const jest = require('jest');

defaults = {}  // forgot what the import code for default object is here

defaults.globals = {
  environmentNumber: 26
}

jest.runCLI(defaults, someOtherObject)
  .then( results => {
    // do something with the result here
  }
}

This is what I wished Mocha has. Because otherwise I'm afraid it's not really usable for my (admittedly very specific) usecase.

I have exactly the same issue with mocha at this moment. I want to run mocha "programmatically" which works fine with hardcoded values. But I want to be able to pass some variables to that process.
Could do this by creating a folder for it in every run and load a dotenv file but that will be an issue when the amount of runs are high (which will happen).
Was thinking of setting a global variable but I'm not sure if mocha.run will be able to access those global variables and this will also force me into running it in a synchronous way.
Anyone has an idea on how to pass some variable (string) without having to create lot of folders, softlinks etc like I would have to when passing the environment variables?

@bramevo you can create a script, e.g., foo.js, and run Mocha with --file foo.js.  foo.js will be loaded first, and you can assign to global there.
// foo.js

someAsyncFunction().then((result) => {
  global.foo = result.bar;
  describe( /* whatever */);
  run();
});

Use mocha --delay --file foo.js <other tests>.  You can also set the delay and file options within a .mocharc.js.

@bramevo you can create a script, e.g., foo.js, and run Mocha with --file foo.js. foo.js will be loaded first, and you can assign to global there.
// foo.js

someAsyncFunction().then((result) => {
  global.foo = result.bar;
  describe( /* whatever */);
  run();
});

Use mocha --delay --file foo.js <other tests>. You can also set the delay and file options within a .mocharc.js.

How would this work when running programmatically? Like described on: https://github.com/mochajs/mocha/wiki/Using-Mocha-programmatically
If somehow I would be able to pass an object through via the Runner class and get that information back in the specs file, that would solve the issue.
When looking into the documentation it seems mocha was never designed to work programmatically.

That is very simple. As run function of Mocha instance, executes the tests in the same process and node context, you could pass the parameters through a global variable.
File handling Mocha programmatic interface:
`
import Mocha from 'mocha' // Mocha -Programmatic rest runner https://github.com/mochajs/mocha/wiki/Using-mocha-programmatically
function runMochaTests(additionalParameters) {
global.additionalParameters = additionalParameters
let mocha = new Mocha(mochaOption)
mocha.run(error => console.error)
}
`
Test file
`
import assert from 'assert'
let testConfig  = global.additionalParameters || {}
console.log(testConfig)
`

That is very simple. As run function of Mocha instance, executes the tests in the same process and Nodejs context, you could pass the parameters through a global variable.
File handling Mocha programmatic interface:
import Mocha from 'mocha' // Mocha -Programmatic rest runner https://github.com/mochajs/mocha/wiki/Using-mocha-programmatically

function runMochaTests(additionalParameters) {
  global.additionalParameters = additionalParameters
  let mocha = new Mocha(mochaOption)
  mocha.run(error => console.error)
}

Test file
import assert from 'assert'

let testConfig  = global.additionalParameters || {}
console.log(testConfig)

This is not elegant, but could work

@myuseringithub I tried the way you have mentioned to pass input to your test files using global variables but it doesn't work if we are using the test parallelly.
 const mocha = new Mocha({
    parallel: true,
    global: ['runnerType'],
    checkLeaks: true
 });

It seems mocha is removing custom added global variables, tried whitelisting it that too didn't work.
Any suggestions anyone?

authors: boneskull bramevo craigtaub laike9m mountainfirefly myuseringithub plroebuck simonsen78


text:
Issue with mocha 'retries'

Hi everyone.
I have a question related to usage of retries inside mocha framework. I am using webdriverio v 5.11 with mocha test runner. Below is a condensed version of my test:
  describe('should set up creating test data', () => {
    this.retries(3) // Retry all tests in this suite up to 3 times
    let respJobList = {}
    it('should succeed on the 3rd try', async () => {
      respJobList = await executeScriptFromPathToGenerateBaseData(
        __dirname,
        '../',
        'data',
        'job_list',
        'jobList.yml'
      )
      expect(respJobList).to.not.equal(undefined)
      recruiterJobListTest = {
        email: respJobList.recruiterJobList.email,
        password: respJobList.recruiterJobList.password
      }
    })
  })

I have described mochaOpts in wdio.conf.js
  mochaOpts: {
    ui: 'bdd',
    require: ['babel-polyfill'],
    compilers: ['js:babel-core/register'],
    timeout: debug ? ONE_HOUR : 300000,
    retries: 3
  }

Actual Results: When I run this test, I get the error TypeError: Cannot read property 'retries' of undefined.
What am I missing here?
Thanks in advance.

arrow functions
You can't use arrow functions with this.timeout / this.retries / this.skip... in Mocha.
Change it to a normal function: describe('should set up creating test data', function() {

thanks a lot. working now.

authors: juergba pk-saxena


text:
fix(browser): failed test icon color

Requirements

Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion.
All new code requires tests to ensure against regressions.

Description of the Change
Due to the wrong variable being referenced in #mocha .test.fail::before, the failed test icon (✖) is currently incorrectly colored as #00d6b2 instead of #cc0000 (light schema) or #ff4444 (dark schema). See for example the screenshot posted in comment https://github.com/mochajs/mocha/pull/4896#issuecomment-1229477806.
Alternate Designs
n/a - this is clearly a bug fix.
Why should this be in core?
n/a - this is clearly a bug fix.
Benefits
Users may now notice test errors more easily, which makes developers, especially the UX/UI ones, happier.
Possible Drawbacks
n/a - this is clearly a bug fix.
Applicable issues
Regressed since https://github.com/mochajs/mocha/pull/4896, mentioned 15 days ago (as a review comment) in https://github.com/mochajs/mocha/pull/4896#discussion_r1008725810, but since there was no response to that, I decided to open this PR.

Coverage remained the same at 94.326% when pulling 6f88dd37c8998caf2bc64137be5c3e331d756f13 on kleisauke:fix-failed-test-icon-color into 3cc9cac7ee040fff1de10af79fb16e7586ea65e8 on mochajs:master.

authors: coveralls kleisauke


text:
Fix package.json to include TypeScript typings

Describe the issue
The package.json files section doesn't include the TypeScript typings so they are not published.
What you are trying to do
Use TypeScript typings.
MockServer version
5.11.0

This is now fixed but not yet released.

repo: mock-server/mockserver-client-node