text:
Enhancement: Provide frozen equivalents of Vec2 and Vec3

What should be added?
Vec2 and other classes were once immutable. They have since been made mutable for convenience and efficiency when changing attributes on objects.
It would be useful to add the following:

A frozen version of these classes that supports hashing in addition to the normal addition and subtraction operations
Convenience methods to make frozen copies of mutable originals

Why
It would be useful for defining immutable constants interoperable with the mutable versions. For example, immutability would prevent people from accidentally changing directions if a constant is passed as an argument to a function (ie game.constants.NORTH, etc):
# will raise errors if any attempt at in-place modification is made with += or similar
NORTH = FrozenVec2(0.0, 1.0)
WEST = FrozenVec(-1.0, 0.0)
NORTHWEST = NORTH + WEST

It would also be useful for defining immutable hashable versions that can be added to sets, dicts, or other containers and re-used later. This would be great for quickly prototyping path finding, but it would also be useful in other cases. For example, although the current version of Vec2 can be converted to tuples for storage, tuples cannot be added to mutable Vec2 instances:
>>> destinations.append(tuple(a_vec2_instance))
...
>>> position = Vec2(0,0)
>>> position += destinations.popleft()
Traceback (most recent call last):
  File "/usr/lib/python3.9/code.py", line 90, in runcode

I'm working on this with two goals:

Implementing all current functionality for FrozenVec* classes
Unit tests for current & new classes so we can make changes safely

This is my current plan for type interactions:
# binary operations always return a new instance
frozen  +  frozen   # creates a new frozen
frozen  +  mutable  # creates a new frozen
mutable +  frozen   # creates a new mutable
mutable +  mutable  # creates a new mutable

# in-place operations get more complicated
frozen  += frozen   # raises AttributeError
frozen  += mutable  # raises AttributeError
mutable += mutable  # modifies in-place
mutable += frozen   # modifies in-place

I'm not sure if I'll add the in-place operation behavior in the first PR.

That looks like a good plan.
Your logic looks sound.

@pushfoo
frozen  +  mutable  # creates a new frozen
mutable +  frozen   # creates a new mutable

Why should these 2 be different?

frozen  += frozen   # raises AttributeError
frozen  += mutable  # raises AttributeError

Why not have these be equivalent to frozen = frozen + frozen and frozen = frozen + mutable?

@pushfoo
frozen  +  mutable  # creates a new frozen
mutable +  frozen   # creates a new mutable

Why should these 2 be different?

My feeling is that the leftmost object is the primary object, due to left -> right order of operations, so it makes sense to use that type for the result. I'm not sure of a specific rule.

frozen  += frozen   # raises AttributeError
frozen  += mutable  # raises AttributeError

Why not have these be equivalent to frozen = frozen + frozen and frozen = frozen + mutable?

I'm not speaking for @pushfoo here, but my opinion:
Since += implies in-place updating, it's probably better to be explicit with the frozen types.
You can of course have += return a new object, but it wouldn't be clear to the user what was happening (unless they checked the object id). Since you would only be using frozen types in places where you don't expect them to be changed, the exception here makes more sense to me.

@TomFryers What Ben said is the exact reason for raising an AttributeError when code tries to use in-place operations with Frozen instances:

Since += implies in-place updating, it's probably better to be explicit with the frozen types.
You can of course have += return a new object, but it wouldn't be clear to the user what was happening (unless they checked the object id). Since you would only be using frozen types in places where you don't expect them to be changed, the exception here makes more sense to me.

I'm rather certain about this design choice because it prevents all sorts of mistakes. For example:
# accidentally replacing a constant with a new value
NORTH += SOUTH

# replacing a local frozen value in-place silently
frozen_instance += mutable_or_frozen

The text of the raised exception can direct the user toward storing their mutable data in a non-frozen Vec class instead. These are mutable by default in master branch, and should be used whenever a vector can reasonably be expected to change.

@olehermanse

frozen  +  mutable  # creates a new frozen
mutable +  frozen   # creates a new mutable

Why should these 2 be different?

I apologize for the delay.
tl;dr: Ben got the gist of it. To my understanding, the alternatives will also be slower to execute, more complicated to implement, and generally more brittle.
I may be wrong and would appreciate any feedback or proposals for alternative behavior you can provide.
As to why I proposed the specific behavior you asked about:
1. Consistency
As Ben mentioned, it follows the general left-right order of operations used in standard infix notation:

the leftmost object is the primary object, due to left -> right order of operations, so it makes sense to use that type for the result.

With the behavior I outlined earlier, the leftmost type always determines what happens, including in-place modification operators and functions involving another vector such as lerp. Documentation can explain it as "freezing from the left".
2. Simple & Reasonably Fast Implementation
Implementing the idea above for binary operators/methods is simple, reasonably fast (no isinstance), and works well with inheritance:
def __add__(self, other: Vec2) -> Vec2:
    return self.__class__(self.x + other.x, self.y + other.y)

3. Support Orthogonal Concerns

Get a basic set of unit tests out the door for Vec types. We have none for Vec types at the moment, and adding them can help us safely make changes to specific behaviors before release.
Encourage using the in-place mutable types for stuff like position += velocity * dt

I see your point re preventing accidental reassignments. My worry is that this is different behaviour to all of the built-in immutable types, which may be unexpected.

@TomFryers Does this look more reasonable?
# binary operations always return a new instance
frozen  +  frozen   # new frozen
frozen  +  mutable  # new mutable
mutable +  frozen   # new mutable
mutable +  mutable  # new mutable

# frozen vectors do not support in-place operations
frozen  += frozen   # raises AttributeError
frozen  += mutable  # raises AttributeError
mutable += frozen   # modifies in-place
mutable += mutable  # modifies in-place

I'm not sure of the best way to implement it efficiently while still allowing inheritance. Maybe discarding inheritance is the way to go for now? I'm also unsure if there's a convenient way to imitate Java's final keyword applied to a class without adding dependencies.

I mean, for int, float, tuple and str, a += b is perfectly valid, and equivalent to a = a + b. Getting an error when DRYing a = a + b to a += b may be unexpected.

I mean, for int, float, tuple and str, a += b is perfectly valid, and equivalent to a = a + b. Getting an error when DRYing a = a + b to a += b may be unexpected.

I agree that it's a little unexpected, but that's intentional. None of the types you mentioned were created with the explicit intent of acting as module-level constants. Adding this behavior highlights that and can inform the user of it. We could also name the proposed Vec subclasses something like ConstantVec*, but that would break the precedent set by frozenset.
For the time being, I think I'll work on separating out the unit tests I've been working on from the preliminary Frozen implementations. We need most of those tests regardless of what we do about this issue since we have zero unit tests for vectors.

I think I'm viewing these types a bit differently. If added, I'd want to use them everywhere in place of their mutable counterparts, since mutability often causes bugs.

I'd want to use them everywhere in place of their mutable counterparts, since mutability often causes bugs.

Thank you for explaining. I understand your viewpoint a lot better now. I think the old Vec implementations were immutable and based on tuples for the same reasons you outlined. Maybe it's worth pursuing doing something similar down the line for FrozenVecs. For now, I'll stick to the current plan of adding unit tests before doing further work.
There might be ways of using advanced class features to accommodate everyone's priorities, but I don't know enough about them yet. One potentially unsatisfying way to deal with this ticket is to put it off until 3.8 is the minimum supported Python version. Doing so would allow the use of typing protocols and make implementation easier, in addition to allowing custom Vec types.

I'm strongly considering changing all Vec types to be immutable (tuple subclasses, like Mat3/4) for the upcoming v2.1 release. This is a breaking release, so a good chance to do it.
There were some usability arguments to having mutable Vec types (for instance player_position.x += dt), but in the end it seems to have caused more issues. In that example, it's probably not a good idea to be treating the Vector as a distinct object anyway.

Update per what ben said today: this is going to be closable on 2.1's release.

authors: TomFryers benmoran56 olehermanse pushfoo

repo: pyglet/pyglet


text:
Example to fix windows inside a viewport : GLFW

Is there an example to fix a window inside a viewport for glfw? and cursor tracking is XOR type in that viewport?
Example:
The Imgui window is locked in position?

Your request about fixing a window in one position seems to be completely unrelated to GLFW?

@learn-more, I'm not so sure(that is partly why I am asking this). When using mujoco-python-viewer , I use viewports inside the window something like this:
graph_viewport= mujoco.MjrRect(
            int(3 * width / 4) + width_adjustment,
            0,
            int(window_width / 4),
            int(window_height / 4),)

This keeps the the location static w.r.t the window
However, I do not know what the underlying MjrRect function does as it renders the graph inside the viewport.
I expected there to be method to lock the position of imgui inside the viewport(or even the window) or keep it locked to the side and the top of the window.

Hi,
imgui provides methods for setting the position of the next window:
https://github.com/pyimgui/pyimgui/blob/74c359702542c4319f4541bab6825597991e980a/imgui/core.pyx#L2651-L2676
Does this provide the functionality you need?
You can also check the examples provided by pyimgui for some use cases:
https://github.com/pyimgui/pyimgui/blob/master/doc/examples/testwindow.py
Best regards
Stephan

To answer these kind of questions you should familiarize yourself with the imgui library. Here is an FAQ, that addresses some of your questions already:
https://github.com/ocornut/imgui/blob/master/docs/FAQ.md
Also the C++ imgui_demo provides a lot of examples on how to use the imgui API.

@rohit-kumar-j This seems more like you want people to read the documentation for you and give you the exact code that you should use.
I suggest that you spend some time reading the documentation yourself.

source: gharchive/issue

authors: learn-more rohit-kumar-j swenninger-1

repo: pyimgui/pyimgui

license_type: permissive

license_source: github-api


text:
Git copy content

From pyiron_workflow, which in turn git-copied from pyiron_base and perhaps other places

I thought I might run into this; the conda merge fails because the env file has no dependencies:
Run python $GITHUB_ACTION_PATH/../.support/condamerge.py $GITHUB_ACTION_PATH/../.support/environment-unittests.yml $GITHUB_ACTION_PATH/../.support/environment-coveralls-codacy.yml .ci_support/environment.yml .ci_support/environment-tests.yml > ./environment.yml
Traceback (most recent call last):
  File "/home/runner/work/_actions/pyiron/actions/v3.1.0/write-environment/../.support/condamerge.py", line 80, in <module>
    merge_env(
  File "/home/runner/work/_actions/pyiron/actions/v3.1.0/write-environment/../.support/condamerge.py", line 68, in merge_env
    'dependencies': merge_dependencies(
  File "/home/runner/work/_actions/pyiron/actions/v3.1.0/write-environment/../.support/condamerge.py", line 43, in merge_dependencies
    add_dict, add_pip_dict = split_dependencies(env_add)
  File "/home/runner/work/_actions/pyiron/actions/v3.1.0/write-environment/../.support/condamerge.py", line 25, in split_dependencies
    for f in env:
TypeError: 'NoneType' object is not iterable

I'm going to just add "python" and see if that minimal change will do it.

Pull Request Test Coverage Report for Build 9199479205
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-10.5%) to 89.535%

Totals

Change from base Build 9198647252:
-10.5%

Covered Lines:
308

Relevant Lines:
344

💛  - Coveralls

@samwaseda since files is a non-trivial chunk by line count, I'm sticking you with review and put you in the author list.
This just (git)copies everything from pyiron_workflow.snippets over to this repo, minus the pyiron test case, which (a) I don't think we really need now that we have other (IMO nicer) ways of testing the docs (cf. the integration test here), and (b) it wants to import numpy and is thus at least a little against the spirit of the business here.
If you're happy with this, I'd prefer to squash merge so the initial content commit is all one piece.

Yeah I was thinking about moving them as well. I’m super happy with the decision

Ah I thought it was pyiron_base. I'm also ok with a new repo, but do we have a clear dependency hierarchy somewhere for all the pyiron modules? I'm a bit concerned that we/I might lose an overview.

This has zero of its own dependencies, so can be mixed in wherever. E.g. one of the first things I want to do with it once it's on conda (besides replacing the code in pyiron workflow) is to search and replace all uses of the Singleton class, which I know is at least in base.

source: gharchive/pull-request

authors: coveralls liamhuber samwaseda


text:
Update utils.py

resolves https://github.com/pyjanitor-devs/pyjanitor/issues/1201
simple typo

PR Description
Please describe the changes proposed in the pull request:

This PR resolves #(put issue number here, and remove parentheses).

PR Checklist

Please ensure that you have done the following:

[ ] PR in from a fork off your branch. Do not PR from <your_username>:dev, but rather from <your_username>:<feature-branch_name>.

[ ] If you're not on the contributors list, add yourself to AUTHORS.md.

[ ] Add a line to CHANGELOG.md under the latest version header (i.e. the one that is "on deck") describing the contribution.

Do use some discretion here; if there are multiple PRs that are related, keep them in a single line.

Automatic checks
There will be automatic checks run on the PR. These include:

Building a preview of the docs on Netlify
Automatically linting the code
Making sure the code is documented
Making sure that all tests are passed
Making sure that code coverage doesn't go down.

Relevant Reviewers

Please tag maintainers to review.

@ericmjl

@ericmjl

@Zeroto521 thoughts?

@joranbeasley I just realized that this PR is a duplicate PR of #1203.
As mentioned in my comment on that PR, I'd like to nonetheless credit you in the changelog for taking the time to make the PR. As such, I will close this PR once the other is merged. That said, you'll be listed alongside @xujiboy for the effort you took here.

authors: ericmjl joranbeasley samukweku

repo: pyjanitor-devs/pyjanitor

license: MIT


text:
[rojak-app] Initial iOS app project.

Setup project
Setup pods

Thanks suhu 🙇, feel free to merge yeah kalo dah ok 🙇

authors: hidrodixtion pyk

repo: pyk/rojak

license_source: bigquery


text:
数据auc是选testauc还是validauc

请问一下大佬，我们选用数据时候可以用验证集AUC吗 Epoch: 8, validauc: 0.7784, validacc: 0.7796, best epoch: 8, best auc: 0.7784, train loss: 10.460885047912598, emb_type: qid, model: iekt, save_dir:  还是必须用test呢？

您好，感谢对我们工作的支持，我们在训练过程中选用最优模型采用的是validauc（会自动保存最优模型路径save_dir: XX）,最终，如果模型属于que_type_model,我们的test结果选用window_testauc。

您好，感谢对我们工作的支持，我们在训练过程中选用最优模型采用的是validauc（会自动保存最优模型路径save_dir: XX）,最终，如果模型属于que_type_model,我们的test结果选用window_testauc。
是不是说 如果模型属于que_type_model，例如IEKT就选用window_testauc，其他模型就选普通的testauc？

您好，感谢对我们工作的支持，我们在训练过程中选用最优模型采用的是validauc（会自动保存最优模型路径save_dir: XX）,最终，如果模型属于que_type_model,我们的test结果选用window_testauc。
是不是说 如果模型属于que_type_model，例如IEKT就选用window_testauc，其他模型就选普通的testauc？
对于非que_type_model,也就是KC level的模型，在测试的时候会存在标签泄漏问题，我们有对其进行fusion操作，一般来说我们会选择auclate_xx/windowauclate_xx这些结果。可以看看我们的论文哈：https://proceedings.neurips.cc/paper_files/paper/2022/hash/75ca2b23d9794f02a92449af65a57556-Abstract-Datasets_and_Benchmarks.html

还有个问题就是，test_dataset加载时，如果需要的数据window_testauc，那么代码在加载的时候，        elif model_name == "lpkt":
test_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_file"]), at2idx, it2idx, data_config["input_type"], {-1})
test_window_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_window_file"]), at2idx, it2idx, data_config["input_type"], {-1})
elif model_name in que_type_models:
test_dataset = KTQueDataset(os.path.join(data_config["dpath"], data_config["test_file_quelevel"]),
input_type=data_config["input_type"], folds=[-1],
concept_num=data_config['num_c'], max_concepts=data_config['max_concepts'])  是不是需要把data_config["test_file_quelevel"])改成data_config["test_window_file"]

还有个问题就是，test_dataset加载时，如果需要的数据window_testauc，那么代码在加载的时候， elif model_name == "lpkt": test_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_file"]), at2idx, it2idx, data_config["input_type"], {-1}) test_window_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_window_file"]), at2idx, it2idx, data_config["input_type"], {-1}) elif model_name in que_type_models: test_dataset = KTQueDataset(os.path.join(data_config["dpath"], data_config["test_file_quelevel"]), input_type=data_config["input_type"], folds=[-1], concept_num=data_config['num_c'], max_concepts=data_config['max_concepts']) 是不是需要把data_config["test_file_quelevel"])改成data_config["test_window_file"]
我们在加载数据的时候，test_dataset和test_window_dataset都会加载，不需要手动进行修改，在预测的时候，在test_dataset and test_window_dataset 的预测结果都会一并给出。

还有个问题就是，test_dataset加载时，如果需要的数据window_testauc，那么代码在加载的时候， elif model_name == "lpkt": test_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_file"]), at2idx, it2idx, data_config["input_type"], {-1}) test_window_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_window_file"]), at2idx, it2idx, data_config["input_type"], {-1}) elif model_name in que_type_models: test_dataset = KTQueDataset(os.path.join(data_config["dpath"], data_config["test_file_quelevel"]), input_type=data_config["input_type"], folds=[-1], concept_num=data_config['num_c'], max_concepts=data_config['max_concepts']) 是不是需要把data_config["test_file_quelevel"])改成data_config["test_window_file"]
我们在加载数据的时候，test_dataset和test_window_dataset都会加载，不需要手动进行修改，在预测的时候，在test_dataset and test_window_dataset 的预测结果都会一并给出。

但是不改的话，每次跑window_testauc都是-1啊 像下面这样Epoch: 50, validauc: 0.9223, validacc: 0.868, best epoch: 46, best auc: 0.9223, train loss: 3.704118251800537, emb_type: qid, model: iekt, save_dir: saved_model\algebra2005_iekt_qid_saved_model_0.001_42_0_0_64_40_1_10_10_0.93_0_1
testauc: -1, testacc: -1, window_testauc: -1, window_testacc: -1

还有个问题就是，test_dataset加载时，如果需要的数据window_testauc，那么代码在加载的时候， elif model_name == "lpkt": test_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_file"]), at2idx, it2idx, data_config["input_type"], {-1}) test_window_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_window_file"]), at2idx, it2idx, data_config["input_type"], {-1}) elif model_name in que_type_models: test_dataset = KTQueDataset(os.path.join(data_config["dpath"], data_config["test_file_quelevel"]), input_type=data_config["input_type"], folds=[-1], concept_num=data_config['num_c'], max_concepts=data_config['max_concepts']) 是不是需要把data_config["test_file_quelevel"])改成data_config["test_window_file"]
我们在加载数据的时候，test_dataset和test_window_dataset都会加载，不需要手动进行修改，在预测的时候，在test_dataset and test_window_dataset 的预测结果都会一并给出。

但是不改的话，每次跑window_testauc都是-1啊 像下面这样Epoch: 50, validauc: 0.9223, validacc: 0.868, best epoch: 46, best auc: 0.9223, train loss: 3.704118251800537, emb_type: qid, model: iekt, save_dir: saved_model\algebra2005_iekt_qid_saved_model_0.001_42_0_0_64_40_1_10_10_0.93_0_1 testauc: -1, testacc: -1, window_testauc: -1, window_testacc: -1

我们在训练的时候不会进行测试集的训练，我们的测试代码放在了wandb_predict.py中。

您好，如果您还是有些地方不清楚，可以加我微信哈，方便点Zion0613 ---- 回复的原邮件 @.>发送日期2023年11月20日 19:27 @.> @.>@.>主题Re: [pykt-team/pykt-toolkit] 数据auc是选testauc还是validauc (Issue #144)
还有个问题就是，test_dataset加载时，如果需要的数据window_testauc，那么代码在加载的时候， elif model_name == "lpkt": test_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_file"]), at2idx, it2idx, data_config["input_type"], {-1}) test_window_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_window_file"]), at2idx, it2idx, data_config["input_type"], {-1}) elif model_name in que_type_models: test_dataset = KTQueDataset(os.path.join(data_config["dpath"], data_config["test_file_quelevel"]), input_type=data_config["input_type"], folds=[-1], concept_num=data_config['num_c'], max_concepts=data_config['max_concepts']) 是不是需要把data_config["test_file_quelevel"])改成data_config["test_window_file"]
我们在加载数据的时候，test_dataset和test_window_dataset都会加载，不需要手动进行修改，在预测的时候，在test_dataset and test_window_dataset 的预测结果都会一并给出。
但是不改的话，每次跑window_testauc都是-1啊 像下面这样Epoch: 50, validauc: 0.9223, validacc: 0.868, best epoch: 46, best auc: 0.9223, train loss: 3.704118251800537, emb_type: qid, model: iekt, save_dir: saved_model\algebra2005_iekt_qid_saved_model_0.001_42_0_0_64_40_1_10_10_0.93_0_1
testauc: -1, testacc: -1, window_testauc: -1, window_testacc: -1
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.>
[
{
@.": "http://schema.org",
@.": "EmailMessage",
"potentialAction": {
@.": "ViewAction",
"target": "https://github.com/pykt-team/pykt-toolkit/issues/144#issuecomment-1818870712",
"url": "https://github.com/pykt-team/pykt-toolkit/issues/144#issuecomment-1818870712",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
@.***": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]

还有个问题就是，test_dataset加载时，如果需要的数据window_testauc，那么代码在加载的时候， elif model_name == "lpkt": test_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_file"]), at2idx, it2idx, data_config["input_type"], {-1}) test_window_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_window_file"]), at2idx, it2idx, data_config["input_type"], {-1}) elif model_name in que_type_models: test_dataset = KTQueDataset(os.path.join(data_config["dpath"], data_config["test_file_quelevel"]), input_type=data_config["input_type"], folds=[-1], concept_num=data_config['num_c'], max_concepts=data_config['max_concepts']) 是不是需要把data_config["test_file_quelevel"])改成data_config["test_window_file"]
我们在加载数据的时候，test_dataset和test_window_dataset都会加载，不需要手动进行修改，在预测的时候，在test_dataset and test_window_dataset 的预测结果都会一并给出。

但是不改的话，每次跑window_testauc都是-1啊 像下面这样Epoch: 50, validauc: 0.9223, validacc: 0.868, best epoch: 46, best auc: 0.9223, train loss: 3.704118251800537, emb_type: qid, model: iekt, save_dir: saved_model\algebra2005_iekt_qid_saved_model_0.001_42_0_0_64_40_1_10_10_0.93_0_1 testauc: -1, testacc: -1, window_testauc: -1, window_testacc: -1

我们在训练的时候不会进行测试集的训练，我们的测试代码放在了wandb_predict.py中。

所以就是先训练好模型，然后利用wandb_predict.py进行测试得到window_testauc，对吗？

还有个问题就是，test_dataset加载时，如果需要的数据window_testauc，那么代码在加载的时候， elif model_name == "lpkt": test_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_file"]), at2idx, it2idx, data_config["input_type"], {-1}) test_window_dataset = LPKTDataset(os.path.join(data_config["dpath"], data_config["test_window_file"]), at2idx, it2idx, data_config["input_type"], {-1}) elif model_name in que_type_models: test_dataset = KTQueDataset(os.path.join(data_config["dpath"], data_config["test_file_quelevel"]), input_type=data_config["input_type"], folds=[-1], concept_num=data_config['num_c'], max_concepts=data_config['max_concepts']) 是不是需要把data_config["test_file_quelevel"])改成data_config["test_window_file"]
我们在加载数据的时候，test_dataset和test_window_dataset都会加载，不需要手动进行修改，在预测的时候，在test_dataset and test_window_dataset 的预测结果都会一并给出。

但是不改的话，每次跑window_testauc都是-1啊 像下面这样Epoch: 50, validauc: 0.9223, validacc: 0.868, best epoch: 46, best auc: 0.9223, train loss: 3.704118251800537, emb_type: qid, model: iekt, save_dir: saved_model\algebra2005_iekt_qid_saved_model_0.001_42_0_0_64_40_1_10_10_0.93_0_1 testauc: -1, testacc: -1, window_testauc: -1, window_testacc: -1

我们在训练的时候不会进行测试集的训练，我们的测试代码放在了wandb_predict.py中。

所以就是先训练好模型，然后利用wandb_predict.py进行测试得到window_testauc，对吗？

是的

repo: pykt-team/pykt-toolkit


text:
Fix #288: remove deprecated 'model' property in tests.

Closes issue #288.
Removes the FutureWarning in tests caused by a deprecated property:
FutureWarning: Model.model property is deprecated. Just use Model.
    warnings.warn("Model.model property is deprecated. Just use Model.", FutureWarning)

Thanks!

authors: anevolbap drbenvincent

repo: pymc-labs/CausalPy


text:
Save lift tests to InferenceData

This would be required to rebuild same model after using lift test methods.
Currently missing:

save the lift tests to the fit_data group
upon load, add the lift tests to the model likelihood

Would be done here

An alternative to this would be to save the lift test to a separate group. However, this would likely cause some additional complexity and would have to capture additional warnings. The fit_data just runs the risk of having to change the structure of the fit_data group itself and likely have to have a version bump of the model.

The kwargs used with the would have to be saved off as well or the reconstructed model would not be the same
https://github.com/pymc-labs/pymc-marketing/blob/4bddf9450f7a85ade4a03ccb47af55bcefc4b982/pymc_marketing/mmm/delayed_saturated_mmm.py#L1735-L1740
Potentially these kwargs in the attrs like in the save and load methods. This seems to require that each df_lift_test would have to be saved in separate variable? Maybe using another group like lift_tests would be more clear?

If the data is stored with pm.Data then these will appear in the observed_data and constant_data groups that come along with PyMC models.

repo: pymc-labs/pymc-marketing


text:
Add keybord key capture

Describe the bug
A simple case I have is that when search keyword is inputed, then press enter key, search is executed, like Google web search box, it has the same effect as I manually click search bottun.
To Reproduce
Steps to reproduce the behavior:

Code/Link to Repo:

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
** Specifics (please complete the following information):**

Python Version:
Pynecone Version:
OS:
Browser (Optional):

Additional context
Add any other context about the problem here.

I think you can already this. For your specific example, you can simply wrap the pc.input in a pc.form which has the on_submit event that gets triggered when the user presses enter while the form has focus.
Generally, the on_key_down and on_key_up events of pc.input should also pass the pressed key to their event handler, so you could check in the event handler if it is the enter key (or whatever other key you are interested) that is pressed/released.

Is there a component called pc.form?  I found pc.form_control in this Docs page https://pynecone.io/docs/library/forms/formcontrol, and there is no event trigger on_submit.  pc.forms raise an error said module not callable.
Tried on_key_down of input, but this will trigger event each time when a single key is pressed. This causes to much event when typing  search word. Anyway, I'll filter it to see if it's "Enter", then do the search.

Thank you both, I'll try and go back here.

Upgraded to 0.1.31, new problem occurs:

The code works in version 0.1.29:
pc.table(                 pc.tr(                     pc.foreach(State.th_fields, generate_th)                 ),                 pc.foreach(                     State.return_lists,                     generate_tr                 )             ),
`def generate_td(value, index) -> pc.Component:
return pc.cond(
index == State.last_index,
pc.cond(
value == "",
pc.td(pc.icon(tag="download", on_click=lambda x: State.download_pdf(value), _hover={"cursor": "pointer"}), border_width="1.5px", margin="0px", padding="8px", text_align="center"),
pc.td("", border_width="1.5px", margin="0px", padding="8px", text_align="center")
),
pc.td(value, border_width="1.5px", margin="0px", padding="8px")
)
def generate_tr(row) -> pc.Component:
return pc.tr(
pc.foreach(
row,
lambda value, index: generate_td(value, index)
),
_hover={"bg": "lavender"}
)`

repo: pynecone-io/pynecone


text:
"pipenv update " doesn't upgrade package

Hi,
I think that upgrade workflow for a single package described in documentation doesn't work properly in 11.10.3. pipenv update <package> command doesn't change neither Pipfile.lock nor package installed in virtual environment.
<details><summary>$ python -m pipenv.help output</summary>

Pipenv version: `'11.10.3'`

Pipenv location: `'/usr/local/lib/python3.6/site-packages/pipenv'`

Python location: `'/usr/local/opt/python/bin/python3.6'`

Other Python installations in `PATH`:

- `2.7`: `/usr/local/bin/python2.7`
- `2.7`: `/usr/local/bin/python2.7`
- `2.7`: `/usr/local/bin/python2.7`
- `2.7`: `/usr/bin/python2.7`
- `3.6`: `/usr/local/bin/python3.6m`
- `3.6`: `/usr/local/bin/python3.6`
- `3.6`: `/usr/local/bin/python3.6`

- `3.6.5`: `/usr/local/opt/python/libexec/bin/python`
- `2.7.14`: `/usr/local/bin/python`
- `2.7.14`: `/usr/local/bin/python`
- `2.7.10`: `/usr/bin/python`
- `2.7.14`: `/usr/local/bin/python2`
- `2.7.14`: `/usr/local/bin/python2`
- `3.6.5`: `/usr/local/bin/python3`
- `3.6.5`: `/usr/local/bin/python3`

PEP 508 Information:

```
{'implementation_name': 'cpython',
'implementation_version': '3.6.5',
'os_name': 'posix',
'platform_machine': 'x86_64',
'platform_python_implementation': 'CPython',
'platform_release': '17.5.0',
'platform_system': 'Darwin',
'platform_version': 'Darwin Kernel Version 17.5.0: Fri Apr 13 19:32:32 PDT '
                    '2018; root:xnu-4570.51.2~1/RELEASE_X86_64',
'python_full_version': '3.6.5',
'python_version': '3.6',
'sys_platform': 'darwin'}
```

System environment variables:

- `TERM_SESSION_ID`
- `SSH_AUTH_SOCK`
- `Apple_PubSub_Socket_Render`
- `COLORFGBG`
- `ITERM_PROFILE`
- `XPC_FLAGS`
- `LANG`
- `PWD`
- `SHELL`
- `SECURITYSESSIONID`
- `TERM_PROGRAM_VERSION`
- `TERM_PROGRAM`
- `PATH`
- `COLORTERM`
- `COMMAND_MODE`
- `TERM`
- `HOME`
- `TMPDIR`
- `USER`
- `XPC_SERVICE_NAME`
- `LOGNAME`
- `__CF_USER_TEXT_ENCODING`
- `ITERM_SESSION_ID`
- `SHLVL`
- `OLDPWD`
- `ZSH`
- `PAGER`
- `LESS`
- `LC_CTYPE`
- `LSCOLORS`
- `VIRTUALENVWRAPPER_PROJECT_FILENAME`
- `VIRTUALENVWRAPPER_WORKON_CD`
- `VIRTUALENVWRAPPER_SCRIPT`
- `WORKON_HOME`
- `VIRTUALENVWRAPPER_HOOK_DIR`
- `LS_COLORS`
- `ANDROID_HOME`
- `_`
- `PYTHONDONTWRITEBYTECODE`
- `PIP_PYTHON_PATH`

Pipenv–specific environment variables:

Debug–specific environment variables:

- `PATH`: `/usr/local/opt/python/libexec/bin:/Users/kuba/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/usr/local/share/dotnet:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/kuba/Library/Android/sdk/tools:/Users/kuba/Library/Android/sdk/platform-tools`
- `SHELL`: `/bin/zsh`
- `LANG`: `pl_PL.UTF-8`
- `PWD`: `/Users/kuba`

---------------------------

</details>

Expected result
pipenv update <package> should both upgrade the package and keep Pipfile.lock up-to-date with current version.
Actual result
pipenv update <package> upgrades package but no changes to Pipfile.lock are made, so previous version is installed back again.
Steps to replicate

Run pipenv install "django<2.0"
Edit out Django version in Pipfile and make it wildcard
Run pipenv lock --keep-outdated and make sure Django 1.11.x is included in Pipfile.lock
Run pipenv update django which says it's installing 2.x branch
Pipfile.lock is left untouched and pipenv run pip freeze lists original version.

Please update to a more current version and let us know if you still experience this issue.

Version 11.10.3 of Pipenv is the latest one at the moment, correct me if I'm wrong.

@kubazarz Unfortunately not. Version 11.10.4 is the latest one.

Ah sorry I misread in the first place. However you pinned django when you installed which means there is a constraint in your Pipfile. The update command will always obey this constraint (<2.0)

Yes, I know, but I wanted to have outdated version of some package to reproduce. I removed pinned version in the second step.

Also command explicitly shows that it downloads newer version. After I executed pipenv update Django was updated to 2.0.5 as expected. Hope this helps.

I have to note that as a fresh pipenv user I just stumbled over this issue because some of my dev dependencies wouldn't update.
In my case I simply forgot to add --dev to the update/sync/lock etc. commands. Maybe this helps others.

@TauPan I think Your use case is different than mine. I see #2179 touches the same issue and it's waiting to be resolved.

@techalchemy This feels like odd behavior. To upgrade a single package (which is a common workflow in a complete application to make regression testing easier) I have to pin all my packages in the my Pipfile except for the one I want to upgrade, do a pipenv lock to update only my one package?
In the future would the pipenv update work as expected - as the requirements was not locked at the time the Pipfile.lock was generated?
I have to be honest, I would expect an update <package> to update the package (which the output tells me it is doing) and then sync the lock to reflect that single library change.

Just to labor the point. When I do an pipenv update flask-restplus it gives me this output:
Installing flask-restplus…
Collecting flask-restplus
  Using cached https://files.pythonhosted.org/packages/26/cf/8c12fe4ccd1dadb6fa5e7f95065c894740f3a053454daeb8af6ff266de0f/flask_restplus-0.11.0-py2.py3-none-any.whl
Requirement not upgraded as not directly required: jsonschema in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from flask-restplus) (2.6.0)
Requirement not upgraded as not directly required: Flask>=0.8 in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from flask-restplus) (0.12.2)
Requirement not upgraded as not directly required: six>=1.3.0 in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from flask-restplus) (1.10.0)
Requirement not upgraded as not directly required: aniso8601>=0.82 in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from flask-restplus) (1.2.1)
Requirement not upgraded as not directly required: pytz in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from flask-restplus) (2017.2)
Requirement not upgraded as not directly required: Werkzeug>=0.7 in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from Flask>=0.8->flask-restplus) (0.12.2)
Requirement not upgraded as not directly required: click>=2.0 in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from Flask>=0.8->flask-restplus) (6.7)
Requirement not upgraded as not directly required: itsdangerous>=0.21 in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from Flask>=0.8->flask-restplus) (0.24)
Requirement not upgraded as not directly required: Jinja2>=2.4 in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from Flask>=0.8->flask-restplus) (2.9.6)
Requirement not upgraded as not directly required: python-dateutil in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from aniso8601>=0.82->flask-restplus) (2.6.0)
Requirement not upgraded as not directly required: MarkupSafe>=0.23 in /Users/gareth/.local/share/virtualenvs/sp-api-JUg8AgFq/lib/python3.6/site-packages (from Jinja2>=2.4->Flask>=0.8->flask-restplus) (1.0)
Installing collected packages: flask-restplus
  Found existing installation: flask-restplus 0.10.1
    Uninstalling flask-restplus-0.10.1:
      Successfully uninstalled flask-restplus-0.10.1
Successfully installed flask-restplus-0.11.0

Adding flask-restplus to Pipfile's [packages]…
Installing dependencies from Pipfile.lock (f43e09)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 37/37 — 00:00:17
To activate this project's virtualenv, run the following:
 $ pipenv shell```

Which indicates that is has updated the package, and then just undoes that from the lock file (but it does that silently, it's only when I manually check the version I find the issue). One of those behaviors must not be the intention, as they are working against each other.

@thestub yes, this is a (very serious) bug currently, it's quite bad.
It is however fixed in master and will be released shortly.  The behavior should be as follows =>

You pin the new version of (or unpin) whatever package you want in your Pipfile
You run Pipenv update
Pipenv runs pipenv lock as a result in order to make sure that your change is also OK with the rest of the packages you have installed
Pipenv then runs pipenv sync to sync the new lockfile with your environment

We have a regression test for this going forward as well, so we should be able to catch this if it breaks again.  Sorry for the frustration on this one

I experience the same problem  on  version  2018.11.26

2020 the same problem yet? I'm having the same issue. A package has version 2+ available but mine is stuck at 0.22.0 and pipenv update doesn't work.

Still happening in 2021? At least with a GitHub package, this seems to still be an issue... 🤔

authors: TauPan aderchox davidstosik halicki kubazarz malinich techalchemy thestub

repo: pypa/pipenv


text:
Error running: pipenv shell (fish shell)

Issue description
Running pipenv shell results the errors shown further down. If I start bash and then run pipenv shell things work as expected. Appears that my fish shell is freaking out in particular.
Expected result
Expected to be put into my virtual environment shell as usual.
Actual result
Launching subshell in virtual environment…
 source /home/myname/.local/share/virtualenvs/myproject-ws-omHWSt_X/bin/activate.fish
Failed to execute process '/usr/bin/uname'. Reason:
The total size of the argument and environment lists 3.2MB exceeds the operating system limit of 2MB.
Try running the command again with fewer arguments.
Welcome to fish, the friendly interactive shell
Failed to execute process '/usr/bin/basename'. Reason:
The total size of the argument and environment lists 3.2MB exceeds the operating system limit of 2MB.
Try running the command again with fewer arguments.
:  source /home/myname/.local/share/virtualenvs/myproject-ws-omHWSt_X/bin/activate.fish
Failed to execute process '/usr/bin/head'. Reason:
The total size of the argument and environment lists 3.2MB exceeds the operating system limit of 2MB.
Try running the command again with fewer arguments.
test: Missing argument at index 2

After running pipenv --support, I realized my PATH statement is far too long because of some Flutter and Android SDK updates. Please disregard this issue for now.

Issue was related to a bug in my fish PATH configuration, causing it to become extremely long.


text:
Change load_module to exec_module

#2249 changed back from exec_module to the deprecated load_module (so I'll cc @alexhenrie). This PR moves forward to exec_module again. Credit to posts like https://stackoverflow.com/a/41595552/4093019
Also fixes the Pillow compilation error reported in #2261.

Thanks Andrew!

authors: alexhenrie radarhere

repo: pypa/setuptools

license: mit


text:
Fix ReST syntax error in the PEP 517 link

I noticed a ReST formatting problem on the quickstart page:

In this file
``twine <https://pypi.org/project/twine/>``

has the same issue.

Fixed the Twine link too, thank you.

@mgedmin I've made a huge PR recently (#2427) and this one may be unnecessary or needs rebasing.

https://setuptools.readthedocs.io/en/latest/userguide/quickstart.html today reads

The change is driven under 517.

and, uh, I think that should be "PEP 517".

@mgedmin ah, right... The anchor is just 517. Do you want to send a patch?

Okay: https://github.com/pypa/setuptools/pull/2439

authors: imba-tjd mgedmin webknjaz


text:
Added short license notice

PyPI.org packages usually gives short license notice, but setuptools not.
In venv .../site-packages/setuptools-57.4.0.dist-info/METADATA  I can see just:

License: UNKNOWN

Obviously - this amateur PR will fix it ;-).
Thank you for your work and time.
Pax et bonum.

Thanks for the contrib. Unfortunately, I wish to avoid papering over downstream flaws and perpetuate the suboptimal practice of duplicating the license info in the license field counter to the published guidance. See this comment and jaraco/skeleton#9 where we'd considered this previously.

authors: jaraco milos-korenciak


text:
make errors after upgrade to Docker Compose 2.6.0

Steps to reproduce:

Upgrade to Docker Compose 2.6.0
Run make serve for pypa/warehouse.

Error message:
Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "hupper": executable file not found in $PATH: unknown
make: *** [serve] Error 1

When I ran docker-compose run worker bash and echo $PATH I saw my host machine’s $PATH echoed in the container. This appears to be an issue with Docker Compose 2.6.0 mishandling environment variables from env_file:

docker/compose#9521

I’m opening and closing this issue as here in case other Warehouse devs encounter the same issue.

Reverting to Docker Desktop 4.8.2 resolved the issue.

This environment variable issue was fixed by docker/compose#9636 (commit 6d613c8) which was released with Docker Compose 2.8.0 and Docker Desktop 4.12.0.

repo: pypa/warehouse


text:
Account recovery by Email

If you try to do "Request Password Reset" you'll face a problem. The form requires your login that most of the users don't know or forget. It'll be very useful to do account recovery by Email, isn't it?

@RafRaf @di Why not "Request Password Reset" accept username or email? Once the user logged in, they can see the account information.

authors: RafRaf gsb-eng


text:
Filters are sorted ASCIIbetically

To reproduce, go to https://pypi.org/search/?q=beautifulsoup click on "Programming Language" under "filter by classifier" scroll down and see this:

3.0
3.1
3.10
3.2
...
3.9

should be

3.0
3.1
3.2
...
3.9
3.10

#8843

#8843


text:
https://pypi.org/simple/  index seems to be behind by 8k+ updates

I published a new project recently, and it's not showing up on https://pypi.org/simple/ after 18+ hours.
Whereas it is available when referenced directly: https://pypi.org/project/portable-python/
Expected behavior
Project should show on https://pypi.org/simple/ in a reasonable amount of time?
Not sure what a "good time" would be, but I was hoping for less than 12 hours.
To Reproduce
curl -I https://pypi.org/simple/
...
x-pypi-last-serial: 11197400
# no mention of project "portable-python" on https://pypi.org/simple/

curl -I https://pypi.org/simple/portable-python/
...
x-pypi-last-serial: 11205924

x-pypi-last-serial is 8500+ updated behind on main index page.

Closing as a duplicate of #8385 and #7384.

All issues related to this seem to get closed without a resolution.
If it helps: my package eventually showed up, apparently it needs 24+ hours to show up.
Perhaps that's OK... I guess 24 hours isn't that bad.
Posting this update here in case other people run into this, and think their package is stuck: it seems that the main index is simply refreshed with a very long delay.
Note: the main index is typically the source of truth of internal pypi mirrors, so one has to wait those 24+ hours for one's package to show up on an internal mirror basically.

What mirror are you using? Most mirrors use the XML-RPC mirroring APIs and thus don't suffer from this. The simple API and this endpoint is not intended to be a source of truth for mirrors.

We have a custom mirror (working on getting rid of it), which uses artifactory as backend...
If you have a recommended mirror, I'm all ears.

See https://packaging.python.org/guides/index-mirrors-and-caches/

That page says "The XML-RPC API will be deprecated in the future".
Looking at the json API doc, it seems that it can't list all of the projects on pypi (which is needed if one wants to do a mirror that incorporates all of open source pypi, with additional internal packages).
So it seems that there is no better way than looking at https://pypi.org/simple/
bandersnatch seems to be a no-go on that end, devpi maybe (will try it out, but last I tried it a long time ago, it wasn't very stable)

authors: di zsimic


text:
changed default image prefix to comply with multilayer exr Maya Arnold render settings

our defaults for Arnold renderer have conflict of Multilayer exr and <RenderPass> image token

it is actually not working with the

Actually, changing that value in settings wouldn't make it pass that validator either. The used prefix is currently hardcoded, see here.
Anyway, I'd be happy to look into this but feel merging the other render setting tweaks first would likely ease the process. That is:
#3880 and https://github.com/pypeclub/OpenPype/pull/3873

Hi guys, seems like it hasnt been updated yet? Tried to test it out but behaves still same...
@m-u-r-p-h-y  did u already fix the preset or?

As @m-u-r-p-h-y mentioned - the commit I suggested doesn't actually work. Let me look into it with this PR: https://github.com/pypeclub/OpenPype/pull/3880#issuecomment-1251931201

Should have a fix for this now implemented in #3880

I suspect we can close this in favor of #3880

authors: BigRoy LiborBatek m-u-r-p-h-y

repo: pypeclub/OpenPype


text:
Update required cassandra version in docs

Since Cyanite makes use of user-defined types at least Cassandra 2.1 is required.

+1

@eckardt sorry for not acting any sooner!

:+1: :fireworks:

authors: aww-yiss eckardt ifesdjeen pyr

repo: pyr/cyanite

license: isc


text:
How did you come to the idea to write your own static site generator?

Did you know there is alread Coleslaw exists?
I did. And I used Coleslaw a few years until this spring, when I decided to write my own static site generator: StatiCL. I wanted a more generic solution than Coleslaw does.
What is your story behind Crystal?

Hey!
Yep, I knew about Coleslaw. Didn't know about StatiCL, however - thanks for the link.
I'd been using an ad hoc collection of Lisp glue code to publish some static websites for a few years. Crystal is just an attempt to consolidate that stuff in one place.
It's extremely basic. It will probably remain extremely basic.

If something’s missing in Coleslaw, I’m glad to merge your pull requests.


text:
usage of networkx

[x] Remove utilization of networkx in demo/tutorial notebooks as it is simply used for plotting.

[x] Use geopandas for plotting in lieu of networkx.

addressed in #142

repo: pysal/spaghetti


text:
Implement BNG boolean treatment within expressions

BNG treats booleans as integers within expressions (1=True,
0=False), so an expression like (2 > 1) * 4 will get evaluated
to 4. Sometimes these expressions get produced by BNG
local functions and may be present in other imported models as
well. However, doesn't implement multiplication of
BooleanAtoms, which causes a TypeError.
This PR implements the BNG style behaviour on import by
patching BooleanTrue.mul and BooleanFalse.mul.
Not very elegant, but it works.

Codecov Report

Merging #494 into master will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #494      +/-   ##
==========================================
+ Coverage   80.27%   80.29%   +0.01%
==========================================
  Files         101      101
  Lines       10597    10605       +8
==========================================
+ Hits         8507     8515       +8
  Misses       2090     2090

Impacted Files
Coverage Δ

pysb/bng.py
93.15% <100.00%> (+0.09%)
:arrow_up:

pysb/tests/test_bng.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 918e86e...a40f21d. Read the comment docs.

I'd prefer a more robust solution rather than monkey-patching sympy internals.
Do you see literal constant 2 > 1, or is it really x > 1 which is evaluated later? Both can be fixed with a call to replace but we would need one extra step to fix the literal version. If we need to handle the literal one, we can add evaluate=False to the parse_expr call to avoid the "BooleanAtom not allowed in this context" exception at that point. If it's really only symbolic stuff like x > 1 getting parsed then we can skip that step.
Once we have a sympy expression, we can replace all boolean expressions used in Mul operations as below. If other operators need this treatment we can refactor this a bit to make that easier to implement.
# Helper to make the replace function below easier to read
def is_bool_expr(e):
    return isinstance(e, sympy.boolalg.Boolean) and not isinstance(e, sympy.AtomicExpr)

def fix_boolean_multiplication(e):
    args = [
        sympy.Piecewise((1, a), (0, True)) if is_bool_expr(a) else a
        for a in e.args
    ]
    return sympy.Mul(*args)

expr = expr.replace(sympy.Mul, fix_boolean_multiplication)

The expression that was causing the issue was of the form some_expr*(0>0) (with multiple different literal values in variants of the local expression). I got your fix to work with a slight modification:
def fix_boolean_multiplication(*args):
    new_args = [
        sympy.Piecewise((1, a), (0, True)) if is_bool_expr(a) else a
        for a in args
    ]
    return sympy.Mul(*new_args)

This issue seems less likely to occur with Sympy 1.6, where one couldn't write a local expression in the above form in the first place. It's unclear to me whether an expanded local expression read back from BNG could be in that form when the original is not, but it might be possible.
I've pushed the fix as a new commit. Thanks!

Codecov Report

Merging #494 into master will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #494      +/-   ##
==========================================
+ Coverage   80.26%   80.28%   +0.02%
==========================================
  Files         101      101
  Lines       10602    10613      +11
==========================================
+ Hits         8510     8521      +11
  Misses       2092     2092

Impacted Files
Coverage Δ

pysb/bng.py
93.16% <100.00%> (+0.11%)
:arrow_up:

pysb/importers/bngl.py
92.98% <100.00%> (ø)

pysb/tests/test_bng.py
100.00% <100.00%> (ø)

pysb/core.py
88.69% <0.00%> (+<0.01%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f003879...ab6e32c. Read the comment docs.

authors: alubbock codecov-commenter codecov-io jmuhlich

repo: pysb/pysb


text:
Disable form controls on xx-events until ready

This MR addresses py-click or any other polyscript handled event so that form controls won't be usable until the interpreter is actually bootstrapped, avoiding possible errors if the user performs an action for something not bootstrapped yet.

This might be overkill, but is it worth adding the ability distinguish between elements disabled by PyScript and those disabled otherwise? I'm thinking of:
CC Number:
<input name="name" type="text"  disabled/>

I've read the terms and conditions
<input type="checkbox" id="terms" name="terms" value="yes" py-change="processTC">

If I'm understanding right, there would be no way to query for inputs disabled by Polyscript and those disabled by other means while Polyscript is loading, since both are just disabled, yes? Maybe the ones disabled by polyscript get a poly-disabled attribute or something?
May be overthinking this, I know hypotheticals aren't always helpful.

there would be no way to query for inputs disabled by Polyscript and those disabled by other means while Polyscript is loading

any prefixed event hints there was a type (interpreter) or custom type (py / mpy) to wait for ... the disabled state is not touched if elements are already disabled (it's a user intent, we don't do anything in there) and any other form that doesn't have well known prefixed events is also untouched ... + the disable disappears pretty soon so checking the state is really not useful but CSS would work wonderfully if you target input[py-click]:disabled {} as example ... I think we're good, without compromising the logic or making it more complex for unclear use cases.

@JeffersGlass if that's OK, we can revisit this after users' feedback in case your case is really desired, I hope that works!

Of course! No worries @WebReflection, we can visit it again if anyone actually asks for that feature. 👍

authors: JeffersGlass WebReflection

repo: pyscript/polyscript


text:
Bugfix: Calendar.todos() now returns todos without a status

This PR fixes #14. The fix was quite simple but seems to do the trick.

I assume it has been tested.  It's considered best practice to also write some test code that would fail without the pull request, but anyway I'll merge it in now.

I'm reverting this out as for now, ref #14

authors: mayjs tobixen

repo: python-caldav/caldav


text:
show plugin package and version

On startup, log the package and version from which
any override plugins are being loaded.

@Gregory-Pereira this is an alternative way for us to get the SHA of the downstream repo into a build log, because the SHA abbreviation will be part of the package version string for the plugins.

repo: python-wheel-build/fromager


text:
Harmonize the staging and prod services for PyPI files hosting

This primarily refactors the two terraform configurations so that they're not in the same file 😂
The only actual change is renaming the friendly name of our custom VCL in the files service.
Note: The "PyPI Staging File Hosting" service in Fastly currently takes no traffic! Next step is to migrate it to :)
Towards #95

This is ✨ almost ✨ ready, the only thing I'm concerned about is if the production and test services need different aws/gcs crews. to be, addressed :)

In order to deploy this we'll have to deactivate the hostname on the old service before deploying.
We'll throw up a status on https://status.python.org to note that test.pypi.org downloads may be temporarily impacted.

repo: python/pypi-infra


text:
Add the telegram group to development.rst

Could be cool to add the telegram group to have a place to go in case of a questions.
t.me/pythonecuador

Hey guys, would it be cool if I took this issue?

Hi!, I added a link to telegram group. Pull request coming soon :)

authors: Seburath csampedroc jiyoungsin

repo: pythonecuador/lira


text:
🛑 Groups Api is down

In 20bf4b6, Groups Api (https://groups.roblox.com/) was down:

HTTP code: 404
Response time: 76 ms

Resolved: Groups Api is back up in 05692c3 after 32 minutes.

repo: pythoniaweb/statusboom


text:
Is jetpack 6.0 for jetson agx orin supported?

I tried installing torch_tensorrt using jetpack 5.0 WORKSPACE script but it did not work for my system which is currently using jetpack 6.0 on the jetson agx orin

IIRC Jetpack 6.0 is still on 8.6 so you may be able to build an older version of TorchTRT (like 2.2) or the NGC-iGPU branch shipped in the NGC containers https://github.com/pytorch/TensorRT/tree/release/ngc/24.07_igpu. You can also use the containers for Jetson directly which already have Torch-TRT installed https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch

It would appear that the PyTorch NGC containers do not work on Jetson.

iirc you can use this container: https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch/tags. The iGPU tag I believe is targeted at jetson cc: @apbose

When attempting to run a model from the HuggingFace Transformers library (OpenVLA), this error occurs. It appears to be a flash attention related issue, but I also compiled flash attention from source:
RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
Here is the entire stack trace:
INFO:__main__:Loading processor and model from openvla/openvla-7b The model was loaded with use_flash_attention_2=True, which is deprecated and may be removed in a future release. Please use attn_implementation="flash_attention_2"instead. You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU withmodel.to('cuda'). WARNING:transformers_modules.openvla.openvla-7b.e5822cc24559b04e532f49b1c1ddb64376c1a485.modeling_prismatic:Expected transformers==4.40.1andtokenizers==0.19.1but gottransformers==4.44.1andtokenizers==0.19.1; there might be inference-time regressions due to dependency changes. If in doubt, pleaseuse the above versions. You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00,  4.42it/s] generation_config.json: 100%|██████████████████████████████████████████████████████████████████| 136/136 [00:00<00:00, 716kB/s] INFO:__main__:Model loaded on device: cuda:0 INFO:__main__:Starting benchmark We detected that you are passing past_key_valuesas a tuple and this is deprecated and will be removed in v4.43. Please use an appropriateCacheclass (https://huggingface.co/docs/transformers/v4.41.3/en/internal/generation_utils#transformers.Cache) Traceback (most recent call last):   File "/workspace/bit-VLA/inferenceV2.py", line 79, in <module>     main()   File "/workspace/bit-VLA/inferenceV2.py", line 63, in main     generation_times = run_benchmark(processor, vla, image, prompt)   File "/workspace/bit-VLA/inferenceV2.py", line 44, in run_benchmark     action = vla.predict_action(**inputs, unnorm_key="bridge_orig", do_sample=False)   File "/root/.cache/huggingface/modules/transformers_modules/openvla/openvla-7b/e5822cc24559b04e532f49b1c1ddb64376c1a485/modeling_prismatic.py", line 517, in predict_action     generated_ids = self.generate(input_ids, max_new_tokens=self.get_action_dim(unnorm_key), **kwargs)   File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context     return func(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 2024, in generate     result = self._sample(   File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 2982, in _sample     outputs = self(**model_inputs, return_dict=True)   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1552, in _wrapped_call_impl     return self._call_impl(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1561, in _call_impl     return forward_call(*args, **kwargs)   File "/root/.cache/huggingface/modules/transformers_modules/openvla/openvla-7b/e5822cc24559b04e532f49b1c1ddb64376c1a485/modeling_prismatic.py", line 404, in forward     language_model_output = self.language_model(   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1552, in _wrapped_call_impl     return self._call_impl(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1561, in _call_impl     return forward_call(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 1189, in forward     outputs = self.model(   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1552, in _wrapped_call_impl     return self._call_impl(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1561, in _call_impl     return forward_call(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 1001, in forward     layer_outputs = decoder_layer(   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1552, in _wrapped_call_impl     return self._call_impl(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1561, in _call_impl     return forward_call(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 734, in forward     hidden_states, self_attn_weights, present_key_value = self.self_attn(   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1552, in _wrapped_call_impl     return self._call_impl(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1561, in _call_impl     return forward_call(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 556, in forward     attn_output = _flash_attention_forward(   File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_flash_attention_utils.py", line 296, in _flash_attention_forward     attn_output = flash_attn_func(   File "/usr/local/lib/python3.10/dist-packages/flash_attn/flash_attn_interface.py", line 880, in flash_attn_func     return FlashAttnFunc.apply(   File "/usr/local/lib/python3.10/dist-packages/torch/autograd/function.py", line 573, in apply     return super().apply(*args, **kwargs)  # type: ignore[misc]   File "/usr/local/lib/python3.10/dist-packages/flash_attn/flash_attn_interface.py", line 546, in forward     out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = _flash_attn_forward(   File "/usr/local/lib/python3.10/dist-packages/flash_attn/flash_attn_interface.py", line 52, in _flash_attn_forward     out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = flash_attn_cuda.fwd( RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile withTORCH_USE_CUDA_DSAto enable device-side assertions.

Does flash attention have any standalone tests you can run to verify your build?

authors: areddy2022 dhruvmsheth narendasan

repo: pytorch/TensorRT


text:
Missing labels for WAV2VEC2_XLSR53

🐛 Describe the bug
Trying to follow character-level CTC segmentation for forced alignment with wav2vec2
Usually, all the models contain some sort of label manifest, like this like this
The specific XLSR53 does not include a labels manifest, making it unusable for the CTC segmentation
Versions
NA

Hi
WAV2VEC2_XLSR53 is a pre-trained on unlabeled data, but not it's not fine-tuned on any task, thus it does not have a label. (so it cannot be used for FA out-of-box)
If I remember correctly, they don't distribute the fine-tuned model weights.

If you are looking to do multilingual FA, the model from this tutorial can be used.
https://pytorch.org/audio/main/tutorials/ctc_forced_alignment_api_tutorial.html
@nateanl Let's add the model bundle, this should be helpful to many people.

authors: mthrok ntakouris

repo: pytorch/audio


text:
Anyway to finetune GPT2?

Hi, I noticed gpt2 arch is now available from #1264 . But it seems we can only train the GPT2 from zero. Is there any way to finetune the pretrained GPT2 for LM task?

Hi I am also trying to fine-tune gpt2 on lm tasks. Have you tried using the gpt-2 model checkpoint from hugging face transformers?

@MichaelZhouwang  I don't believe this will work since Fairseq has a different class structure than Pytorch transformers
nonetheless, to verify here's what I got
In [6]: custom_lm = TransformerLanguageModel.from_pretrained('./checkpoints/gpt2', 'gpt2-pytor
   ...: ch_model.bin', tokenizer='moses', bpe='gpt2')

KeyError                                  Traceback (most recent call last)
<ipython-input-6-8e8df1ba448b> in <module>----> 1 custom_lm = TransformerLanguageModel.from_pretrained('./checkpoints/gpt2', 'gpt2-pytorch_model.bin', tokenizer='moses', bpe='gpt2')./fairseq/fairseq/models/fairseq_model.py in from_pretrained(cls, model_name_or_path, checkpoint_file, data_name_or_path, **kwargs)
    216             data_name_or_path,
    217             archive_map=cls.hub_models(),
--> 218             **kwargs,
    219         )
    220         logger.info(x["args"])

./fairseq/fairseq/hub_utils.py in from_pretrained(model_name_or_path, checkpoint_file, data_name_or_path, archive_map, **kwargs)
     71     models, args, task = checkpoint_utils.load_model_ensemble_and_task(
     72         [os.path.join(model_path, cpt) for cpt in checkpoint_file.split(os.pathsep)],
---> 73         arg_overrides=kwargs,
     74     )
     75

./fairseq/fairseq/checkpoint_utils.py in load_model_ensemble_and_task(filenames, arg_overrides, task, strict)
    197         if not PathManager.exists(filename):
    198             raise IOError("Model file not found: {}".format(filename))
--> 199         state = load_checkpoint_to_cpu(filename, arg_overrides)
    200
    201         args = state["args"]

./fairseq/fairseq/checkpoint_utils.py in load_checkpoint_to_cpu(path, arg_overrides)
    167         )
    168
--> 169     args = state["args"]
    170     if arg_overrides is not None:
    171         for arg_name, arg_val in arg_overrides.items():

KeyError: 'args'

In [7]:                                                                                In [7]:                                             In [7]:

seems that someone recently implemented a transformation of HF class structure to Fairseq one,
https://github.com/pytorch/fairseq/blob/master/fairseq/models/huggingface/hf_gpt2.py
A quick and dirty  way to make GPT-2 work on this is adapt this model to load a pretrained HF model  instead of loading from a config file as in the fairseq example atm.

I've been trying to adapt Huggingface's GPT2 small model. There are several things I've done to get this to work:

Use a dict.txt where the words are in the same order as their indices in HF's gpt2 tokenizer (i.e. '!' has index 0).
Train fairseq's hf_gpt2 for one dummy epoch to gain access to its checkpoint1.pt file
Load Huggingface's pretrained gpt2 model
Use this code to adapt parameters (we need to skip the first four words in the vocab because they're reserved for bos, pad, eos, and unk, and we need to skip the first positional embedding due to padding)

for name, param in hf_pretrained_gpt2_model.named_parameters():
    if name == 'wte.weight':
        fairseq_gpt2_small['model']['decoder.model.transformer.'+name][4:,:] = param[:]
        fairseq_gpt2_small['model']['decoder.model.lm_head.weight'][:] = fairseq_gpt2_small['model']['decoder.model.transformer.'+name][:]
    elif name == 'wpe.weight':
        fairseq_gpt2_small['model']['decoder.model.transformer.'+name][1:,:] = param[:]
    elif 'decoder.model.transformer.'+name in fairseq_gpt2_small['model'].keys():
        fairseq_gpt2_small['model']['decoder.model.transformer.'+name][:] = param[:]
    else:
        print(name)

Nevertheless, it's still not working (I'm getting starting perplexities of like 10^40 -- so it's clearly doing something, but not what I want).
Has anyone been able to load Huggingface's checkpoints?

Funnily I have done the same procedure +

Editing the dictionary.py to use the same BOS symbols as HF  and giving unk and  the same token.
I ran a dummy epoch to save the model. Code works but generations are completely wrong. I put my trials on pause for now until maybe more support is given by the Fairseq folks.

I ran a dummy epoch to save the model. Code works but generations are completely wrong.

Do you mean you're also getting insane perplexity (like 10^40), or is there some other issue?
I think I fixed my issue last night :), so look forward to adaptations of Huggingface's GPT2's very soon :)
(The issue was those 4 tokens at the start of the fairseq vocab I was leaving randomly initialized -- turns out they were being initialized to relatively extreme values, so GPT2 was always 99.9999% sure the next token would be one of them.)

Trying to do the same but where do I put this code

param_optimizer = list(model.named_parameters())
no_decay = ['bias', 'LayerNorm.bias', 'LayerNorm.weight']
optimizer_grouped_parameters = [
            {'params': [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)], 'weight_decay': 0.01},
            {'params': [p for n, p in param_optimizer if any(nd in n for nd in no_decay)], 'weight_decay': 0.0}
            ]
        num_train_optimization_steps = len(train_dataloader) * args.num_train_epochs
        optimizer = OpenAIAdam(optimizer_grouped_parameters,
                               lr=args.learning_rate,
                               warmup=args.warmup_proportion,
                               max_grad_norm=args.max_grad_norm,
                               weight_decay=args.weight_decay,
                               t_total=num_train_optimization_steps)

@thak123 You do training with fairseq using their command-line interface. See their language modeling example

@dawndrain  thanks for the pointer. Will check it out.

authors: MichaelZhouwang dawndrain hadyelsahar sysu-zjw thak123

repo: pytorch/fairseq


text:
sqrt(): argument 'input' (position 1) must be Tensor, not int

🐛 Bug
When using fairseq-train with RoBERTa model after 37 CUDA OOM's I get this error:
TypeError: sqrt(): argument 'input' (position 1) must be Tensor, not int
Cmd I used
fairseq-train 'data-bin/agora' \
    --task masked_lm --criterion masked_lm \
    --arch roberta_base --sample-break-mode complete --tokens-per-sample 512 \
    --optimizer adam --adam-betas '(0.9,0.98)' --adam-eps 1e-6 --clip-norm 0.0 \
    --lr-scheduler polynomial_decay --lr 0.0005 --warmup-updates 50 \
    --total-num-update 100 \
    --dropout 0.1 --attention-dropout 0.1 --weight-decay 0.01 \
    --max-sentences 6 --update-freq 40 \
    --max-update 100 --log-format simple --log-interval 1 \
    --restore-file 'roberta/checkpoint_best.pt' --save-dir 'my_models/agora' \
    --skip-invalid-size-inputs-valid-test --reset-optimizer

pretrained model downloaded from: https://github.com/sdadas/polish-nlp-resources/releases/download/roberta/roberta.zip

Traceback I got
2020-02-21 13:40:23 | WARNING | fairseq.trainer | attempting to recover from OOM in forward/backward pass
Traceback (most recent call last):
  File "/home/jacek/anaconda3/envs/py36/bin/fairseq-train", line 11, in <module>
    load_entry_point('fairseq', 'console_scripts', 'fairseq-train')()
  File "/home/jacek/Projects/fairseq/fairseq_cli/train.py", line 307, in cli_main
    main(args)
  File "/home/jacek/Projects/fairseq/fairseq_cli/train.py", line 102, in main
    train(args, trainer, task, epoch_itr)
  File "/home/jacek/anaconda3/envs/py36/lib/python3.6/contextlib.py", line 52, in inner
    return func(*args, **kwds)
  File "/home/jacek/Projects/fairseq/fairseq_cli/train.py", line 171, in train
    log_output = trainer.train_step(samples)
  File "/home/jacek/anaconda3/envs/py36/lib/python3.6/contextlib.py", line 52, in inner
    return func(*args, **kwds)
  File "/home/jacek/Projects/fairseq/fairseq/trainer.py", line 369, in train_step
    grad_norm = self.optimizer.clip_grad_norm(self.args.clip_norm)
  File "/home/jacek/Projects/fairseq/fairseq/optim/fairseq_optimizer.py", line 91, in clip_grad_norm
    return utils.clip_grad_norm_(self.params, max_norm)
  File "/home/jacek/Projects/fairseq/fairseq/utils.py", line 243, in clip_grad_norm_
    sum(p.grad.data.norm()**2 for p in params if p.grad is not None)
TypeError: sqrt(): argument 'input' (position 1) must be Tensor, not int

Environment

fairseq Version (e.g., 1.0 or master): master
PyTorch Version (e.g., 1.0) 1.4.0
OS (e.g., Linux): Ubuntu 18.04
How you installed fairseq (pip, source): pip install --editable .
Python version: 3.6.9 Anaconda
CUDA/cuDNN version: ?
GPU models and configuration: 750 TI

Your training command works for me. Are you sure you're on pytorch 1.4?

Yes it is 1.4. I will add tomorrow data needed to reproduce it. The thing is that command works, it trains model for about a minute and then throws an exception.

You mentioned seeing numerous OOMs. Does it work if you reduce the batch size so you don't OOM?

After changing to better GPU I didn't have any OOMs, and that problem vanished as well.


text:
Loading data from two data loaders with two training data iterators

Motivation
Multitaks learning and Semi-Supervised Learning is getting so popular. So it is nice to have a feature that can load data from two uneven datasets in the sample time.
Pitch
In the training loop, right now we get data from a group iterator. I want to get data from two group iterators in multi GPU training. Right now Multi-GPU setting is failing.

Duplicate of #1746 :)

authors: myleott shamanez


text:
Training Translation models using pipeline parallism with multi nodes and multi GPUs

What is your question?
How can we training translation models using pipeline parallism with multi nodes and multi GPUs?
What have you tried?
I tried with the following command and it works well in one node and 4 gpus, just hope this issue can collect ideas about pipeline parallism training.
fairseq-train $TRAIN_DIR --arch transformer_base_pipeline_parallel ...... --distributed-world-size 4 --pipeline-model-parallel --pipeline-balance '[4,4,4,4]' --pipeline-devices '[0, 1, 2, 3]' --pipeline-chunks 4
What's your environment?

fairseq Version:0.10.0
PyTorch Version:1.6
OS (e.g., Linux):Linux
How you installed fairseq (pip, source): source
Python version:3.6
CUDA/cuDNN version:10.0
GPU models and configuration:V100

@shruti-bh are you able to add a brief section to the m2m-100 docs on how to use the pipeline parallel flags for training?

authors: myleott zherowolf


text:
What does 1 update in the log file correspond to? Does it refer to update after a pass through a batch?

What does 1 update in the log file correspond to? Does it refer to update after a pass through a batch?

What is log_interval set to in your config?

100

Then it's every 100 updates, as you should see from what is bring printed (num_updates should be increasing by 100 each time).

I might be wrong but it seems to be number of optimizer.step() calls.
One can control update frequency in optimization part of config and if one sets it to be higher than 1 training would be slower (same amount of num updates but more time)


text:
GeneralizedRCNNTransform doesn't work with four-channel inputs

When I modify the input channel of FasterRCNN from 3 to 4, GeneralizedRCNNTransform doesn't work.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py", line 47, in forward
    images, targets = self.transform(images, targets)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/transform.py", line 40, in forward
    image = self.normalize(image)
  File "/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/transform.py", line 55, in normalize
    return (image - mean[:, None, None]) / std[:, None, None]
RuntimeError: The size of tensor a (4) must match the size of tensor b (3) at non-singleton dimension 0

I solved this problem by setting the same dimension for image_std and image_mean.

That's indeed the correct solution. Let us know if you have further issues

authors: ZhiangChen fmassa

repo: pytorch/vision


text:
[Bug?]  roialign operation returning incorrect numerics

torchvision.ops.roialign is returning incorrect results for a simple test case-
# x: tensor of size (1,1,3,3)
x= torch.tensor([[[[1,2,3],[4,5,6],[7,8,9]]]], dtype=torch.float)
boxes = torch.tensor(([[0, 0, 2, 2, 0]]), dtype=torch.float)
z = torchvision.ops.roi_align(x, boxes, (2,2),sampling_ratio=1)

returns z as -
tensor([[[[7.5000, 8.5000],
          [7.5000, 8.5000]]]])

shouldn't this be
tensor([[[[3.0000 4.0000],
          [6.0000, 7.0000]]]])

Hi,
It looks like your box is degenerate, as it's y2 coordinate is smaller than the y1 coordinate (it's [roi, x1, y1, x2, y2].
In this case, the current implementation just clamps the box so that the dimension has at least a size of 1.
In detectron2, they implemented an aligned version, which in your case would just raise an error.
Given that this is a degenerate box case, I believe we should keep the current implementation as is for backwards-compatibility.
Let me know if you disagree.

authors: coderAddy fmassa


text:
to_pil_image different results depending on numpy/torch input

🐛 Describe the bug
to_pil_image has different behaviour depending on torch or numpy input. This is not documented as far as I can see. There is a note that numpy is expected to be HWC, whereas torch is expected to be CHW, but that's not relevant here.
import torch
from torchvision.transforms.functional import to_pil_image
a = torch.rand((100, 101))
print(to_pil_image(a).mode)
# L
print(to_pil_image(a.numpy()).mode)
# F

This is not documented, nor is there any warning, so errors due to this are hard to track down. The problematic code is this section:
    if isinstance(pic, torch.Tensor):
        if pic.is_floating_point() and mode != "F":
            pic = pic.mul(255).byte()

in which the torch.tensor is rescaled. Can we mirror functionality for numpy arrays? (pic * 255).round().astype(np.uint8)
Versions
all versions

Thanks for the report @rb-synth . @vfdev-5 @pmeier  any chance you remember if there's any reason for the current behaviour? Otherwise, any concern with treating this as a bugfix and align both?

@pmeier I have no issue with your suggestion, but that would break backwards compatibility, since documentation states that numpy is expected HWC and torch is expected CHW. If we convert numpy directly to torch, then they'd both be expected to have CHW shape. I'd be in favour of unifying, but realise this would be a breaking change.

Sorry, I missed that in your comment. In that case, let's do
if isinstance(input, np.ndarray):
    input = torch.from_numpy(input).permute((2, 0, 1)).contiguous()

My point here is that we should eliminate all other branching that happens based on numpy vs torch. If we get a numpy image we convert it to a tensor and after that, everything is the same.

LMK what you think on https://github.com/pytorch/vision/pull/8091.
It best not to convert from numpy to pytorch BTW, because the current implementation already does the opposite (convert the tensor to numpy) and a lot of the code assumes numpy arrays, not tensors.

authors: NicolasHug pmeier rb-synth


text:
Add MovingMNIST

PR in reference to https://github.com/pytorch/vision/issues/2676. Currently in draft since the dataset train-test split needs discussion. Inputs regarding how to better handle the train-test arguments is appreciated! @vfdev-5 @pmeier @fmassa

To complement @pmeier answer, if you could let the dataset return a single tensor of shape [num_frames, channels, height, width] it would be great

@pmeier  Somebody working on closing this PR? If no, I can help.

@avijit9 No, as far as I'm aware, no one is currently working on this. Still, lets wait how #3562 plays out. We are currently not sure which datasets we want to add.

This is a new dataset request and will be added to the new dataset API once it is stable enough. In the meantime, we will close the current PR (don't worry, the code will be reused at some point in the new dataset design to integrate it). @pmeier

Thanks @vballoli for this great contribution and sorry for the long time to get a new reply!
Unfortunately, there is a new datasets API in the works here and new datasets should follow the new API design.
That being said, rest assured, there are at least two options:

Given that you already put in the work in this PR,  a new PR adding this to the new API is more than welcome (don't hesitate to ask for help, for example @pmeier is very knowledgable). As a starting point, you can refer to this guide on how to do that.

Someone else could take this over and build on what you have done. You will receive proper credit of course. :+1:

Hope one of the two options works for you @vballoli. :ok_hand:

authors: avijit9 fmassa pmeier vballoli yassineAlouini


text:
[RFC] [NOMERGE] Registration mechanism for models

As discussed at #5088, a model registration mechanism is needed to be able to quickly list the available modules. This mechanism will be responsible for:

Registering models under specific names
Listing the names of all available models under a specific module
Loading (initializing) from its name

Examples:
Listing all available models:
>>> torchvision.models.list_models()
['alexnet', 'mobilenet_v3_large', 'mobilenet_v3_small', 'quantized_mobilenet_v3_large', ...]

Listing available models from a specific module:
>>> torchvision.models.list_models(module=torchvision.models)
['alexnet', 'mobilenet_v3_large', 'mobilenet_v3_small', ...]
>>> torchvision.models.list_models(module=torchvision.models.quantization)
['quantized_mobilenet_v3_large', ...]

Initializing a model with pre-trained weights:
>>> torchvision.models.load("quantized_mobilenet_v3_large", weights="DEFAULT")
QuantizableMobileNetV3(
  (features): Sequential(
   ....
   )
)

To register a model we use a special decorator and provide an arbitrary name:
@register_model(name="mobilenet_v3_large")
def mobilenet_v3_large(
    *, weights: Optional[MobileNet_V3_Large_Weights] = None, progress: bool = True, **kwargs: Any
) -> MobileNetV3:
    pass

@register_model(name="quantized_mobilenet_v3_large")
def mobilenet_v3_large(
    *,
    weights: Optional[Union[MobileNet_V3_Large_QuantizedWeights, MobileNet_V3_Large_Weights]] = None,
    progress: bool = True,
    quantize: bool = False,
    **kwargs: Any,
) -> QuantizableMobileNetV3:
    pass

If you are registering a method under the same as its method name, you can omit passing the name. The following call will register the method under the name "some_model":
@register_model()
def some_model():
    pass

Registering a model under an existing name throws an error but one can overwrite it using:
@register_model(name="some_model")
def some_model():
    return 1

@register_model(name="some_model", overwrite=True)
def some_model():
    return 2

@NicolasHug Thanks for the feedback.

define which weight correspond to pretrained=True so we can document this automatically

I don't think this is something that the registration mechanism should handle because the pretrained=True will go away in a few versions. I think this is still useful but probably needs to be implemented on a different annotation than register_model.

get the available weights from a model name / builder

Yes we can do that. We already have a private method that can achieve exactly this called _get_enum_from_fn() but we can make one public using the string names.

Thanks everyone for their feedback and input. I will close this PR and send a new one with all the proposed changes. This way we will keep this live RFC doc easy to read.


text:
usb.core.USBError: [Errno 22] Invalid parameter while using dev.ctrl_transfer in Windows

I'm trying to run simple GET_REPORT in HID Class example in Raspberry Pi Pico using TinyUSB:
When I try to run following script in Linux, it works perfectly and I get desired output:
import usb.core
from usb.util import *
import sys

dev = usb.core.find(idVendor=0xcafe, idProduct=0x4004)

cfg = dev[0]

class ReportType:
    GET_REPORT      = 0x01
    GET_IDLE        = 0x02
    GET_PROTOCOL    = 0x03
    SET_REPORT      = 0x09
    SET_IDLE        = 0x0A
    SET_PROTOCOL    = 0x0B

st = dev.ctrl_transfer(
                ENDPOINT_IN | CTRL_TYPE_CLASS | CTRL_RECIPIENT_INTERFACE,
                ReportType.GET_REPORT,
                0x0200,
                0,
                64
                )

print("".join(map(chr, st)))

It prints correct output in print statement while running in linux but in windows I'm getting following error:
2023-03-19 15:45:57,451 DEBUG:usb.backend.libusb1:_LibUSB.__init__(<WinDLL 'C:\Users\devan\Downloads\libusb-1.0.26-binaries\VS2015-x64\dll\libusb-1.0.dll', handle 7ffe7eb50000 at 0x1c3a133b3a0>)
2023-03-19 15:45:57,459 INFO:usb.core:find(): using backend "usb.backend.libusb1"
2023-03-19 15:45:57,459 DEBUG:usb.backend.libusb1:_LibUSB.enumerate_devices()
  File "C:\Users\devan\Downloads\src\pyusb\usb\core.py", line 1090, in ctrl_transfer
    ret = self._ctx.backend.ctrl_transfer(
  File "C:\Users\devan\Downloads\src\pyusb\usb\_debug.py", line 62, in do_trace
    return f(*args, **named_args)
  File "C:\Users\devan\Downloads\src\pyusb\usb\backend\libusb1.py", line 900, in ctrl_transfer
    ret = _check(self.lib.libusb_control_transfer(
  File "C:\Users\devan\Downloads\src\pyusb\usb\backend\libusb1.py", line 606, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 22] Invalid parameter

I've also run script with PYUSB_DEBUG=debug and it shows following logs:
2023-03-19 15:55:35,153 DEBUG:usb.backend.libusb1:_LibUSB.__init__(<WinDLL 'C:\Users\devan\Downloads\libusb-1.0.26-binaries\VS2015-x64\dll\libusb-1.0.dll', handle 7ffe8ae50000 at 0x187e0e0aef0>)
2023-03-19 15:55:35,169 INFO:usb.core:find(): using backend "usb.backend.libusb1"
2023-03-19 15:55:35,169 DEBUG:usb.backend.libusb1:_LibUSB.enumerate_devices()
2023-03-19 15:55:35,201 DEBUG:usb.backend.libusb1:_LibUSB.get_device_descriptor(<usb.backend.libusb1._Device object at 0x00000187E2D79240>)
2023-03-19 15:55:35,201 DEBUG:usb.backend.libusb1:_LibUSB.get_device_descriptor(<usb.backend.libusb1._Device object at 0x00000187E2D793C0>)
2023-03-19 15:55:35,201 DEBUG:usb.backend.libusb1:_Device._finalize_object()
2023-03-19 15:55:35,201 DEBUG:usb.backend.libusb1:_LibUSB.get_device_descriptor(<usb.backend.libusb1._Device object at 0x00000187E2D792A0>)
2023-03-19 15:55:35,209 DEBUG:usb.backend.libusb1:_Device._finalize_object()
2023-03-19 15:55:35,209 DEBUG:usb.backend.libusb1:_LibUSB.get_device_descriptor(<usb.backend.libusb1._Device object at 0x00000187E2D79420>)
2023-03-19 15:55:35,209 DEBUG:usb.backend.libusb1:_Device._finalize_object()
2023-03-19 15:55:35,209 DEBUG:usb.backend.libusb1:_LibUSB.get_device_descriptor(<usb.backend.libusb1._Device object at 0x00000187E2D79240>)
2023-03-19 15:55:35,209 DEBUG:usb.backend.libusb1:_Device._finalize_object()
2023-03-19 15:55:35,209 DEBUG:usb.backend.libusb1:_LibUSB.get_configuration_descriptor(<usb.backend.libusb1._Device object at 0x00000187E2D79240>, 0)
2023-03-19 15:55:35,209 DEBUG:usb.backend.libusb1:_LibUSB.open_device(<usb.backend.libusb1._Device object at 0x00000187E2D79240>)
2023-03-19 15:55:35,273 DEBUG:usb.backend.libusb1:_LibUSB.claim_interface(<usb.backend.libusb1._DeviceHandle object at 0x00000187E0E09630>, 0)
2023-03-19 15:55:35,273 DEBUG:usb.backend.libusb1:_LibUSB.ctrl_transfer(<usb.backend.libusb1._DeviceHandle object at 0x00000187E0E09630>, 161, 1, 512, 0, array('B', [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 1000)
2023-03-19 15:55:35,273 DEBUG:usb.backend.libusb1:
--------- Length of Packet : 64 -----------
Traceback (most recent call last):
  File "C:\Users\devan\Downloads\usb_test.py", line 19, in <module>
    st = dev.ctrl_transfer(
  File "C:\Users\devan\Downloads\src\pyusb\usb\core.py", line 1090, in ctrl_transfer
    ret = self._ctx.backend.ctrl_transfer(
  File "C:\Users\devan\Downloads\src\pyusb\usb\_debug.py", line 62, in do_trace
    return f(*args, **named_args)
  File "C:\Users\devan\Downloads\src\pyusb\usb\backend\libusb1.py", line 900, in ctrl_transfer
    ret = _check(self.lib.libusb_control_transfer(
  File "C:\Users\devan\Downloads\src\pyusb\usb\backend\libusb1.py", line 606, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 22] Invalid parameter
2023-03-19 15:55:35,287 DEBUG:usb.backend.libusb1:_LibUSB.release_interface(<usb.backend.libusb1._DeviceHandle object at 0x00000187E0E09630>, 0)
2023-03-19 15:55:35,287 DEBUG:usb.backend.libusb1:_LibUSB.close_device(<usb.backend.libusb1._DeviceHandle object at 0x00000187E0E09630>)
2023-03-19 15:55:35,287 DEBUG:usb.backend.libusb1:_Device._finalize_object()
2023-03-19 15:55:35,287 DEBUG:usb.backend.libusb1:_LibUSB._finalize_object()

I'm not getting the reason of this error

It points to some limitation of the backend: in this instance, libusb.
It might be relevant that the device is a HID: assuming you haven't messed with swapping the driver, libusb actually emulates raw USB I/O over the Microsoft Generic HID driver. While GET_REPORT would be expected to work, maybe libusb for some reason failed to detect/handle it?
In any case, you should check the libusb docs for that specific error, and also enable libusb debug info. If the issue is still unclear after doing those things, maybe try to reaching out to the libusb folks.
P.S. Closing this as it doesn't appear to be related to PyUSB (for now).

repo: pyusb/pyusb


text:
[FEAT] External Login

Is your feature request related to a problem? Please describe.
Currently there is only a Form or None Authoization provided, anyway in many cases there are more complex login pages (2 Factor) or Login via AWS / Google (Firebase) which could currently not used.
Describe the solution you'd like
Create Electron App which shows a website where we can login.
Describe alternatives you've considered
None

can be allready tested and should working only need some settings changes.

repo: q-masters/vsqlik