text:
RFC: make asdf the default version manager

Proposal
Use asdf as our default version manager. It makes it easy to handle multiple tools and their versions.
Reasoning
Many people at artsy have started using asdf. We have also added it to eigen and energy and it has been great. It would be great to extend the usage to all our repos, so we have more consistency. Some people might still want to use their version manager of choice. That's fine, we can keep all the dotfiles for both asdf and other version managers. asdf has made it easy to work with multiple tools, like node, ruby, java, and we even had it for python at some point with detect-secrets. It makes this easy, and it behaves well and consistent for all tools. nvm works differently from rbenv for example. asdf is working the same for all tools, and only requires one change in the shell's rc file.
How is this RFC resolved?
Adding .tool-versions in all our repos, and later changing our setup scripts to use asdf.

I like the idea, but I have had several issues having rbenv and asdf at the same time and I think if we go with asdf we need to go with it all the way and make a versions file for all the repos and make sure we don't require rbenv for any project.

I remember suffering from some ruby Artsy repos with rbenv and asdf, and I guess most of them use rbenv.

I use asdf and like it- i think everyone should switch - but I don't know about requiring this class of tooling for all developers.  asdf supports legacy tool version files: Just add legacy_version_file = yes to ~/.asdfrc. My initial thought is that using legacy version files and telling asdf to adapt seems preferable to me to adding another place to maintain these versions or requiring folks to change version managers.

I'm not sure we can switch Python projects over from pyenv and pip, since asdf doesn't seem to support virtualenvs. Pyenv supports both version and virtualenv management and we use the latter feature heavily. I may be wrong, but I haven't found good documentation on how to do that with asdf.

On the topic of setup scripts, I'm personally in favor of just expecting developers to perform this step themselves. I always read them before running them and have a nagging fear of something like this sneaking in and messing up my machine.

I don't think this is a good idea considering the number of engineers that have to work together on the number of repos we have. In their private tinkering projects, devs can feel free to set up anything themselves how they want to, but in this collaborative context the restrictions of agreeing on a convention can reduce chaos and annoying setup/installation problems. The setup scripts are there to free up time for actual development that would otherwise be spent on installation. In terms of developer velocity definitely better to have a very straight forward setup script in every repo and to agree on some formal conventions for the tools and versions of tools we use.

The setup scripts are there to free up time for actual development that would otherwise be spent on installation. [@kajatiger]

I have had enough bad experiences over the years that I prefer to check these installation scripts before running - it seems very common for something to get out of date with a new architecture, or broken by a version update of some dependency, and when it does it often effects other projects as well; I just don't think it is a cure-all to rely on the scripts and developers should be expected to take some responsibility for maintaining system-level tools. But I agree with your general point that it wouldn't be any easier for developers to manage this all themselves across potentially dozens of repos.
I think I would support

saying 'asdf is our preferred/supported version manager for ruby, nodejs, kubectl, x, y, and z'
updating setup scripts to reflect that
Regarding .tool-versions, either .gitignoreing the legacy version file names so people who opt to not use asdf can continue to, or enabling the legacy version file setting in asdf as part of the setup.

Thanks for all the comments!
@erikdstock I think your bullets are a good next step for this RFC if there is no major opposition. This would not be a personal mandate, if someone prefers to use a different version manager they can continue to do so, long term though asdf would be the default where applicable and they may have to adapt a bit when setting up a repo to use their preferred version manager. We can of course not make this a hard switch and still keep support around for the other version managers in repos in the short term.
For the python concern, I think if there is a language or use case that asdf does not work well enough for our purposes it is fine to not use it. It would be a good enough win if 90% of our use cases can be covered by one version manager.
So e.g. nodejs, ruby, java we can support with asdf, so those repos we could add support for.

I'm open to using asdf instead of nvm and rbenv if it works at least as well. Fewer tools are better than more tools, other things being equal.
In addition to updating the base setup script and at least one project to adopt this preference, I'd like to see the steps for cleanly switching from the previous defaults to this new default (e.g. unhooking rbenv and nvm shims), since this will affect multiple developers.
I'd also warn against trying to make the setup scripts cooperate with all possible preferences. Simpler to align them with a single default and clearly call that out.
P.S. This PR should probably update this playbook accordingly.

authors: araujobarret brainbicycle cavvia erikdstock joeyAghion kajatiger pvinis

repo: artsy/README


text:
Adds timeOnPage

As a part of the Segment audit, we decided to remove timeOnPage events for 30 seconds, 1 min, and 3 mins. We are also using this as an opportunity to move timeOnPage into cohesion.
The current time_on_page event has two properties - category for the time and message for the page path. I decided to get rid of message in favor of the context_page_owner_[x] properties. Lmk what you think!

Pulling the latest from master should resolve your conflicts.

source: gharchive/pull-request

authors: abhitip eessex

repo: artsy/cohesion

license: MIT

license_type: permissive

license_source: github-api


text:
Fix NSInvalidArgumentException in - [ARSwitchBoard(Eigen) loadLiveAuction:], line 73

Version: 2.6.3 (2016.06.24.09) | net.artsy.artsy
Stacktrace (first of multiple)
ARSwitchBoard(Eigen);loadLiveAuction:;ARSwitchboard+Eigen.m;73
-;Artsy.AuctionViewController.(viewWillAppear (Swift.Bool) -> ()).(closure #1);AuctionViewController.swift;57
-;(extension in Artsy):Interstellar.Observable.(merge  (Interstellar.Observable) -> Interstellar.Observable<(A, A1)>).(closure #2);InterstellarExtensions.swift;13
-;Artsy.AuctionSaleArtworksNetworkModel.(fetchSaleArtworks (Swift.String) -> Interstellar.Observable>).(closure #1);AuctionSaleArtworksNetworkModel.swift;23
-;Artsy.((fetchPage in _A2CC32B3077A45794C7542EE6485D1A5) (Swift.Int, forSaleID : Swift.String, alreadyFetched : [__ObjC.SaleArtwork], callback : (Interstellar.Result<[__ObjC.SaleArtwork]>) -> ()) -> ()).(closure #1);AuctionSaleArtworksNetworkModel.swift;47
Reason
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[NSURL URLByAppendingPathComponent:]: component, components, or pathExtension cannot be nil.'
Link to HockeyApp

https://rink.hockeyapp.net/manage/apps/37029/crash_reasons/137110204

Issue moved to artsy/auctions #174 via ZenHub

source: gharchive/issue

authors: artsyit ashfurrow

repo: artsy/eigen


text:
Fix crash in tiled_layer_render(_CAImageProvider*, unsigned int, unsigned int, unsigned int, unsigned int, void*)

Version: 3.1.0 (2017.01.24.16) | net.artsy.artsy
Reason
objc_msgSend() selector name: release
Link to HockeyApp

https://rink.hockeyapp.net/manage/apps/37029/crash_reasons/158321338

Closing in favour of #2091.

authors: alloy artsyit


text:
build(deps): @sentry/cli from 2.26.0 to 2.31.0

This PR resolves []
Description
Bumps sentry cli version from 2.30.2 to 2.31.0

PR Checklist

[x] I have tested my changes on iOS and Android.
[x] I hid my changes behind a feature flag, or they don't need one.
[x] I have included screenshots or videos, or I have not changed the UI.
[x] I have added tests, or my changes don't require any.
[x] I added an app state migration, or my changes do not require one.
[x] I have documented any follow-up work that this PR will require, or it does not require any.
[x] I have added a changelog entry below, or my changes do not require one.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
Changelog updates

Cross-platform user-facing changes

iOS user-facing changes

Android user-facing changes

Dev changes

Bumps sentry cli version from 2.30.2 to 2.31.0 - gkartalis

Need help with something? Have a look at our docs, or get in touch with us.

This PR contains the following changes:

Dev changes (Bumps sentry cli version from 2.30.2 to 2.31.0 - gkartalis)

  Generated by :no_entry_sign: dangerJS against dbfc2b13ca28bcc6a51b4138d6b7e498c714414e

authors: ArtsyOpenSource gkartalis


text:
[WIP] feat(FX-2907): Convert artist series filter to static button

The type of this PR is: Feature
This PR resolves FX-2907
Description
This PR converts the artist series screen from the floating "pill" filter button to a static button in the header.
I converted the ArtistSeries screen from a Flatlist to a ScrollView to deal with a weird sticky header bug I was having (see gif). Is this a terrible idea? Open to input! My thinking was that this page isn't massive so it wasn't a big deal if we render all of it (though it could get quite long if the user hits "load more artworks" a bunch - does the Flatlist in InfiniteScrollArtworksGridContainer handle that?). If converting is a bad idea, I'm sure there's a different way to handle this bug.
Sticky header w/ flatlist (buggy):

Sticky header w/ scrollview:
PR Checklist (tick all before merging)

[x] I have included screenshots or videos to illustrate my changes, or I have not changed anything that impacts the UI.
[x] I have added tests for my changes, or my changes don't require testing, or I have included a link to a separate Jira ticket covering the tests.
[x] I have documented any follow-up work that this PR will require, or it does not require any.
[x] I have added an app state migration, or my changes do not require one. (What are migrations?)
[x] I have added a CHANGELOG.yml entry or my changes do not require one.

Closing - deprioritized for saved search v1.0


text:
fix: Adjust sticky tab page header animation

The type of this PR is: Bugfix

This PR resolves CX-1985
Description
Adjusts artist header animation
PR Checklist (tick all before merging)

[ ] I have included screenshots or videos to illustrate my changes, or I have not changed anything that impacts the UI.
[ ] I have tested my changes on iOS and Android.
[ ] I have added tests for my changes, or my changes don't require testing, or I have included a link to a separate Jira ticket covering the tests.
[ ] I have added a feature flag, or my changes don't require a feature flag. (How do I add one?)
[ ] I have documented any follow-up work that this PR will require, or it does not require any.
[ ] I have added an app state migration, or my changes do not require one. (What are migrations?)
[ ] I have added a changelog entry below or my changes do not require one.

Changelog updates
Changelog updates

Cross-platform user-facing changes

iOS user-facing changes

Android user-facing changes

Dev changes

Warnings

:warning:

❌ An error occurred while validating your changelog, please make sure you provided a valid changelog.

  Generated by :no_entry_sign: dangerJS against 9ae50cb5cd60ffdec4e462e37aa45e64e2e4e159

Closing PR for inactivity - please feel free to reopen it if needed 👌


text:
Use latest fonts repos

Uses the same font techniques introduced in https://github.com/artsy/emission/pull/471 https://github.com/artsy/emission/pull/465 https://github.com/artsy/eigen/pull/2241

Turns out I can't GitHub right now

Looks like it provides a wider font for the avant garde: https://eigen-ci.s3.amazonaws.com/307/index.html
Will update snapshots.

          1 Warning

:warning:
Needs testing on a Phone if change is non-trivial

  Generated by :no_entry_sign: Danger

OK

RIP git repo size

authors: ArtsyOpenSource orta

repo: artsy/energy

license: mit

license_source: bigquery


text:
feature(NewForYou): implement artwork grid w/ artworksForUser query

The type of this PR is: Feature

This PR solves GRO-918 & GRO-920
Description
Add the new for you grid structure & associated query (which will fetch backfill artworks if the user has none).
Backfill work is in flight, per @jonallured.

Hrm the robots did not listen to that merge on green label so I'll push the button!

authors: jonallured laurabeth

repo: artsy/force


text:
⚠ sync schema after removing published

⚠ MP PR for the updated schema: https://github.com/artsy/metaphysics/pull/2605 ⚠
⚠ Gravity PR for this schema change: https://github.com/artsy/gravity/pull/13360 ⚠
⚠ Volt PR to stop using published: https://github.com/artsy/volt/pull/4528 ⚠

@pvinis - can this be merged or closed?

I'll answer here but it's for all four PRs. They should be good to go, but in order:

force and volt get deployed first
mp gets deployed second
gravity gets deployed last

This is for staging as well as production.
I'm off for a few days without my laptop, but @ansor4 and @dblandin can help if you need. They have context of this change.

Bump @pvinis

authors: damassi pvinis


text:
[GRO-834][WIP- dont merge] add Trove Rail to the Artsy homepage

This PR solves: GRO-834
To get more artwork on the homepage and available for users to check out!

Note: We're gonna want to remove Trove as an item on the carousel for logged in / logged out states which is something an admin does.
Gotta coordinate with Jenna before pushing this live.

My graphql query is supposed to pull the first 12 artworks but I only see a few, should I switch isClosed (for bidding I think) to false??


text:
Stell is taken on Rubygems, rename?

The stella gem is very old, but still taken :(
Maybe estella? I'm happy to do the work and make a release.

I'm aware it's taken as i tried to register it. Do you see it as a serious barrier to its adoption/deployment? I decided it wasn't worth changing the name but I'm open to revising that stance.

Yes. Until you have a rubygems release you have to use github references. Nobody should be doing that in production: any repo owner can force push a new tag and break everyone or worse, execute code remotely on people's servers, the repo can disappear, etc. Rubygems is a write forward system, the worst thing you can do is yank a gem.

I wasn't aware it was write forward, that does make a difference - agreed. We could use artsy-stella, stella-search, elastic-stella or another name altogether.

Want to make a call? I like the name stella, would "estella" work?
_Estella is a pretty Latin name that's sounding more and more stylish, remembered as the ward of Miss Haversham in Dickens's Great Expectations. _

I think estella would be good! We'd need to update Gravity and maybe add a link in the readme here to the new repo. I've already submitted artsy/stella to the elastic doc elastic/elasticsearch#22765 so i'll update that too once we have the new repo.

When we rename this repo, Github will keep redirects as long as we don't create another estella repo. I'll take care of it and make a gem release today.

OK great thanks a lot for picking this up - note the module naming scheme Stella:: is used throughout and also referenced in Gravity.

That should work without changes (and is easy to upgrade), we reference a tagged version.

Sure, I can also apply the Gravity changes (tomorrow) once you're done.

authors: cavvia dblock

repo: artsy/stella


text:
Rule ForbiddenComment - regexp support

I think that the rule ForbiddenComment is not flexible enough. In my case I'd like to flag every usage of TODO,TODO:,etc... but if it contains a ticket number then it should pass the check.
Noncompliant code:
// TODO remove this code later
fun deprecatedFunction() {}

Compliant code:
// TODO DTK-2019 remove this code later

The main difference for me is that I can disallow usage of TODOs unless a task is tracked in any task system.
Does it make sense to have something like that in Detekt or I should create a custom rule?

There's no need to prefix a ticket with TODO. Hence I'd just reference the ticket and leave the todo as is. A ticket implies that there is something missing.

@vanniktech makes sense. I guess I can adjust IDEA to show such comments in the same way it shows TODOs. I'll leave this ticket open if someone has other ideas, but feel free to close it.

Yes you can do that. I'm coloring links for instance:

That might be a new rule, feel free to open an issue for that. It will need reference solving which is not yet used in detekt though.

Is there any way to suppress suppress ForbiddenComment? We have several distinct todo's we dealt
with and bound them to TD tickets. I got @vanniktech 's point that we do not need "todo" in a such kinda of comments. I almost agree with that. However

I worked on a several projects with different teams in different domains and can prove it is a quite common pattern to leave "//TODO TICKET-1234" comments when they agreed upon during a code review stage. It is convenient and has IDE support out of the box.
Given (1) I have to force all team members to share the same IDE config and argue with them on why we can't use an approach they used to use for years.

I agree with @mikekudzin and @AndreiZaitcev here.
Next to values there should be allowedPatterns to handle ticket numbers.

authors: AndreiZaitcev arturbosch mikekudzin vanniktech

repo: arturbosch/detekt


text:
🛑 Plex Server is down

In a523215, Plex Server (www.lifefuckingsucks.org) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Plex Server is back up in e63d698.


text:
Upgrades nan for io.js compatibility

Also adds node 0.12 and io.js 1.1.0 and 1.2.0 to Travis configuration.

Cross-reference: iojs/io.js#456

Thanks.

Released with 0.6.0
On Mon Feb 16 2015 at 8:38:53 PM Alex Potsides notifications@github.com
wrote:

Cross-reference: iojs/io.js#456 https://github.com/iojs/io.js/issues/456
—
Reply to this email directly or view it on GitHub
https://github.com/arunoda/node-usage/pull/43#issuecomment-74521826.

authors: achingbrain arunoda

repo: arunoda/node-usage


text:
digest: avoid timeouts

see in admin emails...
[Django] zulip-prod: Timed out in digest_emails after 30 seconds processing 1 events

fix pushed - we'll see tomorrow... cc @otrajman

repo: asah/zulip


text:
Todoアプリのリファクタリング: -> を → に変更する

URL : https://github.com/asciidwango/js-primer/blob/master/source/use-case/todoapp/final/README.md

-> よりも → の方が適切な気がするため。

他に残ってなかればクローズ

地味に -> を使ってる箇所は他にもありそうだった。
(GitHubの検索だとかからない…)

私のほうで -> を使ってる箇所を探したところ、次の4か所が見つかりました。
修正しようと思うのですが、いかがでしょうか。

URL: https://github.com/asciidwango/js-primer/blob/master/source/basic/async/README.md

URL: https://github.com/asciidwango/js-primer/blob/master/source/basic/function-scope/README.md

URL: https://github.com/asciidwango/js-primer/blob/master/source/basic/implicit-coercion/README.md

次と同じようなものが4か所存在

URL: https://github.com/asciidwango/js-primer/blob/master/source/basic/prototype-object/README.md

全部 → に置き換えて問題ない気がしますね。

ご回答ありがとうございます。
対応いたします。

https://github.com/asciidwango/js-primer/blob/master/prh.yml#L596
後 prhの辞書の追加しちゃってもいい気がしますね。
文中で -> と書く場合は多分ないと思われるので、今後統一していくためにも。

ご教授ありがとうございます。
併せて対応させていただきます。

すべて書き換わったはずなのでクローズします。
@gogogodx ありがとうございました！

authors: azu gogogodx

repo: asciidwango/js-primer


text:
JctFileManagerFactoryImpl tests

Adds tests for JctFileManagerFactoryImpl
Adds a @VisibleForTestingOnly annotation internally.

Codecov Report

Merging #296 (9e37811) into main (b5ba122) will increase coverage by 0.01%.
The diff coverage is 100.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main     #296      +/-   ##
==========================================
+ Coverage   70.69%   70.71%   +0.01%
==========================================
  Files          89       89
  Lines        2744     2745       +1
  Branches      237      237
==========================================
+ Hits         1940     1941       +1
  Misses        725      725
  Partials       79       79

Impacted Files
Coverage Δ

...pes/jct/diagnostics/TracingDiagnosticListener.java
100.00% <ø> (ø)

...t/filemanagers/impl/JctFileManagerFactoryImpl.java
100.00% <ø> (ø)

...thub/ascopes/jct/junit/JavacCompilersProvider.java
100.00% <ø> (ø)

...opes/jct/filemanagers/impl/JctFileManagerImpl.java
43.65% <100.00%> (+0.45%)
:arrow_up:

authors: ascopes codecov-commenter

repo: ascopes/java-compiler-testing


text:
Add new plugin type for custom schema validators

This adds a new plugin type for custom schema validators.  We need this in order to implement support for ndarray using the Converter API.
Resolves #1012

@CagtayFabry will this work for your use case?  The URI patterns seem to nicely resolve the issues we were discussing around the scope of the custom validator.

thank you for starting this @eslavich
I will have to take a closer look but from a quick scan it looks good to me.
Regarding URI pattern association: in the example listed if I wanted to 'enable' (associate?) the MaxAreaValidator for all schemas listed unter /tags/ I would do this?
from asdf.extension import Validator
from asdf.exceptions import ValidationError

class MaxAreaValidator(Validator):
    schema_property = "max_area"
    tags = ["asdf://example.com/example-project/tags/**"]

Regarding URI pattern association: in the example listed if I wanted to 'enable' (associate?) the MaxAreaValidator for all schemas listed unter /tags/ I would do this?

Yes, that's right!

Great, with careful versioning syntax this should also effectively enable versioning of the validators :+1:
I'm quite busy these days but I will try a quick demo implementation with our weldx validators in the next 2 weeks to give feedback, would that work @eslavich ?

I'm quite busy these days but I will try a quick demo implementation with our weldx validators in the next 2 weeks to give feedback, would that work @eslavich ?

Yes, that'd be super helpful, thank you!  We might merge this in the meantime, but it's only going into an epic branch and not into master, so there will still be plenty of time to adjust the code based on your feedback.

a few notes from me @eslavich
I managed to implement the new API in a quick draft, so far it seems to work 👍
One issue that came up where I am not sure if the behavior was actually a a bug in the legacy API:
with the old API the validator would trigger on any node in the ASDF tree that had the keyword/validator-property.
Tha also meant that the validator would be in effect for non tagged nodes, here is a simple example from my test cases:
%YAML 1.1
---
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"
id: "asdf://weldx.bam.de/weldx/schemas/debug/test_unit_validator-0.1.0"

type: object
properties:
  length_prop:
    description: |
      a simple length quantity with unit validator
    tag: "asdf://weldx.bam.de/weldx/tags/units/quantity-0.1.*"
    wx_unit: "m"

  simple_prop:
    description: simple property without any tag and unit validator
    type: object
    properties:
      value:
        type: number
      units:
        type: string
    wx_units: "kg"

Previously the wx_unit validator would trigger on both the length_prop and the simple_prop
Now it only gets triggered on the length_prop, when debugging it never enters the validate function.
Is this intended? The ability to enable the validator on everything was quite useful.
Do you see any way to also implement this old behavior (I tried not setting anything for tags in Validator but no luck, I didn't look into your code yet)
On a related note: For some reason the validator gets triggered when the keyword is put on the top level of the schema (instead of next to the individual properties)
%YAML 1.1
---
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"
id: "asdf://weldx.bam.de/weldx/schemas/debug/test_unit_validator-0.1.0"

type: object
properties:
  length_prop:
    description: |
      a simple length quantity with unit validator
    tag: "asdf://weldx.bam.de/weldx/tags/units/quantity-0.1.*"
    wx_unit: "m"

  simple_prop:
    description: simple property without any tag and unit validator
    type: object
    properties:
      value:
        type: number
      units:
        type: string
    wx_units: "kg"

wx_unit:
  length_prop: "m"
  simple_prop: "kg"

The final wx_unit in this gets validated.

Thanks for the notes @CagtayFabry!

Is this intended?

Yes, we wanted to support validators scoped to certain tags, so that different developers could share the same validator name without worrying as much about collisions.

Do you see any way to also implement this old behavior

The wildcard tag pattern "**" should match anything, see https://github.com/asdf-format/asdf/blob/asdf-3.0-epic/asdf/extension/_manager.py#L291.  Thinking about this again I was a bit lazy there and should have used built-in tags like tag:yaml.org,2002:map for dict, list, etc, so that the tag pattern could select one of those specifically.  I'll make a note to fix that...

For some reason the validator gets triggered when the keyword is put on the top level of the schema

That's really strange!  I'll have to look into that one.

authors: CagtayFabry eslavich

repo: asdf-format/asdf


text:
Typos in directory writer API docs

In the directory writer API doc page, the headline is,

The Directory Reader APIs are in the aserto.directory.reader.v2.Reader service.

which seems wrong.

Good catch! Thanks for reporting it.
This should read aserto.directory.writer.v2.Writer of course.
We'll fix it shortly.

repo: aserto-dev/topaz


text:
🛑 SSO is down

In 8995a12, SSO (https://id.ashhhleyyy.dev/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: SSO is back up in bc3db64.

repo: ashhhleyyy/status-page


text:
🛑 Gitea is down

In bccdada, Gitea (https://git.ashhhleyyy.dev/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Gitea is back up in 3f649de.


text:
cannot do distributed training

Hello,
Thanks for open sourcing!
I try to run distributed training for pretraining. Without distributed training, it works fine.
I get the below error. I tried with pytorch versions 1.7.0, 1.7.1 and 1.8.0 They get below error. Version 1.9 gets ImportError: cannot import name '_new_empty_tensor' from 'torchvision.ops' **(/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torchvision/ops/__init__.py)** ``
I tried changing this line to losses.backward(retain_graph=True), it did not fix.
Let me know if you have any suggestions on how to address this issue.
Traceback (most recent call last):
  File "main.py", line 643, in <module>
    main(args)
  File "main.py", line 546, in main
    train_stats = train_one_epoch(
  File "/work/vcirik/mdetr/engine.py", line 100, in train_one_epoch
    losses.backward()
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/autograd/__init__.py", line 130, in backward
    Variable._execution_engine.run_backward(
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [1, 10]] is at version 3; expected version 2 instead. Hint: enable anomaly detection to find the operati\
on that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

Similar error happens to me with command

$ python -m torch.distributed.launch --nproc_per_node=2 --use_env main.py --dataset_config configs/pretrain.json --ema

I enabled detect_anomaly and got the error log below.

/home/toku/mdetr/engine.py:53: UserWarning: Anomaly Detection has been enabled. This mode will increase the runtime and should only be enabled for debugging.
with torch.autograd.detect_anomaly():
[W python_anomaly_mode.cpp:104] Warning: Error detected in EmbeddingBackward. Traceback of forward call that caused the error:
File "main.py", line 643, in
main(args)
File "main.py", line 558, in main
model_ema=model_ema,
File "/home/toku/mdetr/engine.py", line 69, in train_one_epoch
memory_cache = model(samples, captions, encode_and_save=True)
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 705, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/toku/mdetr/models/mdetr.py", line 143, in forward
text_attention_mask=None,
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/toku/mdetr/models/transformer.py", line 121, in forward
encoded_text = self.text_encoder(**tokenized)
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/site-packages/transformers/models/roberta/modeling_roberta.py", line 833, in forward
past_key_values_length=past_key_values_length,
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/site-packages/transformers/models/roberta/modeling_roberta.py", line 132, in forward
token_type_embeddings = self.token_type_embeddings(token_type_ids)
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 147, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File "/usr/local/lib/python3.6/site-packages/torch/nn/functional.py", line 1913, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
(function _print_stack)
Traceback (most recent call last):
File "main.py", line 643, in
main(args)
File "main.py", line 558, in main
model_ema=model_ema,
File "/home/toku/mdetr/engine.py", line 101, in train_one_epoch
losses.backward()
File "/usr/local/lib/python3.6/site-packages/torch/tensor.py", line 245, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/usr/local/lib/python3.6/site-packages/torch/autograd/init.py", line 147, in backward
allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [2, 14]] is at version 3; expected version 2 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

I tried python 3.8 and all torch versions(1.7.0,1.8.0 and 1.9.0) but got same error.
any help?
p.s.
@volkancirik About the error with torch==1.9.0,  #15  might help you.

After fixing 1.9.0 bug, I get a new error.
Traceback (most recent call last):   File "main.py", line 643, in <module>     main(args)   File "main.py", line 546, in main     train_stats = train_one_epoch(   File "/work/vcirik/mdetr/engine.py", line 79, in train_one_epoch     loss_dict.update(criterion(outputs, targets, positive_map))   File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl     return forward_call(*input, **kwargs)   File "/work/vcirik/mdetr/models/mdetr.py", line 679, in forward     losses.update(self.get_loss(loss, outputs, targets, positive_map, indices, num_boxes))   File "/work/vcirik/mdetr/models/mdetr.py", line 655, in get_loss     return loss_map[loss](outputs, targets, positive_map, indices, num_boxes, **kwargs)   File "/work/vcirik/mdetr/models/mdetr.py", line 487, in loss_labels     eos_coef[src_idx] = 1 RuntimeError: linearIndex.numel()*sliceSize*nElemBefore == value.numel()INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1623448278899/work/aten/src/ATen/native/cuda/Indexing.cu":253, please report a bug to PyTorch. number of flat\ tened indices did not match number of elements in the value tensor31 ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 64656) of binary: /work/vcirik/anaconda3/envs/mdetr/bin/python ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed

I tried pretraining with single GPU with the command below, and the error did not happen.

$ python main.py --dataset_config configs/pretrain.json --ema

If the problem is in-place operation, why the error did not happen with single GPU?

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.LongTensor [2, 14]] is at version 3; expected version 2 instead.

This may be the cause of this issue.
https://github.com/pytorch/pytorch/issues/22095
The fix will be
diff --git a/main.py b/main.py
--- a/main.py
+++ b/main.py
@@ -320,7 +320,8 @@ def main(args):
     model_ema = deepcopy(model) if args.ema else None
     model_without_ddp = model
     if args.distributed:
-        model = torch.nn.parallel.DistributedDataParallel(model, device_ids=[args.gpu], find_unused_parameters=True)
+        model = torch.nn.parallel.DistributedDataParallel(model, device_ids=[args.gpu], find_unused_parameters=True, broadcast_buffers=False)
         model_without_ddp = model.module
     n_parameters = sum(p.numel() for p in model.parameters() if p.requires_grad)
     print("number of params:", n_parameters)

The error disappeared, but I am not yet sure whether it will affect training process.

@toku-n , after broadcast_buffers=False fix I still get RuntimeError: linearIndex.numel()*sliceSize*nElemBefore == value.numel()INTERNAL ASSERT FAILED (screenshot below)
What's your torch version where single GPU distributed training is able to run.

Hi @volkancirik
I think torch==1.9.0 causes the error  RuntimeError: linearIndex.numel()*sliceSize*nElemBefore == value.numel()INTERNAL ASSERT FAILED
Here is my environment @alcinos
TITAN RTX
python 3.8.7
cuda 11.1.1
cudnn 8.2.1

and I installed python libs with
$ pip3 install torch==1.8.1+cu111 torchvision==0.9.1+cu111 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
$ pip3 install -r requirements.txt

and my pip libs are
certifi==2021.5.30
chardet==4.0.0
click==8.0.1
cloudpickle==1.6.0
cycler==0.10.0
Cython==0.29.23
filelock==3.0.12
flatbuffers==2.0
huggingface-hub==0.0.12
idna==2.10
joblib==1.0.1
kiwisolver==1.3.1
matplotlib==3.4.2
numpy==1.21.0
onnx==1.9.0
onnxruntime==1.8.1
packaging==21.0
panopticapi==0.1
Pillow==8.3.1
prettytable==2.1.0
protobuf==3.17.3
pycocotools==2.0
pyparsing==2.4.7
python-dateutil==2.8.1
PyYAML==5.4.1
regex==2021.7.6
requests==2.25.1
sacremoses==0.0.45
scipy==1.7.0
six==1.16.0
submitit==1.3.3
timm==0.4.12
tokenizers==0.10.3
torch==1.8.1+cu111
torchvision==0.9.1+cu111
tqdm==4.61.2
transformers==4.8.2
typing-extensions==3.10.0.0
urllib3==1.26.6
wcwidth==0.2.5
xmltodict==0.12.0

I changed my code with  #15 and the  broadcast_buffers=False patch I shared in the comment above.
Will this help you?

@toku-n Your env looks fine.  I wouldn't go the  broadcast_buffers=False route because I can't guarantee, from the top of my head, that it will not have unwanted side-effects. And we definitely didn't require it when training with pythorch 1.8.1
My best guess is the transformers version, try downgrading to 4.5.1 as I suggested above and see if this helps.

I can run when nproc_per_node=1 but cannot run when using multiple gpus.
TITAN RTX
python 3.8.10
cuda 11.1
cudnn 8005
pytorch 1.8.0

Log
$ CUBLAS_WORKSPACE_CONFIG=:16:8 python -m torch.distributed.launch --nproc_per_node=4 --use_env main.py --dataset_config configs/pretrain.json --ema
<de=4 --use_env main.py --dataset_config configs/pretrain.json --ema
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
| distributed init (rank 2): env://
| distributed init (rank 1): env://| distributed init (rank 3): env://

| distributed init (rank 0): env://
git:
  sha: N/A, status: clean, branch: N/A

Namespace(GT_type='separate', aux_loss=True, backbone='resnet101', batch_size=1, bbox_loss_coef=5, ce_loss_coef=1, clevr_ann_path='', clevr_img_path='', clip_max_norm=0.1, coco_path='/work/vcirik/all_data/mscoco/', combine_datasets=['flickr'], combine_datasets_val=['flickr'], contrastive_align_loss=True, contrastive_align_loss_coef=1, contrastive_loss=False, contrastive_loss_coef=0.1, contrastive_loss_hdim=64, dataset_config='configs/pretrain.json', dec_layers=6, device='cuda', dice_loss_coef=1, dilation=False, dim_feedforward=2048, dist_backend='nccl', dist_url='env://', distributed=True, do_qa=False, dropout=0.1, ema=True, ema_decay=0.9998, enc_layers=6, eos_coef=0.1, epoch_chunks=-1, epochs=40, eval=False, eval_skip=1, flickr_ann_path='mdetr_annotations', flickr_dataset_path='/work/vcirik/flickr30k/github_flickr30k_entities/', flickr_img_path='/work/vcirik/flickr30k/images/', fraction_warmup_steps=0.01, freeze_text_encoder=False, frozen_weights=None, giou_loss_coef=2, gpu=0, gqa_ann_path='mdetr_annotations', hidden_dim=256, load='', lr=0.0001, lr_backbone=1e-05, lr_drop=35, mask_loss_coef=1, mask_model='none', masks=False, modulated_lvis_ann_path='', nheads=8, no_detection=False, num_queries=100, num_workers=5, optimizer='adam', output_dir='', pass_pos_and_query=True, phrasecut_ann_path='', phrasecut_orig_ann_path='', position_embedding='sine', pre_norm=False, predict_final=False, qa_loss_coef=1, rank=0, refexp_ann_path='mdetr_annotations', refexp_dataset_name='all', remove_difficult=False, resume='', run_name='', schedule='linear_with_warmup', seed=42, set_cost_bbox=5, set_cost_class=1, set_cost_giou=2, set_loss='hungarian', split_qa_heads=False, start_epoch=0, temperature_NCE=0.07, test=False, test_type='test', text_encoder_lr=5e-05, text_encoder_type='roberta-base', vg_ann_path='', vg_img_path='/work/vcirik/all_data/gqa/images/', weight_decay=0.0001, world_size=4)
/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/__init__.py:421: UserWarning: torch.set_deterministic is deprecated and will be removed in a future release. Please use torch.use_deterministic_algorithms instead
  warnings.warn((
[W Context.cpp:70] Warning: torch.use_deterministic_algorithms is in beta, and its design and functionality may change in the future. (function operator())
/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/__init__.py:421: UserWarning: torch.set_deterministic is deprecated and will be removed in a future release. Please use torch.use_deterministic_algorithms instead
  warnings.warn((
[W Context.cpp:70] Warning: torch.use_deterministic_algorithms is in beta, and its design and functionality may change in the future. (function operator())
/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/__init__.py:421: UserWarning: torch.set_deterministic is deprecated and will be removed in a future release. Please use torch.use_deterministic_algorithms instead
  warnings.warn((
[W Context.cpp:70] Warning: torch.use_deterministic_algorithms is in beta, and its design and functionality may change in the future. (function operator())
/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/__init__.py:421: UserWarning: torch.set_deterministic is deprecated and will be removed in a future release. Please use torch.use_deterministic_algorithms instead
  warnings.warn((
[W Context.cpp:70] Warning: torch.use_deterministic_algorithms is in beta, and its design and functionality may change in the future. (function operator())
Some weights of the model checkpoint at roberta-base were not used when initializing RobertaModel: ['lm_head.dense.bias', 'lm_head.layer_norm.bias', 'lm_head.layer_norm.weight', 'lm_head.dense.weight', 'lm_head.bias', 'lm_head.decoder.weight']
- This IS expected if you are initializing RobertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of the model checkpoint at roberta-base were not used when initializing RobertaModel: ['lm_head.layer_norm.weight', 'lm_head.bias', 'lm_head.decoder.weight', 'lm_head.layer_norm.bias', 'lm_head.dense.weight', 'lm_head.dense.bias']
- This IS expected if you are initializing RobertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of the model checkpoint at roberta-base were not used when initializing RobertaModel: ['lm_head.layer_norm.weight', 'lm_head.dense.bias', 'lm_head.dense.weight', 'lm_head.layer_norm.bias', 'lm_head.bias', 'lm_head.decoder.weight']
- This IS expected if you are initializing RobertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of the model checkpoint at roberta-base were not used when initializing RobertaModel: ['lm_head.layer_norm.weight', 'lm_head.bias', 'lm_head.layer_norm.bias', 'lm_head.dense.bias', 'lm_head.dense.weight', 'lm_head.decoder.weight']
- This IS expected if you are initializing RobertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
number of params: 185160324
loading annotations into memory...
Done (t=16.14s)
creating index...
/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/utils/data/dataloader.py:474: UserWarning: This DataLoader will create 5 worker processes in total. Our suggested max number of worker in current system is 1, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  warnings.warn(_create_warning_msg(
index created!
/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/utils/data/dataloader.py:474: UserWarning: This DataLoader will create 5 worker processes in total. Our suggested max number of worker in current system is 1, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  warnings.warn(_create_warning_msg(
/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/utils/data/dataloader.py:474: UserWarning: This DataLoader will create 5 worker processes in total. Our suggested max number of worker in current system is 1, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  warnings.warn(_create_warning_msg(
/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/utils/data/dataloader.py:474: UserWarning: This DataLoader will create 5 worker processes in total. Our suggested max number of worker in current system is 1, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  warnings.warn(_create_warning_msg(
loading annotations into memory...
Done (t=0.49s)
creating index...
index created!
Start training
Starting epoch 0
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1614378083779/work/aten/src/THC/generic/THCTensorMath.cu line=29 error=715 : an illegal instruction was encountered
Traceback (most recent call last):
  File "main.py", line 682, in <module>
    main(args)
  File "main.py", line 582, in main
    train_stats = train_one_epoch(
  File "/work/vcirik/mdetr/engine.py", line 68, in train_one_epoch
    memory_cache = model(samples, captions, encode_and_save=True)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 705, in forward
    output = self.module(*inputs[0], **kwargs[0])
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/vcirik/mdetr/models/mdetr.py", line 129, in forward
    features, pos = self.backbone(samples)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/vcirik/mdetr/models/backbone.py", line 169, in forward
    xs = self[0](tensor_list)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/vcirik/mdetr/models/backbone.py", line 74, in forward
    xs = self.body(tensor_list.tensors)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torchvision/models/_utils.py", line 63, in forward
    x = module(x)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/container.py", line 119, in forward
    input = module(input)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torchvision/models/resnet.py", line 128, in forward
    out = self.conv2(out)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 399, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 395, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: cuda runtime error (715) : an illegal instruction was encountered at /opt/conda/conda-bld/pytorch_1614378083779/work/aten/src/THC/generic/THCTensorMath.cu:29
terminate called after throwing an instance of 'c10::Error'
  what():  CUDA error: an illegal instruction was encountered
Exception raised from create_event_internal at /opt/conda/conda-bld/pytorch_1614378083779/work/c10/cuda/CUDACachingAllocator.cpp:733 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7ff3253662f2 in /work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x5b (0x7ff32536367b in /work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #2: c10::cuda::CUDACachingAllocator::raw_delete(void*) + 0x809 (0x7ff3255bf219 in /work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/lib/libc10_cuda.so)
frame #3: c10::TensorImpl::release_resources() + 0x54 (0x7ff32534e3a4 in /work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #4: <unknown function> + 0x6e0dda (0x7ff37c2c2dda in /work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
frame #5: <unknown function> + 0x6e0e71 (0x7ff37c2c2e71 in /work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
frame #6: <unknown function> + 0x1932c6 (0x55c734ddf2c6 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #7: <unknown function> + 0x15878b (0x55c734da478b in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #8: <unknown function> + 0xe81c4 (0x55c734d341c4 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #9: <unknown function> + 0x15893b (0x55c734da493b in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #10: <unknown function> + 0x193141 (0x55c734ddf141 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #11: <unknown function> + 0x15878b (0x55c734da478b in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #12: <unknown function> + 0xe81c4 (0x55c734d341c4 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #13: <unknown function> + 0x15893b (0x55c734da493b in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #14: <unknown function> + 0x193141 (0x55c734ddf141 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #15: <unknown function> + 0x15878b (0x55c734da478b in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #16: <unknown function> + 0xe81c4 (0x55c734d341c4 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #17: <unknown function> + 0x15893b (0x55c734da493b in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #18: <unknown function> + 0x193141 (0x55c734ddf141 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #19: <unknown function> + 0x15878b (0x55c734da478b in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #20: <unknown function> + 0xe81c4 (0x55c734d341c4 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #21: <unknown function> + 0x15893b (0x55c734da493b in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #22: <unknown function> + 0x193141 (0x55c734ddf141 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #23: <unknown function> + 0x1592ac (0x55c734da52ac in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #24: <unknown function> + 0x158e77 (0x55c734da4e77 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #25: <unknown function> + 0x158e60 (0x55c734da4e60 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #26: <unknown function> + 0x176057 (0x55c734dc2057 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #27: PyDict_SetItemString + 0x61 (0x55c734de33c1 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #28: PyImport_Cleanup + 0x9d (0x55c734e21aad in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #29: Py_FinalizeEx + 0x79 (0x55c734e53a49 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #30: Py_RunMain + 0x183 (0x55c734e55893 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #31: Py_BytesMain + 0x39 (0x55c734e55ca9 in /work/vcirik/anaconda3/envs/mdetr/bin/python)
frame #32: __libc_start_main + 0xf5 (0x7ff3c0c50c05 in /lib64/libc.so.6)
frame #33: <unknown function> + 0x1e21c7 (0x55c734e2e1c7 in /work/vcirik/anaconda3/envs/mdetr/bin/python)

Killing subprocess 17070
Killing subprocess 17071
Killing subprocess 17072
Killing subprocess 17073
Traceback (most recent call last):
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/distributed/launch.py", line 340, in <module>
    main()
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/distributed/launch.py", line 326, in main
    sigkill_handler(signal.SIGTERM, None)  # not coming back
  File "/work/vcirik/anaconda3/envs/mdetr/lib/python3.8/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
    raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/work/vcirik/anaconda3/envs/mdetr/bin/python', '-u', 'main.py', '--dataset_config', 'configs/pretrain.json', '--ema']' died with <Signals.SIGABRT: 6>.

Hi @alcinos,
The error has gone with transformers==4.5.1 !
Now I don't need the patch of broadcast_buffers=False.
Thank you so much !

@alcinos, thank you for your suggestion, but the error seems to have remained... I ran finetuning on the "all" split and got:
��G�error�X6Traceback (most recent call last):
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/submitit/core/submission.py", line 53, in process_job
    result = delayed.result()
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/submitit/core/utils.py", line 128, in result
    self._result = self.function(*self.args, **self.kwargs)
  File "run_with_submitit.py", line 98, in __call__
    detection.main(self.args)
  File "/home/pchelintsev/MDETR/mdetr/main.py", line 546, in main
    train_stats = train_one_epoch(
  File "/home/pchelintsev/MDETR/mdetr/engine.py", line 73, in train_one_epoch
    loss_dict.update(criterion(outputs, targets, positive_map))
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/pchelintsev/MDETR/mdetr/models/mdetr.py", line 679, in forward
    losses.update(self.get_loss(loss, outputs, targets, positive_map, indices, num_boxes))
  File "/home/pchelintsev/MDETR/mdetr/models/mdetr.py", line 655, in get_loss
    return loss_map[loss](outputs, targets, positive_map, indices, num_boxes, **kwargs)
  File "/home/pchelintsev/MDETR/mdetr/models/mdetr.py", line 487, in loss_labels
    eos_coef[src_idx] = 1
RuntimeError: linearIndex.numel()*sliceSize*nElemBefore == value.numel()INTERNAL ASSERT FAILED at "/pytorch/aten/src/ATen/native/cuda/Indexing.cu":253, please report a bug to PyTorch. number of flattened indices did not match number of elements in the value tensor51

in 10562_0_result.pkl file in experiments/ directory.

@TopCoder2K This error is pretty uninformative. Please try debugging on cpu first (--device cpu) to see if you get more information.

@alcinos, thank you for the advice! I got the following output in job_number_0_log.err:
/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/__init__.py:471: UserWarning: torch.set_deterministic is deprecated and will be removed in a future release. Please use torch.use_deterministic_algorithms instead
  warnings.warn((
/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at  /pytorch/aten/src/ATen/native/BinaryOps.cpp:467.)
  return torch.floor_divide(self, other)
submitit ERROR (2021-09-18 16:41:20,879) - Submitted job triggered an exception
Traceback (most recent call last):
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/submitit/core/_submit.py", line 11, in <module>
    submitit_main()
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/submitit/core/submission.py", line 71, in submitit_main
    process_job(args.folder)
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/submitit/core/submission.py", line 64, in process_job
    raise error
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/submitit/core/submission.py", line 53, in process_job
    result = delayed.result()
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/submitit/core/utils.py", line 128, in result
    self._result = self.function(*self.args, **self.kwargs)
  File "run_with_submitit.py", line 98, in __call__
    detection.main(self.args)
  File "/home/pchelintsev/MDETR/mdetr/main.py", line 546, in main
    train_stats = train_one_epoch(
  File "/home/pchelintsev/MDETR/mdetr/engine.py", line 69, in train_one_epoch
    outputs = model(samples, captions, encode_and_save=False, memory_cache=memory_cache)
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/pchelintsev/MDETR/mdetr/models/mdetr.py", line 154, in forward
    hs = self.transformer(
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/pchelintsev/MDETR/mdetr/models/transformer.py", line 168, in forward
    hs = self.decoder(
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/pchelintsev/MDETR/mdetr/models/transformer.py", line 232, in forward
    output = layer(
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/pchelintsev/MDETR/mdetr/models/transformer.py", line 448, in forward
    return self.forward_post(
  File "/home/pchelintsev/MDETR/mdetr/models/transformer.py", line 383, in forward_post
    tgt2 = self.cross_attn_image(
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/modules/activation.py", line 1031, in forward
    attn_output, attn_output_weights = F.multi_head_attention_forward(
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/functional.py", line 5082, in multi_head_attention_forward
    attn_output, attn_output_weights = _scaled_dot_product_attention(q, k, v, attn_mask, dropout_p)
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/functional.py", line 4830, in _scaled_dot_product_attention
    attn = dropout(attn, p=dropout_p)
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/functional.py", line 1168, in dropout
    return _VF.dropout_(input, p, training) if inplace else _VF.dropout(input, p, training)
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
    _error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 28712) is killed by signal: Killed.

What can be wrong with the dataloader?

The worker may have been called because of the host is out of ram.
I’d suggest running locally first:
python main.py --dataset_config configs/gqa.json --ema --epochs 125 --epoch_chunks 25 --do_qa --split_qa_heads --lr_drop 150 --load pretrained_resnet101_checkpoint.pth  --batch_size 4 --no_aux_loss --qa_loss_coef 25 --lr 1.4e-4 --lr_backbone 1.4e-5 --text_encoder_lr 7e-5 --device cpu --num_workers 0

The worker may have been called because of the host is out of ram.
I’d suggest running locally first:
python main.py --dataset_config configs/gqa.json --ema --epochs 125 --epoch_chunks 25 --do_qa --split_qa_heads --lr_drop 150 --load pretrained_resnet101_checkpoint.pth  --batch_size 4 --no_aux_loss --qa_loss_coef 25 --lr 1.4e-4 --lr_backbone 1.4e-5 --text_encoder_lr 7e-5 --device cpu --num_workers 0

Hmmm, I also encountered on that advice in pytorch issues, so I checked the RAM usage with free -m during the job was running and found out that 60 of 64 Gbs were used. Suspiciously...
I also tried to run the command with --num_workers 0 as you suggested, here the full output:
Not using distributed mode
git:
  sha: dda257d51a9944ee3e4201e7e52e50e5f9faec60, status: has uncommited changes, branch: main

Namespace(aux_loss=False, backbone='resnet101', batch_size=4, bbox_loss_coef=5, ce_loss_coef=1, clevr_ann_path='', clevr_img_path='', clip_max_norm=0.1, coco_path='', combine_datasets=['gqa'], combine_datasets_val=['gqa'], contrastive_align_loss=True, contrastive_align_loss_coef=1, contrastive_loss=False, contrastive_loss_coef=0.1, contrastive_loss_hdim=64, dataset_config='configs/gqa.json', dec_layers=6, device='cpu', dice_loss_coef=1, dilation=False, dim_feedforward=2048, dist_url='env://', distributed=False, do_qa=True, dropout=0.1, ema=True, ema_decay=0.9998, enc_layers=6, eos_coef=0.1, epoch_chunks=25, epochs=125, eval=False, eval_skip=1, fraction_warmup_steps=0.01, freeze_text_encoder=False, frozen_weights=None, giou_loss_coef=2, gqa_ann_path='mdetr_annotations/', gqa_split_type='all', hidden_dim=256, load='pretrained_resnet101_checkpoint.pth', lr=0.00014, lr_backbone=1.4e-05, lr_drop=150, mask_loss_coef=1, mask_model='none', masks=False, modulated_lvis_ann_path='', nheads=8, no_detection=False, num_queries=100, num_workers=0, optimizer='adam', output_dir='', pass_pos_and_query=True, phrasecut_ann_path='', phrasecut_orig_ann_path='', position_embedding='sine', pre_norm=False, predict_final=False, qa_loss_coef=25.0, remove_difficult=False, resume='', run_name='', schedule='linear_with_warmup', seed=42, set_cost_bbox=5, set_cost_class=1, set_cost_giou=2, set_loss='hungarian', split_qa_heads=True, start_epoch=0, temperature_NCE=0.07, test=False, test_type='test', text_encoder_lr=7e-05, text_encoder_type='roberta-base', vg_ann_path='', vg_img_path='images/', weight_decay=0.0001, world_size=1)
/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/__init__.py:471: UserWarning: torch.set_deterministic is deprecated and will be removed in a future release. Please use torch.use_deterministic_algorithms instead
  warnings.warn((

number of params: 185879918
loading annotations into memory...
^[[A^[[BDone (t=233.60s)
creating index...
index created!
loading annotations into memory...
Done (t=45.16s)
creating index...
index created!
Splitting the training set into {args.epoch_chunks} of size approximately  652688
loading annotations into memory...
Done (t=0.45s)
creating index...
index created!
loading from pretrained_resnet101_checkpoint.pth
Start training
Starting epoch 0, sub_epoch 0
/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at  /pytorch/aten/src/ATen/native/BinaryOps.cpp:467.)
  return torch.floor_divide(self, other)
Epoch: [0]  [     0/163172]  eta: 23 days, 4:50:57  lr: 0.000140  lr_backbone: 0.000014  lr_text_encoder: 0.000000  loss: 553.8007 (553.8007)  loss_ce: 2.4964 (2.4964)  loss_bbox: 0.3178 (0.3178)  loss_giou: 0.5582 (0.5582)  loss_contrastive_align: 1.5884 (1.5884)  loss_answer_type: 40.6496 (40.6496)  loss_answer_obj: 0.0000 (0.0000)  loss_answer_attr: 132.8757 (132.8757)  loss_answer_rel: 204.5815 (204.5815)  loss_answer_global: 0.0000 (0.0000)  loss_answer_cat: 170.7331 (170.7331)  loss_ce_unscaled: 2.4964 (2.4964)  loss_bbox_unscaled: 0.0636 (0.0636)  loss_giou_unscaled: 0.2791 (0.2791)  cardinality_error_unscaled: 1.0000 (1.0000)  loss_contrastive_align_unscaled: 1.5884 (1.5884)  loss_answer_type_unscaled: 1.6260 (1.6260)  accuracy_answer_type_unscaled: 0.0000 (0.0000)  loss_answer_obj_unscaled: 0.0000 (0.0000)  accuracy_answer_obj_unscaled: 1.0000 (1.0000)  loss_answer_attr_unscaled: 5.3150 (5.3150)  accuracy_answer_attr_unscaled: 0.0000 (0.0000)  loss_answer_rel_unscaled: 8.1833 (8.1833)  accuracy_answer_rel_unscaled: 0.0000 (0.0000)  loss_answer_global_unscaled: 0.0000 (0.0000)  accuracy_answer_global_unscaled: 1.0000 (1.0000)  loss_answer_cat_unscaled: 6.8293 (6.8293)  accuracy_answer_cat_unscaled: 0.0000 (0.0000)  accuracy_answer_total_unscaled: 0.0000 (0.0000)  time: 12.2855  data: 0.2388  max mem: 0
Epoch: [0]  [    10/163172]  eta: 30 days, 5:15:36  lr: 0.000140  lr_backbone: 0.000014  lr_text_encoder: 0.000000  loss: 406.9724 (418.6352)  loss_ce: 2.1498 (2.1541)  loss_bbox: 0.2972 (0.2831)  loss_giou: 0.4867 (0.4779)  loss_contrastive_align: 1.3373 (1.4061)  loss_answer_type: 39.5703 (38.9707)  loss_answer_obj: 12.8333 (16.3485)  loss_answer_attr: 153.8699 (140.3258)  loss_answer_rel: 187.6746 (186.9401)  loss_answer_global: 0.0000 (0.0000)  loss_answer_cat: 0.0000 (31.7289)  loss_ce_unscaled: 2.1498 (2.1541)  loss_bbox_unscaled: 0.0594 (0.0566)  loss_giou_unscaled: 0.2433 (0.2390)  cardinality_error_unscaled: 1.0000 (0.9091)  loss_contrastive_align_unscaled: 1.3373 (1.4061)  loss_answer_type_unscaled: 1.5828 (1.5588)  accuracy_answer_type_unscaled: 0.2500 (0.2727)  loss_answer_obj_unscaled: 0.5133 (0.6539)  accuracy_answer_obj_unscaled: 1.0000 (0.7273)  loss_answer_attr_unscaled: 6.1548 (5.6130)  accuracy_answer_attr_unscaled: 0.0000 (0.0909)  loss_answer_rel_unscaled: 7.5070 (7.4776)  accuracy_answer_rel_unscaled: 0.0000 (0.0000)  loss_answer_global_unscaled: 0.0000 (0.0000)  accuracy_answer_global_unscaled: 1.0000 (1.0000)  loss_answer_cat_unscaled: 0.0000 (1.2692)  accuracy_answer_cat_unscaled: 1.0000 (0.8182)  accuracy_answer_total_unscaled: 0.0000 (0.0000)  time: 16.0021  data: 0.0998  max mem: 0
Epoch: [0]  [    20/163172]  eta: 28 days, 19:04:06  lr: 0.000140  lr_backbone: 0.000014  lr_text_encoder: 0.000000  loss: 376.5505 (392.1563)  loss_ce: 2.1498 (2.3245)  loss_bbox: 0.2712 (0.2893)  loss_giou: 0.4867 (0.5187)  loss_contrastive_align: 1.4424 (1.5074)  loss_answer_type: 36.7327 (36.6190)  loss_answer_obj: 12.8333 (15.8707)  loss_answer_attr: 136.2527 (106.2132)  loss_answer_rel: 183.5641 (184.4620)  loss_answer_global: 0.0000 (11.7262)  loss_answer_cat: 0.0000 (32.6253)  loss_ce_unscaled: 2.1498 (2.3245)  loss_bbox_unscaled: 0.0542 (0.0579)  loss_giou_unscaled: 0.2433 (0.2594)  cardinality_error_unscaled: 1.0000 (0.9286)  loss_contrastive_align_unscaled: 1.4424 (1.5074)  loss_answer_type_unscaled: 1.4693 (1.4648)  accuracy_answer_type_unscaled: 0.5000 (0.3929)  loss_answer_obj_unscaled: 0.5133 (0.6348)  accuracy_answer_obj_unscaled: 1.0000 (0.6667)  loss_answer_attr_unscaled: 5.4501 (4.2485)  accuracy_answer_attr_unscaled: 0.0000 (0.2857)  loss_answer_rel_unscaled: 7.3426 (7.3785)  accuracy_answer_rel_unscaled: 0.0000 (0.0159)  loss_answer_global_unscaled: 0.0000 (0.4690)  accuracy_answer_global_unscaled: 1.0000 (0.9048)  loss_answer_cat_unscaled: 0.0000 (1.3050)  accuracy_answer_cat_unscaled: 1.0000 (0.8095)  accuracy_answer_total_unscaled: 0.0000 (0.0119)  time: 15.3968  data: 0.0852  max mem: 0

And it continues to work whereas when I didn't use --num_workers 0 job failed during Epoch: [0]  [    0/163172] not reaching Epoch: [0]  [    10/163172].

Hmmmmm, I really confused... It seems I have different mistakes simulteneously, that's why I see different output every time...
I've just rerun the command python main.py --dataset_config configs/gqa.json --ema --epochs 125 --epoch_chunks 25 --do_qa --split_qa_heads --lr_drop 150 --load pretrained_resnet101_checkpoint.pth  --batch_size 4 --no_aux_loss --qa_loss_coef 25 --lr 1.4e-4 --lr_backbone 1.4e-5 --text_encoder_lr 7e-5 --device cpu --num_workers 0 and got
Not using distributed mode
git:
  sha: dda257d51a9944ee3e4201e7e52e50e5f9faec60, status: has uncommited changes, branch: main

Namespace(aux_loss=False, backbone='resnet101', batch_size=4, bbox_loss_coef=5, ce_loss_coef=1, clevr_ann_path='', clevr_img_path='', clip_max_norm=0.1, coco_path='', combine_datasets=['gqa'], combine_datasets_val=['gqa'], contrastive_align_loss=True, contrastive_align_loss_coef=1, contrastive_loss=False, contrastive_loss_coef=0.1, contrastive_loss_hdim=64, dataset_config='configs/gqa.json', dec_layers=6, device='cpu', dice_loss_coef=1, dilation=False, dim_feedforward=2048, dist_url='env://', distributed=False, do_qa=True, dropout=0.1, ema=True, ema_decay=0.9998, enc_layers=6, eos_coef=0.1, epoch_chunks=25, epochs=125, eval=False, eval_skip=1, fraction_warmup_steps=0.01, freeze_text_encoder=False, frozen_weights=None, giou_loss_coef=2, gqa_ann_path='mdetr_annotations/', gqa_split_type='all', hidden_dim=256, load='pretrained_resnet101_checkpoint.pth', lr=0.00014, lr_backbone=1.4e-05, lr_drop=150, mask_loss_coef=1, mask_model='none', masks=False, modulated_lvis_ann_path='', nheads=8, no_detection=False, num_queries=100, num_workers=0, optimizer='adam', output_dir='', pass_pos_and_query=True, phrasecut_ann_path='', phrasecut_orig_ann_path='', position_embedding='sine', pre_norm=False, predict_final=False, qa_loss_coef=25.0, remove_difficult=False, resume='', run_name='', schedule='linear_with_warmup', seed=42, set_cost_bbox=5, set_cost_class=1, set_cost_giou=2, set_loss='hungarian', split_qa_heads=True, start_epoch=0, temperature_NCE=0.07, test=False, test_type='test', text_encoder_lr=7e-05, text_encoder_type='roberta-base', vg_ann_path='', vg_img_path='images/', weight_decay=0.0001, world_size=1)
/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/__init__.py:471: UserWarning: torch.set_deterministic is deprecated and will be removed in a future release. Please use torch.use_deterministic_algorithms instead
  warnings.warn((
number of params: 185879918
loading annotations into memory...
Done (t=216.72s)
creating index...
index created!
loading annotations into memory...
Done (t=42.99s)
creating index...
index created!
Splitting the training set into {args.epoch_chunks} of size approximately  652688
loading annotations into memory...
Done (t=0.44s)
creating index...
index created!
loading from pretrained_resnet101_checkpoint.pth
Start training
Starting epoch 0, sub_epoch 0
/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at  /pytorch/aten/src/ATen/native/BinaryOps.cpp:467.)
  return torch.floor_divide(self, other)
Epoch: [0]  [     0/163172]  eta: 26 days, 3:45:42  lr: 0.000140  lr_backbone: 0.000014  lr_text_encoder: 0.000000  loss: 553.8007 (553.8007)  loss_ce: 2.4964 (2.4964)  loss_bbox: 0.3178 (0.3178)  loss_giou: 0.5582 (0.5582)  loss_contrastive_align: 1.5884 (1.5884)  loss_answer_type: 40.6496 (40.6496)  loss_answer_obj: 0.0000 (0.0000)  loss_answer_attr: 132.8757 (132.8757)  loss_answer_rel: 204.5815 (204.5815)  loss_answer_global: 0.0000 (0.0000)  loss_answer_cat: 170.7331 (170.7331)  loss_ce_unscaled: 2.4964 (2.4964)  loss_bbox_unscaled: 0.0636 (0.0636)  loss_giou_unscaled: 0.2791 (0.2791)  cardinality_error_unscaled: 1.0000 (1.0000)  loss_contrastive_align_unscaled: 1.5884 (1.5884)  loss_answer_type_unscaled: 1.6260 (1.6260)  accuracy_answer_type_unscaled: 0.0000 (0.0000)  loss_answer_obj_unscaled: 0.0000 (0.0000)  accuracy_answer_obj_unscaled: 1.0000 (1.0000)  loss_answer_attr_unscaled: 5.3150 (5.3150)  accuracy_answer_attr_unscaled: 0.0000 (0.0000)  loss_answer_rel_unscaled: 8.1833 (8.1833)  accuracy_answer_rel_unscaled: 0.0000 (0.0000)  loss_answer_global_unscaled: 0.0000 (0.0000)  accuracy_answer_global_unscaled: 1.0000 (1.0000)  loss_answer_cat_unscaled: 6.8293 (6.8293)  accuracy_answer_cat_unscaled: 0.0000 (0.0000)  accuracy_answer_total_unscaled: 0.0000 (0.0000)  time: 13.8501  data: 0.6887  max mem: 0
Epoch: [0]  [    10/163172]  eta: 29 days, 3:12:39  lr: 0.000140  lr_backbone: 0.000014  lr_text_encoder: 0.000000  loss: 406.9724 (418.6352)  loss_ce: 2.1498 (2.1541)  loss_bbox: 0.2972 (0.2831)  loss_giou: 0.4867 (0.4779)  loss_contrastive_align: 1.3373 (1.4061)  loss_answer_type: 39.5703 (38.9707)  loss_answer_obj: 12.8333 (16.3485)  loss_answer_attr: 153.8699 (140.3258)  loss_answer_rel: 187.6746 (186.9401)  loss_answer_global: 0.0000 (0.0000)  loss_answer_cat: 0.0000 (31.7289)  loss_ce_unscaled: 2.1498 (2.1541)  loss_bbox_unscaled: 0.0594 (0.0566)  loss_giou_unscaled: 0.2433 (0.2390)  cardinality_error_unscaled: 1.0000 (0.9091)  loss_contrastive_align_unscaled: 1.3373 (1.4061)  loss_answer_type_unscaled: 1.5828 (1.5588)  accuracy_answer_type_unscaled: 0.2500 (0.2727)  loss_answer_obj_unscaled: 0.5133 (0.6539)  accuracy_answer_obj_unscaled: 1.0000 (0.7273)  loss_answer_attr_unscaled: 6.1548 (5.6130)  accuracy_answer_attr_unscaled: 0.0000 (0.0909)  loss_answer_rel_unscaled: 7.5070 (7.4776)  accuracy_answer_rel_unscaled: 0.0000 (0.0000)  loss_answer_global_unscaled: 0.0000 (0.0000)  accuracy_answer_global_unscaled: 1.0000 (1.0000)  loss_answer_cat_unscaled: 0.0000 (1.2692)  accuracy_answer_cat_unscaled: 1.0000 (0.8182)  accuracy_answer_total_unscaled: 0.0000 (0.0000)  time: 15.4274  data: 0.1400  max mem: 0
Epoch: [0]  [    20/163172]  eta: 27 days, 14:41:18  lr: 0.000140  lr_backbone: 0.000014  lr_text_encoder: 0.000000  loss: 376.5505 (392.1563)  loss_ce: 2.1498 (2.3245)  loss_bbox: 0.2712 (0.2893)  loss_giou: 0.4867 (0.5187)  loss_contrastive_align: 1.4424 (1.5074)  loss_answer_type: 36.7327 (36.6190)  loss_answer_obj: 12.8333 (15.8707)  loss_answer_attr: 136.2527 (106.2132)  loss_answer_rel: 183.5641 (184.4620)  loss_answer_global: 0.0000 (11.7262)  loss_answer_cat: 0.0000 (32.6253)  loss_ce_unscaled: 2.1498 (2.3245)  loss_bbox_unscaled: 0.0542 (0.0579)  loss_giou_unscaled: 0.2433 (0.2594)  cardinality_error_unscaled: 1.0000 (0.9286)  loss_contrastive_align_unscaled: 1.4424 (1.5074)  loss_answer_type_unscaled: 1.4693 (1.4648)  accuracy_answer_type_unscaled: 0.5000 (0.3929)  loss_answer_obj_unscaled: 0.5133 (0.6348)  accuracy_answer_obj_unscaled: 1.0000 (0.6667)  loss_answer_attr_unscaled: 5.4501 (4.2485)  accuracy_answer_attr_unscaled: 0.0000 (0.2857)  loss_answer_rel_unscaled: 7.3426 (7.3785)  accuracy_answer_rel_unscaled: 0.0000 (0.0159)  loss_answer_global_unscaled: 0.0000 (0.4690)  accuracy_answer_global_unscaled: 1.0000 (0.9048)  loss_answer_cat_unscaled: 0.0000 (1.3050)  accuracy_answer_cat_unscaled: 1.0000 (0.8095)  accuracy_answer_total_unscaled: 0.0000 (0.0119)  time: 14.6610  data: 0.0850  max mem: 0
Epoch: [0]  [    30/163172]  eta: 27 days, 8:44:25  lr: 0.000140  lr_backbone: 0.000014  lr_text_encoder: 0.000000  loss: 323.8377 (359.1600)  loss_ce: 2.3897 (2.5281)  loss_bbox: 0.2712 (0.2944)  loss_giou: 0.4717 (0.5140)  loss_contrastive_align: 1.5241 (1.5773)  loss_answer_type: 34.4717 (35.7402)  loss_answer_obj: 15.2606 (14.9681)  loss_answer_attr: 121.9345 (104.8293)  loss_answer_rel: 166.2059 (164.8905)  loss_answer_global: 0.0000 (11.7170)  loss_answer_cat: 0.0000 (22.1010)  loss_ce_unscaled: 2.3897 (2.5281)  loss_bbox_unscaled: 0.0542 (0.0589)  loss_giou_unscaled: 0.2358 (0.2570)  cardinality_error_unscaled: 1.0000 (1.2016)  loss_contrastive_align_unscaled: 1.5241 (1.5773)  loss_answer_type_unscaled: 1.3789 (1.4296)  accuracy_answer_type_unscaled: 0.5000 (0.4032)  loss_answer_obj_unscaled: 0.6104 (0.5987)  accuracy_answer_obj_unscaled: 1.0000 (0.6935)  loss_answer_attr_unscaled: 4.8774 (4.1932)  accuracy_answer_attr_unscaled: 0.5000 (0.3387)  loss_answer_rel_unscaled: 6.6482 (6.5956)  accuracy_answer_rel_unscaled: 0.0000 (0.1398)  loss_answer_global_unscaled: 0.0000 (0.4687)  accuracy_answer_global_unscaled: 1.0000 (0.9032)  loss_answer_cat_unscaled: 0.0000 (0.8840)  accuracy_answer_cat_unscaled: 1.0000 (0.8710)  accuracy_answer_total_unscaled: 0.0000 (0.0403)  time: 13.9777  data: 0.0829  max mem: 0
Epoch: [0]  [    40/163172]  eta: 27 days, 14:40:18  lr: 0.000140  lr_backbone: 0.000014  lr_text_encoder: 0.000000  loss: 312.4061 (340.1148)  loss_ce: 2.3961 (2.5253)  loss_bbox: 0.3259 (0.3168)  loss_giou: 0.4626 (0.5189)  loss_contrastive_align: 1.3347 (1.5488)  loss_answer_type: 35.9053 (35.5551)  loss_answer_obj: 16.3921 (14.5897)  loss_answer_attr: 110.3442 (100.5373)  loss_answer_rel: 140.3934 (150.9077)  loss_answer_global: 0.0000 (8.8592)  loss_answer_cat: 0.0000 (24.7559)  loss_ce_unscaled: 2.3961 (2.5253)  loss_bbox_unscaled: 0.0652 (0.0634)  loss_giou_unscaled: 0.2313 (0.2595)  cardinality_error_unscaled: 1.0000 (1.1037)  loss_contrastive_align_unscaled: 1.3347 (1.5488)  loss_answer_type_unscaled: 1.4362 (1.4222)  accuracy_answer_type_unscaled: 0.5000 (0.3902)  loss_answer_obj_unscaled: 0.6557 (0.5836)  accuracy_answer_obj_unscaled: 1.0000 (0.7154)  loss_answer_attr_unscaled: 4.4138 (4.0215)  accuracy_answer_attr_unscaled: 0.5000 (0.3780)  loss_answer_rel_unscaled: 5.6157 (6.0363)  accuracy_answer_rel_unscaled: 0.0000 (0.1911)  loss_answer_global_unscaled: 0.0000 (0.3544)  accuracy_answer_global_unscaled: 1.0000 (0.9268)  loss_answer_cat_unscaled: 0.0000 (0.9902)  accuracy_answer_cat_unscaled: 1.0000 (0.8537)  accuracy_answer_total_unscaled: 0.0000 (0.0488)  time: 14.6253  data: 0.0905  max mem: 0
Traceback (most recent call last):
  File "main.py", line 643, in <module>
    main(args)
  File "main.py", line 546, in main
    train_stats = train_one_epoch(
  File "/home/pchelintsev/MDETR/mdetr/engine.py", line 100, in train_one_epoch
    losses.backward()
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/_tensor.py", line 255, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/pchelintsev/anaconda3/envs/mdetr_env/lib/python3.8/site-packages/torch/autograd/__init__.py", line 147, in backward
    Variable._execution_engine.run_backward(
RuntimeError: [enforce fail at CPUAllocator.cpp:71] . DefaultCPUAllocator: can't allocate memory: you tried to allocate 244470272 bytes. Error code 12 (Cannot allocate memory)

And indeed the memory ran out (I saw it with free -m), so it's not enough 64 Gb. But why do strange mistakes attack me earlier than Epoch: [0]  [    40/163172] when I use GPU?

I also tried to run evaluation on CLEVR, which is much smaller, using python main.py --batch_size 64 --dataset_config configs/clevr.json --num_queries 25 --text_encoder_type distilroberta-base --backbone resnet18  --resume https://zenodo.org/record/4721981/files/clevr_checkpoint.pth  --eval and again got RuntimeError: linearIndex.numel()*sliceSize*nElemBefore == value.numel()INTERNAL ASSERT FAILED at "/pytorch/aten/src/ATen/native/cuda/Indexing.cu":253, please report a bug to PyTorch. number of flattened indices did not match number of elements in the value tensor1421.
But I have 64 Gb RAM and 32 Gb GPU, so the whole dataset and the models can be easily allocated. It seems this is not a bug with resources... And then I run on CPU, it seems to work, consuming only about 11 Gb RAM.

Try setting this line to false:
https://github.com/ashkamath/mdetr/blob/0b747b99e2995c3c429f1391cb8e6104eaec7f21/main.py#L309
Also, could you paste the output of python -m torch.utils.collect_env ?

@alcinos, thank you for your support!

I changed the line on torch.set_deterministic(False) and the evaluation on CLEVR went successfully!

Accumulating evaluation results...
DONE (t=106.84s).
IoU metric: bbox
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.828
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.990
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.988
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.718
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.830
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.907
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.373
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.872
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.872
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.786
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.874
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.934
{'test_clevr_loss': 43.043012584842515, 'test_clevr_loss_ce': 5.731913995213883, 'test_clevr_loss_bbox': 0.05494903914992771, 'test_clevr_loss_giou': 0.18080522204549035, 'test_clevr_loss_contrastive_align': 0.739596603851473, 'test_clevr_loss_ce_0': 6.165245272397181, 'test_clevr_loss_bbox_0': 0.06370836580744324, 'test_clevr_loss_giou_0': 0.21175039811345905, 'test_clevr_loss_contrastive_align_0': 1.6258955893866438, 'test_clevr_loss_ce_1': 5.9863008091881005, 'test_clevr_loss_bbox_1': 0.06160141016855683, 'test_clevr_loss_giou_1': 0.20656590048007592, 'test_clevr_loss_contrastive_align_1': 1.2556593710804556, 'test_clevr_loss_ce_2': 5.867624813379281, 'test_clevr_loss_bbox_2': 0.059281312062726084, 'test_clevr_loss_giou_2': 0.19940767474091095, 'test_clevr_loss_contrastive_align_2': 1.0192198974077205, 'test_clevr_loss_ce_3': 5.771324536906167, 'test_clevr_loss_bbox_3': 0.05730514128536698, 'test_clevr_loss_giou_3': 0.194069205697486, 'test_clevr_loss_contrastive_align_3': 0.817971428532039, 'test_clevr_loss_ce_4': 5.741090293833827, 'test_clevr_loss_bbox_4': 0.05582752994519147, 'test_clevr_loss_giou_4': 0.18599585877292799, 'test_clevr_loss_contrastive_align_4': 0.7558641342681423, 'test_clevr_loss_answer_type': 3.604563896833151e-06, 'test_clevr_loss_answer_binary': 0.00632767150950992, 'test_clevr_loss_answer_reg': 0.025332900922053852, 'test_clevr_loss_answer_attr': 0.0023749297052867175, 'test_clevr_loss_ce_unscaled': 5.731913995213883, 'test_clevr_loss_bbox_unscaled': 0.010989807827442681, 'test_clevr_loss_giou_unscaled': 0.09040261102274517, 'test_clevr_cardinality_error_unscaled': 0.01639825085324232, 'test_clevr_loss_contrastive_align_unscaled': 0.739596603851473, 'test_clevr_loss_ce_0_unscaled': 6.165245272397181, 'test_clevr_loss_bbox_0_unscaled': 0.012741673153303818, 'test_clevr_loss_giou_0_unscaled': 0.10587519905672953, 'test_clevr_cardinality_error_0_unscaled': 1.9568324677976732, 'test_clevr_loss_contrastive_align_0_unscaled': 1.6258955893866438, 'test_clevr_loss_ce_1_unscaled': 5.9863008091881005, 'test_clevr_loss_bbox_1_unscaled': 0.012320282033075652, 'test_clevr_loss_giou_1_unscaled': 0.10328295024003796, 'test_clevr_cardinality_error_1_unscaled': 1.115309238815267, 'test_clevr_loss_contrastive_align_1_unscaled': 1.2556593710804556, 'test_clevr_loss_ce_2_unscaled': 5.867624813379281, 'test_clevr_loss_bbox_2_unscaled': 0.011856262410399679, 'test_clevr_loss_giou_2_unscaled': 0.09970383737045548, 'test_clevr_cardinality_error_2_unscaled': 0.5807346361705162, 'test_clevr_loss_contrastive_align_2_unscaled': 1.0192198974077205, 'test_clevr_loss_ce_3_unscaled': 5.771324536906167, 'test_clevr_loss_bbox_3_unscaled': 0.01146102826025197, 'test_clevr_loss_giou_3_unscaled': 0.097034602848743, 'test_clevr_cardinality_error_3_unscaled': 0.1629644974624363, 'test_clevr_loss_contrastive_align_3_unscaled': 0.817971428532039, 'test_clevr_loss_ce_4_unscaled': 5.741090293833827, 'test_clevr_loss_bbox_4_unscaled': 0.01116550598355525, 'test_clevr_loss_giou_4_unscaled': 0.09299792938646399, 'test_clevr_cardinality_error_4_unscaled': 0.04800581955033078, 'test_clevr_loss_contrastive_align_4_unscaled': 0.7558641342681423, 'test_clevr_loss_answer_type_unscaled': 3.604563896833151e-06, 'test_clevr_accuracy_answer_type_unscaled': 1.0, 'test_clevr_loss_answer_binary_unscaled': 0.00632767150950992, 'test_clevr_accuracy_answer_binary_unscaled': 0.9983339897855964, 'test_clevr_loss_answer_reg_unscaled': 0.025332900922053852, 'test_clevr_accuracy_answer_reg_unscaled': 0.9925868332182588, 'test_clevr_loss_answer_attr_unscaled': 0.0023749297052867175, 'test_clevr_accuracy_answer_attr_unscaled': 0.9996434087283376, 'test_clevr_accuracy_answer_total_unscaled': 0.9974136092150171, 'test_clevr_coco_eval_bbox': [0.8280577103267962, 0.9900612713060782, 0.9877238191805104, 0.7180399374430324, 0.8298110005371897, 0.9071876701127315, 0.37329648287548844, 0.8716828824051552, 0.8719843337521758, 0.7862159823816266, 0.8740682349128326, 0.9336712955657251], 'n_parameters': 111200939}

Here is the output of python -m torch.utils.collect_env:

Collecting environment information...
PyTorch version: 1.9.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.27

Python version: 3.8 (64-bit runtime)
Python platform: Linux-4.15.0-156-generic-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 9.1.85
GPU models and configuration: GPU 0: Tesla V100-PCIE-32GB
Nvidia driver version: 470.63.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.3
[pip3] torch==1.9.0
[pip3] torchvision==0.10.0
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.3.0           h06a4308_520
[conda] mkl-service               2.4.0            py38h7f8727e_0
[conda] mkl_fft                   1.3.0            py38h42c9631_2
[conda] mkl_random                1.2.2            py38h51133e4_0
[conda] numpy                     1.20.3           py38hf144106_0
[conda] numpy-base                1.20.3           py38h74d4b33_0
[conda] torch                     1.9.0                    pypi_0    pypi
[conda] torchvision               0.10.0                   pypi_0    pypi

I also try VQA2 again a bit later but this gave me hope! Thank you :)

authors: TopCoder2K alcinos toku-n volkancirik

repo: ashkamath/mdetr


text:
dist tags

npm publish --tag my-tag
latest is default tag
allows betas
reference @boennemann's blog post

Hey @ashleygwilliams,
thanks for referencing it here, the post is public now :)
https://medium.com/greenkeeper-blog/one-simple-trick-for-javascript-package-maintainers-to-avoid-breaking-their-user-s-software-and-to-6edf06dc5617

authors: ashleygwilliams boennemann

repo: ashleygwilliams/you-dont-know-npm


text:
[Bug]: Issues with the radar

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug
The radar chart is not correctly responding to the system's color (System mode is not working). Also There's a small typo in the data mapping for issues that's why it is not getting pointed in the radar.
Expected behavior
after the correction it will work properly like this in system mode as well

Add ScreenShots
No response
On which device are you experiencing this bug?
No response
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I have starred the repository

Thank you for creating this issue! 🎉 We'll look into it as soon as possible. In the meantime, please make sure to provide all the necessary details and context. If you have any questions or additional information, feel free to add them here. Your contributions are highly appreciated! 😊
You can also check our CONTRIBUTING.md for guidelines on contributing to this project.

this is what it looks now... now i realized what's the mistake. i forgot to set the theme for systems here and also there is a typo due to which it will display only 3 parameters instead of 4.

GO ahead @Swastik19Nit

Hey @Swastik19Nit,
Do the changes to your issue after #336 this PR gets merged. Or else there will be merge conflicts.
@ashutosh-rath02 give #336 a priority. So, Swastik can work on this issue.

@dhairyathedev Your recent issue aligns much with this one. You can guide @Swastik19Nit what needs to be done here only

i have solved the issue of the ticks as well.. now it will be dynamically calculated on the radar and will be displayed neat

Ok raise a PR @Swastik19Nit

 i have solved the issue of the ticks as well.. now it will be dynamically calculated on the radar and will be displayed neat

How are you setting the tick count? Fixed count or programmatically? @Swastik19Nit

The calculateStepSize function uses logarithms and powers of 2 to find a step size that's easy to read (like 20, 50, 100, etc.) and that dividdes the max value into a reasonable number of steps. This makes sure that the radar chart remains readable regardless of whether the users stats are in tens, hundreds, or thousands.
@ashutosh-rath02

I have raised a common PR

Hello @Swastik19Nit! Your issue #332 has been closed. Thank you for your contribution!


text:
Optimize Rust impls

Related to #107
Optimizes the native implementation in a way that the compiler can actually vectorize the implementations despite the IEE rules.
Although not the simplest, it is more realistic of a 'native' implementation if you are trying to get the maximum speed by going down to intrinsic instructions like AVX.

Hi @ChillFish8! Thanks for your contribution!
Indeed, your loop-unrolled variant is much faster than the naive Rust approach, even the procedural code.
     Running rust/benches/cosine.rs (target/release/deps/cosine-e0cccefbe212a606)
Gnuplot not found, using plotters backend
SIMD Cosine/SimSIMD/0   time:   [91.178 ns 91.296 ns 91.444 ns]
Found 10 outliers among 100 measurements (10.00%)
  2 (2.00%) low severe
  4 (4.00%) high mild
  4 (4.00%) high severe
SIMD Cosine/Rust Procedural/0
                        time:   [793.02 ns 796.96 ns 802.25 ns]
Found 17 outliers among 100 measurements (17.00%)
  5 (5.00%) high mild
  12 (12.00%) high severe
SIMD Cosine/Rust Functional/0
                        time:   [794.70 ns 797.24 ns 801.14 ns]
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severe
SIMD Cosine/Rust Unrolled/0
                        time:   [208.64 ns 209.64 ns 211.12 ns]
Found 14 outliers among 100 measurements (14.00%)
  5 (5.00%) high mild
  9 (9.00%) high severe

I am mostly working on recent CPUs and on Intel Sapphire Rapids SimSIMD currently wins thanks to AVX-512 support. I wouldn't expect much difference for f32 on AVX2-only machines. For other types, it may be noticeable. Maybe it makes sense to add benchmarks for i8, the wins can be very noticeable there 🤗

:tada: This PR is included in version 4.3.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

@ashvardanian Do you have a rough idea what the performance difference is between the Intel rapids AVX512 vs something like an AMD 7700 or Epyc chip? Just curious since I develop mostly on AMD type CPUs which can be a bit difficult to predict how performance goes on intel chipsets.

@ChillFish8 on Zen4 most of AVX512 is available, except for FP16 extensions. Everything except for that should work great.
If you are on Zen3 or older, SimSIMD will use F16C extensions for FMA. They are quite slow, but still much better than serial code for half-precision, as modern compilers can't handle that type well. For single-precision you may not get any gains on older CPUs.
For int8, SimSIMD should work great on both old and new CPUs. That type is often used in heavily-quantized embedding models.

repo: ashvardanian/SimSIMD


text:
Remove MinRequestBodyDataRate with gRPC from docs

Addresses https://github.com/grpc/grpc-dotnet/issues/15
Merge should wait on https://github.com/grpc/grpc-dotnet/pull/259

@JamesNK I put a "WIP" in the title just to make sure this isn't prematurely merged.

Ready to approve/merge.

authors: JamesNK guardrex


text:
Testing something with the msftbot

I'm suspicious that it isn't removing the "Needs: Author Feedback" label properly, so I'm testing something here. Disregard this issue. Sorry for the spam...

Testing

Testing again

Testing again.

Fixed!

repo: aspnet/AspNetCore


text:
ConfigurationBuilder's AddJsonFile not Working in 3.1?

I noticed that .AddJsonFile() is called automatically for appsettings.json and appsettings.{Environment}.json. I'm not sure if this is new behavior, but I do know that calling AddJsonFile() in ConfigureAppConfiguration() doesn't seem to have an affect anymore, but should it?
I read through the docs but don't see where this should be done differently. Here is my Main():
public static void Main(string[] args)
{
	CreateHostBuilder(args).Build().Run();

	var host = CreateHostBuilder(args)

		.ConfigureAppConfiguration((hostingContext, config) =>
													{
			IHostEnvironment env = hostingContext.HostingEnvironment;

			//last registration wins here
			config
				.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
				.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: false, reloadOnChange: true)
				.AddJsonFile($"appsettings.{Environment.MachineName}.json", optional: true, reloadOnChange: true)
				.AddEnvironmentVariables();
			})
		.Build();

	host.Run();
}

Further technical details

ASP.NET Core version: 3.1
Include the output of dotnet --info

.NET Core SDK (reflecting any global.json):
Version:   3.1.100
Commit:    cd82f021f4

Runtime Environment:
OS Name:     Mac OS X
OS Version:  10.15
OS Platform: Darwin
RID:         osx.10.15-x64
Base Path:   /usr/local/share/dotnet/sdk/3.1.100/

Host (useful for support):
Version: 3.1.0
Commit:  65f04fb6db

.NET Core SDKs installed:
3.1.100 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.App 3.1.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Visual Studio for Mac 8.4 Preview build 2653

CreateHostBuilder(args).Build().Run();

The above line of code blocks forever so the second line of code that configures a new host with new configuration will never run.

Yes, deleting the extra line CreateHostBuilder(args).Build().Run(); fixed the issue. It's not clear how it got added, but this issue can be fully deleted if you want.

authors: davidfowl mrlife