text:
Razon - English Translation

Hello, I am attempting to translate "RAZON!!" into English.
Does anyone know who the author so, I can reach out and give them the Localisation file when it is complete?

https://github.com/EtiCarasoft

Hey Bruce, thanks for that. Is the quest data in your report? If so how do I go about forking it and adding the file?
@EtiCarasoft just wanted to tag you in on this. I will attach a localisation.english file once I have completed the translation.

Perfect, thanks a lot for your work !!!

The original tittle is:  At 22.30 at the birds shop.

This scenario was originally added before I had support for user controlled packages. I'd prefer not to update the existing one, other than to change it to the new system:
https://github.com/NPBruce/valkyrie/wiki/QuestPublish

I do not speak Spanish,and I am translating using google translate and then cleaning it up. There a few bits that will probably not translate well, i'll do my best and ask here if I am not sure on something.

Hi @EtiCarasoft ,
I am having trouble translating this line:

El niño te abraza con fuerza y te susurra al oído:
Yo soy tu y tu eres yo, ahora lo verás.
Te ves a ti mismo tumbado, pareces dormido y debería haber alguien llorando a tu lado, pero estas solo, en medio de la nada, todo es oscuridad y te falta el aire. Te diriges hacia un punto de luz que no quieres alcanzar y ves la cara de ese hombre, con los ojos rasgados, te agarra la cabeza con sus enormes y frías manos para arrastrarte hacia la luz. Te señala la cortina roja y una bandada de pájaros de colores la atraviesa dirigiéndose directo hacia ti.
Cierras los ojos y oyes la voz de un niño pequeño:
-Bienvenido a casa, Pajarito.
Ganas el hechizo Alimentar la mente.
El investigador descarta todas sus cartas de Horror boca abajo y coge una carta de Locura, pasa a estar LOCO.
Si el investigador ya estaba loco, este es retirado de la partida y los investigadores disponen de un turno adicional para completar la investigación.

Here is my first attempt, but I am not sure I have the context correct.

The child hugs you tightly and whispers in your ear:
I am you and you are me, now you will see it .
You see yourself lying down, you appear to be asleep and where you would expect someone to be crying at your side, you realise that you are alone, in the middle of nowhere. Everything is dark and you don't seem to be breathing. You go towards a pinprick of light, though youfeel a great sense of forboding as you approach. You see the face of a man with scared and torn eyes, he grabs your head with giant, cold hands and drags you towards the light. It shows you a red curtain and a flock of colorful birds bursts into flight coming straight towards you.
You close your eyes and hear the voice of a small child:

 Welcome home, Pajarito .- .

You gain the spell  Feeding the mind .
The Investigator discards all his facedown Horror cards and becomes  Insane .
If the Investigator was already insane, he is defeated and the Investigators have one additional turn to complete the investigation.

Is Pajarito a name?

I see it pretty cool!
Pajarito is little bird

I tried to add a bit of poetic licence I hope you don't mind:
The line translated to:
I am you and you are me, now you will see it.
I made it:
I am you and you are me, now you will see what we both can see.
I felt the creepy child would work well with a creepy poem! Haha.
When it's done I'd appreciate if you would read it over to make sure I've got the correct context for events. Google translate is amazing and I'm translating event my event as they would unfolds in game to learn the story, but in some instances it's quite hard to maintain your writing style (which I can tell it's very elequent) through the rough Google translation!

@scrubbless don't know if you have seen it, but I just realized there are some pending changes here for this scenario : https://github.com/NPBruce/valkyrie-questdata/pull/34/files
And BTW, I'd like to have a look at your translations too, if you don't mind.
It could help me improve my version in French and I may have some comments as I may have a different understanding on some part.

A quick suggestion for your translation:
"you appear to be asleep and where you would expect someone to be crying at your side, you realise that you are alone, in the middle of nowhere."
I would suggest:
"you appear to be asleep and you think you hear someone crying by your side, but you are alone, in the middle of nowhere."
I'm not fluent in english but I know the first version did not sound so good in French, so I used something similar to this.

Thanks for the tip, I agree with you there. I'm not sure if it was intentional orif it has just translated strange.
The dialogue appears to suggest that in your 'consciousness' mind you expect to be next to the child, but whatever phychosis you are experiencing now depicts you alone.

@scrubbless don't know if you have seen it, but I just realized there are some pending changes for this scenario : > https://github.com/NPBruce/valkyrie-questdata/pull/34/files

And BTW, I'd like to have a look at your translations too when you are done, if you don't mind.
It could help me improve my version in French and I may have some comments as I may have a different understanding on some parts.
Absolutely, I'd appreciate that!

Sorry for the delay, I would usually make quicker progress. However I am trying to push my latest scenario to a near final polish before my last major play test.
Assuming that goes well I'll probably be releasing it and will have more time to work on this translation.
I also plan to take another parse on the tutorial once this translation is done, as I have some really cool techniques s I'd like to share. You'll see them in "A Strain on Reality" once it is released.

Have anyone of your test been succesful on getting trhow the scenario? I mean winning? @scrubbless, @BenRQ ? This weekend my partner and I made a few ideas for the second one. We'll be writting the script this august to program it as son as posible. We also think on downgrading the difficulty of the scenario. The events are very narrative, but are not so random so having to repeat it much will make it less interesting. We want to try it this week-weekend, if we adjunt anything we will tell you here which mythos have been modified and the values, not much change but some adjusting.

Hi @EtiCarasoft
I did not successfully finish it yet, but I'm planning to try it again with some friends this Thursday.
I agree it is very difficult as it is, but I did expect a least one investigator to get 'LOCO' (mad) before the end of the game
Also I'm wondering if the difficulty depends a lot on the number of players.
I tried it with 2 investigators, and will try it with 4 next time.
I guess this will make this scenario easier.
Regarding replaying it, it can be improve for sure, but I like the fact that every choice really has two viable option. So you can replay it and still find some new things to try.
Maybe only once it is a bit to extreme as 'all other investigator are dead'.
It could be changed to 'all investigator get X horrors', as it would made everyone mad, but the game would not be ended by one bad decision.
As you are having a look at the scenario, here some minor points I have spotted:
TokenFantasmaNaranja1.button3,
{action} Rezar (Pray)
This action does nothing, it makes sense but feels a bit strange.
Maybe a small text 'nothing happens' would be good
EventMythosCocina.text,
cuchil 'calvado' or 'clavado'
Probably a mistype here
EventMythosSoft03.text,
Should the object be lose randomly or can the investigator choose ?
EventMythosSoft08.text,
Are you sure this is the right tile name ?
From what I've seen, it should be "la Esquina de calle 1" instead of "Parcela"
TokenPuertaAzulLuzON.text,
There are no option to open the door, and no text to explain why.
Maybe just saying it is closed is enough.
EventMythosHABOUT02.text,
You take a punch in the head, but get horror and no damage.
And by the way, I'm looking forward to test your next scenario 👍 👍

It does a hide action i'll look to make it appear as nothing happens makes sense.
... Cocina is a mistype i've fixed it. Changed.
Soft03 should be random I add it in spanish. Changed
I don't know why it changed the tile it should be SideYard2. Changed.
HABOUT Should Be Damage instead of Horror. Changed.
EventRezaryLlorar = "Parece que no sucede nada, però te sientes aliviado." -
Translation "It seems that nothing happens, but you feel relieved."
Localization.Spanish.txt
This is the file uploaded and quite balanced it's still hard but not so.
Events Changed:

EventMythosHard03A
EventMythosHard03B
EventMythosHard01
EventMythosSotano01
EventMythosFlavor04
EventMythosSoft01
EventMythosSoft03
EventMythosSoft04
EventMythosSoft06T
EventMythosSoft07
EventMythosSoft08
EventMythosSoft09
EventMythosCocina
EventMythosHypnosBad
EventSustoBiblioteca
EventRes2MarcadorFake4
EventResMarcadorFake5
EventPuertaFake2
EventMostrarPuertaOculta1
EventAbrirPuertaCampanario

Maybe only once it is a bit to extreme as 'all other investigator are dead'.
It could be changed to 'all investigator get X horrors', as it would made everyone mad, but the game would not be ended by one bad decision.
This is the previous to last mythos phase this adventure doesn't let you stay in random events, it also makes sense when your taking the last talk. Just don't worry much about this event this decision just makes the game 1-2 turns shorter.

This is the previous to last mythos phase this adventure doesn't let you stay in random events, it also makes sense when your taking the last talk. Just don't worry much about this event this decision just makes the game 1-2 turns shorter. Also enables one alternative ending but that's a spoiler XD

OK makes sense then :) :)

@eticarasoft
Did you know you can use {c:TileTheTilesNAME} to display the tile, the game will enter the tiles name and icon for you.
E.g
The tiles name is Tile001 is the study.
You'd use {c:Tile001} in the text and it would display (in the correct language) the tiles name.
You can do the same with items:
{c:QItemTheitemsName}

@EtiCarasoft I'm planning to play it tomorrow evening, so would be interesting to have the latest version.
Can you please share your latest version here so that I could test?
A zip or the package is good enough for me for this test.
And if you agree, I will try to apply the change proposed by Scrubbless regarding tile names, it should look nicer.

shure no problem @BenRQ .
@scrubbless That script i didn't know it existed when i created the scenario first time.
EditorEscenario1.zip
This is the lattest version. I'm working on a bug i think i've found with some vars with the blue light door. I've solved one of the problems, but i have to research a little how to solve the other one. Sorry. The colour buttons are the most difficult part for me to program correctly.
@BenRQ for your test porpouse use the new mythos difficult used in spanish I would make the scenario  much less hopeless. I listed the mythos before, just get the values and the type of damage.

@scrubbless That script i didn't know it existed when i created the scenario first time.

There's a good chance that it didn't when you created yours, the earlier versions of Valkyrie didn't have as many features! Ive seen Razon on my list ever since I started using Valkyrie!

Thanks @EtiCarasoft, I've now integrated your changes on difficulty in the French translation.
I will play with these changes.

The secrets doors do not snap the same was as doors so have to be manually placed, which is a nightmare!

HUGE SPOILER
There are only 5 mythos Hard it ends after those.
I've fixed the free position of the wall, thanks @scrubbless It's been difficult to align.
TokenPuertaVerde01.text,Una puerta con la luz verde debajo explicat bé
I can't translate 'explicat bé'. Is it a typo ?

Sorry it's a mistake from the previous text I will fix it and post here the correct text. (explica't bé) means well explained in Catalan

With the last question from the cat, to provide an object, there is an option to "recapitular" ( TokenBastEntrega01.button5). This button does nothing.
What did you mean by 'recapitular' ?

Recapitular is the same as cancel, but nicely writing to say you have to think about what to give.

We went anyway to the last question with the cat, and they were all a bit lost with the question.
They did two bad choices. Is it pure luck or is there is a tip somewhere to know what is the object you should provide?

Same problem as token puerta verde, it's a concept problem we have with the second escenario that was discarded. We will add this week some hard way to figure it out from the beggining making players to open fake clue tokens to find the correct item.

The **** doors are the worst part of the scenario, there's no way to make it work properly XD. I have to rework on it to solve all the variable solutions and options.
Thanks @BenRQ for all your work and that extense review we really apreciate it. It's really really difficult to all players to leave the house together even knowing what to do and the correct steps to follow that's why I left the option It's just me who survived, to leave a friend behind 🗡 ...

Thanks @BenRQ for all your work and that extense review we really apreciate it. It's really really difficult to all players to leave the house together even knowing what to do and the correct steps to follow that's why I left the option It's just me who survived, to leave a friend behind 🗡 ...

You are welcome @EtiCarasoft.
I like that you have this choice available 👍

The **** doors are the worst part of the scenario, there's no way to make it work properly XD. I have to rework on it to solve all the variable solutions and options.

Looks like it works in the editor, and not in the game, doesn't it ?
Maybe we can help you fix this.
(Also an easy way to fix this would be to just open each door after each button is pressed, and never close it again, but that's maybe not what you want).
Also I've seen one small improvement:
The skill symbol for the puzzle in the library [PuzzlePuertaBIBLIO], appears twice. Probably a bug of a previous editor.
Selecting the skill 'influence' again in the editor fix this.

I think I've solved all the bugs ^^!! The doors solution is not super elegant but i think it works.
We have also come across with an idea to fix the ending and lapsing to the next scenario.
Forget about TokenPuertaVerde01.text this token is deleted.
My partner is working on 2 new text to improve the scenario. We are thinking on adding a Qitem with interaction and a QUI to explain the idea. I'll posthere the texts also with the file. I want to have it all finished for this week. I'll post also the list of events/tokens/puzzles that will need to be translated. Sorry for the delay and extra work.
Info for @BenRQ & @scrubbless

@EtiCarasoft very good news  👍
I'm ready to translate !  :)

It's been a while, sorry we made a few changes to improve the scenario. We set a $mythosMajor between basement Event and the Greenhouse making the greenhouse the end triggerer.
Fixed Some bug based on this changes and make a new search token which gives a Qitem with precious information to conclude the mision ;).
List of events and text changes:
TokenzzINFO
EventResolucionPuzzleBast
EventDesarrolloPasillo1A
ResObjetoEsquina
TextoPruebaVez01
TextoPruebaVez02
EventInspeccionarNota01
EventInspeccionarNota02
Some buttons atached to this events and one UI element as a button. I think that's all.
We're drawing the image of the puzzle I hope we can have it finished soon. I'll do a package as soon as It's done.
I attach the latest text file:
Localization.Spanish.txt

Hi @eticarasoft,
Nice ! We will have a unique drawing for this scenario :)
Do you think you are done with all the other changes you wanted to make?
I will work on the translation when I'm back home in two weeks.

I think I'm done with the chages with this last one I attach HERE:
Localization.Spanish.txt
On Qitem was changed.
We have tested some critical points with no bugs detected ;)

I've not forgotten about this, I am working on the mythos events and will check I have the latest version before I start error checking.
Works been manic and the small amount of time I've had has been tied up with A Strain on Reality!

Hi @EtiCarasoft, i've nearly finished the english translation, however I can't work out what the following line is:
-Pensaba que estaría enrollado entro. Esto no es lo que estoy buscando, no me hagas perder el tiempo.
It translates to:
 -I thought it would be coiled in. This is not what I'm looking for, do not waste my time .-

Ok here is the first draft, I need to play it through a bit to make sure it flows (and to add that translation above)
Localization.English.txt

downloaded it. changed the Quest.ini
We have the black of the illustration missing, my partner is inking it i hope it's done soon. As soon as it's done i'll change the file, do the package and update the adventure.
info also for @BenRQ

Hi @EtiCarasoft
Can you help with the translation above my last message. I have done some proof reading and corrected some parts. But I will need that translation to finish it off.
If you have a repo could I pull(?) That and do much changes on your scenario? Will mean I'm working on your latest version.

The translation is correct, coiled in or coiled inside i found both to be correct i will also correct the spanish one a D letter is missing.
thx @scrubbless

I thought it (the item the cats after) would be coiled inside.. Now I understand!

Hi @EtiCarasoft
Here is the latest translated file:
Localization.English.txt

Hello @EtiCarasoft
would you have some time to copy your scenario with English and French update in your own repo ?
Valkyrie Release 2.1 is coming out very soon, and I was thinking about sharing the info with the French community. As your scenario is a very nice, I wanted to list it in my "commercial for Valkyrie" on forums :)

Hi I've been quite busy on work, This night i'll pass here the package to uploaded  I think it will be the fastest way to get it to work properly.

escenario package.zip
Escenario full info.zip
@BenRQ I upload here the package and the full folder with all the scenario information just in case you need it to solve any problem with it.
Thks for the patience.

@NPBruce or @BenRQ I don't have the desktop application and the scenario was created before some major changes. How do we can upload all this new files?
Maybe the best way is to delete the old one and upload it again.

Hi @EtiCarasoft
thanks for sharing the files.
We can take it from here but if you could just create a repo, and share access with us it would be perfect.
Look here on how to do it: #924
That way, you can modify your scenario in the future if required.

I've done a repo with the tutorial:
https://github.com/EtiCarasoft/22.30-Birds-Shop

Thanks @EtiCarasoft we are almost ready to publish the update of your scenario with translation.
Can you please share the repo access with me and @scrubbless  so we can do a bit of renaming for the directory ? We should not use the name "EditorEscenario" as it can conflict with other scenarios.
Access rights can be shared from the settings, collaborators, add collaborator :

You can also rename it directly of course.

https://raw.githubusercontent.com/EtiCarasoft/**22.30**-Birds-Shop/master/
This doesn't link in the Valkyrie manifest. I suspect the . is causing issues.
@EtiCarasoft  or @BenRQ any chance one of you can remove that . ?

Ok that looks good. Looks like I've missed some translated events, I may have translated an older version. I'll take a look when I get the chance and update the translation.

I also have a small update on French translation, with this latest version :
Localization.French.txt
@scrubbless, can you take it with your update of translations ?
Also @scrubbless , maybe you should delete the old scenario from the Manifest.ini : [2230hAtTheBirdsPetShop]
external=https://raw.githubusercontent.com/NPBruce/valkyrie-store/master/MoM/2230hAtTheBirdsPetShop/
I have rename the existing ratings to the new scenario name.

French translation updated.

can we delete the old scenario some way?

Hi Joan (@EtiCarasoft )
I have fixed a bunch of syntax errors with the {rnd:hero} code that google translate messed up (it added a space making it {rnd: hero} which is next to impossible to see in the editor!
Could you add this to your scenario and republish when you get the chance. There are a lot of complaints in the feedbacks sheet about it.
https://goo.gl/xgxdMb
Sorry for this!

Hi @EtiCarasoft
What issue are you having? The scenario is in your repo now, so you just need to commit to it:
https://github.com/EtiCarasoft/22-30-Birds-Shop.
Here is the manifest entry:

[BirdsShop]
external=https://raw.githubusercontent.com/EtiCarasoft/22-30-Birds-Shop/master/

I have contribute to your repository so should be able to help.

I updated the english file but in the app it doesn't appear to update.

Did you commit the change and push it to your repo?

I updated via desktop app:

https://github.com/EtiCarasoft/22-30-Birds-Shop/tree/master/BirdsShop
I see here the changes.
I updated .DS_Store also

Hi @EtiCarasoft
You need to create your scenario package and upload that.

The manifest references the .ini file and .valkyrie package file. The folder you have updated is the unpacked scenario that is used in the editor.

In Valkyrie load the scenario up in the editor, then hit tools and create package. It will create a folder on the desktop, copy those files into your repo in the root and commit the changes.
The update will show in the scenario screen (the round arrow), however if you added Synopsis or author metadata it will not show up on the scenario lists main screen for up to an hour. This is just cosmetic, if anyone downloads the scenario they will be playing your latest version.

Hi guys, please note that you have to wait one hour before you can see the update in the app.
Refresh of scenario list is done once per hour.

I updated the scenario adding you english file, created a package updated in the repo. Updated DS.
now we have to wait 1hour to download it correct?

I'll check it in an hour. Thx all

Hi guys, please note that you have to wait one hour before you can see the update in the app.
Refresh of scenario list is done once per hour.

The "updated" icon actually shows up right away. Its just the banner that is created that takes an hour from what I've observed.
You can open the scenario, then click the Grey "delete" and redownload it, and will get the new version.

I've launched the refresh process manually, so you should see it now.

There is nothing obvious to see if it has updated. I would recommend adding a version number to one of your sections (I put it in author) so that you can checksum updates like this easily.

How can we add the version in the title? via editin scenario and change title in each .txt?

You can just edit the scenario in Valkyrie, change the scenario title in the quest section.

I also put a more granular version in the Authors but thats personal preference.

Updating files:

@scrubbless We're working on the next scenario and made some improvements on the Birdshop one, better event presentation and gameplay, also some little narrative improvements. Are you interested in updating it in english? Thanks

authors: BenRQ EtiCarasoft NPBruce scrubbless


text:
OCR多显示器BUG

在非主显示器上启动截图的时候会出现窗体放大的bug，在主显示器上则不会

感谢您提出的bug，这个问题在最新的 1.13.0.0 版本修复了

source: gharchive/issue

authors: NPCDW ZGGSONG

repo: NPCDW/WpfTool

license: MIT

license_type: permissive

license_source: github-api


text:
Ubuntu 14.04 install fail, access denied

Trying to follow the installer instructions, but:
$ sudo apt-get install api-umbrella
...
E: Failed to fetch https://dl.bintray.com/nrel/api-umbrella-ubuntu/pool/main/a/api-umbrella/api-umbrella_0.13.0-1~trusty_amd64.deb  HttpError401

Very strange, sorry for the trouble, but thanks for bringing this to our attention. It looks like some of the v0.13 packages got published, but not all. We'll try to re-upload (but it unfortunately might be a few days).

This should be fixed now. Thanks again for letting us know!

authors: GUI g-borgulya

repo: NREL/api-umbrella

license: mit

license_source: bigquery


text:
[dev] Not starting NFS kernel daemon: no exports.

As a note, when setting up the development server on Ubuntu 14.10, I get the following message when installing the NFS dependencies:
Creating config file /etc/default/nfs-kernel-server with new version
 * Not starting NFS kernel daemon: no exports.

I think this has been addressed by updates to the documentation noting the NFS requirements.

authors: GUI brylie


text:
Add network --host to build script

First confirm with group this is okay (in a pr to recipes)

➤ Julien Lhermitte commented:
This was an issue I had here when playing on thuban:
Testing on thuban - packaging and deployment ( https://3.basecamp.com/3733838/buckets/4534819/messages/1114837477 )
I noticed that there was some issue with the network bridge. When I removed it by adding the "--network host" flag, everything worked fine. If there are issues with docker not being able to see the outside world, then I suggest adding that flag here:
https://github.com/NSLS-II/lightsource2-recipes/blob/master/scripts/nsls2-tag-build.sh#L36
I don't remember more than this and don't have time to try to recall so I will leave it at that. @Maksim has taken charge of the build sytem so i will assign him and decide if this is needed

I don't think it's relevant anymore.


text:
Logged_events needs to be queried based on path id, activity id, course id & cohort id

Provide options to save  path id , activity id ,course id and cohort id directly under the logged_events node.

https://github.com/NUS-ALSET/achievements/pull/950
https://github.com/NUS-ALSET/achievements/pull/954

repo: NUS-ALSET/achievements


text:
[BUG] Timestamp after evaluation is off during model fit

Describe the bug
It appears HugeCTR combines the time taken after a set of iterations with the time taken to do an evaluation and reports that - this results in timestamps of 90s or 70s, etc.

To Reproduce
Run any HugeCTR model.
Expected behavior
Perhaps timestamp after the last training iteration. Then a timestamp for evaluation finish. Then timestamp for next training iteration.
Screenshots
See above.
Environment (please complete the following information):
nvcr.io/nvidia/merlin/merlin-training:21.09
Additional context
Add any other context about the problem here.

@paulhendricks thanks for bringing this to our attention. Despite our best efforts, we were unsuccessful to provoke this issue in our test environment. However, we found a potential race condition in the logging code that could cause faulty rendering of timestamps. A fix is included in the v3.4 release of HugeCTR.
Closing this issue for now. If you still experience this or a likewise issue with HugeCTR v3.4 or later, please do not hesitate to re-open this or create a new issue.

authors: bashimao paulhendricks

repo: NVIDIA-Merlin/HugeCTR


text:
Can FasterTransformer support a streaming request？

Can FasterTransformer be used in conjunction with Triton to support a streaming request mode for GPT models, similar to how ChatGpt can provide results in a streaming manner?
Thanks!

Yes, this is supported for T5 and GPT-NeoX. Please refer to the FT-Triton examples.

T5/GPT streaming inference based on GRPC.
tritonserver configuration:
# config.pbtxt
model_transaction_policy {
  decoupled: True
}

# t5 streaming
https://github.com/NVIDIA/FasterTransformer/pull/421/commits/281cbc16ae07f03b530828cd0c9079380b9af47c
https://github.com/NVIDIA/FasterTransformer/blob/main/src/fastertransformer/triton_backend/t5/T5TritonModelInstance.cc#L210

# gpt streaming
https://github.com/NVIDIA/FasterTransformer/blob/main/src/fastertransformer/triton_backend/multi_gpu_gpt/ParallelGptTritonModelInstance.cc#L242

client configuration:
https://github.com/triton-inference-server/fastertransformer_backend/blob/main/tools/issue_request.py

Hi, are there any pytorch examples to run streaming output in FastTransformer. As the streaming output is implemented in triton_backend part, does it mean that we can not directly use pytorch with streaming output?

authors: Jun-jie-Huang hongqing1986 michaelroyzen songkq

repo: NVIDIA/FasterTransformer


text:
Fix interleaved_numeric_conversion.h

uint4 fast cast should minus 64.5 for elt_45 and elt_67, which is 1032 / 16

Can you explain what problem do you encounter? How do reproduce the issue you encounter?

@yz-chen18 In addition to BoYang's question, I am not sure your suggestion is correct. Due to the bit positioning, we have:
elt_45/elt_67 = 1024 + 16X
Then, we divide by 16 to give us 64 + X. However, X = O + 8 where O is the original, signed value. So to extract the final value we must do -64 - 8 which gives -72. Do you have an example where that produces incorrect results?

source: gharchive/pull-request

authors: byshiue rhenry-nv yz-chen18


text:
Qwen-VL inference errors

Hello, I deployed the model based on examples/qwenvl/README.md, but the model inference result of running run.py was incorrect. What is the problem?

Input: "[{'image': './pics/demo.jpeg'}, {'text': 'Describe the picture'}]"
Output: "On the beach, a dog and its owner sit side by side, with the owner holding a cup of coffee."

I have this problem too. I changed the Input text to "Analyze style of the picture", the answer is not relevant.
CUDA_VISIBLE_DEVICES=1 python3 run.py     --tokenizer_dir=/home/Qwen-VL-Chat     --qwen_engine_dir=./trt_engines/Qwen-VL-7B-Chat     --vit_engine_dir=./plan     --images_path='{"image": "./pics/demo.jpeg"}'     --input_dir='{"image": "image.pt"}'
Input: "[{'image': './pics/demo.jpeg'}, {'text': 'Analyze style of the picture.'}]"
Output: "The style of the picture is quite unique and striking. It features a close-up shot of a person's feet on a sandy beach, with the sea in the background. The person is wearing flip flops, which adds to the casual and relaxed atmosphere of the scene. The composition of the picture is simple yet effective, focusing on the interplay between the person's feet and the natural surroundings. The use of negative space also adds to the visual impact of the image, making it more engaging and memorable. Overall, the style of the picture is both artistic and evocative, capturing a sense of leisure and freedom in a beautiful coastal setting."
PLUS, when I tried some other input texts, it answered that "no images are given" several times, very confusing.

Hi @jdmdj1999 @chiquitita-101 what TRT version you are using and what kind of quantization method you're using for Qwen?

@sunnyqgg

Hi@sunnyqgg
tensorrt-llm 0.9.0.dev2024040200
Qwen-VL-Chat
I deployed according to examples/qwenvl/README.md

I found that the error was caused by deployment and inference on different type of GPU

repo: NVIDIA/TensorRT-LLM


text:
Failed to build with CUDA 11.8 due to change in cudaGraphExecUpdate parameter

In CUDA 11.8, cudaGraphExecUpdate signature is:
__host__cudaError_t cudaGraphExecUpdate ( cudaGraphExec_t hGraphExec, cudaGraph_t hGraph, cudaGraphNode_t* hErrorNode_out, cudaGraphExecUpdateResult ** updateResult_out )

And in CUDA 12
cudaGraphExecUpdate signature is:
__host__cudaError_t cudaGraphExecUpdate ( cudaGraphExec_t hGraphExec, cudaGraph_t hGraph, cudaGraphExecUpdateResultInfo* resultInfo )

This change cause compiling error in tensorrt_llm/runtime.

TensorRT-LLM depends on CUDA 12.2. Is it impossible for you to upgrade to CUDA 12.2?

Just changing the function signature do work, and it can execute via CUDA 11.8 by far.

Just changing the function signature do work, and it can execute via CUDA 11.8 by far.

TensorRT: 9.1.0.4
CUDA : 11.8
CUDNN: 8.6.0
NCCL: 2.18.5

I changed the signature and successfully built a tensorrt engine.

But it throws an exception when executing:
python3 examples/llama/run.py --engine_dir=./llama2-7b-trt/f16-1gpu/ --max_output_len 100 --tokenizer_dir ../llama-recipes/llama-2-7b-hf-4330/ --input_text "how do i count to nine in Frech?"

the error is
CUDA Error: CUDA_ERROR_INVALID_IMAGE: TensorRT-LLM/cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention/fused_multihead_attention_v2.h 99
CUDA Error: CUDA_ERROR_INVALID_IMAGE: TensorRT-LLM/cpp/tensorrt_llm/kernels/contextFusedMultiHeadAttention/fused_multihead_attention_v2.h 105

can we solve it?
@jdemouth-nvidia

Thanks for the update @JosephChenHub - I'm afraid it's because our CUBINs for the Context FMHA are built using CUDA 12.x. I'll check with the engineer in charge of that part of the code if he can give it a try to make sure that's the problem. And, if there's a way to work around it.

Hi @JosephChenHub , I’ve verified with our CUDA team. A CUBIN built with CUDA 12.x will not load in CUDA 11.x. CUDA 12.x is required to use TensorRT-LLM.

Hi @JosephChenHub , I’ve verified with our CUDA team. A CUBIN built with CUDA 12.x will not load in CUDA 11.x. CUDA 12.x is required to use TensorRT-LLM.

my Driver Version: 450.102.04 ,cuda version is 11.0，can i install cuda-compat-12-2 to support 12.x and install trt-llm？

Hi @JosephChenHub , I’ve verified with our CUDA team. A CUBIN built with CUDA 12.x will not load in CUDA 11.x. CUDA 12.x is required to use TensorRT-LLM.

my Driver Version: 450.102.04 ,cuda version is 11.0，can i install cuda-compat-12-2 to support 12.x and install trt-llm？

You must update your CUDA toolkit and update the driver. There’s no way around it.

authors: JosephChenHub a1164714 haojiwei jdemouth-nvidia


text:
Fix zombie process when querying TE install path

This is a modification of the bugfix in https://github.com/NVIDIA/TransformerEngine/pull/120, with some small changes for safety and style.
Pinging @ngoyal2707. Closes https://github.com/NVIDIA/TransformerEngine/pull/120.

Thanks, I was on slightly older version of TE,
now we also need same fix here: https://github.com/NVIDIA/TransformerEngine/blob/main/transformer_engine/pytorch/transformer.py#L45

/te-ci

/te-ci

/te-ci

/te-ci

repo: NVIDIA/TransformerEngine


text:
Documentation for advanced performance optimizations

This adds a Jupyter notebook with text for multi-GPU training, wgrad fusion, and FP8 weight caching. The code itself still needs to be tested.

The notebook now runs.
The parallel model has timing results on an H100 PCIe.
The wgrad fusion model runs but does not have timing results.
There is still no demo for FP8 weight caching.

I've modified the timing function to fix a bug where we were running the backward pass within an fp8_autocast. This fixed some runtime errors I was encountering.
I've added text and examples for FP8 weight caching and sequence parallelism, so I think this covers the main optimizations we want to feature.

@timmoon10 Please sign your commits (see https://github.com/NVIDIA/TransformerEngine/blob/main/CONTRIBUTING.rst for details).

/te-ci

/te-ci


text:
Small modif done in Theano version.

This allow Theano to continue to use hidden symbol by default.
This is needed to work around strange default on Mac.

Merged! Thanks a lot!

authors: jdemouth nouiz

repo: NVIDIA/cnmem


text:
Failed to install gpu-helm-charts/dcgm-exporter

I follow the instructions here. Everything went well before failing to install gpu-helm-charts/dcgm-exporter.
➜  ~ helm install \
   --generate-name \
   gpu-helm-charts/dcgm-exporter
Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"

I tried a few times and even rebooted the system. But I always got the error. I have no idea and ask for your help.
More information:

OS: ubuntu 18.04lts
Docker: nvidia-docker2 19.03.13, API version 1.40.
Kubernetes: 1.19
Helm: 3.4.1

I solved it myself. It turns out that I have to set up prometheus first(https://docs.nvidia.com/datacenter/cloud-native/kubernetes/dcgme2e.html#setting-up-prometheus)

repo: NVIDIA/gpu-monitoring-tools


text:
No such file or directory

when i  run this:  python -m runx.runx scripts/dump_folder.yml -i ,  go wrong .

but the path is available.
how this?

@ajtao
the path logs/dump_folder/hidden-puffin_2021.04.27_17.17/code  is available

Notice that we're cd-ing into the logs/ directory but the command executed, as defined in scripts/dump_folder.yml is:
python -m torch.distributed.launch --nproc_per_node=1 train.py

However, train.py is defined at the project directory level. Changing the relative path to an absolute path should fix this issue. That is:
CMD: "python -m torch.distributed.launch --nproc_per_node=1 {absolute path to train.py}"

In my case this was:
CMD: "python -m torch.distributed.launch --nproc_per_node=1 /home/ubuntu/semantic-segmentation/train.py"

There's something wrong with (both) of your setups. What runx is attempting to do is:

Copy the codebase into a new 'run' directory under LOGROOT (defined in .runx). What @NarutoZhao's output showed is that this copy is failing. You should try to diagnose that problem. Try to define LOGROOT with an absolute path.
@atharvas Your comment is wrong. train.py is invoked after runx cd's into the run directory, where the code has just been copied. So you should not need to refer to the absolute path of train.py. In fact, the purpose of copying the code to the run directory is to serve as both (a) an archive of the state of the code and (b) to allow you to further modify your project repo after launching jobs.

See also #86

authors: NarutoZhao YuNaruto ajtao atharvas msseibel

repo: NVIDIA/semantic-segmentation


text:
Publish Optuna-Spark Examples

Contributing new examples demonstrating hyperparameter tuning with Optuna on Spark.
Approved by OSRB.

Please remove the private links.

Please remove the private links.

Sorry which links are you referring to? I removed those in the PR description if that's what you meant.

in the PR description

Yes, that's what I mean

change target branch to 24.12

For simplicity, I removed benchmarking scripts and just included a runtime graph comparison. The scripts can be added back in if we feel they're necessary.

repo: NVIDIA/spark-rapids-examples


text:
[BUG] GpuRunningWindowExecMeta is missing ExecChecks for partitionSpec in databricks runtime

Describe the bug
related to https://github.com/NVIDIA/spark-rapids/pull/3183, failed 141 cases in databricks runtime 7.3 and 8.2
GpuRunningWindowExecMeta is missing ExecChecks for partitionSpec

failures list,
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Boolean-1000]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Boolean-1g]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Date-1000]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Date-1g]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Timestamp-1000]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Timestamp-1g]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Null-1000]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Null-1g]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Decimal(7,3)-1000]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Decimal(7,3)-1g]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_running_float_sum_no_part[1000][APPROXIMATE_FLOAT]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_running_float_sum_no_part[1g][APPROXIMATE_FLOAT]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Byte]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Short]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Integer]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Long]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Float]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Double]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:String]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Boolean]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Date]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Timestamp]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Null]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank_no_part[data:Decimal(7,3)]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Byte][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Short][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Integer][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Long][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Float][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Double][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[String][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Boolean][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Byte-1000]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Date][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Byte-1g]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Timestamp][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Short-1000]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Null][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.628Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Short-1g]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_rank[Decimal(7,3)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Integer-1000]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Integer-1g]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Long0-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Long0-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Long-1000]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Long-1g]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Double-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Float-1000]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Double-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Float-1g]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Double-1000]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:Double-1g]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-String-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:String-1000]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-String-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Byte-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_no_part[data:String-1g]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Byte-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Timestamp-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Decimal(18,1)-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Timestamp-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Byte-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Byte-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Short-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Decimal(18,1)-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Short-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Timestamp-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Integer-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Null-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Integer-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Null-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Long1-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Decimal(7,3)-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Long-Long1-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Decimal(7,3)-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Float-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_float_decimal_sum[1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Float-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Double-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running_float_decimal_sum[1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Double-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.629Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[String-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[String-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Boolean-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Boolean-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Date-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Date-Long-1g][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_running[Timestamp-Long-1000][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_running_window_function_exec_for_all_aggs[IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Short][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Integer][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Long][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Float][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Double][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[String][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Boolean][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Date][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Timestamp][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Null][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Decimal(18,0)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Decimal(7,-3)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Decimal(7,3)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Decimal(7,7)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Decimal(12,2)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Byte][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Byte)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Short)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Integer)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Float)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Long)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Double)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Date)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Timestamp)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(String)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Null)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Boolean)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Decimal(18,0))][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Decimal(7,-3))][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Decimal(7,3))][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Decimal(7,7))][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Array(Short))][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Decimal(12,2))][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Struct(['child0', Byte],['child1', String],['child2', Float]))][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.630Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long],['child4', Float],['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Struct(['child0', Array(Short)],['child1', Double])][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Struct()][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(Boolean(not_null),Boolean)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Array(Array(String))][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(Byte(not_null),Byte)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Struct(['child0', Byte],['child1', Struct(['child0', Byte],['child1', Short],['child2', Integer],['child3', Long],['child4', Float],['child5', Double],['child6', String],['child7', Boolean],['child8', Date],['child9', Timestamp],['child10', Null])])][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(Short(not_null),Short)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(Integer(not_null),Integer)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(Long(not_null),Long)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(Float(not_null),Float)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(Double(not_null),Double)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(Date(not_null),Date)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(Timestamp(not_null),Timestamp)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(String(not_null),String)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(RepeatSeq(not_null)(Integer(not_null)),Long)][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(String(not_null),Array(String))][IGNORE_ORDER({'local': True})]
[2021-08-19T04:32:18.631Z] FAILED ../../src/main/python/window_function_test.py::test_window_ride_along[Map(String(not_null),Map(String(not_null),String))][IGNORE_ORDER({'local': True})]

Thanks @pxLi ... I filed https://github.com/NVIDIA/spark-rapids/pull/3258 which simply copies the ExecChecks from the base GpuOverrides. Is it easy for you to test this out? I haven't used the Databricks env in a while but I can test there if needed.

authors: andygrove pxLi

repo: NVIDIA/spark-rapids


text:
[BUG]java.lang.ClassNotFoundException: org.apache.spark.sql.rapids.execution.SerializeConcatHostBuffersDeserializeBatch

When GpuKyro is used, Spark-SQL queries will fail:
java.lang.ClassNotFoundException: org.apache.spark.sql.rapids.execution.SerializeConcatHostBuffersDeserializeBatch

Env:
rapids-4-spark_2.12-22.10.0-20220826.162157-17.jar
Note: using 22.08 jar works fine.
Reproduce:
Set below settings in spark-defaults.conf:
spark.serializer org.apache.spark.serializer.KryoSerializer
spark.kryo.registrator com.nvidia.spark.rapids.GpuKryoRegistrator
spark.kryoserializer.buffer.max 512m

Any query will fail:
22/08/29 15:57:00 ERROR SparkSQLDriver: Failed in [select * from test_orc limit 10]
java.lang.NoClassDefFoundError: org/apache/spark/sql/rapids/execution/SerializeConcatHostBuffersDeserializeBatch
	at com.nvidia.spark.rapids.GpuKryoRegistrator.registerClasses(GpuKryoRegistrator.scala:27)
	at org.apache.spark.serializer.KryoSerializer.$anonfun$newKryo$8(KryoSerializer.scala:180)
	at org.apache.spark.serializer.KryoSerializer.$anonfun$newKryo$8$adapted(KryoSerializer.scala:180)
	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
	at org.apache.spark.serializer.KryoSerializer.$anonfun$newKryo$5(KryoSerializer.scala:180)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.apache.spark.util.Utils$.withContextClassLoader(Utils.scala:231)
	at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:171)
	at org.apache.spark.serializer.KryoSerializer$$anon$1.create(KryoSerializer.scala:102)
	at com.esotericsoftware.kryo.pool.KryoPoolQueueImpl.borrow(KryoPoolQueueImpl.java:48)
	at org.apache.spark.serializer.KryoSerializer$PoolWrapper.borrow(KryoSerializer.scala:109)
	at org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:346)
	at org.apache.spark.serializer.KryoSerializationStream.<init>(KryoSerializer.scala:266)
	at org.apache.spark.serializer.KryoSerializerInstance.serializeStream(KryoSerializer.scala:432)
	at org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:319)
	at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:140)
	at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:95)
	at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
	at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:74)
	at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1515)
	at org.apache.spark.sql.rapids.GpuFileSourceScanExec.readerFactory$lzycompute(GpuFileSourceScanExec.scala:559)
	at org.apache.spark.sql.rapids.GpuFileSourceScanExec.readerFactory(GpuFileSourceScanExec.scala:553)
	at org.apache.spark.sql.rapids.GpuFileSourceScanExec.getFinalRDD(GpuFileSourceScanExec.scala:548)
	at org.apache.spark.sql.rapids.GpuFileSourceScanExec.createNonBucketedReadRDD(GpuFileSourceScanExec.scala:535)
	at org.apache.spark.sql.rapids.GpuFileSourceScanExec.inputRDD$lzycompute(GpuFileSourceScanExec.scala:337)
	at org.apache.spark.sql.rapids.GpuFileSourceScanExec.inputRDD(GpuFileSourceScanExec.scala:313)
	at org.apache.spark.sql.rapids.GpuFileSourceScanExec.doExecuteColumnar(GpuFileSourceScanExec.scala:414)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeColumnar$1(SparkPlan.scala:211)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:222)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:219)
	at org.apache.spark.sql.execution.SparkPlan.executeColumnar(SparkPlan.scala:207)
	at com.nvidia.spark.rapids.GpuColumnarToRowExec.doExecute(GpuColumnarToRowExec.scala:320)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:184)
	at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:222)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:219)
	at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:180)
	at org.apache.spark.sql.execution.SparkPlan.getByteArrayRdd(SparkPlan.scala:325)
	at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:443)
	at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:429)
	at org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:48)
	at org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:421)
	at org.apache.spark.sql.execution.HiveResult$.hiveResultString(HiveResult.scala:76)
	at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.$anonfun$run$2(SparkSQLDriver.scala:69)
	at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:103)
	at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163)
	at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:90)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
	at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
	at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:69)
	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:384)
	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.$anonfun$processLine$1(SparkSQLCLIDriver.scala:504)
	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.$anonfun$processLine$1$adapted(SparkSQLCLIDriver.scala:498)
	at scala.collection.Iterator.foreach(Iterator.scala:943)
	at scala.collection.Iterator.foreach$(Iterator.scala:943)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
	at scala.collection.IterableLike.foreach(IterableLike.scala:74)
	at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processLine(SparkSQLCLIDriver.scala:498)
	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:287)
	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
	at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
	at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
	at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
	at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
	at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.rapids.execution.SerializeConcatHostBuffersDeserializeBatch
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 77 more

what type of spark cluster?  Our CDH tests run on yarn and run with the Kryo serializer turned on.

@tgravescs my reproduce is a single node spark standalone cluster

The kryoserializer uses a different classloader:
val classLoader = defaultClassLoader.getOrElse(Thread.currentThread.getContextClassLoader)

Utils.withContextClassLoader(classLoader) {

this is a bit weird case where we have GpuKryoRegistrator outside of the parallel world type classes (com/nvidia/spark/rapids/GpuKryoRegistrator.class) but then in the code it calls it registers classes that are inside the parallel world loaders, like:
spark3xx-common/org/apache/spark/sql/rapids/execution/SerializeConcatHostBuffersDeserializeBatch.class
Since we are calling out to kryo register with this it seems like this should be public classes not in the parallel world but investigating more

22/08/31 13:37:27 WARN ShimLoader: loader is: sun.misc.Launcher$AppClassLoader@4bf558aa
22/08/31 13:37:34 WARN KryoSerializer: classloader is: scala.tools.nsc.interpreter.IMain$TranslatingClassLoader@50b0afd7 with parent scala.reflect.internal.util.ScalaClassLoader$URLClassLoader@65d23aa3, current is scala.tools.nsc.interpreter.IMain$TranslatingClassLoader@50b0afd7
As above in this case its using the repl classloader

authors: tgravescs viadea


text:
[Spark 4.0] Address test failures in cast_test.py [databricks]

Fixes #11009.
This commit addresses the test failures in cast_test.py, on Spark 4.0.
These generally have to do with changes in behaviour of Spark when
ANSI mode is enabled.  In these cases, the tests have been split out into ANSI=on and ANSI=off.
The bugs uncovered from the tests have been spun into their own issues;  fixing all of them was beyond the scope of this change.

Build

Build

Build

I've merged this change.  Thank you for the review, @gerashegalov.


text:
Make InternalExclusiveModeGpuDiscoveryPlugin as protected class.

This fixes https://github.com/NVIDIA/spark-rapids/issues/4022.
Went through the docs and checked all the public classes which can be set. In this PR, we are making InternalExclusiveModeGpuDiscoveryPlugin as protected as it should be loaded only through reflection.
Updated the comments for  ParquetCachedBatchSerializer as well.
Rest of the public classes - GpuKryoRegistrator and RapidsShuffleManager doesn't require any changes.

from the classes referenced in ShimLoader via newInstanceOf it looks we need to cover com.nvidia.spark.rapids.ExplainPlanImpl as well.

from the classes referenced in ShimLoader via newInstanceOf it looks we need to cover com.nvidia.spark.rapids.ExplainPlanImpl as well.

Thanks, @gerashegalov  for pointing this out. Updated the PR.  PTAL.

build

Thanks @gerashegalov  for the review.

build


text:
Move to the cudf-0.15 release

Changes our dependency to be the official cudf 0.15 release,

build

Can we update all the 0.15-SNAPSHOT references in the docs while we're at it?

There are also references to 0.15-SNAPSHOT under jenkins/

Can we update the jenkins and docs files for this version change?

find . -type f | xargs grep -l 0.15-SNAPSHOT returns nothing.

build

authors: jlowe kuhushukla revans2


text:
Enable automerge 22.12 to 23.02 [skip ci]

Signed-off-by: Peixin Li pxli@nyu.edu
Automerge from branch-22.12 to branch-23.02

build


text:
Add nvidia-kmod-headers to "Artifacts" list for each stream

Without this, installing the "src" profile for any branch would
always pull in the latest available nvidia-kmod-headers package.
For instance, "dnf module install nvidia-driver;450/src" would
install nvidia-kmod-headers-470.57.02-1.el8.x86_64.rpm as of
today (August 9, 2021).
Signed-off-by: Jamie Nguyen jamien@nvidia.com

Hi Jamie, it looks good but I will need to wait until Wednesday to test and merge the change.

For what it's worth, here's some output from my own unit test:
[lab@rhel8-kvm-1v-42 ~]$ sudo dnf module install nvidia-driver:470/src
Updating Subscription Management repositories.
custom                                                                                3.4 MB/s | 3.5 kB     00:00
Dependencies resolved.
=========================================================================================================================================
 Package                                Architecture              Version                                Repository                 Size
=========================================================================================================================================
Installing group/module packages:
 nvidia-kmod-headers                    x86_64                    3:470.57.02-1.el8                      custom                     25 M
Installing module profiles:
 nvidia-driver/src
Enabling module streams:
 nvidia-driver                                                    470

Transaction Summary
=========================================================================================================================================
Install  1 Package

...
[lab@rhel8-kvm-1v-42 ~]$ sudo dnf module install nvidia-driver:460/src
Updating Subscription Management repositories.
Last metadata expiration check: 0:00:11 ago on Tue 10 Aug 2021 07:04:40 AM PDT.
Dependencies resolved.
=========================================================================================================================================
 Package                                Architecture              Version                                Repository                 Size
=========================================================================================================================================
Installing group/module packages:
 nvidia-kmod-headers                    x86_64                    3:460.91.03-1.el8                      custom                     24 M
Installing module profiles:
 nvidia-driver/src
Enabling module streams:
 nvidia-driver                                                    460

Transaction Summary
=========================================================================================================================================
Install  1 Package

...
[lab@rhel8-kvm-1v-42 ~]$ sudo dnf module install nvidia-driver:450/src
Updating Subscription Management repositories.
Last metadata expiration check: 0:00:23 ago on Tue 10 Aug 2021 07:04:40 AM PDT.
Dependencies resolved.
=========================================================================================================================================
 Package                                Architecture              Version                                Repository                 Size
=========================================================================================================================================
Installing group/module packages:
 nvidia-kmod-headers                    x86_64                    3:450.142.00-1.el8                     custom                     12 M
Installing module profiles:
 nvidia-driver/src
Enabling module streams:
 nvidia-driver                                                    450

Transaction Summary
=========================================================================================================================================
Install  1 Package

Validated the resultant YAML, merging. Thank you Jamie!

authors: jamieNguyenNVIDIA kmittman

repo: NVIDIA/yum-packaging-precompiled-kmod


text:
peter jacksons king kong unexpected error

Describe the bug
peter jacksons king kong has unexpected error when booting nvidia remix
Remix 0.2
rtx 3070 ti
latest GPU drivers
other
Attach logs!
NvRemixBridge.log
d3d9.log
SettingsApplication_d3d9.log
KingKong8_d3d9.log
CheckApplication_d3d9.log
there are 3 ways of opening the game so included all 3 logs
To Reproduce
Steps to reproduce the behavior:
click on exe
Expected behavior
game should run
Screenshots
If applicable, add screenshots to help explain your problem.

Hello! Will you please test this on the latest release and let us know if the issue persists? Thank you!

Thank you for reaching out. We're going to close this ticket due to an extended period with no response. If the issue persists for you on the latest build, please open a new ticket.

authors: NV-LL howdoicodeplease

repo: NVIDIAGameWorks/rtx-remix


text:
Change: don't error log for 404s

Changes what is logged when bib id doesn't match a bib to issue a INFO
level log rather than an ERROR level log (which was causing lots of
false positives in our alarms). Includes tests asserting the new policy.
Also cleans up README to clarify installation and contributing flow. Also
cleans up swagger document to add several missing params, retire unused
swagger files, and rename it to better reflect its relationship with the
app version.

I think you're right that the call to logger.info should happen in the common error handler because we probably want all 404s treated in this way - and moving it there places it next to the logger.error call we're making for more fatal errors. I moved it.

repo: NYPL/discovery-api


text:
DSD-400: Jest storybook integration

Fixes JIRA ticket DSD-400
This PR does the following:

Adds a plugin to Storybook to integrate it with our current Jest unit tests.
This doesn't add any new tests but only combines a Storybook component with its unit tests. They can be found in the addons panel under the "Test" tab:

While you can't see the test code itself, you can see the description of a test. Altogether, you can see how a component is being tested and it will help us figure out if we're missing any tests for a component.

How has this been tested?
Locally through Storybook.
Checklist:

[ ] I have updated the Storybook documentation accordingly.
[ ] All new and existing tests passed.

Front End Review:

[ ] View the example in Storybook

As far as I could tell, no, there's no way to format it unless if we update a test file's describe and it clauses. The way it auto-format it could also let us know if any tests don't make sense or if we should update the language so it's consistent.
The last commit makes the tests show up in the preview build. The development workflow automatically picks up the test name but for production, it needs to be explicit. In the last commit, you can see the integration for every test file. It's a bit annoying but after this first step, it should be smoother for newer components.

repo: NYPL/nypl-design-system


text:
new problem with velocity

java.lang.NullPointerException: Cannot invoke "com.google.gson.JsonElement.getAsString()" because the return value of "com.google.gson.JsonObject.get(String)" is null
        at com.namelessmc.plugin.lib.nameless-api.Website.<init>(Website.java:45) ~[?:?]
        at com.namelessmc.plugin.lib.nameless-api.NamelessAPI.getWebsite(NamelessAPI.java:83) ~[?:?]
        at com.namelessmc.plugin.common.ApiProvider.api(ApiProvider.java:94) ~[?:?]
        at com.velocitypowered.proxy.scheduler.VelocityScheduler$VelocityTask.lambda$run$1(VelocityScheduler.java:203) ~[velocity.jar:3.1.2-SNAPSHOT (git-3ae93875-b138)]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]
>

Originally posted by @Khikki in https://github.com/NamelessMC/Nameless-Plugin/issues/254#issuecomment-1126657556

Your website is not sending the "locale" in the json response, this was added a couple weeks ago. Please note that the development version of the plugin is only compatible with the latest development version of NamelessMC

Can you tell me how can I install the latest version of nameless?

Development releases can be downloaded from https://ci.rkslot.nl/view/NamelessMC/job/Nameless/ these should not be used for anything other than testing purposes. Upgrades are not supported, frequent reinstallations are required

I tried to install my theme and module in the new version for test, otherwise localization will be called?
https://debug.namelessmc.com/s4lylEhoPK

Templates and modules made for pr12 are very likely incompatible with the development version. In this case it looks like it is missing files for the new language system

I'll close this issue since it seems to be resolved

authors: Derkades Khikki

repo: NamelessMC/Nameless-Plugin


text:
Parallel decryption in case of many items

Hello guys,
We have noticed that our app startup is very very slow (~23sec in case of 33 items). The problem is because of credstash config building.
With this change we were able to reduce it to ~5sec.
From our tests it looks like that like 10 is the correct number for degree of parallelism.
Thanks,
Peter

I'm not 100% sure I like parallel hitting the table as Dynamo can decide to error out if a request limit is hit.
I guess it doesn't hurt to make the degree of parallelism be a config value though.
I see you renamed async functions to be *Async.  I guess I should follow that convention :)

@adamhathcock you are right. the read unit capacity is important.

I added it as a configuration option.
Added a 2 more overloads for IConfigurationBuilder building
I've added xml description to the dop option, so i've ended up adding explanation for the rest
Visual Studio kept asking me for fix line endings, so I clicked OK in the popup

"you renamed async functions to be *Async"
Yeah, that's a general C# convention.
Thanks for the review!
Peter

authors: adamhathcock peterdeme


text:
Uncaught Exception, AddressOutOfBoundsException: Address Overflow in add: ffffefff + 0xefff

Describe the bug
Given the following partially defined memory map:

I'm trying to define the peripherals/IO section in Ghidra as spec'd on the IC's datasheet:

I'm getting the following unhandled exception:

In textual form, for reference:
Address Overflow in add: ffffefff + 0xefff
ghidra.program.model.address.AddressOutOfBoundsException: Address Overflow in add: ffffefff + 0xefff
	at ghidra.program.model.address.AbstractAddressSpace.add(AbstractAddressSpace.java:438)
	at ghidra.program.model.address.GenericAddressSpace.add(GenericAddressSpace.java:21)
	at ghidra.program.model.address.GenericAddress.add(GenericAddress.java:237)
	at ghidra.app.plugin.core.memory.AddBlockModel.hasNoMemoryConflicts(AddBlockModel.java:393)
	at ghidra.app.plugin.core.memory.AddBlockModel.validateInfo(AddBlockModel.java:297)
	at ghidra.app.plugin.core.memory.AddBlockModel.execute(AddBlockModel.java:240)
	at ghidra.app.plugin.core.memory.AddBlockDialog.okCallback(AddBlockDialog.java:338)
	at docking.DialogComponentProvider.lambda$addOKButton$0(DialogComponentProvider.java:447)
	at java.desktop/javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1967)
	at java.desktop/javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2308)
	at java.desktop/javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405)
	at java.desktop/javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262)
	at java.desktop/javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:279)
	at java.desktop/java.awt.Component.processMouseEvent(Component.java:6631)
	at java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3342)
	at java.desktop/java.awt.Component.processEvent(Component.java:6396)
	at java.desktop/java.awt.Container.processEvent(Container.java:2263)
	at java.desktop/java.awt.Component.dispatchEventImpl(Component.java:5007)
	at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2321)
	at java.desktop/java.awt.Component.dispatchEvent(Component.java:4839)
	at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4918)
	at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4547)
	at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4488)
	at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2307)
	at java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2772)
	at java.desktop/java.awt.Component.dispatchEvent(Component.java:4839)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:772)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:715)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:95)
	at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:745)
	at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:743)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
	at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:742)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:117)
	at java.desktop/java.awt.WaitDispatchSupport$2.run(WaitDispatchSupport.java:190)
	at java.desktop/java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:235)
	at java.desktop/java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:233)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.desktop/java.awt.WaitDispatchSupport.enter(WaitDispatchSupport.java:233)
	at java.desktop/java.awt.Dialog.show(Dialog.java:1070)
	at java.desktop/java.awt.Component.show(Component.java:1716)
	at java.desktop/java.awt.Component.setVisible(Component.java:1663)
	at java.desktop/java.awt.Window.setVisible(Window.java:1031)
	at java.desktop/java.awt.Dialog.setVisible(Dialog.java:1005)
	at docking.DockingDialog.setVisible(DockingDialog.java:354)
	at docking.DockingWindowManager.lambda$doShowDialog$6(DockingWindowManager.java:1701)
	at ghidra.util.Swing.doRun(Swing.java:292)
	at ghidra.util.Swing.runNow(Swing.java:208)
	at ghidra.util.Swing.runNow(Swing.java:163)
	at docking.DockingWindowManager.doShowDialog(DockingWindowManager.java:1705)
	at docking.DockingWindowManager.showDialog(DockingWindowManager.java:1739)
	at docking.DockingWindowManager.showDialog(DockingWindowManager.java:1680)
	at ghidra.framework.plugintool.PluginTool.showDialog(PluginTool.java:1438)
	at ghidra.app.plugin.core.memory.AddBlockDialog.showDialog(AddBlockDialog.java:325)
	at ghidra.app.plugin.core.memory.MemoryMapProvider.showAddBlockDialog(MemoryMapProvider.java:663)
	at ghidra.app.plugin.core.memory.MemoryMapProvider$1.actionPerformed(MemoryMapProvider.java:194)
	at docking.menu.ToolBarItemManager.lambda$actionPerformed$0(ToolBarItemManager.java:128)
	at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:715)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
	at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:740)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)

---------------------------------------------------
Build Date: 2020-nov.-24 1908 AEDT
Ghidra Version: 9.3
Java Home: /Library/Java/JavaVirtualMachines/amazon-corretto-11.jdk/Contents/Home
JVM Version: Amazon.com Inc. 11.0.7
OS: Mac OS X 10.15.7 x86_64
Workstation: localhost

The stack trace appears to be a problem in the error detection (which I will correct), however based upon your dialog it appears you attempted to create a block larger than the available space.  With a start of 0xFFFFF000 (and a 32-bit address space) the largest block would be 0x1000 in length.  A length 0xF000 would extend beyond the bounds of the address space.  The block you are attempting to define (based on the graphic map you specified) should have a start address of 0xFFFFF000 with a length of 0x1000

Error checking fix will be included in 9.2.2

Thanks @ghidra1!
I suspect that what confused me were the negative addresses and how to properly defined them in Ghidra's memory mapper (those below the ROM region)?
In principle the figure 3-14 shown above should be possible to define in Ghidra, right? I've heard that certain microcontroller memory arrangements/mappings are still not possible in Ghidra, but I'm not fully aware of the current limitations and specific corner cases where that's not possible?

Thanks @ghidra1!
I suspect that what confused me were the negative addresses and how to properly defined them in Ghidra's memory mapper (those below the ROM region)?
In principle the figure 3-14 shown above should be possible to define in Ghidra, right? I've heard that certain microcontroller memory arrangements/mappings are still not possible in Ghidra, but I'm not fully aware of the current limitations and specific corner cases where that's not possible?

Mapping memory regions may help in the listing view but frequently have issues with pointer de-referencing and decompiler results.  In addition, such architectures also present challenges for the image Loaders which may not properly handle the virtual vs. physical memory distinction and the various processor variants which may have an altered memory arrangement.

Mapping memory regions may help in the listing view but frequently have issues with pointer de-referencing and decompiler results.  In addition, such architectures also present challenges for the image Loaders which may not properly handle the virtual vs. physical memory distinction and the various processor variants which may have an altered memory arrangement.

Thanks @ghidra1, what you write makes sense and I think I understand what you are alluding to w.r.t to pointer de-ref & co (related to decompiling).
But my issue is before reaching analysis stage.
For instance, given this alternative linear address space for the same processor (without the "weird" wrap-around I mentioned above):

It's not easily definable as-is on the memory manager, see:
>>> memory = currentProgram.getMemory()
>>> fb = memory.getAllFileBytes()
>>> blk = memory.getBlock(toAddr(0x0))
>>> memory.removeBlock(blk, monitor)
>>>
>>> memory.createUninitializedBlock("internal_ram",toAddr(0x3ff8000),0x2fff,False)
>>> memory.createUninitializedBlock("peripherals", toAddr(0x3ffefff),0xfff, False)
>>> memory.createInitializedBlock("rom", toAddr(0x0), fb[0], 0, 0x1000000, False)
Traceback (most recent call last):
  File "python", line 1, in <module>
	at ghidra.program.database.mem.MemoryMapDB.checkBlock(MemoryMapDB.java:1043)
	at ghidra.program.database.mem.MemoryMapDB.removeBlock(MemoryMapDB.java:1850)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Blocks do not belong to this program
ghidra.program.database.mem.MemoryBlockDB@6f03ae13
ghidra.program.database.mem.MemoryBlockDB@38d5ac1c
ghidra.program.database.mem.MemoryBlockDB@6ad8b9ac

What would be the correct sequence of arguments/flags/(instruction ordering?) to have a correct representation of a loaded V850 16MB firmware (size 0x1000000) on Ghidra?

I suspect blk is null and is the cause of the error above.
I am rather confused by what you are attempting to achieve with your script and what your initial state is before running the script.  You will really need to debug your script.

Argh, true, thanks @ghidra1. The script assumes the memory map has the initial default mapping, but when it's empty it obviously goes that that way.
Now, I'll carry on and script the rewrite of instruction opcodes so they reference to the new remapped mem regions, yay :)

repo: NationalSecurityAgency/ghidra


text:
CParser documentation incorrectly states CParser will use archive files for undefined types

The help file notes:
https://github.com/NationalSecurityAgency/ghidra/blob/2b08598dbabe6a9c911e712bd928e5f84fb00ee8/Ghidra/Features/Base/src/main/help/help/topics/CParserPlugin/CParser.htm#L155-L156
However, @emteere notes in https://github.com/NationalSecurityAgency/ghidra/issues/716#issuecomment-505419863:

Currently parsing doesn't pull the types from the already open archives until the header files are all parsed. Then a reconciliation of equivalent types is done on the newly parsed data types.

and #716 provides an example and workaround. Am I interpreting this correctly as an error in documentation? (At least, that is, unless the suggestion from @emteere in that issue is implemented?) Happy to submit a simple documentation PR if others agree.

Parsing headers would get a lot easier if you could just use the generic c lib archives for the types in standard headers.

Yes, that’s true, that’s what @emteere suggested in #716, and that would be a useful addition. In the meantime, it seems that’s what is stated in the help document without being what actually happens. I’m recommending starting by fixing the documentation rather than waiting for that to be implemented. If others think that’s reasonable, I’m happy to edit and submit a PR.

I think a PR to update the documentation would be really helpful, right now it's very misleading (and did cost me quite some time...)

This will be addressed in an upcoming branch to be pushed to master.

authors: astrelsky emteere ljrk0 therealchjones


text:
Will this work if I use JavaScript instead of TypeScript?

Hi,
I want to use this npm package and I love it. I am wondering if this will work with JS in react?
Furthermore, I went through the usage example and converted my code to JS, but I get a lot of errors.
Also, I am wondering if the usage example is correct?
Thanks!

Yes, thank you for your input, I had a small fix to make in th documentation and the demo,
please install v1.1.25 latest version.
Here is a working example using react-vite with javascript only -
`import { CardSwiper } from "react-card-swiper"
import ReactSvg from "../src/assets/react.svg"
const Content = () => (

    Lorem ipsum dolor sit amet.

)
function App() {
const mockData = [{ id: '88552078', meta: { apk: 'some-apk-a.apk' }, src: ReactSvg, content:  }]
return (

<CardSwiper data={mockData} emptyState={Empty State} />

)
}
export default App
`

repo: NativSibony/react-card-swiper


text:
ngOnDestroy not called on Android back button

We ran into an issue with ngOnDestroy not being called when we use the Android back button.
I've provided a sample application:
https://github.com/dennis-montana/back-button-issue
If you run the app on Android and click the back button you can see that ngOnDestroy isn't called while the native activityDestroyedEvent from Android is being called.
The app is still running in a sleeping state so everything created in ngOnInit is being created again every time a user uses the back button.
If you run the demo app you can see the following output:
### Opened the app 1st time
JS: AppComponent ngOnInit

### Clicked back button
JS: Android activity destroyed

### Opened the app 2nd time
JS: handleResumeEvent
JS: AppComponent ngOnInit

### Clicked back button
JS: Android activity destroyed
JS: Android activity destroyed

### Opened the app 3rd time
JS: handleResumeEvent
JS: handleResumeEvent
JS: AppComponent ngOnInit

### Clicked back button
JS: Android activity destroyed
JS: Android activity destroyed
JS: Android activity destroyed

As you can see the calls are being stacked.

@dennis-montana @spike1292
The observed behaviour is by design and expected as documented here

With router-outlet new instance of the component is created with each navigation and is destroyed when you navigate to another component. The component's constructor and its init hooks will be called every time you navigate to the component and ngOnDestroy() will be called every time you navigate away from it.
With page-router-outlet when you navigate forward, the current page and views are saved in the native navigation stack. The corresponding component is not destroyed. It is cached, so that it can be shown when you go back to the same page. It will still be connected to the views that were cached natively.

@NickIliev thank you for your reply.
I've read the documentation you've send but it isn't completely clear to me how it describes the Android back button behaviour when there isn't really a component to go back to. When you click back you go the the android home screen.
So when I click back I expect ngOnDestroy to be called or ngOnInit shouldn't be called when I open the app.
Could you elaborate on how to solve this issue?

@dennis-montana perhaps, instead of the Angular life hooks, you can use the NativeScript application events for this case (or directly the Android events).

I think the solution to that is to handle the activity-destroyed application event and actually kill the angular application when the event is fired. This should call the ngOnDestroy hook for all active components.

I solved this in our app by directly using the Android application event to call ngOnDestroy.
android.on(AndroidApplication.activityDestroyedEvent, (args: AndroidActivityEventData) => {
   if (args.activity.toString().startsWith('com.tns.NativeScriptActivity')) {
      this.ngOnDestroy();
   }
});

Hi @dennis-montana  ,
Facing the same issue you described above, where exactly have you declared this event ?

@hdrdiab you can place this within your app.component.ts.

For now i've placed it inside the ngOnInit method, and it is working fine.

Hi,
The solution @dennis-montana proposed is not working anymore for me.
My component have multiple child components in its view , calling ngOnDestroy on the parent component isn't destroying the children.
Can we can expose the clearCache() method of the NSRouteReuseStrategy ? I think using it will get us a reliable way to kill the angular application.

Status on this issue?

@nuvoPoint
This is still an issue.
We've run into it in our application.
I've created a test project here:
https://github.com/m-abs/tns-ng-app-lifecycle
Steps to reproduce:

Run the app with tns run android --emulator --bundle --env.aot
Navigate to one of the items. You should see the ngOnInit() logged to console for the components.
Tap the native back button until the app exists. Notice that ngOnDestroy() is NOT logged to the console.

Now if you run npm run apply-patch (this requires bash) and repeat.
Now after step 3 you should see the ngOnDestroy() in the console.

I wrote an article about this. I solved it in generic way with the decorator.

authors: NickIliev Stanteq dennis-montana hdrdiab jogboms m-abs nuvoPoint vakrilov

repo: NativeScript/nativescript-angular