text:
NullPointerException on PhpStorm startup

After installing gitignore 1.7.5, I get the following when launching PhpStorm.
PhpStorm 2016.3.2
Build #PS-163.10504.2, built on December 20, 2016
JRE: 1.8.0_112-release-408-b6 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Plugin 'mobi.hsz.idea.gitignore' failed to initialize and will be disabled.  Please restart PhpStorm.

java.lang.NullPointerException
    at mobi.hsz.idea.gitignore.IgnoreApplicationComponent.initComponent(IgnoreApplicationComponent.java:59)
    at com.intellij.openapi.components.impl.ComponentManagerImpl$ComponentConfigComponentAdapter.getComponentInstance(ComponentManagerImpl.java:501)
    at com.intellij.openapi.components.impl.ComponentManagerImpl.createComponents(ComponentManagerImpl.java:125)
    at com.intellij.openapi.application.impl.ApplicationImpl.a(ApplicationImpl.java:444)
    at com.intellij.openapi.progress.impl.CoreProgressManager$3.run(CoreProgressManager.java:179)
    at com.intellij.openapi.progress.impl.CoreProgressManager.a(CoreProgressManager.java:568)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:519)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:54)
    at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:164)
    at com.intellij.openapi.application.impl.ApplicationImpl.createComponents(ApplicationImpl.java:451)
    at com.intellij.openapi.components.impl.ComponentManagerImpl.init(ComponentManagerImpl.java:109)
    at com.intellij.openapi.application.impl.ApplicationImpl.load(ApplicationImpl.java:403)
    at com.intellij.openapi.application.impl.ApplicationImpl.load(ApplicationImpl.java:389)
    at com.intellij.idea.IdeaApplication.run(IdeaApplication.java:190)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.idea.MainImpl$1$1.a(MainImpl.java:69)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
    at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:363)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

same here:
IntelliJ IDEA 2017.1 EAP
Build #IU-171.3019.7, built on February 7, 2017
IntelliJ IDEA EAP User
Expiration date: March 9, 2017
JRE: 1.8.0_112-release-b702 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Mac OS X 10.12.3

version 1.7.5 works as expected for me:
IntelliJ IDEA (Community Edition) 2017.1 EAP
Build #IC-171.3019.7, built on February 7, 2017
JRE: 1.8.0_121-b13 amd64
JVM: Java HotSpot(TM) 64-Bit Server VM by Oracle Corporation
Windows 10 10.0

It doesn't work for me:
WebStorm 2016.3.3
Build #WS-163.12024.17, built on January 31, 2017
Licensed to ***
Subscription is active until ***
For educational use only.
JRE: 1.8.0_112-release-408-b6 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o

A had the same issue earlier with WebStorm (last version) on Windows but I don't have this computer at hand at the moment to post its WebStorm version.

Doesn't work for me in version 1.7.5 on Android Studio:
Android Studio 2.2.3
Build #AI-145.3537739, built on December 2, 2016
JRE: 1.8.0_112-release-b05 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o

Me again.... version 1.7.5 works like a charm on Android Studio 2.3 Beta 3 / Windows 10:
Android Studio 2.3 Beta 3
Build #AI-162.3671887, built on January 25, 2017
JRE: 1.8.0_112-release-b06 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o

Also doesn't work:
PhpStorm 2016.3.2
Build #PS-163.10504.2, built on December 20, 2016
JRE: 1.8.0_112-release-408-b6 x86
JVM: OpenJDK Server VM by JetBrains s.r.o

Crashing for me too:
WebStorm 2016.3.3
Build #WS-163.12024.17, built on January 31, 2017
Subscription is active until <Someday>
JRE: 1.8.0_112-release-408-b6 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o

On Linux, the stacktrace is the following:
Plugin 'mobi.hsz.idea.gitignore' failed to initialize and will be disabled.  Please restart WebStorm.

java.lang.NullPointerException
    at mobi.hsz.idea.gitignore.IgnoreApplicationComponent.initComponent(IgnoreApplicationComponent.java:59)
    at com.intellij.openapi.components.impl.ComponentManagerImpl$ComponentConfigComponentAdapter.getComponentInstance(ComponentManagerImpl.java:501)
    at com.intellij.openapi.components.impl.ComponentManagerImpl.createComponents(ComponentManagerImpl.java:125)
    at com.intellij.openapi.application.impl.ApplicationImpl.a(ApplicationImpl.java:444)
    at com.intellij.openapi.progress.impl.CoreProgressManager$3.run(CoreProgressManager.java:179)
    at com.intellij.openapi.progress.impl.CoreProgressManager.a(CoreProgressManager.java:568)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:519)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:54)
    at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:164)
    at com.intellij.openapi.application.impl.ApplicationImpl.createComponents(ApplicationImpl.java:451)
    at com.intellij.openapi.components.impl.ComponentManagerImpl.init(ComponentManagerImpl.java:109)
    at com.intellij.openapi.application.impl.ApplicationImpl.load(ApplicationImpl.java:403)
    at com.intellij.openapi.application.impl.ApplicationImpl.load(ApplicationImpl.java:389)
    at com.intellij.idea.IdeaApplication.run(IdeaApplication.java:190)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.idea.MainImpl$1$1.a(MainImpl.java:68)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
    at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:363)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

I have php Storm at home with version 2016.3.2 and worked. I updated to 2016.3.3 and still works.
But here at work it not working. Both are Ubuntu. Maybe a incompatibility with some other plugin?
PhpStorm 2016.3.1
Build #PS-163.9735.1, built on December 6, 2016
JRE: 1.8.0_112-release-408-b2 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
PhpStorm 2016.3.2
Build #PS-163.10504.2, built on December 20, 2016
JRE: 1.8.0_112-release-408-b6 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o

Same here
PyCharm Community 2016.3.2
Older version is working
Plugin 'mobi.hsz.idea.gitignore' failed to initialize and will be disabled.  Please restart PyCharm Community Edition.

java.lang.NullPointerException
    at mobi.hsz.idea.gitignore.IgnoreApplicationComponent.initComponent(IgnoreApplicationComponent.java:59)
    at com.intellij.openapi.components.impl.ComponentManagerImpl$ComponentConfigComponentAdapter.getComponentInstance(ComponentManagerImpl.java:501)
    at com.intellij.openapi.components.impl.ComponentManagerImpl.createComponents(ComponentManagerImpl.java:125)
    at com.intellij.openapi.application.impl.ApplicationImpl.lambda$createComponents$8(ApplicationImpl.java:444)
    at com.intellij.openapi.progress.impl.CoreProgressManager$3.run(CoreProgressManager.java:179)
    at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:568)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:519)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:54)
    at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:164)
    at com.intellij.openapi.application.impl.ApplicationImpl.createComponents(ApplicationImpl.java:451)
    at com.intellij.openapi.components.impl.ComponentManagerImpl.init(ComponentManagerImpl.java:109)
    at com.intellij.openapi.application.impl.ApplicationImpl.load(ApplicationImpl.java:403)
    at com.intellij.openapi.application.impl.ApplicationImpl.load(ApplicationImpl.java:389)
    at com.intellij.idea.IdeaApplication.run(IdeaApplication.java:190)
    at com.intellij.idea.MainImpl$1.lambda$null$0(MainImpl.java:47)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
    at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:363)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

The same happens in Android Studio

IntelliJ IDEA 2016.3.4
Build #IU-163.12024.16, built on January 31, 2017
Crashes every time.

same here in Android Studio 2.3 Beta 4, trying to freshly install .ignore 1.7.5
Plugin 'mobi.hsz.idea.gitignore' failed to initialize and will be disabled.  Please restart Android Studio.
java.lang.NullPointerException
at mobi.hsz.idea.gitignore.IgnoreApplicationComponent.initComponent(IgnoreApplicationComponent.java:59)
at com.intellij.openapi.components.impl.ComponentManagerImpl$ComponentConfigComponentAdapter.getComponentInstance(ComponentManagerImpl.java:518)
at com.intellij.openapi.components.impl.ComponentManagerImpl.createComponents(ComponentManagerImpl.java:119)
at com.intellij.openapi.application.impl.ApplicationImpl.lambda$createComponents$8(ApplicationImpl.java:429)
at com.intellij.openapi.progress.impl.CoreProgressManager$3.run(CoreProgressManager.java:170)
at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:494)
at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:443)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:54)
at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:155)
at com.intellij.openapi.application.impl.ApplicationImpl.createComponents(ApplicationImpl.java:436)
at com.intellij.openapi.components.impl.ComponentManagerImpl.init(ComponentManagerImpl.java:103)
at com.intellij.openapi.application.impl.ApplicationImpl.load(ApplicationImpl.java:392)
at com.intellij.openapi.application.impl.ApplicationImpl.load(ApplicationImpl.java:384)
at com.intellij.idea.IdeaApplication.run(IdeaApplication.java:196)
at com.intellij.idea.MainImpl$1.lambda$null$0(MainImpl.java:47)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.java:358)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

Same exact bug and stack-trace as OP, except for Intellij IDEA on macOS Sierra 10.12.3.

IntelliJ IDEA 2016.3.4
Build #IU-163.12024.16, built on January 31, 2017
Licensed to Doctor Stranger
Subscription is active until November 15, 2017
JRE: 1.8.0_112-release-408-b6 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o

It doesn't work.
1.7.5 on Arch Linux with PhpStorm 2016.3 and 2017.1 EAP

The same situation with me on Ubuntu 16.04 with PphStorm 2016.3

Finally after a quite long time of unexpected absence in this project I was able to go back to GitHub and release this bugfix.
v1.7.6 has just arrived to the plugins repository
_Because of a plenty of duplicates, I'll just call the others (upvoters, reporters, commenters...) to just let you know.
@abcoep @ahonymous @aimarchenko @akshaybabloo @andjelx @aliuspetraska @amiram  @c-jonua @derentenpopel @dericcain @Didldu-Florian @digish @dmlo @Erchard @epfremmer @evandertino @evgen-d @fatcats @gabrielgio @Grafe89 @Hexeption @hosseinkhaledi @ingvard @jason-chang @k4reln @keqingyuan @khs1994 @kraluk @krarey @mafzst @marcelopm @MarcelTon @mhz-tamb @minhphuc429 @mseleng @Nemmo @ngdo-pro @nunesvictor @OndrejSpanel @Pafishek @Pagestylist @pedroadame @philippe2803 @plombardi89 @planet17 @pymob @rcapile @RazorNd @rayk @RIP21 @sadig @sambadevi @ser-kletrov @skobeeffdev @snooze6 @sparsick @szewczyk-it @TangRufus @tonystatic @tknarr @Trak-X @trixanapp @Xhelas @XuluWarrior @yanzhenxiang @yizeliang @wangcongyi @wizaord @techstay @petenattress @mark-orussa @zeroarst @bllfrnch @martinezleoml @joaquingatica @dejan2609 @mqwerty @romic96 @pschichtel @rusAnthrax @auval @einnuj @davidmpaz
_

cc @abcoep @ahonymous @aimarchenko @akshaybabloo @andjelx @aliuspetraska @amiram  @c-jonua @derentenpopel @dericcain @Didldu-Florian @digish @dmlo @Erchard @epfremmer @evandertino @evgen-d @fatcats @gabrielgio @Grafe89 @Hexeption @hosseinkhaledi @ingvard @jason-chang @k4reln @keqingyuan @khs1994 @kraluk @krarey @mafzst @marcelopm @MarcelTon @mhz-tamb @minhphuc429 @mseleng @Nemmo @ngdo-pro @nunesvictor @OndrejSpanel @Pafishek @Pagestylist @pedroadame @philippe2803 @plombardi89 @planet17 @pymob @rcapile @RazorNd @rayk @RIP21 @sadig @sambadevi @ser-kletrov @skobeeffdev @snooze6 @sparsick @szewczyk-it @TangRufus @tonystatic @tknarr @Trak-X @trixanapp @Xhelas @XuluWarrior @yanzhenxiang @yizeliang @wangcongyi @wizaord @techstay @petenattress @mark-orussa @zeroarst @bllfrnch @martinezleoml @joaquingatica @dejan2609 @mqwerty @romic96 @pschichtel @rusAnthrax @auval @einnuj @davidmpaz

@sadig @sambadevi @ser-kletrov @skobeeffdev @snooze6 @sparsick @szewczyk-it @TangRufus @tonystatic @tknarr @Trak-X @trixanapp @Xhelas @XuluWarrior @yanzhenxiang @yizeliang @wangcongyi @wizaord @techstay @petenattress @mark-orussa @zeroarst @bllfrnch @martinezleoml @joaquingatica @dejan2609 @mqwerty @romic96 @pschichtel @rusAnthrax @auval @einnuj @davidmpaz

Perfect. No longer crashing my Android Studio. 👍

@hsz Working like a charm! Thanks for the effort and the fast pace!

Intellij IDEA 2016.3.4 - fine. Thanks for fast update!

@hsz big thanks for fix and update! Now everything is working perfectly :)

authors: Trak-X auval davidmpaz dejan2609 einnuj hsz joaquingatica kraluk martinezleoml mqwerty pedroadame pschichtel pschulten rcapile romic96 rusAnthrax snooze6 szewczyk-it

repo: hsz/idea-gitignore


text:
Polish char coding

Encoding of Polish characters did not work.  In place of Polish characters, incomprehensible letters appear

Hi Kinpio, thanks for reporting the issue. Could you give me some examples of prompts that you used so I can use them to test some potential fixes?

Hi,
just type "polish alphabet" and use Win+O.
"
A, �?, B, C, �?, D, E, Ę, F, G, H, I, J, K, L, ?�, M, N, ?�, O, ??, P, R,
S, ??, T, U, W, Y, Z, ??, ?�
"
it should look like:
"
A, Ą, B, C, Ć, D, E, Ę, F, G, H, I, J, K, L, Ł, M, N, Ń, O, Ó, P, R, S, Ś,
T, U, W, Y, Z, Ż, Ź
"
--
Piotr
Niniejsza korespondencja przeznaczona jest wyłącznie dla osoby lub
podmiotu, do którego jest zaadresowana i może zawierać treści chronione
przepisami prawa. Wgląd w treść wiadomości otrzymanej omyłkowo, dalsze jej
przekazywanie, rozpowszechnianie lub innego rodzaju wykorzystanie, bądź
podjęcie jakichkolwiek działań w oparciu o zawarte w niej informacje przez
osobę lub podmiot nie będący adresatem, jest niedozwolone. Odbiorca
korespondencji, który otrzymał ją omyłkowo, proszony jest o zawiadomienie
nadawcy i usunięcie tego materiału z komputera.
niedz., 5 lut 2023 o 11:17 Tadashi @.***> napisał(a):

Hi Kinpio, thanks for reporting the issue. Could you give me some examples
of prompts that you used so I can use them to test some potential fixes?
—
Reply to this email directly, view it on GitHub
https://github.com/htadashi/GPT3-AHK/issues/2#issuecomment-1417338903,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANIPBKXCCO5BQSYQAJ5GJ2LWV543NANCNFSM6AAAAAAURVXXKU
.
You are receiving this because you authored the thread.Message ID:
@.***>

Hi @kinpio. https://github.com/htadashi/GPT3-AHK/commit/896feb88df5ebcb53a12da2045963f0a46e76ae8 seems to fix the issue.

I confirm, issue is a thing of the past
Piotr
Piotr
Niniejsza korespondencja przeznaczona jest wyłącznie dla osoby lub
podmiotu, do którego jest zaadresowana i może zawierać treści chronione
przepisami prawa. Wgląd w treść wiadomości otrzymanej omyłkowo, dalsze jej
przekazywanie, rozpowszechnianie lub innego rodzaju wykorzystanie, bądź
podjęcie jakichkolwiek działań w oparciu o zawarte w niej informacje przez
osobę lub podmiot nie będący adresatem, jest niedozwolone. Odbiorca
korespondencji, który otrzymał ją omyłkowo, proszony jest o zawiadomienie
nadawcy i usunięcie tego materiału z komputera.
niedz., 5 lut 2023 o 18:57 Tadashi @.***> napisał(a):

Hi @kinpio https://github.com/kinpio. 896feb8
https://github.com/htadashi/GPT3-AHK/commit/896feb88df5ebcb53a12da2045963f0a46e76ae8
seems to fix the issue.
—
Reply to this email directly, view it on GitHub
https://github.com/htadashi/GPT3-AHK/issues/2#issuecomment-1418201688,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANIPBKTIVU4G4VPUCK3EZW3WV7SX5ANCNFSM6AAAAAAURVXXKU
.
You are receiving this because you were mentioned.Message ID:
@.***>

source: gharchive/issue

authors: htadashi kinpio

license: MIT

license_type: permissive

license_source: github-api


text:
Fix image offset for iOS 12+

This pull request fixes the image offset, based on the work in the cpbitmap.github.io project.
Before:

After:

Thx @dnicolson
Does it mean for ios prior to 12 it will fail now ?
Can it be detected or support prior ios 12 version behind a flag ?

Based on the linked project it probably won't work for iOS 11 and earlier. Similar to the linked project, a command line switch could be added to support earlier versions of iOS but default to iOS 12+.

@dnicolson can you add switch to support earlier versions of iOS but default to iOS 12+, please.

The reasons is I made this script to save my dad old iPhone wallpaper initially.

source: gharchive/pull-request

authors: dnicolson hthetiot

repo: hthetiot/cpbitmap-to-png


text:
Testing using webcam

Can I do the testing through my webcam?
Is there a way to shorten the testing process? It's takes too long.

Can I do the testing through my webcam?
Is there a way to shorten the testing process? It's takes too long.

Hey @srtcdemir did you successfully implement it?

Can I do the testing through my webcam?
Is there a way to shorten the testing process? It's takes too long.

Hey @srtcdemir did you successfully implement it?

I'm also interested in..

no i couldn't. I have not been interested in this project for a long time. @geadalfa

authors: geadalfa srtcdemir thanhsang298


text:
attr-no-unsafe-char line feed

Why is the line feed marked as an unsafe attribute character?
Pretty sure this perfectly valid HTML:

The example is for science by the way.

It's definitely valid, and I'm pretty sure there's no other reason to complain about it (as there would be with a more unusual character, which should be encoded with an escape). I can't find any justification at all for the particular character ranges used by attr-no-unsafe-char. I'll definitely remove LF and CR from those, and possibly some other characters.

Thank you @mlochbaum (:

authors: alfaproject mlochbaum

repo: htmllint/htmllint

license: isc

license_source: bigquery


text:
🛑 Manhuagui is down

In 9870aea, Manhuagui (https://www.manhuagui.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Manhuagui is back up in 3e57e0a after 18 minutes.


text:
Update http2-server to 9.4.35.v20201120

Updates org.eclipse.jetty.http2:http2-server from 9.4.34.v20201102 to 9.4.35.v20201120.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.eclipse.jetty.http2", artifactId = "http2-server" } ]

labels: library-update

Superseded by #3912.

authors: rossabaker scala-steward


text:
Respect the SSL context's verify_hostname value

I stumbled upon an issue where http.rb wouldn't respect the verify_hostname value of the current OpenSSL::SSL::SSLContext.
Previously we would unconditionally call post_connection_check, whose only purpose is to verify hostnames, regardless of what verify_hostname was set to. This results in quite a bit of user confusion since one expects hostname verification to be handled by OpenSSL itself rather than by the HTTP library.
Digging through the blames, it looks like this was never implemented properly. Ruby's Net::HTTP fixed the same issue with this pull in January.
This simple change fixes the issue for me and brings http.rb in line with the other Ruby HTTP libraries.

This results in quite a bit of user confusion since one expects hostname verification to be handled by OpenSSL itself rather than by the HTTP library.

Yeah, it's a big mess. OpenSSL itself didn't implement hostname verification at all until quite recently, leaving all users of it to reimplement it poorly themselves (and as it were, I ended up rewriting the hostname verification code used by Ruby itself due to numerous problems).
Anyway, approved! Thanks for the fix.

Per #633 CI is broken (thanks Travis), but I'm going to go ahead and merge this.

Seems like this PR broke jRuby:

@ixti aah, unfortunate. Sorry about that.
Seems like it can at least be worked around with reflection.

jRuby has older version of OpenSSL gem bundled in.

https://github.com/httprb/http/pull/634/commits/f8dfb38181be02c973194657242590754eeed51e

I was suggesting something like:
if ssl_context.respond_to?(:verify_hostname) && ssl_context.verify_hostname
  @socket.post_connection_check(host)
end

@tarcieri :+1: Will do so!

authors: colemannugent ixti tarcieri

repo: httprb/http

license: mit


text:
httprunner解析json返回值报错：value is not a valid dict (type=type_error.dict)

问题描述
请求接口返回如下格式（比较奇怪的json）：
[{
"classId": 27642,
"classUnit": [{
"classNamePy": "",
"classUnitName": "教辅用书",
"confirmType": 10,
"detailId": 1,
"id": 27642,
"isneed": "Y",
"limitE": 0,
"limitS": 0,
"price": 700.0000,
"qty": 0
}, {
"classNamePy": "",
"classUnitName": "总课时费",
"confirmType": 20,
"detailId": 3,
"id": 27642,
"isneed": "Y",
"limitE": 0,
"limitS": 0,
"price": 24000.0000,
"qty": 0
}],
"comments": "测试",
"createBy": "2693",
"createDate": 1599882667000,
"deleteFlag": 0,
"effectiveDate": 1599840000000,
"employeeId": 0,
"expiredDate": 1635955200000,
"hasCentraSub": 0,
"hasChannelSub": 0,
"hasClassSub": 0,
"hasGiftSub": 0,
"hasLevelSub": 0,
"hasOrgSub": 0,
"hasRalationSub": 0,
"hasSpecialSub": 0,
"id": 10539,
"orgId": 4,
"orgSubIds": "4",
"orgType": 0,
"org_type": 0,
"plannedDate": 1635955200000,
"price": 24700.0000,
"priceListCode": "PRICELIST20200912116058",
"priceListName": "S1 192版 常规班-测试专用0912",
"priceListNamePy": "S1192bcgbcszy0912",
"pristlistType": 1,
"status": 4,
"updatedDate": 1599882667000,
"usageNum": 0
}]
版本信息
请提供如下版本信息：

操作系统类型: [e.g. macos]
Python 版本 [3.7.5]
HttpRunner 版本 [3.1.4]

执行返回信息：

=================================== FAILURES ===================================
_____________________ TestCasePricelistbyclass.test_start ______________________

self = <cos_api_test.har.pricelistbyclass_test.TestCasePricelistbyclass object at 0x7f9ddcdc0750>
param = None

    def test_start(self, param: Dict = None) -> "HttpRunner":
        """main entrance, discovered by pytest"""
        self.__init_tests__()
        self.__project_meta = self.__project_meta or load_project_meta(
            self.__config.path
        )
        self.__case_id = self.__case_id or str(uuid.uuid4())
        self.__log_path = self.__log_path or os.path.join(
            self.__project_meta.RootDir, "logs", f"{self.__case_id}.run.log"
        )
        log_handler = logger.add(self.__log_path, level="DEBUG")

        # parse config name
        config_variables = self.__config.variables
        if param:
            config_variables.update(param)
        config_variables.update(self.__session_variables)
        self.__config.name = parse_data(
            self.__config.name, config_variables, self.__project_meta.functions
        )

        if USE_ALLURE:
            # update allure report meta
            allure.dynamic.title(self.__config.name)
            allure.dynamic.description(f"TestCase ID: {self.__case_id}")

        logger.info(
            f"Start to run testcase: {self.__config.name}, TestCase ID: {self.__case_id}"
        )

        try:
            return self.run_testcase(
>               TestCase(config=self.__config, teststeps=self.__teststeps)
            )

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/runner.py:456:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/runner.py:357: in run_testcase
    extract_mapping = self.__run_step(step)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/runner.py:295: in __run_step
    step_data = self.__run_step_request(step)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/runner.py:166: in __run_step_request
    resp = self.__session.request(method, url, **parsed_request_dict)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/client.py:209: in request
    get_req_resp_record(resp_obj) for resp_obj in response_list
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/client.py:209: in <listcomp>
    get_req_resp_record(resp_obj) for resp_obj in response_list
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/client.py:99: in get_req_resp_record
    body=response_body,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   pydantic.error_wrappers.ValidationError: 3 validation errors for ResponseData
E   body
E     str type expected (type=type_error.str)
E   body
E     byte type expected (type=type_error.bytes)
E   body
E     value is not a valid dict (type=type_error.dict)

我也遇见了这个问题，怎样解决？

好像解析不了列表嵌套的json

问题描述
请求接口返回如下格式（比较奇怪的json）：
[{
"classId": 27642,
"classUnit": [{
"classNamePy": "",
"classUnitName": "教辅用书",
"confirmType": 10,
"detailId": 1,
"id": 27642,
"isneed": "Y",
"limitE": 0,
"limitS": 0,
"price": 700.0000,
"qty": 0
}, {
"classNamePy": "",
"classUnitName": "总课时费",
"confirmType": 20,
"detailId": 3,
"id": 27642,
"isneed": "Y",
"limitE": 0,
"limitS": 0,
"price": 24000.0000,
"qty": 0
}],
"comments": "测试",
"createBy": "2693",
"createDate": 1599882667000,
"deleteFlag": 0,
"effectiveDate": 1599840000000,
"employeeId": 0,
"expiredDate": 1635955200000,
"hasCentraSub": 0,
"hasChannelSub": 0,
"hasClassSub": 0,
"hasGiftSub": 0,
"hasLevelSub": 0,
"hasOrgSub": 0,
"hasRalationSub": 0,
"hasSpecialSub": 0,
"id": 10539,
"orgId": 4,
"orgSubIds": "4",
"orgType": 0,
"org_type": 0,
"plannedDate": 1635955200000,
"price": 24700.0000,
"priceListCode": "PRICELIST20200912116058",
"priceListName": "S1 192版 常规班-测试专用0912",
"priceListNamePy": "S1192bcgbcszy0912",
"pristlistType": 1,
"status": 4,
"updatedDate": 1599882667000,
"usageNum": 0
}]
版本信息
请提供如下版本信息：

操作系统类型: [e.g. macos]
Python 版本 [3.7.5]
HttpRunner 版本 [3.1.4]

执行返回信息：

=================================== FAILURES ===================================
_____________________ TestCasePricelistbyclass.test_start ______________________

self = <cos_api_test.har.pricelistbyclass_test.TestCasePricelistbyclass object at 0x7f9ddcdc0750>
param = None

    def test_start(self, param: Dict = None) -> "HttpRunner":
        """main entrance, discovered by pytest"""
        self.__init_tests__()
        self.__project_meta = self.__project_meta or load_project_meta(
            self.__config.path
        )
        self.__case_id = self.__case_id or str(uuid.uuid4())
        self.__log_path = self.__log_path or os.path.join(
            self.__project_meta.RootDir, "logs", f"{self.__case_id}.run.log"
        )
        log_handler = logger.add(self.__log_path, level="DEBUG")

        # parse config name
        config_variables = self.__config.variables
        if param:
            config_variables.update(param)
        config_variables.update(self.__session_variables)
        self.__config.name = parse_data(
            self.__config.name, config_variables, self.__project_meta.functions
        )

        if USE_ALLURE:
            # update allure report meta
            allure.dynamic.title(self.__config.name)
            allure.dynamic.description(f"TestCase ID: {self.__case_id}")

        logger.info(
            f"Start to run testcase: {self.__config.name}, TestCase ID: {self.__case_id}"
        )

        try:
            return self.run_testcase(
>               TestCase(config=self.__config, teststeps=self.__teststeps)
            )

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/runner.py:456:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/runner.py:357: in run_testcase
    extract_mapping = self.__run_step(step)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/runner.py:295: in __run_step
    step_data = self.__run_step_request(step)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/runner.py:166: in __run_step_request
    resp = self.__session.request(method, url, **parsed_request_dict)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/client.py:209: in request
    get_req_resp_record(resp_obj) for resp_obj in response_list
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/client.py:209: in <listcomp>
    get_req_resp_record(resp_obj) for resp_obj in response_list
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/httprunner/client.py:99: in get_req_resp_record
    body=response_body,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   pydantic.error_wrappers.ValidationError: 3 validation errors for ResponseData
E   body
E     str type expected (type=type_error.str)
E   body
E     byte type expected (type=type_error.bytes)
E   body
E     value is not a valid dict (type=type_error.dict)

解决了么？求助

我也碰到这个问题了，最后怎么解决的呢

在 8930bc4ca97e5294911d8aeeb88d2e9ba92c59c3 commit 中已经解决这个问题。

authors: Jamesparagon JasenWujy carrieyummy liuyunzhi tialias

repo: httprunner/httprunner


text:
hrp convert pytest scripts

feat: prepare python3 venv in ~/.hrp/venv before running #1262
feat: convert YAML/JSON testcases to pytest scripts with hrp convert #1261

Codecov Report

Merging #1269 (2a14da0) into master (5d7a77f) will decrease coverage by 12.55%.
The diff coverage is 4.34%.

@@             Coverage Diff             @@
##           master    #1269       +/-   ##
===========================================
- Coverage   63.46%   50.91%   -12.56%
===========================================
  Files          70       40       -30
  Lines        8326     5492     -2834
===========================================
- Hits         5284     2796     -2488
+ Misses       2819     2470      -349
- Partials      223      226        +3

Flag
Coverage Δ

unittests
50.91% <4.34%> (-12.56%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

hrp/cmd/root.go
0.00% <ø> (ø)

hrp/cmd/scaffold.go
18.51% <0.00%> (ø)

hrp/convert.go
0.00% <0.00%> (ø)

hrp/internal/builtin/assertion.go
32.82% <0.00%> (-1.04%)
:arrow_down:

hrp/internal/builtin/utils.go
0.00% <0.00%> (ø)

hrp/runner.go
62.61% <0.00%> (-0.30%)
:arrow_down:

hrp/internal/scaffold/main.go
42.06% <16.66%> (+0.07%)
:arrow_up:

hrp/cmd/convert.go
18.18% <18.18%> (ø)

hrp/boomer.go
69.29% <0.00%> (-15.75%)
:arrow_down:

hrp/step_request.go
46.64% <0.00%> (-0.79%)
:arrow_down:

... and 32 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5d7a77f...2a14da0. Read the comment docs.

authors: codecov-commenter debugtalk


text:
作者你好，你的代码和论文有些出入，能帮忙解决一下么，感激不尽

在这个代码中：
class rtRouter(nn.Module):
def forward(self, x, prev=None, pprev=None):
out = F.avg_pool2d(x, x.size(2))
out = out.view(out.size(0), -1)     #[N,C]
out = F.relu(self.laybn0(self.layer0(out))) #[N,C]
out = self.layer1(out)  #[N,option]
    out = self.gcn_layers(out, prev, pprev) #[N,self.fea_dim]
    ln = self.layer2(out)   [N,option]

这里面的layer1的输出应该是N,self.fea_dim吧，而不是[N,option]

请问是以论文为主，还是以代码为主

按您的公式来说，应该也是[N,self.fea_dim]吧

可能是注释写的有问题吧。 这个输出的维度是fea_dim
-----原始邮件-----
发件人:Q @.>
发送时间:2022-11-17 16:00:03 (星期四)
收件人: huanyuhello/RDI-Net @.>
抄送: Subscribed @.***>
主题: [huanyuhello/RDI-Net] 作者你好，你的代码和论文有出入，能帮忙解决一下么，感激不尽 (Issue #4)
在这个代码中：
class rtRouter(nn.Module):
def forward(self, x, prev=None, pprev=None):
out = F.avg_pool2d(x, x.size(2))
out = out.view(out.size(0), -1) #[N,C]
out = F.relu(self.laybn0(self.layer0(out))) #[N,C]
out = self.layer1(out) #[N,option]
out = self.gcn_layers(out, prev, pprev) #[N,self.fea_dim]
ln = self.layer2(out)   [N,option]

这里面的layer1的输出应该是N,self.fea_dim吧，而不是[N,option]
请问是以论文为主，还是以代码为主
按您的公式来说，应该也是[N,self.fea_dim]吧
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

非常感谢您的解答。我测试的您代码中‘Sn波浪’是维度为3的向量，然后经过图卷积以后变成了256维的向量。

‘Sn波浪’不应该也是256维的么

这个版本是直接把决策向量放进GCN里面传播了。这个实现的应该是与文章不对应的，我之后会修改。最简单的方法，你可以直接把 gcn 出来的特征传出来，对应我们文章里面就是256 ，同时把layer1 的输出维度改成256.
-----原始邮件-----
发件人:Q @.>
发送时间:2022-11-17 20:27:25 (星期四)
收件人: huanyuhello/RDI-Net @.>
抄送: huanyuhello @.>, Comment @.>
主题: Re: [huanyuhello/RDI-Net] 作者你好，你的代码和论文有些出入，能帮忙解决一下么，感激不尽 (Issue #4)
‘Sn波浪’不应该也是256维的么
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

对，按照论文就是把gcn 出来的特征传出来。您这样实现是把决策向量放进GCN里面传播。
我还有些问题想向您请教一下，
1.RDI-net如果我跳层的话，会不会有实际的加速效果？我看这个路由器有好几层，好几层的话，而且还涉及到采样，我感觉有可能没有卷积快呀（这只是我猜的），我还没具体测试。请问RDI-net和普通的Resnet相比，实际速度可以提高么？
2.把决策向量放进GCN里面传播这样的效果是不是比传播256这个效果好点？
谢谢您的回答，感激不尽！抱拳~

第一个问题你看下论文里面 关于计算量的部分。 这里面的加速是计算量减少（推理时间并不一定加快，这一点与硬件相关）。
第二个问题我不知道啊，你跑一下结果看看。github里面的代码是学弟整理的。我也不知道为啥是直接传播的决策向量。
-----原始邮件-----
发件人:Q @.>
发送时间:2022-11-17 21:07:24 (星期四)
收件人: huanyuhello/RDI-Net @.>
抄送: huanyuhello @.>, Comment @.>
主题: Re: [huanyuhello/RDI-Net] 作者你好，你的代码和论文有些出入，能帮忙解决一下么，感激不尽 (Issue #4)
对，按照论文就是把gcn 出来的特征传出来。您这样实现是把决策向量放进GCN里面传播。
我还有些问题想向您请教一下，
1.RDI-net如果我跳层的话，会不会有实际的加速效果？我看这个路由器有好几层，好几层的话，而且还涉及到采样，我感觉有可能没有卷积快呀（这只是我猜的），我还没具体测试。请问RDI-net和普通的Resnet相比，实际速度可以提高么？
2.把决策向量放进GCN里面传播这样的效果是不是比传播256这个效果好点？
谢谢您的回答，感激不尽！抱拳~
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

repo: huanyuhello/RDI-Net


text:
in-hubble documentation

From @basepi on March 7, 2017 22:51
From @cedwards on July 3, 2016 1:22
Can we replicate Salt's sys.doc within Hubble? ie; hubble.doc [module].
Copied from original issue: hubblestack/nova#185
Copied from original issue: hubblestack/hubble-salt#36

Probably. May take some loader modifications, however.

I think our time would be better spent on the sphinx docs, so I'm going to close this one for now.

repo: hubblestack/hubble


text:
Adding index file content

Fixing the issue described in https://github.com/hubot-scripts/hubot-freshdesk/issues/1

Hey,
Sorry for not responding! I missed this notification.
Thanks a lot for your PR. 👍

repo: hubot-scripts/hubot-freshdesk


text:
Sanic fix #1139 the status of response to request with range should be 206 and  shuold contain range's last byte

fix these below:

Http request with a range header return status 200, which should be 206.
Range header Range: x-y means content contains last byte content[y],  but sanic handler it like slice.

Looks great! Thanks @ryanrain2016

@ashleysommer can you upcheck this again when you get a chance (after the branch update is complete)

I ran git rebase -i e89342b36c24afe876bf925ae67b1511d050969b to try and squash the commits for @ryanrain2016, ... but it just added them as a new commit at the end.
@ryanrain2016, Can you try and undo my changes on your branch and then submit the file changes as a single commit?

Closing in favor of #1402

authors: ahopkins ashleysommer ryanrain2016 sjsadowski

repo: huge-success/sanic


text:
Exception in thread "main" class java.lang.IllegalStateException: Can't find task scheduler for graph 'hugegraph[hugegraph]'

在执行以下命令时都报与task scheduler相关的问题，但我从未设置过与task相关的任务，请问如何解决？ 谢谢！！！ @javeme
$ ./bin/hugegraph graph-clear -c "I'm sure to delete all data"
Exception in thread "main" class java.lang.IllegalStateException: Can't find task scheduler for graph 'hugegraph[hugegraph]'
at com.baidu.hugegraph.exception.ServerException.fromResponse(ServerException.java:44)
at com.baidu.hugegraph.client.RestClient.checkStatus(RestClient.java:100)
at com.baidu.hugegraph.rest.RestClient.delete(RestClient.java:232)
at com.baidu.hugegraph.api.graphs.GraphsAPI.clear(GraphsAPI.java:82)
at com.baidu.hugegraph.driver.GraphsManager.clear(GraphsManager.java:54)
at com.baidu.hugegraph.manager.GraphsManager.clear(GraphsManager.java:44)
at com.baidu.hugegraph.cmd.HugeGraphCommand.execute(HugeGraphCommand.java:222)
at com.baidu.hugegraph.cmd.HugeGraphCommand.main(HugeGraphCommand.java:379)
$ ./bin/hugegraph task-list
Exception in thread "main" class java.lang.IllegalStateException: Can't find task scheduler for graph 'hugegraph[hugegraph]'
at com.baidu.hugegraph.exception.ServerException.fromResponse(ServerException.java:44)
at com.baidu.hugegraph.client.RestClient.checkStatus(RestClient.java:100)
at com.baidu.hugegraph.rest.RestClient.get(RestClient.java:212)
at com.baidu.hugegraph.api.task.TaskAPI.list(TaskAPI.java:66)
at com.baidu.hugegraph.driver.TaskManager.list(TaskManager.java:54)
at com.baidu.hugegraph.manager.TasksManager.list(TasksManager.java:52)
at com.baidu.hugegraph.cmd.HugeGraphCommand.execute(HugeGraphCommand.java:261)
at com.baidu.hugegraph.cmd.HugeGraphCommand.main(HugeGraphCommand.java:379)
$ ./bin/hugegraph task-clear --force
Exception in thread "main" class java.lang.IllegalStateException: Can't find task scheduler for graph 'hugegraph[hugegraph]'
at com.baidu.hugegraph.exception.ServerException.fromResponse(ServerException.java:44)
at com.baidu.hugegraph.client.RestClient.checkStatus(RestClient.java:100)
at com.baidu.hugegraph.rest.RestClient.get(RestClient.java:212)
at com.baidu.hugegraph.api.task.TaskAPI.list(TaskAPI.java:66)
at com.baidu.hugegraph.driver.TaskManager.list(TaskManager.java:54)
at com.baidu.hugegraph.manager.TasksManager.list(TasksManager.java:52)
at com.baidu.hugegraph.manager.TasksManager.clear(TasksManager.java:72)
at com.baidu.hugegraph.cmd.HugeGraphCommand.execute(HugeGraphCommand.java:288)
at com.baidu.hugegraph.cmd.HugeGraphCommand.main(HugeGraphCommand.java:379)

@cyberpsyche 因为这些命令都是跟task相关的。可以说明一下是怎么触发的吗？服务端日志中是否有错误信息？

authors: cyberpsyche javeme

repo: hugegraph/hugegraph-tools


text:
使用Restful API时出现跨域访问问题

具体报错如下：
Access to XMLHttpRequest at 'http://XXX.XXX.XXX.XXX.:8080/gremlin' from origin 'http://XXX.XXX.XXX.XXX.:9999' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
我想请教一下相关hugegraph服务器代码应该在哪个目录中修改

版本是hugegraph-server-0.9.2

代码位于 hugegraph-api 模块下，server 包是关于 RestServer 本身的配置，api 包下是所有的 API。

authors: Linary LuciaRan

repo: hugegraph/hugegraph


text:
Program stuck at 'pad_across_processes'

System Info
Copy-and-paste the text below in your GitHub issue

- `Accelerate` version: 0.14.0
- Platform: Linux-4.15.0-194-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.3
- Numpy version: 1.21.6
- PyTorch version (GPU?): 1.9.1+cu111 (True)
- `Accelerate` default config:
        - compute_environment: LOCAL_MACHINE
        - distributed_type: DEEPSPEED
        - mixed_precision: no
        - use_cpu: False
        - num_processes: 4
        - machine_rank: 0
        - num_machines: 1
        - gpu_ids: None
        - main_process_ip: None
        - main_process_port: None
        - rdzv_backend: static
        - same_network: True
        - main_training_function: main
        - deepspeed_config: {'gradient_accumulation_steps': 1, 'offload_optimizer_device': 'cpu', 'offload_param_device': 'cpu', 'zero3_init_flag': True, 'zero3_save_16bit_model': False, 'zero_stage': 3}
        - fsdp_config: {}
        - megatron_lm_config: {}
        - downcast_bf16: no
        - tpu_name: None
        - tpu_zone: None
        - command_file: None
        - commands: None

Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
[ ] My own task or dataset (give details below)

Reproduction

The code at here

Can replace the datasets with own small datasets for debugging.

3.Model just stuck at line674 and finally said time limit error.
Expected behavior
Can run successfully but program stuck at pad_across_devices

Hello @ZeyiLiao, when using DeepSpeed Zero Stage-3, the generate() call in eval loop hangs because it is missing synced_gpus=True argument. Please refer  issue https://github.com/huggingface/transformers/issues/20130 which details what's happening and how to resolve it.

@pacman100 Gotcha! Reaaly helps!!
And would you mind checking why the program can not exit correctly with --report_to wandb --with_tracking?
And I just simply modify the code like this:
    if args.with_tracking:
        if accelerator.is_main_process:
            experiment_config = vars(args)
            # TensorBoard cannot log Enums, need the raw value
            experiment_config["lr_scheduler_type"] = experiment_config["lr_scheduler_type"].value
            tmp = {"entity":"zeyiliao","name":str(args.output_dir).replace("./","")}
            accelerator.init_trackers("t5-finetune", experiment_config,init_kwargs={"wandb":tmp})

error looks like
Configuration saved in ./t5-large-79a06195-d1b9-4b5a-8b7f-b2c0181fced8/config.json
Model weights saved in ./t5-large-79a06195-d1b9-4b5a-8b7f-b2c0181fced8/pytorch_model.bin
tokenizer config file saved in ./t5-large-79a06195-d1b9-4b5a-8b7f-b2c0181fced8/tokenizer_config.json
Special tokens file saved in ./t5-large-79a06195-d1b9-4b5a-8b7f-b2c0181fced8/special_tokens_map.json
Copy vocab file to ./t5-large-79a06195-d1b9-4b5a-8b7f-b2c0181fced8/spiece.model
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11/11 [01:21<00:00,  7.44s/it]
Exception in thread SockSrvRdThr:
Traceback (most recent call last):
  File "/home/zeyi/miniconda3/envs/longtail/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/home/zeyi/miniconda3/envs/longtail/lib/python3.7/site-packages/wandb/sdk/service/server_sock.py", line 112, in run
    shandler(sreq)
  File "/home/zeyi/miniconda3/envs/longtail/lib/python3.7/site-packages/wandb/sdk/service/server_sock.py", line 173, in server_record_publish
    iface = self._mux.get_stream(stream_id).interface
  File "/home/zeyi/miniconda3/envs/longtail/lib/python3.7/site-packages/wandb/sdk/service/streams.py", line 199, in get_stream
    stream = self._streams[stream_id]
KeyError: 'cs1ulxr6'

All the checkpoints can save correctly and wandb can also log correctly, just the code can not exit correcly.

Hello @ZeyiLiao, this is different issue, anyways, I don't know the root cause for this but to resolve it just remove the following lines
- if args.with_tracking:
-        accelerator.end_training()

If this is resolved, can you please go ahead and close the issue?

Sure!

repo: huggingface/accelerate


text:
WIP chapter 2

Translated the introduction to chapter 2. This is a first draft to just get in sync with other contributors and find our common tone in translation. Also created glossary.csv in the root of 'fa' folder to standardize translations of some commonly mistranslated words--this too is a first draft open to suggestions from other contributors.
Related to #49

@lewtun can we get this through I have another page incoming and need to add terms to the glossary but don't want to invalidate this review. :)

I have chapter 0 ready for merge in a separate branch on my fork. Will submit in separate PR for review.

@lewtun can we get this through I have another page incoming and need to add terms to the glossary but don't want to invalidate this review. :)

Hey @jowharshamshiri happy to merge this once we can get the docs to build. I've left some suggestions on trying this yourself, but please let me know if you need some help!

Hey @lewtun, I experimented with DISABLE-FRONTMATTER-SECTIONS and also adding an empty code block but alas to no avail. I want to open an issue on doc-builder, whereby we add a 'direction' arg or otherwise automatically figure out the direction of languages. That info is then used to automatically handle the direction of the generated divs and whatnot. If we do that we can dispense with the extra RTL divs and spans everywhere in Persian and also other RTL-language docs. Is that something that sounds worthwhile to you or am I confused?

Thanks a lot for diving into this @jowharshamshiri - I had no idea it would be so tricky 😬 .
I think opening an issue on doc-builder would make a lot of sense as I can already see we're running into the same issue with other RTL languages. I suggest providing a super simple example of an MDX file and corresponding _toctree.yml file that reproduces the problem and separately shows the hacks you've needed to implement to make it work.
In the meantime, I'll test I can build the current PR!

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

Hey @lewtun, while trying to file the issue and reproduce the problem I found an simpler workaround which I included in new commits. Please kindly merge. The first header now renders correctly as RTL. I filed an issue on doc-builder here to automate this.
P.S. I think maybe in the CI build you have the lang arg still set to en. It is building the original English docs.

Thank you for finding a nice fix! Will merge this now 🤗

P.S. I think maybe in the CI build you have the lang arg still set to en. It is building the original English docs.

Yeah that's a current limitation with our UI - we need to add a dropdown button to view the PR in language X, hopefully will have that soon!

repo: huggingface/course


text:
Fix key name for split names

Split steps were not visible on dataset state

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

authors: AndreaFrancis HuggingFaceDocBuilder

repo: huggingface/datasets-server


text:
Adding english plaintext jokes dataset

This PR adds a dataset of 200k English plaintext Jokes from three sources: Reddit, Stupidstuff, and Wocka.
Link: https://github.com/taivop/joke-dataset
This is my second PR.
First was: #1269

Hi @purvimisal, thanks for your contributions!
This jokes dataset has come up before, and after a conversation with the initial submitter, we decided not to add it then. Humor is important, but looking at the actual data points in this set raises several concerns :)
The main issue is the Reddit part of the dataset which has most of the examples. A cursory look at the data shows a large number of highly offensive jokes that reproduce some pretty harmful biases (the second one from the top is a Holocaust joke).
The other two sources have similar issues (especially the "Blond Jokes") to a slightly lesser extent.
While such datasets can be useful in the right context, there is a real concern that people using the library might miss some of this context (however much we outline it), and end up unwittingly training models that end up having some pretty racist and sexist associations.
We would recommend skipping this dataset altogether.
If you feel really strongly about having a joke dataset, then we would ask that you:

remove the Reddit part of the dataset altogether
write an in-depth description of the social biases present in the remaining data

Let us know which of the two you decide! And if you want recommendations on other datasets to add, hit us up on Slack 🤗

Hi @yjernite, thanks so much. I should've totally thought about this earlier. The harmful biases make so much sense. I should've consulted before making a PR.
I will be closing this one and skipping this dataset altogether.
Thanks again

authors: purvimisal yjernite

repo: huggingface/datasets


text:
Update Huggy Link

The Hugging Space have some problems with Unity demos.
Since a lot of people are starting the course I want them to be able to play correctly with Huggy so I added it to itch.io and updated the links.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

authors: HuggingFaceDocBuilderDev simoninithomas

repo: huggingface/deep-rl-class


text:
Trt llm surport question

The project shows that it supports trt llm, but the model NousResearch/Llama-2-7b-hf in the example trt_llama is not the engine file of trt_llama. When I built the model using trt llm and uploaded the generated engine file to huggingface for testing, a problem occurred.

Hi! trt-llm is supported through optimum-nvidia where the engine is created from HuggingFace Hub checkpoints.

the problem you're seeing is due to missing pretrained config in your repo

So，does Best Benchmark support images in this format? Among them, files such as llama_float16_tp1_rank0.engine are created by Tensorrt-LLM.

When I use the above config.yaml file to evaluate my Tensorrt-llm engine model like joostinyi-baseten/llama7b-fp16-trtllm, this error ModuleNotFoundError: No module named 'optimum.nvidia' appears.
I still get this error after directly pip install optimum, and I directly pip install optimum- nvidia  shows that this library cannot be loaded. How can I solve it?

Hi！
Do I need to upload my Tensert-llm engine files like the llama_float16_tp1_rank0.engine in the picture to huggingface to evaluate my model?
Or did I upload the wrong model file? What is the structure of the Tensorrt-llm model file supported by optimum-benchmark?
Can you give me a correct huggingface model connection for the Tensorrt-llm model that is supported by optimization-benchmark?
I checked the content in examples/trt_llama.yaml and found that model: NousResearch/Llama-2-7b-hf is not a model in trt format.
After I use trt-llm to optimize my llm model, what should I do to use optimization-benchmark for evaluation?
I have encountered various errors so I have a lot of questions. I sincerely hope to get your reply. Thank you very much!

You should have a working optimum-nvidia environment, optimum-benchmark only benchmarks it, make sure you can load your model using https://github.com/huggingface/optimum-nvidia/tree/main

When I run the example of trt_llama, the following error appears

How do I solve this problem?

make sure the example in https://github.com/huggingface/optimum-nvidia/tree/main?tab=readme-ov-file#generate works with your model, this seems like a problem with how you set up your model in the hub, please report errors related to the backend (trt-llm/optimum-nvidia) in optimum-nvidia

so apparently loading a trt-llm engine from the hub is still not supported in optimum-nvidia. closing this issue.

authors: IlyasMoutawwakil lemon-little

repo: huggingface/optimum-benchmark


text:
Fix torch.full issue below when running deepspeed z3 for llama

Fix torch.full issue below when running deepspeed z3 for llama. It's introduced by transformers 438 upgrade.
RuntimeErrorwork = default_pg.broadcast([tensor], opts) : 0 INTERNAL ASSERT FAILED at "/npu-stack/pytorch-fork/torch/csrc/jit/ir/alias_analysis.cpp":615, please report a bug to PyTorch. We don't have an op for aten::full but it isn't a special case.  Argument types: int[], bool, int, NoneType, Device, bool,
What does this PR do?

Fixes # (issue)
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

authors: HuggingFaceDocBuilderDev libinta

repo: huggingface/optimum-habana


text:
MPMD errors when enabling pipeline parallel for fine-tuning llama 3 8B model

System Info
optimum-neuron==0.0.22
transformers == 4.36.2
python==3.10
torch==2.1.2
optimum=1.18.*

Training image in SageMaker:
https://github.com/aws-neuron/deep-learning-containers/blob/2.19.1/docker/pytorch/training/2.1.2/Dockerfile.neuronx

Who can help?
@michaelbenayoun @JingyaHuang @philschmid
Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)
https://github.com/aws-samples/ml-specialized-hardware/blob/main/tutorials/06_FinetuneLLMs/01_Finetune_LLMs.ipynb
The training is based on the above notebook. I used tp=8, pp=2, 2 trn1.32xlarge instances. Official LLama 3 8B model.
Expected behavior
The following errors showed up during the finetuning of the model:
2024-Jul-31 14:47:33.0955282024-Jul-31 14:47:33.0955282024-Jul-31 14:47:33.0955362024-Jul-31 14:47:33.0955372024-Jul-31 14:47:33.0955352024-Jul-31 14:47:33.0955382024-Jul-31 14:47:33.0955352024-Jul-31 14:47:33.0955402024-Jul-31 14:47:33.095544    98:250   ERROR  TDRV:v2_cc_execute                              104:258   ERROR  TDRV:v2_cc_execute                           2024-Jul-31 14:47:33.095545   114:253   ERROR  TDRV:v2_cc_execute                              120:261   ERROR  TDRV:v2_cc_execute                              125:286   ERROR  TDRV:v2_cc_execute                              112:277   ERROR  TDRV:v2_cc_execute                           [nec_dev 4, gid 4] MPMD detected but reload is not supported yet    96:263   ERROR  TDRV:v2_cc_execute                              117:274   ERROR  TDRV:v2_cc_execute                              115:273   ERROR  TDRV:v2_cc_execute                           2024-Jul-31 14:47:33.095550[nec_dev 31, gid 31] MPMD detected but reload is not supported yet[nec_dev 18, gid 18] MPMD detected but reload is not supported yet[nec_dev 23, gid 23] MPMD detected but reload is not supported yet2024-Jul-31 14:47:33.0955582024-Jul-31 14:47:33.0955442024-Jul-31 14:47:33.095567[nec_dev 21, gid 21] MPMD detected but reload is not supported yet2024-Jul-31 14:47:33.095543   116:256   ERROR  TDRV:v2_cc_execute                           2024-Jul-31 14:47:33.095573

Try installing optimum-neuron from source, recent changes have fixed several issues, including some MPMD errors.
pip install git+https://github.com/huggingface/optimum-neuron.git

authors: bingchen-liu jonetiz

repo: huggingface/optimum-neuron


text:
Module 'tensorflow' has no attribute 'contrib'

System Info
Optimum version: 1.13.2
Python 3.9.16
Tensorflow 2.12.0'

Who can help?
@JingyaHuang @echarlaix
Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)
I am following the inpainting example in the docs and when I try:
import PIL
import requests
import torch
from io import BytesIO
from optimum.onnxruntime import ORTStableDiffusionInpaintPipeline

I get the following error:
...

RuntimeError: Failed to import optimum.onnxruntime.modeling_diffusion because of the following error (look up to see its traceback):
Failed to import transformers.pipelines because of the following error (look up to see its traceback):
module 'tensorflow' has no attribute 'contrib'

I understand that it is related with my TF version. Would you have support for this version?
Expected behavior
The imported libraries.

@juancopi81 Thank you. Could you share the full traceback?

Sure! Thanks for your answer. Here it is:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File /opt/conda/envs/venv/lib/python3.9/site-packages/transformers/utils/import_utils.py:1110, in _LazyModule._get_module(self, module_name)
   1109 try:
-> 1110     return importlib.import_module("." + module_name, self.__name__)
   1111 except Exception as e:

File /opt/conda/envs/venv/lib/python3.9/importlib/__init__.py:127, in import_module(name, package)
    126         level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1030, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1007, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:986, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:680, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:850, in exec_module(self, module)

File <frozen importlib._bootstrap>:228, in _call_with_frames_removed(f, *args, **kwds)

File /opt/conda/envs/venv/lib/python3.9/site-packages/transformers/pipelines/__init__.py:73
     72 from .question_answering import QuestionAnsweringArgumentHandler, QuestionAnsweringPipeline
---> 73 from .table_question_answering import TableQuestionAnsweringArgumentHandler, TableQuestionAnsweringPipeline
     74 from .text2text_generation import SummarizationPipeline, Text2TextGenerationPipeline, TranslationPipeline

File /opt/conda/envs/venv/lib/python3.9/site-packages/transformers/pipelines/table_question_answering.py:27
     25 import tensorflow as tf
---> 27 import tensorflow_probability as tfp
     29 from ..models.auto.modeling_tf_auto import (
     30     TF_MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING,
     31     TF_MODEL_FOR_TABLE_QUESTION_ANSWERING_MAPPING,
     32 )

File /opt/conda/envs/venv/lib/python3.9/site-packages/tensorflow_probability/__init__.py:78
     77 # from tensorflow_probability.google import staging  # DisableOnExport
---> 78 from tensorflow_probability.python import *  # pylint: disable=wildcard-import
     79 from tensorflow_probability.python.version import __version__

File /opt/conda/envs/venv/lib/python3.9/site-packages/tensorflow_probability/python/__init__.py:21
     19 from __future__ import print_function
---> 21 from tensorflow_probability.python import bijectors
     22 from tensorflow_probability.python import distributions

File /opt/conda/envs/venv/lib/python3.9/site-packages/tensorflow_probability/python/bijectors/__init__.py:46
     45 from tensorflow_probability.python.bijectors.matrix_inverse_tril import MatrixInverseTriL
---> 46 from tensorflow_probability.python.bijectors.matveclu import MatvecLU
     47 from tensorflow_probability.python.bijectors.normal_cdf import NormalCDF

File /opt/conda/envs/venv/lib/python3.9/site-packages/tensorflow_probability/python/bijectors/matveclu.py:24
     23 from tensorflow_probability.python.bijectors import bijector
---> 24 from tensorflow_probability.python.math.linalg import lu_reconstruct
     25 from tensorflow_probability.python.math.linalg import lu_solve

File /opt/conda/envs/venv/lib/python3.9/site-packages/tensorflow_probability/python/math/__init__.py:22
     21 from tensorflow_probability.python.math.custom_gradient import custom_gradient
---> 22 from tensorflow_probability.python.math.diag_jacobian import diag_jacobian
     23 from tensorflow_probability.python.math.interpolation import batch_interp_regular_1d_grid

File /opt/conda/envs/venv/lib/python3.9/site-packages/tensorflow_probability/python/math/diag_jacobian.py:24
     22 import tensorflow as tf
---> 24 tfe = tf.contrib.eager
     26 __all__ = [
     27     'diag_jacobian',
     28 ]

AttributeError: module 'tensorflow' has no attribute 'contrib'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
File /opt/conda/envs/venv/lib/python3.9/site-packages/transformers/utils/import_utils.py:1110, in _LazyModule._get_module(self, module_name)
   1109 try:
-> 1110     return importlib.import_module("." + module_name, self.__name__)
   1111 except Exception as e:

File /opt/conda/envs/venv/lib/python3.9/importlib/__init__.py:127, in import_module(name, package)
    126         level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1030, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1007, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:986, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:680, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:850, in exec_module(self, module)

File <frozen importlib._bootstrap>:228, in _call_with_frames_removed(f, *args, **kwds)

File /opt/conda/envs/venv/lib/python3.9/site-packages/optimum/onnxruntime/modeling_diffusion.py:43
     42 from ..onnx.utils import _get_external_data_paths
---> 43 from ..pipelines.diffusers.pipeline_stable_diffusion import StableDiffusionPipelineMixin
     44 from ..pipelines.diffusers.pipeline_stable_diffusion_img2img import StableDiffusionImg2ImgPipelineMixin

File /opt/conda/envs/venv/lib/python3.9/site-packages/optimum/pipelines/__init__.py:16
      1 # coding=utf-8
      2 # Copyright 2023 The HuggingFace Team. All rights reserved.
      3 #
   (...)
     13 # See the License for the specific language governing permissions and
     14 # limitations under the License.
---> 16 from .pipelines_base import (
     17     MAPPING_LOADING_FUNC,
     18     ORT_SUPPORTED_TASKS,
     19     load_bettertransformer,
     20     load_ort_pipeline,
     21     pipeline,
     22 )

File /opt/conda/envs/venv/lib/python3.9/site-packages/optimum/pipelines/pipelines_base.py:19
     17 from typing import Any, Dict, Optional, Union
---> 19 from transformers import (
     20     AudioClassificationPipeline,
     21     AutoConfig,
     22     AutomaticSpeechRecognitionPipeline,
     23     FeatureExtractionPipeline,
     24     FillMaskPipeline,
     25     ImageClassificationPipeline,
     26     ImageSegmentationPipeline,
     27     ImageToTextPipeline,
     28     Pipeline,
     29     PreTrainedTokenizer,
     30     PreTrainedTokenizerFast,
     31     QuestionAnsweringPipeline,
     32     SequenceFeatureExtractor,
     33     SummarizationPipeline,
     34     Text2TextGenerationPipeline,
     35     TextClassificationPipeline,
     36     TextGenerationPipeline,
     37     TokenClassificationPipeline,
     38     TranslationPipeline,
     39     ZeroShotClassificationPipeline,
     40 )
     41 from transformers import pipeline as transformers_pipeline

File <frozen importlib._bootstrap>:1055, in _handle_fromlist(module, fromlist, import_, recursive)

File /opt/conda/envs/venv/lib/python3.9/site-packages/transformers/utils/import_utils.py:1100, in _LazyModule.__getattr__(self, name)
   1099 elif name in self._class_to_module.keys():
-> 1100     module = self._get_module(self._class_to_module[name])
   1101     value = getattr(module, name)

File /opt/conda/envs/venv/lib/python3.9/site-packages/transformers/utils/import_utils.py:1112, in _LazyModule._get_module(self, module_name)
   1111 except Exception as e:
-> 1112     raise RuntimeError(
   1113         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1114         f" traceback):\n{e}"
   1115     ) from e

RuntimeError: Failed to import transformers.pipelines because of the following error (look up to see its traceback):
module 'tensorflow' has no attribute 'contrib'

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Cell In[7], line 5
      3 import torch
      4 from io import BytesIO
----> 5 from optimum.onnxruntime import ORTStableDiffusionInpaintPipeline

File <frozen importlib._bootstrap>:1055, in _handle_fromlist(module, fromlist, import_, recursive)

File /opt/conda/envs/venv/lib/python3.9/site-packages/transformers/utils/import_utils.py:1100, in _LazyModule.__getattr__(self, name)
   1098     value = self._get_module(name)
   1099 elif name in self._class_to_module.keys():
-> 1100     module = self._get_module(self._class_to_module[name])
   1101     value = getattr(module, name)
   1102 else:

File /opt/conda/envs/venv/lib/python3.9/site-packages/transformers/utils/import_utils.py:1112, in _LazyModule._get_module(self, module_name)
   1110     return importlib.import_module("." + module_name, self.__name__)
   1111 except Exception as e:
-> 1112     raise RuntimeError(
   1113         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1114         f" traceback):\n{e}"
   1115     ) from e

RuntimeError: Failed to import optimum.onnxruntime.modeling_diffusion because of the following error (look up to see its traceback):
Failed to import transformers.pipelines because of the following error (look up to see its traceback):
module 'tensorflow' has no attribute 'contrib'

repo: huggingface/optimum


text:
Removing unused decoder for TransformedModel class.

Removing unused decoder in the __init__ of the TransformerModel class as mentioned in #32.

Yes!

authors: rodgzilla thomwolf


text:
Added Squad 2.0

Accidentally closed the last pull request. Created Separate file for Squad 2.0 run with
python3 run_squad.py
--bert_model bert-large-uncased_up
--do_predict
--do_lower_case
--train_file squad/train-v2.0.json
--predict_file squad/dev-v2.0.json
--learning_rate 3e-5
--num_train_epochs 2
--max_seq_length 384
--doc_stride 128
--output_dir squad2_diff
--train_batch_size 24
--fp16
--loss_scale 128
--null_score_diff_threshold -2.6929588317871094
If null score not defined default value is 0.0

how much time does it take to train?

authors: abeljim eric-haibin-lin

repo: huggingface/pytorch-pretrained-BERT


text:
Error loading converted pytorch checkpoint

I am using BioBert. After converting the tensorflow checkpoint to pytorch checkpoint, I want to load it to Bert model. I found that the old pytorch-pretrained-bert works perfect but the new pytorch-transformer fails.
Here is the successful run using pytorch-pretrained-bert:

from pytorch_pretrained_bert import BertForSequenceClassification
model = BertForSequenceClassification.from_pretrained("biobert_v1.1_pubmed/", num_labels=1)
exit()

Here's the failure in pytorch_transformers:

from pytorch_transformers import BertForSequenceClassification
model = BertForSequenceClassification.from_pretrained("biobert_v1.1_pubmed/", num_labels=1)
Model name 'biobert_v1.1_pubmed/' was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc). We assumed 'biobert_v1.1_pubmed/config.json' was a path or url but couldn't find any file associated to this path or url.
Traceback (most recent call last):
File "", line 1, in
File "/home/michael/.local/lib/python3.7/site-packages/pytorch_transformers/modeling_utils.py", line 403, in from_pretrained
model = cls(config)
File "/home/michael/.local/lib/python3.7/site-packages/pytorch_transformers/modeling_bert.py", line 958, in init
super(BertForSequenceClassification, self).init(config)
File "/home/michael/.local/lib/python3.7/site-packages/pytorch_transformers/modeling_bert.py", line 548, in init
super(BertPreTrainedModel, self).init(*inputs, **kwargs)
File "/home/michael/.local/lib/python3.7/site-packages/pytorch_transformers/modeling_utils.py", line 206, in init
self.class.name, self.class.name
ValueError: Parameter config in BertForSequenceClassification(config) should be an instance of class PretrainedConfig. To create a model from a pretrained model use model = BertForSequenceClassification.from_pretrained(PRETRAINED_MODEL_NAME)

Do you have a file named biobert_v1.1_pubmed/config.json as mentioned in the error?

Oh thanks. There is a config file name "bert_config.json" in the directory "biobert_v1.1_pubmed/". I changed the file name to "config.json" and it works!
I am wondering why the pytorch-pretrained-bert can load the checkpoint. Maybe it reads "bert_config.json" and pytorch-transformer reads "config.json"?

repo: huggingface/pytorch-transformers


text:
Pr 2395 ci run

This PR reopens https://github.com/huggingface/text-generation-inference/pull/2395 to run CI and small changes to add tests for pushing to a single message and applying a chat template (to trigger an append)

Thank you @srossi93 🙏

repo: huggingface/text-generation-inference


text:
Fix Document QA pipeline

Closes #218

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

authors: HuggingFaceDocBuilderDev xenova


text:
RLOOTrainer & PPOv2Trainer - Modify Name for W&B Logged Table

Feature request
RLOOTrainer and PPOv2Trainer both log a table to W&B titled "completions".  I would like the name to be changed to f"completions_step={self.state.global_step}" or something like this.
Motivation
Due to how W&B displays tables in the WebUI, logging the table under the same name "completions" means that one can only see the most recent table of completions, rather than tables of completions at different points during learning.
I would like to be able to manually inspect how completions are changing during learning.
I can probably pull this information out of the W&B API, but this is too much effort. I want to be able to manually inspect easily.
Your contribution
I would be happy to submit a PR for this.

You can try. I recently tried to implement this too, but it didn't work. The way wandb handles tables is rather strange, and doesn't seem to be completely stable.

I've been using my own custom PPOv2 and RLOO trainers because I want to make changes like this, and it appears to work fine?

Looks nice, it makes things easier to compare! feel free to open a PR. Thanks

authors: RylanSchaeffer qgallouedec

repo: huggingface/trl


text:
[Data] Implement dataset mixer for combining datasets in training

Feature request
In the alignment-handbook, we implemented a "dataset mixer" that allows one to easily combine datasets in varying proportions, provided they all share the same schema.
It could be interesting to port this mixer to TRL, so that users can easily combine datasets during training. The only caveat I see is that to support the CLI training e.g. trl sft ... we'd need a data structure that is compatible because dict objects don't place nice with CLIs.
Motivation
Advanced post-training typically combines different datasets / proportions. Supporting this in TRL would allow us to gradually deprecate the handbook in favour of using the lib directly.
Your contribution
Open to discussion :)

@lewtun
Shouldn't this be a part of Datasets instead?
Datasets already has Interleaves, which also mixes datasets in a similar way. It's not quite integrable as it is, but may be useful.
As for integrating it into the CLI, I think the best way to do it would be with a config file, like a JSON, something like this::
trl sft --dataset_mixer  --mix_config mix_config.json
mix_config.json:
{
  "dataset_mixer": {
    "dataset_1": 0.4,
    "dataset_2": 0.3,
    "dataset_3": 0.2
  },
  "splits": ["train", "train", "test"],
  "configs": ["main", "math", "logic"],
  "columns_to_keep": ["text", "input", "text"],
  "shuffle": true
}

or a simpler JSON:
[
  ["dataset_1", 0.4, "main", "train", "text"],
  ["dataset_2", 0.3, "math", "train", "input"],
  ["dataset_3", 0.2, "logic", "test", "text"],
]

if the parsing doesn't get too complicated.
How does that sound?

Yes good point about whether this should live in datasets or trl. Gently pinging @lhoestq for his thoughts on this.
Context for Quentin: a common workflow in LLM fine-tuning is to mix datasets with mixed proportions (e.g. 30% of dataset together with 50% of dataset B). The interleave_datasets() method in datasets does something similar, but uses sampling and a stopping strategy which isn't quite the same thing we want here because we don't want to stop when one dataset is exhausted and we don't want to oversample either. One possibility would be to add a new stopping strategy like none_exhausted which samples the precise proportions (or less if there are not enough samples)

This works no ? And it feels easier to understand that a mixer (are the examples taken at random ? is the fraction representing the proportion of the original dataset or the resulting dataset ?)
concatenate_datasets([
    dataset.select(range(int(frac * len(dataset))))
    for dataset, frac in zip(datasets, fracs)
])

This works no ? And it feels easier to understand than a mixer (are the examples taken at random ? is the fraction representing the proportion of the original dataset or the resulting dataset ?)
concatenate_datasets([
    dataset.select(range(int(frac * len(dataset))))
    for dataset, frac in zip(datasets, fracs)
])

Yes, that is effectively what we did in the alignment-handbook here. In our case, the fractions represent the proportion per dataset and the samples are just taken from the first N rows.
My question is more about whether it makes sense for datasets to have a utility method to load and combine multiple datasets from the Hub? Here's how we did it in the handbook (link) - happy to keep this logic in trl if it's too niche!

Since the mixing logic is kinda specific I don't think it should be in datasets. Though if there is something more general / explicit we can reconsider

authors: August-murr lewtun lhoestq


text:
期望 av-canvas 可以提供 sprite 的更新事件

希望可以实时拿到sprite 的宽高、位置、旋转等信息，用来实现撤销、恢复功能

希望可以实时拿到sprite 的宽高、位置、旋转等信息，用来实现撤销、恢复功能

next 版本已支持

propsChange 事件，参考单测代码
test('zIndex change event', () => {
  const spr = createSprite();
  const handler = vi.fn();
  spr.on('propsChange', handler);

  spr.zIndex = 10;
  expect(handler).toBeCalledWith({ zIndex: 10 });
});

test('rect change event', async () => {
  const spr = createSprite();
  const handler = vi.fn();
  spr.on('propsChange', handler);

  spr.rect.x = 10;
  spr.rect.y = 10;
  expect(handler).toBeCalledWith({ rect: { x: 10 } });
  expect(handler).toBeCalledWith({ rect: { y: 10 } });
  expect(handler).toBeCalledTimes(2);
});

repo: hughfenghen/WebAV


text:
Seconds to H:M:S

220 to 00:03:40,   is it possible?
{
  "expected_update_period_in_days": "2",
  "url": "https://www.youtube.com/watch?v=HdG6rPpA8Pg",
  "type": "html",
  "mode": "on_change",
  "extract": {
    "title": {
      "xpath": "//span[@id='eow-title']",
      "value": "normalize-space(.)"
    },
    "duration": {
      "css": "html"
    },
    "description": {
      "xpath": "//p[@id='eow-description']"
    }
  },
  "template": {
    "duration": "{{duration | strip_newlines | regex_replace: '[\\s\\S]*lengthSeconds|ownerProfileUrl[\\s\\S]*|\\\"|\\,|\\:|\\x5c', ''}}"
  }
}

This is something I would also find useful.
Currently, I'm dividing the seconds to minutes and rounding, so something more accurate would be great!

Take a look at this, it's much easier: https://github.com/huginn/huginn/wiki/Formatting-Events-using-Liquid
In that page, search for "More Examples."

authors: 10362227 deathbybandaid virtadpt

repo: huginn/huginn


text:
Add functionality for demo

Ideas:

[x] Organization search
[ ] Organization profile
[ ] Subforum index page (list of threads)
[ ] Thread page

Well, we sorta one of these, so nice!

repo: humanitarians-united/hu


text:
Ensure the app can be compiled without composer.json or composer.lock

When used with PHP-Scoper however, which requires to dump the dependencies, throw an error

Closed by #122 and #124

repo: humbug/box


text:
Fix groups style

Fixes some styling for the new groups stuff.
Changes proposed in this pull request:

Adds more spacing around group neighbors and rounds image corners
Fix overflow in the group leaders list
Fix group name overflow in the "My Groups" widget (more noticeable on /groups)
Increases the height of the groups and tickets searchbar to match the sorting dropdown

/cc @hummingbird-me/staff

Looks great! Thanks as always!

authors: JoshFabian synthtech

repo: hummingbird-me/hummingbird-client


text:
GateIo incomplete order book

Describe the bug
Gate io has incomplete order book

Steps To Reproduce
// A concise description of the steps to reproduce the buggy behavior:
run orderbook --live and check against exchange order book
Here are the steps to reproduce the issue (see attachments in the section below):
start any strategy and run orderbook of gate io
Screenshots
// If applicable, add screenshots to help explain your problem.
Release version
dev-1.0.0
Attachments
// Please attach your config file and log file to help us debug. It would be difficult for us to help you without those!
WARNING: Do NOT publish any exchange API keys or your wallet's private key. Whoever has access to them may steal your assets!
Optional: your discord username:

This issue is still the same issue as https://github.com/hummingbot/hummingbot/issues/4710
Based on the comments, it seems there is still a desynchronization issue.
Could further efforts be made to continue fixing this issue?
This issue made many gate io markets unusable for me as I rely mostly on the order book depth.
I am not familiar with the connector part of the code yet so if can let me know a rough idea of what might be the problem and I can also try to debug from there.
I tried further increasing the past diff window but it does not help.
https://github.com/hummingbot/hummingbot/pull/4756

Hi @leastchaos, just checking again if this is still an ongoing issue? We looked in this issue on our staging branch and seems to be working ok atm. Please let us know if this persists. Thank you

authors: leastchaos rapcmia

repo: hummingbot/hummingbot


text:
ScrollIn event in Scroller fires too early

When scrolling down, the scrollIn event fires too early / scrollOut event does not fire, because isIntersecting is still true and the ratio value is 0.09xxxx. Even if the section is not visible on the screen anymore.

Checking now if isIntersecting is true AND the ratio is above ore equal 0.1 to ensure scrollOut event is fired.


text:
back command - rewinds n number of frames

similar to time command, but rewinds an amount of frames from the current position to create stutters and other rhythmic effects

Can you call it rewind, and put it on line 12.
Also, don't use a method to get the frame, just do .f

renamed, added a forward command 'skip'
example usage
7U8..................
.*:01D...............
3u8..................
..:01C......gC.......
5ub..........44TCAFE.
..:01F.......aVC.....
.....................
.....................
.....C8..Cb..Cg......
.....4F0.9F0.4F0.....
.....................
......J...J...J...Va.
...............:13C..
......J...J...Va.....
...........:14C......
......J...Va.........
.......:14C..........
.....................
.....2D8.............
.......;rewind:0.....
.....................
......D8.............
.......;skip:0.......

authors: echophon neauoire

repo: hundredrabbits/Orca


text:
How to change GPU requirement configuration.

I was trying out yolov4-tensorflow on CUDA==10.1 as tensorflow-gpu supports only this cuda version.
So I wrote the following command in cmd,
python detect_video.py --weights ./checkpoints/yolov4-416 --size 416 --model yolov4 --video 0 --output ./detections/results_new.avi
But this was prompting that 2020-10-08 17:26:22.940638: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
It's wanting CUDA==11. How to resolve this issue?

@hunglc007 please look into the issue!

@dragonsongohan please look into the issue!

@paroque28 please look into the issue!

@jzoker please look into the issue!

no one reviews the issues

You should install Cuda 11.0


text:
"wc-waterfall" has already been used with this registry

感谢库主提供该库。
通过NPM使用在react项目中时偶现component重复定义
Failed to execute 'define' on 'CustomElementRegistry': the name "wc-waterfall" has already been used with this registry

https://github.com/huodoushigemi/wc-flow-layout/blob/c8fb3d3ab8038d4ad1a8e1d18b3c434fea1a9923/src/index.ts#L6C39-L6C55
建议改为
if (!customElements.get('wc-waterfall')) { customElements.define('wc-waterfall', WaterfallElement); }

需要的话可提PR。

👍

Thanks Buddy. It would be great if you don't mind to push the newest release to the NPM. :)

repo: huodoushigemi/wc-flow-layout


text:
🛑 Coach is down

In c9f7010, Coach (https://coach.craftstudios.shop/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Coach is back up in 682c2c6 after 16 minutes.

repo: hupratt/upptime


text:
Issues importing Malaya

Assalamualaikum,
saya cuba untuk import Malaya tapi ada Importerror yang keluar:
ImportError: cannot import name 'nx' from 'networkx'
Saya sudah cuba uninstall Networkx dan install balik,tetapi masalah ini masih berlaku. Ada cadangan untuk menyelesaikan masalah ini?
Terima kasih.

networkx recently released 2.6.2, install networkx <= 2.5.1 should solved the problem.


text:
405 Response When URL Parameter is empty

There is a problem with the tree insertion, where if a user-agent requests a url that has an empty url parameter, a 405 response status code occurs.
example:
router.Get("/v:version/test/:testid", f)

curl -XGET http://localhost/v3/test/ -D -
HTTP/1.1 405 Method Not Allowed
Allow: GET
Date: Wed, 23 Sep 2015 21:51:12 GMT
Content-Length: 18
Content-Type: text/plain; charset=utf-8

Believe the issue is happening due to the split node in the insertion.

https://github.com/husobee/vestigo/commit/c1193e00eb9b9357d466ab9dc24cc0f380cc65a3 <- this commit fixes this bug, implemented a "clean" mechanism on the resource which cleans up the allowMethods string artifact from split nodes.  Also added in a flag to AllowTrace.

repo: husobee/vestigo


text:
Add Myanmar Localization

Hello, here is my translation to timesago library!
local_code for Myanmar language is "my"

Myanmar language is lovely.

authors: hustcc kaungmyatlwin


text:
Fix pack commands with gh-action using buildpacks-action

AsIs
Noted in https://github.com/hwakabh/bennu-official/pull/45#issuecomment-1865410913 and https://github.com/hwakabh/bennu-official/pull/45#discussion_r1433411324, ANALYZING phase would be failed with accessing GHCR.
ToDo
This is because we have to run docker login with PAT, before executing pack commands, so add logics of this to gh-action configs
(Optional) Acceptance criterial
Confirm GitHub actions job would be success.

As the job have been still failed even though adding docker login actions before executing pack command, reopened the issue.

Error on ANALYZE phase with pack:
https://github.com/hwakabh/bennu-official/actions/runs/7297246311/job/19886209158

repo: hwakabh/bennu-official


text:
Issue: Set up a Chroma retriever for client-server configuration of Chroma

Issue you'd like to raise.
I am using Chroma's client-server configuration and I have trouble setting up a retriever for ConversationalRetrievalChain.from_llm() function.
I can't find anything related to this. Can someone guide me on how can I do that or if someone has any solution?
For the locally stored database you cust called db.as_retriever() and that was it.
But now, I can't find a solutionf or passing a retriever to from_llm() function.
My code snippet is:
`def askQuestion(self, thread_id, question):
        collection = self.chroma_client.get_collection(name="my_collection5", embedding_function=self.embedding)
        self.llm = ChatOpenAI(model_name=self.model_name, temperature=self.temperature,
        openai_api_key=os.environ.get('OPENAI_API_KEY'))
        self.memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True,
        output_key='answer')
        self.chain = ConversationalRetrievalChain.from_llm(self.llm, collection.as_retriever(),return_source_documents=True,verbose=VERBOSE, memory=self.memory)

        result = self.chain({"question": question})
        res_dict = {
            "answer": result["answer"],
        }

        res_dict["source_documents"] = []

        # add source docs
        for source in result["source_documents"]:
            res_dict["source_documents"].append({
                "page_content": source.page_content,
                "metadata":  source.metadata
            })

        return res_dict`

Suggestion:
No response

@zigax1 , have you found a solution for this? I'm, facing similar issues when using chroma in server/client mode

Hi dosubot, I'm also looking at this and trying to move from a local vectorstore or persistant vectorstor to a vectorstor created remotely and accessed through the chromadb.HttpClient() call.  The langchain vectorstore contains interfaces for from_documents() and from_texts() and there is documentation that refers to fromExistingCollection() but this last is not present in the code.  Can you point me at the approach to create an Retriever interface from the HttpClient?

authors: bheather tyatabe zigax1


text:
Getting "ValueError: Unsupported chat history format:" while using ConversationalRetrievalChain with memory type VectorStoreRetrieverMemory

System Info
LangChain-0.0.207, Windows, Python-3.9.16
Memory (VectorStoreRetrieverMemory) Settings:
dimension = 768
index = faiss.IndexFlatL2(dimension)
embeddings = HuggingFaceEmbeddings()
vectorstore = FAISS(embeddings.embed_query, index, InMemoryDocstore({}), {})
retriever = vectorstore.as_retriever(search_kwargs=dict(k=3))
memory = VectorStoreRetrieverMemory(
memory_key="chat_history",
return_docs=True,
retriever=retriever,
return_messages=True,
)
ConversationalRetrievalChain Settings:
_template = """Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question, in its original language.
Make sure to avoid using any unclear pronouns.
Chat History:
{chat_history}
(You do not need to use these pieces of information if not relevant)
Follow Up Input: {question}
Standalone question:"""
CONDENSE_QUESTION_PROMPT = PromptTemplate(
input_variables=["chat_history", "question"], template=_template
)
question_generator = LLMChain(llm=llm, prompt=CONDENSE_QUESTION_PROMPT)
doc_chain = load_qa_with_sources_chain(
llm, chain_type="refine"
)  # stuff  map_reduce  refine
qa = ConversationalRetrievalChain(
retriever=chroma.as_retriever(search_kwargs=dict(k=5)),
memory=memory,
combine_docs_chain=doc_chain,
question_generator=question_generator,
return_source_documents=True,
# verbose=True,
)
responses = qa({"question": user_input})
ISSUE: At first cal, it is giving results. BUT in second call, it is throwing an error as follows:
ValueError: Unsupported chat history format: <class 'langchain.schema.Document'>. Full chat history:
What am I doing wrong here?
Who can help?
No response
Information

[ ] The official example notebooks/scripts
[X] My own modified scripts

Related Components

[X] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[X] Vector Stores / Retrievers
[X] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction

Memory (VectorStoreRetrieverMemory) Settings:

dimension = 768
index = faiss.IndexFlatL2(dimension)
embeddings = HuggingFaceEmbeddings()
vectorstore = FAISS(embeddings.embed_query, index, InMemoryDocstore({}), {})
retriever = vectorstore.as_retriever(search_kwargs=dict(k=3))
memory = VectorStoreRetrieverMemory(
memory_key="chat_history",
return_docs=True,
retriever=retriever,
return_messages=True,
)

ConversationalRetrievalChain Settings:

_template = """Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question, in its original language.
Make sure to avoid using any unclear pronouns.
Chat History:
{chat_history}
(You do not need to use these pieces of information if not relevant)
Follow Up Input: {question}
Standalone question:"""
CONDENSE_QUESTION_PROMPT = PromptTemplate(
input_variables=["chat_history", "question"], template=_template
)
question_generator = LLMChain(llm=llm, prompt=CONDENSE_QUESTION_PROMPT)
doc_chain = load_qa_with_sources_chain(
llm, chain_type="refine"
)  # stuff  map_reduce  refine
qa = ConversationalRetrievalChain(
retriever=chroma.as_retriever(search_kwargs=dict(k=5)),
memory=memory,
combine_docs_chain=doc_chain,
question_generator=question_generator,
return_source_documents=True,
# verbose=True,
)
responses = qa({"question": user_input})
ISSUE: At first cal, it is giving results. BUT in second call, it is throwing an error as follows:
ValueError: Unsupported chat history format: <class 'langchain.schema.Document'>. Full chat history:
What am I doing wrong here?
Expected behavior
Chat history should be injected in chain

Now getting following error:
ValueError: Unsupported chat history format: <class 'str'>

@mail2mhossain did you figure this out?

Not Yet

+1
from langchain.memory import VectorStoreRetrieverMemory
from langchain.vectorstores import Qdrant
from langchain.embeddings import SentenceTransformerEmbeddings
from langchain.llms import OpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.retrievers.multi_query import MultiQueryRetriever

llm = OpenAI(openai_api_key=OPEN_AI_API_KEY)

class AnswerMachine:
    def __init__(self):
        self.embeddings = SentenceTransformerEmbeddings(model_name="sentence-transformers/msmarco-distilbert-dot-v5")
        self.qdrant_client = QdrantClient(QDRANT_URL)
        self.qdrant = Qdrant(embeddings=self.embeddings,
                                client=self.qdrant_client,
                                collection_name='history',
                                vector_name='text')
        self.retriever = self.qdrant.as_retriever()
        self.memory = VectorStoreRetrieverMemory(retriever=self.retriever, memory_key="chat_history", return_docs=False, return_messages=True)

    def answer(self, text):
        retriever = MultiQueryRetriever.from_llm(retriever=self.qdrant.as_retriever(search_type='mmr'), llm=llm)

        qa_chain = ConversationalRetrievalChain.from_llm(llm, retriever=retriever, memory=self.memory, return_source_documents=True)
        answer = qa_chain({"question": text})

        return answer

Also returns ValueError: Unsupported chat history format: when called the second time.

Following some examples in #2303
I used: qa_chain = ConversationalRetrievalChain.from_llm(llm, retriever=retriever, memory=self.memory, return_source_documents=True, get_chat_history=lambda h : h)
and it works now.


text:
Adds input parsers for validating and documenting LLM inputs

While working with LLMs as a step in a pipeline with other APIs, I found it useful several times to use a prompt like this in conjunction with an output parsers:
The JSON array below represents X. Each item in the array is an object containing two fields:
  - foo, a string that represents Y
  - bar, a number that represents Z

I've framework-itized it as a StructuredInputParser in this PR. In addition to giving the LLM more context as to what each field does beyond just the field name and overall structure, there's the nice benefit of encouraging people to define input types, more easily support complex inputs, and generally think more about what they're passing into LLMs.
I've included a unit and integration test that illustrate what using such a parser looks like in practice - here's an example of what I'm envisioning:
const chat = new ChatOpenAI({ modelName: "gpt-3.5-turbo", temperature: 0 });
const zodSchema = z
  .array(
    z.object({
      id: z.string().describe("the unique record id"),
      created_at: z.string().describe("creation date of the database record"),
      fields: z.object({
        title: z.string().describe("the book title"),
        seriesId: z.string().describe("id of the series the book belongs to"),
        createdAt: z.string().describe("publish date of the book"),
      }),
    })
  )
  .describe("an array of database records, each representing a book");
const inputParser = StructuredInputParser.fromZodSchema(zodSchema);
const outputParser = StructuredOutputParser.fromZodSchema(zodSchema);
const chain = new LLMChain({
  llm: chat,
  prompt: new PromptTemplate({
    template: `Answer the user's question factually and as best you can.

{input_format_instructions}

{input}

{output_format_instructions}

{query}`,
    inputVariables: ["query", "input"],
    partialVariables: {
      input_format_instructions: inputParser.getFormatInstructions(),
      output_format_instructions: outputParser.getFormatInstructions(),
    },
  }),
});

const res = await chain.call({
  query: `List all the books published before 2000.`,
  input: await inputParser.parse([
    {
      id: "rec0",
      created_at: "2021-11-06T00:00:00.000Z",
      fields: {
        title: "Book 1",
        seriesId: "1",
        createdAt: "1996-08-01T00:00:00.000Z",
      },
    },
    {
      id: "rec1",
      created_at: "2022-08-17T00:00:00.000Z",
      fields: {
        title: "Book 2",
        seriesId: "1",
        createdAt: "1999-08-01T00:00:00.000Z",
      },
    },
    {
      id: "rec3",
      created_at: "1994-01-08T00:00:00.000Z",
      fields: {
        title: "Novel 1",
        seriesId: "2",
        createdAt: "2010-08-31T00:00:00.000Z",
      },
    },
    {
      id: "rec4",
      created_at: "1995-01-08T00:00:00.000Z",
      fields: {
        title: "Novel 2",
        seriesId: "2",
        createdAt: "2011-09-01T00:00:00.000Z",
      },
    },
  ]),
});

console.log(await outputParser.parse(res.text));

I also made a small tweak to the output parser prompt to more strongly hint at the required output format as I noticed some flakiness while writing my tests (s/The output should/Your output must).
I didn't want to spend too much time writing docs as I'm sure there'll be a lot of feedback and requested changes, but am happy to do so as well!

Maybe StructuredInputParameterParser is a clearer name?

Found that gpt-3.5-turbo was a little bit worse at picking up the "description" with this new JSON Schema format - kept getting silly responses like:
Based on the input data provided, it is not possible to determine which books were published before 2000 as the "createdAt" field in the "fields" property only specifies the publication date of each book, not the year it was published.
gpt-4 worked better but decided to simplify the integration test to make it a bit less intentionally misleading for the LLM and it worked fine.