text:
Remove cardano-wallet-byron package

Context
ADP-356
With the hard fork combinator we have a wallet which works on both byron and shelley networks. This will be in the cardano-wallet-shelley package.
We no longer require wallets that can only work with single-era nodes.
Decision
Remove the cardano-wallet-byron package.
Salvage any tests or benchmarks and copy them across to the cardano-wallet-shelley package.

Restore benchmarks ⇒ #1695.

Note: grep is your friend, especially when tracking down byron references in the nix build.
Acceptance Criteria

There must be only one cardano-node backed cardano-wallet.
Test cases, nightly tests, benchmarks, etc should not be lost because we need more automated QA not less.

Development

QA

I is this completed? I suppose there are integration tests to be "merged" to shelley, e.g. -> https://github.com/input-output-hk/cardano-wallet/blob/master/lib/byron/test/integration/Main.hs#L128, not sure about other stuff?
QA section filled would be helpful :sweat_smile:

@piotr-iohk This is not in the QA column. Are you talking about another ticket?

Sorry, my bad. I thought I was it in QA... Jesus :tired_face:

Test cases, nightly tests, benchmarks, etc should not be lost because we need more automated QA not less.

Perhaps that is not a result of this change but Benchmarks and Windows tests are failing for quite a while now. (Although benchmarks are hopefully fixed :crossed_fingers: )

As per https://github.com/input-output-hk/cardano-wallet/issues/1970#issuecomment-672576025 this item was supposed to "fix" Windows tests...

The windows tests were fixed but then there was a new failure. I have opened #2067 for it.

lgtm

authors: Anviking piotr-iohk rvl


text:
Add fingerprint assertions to list/get endpoints in integration tests

Issue Number
ADP-703
Overview

[x] Add fingerprint assertions to list/get endpoints in integration tests

Comments

bors r+

bors r+

bors r+

bors r+

source: gharchive/pull-request

authors: piotr-iohk rvl

repo: input-output-hk/cardano-wallet

license_type: permissive

license_source: github-api


text:
LoggingSpec: Fix error handling and remove problem test cases

Issue Number
ADP-562
Overview
The test cases were getting stuck if ever there was an exception while setting up the warp server.
A common reason for this could be that the random port which was selected is no longer available.
Also I am removing the two tests which have caused so much flakiness, because we are pretty confident that the code works and will continue to work.

[x] Handle errors in the warp server setup.
[x] Delete the problem test cases.
[x] Re-enable these tests on macos.
[x] Fix some Haddock syntax errors that HLS wanted to tell me about.

bors r+

So this should fix #2368 (trivially, by removing the property), and make the failure reason for #2217 clearer.

I don't think this fixes our #2472 problems

In support of that belief, a failure on this PR:
https://hydra.iohk.io/build/6735052
Coverage checks for generators
    TokenQuantity
      +++ OK, passed 100 tests:
      39% token quantity is massive
      27% token quantity is large
      20% token quantity is small
      14% token quantity is zero
Cardano.Wallet.DB.Sqlite
  State machine test (SeqState 'Mainnet ShelleyKey)
    Validate generators & shrinkers
      Generator for CreateWallet
      Shrinker for CreateWallet
      Generator for RemoveWallet
      Shrinker for RemoveWallet
      Generator for ListWallets
      Shrinker for ListWallets
      Generator for PutCheckpoints
      Shrinker for PutCheckpoints
      Generator for ReadCheckpoint
      Shrinker for ReadCheckpoint
      Generator for ListCheckpoints
      Shrinker for ListCheckpoints
      Generator for PutWalletMeta
      Shrinker for PutWalletMeta
      Generator for ReadWalletMeta
      Shrinker for ReadWalletMeta
      Generator for PutDelegationCertificate
      Shrinker for PutDelegationCertificate
      Generator for IsStakeKeyRegistered
      Shrinker for IsStakeKeyRegistered
      Generator for PutTxHistory
      Shrinker for PutTxHistory
      Generator for ReadTxHistory
      Shrinker for ReadTxHistory
      Generator for ReadLocalTxSubmissionPending
      Shrinker for ReadLocalTxSubmissionPending
      Generator for UpdatePendingTxForExpiry
      Shrinker for UpdatePendingTxForExpiry
      Generator for RemovePendingOrExpiredTx
      Shrinker for RemovePendingOrExpiredTx
      Generator for PutPrivateKey
      Shrinker for PutPrivateKey
      Generator for ReadPrivateKey
      Shrinker for ReadPrivateKey
      Generator for RollbackTo
      Shrinker for RollbackTo
      Generator for ReadGenesisParameters
      Shrinker for ReadGenesisParameters
building of '/nix/store/wgf2560pp44yhdlnlxb02k2djgy7gf6s-cardano-wallet-core-test-unit-2021.6.11-check' timed out after 900 seconds of silence

Yep. The more flaky tests we fix, the easier things get.

authors: Anviking rvl


text:
network information CLI command

Issue Number

#819
Overview

[x] I have added a network information command to the CLI
[x] I have added a basic CLI integration test
[x] I have added an API integration scenario for methods not allowed

Comments

Feel free to add some more tests.

bors r+

bors r+

bors r-

bors r+

bors r+


text:
Replace the mini-protocol framework by a program-based one

This resolves #57.

I give this pull request draft status for the time being, since it has turned out that my preliminary implementation of the ping-pong protocol based on this framework doesn’t work with the version above.

@javierdiaz72, do you think it would be better to disband the Protocols section and define the individual locales in the subsections to which they are related: protocol_possibilities in Possibilities, protocol_state_machines in State Machines, and protocol_programs in Programs?

@javierdiaz72, do you think it would be better to disband the Protocols section and define the individual locales in the subsections to which they are related: protocol_possibilities in Possibilities, protocol_state_machines in State Machines, and protocol_programs in Programs?

Yes, I think it would be clearer that way.

@javierdiaz72, I don’t plan to add more commits for now; so you are encouraged to write a review for the present version of this new mini-protocol framework. Please take care to take all commits into account. Thank you.


text:
Avoid needing instance for GetPrevHash for both headers and blocks

Make headerPrevHash the primitive, and blockPrevHash derived. This means we only need to give/require the instance for GetPrevHash for headers, and don't need to duplicate it for blocks (which is now just a single function blockPrevHash).

bors merge

repo: input-output-hk/ouroboros-network


text:
Verify that server rethrows exceptions

net-sim: simulate accept errors
server-test: do not link threads inside withBidirectionalConnectionManager
server-test: check that server re-throws exceptions
server: fixed a typo

implements #3384
bors merge


text:
Include pre-populated traces with the demos

Now we can persist traces and have multiple traces, why not have the demos come with some pre-prepared scenarios demonstrating the contracts?

If someone can come up with some scenarios, publish them as Gists, and stick the IDs here, I'll integrate into the frontend.

Addressed by #1710.

source: gharchive/issue

authors: krisajenkins michaelpj

repo: input-output-hk/plutus


text:
Multiple/predicate

Changing predicate in multiple elements from 'parts' to 'part'

Closing pull request since ticket changed.

repo: inrupt/solid-sdk-forms

license: MIT


text:
[RFE] Enhance snapshot socket gadget with PID and container info

Current situation
The snapshot socket gadget does not include information about the PIDs/Containers using the socket.
Impact
It is particularly useful for local-gadget when the POD information cannot be available because the container might not be created via Kubernetes, making the output of local-gadget not very useful.
Ideal future situation
As discussed with @alban:

In usual use cases, the fd would belong to only one process and therefore one container. But it is possible to have the fd in several pods, for example if the different pods have a host volume to access the same unix socket file and pass fds around. That would be unusual though. In other words, the netns of a socket could be different than the netns of the processes that hold the socket fd.

However, we decided to implement only the simplest case for now and create an issue to track that limitation. Therefore, we will start by iteratating over /proc until we get the first PID using the inode within the same network namespace where we found the inode.
For IG, it also ensures that the container found will always be coherent with the pod so we can have a clean UI (with Kubernetes metadata for kubectl-gadget):
$ sudo local-gadget snapshot socket --proto tcp -e
CONTAINER    PROTOCOL    LOCAL                 REMOTE                STATUS         INODE    PID
myCont1      TCP         192.168.0.41:58202    10.96.0.1:443         ESTABLISHED    39380    1234
myCont2      TCP         10.0.2.15:22          10.0.2.2:45256        ESTABLISHED    32683    1455
myCont3      TCP         10.0.2.15:22          10.0.2.2:45264        ESTABLISHED    40037    8743

Of course, in the documentation of the gadget, we should specify that the container and PID information will be only displayed if the socket is being used by a process within the same network namespace of the pod. And in the case of multiple processes within the same network namespace using the same fd, only one of them will be displayed.
Implementation options
Study how we can iterate over /proc by using eBPF iterators. @flyth

Perhaps we can use https://github.com/inspektor-gadget/inspektor-gadget/pull/1201 for this one?

I think it can be done with the socket enricher (pkg/gadgets/internal/socketenricher). The pid-related information will be from the process creating the socket, but I think that's good enough. @alban any special considerations on this one?

authors: blanquicet mauriciovasquezbernal

repo: inspektor-gadget/inspektor-gadget


text:
Should support editing ENV for the deployed services.

In GitLab by @luotian-github on Jun 22, 2018, 17:45
operation audit sql

In GitLab by @yhua123 on Nov 8, 2019, 07:53
Duplicate #772

In GitLab by @sokril on Nov 8, 2019, 08:24
closed

In GitLab by @sokril on Nov 8, 2019, 08:24
assigned to @sokril

repo: inspursoft/board


text:
when stop node ,the node status is ready and schedulingdisabled,and always this status.the pods in this node no changed [REPLACEMENT ISSUE]

The original issue
Id: 1543
Title: when stop node ,the node status is ready and schedulingdisabled,and always this status.the pods in this node no changed

could not be created.
This is a dummy issue, replacing the original one. It contains everything but the original issue description. In case the gitlab repository is still existing, visit the following link to show the original issue:
TODO

In GitLab by @luotian-github on Nov 8, 2019, 07:55
Need to support both unschedulable and stopped, two types of node status.

In GitLab by @luotian-github on Nov 8, 2019, 07:55
schedulable/unschedulable
stopped/running

In GitLab by @liyanq528 on Nov 8, 2019, 07:55
fixed by pull request #1599

In GitLab by @cuiaq123 on Nov 8, 2019, 07:55
just icon changed in
1.0-767-g9cca511

In GitLab by @liyanq528 on Nov 8, 2019, 07:55
fixed by pull request #1609

In GitLab by @wknet123 on Nov 8, 2019, 07:55
fixed issue #1543

In GitLab by @sokril on Nov 8, 2019, 08:25
closed

In GitLab by @sokril on Nov 8, 2019, 08:25
assigned to @cuiaq123

In GitLab by @liyanq528 on Nov 12, 2019, 02:50
mentioned in merge request !1046

In GitLab by @liyanq528 on Nov 13, 2019, 05:01
mentioned in merge request !838

In GitLab by @liyanq528 on Nov 13, 2019, 09:18
mentioned in merge request !1287


text:
please add info-icon at node-list operation like service-list

In GitLab by @wknet123 on Dec 4, 2017, 16:43
This upload file name should be cleaned when failed to upload file.

In GitLab by @yhua123 on Nov 8, 2019, 07:41
fixed, close

In GitLab by @sokril on Nov 8, 2019, 08:15
closed

In GitLab by @sokril on Nov 8, 2019, 08:15
assigned to @liyanq528


text:
Добавил swagger

Добавил swagger с использованием springdoc
Добавил в swagger способы аутентификации через basic auth и bearer auth

Еще одна крутая штука!

authors: Andrey-Kudryavtsev papanage

repo: intabia-intership/wikitabia-server


text:
🛑 Shopintake is down

In 1626e8d, Shopintake (https://www.shopintake.com) was down:

HTTP code: 403
Response time: 508 ms

Resolved: Shopintake is back up in 20f0660 after 12 minutes.


text:
Add and use dependencies from another playground permalink

I would like to reuse code saved in one playground in another, different playground without having to copy-paste the contents of the original, or having to publish it. This could be useful in several scenarios:

📚 Publishing some light library and reusing it in multiple pastes. If there are many copies of a paste with small differences (which I suspect is a common pattern), this could reduce total paste size and improve compile times for the playground runners by caching the dependencies' object files.
🐛 If someone is using a paste to demonstrate an issue with their code, reusing it as a dependency with some small tweak would naturally highlight the changes made and may aid their comprehension of the solution.
👨🏫 For education purposes, I could see a teacher create a paste that exposes public evaluation functions which must be called to clear some assignment. This would enable the student to focus on the assignment tasks and enables the teacher to know that the evaluation code was not tampered with.

👋 To use it, the playground could reserve a special module name pattern that would dynamically pull in other gists when referencing public items within it. 🚲 Bikeshedding notwithstanding, perhaps something like:
use gist_16aa5441c37dc67c7ce0c8b4b48c194b::{some_pub_fn, other_pub_fn};
// or under some namespace:
use playground::gist_16aa5441c37dc67c7ce0c8b4b48c194b::{some_pub_fn, other_pub_fn};

👩💻 I'm unsure exactly how this could be implemented. Maybe proc macros? A build.rs script that downloads the gist file before compilation? (this could even be reused outside of a playground environment [very carefully]). If using these features introduces undesirable attack surface, parsing the paste for use statements ahead of time could still be workable.
🔐 I don't think this would cause a material change to possible abuse cases. Obviously, gists saved by the playground are never changed so using another gist as a dependency could not cause pastes to break or cause some observable change in behavior. Total compilation+runtime would still be bounded by the playground runner. Dependencies would still have to be trees, and any code that uses this "paste splitting" feature could be automatically rewritten into a single file to review manually if needed.
Thoughts?

An interesting idea! Allow me to respond inline with a stream of consciousness...

Publishing some light library and reusing it in multiple pastes.

Believable, but it's hard to know how many people would use such a feature, and thus how much of a return-on-investment we might get.

If there are many copies of a paste with small differences (which I suspect is a common pattern), this could reduce total paste size

I don't exactly follow how — are you saying that someone would perform the work to extract the common code into this new library style and then just reference that?

improve compile times for the playground runners by caching the dependencies' object files.

I don't think this is a likely outcome. We really never want to share things between compilation sessions of different users.

If someone is using a paste to demonstrate an issue with their code, reusing it as a dependency with some small tweak would naturally highlight the changes made and may aid their comprehension of the solution.

Having a diff of sorts does seem useful, but I'm not following how it would work with this solution. If someone brings it in as a dependency, how would they then edit the dependency?

I could see a teacher create a paste that exposes public evaluation functions which must be called to clear some assignment. This would enable the student to focus on the assignment tasks

I agree (and I use the playground a lot for teaching purposes!).

and enables the teacher to know that the evaluation code was not tampered with.

How does this co-exist with the previous "with some small tweak"?

I don't think this would cause a material change to possible abuse cases.

I agree in general. The main problem I could see would be some form of DOS attack if we allowed recursive dependencies (especially if they were self-referential). That should be avoidable by only going one level deep.

Implementation wise, I can share some semi-planned future work that might make this more achievable. For a long time, people have been wanting to create multiple files, both for modules as well as multiple crates. I've been thinking about extending the playground's gist format to include a special TOML file (or something) that would allow us to store multiple files / a Cargo.toml / etc. If that happens, we could load that and rename the crate, leveraging normal Cargo packages.
That would make this feature mostly a front-end task focused on shuffling around the files appropriately.

We really never want to share things between compilation sessions

Reasonable.

If someone brings it in as a dependency, how would they then edit the dependency?

Maybe this was a bit of a stretch, basically the 'library' code would have to be organized in a way to enable dependents to swap out parts. This should be possible to design intentionally, but I agree that it would not work for arbitrary code.

I've been thinking about extending the playground's gist format to .. store multiple files .. If that happens, we could load [a previous gist] and rename the crate, leveraging normal Cargo packages. That would make this feature mostly a front-end task focused on shuffling around the files appropriately.

This sounds like a great way to implement this feature.

authors: infogulch shepmaster

license_source: bigquery


text:
Evaluate substrate externalities

Our externalities are based on a very old substrate version. In the meantime they have upgraded quite a bit:
https://github.com/paritytech/substrate/blob/7de3d7870f63a525977fd4e91cda16039e7b1d60/primitives/externalities/src/lib.rs#L82
They have quite some background things, such as the state-machine:
https://github.com/paritytech/substrate/tree/7de3d7870f63a525977fd4e91cda16039e7b1d60/primitives/state-machine/src
To be evaluated: Should we update our externalities to match the ones of the substrate, such that we can make use of the full sp-io?

Evaluation criteria:

is it too complex
how easy can it be ported to sgx

authors: clangenb haerdib

repo: integritee-network/worker


text:
chroma-config setup command results in unresponsive IML

On an existing IML 4.0.0 installation, running chroma-config setup with or without further parameters, seems to complete successfully but then when trying to bring up the webpage for IML, nothing appears and eventually a 502 bad gateway is returned.
This was experienced on lotus with Jenkins IML build 2353 but will probably occur on master given the build patch changes seem unrelated.

Need more info on this one, this is a really core issue that we should see in testing.
Can you trace what is causing that 502?

this is when running the command after already having configured and set up IML, it sounds like this should only be run to initially set up IML and in that case , this is not a valid use case (and that should be made clear in documentation)

Even then it shouldn't cause this issue. Can you reproduce / paste the resulting stacktrace behind the 502?

Here is the stack trace provided in the browser:
Exception rendering resources: Error: {"error_message": "Sorry, this request could not be processed. Please try again later."} From GET request to /api/filesystem/?limit=0
    at ResponseError (/usr/share/chroma-manager/ui-modules/node_modules/@iml/view-server/dist/bundle.js:4068:5)
    at ClientRequest.transport.request.r (/usr/share/chroma-manager/ui-modules/node_modules/@iml/view-server/dist/bundle.js:4085:42)
    at ClientRequest.g (events.js:292:16)
    at emitOne (events.js:96:13)
    at ClientRequest.emit (events.js:188:7)
    at HTTPParser.parserOnIncomingClient (_http_client.js:474:21)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:99:23)
    at TLSSocket.socketOnData (_http_client.js:363:20)
    at emitOne (events.js:96:13)
    at TLSSocket.emit (events.js:188:7)

chroma-config validate reports that services are all running (OK)

chroma-config restart results in GUI responding as normal
chroma-config setup admin examplepassword localhost results in the following exceptions for threads 1-4 in /var/log/chroma/gunicorn-error.log:
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner
    self.run()
  File "/usr/share/chroma-manager/chroma_core/services/rpc.py", line 299, in run
    connection.drain_events(timeout = 1)
  File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 275, in drain_events
    return self.transport.drain_events(self.connection, **kwargs)
  File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 91, in drain_events
    return connection.drain_events(**kwargs)
  File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 320, in drain_events
    return amqp_method(channel, args)
  File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 523, in _close
    self._x_close_ok()
  File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 551, in _x_close_ok
    self._send_method((10, 51))
  File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 56, in _send_method
    self.channel_id, method_sig, args, content,
  File "/usr/lib/python2.7/site-packages/amqp/method_framing.py", line 221, in write_method
    write_frame(1, channel, payload)
  File "/usr/lib/python2.7/site-packages/amqp/transport.py", line 177, in write_frame
    frame_type, channel, size, payload, 0xce,
  File "/usr/lib64/python2.7/site-packages/gevent/socket.py", line 458, in sendall
    data_sent += self.send(_get_memory(data, data_sent), flags)
  File "/usr/lib64/python2.7/site-packages/gevent/socket.py", line 435, in send
    return sock.send(data, flags)
error: [Errno 104] Connection reset by peer

Then , when refreshing the IML UI webpage (chrome browser), after a while of inactivity, 502: Bad Gateway nginx message is displayed.
Surprisingly, /var/log/nginx/error.log is empty and presumably these messages are redirected somewhere else.

authors: jgrund tanabarr

repo: intel-hpdd/intel-manager-for-lustre

license: mit


text:
src/java/CMakeLists.txt: Modified path for javac and jar

Signed-off-by: Andrei Vasiliu andrei.vasiliu@intel.com

Merged, please add more details on why you do things like this in the commit msg next time.

Hello.
On 9 Oct 2015 4:38 pm, "Brendan Le Foll" notifications@github.com wrote:

Merged, please add more details on why you do things like this in the
commit msg next time.
Sorry about that. I will be more explicit next time.

Andrei

—
Reply to this email directly or view it on GitHub.

repo: intel-iot-devkit/mraa


text:
Memory leak in service_topo.cpp on Windows x64 with core count more 32

On Windows x64 system in service_topo.h type LNX_PTR2INT is defined as follows:
#ifdef  _x86_64
#define LNX_PTR2INT __int64
#define LNX_MY1CON 1LL
#else
#define LNX_PTR2INT unsigned int
#define LNX_MY1CON 1
#endif
Under Windows x64 macro _M_X64 should be used.
Because LNX_PTR2INT is defined as unsigned int which on Windows is 32 bit
then function static int __internal_daal_countBits(DWORD_PTR x) return maximum of 32,
then functiion static void  __internal_daal_setChkProcessAffinityConsistency( unsigned int lcl_OSProcessorCount )
in statement on line 328
if( sum != lcl_OSProcessorCount ) // check cumulative bit counts matches processor count
determines inconsistensy;
then in function static int __internal_daal_queryParseSubIDs(void) in statement on line 1346
if( glbl_obj.error )
return -1;
error is detected which is returned to function static void __internal_daal_buildSystemTopologyTables()
which leads to exit without initialization
In function static void __internal_daal_buildSystemTopologyTables()
tables are allocated on line 1683 and should be deallocated in case of an error
before returns on lines: 1687, 1689, 1694
function __internal_daal_buildSystemTopologyTables is called from
__internal_daal_buildSystemTopologyTables which is called from
__internal_daal_initCpuTopology which is called from
__internal_daal_GetSysProcessorCoreCount which is called from
GetL1CacheSize which is called in evere call to compute

Hello Alexander!

now we are thinking about disabling TheradPinning functionality at Windows. Do you use TheradPinning?
We will fix it, if it will be decided to keep TheradPinning.
glktsn has destructor (glktsn::FreeArrays() in the same source). all allocated tables free in it.
Andrey

Hello!
Ok. Now I understand the usage case and see where problem is.
If you use open source DAAL, as quick workaround, I suggest to build libs with define DAAL_CPU_TOPO_DISABLED.
Andrey

Hello, Andrey
Yes, there are two problems:

Using macro _x86_64 instead of _M_X64 in service_topo.h (at least on with Windows with MSVC compiler)

Incorrect clean up after error.

I have corrected issue #1 and everithing is working (at least for less than 64 cores)
Thanks,
Alexander

authors: VasilyevAlexander dr-pain

repo: intel/daal


text:
Multiple issues with setting up the text chatbot service on SPR

Hi,
I have followed this instruction intel-extension-for-transformers/intel_extension_for_transformers/neural_chat/docs/notebooks/setup_text_chatbot_service_on_spr.ipynb at main · intel/intel-extension-for-transformers (github.com) and written down a couple of issues with potential solutions, which you may want to consider implementing.
I am using Ubuntu 22.04 LTS and Python 3.10.12.

Setup backend / Setup environment
!git clone https://github.com/intel/intel-extension-for-transformers.git

The instruction says to used HEAD on the master branch, even though the repository is being actively developed. This causes problems like this 422 Unprocessable Entity using Neural Chat via OpenAI interface with meta--lama/llama-2-7b-chat-hf · Issue #1288 · intel/intel-extension-for-transformers (github.com)
I have also encountered the aforementioned issue, and then decided to use the latest release which is v1.3.1.
It would be useful to add information to the instruction about a commit/release it was validated with.
I was continuing with v1.3.1 from now on.

Setup backend / Setup environment

%cd ./intel-extension-for-transformers/intel_extension_for_transformers/neural_chat/
!pip install -r requirements.txt

The requirements.txt installs torch==2.1.0, but once the backend server is started, it complains about torch compatility:

ERROR! Intel® Extension for PyTorch* needs to work with PyTorch 2.2.*, but PyTorch 2.1.0+cu121 is found. Please switch to the matching version and run again.

The work-around is to reinstall torch and its dependencies manually to get compatible versions:

pip uninstall torch torchaudio torchvision xformers -y
pip install torch torchaudio torchvision xformers

Setup backend / Setup environment
!pip install nest_asyncio

It is a bit confusing that nest_syncio has to be installed manually and is not added to requirements.txt for the backend.

Deploy frontend on your server / Install the required Python dependencies
!pip install -r ./examples/deployment/textbot/frontend/requirements.txt

The requirements.txt again installs torch==2.1.0, which makes the backend unusable. Please consider using compatible packages for both components, or maybe suggest in the instruction to create a separate Python virtual environment for each component if the same host is used.

Deploy frontend on your server / Run the frontend
!nohup python app.py &

There is an issue with fastchat.utils when starting app.py:

Traceback (most recent call last):
File "[…]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/app.py", line 38, in
from fastchat.utils import (
ImportError: cannot import name 'violates_moderation' from 'fastchat.utils' ([…]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/fastchat/utils.py)

I have worked around this by removing the reference to violates_moderation in app.py, but you may want to investigate the problem further.

Deploy frontend on your server / Run the frontend
!nohup python app.py &

There is an issue with gradio package which occurs when a NeuralChat URL is loaded in a browser:

2024-02-21 13:18:12 | INFO | gradio_web_server | Models: ['Intel/neural-chat-7b-v3-1']
2024-02-21 13:18:13 | ERROR | stderr | sys:1: GradioDeprecationWarning: The style method is deprecated. Please set these arguments in the constructor instead.
2024-02-21 13:18:13 | INFO | stdout | Running on local URL:  http://0.0.0.0:8080/
2024-02-21 13:18:13 | INFO | stdout |
2024-02-21 13:18:13 | INFO | stdout | To create a public link, set share=True in launch().
2024-02-21 13:18:48 | INFO | stdout | 1 validation error for PredictBody
2024-02-21 13:18:48 | INFO | stdout | event_id
2024-02-21 13:18:48 | INFO | stdout |   Field required [type=missing, input_value={'data': [{}], 'event_dat...on_hash': 'w9eie3cvduh'}, input_type=dict]
2024-02-21 13:18:48 | INFO | stdout |     For further information visit https://errors.pydantic.dev/2.6/v/missing

The solution to this issue is updating gradio to at least version 3.50.2.
The incompatible version is installed in the source code of app.py, so this line has to be changed:

os.system("pip install gradio==3.36.0")

Deploy frontend on your server / Run the frontend
 !nohup python app.py &

After the NeuralChat URL successfully loads in the browser, the chat replies only with:

NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE. (error_code: 4)

It is cause by the error in the backend:

ModuleNotFoundError: No module named 'neural_speed'

To fix this, one may add neural_speed to intel-extension-for-transformers/intel_extension_for_transformers/neural_chat/requirements.txt, or install the package manually.

Just wondering what is the reason that the frontend is able to handle Intel/neural-chat-7b-v3-1 and meta-llama/Llama-2-7b-chat-hf, but for example fails with meta-llama/Llama-2-13b-chat-hf:

2024-02-21 13:39:08 | ERROR | stderr | Traceback (most recent call last):
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/gradio/queueing.py", line 407, in call_prediction
2024-02-21 13:39:08 | ERROR | stderr |     output = await route_utils.call_process_api(
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/gradio/route_utils.py", line 226, in call_process_api
2024-02-21 13:39:08 | ERROR | stderr |     output = await app.get_blocks().process_api(
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/gradio/blocks.py", line 1550, in process_api
2024-02-21 13:39:08 | ERROR | stderr |     result = await self.call_function(
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/gradio/blocks.py", line 1199, in call_function
2024-02-21 13:39:08 | ERROR | stderr |     prediction = await utils.async_iteration(iterator)
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/gradio/utils.py", line 519, in async_iteration
2024-02-21 13:39:08 | ERROR | stderr |     return await iterator.anext()
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/gradio/utils.py", line 512, in anext
2024-02-21 13:39:08 | ERROR | stderr |     return await anyio.to_thread.run_sync(
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
2024-02-21 13:39:08 | ERROR | stderr |     return await get_async_backend().run_sync_in_worker_thread(
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
2024-02-21 13:39:08 | ERROR | stderr |     return await future
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 851, in run
2024-02-21 13:39:08 | ERROR | stderr |     result = context.run(func, *args)
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/gradio/utils.py", line 495, in run_sync_iterator_async
2024-02-21 13:39:08 | ERROR | stderr |     return next(iterator)
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/veee/lib/python3.10/site-packages/gradio/utils.py", line 649, in gen_wrapper
2024-02-21 13:39:08 | ERROR | stderr |     yield from f(*args, **kwargs)
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/app.py", line 331, in http_bot
2024-02-21 13:39:08 | ERROR | stderr |     new_state = get_conv_template(model_name.split('/')[-1])
2024-02-21 13:39:08 | ERROR | stderr |   File "[...]/intel-extension-for-transformers-1.3.1/intel_extension_for_transformers/neural_chat/ui/gradio/basic/./conversation.py", line 300, in get_conv_template
2024-02-21 13:39:08 | ERROR | stderr |     return conv_templates[name].copy()
2024-02-21 13:39:08 | ERROR | stderr | KeyError: 'Llama-2-13b-chat-hf'

The backend seems to load meta-llama/Llama-2-13b-chat-hf correctly. Maybe enabling other models from the same family in the frontend would not require many changes.
Thank you for taking the time to read trough all this text :)

@tbykowsk,
Apologies for the delayed response regarding this issue. I hadn't realized it was assigned to me.
You can refer to the quick start example for setuping the chatbot service. I've updated the notebook via PR 1482, you can also try the notebook using this branch.
If you have any questions, please feel free to ask, and you can reach me directly on Teams, which should be more efficient.
Thanks.

Thank you @lvliang-intel,
I can confirm that issues 1-7 for instruction notebooks/setup_text_chatbot_service_on_spr.ipynb are resolved.
The 8th issue involves a model which is still not supported, so maybe it is beyond the scope of this project.
However, the instruction lacks information where to execute the step "Startup the backend server". A line could be added that the server has to be run from the main directory of repository (./intel-extension-for-transformers). The previous step changes the current directory to ./intel-extension-for-transformers/intel_extension_for_transformers/neural_chat/ and the server reports missing dependencies if started from there.
For clarity, a full path could be provided again in the step "Deploy frontend on your server":
%cd ./intel-extension-for-transformers/intel_extension_for_transformers/neural_chat/ui/gradio/basic
========================================================================
I also followed the quick start example which you referred, focusing on steps for CPU.
Sections "1. Setup Environment" and "2. Run the chatbot in command mode" executed without any issues.
In the section "3. Run chatbot in server mode with UI" in step "3.1.1 Verify the client connection to server is OK." I got this error:
$ curl -vv -X POST http://127.0.0.1:8000/v1/chat/completions
* Uses proxy env variable no_proxy == '10.0.0.0/8,localhost,.local,127.0.0.0/8'
*   Trying 127.0.0.1:8000...
* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
> POST /v1/chat/completions HTTP/1.1
> Host: 127.0.0.1:8000
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 422 Unprocessable Entity
< date: Fri, 19 Apr 2024 07:34:50 GMT
< server: uvicorn
< content-length: 81
< content-type: application/json
<
* Connection #0 to host 127.0.0.1 left intact
{"detail":[{"loc":["body"],"msg":"field required","type":"value_error.missing"}]}

Output from the server:
$ python chatbot_server.py
[…]
2024-04-19 07:28:28,318 - root - INFO - Model loaded.
Loading config settings from the environment...
INFO:     Started server process [935582]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000/ (Press CTRL+C to quit)
INFO:     127.0.0.1:53466 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity

However, the next step "3.1.2 Test request command at client side" executed correctly.
Also sections "3.2 Set up Server mode UI" and "3.3 Start the web service" did not return any errors.
The main issue which I noticed with this quick start example, is that NeuralChat replies over 10 times slower than NeuralChat from the previous instruction notebooks/setup_text_chatbot_service_on_spr.ipynb.
Both times I used the same question:

Tell me about Intel Xeon Scalable Processors.

and received very similar answers. NeuralChat from the quick start example was generating a response for around 13 minutes, whereas NeuralChat from the previous instruction took only around 50 seconds. I ran this experiment on the same platform.
Log from the quick start example:
2024-04-19 08:27:34 | INFO | gradio_web_server | ==== request ====
{'model': 'Intel/neural-chat-7b-v3-1', 'messages': [{'role': 'system', 'content': '### System:\n    - You are a helpful assistant chatbot trained by Intel.\n    - You answer questions.\n    - You are excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.\n    - You are more than just an information source, you are also able to write poetry, short stories, and make jokes.</s>\n'}, {'role': 'user', 'content': 'Tell me about Intel Xeon Scalable Processors.'}], 'temperature': 0.001, 'top_p': 0.95, 'max_tokens': 512, 'stream': True}
2024-04-19 08:40:42 | INFO | gradio_web_server | Intel Xeon Scalable Processors, also known as the "Cascade Lake" family, are a series of high-performance central processing units (CPUs) designed for data centers, cloud computing, and other demanding workloads. These processors are built upon Intel's latest 14nm process technology and feature a modular design that allows for flexible configurations to meet various needs. The Xeon Scalable Processors offer significant improvements in performance, efficiency, and scalability compared to their predecessors. They are designed to handle a wide range of tasks, from general-purpose computing to specialized applications like artificial intelligence, high-performance computing, and virtualization. Some key features of the Intel Xeon Scalable Processors include: 1. Scalable architecture: The modular design allows for customization, enabling users to choose the right combination of cores, memory, and I/O capabilities to match their specific requirements. 2. High-performance cores: The processors feature Intel's latest high-performance cores, which deliver increased performance and efficiency for a variety of workloads. 3. Advanced security features: The Xeon Scalable Processors come with built-in security features, such as Intel Software Guard Extensions (SGX), which help protect sensitive data and code from unauthorized access. 4. Enhanced memory support: The processors support up to 4TB of memory, enabling them to handle large datasets and complex workloads efficiently. 5. Improved I/O capabilities: The Xeon Scalable Processors offer advanced I/O capabilities, including support for PCIe 3.0 and 4.0, which can significantly improve data transfer speeds and overall system performance. In summary, the Intel Xeon Scalable Processors are a powerful and versatile family of CPUs designed to meet the demands of modern data centers and high-performance computing environments. They offer a range of features that make them suitable for various applications, from general-purpose computing to specialized tasks like artificial intelligence and virtualization.

Log from the previous instruction:
2024-04-19 08:51:33 | INFO | gradio_web_server | ==== request ====
{'model': 'Intel/neural-chat-7b-v3-1', 'messages': [{'role': 'system', 'content': '### System:\n    - You are a helpful assistant chatbot trained by Intel.\n    - You answer questions.\n    - You are excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.\n    - You are more than just an information source, you are also able to write poetry, short stories, and make jokes.</s>\n'}, {'role': 'user', 'content': 'Tell me about Intel Xeon Scalable Processors.'}], 'temperature': 0.001, 'top_p': 0.95, 'max_tokens': 512, 'stream': True}
2024-04-19 08:51:33 | INFO | httpx | HTTP Request: POST http://127.0.0.1:8000/v1/chat/completions "HTTP/1.1 200 OK"
[…]
2024-04-19 08:52:24 | INFO | httpx | HTTP Request: POST http://localhost:8080/api/predict "HTTP/1.1 200 OK"
2024-04-19 08:52:24 | INFO | gradio_web_server | Intel Xeon Scalable Processors, also known as the "Cascade Lake" family, are a series of high-performance central processing units (CPUs) designed for data centers, cloud computing, and other demanding workloads. These processors are built upon Intel's latest 14nm process technology and feature a modular design that allows for flexible configurations to meet various needs. The Xeon Scalable Processors offer significant improvements in performance, efficiency, and scalability compared to their predecessors. They are designed to handle a wide range of tasks, from general-purpose computing to specialized applications like artificial intelligence, high-performance computing, and virtualization. Some key features of the Intel Xeon Scalable Processors include: 1. Scalable architecture: The modular design allows for customization, enabling users to choose the right combination of cores, memory channels, and other features to optimize performance for their specific workloads. 2. High-speed memory support: The processors support up to 6 memory channels, enabling faster data access and improved performance for memory-intensive applications. 3. Advanced security features: The Xeon Scalable Processors come with built-in security features like Intel Software Guard Extensions (SGX) and Intel Trusted Execution Technology (TXT) to protect sensitive data and prevent unauthorized access. 4. Enhanced virtualization capabilities: The processors are designed to support multiple virtual machines, allowing for efficient resource utilization and improved performance in virtualized environments. 5. Improved power efficiency: The Xeon Scalable Processors are designed to optimize power consumption, reducing operational costs and minimizing environmental impact. In summary, the Intel Xeon Scalable Processors are a powerful and versatile family of CPUs that cater to the needs of various industries and applications. They offer scalability, high performance, and advanced security features, making them a popular choice for data centers and other demanding computing environments.

Please notice the time difference between the question and reply. Are there any optimizations missing in the quick start example?
========================================================================
I have also read the main NeuralChat Reame for completeness.
In the step Launch OpenAI-compatible Service I was not able to run
neuralchat_server start --config_file ./server/config/neuralchat.yaml
and I tried from different directories within the repository . The application always ended up with this error:
[...]
2024-04-19 10:19:19,158 - root - INFO - Model loaded.
Loading config settings from the environment...
[2024-04-19 10:19:19,192] [   ERROR] - Failed to start server.
[2024-04-19 10:19:19,192] [   ERROR] - No module named 'langchain_community'

I received the same error while running the example Python Code from ./intel-extension-for-transformers/intel_extension_for_transformers/neural_chat directory:
[...]
    if self.init(config):
  File "[…]/intel-extension-for-transformers/my2_venv/lib/python3.10/site-packages/intel_extension_for_transformers/neural_chat/server/neuralchat_server.py", line 326, in init
    from .restful.api import setup_router
  File "[…]/intel-extension-for-transformers/my2_venv/lib/python3.10/site-packages/intel_extension_for_transformers/neural_chat/server/restful/api.py", line 26, in <module>
    from .retrieval_api import router as retrieval_router
  File "[…]/intel-extension-for-transformers/my2_venv/lib/python3.10/site-packages/intel_extension_for_transformers/neural_chat/server/restful/retrieval_api.py", line 38, in <module>
    from ...pipeline.plugins.retrieval.parser.context_utils import clean_filename
  File "[…]/intel-extension-for-transformers/my2_venv/lib/python3.10/site-packages/intel_extension_for_transformers/neural_chat/pipeline/plugins/retrieval/parser/context_utils.py", line 21, in <module>
    from langchain_community.document_loaders import UnstructuredMarkdownLoader
ModuleNotFoundError: No module named 'langchain_community'

The Python code only worked when executed from the root directory of the repository (./intel-extension-for-transformers).  Once the server started, I was able to connect to it using all methods from Asses the Service section.
Is there a configuration step missing in this Readme (analogically like in the first instruction) regarding from where the neuralchat_server or the Python code have to be run? I create a separate Python virtual environment for each experiment to maintain order on the machine. Could there be something specific in my execution environment?
Thanks!

Quick update, I have taken the yesterday's release of v1.4.1 and can confirm that both issues with paths are resolved (for instruction notebooks/setup_text_chatbot_service_on_spr.ipynb and for the main NeuralChat Reame).
The sample code of backend server runs without any errors from ./intel-extension-for-transformers/intel_extension_for_transformers/neural_chat directory, as well as neuralchat_server executable.
The performance issue with quick start example is also fixed. The only problem remains in step "3.1.1 Verify the client connection to server is OK", unless receiving 422 Unprocessable Entity there is expected. Thanks!

@lvliang-intel,
could you please clarify the expected outcome in step 3.1.1 Verify the client connection to server is OK? Receiving error 422 Unprocessable Entity when following the instruction might be misleading

Hi @tbykowsk,
The output of 422 Unprocessable Entity in step 3.1.1 Verify the client connection to server is OK is as expected.
Further explanation

422 Unprocessable Entity means that you are able to connection to backend client and receive an HTTP status code.
It is because there's no -d {} statement in this curl command, and the backend did not receive the correct data format.

Thank you @letonghan for the clarification.
I have noticed that the information about 422 error was also updated in the README, so I'm closing the issue.
Thank you all for addressing my comments!

authors: letonghan lvliang-intel tbykowsk

repo: intel/intel-extension-for-transformers


text:
[SQ bug] unifiy weight_amax for modules same input

Type of Change
bug fix
Description
unifiy weight_amax for modules same input, or qkv will get different scale and got wrong accuracy.
Expected Behavior & Potential Risk
facebook/opt125m gets correct accuracy on ipex2.1, folding=False.
How has this PR been tested?
local test

Please add UT and also fix the coverage gap.

repo: intel/neural-compressor


text:
deploy: fix 1.14 and 1.15, webhook was missing

PR #530 broke deployment on 1.14 and 1.15 because it made all
components of a deployment mandatory but didn't add the "webhook"
link.

This is intentionally a branch in the main repo to get full testing:
https://cloudnative-k8sci.southcentralus.cloudapp.azure.com/job/pmem-csi/job/fix-1.14/

Support for objectSelector was only added in 1.15:

1.14: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.14/#webhook-v1beta1-admissionregistration-k8s-io
1.15: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.15/#mutatingwebhook-v1beta1-admissionregistration-k8s-io

I'll add a base configuration of the webhook which doesn't use it, which means that on 1.14 one can only exclude PMEM-CSI itself by installing it in a namespace that is excluded, like kube-system. For all other releases I'll patch in the objectSelector, so nothing will change there.

which means that on 1.14 one can only exclude PMEM-CSI itself by installing it in a namespace that is excluded

This implies that at least on 1.14, we either need to install it in kube-system or exclude the "default" namespace. I'm leaning towards the later because it is simpler and keeps the deployment similar to other releases - unless we want to use a non-default namespace also there?

A test run with all tests (i.e. not just pre-merge) is currently on-going: https://cloudnative-k8sci.southcentralus.cloudapp.azure.com/job/pmem-csi/job/fix-1.14-II/

I had to create a second branch for that, somehow Jenkins started to ignore the existing one.

repo: intel/pmem-csi


text:
patch_sklearn() is not working but command line python3 -m sklearnex <file.py> is working

Please fix the issue cause this package may have many potentials

Can you please share details on versions used and code for reproduction.

as referred by documentation
python 3.11
sklearn 1.3.0
sklearnex 2023

sklearnex 2023.* is outdated, 2024.0.1 is available in all channels and 2024.1.0 was just released in PyPI (pip) yesterday.

sklearnex 2023.* is outdated, 2024.0.1 is available in all channels and 2024.1.0 was just released in PyPI (pip) yesterday.

But still 2024.1.0 does not support python 3.12.0

sklearnex 2023.* is outdated, 2024.0.1 is available in all channels and 2024.1.0 was just released in PyPI (pip) yesterday.

But still 2024.1.0 does not support python 3.12.0

python 3.12 support will be added in the next releases.

sklearnex 2023.* is outdated, 2024.0.1 is available in all channels and 2024.1.0 was just released in PyPI (pip) yesterday.

But still 2024.1.0 does not support python 3.12.0

already supported in pypi and conda intel channel

authors: Alexsandruss AnuraagaNath napetrov samir-nasibli

repo: intel/scikit-learn-intelex


text:
Add Aten::multilabel_margin_loss forward/backward op

[x] multilabel_margin_loss forward
[x] multilabel_margin_loss backward

authors: chunhuanMeng xytintel

repo: intel/torch-xpu-ops


text:
Update plugin to work with latest ILP Kit

Uses binary packet and fixes many bugs. Also adds RPC-based messaging, which is currently a work-in-progress.

Didn't actually look at the code changes, just saw that sharafian/master is the version used in the seven-ledger demo, so that's probably what we want to have in this repo before the https://github.com/interledgerjs/ilp-plugin-payment-channel-framework refactor starts.

authors: michielbdejong sharafian

repo: interledgerjs/ilp-plugin-ethereum


text:
Twig Migration: Run new twig tests in github actions

Currently the tests are run against the legacy twig it should be changed to run against the new twig

@bashlk so can we close this?

Yeah we can close this @justintemps

authors: Shashika6 bashlk justintemps


text:
Add support for using OS environement variables

This is more related to private registries and applications, but can be useful on mudles too.
Sometimes configuration like paths to XSD files and certs are set at the OS level in the environment variables, and at times this might even be pulled through to the OS from secrets on your cloud solution.
It will be helpful is these environment variables are accessible by the module.xml.

IPM already supports expressions, and you can use it this way, for instance in this case we read environment variable ISC_PACKAGE_INSTALLDIR, " needs instead of "
"#{$system.Util.GetEnviron("ISC_PACKAGE_INSTALLDIR")}"

Thank you. I did not see this on an example or documentation. And working through all the code takes too much time.

authors: csoft-sc daimor

repo: intersystems/ipm


text:
[Bug]: chat window sluggish/lags

Before Reporting

[X] I have checked FAQ, and there is no solution to my issue
[X] I have searched the existing issues, and there is no existing issue for my issue

What happened?
after pasting around 37 lines of code into the chat window the window becomes sluggish/laggy when trying to type into the chat box.
How to reproduce the bug.
I found after pasting around 37 lines of code into the chat box and then trying to type or paste more the window becomes sluggish and typing is laggy (character by character).
Relevant log output
No response
macOS version
13.1
Xcode version
14.3
Copilot for Xcode version
0.14.1
Node version
No response

Released in 0.15.0

authors: DatZig intitni

repo: intitni/CopilotForXcode


text:
🛑 Bonnefanten is down

In 6b09fc0, Bonnefanten (https://www.bonnefanten.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Bonnefanten is back up in 6c8ec0a after 17 minutes.

repo: intk/intk-upptime


text:
🛑 Bonnefanten is down

In 75b24b7, Bonnefanten (https://www.bonnefanten.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Bonnefanten is back up in 3bac03a after 12 minutes.


text:
0.3.3 release with Rails 3.2 compatibility

Any chance we can get one last 0.3.3 release that works with Rails 3.2?

@fbjork What's stopping you from upgrading to 1.0?

Just time spent testing all the providers we use that are now separate gems. Eventually we'll go 1.x, but I'm sure there are others in the same situation that cannot upgrade to Rails 3.2.x for this very reason.

authors: fbjork sferik

repo: intridea/omniauth


text:
Unable to record/transcribe

Hi,
I'm running ODAS on a Raspberry Pi with Respeaker 4-mic array and running ODAS Web on a remote laptop running WSL on Windows 10. I'm somehow just not able to save any recordings once ODAS Web is running. I check the "Determine if separated audio is recorded" box. As soon as I start speaking, the microphone array picks up my voice and the terminal on my remote machine shows that "Recorder 0 started", but then shows "Recorder 0 was false active" - it then goes "Write stream 0 full", "Holding samples" and "Recorder 0 ended". The files that get saved are either of duration -1.00 seconds like in another issue here, or 1.54 seconds long, with barely any discernible sound. Also as soon as the recorder begins, the ODAS Web GUI really slows down, and I can't even kill the process on the Raspberry Pi easily.
Plus, the transcription function is not working either, although I'm not sure if that's due to this recorder problem or a separate issue of its own. I have an API key for Google's speech API (at least, I think so) but I'd like to know where one should download it from. I'm attaching my API key, you guys could confirm if it's correct or not.
SpeechTranscription-f7dd9e0aeebf.zip
Thanks!

Hi, let's start with the recording issue.
It looks like the ODAS Web process cannot write the audio file fast enough. I never used the WSL so I can't tell if the issue comes from there. ODAS Web then hangs because the buffer are getting out of hands (the slow GUI part) and stops receiving audio samples, which leads to ODAS freezing (that's why the process is hard to kill on the Pi). What is your ODAS sink config? Maybe the sample rate is juste too high and it can't write fast enough (parsing raw audio in a javascript loop is a rather suboptimal process). 16 000 is normally a safe choice.
For the transcription part, I will refer you to the Google Cloud documentation for getting your key file. However if recording is not working I'm not sure text to speech will, so I would try to sort one issue at the time.

I don't know what you mean exactly by "sink config" but the config file I run for ODAS on the RPi has the sampling rate for "raw" as 16000. However the sampling frequency for "separated" and "postfiltered" are 44100.

Yes those are the ones. They "sink" data out of ODAS. Try setting them at 16000 like your RAW (and adjust the ODAW Web configuration accordingly).

Okay, that makes sense. I'll try!

Also, should they be the same as RAW even when I'm recording on the raspberry Pi itself (without ODAS Web, just saving it to a file on the Pi)?

Except if your application really requires a specific sample rate they should. There is no advantage to upsampling above the RAW sample rate that I know of, because you can't "create" new information that was not present in the source signal.

I see.
So I changed the sampling frequency to 16000, but the problem persists. The audio files have recorded but are of only 1.54 seconds length, however now I can clearly hear my voice in them. But the length problem is still there. I'm going to try and upload a screenshot of my laptop terminal.
I have no idea how "Recorder 23 was defined" turns up there. Then there's a bunch of (what I think are) Javascript messages.

I've also noticed that it closes the connection by itself after showing that the Write Stream is full. I haven't touched anything on the Pi end. The connection got closed somehow, even though there was no error message on the Pi's terminal.

Once the connection closes, it starts outputting this on the terminal on my laptop

At this point, two 1.54s recordings have been processed and can be played back. Two are still processing, by the looks of things.

And at the moment it seems like they aren't going to be processed. There's a request timeout.

The connection closing on the Pi is expected as ODAS simply stops without any message when it can't sink in real time.
There is a mix of request timeout from Google Speech and recording buffer full in the ODAS Studio output. Can you disable the Google Speech Transcription for now it will isolate things and make the output cleaner. Also if you could get a terminal output with proper line termination it would certainly improve readability.
As for recorder up to 34, that is really strange as recorders are created in a loop from 0 to 3.

Yeah the line termination drives me nuts too. Sorry. I didn't know if I can change that. Not sure if it's an electron thing or npm thing (it was a huge pain getting those two things set up on my laptop).
Anyway I disabled the Google Speech Transcription and hey presto, the recorder started working again! I guess there has to be something wrong with the API key or something, I'll go through the link you shared. Thanks!

Glad to hear it’s working. From what I seen your key file seems good, the error

Error 14: UNAVAIBLE

Leads me to think there is a network connectivity problem from the laptop to the google API.
A quick search turned out this thread.
I don’t know how networking works trough the Windows Linux subsystem but Google API seems to dislike proxies so maybe, but this is a far fetched maybe, it comes from there.

authors: GodCed chejmadi


text:
Issue with #notnull passing when it should fail

The below test passes when it shouldn't (different behavior in the previous version).
    @Test
    void testMatchXmlXpath() {
        run(
            "xml myXml = <root><foo>bar</foo><hello><text>hello \"world\"</text></hello><hello><text>hello \"moon\"</text></hello></root>",
                "match myXml //myXml2/root/text == '#notnull'"
        );
    }

From what I could gathered it has to do with the actual Value being interpreted as '#notpresent' and then the '#notnull' applies to that interpreted value.
Hope the screenshot below helps a bit but I'm not too familiar with this part of the framework.

a recent change must be responsible: https://github.com/intuit/karate/commit/10be897b74b7128f0ceb4ebc92e871b50746ed3f
see if you can figure else I will look at it tomorrow

a recent change must be responsible: https://github.com/intuit/karate/commit/10be897b74b7128f0ceb4ebc92e871b50746ed3f
see if you can figure else I will look at it tomorrow

Confirmed that the fix is working after the changes in the develop branch.

Can you confirm what would be your expectation for the following unit test @ptrthomas ?
    @Test
    void testOptional() {
        match("{ number: '1234' }", EQUALS, "{ number: '##regex \\\\d+' }");
        match("{ }", EQUALS, "{ number: '##regex \\\\d+' }");
        match("{ 'foo': 'bar' }", EQUALS, "{ foo: '#string', number: '##regex \\\\d+' }");
        match("{ number: null }", EQUALS, "{ number: '##regex \\\\d+' }");
    }

It fails on the last match (number: null) but I think it should pass as the regex is optional. If my understanding is correct the below logic is missing a few conditions on the regex condition but also on the remaining to handle null values (actual.isNull()).

@joelpramos yes please submit PR

1.0 released

authors: joelpramos ptrthomas

repo: intuit/karate


text:
bibclassify: text needs preprocessing

Originally by jpcorral on 2011-04-05
When the text is extracted from a PDF using pdftotext can appear some ligatures like this: [...]role of medium eﬀects was emphasized.[...]
After the solution is applied: [...]role of medium effects was emphasized.[...]
The text can be normalize in the function normalize_fulltext of bibclassify_text_normalizer.py
Reference: this ticket #317 show the problem bibdocfile module with ligatures

Not really solved into magpie, but indeed no more useful. The general idea remains that both bibclassify and magpie needs text cleanup before hand (something that is done by http://gituhub.com/inspirehep/refextractutils)

authors: invenio-developers kaplun

repo: inveniosoftware/invenio


text:
generalization of second-order operators in SEQP

Originally by valkyrie (@valkyriesavage) on 2011-06-17
currently, refersto and citedby are hard-coded in several places, but in order to facilitate inter-collection searches (for #420) we will need to generalize it to look in a configuration file.

The issue seems to be stalled for 5 years, hence closing. cc @inveniosoftware/triagers

authors: invenio-developers jirikuncar


text:
Re-Add Custom views (calendar / tree) for orders

Re-implement "calendar" view for the various order types, in PUI

I think fullcalender would be optimal for the calender part. Here is a sample with typescript: https://github.com/fullcalendar/fullcalendar-examples/blob/main/react-typescript/src/DemoApp.tsx

@SchrodingersGat is this still valid / open?

Yes still planning to implement this (or happy for someone else to)

authors: SchrodingersGat matmair

repo: inventree/InvenTree


text:
Add welcome messages

This PR adds an action that will welcome new contributors/issue creators on their first PR / issue and provide links to our contributing / install docs to bring them up to speed faster.

Part of #2493


text:
Incorrect documentation for required_fields option

In TUTORIAL.md, required_fields is listed as an option, nested below the options key. I couldn't get this to work so I looked into the code and it seems required_fields should actually be at the same level as options. Is the documentation erroneous or should the code be changed to match what the docs say?

Which section are you referring to?
It is listed as "optional'.
In the section of simple invoice template there is an example.
issuer: Microsoft Regional Sales Corporation
keywords:

Microsoft
M9-0002526-N
exclude_keywords:
Already\s+paid
Do not pay
fields:
amount: GrandTotal(\d+.\d+)HKD
date: DocumentDate:(\d{1,2}/\d{1,2}/\d{4})
invoice_number: InvoiceNo.:(\w+)
options:
remove_whitespace: true
currency: HKD

authors: bosd flowrean