text:
[dv,dvsim,tooling] Questa Support

Description
This is a bucket-issue for improving Questa support in OpenTitan.

I want to use this issue to drive support forwards by gathering user feedback into a working branch of fixes, which can later be merged all at once.
If you want to help, please see the steps you can take below!

Motive
I would like to get Questa support into a better place across the project.
Questa is a simulator that many people have access to, and a part of maximizing value of the OpenTitan project is to keep the barrier to entry as low as possible for anyone who is interested.
As lowRISC currently does not have access to questa licenses, nor is anyone regularly developing on OpenTitan with it or regressing the DV suite using it, the support there has always been limited and liable to break without notice.
I hope one day we can test our codebase against more simulators and catch breaking changes before they are merged, but up to now it has been a best-effort approach.
Ctx
Questa support was added initially to our DV test tool 'dvsim' in #10574 - [hw/dv] Feature/questa dv, though I am not sure to what extent it was functional at the time. (i.e. which testbenches could be compiled/simulated, and which were incompatible).
Our signoff simulator for blocks/tops is primarily VCS (for example. see the earlgrey top-level testbench config file chip_sim_cfg.hjson#L15), although some blocks now use Xcelium for signoff as well.
'dvsim' is extensible, and the support added above allows for invoking simulations using the questa CLI when you use --tool=questa.
Fundamentally, dvsim should be able to assemble a valid CLI invocation for any tool it supports, with any number of arguments that are specific to the quirks of any program.
Some parts of this process are quite generic between tools. For example, we use fusesoc .core files to package blocks and components, and dvsim will invoke fusesoc to generate output collateral such as a specific filelist for the simulation/testbench of interest.
The tool-specific file /hw/dv/tools/dvsim/questa.hjson is then responsible for tying-up the generic parts of the flow with the implementation details of a tool.
Existing Questa Tickets
I did a little search of previous issues and pull requests related to questa support. I've pulled out the ones I think might be useful as a reference here...
Issues
https://github.com/lowRISC/opentitan/issues?q=is%3Aissue+questa+

Ticket
Desc.

OPEN

#24174 - [questa] Pls update questa.hjson file
Questa support request

#23293 - [siemens] updated questa.hjson
General Questa questions

#23268 - [documentation] generating scripts used to run Jasper Gold
Porting Jasper formal scripts to QuestaFormal

#22755 - gpio_tl_intg_err test failing on Questa
Test passes on Xcelium, but not in Questa

#22243 - [dvsim] Various questions
Questa/CentOS7 support questions, also dvsim/bazel questions

#22217 - [dvgen] generating testbench w/in Ubuntu container but simulating in CentOS
Questa/CentOS7 support questions

#18039 - [dvsim] Riviera/Questa support is broken - around otbn_memutil
Requesting Questa/Riviera support

CLOSED

#22162 - [ci] No module named 'google_verible_verilog_syntax_py (2024)
General questa/bazel support questions

#21518 - [dvsim] Questa uart_smoke failing at tl_agent_pkg load
Questa SV / LRM incompatabilities, not sure a fix was merged but it may have been elsewhere.

#16994 - [aes] aes_core simulation using ModelSim
Request for questa/modelsim support

#9514  - Edalize and dvsim
Discusses some initial SV-incompatabilites before questa.hjson was merged in #10574

#4541  - [DV] missing has_edn check in cip_base_env.sv?
SV issue found while adding questa support. Fixed.

#4529  - Sim: Illegal %s format specififer for class object
Questa incompatabilites -> fixed by adding '-suppress vsim-8323' in #10574

#4528  - Sim: Dangling virtual interface - Questa support
Questa SV incompatabilities -> fixed by adding '-permit_unmatched_virtual_intf' in #10574

#4427  - Sim: tlul_assert: SVA errors - QSTA UART Smoke test
Questa SV LRM incompatabilites in tl_host_driver.sv - closed due to lack of feedback

#4377  - SIM: Questa support: FATAL error from RAL
OpenTitan does not support pre-compiled UVM -> problem found when trying to add Questa build support

#4355  - Sim: Data type mismatch issue
Questa SV Support issue - closed as wontfix, pending vendor feedback on questa LRM interpretation

#4340  - dvsim: QSTA support - sim_tops needs to be moved to run_opts
Some initial design questions about adding dvsim questa support

#4334  - Renaming dir sim-vcs inside fusesoc.hjson
More dvsim+fusesoc oddities than questa

#4332  - dvsim: Questa support - prim_clock_gating.v - is this missing in file-list?
Adding initial questa.hjson

#4153  - EalrGrey Questa support?
Questa support request

PRs
https://github.com/lowRISC/opentitan/pulls?q=is%3Apr+questa+

Ticket
Desc.

OPEN

CLOSED

#10574 - [hw/dv] Feature/questa dv
Initial questa support in dvsim (questa.hjson) + some docs.

#10365 - [hw/dv] Changed set_response_queue to be umbounded due to queue overflow issues in Questa
Fix for #9514 ( a questa issue )

#4435  - [dv] Fix Questa warning and remove unused var
Fixes for #4377/#4398

#4366  - [dv] Move sim_tops to {tool}.hjson
Preparing for initial questa support

#240   - [alert_handler] Add alert_handler RTL implementation

Fixing outstanding issues
To get our DV flows working with Questa more broadly, the bulk of the work will be in SV / LRM differences in how tools interpret our codebase. A familiar hurdle for sure!
This process will probably raise a number of issues in dv base classes to begin with, and then follow onto testbench code specific to each block.
Probably the best approach will be to work block-by-block through the individual testbenches, first fixing any build issues, followed by any runtime differences.
The following table is a good starting list of some block level testbenches for non-parameterized blocks, and suggested invocations to test them.
(I've just worked down the list of IP in hw/ip to make a start. This list can become a more comprehensive checklist of tested-blocks in the future.)

block
simulation command

adc_ctrl
./util/dvsim/dvsim.py hw/ip/adc_ctrl/dv/adc_ctrl_sim_cfg.hjson -i adc_ctrl_smoke --tool questa

aes
./util/dvsim/dvsim.py hw/ip/aes/dv/aes_masked_sim_cfg.hjson -i aes_smoke --tool questa

aon_timer
./util/dvsim/dvsim.py hw/ip/aon_timer/dv/aon_timer_sim_cfg.hjson -i aon_timer_smoke --tool questa

csrng
./util/dvsim/dvsim.py hw/ip/csrng/dv/csrng_sim_cfg.hjson -i aon_timer_smoke --tool questa

edn
./util/dvsim/dvsim.py hw/ip/edn/dv/edn_sim_cfg.hjson -i edn_smoke --tool questa

entropy_src
./util/dvsim/dvsim.py hw/ip/entropy_src/dv/entropy_src_sim_cfg.hjson -i entropy_src_smoke --tool questa

gpio
./util/dvsim/dvsim.py hw/ip/gpio/dv/gpio_sim_cfg.hjson -i gpio_smoke --tool questa

hmac
./util/dvsim/dvsim.py hw/ip/hmac/dv/hmac_sim_cfg.hjson -i hmac_smoke --tool questa

i2c
./util/dvsim/dvsim.py hw/ip/i2c/dv/i2c_sim_cfg.hjson -i i2c_smoke --tool questa

keymgr
./util/dvsim/dvsim.py hw/ip/keymgr/dv/keymgr_sim_cfg.hjson -i keymgr_smoke --tool questa

kmac
./util/dvsim/dvsim.py hw/ip/kmac/dv/kmac_sim_cfg.hjson -i kmac_smoke --tool questa

The top-level testbench would eventually be desirable to fix, though I suspect it will be the most work to get there.

top
simulation command

earlgrey
./util/dvsim/dvsim.py hw/top_earlgrey/dv/chip_sim_cfg.hjson -i chip_sw_rv_timer_irq --tool questa

After running smoketests for each block, probably the next thing would be to run 'all_once' regressions in each block.
E.g. ./util/dvsim/dvsim.py hw/<ip>/dv/<ip>_sim_cfg.hjson -i all_once --tool questa
This should exercise all of the stimulus vseq's listed in the sim_cfg.hjson configuration file.
Stimulus sequences are located in the following directory for each block-level testbench : hw/<ip>/dv/env/seq_lib/.
Our existing convention has been to avoid #ifdefs unless absolutely necessary, so work in this area will need to prioritize finding common language constructs the tools agree on.
There are a very small number of #ifdef XCELIUM or #ifdef VCS switches in the codebase, and adding more of these should be considered a last-resort.
However, I think absorbing changes that use #ifdefs in the short-term, tracked in issues with pending vendor support feedback, may be workable.
For the working branch described below, short-term solutions using #ifdefs will be acceptable.
How you can help
To start with, I've created a draft PR to accumulate a working set of changes to fix bugs as they are found.
As bugs and suggested fixes come in, we can discuss them in this issue, and then propagate them over to that PR.
Steps:

Setup the opentitan repository & dependencies by following the instructions in steps 1-4 here : https://opentitan.org/book/doc/getting_started/index.html

Test the above setup by using bazel to run a verilator simulation : bazel test --test_output=streamed //sw/device/tests:uart_smoketest_sim_verilator
Ensure questa is on your path, and that QUESTA_HOME is set appropriately.

Checkout the changes from the working PR : https://github.com/lowRISC/opentitan/pull/24331
Run one of the simulation commands above, and report back the failure modes in comments on this issue with code/log snippets.

We can discuss the failure modes to propose fixes or workarounds.
Fixes will be propagated to the working PR to unblock further work, with commits/comments refering back to the discussion here.
If an immediate fix is not obvious, or requires some external feedback, then we can add temporary workarounds to the PR to unblock further work while a better fix is pending.

This will probably take a bit of time to get sorted out, as the feedback loop is inherently manual, and I don't have too many free cycles to put into this.
However, I hope that centralizing the discussions here will keep the process moving forwards, and eventually we can get up to a good parity with the other tools.
Thanks!

@lmg260a
(Moving my reply to your comment from 23293 over here...)

Hi,
That's all great news!
What's the best approach for submitting code changes? Create a branch and then propose a merge, email files, something else?
<...>
So how should I raise those? As separate tickets? I've found that lots of small tickets are easier to track.
Also: should I use "[siemens]" as the ticket-group in the subject line, to make things easier?

I'd like to try and keep things centralized to this issue to start with. If there are common design patterns in the codebase causing problems, we can always spin discussions about them out into individual issues/tickets if it becomes too noisy within this thread. If we do open new issues, probably tagging with [siemens] or [questa] and linking back to this one will be best.
I think the best way to submit/suggest code changes would be to post a link to a branch on your github, or to paste the diff into this issue as a comment. I want to integrate all the changes into the working PR #24331 so I can regress it against all the tools I do have access to in one go before merging, so raising individual pull requests for changes is probably going to be quite a lot of noise at this point.

I've also found things which might be exactly what's wanted, but if not clearly documented it would be really easy for people to break things. For example, both a parent and child classes have constraints named the same. Per LRM, the child constraint overwrites the constraint in the parent. My problems are (1) I can't tell if that's deliberate or a bug. It should be clearly documented if it IS deliberate. (2) it may be that whatever simulator you're using doesn't interpret the LRM the same way I do, and it could be that (2.a) I'm wrong or (2.b) the LRM is subtlely vague - and so we're both wrong (or at least, neither of us can claim we're right).

Thanks for bringing this up. In the OpenTitan codebase, it's a common pattern to redefine a constraint in a child class intentionally to overwrite the parent class constraint. As you say, that is what the LRM specifies the behaviour should be. Based on your comment (1), is your previous experience that this should always be documented at every instance? Or is a comment about this something that should be incorporated into a project level style-guide?
I'm interested in any comments about style generally, and if there is any documentation holes we can fill to clarify things. I don't think I have heard the comment about always commenting overridden class constraints before, but everyone has different industry experiences of course. Any docs we can provide to get everyone on the same page as quickly as possible are valuable!

So one thing I'm debugging is that one of the clock period is being set to zero, then being used to do a divide, which of course results in a divide-by-zero. So there's a race-condition someplace going on.

Can you checkout the working PR #24331, and share the dvsim command you are using to generate the test? Also could you share the variable/error, plus which assignment / randomization you suspect is incorrectly generating 0? I can run it locally and share some logs / values, or suggest where things might be going wrong.
Thanks!

@lmg260a
(Moving my reply to your comment here https://github.com/lowRISC/opentitan/pull/24331#issuecomment-2303289278 over to the discussion issue...)
The example dvsim command from your comment (util/dvsim/dvsim.py hw/top_earlgrey/dv/chip_sim_cfg.hjson -i chip_sw_uart_tx_rx -n --tool questa) won't run anything because -n is a dry-run switch, which just logs the build commands and exits.
Could you try running without that switch, and let me know what is broken? Running via dvsim is the only supported flow, as it populates all of the defines etc. automatically, and that way we can know everyone is on the same page.
For example, if I checkout #24331 and run dvsim I get the following:
// I don't have questa, so set a dummy variable...
$ export QUESTA_HOME=/home/harry/questa/dummy
$ util/dvsim/dvsim.py hw/top_earlgrey/dv/chip_sim_cfg.hjson -i chip_sw_uart_tx_rx --tool questa
<EVERYTHING FAILS>

// The relevant logfile is at `<opentitan>/scratch/<branch>/chip_earlgrey_asic-sim-questa/default/build.log`, which shows all the commands which were run. Mine looks normal until...
[make]: build
cd /home/harry/projects/opentitan/scratch/dvsim_questa_fixes/chip_earlgrey_asic-sim-questa/default/sim-vcs && /home/harry/dummy/questasim/linux_x86_64/qrun -optimize +define+TOP_LEVEL_DV +define+UVM +define+UVM_NO_DEPRECATED +define+UVM_REGEX_NO_DPI +define+UVM_REG_ADDR_WIDTH=32 +define+UVM_REG_DATA_WIDTH=64 +define+UVM_REG_BYTENABLE_WIDTH=8 +define+SIMULATION +define+DUT_HIER=tb.dut -timescale 1ns/1ps -outdir /home/harry/projects/opentitan/scratch/dvsim_questa_fixes/chip_earlgrey_asic-sim-questa/default/qrun.out -uvm -uvmhome /home/harry/dummy/questasim/verilog_src/uvm-1.2 -mfcu -f /home/harry/projects/opentitan/scratch/dvsim_questa_fixes/chip_earlgrey_asic-sim-questa/default/sim-vcs/lowrisc_dv_chip_sim_0.1.scr -top clkmgr_bind -top pwrmgr_bind -top rstmgr_bind -top sec_cm_prim_onehot_check_bind -top sec_cm_prim_sparse_fsm_flop_bind -top spi_host_bind -top top_earlgrey_error_injection_ifs_bind -top top_earlgrey_bind -top xbar_main_bind -top xbar_peri_bind -top tb -voptargs="+acc=nr"
bash: line 1: /home/harry/dummy/questasim/linux_x86_64/qrun: No such file or directory

Printed with newlines, the invocation was:
cd /home/harry/projects/opentitan/scratch/dvsim_questa_fixes/chip_earlgrey_asic-sim-questa/default/sim-vcs &&
/home/harry/dummy/questasim/linux_x86_64/qrun
-optimize
+define+TOP_LEVEL_DV
+define+UVM
+define+UVM_NO_DEPRECATED
+define+UVM_REGEX_NO_DPI
+define+UVM_REG_ADDR_WIDTH=32
+define+UVM_REG_DATA_WIDTH=64
+define+UVM_REG_BYTENABLE_WIDTH=8
+define+SIMULATION
+define+DUT_HIER=tb.dut
-timescale 1ns/1ps
-outdir /home/harry/projects/opentitan/scratch/dvsim_questa_fixes/chip_earlgrey_asic-sim-questa/default/qrun.out
-uvm
-uvmhome /home/harry/dummy/questasim/verilog_src/uvm-1.2
-mfcu
-f /home/harry/projects/opentitan/scratch/dvsim_questa_fixes/chip_earlgrey_asic-sim-questa/default/sim-vcs/lowrisc_dv_chip_sim_0.1.scr
-top clkmgr_bind
-top pwrmgr_bind
-top rstmgr_bind
-top sec_cm_prim_onehot_check_bind
-top sec_cm_prim_sparse_fsm_flop_bind
-top spi_host_bind
-top top_earlgrey_error_injection_ifs_bind
-top top_earlgrey_bind
-top xbar_main_bind
-top xbar_peri_bind
-top tb
-voptargs="+acc=nr"

This CLI invocation looks okay to me at a pass. Do you see the same failure mode as you described in the other issue when invoking this way, or something else?

Some best practices that would help:

4-state logic was shown to generate both optimistic (false-pass) and pessimistic (false-fail) results in simulation.  This was back in the 90s.  So it's much better to use 2-state, plus static tools (CDC/RDC) and property checking.  Most simulators have the ability to initialize all flops to random 0/1 values at time0 - so you can use that if you are unsure about Xs.
Some defensive programming I've learned helps over the years:
A) use explicit logic to convert from 4-state to an enum with 4 states, or two 2-state bits; bugs are common when using casting b/c often you want to know that a signal is 'Z' or 'X', and the implicit cast just returns 0.
B) use unit-testing (svunit) to verify code before it gets committed.  One company reported that when they switched to unit-testing in 12 months they saw a 97% drop in bugs reported in the field.
C) Ideally, use test-driven development: the main thing here is that it'll help you spot architectural or interface problems that are just going to be very expensive to verify, and really expensive to debug/maintain.  Before you do any coding.
D) Best practice in Agile is each developer spends 20% of their time just refactoring code: a couple of hours spent rewriting an interface can save days in verifying a really awkward interface design.
E) Put all generated files outside the repo clone - that way, "git status" should only show the files that should be committed.
F) I've seen a couple of schools of thought on big-picture repos: one is that you have one huge repo (but then any change means you have to rerun the full regression), or one for RTL, another for sim testbench, another for static/formal scripts.  There's arguments for both - I just bring it up in case you find the idea useful. The other is that you have one really small repo per IP (so a 'opentitan_uart' repo, etc.).  This way you have all the files involved in an IP in one location, and it greatly simplifies reuse and enhancement.
G) For debugging purposes: I've found it really useful to have the verification environment emit a script (including all environment variable settings) that runs the work.  This way, if anything goes wrong, you just take that script and attach it to the ticket so it can 100% be reproduced.  Note: it really helps if the paths in the script are set using environment variable-names instead of being replaced with their values.  I.e. the path to a file should be "$FOO/...", not "/a/b/c/...".  With $FOO being set earlier in the script.
H) The saying is: 10% documentation, 10% coding, and 80% verification.  So really project success is all about rewriting the code to reduce verification cost.

I'd be happy to do any sort of online class or talk that might help your team on this - I think you've got an amazing thing here, and I'm just trying to do what I can to help.  The problem with greatness is you can't ever stop improving, or else you just eventually wind up back at "good".

Hi Harry,
Some things to change:
-uvm
-uvmhome /home/harry/dummy/questasim/verilog_src/uvm-1.2
to:
-uvm
-uvmhome uvm-1.2
This path has "sim-vcs" instead of "sim-questa" (happens multiple places)
/home/harry/projects/opentitan/scratch/dvsim_questa_fixes/chip_earlgrey_asic-sim-questa/default/sim-vcs
Questa is stricter to the LRM than VCS, so a series of switches are required to tell Questa to relax rules to be VCS-compatible.  This is one of them:
-svext=hierbinsof
This shouldn't be required; in any case, +acc is going obsolete in 2025 to be replaced by other switches.
-voptargs="+acc=nr"
Please add
-permit_unmatched_virtual_intf
to all simulation cases.
General questions:

why are all those "-top" required?  For Questa, there's only one: "-top tb", since one only simulates one module in one sim.
why the full path to 'qrun'?   /home/harry/dummy/questasim/linux_x86_64/qrun
using "-F" makes things more reusable, because all files are searched relative to where the .f file is located.  "-f" means all paths are searched relative to where the qrun/vcs is being invoked.  so a .f file has to have hardcoded paths instead of relative paths.

Could you please include what you get for running the sim in
a) regression mode (don't dump any waveforms)
b) batch (dump waveforms for later analysis)
c) live (run interactive)
d) post-processing (view results generated in batch mode)
Those will need changes as well.

repo: lowRISC/opentitan


text:
[usbdev][usbuart] Change USB PHY interface to pass properties back to IP?

The USB PHY interface we are using today relies on the USB IP to be configured correctly for the PHY capabilities, which ultimately involves the device driver understanding the PHY. This isn't so convenient for portable software.
All signal names below are as seen at IP interface
We should consider reversing this and adding two signals from the PHY to the IP and removing one:

cio_usbphy_tx_d_se0_i: set if PHY transmit side uses {cio_d_o, cio_d_en_o, cio_se0_o} and clear if it uses {cio_dp_o, cio_dp_en_o, cio_dn_o, cio_dn_en_o}.
cio_usbphy_has_diff_rx_i: set if PHY has a differential receiver that drives {cio_d_i, cio_dp_i, cio_dn_i} and clear if it is only single ended and only drived {cio_dp_i, cio_dn_i}.
Remove cio_tx_mode_se_o / cio_tx_mode_se_en_o which currently propagates the mode being used by the IP for TX.

The phy_config register bits tx_differential_mode and rx_differential_mode would become status bits reflecting the values from the PHY.  A bit for debug would be added to force RX to only use the single ended signals (it is nonsense to force use of differential signal if the phy does not provide it).
This would be of even more use in the usbuart case to make it possible to use the IP without software being aware that it is any different from a regular uart.

@vogelpi @stefanlippuner would be interested in your thoughts on this. I am happy to make the changes if you agree.

@vogelpi @stefanlippuner would be interested in your thoughts on this. I am happy to make the changes if you agree.

I think it's a very good idea to add the information about the single-ended / differential capability in hardware and expose it to the software. I would probably leave the final decision up to the software for both TX and RX (in that case it would not require changes to the current mode registers). For us it would be important to have full control for both directions separately from software (e.g. we have the option to bypass the USB PHY via the iomux).

I think it's a very good idea to add the information about the single-ended / differential capability in hardware and expose it to the software. I would probably leave the final decision up to the software for both TX and RX (in that case it would not require changes to the current mode registers). For us it would be important to have full control for both directions separately from software (e.g. we have the option to bypass the USB PHY via the iomux).

This sounds good to me. It would be good to consistently name those signals, e.g., cio_usbphy_rx/tx_diff_mode_i (ideally aligned with the bits in the phy config register).
As noted by Stefan, it may be useful to give software full control over this. I suggest to add a bit in the phy config register to hand over control to software. By default, hardware would be in control and the rx/tx_differential_mode bits would be read-only for software.
About removing cio_tx_mode_se_o/cio_tx_mode_se_en_o (actually related to #2704): This sounds good to me as well as we currently don't use this in OpenTitan. However, it has originally been added by Stefan. Do you still need that?

This sounds good to me. It would be good to consistently name those signals, e.g., cio_usbphy_rx/tx_diff_mode_i (ideally aligned with the bits in the phy config register).
As noted by Stefan, it may be useful to give software full control over this. I suggest to add a bit in the phy config register to hand over control to software. By default, hardware would be in control and the rx/tx_differential_mode bits would be read-only for software.
About removing cio_tx_mode_se_o/cio_tx_mode_se_en_o (actually related to #2704): This sounds good to me as well as we currently don't use this in OpenTitan. However, it has originally been added by Stefan. Do you still need that?

@a-will can you PTAL and relabel as appropriate?

@a-will what do you think? i don't think we need to do this now, but it seems nice to allow transceivers advertise what it can do.

Given that our ASIC and FPGA implementations use exactly the same configuration (and that it's relatively trivial to map any static connection to that configuration), I'm thinking we don't need this. tx_mode_se is essentially inert, since both TX interfaces are always available now, and use_diff_rcvr = 1 can apply to both.
In addition, with the current connectivity at the IP level and templates at the top level, these CSRs may no longer be needed.

For us, I think it's nice to have the single-ended mode as a fall-back and for validation. There the tx_mode_se_o signal is useful to indicate, which TX mode is being used. Signaling the hardware capabilities might be nice, but does not seem to be required, as it is fixed for a hardware configuration.

CC @alees24
I am not sure we need this for Earlgrey PROD due to the reasons brought up by @a-will above.
Whether or not a diff receiver is present seems to be an attribute of the integration and not only of the USBDEV itself, hence it might be better to have the software modulate it's behavior based on platform information?
E.g., in OpenTitan, this could be inferred by reading the product ID / version from the life cycle controller.

source: gharchive/issue

authors: a-will mdhayter msfschaffner stefanlippuner tjaychen vogelpi

license_type: permissive

license_source: github-api


text:
[CI] Expose Vivado build logs as artifacts

When Vivado fails, the info on stdout is not useful. It would be good to expose the more detailed log files as artifacts for people to review. The fact this is probably reliant on having access to the Azure org makes it a bit inconvenient, but there may be other options (e.g. scp to bubble.servers.lowrisc.org/ci_artifacts/prNNN/).
From @imphil on zulip:

Philipp Wagner: yes, Vivado doesn't output synthesis results on stdout (it does output implementation results ...) You need to look into the *.runs/synth_1/runme.log file. This and other *.rpt files from the impl_1 and synth_1 directories should be available as artifacts, otherwise debugging is almost impossible

Philipp Wagner: This is what we do for optimsoc: https://github.com/optimsoc/optimsoc-autobuild/blob/master/.gitlab-ci.yml#L130-L133

@imphil has this been addressed by now?

Yes

authors: asb imphil msfschaffner


text:
[dv/sysrst] Sysrst z3 test

Only the last commit is relevant.
This PR is based on comment: https://github.com/lowRISC/opentitan/pull/17247/files#r1102650481
@crteja please let me know if this meets your suggestion?

@cindychip , Thanks for the change, this should cover the timing check for z3_wakeup_o. Looks good to me.

Looks good to me

source: gharchive/pull-request

authors: cindychip crteja


text:
[opentitantool] Add JTAG support to HyperDebug backend

HyperDebug now supports the CMSIS-DAP protocol which OpenOCD already has a transport driver for.  With this change, the HyperDebug backend driver is able to start up OpenOCD with appropriate parameters to have it connect to the same HyperDebug.

By the way.  I have just verified that with the HyperDebug firmware included in this patch, I am able to use OpenOCD and gdb to debug (single step) a program running on the CW310 board, all using HyperDebug.  That is, HyperDebug can replace the Olimex JTAG dongle, and we should be able to do tests involving JTAG with the HyperDebug bitstream. This should be good preparation for running tests on the future ASIC shield, driven by HyperDebug.

We have some jtag tests in CI. When this is merge, it would be good to add a hyperdebug/jtag tests as well (driven by opentitanlib). Probably it's just a matter of copying the existing one and switching the interface to hyperdebug.

Ok, I have changed this PR such that the path to openocd is still passed via a field in JtagParams, for all backends.  The path to the Olimex adapter config is now passed via a field in ChipWhispererOpts, as it currently does not apply to other backends.
The other OpenOCD config files, the ones about OpenTitan JTAG TAPs, and the one for the CMSIS-DAP compatible adapters (HyperDebug), are compiled into opentitanlib, and passed as strings on the command line to openocd via the -c switch.  (The combined command line probably exceeds a kilobyte.)
The new HyperDebug firmware included in this PR seems to trigger a bug in the I2C backend transport.  I will look at that Monday, but I feel that the JTAG changes in this CL is fully ready for review.

Thanks @jesultra I think it's much better this way. If the command-line becomes a problem, we might look into creating a temporary file to pass it to OpenOCD.

authors: jesultra pamaury


text:
[opentitantool] improve user experience

When 'opentitiantool image manifest update ...' fails due to any input or output file name mistyped, or any of the input files does not exist, the error message is the same:
Error: No such file or directory (os error 2)
Which does not help much the operator. This patch adds some code which will print the name of the file causing the error.
Tested by invoking 'image manifest update' with incorrectly specified various input files. In case of the wrong SPX key file name the file name is printed twice, since the code is trying to read the file first trying to retrieve a private key, and then if unsuccessful - one more time trying to retrieve the public key.
This double print could be eliminated by proper return error processing, but is left as is at this time.

A mostly cosmetic change making it easier to figure out the reason for opentitnatool invocation failures.

I did not add anyhow to the sphincs library, seems more trouble than it'w worth


text:
[ibex,fpga] Apply Ibex FPGA patches

This PR applies two patches on top of Ibex that fix the following two FPGA related issues:

Use WordZeroVal instead of 0 for initializing the FPGA register file
Use async reset when not using a DSP for the Ibex FPGA counter

A patch is needed because:

Without these fixes the SecureIbex parameter cannot be used on CW340 - this is needed for some SiVal tests
These patches only touch the FPGA implementation. If we would vendor in the changes, we would get additional RTL changes that also affect the ASIC

I believe we still don't have bitstream caching enabled for earlgrey_1.0.0. @a-will can provide details on how to enable this.

Bitstreams are being cached, it looks like. The only thing to avoid would be using the latest hacky reference. bitstreams_workspace.py is currently hard-coded to look at master/latest.txt in the bucket for that. In general, though, the default value of "HEAD" should be used from a git repo.

CHANGE AUTHORIZED: hw/vendor/lowrisc_ibex/rtl/ibex_counter.sv blocked
CHANGE AUTHORIZED: hw/vendor/lowrisc_ibex/rtl/ibex_register_file_fpga.sv

CHANGE AUTHORIZED: hw/vendor/lowrisc_ibex/rtl/ibex_counter.sv blocked
CHANGE AUTHORIZED: hw/vendor/lowrisc_ibex/rtl/ibex_register_file_fpga.sv

CHANGE AUTHORIZED: hw/vendor/lowrisc_ibex/rtl/ibex_counter.sv
CHANGE AUTHORIZED: hw/vendor/lowrisc_ibex/rtl/ibex_register_file_fpga.sv
This only changes FPGA logic; ASIC logic isn't affected.

authors: a-will andreaskurth nasahlpa vogelpi


text:
[otbn] Properly parse optional arguments in instruction YAML

The interesting patch is the second in the series. This teaches insn_yaml.py to actually understand the square brackets we have in our instruction syntax.
Note that bn.mulqacc.so has changed syntax: the destination register is now written w0.L or w0.U, rather than just w0L or w0U as previously. This makes parsing much easier: @GregAC, can you check this looks sane?

Crikey. CI took 1075m and then failed with something to do with the Verilator build. Closing/Re-opening and hoping for better luck this time :-)

This went in as part of https://github.com/lowRISC/opentitan/pull/2862, didn't it?

Ah, yes! I was really confused when I looked at this patch again and didn't understand why the otbn-as patch didn't depend on it. Duh...

authors: imphil rswarbrick


text:
[otbn] Fix RIG model's _inc_gpr logic

This wasn't quite right, because we call _inc_gpr after
_generic_update_for_insn. As usual, that's fine for everything except
x1. For x1, the get_reg() call inside _inc_gpr was causing a double
pop.

More and more reasons to get rid of that call stack ...

Yeah, and generating loops is going to be rather painful too!

More and more reasons to get rid of that call stack ...

Yeah, and generating loops is going to be rather painful too!


text:
[analysis] Add trace preprocessing

This PR adds a trace preprocessing script allowing the user to (a) filter traces using a max. amount of tolerable deviation from the trace average and (b) align the traces using a reference window. Currently, the script is tested with the new trace database format. Moreover, I've decided to implement this functionality outside of the TVLA script as this script is already really overblown.
Input trace set:

After filtering:

After aligning:

The aligning (which uses functionality from ChipWhisperer) heavily depends on the selected alignment window.

Done. Thanks for the review @vrozic!

repo: lowRISC/ot-sca

license_source: bigquery


text:
Compilation Error - leptos_use

Hello,
Could I please get some commentary regarding the following error?

same to me, it seems to be incompatible with nightly

Yes, this is currently a limitation tracked in https://github.com/lpotthast/leptonic/issues/17. Looking into it.

authors: MinaMatta98 lighthx lpotthast

repo: lpotthast/leptonic


text:
Laura HTML file

Modifying HTML file to better fit my portfolio's description (will add more in depth information later)

@Tetheta Starting review

authors: LukeGMTravis lqueipom

repo: lqueipom/step


text:
Allow Tutors to provide General Feedback to Text Exercises

Checklist

[X] I run yarn run webpack:build:main: the project builds without errors.
[X] I run yarn lint: the project builds without code style warnings.
[x] I tested the changes and all related features on the test server https://artemistest.ase.in.tum.de.

Motivation and Context
#369
Description
Allow Tutors to provide general feedback to text submissions at the bottom of the assessment page.
Feedback is displayed to students underneath their submission (with inline feedback).
Steps for Testing

Navigate to text exercise and open a submission for assessment
add feedback to text blocks to distribute credits (as before)
add a general feedback text at the very bottom
Log in as the student and view the submission with feedback
find the feedback at the bottom of the page in the blue box

Screenshots

@jpbernius Please review my changes as well

Closing this in favor of PR #432, including these commits, but also adding the same functionality for modeling exercises.

authors: SiggZ jpbernius

license: mit


text:
General: Fix course card style and improve course overview top bar

Checklist
General

[ ] I tested all changes and their related features with all corresponding user types on a test server.
[x] I chose a title conforming to the naming conventions for pull requests.

Client

[x] I followed the coding and design guidelines.
[x] I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.
[x] I documented the TypeScript code using JSDoc style.
[x] I added multiple screenshots/screencasts of my UI changes.

Motivation and Context

We've seen two issues:

Overflow of "due date" indicator on course cards on small screens

Bad responsiveness in course overview top bar

Additionally, the refresh button was not visible anymore if the Exam tab is enabled.

Description

Easy fix by applying no-wrap styles.

For some reason all the controls of the inner components (like the exercise list) are currently rendered by itself, and then moved up into the top bar using absolute pixel offsets. This PR renders them in the top bar natively.

https://user-images.githubusercontent.com/23171488/145895599-6f0a9f75-0a5f-4d87-bb5d-a4b30fa88845.mov
Steps for Testing

1 Course with at least one Exercise that has a due date in the near future

Go to "Course Overview" and check that the Due Date indicator in the bottom right of the course card is rendered correctly on iPhone screen sizes (does not break into 4 lines or sth)
Go into the course. Verify that the Filter and Sorting buttons in the top right of the list are available and functional.
Scale your view port horizontally (make it smaller) and verify that responsiveness works (like in the video above)
Switch to "Lectures". Verify that the buttons in the bar changes to the "Sort lectures" dropdown and that it works correctly.
Switch to "Statistics". Verify that the "Hide optional" checkbox is rendered next to the Refresh button and that it still works.
Change to "Discussions". Verify that no other controls than the Refresh button are rendered in the top bar.

Review Progress

Code Review

[ ] Review 1
[ ] Review 2

Manual Tests

[ ] Test 1
[ ] Test 2

@willergeorg I've addressed your suggestions in dd8a269. I can't find any unused imports, though 🤔

license: MIT


text:
Taxonomy v3.9 might not be working properly with Kraken

Hi,
I'm testing the Kalamari DB with Kraken2 and I think the taxonomy v3.9 is not working as intended (at least not working as described in the Database formatting instructions):

Following the Kraken2 instructions, Kraken creates the database only with 2 sequences

Candidatus Korarchaeum cryptofilum	CP000968	374847
Halobacterium salinarum	AM774415	478009

(from the chromosomes file)

Using taxonomy taxonomy v3.5, the database is then created with 284 sequences.
If I concatenate both taxonomy versions, one entry is added to the database: Betacoronavirus coronavirus	MT233526	2697049.
If we ignore the 19 entries found in chromosomes file that don't have accession numbers (and therefore not downloaded), there are still 3 entries that are not included in Kraken's database both with taxonomy v3.5 or taxonomy v3.5+v3.9

Campylobacter hyointestinalis hyointestinalis	CP015575	91352
Campylobacter hyointestinalis lawsonii	CP015575	91353
Escherichia albertii	CP024282	208961

The Escherichia entry (the taxonomy id) doesn't have a matching entry in taxonomy files. I searched for that taxonomy id and it was Influenza virus. Escherichia albertii seems to be 208962. I tried to replace 208961 in the sequence header by 208962, but Kraken is still ignoring this entry (I might be missing something or doing something wrong).
For the Campylobacter entries, different taxonomy ids use the same accession number (which might forces Kraken to not include those into the database).

My idea here is to let you know the problems I found, and try to understand how does everything work, so I could contribute and try to increase/improve the Kalamari DB.
Thanks a lot,
Best regards,
Miguel

Thanks for letting me know!  I will investigate when possible.  Thank you for being so in depth and giving me easy options to follow up on :)

Trying to address this in v3.9.2 with some more rigorous scripting, in branch custom-taxdump

https://github.com/lskatz/Kalamari/runs/3147121557?check_suite_focus=true
It seems to have fixed it but I'm out of time for this work week.  Let me know if you get a chance to try it?  You were totally right about the Escherichia species taxid and so thank you for finding that.

I think it has been fixed and I am now monitoring it with Github Actions.  Thank you for pointing it out!

authors: lskatz miguelpmachado

repo: lskatz/Kalamari


text:
all classes inside Dpdt.Injector.Src should be internal

also we need to use an attribute InternalsVisibleTo to make these classes visibile to Dpdt.Generator(.Core) and extension.
it is unclear now hot it will affect unit tests.

as I know now it's a very VERY tricky task! :)

repo: lsoft/DpdtInject


text:
Allow configuring containers with no network

Allow configuring individual LSPs to have no network access. By default,
LPSs will have no network access, but those that require it can have a
network="bridge" attribute (see gopls as an example).
This setting can also be overridden, so users with custom needs can
override the defaults, and it can also be specified for custom LSPs.

I've been using this with networked gopls and other LSPs without network. All work great.
It's possible some LSP that I'm not using will require network be default tho.

I've actually been considering making this change as well because I am worried that all of  Docker's "magic" is sometimes too slow for the quick fork that NVIM is looking for. The networking stack does not need to be complex or literally existent at all for most cases.

yamlls requires a network connection to fetch schemas AFAIK

@mjlbach You're right, I've enabled network for yamlls by default.
It can be disabled in individual setups tho by simply adding network="none" to the options passed.

CI failures seem unrelated.

CI is now passing fine :+1:
Looks like there was a transient error fetching the nightly.

Any idea why CI is failing?
There's a bug in master where using additional languages fails. I've a fix for it in 523dddde2715b6aaa93bc020604be3eb8d5e5dd7, but that's kinda dependant on this PR, since the changes are built on top of it.

It looks like some of the tests were failing due to bad paths and other outdated code. I've fixed it and begun to catch up merging these requests. We will need this conflict fixed before we can merge it.

Thanks for the update!

authors: WhyNotHugo erkrnt mjlbach


text:
update version info

Should be done before and after each release :)

Thanks! :)

authors: lstrojny pierrejoye

repo: lstrojny/php-cld


text:
🛑 Mirror-爬 is down

In cd7eb77, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 1088 ms

Resolved: Mirror-鱼露 is back up in 7061fe0.


text:
🛑 Mirror-爬 is down

In 207b3ad, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 1130 ms

Resolved: Mirror-鱼露 is back up in d8234e3.


text:
🛑 Mirror-爬 is down

In 825b8b7, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 1052 ms

Resolved: Mirror-鱼露 is back up in 8e68d22.


text:
🛑 Mirror-爬 is down

In 8d28c3a, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 926 ms

Resolved: Mirror-鱼露 is back up in 98f98b7.


text:
🛑 Mirror-木生 is down

In 82a502e, Mirror-木生 (https://t.lsy223622.com/ncov-xidian-times) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Mirror-爬 is back up in 3f6ac9b.


text:
🛑 Mirror-爬 is down

In 612fc12, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 1604 ms

Resolved: Mirror-鱼露 is back up in 0b8919e.


text:
🛑 Mirror-爬 is down

In 4cf3b0d, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 1392 ms

Resolved: Mirror-鱼露 is back up in 1805e51.


text:
🛑 Mirror-爬 is down

In 47eb3c2, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 1232 ms

Resolved: Mirror-鱼露 is back up in aa02d7a.


text:
🛑 Mirror-爬 is down

In 32aea4c, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 727 ms

Resolved: Mirror-鱼露 is back up in c1628e4.


text:
🛑 Mirror-爬 is down

In 75437b9, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 1061 ms

Resolved: Mirror-鱼露 is back up in 705987d.


text:
🛑 Mirror-爬 is down

In addc77e, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 953 ms

Resolved: Mirror-鱼露 is back up in 450b01e.


text:
🛑 Mirror-爬 is down

In 408ac16, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 1000 ms

Resolved: Mirror-鱼露 is back up in 5f084e9.


text:
🛑 Mirror-爬 is down

In 175e30d, Mirror-爬 (https://ncov.zhouym.tech/) was down:

HTTP code: 404
Response time: 669 ms

Resolved: Mirror-鱼露 is back up in 8e2b2cb.


text:
Add more unit tests for continuous integration

[ ] Add more Mock platform tests

[ ] DownloadCache tests
[ ] Better ImageService tests coverage

[ ] Add Android Platform specific tests

[ ] MemoryCache tests
[ ] DiskCache tests

[ ] Add iOS Platform specific tests

[ ] MemoryCache tests
[ ] DiskCache tests

[ ] Add Mac Platform specific tests

[ ] MemoryCache tests
[ ] DiskCache tests

[ ] Add Windows Platform specific tests

[ ] MemoryCache tests
[ ] DiskCache tests

[ ] Add Tizen Platform specific tests

[ ] MemoryCache tests
[ ] DiskCache tests

https://github.com/luberda-molinet/FFImageLoading/tree/master/source/Tests/FFImageLoading.Tests

I met the same problem with FFImageLoading.Transformations, is there any quick way to get rid of this?
Like inject different instance etc?

Could you describe what the problem is? Is it related to unit tests? Usually deleting bin/obj packages and reopening solution should fix a lot of VS issues

Yes, we do the unit test, we use Xamarin.Forms.Mocks to mock Xamarin.Forms. In our XF project, we use transformation like this.
                <ffimageloading:CachedImage.Transformations>
                    <fftransformations:CircleTransformation BorderSize="0" />
                </ffimageloading:CachedImage.Transformations>

And it throws this exception FFImageLoading.Transformations.Common.DoNotReferenceMessage
I used 2.4.3.840

@jessejiang0214 Hi. It should be fixed with: https://github.com/luberda-molinet/FFImageLoading/commit/f3c2cb8882738101a730dfd3985908fad8a4224d
Also please make sure you follow this:
https://github.com/luberda-molinet/FFImageLoading/wiki/Unit-Testing----how-to-enable

Hi @daniel-luberda , cool, is there any nuget contain that fix?

Hi @daniel-luberda I tested 2.4.10.966-pre in pipline, it fixed this issue, thanks.

repo: luberda-molinet/FFImageLoading


text:
SVG not render properly

Description
I can´t make a svg render properly with a svg exported from sketch
Expected Behavior

Actual Behavior

Basic Information
the svg source is here
<?xml version="1.0" encoding="UTF-8"?> <svg width="52px" height="38px" viewBox="0 0 52 38" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">     <!-- Generator: Sketch 49 (51002) - http://www.bohemiancoding.com/sketch -->     <title>Page 1</title>     <desc>Created with Sketch.</desc>     <defs></defs>     <g id="Page-1" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">         <g id="conta" transform="translate(-664.000000, -617.000000)" fill="#00A0E9">             <path d="M672.272727,647.727273 L688.818182,647.727273 C689.470545,647.727273 690,647.199 690,646.545455 C690,645.893091 689.470545,645.363636 688.818182,645.363636 L672.272727,645.363636 C671.620364,645.363636 671.090909,645.893091 671.090909,646.545455 C671.090909,647.199 671.620364,647.727273 672.272727,647.727273 L672.272727,647.727273 Z M672.272727,643 L693.545455,643 C694.197818,643 694.727273,642.471727 694.727273,641.818182 C694.727273,641.165818 694.197818,640.636364 693.545455,640.636364 L672.272727,640.636364 C671.620364,640.636364 671.090909,641.165818 671.090909,641.818182 C671.090909,642.471727 671.620364,643 672.272727,643 L672.272727,643 Z M666.363636,624.090909 L713.636364,624.090909 L713.636364,619.363636 L666.363636,619.363636 L666.363636,624.090909 Z M666.363636,652.454545 L713.636364,652.454545 L713.636364,633.545455 L666.363636,633.545455 L666.363636,652.454545 Z M713.636364,617 L666.363636,617 C665.058909,617 664,618.058909 664,619.363636 L664,652.454545 C664,653.759273 665.058909,654.818182 666.363636,654.818182 L713.636364,654.818182 C714.941091,654.818182 716,653.759273 716,652.454545 L716,619.363636 C716,618.058909 714.941091,617 713.636364,617 L713.636364,617 Z M703,647.727273 L707.727273,647.727273 C708.379636,647.727273 708.909091,647.199 708.909091,646.545455 L708.909091,641.818182 C708.909091,641.165818 708.379636,640.636364 707.727273,640.636364 L703,640.636364 C702.347636,640.636364 701.818182,641.165818 701.818182,641.818182 L701.818182,646.545455 C701.818182,647.199 702.347636,647.727273 703,647.727273 L703,647.727273 Z" id="Page-1"></path>         </g>     </g> </svg>
maybe it's an error from parser?

Can you cut this until you find the simplest string that renders incorrectly?

authors: SergioBarbosa charlesroddie


text:
🛑 Školský šport is down

In 56dda34, Školský šport (https://skolskysport.sk/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Školský šport is back up in f0e2971 after 7 minutes.

repo: lubosm/minedusk-uptime


text:
🛑 Školský šport is down

In 5099e0e, Školský šport (https://skolskysport.sk/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Školský šport is back up in a4e1493 after 7 minutes.


text:
Not picking up own handle in Twitter profile in URL field

Not sure if maybe I'm doing something dumb or if something with the tld may be interfering.
Twitter profile: @Vchat20
Mastodon handle: https://neurodifferent.me/@Vchat20
I've got this entered in the URL field in my profile but under step 2 Fedifinder consistently says no handles were found.

The filter to remove telegram urls was too aggressive. Fixed with the new version.

repo: lucahammer/fedifinder


text:
Implement persistent data storage to reduce cold start time

Currently, osscontribute.com is hosted on Google Cloud Run, which uses an in-memory filesystem that does not persist data across container restarts.
Data is currently being written to a file in this in-memory filesystem, but because the file is not persisted across restarts, cold starts are slower than they should be because data has to be fetched from the GitHub API. This API call is particularly expensive because it is performing complex search queries on a massive dataset.
The JSON data returned from the GitHub API is relatively small and will not be accessed on a highly frequent basis, so storing the JSON data as a file in object storage should be sufficient for this use case. Since the service is running on GCP, it makes sense to use Google Cloud Storage (GCS).
It makes sense to decouple the web server logic from the data fetching/storage logic for this use case. Data only needs to refreshed roughly every 24 hours. This "job" is currently implemented as a background goroutine that runs every 24 hours, but due to restarts, the job never actually has a chance to execute because the timer gets reset on every restart.
To keep it simple and leverage tools already in use, GitHub Actions and Google Cloud Run will be considered for running a scheduled job to fetch and store GitHub data in a GCS bucket:
GitHub Actions: https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#schedule
Google Cloud Run: https://cloud.google.com/run/docs/execute/jobs-on-schedule
GitHub Actions is preferred because it is free for public repositories.

[ ] Create a dedicated GCS bucket to store GitHub data.
[ ] Implement scheduled job to run every 24 hours that fetches data from the GitHub API and stores it in the GCS bucket.
[ ] Remove background goroutine logic.
[ ] Refactor RepoData function to fetch GitHub data from GCS and cache it in memory.

Helpful links:
https://cloud.google.com/storage/docs/reference/libraries#use
https://github.com/google-github-actions/auth


text:
[todo] positional encoding

either use the RPE scheme as in the paper, or see if one can use rotary embeddings where possible, and T5 RPE bias in other places

the RPE used in the paper is good, explains why they use it to bias all other attention in the network


text:
Conducting experiments on ImageNet using the implementation of FSQ from this repository.

Many thanks to the authors for implementing the Finite Scalar Quantizer (FSQ). We further conduct experiments on ImageNet, and FSQ proves to be simple and effective.
This is the repo: https://github.com/duchenzhuang/FSQ-pytorch

@duchenzhuang nice! what would be helpful is to also try running it against LFQ. i'm curious which one comes out on top

In my current experimental setting, with all other configurations such as network architecture held constant, I observe that switching the quantizer to LFQ didn't work well. Even during the training of the auto-encoder, the reconstruction loss on the training set remained at a relatively high level and did not decrease.
Additionally, I found a contradiction between the papers on FSQ and LFQ. The FSQ paper suggests that it is advisable for each dimension to have values greater than or equal to 5 (Figure 10), whereas LFQ essentially allows each dimension to have only two values, either +1 or -1.

thank you Chenzhuang!
disappointing to hear but good to know

@Lijun-Yu would you like to do one more code review of LFQ, just to make sure my implementation of your work is fair

For sure, at a glance I see an issue in calculating the avg_prob and in turn codebook_entropy. I'll do a full review and potentially send a pr to fix shortly

@Lijun-Yu thank you! 🙏

For sure, at a glance I see an issue in calculating the avg_prob and in turn codebook_entropy. I'll do a full review and potentially send a pr to fix shortly

I'm eagerly looking forward to your comprehensive review of LFQ's implementation. Following that, I plan to rerun experiments on my code to validate the Lookup Free Quantizer.

@Lijun-Yu could you possibly put the issue you observe into writing? probably faster that i just put in the patch

Sorry for the delay. I've started running verification with the same auto encoder architecture following your examples. And I found more issues than I expected - the current implementation is giving 1% utilization with LFQ. Let me investigate more and update here in a few hours.

In addition, I've noticed that when I set the auto-encoder's downsampling factor to 16 (e.g., compressing a 256 * 256 image into a 16 * 16 latent space), the reconstruction quality is significantly poorer.
I've observed that both VQGAN and MagViTv2 use a downsampling factor of 16, but even though VQGAN has open-sourced their code, it seems challenging to reproduce results similar to their paper.
Do you have any recommendations or references to address this issue? @Lijun-Yu @lucidrains

256x256 -> 16x16 is definitely doable as have been done in VQGAN and MaskGIT. And MAGVITv2 also show that we can do 512x512 -> 16x16 with SOTA generation quality. However, such high compression rate puts higher pressure on the model - in general you need more blocks/layers to do more down/upsampling and also to build a deeper/larger model. You may find Fig 7 from arxiv.org/pdf/2212.05199 helpful if you change all 3D conv to 2D and adapt it for 256x256

In addition, I've noticed that when I set the auto-encoder's downsampling factor to 16 (e.g., compressing a 256 * 256 image into a 16 * 16 latent space), the reconstruction quality is significantly poorer.
I've observed that both VQGAN and MagViTv2 use a downsampling factor of 16, but even though VQGAN has open-sourced their code, it seems challenging to reproduce results similar to their paper.
Do you have any recommendations or references to address this issue? @Lijun-Yu @lucidrains

256x256 -> 16x16 is definitely doable as have been done in VQGAN and MaskGIT. And MAGVITv2 also show that we can do 512x512 -> 16x16 with SOTA generation quality. However, such high compression rate puts higher pressure on the model - in general you need more blocks/layers to do more down/upsampling and also to build a deeper/larger model. You may find Fig 7 from arxiv.org/pdf/2212.05199 helpful if you change all 3D conv to 2D and adapt it for 256x256

Thank you once again for your response. I would like to inquire about the necessity of GAN loss when training a VQVAE with high compression ratios (such as 256 * 256->16 * 16 or 512 * 512->16 * 16). In previous experiments, I did not include GAN loss due to its instability. Therefore, I would like to seek your insights on the importance of GAN loss.

In addition, I've noticed that when I set the auto-encoder's downsampling factor to 16 (e.g., compressing a 256 * 256 image into a 16 * 16 latent space), the reconstruction quality is significantly poorer.
I've observed that both VQGAN and MagViTv2 use a downsampling factor of 16, but even though VQGAN has open-sourced their code, it seems challenging to reproduce results similar to their paper.
Do you have any recommendations or references to address this issue? @Lijun-Yu @lucidrains

256x256 -> 16x16 is definitely doable as have been done in VQGAN and MaskGIT. And MAGVITv2 also show that we can do 512x512 -> 16x16 with SOTA generation quality. However, such high compression rate puts higher pressure on the model - in general you need more blocks/layers to do more down/upsampling and also to build a deeper/larger model. You may find Fig 7 from arxiv.org/pdf/2212.05199 helpful if you change all 3D conv to 2D and adapt it for 256x256

Thank you once again for your response. I would like to inquire about the necessity of GAN loss when training a VQVAE with high compression ratios (such as 256 * 256->16 * 16 or 512 * 512->16 * 16). In previous experiments, I did not include GAN loss due to its instability. Therefore, I would like to seek your insights on the importance of GAN loss.

In general, GAN loss leads to better visual quality if that's what you care about. And the architecture/capacity of the discriminator matters quite a bit, e.g. StyleGAN would be better/more stable than PatchGAN. In addition, GAN loss has a significant impact on FID/FVD scores if you care about those metrics.

I fixed a few issues in #89 , @duchenzhuang maybe you can have a try on ImageNet as well
The default parameters in lookup_free_quantization.py should be reasonable enough to work. But you may need to train a generation model to fully see its benefits compared to other methods.

@Lijun-Yu thank you for the PR! very excited to try breeding LFQ latents in a genetic algorithm setting

I fixed a few issues in #89 , @duchenzhuang maybe you can have a try on ImageNet as well The default parameters in lookup_free_quantization.py should be reasonable enough to work. But you may need to train a generation model to fully see its benefits compared to other methods.

Thank you, Lijun. I will try it as soon as possible.

Thank you very much, @Lijun-Yu @lucidrains! The Lookup Free Quantizer works effectively in my experiment!
I would like to inquire with Lijun @Lijun-Yu  about the selection of hyperparameters for LFQ.
Is commitment_loss_weight fixed at 0.25 for all experiments?
I noticed that entropy_loss_weight is defaulting to 0.1 in LFQ, but in your example, the weight for entropy loss is even smaller. Could you please advise on a suitable value for entropy_loss_weight, especially for larger datasets like ImageNet?
As the Codebook size increases, how should the weights for entropy loss be adjusted?
Thank you.

By the way, @Lijun-Yu , in your  experiments, did you only use perceptual loss and adversarial loss without employing L2 loss or L1 loss for image reconstruction when training the tokenizer?

Thank you very much, @Lijun-Yu @lucidrains! The Lookup Free Quantizer works effectively in my experiment!
I would like to inquire with Lijun @Lijun-Yu about the selection of hyperparameters for LFQ. Is commitment_loss_weight fixed at 0.25 for all experiments? I noticed that entropy_loss_weight is defaulting to 0.1 in LFQ, but in your example, the weight for entropy loss is even smaller. Could you please advise on a suitable value for entropy_loss_weight, especially for larger datasets like ImageNet? As the Codebook size increases, how should the weights for entropy loss be adjusted?
Thank you.

In all our experiments in the paper, we use commitment loss weight 0.25 and entropy loss weight 0.1, which has been tested for ImageNet/Kinetics/larger datasets across vocabulary sizes from 2^10 to 2^40. (In the next arxiv revision we'll provide a hyperparameter list to clarify these.)

By the way, @Lijun-Yu , in your experiments, did you only use perceptual loss and adversarial loss without employing L2 loss or L1 loss for image reconstruction when training the tokenizer?

We use L2 + perceptual + adversarial. You may find many implementation details in the official repo for magvitv1 https://github.com/google-research/magvit/tree/main. Magvitv2 is similar unless specifically noted in our paper.

but in your example, the weight for entropy loss is even smaller

The toy example uses much smaller resolution/model/dataset, so it's not representative enough. It's still okay to use 0.1 in this toy setup, but the training would focus too much on optimizing the entropy and lead to somewhat worse reconstruction, which is why I reduced the weight. This is usually not the case when you train deeper model on larger datasets.
In addition, I want to remind you of the entropy annealing schedule which is quite useful for large vocabularies. @duchenzhuang

but in your example, the weight for entropy loss is even smaller

The toy example uses much smaller resolution/vocabulary/model/dataset, so it's not representative enough. It's still okay to use 0.1 in this toy setup, but the training would focus too much on optimizing the entropy and lead to somewhat worse reconstruction, which is why I reduced the weight. This is usually not the case when you train deeper model on larger datasets with perceptual and adversarial losses.
In addition, I want to remind you of the entropy annealing schedule which is quite useful for large vocabularies. @duchenzhuang

Thank you, Lijun @Lijun-Yu .

@Lijun-Yu congrats Lijun! https://github.com/lucidrains/meshgpt-pytorch/issues/9#issuecomment-1852588734

authors: Lijun-Yu duchenzhuang lucidrains

repo: lucidrains/vector-quantize-pytorch


text:
增加隐藏板块功能

希望可以增加隐藏板块的功能，比如隐藏底栏的频道，主页的直播。平时个人最常用的是动态，推荐，热门，排行榜，搜索这几个板块。可以考虑一下，多谢，开发者辛苦了！

1.0.8版本可以在设置中关闭热搜和搜索栏推荐

authors: amokiel lucinhu


text:
Issue with NukkitX preventing plugin from working

https://i.gyazo.com/5b54dfb54ef1a38a200e706b09704a37.png
plugin doesnt appear to be working, only other plugin on the server is https://github.com/funniray/EssentialsNK which should format the chat. (error occurs even without essentialsnk)
according to dev of nukkitx you are trying to inject into a final field (or something like that)

Issue was because Nukkit uses a Set, not a list :)
Thanks for reporting

authors: PhanaticD lucko

repo: lucko/LuckPerms


text:
Server Hanging/Crashing

https://pastebin.com/ZsfaJrnU
I seem to be getting this once in a blue moon, and then on top of that my server conveniently decides to just stop error reporting in its crash once in a blue moon. Any Ideas?

Going to close this as I just didn't have enough Metaspace memory configured, and hence most likely nothing to do with LP, just that LP was trying to do something when the server ran out of metaspace.  (For future concerns from anyone else)


text:
AdvancedBan Permissions not Showing Up with LuckPerms

Steps to Reproduce the problem

Download LuckPerms and AdvancedBan
Put AdvancedBan in your BungeeCord
Plugins folder (not the individual spigot servers) just the Bungeecord Plugins folder
Put the LuckPerms plugin in any server plugin folder you like.
Put LuckPermsBungee in your bungee cord plugin folder
Lauch MC
Go into your BungeeCord server and do /server (Server you put luck perms in)
Enter the console command /lp user permission set luckperms.* true
Do /lp editor.
Open the assigned link
Try to do any ab. perms. (It won't autofill)
If it doesn't autofill, try just inputting any perm manually like ab.ban.perma
Click the save changes at the top of the luck perms editor, and copy and paste the command it gives you into your game (player command).

When you try to do /ban
It will say,
AdvancedBan You don't have perms for that (or somthing similar)
When I try to add a perm for AdvancedBan, I try to add a perm to luck perms web editor by just doing ab. usually the web editor will complete the perm I am trying to input, but nothing shows up for me with AdvancedBan at all.
Even when I manually type a perm into luck perms such as ab.ban.perma, when I apply the edits, in game it still doesn't work. I can do the command from the console so I know the commands work I just don't know why perms doesn't
Advanced ban is in my bungee plugin folder.
While luck perms is in my spigot server.
Any help?
Sincerely
Shreyas Ayyengar

Have you synced all your LuckPerms on all server (including bungeecord) to a common sql database?

Ensure your bungee network has been properly synced as per this guide: https://luckperms.net/wiki/Network-Installation.


text:
User meta issue

Description
When a group is set to a user, the user's meta gets set to the previous group's.
Reproduction steps

Run the command /lp bulkupdate users delete permissions (to clear permissions and groups)
The user now has the default group set, with a clear user meta
Set the user's parent to another group
The user now has the default group meta set as user meta and in order to show the new group info, /lp user  meta clear is required.

Expected behavior
When I set a user to a group, I expect the plugin to not set the previous group's meta to the user.
Environment details

Server type/version: YAPFA (PaperSpigot) 1.15.2
LuckPerms version: v5.2.38

Bump

Does this happen when LuckPerms is the only plugin installed?
Screenshot /lp info and two /lp editor links from before and after running the bulk update command please.

Sorry for the late reply..

Sorry for the late reply..


text:
Luckperms expire time

Description
So if i add temp group to player, and i type somewhere placeholder %luckperms_expire_time_group.master% (it can be whatever) the placeholder says not 30 days but 5d 59h 59m 59s.

Expected behaviour
Expected is 30d not 5d 59h 59m 59s.

Environment details

Server type/version: PaperSpigot running version 1.15.2 build git-Paper-390
LuckPerms version: v5.3.0 - i also tried 5.1.26

Picture https://i.imgur.com/1CLls54.png

Update PAPI to a dev build https://ci.extendedclip.com/job/PlaceholderAPI/

thanks it worked


text:
Windows Support

From what I can tell, as of this commit (latest), Windows is supported and working on Crystal 1.7.1. Ameba, however, does not seem to support windows yet, or at least I can't run shards install for it successfully.
I guess the least we can do is just add Windows in a Github Action to validate it compiles and runs.

Yeah, would be nice to start getting these smaller shards out of the way sooner. Hopefully it doesn't take a ton of effort.

authors: jwoertink mdwagner


text:
Add recorded demonstration

Provide a demo with a video showing bufferline and lualine click support along with usage of keymaps and commands.
Scenario
Open the default project with:

bufferline left icons (file explorer, git, ...)
bufferline right icons (overseer, outline, ...)
lualine bottom icons (toggleterm, trouble, spectre, ...)

Click support:

Click on icons to open the some views from each edge.
Click on icons to select next/previous group.
Open several toggleterm and change group.

Command:

Use EdgeGroupSelect to open few groups

Keymaps:

Navigate on few groups with keymaps

Done in 3a74f23


text:
Got crash

on windows, this is crash log:
KERNELBASE
VCRUNTIME140
UnrealEditor_AzSpeech!std::_Throw_bad_weak_ptr() [d:\build\AutoSDK\Sync\HostWin64\Win64\VS2022\14.34.31942\INCLUDE\memory:1050]
UnrealEditor_AzSpeech!Microsoft::CognitiveServices::Speech::SpeechSynthesizer::StopSpeakingAsync() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\AzSpeech\Source\ThirdParty\AzureWrapper\include\cxx_api\speechapi_cxx_speech_synthesizer.h:461]
UnrealEditor_AzSpeech!FAzSpeechSynthesisRunnable::Exit() [D:\build\U5M-Marketplace\Sync\LocalBuilds\PluginTemp\HostProject\Plugins\AzSpeech\Source\AzSpeech\Private\AzSpeech\Runnables\Synthesis\AzSpeechSynthesisRunnable.cpp:92]
UnrealEditor_Core
UnrealEditor_Core
kernel32
ntdll

Also getting this crash, happens randomly when generating text to speech with same call stack

repo: lucoiso/UEAzSpeech


text:
🛑 CCO Lab is down

In eeceb8b, CCO Lab (https://lab.lukeslabs.xyz) was down:

HTTP code: 530
Response time: 118 ms

Resolved: CCO Lab is back up in 015dbd1 after 1 day, 1 hour, 3 minutes.

repo: lucolvin/Upptime


text:
How to handle katex errors.

How can I handle errors which are throwing to console by katex itself? I want to show this error messages in my vue component directly to users.
Example:

To render latex I am using:
<div v-katex:auto="{options}" :key="item.text">.... </div>

Why is this closed? Do you find the solution @sabadoryo ?

Indeed, I think we can do better. I wanted to update the plugin for Vue 3 and keep compatibility with Vue 2 as well. I will add this feature that is handy. I cannot commit to a date. For specific use of katex, it's also possible to create your own component, this lib being a very light wrapper around the original katex library.

authors: lucpotage sabadoryo tuanalumi

repo: lucpotage/vue-katex


text:
Login persistence: not persistent

There are a variety of situations where you become no longer logged in.

Is there anything I can do to avoid those situations and stay logged in? It seems to log me out whenever I'm not looking, even when it's open in a tab :p

Not yet. It's literally on my end (I implemented it wrong). This might be the source of a bunch of pain this weekend. :cry:

This should be fixed now.

authors: kamjau povrazor

repo: ludumdare/ludumdare


text:
Weird '/' on the main page

Not really sure what it's meant to say, but it looks like you missed one of the link's text.

It's part of the sentence "To check when the event is ending look at the countdown clock in the sidebar of / site. The Compo finishes 48 hours after the theme was announced and the Jam ends 72 hours after the theme was announced."

Ok, looks like the page has since been updated. Maybe I was looking at it mid edit.

I think you were thinking of this page:
https://ldjam.com/events/ludum-dare/hosting-guide

authors: Jezzamonn povrazor


text:
fix typo: exponental => exponential

Fix comment typo

Thank you!

authors: laqiiz lufia

repo: lufia/backoff


text:
Environment Variables keeps reading from .env

running ./gradlew bundleRelease to generate app bundles to release to google play store.
Debug mode reads correct environment variants with respective .env files. However, all variant's app bundles are being generated with only the .env file.
Root

app/build.gradle

tried adding proguard override -keep class com.dottid.BuildConfig { *; }
tried ENVFILE=.env.prod ./gradlew bundleRelease, but getting a bunch of gradle errors, ... without declaring an explicit or implicit dependency
tried changing name of .env to .env.developer but then getting a missing env error when i run command ./gradlew bundleRelease

Thank you for your time

are you using monterey?

monterey

I am using Bumblebee but get the same error. without .env file, build fails.

The specific issue from the topic starter is likely because he is building using ./gradlew bundleRelease to build all the release variants at one go instead of specifically specifying the full variant eg. ./gradlew bundleProductionRelease.
You can look at https://github.com/luggit/react-native-config/blob/master/android/dotenv.gradle in this repository to see the logic for how the env file is determined. From the logic I don't think it is possible for it to determine the correct env file to use if you are building more than one build variant at a time. In which case it will fallback on .env.
As a sanity check you can after building see android/app/build/generated/source/buildConfig/{productFlavor}/{buildType}/{package}/BuildConfig.java to check if the values are as expected.

I just updated Android studio & command line tools. Since then, I'm not receiving the the ENVFILE variable (with System.env['ENVFILE']) when dotenv.gradle is executed (Checked by printout output of System.env in dotenv.gradle).
Could there be a change in the new version of either android studio or command line tools which affects the ability to read this variable?

Just tried. Using Android Studio Chipmunk on Windows and tested with SET ENVFILE=.env.staging && react-native run-android in package.json and it works. That said I made sure that I didn't have any pre-existing gradle daemons running. There might be potential issues with a pre-existing gradle daemon and reading the current environment variables. I actually don't use the ENVFILE environment variable and just configure variants for Android and schemes for iOS.

That said I made sure that I didn't have any pre-existing gradle daemons running. There might be potential issues with a pre-existing gradle daemon and reading the current environment variables.

Good point. After retrying without pre-existing daemon running, the ENVFILE variable was read correctly.
This may seem pretty common sense in retrospect, but might still be worth adding a small note to the README for anyone who encounters the same issue.

Similar thing to what @Mr0cket reports happening here. The ENVFILE=.env.something way of setting the env file seems to have stopped working as expected.
Up till now I have had different npm scripts, each doing ENVFILE=.env.something react-native run-android, and the envfile would be correctly changed if I ran two different builds in a row.
Now the first run will pick the correct env file, and the ones following will use the same ENVFILE variable, unless the gradle daemon is stopped with ./gradlew --stop before the build.
Not sure if it can have something to do but I've recently updated my JDK version from JDK 8 to 11 (Zulu OpenJDK as specified in the latest RN docs) and also the Xcode Command Line Tools to 13.2.0.
I'm running in Mac M1
react-native: 0.67.2
react-native-config: 1.4.6
macOS: BigSur (11.6.2)

my solution is cp .env.prod .env before run a script, and .env was ignored from git

my solution is cp .env.prod .env before run a script, and .env was ignored from git

only this worked.
project.ext.envConfigFiles and ENVFILE=.env.something are ignored
"react-native-config": "^1.4.6"

./gradlew --stop

This one worked for me finally!

right above project.ext.envConfigFiles, set this:
project.ext.defaultEnvFile ="../path/to/.env.file"

Yes super annoying, seems to be an issue with m1 or montery keeping gradle deamon in memory. For fastlane added the -- stop command in the build script
gradle(
  task: "--stop"
)

Hi,
I had run into the same issue and did the following approach.
Instead of providing envConfigFiles at top level, what I did was provide default env file in the product flavor itself
    flavorDimensions "phase"
    productFlavors {
        dev {
            dimension "phase"
            project.ext.defaultEnvFile = ".env.dev"
            apply from: project(':react-native-config').projectDir.getPath() + "/dotenv.gradle"
            applicationId project.env.get("APP_ID")
        }
        stage {
            dimension "phase"
            project.ext.defaultEnvFile = ".env.stage"
            apply from: project(':react-native-config').projectDir.getPath() + "/dotenv.gradle"
            applicationId project.env.get("APP_ID")
        }
        prod {
            dimension "phase"
            project.ext.defaultEnvFile = ".env.prod"
            apply from: project(':react-native-config').projectDir.getPath() + "/dotenv.gradle"
            applicationId project.env.get("APP_ID")
        }
    }

This should allow to build multiple variants with one command.
Since apply is getting invoked multiple times there might be a performance hit. Maybe someone with gradle experience can comment here
One drawback here is that env variable should be used only within flavor definition. Once it goes outside the block, it may be overwritten by the next flavor.
Also more complex combination of flavors may become difficult to handle.

I just stumbled on this as well... the strange this is that it's only affecting one of my projects, not both. ./gradlew --stop worked for me so I've added it to my fastlane as per #issuecomment-1230403293

right above project.ext.envConfigFiles, set this:
project.ext.defaultEnvFile ="../path/to/.env.file"

This one line worked like a charm in my case

When running it locally the java process also stays running forever on the m1 new mac books, taking quite some memory, so works better to stop the process.
gradle(
task: "--stop"
)

from your root folder try this
cd android && ./gradlew clean && ENVFILE=../.env.production ./gradlew bundleRelease && cd ..

my solution is cp .env.prod .env before run a script, and .env was ignored from git

I have a few apps using the same codebase and was attempting to keep the .env files in separate folders. Using the ENVFILE=app1/.env.development command would not work for me at all. Only the above has worked correctly

my solution is cp .env.prod .env before run a script, and .env was ignored from git

only this worked. project.ext.envConfigFiles and ENVFILE=.env.something are ignored
"react-native-config": "^1.4.6"
"./gradlew --stop" - this helped to reset to .env from .env.beta (no idea how it .env.beta was selected before 🤨)

Yes, worked for me

Hi , I am still facing this issue !
The release builds always picks up .env config instead of any other extenstion.

add defaultEnvFile=.env.prod in to file gradle.properties

In my case this issue was caused by calling
apply from: project(':react-native-config').projectDir.getPath() + "/dotenv.gradle"
too early. It must be called AFTER:
project.ext.envConfigFiles = [     debug: ".env.development",     release: ".env.production",     anothercustombuild: ".env", ]

authors: Abhijeetk28 AdamGerthel Beka-Bat1 BoilingOil CaioSantos Mr0cket Stas-Buzunko ToshKoevoets anees17861 ashcode1 chamaldesilva fahry-mohammed jjdp justin-tay mahardikaindra my-name-is-nheo personalnadir santitopo viet795 vinhtnk

repo: luggit/react-native-config


text:
Subscriber clients won't reconnect

Subscriber clients won't reconnect after a timeout when redis server removes the client based on tcp-keepalive setting (default is 300 seconds).
This is only related to a remote redis connection. Locally it works.
I ran into this because I put my system to sleep instead of shutting it down. But instead of sleep it could be a broken uplink, switch failure, power outage and so on.
The subscriber should automatically reconnect, but in this case he doesn't.
Reproduce

Use a remote redis server
Cut your uplink for at least 300 seconds (tcp-keepalive value) or put your system to sleep/hibernate

You can reduce this time when you reduce the tcp-keepalive value on your redis server

const Redis = require("ioredis");

(async () => {
  const client = new Redis({
    host: process.env.REDIS_HOST,
    port: process.env.REDIS_PORT,
    username: process.env.REDIS_USERNAME,
    password: process.env.REDIS_PASSWORD,
  });

  client.on("error", (err) => {
    console.log("client error", new Date(), err);
  });

  const publisher = new Redis({
    host: process.env.REDIS_HOST,
    port: process.env.REDIS_PORT,
    username: process.env.REDIS_USERNAME,
    password: process.env.REDIS_PASSWORD,
  });

  publisher.on("error", (err) =>
    console.log("publisher error", new Date(), err)
  );

  const subscriber = new Redis({
    host: process.env.REDIS_HOST,
    port: process.env.REDIS_PORT,
    username: process.env.REDIS_USERNAME,
    password: process.env.REDIS_PASSWORD,
  });

  subscriber.on("error", (err) => {
    console.log("subscriber error", new Date(), err);
  });

  subscriber.subscribe("ioredis:msg");

  subscriber.on("message", (channel, message) => {
    console.log(new Date(), message);
  });

  setInterval(() => {
    publisher.publish("ioredis:msg", "published @ " + new Date());
  }, 2500);
})();

I noticed that the error event for the subscriber doesn't get fired every time.
subscriber.on("error", (err) => {
  console.log("subscriber error", new Date(), err);
});
If it would, I could initiate the ping from from there.

My working fix
Ping the server regularly.
setInterval(() => {
  subscriber.ping()
}, 2500);

The issue is most likely that there's no keepAlive configured. And for the keepalive itself we have another issue open: https://github.com/luin/ioredis/issues/1339

question is why isn't a connection abort detected? I think it should resubscribe.
Sorry, will reopen that.

Could you test that with ioredis v5?

Is this somehow tackled at the moment? The README says that a subscribed client will resubscribe to a channel, but for this the connection loss must be detected. The ping trick does its thing, but it would be cooler if a connection loss would be detected.

Ah I see. We use node.js LTS Version, which is 16.x.

The applied fix in ioredis v5.03 should work for all the node.js versions.
The mention of 17.7 was just about another new configuration option in the socket api, but that's not in use at the moment.

Closing as we can assume the keepalive change fixed it

authors: Mall0c marcbachmann xairoo

repo: luin/ioredis


text:
relation "shedlock" does not exist

Describe the bug

Which version do you use 4.21.0
Which Lock Provider JdbcTemplateLockProvider
ShedLock configuration

    @Bean
    public LockProvider lockProvider(DataSource dataSource) {
        return new JdbcTemplateLockProvider(
                JdbcTemplateLockProvider.Configuration.builder()
                        .withJdbcTemplate(new JdbcTemplate(dataSource))
                        .withTableName("shedlock")
                        .usingDbTime()
                        .build()
        );

ShedLock logs

Caused by: org.postgresql.util.PSQLException: ERROR: relation "shedlock" does not exist
  Position: 8
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2553)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2285)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:323)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:473)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:393)
	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:164)
	at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:130)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
	at org.springframework.jdbc.core.JdbcTemplate.lambda$update$0(JdbcTemplate.java:867)
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:617)
	... 34 common frames omitted

Expected behavior
Shedlock finds the table and acquires lock
Actual behavior
Throws exception about table not found

Hi, can you please describe what did you do to debug it yourself?

So I tried passing in my own custom implementation of datasource Bean and doing so while debugging I noticed that the bean is created fine but the jdbcTemplate is connected to a database called default (no idea yet where this is coming from)

Hi, it seems that you data-source is misconfigured. In postgress it's quite tricky to get it right, I would recommned you to play with the JDBC connection String. If everything fails and it's an issue with different schema, you can specify the table name in LockProvider config in form 'schema.shedlock'.
As it is an issue with your configuration I will close this ticket. Good luck.

authors: MSaifAsif lukas-krecan

repo: lukas-krecan/ShedLock


text:
Fix swapped GH username and IRC nick

Fixes #20

Coverage remained the same at 100.0% when pulling 409f4757374be469e546eda347380b414df88159 on bug/20 into e2a05a6da73e3e7bce7d036538c6c1044df92377 on master.

authors: coveralls lukegb

repo: lukegb/SpongeAuth


text:
automatic inversion of x axis with pandas dataframe read from a csv

Description
The x axis of histogram is automatically inverted when plotted with a pandas dataframe read from a csv file. It's fine when the data is not from a csv file as far as I observed.
Steps to reproduce
import numpy as np
import pandas as pd
import proplot as plot

rng = np.random.default_rng()
x = rng.standard_normal(500)
x = pd.DataFrame(x, columns=['col'])

f = 'test.csv'
x.to_csv(f)
x1 = pd.read_csv(f, index_col=0)
fig, ax = plot.subplots()
ax.hist(x1.col.values)
plot.show()

Expected behavior: [What you expected to happen]
Actual behavior: [What actually happened]

Equivalent steps in matplotlib
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.hist(x1.col.values)

Proplot version
0.6.4

Thanks for the report, this was a very simple fix: 1290536

This seems to happen when I specify the number of bins?
But doesn't without that...

authors: kinyatoride lukelbd scottstanie

repo: lukelbd/proplot


text:
tor-browser: update to version 8.5.1

changed checkver regex url

There seems to be a pull request for tor-browser that's under consideration already.

repo: lukesampson/scoop-extras

license: Unlicense


text:
After updating scoop's Google Chrome it went nuts

Mainly 40+ tabs went into the wild - not saved in history or opened after closing, updating and starting the browser
Next time please test the apps BEFORE added to scoop's ecosystem. Thank you very much

Now more on the problem. If you set
D:\scoop\apps\googlechrome\current\chrome.exe --user-data-dir=d:\scoop\apps\googlechrome\**current**\User
it's ok to expect that scoop will MOVE User subdir from symlinked chromeversiondirname as current to the newer one chromeNEWERversiondirname, not leaving it in the old and waiting chrome to create a new one. In this case scoop MUST move the User dir from 77.0.3865.75 dir to 77.0.3865.90

If you want a portable chrome by setting D:\scoop\apps\googlechrome\current\chrome.exe --user-data-dir=d:\scoop\apps\googlechrome\current\User , please create a new manifest googlechrome-porable or something else. Scoop does not know there is a User directory which need to be persist unless you tell it by persist .

Portable settings configured with parameter are pain for many reason, but main is the obvious one, which will make it unusable.
You cannot set parameter to Window's Default Apps. This will make it junk.

If you open it from start menu / cmd, it will work with parameter.
If some application will spawn & url.url (how all application spawn browser) browser will be opened without parameter, which will result in completely different profile located in AppData
If you set it as open with, then again you have no parameter.

As I already told you in issue before. Chrome is using AppData. This indicates it is not portable by default. You created your own problem, which has no solution, which will fit most users use cases. Create custom manifest in custom bucker for portable chrome, but it will not work as you want.

Already fixed the problem but I'm too lazy to add parameters to the shortcut in C:\Users\UserName\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Scoop Apps :)

https://github.com/lukesampson/scoop/issues/3657#issuecomment-533855636
informative post.
github should have a " informative" reaction.

The final fix to make Chrome somewhat portable in scoop is to associate http and https urls when opened from your mail client or any app that opens them or files with the DEFAULT browser in the system. Just open in the registry HKEY_CLASSES_ROOT\http\shell\open and change the registry at HKEY_CLASSES_ROOT\ChromeHTML.QLSDFSVFTHARQA4CM4IU7I35SY\shell\open\command to be

"yourdrivename:\scoopdirectory\apps\googlechrome\current\chrome.exe" "%1" "--user-data-dir=yourdrivename:\scoopdirectory\apps\googlechrome\yourUserDataDirectory"

instead of

"yourdrivename:\scoop\apps\googlechrome\current\chrome.exe" -- "%1"

https://github.com/lukesampson/scoop/issues/3657#issuecomment-533850089

from https://github.com/lukesampson/scoop/issues/3657#issuecomment-533855636

You cannot set parameter to Window's Default Apps

this applies only if user tries to make google-chrome work in portable mode?

If I remember correctly if you need chrome to output to stdout you must enable logging with a switch after the exe:
chrome.exe --enable-logging --v=1

you must enable logging with a switch

good to know. but i'm talking about what curl.exe -hwould do.

There's no /help or /? switches implemented for chrome.exe

repo: lukesampson/scoop

license: unlicense


text:
Finalize bucket extraction

Closes #1328

I tested it today and haven't found any bug, which need to postpone bucket extraction.
/cc @r15ch13 @niheaven @h404bi @rasa @brandon93s @scoopinstaller @se35710 Did you encounter any bugs?
SideNote: Remove Path alias for binaries as discussed in discord
Fixes from decompress refactor:

Closes #3080
Closes #3120
Closes #3198
Closes #3230
Closes #3287
Closes #3339
Closes #3374
Closes #3388

Don't make duplicated commits from other branches to master please. Just push your finial changes to develop than ask @r15ch13 to release these updates from develop branch to master.


text:
Fix typos

Found via typos --format brief.

:partying_face: :partying_face: :partying_face: :partying_face: :partying_face:

repo: lunatic-solutions/submillisecond


text:
code block backgound

I am using the SyntaxHighlighting extension but it generates a white background for the code block.  I tried passing in StyleDictionarly.DefaultDark but that gave a black background and most of the code is invisible.
I simply would like each code block to have a light grey like you see on every blog post on line
here is the code I am using
private MarkupString ConvertStringToMarkupString(string value)
{
    if (!string.IsNullOrWhiteSpace(_content))
    {
        var pipeline = new MarkdownPipelineBuilder()
        .UseAdvancedExtensions()
        .UseSyntaxHighlighting()
        .Build();
        var html = Markdig.Markdown.ToHtml(value, pipeline);

        var sanitizedHtml = HtmlSanitizer.Sanitize(html);

        return new MarkupString(sanitizedHtml);
    }

    return new MarkupString();

}

That's a CSS issue/SyntaxHighlighting on your side that should be solved with CSS. Markdig doesn't enforce any rendering rules

authors: xoofx zoinkydoink

repo: lunet-io/markdig


text:
Update mmu.md

level  1 chunk size should be 1GB

Oops sorry about that, thank you so much! :-)


text:
PyTorch: l.slice is not a function in

Netron app and version: 4.6.3 Desktop

Steps to Reproduce:
import torch
model = torch.hub.load('pytorch/vision:v0.7.0', 'deeplabv3_resnet50', pretrained=True)
model.eval()
torch.jit.save(torch.jit.script(model), 'deeplabv3_scripted.pt')

deeplabv3_scripted.pt.zip
pytorch/pytorch#48525

pytorch/pytorch#48525

how do you solve this unsupported issue? why closed?

repo: lutzroeder/netron


text:
Added poe tcp streaming demos

TODO: add readmes

Modify such that PoE device is the host and other relevant changes

@themarpe Both options (OAK host or client) are in the folder. What relevant changes?

Hi, my name is Ryan.
i have still not clear how to modify those files. So i have questions.
First: Which one is server? OAK-PoE?
Second: How to put the code on OAK-PoE?
If i am on the wrong way please correct me.
Best regards,

Hello @reallyedutech ,

Files can be found here
As you can see on the link above, there are 2 folders; one where OAK is host and one where OAK is the client
When you run oak.py, the pipeline will get uploaded to the (connected) OAK camera. Simultaneously you have to run the host.py script, so the TCP communication starts.

authors: Erol444 reallyedutech themarpe

repo: luxonis/depthai-experiments