text:
Chapter6 DQN Pong can't calculate loss

I'm running the exact same code yet i get this error:
C:\Users\Utilizador\anaconda3\python.exe "C:/Users/Utilizador/Thesis/Deep Reinforcement Learning Hands-On/Chapter6/02_dqn_pong.py"
DQN(
(conv): Sequential(
(0): Conv2d(4, 32, kernel_size=(8, 8), stride=(4, 4))
(1): ReLU()
(2): Conv2d(32, 64, kernel_size=(4, 4), stride=(2, 2))
(3): ReLU()
(4): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1))
(5): ReLU()
)
(fc): Sequential(
(0): Linear(in_features=3136, out_features=512, bias=True)
(1): ReLU()
(2): Linear(in_features=512, out_features=6, bias=True)
)
)
880: done 1 games, mean reward -21.000, eps 0.99, speed 720.31 f/s
1848: done 2 games, mean reward -21.000, eps 0.98, speed 659.82 f/s
2670: done 3 games, mean reward -21.000, eps 0.97, speed 642.90 f/s
3492: done 4 games, mean reward -21.000, eps 0.97, speed 666.83 f/s
4659: done 5 games, mean reward -20.600, eps 0.95, speed 671.71 f/s
Best mean reward updated -21.000 -> -20.600, model saved
5765: done 6 games, mean reward -20.167, eps 0.94, speed 629.02 f/s
Best mean reward updated -20.600 -> -20.167, model saved
6682: done 7 games, mean reward -20.143, eps 0.93, speed 645.23 f/s
Best mean reward updated -20.167 -> -20.143, model saved
7668: done 8 games, mean reward -20.125, eps 0.92, speed 628.90 f/s
Best mean reward updated -20.143 -> -20.125, model saved
8648: done 9 games, mean reward -20.000, eps 0.91, speed 606.56 f/s
Best mean reward updated -20.125 -> -20.000, model saved
9919: done 10 games, mean reward -19.700, eps 0.90, speed 623.48 f/s
Best mean reward updated -20.000 -> -19.700, model saved
Traceback (most recent call last):
File "C:/Users/Utilizador/Thesis/Deep Reinforcement Learning Hands-On/Chapter6/02_dqn_pong.py", line 169, in
loss_t = calc_loss(batch, net, tgt_net, device=device)
File "C:/Users/Utilizador/Thesis/Deep Reinforcement Learning Hands-On/Chapter6/02_dqn_pong.py", line 96, in calc_loss
state_action_values = net(states_v).gather(1, actions_v.unsqueeze(-1)).squeeze(-1)
RuntimeError: gather_out_cpu(): Expected dtype int64 for index
Process finished with exit code 1
I'm relatively new to Python and couldn't find any solution to this kind of problem on other sources. Anyone had this same problem or knows how to fix it?

I just found that if you're on newer versions of PyTorch you should define your calc_loss like this:
`def calc_loss(batch, net, tgt_net, device="cpu"):
states, actions, rewards, dones, next_states = batch
states_v = torch.tensor(states).to(device)
next_states_v = torch.tensor(next_states).to(device)
actions_v = torch.tensor(actions).to(device, dtype=torch.int64)
rewards_v = torch.tensor(rewards).to(device)
done_mask = torch.tensor(dones).to(device, dtype=torch.bool)

state_action_values = net(states_v).gather(1, actions_v.unsqueeze(-1)).squeeze(-1)
next_state_values = tgt_net(next_states_v).max(1)[0]
next_state_values[done_mask] = 0.0
next_state_values = next_state_values.detach()

expected_state_action_values = next_state_values * GAMMA + rewards_v
return nn.MSELoss()(state_action_values, expected_state_action_values)`

The only differences between this and the code in the book are the declarations of vars actions_v and done_mask

source: gharchive/issue

license: MIT

license_type: permissive

license_source: github-api


text:
Section 2: Files and content issue

The Files are not uploaded with correct content as explained in video.
Also the correct code to make the REST API work using /post will be this :
@app.post("/post", response_model=UserPost)
async def create_post(post: UserPostIn):
data = post.dict()
last_record_id = len(post_table)
new_post = {**data, "id": last_record_id}
post_table[last_record_id] = new_post
return new_post
Its surprising how it worked in the Video, which is wrong.

Hi @sb1975
We have updated the course.
Happy learning!


text:
tiny和aishell，训练报错

一共两个报错信息，tiny和aishell，报错信息不一样
no.1
tiny：
bash run.sh
============  Configuration Arguments ============
alpha_from: 1.0
alpha_to: 3.2
batch_size: 128
beam_size: 500
beta_from: 0.1
beta_to: 0.45
cutoff_prob: 1.0
cutoff_top_n: 40
error_rate_type: wer
lang_model_path: /root/DeepSpeech/examples/tiny/../..//models/lm/common_crawl_00.prune01111.trie.klm
mean_std_path: data/mean_std.npz
model_path: /root/DeepSpeech/examples/tiny/../..//models/librispeech
num_alphas: 45
num_batches: -1
num_betas: 8
num_conv_layers: 2
num_proc_bsearch: 12
num_rnn_layers: 3
rnn_layer_size: 2048
share_rnn_weights: 1
specgram_type: linear
trainer_count: 8
tune_manifest: data/manifest.dev-clean
use_gpu: 1
use_gru: 0
vocab_path: data/vocab.txt
================================================----
2021-02-18 12:11:32,090-INFO: begin to initialize the external scorer for decoding
2021-02-18 12:11:39,547-INFO: language model: is_character_based = 0, max_order = 5, dict_size = 400000
2021-02-18 12:11:39,620-INFO: end initializing scorer
2021-02-18 12:11:39,621-INFO: start tuning ...
W0218 12:11:45.212179 23717 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 75, Driver API Version: 11.1, Runtime API Version: 10.0
W0218 12:11:45.224212 23717 device_context.cc:260] device: 0, cuDNN Version: 8.0.
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 0 [128/?], current opt (alpha, beta) = (3.200, 0.100),  min [wer] = 0.973516
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 1 [256/?], current opt (alpha, beta) = (3.200, 0.150),  min [wer] = 0.972130
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 2 [384/?], current opt (alpha, beta) = (3.200, 0.150),  min [wer] = 0.975304
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 3 [512/?], current opt (alpha, beta) = (3.200, 0.150),  min [wer] = 0.974737
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 4 [640/?], current opt (alpha, beta) = (3.200, 0.150),  min [wer] = 0.977027
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 5 [768/?], current opt (alpha, beta) = (3.200, 0.150),  min [wer] = 0.976478
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 6 [896/?], current opt (alpha, beta) = (3.200, 0.150),  min [wer] = 0.974615
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 7 [1024/?], current opt (alpha, beta) = (3.200, 0.150),  min [wer] = 0.974513
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 8 [1152/?], current opt (alpha, beta) = (3.200, 0.100),  min [wer] = 0.976047
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 9 [1280/?], current opt (alpha, beta) = (3.200, 0.100),  min [wer] = 0.975835
finish initing model from pretrained params from /root/DeepSpeech/examples/tiny/../..//models/librispeech
....................................................................................................................................................................................
Batch 10 [1408/?], current opt (alpha, beta) = (3.200, 0.100),  min [wer] = 0.975548
./local/run_tune.sh: line 30: 23717 Killed                  CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -u $MAIN_ROOT/tools/tune.py --num_batches=-1 --batch_size=128 --beam_size=500 --num_proc_bsearch=12 --num_conv_layers=2 --num_rnn_layers=3 --rnn_layer_size=2048 --num_alphas=45 --num_betas=8 --alpha_from=1.0 --alpha_to=3.2 --beta_from=0.1 --beta_to=0.45 --cutoff_prob=1.0 --cutoff_top_n=40 --use_gru=False --use_gpu=True --share_rnn_weights=True --tune_manifest="data/manifest.dev-clean" --mean_std_path="data/mean_std.npz" --vocab_path="data/vocab.txt" --model_path="$MAIN_ROOT/models/librispeech" --lang_model_path="$MAIN_ROOT/models/lm/common_crawl_00.prune01111.trie.klm" --error_rate_type="wer" --specgram_type="linear"
Failed in tuning!
============
no.2
aishell：
bash run.sh
Unpacking /root/DeepSpeech/examples/aishell/../..//dataset/aishell/data_aishell/wav/S0598.tar.gz ...
Creating manifest data/manifest ...
============  Configuration Arguments ============
count_threshold: 0
manifest_paths: ['data/manifest.train', 'data/manifest.dev']
vocab_path: data/vocab.txt
================================================----
============  Configuration Arguments ============
manifest_path: data/manifest.train
num_samples: 2000
output_path: data/mean_std.npz
specgram_type: linear
================================================----
Aishell data preparation done.
Download language model ...
--2021-02-18 16:21:42--  https://deepspeech.bj.bcebos.com/zh_lm/zh_giga.no_cna_cmn.prune01244.klm
Resolving deepspeech.bj.bcebos.com (deepspeech.bj.bcebos.com)... 2409:8c00:6c21:10ad:0:ff:b00e:67d, 220.181.33.43, 220.181.33.44
Connecting to deepspeech.bj.bcebos.com (deepspeech.bj.bcebos.com)|2409:8c00:6c21:10ad:0:ff:b00e:67d|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2953395058 (2.8G) [application/octet-stream]
Saving to: './zh_giga.no_cna_cmn.prune01244.klm'
./zh_giga.no_cna_cmn.prune01244. 100%[=======================================================>]   2.75G  27.7MB/s    in 1m 45s
2021-02-18 16:23:28 (26.7 MB/s) - './zh_giga.no_cna_cmn.prune01244.klm' saved [2953395058/2953395058]
Download Aishell model ...
--2021-02-18 16:23:34--  https://deepspeech.bj.bcebos.com/mandarin_models/aishell_model_fluid.tar.gz
Resolving deepspeech.bj.bcebos.com (deepspeech.bj.bcebos.com)... 2409:8c00:6c21:10ad:0:ff:b00e:67d, 220.181.33.44, 220.181.33.43
Connecting to deepspeech.bj.bcebos.com (deepspeech.bj.bcebos.com)|2409:8c00:6c21:10ad:0:ff:b00e:67d|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 227293290 (217M) [application/x-gzip]
Saving to: './aishell_model_fluid.tar.gz'
./aishell_model_fluid.tar.gz     100%[=======================================================>] 216.76M  27.2MB/s    in 12s
2021-02-18 16:23:46 (18.7 MB/s) - './aishell_model_fluid.tar.gz' saved [227293290/227293290]
mean_std.npz
README.md
vocab.txt
params.pdparams
============  Configuration Arguments ============
alpha: 2.6
batch_size: 128
beam_size: 300
beta: 5.0
cutoff_prob: 0.99
cutoff_top_n: 40
decoding_method: ctc_beam_search
error_rate_type: cer
lang_model_path: /root/DeepSpeech/examples/aishell/../..//models/lm/zh_giga.no_cna_cmn.prune01244.klm
mean_std_path: /root/DeepSpeech/examples/aishell/../..//models/aishell/mean_std.npz
model_path: /root/DeepSpeech/examples/aishell/../..//models/aishell
num_conv_layers: 2
num_proc_bsearch: 8
num_rnn_layers: 3
rnn_layer_size: 1024
share_rnn_weights: 0
specgram_type: linear
test_manifest: data/manifest.test
use_gpu: 1
use_gru: 1
vocab_path: /root/DeepSpeech/examples/aishell/../..//models/aishell/vocab.txt
================================================----
2021-02-18 16:23:50,817-INFO: begin to initialize the external scorer for decoding
2021-02-18 16:23:51,138-INFO: language model: is_character_based = 1, max_order = 5, dict_size = 0
2021-02-18 16:23:51,138-INFO: end initializing scorer
2021-02-18 16:23:51,138-INFO: start evaluation ...
W0218 16:23:55.017011  4562 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 75, Driver API Version: 11.1, Runtime API Version: 10.0
W0218 16:23:55.025905  4562 device_context.cc:260] device: 0, cuDNN Version: 8.0.
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (128/?) = 0.055526
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (256/?) = 0.051899
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (384/?) = 0.052061
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (512/?) = 0.056111
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (640/?) = 0.053693
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (768/?) = 0.055531
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (896/?) = 0.056642
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (1024/?) = 0.058581
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (1152/?) = 0.058872
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (1280/?) = 0.059066
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (1408/?) = 0.060236
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (1536/?) = 0.063794
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (1664/?) = 0.067118
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (1792/?) = 0.070420
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (1920/?) = 0.069857
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (2048/?) = 0.068976
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (2176/?) = 0.068703
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (2304/?) = 0.068002
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (2432/?) = 0.066791
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (2560/?) = 0.066410
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (2688/?) = 0.067141
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (2816/?) = 0.067251
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (2944/?) = 0.068165
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (3072/?) = 0.068429
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (3200/?) = 0.068761
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (3328/?) = 0.069398
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (3456/?) = 0.070203
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (3584/?) = 0.071271
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (3712/?) = 0.073296
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (3840/?) = 0.074478
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (3968/?) = 0.076443
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (4096/?) = 0.075874
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (4224/?) = 0.075810
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (4352/?) = 0.075598
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (4480/?) = 0.076835
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (4608/?) = 0.078134
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (4736/?) = 0.079075
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (4864/?) = 0.078314
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (4992/?) = 0.077751
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (5120/?) = 0.077168
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (5248/?) = 0.076967
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (5376/?) = 0.076557
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (5504/?) = 0.075942
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (5632/?) = 0.075414
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (5760/?) = 0.075584
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (5888/?) = 0.076243
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (6016/?) = 0.077038
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (6144/?) = 0.078038
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (6272/?) = 0.079831
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (6400/?) = 0.081442
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (6528/?) = 0.082090
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (6656/?) = 0.081777
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (6784/?) = 0.081388
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (6912/?) = 0.081175
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (7040/?) = 0.080895
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (7168/?) = 0.080448
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Error rate [cer] (7176/?) = 0.080447
Final error rate [cer] (7176/7176) = 0.080447
2021-02-18 16:54:37,277-INFO: finish evaluation
Download language model ...
./zh_giga.no_cna_cmn.prune01244.klm already exists, download skipped.
Download Aishell model ...
./aishell_model_fluid.tar.gz already exists, download skipped.
mean_std.npz
README.md
vocab.txt
params.pdparams
============  Configuration Arguments ============
alpha: 2.6
beam_size: 300
beta: 5.0
cutoff_prob: 0.99
cutoff_top_n: 40
decoding_method: ctc_beam_search
error_rate_type: cer
infer_manifest: data/manifest.test
lang_model_path: /root/DeepSpeech/examples/aishell/../..//models/lm/zh_giga.no_cna_cmn.prune01244.klm
mean_std_path: /root/DeepSpeech/examples/aishell/../..//models/aishell/mean_std.npz
model_path: /root/DeepSpeech/examples/aishell/../..//models/aishell
num_conv_layers: 2
num_proc_bsearch: 8
num_rnn_layers: 3
num_samples: 10
rnn_layer_size: 1024
share_rnn_weights: 0
specgram_type: linear
use_gpu: 0
use_gru: 1
vocab_path: /root/DeepSpeech/examples/aishell/../..//models/aishell/vocab.txt
================================================----
2021-02-18 16:54:45,732-INFO: begin to initialize the external scorer for decoding
2021-02-18 16:54:45,823-INFO: language model: is_character_based = 1, max_order = 5, dict_size = 0
2021-02-18 16:54:45,823-INFO: end initializing scorer
2021-02-18 16:54:45,823-INFO: start inference ...
finish initing model from pretrained params from /root/DeepSpeech/examples/aishell/../..//models/aishell
Target Transcription: 维护正常的市场价格秩序
Output Transcription: 维护正常的市场价格秩序
Current error rate [cer] = 0.000000
Target Transcription: 研究者们选取了一些年龄大的跑者作为研究对象
Output Transcription: 研究者们选取了一些年龄大的跑者作为研究对象
Current error rate [cer] = 0.000000
Target Transcription: 在经济下行压力加大的背景下
Output Transcription: 在经济下行压力加大的背景下
Current error rate [cer] = 0.000000
Target Transcription: 公司已将原有的直销模式改为经销模式
Output Transcription: 公司已将原有的直销模式改为经销模式
Current error rate [cer] = 0.000000
Target Transcription: 记者从首都机场公安分局航站区派出所获悉
Output Transcription: 记者从首都机场公安分局行战区派出所获悉
Current error rate [cer] = 0.105263
Target Transcription: 年龄大跑者的步幅明显短于年轻人
Output Transcription: 年龄大跑者的不服明显短于年轻人
Current error rate [cer] = 0.133333
Target Transcription: 影片将在二零一五年一月在慕尼黑正式开机
Output Transcription: 影片将在二零一五年一月三目一黑正式开机
Current error rate [cer] = 0.157895
Target Transcription: 在市场整体从高速增长进入中高速增长区间的同时
Output Transcription: 在市场整体从高速增长进入中高速增长区间的同时
Current error rate [cer] = 0.000000
Target Transcription: 成功处置深航机上纵火事件
Output Transcription: 成功处置深航机上作火事件
Current error rate [cer] = 0.083333
Target Transcription: 一线城市签约十七万套
Output Transcription: 一线城市签约十七万套
Current error rate [cer] = 0.000000
2021-02-18 16:54:57,788-INFO: finish inference
============  Configuration Arguments ============
augment_conf_path: /root/DeepSpeech/examples/aishell/../..//conf/augmentation.config
batch_size: 64
dev_manifest: data/manifest.dev
init_from_pretrained_model: None
is_local: 1
learning_rate: 0.0005
max_duration: 27.0
mean_std_path: data/mean_std.npz
min_duration: 0.0
num_conv_layers: 2
num_epoch: 50
num_iter_print: 100
num_rnn_layers: 3
num_samples: 120000
output_model_dir: ./checkpoints
rnn_layer_size: 1024
save_epoch: 1
share_rnn_weights: 0
shuffle_method: batch_shuffle_clipped
specgram_type: linear
test_off: 0
train_manifest: data/manifest.train
use_gpu: 1
use_gru: 1
use_sortagrad: 1
vocab_path: data/vocab.txt
================================================----
W0218 16:54:59.143281 14427 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 75, Driver API Version: 11.1, Runtime API Version: 10.0
W0218 16:54:59.152488 14427 device_context.cc:260] device: 0, cuDNN Version: 8.0.
epoch: 0, batch: 0, train loss: 418.396118
epoch: 0, batch: 100, train loss: 62.416603
epoch: 0, batch: 200, train loss: 63.825356
epoch: 0, batch: 300, train loss: 54.420654
epoch: 0, batch: 400, train loss: 37.008553
epoch: 0, batch: 500, train loss: 26.452053
epoch: 0, batch: 600, train loss: 24.803284
epoch: 0, batch: 700, train loss: 23.048244
epoch: 0, batch: 800, train loss: 21.241261
W0218 17:14:01.788877 14487 operator.cc:187] warpctc raises an exception paddle::memory::allocation::BadAlloc,
================================================
C++ Call Stacks (More useful to developers):
0   std::string paddle::platform::GetTraceBackStringstd::string(std::string&&, char const*, int)
1   paddle::memory::allocation::CUDAAllocator::AllocateImpl(unsigned long)
2   paddle::memory::allocation::AlignedAllocator::AllocateImpl(unsigned long)
3   paddle::memory::allocation::AutoGrowthBestFitAllocator::AllocateImpl(unsigned long)
4   paddle::memory::allocation::Allocator::Allocate(unsigned long)
5   paddle::memory::allocation::RetryAllocator::AllocateImpl(unsigned long)
6   paddle::memory::allocation::AllocatorFacade::Alloc(paddle::platform::Place const&, unsigned long)
7   paddle::memory::Alloc(paddle::platform::Place const&, unsigned long)
8   paddle::memory::Alloc(paddle::platform::DeviceContext const&, unsigned long)
9   paddle::framework::Tensor paddle::framework::ExecutionContext::AllocateTmpTensor<float, paddle::platform::CUDADeviceContext>(paddle::framework::DDim const&, paddle::platform::CUDADeviceContext const&) const
10  paddle::operators::WarpCTCFunctorpaddle::platform::CUDADeviceContext::operator()(paddle::framework::ExecutionContext const&, float const*, float*, int const*, int const*, int const*, unsigned long, unsigned long, unsigned long, float*)
11  paddle::operators::WarpCTCKernel<paddle::platform::CUDADeviceContext, float>::Compute(paddle::framework::ExecutionContext const&) const
12  std::_Function_handler<void (paddle::framework::ExecutionContext const&), paddle::framework::OpKernelRegistrarFunctor<paddle::platform::CUDAPlace, false, 0ul, paddle::operators::WarpCTCKernel<paddle::platform::CUDADeviceContext, float> >::operator()(char const*, char const*, int) const::{lambda(paddle::framework::ExecutionContext const&)#1}>::_M_invoke(std::_Any_data const&, paddle::framework::ExecutionContext const&)
13  paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&, paddle::framework::RuntimeContext*) const
14  paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&) const
15  paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, paddle::platform::Place const&)
16  paddle::framework::details::ComputationOpHandle::RunImpl()
17  paddle::framework::details::FastThreadedSSAGraphExecutor::RunOpSync(paddle::framework::details::OpHandleBase*)
18  paddle::framework::details::FastThreadedSSAGraphExecutor::RunOp(paddle::framework::details::OpHandleBase*, std::shared_ptr<paddle::framework::BlockingQueue > const&, unsigned long*)
19  std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result, std::__future_base::_Result_base::_Deleter>, void> >::_M_invoke(std::_Any_data const&)
20  std::__future_base::_State_base::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>&, bool&)
21  ThreadPool::ThreadPool(unsigned long)::{lambda()#1}::operator()() const
========================
Error Message Summary:
ResourceExhaustedError:
Out of memory error on GPU 0. Cannot allocate 142.958496MB memory on GPU 0, available memory is only 24.562500MB.
Please check whether there is any other process using GPU 0.

If yes, please stop them, or start PaddlePaddle on another GPU.
If no, please decrease the batch size of your model.

at (/paddle/paddle/fluid/memory/allocation/cuda_allocator.cc:69)
F0218 17:14:01.791173 14487 exception_holder.h:37] std::exception caught,
================================================
C++ Call Stacks (More useful to developers):
0   std::string paddle::platform::GetTraceBackStringstd::string(std::string&&, char const*, int)
1   paddle::memory::allocation::CUDAAllocator::AllocateImpl(unsigned long)
2   paddle::memory::allocation::AlignedAllocator::AllocateImpl(unsigned long)
3   paddle::memory::allocation::AutoGrowthBestFitAllocator::AllocateImpl(unsigned long)
4   paddle::memory::allocation::Allocator::Allocate(unsigned long)
5   paddle::memory::allocation::RetryAllocator::AllocateImpl(unsigned long)
6   paddle::memory::allocation::AllocatorFacade::Alloc(paddle::platform::Place const&, unsigned long)
7   paddle::memory::Alloc(paddle::platform::Place const&, unsigned long)
8   paddle::memory::Alloc(paddle::platform::DeviceContext const&, unsigned long)
9   paddle::framework::Tensor paddle::framework::ExecutionContext::AllocateTmpTensor<float, paddle::platform::CUDADeviceContext>(paddle::framework::DDim const&, paddle::platform::CUDADeviceContext const&) const
10  paddle::operators::WarpCTCFunctorpaddle::platform::CUDADeviceContext::operator()(paddle::framework::ExecutionContext const&, float const*, float*, int const*, int const*, int const*, unsigned long, unsigned long, unsigned long, float*)
11  paddle::operators::WarpCTCKernel<paddle::platform::CUDADeviceContext, float>::Compute(paddle::framework::ExecutionContext const&) const
12  std::_Function_handler<void (paddle::framework::ExecutionContext const&), paddle::framework::OpKernelRegistrarFunctor<paddle::platform::CUDAPlace, false, 0ul, paddle::operators::WarpCTCKernel<paddle::platform::CUDADeviceContext, float> >::operator()(char const*, char const*, int) const::{lambda(paddle::framework::ExecutionContext const&)#1}>::_M_invoke(std::_Any_data const&, paddle::framework::ExecutionContext const&)
13  paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&, paddle::framework::RuntimeContext*) const
14  paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&) const
15  paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, paddle::platform::Place const&)
16  paddle::framework::details::ComputationOpHandle::RunImpl()
17  paddle::framework::details::FastThreadedSSAGraphExecutor::RunOpSync(paddle::framework::details::OpHandleBase*)
18  paddle::framework::details::FastThreadedSSAGraphExecutor::RunOp(paddle::framework::details::OpHandleBase*, std::shared_ptr<paddle::framework::BlockingQueue > const&, unsigned long*)
19  std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result, std::__future_base::_Result_base::_Deleter>, void> >::_M_invoke(std::_Any_data const&)
20  std::__future_base::_State_base::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>&, bool&)
21  ThreadPool::ThreadPool(unsigned long)::{lambda()#1}::operator()() const
========================
Error Message Summary:
ResourceExhaustedError:
Out of memory error on GPU 0. Cannot allocate 142.958496MB memory on GPU 0, available memory is only 24.562500MB.
Please check whether there is any other process using GPU 0.

If yes, please stop them, or start PaddlePaddle on another GPU.
If no, please decrease the batch size of your model.

at (/paddle/paddle/fluid/memory/allocation/cuda_allocator.cc:69)
*** Check failure stack trace: ***
@     0x7f0a4b55676d  google::LogMessage::Fail()
@     0x7f0a4b55a21c  google::LogMessage::SendToLog()
@     0x7f0a4b556293  google::LogMessage::Flush()
@     0x7f0a4b55b72e  google::LogMessageFatal::~LogMessageFatal()
@     0x7f0a4e6acec8  paddle::framework::details::ExceptionHolder::Catch()
@     0x7f0a4e74808e  paddle::framework::details::FastThreadedSSAGraphExecutor::RunOpSync()
@     0x7f0a4e745a2f  paddle::framework::details::FastThreadedSSAGraphExecutor::RunOp()
@     0x7f0a4e745cf4  _ZNSt17_Function_handlerIFvvESt17reference_wrapperISt12_Bind_simpleIFS1_ISt5_BindIFZN6paddle9framework7details28FastThreadedSSAGraphExecutor10RunOpAsyncEPSt13unordered_mapIPNS6_12OpHandleBaseESt6atomicIiESt4hashISA_ESt8equal_toISA_ESaISt4pairIKSA_SC_EEESA_RKSt10shared_ptrINS5_13BlockingQueueImEEEEUlvE_vEEEvEEEE9_M_invokeERKSt9_Any_data
@     0x7f0a4b5b3c33  std::_Function_handler<>::_M_invoke()
@     0x7f0a4b3b11b7  std::__future_base::_State_base::_M_do_set()
@     0x7f0b0ba3a997  __pthread_once_slow
@     0x7f0a4e741ec2  _ZNSt13__future_base11_Task_stateISt5_BindIFZN6paddle9framework7details28FastThreadedSSAGraphExecutor10RunOpAsyncEPSt13unordered_mapIPNS4_12OpHandleBaseESt6atomicIiESt4hashIS8_ESt8equal_toIS8_ESaISt4pairIKS8_SA_EEES8_RKSt10shared_ptrINS3_13BlockingQueueImEEEEUlvE_vEESaIiEFvvEE6_M_runEv
@     0x7f0a4b3b3614  _ZZN10ThreadPoolC1EmENKUlvE_clEv
@     0x7f0a89db9421  execute_native_thread_routine_compat
@     0x7f0b0ba32fa3  start_thread
@     0x7f0b0b9634df  clone
@              (nil)  (unknown)
./local/run_train.sh: line 33: 14427 Aborted                 CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python3 -u ${MAIN_ROOT}/train.py --batch_size=64 --num_epoch=50 --num_conv_layers=2 --num_rnn_layers=3 --rnn_layer_size=1024 --num_iter_print=100 --save_epoch=1 --num_samples=120000 --learning_rate=5e-4 --max_duration=27.0 --min_duration=0.0 --test_off=False --use_sortagrad=True --use_gru=True --use_gpu=True --is_local=True --share_rnn_weights=False --train_manifest="data/manifest.train" --dev_manifest="data/manifest.dev" --mean_std_path="data/mean_std.npz" --vocab_path="data/vocab.txt" --output_model_dir="./checkpoints" --augment_conf_path="${MAIN_ROOT}/conf/augmentation.config" --specgram_type="linear" --shuffle_method="batch_shuffle_clipped"
Failed in training!
Download language model ...
./zh_giga.no_cna_cmn.prune01244.klm already exists, download skipped.
============  Configuration Arguments ============
alpha: 2.6
batch_size: 128
beam_size: 300
beta: 5.0
cutoff_prob: 0.99
cutoff_top_n: 40
decoding_method: ctc_beam_search
error_rate_type: cer
lang_model_path: /root/DeepSpeech/examples/aishell/../..//models/lm/zh_giga.no_cna_cmn.prune01244.klm
mean_std_path: data/mean_std.npz
model_path: checkpoints/step_final
num_conv_layers: 2
num_proc_bsearch: 8
num_rnn_layers: 3
rnn_layer_size: 1024
share_rnn_weights: 0
specgram_type: linear
test_manifest: data/manifest.test
use_gpu: 1
use_gru: 1
vocab_path: data/vocab.txt
================================================----
2021-02-18 17:14:08,242-INFO: begin to initialize the external scorer for decoding
2021-02-18 17:14:08,423-INFO: language model: is_character_based = 1, max_order = 5, dict_size = 0
2021-02-18 17:14:08,423-INFO: end initializing scorer
2021-02-18 17:14:08,423-INFO: start evaluation ...
W0218 17:14:09.493114 19569 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 75, Driver API Version: 11.1, Runtime API Version: 10.0
W0218 17:14:09.498034 19569 device_context.cc:260] device: 0, cuDNN Version: 8.0.
checkpoints/step_final
Traceback (most recent call last):
File "/root/DeepSpeech/examples/aishell/../..//test.py", line 159, in
main()
File "/root/DeepSpeech/examples/aishell/../..//test.py", line 155, in main
evaluate()
File "/root/DeepSpeech/examples/aishell/../..//test.py", line 123, in evaluate
feeding_dict=data_generator.feeding)
File "/root/DeepSpeech/model_utils/model.py", line 420, in infer_batch_probs
self.init_from_pretrained_model(exe, infer_program)
File "/root/DeepSpeech/model_utils/model.py", line 164, in init_from_pretrained_model
raise Warning("The pretrained params do not exist.")
Warning: The pretrained params do not exist.
Failed in evaluation!
Download language model ...
./zh_giga.no_cna_cmn.prune01244.klm already exists, download skipped.
============  Configuration Arguments ============
alpha: 2.6
beam_size: 300
beta: 5.0
cutoff_prob: 0.99
cutoff_top_n: 40
decoding_method: ctc_beam_search
error_rate_type: cer
infer_manifest: data/manifest.test
lang_model_path: /root/DeepSpeech/examples/aishell/../..//models/lm/zh_giga.no_cna_cmn.prune01244.klm
mean_std_path: data/mean_std.npz
model_path: checkpoints/step_final
num_conv_layers: 2
num_proc_bsearch: 8
num_rnn_layers: 3
num_samples: 10
rnn_layer_size: 1024
share_rnn_weights: 0
specgram_type: linear
use_gpu: 1
use_gru: 1
vocab_path: data/vocab.txt
================================================----
2021-02-18 17:14:16,196-INFO: begin to initialize the external scorer for decoding
2021-02-18 17:14:16,292-INFO: language model: is_character_based = 1, max_order = 5, dict_size = 0
2021-02-18 17:14:16,292-INFO: end initializing scorer
2021-02-18 17:14:16,292-INFO: start inference ...
W0218 17:14:16.331327 19689 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 75, Driver API Version: 11.1, Runtime API Version: 10.0
W0218 17:14:16.332828 19689 device_context.cc:260] device: 0, cuDNN Version: 8.0.
checkpoints/step_final
Traceback (most recent call last):
File "/root/DeepSpeech/examples/aishell/../..//infer.py", line 159, in
main()
File "/root/DeepSpeech/examples/aishell/../..//infer.py", line 155, in main
infer()
File "/root/DeepSpeech/examples/aishell/../..//infer.py", line 131, in infer
feeding_dict=data_generator.feeding)
File "/root/DeepSpeech/model_utils/model.py", line 420, in infer_batch_probs
self.init_from_pretrained_model(exe, infer_program)
File "/root/DeepSpeech/model_utils/model.py", line 164, in init_from_pretrained_model
raise Warning("The pretrained params do not exist.")
Warning: The pretrained params do not exist.
Failed in inference!

麻烦确保run.sh的脚本每一步都执行正确后再执行下一步，目前还没有做错误check，run.sh只是流程实例，可以分开执行的。

第一个问题是 ./local/run_tune.sh 可能占用大量的内存 OOM 了，或者是显存不够等。

第二个是训练的时候显存不够了
W0218 17:14:01.788877 14487 operator.cc:187] warpctc raises an exception paddle::memory::allocation::BadAlloc
Out of memory error on GPU 0. Cannot allocate 142.958496MB memory on GPU 0, available memory is only 24.562500MB

显存每次都是占用1-2G，就报错了
显存一共6G
我需要在哪里调整哪些参数吗
@zh794390558

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.45.01    Driver Version: 455.45.01    CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 2060    Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   57C    P2    32W /  N/A |   1062MiB /  5934MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      3329      C   python3                          1059MiB |
+-----------------------------------------------------------------------------+

@zh794390558

已经解决了，减小--batch_size即可

嗯嗯，一般显存爆的化，不该模型就只能减小batch了。

repo: PaddlePaddle/DeepSpeech


text:
图像分类针对未知分类问题如何解决？

比如图像分类中的二分类，假设我只想区分某张图片中是否有火车（有火车就被标记为0类，没火车就被标记为1类）；
现实生活中的不包含火车的图片包罗万象，不可能都被拿来作为训练集；
如果测试集中出现了未知的物体，但一定不是火车，如果更准确的把这张图片推理为1类呢？
训练数据或定义标签时可否定义一个blank来解决这个问题呢？
望各位大神慷慨解答

可以使用PP-ShiTu，将火车数据建立一个特征库，进行识别

repo: PaddlePaddle/PaddleClas


text:
paddlex 训练好的模型可以直接用于Paddle.js吗？

将训练好的模型放到Paddle.js上使用，请问该如何处理？

请问你需要使用的是什么模型

MobilenetV2

MobileNetV2目前可以支持部署到Paddle.js，但在导出时需要注意一下
1. 如果你是使用PaddleX GUI训练模型

请在工作目录中找到项目所在的路径，从output目录下，找到各轮训练保存的模型，例如epoch_10或best_model目录
在本地使用pip install paddlex， 通过paddlex如下命令导出部署模型
paddlex --export_inference --model_dir output/best_model --fixed_input_shape 224,224 --save_dir ./infer_model

如果你是在Python环境使用PaddleX API训练的模型

直接使用如下命令导出部署模型
``paddlex --export_inference --model_dir output/best_model --fixed_input_shape 224,224 --save_dir ./infer_model`

上面命令中 --fixed_input_shape为训练时指定图像预处理完成后的输入大小，具体根据自己的情况设置。
Paddle.js使用请参考Paddle.js文档 https://github.com/PaddlePaddle/Paddle.js/blob/master/README_cn.md

感谢

repo: PaddlePaddle/PaddleX


text:
【Docs】Markdown 格式的文档，表格和标题之间需要加空行

一、背景
飞桨官网的使用指南中，存在表格呈现形式错误问题。
例如：

Markdown 格式的文档储存在GitHub中的 docs 仓库下，具体路径为 PaddlePaddle/docs/docs/guides

每一个表格在Markdown 格式的文档中都有对应的描述

二、原因分析
Markdown 格式的文档中表格和标题之间缺失空行，官网渲染问题。
例如：

三、解决方法
在表格和标题之间的位置添加空行。
例如：

如上，添加空行后，渲染成功，表格形式正确。

已修复

repo: PaddlePaddle/docs


text:
add custom ccl docs

add custom ccl docs
http://10.136.157.23:8090/documentation/docs/zh/dev_guides/custom_device_docs/custom_runtime_cn.html?reviewVersion=jenkins-doc-review-2-557

文档风格被卡住了，需要用pre-commit修正一下

source: gharchive/pull-request


text:
add cn doc for prior_box

add CN doc for paddle.vision.ops.prior_box
paddle pr : https://github.com/PaddlePaddle/Paddle/pull/46398

paddle.vision.ops.prior_box 检查一下英文文档，中文这两个问题是英文文档生成的问题，英文是因为py源代码的问题

paddle.vision.ops.prior_box 检查一下英文文档，中文这两个问题是英文文档生成的问题，英文是因为py源代码的问题

done


text:
can i use the database via localhost?

Can I use the database through my Hosting's localhost? And how do I make them compatible with your source?

You can set up your own DB for free, check out the README for instructions: https://github.com/PaeP3nguin/honkai-guide#setting-up-cloud-firestore


text:
Links broken when viewing directly in GitHub.

Viewing the docs directly in GitHub results in broken links. e.g. https://github.com/PagerDuty/incident-response-docs/blob/master/docs/getting_started.md
I ran into issues using relative links, so changed to absolute links. Unfortunately this breaks viewing the docs in GitHub. I found it hard to get the links to work in GitHub and on the rendered website at the same time. I'd fix one, and the other would break. For now, I focussed on making sure the rendered website at https://response.pagerduty.com works properly.
This issue is to find a way to get the GitHub links working again at the same time as the website.

Changing this to a "Won't Fix". We'll ensure that the website always works, but having it work natively in GitHub will be considered a "Nice to have" feature rather than a requirement.
We'll fix whatever issues we can, but we aren't going to invest significant effort to get it to work in both places.

repo: PagerDuty/incident-response-docs


text:
docker compose quick-start demo

DRAFT
#!/bin/bash
git clone git@github.com:PaimaStudios/paima-game-templates.git paima-quickstart
cd paima-quickstart
git checkout feature/quick-start
curl -L -O https://github.com/PaimaStudios/paima-engine/releases/download/v1.0.0/paima-engine-linux
cd chess
./quick-start.sh
# open http://localhost:9000

I would put all of this in @paima/build-utils so that it can be used from any game (and not just chess) and people can use it from a simple npx command

authors: SebastienGllmt acedward

repo: PaimaStudios/paima-game-templates


text:
commit-file.yml

file.yml to tests

is test


text:
UTIL type=ALL - loading dynamic Addressgroup with filter, where tag name has characters '(' or ')' in

Describe the bug

<entry name="dummy(text)">
  <color>color42</color>
</entry>

<entry name="addrDUMMY">
  <dynamic>
    <filter>'dummy(text)' or 'DEMO'</filter>
  </dynamic>
</entry>

Expected behavior

handle DAG with filter of tag, where name include characters like '(' or ')' correctly
Current behavior

DAG with this tag filter are not correctly checked for their members

fixed with PR:
#561

repo: PaloAltoNetworks/pan-os-php

license: ISC


text:
您好，请问下你线上的预览的这个是怎么打包的，我也想把模拟数据打包进去

在打包的时候我也想把mock 的数据打包进去 因为我们这边的接口还没有开发完成但是又想看效果，所以想把mock 的数据一起打包进去 然后build 发布到网络上。可是我打包了好多次都没打包成功，打包成功之后点击登录 没有反应。

你是本地预览的把，需要自己本地启一个服务

自己的服务器？是打包完之后把dist的文件夹挂上的那种服务器么？

本地启一个服务。 如 serve 这种库。

抱歉，我没明白你的意思，如果服务器的话  我这边是用iis 的，但是这个和打包完之后的有什么关联么？我是想把我的mock 数据 和项目文件一起打包发布到网上 不止给本地的人预览 还要能给线上的人预览的那种。

您的意思是把mock数据在本地部署一个服务器？

你回答了你本地登录不了的原因，如果是 localhost的文件不能访问cookie，所以登录不了。
mock的文档，文档写的很清楚了。 可以使用mockjs 纯前端mock 不需要任何后端服务。或者用easy-mock或者自己架一个mock服务。
请先自己看文档。

我的所有mock数据都是本地的mock.js 创建出来的，那如果不是mock 数据的问题  那就是因为localhost 的文件访问不了cookie 所以登录不了 的原因咯？那我需要 或者应该怎么修改呢？ 谢谢您！

https://github.com/PanJiaChen/vue-element-admin/issues/1030

谢谢 谢谢 谢谢！万分感谢！感谢您花时间回答了我的问题！

authors: Heendy PanJiaChen

repo: PanJiaChen/vue-admin-template


text:
Plans to support Forge?

Curious if this is something you're working on. I may be able to chip away at it.
I would also appreciate any pointers if there is already a forge mod that does this for 1.18 that I'm missing!

Sorry, but I have too many problems with loading the same mixins on Forge. That's why I decided not to release the Forge version. You can use the shared part and try to create it yourself, I allow.

repo: PanSzelescik/proxy-protocol-support


text:
🛑 Pandaily is down

In 8a18815, Pandaily (https://pandaily.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Pandaily is back up in 9c9c548.

repo: PandaDaily/uptime


text:
NBT data loss for items on 1.20.1

I have a datapack that allows for smithing templates to be applied to tools. With specifically Paper (I am on 1.20.1-69) the NBT data is semi-randomly cleared.
Steps I took to replicate:

booted fresh 1.20.1-69 server on local
installed datapack to allow for tools to have armor trim applied (nothing else, no plugins, no config changes, etc)
restarted server
gave myself op, changed gamemode to creative, gave myself needed items for recipe
apply template to tool
quickly randomly move tool around inventory/rapidly click on it (if its slow enough it doesnt seem to happen)
NBT data for trim is lost.

Notes:

I was not able to replicate this on a vanilla local server, or singleplayer
I also tested it in survival and experienced the same NBT data loss
all tests were done with a vanilla client (resource packs only)
other NBT data (for enchants, names, etc) is not lost

Attached (redacted) log and datapack.
dp-trimmable-tools-mc1.20-v1.0.4.zip
latest.log

Issue created in docs instead of development, issue closing and creating in correct location

repo: PaperMC/docs


text:
loading  configuration got stuck at 30%

Expected Behavior
Able to access to the settings of web ui.
Current Behavior
Ui got stuck with message: Loading Configuration 30%
can't navigate to Settings section.
Steps to Reproduce
Not clear. I've enabled several options. But since I can't reach to settings, I have no idea what's changed.
Possible Solution
Nope
Already downloaded lot's of bytes. How can revert last changes?
Context
btw SPDisplaysDataType Metal Support result is  Metal 3.
system_profiler SPDisplaysDataType
Graphics/Displays:

    Apple M3 Max:

      Chipset Model: Apple M3 Max
      Type: GPU
      Bus: Built-In
      Total Number of Cores: 30
      Vendor: Apple (0x106b)
      Metal Support: Metal 3
      Displays:
        Color LCD:
          Display Type: Built-in Liquid Retina XDR Display
          Resolution: 3024 x 1964 Retina
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Automatically Adjust Brightness: Yes
          Connection Type: Internal

Here is pip list command result:
└─▪ pip list
\Package             Version     Editable project location
------------------- ----------- ------------------------------------------------------------------------
accelerate          1.2.1
aiofiles            24.1.0
aiohappyeyeballs    2.4.4
aiohttp             3.11.10
aiosignal           1.3.2
annotated-types     0.7.0
anyio               4.7.0
ascii-colors        0.4.2
attrs               24.3.0
autopep8            2.3.1
beautifulsoup4      4.12.3
bidict              0.23.1
bs4                 0.0.2
certifi             2024.12.14
cffi                1.17.1
charset-normalizer  3.4.0
click               8.1.7
colorama            0.4.6
colorclass          2.2.2
compressed-rtf      1.0.6
cryptography        44.0.0
defusedxml          0.7.1
diffusers           0.31.0
distro              1.9.0
dpkt                1.9.8
easygui             0.98.3
ebcdic              1.1.1
et_xmlfile          2.0.0
extract-msg         0.52.0
fastapi             0.115.6
filelock            3.16.1
freedom_search      0.2.0
frozenlist          1.5.0
fsspec              2024.10.0
gitdb               4.0.11
GitPython           3.1.43
googlesearch-python 1.2.5
h11                 0.14.0
httpcore            1.0.7
httpx               0.28.1
huggingface-hub     0.26.5
idna                3.10
importlib_metadata  8.5.0
iniconfig           2.0.0
Jinja2              3.1.4
jiter               0.8.2
joblib              1.4.2
lark                1.1.9
llvmlite            0.43.0
lollms              10.1.0      /Users/kunthar/development/brainz/lollama-webui/lollms-webui/lollms_core
lollms_client       0.7.7
lollmsvectordb      1.2.8
lxml                5.3.0
markdown-it-py      3.0.0
MarkupSafe          3.0.2
mdurl               0.1.2
more-itertools      10.5.0
mpmath              1.3.0
msoffcrypto-tool    5.4.2
multidict           6.1.0
networkx            3.4.2
numba               0.60.0
numpy               1.26.3
olefile             0.47
oletools            0.60.2
openai              1.57.4
openai-whisper      20240930
opencv-python       4.10.0.84
openpyxl            3.1.5
outcome             1.3.0.post0
packaging           24.2
pandas              2.2.3
pcodedmp            1.2.6
pillow              11.0.0
pip                 24.3.1
pipmaster           0.4.0
pluggy              1.5.0
propcache           0.2.1
protobuf            5.29.1
psutil              6.1.0
pycodestyle         2.12.1
pycparser           2.22
pydantic            2.10.3
pydantic_core       2.27.1
Pygments            2.18.0
pyparsing           3.2.0
PyPDF2              3.0.1
PyQt5               5.15.11
PyQt5-Qt5           5.15.15
PyQt5_sip           12.16.1
PySocks             1.7.1
pytest              8.3.4
python-dateutil     2.9.0.post0
python-docx         1.1.2
python-engineio     4.10.1
python-multipart    0.0.19
python-socketio     5.11.4
pytz                2024.2
PyYAML              6.0.2
red-black-tree-mod  1.20
regex               2024.11.6
requests            2.32.3
rich                13.9.4
RTFDE               0.1.2
safetensors         0.4.5
scikit-learn        1.6.0
scipy               1.14.1
scrapemaster        0.2.2
selenium            4.27.1
sentencepiece       0.2.0
setuptools          70.2.0
simple-websocket    1.1.0
six                 1.17.0
smmap               5.0.1
sniffio             1.3.1
sortedcontainers    2.4.0
sounddevice         0.5.1
soupsieve           2.6
starlette           0.41.3
sympy               1.13.1
threadpoolctl       3.5.0
tiktoken            0.8.0
tokenizers          0.21.0
torch               2.5.1
torchaudio          2.5.1
torchvision         0.20.1
tqdm                4.67.1
transformers        4.47.0
trio                0.27.0
trio-websocket      0.11.1
typing_extensions   4.12.2
tzdata              2024.2
tzlocal             5.2
urllib3             2.2.3
uvicorn             0.34.0
Wave                0.0.2
websocket-client    1.8.0
wget                3.2
wheel               0.45.1
wsproto             1.2.0
yarl                1.18.3
zipfile36           0.1.3
zipp                3.21.0

Screenshots

If applicable, add screenshots to help explain the issue.

Found the root cause.
brainz/lollama-webui/personal_data/discussion_databases/

moved folder 2 and database.db to force recreate.
application recreated and run the rest smoothly.
it seems there is a db column creation or write error. I'll close this because it was random.

Can't reproduce closing.

repo: ParisNeo/lollms-webui


text:
Minor code clean ups

Depends on #336.

Thank you @Kijewski!

authors: Kijewski ParkMyCar


text:
Load bundle before resgistering subclasses

Intended to address https://github.com/ParsePlatform/Parse-SDK-iOS-OSX/issues/1006

Closing this because of reports that it does not solve the problem.

repo: ParsePlatform/Parse-SDK-iOS-OSX

license_source: bigquery


text:
Clients aren't allowed to perform the create operation on the role collection. (Code: 119, Version: 1.12.0)

Hello Team,
I am struggling with an issue which is related with PFRole Parse SDK Ver 1.12.0.
I have a parse app running on Parse server with iOS SDK 1.7.5 on PRODUCTION and in that app we are creating role from iOS app directly using PFRole, which is working fine.
But after migrating my Parse cloud app to Amazon + Mango DB and changed the Client iOS SDK to 1.12.0, I am getting issue and unable to create the PFRole.
The same code was working properly with iOS SDK 1.7.5. Will you please help me out to fix this issue

Did you check your class level permissions?

Request you to please more explain about class level permission. What I need to change in schema

hi @hramos
here is my class level permission (From MongoDB )for the Role-
{
"_id": "_Role",
"_metadata": {
"class_permissions": {
"get": {
"": true
},
"find": {
"": true
},
"update": {
"": true
},
"create": {
"": true
},
"delete": {
"": true
},
"addField": {
"": true
},
"readUserFields": [],
"writeUserFields": []
}
},
"role": "string",
"roles": "relation<_Role>",
"users": "relation<_User>",
"name": "string",
"organizationId": "string"
}
And from my iOS application I am trying to add the role like below:
    PFACL *roleACL = [PFACL ACL];
    [roleACL setPublicReadAccess:YES];
    [roleACL setPublicWriteAccess:YES];

    PFRole *roleAdmin = [PFRole roleWithName:roleNameAdmin acl:roleACL];
    [roleAdmin.users addObject:currentUser];
    [roleAdmin setValue:@"admin" forKey:@"role"];
    [roleAdmin setValue:orgId forKey:@"organizationId"];

will you please guide me what I am doing wring at Class level Permission.

Hi @hramos
Any guess ??

+1
I have the same problem here. I tried to find where this error message is coming from but I didn't find this string in parse-server or in the iOS SDK source code. Why does this error even exist? What's the reason behind not allowing us to query roles from the client app? This is a necessity in my application.

Oh here it is https://github.com/ParsePlatform/parse-server/blob/d33dd68cc521e63be68a93ab1a00cfdb7a7d243d/src/rest.js#L118
What is the point of enforeRoleSecurity? I don't want to enforce role security.

Oh wait I think I don't have the latest version... this was actually removed here https://github.com/ParsePlatform/parse-server/commit/dd7fad8e96c93b933d4515fc2dc5c7aebaaff115#diff-62e69722264e5a578707565ff5fcf41c

We have the same issue. Application gets:

Clients aren't allowed to perform the find operation on the installation collection. (Code: 119, Version: 1.12.0)

@drorsun Update your parse-server version.

@xissburg I am on version 2.2.7 which is the latest

I am having the same issue as @drorsun and am also on v2.2.7.
PFInstallation.currentInstallation().fetchInBackgroundWithBlock {......   returns error code 119.

Check what your have in enforeRoleSecurity at node_modules/parse-server/lib/rest.js

@xissburg the function is:
// Disallowing access to the _Role collection except by master key
function enforceRoleSecurity(method, className, auth) {
if (className === '_Installation' && !auth.isMaster) {
if (method === 'delete' || method === 'find') {
var error = 'Clients aren't allowed to perform the ' + method + ' operation on the installation collection.';
throw new Parse.Error(Parse.Error.OPERATION_FORBIDDEN, error);
}
}
}

@xissburg I also have the same code as @brianyyz does

Guys,
I updated my Parse-Server version and also the latest iOS SDK.
I setup my installation locally and now everything is working fine the issue is resolved

authors: brianyyz drorsun hclviv hramos xissburg

repo: ParsePlatform/parse-server


text:
Password reset works, but email verification does not work

Hi,
I successfully migrate Parse server to Herouku and for email verification and password reset I am using Mailgun.
I managed to get email for password verification with link to reset password and it is working fine, but I do not get email for email verification.
What I could do about this?
Here is code: http://pastebin.com/yNQ0z7Mn

I am having the same problem. Did you manage to find a solution on that. I have been trying to find a solution for almost a month without luck.

authors: Zookey zirinisp


text:
How to change maxUploadSize in Azure ?

I want to change maxUploadSize , my prase server is in Azure
but I can't find FilesRouter.js file
I has asked in Stackoverflow : here

thanks for asking in stack overflow.  I'm going to close this issue here.


text:
Posting file pointers to REST API

Hey. We're transitioning onto parse-server. We had old code POSTing images to the REST API, immediately followed by POSTing data to another collection, with a pointer to that earlier image file.
When we were fully on Parse, before using parse-server, we supplied data like this:
'imageObject': {
  'name': data.name,
  '__type': "File"
}

where only a name and a __type were necessary to define that relationship.
Once we moved to parse-server, this code was breaking. It boiled down to this line which also looks for a url. We needed to change out the data we're POSTing to something like this:
'imageObject': {
  'name': data.name,
  'url': data.url,
  '__type': "File"
}

Maybe this information could be provided in migration docs, since we didn't see it there.
It might also be worth mentioning that we're now using the S3Adapter to store our newly-created files in a bucket.
Thanks for everything! Overall, this migration is going far smoother than we could've expected.
cc @kyleseaman

I think we could probably just remove the check for the URL. @nlutsenko thoughts?

Files are required to have a url at the moment of submitting them via REST API.
There are a lot of things that could go wrong when this is not true, like storage mechanisms or beforeSave/afterSave hooks. This is true for every single endpoint and is specified in the REST API guide.
All this comes down to the fact that file uniqueness is not enforced by name, but it is by url, meaning that all files can have duplicate names when they are stored.
This said, we could remove the limitation here, but I would actually recommend moving your file pointers to use a different system of pointers for files, or sticking a URL there might also be the case.

How would their code have worked on Parse.com then? Would the pointer have just referenced the most recent file with that name?

Actually, I take it back - we do enforce uniqueness of files by name, so we might as well remove this restriction. And if we can't serve the file from a URL - then, yeah, that actually is absolutely fine, but to match it closer to api.parse.com - going to remove the restriction.

authors: brandly drew-gross nlutsenko


text:
Client.py - KafkaClient, source ip address and port argument

Can KafkaClient in client.py be enhanced to take a source ip address and port argument. https://github.com/Parsely/pykafka/blob/master/pykafka/client.py#L38
If i am not wrong, KafkaClient constructor creates a tcp socket internally using socket.create_connection which supports source ip address/port argument (since python 2.7)
This will be greatly helpful when running consumers on public servers where we don't prefer to keep many ports open.

This seems reasonable to me. It doesn't really affect any users who don't want to set it, and it's helpful for those who do.

Pull requested here https://github.com/Parsely/pykafka/pull/221

repo: Parsely/pykafka


text:
Enforce that user passes a working_dir if it is needed

Reported by @mgovoni-devel.
In this case, Marco did not have working_dir set for the relevant executor. But that defaults to working_dir=None. If working_dir indeed a requirement, it shouldn't be a keyword argument. We need to think a bit about how to implement this, because if it is only a requirement when storage_access != None, then we should probably make it part of the Scheme specification. I want to avoid making it a requirement for users that don't need it (i.e., ones that are not using staging.)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-dc2b9795abfc> in <module>()
      1 ## RUN THE APP ON MIDWAY
      2 qb_input = []
----> 3 qb_input.append(File("globus://"+data["midway"]["globus_uuid"]+data["midway"]["globus_path"]+"qb.in"))

~/parsl/parsl/data_provider/files.py in __init__(self, url, dman, cache, caching_dir)
     43         self.data_future = {}
     44         if self.scheme == 'globus':
---> 45             self.dman.add_file(self)
     46
     47         self.cache = cache

~/parsl/parsl/data_provider/data_manager.py in add_file(self, file)
     77             # keep a list of all remote files for optimization purposes (TODO)
     78             self.files.append(file)
---> 79             self._set_local_path(file)
     80
     81     def _set_local_path(self, file):

~/parsl/parsl/data_provider/data_manager.py in _set_local_path(self, file)
     80
     81     def _set_local_path(self, file):
---> 82         globus_ep = self._get_globus_endpoint()
     83         file.local_path = os.path.join(globus_ep['working_dir'], file.filename)
     84

~/parsl/parsl/data_provider/data_manager.py in _get_globus_endpoint(self, executor_label)
     88                 for scheme in executor.storage_access:
     89                     if isinstance(scheme, GlobusScheme):
---> 90                         working_dir = os.path.normpath(executor.working_dir)
     91                         if scheme.endpoint_path and scheme.local_path:
     92                             endpoint_path = os.path.normpath(scheme.endpoint_path)

~/.conda/envs/parsl_py36/lib/python3.6/posixpath.py in normpath(path)
    331 def normpath(path):
    332     """Normalize path, eliminating double slashes, etc."""
--> 333     path = os.fspath(path)
    334     if isinstance(path, bytes):
    335         sep = b'/'

TypeError: expected str, bytes or os.PathLike object, not NoneType

@lukaszlacinski, what do you think? Does it make sense to move this to the Scheme?

PR #789 makes working_dir mandatory when using the globus scheme - it throws an exception at staging time, rather than at config/initialisation time. It would be nicer to throw a config-time error, perhaps.

authors: annawoodard benclifford

repo: Parsl/parsl


text:
BUG: Scroll To Top Button Visibility

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
The scroll to top button in the (/companies page)  should not be visible when we are at the top of the page, it should be rendered when we are not at the top of the page I can fix this and also make the ui of the button better.

Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@PatilHarshh can u assign me this?

@PatilHarshh can u review it and assign this?

@PatilHarshh pls review it!!

repo: PatilHarshh/Kaam-Do


text:
Upgraded the projects to .NET 8 and included an extra argument in the…

Removed both obj and bin folders from the repository, as a cleanup measure.
Updated the projects' settings to have the .NET 8 framework as target.
Included a new parameter named replyTo in the method sendEmail_Ext, allowing to fill in the Message.ReplyToList field with the provided email addresses.

Changes have been merged to build number 20241116065550. Thanks!

authors: PaulHoOutsystems jnamartins


text:
Producing meta modules for packages like wrf and roms

This PR is currently a draft as I am testing ideas on generating modules that load all the dependencies need by a package.

Thanks Pascal, I will review as soon as I am back to work.
One thing that I wanted to share with you on the wrf/roms meta-modules, which is actually just a heads up, because I think your current implementation is fine with that.
The strings wrf-dependencies and roms-dependencies may negatively interact with the string filtering that happens in the module template:
https://github.com/PawseySC/pawsey-spack-config/blob/main/setonix/templates_setonix/modules/modulefile.lua#L73
However, yout current implementation is OK, because you have a .lua extension, wrf-dependencies.lua, whereas the filter strips out dependencies/, with a final slash.

Thanks @marcodelapierre  for the point. I had thought of that and wanted to avoid dependencies but that is what the module provides.

Oh, one extra though on this, sorry my brain is slower than usual today. It might get tricky in case we need to change the naming convention for these modules in the future, e.g. from <tool>-dependencies.lua to <tool>-dependencies/<version>/module.lua, to account for versions. then, the filter will play dirty tricks.
Then, one alternative I was thinking of is <tool>-dependency-set, because in general it will provide a set of dependency.

So now the module produced is something like wrf-dependency-set/<version>/module.lua
-- -*- lua -*-
-- Module file created by Pawsey
--

whatis([[Name : wrf-dendencies]])
whatis([[Package version and spec used to generate deps:   wrf@4.2%gcc@10.3.0 cflags="-O3" cppflags="-O3" fflags="-O3 -fallow-argument-mismatch" +pnetcd
f build_type=dmpar compile_type=em_real nesting=basic patches=01c0f71a4c742603ca6418977bdc71c8c5ccf882e54e6c193ec2adaf98e31511,0b093750c1344ecb15061ad23
634055a39cc79698a396bf6f6c015a6c856c823,68548f61600d8503bf14a1ab8b90e0e20375fe9466fa1e205b8d835174fcdbef,b3f063c60c61ac6655f91cbb266e3a0dafc06449ebb701c
b56541c5e9bff65e9,b3fc8bb1e6924f9441e746d0de7dc2b2421b064272a6cc33e8cf8ce8c1ad20a9,c47ec9cba9b7396186801f9eaae794fe6db53570486095d1bda4f58e327b4eec,d019
b9605304d5728d6eceda24b1ef4398227fd978f3c1b32a91fbc4f5069e8d,e07c39c644dab0cad373e4ff3809db22128b376c139416d9e774b96e1f86e7e7,f3dd50d3de8dc26d12ab5bceea
a80ec05a722834dde97107d7bb194c9f67045a,f85f29a41d535b1ca612bcf8217a3ea4975fb0b95dcd69e2bf0b26605795261c arch=cray-sles15-zen2 ]])

whatis([[Short description : Meta module to load dependencies to build wrf. ]])
help([[Meta module that loads all dependencies need to build local wrf]])

-- TODO: figure out way of spack generating dependencies
-- currently would be to make use of the spack.lock
-- in the related environment.
load("dependencies/.cray-mpich/7.7.0-rpq4ivk/module")
load("libraries/hdf5/1.12.1-parallel-api-v112/module")
load("utilities/cmake/3.21.4/module")
load("dependencies/.ncurses/6.1-f5xbinw/module")
load("dependencies/.openssl/1.1.0i-fips-kl6ezjb/module")
load("dependencies/.libaec/1.0.5-wa6rg3n/module")
load("dependencies/.numactl/2.0.11-tinrgvs/module")
load("dependencies/.pkg-config/0.29.2-3tlf6jx/module")
load("dependencies/.zlib/1.2.11-fhca6le/module")
load("dependencies/.jasper/2.0.32-dfek24i/module")
load("dependencies/.libjpeg-turbo/2.1.0-aygwfi2/module")
load("dependencies/.nasm/2.15.05-4ooqntw/module")
load("dependencies/.libpng/1.6.37-6cux2z5/module")
load("dependencies/.libtirpc/1.2.6-tl666wp/module")
load("dependencies/.krb5/1.19.2-5lln5yk/module")
load("dependencies/.libtool/2.4.6-jlvb3j2/module")
load("libraries/netcdf-c/4.8.1/module")
load("dependencies/.hdf/4.2.15-2zux5tl/module")
load("libraries/parallel-netcdf/1.12.2/module")
load("programming-languages/perl/5.34.0/module")
load("dependencies/.berkeley-db/18.1.40-ihkuhwv/module")
load("dependencies/.bzip2/1.0.6-t34oqcl/module")
load("dependencies/.gdbm/1.19-72vdi62/module")
load("dependencies/.readline/8.1-pdzbirz/module")
load("libraries/netcdf-fortran/4.5.3/module")
load("dependencies/.tcsh/6.22.02-d7xyudn/module")

Just now need to strip the key words from the lua file.

Now
-- -*- lua -*-
-- Module file created by Pawsey
--

whatis([[Name : wrf-dependency-set]])
whatis([[Package version and spec used to generate deps:   wrf@4.2%gcc@10.3.0 cflags="-O3" cppflags="-O3" fflags="-O3 -fallow-argument-mismatch" +pnetcd
f build_type=dmpar compile_type=em_real nesting=basic patches=01c0f71a4c742603ca6418977bdc71c8c5ccf882e54e6c193ec2adaf98e31511,0b093750c1344ecb15061ad23
634055a39cc79698a396bf6f6c015a6c856c823,68548f61600d8503bf14a1ab8b90e0e20375fe9466fa1e205b8d835174fcdbef,b3f063c60c61ac6655f91cbb266e3a0dafc06449ebb701c
b56541c5e9bff65e9,b3fc8bb1e6924f9441e746d0de7dc2b2421b064272a6cc33e8cf8ce8c1ad20a9,c47ec9cba9b7396186801f9eaae794fe6db53570486095d1bda4f58e327b4eec,d019
b9605304d5728d6eceda24b1ef4398227fd978f3c1b32a91fbc4f5069e8d,e07c39c644dab0cad373e4ff3809db22128b376c139416d9e774b96e1f86e7e7,f3dd50d3de8dc26d12ab5bceea
a80ec05a722834dde97107d7bb194c9f67045a,f85f29a41d535b1ca612bcf8217a3ea4975fb0b95dcd69e2bf0b26605795261c arch=cray-sles15-zen2 ]])

whatis([[Short description : Meta module to load dependencies to build wrf@4.2. ]])
help([[Meta module that loads all dependencies need to build local wrf@4.2]])

-- TODO: figure out way of spack generating dependencies
-- currently would be to make use of the spack.lock
-- in the related environment.
load(.cray-mpich/7.7.0-rpq4ivk/module")
load(hdf5/1.12.1-parallel-api-v112/module")
load(cmake/3.21.4/module")
load(.ncurses/6.1-f5xbinw/module")
load(.openssl/1.1.0i-fips-kl6ezjb/module")
load(.libaec/1.0.5-wa6rg3n/module")
load(.numactl/2.0.11-tinrgvs/module")
load(.pkg-config/0.29.2-3tlf6jx/module")
load(.zlib/1.2.11-fhca6le/module")
load(.jasper/2.0.32-dfek24i/module")
load(.libjpeg-turbo/2.1.0-aygwfi2/module")
load(.nasm/2.15.05-4ooqntw/module")
load(.libpng/1.6.37-6cux2z5/module")
load(.libtirpc/1.2.6-tl666wp/module")
load(.krb5/1.19.2-5lln5yk/module")
load(.libtool/2.4.6-jlvb3j2/module")
load(netcdf-c/4.8.1/module")
load(.hdf/4.2.15-2zux5tl/module")
load(parallel-netcdf/1.12.2/module")
load(perl/5.34.0/module")
load(.berkeley-db/18.1.40-ihkuhwv/module")
load(.bzip2/1.0.6-t34oqcl/module")
load(.gdbm/1.19-72vdi62/module")

thanks @pelahi , very important piece of work for the Setonix Spack stack

authors: marcodelapierre pelahi

repo: PawseySC/pawsey-spack-config


text:
Rename hungry_dragon to baby_dragon & Rename minipekka to mini_pekka

Baby Dragon:
Rename hungry_dragon to baby_dragon:

In the characters.csv, it is listed as baby_dragon
Easier to write

Mini Pekka:
Rename minipekka to mini_pekka:

In the characters.csv, it is listed as mini_pekka
It has a space and it is the only card with two words that does not have a space

Where to change:

In the code where the values appear
In the Wiki
CSV
Image Name

characters.csv

I am hesitant to do this atm because I would also have to change these names in various data sets and config files that I have outside of the repo.

authors: Chi-EEE Pbatch

repo: Pbatch/ClashRoyaleBuildABot


text:
Initial frontend elements

Changes

Uses a simple theme with color variables taken from the Figma design
Added placeholder buttons and search bar on top of the Leaflet map

Upcoming changes

Separating styling into their own files

Look into MUI's own styled components. https://mui.com/system/styled/

Good call! I'd been avoiding styled components since I wasn't fully aware of the benefits, but after reading up, I can see it's definitely the better choice. Thanks, I'll change it up! 😊

All suggested changes are now done! ✨

repo: Pearl-image-comparer/Pearl


text:
[Snyk] Security upgrade golang from 1.16 to 1.22.1

This PR was automatically created by Snyk using the credentials of a real user.Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.
Changes included in this PR

owasp-top10-2021-apps/a1/camplake-api/deployments/api.Dockerfile

We recommend upgrading to golang:1.22.1, as this image has only 92 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
Some of the most important vulnerabilities in your base image include:

Severity
Priority Score / 1000
Issue
Exploit Maturity

823
Out-of-bounds Write SNYK-DEBIAN11-GLIBC-5927133
Mature

823
Out-of-bounds Write SNYK-DEBIAN11-GLIBC-5927133
Mature

823
Out-of-bounds Write SNYK-DEBIAN11-GLIBC-5927133
Mature

823
Out-of-bounds Write SNYK-DEBIAN11-GLIBC-5927133
Mature

561
Resource Exhaustion SNYK-DEBIAN11-NGHTTP2-5953384
Mature

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Resource Exhaustion

Checkmarx One – Scan Summary & Details – 311ec325-0dcb-4c9f-9b4c-a982065a0dab
New Issues

Severity
Issue
Source File / Package
Checkmarx Insight

CVE-2021-32050
Npm-mongodb-3.3.2
Vulnerable Package

CVE-2022-26592
Npm-node-sass-4.9.3
Vulnerable Package

CVE-2022-26592
Npm-node-sass-4.11.0
Vulnerable Package

CVE-2022-43357
Npm-node-sass-4.9.3
Vulnerable Package

CVE-2022-43357
Npm-node-sass-4.11.0
Vulnerable Package

CVE-2022-43358
Npm-node-sass-4.9.3
Vulnerable Package

CVE-2022-43358
Npm-node-sass-4.11.0
Vulnerable Package

CVE-2023-24998
Maven-org.apache.tomcat.embed:tomcat-embed-core-9.0.38
Vulnerable Package

CVE-2023-39325
Go-golang.org/x/net-v0.0.0-20190404232315-eb5bcb51f2a3
Vulnerable Package

CVE-2023-39325
Go-golang.org/x/net-v0.0.0-20220412020605-290c469a71a5
Vulnerable Package

CVE-2023-39325
Go-golang.org/x/net-v0.0.0-20210226172049-e18ecbb05110
Vulnerable Package

CVE-2023-39325
Go-golang.org/x/net-v0.0.0-20180724234803-3673e40ba225
Vulnerable Package

CVE-2023-39325
Go-golang.org/x/net-v0.0.0-20190620200207-3b0461eec859
Vulnerable Package

CVE-2023-42282
Npm-ip-1.1.5
Vulnerable Package

CVE-2023-42818
Go-golang.org/x/crypto-v0.0.0-20181112202954-3d3f9f413869
Vulnerable Package

CVE-2023-42818
Go-golang.org/x/crypto-v0.0.0-20190103213133-ff983b9c42bc
Vulnerable Package

CVE-2023-42818
Go-golang.org/x/crypto-v0.0.0-20181203042331-505ab145d0a9
Vulnerable Package

CVE-2023-42818
Go-golang.org/x/crypto-v0.0.0-20220411220226-7b82a4e95df4
Vulnerable Package

CVE-2023-42818
Go-golang.org/x/crypto-v0.0.0-20200414173820-0848c9571904
Vulnerable Package

CVE-2023-42818
Go-golang.org/x/crypto-v0.0.0-20201002170205-7f63de1d35b0
Vulnerable Package

CVE-2023-42818
Go-golang.org/x/crypto-v0.0.0-20190605123033-f99c8df09eb5
Vulnerable Package

CVE-2023-42818
Go-golang.org/x/crypto-v0.0.0-20210921155107-089bfa567519
Vulnerable Package

CVE-2023-44487
Maven-org.apache.tomcat.embed:tomcat-embed-core-9.0.38
Vulnerable Package

CVE-2023-44487
Go-google.golang.org/grpc-v1.21.1
Vulnerable Package

CVE-2023-45133
Npm-@babel/traverse-7.4.5
Vulnerable Package

CVE-2023-45133
Npm-@babel/traverse-7.11.5
Vulnerable Package

CVE-2023-45133
Npm-@babel/traverse-7.18.0
Vulnerable Package

CVE-2023-45311
Npm-fsevents-1.2.7
Vulnerable Package

CVE-2023-45311
Npm-fsevents-1.2.4
Vulnerable Package

CVE-2023-46136
Python-Werkzeug-0.14.1
Vulnerable Package

CVE-2023-46234
Npm-browserify-sign-4.2.1
Vulnerable Package

CVE-2023-46589
Maven-org.apache.tomcat.embed:tomcat-embed-core-9.0.38
Vulnerable Package

CVE-2023-6378
Maven-ch.qos.logback:logback-classic-1.2.3
Vulnerable Package

CVE-2023-6378
Maven-ch.qos.logback:logback-core-1.2.3
Vulnerable Package

CVE-2023-6481
Maven-ch.qos.logback:logback-core-1.2.3
Vulnerable Package

CVE-2024-22243
Maven-org.springframework:spring-web-5.2.9.RELEASE
Vulnerable Package

CVE-2024-22259
Maven-org.springframework:spring-web-5.2.9.RELEASE
Vulnerable Package

CVE-2024-23672
Maven-org.apache.tomcat.embed:tomcat-embed-core-9.0.38
Vulnerable Package

CVE-2024-23672
Maven-org.apache.tomcat.embed:tomcat-embed-websocket-9.0.38
Vulnerable Package

CVE-2024-27088
Npm-es5-ext-0.10.53
Vulnerable Package

CVE-2024-27088
Npm-es5-ext-0.10.47
Vulnerable Package

CVE-2024-29180
Npm-webpack-dev-middleware-5.3.3
Vulnerable Package

CVE-2024-29180
Npm-webpack-dev-middleware-3.7.2
Vulnerable Package

Client_DOM_XSS
/owasp-top10-2021-apps/a8/amarelo-designs/app/static/User/vendor/raphael/raphael.js: 6006
Attack Vector

Client_DOM_XSS
/owasp-top10-2021-apps/a6/cimentech/app/html/modules/overlay/overlay-parent.js: 610
Attack Vector

Cxab55612e-3a56
Npm-braces-3.0.2
Vulnerable Package

Cxab55612e-3a56
Npm-braces-1.8.5
Vulnerable Package

Cxab55612e-3a56
Npm-braces-2.3.2
Vulnerable Package

Cxca84a1c2-1f12
Npm-micromatch-4.0.2
Vulnerable Package

Cxca84a1c2-1f12
Npm-micromatch-3.1.10
Vulnerable Package

Cxca84a1c2-1f12
Npm-micromatch-4.0.5
Vulnerable Package

Cxca84a1c2-1f12
Npm-micromatch-2.3.11
Vulnerable Package

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a1/tictactoe/src/app.js: 154
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a1/tictactoe/src/app.js: 89
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb-core/lib/auth/scram.js: 242
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb-core/lib/auth/scram.js: 242
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a1/tictactoe/src/app.js: 153
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a1/tictactoe/src/app.js: 88
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a1/tictactoe/src/app.js: 163
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb-core/lib/auth/scram.js: 133
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a1/tictactoe/src/app.js: 111
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a1/tictactoe/src/db.js: 103
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a1/tictactoe/src/db.js: 95
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a1/tictactoe/src/db.js: 76
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb-core/lib/auth/mongocr.js: 18
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb-core/lib/auth/mongocr.js: 17
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb-core/lib/auth/mongocr.js: 17
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb/lib/operations/db_ops.js: 88
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb/lib/operations/db_ops.js: 88
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb-core/lib/connection/connection.js: 126
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb/lib/operations/db_ops.js: 96
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb/lib/operations/db_ops.js: 96
Attack Vector

Insecure_Storage_of_Sensitive_Data
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/mongodb-core/lib/auth/mongocr.js: 35
Attack Vector

MongoDB_NoSQL_Injection
/owasp-top10-2021-apps/a3/mongection/src/app.js: 59
Attack Vector

Passwords And Secrets - Generic Password
/docker-compose.yml: 31
Query to find passwords and secrets in infrastructure code.

Second_Order_SQL_Injection
/owasp-top10-2021-apps/a1/tictactoe/src/db.js: 13
Attack Vector

Absolute_Path_Traversal
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/send/index.js: 613
Attack Vector

Absolute_Path_Traversal
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/send/index.js: 613
Attack Vector

Absolute_Path_Traversal
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/send/index.js: 613
Attack Vector

Absolute_Path_Traversal
/owasp-top10-2021-apps/a5/stegonography/app/node_modules/dotenv/config.js: 8
Attack Vector

Broken_or_Risky_Hashing_Function
/owasp-top10-2021-apps/a6/cimentech/app/html/modules/openid/openid.inc: 520
Attack Vector

Broken_or_Risky_Hashing_Function
/owasp-top10-2021-apps/a6/cimentech/app/html/modules/openid/openid.inc: 462
Attack Vector

Broken_or_Risky_Hashing_Function
/owasp-top10-2021-apps/a6/cimentech/app/html/modules/openid/openid.inc: 461
Attack Vector

Broken_or_Risky_Hashing_Function
/owasp-top10-2021-apps/a6/cimentech/app/html/modules/openid/openid.inc: 455
Attack Vector

CSRF
/owasp-top10-2021-apps/a6/cimentech/app/html/modules/simpletest/drupal_web_test_case.php: 1815
Attack Vector

CVE-2019-11841
Go-golang.org/x/crypto-v0.0.0-20190103213133-ff983b9c42bc
Vulnerable Package

CVE-2019-11841
Go-golang.org/x/crypto-v0.0.0-20181203042331-505ab145d0a9
Vulnerable Package

CVE-2019-11841
Go-golang.org/x/crypto-v0.0.0-20181112202954-3d3f9f413869
Vulnerable Package

CVE-2019-18799
Npm-node-sass-4.9.3
Vulnerable Package

CVE-2019-18799
Npm-node-sass-4.11.0
Vulnerable Package

CVE-2023-26159
Npm-follow-redirects-1.15.0
Vulnerable Package

CVE-2023-26159
Npm-follow-redirects-1.5.7
Vulnerable Package

CVE-2023-26159
Npm-follow-redirects-1.13.0
Vulnerable Package

CVE-2023-26159
Npm-follow-redirects-1.7.0
Vulnerable Package

CVE-2023-29407
Go-golang.org/x/image-v0.0.0-20190802002840-cff245a6509b
Vulnerable Package

CVE-2023-29408
Go-golang.org/x/image-v0.0.0-20190802002840-cff245a6509b
Vulnerable Package

CVE-2023-33202
Maven-org.bouncycastle:bcprov-jdk15on-1.56
Vulnerable Package

CVE-2023-34055
Maven-org.springframework.boot:spring-boot-2.3.4.RELEASE
Vulnerable Package

CVE-2023-3978
Go-golang.org/x/net-v0.0.0-20180724234803-3673e40ba225
Vulnerable Package

CVE-2023-3978
Go-golang.org/x/net-v0.0.0-20190620200207-3b0461eec859
Vulnerable Package

CVE-2023-3978
Go-golang.org/x/net-v0.0.0-20220412020605-290c469a71a5
Vulnerable Package

CVE-2023-3978
Go-golang.org/x/net-v0.0.0-20210226172049-e18ecbb05110
Vulnerable Package

CVE-2023-3978
Go-golang.org/x/net-v0.0.0-20190404232315-eb5bcb51f2a3
Vulnerable Package

CVE-2023-41080
Maven-org.apache.tomcat.embed:tomcat-embed-core-9.0.38
Vulnerable Package

CVE-2023-42795
Maven-org.apache.tomcat.embed:tomcat-embed-core-9.0.38
Vulnerable Package

CVE-2023-44270
Npm-postcss-8.4.14
Vulnerable Package

CVE-2023-44270
Npm-postcss-7.0.39
Vulnerable Package

CVE-2023-44270
Npm-postcss-7.0.35
Vulnerable Package

CVE-2023-44270
Npm-postcss-7.0.32
Vulnerable Package

CVE-2023-44270
Npm-postcss-7.0.21
Vulnerable Package

CVE-2023-44270
Npm-postcss-7.0.14
Vulnerable Package

CVE-2023-44270
Npm-postcss-7.0.2
Vulnerable Package

CVE-2023-45648
Maven-org.apache.tomcat.embed:tomcat-embed-core-9.0.38
Vulnerable Package

CVE-2023-45857
Npm-axios-0.20.0
Vulnerable Package

CVE-2023-45857
Npm-axios-0.17.1
Vulnerable Package

CVE-2023-45857
Npm-axios-0.21.4
Vulnerable Package

CVE-2023-48795
Go-golang.org/x/crypto-v0.0.0-20181112202954-3d3f9f413869
Vulnerable Package

CVE-2023-48795
Go-golang.org/x/crypto-v0.0.0-20181203042331-505ab145d0a9
Vulnerable Package

CVE-2023-48795
Go-golang.org/x/crypto-v0.0.0-20201002170205-7f63de1d35b0
Vulnerable Package

CVE-2023-48795
Go-golang.org/x/crypto-v0.0.0-20210921155107-089bfa567519
Vulnerable Package

CVE-2023-48795
Go-golang.org/x/crypto-v0.0.0-20200414173820-0848c9571904
Vulnerable Package

CVE-2023-48795
Go-golang.org/x/crypto-v0.0.0-20190605123033-f99c8df09eb5
Vulnerable Package

CVE-2023-48795
Go-golang.org/x/crypto-v0.0.0-20190103213133-ff983b9c42bc
Vulnerable Package

CVE-2023-48795
Go-golang.org/x/crypto-v0.0.0-20220411220226-7b82a4e95df4
Vulnerable Package

CVE-2023-51074
Maven-com.jayway.jsonpath:json-path-2.4.0
Vulnerable Package

CVE-2024-21733
Maven-org.apache.tomcat.embed:tomcat-embed-core-9.0.38
Vulnerable Package

CVE-2024-22195
Python-Jinja2-2.10
Vulnerable Package

CVE-2024-24549
Maven-org.apache.tomcat.embed:tomcat-embed-core-9.0.38
Vulnerable Package

CVE-2024-25710
Maven-org.apache.commons:commons-compress-1.12

More results are available on AST platform

authors: svc-pe-appsec vrangsr

repo: PearsonAppsec/secDevLabs


text:
How to install using gradle?

As title, i have search the internet to find tutorial / examples on how to install the template using gradle but cant find anything.
I want to use Pebble with Vert.x
Can someone help me?

Just add in your build.gradle this dependency
dependencies {
compile 'io.pebbletemplates:pebble:2.6.0'
}

repo: PebbleTemplates/pebble


text:
Tx.go updates / simplify Validate func

Title
This PR aims to remove some unused func from tx.go: UnmarshallTx and remove the use of that func in some tests files.
It also simplifies the Validate function as there is no need to check if a field is a string or not as it is enforced by Go types.
Tx.go has 100% test coverage.
I will update in a later PR the coverage for AMMBid which seems to have decreased.

@GuillemGarciaDev I removed the other UnmarshallJSON function from the remaning transaction types and the test files. It's ready for another review

repo: Peersyst/xrpl-go


text:
Support getting the most recent block to a slot from storage

PR Description
Modifies HistoricalChainData to return the most recent block prior to or in a finalised slot rather than returning empty when the slot did not have a block.  Also explicitly returns empty the slot is after the finalised checkpoint as there may be multiple blocks applying at that point.
Note: Breaks backwards compatibility of our own disk storage.

Bytes32 and UnsignedLong values are now stored in binary format instead of hex strings
A sorted map is now used for block root references so we can  efficiently find the most recent root.

We can now create in-memory instance of Database for testing.

Oh also, do you mind explaining why do we need Historical Chain data to return the most recent block rather than returning empty?

authors: ajsutton cemozerr

repo: PegaSysEng/artemis


text:
qjit(static_argnums=...) fails when the marked static argument has a default value

Context
When jit-compiling a python function, the arguments of the compiled function lose their concrete values and are replaced by tracers, which at a high level means abstract variables that have the same type and shape as the concrete variable. A compiled program, called the jaxpr, uses these abstract tracers to represent how the arguments of the function are used.
The below example shows how to use Catalyst to jit-compile a function, and how to inspect the compiled jaxpr.
import catalyst
from catalyst import qjit

@qjit
def f(x, y):
  return x+y

result = f(10, 20)
print(result)
print(f.jaxpr)

result = f(1.1, 2.2)
print(result)
print(f.jaxpr)

30
{ lambda ; a:i64[] b:i64[]. let
    c:i64[] = add a b
  in (c,) }
3.3
{ lambda ; a:f64[] b:f64[]. let
    c:f64[] = add a b
  in (c,) }

Notice that in the jaxpr, the type of the arguments to the function, i64 and f64, are the same as the type of the concrete arguments of their corresponding calls. The process of converting python to jaxpr is called tracing.
One issue with arguments being abstract is when their concrete value is needed, for example when being compared to other concrete values, tracing will fail, since abstract tracers cannot be interpreted as concrete values. See here for more details.
@qjit
def f(x, y):
  if x < 100:
    return x+y
  return 42

result = f(10, 20)
print(result)
print(f.jaxpr)

jax.errors.TracerBoolConversionError: Attempted boolean conversion of traced array with shape bool[]..

To avoid this problem, some of the function arguments can be marked static, which essentially means when tracing, keep their concrete values, and don't replace them with tracers. This marking can be done by the static_argnums keyword argument of qjit, which takes in a list of argument indices to be marked static.
@qjit(static_argnums=[0])
def f(x, y):
  if x < 100:
    return x+y
  return 42

result = f(10, 20)
print(result)
print(f.jaxpr)

result = f(1000, 20)
print(result)
print(f.jaxpr)

30
{ lambda ; a:i64[]. let  ; b:i64[] = add 10 a in (b,) }
42
{ lambda ; a:i64[]. let  ;  in (42,) }

However, currently in Catalyst, arguments with default values cannot be marked as static_argnum:
@qjit(static_argnums=[1])
def f(y, x=9):
    if x < 10:
        return x + y
    return 42000

res = f(20)
print(res)

catalyst.utils.exceptions.CompileError: argnum 1 is beyond the valid range of [0, 1).

Goal
We would like to support static_argnums in qjit to mark arguments with default values, as this is supported by native jax.jit:
from functools import partial
import jax
@partial(jax.jit, static_argnums=[1])
def f(y, x=9):
    if x < 10:
        return x + y
    return 42000

print(f(20), f(20, 3), f(20, 30000))

29 23 42000

Requirements:

We would like to support marking arguments with default values as static, and match all behavior observed from jax.jit. Explicitly:

If a value is not provided for the default-valued argument in the call, the default concrete value should be used in the jaxpr.
If a value is provided for the default-valued argument in the call, then the provided concrete value should be used in the jaxpr.

Technical details

Due to reasons that do not concern us here, all jaxprs produced by qjit will carry a transform_named_sequence. You can safely ignore it.

The qjit function takes in a python function and returns a QJIT object, which is a callable. In the QJIT object, there is a capture method that determines how a python function is traced into a jaxpr. See frontend/catalyst/jit.py.

It should be possible to implement this functionality completely in the capture layer, without delving into the actual underlying machinery of the trace_to_jaxpr methods. For example, one option is to create two versions of the function, both without any default-valued arguments, adjusted to behave correctly, and trace these two functions depending on whether a default value was supplied by the user's call. Other options might be possible too.

There are many cases potentially possible for how a user might call a function. For the purpose of this assessment, only the simplest case is required, aka the example above that worked with pure jax (though bonus points if you can make more complicated patterns work).

Installation help
To save time, instead of installing Catalyst from source it is also be possible to download the PyPI wheel and extract it into the frontend directory of a cloned catalyst repository (taking care to match git tags before hand), followed by make frontend. This then allows modifying the Python files in-place.
Alternatively, complete instructions to install Catalyst from source can be found here, but due to the size of the llvm-project it can take a while (~3 hrs on a personal laptop) to compile.

I think this is because the static_argnum tracks the argument indices at the call, not at the definition

Hi! I am Aniket, a PhD candidate at Duke interviewing for a role on the compiler team. I was given this issue as a technical challenge. Apart from the details mentioned in this issue, are there any other pointers that might help me tackle this issue? Thank you!

Hi @AniketDalvi , do you have a more specific question in mind? I would love to give out more pointers and resolve any confusion you may have!

Hi! So for installation - it says to download the PyPI wheel. Which version should I be downloading the wheel for? pip says there are 3 available versions - 0.1.0, 0.1.1 and 0.1.2.

Hey @AniketDalvi, I am not too sure why your pip says that there are only 3 versions available. We have version 0.8.1 in pypi and will be releasing version 0.9.0 soon.

Okay I am just going to download the .whl file from the above link. I will then extract it in the frontend directory as was directed in the installation instructions

Hi! Okay, so I have followed the instruction and seemed to have successfully installed the repository. Is there a quick sanity check experiment/file I can run to verify the installation?

You can run the tests. pytest frontend/test/pytest -n auto. I think it is possible that some debugging tests fail in your machine as they expect a specific path. But if the vast majority are passing, then it should be good.

Running pytest gave me an error stating that there was an interpreter mismatch. The interpreter is Python 3.10 while the package is compatible only with 3.12

Running pytest gave me an error stating that there was an interpreter mismatch. The interpreter is Python 3.10 while the package is compatible only with 3.12

When you download via pip, pip enforces this compatibility check, but manually downloading it you need to make sure to download the appropriate one for you. See here for a list of several wheels with different python version compatibility.
Maybe it would be easier to install from source? It just takes a long time to build LLVM initially.

Understood, that makes sense. I might re-try it with a different wheel with a compatible python version. If not, I will resort to installing from source.

Okay when trying to run the make frontend command with the new wheel, I get this error - error: command '/usr/bin/g++' failed with exit code 1. I have tried all solutions that stack overflow has to offer, but to no avail. Any thoughts on what this could be?

There's some required packages before building Catalyst. Maybe some of them are missing? See the build from source guide

Yup installed all the required packages, but the error persists. I am now just going to build from source instead.

Yup installed all the required packages, but the error persists. I am now just going to build from source instead.

To avoid all package version issues, I also recommend using a fresh virtual environment when developing, e.g.
python3 -m venv pyenv
source ./pyenv/bin/activate

after which you can pip install all the requirements and make all from source.

Okay I seemed to have gotten it to work from source. Most tests pass, some are skipped, and 4 debugging tests fails (as @erick-xanadu said is expected). I am running all of this from with within a conda environment on a linux machine.

Okay I seemed to have gotten it to work from source. Most tests pass, some are skipped, and 4 debugging tests fails (as @erick-xanadu said is expected).

Yeah, a bunch are skipped, 4 failing ones.

I am running all of this from with within a conda environment on a linux machine.

Awesome! We don't normally use conda so I am happy to hear this worked for you :)

Hi! So from my initial analysis, I traced the issue down to this check that throws an exception - https://github.com/PennyLaneAI/catalyst/blob/75dc517a6e2f2583da890b9e8198241933a7aef5/frontend/catalyst/tracing/type_signatures.py#L121.
It appears that it checks the index used to specify the static argument with the number of args passed to the function. However, in the case of a default parameter, the static argument index is likely to be greater than len(args) as the default parameter may not be passed when the function is called. Considering this, my initial proposal for a solution would be to compare the static argument index to the number of arguments in the function signature, as opposed to the number in the function itself.
Would like to understand your thoughts on this!

Hi! So from my initial analysis, I traced the issue down to this check that throws an exception -
https://github.com/PennyLaneAI/catalyst/blob/75dc517a6e2f2583da890b9e8198241933a7aef5/frontend/catalyst/tracing/type_signatures.py#L120
.
It appears that it checks the index used to specify the static argument with the number of args passed to the function. However, in the case of a default parameter, the static argument index is likely to be greater than len(args) as the default parameter may not be passed when the function is called. Considering this, my initial proposal for a solution would be to compare the static argument index to the number of arguments in the function signature, as opposed to the number in the function itself.
Would like to get your thoughts on this!

Hi! Usually what we do for these challenges is you can fork the catalyst repo, push your changes, and open a PR. It will be easier to review.
It's good if it turns out to be a simple fix, but one thing I'm afraid of is whether loosening the verification would allow in some errors. Can you test the frontend test suite to make sure this does not happen? You can run make test-frontend in the root catalyst directory.

Hi! Okay that sounds good. I am working off of a my branch. Does that work, or does it have to be a fork?

It doesn't really matter how you develop, as long as you are able to push a pull request :). I think to do that, you do need a fork. But you can always just add a new remote to your local git workspace.
git clone $pennylane/catalyst
# work on the issue
# fork $pennylane/catalyst to $yourrepo/catalyst
git remote add myrepo $yourrepo/catalyst
git push myrepo $yourbranch
# open a PR from $yourrepo/catalyst to $pennylane/catalyst

authors: AniketDalvi erick-xanadu paul0403

repo: PennyLaneAI/catalyst


text:
Improve API for third party compilers.

In order to implement CUDA, I had to override some functionality in Catalyst.
        with Patcher(
            (catalyst.pennylane_extensions.QFunc, "extract_backend_info", cudaq_backend_info),
            (qml.QNode, "__call__", QFunc.__call__),
        ):

I think third party compilers might have to do something similar if they follow a similar approach to CUDA-quantum's implementation.

I'm not sure this actually concerns Catalyst, but is something to look at for PennyLane's compiler API.