text:
run run_chatbot.sh RuntimeError: Tensors must be contiguous

Hello, I downloaded the gpt-neo-2.7B model and ran run_chatbot.sh, which displayed the following error? How can I resolve it?
(lmflow) root@shenma:~/LMFlow# ./scripts/run_chatbot.sh output_models/gpt-neo-2.7B
[2023-04-06 09:55:55,760] [WARNING] [runner.py:186:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only.
Detected CUDA_VISIBLE_DEVICES=0: setting --include=localhost:0
[2023-04-06 09:55:55,769] [INFO] [runner.py:550:main] cmd = /root/anaconda3/envs/lmflow/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMF19 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None examples/chatbot.py --deepspeed configs/ds_config_chatbot.json --model_name_or_path output_models/gpt-neo-2.7B
[2023-04-06 09:55:56,692] [INFO] [launch.py:142:main] WORLD INFO DICT: {'localhost': [0]}
[2023-04-06 09:55:56,692] [INFO] [launch.py:148:main] nnodes=1, num_local_procs=1, node_rank=0
[2023-04-06 09:55:56,692] [INFO] [launch.py:161:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0]})
[2023-04-06 09:55:56,692] [INFO] [launch.py:162:main] dist_world_size=1
[2023-04-06 09:55:56,692] [INFO] [launch.py:164:main] Setting CUDA_VISIBLE_DEVICES=0
Traceback (most recent call last):
  File "/root/LMFlow/examples/chatbot.py", line 117, in <module>
    main()
  File "/root/LMFlow/examples/chatbot.py", line 44, in main
    model = AutoModel.get_model(
  File "/root/LMFlow/src/lmflow/models/auto_model.py", line 14, in get_model
    return HFDecoderModel(model_args, *args, **kwargs)
  File "/root/LMFlow/src/lmflow/models/hf_decoder_model.py", line 224, in __init__
    self.ds_engine = deepspeed.initialize(model=self.backend_model, config_params=ds_config)[0]
  File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/deepspeed/__init__.py", line 125, in initialize
    engine = DeepSpeedEngine(args=args,
  File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/deepspeed/runtime/engine.py", line 297, in __init__
    self._configure_distributed_model(model)
  File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/deepspeed/runtime/engine.py", line 1182, in _configure_distributed_model
    self._broadcast_model()
  File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/deepspeed/runtime/engine.py", line 1105, in _broadcast_model
    dist.broadcast(p,
  File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/deepspeed/comm/comm.py", line 123, in log_wrapper
    return func(*args, **kwargs)
  File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/deepspeed/comm/comm.py", line 228, in broadcast
    return cdb.broadcast(tensor=tensor, src=src, group=group, async_op=async_op)
  File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/deepspeed/comm/torch.py", line 78, in broadcast
    return torch.distributed.broadcast(tensor=tensor,
  File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py", line 1436, in wrapper
    return func(*args, **kwargs)
  File "/root/anaconda3/envs/lmflow/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py", line 1555, in broadcast
    work = group.broadcast([tensor], opts)
RuntimeError: Tensors must be contiguous
[2023-04-06 09:56:28,731] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 439575
[2023-04-06 09:56:28,731] [ERROR] [launch.py:324:sigkill_handler] ['/root/anaconda3/envs/lmflow/bin/python', '-u', 'examples/chatbot.py', '--local_rank=0', '--deepspeed', 'configs/ds_config_chatbot.json', '--model_name_or_path', 'output_models/gpt-neo-2.7B'] exits with return code = 1

deepspeed                0.8.3

Thanks for your interest in LMFlow! As the chatbot works fine in Google Colab, it is possible this is caused by some local incompatibility problems. Someone has reported similar issues when using huggingface transformers. It appears to be a problem of deepspeed. One way to walk around is to make sure every tensor is continuous right after model initialization, as recommended in this link.
for p in model.backend_model().parameters():
    p = p.contiguous()

It you don't want to modify the code and simply want to have a try, we recommend to run this experiment on Google Colab, which has the environment set up for you. You may also try docker, which has much less dependency problems.
Hope that can resolve your issue. Thanks 😄

Thank you. I also found this solution on Google, but I cannot solve it

I still encountered this error using the Docker version
#system
Ubuntu 20.04.5 LTS x86_64
#Graphics card
NVIDIA GeForce RTX 3060

If the docker version doesn't work, it is highly possible that it was caused by some hardware dependency problems. Could you please provide your CUDA version and CUDA driver version, so we may check that for you? Also, it would be nice if you could try on some small models and check if the problem occurs as well. Thanks!

Hi, I made the changes according to your suggestion. But issue is still there. Below are changes I have made to the code.
src/lmflow/models/hf_decoder_model.py, starting from line 198:
        ##### George: modified to deal with the issue of Tensors must be contiguous according to issue 144
        model = self.backend_model
        for p in model.parameters():
            p = p.contiguous()
        self.ds_engine = deepspeed.initialize(model=model, config_params=ds_config)[0]
        ##### George: end modification

deepspeed.initialize(model=model, config_params=ds_config)[0] still comes back with error message: Tensors must be contiguous
Is above modification correct?
Thanks
George

By the way, below is my hardware and conda env:
Driver Version: 510.108.03   CUDA Version: 11.6
name: lmflow
channels:

defaults
dependencies:
_libgcc_mutex=0.1=main
_openmp_mutex=5.1=1_gnu
ca-certificates=2023.01.10=h06a4308_0
certifi=2022.12.7=py39h06a4308_0
ld_impl_linux-64=2.38=h1181459_1
libffi=3.4.2=h6a678d5_6
libgcc-ng=11.2.0=h1234567_1
libgfortran-ng=7.5.0=ha8ba4b0_17
libgfortran4=7.5.0=ha8ba4b0_17
libgomp=11.2.0=h1234567_1
libstdcxx-ng=11.2.0=h1234567_1
mpi=1.0=mpich
mpi4py=3.1.4=py39hfc96bbd_0
mpich=3.3.2=hc856adb_0
ncurses=6.4=h6a678d5_0
openssl=1.1.1t=h7f8727e_0
pip=23.0.1=py39h06a4308_0
python=3.9.16=h7a1cb2a_2
readline=8.2=h5eee18b_0
setuptools=65.6.3=py39h06a4308_0
sqlite=3.41.1=h5eee18b_0
tk=8.6.12=h1ccaba5_0
tzdata=2022g=h04d1e81_0
wheel=0.38.4=py39h06a4308_0
xz=5.2.10=h5eee18b_1
zlib=1.2.13=h5eee18b_0
pip:

accelerate==0.17.1
aiohttp==3.8.4
aiosignal==1.3.1
appdirs==1.4.4
async-timeout==4.0.2
attrs==22.2.0
charset-normalizer==3.1.0
click==8.1.3
cmake==3.26.0
datasets==2.10.1
deepspeed==0.8.3+e73de8ce
dill==0.3.6
docker-pycreds==0.4.0
filelock==3.10.0
flask==2.2.3
flask-cors==3.0.10
frozenlist==1.3.3
fsspec==2023.3.0
gitdb==4.0.10
gitpython==3.1.31
hjson==3.1.0
huggingface-hub==0.13.2
idna==3.4
importlib-metadata==6.1.0
itsdangerous==2.1.2
jinja2==3.1.2
lit==15.0.7
markupsafe==2.1.2
mpmath==1.3.0
multidict==6.0.4
multiprocess==0.70.14
networkx==3.0
ninja==1.11.1
numpy==1.24.2
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-cupti-cu11==11.7.101
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cudnn-cu11==8.5.0.96
nvidia-cufft-cu11==10.9.0.58
nvidia-curand-cu11==10.2.10.91
nvidia-cusolver-cu11==11.4.0.1
nvidia-cusparse-cu11==11.7.4.91
nvidia-nccl-cu11==2.14.3
nvidia-nvtx-cu11==11.7.91
packaging==23.0
pandas==1.5.3
pathtools==0.1.2
peft==0.3.0.dev0
protobuf==4.22.1
psutil==5.9.4
py-cpuinfo==9.0.0
pyarrow==11.0.0
pydantic==1.10.6
python-dateutil==2.8.2
pytz==2022.7.1
pyyaml==6.0
regex==2022.10.31
requests==2.28.2
responses==0.18.0
sentencepiece==0.1.97
sentry-sdk==1.17.0
setproctitle==1.3.2
six==1.16.0
smmap==5.0.0
sympy==1.11.1
tokenizers==0.13.2
torch==2.0.0
tqdm==4.65.0
transformers==4.28.0.dev0
triton==2.0.0
trl==0.4.2.dev0
typing-extensions==4.5.0
urllib3==1.26.15
wandb==0.14.0
werkzeug==2.2.3
xxhash==3.2.0
yarl==1.8.2
zipp==3.15.0

Thanks! Could you check if the following code piece works?
for p in model.parameters():
    p.data.copy_(p.contiguous().data)

I checked that the aforementioned code may not be able to replace the original parameters, this one may work. According to pytorch's official documentation, p.contiguous() only returns a memory-contiguous tensor, but may not affect the original tensor.
Also, it would be nice if you can check the problem when running on smaller models. I am wondering if that's related to model size. Thanks!

The above changed code still not working. It works when using gpt2 as the model.

Hi, I meet a similar problem when the remaining GPU memory is not sufficient. My guess is that RAM optimized load splits tensors and moves its around in different GPUs, thus triggers the error. You may try to add --use_ram_optimized_load in scripts/chatbot.sh to see if the problem is resolved. Thanks 😄

Hello, here are my cuda and cuda driver versions
NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6

I think that it is highly probable that the problem is caused by RAM optimized load. Since small models don't have this issue, it is  possible that gpt-neo-2.7B triggers the RAM optimize mechanism and results in model splits among different devices, then leads to this error. You may turn off this RAM optimization via adding --use_ram_optimized_load False  in scripts/chatbot.sh to see if the problem can be resolved. Thanks 😄

I think that it is highly probable that the problem is caused by RAM optimized load. Since small models don't have this issue, it is possible that gpt-neo-2.7B triggers the RAM optimize mechanism and results in model splits among different devices, then leads to this error. You may turn off this RAM optimization via adding --use_ram_optimized_load False in scripts/chatbot.sh to see if the problem can be resolved. Thanks 😄

Thank you for your reply. This will directly kill the program

I think the bottleneck here is RAM. Normally a setup similar to our Google Colab example should work, which has approximately  13G RAM and 16G GPU memory (T4). May I ask how much RAM there is in your server? Thanks!

Thanks for providing more details! Looks like the problem was caused by multiple factors. Since loading gpt-neo-2.7B requires more than 16GB memory during the loading process, turning on RAM optimized loading becomes necessary. However, this option will result in splits of model tensors when the GPU memory is not enough, which leads to the Runtime Error here Tensors must be continguous.
To avoid this error, there are several options:

You may add more RAM to your machine/server, and turn off --use_ram_optimized_load False.
You may use a GPU with larger GPU memory, 16G will be sufficient.
You may use a CPU-only server with large RAM and run ./scripts/run_chatbot_cpu.sh, which supports CPU-only inference (although the speed may be much slower when compared with GPU).

We recommend Google Colab for running this type of experiments. Hope that answers your question. Thanks 😄

Thanks for providing more details! Looks like the problem was caused by multiple factors. Since loading gpt-neo-2.7B requires more than 16GB memory during the loading process, turning on RAM optimized loading becomes necessary. However, this option will result in splits of model tensors when the GPU memory is not enough, which leads to the Runtime Error here Tensors must be continguous.
To avoid this error, there are several options:

You may add more RAM to your machine/server, and turn off --use_ram_optimized_load False.
You may use a GPU with larger GPU memory, 16G will be sufficient.
You may use a CPU-only server with large RAM and run ./scripts/run_chatbot_cpu.sh, which supports CPU-only inference (although the speed may be much slower when compared with GPU).

We recommend Google Colab for running this type of experiments. Hope that answers your question. Thanks 😄

Okay, thank you. I closed the issue

repo: OptimalScale/LMFlow


text:
How to use position interpolation

Thank you for your work. I noticed that you have added support for positional interpolation! Could you provide a specific example or instructions such as the example of config file on how to use this feature? Additionally, while reading the code, I found that your implementation differs from LongChat, especially in using two parameters. Could you please explain in detail the purpose and design approach of these parameters? Thank you very much.

I was wondering if anyone could assist me？

@yaoguany Could you please look into this problem?

How to use this feature:
You can check out this link to see how to use this feature:
Position_Interpolation
Two parameters
We support both linear and NTK interpolation so there will be two parameters. More advanced than the official library, our codes allows you to simultaneously use both linear and NTK interpolation. Just set the ratio as:
--rope_pi_ratio 2 \
--rope_ntk_ratio 4 \

Thanks for your reply! I want to confirm two thing

If the original input length is 1024 and I want to extend it to 4096 during the training stage, is the configuration below correct?
--truncate_to_model_max_length False --do_rope_scaling True --rope_pi_ratio 2 \ --rope_ntk_ratio 4 \ --block_size 4096

Is the example you provided below for extending the input length from 512 to 4098 during the inference stage?
'
CUDA_VISIBLE_DEVICES=0
deepspeed examples/evaluation.py
--answer_type text
--model_name_or_path pinkmanlove/llama-7b-hf
--dataset_path data/wiki_en_eval
--deepspeed examples/ds_config.json
--inference_batch_size_per_device 1
--truncate_to_model_max_length False
--block_size 4096
--use_flash_attention True
--do_rope_scaling True
--rope_pi_ratio 2
--rope_ntk_ratio 4
--metric ppl
‘

1.No,the scaling ratio is pi_ratio*ntk_ratio. So in this case you are scaling to 8192
2.Same as 1, you can set either pi_ratio or ntk_ratio to 4, or set both pi_ratio and ntk_ratio to2

so in the question 1 should I set block_size as 8192 if the target input length is 8192?
does it mean that because the scaling ratio is 8, the model now supports the max length is 20488 and the largest number of block_size I can set is 20488?

No,you can still set the block size to 4096 even if scaling length is 8192
The model supports max length is 16384 for that case ,but if you set the block size to 16384, there may be some drops on the performance because the length is much longer than the original one.
In our test, the model still works well on 7000 block size under this scaling ratio set,

Thank for your patient reply! I still have a little confusion that why I can set the block_size to 4096 even if the target length is 8192. Does it mean that if my target length is 8192, the model can work even the block_size is 4096, but I can get better performance if I set it to 8192? Thanks again!

Yse,you can check this link

Thank you very much! I will check it~

source: gharchive/issue

authors: pangwenfeng research4pan yaoguany

license_type: permissive

license_source: github-api


text:
Global Header: Supra Bar with language options

#700

Here is the designer feedback on the Supra bars.
I gave him https://codepen.io/julien-deramond/pen/LYywVYz to be able to review the behavior of those bars.

Change Height of Supra to 40px
Change Personal and Business label sizes as well as the language options to 14px
Include a 1px grey horizontal line at the bottom of the Supra Bar (within the 40px height of the Supra bar). Grey colour #666666
Change the spacing of the label positions. 1. Change the space between the labels and the bottom of the Supra bar. 2. Change the space between the labels.
Below is an image from Zeplin showing the pixel spacing sizes.
Please note, In Zeplin and Sketch the positioning and specifications of label positioning is based from the Font Cell (the outline surrounding the labels).

Regarding this feedback, you'll need to evaluate if it is possible to fix that only with basic classes or if you need to create a custom component (e.g. a supra class as it was done in v4).

source: gharchive/pull-request

authors: Nurovek julien-deramond

license: mit

license_source: bigquery


text:
fix the Lucene missing Orchard & bring about a module named Orchard.UserCenter

fix the Lucene missing Orchard & bring about a module named Orchard.UserCenter that I had asked at issues 512

https://github.com/OrchardCMS/Orchard2/issues/512


text:
Detect Safe wallet contracts outside of connected apps

Safe contracts are detected when a user connects via a connected app in their frontend. But if she uses WalletConnect and their mobile app, no banner is displayed indicating that opting-in is necessary to receive yield.
https://www.loom.com/share/293b59d5f321478f8db5cccd5964a5d0

this feature has been migrated to new origin-defi repo where the Safe wallet is properly detected

authors: micahalcorn toniocodo

repo: OriginProtocol/origin-dollar

license: MIT


text:
allow to specify environment variables for the script

env:
  TR_APP_VERSION: 0.1.0
arguments: --some-flag
protocol:
  - bla

nah, i was just being mindless when writing the example in #44, object syntax is 👍

authors: matthandlersux soenkehahn

repo: Originate/check-protocols


text:
Kafka message format 0.11.x support

Currently Oryx is not supporting Kafka message format 0.11.x
Details: https://community.cloudera.com/t5/Data-Science-and-Machine/NoClassDefFoundError-in-Oryx-batch-and-speed-layers/m-p/59235#M765
Any suggestions how this can be fixed/added support @srowen, I will be happy to help with this.

Your issue does not suggest something specific to 0.11. This is how other Kafka version incompatibilities look at runtime. Do you know 0.11 isn't compatible with the 0.10.2 client? I hope not because that would mean yet another branch just to support mutually incompatible Kafka versions. I thought they'd gotten better after the 0.10.0 - 0.10.2 error.

I'm not sure whether 0.11 isn't compatible with 0.10.2 client. I can try it once master build is finished (I see it's failing in CI).

You can ignore the build failures -- that's me fixing it to work with recent travis changes. It actually works fine.

I've tested 2.5.0-SNAPSHOT. Not working with 0.11, I was trying to build it with:
<kafka.version>0.11.0-kafka-3.0.0</kafka.version> and <kafka.version>0.11.0.0</kafka.version>, but build is failing...

I saw that too when I tried it, but it looked like it was just because some methods are deprecated now and the build sets -Werror to make them fail the build. You could just set it to not fail on warning.

Still getting the same exception :(
My pom.xml looks like this:
...
      <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka_${scala.minor.version}</artifactId>
        <version>0.11.0.0</version>
        <scope>provided</scope>
        <exclusions>
          <!-- later Kafka versions bring this in -->
          <exclusion>
            <groupId>javax.servlet</groupId>
            <artifactId>servlet-api</artifactId>
          </exclusion>
        </exclusions>
      </dependency>
      <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-clients</artifactId>
        <version>0.11.0.0-cp1</version>
      </dependency>
...```

Note: `-cp1` is Confluent platform kafka-client version. Build is successful with disabled `-Werror`, but still getting `java.lang.NoClassDefFoundError: kafka/admin/RackAwareMode` exception.

Hmmm. Looking into code, this part of code is not even executed: https://github.com/OryxProject/oryx/blob/master/framework/kafka-util/src/main/java/com/cloudera/oryx/kafka/util/KafkaUtils.java#L89
Strange. It seems like there is a problem with kafka.admin.AdminUtils

You shouldn't have a class version incompatibility if you really are building and running the exact same Kafka version. I think that indicates you still have some mismatch somewhere.
However I did manage to run the tests with 0.11 and it doesn't seem to work as-is. Something else in the behavior has changed.
The 2.5 branch is branched now so master could move to 0.11. I don't have the cycles to work on getting it working with 0.11 (not sure Spark will work with it either) but if you do get it working, feel free to propose a patch.

It seems like kafka-clients and it's relevant CP versions are not compatible. There is no AdminUtils class though. Even if I get it running by workaround to don't check topic existence, it fails on Spark Streaming Kafka integration which supports only 0.10 Kafka. Hmm, I am not sure what should I do now.

I've opened discussion in Spark dev mailing list, hopefully it will get accepted.

My guess is that in fact you are getting Kafka dependencies from Spark dependencies, and yes it doesn't support 0.11 formally. I think that could be a blocker, because most workarounds I can think of would inevitably run into a problem of that form.
I haven't followed Kafka 0.11 / 1.0 (though I work on Spark) so I'm not sure, does anyone have Spark Streaming working with Kafka 0.11? if so then there's hope.

I think the first question is, does Spark 2.2 even work with Kafka 0.11? if not, then I don't know if there's a workaround here. You can run it on a separate cluster, sure, with Kafka 0.10.2. You need storage like HDFS too anyway, and it's intended to run on CDH, so a small CDH installation would provide all the pieces too.

That's what we got. We have CDH installation on 5 servers, the only one difference was that we manually installed Confluent platform as a Kafka cluster. I will try to install Kafka 0.10 via parcels and get this working until Spark supports 0.11 or 1.0
Thanks for help Sean.

I've been asking in Kafka community about 0.10 and 0.11 Spark support. They say Spark should work with 0.11 without any problems. This is response of Aymeric Bouvet:

0.11 introduced some new protocols (say a v3), which cant be understand by old clients (which only know v2 and v1). When clients send request (fetch, produce, or any other request), it send it with the protocol it know (so v2 in your case)
Broker will in this case answer with v2 protocol also, so the client can understand it. So in v1, where there were no timestamp, kafka would only serve data without timestamp to an appliation which request v1, and would serve timestamp also for client which request v2.
If the client is updated to v3, it will add headers for example

If a client know v3 but broker is old (0.10.0 and know only v2), if client send v3 it will be rejected. But clients can also know what version brokers support (this is known via an apiversionrequest), so client may first check what broker support, and use the best protocol in this case.
This is automatically done by java clients (which are used by kafka spark)

Yeah the wire protocol may be compatible, but not necessarily the client libs at compile time. For Spark, I honestly don't know, haven't tried it yet (I'm actually working on starting to dislodge 0.8 support in Spark).
The particular problem here is that Spark Streaming is binding against Kafka libraries and it seems like it doesn't like the 0.11 libraries. It's not clear.
For here, for now, I'd definitely use the CDH libraries as a whole as it's definitely supposed to work with those. I'm releasing 2.5.0 which is intended to be compatible with all the latest CDH components (which is what's on my cluster). You can see the 2.5.x branch now and build it, even.

I'll wait for released 2.5.0 and use that. Going to downgrade Kafka cluster to 0.10
Hopefully everything will work fine.

OK I cut the release: https://github.com/OryxProject/oryx/releases/tag/oryx-2.5.0

This is really strange. I have tried 2.5.0 and it works with Kafka 0.11 - Batch layer creates model successfully, speed layer is working also and sending updates.
The problem is only with serving layer. This is full stacktrace when starting serving layer:
2017-09-12 11:36:14,445 INFO  ServingLayer:94 Configuration:
oryx {
    als {
        decay {
            factor=1
            zero-threshold=0
        }
        hyperparams {
            alpha=1
            epsilon=1.0E-5
            features=10
            lambda=0.001
        }
        implicit=true
        iterations=10
        logStrength=false
        no-known-items=false
        rescorer-provider-class=null
        sample-rate=1
    }
    batch {
        storage {
            data-dir="hdfs:///user/mcimerman/als-example/data/"
            key-writable-class="org.apache.hadoop.io.Text"
            max-age-data-hours=-1
            max-age-model-hours=-1
            message-writable-class="org.apache.hadoop.io.Text"
            model-dir="hdfs:///user/mcimerman/als-example/model/"
        }
        streaming {
            config {
                spark {
                    io {
                        compression {
                            codec=lzf
                        }
                    }
                    logConf=true
                    serializer="org.apache.spark.serializer.KryoSerializer"
                    speculation=true
                    ui {
                        showConsoleProgress=false
                    }
                }
            }
            deploy-mode=client
            driver-memory="1g"
            dynamic-allocation=false
            executor-cores=2
            executor-memory="4g"
            generation-interval-sec=300
            master=yarn
            num-executors=1
        }
        ui {
            port=4040
        }
        update-class="com.cloudera.oryx.app.batch.mllib.als.ALSUpdate"
    }
    default-streaming-config {
        spark {
            io {
                compression {
                    codec=lzf
                }
            }
            logConf=true
            serializer="org.apache.spark.serializer.KryoSerializer"
            speculation=true
            ui {
                showConsoleProgress=false
            }
        }
    }
    id=ALSExample
    input-schema {
        categorical-features=null
        feature-names=[]
        id-features=[]
        ignored-features=[]
        num-features=0
        numeric-features=null
        target-feature=null
    }
    input-topic {
        broker="exp-slon-1.maas:9092,exp-slon-2.maas:9092,exp-slon-3.maas:9092,exp-slon-4.maas:9092,exp-slon-5.maas:9092"
        lock {
            master="exp-slon-2.maas:2181,exp-slon-3.maas:2181,exp-slon-4.maas:2181"
        }
        message {
            key-class="java.lang.String"
            key-decoder-class="org.apache.kafka.common.serialization.StringDeserializer"
            message-class="java.lang.String"
            message-decoder-class="org.apache.kafka.common.serialization.StringDeserializer"
            topic=OryxInput
        }
    }
    kmeans {
        evaluation-strategy=SILHOUETTE
        hyperparams {
            k=10
        }
        initialization-strategy="k-means||"
        iterations=30
        runs=3
    }
    ml {
        eval {
            candidates=1
            hyperparam-search=random
            parallelism=1
            test-fraction=0.1
            threshold=null
        }
    }
    rdf {
        hyperparams {
            impurity=entropy
            max-depth=8
            max-split-candidates=100
            min-info-gain-nats=0.001
            min-node-size=16
        }
        num-trees=20
    }
    serving {
        api {
            context-path="/"
            key-alias=null
            keystore-file=null
            keystore-password=*****
            password=*****
            port=8080
            read-only=false
            secure-port=443
            user-name=null
        }
        application-resources="com.cloudera.oryx.app.serving,com.cloudera.oryx.app.serving.als"
        memory="4000m"
        min-model-load-fraction=0.8
        model-manager-class="com.cloudera.oryx.app.serving.als.model.ALSServingModelManager"
        no-init-topics=false
        yarn {
            cores="4"
            instances=1
        }
    }
    speed {
        min-model-load-fraction=0.8
        model-manager-class="com.cloudera.oryx.app.speed.als.ALSSpeedModelManager"
        streaming {
            config {
                spark {
                    io {
                        compression {
                            codec=lzf
                        }
                    }
                    logConf=true
                    serializer="org.apache.spark.serializer.KryoSerializer"
                    speculation=true
                    ui {
                        showConsoleProgress=false
                    }
                }
            }
            deploy-mode=client
            driver-memory="512m"
            dynamic-allocation=false
            executor-cores=4
            executor-memory="1g"
            generation-interval-sec=10
            master=yarn
            num-executors=2
        }
        ui {
            port=4041
        }
    }
    update-topic {
        broker="exp-slon-1.maas:9092,exp-slon-2.maas:9092,exp-slon-3.maas:9092,exp-slon-4.maas:9092,exp-slon-5.maas:9092"
        lock {
            master="exp-slon-2.maas:2181,exp-slon-3.maas:2181,exp-slon-4.maas:2181"
        }
        message {
            decoder-class="org.apache.kafka.common.serialization.StringDeserializer"
            encoder-class="org.apache.kafka.common.serialization.StringDeserializer"
            max-size=16777216
            topic=OryxUpdate
        }
    }
}

2017-09-12 11:36:14,451 INFO  ServingLayer:123 Starting Serving Layer ALSExample
Sep 12, 2017 11:36:15 AM org.apache.coyote.http11.AbstractHttp11Protocol configureUpgradeProtocol
INFO: The ["http-nio2-8080"] connector has been configured to support HTTP upgrade to [h2c]
Sep 12, 2017 11:36:15 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-nio2-8080"]
Sep 12, 2017 11:36:15 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service [Tomcat]
Sep 12, 2017 11:36:15 AM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/8.5.20
2017-09-12 11:36:15,407 INFO  ModelManagerListener:103 ModelManagerListener initializing
Sep 12, 2017 11:36:15 AM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class [com.cloudera.oryx.lambda.serving.ModelManagerListener]
java.lang.NoClassDefFoundError: kafka/admin/RackAwareMode
	at com.cloudera.oryx.lambda.serving.ModelManagerListener.contextInitialized(ModelManagerListener.java:108)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4745)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5207)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: kafka.admin.RackAwareMode
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 10 more

Sep 12, 2017 11:36:15 AM org.apache.catalina.core.StandardContext startInternal
SEVERE: One or more listeners failed to start. Full details will be found in the appropriate container log file
Sep 12, 2017 11:36:15 AM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [Oryx] startup failed due to previous errors
2017-09-12 11:36:15,447 INFO  ModelManagerListener:153 ModelManagerListener destroying
2017-09-12 11:36:15,447 INFO  ModelManagerListener:173 ModelManagerListener closing
Sep 12, 2017 11:36:16 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-nio2-8080"]

The thing about the serving layer is that it needs the classpath set up differently, manually, via compute-classpath.sh, because it's not a Spark app. Check that the classpath makes sense for your env? it should if it's CDH.

This is result of compute-classpath.sh:
 mcimerman@hadoop-1  /opt/oryx  ./compute-classpath.sh
/opt/cloudera/parcels/SPARK2/lib/spark2/jars/scala-library-2.11.8.jar
/opt/cloudera/parcels/SPARK2/lib/spark2/jars/scala-parser-combinators_2.11-1.0.4.jar
/opt/cloudera/parcels/CDH/jars/zookeeper-3.4.5-cdh5.12.0.jar
/opt/cloudera/parcels/CDH/jars/hadoop-auth-2.6.0-cdh5.12.0.jar
/opt/cloudera/parcels/CDH/jars/hadoop-common-2.6.0-cdh5.12.0.jar
/opt/cloudera/parcels/CDH/jars/hadoop-hdfs-2.6.0-cdh5.12.0.jar
/opt/cloudera/parcels/CDH/jars/commons-cli-1.2.jar
/opt/cloudera/parcels/CDH/jars/commons-collections-3.2.2.jar
/opt/cloudera/parcels/CDH/jars/commons-configuration-1.7.jar
/opt/cloudera/parcels/CDH/jars/protobuf-java-2.5.0.jar
/opt/cloudera/parcels/CDH/jars/snappy-java-1.0.4.1.jar
/opt/cloudera/parcels/CDH/jars/hadoop-yarn-applications-distributedshell-2.6.0-cdh5.12.0.jar

Seems like it make sense.
However, maybe this could be a problem: /opt/cloudera/parcels/CDH/jars/zookeeper-3.4.5-cdh5.12.0.jar sine we've got Confluent Platform installed as an Kafka distribution, but still using Zookeeper installed from CDH Parcels though.

Hm, for what it's worth, it all works fine for me with the runner script. I'm using 2.5.0 and CDH 5.12.0. Just tried it now.

Okay. Going to try with your build (I've been trying with mine 2.5.0 build before you've cut release).

Still getting the same error 😞. No idea why...could it be linked to global Kafka binaries?

So it seems like everything is working fine with CDH Kafka 0.10.2. Apparently I will have to keep separate CDH Kafka cluster for Oryx.

I managed to get it working with 0.10.2.1-cp2 Kafka packages resolved from Confluent Maven repository. It's working fine with Confluent platform 3.3.0.1 which uses Kafka 0.11.0.0
I've opened PR: https://github.com/OryxProject/oryx/pull/338

@cimox I updated master to use Kafka 0.11 directly. Works OK for me after some light testing so far.
https://github.com/OryxProject/oryx/commit/80e16366b85e1cfa1634a53cba6945b7531d549b

authors: cimox srowen

repo: OryxProject/oryx


text:
🛑 CrdPro is down

In a7b98cd, CrdPro (https://crdpro.cc) was down:

HTTP code: 502
Response time: 481 ms

Resolved: CrdPro is back up in d3087c9.

repo: OsintUK/Up-or-Down


text:
width & height custom config scale

Would be great to have config also for width and height similar like tailwindcss has.
Also, it would be great if width and height could support percentage values. e.q. w-1/2

Would be great to have config also for width and height similar like tailwindcss has.
Also, it would be great if width and height could support percentage values. e.q. w-1/2

for width & height percentage, it's already available w/{percentage-amount}
example:
w/50 the output is 50% width based on window screen
but for percentage width string like { width: '50%' } that not based on window screen, we will update in next release

authors: DeVoresyah branislaav

repo: OsmiCSX/osmicsx


text:
Capstone home and about page

These are the things that I did on Capstone-Project-Concert --Page Repo:

Create a Feature Branch.
Add index.html file.
Add style.css file.
Add Javascript file.
Add a descriptive readme file.
Add gitignore on node modules.
Add images folder.
Render the speakers dynamically.
Add functionality to the menu icon using Javascript.
Build a separate About page.
Install and fix the Linter error.
Push to Remote
Record a video and Add it to READme.MD file.
Deploy the Pages in Git and add the link in READme.md file.

Loom Video Link

Video-Link

@Ismailco
Approved the changes that I made as per the previous reviewer @Lucas-Masaba


text:
Docker 2

Выполнены задания docker, docker-2, включая задания со звёздочками.
Такими командами можно развернуть монолит у себя на облаке:
pushd docker-monolith/infra/packer/
# You may create variables file from variables.json.example
packer build -var-file=variables.json docker.json
popd

pushd docker-monolith/infra/terraform/
# Terraform requires variables, see terraform.tfvars.example
terraform apply -auto-approve
popd

pushd docker-monolith/infra/ansible/
# Inventory populates with terrafrom apply
ansible-playbook monolith.yml --tags app
popd

@mrgreyves
Владимир, а где это описано? В тексте ДЗ не нашёл. Там тот же пайплайн для GA, что и в {{ username }}_infra?

@mrgreyves
Прошляпил, исправился.


text:
Docker 3

Выполнено ДЗ к занятию №17: Docker образы. Микросервисы

[X] Основное ДЗ
[X] Задание со *

В процессе сделано:

Наше приложение разбито на несколько компонентов
Запущено  микросервисное приложение
Произведен запуск контейнеров с другими сетевыми алиасами
Собран образ сервиса UI на основе Alpine Linux
Произведен запуск приложения с volume

Как запустить проект:

подробные инструкции приведены в README.md

Как проверить работоспособность:

Например, перейти по ссылке http://51.250.5.113:9292/

PR checklist

[X] Выставил label с номером домашнего задания
[X] Выставил label с темой домашнего задания

Приветствую! Заапрувьте пожалуйста ) Пока еще можно ))


text:
what does it do

what does it do
“原生之初”?

@baka4n

repo: Over-Run/overrunwindow


text:
Supplying references to equal and compare

I'm putting structs into a list like so:
typedef struct __MyData {
    int data;
} MyData;

inline bool MyData__equals(const MyData z1, const MyData z2) {
    return z1.data == z2.data;
}

LIST_DEF(MyList, MyData, M_OPEXTEND(M_POD_OPLIST, EQUAL(MyData__equals)))

When comparing two list elements, the MyData__equals function gets supplied two elements copied by value. Is it somehow possible to supply them as pointers, so they won't be copied through the stack? I have tried to use M_IPTR which gets me this:
inline bool MyData__equals(const MyData *z1, const MyData z2) {
    return z1->data == z2.data;
}

LIST_DEF(MyList, MyData, M_OPEXTEND(M_POD_OPLIST, EQUAL(MyData__equals M_IPTR)))

So I'm half-way there. Is there a way to also get the second value passed as pointer?

The best and simple solution will be to update your definition of the structure (to use the [1] trick) like this:
typedef struct __MyData {
    int data;
} MyData[1];

inline bool MyData__equals(const MyData z1, const MyData z2) {
    return z1->data == z2->data;
}

LIST_DEF(MyList, MyData, M_OPEXTEND(M_A1_OPLIST, EQUAL(MyData__equals)))

You will still allocate the memory by declaring a variable (global or local) and you will always pass the structure by reference. A consequence is that you'll always need to access the data using '->' instead of '.'.

Another way to do it for only some targeted function is through macro encapsulation:
inline bool MyData__equals(const MyData *z1, const MyData *z2) {
    return z1.data == z2.data;
}
#define MyData__equals_val(z1, z2) MyData__equals(&(z1),&(z2))
LIST_DEF(MyList, MyData, M_OPEXTEND(M_POD_OPLIST, EQUAL(MyData__equals_val)))

A third solution is to use the new available API transformation:
inline bool MyData__equals(const MyData *z1, const MyData *z2) {
   return z1->data == z2->data;
}
LIST_DEF(MyList, MyData, M_OPEXTEND(M_POD_OPLIST, EQUAL(API_6(MyData__equals ))))

authors: P-p-H-d jens-totemic

repo: P-p-H-d/mlib


text:
fatal design flaw

This looks like a great project and I am glad you have been thinking about decentralization. I have some issues with this though (only one really). It is built on IOTA.
IOTA run by people (come_from_beyond and other questionable characters whose understanding of Math can best be summed up as "Numberwang"). People who think that collision resistance doesn't have to be a primary feature of a hash functions.
the IOTA guys have been ignorant towards any criticism from the cryptography experts (ask Matthew D. Green and many others).  IOTA is a total clusterfuck. Don't use it. Or if you do please check at least twitter to see what people with an actual idea of cryptography are saying.
Please consider rewriting this or alternatively push your commits to /dev/null ... thanks.
https://twitter.com/peterktodd/status/938397135862714373
https://twitter.com/matthew_d_green/status/1027197175456034818
https://twitter.com/dionyziz/status/968088282457354240
PS: crypto means cryptography

This is not how you constructively interact with an open source project.

Hey @jbauernberger !
Would you mind elaborating your points? While being open to constructive criticism I myself don't consider twitter as a valid source of information, but rather like to see scientific proof. I'd be pleased if you were able to cite such sources.
Anyway I am well aware that there have been some rumors about a year ago. I'd request you to deepen my understanding of why they are relevant for this project and how they are an issue.
Please take into account that:

The rumors were about IOTAs signing scheme for transactions that carry value
The dWeb project does not use any transactions carrying value
The dWeb project does not on rely nor even use the IOTA signing scheme

In addition please make a proposal what the alternative would be.
Please take into account that:

A decentralized approach is needed
A feeless approach is mandatory

I’m closing this issue, since the complain is about a feature of IOTA, which is on the one hand fixed by IOTA and on the other hand not even  integrated in Dweb.page. But in general I welcome any kind of discussion here!

authors: Felandil Noc2 jbauernberger tom-james-watson


text:
Update ml_beginners.md

type conversion to fix #760

This change is

I signed it!

Thank you for the fix!

Review status: 0 of 1 files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

authors: OliverLonghi nsthorat

repo: PAIR-code/deeplearnjs


text:
Custom format for only one language

With the upgrade to v3, unfortunately my Sonarr no longer works the way it did before regards languages. I always want to download tv shows in German. I don't care whether other languages are integrated, the important thing for me is German. How can I configure this? I have followed your guide, but German is only preferred and every second download is in English.
Thanks.

I am also interested in, how download only german dl, or germany.
If there is no matching reault, it should not download english.
Any advice, how to do that?

Easy!
Import this custom format
{   "name": "NOT German",   "includeCustomFormatWhenRenaming": false,   "specifications": [     {       "name": "NOT German",       "implementation": "LanguageSpecification",       "negate": true,       "required": true,       "fields": {         "value": 4       }     }   ] }
In your quality profile(s) give it a score of -50000

Thanks for the quick answer.
Seems to work fine.
You can close this issue.

Still only finding other languages, not german for me.

Still only finding other languages, not german for me. Radarr will only find, what i don't want. But i know german is available on the indexer.

Can you share your quality profile? Also, is your indexer SceneNZBs?

{
"name": "German DL",
"includeCustomFormatWhenRenaming": true,
"specifications": [
{
"name": "German DL",
"implementation": "ReleaseTitleSpecification",
"negate": false,
"required": true,
"fields": {
"value": "(?i)german\s*\.?dl|(?<=\bGerman\b.)(?<!\bWEB[-_. ])\bDL\b|\[DE\+[a-z]{2}\]|\[[a-z]{2}\+DE\]|ger,\s[a-z]{3}\]|\[[a-z]{3}\s*,\s*ger\]"
}
}
]
}
And yes, scnenenzb. :)

@Deschne that is not the quality profile, it's the custom format. The quality profile is where you assign a score to the custom format.
Kannst auch auf deutsch schreiben sonst

Oh, sorry, bei dem Profil hab ich alles ausgewählt. Bei Scenenzbs gibts auf jeden Fall deutsche Titel, Rambo 1 z.B.
Die findet Radarr aber gar nicht erst.

Settings>Indexer
Dort SceneNZBs anklicken und in Sonarr bei den Kategorien TV - DE anklicken, in Radarr Movies - DE anklicken.

Ha, du bist der beste! Danke, darauf wäre ich im Leben nicht gekommen...Spende geht gleich raus.

authors: BenRoe Deschne PCJones elmero1


text:
assertion failed at: "shared/cfe/edgcpfe/lower_name.c", line 9640

libcifpp-2.0.0 from GitHub releases
Environment:

Red Hat Enterprise Linux 8.1
Intel compiler icpc 19.1.3.304
Boost 1.75.0
cmake 3.19.5

[  5%] Built target COMPONENTS
[ 11%] Building CXX object CMakeFiles/cifpp.dir/src/Cif++.cpp.o
/ifs/opt/src/libcifpp-2.0.0/src/Cif++.cpp(3696): internal error: assertion failed at: "shared/cfe/edgcpfe/lower_name.c", line 9640

compilation aborted for /ifs/opt/src/libcifpp-2.0.0/src/Cif++.cpp (code 4)
make[2]: *** [CMakeFiles/cifpp.dir/build.make:108: CMakeFiles/cifpp.dir/src/Cif++.cpp.o] Error 4
make[1]: *** [CMakeFiles/Makefile2:152: CMakeFiles/cifpp.dir/all] Error 2
make: *** [Makefile:160: all] Error 2

Although I'm the last one to claim there are no bugs in my code, this clearly is a problem with the Intel compiler. The assertion failure is a message from within the compiler and it most likely stumbles upon generating template implementations since the line it complains about is the very last line of the source file.
Anyway, this is not something I can fix. You should report this problem to Intel if you want to receive a fix.
regards, -maarten

authors: mhekkel prehensilecode

repo: PDB-REDO/libcifpp


text:
It dosen't work...

It dose absoloutly nothing... the file is empty ._. im so confused help.

Unfortunately PermissionsEx is currently unmaintained and no further updates are planned. Because of this, all pull requests are being closed. If you are looking for an actively maintained permissions plugin, take a look at LuckPerms. It is compatible with major server platforms and can import data from PEX.

repo: PEXPlugins/PermissionsEx


text:
unwrap optionals

Hello. Could not build it on xcode 9 beta 4 . Force unwrapping optionals is probably not the best solution but it fixes it.

Yes you are right, seems like slightly change in SpriteKit API. Thanks for contributing.


text:
endBlock error

<?php $this->startBlock('header', true); ?>

Quickstart

<?php $this->endBlock(); ?>

if i wrote like this, i get
endBlock() called too many times in 'app:quickstart/layout'

There's a bug in the docs, it should be:
<?php if($this->startBlock('header', true)): ?>

Quickstart

<?php $this->endBlock(); endif; ?>

Thanks for respond

authors: dracony naneutral

repo: PHPixie/Demo-Quickstart


text:
Update Application Settings Wiki

Issue description
I'm using v2.0.2, the Wiki is not up to date with the actual settings.config file. Please update.
Steps to reproduce
In my settings.config is can see:
DefaultCentralSslPath
DefaultCentralSslPassword
In the Wiki, I see:
DefaultCentralSslStore
DefaultCentralSslPfxPassword
DefaultPemFilesPath
I suppose the first two are just renames ?
And DefaultPemFilesPath is just missing or not available anymore.

Thanks for noticing that!
The Wiki is actually correct, it was the (default) config that was out of date. You can make the modifications yourself or download the latest version from https://github.com/PKISharp/win-acme/blob/84df8a5b8c5003465020bf5b1d8e3b417ed40eca/src/main/settings_default.config (and rename it to settings.config).
I'll make sure that the correct template is included in the next build.

Fixed in 2.0.4

authors: WouterTinus qpple

repo: PKISharp/win-acme


text:
Feature Request: Ability to delete broswser history

Feature Requested
Thats it. I wanna delete my browser history.
Relevant Screenshot
No response
Checklist

[X] The feature would be useful to more users than just me.

Supported.

authors: Quorafind ianTevesAcc

repo: PKM-er/Obsidian-Surfing


text:
How to perform 1024 or higher resolution training?

Hi, thanks for your work.
I have a question that whether I just change the "--max_image_size 1024 " is enough or not?

We don't support 1024 resolution generation in this version, you can train it by yourself, we plan to release it in the next next version.

repo: PKU-YuanGroup/Open-Sora-Plan


text:
Add mechanism to control application of .nice()

If the forceTickCount property is set on the axis, the call to .nice(...) will take the specified tickCount into account when generating the scale function for the axis. This prevents an issue where the specified min/max value is overridden in the axis scale by the call to .nice() as the scale function is constructed.
An example of the issue may be seen here http://jsbin.com/menejiwuxu where there is a gap between the min-value of the x-axis and the origin of the axis.

For posterity or in case JSBin falls over, the code for the example illustrating the buggy behaviour on the x-axis is as follows:
<!DOCTYPE html>

  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width">
  <title>JS Bin</title>

 scatter chart with axis bug demo

    var xAxisProperty = "x_value";
    var yAxisProperty = "y_value";

    var data = [
          {x_value: 9, y_value: 0.9},
          {x_value: 10.9, y_value: 2},
          {x_value: 15.6, y_value: 0.5},
          {x_value: 23, y_value: 1.8}
        ];

    var svg = dimple.newSvg("#chartContainer", 590, 400);
    var myChart = new dimple.chart(svg);

    var xAxis = myChart.addMeasureAxis("x", xAxisProperty);
    xAxis.overrideMin = 5;
    xAxis.overrideMax = 25;
    xAxis.ticks = 5;

    var yAxis = myChart.addMeasureAxis("y", yAxisProperty);
    yAxis.overrideMin = 0.4;
    yAxis.overrideMax = 2.4;
    yAxis.ticks = 4;

    var scatterSeries = myChart.addSeries(
      [xAxisProperty, yAxisProperty, "Example Series"],
      dimple.plot.bubble,
      [xAxis, yAxis]);
    scatterSeries.data = data;

    myChart.addLegend(130, 10, 400, 35, "right");
    myChart.draw();

Output chart:

In this output chart, the xAxis can be seen to start with a min-value below 5, despite that being explicitly set with the overrideMin property.

repo: PMSI-AlignAlytics/dimple


text:
Bugfix/30

Fixed non-unique attribute variable name in query message

I've tested and confirmed that those two sparql queries seem to be the only ones that were broken. I've tested with the ieee13nodeckt, ieee13nodecktassets, and ieee123. @AAndersn this PR can be merged and issue 30 can be closed.

repo: PNNL-CIM-Tools/CIM-Graph


text:
Using two 16 line radar fusion data as input

Hello, I'm going to use the fusion data of two 16 line lidars as the input. How can I choose the program parameters?

The code in the repo is designed to work with a single LiDAR. This limitation is also described in the corresponding papers. The reason for this is that the points it would generate must fall exactly on the pixels of the range image. You can try to project the fused point cloud onto a custom range image but this is prone to failure as there will be gaps in the range data. If you manage to create a range image without the gaps from your fused LiDAR setup the algorithm should work.

authors: GuoliangPeng niosus


text:
VS Code extension shows 0.9.1

Using VS Code (up to date) on a Mac, I note that the VS Code extension for PRQL has the version number in its documentation of 0.9.2. But the code it generates from from invoices shows a version comment of 0.9.1:
SELECT
  *
FROM
  invoices

-- Generated by PRQL compiler version:0.9.1 (https://prql-lang.org)

https://github.com/PRQL/prql-vscode/releases/tag/0.9.2 doesn't contain prql-js 0.9.2.
I think @max-sixty has misnamed the tag.

Sorry! That was not my best day.
I think this is fine to leave until the next release though — there's no actual difference in the code.

I agree it's OK to leave. How shall we remember to do this when the next release comes out? Leave this issue open?

Best way is probably dependabot, which does it automatically...

authors: eitsupi max-sixty richb-hanover

repo: PRQL/prql-vscode


text:
Join tuple equivalence

What's up?
Currently, the second arg of join takes either:
# - scalars
join rel cond1

# - tuples
join rel {cond1, cond2, cond3}

In #469 we changed filter not to accept tuples and force the user to either use && or multiple filters.
The reasoning for this change is based on a move away from implicit && that are inserted into the tuple to convert it into a boolean expression. One may incorrectly assume the tuple means "any of the conditions" and not "all of the conditions".
@snth left a comment about this here.

Examples of the proposed change:
   from a
-  join b {==a_id}
+  join b (==a_id)

   from a
-  join b {a.name == b.name, a.number = b.number}
+  join b (a.name == b.name && a.number = b.number)

PS: in the first example, parenthesis are required because without them, the join would be parsed as join (b == a_id)

I'm +0.5

Oh, and @max-sixty left a comment here.
I forgot to mention that I'm +1 on this, which is a majority.
It turns out that it issue is just documenting something we already agreed on :D

I would add — it would be great to make a good error message! Even for those not using join already, I'm sure they'll make the mistake given how prevalent the singleton / tuple equivalence is...

I'm +1 on replacing {} with () for join conditions for the reasons mentioned.
One clarification please: does this mean the () is essentially mandatory for any join that has a join condition, i.e. anything other than a CROSS JOIN? I think that would be a good thing because it gives a visual consistency.

I'm -1 on the idea I floated in https://github.com/PRQL/prql/issues/2124#issuecomment-1563462793 of delineating the join conditions with a keyword like on or where.  I don't like how those examples looked and the examples with () are more visually consistent with the rest of the language.

One clarification please: does this mean ...

Yes, () would be mandatory, because you'd be using binary or unary operators, which all require () when used as arguments to functions. It would not be required for cross join, as one can use just true:
from a
join b true

To be honest, I didn't quite understand what you meant in the PR comment, about on or where. Now that I do, I also don't like adding keywords specifically for the join function.

authors: aljazerzen max-sixty snth

repo: PRQL/prql


text:
fix: missing main pipeline

Closes #2581

Thanks a lot! You're on a tear...

authors: aljazerzen max-sixty


text:
refactor: Should prql-lib use char?

I'm less strong on this stuff — I saw a warning and made this change — is it good?

Yeah, probably. I doesn't really matter - I know cases where people just use void * for all pointers - but still, char gives more info that these are strings.

Keep in mind that the libprql_lib.h header file was autogenerated with cbindgen. That there is also a header file for C++ called libprql_lib.hpp which you did not touch.

Keep in mind that the libprql_lib.h header file was autogenerated with cbindgen. That there is also a header file for C++ called libprql_lib.hpp which you did not touch.

Ah, thanks for spotting @vanillajonathan — you're right, it says it at the top of the file 🤦 .
Do you happen to know whether we can change it to generate char? Otherwise I guess this is going to be overwritten next time we autogen...

Do you happen to know whether we can change it to generate char? Otherwise I guess this is going to be overwritten next time we autogen...

I generated a new header file and saw that it generated char, I don't know why.

Ah interesting.
If we can add the command to generate a new header to task build-lib (or task build-c), that would be a good reference, and ensure it keeps up to date...

authors: aljazerzen max-sixty vanillajonathan


text:
Create separate function to compute disposable income

The function  for consumer demand for commodity i, CGE_equations.eqXp(), is given as:
Xp = alpha * ((pf * Ff).sum() - Sp - Td - Fsh + Trf) / pq

The term in brackets, ((pf * Ff).sum() - Sp - Td - Fsh + Trf) is equal to I.
It might be easier to think about how this function matches up to the theory if I were computed in a separate function and passed to the eqXp() equation.

Do you want me to add this function to household.py as well since the functions are split into separate modules?

@duncanhobbs, That'd be great.  I think we will be removing equations.py soon and keeping the functions in separate modules to help with organization.

authors: duncanhobbs jdebacker

repo: PSLmodels/CGE


text:
Wrong slot texture in induction Furnace

Thank you very much for reporting!
the issue will be fixed in the next release


text:
"Client::AUTH_HTTP_TOKEN" authentication method is deprecated

INFO   | PHP    User Deprecated: Using the "Client::AUTH_HTTP_TOKEN" authentication method is deprecated in knp-labs/php-github-api v2.15 and will be removed in knp-labs/php-github-api v3.0. Use "Client::AUTH_ACCESS_TOKEN" instead.
what do you think if we upgrade the client version?

IMHO we can try to upgrade the client version.

it has been fixed! 🚀

authors: AlessandroMinoccheri JellyBellyDev

repo: PUGX/badge-poser


text:
open telem 1 to write data

I want to open("ttyS1",O_RDWR | O_NOCTTY),but it is failed, I can't understand.but if it is ttyS3 and other that is success.and i send 0x0a to serial4/5,but it output 0x0d 0x0a.I don't want 0x0d,how should I do

        int fd= open("/dev/ttyS5",O_RDWR | O_NOCTTY);
        printf("fd = %d",fd);
        cmd_t cmd = {0x81,0x01,0x0a,0x10,0x00,0x32,0x03,0x01,0xFF};
        write(fd,(void *)&cmd,sizeof(cmd));

        close(fd);

this is code

It output 0x81 0x01 0x0d 0x0a...How should I do to cut out 0x0d?

I have comment out the " mavlink start ${MAVLINK_F} " in rcs

I want to open("ttyS1",O_RDWR | O_NOCTTY),but it is failed, I can't understand.but if it is ttyS3 and other that is success.and i send 0x0a to serial4/5,but it output 0x0d 0x0a.I don't want 0x0d,how should I do

ttys1 and ttys2 all failed

did you stop mavlink before you use ttys1?
As we know, in rcS, px4 will open ttys1 for mavlink

did you stop mavlink before you use ttys1?
As we know, in rcS, px4 will open ttys1 for mavlink

I have comment out the "mavlink start ${MAVLINK_F} " in rcs.And ttys1 have stopped mavlink

could you test
mavlink stop-all
at the end of rcS to confirm stop all the mavlink message

I have input mavlink status,the telemetry is ttys2 only.

Can you solve that 0x0a and 0x0d ?  I have add the code  but it is no change
uart_config.c_oflag &= ~(ONLRET | OPOST);
//    uart_config.c_oflag &= OPOST;
uart_config.c_iflag &= ~(IGNBRK | BRKINT | PARMRK | ISTRIP | INLCR | IGNCR | ICRNL | IXON);
//    uart_config.c_oflag &= ~ONLRET;
@jinchengde,please

so have you try ttyS6 through serial5 or other serial port ?
both the same output?

@roseyangyu
Target output ： 0x81,0x01,0x0a,0x10,0x00,0x32,0x03,0x01,0xFF
Actual output ： 0x81,0x01,0x0d，0x0a,0x10,0x00,0x32,0x03,0x01,0xFF
Right？  By the way, the question you asked is not very clear. What you want to ask is that serial port 1 failed to open? Still more output ‘0x0d’?

I'm sorry,I will mention my question again.
I want to use telem1 to output other message,but open ttys1 is failed.So I change serial5 to output.but output  more than one 0x0d in front of 0x0a.I am more anxious to solve 0x0d than that open ttys1 is failed.

all port is more than one  0x0d in front of 0x0a @jinchengde

@roseyangyu  maybe help： https://blog.csdn.net/CNHK1225/article/details/77883701

I try it, hope I can solve it.Thanks@xdwgood

Please use http://discuss.px4.io/ for discussions not related to core flightstack bugs.

authors: jinchengde mhkabir roseyangyu xdwgood


text:
FW SITL intermittent failure

http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/master_nuttx_7.28%2B/3/pipeline
Exiting early?

https://logs.px4.io/plot_app?log=214c70d1-d6c9-4cf7-a655-5faab83328e1
Examples

http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/master_nuttx_7.28%2B/3/pipeline
http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/master_nuttx_7.28%2B/5/pipeline
http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/master_nuttx_7.28%2B/6/pipeline
http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/PR-11234/9/pipeline
http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/master_nuttx_7.28%2B/9/pipeline
http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/PR-11274/2/pipeline/
http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/PR-11277/2/pipeline
http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/improve-slow-landing/3/pipeline
http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/master_nuttx_7.28%2B/14/pipeline

I think the main failure this issue was opened to address has been resolved. Continue the "poll timeout" failure in https://github.com/PX4/Firmware/issues/11380.

What exactly fixed this? Removing the camera?

Mostly, yes.

What does that mean?

authors: dagar julianoes


text:
Tuning sliders (e.g Position control sensitivity) don´t keep their position, return to left att.

No problem to adjust them, but once you leave the tuning page and return, they are back @ left position.
Furthermore, it would be great, if it would be possible to assign them by choice to an Aux channel (potentiometer), so in flight tuning would be easier.

This is in QGC I assume. This should be a QGC issue.

Thx. Do I have to close this, and place it @ QGC issues, or wait till its sure ?

Tuning sliders are just changing parameters, so nothing to do with firmware directly. You'll need to close this and reopen on the QGC side.

authors: DonLakeFlyer GervaisK


text:
master_new_archs_nuttx_upgrade build issues

@dagar master_new_archs_nuttx_upgrade

[ ] build issues VN
[ ] V not working
[ ] Hardlink edit not working edit file in build is not reflected in src
[ ] check nuttx deps
[ ] edit a defconfig does not rebuild

Can you expand on hardlink edit not working?

@dagar updated

Hardlink copy was accidentally disabled (likely from testing OSX).
Touching defconfig and triggering a rebuild is working for me (with and without hardlink edit). Are you talking about through menuconfig?

@dagar - no it was a disable of a I2C I was testing.

Verbose mode readded. You can use V=1 or VERBOSE=1.

@dagar - still some defconfig issues
defconfig changes not in build
make clean
make px4fmu-v3_default
vim # CONFIG_NSH_DISABLE_FREE not used  => CONFIG_NSH_DISABLE_FREE=y
make px4fmu-v3_default
cat build/px4fmu-v3_default/NuttX/nuttx/.config | grep FREE
CONFIG_NSH_DISABLE_FREE=y

nsh> ?
help usage:  help [-v] [<cmd>]

  [           date        free        mount       rmdir       true
  ?           df          help        mv          set         umount
  break       echo        kill        mw          sh          unset
  cat         exec        ls          ps          sleep       usleep
  cd          exit        mkdir       pwd         test
  cp          false       mkfatfs     rm          time

It's rebuilding for me.
 % make px4fmu-v3_default
ninja: no work to do.

 % vim nuttx-configs/px4fmu-v3/nsh/defconfig

 % make px4fmu-v3_default
 [868/868] Creating nuttx_px4fmu-v3_default.px4

% grep -nr CONFIG_NSH_DISABLE_FREE build/px4fmu-v3_default
build/px4fmu-v3_default/NuttX/nuttx/.config:1555:CONFIG_NSH_DISABLE_FREE=y

@dagar but when you run that version free should be gone and it is not.

Oh okay, so it's rebuilding everything, but for each NuttX lib it's simply make in each directory. NuttX itself doesn't recognize that dependency.

I believe all of these issues are resolved.


text:
Update rc.axialracing_ax10_defaults

back to original PWM default values

Thanks!

000

authors: LorenzMeier mhc994 mstuettgen


text:
WIP: Add baro bias state to LPE

This adds a state for baro bias random walk. This allows us to lower baro noise since the low frequency offset is a state.

Here is a plot to show results with GPS on, note how low frequency baro drift doesn't impact altitude estimate.

Without GPS it doesn't move much since it isn't observable without multiple measurements of altitude. I'm going to try with lidar next.

I'm testing this for the baro walk.
The rest are probably going to be improvements for me since I'm using optical flow. I think it would be easier to understand if each improvement was a separate PR though
What was the bug that https://github.com/PX4/Firmware/pull/5381/commits/58d4aeb1bc57433ef260ade3ebce58e21d868510 solves?  (curious)

@JochiPochi this is a work in progress,  I really haven't made a conclusion on what I'm keeping yet. So use at your own risk. 58d4aeb  is only relevant when you have GPS and flow on at the same time or you lose flow and it comes back. In one spot it wasn't resetting the coutn to zero, so the init was immediately passing, causing an init deinit loop.

Right now I'm dealing with an observability issue. How can I detect faults in the distance sensors while simultaneously allowing discrete jumps in terrain altitude as you fly over boxes etc. The fault detection is setup to trigger on large jumps, but boxes obviously cause this as well.  The code currently is setup to allow a smoothly changing ground, and that works, but I'm not seeing terrain changes going purely into the terrain state either, some of it bleeds into the altitude state, so I need to pin that down.

Please rebase on master to fix CI.

Can one of the admins verify this patch?

Ide

authors: JochiPochi LorenzMeier PX4BuildBot jgoppert


text:
如何标注相邻的两个物体

比如上衣和裤子，标注出上衣后标裤子的时候上衣区域不能再标为负样本点，裤子的整体轮廓会包含上衣点集

比如上衣和裤子，标注出上衣后标裤子的时候上衣区域不能再标为负样本点，裤子的整体轮廓会包含上衣点集

空格可以改变标注状态，当标注完上衣后空格生成多边形，此时内部是有颜色的，再次使用空格内部就会变成透明的，此时标注裤子时就可以在上衣的区域添加负点了

repo: PaddleCV-SIG/EISeg


text:
[Feature]: Prefill customer name

Tell us about your feature request
It appears you cannot pass a customer name into Paddle checkout. I'd like to be able to prefill a customer name in the same way that I do the email address so that it appears in the dashboard with the other customer information once a checkout is completed.
What problem are you looking to solve?
It is expected to be able to prefill basic customer data fields
Additional context
No response
How important is this suggestion to you?
Important

Hi @jamescrewdentials,
Thank you for reaching out to us.
This is a very good feedback, we are discussing about this internally and will get back to you once we have more information.

Yes please, this is important in keeping track of users. Also the Customer name column being empty looks weird. Ideally should be filled with the card name used during checkout

I agree, this would greatly help. This is useful for sending the customer emails and having the data in the customer name column being entered correctly. Just one additional field on the checkout (maybe optionally activated) for the full name of the customer or business name to be passed to Paddle after checkout would fix this. Similar to how Stripe is handling that during checkout.

authors: bhupeshpr25 gillesvink jamescrewdentials vijayasingam-paddle

repo: PaddleHQ/paddle-js-wrapper


text:
[Agent] Enhance FileManager and simplify the use of file managers

主要改动如下：

优化FileManager的使用方式。具体而言：对于初级用户，修改全局file manager的使用方式，提供配置全局file manager的功能；对于进阶用户，提供基础的资源管理功能，允许更精细化的资源、对象生命周期控制。以下是对于初级用户的示例用法。用户不需要关心file manager，只需要设置环境变量或全局配置一次。

import os

# 设置环境变量
os.environ["EB_AGENT_ACCESS_TOKEN"] = "xxx"

# 构建和使用各组件，不需要设置access_token，也不需要构建或传递file_manager

# 未设置环境变量

from erniebot_agent.file_io import configure_global_file_manager

async def main():
    # 文件保存在指定位置，且文件夹不会在程序结束后被自动清理
    await configure_global_file_manager(access_token="xxx", save_dir="my/save/dir", prune_on_close=False)

    # 构建和使用各组件，不需要构建或传递file_manager

支持从环境变量读取全局access token等配置项。
新增File.write_contents_to方法，用于将文件内容转存到本地文件中。
修复Makefile中的bug。

更多细节详见comments。

Codecov Report
Attention: 116 lines in your changes are missing coverage. Please review.

Comparison is base (eb4069d) 61.83% compared to head (f4f61e6) 61.40%.
Report is 4 commits behind head on develop.

Files
Patch %
Lines

...t-agent/src/erniebot_agent/file_io/file_manager.py
75.25%
24 Missing :warning:

...ot-agent/src/erniebot_agent/file_io/remote_file.py
36.36%
21 Missing :warning:

...niebot-agent/src/erniebot_agent/utils/temp_file.py
0.00%
18 Missing :warning:

...-agent/src/erniebot_agent/file_io/file_registry.py
58.33%
10 Missing :warning:

.../src/erniebot_agent/file_io/global_file_manager.py
72.22%
10 Missing :warning:

erniebot-agent/src/erniebot_agent/utils/mixins.py
52.63%
9 Missing :warning:

...t-agent/src/erniebot_agent/tools/remote_toolkit.py
62.50%
6 Missing :warning:

erniebot-agent/src/erniebot_agent/utils/logging.py
28.57%
5 Missing :warning:

erniebot-agent/src/erniebot_agent/file_io/base.py
57.14%
3 Missing :warning:

erniebot-agent/src/erniebot_agent/agents/base.py
83.33%
2 Missing :warning:

... and 5 more

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #193      +/-   ##
===========================================
- Coverage    61.83%   61.40%   -0.44%
===========================================
  Files           56       58       +2
  Lines         2780     2920     +140
===========================================
+ Hits          1719     1793      +74
- Misses        1061     1127      +66

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

更新之后factory的get_file_manager好像没有地方使用了，这个文件可以删除吗

更新之后factory的get_file_manager好像没有地方使用了，这个文件可以删除吗

已删除

repo: PaddlePaddle/ERNIE-Bot-SDK


text:
转换ONNX模型时出现量化参数不匹配的问题

在使用Paddle2ONNX转换模型时出现量化参数不对齐的问题
模型链接: PPOCRV3_rec_QAT.zip

@yeliang2258

具体可以见p2o.Add.98节点，存在2个量化参数

https://github.com/PaddlePaddle/Paddle2ONNX/pull/1071


text:
在 Python 代码中预测报错

欢迎您使用PaddleClas并反馈相关问题，非常感谢您对PaddleClas的贡献！
提出issue时，辛苦您提供以下信息，方便我们快速定位问题并及时有效地解决您的问题：

paddlepaddle       2.4.1
paddleclas         2.4.0
训练环境信息：
a. 具体操作系统：Windows
b. Python版本号：Python3.7
c. 虚拟环境：Anaconda

源码如下：
from paddleclas import PaddleClas
import warnings
warnings.filterwarnings('ignore')
infer_imgs = r'D:\Files\python\images\fff587897edb4a49a2497fe10610d335.jpg'
clas = PaddleClas(
model_name='PPLCNet_x0_25',
inference_model_dir='D:/Project/My/python/paddleclas-demo/model/PPLCNet_x0_25',
class_id_map_file='D:/Project/My/python/paddleclas-demo/model/ccj_label_list.txt',
topk=2,
use_gpu=False
)
result = clas.predict(infer_imgs)
print(next(result))
报错代码：
Traceback (most recent call last):
File "D:/Project/My/python/paddleclas-demo/main.py", line 37, in
print(next(result))
File "D:\ProgramData.conda\envs\paddleclas\lib\site-packages\paddleclas\paddleclas.py", line 573, in predict
preds = self.cls_predictor.predict(img_list)
File "D:\ProgramData.conda\envs\paddleclas\lib\site-packages\paddleclas\deploy\python\predict_cls.py", line 109, in predict
batch_output = self.postprocess(batch_output)
File "D:\ProgramData.conda\envs\paddleclas\lib\site-packages\paddleclas\deploy\python\postprocess.py", line 50, in call
tmp = func(x, image_file)
File "D:\ProgramData.conda\envs\paddleclas\lib\site-packages\paddleclas\deploy\python\postprocess.py", line 127, in call
label_name_list.append(self.class_id_map[i.item()])
KeyError: 605
ccj_label_list.txt
0 gun
1 signal
在源码postprocess.py第123行报错
        for i in index:
            clas_id_list.append(i.item())
            score_list.append(probs[i].item())
            if self.class_id_map is not None:
                label_name_list.append(self.class_id_map[i.item()])

这里的index为[605,704]，正常应该是ccj_label_list.txt里[0,1]，请问是哪里配置错误？

已经解决。

windows系统
在"C:\Users\用户名.paddleclas\inference_model\IMN\PPLCNet_x0_25"文件夹的inference.pdiparams、inference.pdiparams.info和inference.pdmodel三个文件替换自己的模型，这里的PPLCNet_x0_25对应训练使用的模型名称。

linux系统
在"/root/.paddleclas/inference_model/IMN/PPLCNet_x0_25"文件夹的inference.pdiparams、inference.pdiparams.info和inference.pdmodel三个文件替换自己的模型，这里的PPLCNet_x0_25对应训练使用的模型名称。

repo: PaddlePaddle/PaddleClas


text:
软件环境
- paddlenlp: 2.4.1
- paddlepaddle-gpu: 2.3.2

重复问题

[X] I have searched the existing issues

错误描述
多卡训练时，如果模型tokenizer没有added_tokens.json和tokenizer_config.json两个文件时，训练会hang住，hang在下载处。

稳定复现步骤 & 代码
--

我这边用以下代码能够复现出你的问题：
from paddlenlp.transformers import BertModel, AutoTokenizer
model_name = "IDEA-CCNL/Randeng-Pegasus-523M-Summary-Chinese"
AutoTokenizer.from_pretrained(model_name)

subprocess.check_output(f"rm ~/.paddlenlp/models/{model_name}/*".split()).decode("utf-8")
AutoTokenizer.from_pretrained(model_name)

后续会针对于这个问题来提交 PR Fix。

authors: gongel wj-Mcat

repo: PaddlePaddle/PaddleNLP


text:
❣ [TTS] MFA 报错 No such file or directory: "xx/xx/xx/train/mfcc/raw_mfcc.0.scp"

一般是因为系统环境问题，可能是某个系统库没有安装，可以查看 --temp_directory 路径下的 logging 文件, 如果使用 MFA 时未指定 --temp_directory, 默认路径在 ${HOME}/Documents/MFA

后台日志：copy-feats: error while loading shared libraries: libgfortran.so.3: cannot open shared object file: No such file or directory
compute-mfcc-feats: error while loading shared libraries: libgfortran.so.3: cannot open shared object file: No such file or directory
需要安装libgfortran3
1、apt-get update
2、sudo apt-get install libgfortran3

find libgfortran.so.3 in your os, copy it to MFA lib. and add MFA lib to LD_LIBRARY_PATH of your conda env variables

加了这个库依然报错，不知道遇到过嘛

我的错误和该issue是一样的，但我的logging文件夹中只有一个文件corpus.log，内容为： Setting up corpus information... Number of speakers in corpus: 1, average number of utterances per speaker: 198.0 Setting up training data... 这要怎么处理呢？

同问，请问你解决了吗，我看上面要安装kaldi前置依赖，但是这个依赖不知道怎么安装。

回复一下，其实这种情况除了log中提到缺少的库，主要还是缺少kaldi前置环境的依赖，使用mfa之前需要先安装kaldi就基本解决了

用官方的方法没办法安装kaldi，请问你是怎么安装的

我的错误和该issue是一样的，但我的logging文件夹中只有一个文件corpus.log，内容为： Setting up corpus information... Number of speakers in corpus: 1, average number of utterances per speaker: 198.0 Setting up training data... 这要怎么处理呢？

同问，请问你解决了吗，我看上面要安装kaldi前置依赖，但是这个依赖不知道怎么安装。

你全局查找一下“make_mfcc.0.log”这个日志，我的日志是在
“/root/Documents/MFA/${root_...}/train/mfcc/log/” 这个目录下，然后cat make_mfcc.0.log 查看这个日志文件的信息，我的报错是“compute-mfcc-feats: error while loading shared libraries: libgfortran.so.3: cannot open shared object file: No such file or directory”，所以使用apt-get install libgfortran3 安装对应的库之后就可以正常使用了

cmd输入
sudo find / -name libgfortran.so.3
找到路径以后
export LD_LIBRARY_PATH=你自己的路径:$LD_LIBRARY_PATH
如：
export LD_LIBRARY_PATH=/home/user/miniconda3/envs/paddle/lib/python3.9/site-packages/paddle/libs/:$LD_LIBRARY_PATH
注意：以上是临时添加，我不会永久添加，请自行研究

repo: PaddlePaddle/PaddleSpeech


text:
paddlespeech唤醒词实例运行

General Question

运行hey_snips模型，需要有hey_snip的数据集，数据集现在无法下载，请问有没有分享的hey_snip数据集。如果要做自定义唤醒词，应该怎么制造数据集。

同问

我也想知道，如何在网页例子中添加唤醒词

有没有相关的能下载的语料，有没有预训练模型


text:
OpenCL实现支持

看到operator下面大部分cl文件都是空的，是没有做这方面的支持是么？想了解一下opencl对gemm的实现情况，和使用clblas，clblast的性能对比

@codeWorm2015 有具体的性能数据可以共享出来么？

same issue
arm-linux开启GPU运行显示 Segmentation fault · Issue #1593 · PaddlePaddle/paddle-mobile
https://github.com/PaddlePaddle/paddle-mobile/issues/1593

repo: PaddlePaddle/paddle-mobile


text:
Make sure protoc installed.

Resolves #14

Oh guys I intentionally had chosen ONNX 1.0. 1.1 has some issues on Mac due to protoc compatibility. Is it possible to reconsider this?

Can you upgrade protoc? I found the 1.0 version is too older, so change to the latest one.

authors: kuke pkuyym varunarora

repo: PaddlePaddle/paddle-onnx


text:
updated version of php to 5.3

This change is

Reviewed 3 of 3 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

repo: PagaMasTarde/autoindex


text:
Considering using Scheduler but... is it still maintained

Hi
I am evaluating different alternatives to schedule tasks, and this is a good candidate as we are basing a lot of out work on Scala, Akka, Cassandra and Kafka.
But when I browse through, I do not see any active work on it, quite some pieces of the code is 3 years old, the Cassandra migration directory uses deprecated cassandra calls.
So is this project still maintained ?
Is the solution still in use ?
-Tobias Eriksson

Hi @tobiaseriksson, scheduler is still in use but isn't receiving regular updates. If you want to use it, you should be prepared to take on some maintenance yourself. Note that the lib hasn't been upgraded to Scala 2.12 or 2.13.

Thanx for quick reply David.
Trying to read inbetween the lines :-) are you saying that inhouse you still maintain it, but the github repo is not that frequently updated ?
From what I can see, Scala needs to be upgraded, kafka was like 0.6 or was it 0.10 and now they just released 2.4, and cassandra I am having a feeling you base it off of 2.x something, and it is 3.11. So a little bit of work then.
Just cloned it and built it, but got some errors, haven't had the time to look more into it
-Tobias

I'm afraid that in-house it does not see frequent maintenance :-(. The public GH repo is what we use internally as well.
I believe we're running it against Cass 2.x and Kafka 1.x.x right now.

authors: DWvanGeest tobiaseriksson

repo: PagerDuty/scheduler


text:
Question Anwering Query

name: b) New feature - Question Answering Query
about: Select this if you're adding a new feature to libra.
A query that fits a pre-trained BERT QA model to the dataset (more explained later).
labels: feature

This pull request closes #252
- What I did
I developed a query that fine-tunes a question answering model on the user's dataset.
- How I did it
I used the cdqa library which allowed me to use and fine tune a pre-trained BERT based QA model. Their repo can be found here: https://github.com/cdqa-suite/cdQA
- How to verify it
Go through the instructions in this colab notebook to run the code: https://colab.research.google.com/gist/pranavnt/96b426f9a44f0c0ef6abf38bf81fbded/qa-testing.ipynb

[ ] I updated the docs.
PR to docs will certainly come before the next release, if not within a few days

This pull request adds a new feature to libra. @Palashio, could you please take a look at it?

@pranavnt This looks alright but I have a couple of concerns here that need to be addressed before merging. 1st cdqa is built on top of Pytorch which we decided to not be a part of our Libra for now. Also, you don't seem to be using our common framework of allowing the user to specify column names or extracting those from the command which would be a helpful feature to have in this setting. Also, there seems to be no modularity in the training loop which is inconsistent with other queries. Let's chat more on slack!

Sounds good! I'll close this PR and reopen once I fix those issues

authors: anas-awadalla pranavnt

repo: Palashio/libra


text:
Need for help with PR #176

From this PR #176
For adding the Response Example, I am not able to find the documentation regarding it and what should be updated in the open apni specification for adding the response example.

Hi @snehilms have you read through these docs?
https://swagger.io/docs/specification/adding-examples/

authors: snehilms sserrata


text:
Session timed out error message on plan or apply

Describe the bug
Frequently receive 'Error: Session timed out' when trying to run plan or apply.
Expected behavior
Should finish plan and display changes or apply planned changes, every time.
Current behavior
Terraform quits with error 'Session timed out' frequently but not always.
Possible solution
This problem can be resolved by forcing terraform to only perform one operation (using -parallelism=1 to disable parallel requests) which is not a practical fix. Other possible solutions could include:

Retry with exponential back-off in case it doesn't exist
Tweak the way provider handles parallel requests.

Steps to reproduce
The following is from a redacted/example plan where config matches production but values have been replaced with dummy data. Problem can be reproduced in lab with separate firewall and terraform host. It is not re-producible when Ansible is used with PanOS Collection with the same firewall and host.
Terraform code:

############
tftest2/main.tf
############
module "firewall_rules_app001" {
  source            = "./module/panos-worker"

  app_name          = "APP001"
  pat_port          = "10001"
  internal_endpoint = "app001.internal"
  internal_port     = 443
  internal_protocol = "tcp"
  app_type          = ["web-browsing", "ssl"]
  rule_state        = false # do not disable rule
}

module "firewall_rules_app002" {
  source            = "./module/panos-worker"

  app_name          = "APP002"
  pat_port          = "10002"
  internal_endpoint = "app002.internal"
  internal_port     = 443
  internal_protocol = "tcp"
  app_type          = ["web-browsing", "ssl"]
  rule_state        = false # do not disable rule
}

module "firewall_rules_app003" {
  source            = "./module/panos-worker"

  app_name          = "APP003"
  pat_port          = "10003"
  internal_endpoint = "app003.internal"
  internal_port     = 443
  internal_protocol = "tcp"
  app_type          = ["web-browsing", "ssl"]
  rule_state        = false # do not disable rule
}

module "firewall_rules_app004" {
  source            = "./module/panos-worker"

  app_name          = "APP004"
  pat_port          = "10004"
  internal_endpoint = "app001.internal"
  internal_port     = 443
  internal_protocol = "tcp"
  app_type          = ["web-browsing", "ssl"]
  rule_state        = false # do not disable rule
}

module "firewall_rules_app005" {
  source            = "./module/panos-worker"

  app_name          = "APP005"
  pat_port          = "10005"
  internal_endpoint = "app001.internal"
  internal_port     = 443
  internal_protocol = "tcp"
  app_type          = ["web-browsing", "ssl"]
  rule_state        = false # do not disable rule
}

module "firewall_rules_app006" {
  source            = "./module/panos-worker"

  app_name          = "APP006"
  pat_port          = "10006"
  internal_endpoint = "app001.internal"
  internal_port     = 443
  internal_protocol = "tcp"
  app_type          = ["web-browsing", "ssl"]
  rule_state        = false # do not disable rule
}

module "firewall_rules_app007" {
  source            = "./module/panos-worker"

  app_name          = "APP007"
  pat_port          = "10007"
  internal_endpoint = "app001.internal"
  internal_port     = 443
  internal_protocol = "tcp"
  app_type          = ["web-browsing", "ssl"]
  rule_state        = false # do not disable rule
}

module "firewall_rules_app008" {
  source            = "./module/panos-worker"

  app_name          = "APP008"
  pat_port          = "10008"
  internal_endpoint = "app001.internal"
  internal_port     = 443
  internal_protocol = "tcp"
  app_type          = ["web-browsing", "ssl"]
  rule_state        = false # do not disable rule
}

############################
tftest2/module/panos-worker/main.tf
############################
resource "panos_address_object" "this" {
  name        = "int_${var.app_name}_fqdn"
  vsys        = "vsys1"
  type        = "fqdn"
  value       = var.internal_endpoint
  description = "${var.app_name}"
  tags        = ["dmz"]
}

resource "panos_service_object" "this" {
  name        = "${var.internal_protocol}-${var.pat_port}"
  vsys        = "vsys1"
  protocol    = var.internal_protocol
  description = "${var.app_name}"
  destination_port = var.pat_port
  tags      = ["dmz"]
}

resource "panos_security_rule_group" "this" {
  depends_on = [
    panos_service_object.this
  ]

  vsys                    = "vsys1"
  rule {
    name                  = var.app_name
    description           = "Rule for ${var.app_name}"
    type                  = "universal"
    tags                  = ["dmz"]
    source_zones          = ["untrust"]
    source_addresses      = ["any"]
    negate_source         = false
    source_users          = ["any"]
    hip_profiles          = ["any"]
    destination_zones     = ["trust"]
    destination_addresses = ["any"]
    negate_destination    = false
    applications          = var.app_type
    services              = [panos_service_object.this.name]
    categories            = ["any"]
    action                = "allow"
    log_setting           = "logtoexternal"
    log_start             = false
    log_end               = true
    disabled              = var.rule_state
    schedule              = ""
    virus                 = "default"
    spyware               = "default"
    vulnerability         = "default"
    url_filtering         = ""
    wildfire_analysis     = "default"
    data_filtering        = ""
  }
}

################################
tftest2/module/panos-worker/_variables.tf
################################
variable "app_name" {
  type = string
}

variable "pat_port" {
  type = number
}

variable "internal_endpoint" {
  type = string
}

variable "internal_port" {
  type = number
}

variable "internal_protocol" {
  type = string
}

variable "app_type" {
  type = list(string)
  description = "List of applications - e.g. web-browsing, ssl"
}

variable "rule_state" {
  type = bool
  description = "Set to true if you would like to disable the rules on the firewall"
}

###############################
tftest2/module/panos-worker/_provider.tf
###############################
terraform {
  required_providers {
    panos = {
      source  = "paloaltonetworks/panos"
      version = "~> 1.6"
    }
  }
}

provider "panos" {
  hostname = "xx.xx.xx.xx"
  username = "xxxxxxxxx"
  password = "xxxxxxxxx"
  timeout  = 60
}

Screenshots
USER@HOST:~/gitlab-repos/tftest2 $ terraform apply
module.firewall_rules_app003.panos_address_object.this: Refreshing state... [id=vsys1:int_APP003_fqdn]
module.firewall_rules_app003.panos_service_object.this: Refreshing state... [id=vsys1:tcp-10003]
module.firewall_rules_app003.panos_security_rule_group.this: Refreshing state... [id=vsys1:0::QVBQMDAz]
module.firewall_rules_app004.panos_address_object.this: Refreshing state... [id=vsys1:int_APP004_fqdn]
module.firewall_rules_app004.panos_service_object.this: Refreshing state... [id=vsys1:tcp-10004]
module.firewall_rules_app001.panos_address_object.this: Refreshing state... [id=vsys1:int_APP001_fqdn]
module.firewall_rules_app001.panos_service_object.this: Refreshing state... [id=vsys1:tcp-10001]
module.firewall_rules_app004.panos_security_rule_group.this: Refreshing state... [id=vsys1:0::QVBQMDA0]
module.firewall_rules_app007.panos_service_object.this: Refreshing state... [id=vsys1:tcp-10007]
module.firewall_rules_app007.panos_address_object.this: Refreshing state... [id=vsys1:int_APP007_fqdn]
module.firewall_rules_app001.panos_security_rule_group.this: Refreshing state... [id=vsys1:0::QVBQMDAx]
module.firewall_rules_app006.panos_address_object.this: Refreshing state... [id=vsys1:int_APP006_fqdn]
module.firewall_rules_app006.panos_service_object.this: Refreshing state... [id=vsys1:tcp-10006]
module.firewall_rules_app005.panos_service_object.this: Refreshing state... [id=vsys1:tcp-10005]
module.firewall_rules_app005.panos_address_object.this: Refreshing state... [id=vsys1:int_APP005_fqdn]
module.firewall_rules_app002.panos_address_object.this: Refreshing state... [id=vsys1:int_APP002_fqdn]
module.firewall_rules_app002.panos_service_object.this: Refreshing state... [id=vsys1:tcp-10002]
module.firewall_rules_app008.panos_address_object.this: Refreshing state... [id=vsys1:int_APP008_fqdn]
module.firewall_rules_app008.panos_service_object.this: Refreshing state... [id=vsys1:tcp-10008]
module.firewall_rules_app007.panos_security_rule_group.this: Refreshing state... [id=vsys1:0::QVBQMDA3]
module.firewall_rules_app005.panos_security_rule_group.this: Refreshing state... [id=vsys1:0::QVBQMDA1]
module.firewall_rules_app006.panos_security_rule_group.this: Refreshing state... [id=vsys1:0::QVBQMDA2]
module.firewall_rules_app002.panos_security_rule_group.this: Refreshing state... [id=vsys1:0::QVBQMDAy]
module.firewall_rules_app008.panos_security_rule_group.this: Refreshing state... [id=vsys1:0::QVBQMDA4]

Warning: Interpolation-only expressions are deprecated

  on module/panos-worker/main.tf line 6, in resource "panos_address_object" "this":
   6:   description = "${var.app_name}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.

(and 15 more similar warnings elsewhere)

Error: Session timed out

USER@HOST:~/gitlab-repos/tftest2 $

Context
Trying to configure a number of firewalls with security rules and address objects for multiple apps and running plan/apply multiple times a day. The pipeline is failing frequently and needs to be re-run manually.
Your Environment

Version used:

Tested on Redhat Linux 8 and also from within a docker container on Ubuntu VM
Terraform Version v0.13.4
PanOS Provider v1.6.3
PA-VM VM-100 on v8.1.17 and VM-300 on v9.1.6

Your provider timeout is set to 60 right now - if you increase that (say 120 or whatever) does it finish ok?

Your provider timeout is set to 60 right now - if you increase that (say 120 or whatever) does it finish ok?

Setting it to any higher than 60 results in an error as it is the maximum timeout value.

I'll increase the max from 60 to 600 in the next provider release.

I'll increase the max from 60 to 600 in the next provider release.

Your provider timeout is set to 60 right now - if you increase that (say 120 or whatever) does it finish ok?

Setting it to any higher than 60 results in an error as it is the maximum timeout value.

I've seen this issue as well.

Your provider timeout is set to 60 right now - if you increase that (say 120 or whatever) does it finish ok?

Setting it to any higher than 60 results in an error as it is the maximum timeout value.

I've seen this issue as well.

Ok, v1.7.0 is out now.  I set the timeout to 600 sec now, IIRC, so increase that and let me know the results.

Ok, v1.7.0 is out now.  I set the timeout to 600 sec now, IIRC, so increase that and let me know the results.

My environment:
Tested on macOS
Terraform Version v0.14.5 and v0.15
PanOS Provider v1.7.0
PA-220, version 10.0.3
I see the exact same results

For those still having this issue with 1.7, instead of using username and password try generating an API key using the Go code provided in the TF provider documentation and pass that to TF. This seems to have solved the issue for me.

I slightly misspoke above:  I didn't increase the default timeout to 600 sec, I increased the max timeout to 600 sec.  I'm uncertain if having a max timeout is a useful thing or not..  Will reassess after this issue is resolved.
With regards to what @vabagaria said above - I actually removed that chunk of documentation because I wasn't sure if it was a useful thing to have or not.  So instead of asking people to install pango and compile code to get a simple answer, I've just released another provider version, 1.8.0, which has panos_api_key that you can use to find out the API key for the provider auth you've supplied.

@vabagaria
Seems like this issue is resolved..?

@shinmog I haven't been able to reproduce the issue on tf-plan with 1.8 but I have gotten session time out errors on tf-apply.

So you have the timeout set to 600 seconds and it times out..?  Does it actually sit around and take those 10 minutes or..?

@shinmog I'm also seeing this issues or at least something very similar.

So you have the timeout set to 600 seconds and it times out..? Does it actually sit around and take those 10 minutes or..?

For me, there is definitely not a 10 minute wait before getting the session timeout errors. The duration of the session timeout variables does not seem to impact how quickly these errors are thrown, regardless of whether they are set via terraform configuration or via environment variables read by pango directly.

I've been setting the following in my env to work around this: TF_CLI_ARGS_destroy="-parallelism=2"
Since I see this issue with a few local exec provisioners running python pa libraries which cover configure resources not supplied by this terraform provider I can only assume this is an issue with the PA server or a general issue with both client modules.
I think parallel requests are causing a Session timed out, not any http or tcp/network issues. The Session timed out issue oddly enough happens with a server HTTP response of 200 ok...
To figure out if this was really Parallel requests I setup a test environment which creates many static entries in the PA DNS proxy, this uses python, but again I think the issues are related. When run in series I've completed over 4k requests with no timeout issues, however, running this test in parallel ~10-20 threads, I'm able to occasionally get a session timeout, upping to ~60 threads concurrently creating 10 records I'm able to repro the timeout consistently.
I have created a pr for pan-python which immediately reads and closes the http connection in a context manager.. With that in place I don't see the python issue anymore, but I saw it far less frequently in python than pango/tf to begin with. I believe closing the connections just lowers the likelihood of hitting the problem and I'm seeing it go because there are more resources defined with terraform panos resources.
My guess now is that there's something up with the server, it either struggles with responding to high volumes of requests, or there's some random collision that can occur.
If this is a PA server error would something like the following be possible? Perhaps with exponential backoffs?
https://github.com/PaloAltoNetworks/pango/compare/master...jubrad:retry-on-timeout?expand=1

Ok, if this is a parallelism issue, then I've come across this already in the provider:
https://github.com/PaloAltoNetworks/terraform-provider-panos/issues/264#issuecomment-788118806

authors: harsh-vm jubrad koelee shinmog vabagaria


text:
remove sleep 10

I'm not sure the reasons for having this sleep for 10 s before getting a job, but it seems unnecessary especially in push mode where the job description file is already present. I didn't notice any bad effects from testing jobs without the sleep.

I don't remember - possibly a useless remnant of early testing. This PR was to master so I'll close it, but I have removed the line from the dev version. Thanks.

authors: PalNilsson davidgcameron


text:
FI: JYSK

Brand : JYSK
Category : furniture
Opening rule : shops open as usual
Wikipedia / Wikidata : https://fi.wikipedia.org/wiki/Jysk
Opening hours : https://jysk.fi/myymalat
Source : https://jysk.fi/tietoa-koronaviruksesta

Integrated, thanks :+1:

authors: PanierAvide lehtojaa


text:
Found small bug in test for FileObjects.ofLines

Test was supposed to check the validity of the varargs overload, but it instead tested the Iterable overload. This is fixed now.
Signed-off-by: CC007 Coolcat_the_best2@Hotmail.com

Thanks for the PR!

repo: Pante/elementary


text:
chore(develop): release quetzalcoatl 1.0.2

:robot: I have created a release beep boop
1.0.2 (2024-09-24)
Bug Fixes

quetzalcoatl: remove redundant compose file (#45) (481a13b)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

quetzalcoatl/v1.0.2
:sunflower:

repo: Pantheonix/Asgard