text:
Ansible... and then AWS Provider

Scott,
Everything I did in labs 01 and 02 were using windows 10.  When I got to lab 3 and tried to get ansible installed, I realized that I needed to be on Linux or Mac.  I choose to use WSL running Ubuntu on my windows machine.  Is this ok?
Initially I thought that I would just run Ansible on Ubuntu, and leave the Terraform on my windows machine, but run into issues and abandon that quickly.  I get the following error:
$ python ec2.py
region name: None likely not supported, or AWS is down.  connection to region failed.
instead of TSHOOT the above, I then installed Git, Terraform, and Ansible on my WSL Ubuntu machine.  Ran through the labs again... but again run into the following issues.  See below.  FYI.  I am able to ssh to my instances on AWS from my WSL Ubuntu machine, so I know that is working.  I am using the correct Region and key_pair, but getting the error below.
Issue1:  Locking (Solved... workaround using "-lock=false" flag.  Is this occurring because I have the same GIT project cloned on my windows machine, with terraform init  / plan / apply performed?  I did destroy before implementing the below.
$ terraform plan
Error: Error locking state: Error acquiring the state lock: open terraform.tfstate: permission denied
Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and try
again. For most commands, you can disable locking with the "-lock=false"
flag, but this is not recommended.
Issue 2:  No valid Credentials (provider.aws)
$ terraform plan -lock=false
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
Error: Error refreshing state: 1 error(s) occurred:
provider.aws: No valid credential sources found for AWS Provider.
Please see https://terraform.io/docs/providers/aws/index.html for more information on
providing credentials for the AWS Provider

@melshman The second issue, at least, can be solved by providing proper AWS credentials to Terraform. If you have the AWS CLI installed, configured, and working correctly, then Terraform will automatically pull the appropriate credentials. If not, then you'll need to supply credentials. See https://www.terraform.io/docs/providers/aws/index.html for details on supplying credentials for Terraform to use.
The first issue, on the other hand, isn't quite as clear-cut. If you've destroyed all the infrastructure, then you don't technically need to persist the state. I would first verify that the relevant AWS infrastructure has been destroyed (you can use the AWS console or the AWS CLI, but the console is probably faster/easier), then just delete all the *.tfstate files (as well as *.tfstate.backup and *.tfstate.lock files). This will clear out the Terraform state, which you can then recreate upon the next terraform plan.
I should probably update the docs to indicate that having the AWS CLI installed, configured, and working properly is a prerequisite. If you're feeling adventurous and would like to try your hand at a pull request, feel free to give that a swing.
(Note that my replies may be delayed as I'm out on vacation this week.)

Scott,
This can wait!  Enjoy your vacation.  I am just trying to move forward when I can.  Not rush at all.  I really appreciate the opportunity to get some instruction on this topic.
Success followed by the initial issue.
i installed AWS CLI and confirmed access key and secret.  that worked.  Afterwards i was able to perform the terraform plan and terraform apply and resources properly created in AWS.
Note:  I still could not get around using '-lock=false' in both the terraform plan and terraform apply commands.   Not sure what is going on there.  but not stopping progress with work around so moved forward.
The Issue:  Region not supported - connection to region failed
The issue I am at now is that I am getting the same error I started off this whole battle with back on Windows 10.  first sentence of my first post on this issue above.
$ python ec2.py --list
region name: None likely not supported, or AWS is down.  connection to region failed.
I changed the ec2.ini to use 'us-east-1' instead of auto.  same error upon running, but now it says 'us-east-1'
$ sudo python ec2.py
region name: 'us-east-1' likely not supported, or AWS is down.  connection to region failed.m
Moving on... to run Ansible playbook...
here is what I get when I actually run the playbook, without ever having success with the ec2.py script.
e$ ansible-playbook playbook.yml
[WARNING]:  * Failed to parse /home/melshman/projects/2018-itx-iac-workshop/ansible/ec2.py with script
plugin: Inventory script (/home/melshman/projects/2018-itx-iac-workshop/ansible/ec2.py) had an
execution error: region name: 'us-east-1' likely not supported, or AWS is down.  connection to region
failed.
[WARNING]:  * Failed to parse /home/melshman/projects/2018-itx-iac-workshop/ansible/ec2.py with ini
plugin: /home/melshman/projects/2018-itx-iac-workshop/ansible/ec2.py:3: Error parsing host definition
''''': No closing quotation
[WARNING]: Unable to parse /home/melshman/projects/2018-itx-iac-workshop/ansible/ec2.py as an
inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost
does not match 'all'
PLAY [all] *********************************************************************************************
skipping: no hosts matched
[WARNING]: Could not match supplied host pattern, ignoring: tag_role_node
PLAY [tag_role_node] ***********************************************************************************
skipping: no hosts matched
PLAY RECAP *********************************************************************************************
END----------------------------
here is the ansible info, although ansible is not really the issue here, it is the python script ec2.py:
$ ansible --version
ansible 2.5.2
config file = /home/melshman/projects/2018-itx-iac-workshop/ansible/ansible.cfg
configured module search path = [u'/home/melshman/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /home/melshman/.local/lib/python2.7/site-packages/ansible
executable location = /home/melshman/.local/bin/ansible
python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]

@melshman There's a couple of things you can try:

Try setting the AWS_REGION environment variable to the AWS region you want to use.
Try running aws configure to configure a default region for the AWS CLI.

Try one or both of these and let me know what happens.

Update.  By deleting the git repo on my windows 10 machine and destroying everything in AWS related to this project, I was able to get the locking issue to subside when running from my WSL machine.
I was then able to do the following from my WSL machine successfully without '-lock=false':
terrafom init
terraform plan
terraform apply
All AWS resources were confirmed to have been created.
But when I run the $ python ec2.py --list', I still get the same error.
$ sudo python ec2.py --list
region name: None likely not supported, or AWS is down.  connection to region failed.

Thanks... good but more bad...

Try setting the AWS_REGION environment variable to the AWS region you
want to use.

OK... I changed from auto  to  us-east-1 in the 'ec2.ini' file?
provide the 'regions_exclude' option. If this is set to 'auto',
AWS_REGION or
AWS_DEFAULT_REGION environment variable will be read to determine the
region.
regions = us-east-1
#regions_exclude = us-gov-west-1, cn-north-1

Try running aws configure to configure a default region for the AWS CLI.
$ aws configure
AWS Access Key ID [****************N3LA]:
AWS Secret Access Key [****************Ss7y]:
Default region name [us-east-1]:
Default output format [json]:

*It worked!  I changed the file, redeployed via terraform, ran 'sudo
phython ec2.py --list' command and got correct output.   Then i moved on to
running ansible-playbook command but that failed.  *
Unfortunately... next issue.  - error running ansible-playbook
When I finally run the 'sudo ansible-playbook playbook.yml' command i get:
$ sudo ansible-playbook playbook.yml
sudo: ansible-playbook: command not found
So then I run it without the sudo and get file access errors...
$ ansible-playbook playbook.yml
[ERROR]:
PLAY [all]

TASK [Install Python]

fatal: [34.205.87.184]: UNREACHABLE! => {"changed": false, "msg": "Failed
to connect to the host via ssh: no such identity:
/home/melshman/.ssh/aws_rsa: No such file or directory\r\nPermission denied
(publickey).\r\n", "unreachable": true}
fatal: [54.227.197.71]: UNREACHABLE! => {"changed": false, "msg": "Failed
to connect to the host via ssh: no such identity:
/home/melshman/.ssh/aws_rsa: No such file or directory\r\nPermission denied
(publickey).\r\n", "unreachable": true}
fatal: [54.237.249.177]: UNREACHABLE! => {"changed": false, "msg": "Failed
to connect to the host via ssh: no such identity:
/home/melshman/.ssh/aws_rsa: No such file or directory\r\nPermission denied
(publickey).\r\n", "unreachable": true}
fatal: [54.172.73.95]: UNREACHABLE! => {"changed": false, "msg": "Failed to
connect to the host via ssh: no such identity: /home/melshman/.ssh/aws_rsa:
No such file or directory\r\nPermission denied (publickey).\r\n",
"unreachable": true}
fatal: [34.201.72.124]: UNREACHABLE! => {"changed": false, "msg": "Failed
to connect to the host via ssh: no such identity:
/home/melshman/.ssh/aws_rsa: No such file or directory\r\nPermission denied
(publickey).\r\n", "unreachable": true}
fatal: [54.172.83.227]: UNREACHABLE! => {"changed": false, "msg": "Failed
to connect to the host via ssh: no such identity:
/home/melshman/.ssh/aws_rsa: No such file or directory\r\nPermission denied
(publickey).\r\n", "unreachable": true}
[WARNING]: Could not create retry file
'/home/melshman/projects/2018-itx-iac-workshop/ansible/playbook.retry'.
[Errno 13] Permission denied:
u'/home/melshman/projects/2018-itx-iac-workshop/ansible/playbook.retry'
PLAY RECAP

34.201.72.124              : ok=0    changed=0    unreachable=1    failed=0
34.205.87.184              : ok=0    changed=0    unreachable=1    failed=0
54.172.73.95               : ok=0    changed=0    unreachable=1    failed=0
54.172.83.227              : ok=0    changed=0    unreachable=1    failed=0
54.227.197.71              : ok=0    changed=0    unreachable=1    failed=0
54.237.249.177             : ok=0    changed=0    unreachable=1    failed=0
Tim
On Thu, May 17, 2018 at 12:37 AM, Scott S. Lowe notifications@github.com
wrote:

@melshman https://github.com/melshman There's a couple of things you
can try:

Try setting the AWS_REGION environment variable to the AWS region
you want to use.
Try running aws configure to configure a default region for the AWS
CLI.

Try one or both of these and let me know what happens.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/scottslowe/2018-itx-iac-workshop/issues/4#issuecomment-389742019,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Aetso2hRcr7ZCLyWMyst50AY1qmlliQgks5tzP6RgaJpZM4T84W_
.

I got it!  Ansible was looking in the wrong location for the keypair file.  changed it and got past the above.
It all works now.
BTW... after reviewing the playbook.yml file, i had a quick question.
I understand that the .cfg file is using ec2.py as the inventory.  That makes sense for the task that uses hosts: "all", which affected 6 instances.  My question is what is the difference in the second portion of the playbook when it uses this:  - hosts: tag_role_node.  What is tag_role_node,, is it a keyword, and where does it determine what hosts to use?
I navigated to the /ansible/roles/kubeadm/tasks and see where the tasks are defined for the role 'kubeadm'.  And I also see that a tag is defined.
tags:
- kubernetes
I actually just found this... in the ec2.py -- list output:  Is this what it is looking at?
"ec2_tag_Name": "k8s-2",
"ec2_tag_role": "node",
and also noticed that the ec2 instance that only had python installed does not have a tag name or tag role.  Mystery solved.
Second Question:
Why are these items with "":
apt:
name: "{{ item }}"
state: "present"
with_items:
- "docker.io"
- "kubectl"
- "kubelet"
- "kubeadm"
And why are these items without ""
file:
path: "{{ item }}"
state: directory
with_items:
- /etc/kubernetes
- /etc/kubernetes/audit
Thanks for you help on this tutorial.  It feels great to have it working.

@melshman Sorry about the error with the SSH key; it occurred to me after reading your previous update that the SSH key specified in ansible.cfg was incorrect, but you resolved the error before I could respond here.
As for your question regarding quotes: the quotes around {{ item }} are required (Ansible won't parse it correctly otherwise), but quotes almost everywhere else are optional. I generally try to use them everywhere for consistency, but (as you can see) I didn't live up to that standard here.
Finally, regarding tags and such: the ec2.py inventory script uses AWS tags to create inventory groups (like "tag_role_node"); the tags are assigned by Terraform. This allows you to target specific subsets of your AWS EC2 inventory for certain tasks based on tag. The tags in the Ansible playbook are unrelated; they allow you to limit which tasks should be executed by instructing Ansible to only execute tasks with a particular tag.
I hope this helps!

@melshman I spawned two separate issues (#5 and #6, both addressed already) to fix the issues you brought up in this issue. I'll go ahead and close this, but feel free to re-open or create a new issue if you run into further problems or have more questions. Thanks!

authors: melshman scottslowe


text:
Update to use latest Talos provider

The code is currently using a prerelease/alpha version of a provider using a repository that no longer exists. The code needs to be updated to use the latest provider, which has significant changes from earlier versions.

talosSecrets.ToSecretsTypeOutput().ClientConfiguration(),
and
 client.GetConfigurationOutput(ctx, client.GetConfigurationOutputArgs{
isn't working in your import update branch.

talosSecrets.ToSecretsTypeOutput().ClientConfiguration(),
and
 client.GetConfigurationOutput(ctx, client.GetConfigurationOutputArgs{
isn't working in your import update branch.

Correct. The provider is currently broken, and it's unclear at this time when (if?) a fix will be available.

github.com/pulumiverse/pulumi-talos/sdk v0.2.1-0.20240418152927-2960f4249b3b
somehow this version works.  FYI.

And instead use ToSecretsOutput().ClientConfiguration(), instead of ToSecretsTypeOutput().ClientConfiguration(),

Need to test 0.4.1, which was released a couple weeks ago. It looks like there were significant changes in 0.3.0 which may have fixed the problem.

source: gharchive/issue

authors: drduker scottslowe

repo: scottslowe/talos-aws-pulumi

license_type: permissive

license_source: github-api


text:
The application doesn't open

The application doesn't open with Adobe Air 13.0.
this is the exception that is throwing:

Issue resolved. Closing this ticket. Scout-App 2 no longer relies on Adobe Air or Java.

Scout-App 2 (2016)

authors: TheJaredWilcurt ghost

repo: scout-app/scout-app

license: MIT


text:
Fixes script pack support, hooks up AppDomainAssemblyResolver, and updat...

...es sample to use Logger script pack. Also slight refactoring of MEF sample code
Fix for #2

This adds a few other things.

It adds a post build step to the sample so that all the sample files including the packages are copied to the right place.
Introduces a new param on the catalog for passing custom script args. This sets the args in the script execution environment, which some script packs rely on. In the sample for example I am setting the -loglevel arg which the Logger Script Pack uses.
The AppDomainAssemblyResolver is initialized which handles runtime assembly redirects. This removes the need for bindings in the app.config / removes version conflicts.

Oops, need to update unit tests!

@laedit weird, looks like it failed, but when I viewed the logs, all tests passed, etc. It just had a weird error at the very end.

Don't worry, it's, the code coverage which had an issue only on PR.

Thanks a lot, you are awesome! :smiley:
Also one question, do you think it's possible to make the scriptArgs a default value of null to avoid the use of constructors with too much parameters?

Glad I could help :-). You helped me, as I have been wanting this for a long time.
We should chat about recomposability. I am not sure that's such a good idea....
As to the args, the problem is you have params at the end. So I don't think that will work.

For args we could add more overloads, but then you end up with like 10 constructors, which is kinda fail.
Another idea, is to have an Options object that one passes in. That could get rid of a lot of the overloads.

I was thinking about using an object instead of multiple parameters, you have convinced me.
I am open regarding recomposability, I must admit that I am far from being an expert on this field.

As to the object, in this case it might be good as you can have defaults easily and you only set what you need.
Let's skype or something when you have time and talk about what you are thinking with recomposition. I understand it pretty well from having worked on it :-)

Weird, I can't make it work: the AppDomainAssemblyResolver warn about not finding ScriptCs.Contracts in v0.9.0 and ScripCs.Logger.ScriptPack, the AssemblyResolver.GetAssemblyPaths return an empty enumerable and no ScriptPacks are found from ScriptPackResolver.GetPacks :(
Did I miss something?
Also, about recomposition, I tend to avoid it when I can but in this case since we are working with scripts I thought that it can be nice to allow it.

I see the problem. The full packages were not added into the commits, just the dlls. I'll update the PR.

OK, so the issue is this. Githib has converted the dlls in the package folders into text, so they are no longer loadable. I have to flag the files as binary.
To test though do this. Go into a terminal and go to your bin/debug/scripts folder and delete the scriptcs_packages folder. Then run scriptcs -install to have it install the packages.
Next run the app and it should work. Can you try this?

It's working, thanks!
But when I try your PR #999, everytime I replace scriptServicesBuilder.ScriptName(String.Empty); by scriptServicesBuilder.LoadScriptPacks(); the script packs are lost and I need to reinstall them to make them work with the original scriptServicesBuilder.ScriptName(String.Empty);.
I haven't manage to have the scriptServicesBuilder.LoadScriptPacks(); working yet, I will retry later.

I know why. You need to run scriptcs -install not in bin/debug, that was
wrong.
You need to delete scripts_packages in the
sample\ScriptCompositionSample\Scripts folder and run there.
There is a post deployment script which erases folders and copies files to
bin/debug/Scripts.
So try what I told you just in a diff folder.
On Fri, Mar 27, 2015 at 12:11 AM Jérémie Bertrand notifications@github.com
wrote:

It's working, thanks!
But when I try your PR #999
https://github.com/scriptcs/scriptcs/pull/999, everytime I replace
scriptServicesBuilder.ScriptName(String.Empty); by scriptServicesBuilder.
LoadScriptPacks(); the script packs are lost and I need to reinstall them
to make them work with the original scriptServicesBuilder.
ScriptName(String.Empty);.
I haven't manage to have the scriptServicesBuilder.LoadScriptPacks();
working yet, I will retry later.
—
Reply to this email directly or view it on GitHub
https://github.com/scriptcs-contrib/scriptcs-mef/pull/5#issuecomment-86850394
.

I can't believe I haven't thought of that.
It's working perfectly, thanks a lot! :smiley:

source: gharchive/pull-request

authors: glennblock laedit

repo: scriptcs-contrib/scriptcs-mef

license_source: bigquery


text:
🛑 Github Insights is down

In 12dea88, Github Insights (https://github-insights.atanas.info) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Github Insights is back up in 049456f.

repo: scriptex/uptime


text:
the scroll stable branch

issues:

check_rw_lookup does not work well for RETURN.
we must use parallel iter to calculate witness of keccak circuit

RLP circuit PR: #233

Tx circuit PR: #220,
PI circuit PR: #220

the CI is becoming slower and slower... We may need to re-pick this: https://github.com/scroll-tech/zkevm-circuits/pull/193

Yeah, I think so. Running make test on my local machine(with 72 threads) needs about 9min to finish.

tx_circuit_1tx_2max_tx failed

@silathdiir oh each action run twice... in my above commit (push directly to scroll-stable). Can you help fix that?

wait me to fix this..

authors: kunxian-xia lispc

repo: scroll-tech/zkevm-circuits


text:
Module Name Needs Fix in PowerShell Gallery

Client

OS: Ubuntu 18.04
PSCoreVersion: 6.2.2

Issue
PS> Find-Module VaporShell | select Version, Name, Repository

Version Name       Repository
------- ----       ----------
2.8.0   Vaporshell PSGallery

Vaporshell should be named VaporShell, otherwise the case will break the module path and the module cannot be imported.
Install-Module from the PowerShell Gallery results in:
# Expected Path for VaporShell
/home/$ENV:USER/.local/share/powershell/Modules/VaporShell
# Actual Path is for Vaporshell
/home/$ENV:USER/.local/share/powershell/Modules/Vaporshell

Case sensitivity prevents the module from being imported because the module dir string name differs from the module .psm1 file string name (/Vaporshell/ vs. VaporShell.psm1).
Temp Fix
# Rename Vaporshell dir to VaporShell
mv /home/$ENV:USER/.local/share/powershell/Modules/Vaporshell /home/$ENV:USER/.local/share/powershell/Modules/VaporShell

thanks for opening this up, @ScriptAutomate ! Very fair point, I'll reach out to the Gallery Admins to request the rename!

Request submitted @ScriptAutomate , I'll keep you updated!

hey @ScriptAutomate - all is fixed up on the Gallery side! Let me know how it's working for you. Tested in WSL on my host (Ubuntu 18.04 as well) and it's looking good to me:

authors: ScriptAutomate scrthq

repo: scrthq/VaporShell


text:
No longer builds on Ubuntu 20.04

I was successfully building commit 79ce3f1e13cd1cb283871ec5a9f90cf4062d91d4, but with latest commit 7f179204cc871434d6751259bc0064e5416cb3fa it fails with:
The Meson build system
Version: 0.53.2
Source dir: /usrsctp
Build dir: /usrsctp/build
Build type: native build

meson.build:82:28: ERROR: Expecting rparen got rbracket.
    add_project_arguments([]], language: 'c')
                         ^__^

Apparently something broke in between those 2 snapshots.

It was me... Sorry about that.
It should be fixed in https://github.com/sctplab/usrsctp/commit/2cb9d6e3ccbd11eadaabc0c8af53a917ecb0266f.
Thanks for reporting.

authors: nazar-pc tuexen

repo: sctplab/usrsctp


text:
Support for 64-bit indptr data and multiple GPU？

Hi, thank you for your work, which has made our single cell analysis much faster.
However, when the nnz of the matrix exceeds the maximum limit of 32 bits, the preprocessing process needs to be performed again using native scanpy, which greatly increases our time. I saw in https://github.com/NVIDIA-Genomics-Research/rapids-single-cell-examples/tree/master this work uses dask to distribute large tasks in chunks to multiple GPUs, which makes large The GPU acceleration of the amount of data can be realized, do you have any ideas in this regard? Such as support for multiple GPUs?
Thanks again for your work!

Dear @YH-Zheng,
Thank you for your interest in rapids_singlecell. I want to support 64bit indptr in rsc. However there is currently a limitation with cupy when it comes to the indptr arrays. I tested a bit with using uint32 which doubles the nnzs but that's sadly to unstable. I'm already testing ways to make rapids-singlecell multi gpu enabled. Which back end I'll use is still up in the air. cunumeric is also very promising.

Dear @Intron7 ,
Hi Severin! Thank you for your reply, and I am looking forward to your next work. At present, I will still use the regular scanpy analysis process until I use rsc for harmony integration.
I also noticed that the rsc of the Louvain and Leiden algorithm will generate more clusters, especially Leiden, even if the resolution is reduced, it will still generate hundreds of clusters, but it performs well in the sc (because the amount of data is too large, full iteration Time is too long, only iterate 2 times). What do you think about it?
I have not saved the Leiden result at present, the picture is the Louvain results of rsc, I think resolution=1 produces too many clusters than sc.

See https://github.com/scverse/rapids_singlecell/issues/44 for the clustering issue

So the 64 bit support in cupy might be coming. They made it a high priority https://github.com/cupy/cupy/issues/3513.

Dear @YH-Zheng,
Thank you for your interest in rapids_singlecell. I want to support 64bit indptr in rsc. However there is currently a limitation with cupy when it comes to the indptr arrays. I tested a bit with using uint32 which doubles the nnzs but that's sadly to unstable. I'm already testing ways to make rapids-singlecell multi gpu enabled. Which back end I'll use is still up in the air. cunumeric is also very promising.

Also waiting for supporting for multiple GPUs!

Multi-GPU support with dask has been released


text:
New defaults

"zinb" vs "nb". Negative binomial seems more sensible of a default. Stipulation of changing will be rerunning all tutorials.
encode_covariates=True/False. Typically we set this to True to improve mixing, but would like to check with Pierre since apparently there was a discussion about this before with Pierre and Jonas, and the conclusion was to leave it as False.

We can use the ray tune PR to test for the best hyperparameter settings once it gets in: https://github.com/scverse/scvi-tools/pull/1565

repo: scverse/scvi-tools


text:
Read xenium explorer selection

Any selection from the Xenium Explorer (i.e., lasso or rectangle) can be read by this function. It returns a polygon that we typically use for a polygon_query.
The pixelsize is needed if the Xenium Explorer is used to visualize non-xenium data, for instance if using spatialdata_xenium_explorer.
The function is really short, do we really need this? Maybe we feel more in control to directly use these two lines of code rather than this function. What do you think @LucaMarconato?

Codecov Report
Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (d17b438) 36.83% compared to head (3264ef8) 37.00%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #106      +/-   ##
==========================================
+ Coverage   36.83%   37.00%   +0.16%
==========================================
  Files          15       15
  Lines        1075     1081       +6
==========================================
+ Hits          396      400       +4
- Misses        679      681       +2

Files
Coverage Δ

src/spatialdata_io/__init__.py
100.00% <100.00%> (ø)

src/spatialdata_io/readers/_utils/_utils.py
30.18% <60.00%> (+3.10%)
:arrow_up:

Thanks @quentinblampey, I think even if short it would still be useful for the user to have this, so for instance they don't have to look up the scale factors. Asking also for @giovp opinion. I have moved the function in xenium.py (and also made xenium_aligned_image() visible.

sounds good for me! I would add a link toboth https://github.com/quentinblampey/spatialdata_xenium_explorer and the 10x page maybe

Thanks @giovp! I updated the docstring to add the links you mentioned

Thanks both! Super tiny thing, can I change pixelsize to pixel_size or is pixelsize a named used by Xenium Explorer? I will merge right after in both cases.

Thanks both! Super tiny thing, can I change pixelsize to pixel_size or is pixelsize a named used by Xenium Explorer? I will merge right after in both cases.

I kept the same nomenclature than 10x on this tutorial
But I agree that pixel_size is better, and I think that 10x doesn't use pixelsize elsewhere, so it should be fine? I will also rename it in sopa and spatialdata_xenium_explorer

Thanks for the clarification, I'd use pixel_size indeed then 😊

authors: LucaMarconato codecov-commenter giovp quentinblampey

repo: scverse/spatialdata-io


text:
fixed reading 10x formatted visium mtx files

Description
Replaced Anndata's read_mtx function with scanpys native read_10x_mtx function for reading mtx formatted visium experiments. This solves issues with matching the spot positions to barcodes, as well as missing feature names and barcodes, and proper matrix orientation.
How has this been tested?
all tests ran locally
Closes
closes #802

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (4ad4083) 69.83% compared to head (f3b434a) 69.83%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #803   +/-   ##
=======================================
  Coverage   69.83%   69.83%
=======================================
  Files          39       39
  Lines        5497     5497
  Branches     1034     1034
=======================================
  Hits         3839     3839
  Misses       1363     1363
  Partials      295      295

Files
Coverage Δ

src/squidpy/read/_utils.py
75.00% <66.66%> (ø)

thank you @LinearParadox ! I wonder if it's related to #797 , @Lem-P could you try out this solution and see if it fixes the issue?

I think it's likely a different issue. When squidpy would read the mtx file, the object would have no gene / barcode names, and the failure would likely be noticed at qc (as you would have only a single peak in spot counts, since they'd be all NA). Do you know if the current test datasets in squidpy have the mtx files floating around somewhere? It would be nice to add some tests to make 100% sure the mtx reading is equivalent with reading h5 files.

hi @LinearParadox , you are right, re-reading the issue I think it's due to the header being set wrong.
regarding tests, unfortunately we don;'t have the mtx file in thedatasets, and we don't really have a test suite for readers in squidpy. For spatialdata, we have a robust and rather compute expensive pipeline that ensures that datasets can always be read by the readers and supported by spatialdata objects (set up by @LucaMarconato , cc). I think it's outside the scope of squidpy to set something up like that as we are hoping that users will transition to spatialdata eventually. You can find the (tested) datasets here: https://github.com/scverse/spatialdata-notebooks/tree/main/datasets . I will merge this once CI passes

maybe @LinearParadox can you add a release note? you can add it in the dev release for now (in docs/release_notes) and I will move it to whatever version we do next. Thank you!

Should be done! Let me know if you need me to add more detail or if there's anything else I should modify/add!

authors: LinearParadox codecov-commenter giovp

repo: scverse/squidpy


text:
node.start(wait_other_notice=True, ...) may return too early because implementation of wait_other_notice may observe past state

wait_other_notice in ccmlib/scylla_node.py currently works by looking at the /gossiper/live endpoint API
https://github.com/scylladb/scylla-ccm/blob/bd519ed12cccd8186dedf8613656f21dd156354e/ccmlib/scylla_node.py#L334
https://github.com/scylladb/scylla-ccm/blob/bd519ed12cccd8186dedf8613656f21dd156354e/ccmlib/scylla_node.py#L1361
If other nodes see the node as UP, and the node sees other nodes as UP, the check will succeed.
But consider the following:

2 nodes A, B
kill node A
start node A with wait_other_notice=True

The start call will often return quickly, but not because node B noticed that A restarted
but because node B didn't even notice that A was killed yet!
So node B may still consider A as UP because it was UP before it was killed; B's failure detector just didn't react to that event yet.
That's why the old implementation in ccmlib/node.py used log markers, it watched other nodes' logs from a point after the node was restarted, effectively serializing the events.
This bug was introduced in https://github.com/scylladb/scylla-ccm/pull/462.
cc @temichus

We could fix it by combining the old behavior (with log markers) with the new behavior (watching REST). For example -- we could wait for the "is now UP" message first, then use watch_rest_for_alive.
Although, looking at Scylla code, I don't really understand why "is now UP" is not sufficient; it happens strictly after modifying live_endpoints set in gossiper, which is used by the /gossiper/live REST API.
Perhaps it's necessary for some older versions of Scylla.
And I see that watch_rest_for_alive contains additional logic for watching for NORMAL too, so that is still useful.

Example failure:
https://github.com/scylladb/scylla-dtest/pull/4042/files
the test sometimes fails with
        # starting node1 - it should reconnect and run as is
        node1.start(wait_other_notice=True, wait_for_binary_proto=True)
        #  due to CL=1 it does not make sense to read while not all nodes are UP
        # for node in cluster.nodelist():
        #     wait_for_nodes_status(node, ["UN", "UN", "UN"])

>       result = list(session.execute("SELECT * FROM cf"))

update_cluster_layout_tests.py:1136:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cassandra/cluster.py:2726: in cassandra.cluster.Session.execute
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   cassandra.Unavailable: Error from server: code=1000 [Unavailable exception] message="Cannot achieve consistency level for cl ONE. Requires 1, alive 0" info={'consistency': 'ONE', 'required_replicas': 1, 'alive_replicas': 0}

(https://jenkins.scylladb.com/view/staging/job/scylla-staging/job/artsiom_mishuta/job/dtest-pytest-gating/240/testReport/junit/update_cluster_layout_tests/TestUpdateClusterLayout/test_simple_kill_remained_node_while_decommissioning_96_100_/)
because we execute SELECT too fast. start returned too early, and then node 2 marked node 1 as DOWN, and then we executed SELECT before node2 marked node 1 as UP again.

But consider the following:
* 2 nodes A, B
* kill node A
* start node A with `wait_other_notice=True`

The start call will often return quickly, but not because node B noticed that A restarted but because node B didn't even notice that A was killed yet!

I see that you and Benny liked https://github.com/scylladb/scylla-ccm/pull/564 so I'll merge it, but I want to point out that I think the real problem in your example above is that "kill node A" neglects to wait for node A to be killed. If you want to do "kill node A" followed in quick succession by "start node A", it doesn't make sense to not wait for node A to be killed before starting it again.
The benefit of https://github.com/scylladb/scylla-ccm/pull/564 is that it will allow the test to succeed despite the test bug that it forgets to wait for the "kill node A" to finish, so in that sense it is a good patch because it makes tests less easy to write incorrectly.

but I want to point out that I think the real problem in your example above is that "kill node A" neglects to wait for node A to be killed. If you want to do "kill node A" followed in quick succession by "start node A", it doesn't make sense to not wait for node A to be killed before starting it again.

I see it differently. The semantics of wait_other_notice should be as the name suggests: we wait until other nodes notice this particular start event. (Not some earlier start event.)
The test is correct when written using this assumption, this semantics. (And it was indeed so -- before https://github.com/scylladb/scylla-ccm/pull/462)

authors: kbr-scylla nyh

repo: scylladb/scylla-ccm


text:
feat(gen): const from default

Generates a matching const for each type definition that holds the defaults for the type based on information from CUE defaults.
Status: Useful (I guess)

CUE
TS

package cuetsy

AType: "foo" | "bar" | *"baz" @cuetsy(targetType="type")

Foo: {
  Bar: string | *"ohai"
  Baz: int | *4
  C: AType
} @cuetsy(targetType="interface")

export type AType = 'foo' | 'bar' | 'baz';
export const aTypeDefault: AType = 'baz'
export interface Foo {
  Bar: string;
  Baz: number;
  C: AType;
}
export const fooDefault: Foo = {
  Bar: 'ohai',
  Baz: 4,
  C: aTypeDefault,
}

package cuetsy

OrEnum: "foo" | "bar" | *"baz" @cuetsy(targetType="enum")

StructEnum: {
   Walla: "laadeedaa" @cuetsy(enumDefault)
   run: "OMG"
} @cuetsy(targetType="enum")

export enum OrEnum {
  Bar = 'bar',
  Baz = 'baz',
  Foo = 'foo',
}
export const orEnumDefault: OrEnum = OrEnum.Baz
export enum StructEnum {
  Walla = 'laadeedaa',
  run = 'OMG',
}
export const structEnumDefault: StructEnum = StructEnum.Walla

excellent!  2 questions:

can generating defaults be optional?  We will only want it for some types of objects  (namely, panelOptions and panelFieldConfig)
can the default name be lowercase?
export const fooDefault: Foo ...

@sdboyer @ryantxu afaict it is mostly does what it is supposed to do. There are for sure rough edges, but it should be good enough for a first round of review

The results here look great!
If it is possible to optionally tag interfaces that we want to generate a default that would be best -- for most types we do not want a default and it will just create noise bloat.  It is only the final full options objects where we want.  Perhaps:
} @cuetsy(targetType="interface", generateDefaults=true)

So, months later...
i'm trying to recall why it is that we thought struct representations of enums were a good idea. I think it was because we were looking for a way to encode arbitrary member names easily. But it's just wrong. CUE structs are plainly, not analogous to a TS enum. In a system (like we have in Grafana), using the CUE struct representation directly in some larger CUE structure means that you'll end up with a struct type on the CUE side, and an enum on the TS side. Not what we want.
The rest of this looks good, but i think that part's just not what we need. i introduced an alternative pattern that seems like it should give us the same control (even if it's a bit awkward), while having a CUE type (simple disjunct value) which is harmonious with the TS it will become.

i'm trying to recall why it is that we thought struct representations of enums were a good idea.

Well, it was literally the first issue - #1. Not the fault of this PR. So i'm gonna merge this one as-is, then we can pull that behavior out.

authors: ryantxu sdboyer sh0rez

repo: sdboyer/cuetsy


text:
ghc-8.2 HasField, Type & *

Is it ok to expose these for ghc-8.2?

Yes, this should be good now that we have GHC 8.2. :+1:

repo: sdiehl/protolude

license: mit


text:
get_data creates NaNs on the GPU

input, target = get_data(:exchange_rate, poollength, datalength, horizon) |> gpu works fine on the CPU, but creates some NaNs at the beginning of the dataset (at least on a Jetson Nano).

The actual raw data loads differently on a x64 and arm64, with the latter being wrong.

I had a similar problem. It has nothing to do with GPUs, but does fail more loudly on em.
prepare_data creates an uninitialized array with  similar. and then doesn't fill it all up. Leading to random data being read as float, with some being NaNs.
To make it really obvious I replaced the similar call with zeros(Float32, ... ):
function prepare_data(data, poollength, datalength, horizon; normalise=true)
  extendedlength = datalength + poollength
  extendedlength > size(data, 1) && throw(ArgumentError("datalength $(datalength) larger than available data $(size(data, 1) - poollength)"))
  (normalise == true) && (data = Flux.normalise(data, dims=1))
  features = zeros(Float32, size(data, 2), poollength, 1, datalength)
  for i = 0:poollength - 1
      for j = poollength:datalength
          features[:,i + 1,1,j] = data[j - i,:]
      end
  end
  labels = circshift(data[1:datalength,1], -horizon)
  return features, labels
end

and then
prepare_data(ones(Float32, 510, 3), 10, 500, 7, normalise=false)
> 3×10×1×500 Array{Float32, 4}:
[:, :, 1, 1] =
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0

[:, :, 1, 2] =
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0

[:, :, 1, 3] =
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0

...

[:, :, 1, 498] =
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0

[:, :, 1, 499] =
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0

[:, :, 1, 500] =
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0

There should be no zeros (aka uninit memory), but there is.

Thanks for the quick fix!
This is now the most useful time series analysis package for me. <3

authors: KingBoomie sdobber


text:
fix(cfg): convert signed nums to unsigned if meets unsigned operators

When unsigned operator uop contains a signed number in LLVM IR, we need to convert the signed number to an unsigned number in CrabIR before lowering the unsigned operation to signed. For instance:
%3 = icmp ult i64 %1, -26, !dbg !1010

should appear in crab IR:
@V_15 = (-_0 <= 0);
@V_16 = (_0 <= 18446744073709551589);
@V_17 = true;
_2 = ite(@V_15,@V_16,@V_17);

Just to clarify, to convert to _2 = (_0 <=_u -27) is correct because then it's up to the abstract domain to deal with unsigned comparisons. The problem only arises when we try to lower unsigned comparisons to signed ones.

It would also nice to add a test.

Fixed checking signed operand based on your clarification. Also added some tests for lower unsigned.

This should be fixed in commit d84dee4

authors: LinerSu caballa

repo: seahorn/clam


text:
Can't build seajax on mac

Eights-Book-2:Scripts eight$ npm install
module.js:338
throw err;
^
Error: Cannot find module 'umask'
at Function.Module._resolveFilename (module.js:336:15)
at Function.Module._load (module.js:278:25)
at Module.require (module.js:365:17)
at require (module.js:384:17)
at Object. (/usr/local/lib/node_modules/npm/lib/utils/umask.js:1:75)
at Module._compile (module.js:460:26)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Module.require (module.js:365:17)
Eights-Book-2:Scripts eight$

Using the pivotviewer canvas template, I am getting odd behavior. I am attempting to draw a canvas element with a picture in it. The image loads fine in IE, flickers and can disappear in Firefox, and does not appear at all in Chrome/Safari. Here is a simple template that is not working. I don't want to use just the image template, because I want to draw images and text on the canvas. Can anyone tell me why my .drawimage does not work in the canvas template?
viewer.setTemplates([
{
type: "canvas",
width: 438,
height: 300,
template: function (ctx, x, y, w, h, item) {
ctx.strokeStyle = "black";
ctx.lineWidth = w / 100;
ctx.strokeRect(x + w / 200, y + w / 200, .99 * w, h - .01 * w);
var img = new Image();
img.onload = function () {
ctx.drawImage(img, x, y, w, h);
}
img.src = 'http://www.html5canvastutorials.com/demos/assets/darth-vader.jpg';
}
}
]);

Hi cpaulin,
Thanks for your interest! In order to do animations and such, the Viewer regularly clears its entire view and tells each element to redraw itself. It seems that the particular timing of when the screen repaints is different on various browsers, but in general I'd expect that a solution like the following might be beneficial.
var loadedImages = {}; // a set of images by source URL so we don't have to re-decode
viewer.setTemplates([
    {
        type: "canvas",
        width: 438,
        height: 300,
        template: function (ctx, x, y, w, h, item) {
            ctx.strokeStyle = "black";
            ctx.lineWidth = w / 100;
            ctx.strokeRect(x + w / 200, y + w / 200, .99 * w, h - .01 * w);
            var src = 'http://www.html5canvastutorials.com/demos/assets/darth-vader.jpg'; // would probably vary by item
            var img = loadedImages[src];
            if (!img) {
                var img = loadedImages[src] = new Image();
                img.src = src;
            }
            if (img.complete) {
                ctx.drawImage(img, x, y, w, h);
                // returning true means this item doesn't need to redraw itself next frame (unless viewport moves and everybody must redraw)
                return true;
            }
        }
    }
]);

Since the viewer is trying to redraw every frame anyway, we don't necessarily need to wait for a load event, we can just check the image's complete property at the time we are asked to draw the element. Once the image has loaded and drawn successfully, we can return true to notify the viewer that we don't require an update on the next frame; this can help CPU usage a lot by stopping the redraw cycle until the user does something. A more advanced solution would also include an error handler for the image, so that we can return true after either successful download or failure.

And @eighteight , I'm sorry we just derailed your issue thread. I don't have a Mac and so I can't be of much use to you, but perhaps some other members of the community here could help (summoning @gasi ).

@eighteight Which version of Node.js / npm do you have?
I just tried with a fresh git clone and it worked:
> node --version
v0.10.26

> npm --version
1.4.23

> npm install .
uglify-js@2.4.24 node_modules/uglify-js
├── uglify-to-browserify@1.0.2
├── async@0.2.10
├── source-map@0.1.34 (amdefine@1.0.0)
└── yargs@3.5.4 (decamelize@1.0.0, camelcase@1.2.1, window-size@0.1.0, wordwrap@0.0.2)

jshint@2.8.0 node_modules/jshint
├── strip-json-comments@1.0.4
├── exit@0.1.2
├── shelljs@0.3.0
├── console-browserify@1.1.0 (date-now@0.1.4)
├── minimatch@2.0.10 (brace-expansion@1.1.0)
├── lodash@3.7.0
├── cli@0.6.6 (glob@3.2.11)
└── htmlparser2@3.8.3 (domelementtype@1.3.0, entities@1.0.0, domhandler@2.3.0, readable-stream@1.1.13, domutils@1.5.1)

authors: cpaulin eighteight gasi sethobrien

repo: seajax/seajax


text:
Unable to get deployment logs when service deployment fails: http error 409

Describe the bug

Unable to get deployment logs when
Steps to reproduce

Create a service using terraform-kubernetes-containertask template.
Configure an invalid time zone.

Check the deployment result.

Result

Failed to deploy and cannot get the deployment logs.

Checked database, the recode is null.

Checked walrus logs.

2023-12-01T09:04:31.645Z	W	deployer.tf.config	provider not found, skip provider: terraform
2023-12-01T09:04:31.668Z	D	deployer.tf	k8s job tf-job-apply-489713297898124546 created
2023-12-01T09:04:31.668Z	D	api	200 | HTTP/2.0 |        0 B |        0 B |   64.723053ms |   192.168.181.1 | POST    /v1/projects/489238360397747202/environments/489573508389582082/resources/489699747074979074/rollback?revisionID=489711825143772418
2023-12-01T09:04:32.109Z	D	api	200 | HTTP/2.0 |        0 B |     21 KiB |    2.588186ms |   192.168.181.1 | GET     /v1/projects/489238360397747202/environments/489573508389582082/resources/489699747074979074/revisions/489713297898124546/terraform-states
2023-12-01T09:04:34.109Z	D	api	200 | HTTP/2.0 |        0 B |     21 KiB |    1.573236ms |   192.168.181.1 | GET     /v1/projects/489238360397747202/environments/489573508389582082/resources/489699747074979074/revisions/489713297898124546/terraform-states
2023-12-01T09:04:34.702Z	D	api	409 | HTTP/2.0 |     21 KiB |      152 B |   14.021668ms |   192.168.181.1 | PUT     /v1/projects/489238360397747202/environments/489573508389582082/resources/489699747074979074/revisions/489713297898124546/terraform-states

Exec to walrus backend and viewed the deployment logs.

...
2023-12-01T09:04:34.663Z [INFO]  Starting apply for module.task2.kubernetes_cron_job_v1.task[0]
2023-12-01T09:04:34.684Z [ERROR] provider.terraform-provider-kubernetes_v2.23.0_x5: Response contains error diagnostic: diagnostic_severity=ERROR diagnostic_summary="CronJob.batch "task2-dm5tq" is invalid: spec.timeZone: Invalid value: "CST": unknown time zone CST" tf_provider_addr=registry.terraform.io/hashicorp/kubernetes tf_req_id=89e95e2c-505b-c43d-9f3f-694ea7fc9187 @caller=github.com/hashicorp/terraform-plugin-go@v0.14.2/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto tf_proto_version=5.3 tf_resource_type=kubernetes_cron_job_v1 tf_rpc=ApplyResourceChange diagnostic_detail= timestamp=2023-12-01T09:04:34.684Z
2023-12-01T09:04:34.686Z [ERROR] vertex "module.task2.kubernetes_cron_job_v1.task[0]" error: CronJob.batch "task2-dm5tq" is invalid: spec.timeZone: Invalid value: "CST": unknown time zone CST

Error: Failed to save state

Error saving state: HTTP error: 409

Error: Failed to persist state to backend

The error shown above has prevented Terraform from writing the updated state
to the configured backend. To allow for recovery, the state has been written
to the file "errored.tfstate" in the current working directory.

Running "terraform apply" again at this point will create a forked state,
making it harder to recover.

To retry writing this state, use the following command:
    terraform state push errored.tfstate

Error: CronJob.batch "task2-dm5tq" is invalid: spec.timeZone: Invalid value: "CST": unknown time zone CST

  with module.task2.kubernetes_cron_job_v1.task[0],
  on .terraform/modules/task2/main.tf line 823, in resource "kubernetes_cron_job_v1" "task":
 823: resource "kubernetes_cron_job_v1" "task" {

Expected behavior

Show the deployment logs.
Environment

Walrus version: main 8fc1ee8f23f152bbeaf1aec685c03d15ffbccae1
UI version: dev-d9eaece

Additional context
Add any other context about the problem here.

Environment

Walrus version: main 54608149edeebd87ed53da7b8e51f384ae386650
UI version: dev-b0233e3

Test Result: Cannot be reproduced in main.

repo: seal-io/walrus


text:
refactor: remove isService filter and svc expr substitution

Related Issue:
https://github.com/seal-io/walrus/issues/1770

Rebased


text:
自动清群支持白名单

在提问之前...

[X] 我填写了简短且清晰明确的标题，以便开发者在翻阅 issue 列表时能快速确定大致问题。而不是“一个建议”、“卡住了”等
[X] 我基本确定这是一个新功能/建议，而不是遇到了 bug（不确定的话请附上日志）

说说你遇到的问题？
设定一个白名单, 其上的群不被自动清理.
考虑直接使用群信任等级.
有什么好的想法？
No response
其他内容
No response

Ref #419

repo: sealdice/sealdice-core


text:
feat: Removed requires of consul, use options parameter

Details
Breaking Changes
Interface changed: consul is now part of options parameter.
Example:
const httpServer = require('@sealsystems/http-server');
const consul = require('@sealsystems/consul');

consul.connect(...);

await httpServer.start({
  app: myExpressApp,
  host: '192.168.0.1',
  port: '3000',
  consul
});

References
Introduction to Pull Requests

Please read before submitting your first pull request!

Thank you for creating a pull request. We really appreciate your effort!
Here is some information to help you integrate your changes as easily as possible into the next release.
Title Field
The title is one of the most important parts of a pull request. The content is shown in CHANGELOG.md and release notes. Therefore, be sure to summarize your changes in a short but catchy sentence. Others should be able to easily get an idea of your changes.
By adding keywords at the beginning of the title, you can control the process of merging and releasing your pull request. Here is the list of all available keywords:

WIP:
It is highly recommended to open a pull request as soon as possible. This allows discussions during code development and not just criticism after work is done.
Please start a new pull request with the prefix WIP:. This signals to others that is is not yet ready to be merged into the master branch. It also prevents our bot from merging. (See section "Merging into Master" below)
After the changes and the pull request description have been completed, remove this prefix. In this case, review requests may be automatically generated to ensure, e.g., the compatibility of your description with our release notes format.

feat:
A pull request with this prefix in its title is marked as a new feature. In general this creates a minor release when merging the pull request into the master branch. A label with the expected release type is added to the pull request.

fix:
A pull request with this prefix in its title is marked as a bugfix. In general this creates a patch release when merging the pull request into the master branch. A label with the expected release type is added to the pull request.

chore:
A pull request with this prefix in its title is marked as a maintenance change. In general this creates a patch release when merging the pull request into the master branch. A label with the expected release type is added to the pull request. Only CHANGELOG.md, but not our release notes, will be updated with information about the pull request.

You can specify the kind of pull request when you create a new one. But do not forget to prefix it with WIP:. Here is an example title for such a pull request:
WIP: feat: My super cool new feature

After finishing the work, you can simply remove WIP:.
Please do not format the Title field with Markdown.
Description Field
Besides this introduction, the description field also contains sections with the following headings which contribute to CHANGELOG.md and release notes:

Details
Sometimes it is not possible to explain a change with a single short line in the Title field. In this case, you can add a more detailed description in the Details section.
You may add several paragraphs if needed and unlike the Title field, you can format the content with Markdown.

Breaking Changes
Please fill out this section if and only if your changes affect the public Interface. This creates a major release when merging the pull request into the master branch. Breaking changes are shown in an additional section of CHANGELOG.md and release notes. So two headings will be created: One for the bugfix/feature itself and one for the breaking changes.
All information for the Details section also applies here, with one important exception:
In contrast to the Details section, the first line is used as a title for the breaking change. Therefore, please add a short description first and append paragraphs for more information if needed. Do not start with a large paragraph and do not format the first line.
Here is an example of a longer text for a breaking change:

This is a brief description of the breaking change (without a punctuation mark and formatting)
In another paragraph, I add detailed information of steps needed to adapt to the new interface. I can also format this paragraph using Markdown.
I can add as many paragraphs as needed.

The first line is shown as a heading and the following paragraphs as normal text.
Use this section only to describe changes to the interface and how to adapt to the new version. Use the Details section to provide information about the bugfix/feature itself.

References
Here you can enter links to Jira oder Github. This allows you to provide more information about the background of your code changes.
Add one link per line without any other text.

Please make sure that you leave a blank line between the headings and the beginning and end of your text.
Merging into Master
After all reviews have been successfully completed, you are ready to merge your pull request into the master branch. This is only possible if all required automated tests pass.
You cannot use the green "Merge" button. Instead you must add a special comment containing only the following command:
/merge

This causes our merge bot to merge the pull request using a commit message consisting of the various pieces of information in the Title and Description fields.
Only users with admin or write permissions are allowed to merge a pull request. The branch is automatically deleted after a successful merge.

/merge

:tada: This PR is included in version 4.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

authors: comgit seal-mt

repo: sealsystems/node-http-server


text:
Update build configuration

The build configuration of this project is outdated and may no longer work.
This pull request will be merged automatically if there are no conflicts.

:tada: This PR is included in version 2.4.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

repo: sealsystems/node-tlscert


text:
Initial support of Postgres 9.5 native upsert support

Did some work on getting Postgresql 9.5's new UPSERT feature working.  The code's a little rough (could be organized a bit better) but seems to be working.
I also updated the travis build file to test against both Postgresql 9.4 and 9.5.

@seamusabshere I think I finally got travis happily building again.  Mind taking a look when you have a chance?  I'll try and clean up the code this week too :)

@seamusabshere What do you think behaviour should be if you specify use_native: true but it's not available?  Do we error out or ignore and just use the function?

@seamusabshere See https://travis-ci.org/seamusabshere/upsert/jobs/161380366 - thoughts?

@pnomolos let's do that in a separate PR, feel free to put back the version restriction

@seamusabshere OK, pushed.  Once this is merged onto master I'll do a separate PR up.  What version of AR do you want to track?

@pnomolos when the time comes, probably 5

@pnomolos would it be useful for me to go back and mark which comments i still consider unresolved?

@seamusabshere Yeah, that would be great :)

@pnomolos done

@pnomolos oh you're right, it's like i'm not seeing updates...

will re-review now

@seamusabshere Wait a few, I did miss a few things so pushing more fixes now :)

@pnomolos ok waiting

@seamusabshere OK, pushed 👍

@seamusabshere Specs are passing!

authors: pnomolos seamusabshere

repo: seamusabshere/upsert


text:
Directory Doesn't Exist

If the 'xorg.conf.d' directory isn't already present, an error is thrown.

This is because the use of 'touch', does not create directories, only files.
Instead use 'mkdir' to create the directory before attempting 'touch'.


text:
Python addDataPointBatch bug

Here is the problem:
when I using the example:
# create a random matrix to index
data = numpy.random.randn(10000, 100).astype(numpy.float32)

# initialize a new index, using a HNSW index on Cosine Similarity
index = nmslib.init(method='hnsw', space='cosinesimil')
index.addDataPointBatch(data)
index.createIndex({'post': 2}, print_progress=True)

# query for the nearest neighbours of the first datapoint
ids, distances = index.knnQuery(data[0], k=10)

the ids is :
[   0, 8466, 9355, 5861, 1651,   18, 2474, 9736, 2172, 6289]
But when I add the data point batch by batch:
for i in range(300):
    index.addDataPointBatch(data[i*30:(i+1)*30])

Then build the index and try knn search, the results becomes:
# query for the nearest neighbours of the first datapoint
ids, _  = index.knnQuery(data[0], k=10)
ids
[ 0, 16, 12, 20,  8,  9, 12, 27, 24,  7]

I tried different data, the ids are all in 0-30. What happens here???

So addDataPointBatch takes two parameters: the data and the ids to index the data under:
https://searchivarius.github.io/nmslib/api.html#nmslib.dist.FloatIndex.addDataPointBatch
If the ids aren't set, it defaults to the row of the passed in matrix - which in your second case is always 0-30. If you set this it should work .

@benfred shouldn't we make the IDs parameter mandatory?

@benfred Thanks for your explanation. I didn't notice the description of the ids parameter. I thought it would append the data to the previous added data.
Why do you need id here? I think usually user will add data to the index, then call the create index method when all data are added.
You can close this issue.
Thanks.

@xueyouluo IDs are needed, because they can be different from the insertion order.

PS: @xueyouluo but we shouldn't allow users to shoot in the feet.

authors: benfred searchivarius xueyouluo

repo: searchivarius/nmslib


text:
need to optimize the current workerId generation rule of snowflake id

[x] I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description
at present, the workerId is randomly generated from 0-1023, which is not elegant.I hope someone can solve this problem.
目前workerId 采用随机从0-1023中生成,并不优雅,希望有同学能解决这个问题.
Ⅱ. Describe what happened
If there is an exception, please attach the exception trace:
Just paste your stack trace here!

Ⅲ. Describe what you expected to happen
Ⅳ. How to reproduce it (as minimally and precisely as possible)

xxx
xxx
xxx

Ⅴ. Anything else we need to know?
Ⅵ. Environment:

JDK version :
OS :
Others:

Please assign me the task

repo: seata/seata


text:
optimize: copy jdbc driver to image

Ⅰ. Describe what this PR did

Copy jdbc driver to image

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it

Build an image
Run container by this image
Found jdbc directory under /seata-server/libs/jdbc

Ⅴ. Special notes for reviews

Codecov Report

Merging #2751 into develop will decrease coverage by 0.15%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             develop    #2751      +/-   ##
=============================================
- Coverage      50.89%   50.74%   -0.16%
+ Complexity      2849     2847       -2
=============================================
  Files            567      567
  Lines          18172    18153      -19
  Branches        2166     2127      -39
=============================================
- Hits            9249     9212      -37
+ Misses          8029     8023       -6
- Partials         894      918      +24

Impacted Files
Coverage Δ
Complexity Δ

...obuf/convertor/BranchRegisterRequestConvertor.java
90.47% <0.00%> (-9.53%)
3.00% <0.00%> (ø%)

...otobuf/convertor/BranchReportRequestConvertor.java
90.90% <0.00%> (-9.10%)
3.00% <0.00%> (ø%)

...obuf/convertor/BranchRollbackRequestConvertor.java
92.00% <0.00%> (-8.00%)
3.00% <0.00%> (ø%)

...otobuf/convertor/GlobalBeginResponseConvertor.java
92.59% <0.00%> (-7.41%)
3.00% <0.00%> (ø%)

...otobuf/convertor/GlobalCommitRequestConvertor.java
93.75% <0.00%> (-6.25%)
3.00% <0.00%> (ø%)

...otobuf/convertor/GlobalStatusRequestConvertor.java
93.75% <0.00%> (-6.25%)
3.00% <0.00%> (ø%)

...obuf/convertor/GlobalRollbackRequestConvertor.java
93.75% <0.00%> (-6.25%)
3.00% <0.00%> (ø%)

...protobuf/convertor/RegisterTMRequestConvertor.java
94.73% <0.00%> (-5.27%)
3.00% <0.00%> (ø%)

...protobuf/convertor/RegisterRMRequestConvertor.java
90.47% <0.00%> (-4.77%)
3.00% <0.00%> (ø%)

...tobuf/convertor/BranchReportResponseConvertor.java
95.45% <0.00%> (-4.55%)
3.00% <0.00%> (ø%)

... and 20 more

authors: codecov-commenter helloworlde


text:
Validate Alternative and Experiment names

Closes #14

Coverage increased (+0.3%) to 88.321% when pulling 416815e59b987735744144bb8fdebf104226f40f on validate-alternative-experiment-names into af94ab5ee0343a0f41ec78f3dbfcf0f24fedb76e on master.

authors: coveralls sddamico

repo: seatgeek/sixpack-java


text:
mount: can not  ensure ordered file handle lock and unlock by orderedMutex

Describe the bug
Seaweedfs  can not make IO in ordered  by file handle lock when I test.  The emaphore.NewWeighted is set to math.MaxInt64. The code is below in func newFileHandle(https://github.com/seaweedfs/seaweedfs/commit/22064c342585bddaa7ebdb21e39cac7db87826df)
fh := &FileHandle{
fh:      handleId,
counter: 1,
inode:   inode,
wfs:     wfs,
fh:           handleId,
counter:      1,
inode:        inode,
wfs:          wfs,
orderedMutex: semaphore.NewWeighted(int64(math.MaxInt64)),
}
I think it is a bug. I think semaphore.NewWeighted should be set to 1. Only in this way can we guarantee to prevent concurrency.
System Setup
OS version is ubuntu18.04
weed version : 3.34 ( the fault also happened in latset version )

please add the reason why you think it is a bug.

I think semaphore.NewWeighted should be set to 1.  Only in this way can we guarantee to prevent concurrency.

please add the reason why you think it is a bug.

The real purpose of orderedMutex is to prevent IO concurrency. If it is set to infinite, it is the same as no lock. How can it prevent concurrency? I think your original intention should be that this is 1.

@chrislusf

this is removed in https://github.com/seaweedfs/seaweedfs/pull/4089

the PR https://github.com/seaweedfs/seaweedfs/pull/4089 is merged.

repo: seaweedfs/seaweedfs


text:
Filer S3 500 Error (upload error: unmarshalled error )

Describe the bug
Some s3 upload errors  during maintaince.
System Setup
AlmaLinux 8.7
Seaweedfs-3.57 3 master servers 3 filer servers 20 volume servers.
Expected behavior
No errors during upload
Logs

filer.log
E1019 18:50:13.088244 filer_server_handlers_write_upload.go:203 upload error: unmarshalled error http://192.168.9.136:8082/8218,0e67206774f809db: failed to write to replicas for volume 8218: [192.168.9.134:8082]: unmarshalled error http://192.168.9.134:8082/8218,0e67206774f809db?ts=1697741411&ttl=5d&type=replicate: failed to write to local disk: invalid argument E1019 18:50:13.089497 s3api_object_handlers.go:519 upload to filer error: unmarshalled error http://192.168.9.136:8082/8218,0e67206774f809db: failed to write to replicas for volume 8218: [192.168.9.134:8082]: unmarshalled error http://192.168.9.134:8082/8218,0e67206774f809db?ts=1697741411&ttl=5d&type=replicate: failed to write to local disk: invalid argument E1019 18:50:20.326799 filer_server_handlers_write_upload.go:203 upload error: unmarshalled error http://192.168.9.136:8082/8218,0e6720bc2bc23f0a: failed to write to replicas for volume 8218: [192.168.9.134:8082]: unmarshalled error http://192.168.9.134:8082/8218,0e6720bc2bc23f0a?ts=1697741419&ttl=5d&type=replicate: failed to write to local disk: invalid argument E1019 18:50:20.327412 s3api_object_handlers.go:519 upload to filer error: unmarshalled error http://192.168.9.136:8082/8218,0e6720bc2bc23f0a: failed to write to replicas for volume 8218: [192.168.9.134:8082]: unmarshalled error http://192.168.9.134:8082/8218,0e6720bc2bc23f0a?ts=1697741419&ttl=5d&type=replicate: failed to write to local disk: invalid argument E1019 18:51:21.908460 filer_server_handlers_write_upload.go:203 upload error: unmarshalled error http://192.168.9.136:8082/8218,0e6723df940237cf: failed to write to replicas for volume 8218: [192.168.9.134:8082]: unmarshalled error http://192.168.9.134:8082/8218,0e6723df940237cf?ts=1697741480&ttl=5d&type=replicate: failed to write to local disk: invalid argument E1019 18:51:21.909514 s3api_object_handlers.go:519 upload to filer error: unmarshalled error http://192.168.9.136:8082/8218,0e6723df940237cf: failed to write to replicas for volume 8218: [192.168.9.134:8082]: unmarshalled error http://192.168.9.134:8082/8218,0e6723df940237cf?ts=1697741480&ttl=5d&type=replicate: failed to write to local disk: invalid argument E1019 18:52:06.155634 filer_server_handlers_write_upload.go:203 upload error: unmarshalled error http://192.168.9.136:8082/8218,0e6727280e844e25: failed to write to replicas for volume 8218: [192.168.9.134:8082]: unmarshalled error http://192.168.9.134:8082/8218,0e6727280e844e25?ts=1697741524&ttl=5d&type=replicate: failed to write to local disk: invalid argument E1019 18:52:06.155799 s3api_object_handlers.go:519 upload to filer error: unmarshalled error http://192.168.9.136:8082/8218,0e6727280e844e25: failed to write to replicas for volume 8218: [192.168.9.134:8082]: unmarshalled error http://192.168.9.134:8082/8218,0e6727280e844e25?ts=1697741524&ttl=5d&type=replicate: failed to write to local disk: invalid argument E1019 18:53:28.089785 filer_server_handlers_write_upload.go:203 upload error: unmarshalled error http://192.168.9.136:8082/8218,0e672bbd51b08e9d: failed to write to replicas for volume 8218: [192.168.9.134:8082]: unmarshalled error http://192.168.9.134:8082/8218,0e672bbd51b08e9d?ts=1697741606&ttl=5d&type=replicate: failed to write to local disk: invalid argument
master.log
I1019 18:49:59.443813 volume_layout.go:383 Volume 8218 becomes unwritable I1019 18:49:59.443851 topology_vacuum.go:74 2 Start vacuuming 8218 on 192.168.9.134:8082 I1019 18:49:59.443857 topology_vacuum.go:74 0 Start vacuuming 8218 on 192.168.9.136:8082 I1019 18:49:59.443899 topology_vacuum.go:74 1 Start vacuuming 8218 on 192.168.9.132:8081 I1019 18:49:59.472383 topology_vacuum.go:102 Complete vacuuming 8218 on 192.168.9.132:8081 I1019 18:49:59.480881 topology_vacuum.go:102 Complete vacuuming 8218 on 192.168.9.136:8082 I1019 18:49:59.489700 topology_vacuum.go:102 Complete vacuuming 8218 on 192.168.9.134:8082 I1019 18:49:59.489714 topology_vacuum.go:128 Start Committing vacuum 8218 on 192.168.9.136:8082 I1019 18:49:59.521747 topology_vacuum.go:147 Complete Committing vacuum 8218 on 192.168.9.136:8082 I1019 18:49:59.521754 topology_vacuum.go:128 Start Committing vacuum 8218 on 192.168.9.132:8081 I1019 18:49:59.522652 topology.go:280 removing volume info: Id:8070, Size:0, ReplicaPlacement:020, Collection:vms-mconv, Version:3, FileCount:0, DeleteCount:0, DeletedByteCount:0, ReadOnly:false from 192.168.9.137:8083 I1019 18:49:59.522671 volume_layout.go:396 Volume 8070 becomes writable I1019 18:49:59.550279 topology_vacuum.go:147 Complete Committing vacuum 8218 on 192.168.9.132:8081 I1019 18:49:59.550287 topology_vacuum.go:128 Start Committing vacuum 8218 on 192.168.9.134:8082 I1019 18:49:59.556993 topology_vacuum.go:147 Complete Committing vacuum 8218 on 192.168.9.134:8082 I1019 18:49:59.557707 volume_layout.go:383 Volume 8067 becomes unwritable I1019 18:49:59.557726 topology_vacuum.go:74 2 Start vacuuming 8067 on 192.168.9.137:8084 I1019 18:49:59.557856 topology_vacuum.go:74 0 Start vacuuming 8067 on 192.168.9.132:8081 I1019 18:49:59.557921 topology_vacuum.go:74 1 Start vacuuming 8067 on 192.168.9.135:8081 I1019 18:49:59.579700 volume_layout.go:396 Volume 8218 becomes writable I1019 18:49:59.579967 master_server.go:325 error: copy volume 8218 from 192.168.9.134:8082 to 192.168.9.134:8083: rpc error: code = Unknown desc = failed to copy /data/swfs/hdd3/volume/vms-mconv_8218.dat file: receiving /data/swfs/hdd3/volume/vms-mconv_8218.dat: rpc error: code = Unknown desc = volume 8218 is compacted
volume_server.log
I1019 18:35:51.637028 volume_loading.go:169 updating leveldb medium index /data/swfs/hdd2/volume/vms-mconv_8230.ldb I1019 18:49:59.478831 needle_map_leveldb.go:338 loading idx to leveldb from offset 0 for file: /data/swfs/hdd2/volume/vms-mconv_8218.cpx I1019 18:49:59.550107 volume_vacuum.go:106 Committing volume 8218 vacuuming... I1019 18:49:59.552134 volume_loading.go:91 readSuperBlock volume 8218 version 3 I1019 18:49:59.552149 volume_loading.go:116 open to read file /data/swfs/hdd2/volume/vms-mconv_8218.idx I1019 18:49:59.552230 needle_map_sorted_file.go:24 Start to Generate /data/swfs/hdd2/volume/vms-mconv_8218.sdx from /data/swfs/hdd2/volume/vms-mconv_8218.idx I1019 18:49:59.556161 needle_map_sorted_file.go:26 Finished Generating /data/swfs/hdd2/volume/vms-mconv_8218.sdx from /data/swfs/hdd2/volume/vms-mconv_8218.idx I1019 18:49:59.760297 needle_map_leveldb.go:338 loading idx to leveldb from offset 0 for file: /data/swfs/hdd2/volume/vms-mconv_8232.cpx I1019 18:49:59.836768 volume_vacuum.go:106 Committing volume 8232 vacuuming... I1019 18:49:59.839459 volume_loading.go:91 readSuperBlock volume 8232 version 3 I1019 18:49:59.839501 volume_loading.go:169 updating leveldb medium index /data/swfs/hdd2/volume/vms-mconv_8232.ldb I1019 18:50:00.195395 needle_map_leveldb.go:338 loading idx to leveldb from offset 0 for file: /data/swfs/hdd2/volume/vms-mconv_8230.cpx I1019 18:50:01.446143 volume_vacuum.go:106 Committing volume 8230 vacuuming... I1019 18:50:01.448758 volume_loading.go:91 readSuperBlock volume 8230 version 3 I1019 18:50:01.448809 volume_loading.go:169 updating leveldb medium index /data/swfs/hdd2/volume/vms-mconv_8230.ldb I1019 18:50:08.319939 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:08.320991 common.go:75 response method:POST URL:/8218,0e67206774f809db?ts=1697741408&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:08.799204 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:08.799229 common.go:75 response method:POST URL:/8218,0e67206774f809db?ts=1697741408&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:09.511876 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:09.511901 common.go:75 response method:POST URL:/8218,0e67206774f809db?ts=1697741408&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:09.993843 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:09.993866 common.go:75 response method:POST URL:/8218,0e67206774f809db?ts=1697741409&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:10.469726 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:10.469750 common.go:75 response method:POST URL:/8218,0e67206774f809db?ts=1697741409&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:11.182497 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:11.182521 common.go:75 response method:POST URL:/8218,0e67206774f809db?ts=1697741409&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:11.898277 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:11.898300 common.go:75 response method:POST URL:/8218,0e67206774f809db?ts=1697741411&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:12.374474 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:12.374507 common.go:75 response method:POST URL:/8218,0e67206774f809db?ts=1697741411&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:13.087509 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:13.087534 common.go:75 response method:POST URL:/8218,0e67206774f809db?ts=1697741411&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:15.398189 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:15.398217 common.go:75 response method:POST URL:/8218,0e6720bc2bc23f0a?ts=1697741415&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:15.890478 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:15.890516 common.go:75 response method:POST URL:/8218,0e6720bc2bc23f0a?ts=1697741415&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:16.620043 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:16.620068 common.go:75 response method:POST URL:/8218,0e6720bc2bc23f0a?ts=1697741415&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:17.134436 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:17.134475 common.go:75 response method:POST URL:/8218,0e6720bc2bc23f0a?ts=1697741417&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:17.625789 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:17.625819 common.go:75 response method:POST URL:/8218,0e6720bc2bc23f0a?ts=1697741417&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:18.356396 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:18.356431 common.go:75 response method:POST URL:/8218,0e6720bc2bc23f0a?ts=1697741417&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:19.103147 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:19.103209 common.go:75 response method:POST URL:/8218,0e6720bc2bc23f0a?ts=1697741419&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:19.596009 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:19.596044 common.go:75 response method:POST URL:/8218,0e6720bc2bc23f0a?ts=1697741419&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:20.325799 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:20.325825 common.go:75 response method:POST URL:/8218,0e6720bc2bc23f0a?ts=1697741419&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:28.197396 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:28.197447 common.go:75 response method:POST URL:/8218,0e6721777d3319c4?ts=1697741428&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"} I1019 18:50:28.676267 store_replicate.go:53 failed to write to local disk: invalid argument I1019 18:50:28.676291 common.go:75 response method:POST URL:/8218,0e6721777d3319c4?ts=1697741428&ttl=5d&type=replicate with httpStatus:500 and JSON:{"error":"failed to write to local disk: invalid argument"}
Additional context
WorkArround restart volume servers with problem volume

invalid argument - it means I couldn't get a needle. It means I couldn’t get the needle.
You need to stop weed volume and run weed fix /data/dir

authors: kgalinurov kmlebedev


text:
建议：漏洞列表添加漏洞地址模块

建议在填写漏洞列表时添加一个漏洞地址模块，如果说一个系统多个路径下存在同一种类型的漏洞，不添加详细漏洞地址在导出报告后看起来很怪异

已更新，详见v1.0.9

repo: sec-report/SecReport


text:
怎么用secretflow实现多机集群pir

我想使用docker部署secretflow，生产模式下实现pir，但是我不知道如何通过代码实现，以下是我的一些尝试。
以alice为client，bob为server。
alice.py代码如下：
import os
import sys
import time
import logging
import multiprocess
from absl import app

import spu
import secretflow as sf

sf.shutdown()
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)

alice, bob = sf.PYU('alice'), sf.PYU('bob')

cluster_config ={
    'parties': {
        'alice': {
            'address': '10.229.208.37:3545',
            'listen_addr': '0.0.0.0:3545'
        },
        'bob': {
            'address': '10.229.208.36:3544',
            'listen_addr': '0.0.0.0:3544'
        },
    },
    'self_party': 'alice'
}

cluster_def = {
    'nodes': [
        {
            'party': 'alice',
            'address': f'10.229.208.37:3457',
        },
        {
            'party': 'bob',
            'address':  f'10.229.208.36:3458',
        },
    ],
    'runtime_config': {
        'protocol': spu.spu_pb2.SEMI2K,
        'field': spu.spu_pb2.FM128,
    }
}

link_desc = {
    'recv_timeout_ms': 3600000,
}

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)

sf.init(address='10.229.208.37:3456', cluster_config=cluster_config)

key_columns = ['id']
label_columns = ['register_date','age']
spu = sf.SPU(cluster_def, link_desc)

pir_input_path = '/root/B_PIR_DATA.csv'
pir_oprf_key_path = '/root/oprf_key.bin'
pir_setup_path = '/root/setup_path'

alice_input_path = '/root/A_PIR_ID.csv'

alice_config = {
    'input_path': alice_input_path,
    'key_columns': key_columns,
    'output_path': '/root/sf_pir_out.csv',
}

bob_config = {
    'oprf_key_path': pir_oprf_key_path,
    'setup_path': pir_setup_path,
}

query_config = {
    alice: alice_config,
    bob: bob_config,
}

start = time.time()

reports = spu.pir_query(
    server='bob',
    config=query_config,
    protocol='KEYWORD_PIR_LABELED_PSI'
)

logging.info(f"online pir reports: {reports}")
logging.info(f"pir query cost time: {time.time() - start}")

sf.shutdown()

bob.py：
import logging
import multiprocess

import spu
import secretflow as sf

sf.shutdown()

alice, bob = sf.PYU('alice'), sf.PYU('bob')

cluster_config ={
    'parties': {
        'alice': {
            'address': '10.229.208.37:3545',
            'listen_addr': '0.0.0.0:3545'
        },
        'bob': {
            'address': '10.229.208.36:3544',
            'listen_addr': '0.0.0.0:3544'
        },
    },
    'self_party': 'bob'
}

cluster_def = {
    'nodes': [
        {
            'party': 'alice',
            'address': f'10.229.208.37:3457',
        },
        {
            'party': 'bob',
            'address':  f'10.229.208.36:3458',
        },
    ],
    'runtime_config': {
        'protocol': spu.spu_pb2.SEMI2K,
        'field': spu.spu_pb2.FM128,
        'sigmoid_mode': spu.spu_pb2.RuntimeConfig.SIGMOID_REAL,
    }
}

link_desc = {
    'recv_timeout_ms': 3600000,
}

sf.init(address='10.229.208.36:3556', cluster_config=cluster_config)

key_columns = ['id']
label_columns = ['register_date','age']
spu = sf.SPU(cluster_def=cluster_def)

pir_input_path = '/root/B_PIR_DATA.csv'
pir_oprf_key_path = '/root/oprf_key.bin'
pir_setup_path = '/root/setup_path'

start = time.time()

reports = spu.pir_setup(
    server='bob',
    input_path=pir_input_path,
    key_columns=key_columns,
    label_columns=label_columns,
    oprf_key_path=pir_oprf_key_path,
    setup_path=pir_setup_path,
    num_per_query=1,
    label_max_len=18,
)

logging.info(f"offline pir reports: {reports}")
logging.info(f"setup cost time: {time.time() - start}")

脚本运行后，bob.py退出

alice.py停在这里：

请问我应该如何修改？

第一步，offline 数据预处理
按上面bob 端的脚本，执行offline阶段的数据预处理，调用的是pir_setup，单方任务，提前执行
第二部，online 查询阶段
alice和bob都执行上面alice端的脚本，执行query函数，双方交互完成查询

第一步，offline 数据预处理 按上面bob 端的脚本，执行offline阶段的数据预处理，调用的是pir_setup，单方任务，提前执行 第二部，online 查询阶段 alice和bob都执行上面alice端的脚本，执行query函数，双方交互完成查询

如果直接执行bob端脚本，会报错连接不上alice端，因为sf.init使用的cluster_config中有alice，需要都去掉只留下bob的吗？

第一步，offline 数据预处理 按上面bob 端的脚本，执行offline阶段的数据预处理，调用的是pir_setup，单方任务，提前执行 第二部，online 查询阶段 alice和bob都执行上面alice端的脚本，执行query函数，双方交互完成查询

如果直接执行bob端脚本，会报错连接不上alice端，因为sf.init使用的cluster_config中有alice，需要都去掉只留下bob的吗？

@zhouaihui zhou 看一下，只留bob是不是可以执行单方集群的任务
sf.init(address='10.229.208.36:3556', cluster_config=cluster_config)

可以在sf.init设置参数 enable_waiting_for_other_parties_ready=False，这样不会去等待其他方启动。
具体参考API说明 https://www.secretflow.org.cn/docs/secretflow/latest/zh-Hans/source/secretflow#secretflow.init

可以在sf.init设置参数 enable_waiting_for_other_parties_ready=False，这样不会去等待其他方启动。 具体参考API说明 https://www.secretflow.org.cn/docs/secretflow/latest/zh-Hans/source/secretflow#secretflow.init

此时我将bob端脚本只做了上述init参数的修改，运行时报错：

第一步，offline 数据预处理 按上面bob 端的脚本，执行offline阶段的数据预处理，调用的是pir_setup，单方任务，提前执行 第二部，online 查询阶段 alice和bob都执行上面alice端的脚本，执行query函数，双方交互完成查询

因为你使用了SPU，SPU还是需要互相连接的。 建议按照前面的建议，在bob的脚本上加上online查询。

数据预处理，调用的pir_setup是需要使用spu的，所以setup和query不能分开吗？每次新的client查询都要server端重新运行setup和query？

第一步，offline 数据预处理 按上面bob 端的脚本，执行offline阶段的数据预处理，调用的是pir_setup，单方任务，提前执行 第二部，online 查询阶段 alice和bob都执行上面alice端的脚本，执行query函数，双方交互完成查询

因为你使用了SPU，SPU还是需要互相连接的。 建议按照前面的建议，在bob的脚本上加上online查询。

alice脚本不变，我现在把bob端的setup和query放在了一个脚本中，如下：
import os
import sys
import time
import logging
import multiprocess
from absl import app

import spu
import secretflow as sf

sf.shutdown()
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)

alice, bob = sf.PYU('alice'), sf.PYU('bob')

cluster_config ={
    'parties': {
        'alice': {
            'address': '10.229.208.37:3545',
            'listen_addr': '0.0.0.0:3545'
        },
        'bob': {
            'address': '10.229.208.36:3544',
            'listen_addr': '0.0.0.0:3544'
        },
    },
    'self_party': 'bob'
}

cluster_def = {
    'nodes': [
        {
            'party': 'alice',
            'address': f'10.229.208.37:3457',
        },
        {
            'party': 'bob',
            'address':  f'10.229.208.36:3458',
        },
    ],
    'runtime_config': {
        'protocol': spu.spu_pb2.SEMI2K,
        'field': spu.spu_pb2.FM128,
    }
}

link_desc = {
    'recv_timeout_ms': 3600000,
}

# init log
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)

sf.init(address='10.229.208.36:3556', cluster_config=cluster_config, enable_waiting_for_other_parties_ready=False)

key_columns = ['id']
label_columns = ['register_date','age']
spu = sf.SPU(cluster_def=cluster_def)

# server dataset
pir_input_path = '/root/B_PIR_DATA.csv'
pir_oprf_key_path = '/root/oprf_key.bin'
pir_setup_path = '/root/setup_path'

start = time.time()

# server setup
reports = spu.pir_setup(
    server='bob',
    input_path=pir_input_path,
    key_columns=key_columns,
    label_columns=label_columns,
    oprf_key_path=pir_oprf_key_path,
    setup_path=pir_setup_path,
    num_per_query=1,
    label_max_len=18,
)

#print(f"psi reports: {reports}")
logging.info(f"offline pir reports: {reports}")
logging.info(f"setup cost time: {time.time() - start}")

# client data query
alice_input_path = '/root/A_PIR_ID.csv'

# client config
alice_config = {
    'input_path': alice_input_path,
    'key_columns': key_columns,
    'output_path': '/root/sf_pir_out.csv',
}

# server config
bob_config = {
    'oprf_key_path': pir_oprf_key_path,
    'setup_path': pir_setup_path,
}

query_config = {
    alice: alice_config,
    bob: bob_config,
}

start = time.time()

reports = spu.pir_query(
    server='bob',
    config=query_config,
    protocol='KEYWORD_PIR_LABELED_PSI'
)

logging.info(f"online pir reports: {reports}")
logging.info(f"pir query cost time: {time.time() - start}")

sf.shutdown()

同时运行两方脚本，但是停在了这里不会退出，已经有了查询结果。运行完该如何退出？

bob可以参照下面的代码，执行单方任务，做offline数据预处理，offline完成后双方再执行online query
https://github.com/secretflow/spu/blob/main/examples/python/pir/pir_setup.py

bob可以参照下面的代码，执行单方任务，做offline数据预处理，offline完成后双方再执行online query https://github.com/secretflow/spu/blob/main/examples/python/pir/pir_setup.py

好的，谢谢。我将setup过程单独写了一个脚本，成功进行了预处理。
还有2个问题：
1、具体的查询仍然使用前面的alice脚本，两方同时执行？
2、server端的数据是否只需要一次数据预处理，后期只需要两方执行查询脚本交互？

bob可以参照下面的代码，执行单方任务，做offline数据预处理，offline完成后双方再执行online query https://github.com/secretflow/spu/blob/main/examples/python/pir/pir_setup.py

好的，谢谢。我将setup过程单独写了一个脚本，成功进行了预处理。 还有2个问题： 1、具体的查询仍然使用前面的alice脚本，两方同时执行？ 2、server端的数据是否只需要一次数据预处理，后期只需要两方执行查询脚本交互？

回答：

是的，两方执行上面有pir_query的脚本
是的预处理一次，可以双方可以多次指定query，但需要注意的是server预处理需要一个32B的oprf_key，实际使用时建议定期更新oprf_key并重新执行预处理

bob可以参照下面的代码，执行单方任务，做offline数据预处理，offline完成后双方再执行online query https://github.com/secretflow/spu/blob/main/examples/python/pir/pir_setup.py

好的，谢谢。我将setup过程单独写了一个脚本，成功进行了预处理。 还有2个问题： 1、具体的查询仍然使用前面的alice脚本，两方同时执行？ 2、server端的数据是否只需要一次数据预处理，后期只需要两方执行查询脚本交互？

回答：

是的，两方执行上面有pir_query的脚本
是的预处理一次，可以双方可以多次执行query，但需要注意的是server预处理需要一个32B的oprf_key，实际使用时建议定期更新oprf_key并重新执行预处理

好的，非常感谢各位解答~

authors: hlysf zhanglei486 zhouaihui

repo: secretflow/secretflow


text:
Documentation: Recommended hardware for workloads

Customers have expressed a desire for a minimum acceptable hardware configuration for various workloads.
This task involves:

Starting with 3 OpenFL tutorials (pytorch, tensorflow, jax), Profile how long it takes to run them and make a recommendation for the minimum hardware that should be used (i.e. high core CPU, GPU, memory, etc.). This list should be expanded over time (not as part of this task)
Create a new page in the OpenFL documentation that includes a table of the available tutorials that includes this information

I have read the CLA Document and I hereby sign the CLA

Closing because this issue is stale. We can reopen this as priorities change.

repo: securefederatedai/openfl


text:
fix(KONFLUX-3663): format PipelineRun files and upload SAST results

This update configures the SAST task to upload SARIF results to quay.io for long-term storage
Please note that this PR was automatically generated and may include unrelated changes due to automatic YAML formatting performed by yq
The YAML files will be indented using 2 spaces, if the YAML file uses indentationless list the automation will try to keep this format
The PR contains two separate commits:

Format YAML files: Ensures consistent indentation and formatting of the YAML files
Upload SAST results: Configures the PipelineRun files to enable uploading SARIF results to quay.io

Separating these changes into two commits simplifies the review process. The first commit focuses on indentation and formatting, while the second commit contains the semantic changes
Related:

https://issues.redhat.com/browse/KONFLUX-3663
https://issues.redhat.com/browse/KONFLUX-2263

/approve

/ok-to-test

authors: ccronca lance

repo: securesign/rekor


text:
Unsupported OpenType signature ttcf after changing to Noto Sans CJK TC

/home/user/extratermqt-0.68.0-linux-x64/main/dist/main.cjs:13537
          throw new Error("Unsupported OpenType signature " + signature);
                ^
Error: Unsupported OpenType signature ttcf
    at parseBuffer (/home/user/extratermqt-0.68.0-linux-x64/main/dist/main.cjs:13537:17)
    at /home/user/extratermqt-0.68.0-linux-x64/main/dist/main.cjs:13704:22
    at /home/user/extratermqt-0.68.0-linux-x64/main/dist/main.cjs:13437:11
    at FSReqCallback.readFileAfterClose [as oncomplete] (node:internal/fs/read_file_context:68:3)
panic: Post "/command": unsupported protocol scheme ""

goroutine 1 [running]:
main.runCommand({0x0, 0x0}, 0xc0000bbf08)
        /home/runner/work/extraterm-launcher/extraterm-launcher/cmd/extraterm-launcher/extraterm-launcher.go:200 +0x2f4
main.runOpenWindowAtCommand({0x0, 0x0}, {0xc00001a004, 0x27})
        /home/runner/work/extraterm-launcher/extraterm-launcher/cmd/extraterm-launcher/extraterm-launcher.go:135 +0xd9
main.main()
        /home/runner/work/extraterm-launcher/extraterm-launcher/cmd/extraterm-launcher/extraterm-launcher.go:40 +0x73

This sound like issue #402 . It might already be fixed now since the upgrade to Qt6. Could you retest this issue with the latest Extraterm version?

Fixed in 0.71.1.


text:
README.md mentions Ubuntu installation step

Looks like for Ubuntu 14.04 it's enough to link database file to ~/.dircolors as ~/.bashrc already has statements which execute right eval command.

thanks for the note

repo: seebi/dircolors-solarized


text:
Weird skyplot output

Not sure how this happened:

Ran this on 12 May 2023 at around 9:45pm PDT with the following command:
skyplot --tle ~/Desktop/tles/ISS.tle --time-start "13 May 04:00"

And the following TLE
ISS
1 25544U 98067A   23132.42640824  .00014922  00000-0  26744-3 0  9995
2 25544  51.6400 145.7781 0006355 339.7698 124.4230 15.50147384396243

Hopefully the rest can be reproduced from cached logs and alignment model.

OHHH I didn't even notice that this is showing the mount's actual path, since this was from a recent pass. I didn't even know that feature still worked. I need to adjust the color of that line so it has better contrast with the background.

repo: seeing-things/track


text:
Fxes issue #303 preventing multiple rule parsing

See issue #303 for example, just a simple fix to add elements to an array in the hash key rather than assigning a new array every time a rule was processed.

This looks good, why is it not merged?

:+1:  why indeed?

We merged #308 instead.

authors: avit dimerman joelfperion sethdeckard


text:
Nuxt 3 + Vanilla Extract Vite nuxt build Error

Describe the bug
Running yarn build (nuxt build is the actual command) in a project that uses the Vanilla Extract Vite plugin produces this error:
▶ yarn build
yarn run v1.22.15
$ nuxt build
Nuxt CLI v3.0.0-27255771.f78ec93

 ERROR  [vite:css] /Users/timothymorris/Projects/vanilla-extract-ssr-error/styles.css.ts$$$nuxt3-vanilla-extract-ssr-error.vanilla.css:1:10: Unknown word
file: styles.css.ts$$$nuxt3-vanilla-extract-ssr-error.vanilla.css

 ERROR  /Users/timothymorris/Projects/vanilla-extract-ssr-error/styles.css.ts$$$nuxt3-vanilla-extract-ssr-error.vanilla.css:1:10: Unknown word

  at Input.error (node_modules/postcss/lib/input.js:123:16)
  at Parser.unknownWord (node_modules/postcss/lib/parser.js:518:22)
  at Parser.other (node_modules/postcss/lib/parser.js:149:12)
  at Parser.parse (node_modules/postcss/lib/parser.js:59:16)
  at parse (node_modules/postcss/lib/parse.js:11:12)
  at new LazyResult (node_modules/postcss/lib/lazy-result.js:133:16)
  at Processor.process (node_modules/postcss/lib/processor.js:36:12)
  at compileCSS (node_modules/vite/dist/node/chunks/dep-85dbaaa7.js:20288:10)
  at processTicksAndRejections (internal/process/task_queues.js:95:5)
  at async Object.transform (node_modules/vite/dist/node/chunks/dep-85dbaaa7.js:19876:50)

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Link to reproduction
Repo (Reused from #445, though I don't believe these issues to be related)
System Info
Output of npx envinfo --system --npmPackages @vanilla-extract/css,@vanilla-extract/webpack-plugin,@vanilla-extract/esbuild-plugin,@vanilla-extract/vite-plugin,@vanilla-extract/sprinkles,webpack,esbuild,vite --binaries --browsers:
  System:
    OS: macOS 12.0.1
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 49.77 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.13.0 - ~/.nvm/versions/node/v16.13.0/bin/node
    Yarn: 1.22.15 - /usr/local/bin/yarn
    npm: 8.1.0 - ~/.nvm/versions/node/v16.13.0/bin/npm
  Browsers:
    Chrome: 95.0.4638.54
    Firefox: 89.0.1
    Safari: 15.1
  npmPackages:
    @vanilla-extract/css: ^1.6.3 => 1.6.3
    @vanilla-extract/vite-plugin: ^3.0.0 => 3.0.0

I've updated the example repo with a static css file for comparison in behavior, and upgraded all the deps. Still seeing the above error when trying to build the project.

same error

After a bit more investigation, it appears that during build, this code is being fed into postcss:
import { injectStyles } from '@vanilla-extract/css/injectStyles';                                                                                15:33:58

const inject = (css) => injectStyles({
  fileScope: {"filePath":"styles/reset.css.ts","packageName":"my-app"},
  css
});

inject("/* my css here, omitted for brevity */");

import.meta.hot.on('vanilla-extract-style-update:styles/reset.css.ts$$$my-app', (css) => {
  inject(css);
});

which appears to match the output of @vanilla-extract/vite-plugin/src/index.ts lines 90-103.
Ultimately its become clear that this is a Nuxt 3 issue not a vanilla-extract issue (as this works perfectly with vite out of the box), but does anybody have any ideas on what configuration might cause postcss to receive the above code (rather than what I assume is the output of running that code)?

authors: Ttou tkofh

repo: seek-oss/vanilla-extract