text:
Split dataset by node: index error when sharding iterable dataset

Describe the bug
Context: we're splitting an iterable dataset by node and then passing it to a torch data loader with multiple workers
When we iterate over it for 5 steps, we don't get an error
When we instead iterate over it for 7 steps, we get an IndexError when fetching the data
Steps to reproduce the bug
Here, we have 2 JAX processes (jax.process_count() = 2) which we split the dataset over. The dataset loading script can be found here: https://huggingface.co/datasets/distil-whisper/librispeech_asr/blob/c6a1e805cbfeed5057400ac5937327d7e30281b8/librispeech_asr.py#L310

 Code to reproduce
from datasets import load_dataset
import jax
from datasets.distributed import split_dataset_by_node
from torch.utils.data import DataLoader
from tqdm import tqdm

# load an example dataset (https://huggingface.co/datasets/distil-whisper/librispeech_asr)
dataset = load_dataset("distil-whisper/librispeech_asr", "all", split="train.clean.100", streaming=True)
# just keep the text column -> no need to define a collator
dataset_text = dataset.remove_columns(set(dataset.features.keys()) - {"text"})

# define some constants
batch_size = 256
num_examples = 7
num_workers = dataset_text.n_shards

# try with multiple workers
dataloader = DataLoader(dataset_text, batch_size=batch_size, num_workers=num_workers, drop_last=True)

for i, batch in tqdm(enumerate(dataloader), total=num_examples, desc="Multiple workers"):
    if i == num_examples:
        break

# try splitting by node (we can't do this with `dataset_text` since `split_dataset_by_node` expects the Audio column for an ASR dataset)
dataset = split_dataset_by_node(dataset, rank=jax.process_index(), world_size=jax.process_count())
# remove the text column again
dataset_text = dataset.remove_columns(set(dataset.features.keys()) - {"text"})
dataloader = DataLoader(dataset_text, batch_size=16, num_workers=num_workers // 2, drop_last=True)

for i, batch in tqdm(enumerate(dataloader), total=num_examples, desc="Split by node"):
    if i == num_examples:
        break

# too many workers
dataloader = DataLoader(dataset_text, batch_size=256, num_workers=num_workers, drop_last=True)
for i, batch in tqdm(enumerate(dataloader), total=num_examples, desc="Too many workers"):
    if i == num_examples:
        break

 With 5 examples:
Multiple workers: 100%|███████████████████████████████████████████████████████████████████| 5/5 [00:16<00:00,  3.33s/it]
Assigning 7 shards (or data sources) of the dataset to each node.
Split by node: 100%|██████████████████████████████████████████████████████████████████████| 5/5 [00:13<00:00,  2.76s/it]
Assigning 7 shards (or data sources) of the dataset to each node.
Too many dataloader workers: 14 (max is dataset.n_shards=7). Stopping 7 dataloader workers.
To parallelize data loading, we give each process some shards (or data sources) to process. Therefore it's unnecessary t
o have a number of workers greater than dataset.n_shards=7. To enable more parallelism, please split the dataset in more
 files than 7.
Too many workers: 100%|███████████████████████████████████████████████████████████████████| 5/5 [00:15<00:00,  3.03s/it]

 With 7 examples:
Multiple workers: 100%|███████████████████████████████████████████████████████████████████| 7/7 [00:32<00:00,  4.60s/it]
Assigning 7 shards (or data sources) of the dataset to each node.
Split by node: 100%|██████████████████████████████████████████████████████████████████████| 7/7 [00:13<00:00,  1.89s/it]
Assigning 7 shards (or data sources) of the dataset to each node.
Too many dataloader workers: 14 (max is dataset.n_shards=7). Stopping 7 dataloader workers.
To parallelize data loading, we give each process some shards (or data sources) to process. Therefore it's unnecessary t
o have a number of workers greater than dataset.n_shards=7. To enable more parallelism, please split the dataset in more
 files than 7.
Too many workers: 100%|███████████████████████████████████████████████████████████████████| 7/7 [00:15<00:00,  2.22s/it]
Traceback (most recent call last):
  File "distil-whisper/test_librispeech.py", line 36, in <module>
    for i, batch in tqdm(enumerate(dataloader), total=num_examples, desc="Too many workers"):
  File "/home/sanchitgandhi/hf/lib/python3.8/site-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/home/sanchitgandhi/hf/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 633, in __next__
    data = self._next_data()
  File "/home/sanchitgandhi/hf/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1325, in _next_data
    return self._process_data(data)
  File "/home/sanchitgandhi/hf/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
    data.reraise()
  File "/home/sanchitgandhi/hf/lib/python3.8/site-packages/torch/_utils.py", line 644, in reraise
    raise exception
IndexError: Caught IndexError in DataLoader worker process 7.
Original Traceback (most recent call last):
  File "/home/sanchitgandhi/hf/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/sanchitgandhi/hf/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 32, in fetch
    data.append(next(self.dataset_iter))
  File "/home/sanchitgandhi/datasets/src/datasets/iterable_dataset.py", line 986, in __iter__
    yield from self._iter_pytorch(ex_iterable)
  File "/home/sanchitgandhi/datasets/src/datasets/iterable_dataset.py", line 920, in _iter_pytorch
    for key, example in ex_iterable.shard_data_sources(worker_info.id, worker_info.num_workers):
  File "/home/sanchitgandhi/datasets/src/datasets/iterable_dataset.py", line 540, in shard_data_sources
    self.ex_iterable.shard_data_sources(worker_id, num_workers),
  File "/home/sanchitgandhi/datasets/src/datasets/iterable_dataset.py", line 796, in shard_data_sources
    self.ex_iterable.shard_data_sources(worker_id, num_workers),
  File "/home/sanchitgandhi/datasets/src/datasets/iterable_dataset.py", line 126, in shard_data_sources
    requested_gen_kwargs = _merge_gen_kwargs([gen_kwargs_list[i] for i in shard_indices])
  File "/home/sanchitgandhi/datasets/src/datasets/utils/sharding.py", line 76, in _merge_gen_kwargs
    for key in gen_kwargs_list[0]
IndexError: list index out of range

Expected behavior
Should pass for both 5 and 7 examples
Environment info

datasets version: 2.12.1.dev0
Platform: Linux-5.13.0-1023-gcp-x86_64-with-glibc2.29
Python version: 3.8.10
Huggingface_hub version: 0.14.1
PyArrow version: 12.0.0
Pandas version: 2.0.1

cc @lhoestq in case you have any ideas here!

I am also facing the same problem. Could you let me know if you found a solution for this?

I couldn't reproduce with the latest version of datasets 2.16.1, can you update datasets and try again ?

authors: Munikumar09 lhoestq sanchit-gandhi


text:
~/.huggingface/token is loaded with EOL

Describe the bug
I copy the authentication token from the site and put it into ~/.huggingface/token using vim, nano or what else method.
When I run the code, load_datasets() fails, because a newline character is added to the token.
There is a workaround to solve the problem: truncate -s -1  .huggingface/token
But I believe correct solution would be to drop the finishing "\n" in the library code, for example in HfFolder().get_token(), if it is present, and to drop any finishing space characters too.
Reproduction
from datasets import load_dataset
ds = load_dataset("my-company/my-ds", verification_mode="no_checks", token=True)
Logs
N/A

System info
- huggingface_hub version: 0.16.4
- Platform: Linux-5.15.0-1036-aws-x86_64-with-glibc2.31
- Python version: 3.9.16
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /home/ubuntu/.cache/huggingface/token
- Has saved token ?: True
- Who am I ?: gbr-bst
- Configured git credential helpers:
- FastAI: 2.1.10
- Tensorflow: N/A
- Torch: 1.13.1
- Jinja2: 3.1.2
- Graphviz: N/A
- Pydot: N/A
- Pillow: 9.5.0
- hf_transfer: N/A
- gradio: N/A
- tensorboard: N/A
- numpy: 1.24.3
- pydantic: 1.10.8
- aiohttp: 3.8.5
- ENDPOINT: https://huggingface.co
- HUGGINGFACE_HUB_CACHE: /home/ubuntu/.cache/huggingface/hub
- HUGGINGFACE_ASSETS_CACHE: /home/ubuntu/.cache/huggingface/assets
- HF_TOKEN_PATH: /home/ubuntu/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False

@Wauplin
> do you know that you can save your token using huggingface-cli login instead?
It's that I finally did. But eventually I learned about existence of this option much later than about load_dataset().  (I used the latter much with free datasets until now.)
BTW, the documentation says several times:
token (str or bool, optional) — The token to use as HTTP bearer authorization for remote files. If True, or not specified, will use the token generated when running huggingface-cli login (stored in ~/.huggingface).

That gave me impression that ~/.huggingface is a name of file where the token may be stored.
Is it possible to make it more clear?  According to what I and my coworkers see in our workstations, it should say:
token (str or bool, optional) — The token to use as HTTP bearer authorization for remote files. If True, or not specified, will use the token generated when running huggingface-cli login (stored in ~/.cache/.huggingface/toekn file) or load it from  ~/.huggingface/token file.

> Do you want to open a PR to fix this and add a quick test in [test_utils_hf_folder.py](https://github.com/huggingface/huggingface_hub/blob/main/tests/test_utils_hf_folder.py)?

OK, will do.

@Wauplin PR is created: https://github.com/huggingface/huggingface_hub/pull/1638

Thanks for the quick PR @GBR-613 :) I've approved it and will merge once the CI is green.

source: gharchive/issue

license_type: permissive

license_source: bigquery


text:
Chat bot empty replies

I've made my own chatbot in google colabs using this tutorial: https://youtu.be/UjDpW_SOrlw
I've made some changes like changing the gpt model as I wanted it to speak Russian so I picked up the well trained medium gpt3 model from sberbank.
I pushed it to huggingface https://huggingface.co/life4free96/DialogGPT-med-TeiaMoranta?
So the problem is i send a message, the bot replies to it and when I'm sending the 2nd message it replies with nothing.

And the json output log:
{
"generated_text": "",
"conversation": {
"generated_responses": [
"! Хочешь пойти на утреннюю пробежку вместе? Можно пообедать вместе с тобой",
""
],
"past_user_inputs": [
"Здравствуй",
"Привет"
]
},
"warnings": [
"Conversation input is to long (22), trimming it to (20 - 10)",
"Setting pad_token_id to eos_token_id:50256 for open-end generation."
]
}

Is it possible that your model is just generating an EOS token here (or EOS + special tokens that get removed) ?
In order to debug this, you could use pipeline(model="life4free96/DialogGPT-med-TeiaMoranta") and play around with it.
from transformers import pipeline
from transformers.pipelines.conversational import Conversation

pipe = pipeline(model="life4free96/DialogGPT-med-TeiaMoranta")
conv = Conversation("Привет")
out = pipe(conv)
print(out)
conv.add_user_input("Здравствуй")
out2 = pipe(conv)
print(out2)

Go checkout what happens in the generate function you should get a better grasp of what's happening.

pipeline(model="life4free96/DialogGPT-med-TeiaMoranta")

How can I do it in google colab?

Hi, actually, we should close this, and open a discussion here: https://discuss.huggingface.co/ if you don't mind.
We try to keep conversations in the forum so other users can search an learn from those, which is less convenient on discord (which is more for casual conversation).
I am closing this issue, but please comment with the thread on discuss once you have created it please.
Cheers.

authors: Narsil vexhor


text:
Is there any chance to use this repository under python3.8?

Thanks for the excellent work!
Since my custom environment is under isaac-gym, which is deprecated and only allowed on python3.6-3.8, could you help me easily use lerobot under python3.8?

Hi there, thank you!
Unfortunately, some of our dependencies require python>3.9.
Additionally, Python 3.8 will reach end of life by 2024-10
If you really need 3.8 you can try and downgrade those dependencies until you get it to work but we can't guarantee it. If you manage to get it to work, don't hesitate to paste your .lock file here ;)

authors: aliberts calcualatexzy

repo: huggingface/lerobot

license_source: github-api


text:
Enable Falcon-7b

This PR enables Habana optimized Falcon-7b. This version of the model uses

static input shape along with token_idx and position_ids to avoid repeated graph compilation
fused RoPE kernel

This change requires transformers=4.32.0.dev0
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

There is an issue with the CI, it shouldn't fail. At least not with this error, I'm going to take a look.

Thanks @regisss. I thought the error ModuleNotFoundError: No module named 'transformers.models.falcon' is from using the old Transformers version. However, CI still fails with your patch to upgrade Transformers. If you find any issue with my change, please let me know.

Thanks @regisss. I thought the error ModuleNotFoundError: No module named 'transformers.models.falcon' is from using the old Transformers version. However, CI still fails with your patch to upgrade Transformers. If you find any issue with my change, please let me know.

Your branch is fine and main too. The issue is with the way the CI GitHub workflow is configured, it didn't trigger the tests at the right commit: so basically it skipped my commit of yesterday to upgrade Transformers...
What's weird is that it didn't happen with other PRs :thinking: I'll take a closer look at it!

@schoi-habana Have you also tested Falcon-40B?

@schoi-habana Have you also tested Falcon-40B?

No, this patch was targeted only for Falcon-7B. I was told you are going to work on Falcon-40B

Well the modeling code is the same in Transformers, so not sure there is anything else to do. I'll test it :+1:

@schoi-habana I just piushed a commit to add Falcon in this table: https://github.com/huggingface/optimum-habana#validated-models
You may also see some other commits to see if I can fix this CI issue.

source: gharchive/pull-request

repo: huggingface/optimum-habana


text:
Small fixes

Fix #908 and #901

Codecov Report

Merging #911 into master will increase coverage by 0.01%.
The diff coverage is 90.9%.

@@            Coverage Diff             @@
##           master     #911      +/-   ##
==========================================
+ Coverage   79.03%   79.04%   +0.01%
==========================================
  Files          34       34
  Lines        6234     6242       +8
==========================================
+ Hits         4927     4934       +7
- Misses       1307     1308       +1

Impacted Files
Coverage Δ

pytorch_transformers/tokenization_utils.py
86.56% <90.9%> (+0.03%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c054b5e...7b6e474. Read the comment docs.

Codecov Report

Merging #911 into master will increase coverage by 0.01%.
The diff coverage is 90.9%.

@@            Coverage Diff             @@
##           master     #911      +/-   ##
==========================================
+ Coverage   79.03%   79.04%   +0.01%
==========================================
  Files          34       34
  Lines        6234     6242       +8
==========================================
+ Hits         4927     4934       +7
- Misses       1307     1308       +1

Impacted Files
Coverage Δ

pytorch_transformers/tokenization_utils.py
86.56% <90.9%> (+0.03%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c054b5e...7b6e474. Read the comment docs.

authors: codecov-io thomwolf

repo: huggingface/pytorch-transformers


text:
KTOTrainer Memory Leakage

System Info
I've been running some experiments on KTO using LoRA and noticed that there is a large disparity between the peak allocated and reserved memory, which I suspect is a memory leakage problem.
To my understanding, when using LoRA with KTOTrainer, the target model is also used as the reference model for calculating the reward and approximate KL-divergence by disabling the adapter weights . I am unfamiliar with how gradients and optimizer states might differ when performing KTO compared to SFT, but intuitively I don't think they should be much different, if at all. And so, if we aren't loading a separate model as the reference model, the memory usage should not be that high.
For reference, I am performing KTO with the following parameters, and using a variant of Llama-3-8B-Instruct (called 'aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct'):
training_args = KTOConfig(
            num_train_epochs=2,
            per_device_train_batch_size=4,
            remove_unused_columns=False,
            gradient_accumulation_steps=4,
            learning_rate=5e-7,
            evaluation_strategy="steps",
            beta=0.5,
            max_length=1000,
            gradient_checkpointing=True,
            bf16 = True,
            save_total_limit=0,
            undesirable_weight=1.0,
            max_prompt_length=800,
            max_completion_length=200,
            report_to="none"
)

Tracking memory usage with torch.cuda utilities, for KTO I have:

but for SFT on the same dataset with the same batch size, etc:

Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder
[X] My own task or dataset (give details below)

Reproduction
def torch_print_gpu_memory():
print(f"Memory allocated: {torch.cuda.memory_allocated() / 1024 ** 3:.4f} GB")
print(f"Memory reserved: {torch.cuda.memory_reserved() / 1024 ** 3:.4f} GB")
print(f"Peak memory allocated: {torch.cuda.max_memory_allocated() / 10243:.4f} GB")
print(f"Peak memory reserved: {torch.cuda.max_memory_reserved() / 10243:.4f} GB")
print('Before loading base model')
torch_print_gpu_memory()
print('\n----------\n')
model = AutoModelForCausalLM.from_pretrained(
'aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct',
trust_remote_code=True,
device_map = 'auto',
torch_dtype = torch.bfloat16,
attn_implementation="flash_attention_2"
)
print('\n----------\n')
print('After loading base model')
torch_print_gpu_memory()
print('\n----------\n')
lora_config = LoraConfig(
r=128,
lora_alpha=128,
lora_dropout=0.05,
target_modules= 'all-linear',
)
training_args = KTOConfig(
num_train_epochs=2,
per_device_train_batch_size=4,
remove_unused_columns=False,
gradient_accumulation_steps=4,
learning_rate=5e-7,
evaluation_strategy="steps",
beta=0.5,
max_length=1000,
gradient_checkpointing=True,
bf16 = True,
save_total_limit=0,
undesirable_weight=1.0,
max_prompt_length=800,
max_completion_length=200,
report_to="none"
)
print('Before KTOTrainer Instantiation')
torch_print_gpu_memory()
print('\n----------\n')
trainer = KTOTrainer(
model=model,
args=training_args,
tokenizer=tokenizer,
train_dataset=dataset,
peft_config=lora_config,
)
print('\n----------\n')
print('After KTOTrainer Instantiation')
torch_print_gpu_memory()
print('\n----------\n')
print('\n----------\n')
print('Before KTO Train')
torch_print_gpu_memory()
print('\n----------\n')
trainer.train()
print('\n----------\n')
print('After KTO Train')
torch_print_gpu_memory()
print('\n----------\n')
Expected behavior
Peak memory reserved when using KTOTrainer for PEFT should be significantly closer to peak memory allocated.

Thanks for reporting, please share your system info

Thanks for reporting, please share your system info

Thanks for looking into this!
System:
Debian 11
Python 3.10
1xA100-80GB
Nvidia driver 550.90.07, CUDA 12.4
(running this on a GCP CE instance based on the c0-deeplearning-common-cu123-v20240922-debian-11-py310 image)
Env:
torch==2.4.0
transformers==4.44.0
trl==0.11.3
flash-attn==2.6.3
accelerate==1.0.1

is there any chance that we can run KTO script with 24G GPU

authors: Isaaclgz chenyang399 qgallouedec

repo: huggingface/trl


text:
Update KTO example with good dataset & chat format

This PR removes the custom preprocessing that's applied with the Anthropic dataset in favour of:

A small, high quality dataset that's already suitable to apply a chat template to
A small, but good SFT model that has a chat template

The current script runs in about ~2' on a single node. The current set of hparams show margins going up / loss going down which is sufficient for a demo.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

authors: HuggingFaceDocBuilderDev lewtun


text:
[Feature request] HugoMods module to integrate Sveltia CMS

Requested Feature
A HugoMods module to support the Feature rich Sveltia CMS, Alternative to Netlify/Decap CMS. Modern, fast, lightweight, Git-based headless CMS ... May be implemented as a global Module for headless Decap and Sveltia CMS
What is Sveltia
Alternative to Netlify/Decap CMS. Modern, fast, lightweight, Git-based headless CMS. Free & open source. UX-driven development. Made with Svelte.
Sveltia CMS is a Git-based lightweight headless CMS under active development as a modern, quick replacement for Netlify CMS and [Decap CMS](https://decapcms.org/). In some simple cases, migration is as easy as a single line of code change, although we’re still working on improving compatibility. The free, open source, UX-focused alternative to Netlify/Decap CMS
Third party integration
https://github.com/privatemaker/headless-cms
Hugo Module for headless Decap and Sveltia CMS
Best regards,
Tshitshi

Thanks for the idea, we have module for Decap CMS (larger community) already, https://github.com/hugomods/decap-cms, I'm not planing to implement an alternative.
Btw, I'd suggest sharing ideas on Discussion instead of this repo.

authors: ezzle razonyang

repo: hugomods/base

license: MIT


text:
Mutable requirements checker

Feature Request
It would be really great if you could mute the requirements checker if it doesn't produce an error.
In my current use case I have the situation that I use the verbosity option for my own purposes. However, if I now specify the -vv or -vvv flag, the checker will take the floor.
I can understand the idea behind this, but I would prefer to have a configuration option to completely mute the tool as that output could quickly lead to confusion of the end-user (at least in my case).
e.g.:
{
    "main": "Bin/some-bin",
    "output": "Build/some-bin.phar",
    "chmod": "0777",
    "git-version": "package_version",
    "silent-check": true
}

Let me know what you think about this idea.

I think in your case it would be more appropriate to completely disable the requirement checker (check-requirements: false).
A new option to tweak how the verbosity flags are handled could be done, but I suspect that would be quite difficult and very dirty and I'm not sure I would like to do that just to allow against the conventions of the Symfony Console.

authors: MCStreetguy theofidry

repo: humbug/box


text:
GraphQL - media search

What

Why

Checklist

[ ] All files pass Rubocop
[ ] Any complex logic is commented
[ ] Any new systems have thorough documentation
[ ] Any user-facing changes are behind a feature flag (or: explain why they can't be)

[ ] All the tests pass
[ ] Tests have been added to cover the new code

We do not want to handle this advanced of search on the server-side. This will be left to the client.

repo: hummingbird-me/kitsu-server


text:
(fix) Run IsolatedAsyncio in HB

Before submitting this PR, please make sure:

[x] Your code builds clean without any errors or warnings
[x] You are using approved title ("feat/", "fix/", "docs/", "refactor/")

A description of the changes proposed in the pull request:
Proposing a fix to the impossibility to use IsolatedAsyncio TestCase due to it closing the main event loop and causing all tests afterward to fail with:
RuntimeError: There is no current event loop in thread 'MainThread'.
Using asyncio.run() causes the same issue (used in Coinbase Advanced Trade tests to cleanly isolated async tests)
This proposal corrects the behavior of the previously proposed decorator (async_to_sync) and adds wrappers to 3 event loop use cases:

IsolatedAsyncioTesCase: Simply wraps the class with preservation and re-assignment of the main loop
Class-level usage of an event loop: Creates a local loop while preserving the main loop
Test-level usage of an event loop: One event loop per test

Tests performed by the developer:
Tried to provide an exhaustive test suite to verify the expected behavior
Tips for QA testing:

@fengtality I think this PR is good and will be very helpful to refactor the current tests we already have.
I don't think that we should be using the Isolated approach because as I discussed with Memento we should rewrite all the tests at once, and if we are doing that I prefer to use Pytest, that already has support for async test with the decorator @pytest.mark.asyncio.
Good job @MementoRC

Thanks @cardosofede I realize that I should add some clean-up when tasks are started within the test. Any other cases that need to be investigated?
Given the interest, I created a HIP, I am flexible on the HBOT, I just put a number close to the incentive rewards
https://snapshot.org/#/hbot-ip.eth/proposal/0x94fec9fe725d2bcecdeda86af9cdab6592a414275baeaafb7ce3f18a42d8b910

@MementoRC I think that the requested amount of HBOT's can be higher since it's going to be necessary. What do you think about 50k?

@cardosofede Sounds cool. I used to write the tests in Pytest, but after a year of HB I have forgotten, but it should not be to difficult, actually I often use pytest in Pycharm. I could investigate

authors: MementoRC cardosofede

repo: hummingbot/hummingbot


text:
bumped injective-py to 1.4.x

Before submitting this PR, please make sure:

[x] Your code builds clean without any errors or warnings
[x] You are using approved title ("feat/", "fix/", "docs/", "refactor/")

A description of the changes proposed in the pull request:
Updating injective-py to accomodate https://github.com/hummingbot/hummingbot/pull/6905
Tests performed by the developer:

My trading bot runs injective_v2 with 1.4.1 and making/updating/canceling orders still works. No warnings about deprecated functions appear in the logs. There seems to be no updates needed on this front.

Tips for QA testing:

This version bump should not be done without taking care of removing the deprecated functions

This version bump should not be done without taking care of removing the deprecated functions

Hey there,
I couldn't find any in the logs. You think think there still are?

Closing cause this did a better job: https://github.com/hummingbot/hummingbot/pull/6908
@aarmoa can I ask you how you knew where the deprecated functions were? I didn't find them in the logs when I ran a test strategy. I was hoping you could point me in the right direction for next time

authors: aarmoa peterwilli


text:
Fix: add install requires for pip installation

Include the requirements packages in the setup file in order to be installed in the notifier pip installation.
These requirements must be updated any time a new requirement package be added to the project.

@fortizc, you were right. Read the install_requires from a file outside the setup file is a much better way to solve this!

Now, we use the latest minor and patch release for the current major version of celery...., but are you totally sure that a different minor version couldn't break our code?

Now, we use the latest minor and patch release...
Is mayor.minor.patch
but are you totally sure that a different minor version couldn't break our code?
Yes, see this Python versioning system is compatible with semantic versioning

We're right then!

authors: fortizc pahumadad


text:
Accessibility on FormControl component

The control component seems to add an input.
Accessibility-wise this input causes two issues:

When you tab, it is focused. Should add `tabindex='-1'
When you hover the area, the the pointer events are still shown as pointerEvents is set instead of pointer-events. This seems to be a general problem with the hiddenInputStyle string. Either, should add some "camelToSnake" function or rename the object key from  pointerEvents => "pointer-events" in .../internal/style.ts

<input name="name" style="position:absolute;width:25px;height:25px;opacity:0;margin:0px;pointerEvents:none;transform:translateX(-100%);">

Files

src/lib/components/form-control.svelte
src/lib/internal/style.ts

The input definitely should not be receiving focus or pointer events, you're right about that one. It should be hidden completely from the user as the other component being used as the control should receive focus.

It seems to me that a simple fix would be to just add type="hidden" to the generated input.

authors: arcuo huntabyte wesharper

repo: huntabyte/formsnap


text:
🛑 recipes is down

In 10fffe9, recipes (https://recipes.thekor.eu/) was down:

HTTP code: 502
Response time: 3438 ms

Resolved: recipes is back up in 2f9f9ee after 50 minutes.

repo: hupratt/upptime


text:
🛑 mealie is down

In 3d37123, mealie (https://mealie.thekor.eu/) was down:

HTTP code: 502
Response time: 3441 ms

Resolved: mealie is back up in 2c4ed74 after 9 minutes.


text:
The Space Traveler's Hub

You can find the features added for this Pull Request below. 🧩🧩

React app created, React-Redux, Redux Logger, and React Router installed.
Used the image given for the logo.
Created routes and view components for Rockets, Mission, and Profile sections.
Created directories for all Redux state slice files; rockets, missions.
Upon first render data fetched from the SpaceX API endpoints, filtered and dispatch an action to Redux store with the given key names.
Used useSelector() Redux Hook o select the state slices and render lists of rockets and missions in the corresponding routes.
Render a list of rockets and missions in the corresponding route. For the image of a rocket using the first image in the array of flickr_images. For the missions page, the table is rendered with the mission's data.
Styled the rockets and missions page with basic CSS according to the design provided.
Reservation button was added for missions and rockets routes.
Actions are added for joining and canceling the reservation for both routes.
Reserved badges were added for reserved missions and rockets.
My Missions and My Rockets are now rendering active reservations.
Added snapshot and unit test for the components and reducers.

Project by @Kwentsir @huseyinbiyik
Thank you for your review in advance! 🤓

Approved ✔
Hi there
Congratulations on completing this project! 👏🏼 🎉
You have correctly implemented all of the requirements📃 correctly and without any errors⚠ or bugs🪲.
Cheers and good luck in your next milestones! 🍀
Feel free to leave any questions or comments in the PR thread(don't forget to tag me 😄) if something is not 100% clear.

As described in the Code reviews limits policy  you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.

Thank you for the review! I will merge it now 🎉

authors: huseyinbiyik omar-labana

repo: huseyinbiyik/space-travelers-hub


text:
Eval results with beam search

When i run eval.py,i got the results as bellow. If the above is the original sentence, the following is the sentence after beam search？If so, which one is used to calculate score？
DataLoader loading json file:  data/cocotalk.json
vocab size is  9487
DataLoader loading h5 file:  data/cocobu_fc data/cocobu_att data/cocobu_box data/cocotalk_label.h5
max sequence length in data is 16
read 123287 image features
assigned 113287 images to split train
assigned 5000 images to split val
assigned 5000 images to split test
a man riding a motorcycle down a dirt road
a man riding a motorcycle on a dirt road
a woman sitting at a table with a bowl of food
a woman sitting at a table with a plate of food
a man riding a bike next to a train
a man riding a bike in front of a train
a kitchen with a sink and a window
a kitchen with a sink and a window in it

Both sentences come from beam search. The above has a higher propability, and is used to calculate score.

Thanks for your reply!
But  i got the cider score 128.8 with this code, which was lower than 129.8(in your paper) . So I want to  know dose the configuration in this code can meetsthe performance of the paper? Or tell me which configuration needs to be changed. Thanks!!

Thanks for your reply!
But i got the cider score 128.8 with this code, which was lower than 129.8(in your paper) . So I want to know dose the configuration in this code can meetsthe performance of the paper? Or tell me which configuration needs to be changed. Thanks!!

Hi, have you change the training parameters that the author provided in the train.sh? I got the CIDEr score just 127 with the origin code.

Thanks for your reply!
But i got the cider score 128.8 with this code, which was lower than 129.8(in your paper) . So I want to know dose the configuration in this code can meetsthe performance of the paper? Or tell me which configuration needs to be changed. Thanks!!

RuntimeError: probability tensor contains either inf, nan or element < 0  。 Did you encounter this kind of error during training?

authors: YJsing husthuaan hwbhwbgao n9705

repo: husthuaan/AoANet


text:
Huge AP drop when I change image size

I use default settings to train on my custom data and everything is OK.
For fair comparison, I try to replace the augments as commonly used in other projects and re-train the model. But this leads to ~10 AP drop and it's quite unreasonable. I also tried to remove the CROP augment but it didn't work. So I wonder whether I have omitted something.
The main changes in my configs are as follow.
INPUT:   MAX_SIZE_TEST: 1333   MAX_SIZE_TRAIN: 1333   MIN_SIZE_TEST: 800   MIN_SIZE_TRAIN: (640, 672, 704, 736, 768, 800)   MIN_SIZE_TRAIN_SAMPLING: choice
Do you have some idea about this issue?

Hi @Jtao0818, have you solved this problem? Sorry for the late reply : (

Hi @Jtao0818, did you find out why? I have the same issue on my custom data as well. Thanks!

authors: Jtao0818 chiayuhsu wondervictor

repo: hustvl/SparseInst


text:
Does Client.users not update on its own with respect to User.game?

I wrote some sample code to print out the games of all the users in my channel using the bot's users cache, but the games don't update when they change (although the cache seems to update when users go online/offline).  Is this intended?  I know that the presence event exists, but I didn't know if it was as easy as calling bot.users.update(presenceVar1, presenceVar2) and if that was even necessary.
Thanks!  Keep up the great work. :)

what version of discord.js?

package.json says 8.0.0.

Latest is 8.1.0, try that

Upgrading to 8.1.0 did fix the issue!  Apologies, I should have tried that first.  Thank you for your help!

authors: JaronOrchard abalabahaha briantanner


text:
Profile page followup improvements

[x] We need to have the list of resources sorted by time (coming from the back end) to populate the recent activity list.
[ ] Get a list of recent comments made by the profile user as well to add to the recent activity list.

Feel free to add any suggestions.

@Lizabrazil @Maurier do we still want the recent comments? Alva/Manish are working on Recent Activity now...

Sure, but I think we should likely be doing less on the profile page and more on the dashboard page. Hence - I think we should close this issue and open any appropriate issues related to recent comments on the dashboard. I'm not sure we need to display recent comments on the dashboard, but we probably should be notifying resource owners about comments that have been posted and then notifying comment posters when another comment/response has been posted.

Agreed, let's close this and reassess the ability to add notifications  potentially on the dashboard. Just from my intuition, I would think a user would only want to know when others comment on their resources, and not as much about the comments that they made to other resources.

authors: Lizabrazil Maurier cuahsimarko horsburgh

repo: hydroshare/hydroshare


text:
2502 fix user file

Pull Request Checklist:

[x] Positive Test Case Written by Dev

[ ] Automated Testing

[ ] Sufficient User and Developer Documentation

[ ] Passing Jenkins Build

[ ] Peer Code review and approval

Positive Test Case

use script
./hsctl managepy list_resources
and its many arguments to locate existing resources with various attributes as follows:

--storage=local, --storage=user, --storage=federated: storage model
--access=public, --access=discoverable, --access=private: kind of access.
--type=CompositeResource (etc)
--has_subfolders: identifies resources with subfolders.

For a local resource and a user resource: Try to zip and download a bag.

For a local resource and a user resource with subfolders: Try to download a subfolder.

For a local resource and a user resource: Try to zip a file.

For a composite resource: try to download a file with single-file-aggregate metadata. (should get a zip).

For a composite resource: try to download a file without single-file-aggregate metadata and force a zip with the .zip suffix.

Overview
Correct user file download in 2502-composite-resource-phase-2. This PR has a lot of components and fixes many, many shortcomings of the original solution:

All URLs used for downloads are public URLs without iRODS information.
User and federated resources are handled properly by adding the federation path when processing.
There are three kinds of public requests:

a bag download request. format /bags/{resource_id}.zip
a zipfile request (for a previously created zipfile), format /zips/{date}/{hash}/{resource id}/{path-to-zip}
a path download request (that is automatically zipped if it is a folder, and whose response contains the zipfile path), format /resource/{resource id}/{path-to-target}

special features:

folders are always automatically zipped when used as targets
adding .zip to the end of a file download request zips the file unconditionally. Otherwise the file is returned zipped if it has single-file-aggregate metadata, and unzipped if it doesn't.

other changes:

django_irods is a directory, not a submodule.

There are a number of known issues:

Large file handling does not change the banner about not being able to download large files.
Many composite resource metadata files are missing from existing resources, even after the script to remove generic metadata. I am logging their absence as errors.
Unit tests have mistakes in them; they look for the wrong paths.

As a summary of what this does:

action
old
new

url of a zipped bag
/django_irods/download/bags/{resource id}.zip
/bags/{resource-id}.zip

url of a zipped folder
/django_irods/download/zips/{date}/{resource-id}/{hash}/{path-to-target}
/django_irods/download/zips/{date}/{hash}/{resource-id}/{path-to-target}.zip

zipping a folder
/django_irods/downloads/zips/{resource id}/{path-to-target}.zip
regular path: /resource/{resource-id}/{path-to-target}

zipping a file
no analogue
/resource/{resource id}/{path-to-target}.zip

I backed off from making  /zips/ and /bags/ public endpoints, because there is too much to change in the REST API to make them work properly. So far, changes don't affect the REST API much. We can add convenience endpoints later if desired.

@aphelionz @hyi @sblack-usu @pkdash
Status update:

All tests modified and work.
Cannot conceive of the problem in metadata not being created for Composite Resources. Logical file exists, no corresponding metadata files. Creation code gives the appearance of working. Giving up for now.

@alvacouch

Cannot conceive of the problem in metadata not being created for Composite Resources. Logical file exists, no corresponding metadata files. Creation code gives the appearance of working. Giving up for now.

Is this problem with existing composite resources (prior to phase-2) or new composite resource? How can I reproduce this problem?

I'll take a look at this in the morning.  Took me longer than I thought it would to upgrade to solr6...

@pkdash @sblack-usu The problem with missing metadata is for existing Composite Resources after the upgrade script. Lots of files are supposed to correspond to aggregate metadata and don't. These are reflected in the log when I try to zip the supposed metadata.
Meanwhile, your script regenerate_xml_files_composite_resource crashes with the error
> GENERATING BAG FILES FOR COMPOSITE RESOURCE:582060f00f6b443bb26e896426d9f62a
Traceback (most recent call last):
 File "manage.py", line 10, in <module>
   execute_from_command_line(sys.argv)  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
   utility.execute()  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
   self.fetch_command(subcommand).run_from_argv(self.argv)  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
   self.execute(*args, **cmd_options)
 File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
   output = self.handle(*args, **options)  File "/hydroshare/hs_core/management/commands/regenerate_xml_files_composite_resource.py", line 69, in handle
   generate_bag_files(r)  File "/hydroshare/hs_core/management/commands/regenerate_xml_files_composite_resource.py", line 46, in generate_bag_files
   create_bag_files(resource)
 File "/hydroshare/hs_core/hydroshare/hs_bagit.py", line 205, in create_bag_files
   resource.create_aggregation_xml_documents()
 File "/hydroshare/hs_composite_resource/models.py", line 159, in create_aggregation_xml_documents
   aggregation.create_aggregation_xml_documents()
 File "/hydroshare/hs_file_types/models/raster.py", line 434, in create_aggregation_xml_documents
   super(GeoRasterLogicalFile, self).create_aggregation_xml_documents(create_map_xml)
 File "/hydroshare/hs_file_types/models/base.py", line 1223, in create_aggregation_xml_documents
   raise exUnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 668: ordinal not in range(128)

and the log message:
...
[09/Aug/2018 23:57:47] WARNING [rdflib.term:208] http://www.hydroshare.org/resource/ae791e802ff7401887ad1e552be3bb0a/data/contents/Progress report.pdf does not look like a valid URI, trying to serialize this will break.
[09/Aug/2018 23:57:48] WARNING [rdflib.term:208] http://www.hydroshare.org/resource/ab6c729f441249f9b10824f541cbdb61/data/contents/modified report/modified report.pdf does not look like a valid URI, trying to serialize this will break.
[09/Aug/2018 23:57:48] WARNING [rdflib.term:208] http://www.hydroshare.org/resource/ab6c729f441249f9b10824f541cbdb61/data/contents/Progress report.pdf does not look like a valid URI, trying to serialize this will break.
[09/Aug/2018 23:58:02] ERROR [root:1222] Failed to create aggregation metadata xml file. Error:

SO, there are at least two problems with metadata generation:

Crash while creating metadata.
Failure to convert ' ' to '+' when creating URLS for filenames containing spaces.

@alvacouch

Crash while creating metadata.
Failure to convert ' ' to '+' when creating URLS for filenames containing spaces.

I will try to get a fix for these two above issues tonight. I need help in testing though. Would you be able to run the management command (the command that raised the above 2 issues) using the branch that I create for this fix?

@alvacouch I just remembered that I had a fix for the encoding in this PR: https://github.com/hydroshare/hydroshare/pull/2840 which never got merged.

My testing on https://dev-hs-6.cuahsi.org

Backed up existing config files into ~/backup
Did a fresh git clone of hydroshare on dev-hs-6 into /home/hydro/hydroshare
Checked out 2502-fix-user-file
See code below, now following positive use case instructions

Try to zip and download a bag.

[ ] Local Resource
[ ] User Resource

Try to download a subfolder.

[ ] Local Resoruce
[ ] User Resource

Try to zip a file.

[ ] Local Resoruce
[ ] User Resource

For a composite resource: try to download a file with single-file-aggregate metadata. (should get a zip).

[ ] Local Resoruce
[ ] User Resource

For a composite resource: try to download a file without single-file-aggregate metadata and force a zip with the .zip suffix.

[ ] Local Resoruce
[ ] User Resource

Notes below:
Step 1, using ./hsctl managepy list_resources script with arguments
Step 2:
$ ./hsctl managepy list_resources --storage=local
46383233 0359cab7d93f403ba6ee8726cff74f8a public local GenericResource GIS in Water Resources Term Project 2015
153071782 040dfdc5e0684a40bd13c0259e971bb7 public local GenericResource NFIE Colorado Region
3767595 0559afaa73aa4504b3bbc76a690948f0 private local GenericResource water data
21866 05a29e4ddffd42b88f482bd5be46e88d public local GenericResource Yellowstone Ground Snow Cover Validation
155205454 07a719c26ebb4aea89c0923e56c0df5f private local GenericResource Coweeta WS 14, created by EcohydroLib
65382856 0b7dcbc238c947239a6f949abec869db public local GenericResource NFIE Mid-Atlantic Region
9330159 0cb834d7635943e98d770e9df0c522fe public local GenericResource FRIS Geodatabase Table for Tar River Basin
29241961 0e62b639e17c40259fc969437f33f59b public local NetcdfResource cliamte_model_sample
1738704043 126bb28f05224636a8bc8b3d1bdad6b5 public local GenericResource NFIE-Geo Mississippi Region
190071964 137c5c8f43c847feb213c405bfcd19ca private local GenericResource UEB input package for the Dolores watershed
7904847 15961229b690461f9f60efdf09b280ab private local GenericResource TOPNET input package for the Logan River watershed
75108 17c732b4ec924f54ba73aa77497d8354 public local GenericResource Colorado River @ Lee's Ferry (Discharge, Water Temp, Precip) from 7/31/15 to 8/15/15
6639546 19f4c8aebfcd4c6c86766a6d7f682374 private local GenericResource UEB input package for the LittleBear watershed at Hyrum Reservoir
...

``
./hsctl managepy list_resources --storage=user
[no results]

$ ./hsctl managepy list_resources --storage=local --has_subfolders
10310744 b26090299ec947c692d4ee4651815579 public local GenericResource Great Salt Lake Level and Volume
118889880 0e49df4b97f94247a8d52bac4adeb14a public local GenericResource Thunder Creek Landlab Landslide Example
3427648 250a21e6752b46a48167311b46fa7c71 public local GenericResource Utah Water Voices
10849740 cf8d84ef37964fa3a10f69ce4b9f9586 public local GenericResource Discharge Rating Curve at Red Butte Creek near Foothill Drive Advanced Aquatic Site (RB_FD_AA)
10224083 c0a5a958e95e44b5846a7d8950be7bbd public local GenericResource Discharge Rating Curve at Red Butte Creek near Cottams Grove Basic Aquatic Site (RB_CG_BA)
10452623 0b6d8dad809e4638b039b01adf4b7b74 private local CompositeResource Compo Res-1
87996 ee9ab1df48b14e66ae9ee4c265a96fb5 discoverable local GenericResource Using ODM2API and Bokeh to create result value annotations. Bokeh allows a user to interactively select values from a graph.
39432633 b0dd5943944a4ae8949b881cd0264bf5 public local GenericResource HydroShare Pictures
1932590232 e77d01d7d1084be396bf783bd8d3c7e4 private local GenericResource Testing Sauk - Jim
429903220 d7b7f9d5c7e0411683b76ef45c6bf0d8 public local CompositeResource Logan 10 m Terrain Analysis
2870847 8bb80a0aece94f81b52dcb719ada5a8f public local ModelProgramResource SWAT Calibration Helper v1.0
20 b1b29c30eb7d45bd85b527b4248f418e private local GenericResource Functional Test
20 fe6edc72a982454b8a86aacd7cfbaf74 private local GenericResource Test

``
./hsctl managepy list_resources --storage=user --has_subfolders
[no results]

$ ./hsctl managepy list_resources --storage=local --type=CompositeResource
10452623 0b6d8dad809e4638b039b01adf4b7b74 private local CompositeResource Compo Res-1
4551930 8bcb414386c34d74aeb102dd32e1335c private local CompositeResource Untitled resource
4551930 fbee3a71e353420488573217a7183e62 private local CompositeResource Untitled resource
16117455 b541b6734ada45ce86a1340d7af995af private local CompositeResource MODIS Test 4
429903220 d7b7f9d5c7e0411683b76ef45c6bf0d8 public local CompositeResource Logan 10 m Terrain Analysis
26696322 f0f09e43d0e54e3199cc23d05a59295d public local CompositeResource S. Fk. Flathead River data
70070 15b7d7ceb7514893b2fc758573cc8b11 public local CompositeResource Teton River, MT
86162 3d86775c5e894cd486efff8aff9a401b public local CompositeResource North Fork Sun River, MT
46712 c16d726d99de4886912af8473c881bd1 public local CompositeResource Mission Creek, MT
6556482 6d995a7597ec4dbebc37c118b0a1bade public local CompositeResource Demo Composite Resource
56391920 2e921a60f5914870bb16ca952d90d452 private local CompositeResource MOPEX 2
1555 9cc257a981404131982a58fd91f91d4e private local CompositeResource Time series layer resource created on Mon Apr 10 2017 09:37:59 GMT-0600 (MDT)
5369856 5c129ecc8c59460d8f3668ba1455ae0a private local CompositeResource Time series layer resource created on Mon Apr 10 2017 09:37:59 GMT-0600 (MDT)
1555 7e577559494f4bf88ea07769eeb6a280 private local CompositeResource Time series layer resource created on Mon Apr 10 2017 09:37:59 GMT-0600 (MDT)
3823811880 e8434e287ada4540be814fa7275fb749 public local CompositeResource Supporting files for python tool subset_nwm_netcdf 1.1.3
1555 ba71686c6c3f4daf9709c75e698a8b57 private local CompositeResource Time series layer resource created on Mon Apr 10 2017 09:37:59 GMT-0600 (MDT)

with / without single file aggregate metadata

@sblack-usu I added the fixes from Pabitra and this does not fix the xml metadata file generation.
I get the following crash:
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/hydroshare/hs_core/management/commands/regenerate_xml_files_composite_resource.py", line 69, in handle
    generate_bag_files(r)
  File "/hydroshare/hs_core/management/commands/regenerate_xml_files_composite_resource.py", line 46, in generate_bag_files
    create_bag_files(resource)
  File "/hydroshare/hs_core/hydroshare/hs_bagit.py", line 205, in create_bag_files
    resource.create_aggregation_xml_documents()
  File "/hydroshare/hs_composite_resource/models.py", line 159, in create_aggregation_xml_documents
    aggregation.create_aggregation_xml_documents()
  File "/hydroshare/hs_file_types/models/raster.py", line 435, in create_aggregation_xml_documents
    super(GeoRasterLogicalFile, self).create_aggregation_xml_documents(create_map_xml)
  File "/hydroshare/hs_file_types/models/base.py", line 1224, in create_aggregation_xml_documents
    raise ex
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 668: ordinal not in range(128)

@sblack-usu The plot thickens. In the file metadata somewhere in this particular resource 582060f00f6b443bb26e896426d9f62a, there is a Unicode right quote (\u2019) that gets cast to ASCII by the XML conversion.  My hint about this is that it's used as a foot abbreviation in the phrase  … 4200\u2019 are ....

@sblack-usu The offending quotes are in the method field of the GeoRasterMetaData object.

I pushed the fix to this branch.  I struggled with reproducing this in a unit test so there isn't an extra test around the change.  I thought it best to get this through, unless someone else wants to take a crack at the test.

@sblack-usu @martinseul @aphelionz @pkdash
I have completed my changes due to reviews:

debugging logging all encapsulated in if __debug__ contexts.
Repair turned off for Composite Resources until next release. Existing strategy would corrupt them.
Addressed line-by-line comments individually.

At present, the only major shortcoming for this release is that repair_resource doesn't yet work for composite resources. This can be put in as a hotfix later.

@sblack-usu We are not out of the woods yet:
GENERATING BAG FILES FOR COMPOSITE RESOURCE:99d5f690908b4c18bd362039522bff3f
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/hydroshare/hs_core/management/commands/regenerate_xml_files_composite_resource.py", line 69, in handle
    generate_bag_files(r)
  File "/hydroshare/hs_core/management/commands/regenerate_xml_files_composite_resource.py", line 46, in generate_bag_files
    create_bag_files(resource)
  File "/hydroshare/hs_core/hydroshare/hs_bagit.py", line 94, in create_bag_files
    istorage.saveFile(from_file_name, to_file_name, True)
  File "/hydroshare/django_irods/storage.py", line 220, in saveFile
    self.session.run("imkdir", None, '-p', splitstrs[0])
  File "/hydroshare/django_irods/icommands.py", line 182, in run
    raise SessionException(proc.returncode, stdout, stderr)
django_irods.icommands.SessionException: (SessionException(...), 'Error processing IRODS request: 3. stderr follows:\n\nERROR: mkdirUtil: mkColl of /nfiehydroZone/home/localHydroProxy/99d5f690908b4c18bd362039522bff3f/data error. status = -130000 SYS_INVALID_INPUT_PARAM\n')

This is most likely a space in a filename that is not adequately escaped with \.

@alvacouch Noticed this specific resource is stored in nfiehydroZone which is only federated with www hydroshare iRODS production server, so the dev irods proxy user will not be able to imkdir into that zone path. This was a test resource created by @rayi113 and you can skip it in your testing on a dev VM.

@hyi How can I identify whether a zone is federated with a deployment or not? Is there an iCOMMAND I can use?

@alvacouch It's in the federation field in /etc/irods/server_config.json.

@alvacouch Whether a selected iRODS path is federated with HydroShare is checked with this routine: https://github.com/hydroshare/hydroshare/blob/develop/hs_core/hydroshare/utils.py#L156-L186
As you can see, it checks whether HydroShare iRODS proxy user can list the iRODS path or not, if yes, it is federated, otherwise, it is not.

@alvacouch - are you satisfied with this pull request?  If so we can get it merged into the main composite resource branch and I'll redeploy.

PR addressed https://github.com/hydroshare/hydroshare/issues/1744

authors: alvacouch aphelionz devincowan hyi pkdash rayi113 sblack-usu


text:
🛑 Daisy - Minecraft is down

In 861f222, Daisy - Minecraft (https://mc.hydev.org/dynmap) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Daisy - Minecraft is back up in 856f571.

repo: hykilpikonna/Uptime


text:
Standard Library Reorganization and Serialization

This WIP is part of pre-serializing the standard library ASTs

The "core" library is really the standard library and the parts that are dependent on OS-support are just "extensions".

That's one valid way of looking at things I suppose, but it's not “the truth:” the fact that something is "just extensions" doesn't make it any less of a library.  That's mostly just about scoping, and I'm sure there will one day be entire types, traits, and public functions for which a fully-featured host OS is simply necessary.
You write:

Having two separate entities is an abstraction leak because

and then go on to give reasons I can't relate to my understanding of the term “abstraction leak”. Care to explain the connection?
The example you give of a problem with the separation doesn't seem very compelling:

it is unclear how to implement the precondition that makes indexing safe

If by that you mean you don't know how to stop the program, I can think of a bunch of easy answers:

We say the language is not supported on platforms without whose LLVM trap instruction doesn't stop the program
We require the user to supply a second entry point (other than main) that stops the program and we call that
We run an infinite loop (deadlocks are technically safe)
no. 2 with a default of no. 3

I prefer no. 1; it's simple.

and then go on to give reasons I can't relate to my understanding of the term “abstraction leak”. Care to explain the connection?

It is the fact that the unhosted layer will have to expose parts that I would have expected to be fully encapsulated. For example, the Array.reserve_capacity requires privileged access to the internals of Array. So we can't really define Array in a "core" library and then write an extension that implements reserve_capacity without exposing those internals.
There are other similar examples. The worst case I encountered so far String because the hosted layer has to bring essentially extend the union type representing the string's storage. Happy to delve into the details if necessary.

If by that you mean you don't know how to stop the program, I can think of a bunch of easy answers:

Those answers are great. Last time we chat about it we decided we would do n°2.
The issue is kind of introducing a whole new feature for something that can be solved with conditional compilation, I think. Or we can go with your n°1, which I don't hate.

authors: dabrahams kyouko-taiga

repo: hylo-lang/hylo


text:
Order autocomplete results

Order by occurrence (more common - higher in the list)

where do you see this feature to be added (i.e. src/components/Tune/Dialog/TextField.tsx)?

This is in tune's details section when uploading or editing

People not familiar with code base (beginners) would greatly benefit from knowing the file name :)

Suspect file hypertuner-cloud/src/pages/Upload.tsx
code block:

          <Item name='engineMake' rules={requiredTextRules}>
            <AutoComplete
              options={autocompleteOptions.engineMake}
              onSearch={(search) => searchAutocomplete('engineMake', search)}
              backfill

Correct?

To be specific: https://github.com/hyper-tuner/hypertuner-cloud/blob/master/src/pages/Upload.tsx#L153

ok, so it seems first hurdle to understanding requirement(s) was taken.
Paraphrased in my words:
Envisaged is a global change affecting all text input fields based on count of existing values in hypertuner dataset, without considering user specific count of previously used values.
Correct?

Yes, at least this is what I was thinking when creating this issue
for example: more popular "make" -> higher in the autocomplete list in "make" field

ChatGPTs anwser:
const options = (await autocomplete(attribute, search)).map((record) => record[attribute]);

// Count the occurrences of each value
const valueCounts = {};
options.forEach((value) => {
  valueCounts[value] = (valueCounts[value] || 0) + 1;
});

// Sort values by occurrence frequency (more common - higher in the list)
const unique = [...new Set(options)].map((value) => ({ value })).sort((a, b) => valueCounts[b.value] - valueCounts[a.value]);

ok, so does this ths code work as expected?
On a somewhat related topic, is it desired to have the same engine multiple times in the dataset under the  different car maker brands using it in different model?
Best example is the famous DieselGate EA 189 engine.
This engine is found in cars from VW, AUDI, Porsche  and the count of multi-brand engines is growing strong after 2003.

I haven't checked the code
Regarding other topic, it's and idea, but I'm not sure whether it's worth the time at this stage. However it could be nice to pull data from a good source

It appears to be already handled internally by the <AutoComplete /> component. Thanks @villager88 !

repo: hyper-tuner/hypertuner-cloud


text:
Pass emit to events and make mixins either a function or object.

Recently we've talked about making mixins a function (by convention, not strictly), so instead of:
mixins: [logger]

...prefer:
mixins: [logger()]

This got me thinking about how to simplify mixins.
To me mixins are a no-brainer, but to a newcomer, it might be frustrating how mixins that take options must be a function that returns a function. 🤔
What if we go back to passing emit to events? This would allow us to make mixins either a function that takes some options or just an object.
Instead of passing emit to events, we can try a midway approach, allowing mixins to behave either as a function that returns a function or a function that returns an object.
For example, @okwolf's logger could be changed to the following (since emit is not necessary):
function logger(options) {
  // Initialize...
  return {
    events: {
      action: function(state, actions, action) {
        // ...
      },
      resolve: function(state, actions, result) {
        // ...
      },
      update: function(state, actions, nextState) {
        // ...
      }
    }
  }
}

And if you want emit, then use it as usual (no API break).
Pros

Allow mixins to be simple objects.
Make mixins that don't use emit simpler to write.
No API break.

Cons

Mixins can take more shapes now. All of the following are valid mixins:

// Basic free plan
const a = {
  events: {
    load() {
      console.log("ok")
    }
  }
}

// Basic free plan with options

function a(options) {
  return {
    events: {
      load() {
        console.log("ok")
      }
    }
  }
}

// Premium plan with emit

function a(emit) {
  return {
    events: {
      load() {
        console.log("ok")
      }
    }
  }
}

// Premium plan with emit & options
function a(options) {
  return function(emit) {
    return {
      events: {
        load() {
          console.log("ok")
        }
      }
    }
  }
}

@jbucaran is this how each of the different "plans" would be used or am I missing something?
// Basic free plan
mixins: [a]

// Basic free plan with options
mixins: [a(options)]

// Premium plan with emit
mixins: [a]

// Premium plan with emit & options
mixins: [a(options]

@okwolf Could there be confusion/errors from a user passing in a mixin function that took options and returned a mixin object VS a mixin function that takes emit and returns a mixin object?

Not sure. But  probably yes, there could be.

to a newcomer, it might be frustrating how mixins that take options must be a function that returns a function. :thinking:

Anyone diving into an Elm-like architecture will sooner or later have to deal with higher-order functions. With proper documentation and good explanations, I don't think that this should be an issue.

https://github.com/hyperapp/hyperapp/commit/5b4a705df02595dd6b6a28bcdb205a1ab530b37e
Thank you everyone for your input.

authors: jbucaran lazarljubenovic okwolf

repo: hyperapp/hyperapp

license: mit


text:
Improve examples of modular code design

The Modular Apps section of the documentation doesn't show how one might build a multi page website using one file per page. I'm coming from a C#/Blazor perspective where individual .razor/.cshtml files correspond to an individual pages in the application. The behavior I'm thinking would be something like this:
main.py
import hyperdiv as hd
from home import home_init

router = hd.router()

@router.route("/")
def home():
     home_init()

def app():
    template = hd.template(logo="logo.png", title="title", theme_switcher=True)

    template.add_topbar_links({
        "Home": {"icon": "house", "href": "/"},
        "Settings": {"icon": "gear", "href": "/settings"}
    })

    with template.body():
        router.run()

hd.run(app)

home.py
import hyperdiv as hd

def home_init():
    page = {SOME RENDERABLE CONTAINER}
    page.add(hd.markdown("# Home Page"))
    return page

Where page is some renderable container that has a .add() function to add components to the page and can be returned to the caller to be rendered.
Alternatively something like this:
main.py
import hyperdiv as hd
from home import home_init

router = hd.router()
home_init(router)

def app():
    template = hd.template(logo="logo.png", title="title", theme_switcher=True)

    template.add_topbar_links({
        "Home": {"icon": "house", "href": "/"},
        "Settings": {"icon": "gear", "href": "/settings"}
    })

    with template.body():
        router.run()

hd.run(app)

home.py
import hyperdiv as hd

router = None

def home_init(self, global_router):
    router = global_router

@router.route("/")
def home():
    hd.checkbox("Check Me", checked=True)
    hd.alert("This is an alert", opened=True)
    hd.button("Click Me")

Where an application wide router can be passed between pages as long as the page is initialized with the router beforehand.
If some system like this already exists, including examples of that would be helpful. Otherwise, including it would be a great help from a codebase management perspective (avoiding making the entire web app in 1 file).

I have a pages/_router.py that is just router = hd.router()
And all the pages do from ._router import router and then setup the routes normally.
The main file then imports all the pages with from .pages import * and runs the router

Thanks I'll try to make more clear in the docs how to build multi-page/multi-module apps. But yes the idea is that you instantiate a router in a central module and then import that router into every page module to add routes to it.

repo: hyperdiv/hyperdiv


text:
Router::addRoute 路由的，相当于取别名的，getAttribute取不到值

Router::addRoute 路由的，相当于取别名的，在controller里getAttribute取不到值。
改成真实的路径就可以了。

贴代码

贴代码

还是没懂你啥意思，能不能把代码贴全，你那个 account_role_id 是什么？

还是没懂你啥意思，能不能把代码贴全，你那个 account_role_id 是什么？

这个是在中间件里给赋值的
        $request = $request->withAttribute("account_role_id", $role_id);

这个是在中间件里给赋值的
        $request = $request->withAttribute("account_role_id", $role_id);

中间件是怎么定义的
注解中间件只对注解路由有用，配置中间件只对配置路由有用

中间件是怎么定义的
注解中间件只对注解路由有用，配置中间件只对配置路由有用

中间件如下：

中间件如下：

中间件如何配置的，也截图出来

中间件如何配置的，也截图出来

authors: limingxinleo linjuchen

repo: hyperf/hyperf


text:
建议开发针对腾讯微服务 polarismesh 的支持

建议开发针对腾讯微服务 polarismesh 的支持

赞成

话说这项目还有人管吗  貌似作者都不回复问题了0.0......

话说这项目还有人管吗 貌似作者都不回复问题了0.0......

https://github.com/orgs/hyperf/people 这些都是 Hyperf 组织的成员，同时开源项目并不是项目作者为所有人服务，而是构建一个开源社区让大家参与进来，如果大家对 Polarismesh 的期望很高，我也很期待有人能够 PR 这个组件上来

authors: diankemao fanuser huangzhhui linjuchen


text:
Update TwigEngine.php

added twig auto_reload for auto compile on dev

Thanks for your pr, could you please move it into repository of hyperf.

authors: cmpan limingxinleo

repo: hyperf/view


text:
Support installation on hass.io

Hey everybody,
I've been running my hyperion server on a raspberry pi along with a distribution of Hassbian. Now I switched to the newer hass.io, which seems to be based on the resinos distro(is it a distro?)
Long story short I tried to install the hyperion server via HyperCon and via plain ssh, but it fails. For e.g. there doesn't seem to be apt-get available, not sure what else fails. Can grab a log if it's needed.

It’s unlikely hypercon will get support for this platform (or any others)
Can you install Hyperion manually ?

Not that compftable with linux, is there a guide how to install manually? I also stuggle to find anything on https://hyperion-project.org/ sorry to say :(

You can‘t install Hyperion under resinOS from the console... You have to create a docker container or „Addon“ as hass.io is calling it. This is the wrong forum to ask.
Why would you want that anyway? Hass.io is designed to be the hub for home automation, not Ambylight stuff...

Using the same pi for both, to safe some energy.
And previously it were just different systems on the same OS. I'm back to non resin OS, home assistant btw

I like the idea of a docker container with Hyperion.
SPI, USB and network led devices should be possIble with minimal effort.
Screen grabbing and the PWM leds probably won’t be so easy.

authors: Razzeee pattyland penfold42

repo: hyperion-project/hyperion


text:
fix #674

IMPORTANT: Please don´t commit to master, we will test your changes first at the beta branch!
1. Tell us something about your changes.
fix #674
remove last occurence of ENABLE_PROTOBUF
make it ready for cmake 3 and fix cmake warnings
2. If this changes affect the .conf file. Please provide the changed section
no
3. Reference a issue (optional)
Note: For further discussions use our forum: forum.hyperion-project.org

Nice!
Thank you, i will test this later.

authors: brindosch redPanther


text:
Default sleep_on_errors to false

The behavior in https://docs.rs/hyper/0.12.14/hyper/server/conn/struct.AddrIncoming.html#method.set_sleep_on_errors (PR https://github.com/hyperium/hyper/pull/1450) is speculative. It shouldn’t default to true, since it masks error conditions and does not appreciably reduce the risk of service unavailability, as intended (#1358).
I suggest to default this parameter to false or, better yet, remove this code.

Although the implementer notes an error log entry would be emitted, he/she does not indicate how to control and view this error log. E.g., what env. var. should be set and for which path?

Sleeping on TCP accept errors is actually common in other libraries, like nginx, nodejs, Netty...
Was it masking an actual problem in your server?

nginx: https://github.com/nginx/nginx/blob/1028d71/src/event/ngx_event_accept.c#L99-L127
Netty: https://github.com/netty/netty/commit/dcf78961724abf97fea9e768cca02ba65146b324

authors: sanmai-NL seanmonstar

repo: hyperium/hyper


text:
UpgradedSendStream transmute is super unsound

https://github.com/hyperium/hyper/blob/1d4ff3597b8e76818c8553dbfa4234cf4208c958/src/proto/h2/mod.rs#L389C8-L397
This isn't sound. Neutered<B>  is:
#[repr(transparent)]
struct Neutered<B> {
    _inner: B,
    impossible: Impossible,
}

enum Impossible {}

Firstly, it's unclear what guarantees repr(transparent) confers here, impossible does have size_of == 0 but it's not exactly a zero-sized type. The repr(transparent) RFC is not clear about this, and it's quite shaky ground to rely on, especially since empty ("impossible") types are one place Rust does sometimes perform layout optimizations.
Empty types can and do affect enum representations. SendBuf<B> has the following definition:
#[repr(usize)]
enum SendBuf<B> {
    Buf(B),
    Cursor(Cursor<Box<[u8]>>),
    None,
}

There's nothing preventing the compiler from realizing that Buf(Neutered<...>) is impossible and deciding to not include it as a discriminant choice. The asserts in this code are insufficient to check that.
This code seems to work fine under the current compiler but I wouldn't rely on that.

For historical reference, this was discussed in https://github.com/hyperium/hyper/pull/2831 and determined it wasn't unsound. Could be we got it wrong in there, I'm just including it for context.

Yeah, that discussion seems to focus on whether Neutered<B> is constructed; not about whether the transmute is safe.

Also stepping back a bit this pattern here is extremely weird and has almost no comments explaining what it does or justifying its safety. It's worth considering moving away from this model anyway, if possible.

cc @nox, you'll remember this better than I.

There's nothing preventing the compiler from realizing that Buf(Neutered<...>) is impossible and deciding to not include it as a discriminant choice. The asserts in this code are insufficient to check that.

Agreed.

There is no such thing as “not include it as a discriminant choice” as that would be an observable behaviour for repr(usize) and break things related to such enums. Servo for example include uninhabited variants in repr(u32) enums specifically to skip a few variant discriminants to make the actually inhabited variants have their discriminants coincide with other enums. See discussions on the explicit discriminants Rust RFC. I am on PTO with a smartphone with a broken screen so you’ll excuse me for the lack of links in this comment hopefully.
Ven 22 déc 2023, à 16:09, Ralf Jung a écrit :

There's nothing preventing the compiler from realizing that Buf(Neutered<...>) is impossible and deciding to not include it as a discriminant choice. The asserts in this code are insufficient to check that.

Agreed.
—
Reply to this email directly, view it on GitHub https://github.com/hyperium/hyper/issues/3500#issuecomment-1867795392, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA6BV6JLBF3EMSPAHMNBULYKWPDNAVCNFSM6AAAAABA6V6R4SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRXG44TKMZZGI.
You are receiving this because you were mentioned.Message ID: @.***>

Ah sorry I had missed that this enum is repr(usize). That indeed makes a difference.

@RalfJung does that also work with the transitivity assumption?

Not sure which assumption exactly you mean?
Basically the question is, for two repr($int) enum, if they have the same variants in the same order and all fields have the same layout, then do the two enums have the same layout? I don't see any way this could not be the case; AFAIK layout is completely laid down for repr(usize). So I think just like repr(transparent) is "transitive" through repr(C), it should also be "transitive" through repr($int). At least I'd say we should make such a guarantee; I don't know if we do.
Whether that's actually implemented properly for uninhabited types (https://github.com/rust-lang/unsafe-code-guidelines/issues/485), I don't know.

Okay, good to know, thanks!

So did that extra information mean that this is fine after all? Or do we need to adjust something?

I think it's still potentially an issue given:

Whether that's actually implemented properly for uninhabited types (https://github.com/rust-lang/unsafe-code-guidelines/issues/485), I don't know.

but the bigger issue is a non-issue it seems.

authors: Manishearth RalfJung nox seanmonstar


text:
feat(codec): revert to tokio-util traits

Motivation
See the discussion post I made a couple of weeks ago.
Pull request #208 introduced tonic::codec::{DecodeBuf, Decoder, EncodeBuf, Encoder} in preparation for experimenting with a different decoding strategy, but as it currently stands the in-tree functionality is equivalent to the original implementation using tokio_util::codec::{Decoder, Encoder} and bytes::BytesMut.

Solution
Given that a significant amount of time has passed with no experimentation having happened (as far as I can tell), this commit reverts those changes. Additionally, it also restores efficient use of the underlying buffers, an issue that was brought up later in its respective comment thread.
Benchmarks
The performance impact is negligible, but still worth noting. I ran this on my 2020 13-inch M1 MacBook Pro (16GB RAM), if that matters.
Before:
test chunk_size_100   ... bench:         566 ns/iter (+/- 8) = 1775 MB/s
test chunk_size_1005  ... bench:         361 ns/iter (+/- 13) = 2783 MB/s
test chunk_size_500   ... bench:         412 ns/iter (+/- 13) = 2439 MB/s
test message_count_1  ... bench:         344 ns/iter (+/- 9) = 1468 MB/s
test message_count_10 ... bench:       1,344 ns/iter (+/- 71) = 3757 MB/s
test message_count_20 ... bench:       2,468 ns/iter (+/- 84) = 4092 MB/s
test message_size_10k ... bench:       1,654 ns/iter (+/- 151) = 12097 MB/s
test message_size_1k  ... bench:         495 ns/iter (+/- 30) = 4060 MB/s
test message_size_5k  ... bench:         922 ns/iter (+/- 40) = 10856 MB/s

After:
test chunk_size_100   ... bench:         600 ns/iter (+/- 17) = 1675 MB/s
test chunk_size_1005  ... bench:         394 ns/iter (+/- 14) = 2550 MB/s
test chunk_size_500   ... bench:         442 ns/iter (+/- 18) = 2273 MB/s
test message_count_1  ... bench:         387 ns/iter (+/- 16) = 1304 MB/s
test message_count_10 ... bench:         993 ns/iter (+/- 53) = 5085 MB/s
test message_count_20 ... bench:       1,669 ns/iter (+/- 25) = 6051 MB/s
test message_size_10k ... bench:       1,284 ns/iter (+/- 106) = 15584 MB/s
test message_size_1k  ... bench:         469 ns/iter (+/- 18) = 4285 MB/s
test message_size_5k  ... bench:         788 ns/iter (+/- 39) = 12703 MB/s

(I ran cargo bench 2>&1 | grep bench: to produce these figures, my apologies if I am missing anything)

We will be keeping the tonic codec traits for versioning purposes. Thanks for the contribution!

authors: LucioFranco themaxdavitt

repo: hyperium/tonic


text:
Dev prerequisite update

#523

Just change the PATH. Is it possible to check versions like > rather than equals to the exact version?

I dont think we shall do this. because we dont know if the prev or upcoming versions will not break anything. We should only check for the versions on which we have tested.


text:
Operations Guides » Setting up an ordering node

The doc source: docs/source/orderer_deploy.md

assign me

assign to me

@tomxucnxa 翻译的怎么样了？超过两周提交翻译 issue 可以被重新认领。

@stone-ch 本周完成。

Done

authors: stone-ch tomxucnxa

repo: hyperledger-labs/fabric-docs-cn


text:
feat: support react-native 0.75

Fixes #307

Do you know whether this is backwards compatibel with older versions of React Native?

It should be -> https://github.com/expo/expo/pull/31072/files

Okay thanks, then this looks good! :)
Could you sign off your commits? https://github.com/hyperledger/aries-rfcs/blob/main/contributing.md#signing-off-commits-dco
Also, I'm not sure if you use AnonCreds / Indy, but we need the same changes there https://github.com/hyperledger/indy-vdr/blob/a113c6d938e228efd92325f1401c0dd6ab2e6237/wrappers/javascript/indy-vdr-react-native/indy-vdr.podspec, https://github.com/hyperledger/anoncreds-rs/blob/fe5e7631e276b5ac8c9685d7e9d1f2eefc6ee70c/wrappers/javascript/packages/anoncreds-react-native/anoncreds.podspec

Thanks for the contribution.  Agree with Timo that getting the change into the other location is important.
The DCO update is necessary prior to merging.  See DCO - Developer Certificate of Origin - https://github.com/apps/dco and the “Details” button beside the failed DCO check.

@TimoGlastra Done

authors: DorianMazur TimoGlastra swcurran

repo: hyperledger/aries-askar


text:
Re-factor Mediator support

Re-factor to make mediation a plug-in, or at least a separate repo (separate from aca-py)

Closing. aries-mediator-service exists with multiple implementations and aries-socketdock is the best option for a high-performance mediator.

authors: ianco swcurran

repo: hyperledger/aries-cloudagent-python


text:
[rest] no peer dependencies

"Starting Own Server" section in docs mentions "To get started, make sure you have set up the correct version of Aries Framework JavaScript" with the following command:
npm info "@aries-framework/rest" peerDependencies

But that command doesn't return anything. I wanted to use the extension with the framework, but I was a bit confused about what exact version of framework is compatible? It is the version 0.1.0?

Ah I think we moved away from peer dependencies as you can now also use the rest package as cli, in which case you won't install the node/core packages yourself. We should probably remove this from the README
Are you willing to open a PR for this?
But yes, the current supported version is 0.1.0 (we should probably also add that to the readme!).
@amanji has been working on 0.2.0 support, but not sure on the status of that

Sorry this took a bit of a backseat to work on updates to Bifold. Will continue working on this. @jakubkoci Do you want to coordinate on how to remove the peer dependencies?

Yes It's just an update to the readme. That would be appreciated, thanks!

authors: TimoGlastra amanji jakubkoci


text:
feat: add yml for postgres setup

Added ci setup for Postgres

Related Issue: #553
Related PR for feature: #699

I have created a new action file for Postgres. Does our CI-CD have an author linked to your email @TimoGlastra , I kept it 'timo@animo.id' for now

@TimoGlastra I have also added action for setting up a Postgres wallet plugin as well

Codecov Report

Merging #703 (5a0bb17) into main (549647d) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #703   +/-   ##
=======================================
  Coverage   86.17%   86.17%
=======================================
  Files         343      343
  Lines        7445     7445
  Branches     1153     1153
=======================================
  Hits         6416     6416
  Misses       1028     1028
  Partials        1        1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 549647d...5a0bb17. Read the comment docs.

authors: codecov-commenter sairanjitAW

repo: hyperledger/aries-framework-javascript


text:
FINALLLLYYYY COLOOOOR!

PR description
Wow, log4j is way more frustrating than I imagined. But I got it working!
Try it out with besu --logging=trace
I need a couple of things from a few people:

@CjHare According to the git blame for the teku logging, you're the pro on the log4j programmatic configuration and I'd love if you would be the final reviewer ;) If you don't feel like reviewing it feel free to assign someone else
@atoulme Can you make sure splunk still works? I removed a line from your configuration that could have broken it.
@benjamincburns Could you make sure this isn't absolutely broken for you on windows?
everyone else who wants to bike-shed on this I only ask that you bike shed by providing an alternate format string which can be easily specified using the format found here, scroll down a bit to highlight

Fixed Issue(s)
fixes #543
fixes #540

If you look in the xml, there are a number of settings set via environment variables at the top, that could be useful to you.
I would recommend that an env variable be set to disable colors if besu identifies it's not running in a tty environment.

I think we automatically get env vars for command line options so that still leaves us with the issue of how to change the color behavior at run time.

issue of how to change the color behavior at run time.
Ah, an opportunity for new JSON-RPC endpoint perhaps 😉

Ah, an opportunity for new JSON-RPC endpoint perhaps 😉

Sorry, that was a hint/suggestion to use an endpoint to trigger the change in the color printing (similar to that in Teku), rather then leveraging the log4j config file scanning/hot replace feature

Looking into how to shift code dependencies around to get around the broken unit test. I have a strong suspicion there's nothing wrong in the production code but there's something about that static variable that the tests don't like 🤔

Could you post the contents of the xml file you used? I tried using the two xmls in the teku tree but they didn't cause the double logging.

I'd have expected anything that defines an appender to have issues.  Mine was:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO" monitorInterval="5">
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <PatternLayout
                pattern="%white{%d{yyyy-MM-dd HH:mm:ss.SSS}} | %-25.25t | %-5level | %-30.30c{1} | %msg%n"/>
        </Console>
    </Appenders>
    <Loggers>
        <Logger name="teku-event-log" level="INFO"/>
        <!--        <Logger name="tech.pegasys.teku.datastructures.util.GenesisGenerator" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.services.powchain" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.datastructures.util.BeaconStateUtil" level="DEBUG"/>-->
        <!--        <Logger name="tech.pegasys.teku.pow" level="TRACE"/>-->
        <!--        <Logger name="org.ethereum.beacon.discovery" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.datastructures.util.GenesisGenerator" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.statetransition.genesis.GenesisHandler" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.statetransition.attestation" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.validator.coordinator.ValidatorApiHandler" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.validator.client" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.sync" level="INFO"/>-->
        <Logger name="tech.pegasys.teku.statetransition.genesis.GenesisHandler" level="TRACE"/>
        <Logger name="tech.pegasys.teku.pow.MinimumGenesisTimeBlockFinder" level="TRACE"/>
        <Logger name="org.quartz" level="INFO"/>
        <Logger name="io.netty" level="INFO"/>
        <Logger name="io.libp2p" level="INFO"/>
        <!--        <Logger name="tech.pegasys.teku.pow.Eth1HeadTracker" level="DEBUG"/>-->
        <!--        <Logger name="tech.pegasys.teku.networking.eth2.peers.Eth2Peer" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.networking.eth2.peers.PeerChainValidator" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.pow.Web3jEth1Provider" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.networking.p2p.connection.ConnectionManager" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.networking.p2p.libp2p.LibP2PPeer" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.networking.eth2.peers.Eth2PeerManager" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.networking.eth2.rpc.beaconchain.methods.StatusMessageHandler" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.networking.eth2.rpc.core.Eth2OutgoingRequestHandler" level="TRACE"/>-->
        <!--        <Logger name="tech.pegasys.teku.networking.eth2.rpc.core.Eth2IncomingRequestHandler" level="DEBUG"/>-->
        <!--        <Logger name="tech.pegasys.teku.networking.p2p.libp2p.rpc.RpcHandler" level="TRACE"/>-->
        <Root level="INFO">
            <AppenderRef ref="console"/>
        </Root>
    </Loggers>
</Configuration>

Yup, that config caused the double-logging, pushed up changes to address it.

@atoulme Did you get a chance to see if the splunk stuff still works?

I didn't. What do you need me to do?

If you take a look at the diff, I removed the reference to the appender that you added because it no longer exists at config parse time. It looks suspiciously like it would ruin what you did in your PR

@benjamincburns did you get a chance to see if this works on windows?

Tested on windows under both powershell and the old dos prompt. I see bash color escape codes on both, so the logs are unreadable there. Following the user-driven workarounds mentioned here does make colorization work on Windows, though. That said, this isn't set by default, so if this is merged as-is the default UX for windows will suffer some.

@ajsutton Any other changes you'd like to see?

authors: CjHare RatanRSur ajsutton atoulme benjamincburns

repo: hyperledger/besu