text:
Add type annotations for plt.models.regression

What does this PR do?

I have added type annotations to the module plt.models.regression.  Currently, there are only two files that need to be changed.
I had one doubt but that is unrelated to this PR, that is, in function validation_step(self, batch, batch_idx) the parameter batch_idx is not used anywhere in the function but it is a required argument. So, may I suggest it can be changed to optional argument with None option by default. I am not sure but any guidance would be helpful.
Thank you.
Related to #434
Before submitting

[x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[ ] Did you make sure to update the documentation with your changes? (NA)
[ ] Did you write any new necessary tests? [not needed for typos/docs] (NA)
[x] Did you verify new and existing tests pass locally with your changes?
[ ] If you made a notable change (that affects users), did you update the CHANGELOG? (NA)

PR review

[x] Is this pull request ready for review? (if not, please submit in draft mode)

Hello @jeromepatel! Thanks for opening this PR.

In the file pl_bolts/models/regression/linear_regression.py:

Line 2:32: E241 multiple spaces after ','

In the file pl_bolts/models/regression/logistic_regression.py:

Line 2:32: E241 multiple spaces after ','

Do see the Hitchhiker's guide to code style

I am not sure why CodeFactor and other test are failing but any guidance would be helpful.

@akihironitta mind have look? :]

in function validation_step(self, batch, batch_idx) the parameter batch_idx is not used anywhere in the function but it is a required argument. So, may I suggest it can be changed to optional argument with None option by default.

@jeromepatel Hi, thank you for your contribution! I wouldn't change it to batch_idx: Optional[int] = None because an int value is always passed to batch_idx anyway.

@akihironitta thank you for your reply. The reason seems valid to me. I think there is no need to change it to optional then.   Thanks for reviewing.

I pushed some commits to this branch directly, but as of 9e36df9, we'll get the following 43 errors from mypy... We might need to rethink if we really need to add type hints to the codebase.

mypy errors
$ mypy -V
mypy 0.800
$ mypy
pl_bolts/utils/semi_supervised.py:98: error: Incompatible return value type (got "Tuple[List[List[Any]], List[List[Any]]]", expected "Tuple[ndarray, ndarray]")  [return-value]
        return final_batches_x, final_batches_y
               ^
pl_bolts/utils/semi_supervised.py:139: error: Incompatible return value type (got "Tuple[List[Any], List[Any]]", expected "Tuple[ndarray, ndarray]")  [return-value]
        return X, Y
               ^
pl_bolts/models/regression/logistic_regression.py:44: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "input_dim"  [union-attr]
            self.linear = nn.Linear(in_features=self.hparams.input_dim, out_features=self.hparams.num_classes, bias=bias)
                                                ^
pl_bolts/models/regression/logistic_regression.py:44: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "num_classes"  [union-attr]
            self.linear = nn.Linear(in_features=self.hparams.input_dim, out_features=self.hparams.num_classes, bias=bias)
                                                                                ^
pl_bolts/models/regression/logistic_regression.py:46: error: Signature of "forward" incompatible with supertype "LightningModule"  [override]
        def forward(self, x: Tensor) -> Tensor:
        ^
pl_bolts/models/regression/logistic_regression.py:51: error: Signature of "training_step" incompatible with supertype "LightningModule"  [override]
        def training_step(self, batch: Tuple[Tensor, Tensor], batch_idx: int) -> Dict[str, Tensor]:
        ^
pl_bolts/models/regression/logistic_regression.py:63: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "l1_strength"  [union-attr]
            if self.hparams.l1_strength > 0:
               ^
pl_bolts/models/regression/logistic_regression.py:65: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "l1_strength"  [union-attr]
                loss += self.hparams.l1_strength * l1_reg
                        ^
pl_bolts/models/regression/logistic_regression.py:68: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "l2_strength"  [union-attr]
            if self.hparams.l2_strength > 0:
               ^
pl_bolts/models/regression/logistic_regression.py:70: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "l2_strength"  [union-attr]
                loss += self.hparams.l2_strength * l2_reg
                        ^
pl_bolts/models/regression/logistic_regression.py:76: error: Dict entry 1 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'loss': loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                  ^
pl_bolts/models/regression/logistic_regression.py:76: error: Dict entry 2 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'loss': loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                                           ^
pl_bolts/models/regression/logistic_regression.py:78: error: Signature of "validation_step" incompatible with supertype "LightningModule"  [override]
        def validation_step(self, batch: Tuple[Tensor, Tensor], batch_idx: int) -> Dict[str, Tensor]:
        ^
pl_bolts/models/regression/logistic_regression.py:85: error: Return type "Dict[str, Tensor]" of "validation_epoch_end" incompatible with return type "None" in supertype "LightningModule"
[override]
        def validation_epoch_end(self, outputs: List[Dict[str, Tensor]]) -> Dict[str, Tensor]:
        ^
pl_bolts/models/regression/logistic_regression.py:90: error: Dict entry 1 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'val_loss': val_loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                          ^
pl_bolts/models/regression/logistic_regression.py:90: error: Dict entry 2 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'val_loss': val_loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                                                   ^
pl_bolts/models/regression/logistic_regression.py:92: error: Signature of "test_step" incompatible with supertype "LightningModule"  [override]
        def test_step(self, batch: Tuple[Tensor, Tensor], batch_idx: int) -> Dict[str, Tensor]:
        ^
pl_bolts/models/regression/logistic_regression.py:93: error: Cannot resolve name "x" (possible cyclic definition)  [misc]
            x = x.view(x.size(0), -1)
                ^
pl_bolts/models/regression/logistic_regression.py:95: error: Name 'y' is not defined  [name-defined]
            acc = accuracy(y_hat, y)
                                  ^
pl_bolts/models/regression/logistic_regression.py:96: error: Name 'y' is not defined  [name-defined]
            return {'test_loss': F.cross_entropy(y_hat, y), 'acc': acc}
                                                        ^
pl_bolts/models/regression/logistic_regression.py:98: error: Return type "Dict[str, Tensor]" of "test_epoch_end" incompatible with return type "None" in supertype "LightningModule"  [override]
        def test_epoch_end(self, outputs: List[Dict[str, Tensor]]) -> Dict[str, Tensor]:
        ^
pl_bolts/models/regression/logistic_regression.py:103: error: Dict entry 1 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'test_loss': test_loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                            ^
pl_bolts/models/regression/logistic_regression.py:103: error: Dict entry 2 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'test_loss': test_loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                                                     ^
pl_bolts/models/regression/logistic_regression.py:106: error: Unexpected keyword argument "lr" for "Optimizer"  [call-arg]
            return self.optimizer(self.parameters(), lr=self.hparams.learning_rate)
                   ^
pl_bolts/models/regression/logistic_regression.py:106: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "learning_rate"  [union-attr]
            return self.optimizer(self.parameters(), lr=self.hparams.learning_rate)
                                                        ^
pl_bolts/models/regression/logistic_regression.py:145: error: Cannot instantiate abstract class 'SklearnDataModule' with abstract attributes 'prepare_data', 'setup' and 'transfer_batch_to_device'
 [abstract]
        loaders = SklearnDataModule(X, y, batch_size=args.batch_size, num_workers=0)
                  ^
pl_bolts/models/regression/linear_regression.py:43: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "input_dim"  [union-attr]
            self.linear = nn.Linear(in_features=self.hparams.input_dim, out_features=self.hparams.output_dim, bias=bias)
                                                ^
pl_bolts/models/regression/linear_regression.py:43: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "output_dim"  [union-attr]
            self.linear = nn.Linear(in_features=self.hparams.input_dim, out_features=self.hparams.output_dim, bias=bias)
                                                                                ^
pl_bolts/models/regression/linear_regression.py:45: error: Signature of "forward" incompatible with supertype "LightningModule"  [override]
        def forward(self, x: Tensor) -> Tensor:
        ^
pl_bolts/models/regression/linear_regression.py:49: error: Signature of "training_step" incompatible with supertype "LightningModule"  [override]
        def training_step(self, batch: Tuple[Tensor, Tensor], batch_idx: int) -> Dict[str, Tensor]:
        ^
pl_bolts/models/regression/linear_regression.py:60: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "l1_strength"  [union-attr]
            if self.hparams.l1_strength > 0:
               ^
pl_bolts/models/regression/linear_regression.py:62: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "l1_strength"  [union-attr]
                loss += self.hparams.l1_strength * l1_reg
                        ^
pl_bolts/models/regression/linear_regression.py:65: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "l2_strength"  [union-attr]
            if self.hparams.l2_strength > 0:
               ^
pl_bolts/models/regression/linear_regression.py:67: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "l2_strength"  [union-attr]
                loss += self.hparams.l2_strength * l2_reg
                        ^
pl_bolts/models/regression/linear_regression.py:73: error: Dict entry 1 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'loss': loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                  ^
pl_bolts/models/regression/linear_regression.py:73: error: Dict entry 2 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'loss': loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                                           ^
pl_bolts/models/regression/linear_regression.py:75: error: Signature of "validation_step" incompatible with supertype "LightningModule"  [override]
        def validation_step(self, batch: Tuple[Tensor, Tensor], batch_idx: int) -> Dict[str, Tensor]:
        ^
pl_bolts/models/regression/linear_regression.py:81: error: Return type "Dict[str, Tensor]" of "validation_epoch_end" incompatible with return type "None" in supertype "LightningModule"
[override]
        def validation_epoch_end(self, outputs: List[Dict[str, Tensor]]) -> Dict[str, Tensor]:
        ^
pl_bolts/models/regression/linear_regression.py:85: error: Dict entry 1 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'val_loss': val_loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                          ^
pl_bolts/models/regression/linear_regression.py:85: error: Dict entry 2 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'val_loss': val_loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                                                   ^
pl_bolts/models/regression/linear_regression.py:87: error: Signature of "test_step" incompatible with supertype "LightningModule"  [override]
        def test_step(self, batch: Tuple[Tensor, Tensor], batch_idx: int) -> Dict[str, Tensor]:
        ^
pl_bolts/models/regression/linear_regression.py:92: error: Return type "Dict[str, Tensor]" of "test_epoch_end" incompatible with return type "None" in supertype "LightningModule"  [override]
        def test_epoch_end(self, outputs: List[Dict[str, Tensor]]) -> Dict[str, Tensor]:
        ^
pl_bolts/models/regression/linear_regression.py:96: error: Dict entry 1 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'test_loss': test_loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                            ^
pl_bolts/models/regression/linear_regression.py:96: error: Dict entry 2 has incompatible type "str": "Dict[str, Tensor]"; expected "str": "Tensor"  [dict-item]
            return {'test_loss': test_loss, 'log': tensorboard_logs, 'progress_bar': progress_bar_metrics}
                                                                     ^
pl_bolts/models/regression/linear_regression.py:99: error: Unexpected keyword argument "lr" for "Optimizer"  [call-arg]
            return self.optimizer(self.parameters(), lr=self.hparams.learning_rate)
                   ^
pl_bolts/models/regression/linear_regression.py:99: error: Item "Dict[Any, Any]" of "Union[Dict[Any, Any], Namespace]" has no attribute "learning_rate"  [union-attr]
            return self.optimizer(self.parameters(), lr=self.hparams.learning_rate)
                                                        ^
pl_bolts/models/regression/linear_regression.py:138: error: Cannot instantiate abstract class 'SklearnDataModule' with abstract attributes 'prepare_data', 'setup' and 'transfer_batch_to_device'
[abstract]
        loaders = SklearnDataModule(X, y, batch_size=args.batch_size)
                  ^
Found 47 errors in 3 files (checked 197 source files)

@akihironitta  so, should we revisit this issue after solving the error or maybe close this PR as of now and work on more important issues? Thank you for your reply.

This change is causing tests to fail in master. Please check!
https://github.com/PyTorchLightning/lightning-bolts/pull/595/commits/3bf65da4d611445a9b09399f7bcee68510c8ca13#diff-81b21ad681f00e8e80279aaf4d5e2c3a304b5a587b79850cff8d4472147f7f0aL92

repo: PyTorchLightning/lightning-bolts


text:
Pickle issue with 🤗 tokenizer's encoding

🐛 Bug
Following error is thrown when the 🤗's transformer's tokenizer encode function inside the LightningModule. However, the encoding works perfectly when called outside the  LightningModule:
TypeError: cannot pickle 'Tokenizer' object
Code sample
import nlp
import transformers as tfs
import pytorch_lightning as pl

class Model(pl.LightningModule):
    def __init__(self, ):
        super().__init__()

    def prepare_data(self):
        dataset = nlp.load_dataset('squad', split='train[:1%]')
        tokenizer = tfs.BertTokenizerFast.from_pretrained('bert-base-cased')
        dataset = dataset.map(lambda example: tokenizer.batch_encode_plus(example['context']), batched=True)

    def forward(self):
        pass

    def training_step(self, batch, batch_idx):
        pass

    def configure_optimizers(self):
        pass

    def train_dataloader(self):
        pass

dataset = nlp.load_dataset('squad', split='train[:1%]')
tokenizer = tfs.BertTokenizerFast.from_pretrained('bert-base-cased')
dataset = dataset.map(lambda example: tokenizer.batch_encode_plus(example['context']), batched=True)

model = Model()
trainer = pl.Trainer()
trainer.fit(model)

Environment
* CUDA:
        - GPU:
                - GeForce RTX 2080 Ti
        - available:         True
        - version:           10.2
* Packages:
        - numpy:             1.18.1
        - pyTorch_debug:     False
        - pyTorch_version:   1.5.0
        - pytorch-lightning: 0.7.6
        - tensorboard:       2.2.2
        - tqdm:              4.46.0
* System:
        - OS:                Linux
        - architecture:
                - 64bit
                - ELF
        - processor:         x86_64
        - python:            3.8.3
        - version:           #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019

check again from master?

tried with master, same issue.

AFAIK fast tokenizers are written in Rust and are not pickleable. You could either tokenize offline or use the Python tokenizer

@Anjum48 Where exactly are we doing pickle over here? In the example you see, it's the same code in the main as well as prepare_data.

@tshrjn Are you using multiple workers in your DataLoader? If you are, then there is pickling going on in multiprocessing. You could try setting workers=0 and see if that works.

The error is coming from here:

The error is occurring here:
def prepare_data(self):
        ...
    =>  dataset = dataset.map(lambda example: tokenizer.batch_encode_plus(example['context']), batched=True)

No dataloader is created yet. I am also wondering what's the reason for the error.

Try tokenizers from the master. It's working fine.

I looked into this and noticed it had something to do with difference between Python 3.6's dataclasses and Python >=3.7's built in dataclasses. If you try in a Python 3.6 environment, transformers will install dataclasses for you and it works fine.

This seems to have been resolved when using a single GPU. However, when you use multi-GPU training, you again get a pickle error.
Here is some minimalistic code. You can reproduce the error using:
python main.py --gpus 1, --qa_model distilroberta-base --workers 20 --bs 5 --max_epochs 10 .

there can be problem with a mapping lambda function, some pickable does not like local functions...
@tshrjn mind share what versions of pickle lib are you using?

@Borda I believe I can close this later today, as I've been using tokenizers on multi-gpu. Workin on colab notebook example.

Ok so no colab notebook because I can't test the multi-gpu usage over there. But I used this datamodule in a model and its working fine. I was not able to make fast tokenizers work, but I don't believe that's a lightning issue, so I'm going to close this.
Not the most elegant use of nlp package w/ squad, but it's what I've got for now.
transformers=3.0.2
nlp=0.4.0
pytorch-lightning=0.9.0rc6

from tqdm import tqdm
from pathlib import Path

import torch
from pytorch_lightning import LightningDataModule
from torch.utils.data import DataLoader
from transformers import AutoTokenizer, SquadExample, SquadV1Processor, squad_convert_examples_to_features
from nlp import load_dataset

class SquadDataModule(LightningDataModule):

    def __init__(
        self,
        tokenizer,
        feature_cache_dir: str = './cached_features',
        max_seq_length: int = 384,
        doc_stride: int = 128,
        max_query_length: int = 64,
        threads: int = 8,
        train_batch_size: int = 32,
        eval_batch_size: int = 32,
        num_workers: int = 8,
        **kwargs
    ):
        super().__init__()
        self.tokenizer = tokenizer
        self.feature_cache_dir = Path(feature_cache_dir)
        self.max_seq_length = max_seq_length
        self.doc_stride = doc_stride
        self.max_query_length = max_query_length
        self.threads = threads
        self.train_batch_size = train_batch_size
        self.eval_batch_size = eval_batch_size
        self.num_workers = num_workers

    def _cache_file(self, split: str, collection_name: str):
        # TODO - add model name by grabbing it from tokenizer
        file_name = '{}_{}_{}_{}_{}'.format(
            split,
            collection_name,
            self.max_seq_length,
            self.doc_stride,
            self.max_query_length
        )
        return self.feature_cache_dir / file_name

    def prepare_data(self):
        load_dataset('squad', split='train')
        load_dataset('squad', split='validation')
        self.feature_cache_dir.mkdir(exist_ok=True)
        if not self._cache_file('train', 'dataset').exists():
            nlp_ds_train = load_dataset('squad', split='train')
            train_examples, train_features, train_dataset = self._get_data(nlp_ds_train, is_training=True)
            torch.save(train_examples, self._cache_file('train', 'examples'))
            torch.save(train_dataset, self._cache_file('train', 'dataset'))
            torch.save(train_features, self._cache_file('train', 'features'))

        if not self._cache_file('val', 'dataset').exists():
            nlp_ds_val = load_dataset('squad', split='validation')
            val_examples, val_features, val_dataset = self._get_data(nlp_ds_val, is_training=False)
            torch.save(val_examples, self._cache_file('val', 'examples'))
            torch.save(val_dataset, self._cache_file('val', 'dataset'))
            torch.save(val_features, self._cache_file('val', 'features'))

    def setup(self, stage):

        if hasattr(self, 'train_dataset'):
            return

        self.train_dataset = torch.load(self._cache_file('train', 'dataset'))
        self.train_features = torch.load(self._cache_file('train', 'features'))
        self.train_examples = torch.load(self._cache_file('train', 'examples'))

        self.val_dataset = torch.load(self._cache_file('val', 'dataset'))
        self.val_features = torch.load(self._cache_file('val', 'features'))
        self.val_examples = torch.load(self._cache_file('val', 'examples'))

    def train_dataloader(self):
        return DataLoader(self.train_dataset, batch_size=self.train_batch_size, num_workers=self.num_workers)

    def val_dataloader(self):
        return DataLoader(self.val_dataset, batch_size=self.eval_batch_size, num_workers=self.num_workers)

    def _get_data(self, nlp_dataset, is_training):

        examples = []
        for nlp_ex in tqdm(nlp_dataset):
            example = SquadExample(
                qas_id=nlp_ex['id'],
                question_text=nlp_ex['question'],
                context_text=nlp_ex['context'],
                answer_text=nlp_ex['answers']['text'][0],
                start_position_character=nlp_ex['answers']['answer_start'][0],
                title=nlp_ex['title']
            )
            examples.append(example)

        features, dataset = squad_convert_examples_to_features(
            examples,
            self.tokenizer,
            max_seq_length=self.max_seq_length,
            doc_stride=self.doc_stride,
            max_query_length=self.max_query_length,
            is_training=is_training,
            threads=self.threads,
            return_dataset='pt'
        )

        return examples, features, dataset

if __name__ == '__main__':
    tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
    dm = SquadDataModule(tokenizer)
    dm.prepare_data()
    dm.setup('fit')

source: gharchive/issue

authors: Anjum48 Borda nateraw rohitgr7 tshrjn williamFalcon

repo: PyTorchLightning/pytorch-lightning

license_type: permissive

license_source: github-api


text:
How to use multiple metric monitors in ModelCheckpoint callback?

❓ Questions and Help
What is your question?
How can I use multiple metric monitors in the ModelCheckpoint？ In another way, how can I use multiple ModelCheckpoint callbacks？It seems that the Trainer only accepts a singleModelCheckpoint in the checkpoint_callback argument.
Code
site-packages/pytorch_lightning/trainer/callback_config.py", line 46, in configure_checkpoint_callback
    checkpoint_callback.save_function = self.save_checkpoint
AttributeError: 'list' object has no attribute 'save_function'

What's your environment?

OS: Ubuntu 16.04
Packaging: pip
Version: pytorch-lightning==0.9.0rc12

We currently don't support multiple ModelCheckpoint callbacks.
For monitoring multiple metrics with the same callback, I think you have to use the Results object:
https://pytorch-lightning.readthedocs.io/en/latest/results.html#trainresult

Do you plan to support it? It would be nice to be able to do the following:
on_best_precision = pytorch_lightning.callbacks.ModelCheckpoint(
    filepath=filepath + "epoch={epoch}_precision={precision}",
    monitor="precision",
    save_last=True,
    save_top_k=3,
    mode="max",
)
on_best_recall = pytorch_lightning.callbacks.ModelCheckpoint(
    filepath=filepath + "epoch={epoch}_recall={recall}",
    monitor="recall",
    save_last=True,
    save_top_k=3,
    mode="max",
)
every_five_epochs = pytorch_lightning.callbacks.ModelCheckpoint(
    filepath=filepath + "epoch={}",
    period=5,
    save_top_k=-1,
)
trainer = pl.Trainer(
    checkpoint_callback=[on_best_precision, on_best_recall, every_five_epochs],
)

and something similar for the early_stop_callback flag.

ping @awaelchli

I think yes, one day we will manage that. However, there are soooo many edge cases we need to consider, and we haven't even figured out all of them for the case of a single checkpoint callback. We need to approach this with care. It should be done by someone who is very confident with how model checkpointing works in PL (solely my opinion of course, not saying you or anybody else can't do it, it's just generally a hard task in my opinion, because it connects to many parts of the trainer, e.g. validation, early stopping, resuming from checkpoints, formatting names for checkpoint files, topk management, saving the last, persisting state, ... the list goes on).
cc @PyTorchLightning/core-contributors

Hi, this would be a great addition to Lightning! Do you know roughly when you're planning to include this feature?

@bartmch You can pass in multiple callbacks to the Trainer, just make sure you have the filename set properly in the checkpoint callback so that the checkpoints don't collide.

Any update on this? Is this working? I am trying to log loss and accuracy using 2 ModelCheckpoints, however, with regards to the model checkpoint filenames, logging for accuracy seems not to work as it says always "0.00" for the "best" value.

did you set max for the mode with accuracy? I suppose with mode=min, 0 accuracy would be the best.

Yes, mode is 'max'. So monitoring 2 metrics using 2 ModelCheckpoints should work now?

@carmocca I agree, this feels like an important and general addition. open to suggestions in #6467

Me too, I also think that having multiple instances of model checkpoint is better than supporting multiple monitor values in the same.

We don’t. We are still limited by the fact that we save callbacks by their type, thus overwritting any with the same type

Yes it's true for loading but just saving the checkpoints with multiple instances should work today?
My suggestion is to use an identifier for each callback instance and derive a unique key when saving. This would allow us to restore from a checkpoint even if the user has changed the callback list.
In ModelCheckpoint, the unique identifier could be derived from the monitor key automatically. Same for early stopping and other callbacks that safe state.

Important is to set the filename correctly so that the files always have different names. So I recommend to include the monitor key in the filename.

Thanks - I checked again and recognized a typo in the metric key in the file name - now it's working!
However, I don't see an issue with ModelCheckpoints being overwritten currently... My setup is as follows and working for me:
# define ModelCheckpoint callbacks
model_checkpoint_cb_loss = ModelCheckpoint(
    monitor="loss/valid",
    verbose=False,
    save_last=True,
    save_top_k=3,
    save_weights_only=False,
    mode='min',
    period=1,
    dirpath=None,
    filename="{epoch}-{loss/valid:.2f}"
)
model_checkpoint_cb_acc = ModelCheckpoint(
    monitor="auroc/valid",
    verbose=False,
    save_last=True,
    save_top_k=3,
    save_weights_only=False,
    mode='max',
    period=1,
    dirpath=None,
    filename="{epoch}-{auroc/valid:.2f}"
)

Those callbacks are then passed to trainer.
For testing both metrics, I do:
# fit the model
trainer.fit(model=model, datamodule=dm)

# Test multiple "best" checkpoints
for _ckpt in range(len(trainer.checkpoint_callbacks)):
    logging.info("Testing: monitor metric: {}".format(
        trainer.checkpoint_callbacks[_ckpt].monitor
    ))
    ckpt_path = trainer.checkpoint_callbacks[_ckpt].best_model_path
    logging.info("Best checkpoint path: {}".format(ckpt_path))

    # test the model using current checkpoint
    trainer.test(ckpt_path=ckpt_path)

That's great!

However, I don't see an issue with ModelCheckpoints being overwritten currently...

Yes, the checkpoint files get written and that's all fine, and it saves all the model weights and hyperparameters. Most users will be happy with that. However, one additional thing we save into the checkpoint dictionary is the state of all callbacks that you've used, even the ModelCheckpoint itself. So these are unfortunately not saved correctly, and you will only run into problems if you try to do Trainer(resume_from_checkpoint=...). If you would do that, and run your for loop above again, you would see the same values twice, because one of them got overwritten by the other.

authors: ananthsub angshine awaelchli bartmch carmocca kapsner


text:
Automatic Optimization should be a property of the LightningModule, not the Trainer

🚀 Feature
https://pytorch-lightning.readthedocs.io/en/latest/optimizers.html#manual-optimization
To switch between manual and automatic optimization, we need to toggle this flag on the Trainer. However, all of the user code for optimization is inside the LightningModule. If we had this flag for automatic_optimization on the LightningModule instead, we'd keep all the user-facing optimization pieces in one place.
Pitch
Define a property method for automatic_optimization inside the LightningModule. this would be another model hook
By default:
@property
def automatic_optimization(self) -> bool:
    return True

Then we can set the corresponding properties on the trainer/training loop based on the LightningModule and deprecate the automatic_optimization flag on the Trainer

this makes sense to me! good catch

Please also update the docs regarding this change at: https://pytorch-lightning.readthedocs.io/en/1.0.1/new-project.html

authors: ananthsub titamar williamFalcon


text:
advanced profiler description fails for python 3.6

🐛 Bug
Python 3.6 doesn't have the pstats.SortKey.CUMULATIVE enum so the profiler description breaks.
To Reproduce
Steps to reproduce the behavior:
Use Python 3.6, pass in the AdvancedProfiler, get report at end of a training run.
profiler = AdvancedProfiler(line_count_restriction=10)
trainer = Trainer(profiler=profiler)
trainer.fit(model)

Stack trace:
    164         for action_name, pr in self.profiled_actions.items():
    165             s = io.StringIO()
--> 166             sortby = pstats.SortKey.CUMULATIVE
    167             ps = pstats.Stats(pr, stream=s).strip_dirs().sort_stats(sortby)
    168             ps.print_stats(self.line_count_restriction)

AttributeError: module 'pstats' has no attribute 'SortKey'

Code sample
from pytorch_lightning import Trainer
from pytorch_lightning.profiler import AdvancedProfiler
from argparse import Namespace
from pl_examples.basic_examples.lightning_module_template import LightningTemplateModel

# define model
hparams = {
    "batch_size": 128,
    "in_features": 784,
    "hidden_dim": 512,
    "drop_prob": 0.0,
    "out_features": 10,
    "learning_rate": 5e-3,
    "data_root": "data"
}
hparams = Namespace(**hparams)
model = LightningTemplateModel(hparams)

# overfit on small batch
profiler = AdvancedProfiler(line_count_restriction=10)
trainer = Trainer(profiler=profiler, overfit_pct=0.05, min_epochs=10)
trainer.fit(model)

Expected behavior

Environment
Collecting environment information...
PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.1
OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: version 3.12.0
Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: GPU 0: Tesla P100-PCIE-16GB
Nvidia driver version: 418.67
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
Versions of relevant libraries:
[pip3] numpy==1.17.5
[pip3] pytorch-lightning==0.6.1.dev0
[pip3] torch==1.4.0
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.3.1
[pip3] torchvision==0.5.0
[conda] Could not collect

just wondering, was it also falling while PR #782?

there wasn’t test coverage for this case, it was failing in the describe() method. worked for me on local since i was using 3.7.

authors: Borda jeremyjordan


text:
Update type hints for multiple dataloaders in .fit() and .test()

Before submitting

[ ] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[x] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure to update the docs?
[ ] Did you write any new necessary tests?
[x] If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?
Multiple dataloaders can be pased to fit and test.
The type hints should reflect that
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

/rebase

source: gharchive/pull-request

authors: Borda awaelchli


text:
Bugfix: accumulation and suggestion for learning rate finder

Before submitting

[x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure to update the docs?
[x] Did you write any new necessary tests?
[x] If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?
Fixes https://github.com/PyTorchLightning/pytorch-lightning/issues/1767, by implementing a skip_begin parameter (default 10) for the lr_finder.suggestion method
Fixes https://github.com/PyTorchLightning/pytorch-lightning/issues/1726, by only logging learning rate and loss when optimizer is called (as it should be). This also removes num_accumulation_steps since this is a duplicate of the accumulate_grad_batches trainer arg.
Added tests for both.
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

@SkafteNicki want to sneak this last minute into 0.7.6? we're releasing the rc2 in the next hour

need to get these tests to pass

need to get these tests to pass

this is fixed In master now, just rebase it...

authors: Borda SkafteNicki williamFalcon


text:
UI frontend implementation of notifications into Front-End Login and Signup

The notification service implemented in #151 was implemented in the front end, specifically the login and signup page.
Now, whenever a user enters an invalid input into a field and tries to log in or signup they will be prompted with an error notification.
For example:

Also when a user successfully logs in or signs up they will be prompted by a success notification.
For example:

Everything looks good.
is it possible to remove .idea/inspectionProfiles in root?

authors: j-eltayar sachaelkaim

repo: Pyxsys/ttg-healthcheck

license: MIT


text:
problem importing the image

please check #66 issue

please add the required policy and roles for image import
follow this link ckick here
once the above link is opend then click on prerequisites--> click required service role
please read the documentation and add the required roles and policies.

repo: QT-DevOps/AWSIssues


text:
Removed debugging console.logs

I was attempting to test a plugin of my own and I found these lines were flooding the console.
Thought I'd make a pull request to fix it, even if it's a minor "issue".

maybe change em to console.debug()?

Fixed in v.1.1.0. Plugin redownload needed after latest breaking discord update.

authors: HexCodeFFF QbDesu Refusings

repo: QbDesu/BetterDiscordAddons

license: Unlicense


text:
i have lagg in 1.13.2

please fix this : http://prntscr.com/mjeobr

Known issue I am working on this but I am very busy.  For now, a fix is to use this version https://www.spigotmc.org/resources/wilderness-tp.18431/download?version=252854. If you use FAWE or Kingdoms though the hooks won't work

Believed to have this fixded

authors: Qballl successed

repo: Qballl/WildernessTp

license: mit

license_source: bigquery


text:
Issue #434: Correct type and class names

Fixes #434

The Revision objects need to be serialized using JSON, and it has revisionID as an attribute. Changing this attribute name to revisionId will make it incompatible with existing qbeast tables.

Closing it for now as it introduces incompatibilities with existing tables.

repo: Qbeast-io/qbeast-spark


text:
外置插件编译出来为什么会有1.6MB这么大

我就是简单的一个Activity，可编译出来的插件都有1.6M。这个不知道能不能精简一下，或者将一些RePlugin的组件提前放在Host中呢。
毕竟插件更新的话，小一点速度也快一些

因为你自己 compile 了 support 库吧，你可以试试 provided support 库，然后依赖主工程的 support 库这样应该就 ok 的

是滴，可以用provided 试试，骗过编译器。

感谢 @jonyChina162 的回答，@woolly-headed 你可以反编译一下你你刚刚编译出来的APK，肯定是compile 了一堆库在里面。

是的，我看了就是很多库，多谢

authors: cundong jonyChina162 lizhanzhishang woolly-headed


text:
Pluggable Clifford synthesis for RB circuits

Summary
Make Clifford synthesis algorithms for RB circuits pluggable (implementing it as a HighLevelSynthesisPlugin).
Fixes #1279 and #1023.
Change to accept only Clifford elements that contains instructions supported by the backend as an interleaved element for InterleavedRB.
Details and comments
Previously, for 3Q+ RB circuits, entire circuit is transpiled at once and hence for each of the resulting Cliffords, the initial and the final layout may differ, that means sampled Cliffords are changed during the transpilation. Also in the worst case, the resulting circuit may use physical qubits not in the supplied physical_qubits. To avoid that, this commit changes to transpile an RB sequence Clifford by Clifford. The Clifford synthesis algorithm (rb_default) is implemented as a HighLevelSynthesisPlugin (see RBDefaultCliffordSynthesis in clifford_synthesis.py), which forces the initial layout (i.e. guarantees the initial layout = the final layout) and physical qubits to use.
Depends on https://github.com/Qiskit/qiskit/pull/10477 (qiskit 0.45)

No performance regression by this commit:

Comparable with the main branch for 1q and 2q
About 2x speed-up for 3q (by the improvement in cafe761)

Note that for the final inverse Clifford, you can reduce the depth and CX count by using synth_stabilizer_depth_lnn or synth_stabilizer_layers, since you don't need the entire Clifford, but only part of it. See an example in the following test:
https://github.com/Qiskit/qiskit/blob/7c89b68668b6ba74926bf767293dc8dd51ba3cfb/test/python/synthesis/test_stabilizer_synthesis.py#L78

@ShellyGarion Thanks, good to know, however, I think it's too much for RB to have one more Clifford synthesis option only for the last inversion Clifford.

authors: ShellyGarion itoko

repo: Qiskit-Extensions/qiskit-experiments


text:
Return more helpful exception if Quantum Service is in maintenance

What is the expected feature or enhancement?
In the rare times when the IBM Quantum service is in maintenance, there is no longer any API available to the user, and the site is fully replaced with a maintenance page - html, returns a 200.
The runtime client should catch these items as a specific exception when expecting JSON, and convert to an exception that the service is in maintenance mode with a link to the page.
Acceptance criteria

To make sure I understand, a 200 is expected when in maintenance mode, and there isn't a plan to change it to, eg, 503?
See also https://github.com/Qiskit/qiskit-ibm-runtime/issues/1637

To make sure I understand, a 200 is expected when in maintenance mode, and there isn't a plan to change it to, eg, 503?
See also #1637

When in maintenance mode, there is no service to return a 503. It's a static html page that replaces everything on the domain, done at the Cloudflare level.

After #1640 we now catch the exception and return Unexpected response received from server.
We can update to message to return something like this?
Unexpected response received from server. Please check if the service is in maintenance mode https://docs.quantum.ibm.com/announcements/service-alerts.

authors: ihincks kt474 sdague

repo: Qiskit/qiskit-ibm-runtime


text:
Update primitive options

Summary
Update primitive options to match server side support
Details and comments
Fixes #

Pull Request Test Coverage Report for Build 8133910433
Details

186 of 193 (96.37%)  changed or added relevant lines in 14 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.2%) to 82.659%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit_ibm_runtime/options/options.py
32
33
96.97%

qiskit_ibm_runtime/options/resilience_options.py
17
18
94.44%

qiskit_ibm_runtime/options/utils.py
29
30
96.67%

qiskit_ibm_runtime/options/zne_options.py
27
28
96.43%

qiskit_ibm_runtime/options/layer_noise_learning_options.py
17
20
85.0%

Totals

Change from base Build 8100650231:
0.2%

Covered Lines:
5577

Relevant Lines:
6747

💛  - Coveralls

This PR LGTM. Holding off on approval until we agree on how to synchronize server-side efforts.

authors: coveralls ihincks jyu00


text:
Add resilience.zne.amplifier to EstimatorV2 options

Summary
This PR adds the ZneOptions attribute amplifier which can be one of these strings:

"gate_folding" (default) uses 2-qubit gate folding to amplify noise. If the noise
factor requires amplifying only a subset of the gates, then these gates are chosen
randomly.
"gate_folding_front" uses 2-qubit gate folding to amplify noise. If the noise
factor requires amplifying only a subset of the gates, then these gates are selected
from the front of the topologically ordered DAG circuit.
"gate_folding_back" uses 2-qubit gate folding to amplify noise. If the noise
factor requires amplifying only a subset of the gates, then these gates are selected
from the back of the topologically ordered DAG circuit.

Details and comments
Fixes #1670

Pull Request Test Coverage Report for Build 9084117963
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

2 of 2 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.004%) to 82.767%

Totals

Change from base Build 9039367702:
0.004%

Covered Lines:
6239

Relevant Lines:
7538

💛  - Coveralls

We'll need to add this to the schema as well, cc @gadial


text:
Add support for running functions on GPUs

Summary
Provides support for running a select group of functions on GPU nodes. Functions will need to use distribute_task to run workloads on the GPU (ex: @distribute_task(target={"gpu": 1}).
Details and comments
To set the list of providers allowed to run functions on GPUs, edit the gateway/api/v1/gpu-jobs.json file. A sample config, allowing the mockprovider provider to run on GPUs would look like this:
{
    "gpu-functions": {
        "mockprovider": [ "my-first-pattern" ]
     }
}

The method for getting the list of GPU providers is the same as we previously used for allowlisting dependencies.
In a future iteration, this could also be limited to specific functions by checking the the program title matches an element in the list (in this example, that would be my-first-pattern).
Jobs and ComputeResources using GPUs are tracked via a new field in each model.
There is also a separate queue for provisioning GPU clusters. The current default is only 1 GPU job at a time, but that can be configured by setting a value for the LIMITS_GPU_CLUSTERS on the scheduler deployment.
CPU-only jobs are only allowed to run on CPU nodes, and GPU jobs can only run on GPU nodes. This is enforced by configuring a nodeSelector on the rayclustertemplate, the values of which can be configured in the Helm chart.

Is it using the distribute_task annotation or the entire function will run in GPU?

There's two parts here:
As of now, to run on the GPU, functions will need to use distribute_task to let Ray know they'll need  GPU resources. If we didn't want to use distribute_task, we could alternatively  set entrypoint_num_gpus when the job is submitted.
But either way we go, each job will block a GPU until it completes. Nothing we can really do about that -- as serverless is currently written, we don't have the ability to release resources (CPU or GPU) in the middle of a job.

distribute_task annotation is ok.
Ohh so even if the method where the distribute_task annotation finished the resource is not released to be available again for other function?

Is it possible assign a classical CPU node for the main functions and just assign the GPU nodes per each distribute_task annotation call, so the GPU resource is unblocked when the annotated method finishes? (like Ray is doing :) )

Is it possible assign a classical CPU node for the main functions and just assign the GPU nodes per each distribute_task annotation call, so the GPU resource is unblocked when the annotated method finishes? (like Ray allows :) )

No, not without doing a massive overhaul of serverless.

I'm going to try to give a review today @psschwei , thank you! 👍

confirm that the configuration for the gpu-jobs.json can be done in the private repository

Yes, that config will be done in the internal repo

authors: Tansito pacomf psschwei

repo: Qiskit/qiskit-serverless


text:
Fix instruction inheritance issues

Fixes #1892
Summary
Modified the way in which type-checking is done in instruction.py to allow QuantumRegister and ClassicalRegister to be inherited from.
Details and comments
Type-equality checking is brittle and should be avoided except in the case where inheritance is to be disregarded for a class. That appears to be the case for neither QuantumRegister nor ClassicalRegister.

This PR was made against the stable branch, and I didn't notice that when I merged. It is now reverted. Please reopen the PR against master.

repo: Qiskit/qiskit-terra


text:
Speedup binding circuit params

Summary
Speedup binding and returning circuit parameters by

only sorting parameters when it's really needed
caching the sorted parameters

This should tackle the ~70x slowdown as presented in https://qiskit.github.io/qiskit/#circuit_construction.ParameterizedCircuitBindBench.time_bind_params?machine=dedicated-benchmarking-softlayer-baremetal&os=Linux 4.15.0-46-generic&ram=16GB&p-width=20&p-gates=2048&p-number of params=2048&commits=296a745b.
Details and comments
On a 1000 parameter circuit binding 10 times currently takes
dict took 74.88096404075623s # bind via dict
array took 74.05472993850708s  # bind via array

with this PR its
dict took 1.5851850509643555s
array took 0.6309740543365479s

Fixes #5965

It would be nice to have a more uniform interface for accessing the parameters that would also allow __len__ and __contains__, I agree! @molar-volume also suggested that while working on #5961.
It's not that straightforward though, since

the parameter table doesn't know all parameters (global phase might be parameterized)
the parameter view object was designed to be temporary and is currently only constructed upon returning the parameters
If we instead keep a ParameterView object internally (that also might do more efficient caching like you suggested in the comments), we could improve all these points. But this is likely better done in a follow-up, too 🙂

It would be nice to have a more uniform interface for accessing the parameters that would also allow __len__ and __contains__, I agree! @molar-volume also suggested that while working on #5961.

@Cryoris is there an issue to track these discussions so we can follow up once these PRs are merged? Can you open one if not?

I opened a draft suggesting changes to ParameterView and exploring options for simplifying ParameterTable as well https://github.com/Qiskit/qiskit-terra/pull/6074

authors: Cryoris kdk molar-volume


text:
Allow dynamical decoupling pass to work on circuits with pulse gates

Summary
Allow dynamical decoupling pass to work on circuits with pulse gates
Details and comments
The pass was accessing the circuit calibrations incorrectly, resulting in
an error when one of the gates in the dynamical decoupling sequence had
a pulse gate calibration.

Closes #10833

You need to put the "closes #10833" in the top PR comment for it to have an effect, but you can edit it in and it'll work.

Ah, okay. I put it separately because I don't like it ending up in the commit message, but I guess it's fine.

I put it separately because I don't like it ending up in the commit message, but I guess it's fine.

The PR body isn't used for the commit message in a squash-merge - GitHub just catenates all the commit messages together, so if you've got stuff that's added to the PR comment but not the head commit message, it doesn't get included.  (I think the rules are different if the PR is merged by merge, though.)

The PR body isn't used for the commit message in a squash-merge - GitHub just catenates all the commit messages together, so if you've got stuff that's added to the PR comment but not the head commit message, it doesn't get included. (I think the rules are different if the PR is merged by merge, though.)

Ah, okay, I have not looked closely at the commit messages since this repo switched to using the merge queue. In qiskit-experiments, when we started using the merge queue, we changed the default commit message to "Pull request title and body" so that we could have more control over the commit message. I forgot that wasn't the default behavior. I hope some day the "Merge when ready" button will open a text box for editing the commit message like "Squash and merge" does.

Pull Request Test Coverage Report for Build 6180555583

10 of 11 (90.91%)  changed or added relevant lines in 1 file are covered.
23 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.02%) to 87.262%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit/transpiler/passes/scheduling/padding/dynamical_decoupling.py
10
11
90.91%

Files with Coverage Reduction
New Missed Lines
%

crates/qasm2/src/lex.rs
5
92.17%

crates/qasm2/src/parse.rs
18
96.67%

Totals

Change from base Build 6174101621:
-0.02%

Covered Lines:
74199

Relevant Lines:
85030

💛  - Coveralls

authors: coveralls jakelishman wshanks

repo: Qiskit/qiskit


text:
ParameterVector support for pulse parameter assignment

Summary
This PR proposes an extended use of the current method assign_parameters in the ParameterManagerto enable the assignment of a list of values through ParameterVectorclass. This enables an extended usage for the same methods in the Scheduleand ScheduleBlockobjects.
Details and comments
The PR simply takes the already existing unrolling method from QuantumCircuitand applies it for the pulse level case, to enable a simple extension of the existing methods for parameter assignment.
This can be practical when mixing pulse level calibrations with many parameters.
This PR closes #5233

This PR would solve issue #5233

Pull Request Test Coverage Report for Build 8355685528
Details

15 of 18 (83.33%)  changed or added relevant lines in 2 files are covered.
7 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.008%) to 89.315%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit/pulse/parameter_manager.py
14
17
82.35%

Files with Coverage Reduction
New Missed Lines
%

crates/qasm2/src/expr.rs
1
93.81%

crates/qasm2/src/lex.rs
6
91.44%

Totals

Change from base Build 8350434860:
-0.008%

Covered Lines:
59836

Relevant Lines:
66994

💛  - Coveralls

Thanks @arthurostrauss for the final fixes! This looks great.
@nkanazawa1989 - Why commits from other PRs are marked as differences here?

I think GitHub is confused about the multiple branches/merges. When I look at the diff locally, it looks right.

@arthurostrauss This looks good to me now! Do you want to try to make it look clean in GitHub, so we know it will merge the way we want? I can squash things down and force push to your branch if you want.

@arthurostrauss This looks good to me now! Do you want to try to make it look clean in GitHub, so we know it will merge the way we want? I can squash things down and force push to your branch if you want.

@wshanks Yes if you can do such force push (if necessary) I'd be happy to let you do it, as I'm not sure how to proceed on my side for this.

@arthurostrauss Actually, the diff looks good in GitHub now after the last merge, so I think it is fine to merge as is. I am not sure what messed up the diff view before.

authors: TsafrirA arthurostrauss coveralls wshanks


text:
[WIP] Shell script populate DB with config (task #2048)

shell script that populate the DB
first idea of admin_setting.php file

Codecov Report

Merging #646 into task-2048-Admin-settings will decrease coverage by 0.26%.
The diff coverage is 9.8%.

@@                      Coverage Diff                       @@
##             task-2048-Admin-settings     #646      +/-   ##
==============================================================
- Coverage                       30.79%   30.52%   -0.27%
- Complexity                       1168     1176       +8
==============================================================
  Files                             105      107       +2
  Lines                            4014     4065      +51
==============================================================
+ Hits                             1236     1241       +5
- Misses                           2778     2824      +46

Impacted Files
Coverage Δ
Complexity Δ

src/Shell/SettingsShell.php
0% <0%> (ø)
6 <6> (?)

src/Model/Table/SettingsTable.php
29.41% <29.41%> (ø)
2 <2> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3cd237...6e44b53. Read the comment docs.

authors: codecov-io eliamariutti

repo: QoboLtd/project-template-cakephp


text:
[SDS-617] Fix measurement statements in ProjectQ

Added measurement statements in projectQ algorithm at the end not needed (__add_measure_all_qubits deleted) and even led to a crash crashed when no qubits were measured
With fsp, the simulator will return all qubits as measured, but when there were (removed) measurements in the deterministic algorithm we want to get these measured qubits only in the results (_get_measured_qubit_iterator).

Coverage remained the same at 100.0% when pulling 582208c59dc1263f1aa496f375334bfa69c6745e on fix/SDS-617/measurements_projectq into f4987cd2c56b9e98a618c31e688d7c231302c1da on dev.

authors: QFer coveralls

repo: QuTech-Delft/quantuminspire


text:
Known bugs

Things that need to be implemented (perhaps you can help!)

Add search lyrics functionality when inaccurate or no lyrics can be found with a button under the search list to add your own lyrics to save
Add lyric button for toggling scroll functionality (some of it is in there but not functional)
Add cobalt theme (dark blue(s), green)
Make sure adverts are click-able and take you to their intended place (if they are Spotify playlist adverts - load it up in the player)

Bugs that you may come across

Sometimes when loosing a network connection, Spotify will never reconnect/turn black
Theme inconsistencies

Cobalt theme may be created in future release, adverts implementation has been changed - closing mostly due to irrelevance after git commits


text:
Training only last added person

Is it possible to train only the lastly added person instead of training from the scratch?

I don't know any algorithm which allows a delta training, but probably you can find something online

Can you give me a gist on how I can do so?

What takes the most amount of time is usually the feature extraction. The easiest way to improve the speed of training would be to save the extracted features (were already done) so you don't need to extract them again (if you don't change the settings/parameters, however if you change some parameters then you should repeat the whole extraction again).
https://github.com/elimu-ai/authentication/blob/eabd87855f8fc541ea488b0a84a35c71f1147a1a/app/src/main/java/org/literacyapp/authentication/thread/TrainingThread.java#L103
Another solution would be incremental/online learning, but then you would need to find a library which implements it or you would need to code the whole algorithms yourself (e.g. incremental SVM).

sorry for my late reply... is it possible to do that in python may be.. if so can you please share with me a gist on the same?

@Zumbalamambo  , Great to hear you made it working  ,but I am still facing the same problem to the confidence score on the Android App . I am using KNN. Could you please help me in to achieve  confidence score.

@rautsunil are you able to achieve the confidence score?

authors: Zumbalamambo rautsunil sladomic sunnykinger


text:
Login page next paramemter in url not set properly

When clicking the profile the link query parameter next is not set properly

It gives http://localhost:5000/conda-store/login?nexthttp://localhost:5000/conda-store/static/index.html and should be http://localhost:5000/conda-store/login?next=http://localhost:5000/conda-store/static/index.html

Hi @costrouc
To unblock you quickly, in your. env file, you can set REACT_APP_LOGIN_PAGE_URL with the missing parameter: http://localhost:5000/conda-store/login?next=.
If the variable is not set, it will take the value given from preferences.tsx. The second condition is missing for =.
I am going to create a PR to fix that issue.
Thanks for creating this issue.

PR #174

PR Merged, thanks for the issue @costrouc  and thanks for your reactivity @jujogi  :)

authors: costrouc jujogi pierrotsmnrd

repo: Quansight/conda-store-ui


text:
How to use non-ASCII characters as identifiers?

I have tried using non-ASCII characters as the name of axioms (such as Chinese characters),
but Lean gives an error: "expected token".
def 字符串:="this is a string"

Is there any feasible solution?
(However, it is possible to use fancy characters, such as ℙ)

Hey @hhiim!
Believe def 字符串:="this is a string" is not a valid syntax, at least in py 3.11. The following code works correctly in the QC cloud:
class CalculatingRedOrangeBuffalo(QCAlgorithm):
    def Initialize(self):
        self.Quit(self.字符串())

    def 字符串(self):
        return "this is a string"

Hey @hhiim!
Believe def 字符串:="this is a string" is not a valid syntax, at least in py 3.11. The following code works correctly in the QC cloud:
class CalculatingRedOrangeBuffalo(QCAlgorithm):
    def Initialize(self):
        self.Quit(self.字符串())

    def 字符串(self):
        return "this is a string"

Sorry, It's a misunderstanding. I was not distinguishing "Lean" and "Lean Theorem Prover"...
close the question please --- again sorry!

authors: Martin-Molinero hhiim

repo: QuantConnect/Lean


text:
practical structural issues while manipulating the Cytoscape widget

It's possible to have multiple instances of same node/edge in the CyptoscapeWidget.  The following bit of code is perfectly valid.

>>> G = ipycytoscape.CytoscapeWidget()
>>> G.graph.add_graph_from_networkx(nx.complete_graph(5))
>>> G.graph.add_graph_from_networkx(nx.complete_graph(9))
>>> G.graph.nodes
[Node(data={'id': 0}, position={}),
 Node(data={'id': 1}, position={}),
 Node(data={'id': 2}, position={}),
 Node(data={'id': 3}, position={}),
 Node(data={'id': 4}, position={}),
 Node(data={'id': 0}, position={}),
 Node(data={'id': 1}, position={}),
 Node(data={'id': 2}, position={}),
 Node(data={'id': 3}, position={}),
 Node(data={'id': 4}, position={}),
 Node(data={'id': 5}, position={}),
 Node(data={'id': 6}, position={}),
 Node(data={'id': 7}, position={}),
 Node(data={'id': 8}, position={})]

Just deleting a node messes up the entire graph, as deleting a node doesn't delete the edge itself.

>>> nodes = G.graph.nodes
>>> G.graph.remove_node(nodes[0])
>>> G.graph.nodes
[Node(data={'id': 1}, position={}),
 Node(data={'id': 2}, position={}),
 Node(data={'id': 3}, position={}),
 Node(data={'id': 4}, position={})]
>>> G.graph.edges
[Edge(data={'source': 0, 'target': 1}, position={}),
 Edge(data={'source': 0, 'target': 2}, position={}),
 Edge(data={'source': 0, 'target': 3}, position={}),
 Edge(data={'source': 0, 'target': 4}, position={}),
 Edge(data={'source': 1, 'target': 2}, position={}),
 Edge(data={'source': 1, 'target': 3}, position={}),
 Edge(data={'source': 1, 'target': 4}, position={}),
 Edge(data={'source': 2, 'target': 3}, position={}),
 Edge(data={'source': 2, 'target': 4}, position={}),
 Edge(data={'source': 3, 'target': 4}, position={})]

There is no such thing as an edge id currently (atleast while adding a networkx object?) so the function remove_edge_by_id should be something like def remove_edge_by_id(self, source_id, target_id) rather than def remove_edge_by_id(self, edge_id))

Never really noticed that, but you're completely right.
Thanks @MridulS!

The issues are addressed in #54

This was incorrectly linked to #104 .

fixed in https://github.com/QuantStack/ipycytoscape/pull/54

authors: MridulS marimeireles

repo: QuantStack/ipycytoscape


text:
unweighted bases should be available for array summaries

currently unweighted bases for arrays are skipped in stack.aggregate() if weighted bases are available.
Changing this might also affect qp.ViewManager, qp.ChainManager, ExcelPainter, PowerPointPainter

merged into #1080

repo: Quantipy/quantipy


text:
Overall, ported the godot-console to Godot 3.1 alpha4 baseline.

Fixed newly explicit enum scope/name syntax.
Refactored 'build'er code to a more appropriate scope. 3.1 doesn't seem keen on self replicating classes.
Argument.build() -> Commands.build_argument()
Argument.buildAll() -> Commands.build_arguments()
Command.build() -> Commands.build_command()
Renamed get() to fetch() to deconflict with Object's get() method.

Looks like the Argument setting and fetching is still a bit off in this pull request. I'm working to fix that in my fork at the moment. I also noticed that you've done a bit of refactoring in the dev branch ... but it references a missing Callback.gd that isn't pushed into the branch.

Good evening, @crazychenz
Thanks for your contribution!
The dev branch was rewritten a LOT. And there's more to come. Rn if I remember it right, it is pretty stable.
The Callback is now here

no, i was talking about the "init.gd" scripts in the DEV branch that have a bunch of preloads in them. I have never seen anyone do that in Godot, in Python, yes. Just wondering why you are doing it. Thanks.

I guess I am interested now too! I just looked at the godot core and there is no reference to an init.gd. Looks like GodotExplorer uses this pattern as well. If I had to guess, it looks like a simple pattern for using godot-console as a base for sub-classing or extensions. You can see @QuentinCaffeino 's usage of it in the top level init.gd, but I don't see them used anywhere else.

You're right, @crazychenz, it's just a pattern that I adopted from python. Sometimes it just makes life a little easier.

Many things have been rewritten since, closing this PR as it is no longer required.
Thanks for your attempt to contribute)

authors: QuentinCaffeino crazychenz paulhocker

repo: QuentinCaffeino/godot-console


text:
specify requirements without direct reference

This PR is fixing issues while using this package in symbiflow-arch-defs
There is an existing package in python repository for fasm so it is not needed to fetch it from github.
In case of fasm-utils, we provide the package name before the @ .
Signed-off-by: Paweł Czarnecki pczarnecki@antmicro.com

@lnsharma, @tpagarani - could you please look into that?

repo: QuickLogic-Corp/quicklogic-fasm


text:
Can't resolve 'next-international/middleware'

Describe the bug
Clone repo, go to example, next-app, pnpm i, then pnpm dev met Can't resolve 'next-international/middleware'
To Reproduce
Steps to reproduce the behavior:

Go toexample/next-app/
pnpm i
pnpm dev
See error

Expected behavior
No error
Screenshots

About (please complete the following information):
main branch on 2023-11-02

You have to build both packages/next-international & packages/international-types first - I'll open another issue to add a proper CONTRIBUTING.md documentation.

authors: Crayon-ShinChan QuiiBz

repo: QuiiBz/next-international


text:
Failed to spawn command

This is the specific error I get.
Failed to spawn command /home/christian/anaconda3/bin/python3. Make sure /home/christian/anaconda3/bin/python3 is installed and on your PATH
[YCM] Error
Error: connect ECONNREFUSED 127.0.0.1:42399
at Object.exports._errnoException (util.js:1026:11)
at exports._exceptionWithHostPort (util.js:1049:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1081:14)
I can use anaconda from terminal so I don't see why I am getting this path issue.

I have no idea what's going wrong. You can run which python3 in your shell to see the full path to the python executable.

repo: Qusic/atom-youcompleteme


text:
[BUG] 多GPU LoRA报错：ssertionError: {'id': 323, 'status': 'NOT_AVAILABLE', 'numel': 0, 'ds_numel': 0, 'shape': (0,), 'ds_shape': (0,), 'requires_grad': True, 'grad_shape': None, 'persist': True, 'active_sub_modules': {7}, 'ds_tensor.shape': torch.Size([0])}

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

[X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

[X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior
使用qwenllm/qwen:cu117镜像,对Qwen-14B-Chat在2*v100上进行LoRA微调报错
期望行为 | Expected Behavior
是环境配置不兼容吗？能够帮忙复现排查下~
复现方法 | Steps To Reproduce
No response
运行环境 | Environment
- OS:Ubuntu 20.04
- Python:3.8.10
- Transformers:4.32.0
- PyTorch:2.0.1
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`):11.7

备注 | Anything else?

这个问题我也遇到了

我也遇到了, 通过finetune/finetune_lora_ds.sh对Qwen-14B模型进行finetune时遇到的

我也遇到了, 通过finetune/finetune_lora_ds.sh对Qwen-14B模型进行finetune时遇到的

不用lora这个错误就没了

我也遇到了, 通过finetune/finetune_lora_ds.sh对Qwen-14B模型进行finetune时遇到的

我也是

有什么解决办法嘛？只能做得起lora

有解决的没？我也遇到了

@jiangliqin @nightmaredimple 您好，可以提供一下报错的具体命令吗？

我报错是 运行的 bash finetune/finetune_lora_ds.sh ,使用的是你们提供的deepspeed_stage3， 初次运行的时候报了其他的错误，注释了 device_map="auto"

@jiangliqin @nightmaredimple 您好，可以提供一下报错的具体命令吗？

就是按照官方提供的，执行bash finetune/finetune_lora_ds.sh，diff有：

fp16=True
使用finetune/ds_config_zero3.json
注释finetune.py中的device_map="auto"

不确定是否跟这个issue相关：https://github.com/huggingface/peft/issues/1278

@yawzhe 目前对Base模型使用finetune/finetune_lora_ds.sh进行多机LoRA微调，在使用deepspeed zero3的情况下，会由于peft内部的处理逻辑，导致deepspeed无法正确切分embedding（wte）参数，可参考 #718 的相关信息。
我们正在解决该问题，目前您可以尝试以下方法：改用基于Chat模型微调、改用Q-LoRA微调、将zero3改为zero2配置、或者在finetune.py中手动关闭对wte参数的微调（参见此处）。
如果不是上面的微调配置而仍然报错，您可以尝试更新docker镜像到最新版本然后重试一下，我们这边测试的结果，其他情况下均能正常微调。

我用的就是chat模型微调lora,使用deepspeed,发现报错

我用的就是chat模型微调lora,使用deepspeed,发现报错

如果使用的是chat模型微调，应当是不会出现问题的。
可以更新一下docker镜像，再检查一下运行命令时chat模型挂载到的模型名是否包含“chat”字符串，finetune.py根据这个条件判断是否将wte加入微调之中。

遇到了同样的问题，改成 qwen-14b-chat 就好了

authors: LMXKO fyabc icowan jiangliqin nightmaredimple njupopsicle tangsipeng yawzhe

repo: QwenLM/Qwen


text:
Create a room with error : "Parent entities cannot be found."

I create a room with a client.
I quit the client.
I create a second room.
I have a "Parent entities cannot be found." and the client stop.

Same problem when I try to reconnect to a room

Outdated, refer to my discord message.

authors: LucasTesnier osvegn

repo: R-Type-Epitech-Nantes/R-Type


text:
glados.pt GPU model

I was trying to use a GPU for loading the model, but it didn't worked

glados.pt is an 8bit quantization of the original model, and only intended to be run on CPU. I found that the ForwardTacotron model was so fast that the GPU overhead actually made the thing run slower. I lost the original files for the model due to my laptop going kaboom when playing Valorant but I have a new version of the model that sounds much better. If you'd like to try running that you can add me on Discord or something

Yea sure :D, what's your discord. Mine is pedro021

Mine is r2d2_fish

Already send a request :D


text:
Add option for custom background image and custom font colors

title, customizablity is always great for users.

I would recommend against it, as doing so could affect UI readability and would harm the project's credibility.

thats a user problem if they fuck up their color schemes.
as long as default stays the same there will be no problem


text:
Fix: Today Page Not Updating #126

The issue was that the DOM elements were being updated every second, but the data they displayed was not changing because it was only being set on page load or reload. To resolve this, I modified the code so that the today variable, which holds the current date and time, is refreshed every second within the setInterval function. This ensures that the data is current when it’s used to update the DOM elements. Additionally, I consolidated the getAndSetDomElements and setDomElements functions into a single setDomElements function to streamline the code. Now, this unified function is called within the setInterval method, which not only simplifies the code but also guarantees that both the data and the DOM elements are updated in sync every second.

Looks good. Slight revision can you use arrow functions instead to keep all the functions consistent.

Can you use arrow functions for updateDate and setDomElements.

Yes sir, I'm on it

Fixes #126

Looks good. Go for merge

authors: JohnPatrickAncajas RDjarbeng

repo: RDjarbeng/countdown


text:
📖 DOC: Updated README

[Update] Contributors section in README
Closes #55
Changes made🏗:

[x]  Updated table of content with sub lists.
[x] Updated .gitIgnore to .gitignore
[x] Updated Contributors section.

Awesome! Thank you :)

authors: Codedwells unwls


text:
Lines should not be more than 80 characters

Hello:
I've recently encountered a problem with the R extension in VScode that is causing me a lot of headaches!

When a line of code exceeds 80 characters, a long wavy underline leaves indicating that I've exceeded 80 characters, which may override some other issues, so I've come to ask for help, is there any way to simply and brute force permanently ignore this alert?
I tried using a .lintr file and writing linters in it: with_defaults(line_length_linter=line_length_linter(100)), but another problem occurs with the prompt as shown here:
Failed to run diagnostics: ! in callr subprocess. Caused by error in 'lintr::lint(path, cache = cache, text = content) ' : ! Malformed config file (.lintr): Caused by error in 'load_config()': ! Error from config setting 'linters'. Caused by error: ! Function with_defaults was deprecated in lintr version 3.0.0. Use linters_with_defaults or modify_defaults instead. lintr

Expect a reply soon and help me solve this problem!Thanks.

Hope this helps:
.lintr
linters: linters_with_defaults(
    line_length_linter(142)
  )

Thanks for your prompt reply, my problem has been solved.
Thanks again!

You are welcome.
Please, close the issue. :)

authors: Midreancgle albertosantini

repo: REditorSupport/vscode-R


text:
We can delete the lib/widgets folder

This folder uses widgets no longer needed.

I'll take this issue.

authors: FedericoBruzzone pentidave

repo: RIP-Comm/sossoldi


text:
🛑 Cleanse Drops Kidney is down

In 2774a12, Cleanse Drops Kidney (http://cleansedropskidney.com/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Cleanse Drops Kidney is back up in 6c3cbc7 after 12 minutes.

repo: RJVCA/uptime


text:
🛑 Activated Male is down

In 7d1ee5c, Activated Male (http://activatedmale.com/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Activated Male is back up in 8b8cede after 11 minutes.


text:
🛑 CEOTR Home Page loads is down

In c1c79a8, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in 63069b6 after 2 minutes.

repo: RKTowse/upptime


text:
🛑 CEOTR Home Page loads is down

In 853dd44, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in 20a55a1 after 2 minutes.


text:
🛑 OGEN Log In is down

In 66a936f, OGEN Log In (https://dev.oceangraduate.com/wp-login.php) was down:

HTTP code: 0
Response time: 0 ms

Resolved: OGEN Log In is back up in 3d207c0 after 3 days, 11 hours, 31 minutes.


text:
🛑 Sensor Tracker login is down

In a651ac1, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Sensor tracker login is back up in b16a0fe after 1 minute.


text:
🛑 Sensor Tracker login is down

In 7689abc, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Sensor tracker login is back up in 667d075 after 1 minute.


text:
🛑 CEOTR Home Page loads is down

In 10e00fb, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in dfd62c4 after 2 minutes.


text:
🛑 Sensor Tracker login is down

In b81305b, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 502
Response time: 118 ms

Resolved: Sensor tracker login is back up in 8b1190c.


text:
🛑 CEOTR Home Page loads is down

In e4ec52d, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in db8b118 after 2 minutes.


text:
🛑 Sensor Tracker login is down

In 53925e0, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 502
Response time: 106 ms

Resolved: Sensor tracker login is back up in 101a849.


text:
🛑 Sensor Tracker login is down

In 205fd6b, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 502
Response time: 42 ms

Resolved: Sensor tracker login is back up in fc69e71.


text:
🛑 CEOTR Home Page loads is down

In 6de7d9d, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in 457b41f after 2 minutes.


text:
🛑 CEOTR Home Page loads is down

In 4a21be7, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in a90c41a.


text:
🛑 CEOTR Home Page loads is down

In d93685f, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in b7ce27a.


text:
🛑 CEOTR Home Page loads is down

In d89b265, CEOTR Home Page loads (https://stg.ceotr.ca CEOTR Home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CEOTR Home Page loads is back up in 974c2b7 after 38 days, 21 hours, 49 minutes.


text:
🛑 Sensor Tracker login is down

In 50f361c, Sensor Tracker login (https://stg.ceotr.ca/sensor_tracker) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Sensor tracker login is back up in 13e2dbf after .


text:
A question about open_manipulator_x in ROS gazebo simulation

I meet a big problem when I use open_manipulator_X during the process of gazebo simulation.
I put the problem's video in annex.
because I install the manipulator on the PX4 drone,the control period of the PX4 is 250HZ,so I set the parameter in empty.world like this:
<real_time_update_rate>250.0</real_time_update_rate>
<max_step_size>0.004</max_step_size>
<real_time_factor>1</real_time_factor>
I also check the .xacro files and I only find one parameter in open_manipulator.gazebo.xacro like that:
  <controlPeriod>0.004</controlPeriod>

I modified these two files and then,the question arises:manipulator shaking in the gazebo.
and I find that the lower update_rate,the bigger shake.
I try to modify PX4 files,but only gazebo simulation rate is 250HZ can start the gazbeo.
so my question is whether users can modify the control period of the open_manipulator_x,if we can,than how to do ? ^_^
if we can not modify the parameter 1000hz,when I use material object,will I have the same problem?
This little problem has not been solved for two weeks.I would great aooerciate you if you could help me.
thank you!

Hi @Sunchanghao
I understand the problem.
That vibration problem occurs in the gazebo ros controller.
The reduced control rate prevents the controller from converging properly. You can try to reset the PID gains in gazebo controller to solve this problem. These PID gains are set in open_manipulator_gazebo/config/open_open_manipulator_controller.yaml in open_manipulator_simulation package.
Plaese try it.

@gpwhd07 Thank you! I have realized that I need reset the PID gains in  .yaml
like you say, after I try to reset the PID gains,the model do not shake again, even more stable than the original model.
Thank you!

@Sunchanghao
That's good.
We look forward to seeing your project perform well.


text:
rocFFT and hipFFT examples (part II)

This pull request contains the second batch of the new rocFFT and hipFFT examples. Added samples:

rocFFT

complex_complex
complex_real
real_complex

hipFFT

multi_gpu

I'm having some trouble getting the example to work with the cuda backend; cmake issues show up.

Sorry for the delay - jumping back into this again.
When I configure with
cmake -DCMAKE_CXX_COMPILER=hipcc -DGPU_RUNTIME=CUDA ..
I get errors like
[ 16%] Building CUDA object multi_gpu/CMakeFiles/hipfft_multi_gpu.dir/main.cpp.o
In file included from /home/AMD/marobert/repo/rocm-examples/Libraries/hipFFT/multi_gpu/main.cpp:24:
/home/AMD/marobert/repo/rocm-examples/Libraries/hipFFT/multi_gpu/../../../Common/example_utils.hpp:50:10: fatal error: hip/hip_runtime.h: No such file or directory
   50 | #include <hip/hip_runtime.h>
      |          ^~~~~~~~~~~~~~~~~~~
```

@malcolmroberts Oh I see, but IINM that error should be happening for other samples because it seems the HIP headers are not found. I can reproduce this on my side if for instance I rename the /opt/rocm/include/hip folder to something else. Could you verify if you have the hip headers present on your machine?

Hey, @Beanavil ; I do have /opt/rocm/include/hip populated on my test machine.
However, if there isn't a find_package( hip REQUIRED ) then I wouldn't expect it to include or link the hip runtime.  The files are .cpp, so the cmake setup isn't detecting the hip programming language, and I don't think that the runtime is going to be automatically included.  I'm configuring/compiling from the Libraries/hipFFT subdirectory.

@Beanavil Just to be totally clear here, are you also building these examples under CUDA?

@evetsso @malcolmroberts sorry for the delayed response, I got totally caught up on other tasks. Yes, we build the examples for CUDA and HIP backend. In fact, we use the dockerfiles provided by this repo for this, so I think if you use a container built from the cuda image the build should be successful.
I'm trying to figure out what could be triggering the failures in your end, I don't think a find_package( hip REQUIRED ) is what's missing here because IINM when we link to hipFFT we are already including the dependency on HIP. On my end, for instance, I can see that when I invoke cmake it does add the compiler option for including the hip headers (-isystem=/opt/rocm/include):
developer@2c1b9a595f95:~/rocm-examples/Libraries/hipFFT/build$ cmake --build . -v
...
[ 16%] Building CUDA object multi_gpu/CMakeFiles/hipfft_multi_gpu.dir/main.cpp.o
cd /home/developer/rocm-examples/Libraries/hipFFT/build/multi_gpu && /usr/local/cuda/bin/nvcc
-forward-unknown-to-host-compiler  -I/home/developer/rocm-examples/Libraries/hipFFT/multi_gpu/../../../Common
+ -isystem=/opt/rocm/include --generate-code=arch=compute_52,code=[compute_52,sm_52] -std=c++17 -MD -MT
multi_gpu/CMakeFiles/hipfft_multi_gpu.dir/main.cpp.o -MF CMakeFiles/hipfft_multi_gpu.dir/main.cpp.o.d -x cu
-c /home/developer/rocm-examples/Libraries/hipFFT/multi_gpu/main.cpp -o CMakeFiles/hipfft_multi_gpu.dir/main.cpp.o
...

Could you also show how is the compiler being run on your end? This is done with the -v option that I show in the previous code.

@MKKnorr hi could you ping the right people to get this merged? it's been open for awhile

Sorry for the delay; I'm gong to try and be more on top of this PR.
So, the issue that I'm having is that, on a pure NVIDIA machine, I can install hipfft, and set up cmake paths, but when I try and compile, for example, plan_d2z, I get a compilation error that it cna't find the hip runtime.  Steps are:
cd Libraries/hipFFT/plan_d2z && mkdir build && cd build
export HIP_PLATFORM=nvidia
cmake -DGPU_RUNTIME=CUDA ..
make

then I get
[ 50%] Building CUDA object CMakeFiles/hipfft_plan_d2z.dir/main.cpp.o
In file included from /home/marobert/repo/rocm-examples/Libraries/hipFFT/plan_d2z/main.cpp:24:
/home/marobert/repo/rocm-examples/Libraries/hipFFT/plan_d2z/../../../Common/example_utils.hpp:50:10: fatal error: hip/hip_runtime.h: No such file or directory
   50 | #include <hip/hip_runtime.h>
      |          ^~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/hipfft_plan_d2z.dir/build.make:77: CMakeFiles/hipfft_plan_d2z.dir/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/hipfft_plan_d2z.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

This isn't a problem when compiling on AMD systems.

@malcolmroberts could you perhaps give more details about the previous setup that was done on that machine (ROCm instalation e.g.)? I'm trying to reproduce on a NVIDIA machine inside a container (using this image) and following the setup steps of our dockerfile for cuda (this one) and I cannot reproduce any build errors :(.

I can get this working in the docker image; trying to figure out what the difference might be in the installs.

authors: Beanavil Naraenda evetsso malcolmroberts

repo: ROCm/rocm-examples


text:
Improve API for program parameters

This improves the API usability. For python API, it allows passing a numpy array in directly in the python dict. For C++, the program parameters can be constructed using initializer lists.

Is there a way to support both previous API and the new API change? Otherwise, we will have to update all of our external scripts/documentation as well.

repo: ROCmSoftwarePlatform/AMDMIGraphX


text:
Add Pibrella

Using JamHat as template, added Pibrella so can be used remotely via PiZero GPIOExtender

Codecov Report

Merging #773 into master will decrease coverage by 0.09%.
The diff coverage is 44.44%.

@@            Coverage Diff            @@
##           master     #773     +/-   ##
=========================================
- Coverage   79.44%   79.35%   -0.1%
=========================================
  Files          23       23
  Lines        4472     4481      +9
  Branches      651      651
=========================================
+ Hits         3553     3556      +3
- Misses        864      870      +6
  Partials       55       55

Impacted Files
Coverage Δ

gpiozero/__init__.py
100% <ø> (ø)
:arrow_up:

gpiozero/boards.py
98.3% <44.44%> (-1.05%)
:arrow_down:

gpiozero/mixins.py
97.88% <0%> (-0.43%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9435622...a2c2d95. Read the comment docs.

Closing in favour of #798 - thanks @ForToffee

authors: ForToffee bennuttall codecov-io

repo: RPi-Distro/python-gpiozero


text:
Search Engine RSS Bridges don't work correctly

A RSS feed should contain news from the date of publication, and not with the date when the entry was last found by a search query. The result of this bug is, that the feed checking software always gets RSS feed entries with all the same date of when the Bridge's search was done. So it will always fire all results at once, again and again, even though most of the results were cached months ago by the respective search engine.
The bug could be fixed by first checking the date a resulting URL was first put into the search engine's cache, so all the search results can be sorted chronologically into the then created RSS feed.

Technically, if the RSS entries have the same ID (but different dates), the feed reader should consider them as equal and don't mark them as new.
But you are correct. However, sometimes it's too expensive to fetch the correct date for each item. On "social" sites, you are heavily rate-limited, so doing many more requests to determine this might not be practically useful.

How many results of a search engine are actually considered? they seem to be below 10

authors: gloschtla somini

repo: RSS-Bridge/rss-bridge

license: unlicense


text:
[YoutubeBridge] Removed duration in titles on search mode

Partially(!) fixes https://github.com/RSS-Bridge/rss-bridge/issues/671 - does not append duration to body.
Before: https://feed.eugenemolotov.ru/?action=display&bridge=Youtube&s=doom2&pa=&format=Html
After: https://feed.eugenemolotov.ru/dev/?action=display&bridge=Youtube&s=doom2&pa=&format=Html

Merged, thanks !


text:
Should we drop LOINC from KG2?

The LOINC UMLS source seems to take a very long time to ETL (last report was 29 hours). Unclear what benefit it brings, given the huge time cost to ETL it. Considering dropping it from the KG2.8.5pre build and beyond. We can discuss here.

@edeutsch wrote:

I don't think the SRI Node Normalizer has LOINC identifiers, so we probably won't get LOINC identifiers in queries. But I don't know what kinds of edges it might have that are useful.

@dkoslicki wrote:

I don't know of places where it's vital either; not to say it isn't, I just don't see LOINC stuff that often (which likely speaks to its useful/lessness)

We have decided to drop LOINC starting with KG2.8.4.

The report comparison step noted that all of LOINC was removed in KG2.8.4pre, so I am going to close out this issue.

Farewell, LOINC!

authors: ecwood saramsey


text:
模型微调

你好，请问模型支持微调吗？想用一些数据微调codes15b，微调代码工程使用LLaMA-Factory，然后用微调后的模型再在spider和bird上进行推理，看效果相比codes15b会怎样变化。如果可以的话，可以提示下需要注意的地方吗（比如数据、模板等）？

base模型是支持微调的（codes-1b, codes-3b, codes-7b, 和codes-15b），我们也开放了微调脚本train_causal_lm.py，并在README里写了如何微调。如果你使用LLaMA-Factory进行微调的话，你需要提前将数据处理成<instruction, input, output>三元组，其中input可以是空的，instruction包括database prompt和question，详见load_sft_dataset.py文件。模版只需要保证训练和评估时一致就行。

好的，因为我打算用这个数据（https://huggingface.co/datasets/Healthy13/Text2SQL）进行微调，数据格式就是<instruction, input, output>三元组，所以使用LLaMA-Factory进行微调的话不用进行数据处理了。而如果用您项目里的微调脚本，是需要把数据处理成sft_spider_train_text2sql.json类似的格式吗

是的，如果想用CodeS中提出的database prompt construction方案，就需要把数据处理成sft_spider_train_text2sql.json一样的格式。
如果用这个模版进行微调，推理时需要让text2sql_zero_shot.py（不是text2sql_few_shot.py，这个是few-shot脚本）的输入格式修改成一样。不过我建议您使用LLaMA-Factory自带的推理脚本进行推理，这样就不用考虑格式对齐的问题了。

好的，明白了。
我基于LLaMA-Factory和default模板用上述数据集的200多条数据对codes-15b进行了微调，然后基于微调后的模型使用LLaMA-Factory的推理脚本在spider-dev上进行了推理，而后的评估结果是27.9分。请问准确率这么低会是因为微调数据量比较少吗？但是codes-15b基础模型在few-shot情况下spider上的准确率挺高的呢。

上述数据集的质量好像是挺低的。。。你可以尝试直接使用spider的训练集进行微调，然后在spider的dev集上进行测试。

好的，我试一下，感谢~

您有试过codes-7b-spider微调模型在bird上的推理效果怎么样嘛？

这个没有尝试过，因为spider和bird的标注习惯以及难度不同（其实就是它们的数据分布不同），而codes-7b-spider已经完全拟合了spider的数据分布，因此在bird上可能效果并不是很好。我后来把spider和bird这些训练数据合并到一起，训练了一个7b的模型，名为codes-7b-merged，这个模型可以在spider和bird上都有不错的效果。

针对某个数据集的train集微调，在相应数据集的dev集上可以取得较好效果，但是在其他数据集上效果怎么样呢？

spider和bird都是cross-domain能力的评估，也就是说训练时见到的数据库schema和测试时没有重合。

好，我明白啦，非常感谢！

repo: RUCKBReasoning/codes


text:
RVC GUI will not start

Whenever I try to load RVC-GUI, it doesn't load and gets stuck on "Running with the runtime Python, Please wait."
I ran it before and it worked but I reinstalled it and it doesn't work anymore
I use Python 3.10.11 and I have all of the dependencies installed

NEVERMIND it works now


text:
Mono support

Since Selenium should work in more, check if this library is also going to support this platform.

I've created a Travis CI project to test building and running the project on mono.
https://travis-ci.org/RaYell/selenium-webdriver-extensions
It doesn't build at the moment.
Errors:
/home/travis/build/RaYell/selenium-webdriver-extensions/Selenium.WebDriver.Extensions.sln (default targets) ->
(Build target) ->
/home/travis/build/RaYell/selenium-webdriver-extensions/src/Selenium.WebDriver.Extensions/Selenium.WebDriver.Extensions.csproj (default targets) ->
/usr/lib/mono/xbuild/12.0/bin/Microsoft.CSharp.targets (CoreCompile target) ->
	Models/By.cs(4,30): error CS0234: The type or namespace name `Core' does not exist in the namespace `Selenium.WebDriver.Extensions'. Are you missing an assembly reference?
	Models/By.cs(25,23): error CS0012: The type `Selenium.WebDriver.Extensions.Core.SelectorBase' is defined in an assembly that is not referenced. Consider adding a reference to assembly `Selenium.WebDriver.Extensions.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'
	Models/By.cs(57,23): error CS0246: The type or namespace name `QuerySelector' could not be found. Are you missing an assembly reference?
	Models/By.cs(71,23): error CS0246: The type or namespace name `QuerySelector' could not be found. Are you missing an assembly reference?
	Models/By.cs(81,27): error CS0246: The type or namespace name `ClassNameSelector' could not be found. Are you missing an assembly reference?
	Models/By.cs(91,27): error CS0118: `OpenQA.Selenium.By.CssSelector(string)' is a `method' but a `type' was expected
	Models/By.cs(101,27): error CS0246: The type or namespace name `IdSelector' could not be found. Are you missing an assembly reference?
	Models/By.cs(111,27): error CS0246: The type or namespace name `LinkTextSelector' could not be found. Are you missing an assembly reference?
	Models/By.cs(124,23): error CS0246: The type or namespace name `LinkTextSelector' could not be found. Are you missing an assembly reference?
	Models/By.cs(134,27): error CS0246: The type or namespace name `NameSelector' could not be found. Are you missing an assembly reference?
	Models/By.cs(144,27): error CS0246: The type or namespace name `PartialLinkTextSelector' could not be found. Are you missing an assembly reference?
	Models/By.cs(157,23): error CS0246: The type or namespace name `PartialLinkTextSelector' could not be found. Are you missing an assembly reference?
	Models/By.cs(167,27): error CS0246: The type or namespace name `TagNameSelector' could not be found. Are you missing an assembly reference?
	Models/By.cs(180,27): error CS0246: The type or namespace name `XPathSelector' could not be found. Are you missing an assembly reference?
	 2 Warning(s)
	 14 Error(s)

The source of those errors is wrong SolutionDir value in some csproj files and wrong referenced project id.

The build now failes while trying to download ChromeDriver
/home/travis/build/RaYell/selenium-webdriver-extensions/Selenium.WebDriver.Extensions.sln (default targets) ->
(Build target) ->
/home/travis/build/RaYell/selenium-webdriver-extensions/test/Selenium.WebDriver.Extensions.IntegrationTests/Selenium.WebDriver.Extensions.IntegrationTests.csproj (default targets) ->
(DownloadChromeDriver target) ->
	/home/travis/build/RaYell/selenium-webdriver-extensions/test/Selenium.WebDriver.Extensions.IntegrationTests/Selenium.WebDriver.Extensions.IntegrationTests.csproj: error : Command '%WINDIR%/System32/WindowsPowerShell/v1.0/powershell.exe -NonInteractive -executionpolicy Unrestricted -command "& { &'/home/travis/build/RaYell/selenium-webdriver-extensions/packages/Selenium.WebDriver.ChromeDriver.2.14.0.0/tools/Init.ps1' '/home/travis/build/RaYell/selenium-webdriver-extensions/packages/Selenium.WebDriver.ChromeDriver.2.14.0.0/' '/home/travis/build/RaYell/selenium-webdriver-extensions/packages/Selenium.WebDriver.ChromeDriver.2.14.0.0/tools/'} "' exited with code: 127.

The path is incorrect because there's no WINDIR environment variable. But even if that was set it still wouldn't work as this is Windows version of Chrome. I'll create a new build configuration for mono and exclude the whole integration tests project on it.

The project now build correctly and runs unit tests thanks to Travis CI

repo: RaYell/selenium-webdriver-extensions


text:
Update deploy/os_ova_install.py multiple VMs routine

Fixed OVA installer for new FIT configuration system.
Multiple VM deployments now seed off of the 'ovamac' key in the stack config.
@hohene @jimturnquist @johren @larry-dean

Please also remove the stack number check around line 114.  This is no longer necessary if we are not using that ID to compose the MAC.

@johren Do you the line number correct in your comment above.

Yep, just double-checked.  It checks to make sure the stack ID is a number if numvms > 1.  This is no longer necessary with George's change.

Removed number check for stack ID and tested as requested by @johren

authors: gpaulos jimturnquist johren

repo: RackHD/RackHD