text:
Introduce proper KDoc parsing and Formatting

The first version of Ktfmt uses the java-doc formatter from Google-Java-Format with very few forks to make it palatable. However, this will keep us stumbling onto issues. We need instead to figure a clean way to parse KDoc and output them instead.
@cgrushko pointed out that kotlinc can parse that, so this may be the first step to figuring out a good solution.

We now parse KDoc using the built in lexers. There's definitely bugs and issues to resolve, but the results should be satisfactory for the vast majority of code. We will deal with specific issues from now on,

repo: facebookincubator/ktfmt


text:
Return type of bitwise_xxx_agg doesn't match Presto's

Description
Window fuzzer found that the return type of bitwise_xxx_agg in Presto always return BIGINT, while they in Velox return the same type as the input.
Error Reproduction
Executing query plan:
-- Window[partition by [p0, p1] order by [s0 ASC NULLS LAST] w0 := bitwise_xor_agg(ROW["c0"]) RANGE between UNBOUNDED PRECEDING and CURRENT ROW] -> c0:TINYINT, s0:ARRAY<BOOLEAN>, p0:REAL, p1:BIGINT, row_number:BIGINT, w0:TINYINT
  -- Values[1000 rows in 10 vectors] -> c0:TINYINT, s0:ARRAY<BOOLEAN>, p0:REAL, p1:BIGINT, row_number:BIGINT

Relevant logs
I20231219 20:28:21.981456 2392528 WindowFuzzer.cpp:64] ==============================> Started iteration 36 (seed: 1155920780)
I20231219 20:28:21.987654 2392528 AggregationFuzzerBase.cpp:364] Executing query plan:
-- Window[partition by [p0, p1] order by [s0 ASC NULLS LAST] w0 := bitwise_xor_agg(ROW["c0"]) RANGE between UNBOUNDED PRECEDING and CURRENT ROW] -> c0:TINYINT, s0:ARRAY<BOOLEAN>, p0:REAL, p1:BIGINT, row_number:BIGINT, w0:TINYINT
  -- Values[1000 rows in 10 vectors] -> c0:TINYINT, s0:ARRAY<BOOLEAN>, p0:REAL, p1:BIGINT, row_number:BIGINT
I20231219 20:28:22.011268 2394184 Task.cpp:1062] All drivers (1) finished for task test_cursor 37 after running for 22 ms.
I20231219 20:28:22.011299 2394184 Task.cpp:1746] Terminating task test_cursor 37 with state Finished after running for 22 ms.
I20231219 20:28:22.233783 2392528 Writer.cpp:546] Stripe 0: Flush overhead = 30656, data length = 10744, pre flush mem = 448192, post flush mem = 445056. Closing = true
I20231219 20:28:22.234041 2392528 FileSink.cpp:123] closing file: /var/folders/yl/2mzb3q4s0gn62ffjlfbch0w80000gn/T/PrestoTest2551609741947032008/hive_data/tpch/tmp/fuzzer.dwrf,  total size: 11.62KB
/Users/weihe/Development/velox/velox/exec/tests/utils/QueryAssertions.cpp:1063: Failure
Failed
Types of expected and actual results do not match: ROW(TINYINT, ARRAY, REAL, BIGINT, BIGINT, BIGINT) vs. ROW(TINYINT, ARRAY, REAL, BIGINT, BIGINT, TINYINT)
E20231219 20:28:22.403347 2392528 Exceptions.h:69] Line: /Users/weihe/Development/velox/velox/exec/fuzzer/WindowFuzzer.cpp:179, Function:verifyWindow, Expression: assertEqualResults( expectedResult.value(), plan->outputType(), {resultOrError.result}) Velox and reference DB results don't match, Source: RUNTIME, ErrorCode: INVALID_STATE
libc++abi: terminating due to uncaught exception of type facebook::velox::VeloxRuntimeError: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Velox and reference DB results don't match
Retriable: False
Expression: assertEqualResults( expectedResult.value(), plan->outputType(), {resultOrError.result})
Function: verifyWindow
File: /Users/weihe/Development/velox/velox/exec/fuzzer/WindowFuzzer.cpp
Line: 179

@kagamiori The issue here is slightly different. bitwise_xxx functions in Presto have only 1 signature, but in Velox there are multiple signatures. A Fuzzer may create a query plan using bitwise_and_agg(tinyint). When this plan is converted to SQL,  Presto's query planner applies implicit cast to input argument: tynyint -> bigint. Hence, the result in Presto is bigint, not tinyint.
presto:di> show functions like 'bitwise_%_agg';
    Function     | Return Type | Argument Types | Function Type | Deterministic | Description | Variable Arity | Built In | Temporary | Language
-----------------+-------------+----------------+---------------+---------------+-------------+----------------+----------+-----------+----------
 bitwise_and_agg | bigint      | bigint         | aggregate     | true          |             | false          | true     | false     |
 bitwise_or_agg  | bigint      | bigint         | aggregate     | true          |             | false          | true     | false     |
 bitwise_xor_agg | bigint      | bigint         | aggregate     | true          |             | false          | true     | false     |
(3 rows)

vs. Velox
  for (const auto& inputType : {"tinyint", "smallint", "integer", "bigint"}) {
    signatures.push_back(exec::AggregateFunctionSignatureBuilder()
                             .returnType(inputType)
                             .intermediateType(inputType)
                             .argumentType(inputType)
                             .build());
  }

CC: @aditi-pandit @kgpai

source: gharchive/issue

authors: kagamiori mbasmanova

repo: facebookincubator/velox

license_type: permissive

license_source: github-api


text:
ApproxPercentile fails with xxx away from acceptable range

Description
When running with Nan/Inf support in fuzzer approx_percentile seems to fail with following error . PR : https://github.com/facebookincubator/velox/pull/8732 .
Error Reproduction
Try seed : 46435304  in Aggregation fuzzer against the linked PR above.
Relevant logs
I20240212 15:13:07.450784 19839653 AggregationFuzzerBase.cpp:364] Executing query plan:
-- Aggregation[SINGLE a0 := approx_percentile(ROW["c0"],ROW["c1"]) mask: m0] -> a0:ARRAY<DOUBLE>
  -- Values[1000 rows in 10 vectors] -> c0:DOUBLE, c1:ARRAY<DOUBLE>, m0:BOOLEAN
I20240212 15:13:07.453105 19844862 Task.cpp:1085] All drivers (1) finished for task test_cursor 608 after running for 2 ms.
I20240212 15:13:07.453125 19844862 Task.cpp:1769] Terminating task test_cursor 608 with state Finished after running for 2 ms.
I20240212 15:13:07.460280 19844863 Task.cpp:1085] All drivers (3) finished for task test_cursor 609 after running for 3 ms.
I20240212 15:13:07.460305 19844863 Task.cpp:1769] Terminating task test_cursor 609 with state Finished after running for 3 ms.
E20240212 15:13:07.460641 19839653 ApproxPercentileResultVerifier.h:154] approx_percentile(pct: 0.95, accuracy: 0.0133) is more than 0.0133 away from acceptable range of [0.683386, 0.811912]. Difference: 0.138088
E20240212 15:13:07.460682 19839653 Exceptions.h:69] Line: /Users/kpai/src/velox/velox/exec/fuzzer/AggregationFuzzer.cpp:1084, Function:compareEquivalentPlanResults, Expression: verifier->verify(resultOrError.result) Aggregation results failed custom verification, Source: RUNTIME, ErrorCode: INVALID_STATE
I20240212 15:13:07.486898 19839653 AggregationFuzzerBase.cpp:655] Persisted aggregation plans to /tmp/aggregate_fuzzer_repro/velox_aggregationVerifier_nDkFYV/plan_nodes
libc++abi: terminating due to uncaught exception of type facebook::velox::VeloxRuntimeError: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Aggregation results failed custom verification
Retriable: False
Expression: verifier->verify(resultOrError.result)
Function: compareEquivalentPlanResults
File: /Users/kpai/src/velox/velox/exec/fuzzer/AggregationFuzzer.cpp
Line: 1084

Shall we just throw exception if input contains NaN?  There is no well defined order for NaN so there is no way to get a percentile that makes sense if input contains NaN.

I think this is another NaN behavior problem, CC: @bikramSingh91

authors: Yuhta kgpai


text:
Connection Error on Creating Animation

Hello,
For the past three days I have been trying to create animations from my own drawings but to no avail. The constant headache has proved to be a connection error on the last step on the setup process:
python image_to_animation.py drawings/garlic.png garlic_out
I initially ran through the installation commands in Anaconda and have also tried in Ubuntu. With the following error:

I have also tried installing the packages suggested from this post but this has proved to be no help either:

If anyone has any ideas or solutions I would be most grateful.

What about after you've run python image_to_animation.py drawings/garlic.png garlic_out? If you ping the server again at that point, what response does it give?

What about after you've run python image_to_animation.py drawings/garlic.png garlic_out? If you ping the server again at that point, what response does it give?

ah thankyou I havn't tried that, I'll give that a go later and get back to you

What sort of a machine are you running it on? If the Docker container doesn't have enough RAM (16gb, maybe 32gb....) Torchserve can die suddenly when you try to query it. Try giving it more RAM and see if that helps

See #105

Hello guys, I found that if docker_torchserve runs in a CPU to memory ratio of 1:1 env, it will get stuck and consume all CPU and memory. I think it is caused by torchserve self.
I tested the following cases:

VM
Result

2c4g
✅
slow but works

4c8g
✅
bbox: about 10s

8c16g
✅
bbox: about 3s

16c32g
✅
bbox: about 1s

4c4g
❌
stuck

8c8g
❌
stuck

8c10g
❌
stuck

16c16g
❌
stuck

BTW, docker_torchserve works well on my 2c4g dev env. 🤣
docker run --cpus 2 -m 4g -d --name docker_torchserve -p 8080:8080 -p 8081:8081 docker_torchserve

Can confirm that running with docker run --cpus 2 -m 4g -d --name docker_torchserve -p 8080:8080 -p 8081:8081 docker_torchserve also resolves the issue for me. Ran on Win 11 with an Intel i9, 32GB RAM, 12 GB RTX 3080ti.

On mac I have to increase the memory to 13GB to fix this issue. Why does it take so much more memory than win 11 on mac?

authors: Anthot44 benjaminbojko hjessmith thhung yihleego

repo: facebookresearch/AnimatedDrawings

license: MIT


text:
problem with transcoder

showing below error
File "TransCoder/translate.py", line 171, in
translator = Translator(params)
File "TransCoder/translate.py", line 83, in init
encoder, decoder = build_model(self.reloaded_params, self.dico)
File "/content/TransCoder/XLM/src/model/init.py", line 181, in build_model
enc_path, map_location=lambda storage, loc: storage.cuda(params.local_rank))
File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 608, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 787, in _legacy_load
result = unpickler.load()
File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 743, in persistent_load
deserialized_objects[root_key] = restore_location(obj, location)
File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 830, in restore_location
result = map_location(storage, location)
File "/content/TransCoder/XLM/src/model/init.py", line 181, in
enc_path, map_location=lambda storage, loc: storage.cuda(params.local_rank))
File "/usr/local/lib/python3.7/dist-packages/torch/_utils.py", line 71, in _cuda
with torch.cuda.device(device):
File "/usr/local/lib/python3.7/dist-packages/torch/cuda/init.py", line 225, in enter
self.prev_idx = torch.cuda.current_device()
File "/usr/local/lib/python3.7/dist-packages/torch/cuda/init.py", line 432, in current_device
_lazy_init()
File "/usr/local/lib/python3.7/dist-packages/torch/cuda/init.py", line 172, in _lazy_init
torch._C._cuda_init()

Hi,
Your code contains enc_path, map_location=lambda storage, loc: storage.cuda(params.local_rank)) on line 181 in init.py while the current version is different.
https://github.com/facebookresearch/CodeGen/blob/9720a8bdba18552cc499975e1f1fb6a7eca74612/codegen_sources/model/src/model/__init__.py#L181
Could you pull and try again ?

repo: facebookresearch/CodeGen


text:
Normalization values for CLIP models

Thanks for the great work.
I would like to confirm whether the normalization mean (108.32, 116.74, 104.09) for OpenAI clip-based models is correct.
The official normalization values in OpenAI CLIP is (0.48145, 0.45782, 0.40821). In this case, by multiplying 255, the values should be (122.77, 116.74, 104.09).
It's not a big deal though. The model is robust against small changes to normalization values.

Hi @Jazzcharles ,
You are right. 122.77 should be a more correct one. I'd keep the legacy value since re-training the models is beyond my compute resource. Also, the performance is unlikely to change much.
Best,
Yue

authors: Jazzcharles zhaoyue-zephyrus

repo: facebookresearch/LaViLa


text:
Query regarding --use_valedges_as_input

Hi Dr. Zhang,
Hope you are doing good.
I had a query regarding the use of validation edges as input, which is useful in the ogbl-collab dataset as per the OGB rules. I noticed that when the flag --use_valedges_as_input is True, we have the following code snippet:
if args.use_valedges_as_input:
    val_edge_index = split_edge['valid']['edge'].t()
    if not directed:
        val_edge_index = to_undirected(val_edge_index)
    data.edge_index = torch.cat([data.edge_index, val_edge_index], dim=-1)
    val_edge_weight = torch.ones([val_edge_index.size(1), 1], dtype=int)
    data.edge_weight = torch.cat([data.edge_weight, val_edge_weight], 0)

But, since after hypertuning, we are allowed to use val_edges in training, can't we also update the split_edge as:
split_edge['train']['edge'] = torch.cat([data.edge_index, val_edge_index], dim=-1)

I was wondering if SEAL would benefit from this change/if this change makes sense. Any input is greatly appreciated!
Warm Regards,
Paul

Hi Paul! Adding validation edges to the training set is totally feasible and might increase the performance. The reason I didn't add them was because I anyway sampled only a small portion of edges from the training set. Hope that helps!

Hi Dr. Zhang,
Makes sense! Thank you so much for the quick response.
Marking this issue as resolved.

authors: muhanzhang venomouscyanide


text:
What does Embedding Vectors mean?

In #157 , I asked how to access embedding table with indices.
Now I know that.
But, I have new questions about embedding tables.
In #67, "each unique index maps to a single embedding vector"
so Embedding table is mapped corresponding to the number of categorical feature values.
for example,
If a categorical feature means user's nations in any dataset, features may have about 200 country.
Then an embedding table will be consist of about 200 embedding vectors, right?
Question.
If I set the embedding dimension as d = 16, then what does dimensions mean?
each dimension means meaningful value?
or Are the values of each dimension in the embedding vector composed of 16 random values?

A simple overview of embedding table lookups can be found here. Please take a look at it.
M. Naumov, On the dimensionality of embeddings for sparse features and data, CoRR, 2019 https://arxiv.org/pdf/1901.02103.pdf

Thanks for reply @mnaumovfb
I want to ask more questions.
From my understanding, The embedding vector maps each categorical value.
Then, are the vector weights initialized by a random uniform distribution function?
The reason I think so is because I saw in the code that the table weights are initialized by the random function.
(dlrm_s_pytorch.py 268 lines)
W = np.random.uniform(low=-np.sqrt(1/n), high = np.sqrt(1/n), size = (n,m))
If embedding dimension is d,  is the Initial weight of the vector composed of d-random values?
and Can I change the embedding table weight range? (Ignore the other influences)
Sorry to continue asking similar questions.
Thank you in advance.

authors: Jeongmiu mnaumovfb

repo: facebookresearch/dlrm


text:
where is the data pipeline for training?

I do not find the info for training and testing on the kaggle dataset.
there is only one line for dataset creation in data_utils.py
getKaggleCriteoAdData(datafile="<path-to-train.txt>", o_filename=kaggleAdDisplayChallenge_processed.npz")
however, in the dlrm_s_pytorch.py, there is no where can input a dataset.
    parser.add_argument("--data-set", type=str, default="kaggle")  # or terabyte
    parser.add_argument("--raw-data-file", type=str, default="")
    parser.add_argument("--processed-data-file", type=str, default="")

these 3 lines above are useless, since no ref in the code for these parameters.
what is the purpose to publish the source code that is not able to read any dataset?

Please refer to the detailed description in part 2 of the benchmarking section in the README file.
We do not distribute any datasets with this model. We do support interface with Kaggle Display Advertising Challenge Dataset. In order to use it:

Please download the dataset yourself. Then, you will see a raw data file train.txt
Please pass this file to the model using "--raw-data-file=..." command line argument
The model will generate a processed .npz file, which can be used in subsequent runs
Please refer to the ./bench/dlrm_s_criteo_kaggle.sh script for how it is used.

did you Really go through the code???
--raw-data-file only appear once in L428 in the dlrm_s_pytorch.py code below. There is no where really accept the value of this parameter for train.txt file all at. If no raw data input, how can you create .npz file????
https://github.com/facebookresearch/dlrm/blob/d8154a01186420a567ac38a3c940e9ad665d3ea2/dlrm_s_pytorch.py#L428
Other 2 parameters --data-set and --processed-data-file are the same.

also, dlrm_s_criteo_kaggle.sh does not include --raw-data-file by default as well, right?
The readme file is very unstructured.

FYI, I am NOT one of the authors. But let me add a few comments in the meantime.
First of all, I'd like to say I and my colleagues have been completely fine with running DLRM (while the one before refactoring data loader) based on the readme.
I agree that the script could have had --raw-data-file by default but it's also well described in the https://github.com/facebookresearch/dlrm#benchmarking.
For the earlier one, could you take a look at the following to get to data_utils.py?

https://github.com/facebookresearch/dlrm/blob/master/dlrm_s_pytorch.py#L500
https://github.com/facebookresearch/dlrm/blob/master/dlrm_data_pytorch.py#L42
https://github.com/facebookresearch/dlrm/blob/f57a80f5e23a3ae15260342179fcc558d345dece/data_utils.py#L292

Hope this helps meanwhile.
Yongkee

I have added the --raw-data-file option to the benchmarking script in the latest commit. It should pickup either the raw file or pre-processed file automatically.
I consider this issue resolved.

authors: civilman628 insoulity mnaumovfb


text:
Creating a task with the "Task Code" as a number doesn't work as expected.

After creating a task with the task code as a number, and accepting the task, when users want to navigate to the task, it should ideally take us to a page which says "The task owner still needs to activate this task.", but in this case, we are shown the respective page for a millisecond, and taken back to the home page, which I think is unexpected behaviour.
A demonstration is given in the following screen recording of the same issue.
Steps to reproduce:

Create a task proposal with the "Task Code" field as a number
Accept the task as the admin user.
Now try to click on the respective task from your "Tasks" page. It should just take you back to the homepage.

This seems to happen only for a purely numeric "Task Code" and not for an alphanumeric "Task Code"
https://user-images.githubusercontent.com/48560219/135757335-d98f116f-b7d6-44dc-a1fd-0c8b6fac7c61.mov

Thanks for pointing this out! It would also be good to have a little explanation for what task code actually is. Maybe something like: this is a short string that will be the url for your task; e.g, "nli" for "Natural Language Inference"

The fix is probably to disallow any task code that doesn't contain at least one letter?

Thanks for pointing this out! It would also be good to have a little explanation for what task code actually is. Maybe something like: this is a short string that will be the url for your task; e.g, "nli" for "Natural Language Inference"

Yeah, that makes a lot of sense

authors: TristanThrush anandrajaram21 douwekiela

repo: facebookresearch/dynabench


text:
Distributed training fails with criterion having logging_outputs_can_be_summed to False

🐛 Bug
My understanding is that now criterions containing parameters are supported (as of https://github.com/pytorch/fairseq/commit/ba5f829f64d2adf9de4502dab94abf0a1a54106b).
I tried implementing a custom criterion with parameters and everything looked fine when testing on a single GPU.
Moving to a distributed training on 8 GPUs the training got stuck at the very beginning and never went on.
Debugging the issue, I noticed that all the processes started working on all the GPUs, rather than stick to the one they are assigned to.
Unfortunately, so far I have not been able to understand the root cause of the issue, though.
EDIT: further investigation showed that the problem is not related to the criterion having parameters but to the criterion having logging_outputs_can_be_summed returning False.
To Reproduce
Add a very simple loss containing an attribute like:
@register_criterion("test_loss")
class TestLoss(FairseqCriterion):
    def __init__(self, args, task):
        super().__init__(args, task)
        self.real_criterion = TestLoss.build_real_criterion(args, task)

    @staticmethod
    def build_real_criterion(args, task):
        saved_criterion = args.criterion
        args.criterion = args.underlying_criterion
        assert saved_criterion != args.underlying_criterion
        underlying_criterion = task.build_criterion(args)
        args.criterion = saved_criterion
        return underlying_criterion

    @staticmethod
    def add_args(parser):
        parser.add_argument('--underlying-criterion', type=str, metavar='VAL', required=True,
                            help='underlying criterion to use for the model output loss')

    def forward(self, model, sample, reduce=True, log_probs=True):
        return self.real_criterion(model, sample, reduce=reduce)

    @staticmethod
    def reduce_metrics(logging_outputs):
        """Aggregate logging outputs from data parallel training."""
        loss_sum = utils.item(sum(log.get('loss', 0) for log in logging_outputs))
        sample_size = utils.item(sum(log.get('sample_size', 0) for log in logging_outputs))
        metrics.log_scalar('loss', loss_sum / sample_size / math.log(2), sample_size, round=3)

Train on multiple GPUs (in my case, 8).
The training gets stuck at 0% and never starts. The output of nvidia-smi contains:
+------------------------------------------------------------+
| Processes:                                      GPU Memory |
|  GPU       PID   Type   Process name            Usage      |
|============================================================|
|    0      2632      C   .../fairseq0.9/bin/python  1062MiB |
|    0      2633      C   .../fairseq0.9/bin/python   310MiB |
|    0      2634      C   .../fairseq0.9/bin/python   310MiB |
|    0      2635      C   .../fairseq0.9/bin/python   310MiB |
|    0      2636      C   .../fairseq0.9/bin/python   310MiB |
|    0      2637      C   .../fairseq0.9/bin/python   310MiB |
|    0      2638      C   .../fairseq0.9/bin/python   310MiB |
|    0      2639      C   .../fairseq0.9/bin/python   310MiB |
|    1      2632      C   .../fairseq0.9/bin/python   310MiB |
|    1      2633      C   .../fairseq0.9/bin/python  1016MiB |
|    1      2634      C   .../fairseq0.9/bin/python   310MiB |
|    1      2635      C   .../fairseq0.9/bin/python   310MiB |
|    1      2636      C   .../fairseq0.9/bin/python   310MiB |
|    1      2637      C   .../fairseq0.9/bin/python   310MiB |
|    1      2638      C   .../fairseq0.9/bin/python   310MiB |
|    1      2639      C   .../fairseq0.9/bin/python   310MiB |
|    2      2632      C   .../fairseq0.9/bin/python   310MiB |
|    2      2633      C   .../fairseq0.9/bin/python   310MiB |
|    2      2634      C   .../fairseq0.9/bin/python  1046MiB |
|    2      2635      C   .../fairseq0.9/bin/python   310MiB |
|    2      2636      C   .../fairseq0.9/bin/python   310MiB |
|    2      2637      C   .../fairseq0.9/bin/python   310MiB |
|    2      2638      C   .../fairseq0.9/bin/python   310MiB |
|    2      2639      C   .../fairseq0.9/bin/python   310MiB |
|    3      2632      C   .../fairseq0.9/bin/python   310MiB |
|    3      2633      C   .../fairseq0.9/bin/python   310MiB |
|    3      2634      C   .../fairseq0.9/bin/python   310MiB |
|    3      2635      C   .../fairseq0.9/bin/python  1048MiB |
|    3      2636      C   .../fairseq0.9/bin/python   310MiB |
|    3      2637      C   .../fairseq0.9/bin/python   310MiB |
|    3      2638      C   .../fairseq0.9/bin/python   310MiB |
|    3      2639      C   .../fairseq0.9/bin/python   310MiB |
|    4      2632      C   .../fairseq0.9/bin/python   310MiB |
|    4      2633      C   .../fairseq0.9/bin/python   310MiB |
|    4      2634      C   .../fairseq0.9/bin/python   310MiB |
|    4      2635      C   .../fairseq0.9/bin/python   310MiB |
|    4      2636      C   .../fairseq0.9/bin/python  1094MiB |
|    4      2637      C   .../fairseq0.9/bin/python   310MiB |
|    4      2638      C   .../fairseq0.9/bin/python   310MiB |
|    4      2639      C   .../fairseq0.9/bin/python   310MiB |
|    5      2632      C   .../fairseq0.9/bin/python   310MiB |
|    5      2633      C   .../fairseq0.9/bin/python   310MiB |
|    5      2634      C   .../fairseq0.9/bin/python   310MiB |
|    5      2635      C   .../fairseq0.9/bin/python   310MiB |
|    5      2636      C   .../fairseq0.9/bin/python   310MiB |
|    5      2637      C   .../fairseq0.9/bin/python  1022MiB |
|    5      2638      C   .../fairseq0.9/bin/python   310MiB |
|    5      2639      C   .../fairseq0.9/bin/python   310MiB |
|    6      2632      C   .../fairseq0.9/bin/python   310MiB |
|    6      2633      C   .../fairseq0.9/bin/python   310MiB |
|    6      2634      C   .../fairseq0.9/bin/python   310MiB |
|    6      2635      C   .../fairseq0.9/bin/python   310MiB |
|    6      2636      C   .../fairseq0.9/bin/python   310MiB |
|    6      2637      C   .../fairseq0.9/bin/python   310MiB |
|    6      2638      C   .../fairseq0.9/bin/python  1064MiB |
|    6      2639      C   .../fairseq0.9/bin/python   310MiB |
|    7      2632      C   .../fairseq0.9/bin/python   310MiB |
|    7      2633      C   .../fairseq0.9/bin/python   280MiB |
|    7      2634      C   .../fairseq0.9/bin/python   277MiB |
|    7      2635      C   .../fairseq0.9/bin/python   293MiB |
|    7      2636      C   .../fairseq0.9/bin/python   266MiB |
|    7      2637      C   .../fairseq0.9/bin/python   270MiB |
|    7      2638      C   .../fairseq0.9/bin/python   310MiB |
|    7      2639      C   .../fairseq0.9/bin/python  1058MiB |
+------------------------------------------------------------+

I also tried changing the --num-workers to 0, changing --ddp-backend but the behavior didn't change.
Code sample
See section above.
Expected behavior
The training running fine in a distributed environment too.
Environment

fairseq Version (e.g., 1.0 or master): master (updated at 6a5181509aa1fa7d260985157e77211753da544b)
PyTorch Version (e.g., 1.0): 1.4
OS (e.g., Linux): ubuntu
How you installed fairseq (pip, source): source
Build command you used (if compiling from source): python setup.py install -e
Python version: 3.6
CUDA/cuDNN version: 10.0
GPU models and configuration: 8 K80
Any other relevant information:

Additional context
The bug has been introduced in https://github.com/pytorch/fairseq/commit/4cae68037d4ded8fdc6d9c13edbfbbcb21e39f7f.

cc @jcai1 @okhonko sorry for pinging you, but I saw you worked on the commit I mentioned, so you may know more about this. Thanks.

Are you using a custom dataset? Usually this happens when you have dataloading code that touches the GPU (or uses pinned memory), causing PyTorch to initialize itself on the GPU in the subprocess.
Does it work if you use a standard criterion? What is the rest of your code/model doing?

@myleott thanks for your answer. It works if using any criterion without parameters (eg. cross_entropy), but using the same criterion as "underlying criterion" in the repro above causes the issue.
The rest is just a normal translation task with transformer model. The command I am executing is:
python train.py $data_bin \
    -t $lang -s en \
    --clip-norm 20  \
    --max-source-positions 8000 --max-target-positions 8000 \
    --max-tokens 4500 \
    --save-dir $save_dir \
    --max-epoch 100 \
    --lr 1e-3 \
    --min-lr 1e-08 \
    --lr-scheduler inverse_sqrt \
    --arch transformer \
    --dropout 0.3 \
    --warmup-updates 4000 --warmup-init-lr 1e-7 \
    --update-freq 4 \
    --optimizer adam --adam-betas '(0.9, 0.98)' \
    --task translation \
    --skip-invalid-size-inputs-valid-test \
    --sentence-avg \
    --user-dir $my_ext_dir --criterion test_loss--underlying-criterion cross_entropy

With this command and the test_loss mentioned in the issue description, the problem can be reproduced consistently. So the only custom part is the criterion (which is actually very similar to the composite_loss, so I expect that to be affected by this issue too). Hence, I am confident this is a bug, not a question about how to extend fairseq, but unfortunately, I have not yet been able to understand the root cause. If there is anything I can do to further investigate this, I am very open to any suggestion. Thanks.

Ok, so I managed to understand more about the problem. The issue is caused by https://github.com/pytorch/fairseq/commit/4cae68037d4ded8fdc6d9c13edbfbbcb21e39f7f.
The problem happens only with criterions which don't set logging_outputs_can_be_summed to True (the default is False).  Hence I will update the issue description as the problem is not with parameters but the difference was related to logging_outputs_can_be_summed.
I'll try and find a fix in the next days, but I am not very familiar with this part of code, so if anyone else comes up with a solution, then please go ahead with it as it may take some time for me and I am not sure I would come up with the cleanest possible solution.
I am switching back to label bug as I think it is evident that this is a bug. Thanks.

I see only now that there is already https://github.com/pytorch/fairseq/issues/1826 which is probably the same issue. I am closing this in favor of it. Thanks @myleott .

@mgaido91 Sorry for asking. But since you have a good understanding of criterions.
In the criterion we return the loss, sample_size, logging_outputs.
What is we return loss/sample_size, 1, logging_outputs only i.e., divide the loss by sample_size and just pass 1 in its place?

I think it is not a good idea, as losses are aggregated later among all gpus, so you should avoid that

Why is it not a good idea?
Lets say we have 2 gpus such that 4 and 3 training examples on each.
why not take a mean and then pass instead of passing the loss as a sum?

because you should sum their losses and divide by 7 instead of dividing the first by 4 and the second by 3

@mgaido91 I request you to pelase share any reading material on why this is preferred over the other.
Furthermore, if I do not take a mean before then the loss is rescaled heavily to 4 in the first iteration.

authors: mgaido91 myleott raotnameh

repo: facebookresearch/fairseq


text:
How could I find the Matterport3D (MP3D) dataset used in habitat?

Habitat-Sim version
v0.1.7
❓ Questions and Help
How could I find the Matterport3D (MP3D) dataset used in habitat?
I couldn't find the download_mp.py, and has finished sending email to matterport3d@googlegroups.com.
Now I am waiting email reply, but how can I recognize which dataset is used by habitat?

Hey @GuoPingPan,
There are many scene datasets available for use in Habitat.
The first place to start is DATASETS.md. All download instructions and processes are listed there, including MP3D and the newer HM3D.
If you have specific questions about some step not working feel free to open a new issue.

repo: facebookresearch/habitat-sim


text:
Update Magnum with new scene tools, bindings and plugin additions

Motivation and Context
Continuation of #2100, with another ~10 days and >10 thousand lines of changes on the Magnum side. A large batch of features got finished and merged to master this time, and some breaking changes were needed to keep the API organized -- see below.
What's new:

Fixed some minor build issues with GCC 13
New expanded() and collapsed() APIs on multidimensional strided (bit) array views, similar to what Numpy's reshape() does
New pixelFormat() helper API for conveniently changing component count or sRGB encoding of an arbitrary PixelFormat
(Contributed by a 3rd party) The UfbxImporter plugin now has complete support for FBX skins and animations, allowing @0mdc to replace Assimp's broken FBX loading
New simplifyFailEmpty option in MeshOptimizerSceneConverter that'll make it fail in case simplification produces an empty mesh. By default it produces an empty MeshData, which can't be used to produce a valid glTF file. Related is also a minor fix to a bug that caused the plugin to trigger an assertion in presence of an empty mesh.
New forceBitDepth option for StbImageImporter and PngImporter that allows opening 16-bit images as 8-bit and vice versa. Added to work around a limitation of the Basis library which can only deal with 8-bit-per-pixel images.

New tools, especially of interest to @jturner65:

SceneTools::parentsBreadthFirst() and SceneTools::childrenDepthFirst() utilities for returning a scene hierarchy in various orders depending on desired use case
SceneTools::combineFields() that makes it possible to conveniently compose a SceneData out of loose data arrays.
SceneTools::filterFields(), SceneTools::filterOnlyFields() and SceneTools::filterExceptFields() for removing a particular field (such as mesh assignments, or transformations) from a scene as a whole. These are similar to existing attribute filtering tools for meshes and materials.
SceneTools::filterFieldEntries() for removing individual field entries (for example, all references to a particular flowerpot) from the scene
SceneTools::filterObjects() for removing individual objects (for example, openings objects) from the scene
A unified set of MeshTools::copy(), MaterialTools::copy() and SceneTools::copy() helpers for the rare cases where a mesh, material or scene data instance needs to be copied
Except for combineFields(), all of the above are also exposed to Python in the scenetools and meshtools modules, together with an ability to export scenes through trade.AbstractSceneConverter.

Additions to the magnum-sceneconverter utility, which you however usually use via the prebuilt tools instead of through Habitat:

New --passthrough-on-mesh-converter-failure and --passthrough-on-image-converter-failure options, making it possible to fall back to the original data for a particular failed -M or -P conversion
New --prefer option, adding an ability to choose an alternative plugin implementation, such as SpngImporter instead of the default PngImporter that may be as much as 2x faster
New --set option, allowing to set global plugin options like overriding bit depth of imported images, quality of exported JPEGs or Basis compression options. This used to be only partially possible before, done in a hacky way with [imageConverter] options in GltfSceneConverter, and --set is now the preferred way.
New --object-hierarchy option for --info-objects that'll visualize the object hierarchy by indenting the output. Useful for discovering object groupings or redundant nodes in a scene.

:warning: Breaking changes: :warning:

Trade::LightData::Type is now Trade::LightType so it's no longer needed to include the whole class for an enum declaration
SceneTools::flattenTransformationHierarchy3D() is renamed to SceneTools::absoluteFieldTransformations3D(), same change is done for the scenetools.flatten_transformation_hierarchy3d() Python binding that's now scenetools.absolute_field_transformations3d()
MeshTools::owned() is now MeshTools::copy()

The old C++ APIs are still present but marked as deprecated and thus causing compiler warnings. For the Python bindings however it's a hard change with no fallback. I updated the affected code here, some changes are needed in habitat-lab as well, but there I unfortunately have no easy way to verify the changes. @aclegg3 can you handle the updates on the habitat-lab side? Thank you.
How Has This Been Tested
:green_apple:

This now includes also a GLFW application EGL fix for @eundersander and https://github.com/facebookresearch/habitat-sim/pull/2116.

Assuming the tests pass on https://github.com/facebookresearch/habitat-lab/pull/1356 against this branch I think we can merge both.

source: gharchive/pull-request


text:
Move Conda Nightly builds to Docker

Motivation and Context
https://github.com/facebookresearch/habitat-sim/pull/663 uses Ubuntu 16.04 as base OS to build conda builds. That limits conda build compatibility compare to using Centos7 as base OS.

Added install Docker step to Nightly builds job
Added "mesa-libGL-devel" install to conda Docker to improver build with Display
Moved build conda env creation to docker build phase

How Has This Been Tested

Checked that nightly builds are working good on Colab

Types of changes

[ ] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have completed my CLA (see CONTRIBUTING)
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

Building the conda viewer still doesn't work. :( If just building the headless conda binaries unblock us, that's fine with me. We can add the viewer back in a separate PR.

We should be good to go here.


text:
Remove no_ and keep_ checkpoint flags

Patch Description
Removed most of the no_* and keep_* flags and cleaned up their logic.
Testing steps
I did have some issues with setting up an environment to test this.
In the end I tested with the opt_baselines script and checked if different save_intervals and save_interval_updates  correctly create all the checkpoints:
python3 metaseq/launcher/opt_baselines.py --prefix train.8m --model-size 8m --checkpoints-dir ./test-checkpoint --tensorboard-logdir ./test-checkpoint --num-trials 1 --azure --num-gpus 4 --num-nodes 1 --seed 1 --circleci --local --disable-validation --max-epoch 5 --max-update 200

renamed --save-interval to --save-interval-epochs, otherwise it will be quite confusing with the keep flags as well.

The final flags are now:
save_interval_epochs
save_interval_updates
keep_last_epochs
keep_last_updates

tested multiple combinations of them, as well as --benchmark.

It looks like this diff broke metaseq-internal scripts: metaseq_internal/fb_sweep/sweep_openlm_baselines.py

Ah, some sweep scripts were updated (https://github.com/fairinternal/metaseq-internal/pull/431) but not all.  I just pinned the post in our internal workplace group notifying of the breaking change to sweep scripts - hopefully that can help alert of these changes in this future!

It looks like this diff broke metaseq-internal scripts: metaseq_internal/fb_sweep/sweep_openlm_baselines.py

This is now fixed in https://github.com/fairinternal/metaseq-internal/pull/453

authors: Xirider suchenzang vesko-st

repo: facebookresearch/metaseq


text:
Changing the events.peek() function to show sky image in RA/DEC and event rate

changing peek() to show image in RA/DEC and event rate vs time.
Fixed the double plot issue in jupyter for the peek function only. Double plot is due to return fig in plot routine.

Aha, here it is
:-)
You made the pull request against your own fork at https://github.com/facero/gammapy
Can you please make a pull request against the main Gammapy repo at https://github.com/gammapy/gammapy instead?
You can re-use the same branch, and just make a new pull request and close this one.

Some comments:

Can you remove the plt.show and return fig completely? You currently have it commented out, but I think we know what we want now and can just clean that out.
Rename plot_imageRADEC to plot_image_radec or plot_skyimage (basically, PEP8 style guide wins) :-)

For the sky image, maybe calling SkyImage.empty and to put a local TAN projection, and SkyImage.fill_events and SkyImage.plot would be better?
So it could work like that:
events = EventList.read('...')
events.counts_image(<options?>).plot(<options>)

where the counts_image method would call something like
def counts_image(<options?>):
    image = events._empty_image(<options?>)
    image.fill(self)
    return image

So basically, it would be a convenience, re-exposing the existing methods from SkyCoord on EventList, so that if you have an eventlist in hand, it's quick to get a counts image and plot.
I haven't thought much whether this is a good idea, or if what you do here (re-implementing histogramming, not really having a sky image with a WCS) is better for some reason.
@facero - Thoughts?

authors: cdeil facero

repo: facero/gammapy


text:
is write-file-webpack-plugin still needed?

I see you're using compiler memoryfs to access files...

No. it's to show u what chunks you are getting in development.


text:
Error: filesystem error: copy_file "/factorio/temp/currently-playing/level.dat" failed: Permission denied

Hi, I'm getting persistent permission errors when trying to run the Factorio server on my Macbook.
This is the Dockerfile I'm using:
services:

  factorio-server:
    image: factoriotools/factorio:${FACTORIO_VERSION}
    container_name: "factorio-server"
    restart: "no"
    ports:
      - "34197:34197/udp" # factorio
      - "27015:27015/tcp" # rcon
    volumes:
      - "./factorio-server:/factorio"
    environment:
      PUID: ${FACTORIO_SERVER_USER_ID}
      PGID: ${FACTORIO_SERVER_GROUP_ID}

And the .env file:
FACTORIO_SERVER_GROUP_ID=20
FACTORIO_SERVER_USER_ID=502
FACTORIO_VERSION=2.0.16

However, I can't seem to get this working at all. I've tried owning the ./factorio-server/ dir with the Factorio uid:845 user (and removing the custom PUID and PGID), or as my current user, but I can't figure out the necessary combination...
I've deleted the files and container, and restarted my machine.
Versions

factoriotools/factorio version 2.0.16
Docker Compose version 2.29.2
colima version 0.8.0

Logs
2024-11-16T11:44:34.485784410Z +++ readlink -f /docker-entrypoint.sh
2024-11-16T11:44:34.485997655Z ++ dirname /docker-entrypoint.sh
2024-11-16T11:44:34.486319107Z + INSTALLED_DIRECTORY=/
2024-11-16T11:44:34.487166424Z + FACTORIO_VOL=/factorio
2024-11-16T11:44:34.487177174Z + LOAD_LATEST_SAVE=true
2024-11-16T11:44:34.487182298Z + GENERATE_NEW_SAVE=false
2024-11-16T11:44:34.487185965Z + PRESET=
2024-11-16T11:44:34.487189673Z + SAVE_NAME=
2024-11-16T11:44:34.487193048Z + BIND=
2024-11-16T11:44:34.487196631Z + CONSOLE_LOG_LOCATION=
2024-11-16T11:44:34.487199923Z + mkdir -p /factorio
2024-11-16T11:44:34.487274422Z + mkdir -p /factorio/saves
2024-11-16T11:44:34.487767412Z + mkdir -p /factorio/config
2024-11-16T11:44:34.488901848Z + mkdir -p /factorio/mods
2024-11-16T11:44:34.490234988Z + mkdir -p /factorio/scenarios
2024-11-16T11:44:34.491749416Z + mkdir -p /factorio/script-output
2024-11-16T11:44:34.492794145Z + [[ ! -f /factorio/config/rconpw ]]
2024-11-16T11:44:34.492808353Z + [[ ! -f /factorio/config/server-settings.json ]]
2024-11-16T11:44:34.492813312Z + [[ ! -f /factorio/config/map-gen-settings.json ]]
2024-11-16T11:44:34.492817353Z + [[ ! -f /factorio/config/map-settings.json ]]
2024-11-16T11:44:34.492824061Z ++ find -L /factorio/saves -iname '*.tmp.zip' -mindepth 1
2024-11-16T11:44:34.494663233Z ++ wc -l
2024-11-16T11:44:34.494676774Z + NRTMPSAVES=0
2024-11-16T11:44:34.494680816Z + [[ 0 -gt 0 ]]
2024-11-16T11:44:34.494992268Z + [[ '' == \t\r\u\e ]]
2024-11-16T11:44:34.495214639Z + //docker-dlc.sh
2024-11-16T11:44:34.495357219Z Enable mod elevated-rails for DLC
2024-11-16T11:44:34.518520551Z Enable mod quality for DLC
2024-11-16T11:44:34.544449285Z Enable mod space-age for DLC
2024-11-16T11:44:34.579877664Z + EXEC=
2024-11-16T11:44:34.580392362Z ++ id -u
2024-11-16T11:44:34.580868770Z + [[ 0 == 0 ]]
2024-11-16T11:44:34.580934435Z + usermod -o -u 845 factorio
2024-11-16T11:44:34.581720836Z usermod: no changes
2024-11-16T11:44:34.581816167Z + groupmod -o -g 845 factorio
2024-11-16T11:44:34.584883231Z + chown -R factorio:factorio /factorio
2024-11-16T11:44:34.586898400Z + EXEC='runuser -u factorio -g factorio --'
2024-11-16T11:44:34.586943482Z + [[ -f /bin/box64 ]]
2024-11-16T11:44:34.587015564Z + EXEC='runuser -u factorio -g factorio -- /bin/box64'
2024-11-16T11:44:34.587031564Z + sed -i '/write-data=/c\write-data=\/factorio/' /opt/factorio/config/config.ini
2024-11-16T11:44:34.587988586Z ++ find -L /factorio/saves -iname '*.zip' -mindepth 1
2024-11-16T11:44:34.588451619Z ++ wc -l
2024-11-16T11:44:34.588981108Z + NRSAVES=0
2024-11-16T11:44:34.588994108Z + [[ false != true ]]
2024-11-16T11:44:34.589033107Z + [[ 0 == 0 ]]
2024-11-16T11:44:34.589042732Z + GENERATE_NEW_SAVE=true
2024-11-16T11:44:34.589050607Z + SAVE_NAME=_autosave1
2024-11-16T11:44:34.589112481Z + [[ true == true ]]
2024-11-16T11:44:34.589121939Z + [[ -z _autosave1 ]]
2024-11-16T11:44:34.589128980Z + [[ -f /factorio/saves/_autosave1.zip ]]
2024-11-16T11:44:34.589204979Z + [[ ! -z '' ]]
2024-11-16T11:44:34.589238478Z + runuser -u factorio -g factorio -- /bin/box64 /opt/factorio/bin/x64/factorio --create /factorio/saves/_autosave1.zip --map-gen-settings /factorio/config/map-gen-settings.json --map-settings /factorio/config/map-settings.json
2024-11-16T11:44:34.912607467Z    0.001 2024-11-16 11:44:34; Factorio 2.0.16 (build 80214, linux64, headless, space-age)
2024-11-16T11:44:34.913803527Z    0.002 Operating system: Linux
2024-11-16T11:44:34.914374599Z    0.003 Program arguments: "/opt/factorio/bin/x64/factorio" "--create" "/factorio/saves/_autosave1.zip" "--map-gen-settings" "/factorio/config/map-gen-settings.json" "--map-settings" "/factorio/config/map-settings.json"
2024-11-16T11:44:34.914483138Z    0.003 Config path: /opt/factorio/config/config.ini
2024-11-16T11:44:34.916418017Z    0.004 Read data path: /opt/factorio/data
2024-11-16T11:44:34.916435725Z    0.004 Write data path: /factorio [220900/948584MB]
2024-11-16T11:44:34.916442391Z    0.004 Binaries path: /opt/factorio/bin
2024-11-16T11:44:34.930879438Z    0.019 System info: [CPU:                             Box64 on - @1000 MHz, 2 cores, RAM: 1959 MB]
2024-11-16T11:44:34.931352511Z    0.020 Environment: DISPLAY=<unset> WAYLAND_DISPLAY=<unset> DESKTOP_SESSION=<unset> XDG_SESSION_DESKTOP=<unset> XDG_CURRENT_DESKTOP=<unset> SDL_VIDEODRIVER=<unset> __GL_FSAA_MODE=<unset> __GL_LOG_MAX_ANISO=<unset> __GL_SYNC_TO_VBLANK=<unset> __GL_SORT_FBCONFIGS=<unset> __GL_YIELD=<unset>
2024-11-16T11:44:34.940323375Z    0.029 Running in headless mode
2024-11-16T11:44:34.940598911Z    0.029 Audio is disabled
2024-11-16T11:44:34.947550814Z    0.036 Info ModManager.cpp:434: FeatureFlag expansion-shaders = true
2024-11-16T11:44:34.947832017Z    0.036 Info ModManager.cpp:434: FeatureFlag freezing = true
2024-11-16T11:44:34.947957890Z    0.037 Info ModManager.cpp:434: FeatureFlag quality = true
2024-11-16T11:44:34.948042930Z    0.037 Info ModManager.cpp:434: FeatureFlag rail-bridges = true
2024-11-16T11:44:34.948147928Z    0.037 Info ModManager.cpp:434: FeatureFlag segmented-units = true
2024-11-16T11:44:34.948259717Z    0.037 Info ModManager.cpp:434: FeatureFlag space-travel = true
2024-11-16T11:44:34.948325924Z    0.037 Info ModManager.cpp:434: FeatureFlag spoiling = true
2024-11-16T11:44:34.968766018Z    0.057 Loading mod core 0.0.0 (data.lua)
2024-11-16T11:44:34.992095179Z    0.081 Loading mod base 2.0.16 (data.lua)
2024-11-16T11:44:35.195975315Z    0.285 Loading mod elevated-rails 2.0.16 (data.lua)
2024-11-16T11:44:35.236464085Z    0.325 Loading mod quality 2.0.16 (data.lua)
2024-11-16T11:44:35.260749896Z    0.349 Loading mod space-age 2.0.16 (data.lua)
2024-11-16T11:44:35.577976379Z    0.666 Loading mod base 2.0.16 (data-updates.lua)
2024-11-16T11:44:35.605558256Z    0.694 Loading mod quality 2.0.16 (data-updates.lua)
2024-11-16T11:44:35.654925990Z    0.743 Loading mod space-age 2.0.16 (data-updates.lua)
2024-11-16T11:44:35.814618815Z    0.903 Checksum for core: 1966913147
2024-11-16T11:44:35.815074891Z    0.903 Checksum of base: 3431688266
2024-11-16T11:44:35.815253596Z    0.904 Checksum of kafkatorio-events: 0
2024-11-16T11:44:35.815338136Z    0.904 Checksum of elevated-rails: 2929976872
2024-11-16T11:44:35.815477633Z    0.904 Checksum of quality: 3484764141
2024-11-16T11:44:35.815586173Z    0.904 Checksum of space-age: 3982937588
2024-11-16T11:44:36.378991307Z    1.467 Prototype list checksum: 3872509593
2024-11-16T11:44:36.427215357Z    1.516 Info PlayerData.cpp:68: Local player-data.json unavailable
2024-11-16T11:44:36.427386063Z    1.516 Info PlayerData.cpp:73: Cloud player-data.json unavailable
2024-11-16T11:44:36.439726695Z    1.528 Factorio initialised
2024-11-16T11:44:36.442473817Z    1.531 Info Main.cpp:792: Creating new map /factorio/saves/_autosave1.zip
2024-11-16T11:44:36.461797670Z    1.550 Blueprint storage "blueprint-storage-2.dat" was not found, trying to load previous version storage "blueprint-storage.dat"
2024-11-16T11:44:39.946991007Z Error: filesystem error: copy_file "/factorio/temp/currently-playing/level.dat" to "/factorio/temp/currently-playing/level-init.dat" failed: Permission denied [/factorio/temp/currently-playing/level.dat] [/factorio/temp/currently-playing/level-init.dat]
2024-11-16T11:44:39.947019381Z    5.036 Goodbye

For investigation purposes I tried running sudo chown -R 845:845 ./factorio-server and sudo chmod -R 775 ./factorio-server, but I see a different failure.
2024-11-16T12:38:11.819830351Z +++ readlink -f /docker-entrypoint.sh
2024-11-16T12:38:11.822470441Z ++ dirname /docker-entrypoint.sh
2024-11-16T12:38:11.824062278Z + INSTALLED_DIRECTORY=/
2024-11-16T12:38:11.825827116Z + FACTORIO_VOL=/factorio
2024-11-16T12:38:11.825862241Z + LOAD_LATEST_SAVE=true
2024-11-16T12:38:11.825879991Z + GENERATE_NEW_SAVE=false
2024-11-16T12:38:11.825900866Z + PRESET=
2024-11-16T12:38:11.825906241Z + SAVE_NAME=
2024-11-16T12:38:11.825910574Z + BIND=
2024-11-16T12:38:11.825915116Z + CONSOLE_LOG_LOCATION=
2024-11-16T12:38:11.825918741Z + mkdir -p /factorio
2024-11-16T12:38:11.826225992Z + mkdir -p /factorio/saves
2024-11-16T12:38:11.827173827Z + mkdir -p /factorio/config
2024-11-16T12:38:11.828744331Z + mkdir -p /factorio/mods
2024-11-16T12:38:11.829433166Z + mkdir -p /factorio/scenarios
2024-11-16T12:38:11.830623503Z + mkdir -p /factorio/script-output
2024-11-16T12:38:11.833381843Z + [[ ! -f /factorio/config/rconpw ]]
2024-11-16T12:38:11.833590968Z + [[ ! -f /factorio/config/server-settings.json ]]
2024-11-16T12:38:11.833744469Z + [[ ! -f /factorio/config/map-gen-settings.json ]]
2024-11-16T12:38:11.833866261Z + [[ ! -f /factorio/config/map-settings.json ]]
2024-11-16T12:38:11.834243928Z ++ wc -l
2024-11-16T12:38:11.834371679Z ++ find -L /factorio/saves -iname '*.tmp.zip' -mindepth 1
2024-11-16T12:38:11.839082149Z + NRTMPSAVES=0
2024-11-16T12:38:11.839303066Z + [[ 0 -gt 0 ]]
2024-11-16T12:38:11.839431316Z + [[ '' == \t\r\u\e ]]
2024-11-16T12:38:11.839537775Z + //docker-dlc.sh
2024-11-16T12:38:11.841306987Z Enable mod elevated-rails for DLC
2024-11-16T12:38:11.841609613Z //docker-dlc.sh: line 17: /factorio/mods/mod-list.json.tmp: Permission denied

Did you manage to resolve this? I transferred my server to another device and now I'm getting the exact same error

This looks like a permission issue

This IS a permission issue, but it should not occur because even following the examples in the documentation causes this problem

authors: Fank aSemy icheered

repo: factoriotools/factorio-docker


text:
🛑 acmcimisionesgobar is down

In 69f2dc2, acmcimisionesgobar (https://acmci.misiones.gob.ar/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: acmcimisionesgobar is back up in 574bf20.


text:
Migrate Sieve of Eratosthenes and add unit tests

Move implementation to header file
Add unit tests as cpp file
Update readme
Update build script
Remove old file

@faheel I had to make a few decisions about how to migrate this over so I'll leave the PR open for a day in case you'd like to have a look before merge. Main change was returning primes in a vector rather than sending them to stdout.

repo: faheel/Algos

license: mit

license_source: bigquery


text:
Incorrect Unicode symbols on first login

Instead of the straight lines marking the border of the login window, I'm getting other characters instead.
＝ìììììììììììììììᶇ
ᶉ                ᶉ
ᶉ                ᶉ
ᶉ                ᶉ
¥ìììììììììììììììì』

However, the odd thing is this only happens on first boot. If I log in and back out, the characters will render correctly. I haven't been able to track down why this odd behavior is occurring.
I have scoured the closed issues looking for solutions and nothing has helped so far. I've tried setting the locale in /etc/environment, I've tried changing the language in /etc/ly/config.ini.
System: Fedora 36

Started to have the same issue after I configured Silent Boot. After messing around I found out that if remove FONT from /etc/vconsole.conf it is working correctly, but then you are missing your desired font.

authors: quylaa yavorski

repo: fairyglade/ly

license: WTFPL


text:
Getting started fixes (typos / new graphic)

What type of PR is this?
/kind bug
/kind cleanup
Any specific area of the project related to this PR?
/area documentation
What this PR does / why we need it:

fixes a typo ('and' should have been 'a')
updates the architecture graphic which had a black mark on it
corrected 'falco sidekick' to 'falcosidekick' and added a link to the project

Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Issif, mikegcoleman
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

content/OWNERS [Issif]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 6084c3bdd3c5aeeb847e92fe2db4e187276f8d4b

authors: mikegcoleman poiana

repo: falcosecurity/falco-website


text:
falco community survey blog

images and copy for summary of community survey
What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind user-interface
/kind content

/kind translation

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area blog

/area documentation

/area videos

What this PR does / why we need it:
copy editing
Which issue(s) this PR fixes:
new content summary for community survey

Fixes #
Special notes for your reviewer:
falco community survey blog
copy + images
Signed-off-by: jsalinas29 salinas.jacque@gmail.com

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: jsalinas29
Once this PR has been reviewed and has the lgtm label, please assign vjjmiras for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

content/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.
:memo: Please follow instructions in the contributing guide to update your commits with the DCO
Full details of the Developer Certificate of Origin can be found at developercertificate.org.
The list of commits missing DCO signoff:

c05f34b Merge branch 'falcosecurity:master' into master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.
:memo: Please follow instructions in the contributing guide to update your commits with the DCO
Full details of the Developer Certificate of Origin can be found at developercertificate.org.
The list of commits missing DCO signoff:

c05f34b Merge branch 'falcosecurity:master' into master
3590244 Merge pull request #1 from vjjmiras/community-survey-2023

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: jsalinas29, vjjmiras
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

content/OWNERS [vjjmiras]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


text:
AWS-Region issue when trying to work with EC2-Instance Profiles via KIAM

Describe the bug
Using a Kubernetes Installation on AWS-EC2-VMs. To communicate to AWS (Assume-Roles) we use KIAM (https://github.com/uswitch/kiam)
When trying to remove the Access_Keys and Access_Secrets and make use of EC2-Instance Profiles, Falcosidekick fails when it tries to connect to AWS.
Error Message in the Logs:
2022/06/01 10:30:33 [ERROR] : AWS - MissingRegion: could not find region configuration
How to reproduce it
When removing the following Entries from secret of Falcosidekick:

AWS_ACCESSKEYID:
AWS_SECRETACCESSKEY:
AWS_REGION:

it will show the following error: 2022/06/01 10:30:33 [ERROR] : AWS - MissingRegion: could not find region configuration
When putting the AWS_REGION with the the correct Region (ex. "eu-central-1") then it will show the following error:
2022/06/01 10:50:45 [ERROR] : AWS - Error while getting AWS Token
After a hint from @Issif I tried to define Environment variable: AWS_STS_REGIONAL_ENDPOINTS = true
the following error message is shown:
2022/06/01 11:00:30 [ERROR] : AWS - Error while creating AWS Session
Expected behaviour
Login via Instance-Role Profile should work and should be able to communicate with the AWS-API.
Environment

Falco version:
falco version: 0.31.1
falcosidekick version: 2.25.0
System info:

{
  "machine": "x86_64",
  "nodename": "xxxxxxxxx"
  "release": "5.10.107-flatcar",
  "sysname": "Linux",
  "version": "#1 SMP Tue Mar 22 19:39:53 -00 2022"
}

Wrong repository. Created a new issue on Falcosidekick: https://github.com/falcosecurity/falcosidekick/issues/325

repo: falcosecurity/falco


text:
Add Deckhouse to Falco adopters

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

/kind release

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area engine

/area tests

/area proposals

/area CI

What this PR does / why we need it:
Deckhouse now includes Falco out of the box for its users and can be considered as one of Falco adopters!
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

NONE

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: nabokihms
Once this PR has been reviewed and has the lgtm label, please assign leogr for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Welcome @nabokihms! It looks like this is your first PR to falcosecurity/falco 🎉

LGTM label has been added.  Git tree hash: 6d68f83653ef13cdecf5b6ebbf0f4b5b8a5f4a41

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: jasondellaluce, nabokihms
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/milestone 0.35.0

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, jasondellaluce, nabokihms
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

CircleCI Pipeline — Could not find a usable config.yml, you may have revoked the CircleCI OAuth app.

Why circleCI :/

Closing and reopening for CircleCI
/close

@jasondellaluce: Closed this PR.

In response to this:

Closing and reopening for CircleCI
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/reopen

@jasondellaluce: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

authors: FedeDP jasondellaluce nabokihms poiana


text:
prepare v1.0.1

Signed-off-by: Issif issif_github@gadz.org

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area tests

What this PR does / why we need it:
This is a bug fix release.
Which issue(s) this PR fixes:
When the UI is reached by https, the websocket connection is still in clear, which raises a "Mixed content" error in browser.

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please ask for approval from issif after the PR has been reviewed.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: fjogeleit
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [fjogeleit]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: a3ad8c7a5640ca31472a29c70f2bef539c454fcd

authors: Issif poiana

repo: falcosecurity/falcosidekick-ui


text:
cleanup(libsinsp): swap check order in is_in_pid_namespace()

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area API-version

/area build

/area CI

/area driver-kmod

/area driver-bpf

/area driver-modern-bpf

/area libscap-engine-bpf

/area libscap-engine-gvisor

/area libscap-engine-kmod

/area libscap-engine-modern-bpf

/area libscap-engine-nodriver

/area libscap-engine-noop

/area libscap-engine-source-plugin

/area libscap-engine-savefile

/area libscap-engine-udig

/area libscap

/area libpman

/area libsinsp

/area tests

/area proposals

Does this PR require a change in the driver versions?

/version driver-API-version-major

/version driver-API-version-minor

/version driver-API-version-patch

/version driver-SCHEMA-version-major

/version driver-SCHEMA-version-minor

/version driver-SCHEMA-version-patch

What this PR does / why we need it:
Since this method is now being used in the process tree fix PR, I am not sure if not having the check order the more efficient / robust way aka the way it is in parsers.cpp could cause issues when trying to find the reaper.
Anyhow, it's helping with better consistency at a minimum.
@Andreagit97 @gnosek
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

NONE

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: incertum
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [incertum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, incertum
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,incertum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: c8331d7043ee2db0c462f9ac46c13b03f2c48be9

/milestone 0.13.0

authors: FedeDP incertum poiana

repo: falcosecurity/libs


text:
fix: change CVE-2024-3094 to match liblzma contain instead of endswith

What type of PR is this?
/kind bug
/area rules
/area maturity-incubating
What this PR does / why we need it:
Some of the docs in the wild of how to reproduce XZ vulnerability relies on patching liblzma and having the filename with patch suffix, for example - liblzma.so.5.6.1.patch, thus this Falco rule doesn't pick this up. Changing rule to match contains string resolves the issue.
Some examples of who's using liblzma.so.5.6.1.patch name:

https://github.com/amlweems/xzbot/tree/main/assets
https://github.com/r0binak/xzk8s/blob/main/Dockerfile#L11

Welcome @apsega! It looks like this is your first PR to falcosecurity/rules 🎉

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: apsega
Once this PR has been reviewed and has the lgtm label, please assign leogr for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: apsega, darryk10
Once this PR has been reviewed and has the lgtm label, please assign leogr for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 56a2344a6ca08804aecca91738487a2421b8bc0d

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: apsega, darryk10, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

authors: apsega poiana

repo: falcosecurity/rules


text:
Clippy

All patches here were found by running cargo clippy --all --all-targets --all-features -- -D warnings. This patch should be tested extensively before merging to make sure no regressions were introduced.
However, there remain two unpatched linter findings that warrant further discussion here.

We use a bunch of enums that have a large size difference between their possible values. We could either Box these, or add clippy exceptions for them.
There seems to be a problem with #[display(Debug)] / derive(Display) where we take a reference that is immediately dereffed.

Codecov Report

Merging #185 (0bce073) into main (cc4cee5) will increase coverage by 0.2%.
The diff coverage is 14.2%.

@@           Coverage Diff           @@
##            main    #185     +/-   ##
=======================================
+ Coverage   15.6%   15.7%   +0.2%
=======================================
  Files         27      27
  Lines       6077    5989     -88
=======================================
- Hits         945     941      -4
+ Misses      5132    5048     -84

Impacted Files
Coverage Δ

src/cli/command.rs
42.6% <0.0%> (-0.1%)
:arrow_down:

src/farcasterd/runtime.rs
0.0% <0.0%> (ø)

src/opts.rs
100.0% <ø> (ø)

src/peerd/opts.rs
0.0% <0.0%> (ø)

src/peerd/runtime.rs
0.0% <0.0%> (ø)

src/rpc/client.rs
34.5% <0.0%> (ø)

src/rpc/messages.rs
0.0% <0.0%> (ø)

src/rpc/request.rs
14.8% <0.0%> (ø)

src/service.rs
8.5% <ø> (ø)

src/swapd/opts.rs
0.0% <ø> (ø)

... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cc4cee5...0bce073. Read the comment docs.

I decided to bump the max enum variant size to 4k. Seems too complicated to either start using trait objects everywhere, or boxing the type.

I still don't understand the issue with the formatter reference. After all impl fmt::Display takes a mutable reference as an argument fn fmt(&self, f: &mut fmt::Formatter<'_>). Might this be a false positive?

I still don't understand the issue with the formatter reference. After all impl fmt::Display takes a mutable reference as an argument fn fmt(&self, f: &mut fmt::Formatter<'_>). Might this be a false positive?

I'm getting a few other errors but I don't see this one, what version of clippy/rust are you using?

Clippy version: clippy 0.1.58 (d914f17 2021-11-16)
Rustc: rustc 1.58.0-nightly (d914f17ca 2021-11-16)
Cargo: cargo 1.58.0-nightly (2e2a16e98 2021-11-08)

Oh,

Clippy version: clippy 0.1.58 (d914f17 2021-11-16) Rustc: rustc 1.58.0-nightly (d914f17ca 2021-11-16) Cargo: cargo 1.58.0-nightly (2e2a16e98 2021-11-08)

Oh, you should maybe stay in stable for now, nightly for clippy is somewhat weird sometimes.

authors: TheCharlatan codecov-commenter h4sh3d

repo: farcaster-project/farcaster-node


text:
refactor(StorageRegistry): rename StorageRent to StorageRegistry

Motivation
The contract was inconsistently called Storage and StorageRent in different places. Renaming it to StorageRegistry aligns with the naming convention used in other contracts and clarifies its purpose.
Change Summary

Rename StorageRent to StorageRegistry

Merge Checklist

[x] The PR title adheres to the conventional commits standard
[x] The PR has been tagged with change type label(s) (i.e. documentation, feature, bugfix, or chore)
[x] The PR's changes adhere to all the requirements in the contribution guidelines
[x] This PR does not require changes to the Farcaster protocol

PR-Codex overview
This PR focuses on renaming the StorageRent contract to StorageRegistry.
Detailed summary

Renamed StorageRent contract to StorageRegistry throughout the codebase.
Updated references to StorageRent in test files and gas usage tests.
Updated references to StorageRent in deployment scripts.
Updated references to StorageRent in the README.md file.

The following files were skipped due to too many changes: script/Deploy.s.sol, docs/docs.md, src/Bundler.sol, test/Deploy/Deploy.t.sol, test/Bundler/Bundler.t.sol, test/StorageRent/StorageRent.t.sol

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@varunsrin nitpick: I think just naming this Storage is the best suggestion so far. This was originally named StorageRegistry, but I swapped Registry for Rent since unlike the "registry" contracts, it doesn't expose storage allocations on chain, just through events. "Registry" implies to me that there's something to look up on chain, which is the case for IdRegistry and KeyRegistry.
Not a hill I will die on, though. 😄

That's fair, will change
On Mon, Jul 31, 2023 at 4:23 PM horsefacts @.***> wrote:

@.**** approved this pull request.
—
Reply to this email directly, view it on GitHub
https://github.com/farcasterxyz/contracts/pull/299#pullrequestreview-1555945577,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEBD4SCQCDFCC73PBSQUMLXTA5AVANCNFSM6AAAAAA265VPFE
.
You are receiving this because you were mentioned.Message ID:
@.***>

@horsefacts the one annoying thing about Storage vs StorageRegistry is that storage isn't a valid variable name, so lets keep it Registry for now.

authors: horsefacts varunsrin

repo: farcasterxyz/contracts


text:
[Request] Percentage support/more complex expressions

Hello,
This is an amazing plugin. Would it be possible to add us support for some  more math. i.e. query a+b-c, a/b *100 (percentage) etc.?
Thanks
Karan.

@KaranTrivedi I don't have any plans to add that sort of functionality, but the Singlestat Math plugin sounds very close to what you're looking for, so I would take a look at that and see if it offers what you need.

Hey, thanks for the reply. This was for elastic, not influx, I dont think I can use that.

Ah, hm. I took a loot at the source for it and it looked fairly generic. I think it should work with any data source that outputs time series data. If not, I know that Grafana has had native query math support on the roadmap for quite a while. Hopefully that makes it into the product sometime soon.

Alright, thanks.

authors: KaranTrivedi farski

repo: farski/blendstat-grafana


text:
AWS Sagemaker CloudFormation: Error Template validation error: Internal Failure

As the title says, the template does not launch. Tried in any of the provided regions.
The template, however, works when manually loaded through s3 link

Could you ask on the forum, and if everyone else is seeing the same problem, reopen this? If it's specific to your account, best off asking aws.


text:
indirect dependencies from "install_requirement = 'jupyter'"

Hello everyone,
the Jupyter metapackage 'jupyter' has the install_requirements
        'notebook',
        'qtconsole',
        'jupyter-console',
        'nbconvert',
        'ipykernel',
        'ipywidgets',

Does nbdev really needs all of them? If not, it would be a great idea to list them separately instead.
It could help to reduce the number of indirect dependencies.

nbdev2 will no longer require jupyter.

repo: fastai/nbdev


text:
Fix unregistered error received when celery call task

PLR.

You need to start celery in the backend/app directory.

You need to start celery in the backend/app directory.

I know, it's already done.

Is it the same mistake? Can you provide the startup log?

authors: downdawn wu-clan


text:
Fastlane Boarding still actively maintained?

Question Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Question Subject
Fastlane Boarding
Question Description
Apologies for posting this here. I have been working on some new features for FastLane Boarding, but since making a PR, I haven't heard anything. Could someone please confirm that Boarding is still being actively maintained?

It definitely hasn't been on my radar, but I am following the repo now.
But I also don't see any recent issues or pull requests, so there doesn't seem to much to do right now.

Haven't had the time to make any changes recently, but some are on the way!

authors: FinHorsley janpio

repo: fastlane/fastlane


text:
deliver key not found "value"

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description
Everything works ok, until fastlane goes to deliver step i get this error
key not found : "value"
Command executed

Complete output when running fastlane, including the stack trace and command used

Using fastlane ios beta --verbose

INFO [2019-10-31 14:23:34.22]: Successfully sent Slack notification
WARN [2019-10-31 14:23:34.22]: Lane Context:
INFO [2019-10-31 14:23:34.22]: {:PLATFORM_NAME=>:ios, :LANE_NAME=>"ios beta", :GIT_REPO_WAS_CLEAN_ON_START=>true, :BUILD_NUMBER=>"7", :SIGH_PROFILE_TYPE=>"app-store", :IPA_OUTPUT_PATH=>"/Users/dantecervantes/Documents/projects/watsontapp/watsontapp.ipa", :DSYM_OUTPUT_PATH=>"/Users/dantecervantes/Documents/projects/watsontapp/watsontapp.app.dSYM.zip", :XCODEBUILD_ARCHIVE=>"/Users/dantecervantes/Library/Developer/Xcode/Archives/2019-10-31/watsontapp 2019-10-31 14.18.40.xcarchive"}
ERROR [2019-10-31 14:23:34.22]: key not found: "value"
INFO [2019-10-31 14:23:34.23]: Successfully generated documentation at path '/Users/dantecervantes/Documents/projects/watsontapp/fastlane/README.md'
+------+------------------------+-------------+
|              fastlane summary               |
+------+------------------------+-------------+
| Step | Action                 | Time (in s) |
+------+------------------------+-------------+
| 1    | opt_out_usage          | 0           |
| 2    | ensure_git_branch      | 0           |
| 3    | ensure_git_status_cle  | 0           |
|      | an                     |             |
| 4    | increment_build_numbe  | 0           |
|      | r                      |             |
| 5    | Switch to ios build    | 0           |
|      | lane                   |             |
| 6    | Switch to ios          | 0           |
|      | certificates lane      |             |
| 7    | match                  | 5           |
| 8    | match                  | 5           |
| 9    | gym                    | 289         |
| 💥   | deliver                | 3           |
| 11   | slack                  | 6           |
+------+------------------------+-------------+
ERROR [2019-10-31 14:23:34.29]: fastlane finished with errors
Looking for related GitHub issues on fastlane/fastlane...
Search query: key not found: "value"
URL: https://api.github.com/search/issues?q=key not found%3A "value"+repo:fastlane/fastlane
➡  Fastlane ruby version not found
https://github.com/fastlane/fastlane/issues/15509 [open] 1 💬
2 weeks ago
➡  Arabic language not supported in metadata
https://github.com/fastlane/fastlane/issues/14959 [open] 24 💬
3 days ago
➡  Regenerate Profiles from Different Machine
https://github.com/fastlane/fastlane/issues/15492 [open] 3 💬
2 weeks ago
and 1964 more at: https://github.com/fastlane/fastlane/search?q=key not found%3A "value"&type=Issues&utf8=✓
🔗  You can ⌘ + double-click on links to open them directly in your browser.
/Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:740:in fetch': [!] key not found: "value" (KeyError)         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:740:in block in setup_trailers_for'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:739:in each'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:739:in setup_trailers_for'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:726:in block in setup_trailers'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:724:in each'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:724:in setup_trailers'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:323:in setup'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/base.rb:215:in initialize'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:168:in new'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:168:in factory'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/app_version.rb:189:in find'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/spaceship/lib/spaceship/tunes/application.rb:111:in edit_version'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/deliver/lib/deliver/upload_metadata.rb:153:in verify_available_languages!'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/deliver/lib/deliver/upload_metadata.rb:44:in upload'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/deliver/lib/deliver/runner.rb:62:in upload_metadata'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/deliver/lib/deliver/runner.rb:21:in run'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/actions/deliver.rb:16:in run'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/runner.rb:252:in block (2 levels) in execute_action'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/actions/actions_helper.rb:50:in execute_action'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/runner.rb:230:in block in execute_action'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/runner.rb:226:in chdir'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/runner.rb:226:in execute_action'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/runner.rb:148:in trigger_action_by_name'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/fast_file.rb:146:in method_missing'         from Fastfile:40:in block (2 levels) in parsing_binding'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/lane.rb:33:in call'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/lane.rb:33:in call'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/runner.rb:49:in block in execute'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/runner.rb:45:in chdir'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/runner.rb:45:in execute'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/lane_manager.rb:52:in cruise_lane'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/command_line_handler.rb:30:in handle'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/commands_generator.rb:104:in block (2 levels) in run'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:178:in call'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:178:in call'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:153:in run'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/commander-fastlane-4.4.6/lib/commander/runner.rb:476:in run_active_command'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:40:in run!'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/commander-fastlane-4.4.6/lib/commander/delegates.rb:15:in run!'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/commands_generator.rb:303:in run'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/commands_generator.rb:42:in start'
from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/fastlane/lib/fastlane/cli_tools_distributor.rb:66:in take_off'         from /Users/dantecervantes/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.28.3/bin/fastlane:20:in <top (required)>'
from /Users/dantecervantes/.fastlane/bin/bundle/bin/fastlane:22:in load'         from /Users/dantecervantes/.fastlane/bin/bundle/bin/fastlane:22:in '

Environment

Key
Value

OS
10.14.6

Ruby
2.2.4

Bundler?
false

Git
git version 2.21.0 (Apple Git-122)

Installation Source
~/.fastlane/bin/bundle/bin/fastlane

Host
Mac OS X 10.14.6 (18G103)

Ruby Lib Dir
~/.fastlane/bin/bundle/lib

OpenSSL Version
OpenSSL 1.0.2g  1 Mar 2016

Is contained
false

Is homebrew
true

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
11.1

never mind, i just updated fastlane to last version and it works!