text:
Clean up IA32 Assembler's code_

assembler_ia32.h has a field, code_, which is not present in any of the other assemblers:
https://github.com/dart-lang/sdk/blob/618db8a41d68fd37ff5bb228c69486721be3a636/runtime/vm/compiler/assembler/assembler_ia32.h#L1093
It is only read in one place:
https://github.com/dart-lang/sdk/blob/618db8a41d68fd37ff5bb228c69486721be3a636/runtime/vm/compiler/assembler/assembler_ia32.cc#L2699-L2704
And set in one place:
https://github.com/dart-lang/sdk/blob/618db8a41d68fd37ff5bb228c69486721be3a636/runtime/vm/object.cc#L17254-L17256
Note how we're setting the field through a getter:
https://github.com/dart-lang/sdk/blob/618db8a41d68fd37ff5bb228c69486721be3a636/runtime/vm/compiler/assembler/assembler_ia32.h#L1046
This is already fishy, but this quirk is also causing a different bug, whereby we incorrectly set the Code object in Dart frames to the current Code, instead of the Code being called.
For example, in FfiCallInstr we commonly do the following across all architectures:
https://github.com/dart-lang/sdk/blob/618db8a41d68fd37ff5bb228c69486721be3a636/runtime/vm/compiler/backend/il_ia32.cc#L1039-L1041
Except, on IA32 EnterDartFrame calls the above PushCodeObject which we saw always pushes whatever is in code_, and not what is in CODE_REG, as the comment otherwise claims (which is correct on other archs.):
https://github.com/dart-lang/sdk/blob/618db8a41d68fd37ff5bb228c69486721be3a636/runtime/vm/compiler/assembler/assembler_ia32.cc#L2706-L2709
This ultimately has the effect that FFI ExitFrames have a duplicate FfiTrampoline Code object in them where they should have Null, which in turn affects stack traces, etc.
Aside: This might also affect other users of EnterDartFrame, but I haven't checked.

...  we incorrectly set the Code object in Dart frames to the current Code ...

There may be a misunderstanding here. The code object in a frame is representing the code object of that frame's function (not the caller's code object).
On most of our architectures (x64, arm, arm64) the caller is populating this code object as part of the call sequence: In order to call a function, the caller has to load the Functions Code and from there the Codes entrypoint address. The caller therefore has to load the code object anyway, so it loads it into the specified CODE_REG register - which makes it available to the callee (i.e. the callee can simply push CODE_REG - which is its own code object).
Now on ia32 we do not use code objects as such. Callers (e.g. a static call) will invoke its target directly - on need to get the entrypoint from a code object. The callee therefore has to get it's own code object in a different way, namely in the way you described above.
This is by-design.

Support for ia32 is going to be deprecated soon, this issue will not be addressed.

authors: a-siva ghost mkustermann


text:
ByteData bug: newValue(call getFloat32) is not equal value(call setFloat32)

Dart SDK Version (dart --version)
Dart SDK version: 2.17.1 (stable) (Tue May 17 17:58:21 2022 +0000) on "macos_x64"

Whether you are using Windows, MacOSX, or Linux (if applicable)
MacOSX

Whether you are using Chrome, Safari, Firefox, Edge (if applicable)
Chrome/Safari

import 'dart:typed_data';

import 'package:flutter/foundation.dart';
import 'package:flutter/services.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  test('ByteData Test', () {
    final ByteData byteData = ByteData(4);
    byteData.setFloat32(0, 800.2, Endian.little);
    final double value = byteData.getFloat32(0, Endian.little);
    if (kDebugMode) {
      print('value: $value');
    }
    expect(value == 800.2, isTrue);
  });
}

value: 800.2000122070312
package:test_api                                    expect
package:flutter_test/src/widget_tester.dart 455:16  expect
test/fuck_test.dart 15:5                            main.<fn>

Expected: true
  Actual: <false>

This is working as intended.  This has to do with the underlying 32-bit floating point representation, not Dart.
E.g., you'll see similar with the follow C program:
#include <stdio.h>

int main() {
  float v = 800.2;
  printf("%f\n", v);
}

source: gharchive/issue

repo: dart-lang/sdk

license_type: permissive

license_source: github-api


text:
Release 0.3.8

@rultor release it; the tag is 0.3.8.

@rultor release it; the tag is 0.3.8.

@rafamizes OK, I will release it now. Please check the progress here

@rultor release it; the tag is 0.3.8.

@rafamizes Done! FYI, the full log is here (took me 2min)

authors: rafamizes rultor

license: MIT


text:
[Potential Bug] Highlighted recipe missing a ring

I noticed that when I was using this tool at the beginning of the league, the highlights would sometimes be missing a ring (maybe 50% of the time?). It's possible that I somehow kept missing the highlight of the ring in my quad stash but I'm fairly certain this happened sometimes. I'm busy playing atm so cannot provide logs right now, but it shouldn't be too hard to test it and reproduce it. Just load a quad stash tab with a few recipes.

I tried testing it again but wasn't able to reproduce it. Please keep an eye out for it.

The issue happened again. It keeps missing one ring highlight when you ask it to show the recipe. I THINK it has to do with the regal recipe portions (ilvl 75+) items...that change probably added this bug.
Also, could you please have an option to disable logging or to truncate the log file? The log file grows way really quickly and would like to limit that significantly.

I have very little experience with this script so far. Running from latest code from the master branch. The script only highlighted 1 1h weapon for me.

are you still seeing the issue with 75+ items?

I have not been able to figure out why it happens sometimes. I've been restarting stashvision a bit between hand ins and sometimes I have the "one-ring-issue" multiple times on a quad-vendor-binge and sometimes no issues at all.
I can see if ilvl matters. But now I'm getting into red maps so will see how much more chaos recipe I will do :-)

authors: EirikAskheim darvid thebeast792

repo: darvid/stashvision


text:
Feature Request: shortcut options

As a developer I want to have the ability to treat some options as aliases of others, similar to command line arguments where -q could be equivalent to --quiet
Specifically in scenic the goal is to allow developers to specify the translate (as one example) as either :t or :translate:
https://github.com/boydm/scenic/blob/b2e57b50737443d8fb7a6d7ef2e88ef704cf42c2/lib/scenic/primitive/transform/translate.ex#L21-L29
This is currently implemented using :rename_to which is now deprecated in 0.4:
https://github.com/boydm/scenic/blob/b2e57b50737443d8fb7a6d7ef2e88ef704cf42c2/lib/scenic/primitive/transform/transform.ex#L61-L68
It would be nice to be able to do this without having to post-process the output of nimble_options (one specific downside to this is having to customize the output of NimbleOptions.docs/2).
This is effectively a request to include :other_names mentioned in https://github.com/dashbitco/nimble_options/issues/68#issuecomment-953619287 although I think :aliases would be a better name for it.

@axelson I think aliases (or shortcuts) are much more common in CLI tools arguments rather than options like the ones NimbleOptions is targeting. As you said, doing a pre-pass on the options to rename them before feeding them into nimble_options should be straightforward and, as you mentioned, you can customize the doc: option to mention aliases.
You could even do all of this in a programmatic way by preprocessing the schema too:
schema_with_aliases =
  Enum.map(schema, fn {key, schema} ->
    case Keyword.fetch(schema, :aliases) do
      {:ok, aliases} ->
        doc = "Aliased to: #{inspect(aliases)}"
        Keyword.update(schema, :doc, " #{doc}", &(&1 <> " #{doc}")
  end)

or something along those lines. You get the idea. Thoughts?

Yeah, I think a pre-pass is the way to go too. I understand why it exists on Scenic and agree with it, but we likely don’t want to encourage APIs to have aliases. It is rather the exception than the rule.

authors: axelson josevalim whatyouhide


text:
Wakeup wakeup-pipe when new peers are added

This also allows us to increase the timeout for select/poll, which avoids unnecessary work in the socket handler thread.

Needs rebase

source: gharchive/pull-request

repo: dashpay/dash

license: mit

license_source: bigquery


text:
Implement all arg types for to_datetime

Closes #656

thx

authors: hendrikmakait phofl

repo: dask-contrib/dask-expr


text:
Datafusion IndexError - Return fields from the lhs and rhs of a join

Closes: #584

Codecov Report

:exclamation: No coverage uploaded for pull request base (datafusion-sql-planner@ba50d24). Click here to learn what that means.
The diff coverage is n/a.

@@                    Coverage Diff                    @@
##             datafusion-sql-planner     #599   +/-   ##
=========================================================
  Coverage                          ?   60.81%
=========================================================
  Files                             ?       71
  Lines                             ?     3560
  Branches                          ?      722
=========================================================
  Hits                              ?     2165
  Misses                            ?     1247
  Partials                          ?      148

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

repo: dask-contrib/dask-sql


text:
test_division_or_partition fails on Python 3.12

Describe the issue:
I believe 3.12 is the key factor, but all dependencies were rebuilt for it, so all dependencies are listed as possible causes. If you suspect that there's likely to be a different cause, see here for all the dependencies.
__________________________ test_division_or_partition __________________________
[gw3] linux -- Python 3.12.0 /usr/bin/python3
db = 'sqlite:////tmp/tmpk8po4885'
    def test_division_or_partition(db):
        with pytest.raises(TypeError, match="either 'divisions' or 'npartitions'"):
            read_sql_table(
                "test",
                db,
                columns=["name"],
                index_col="number",
                divisions=[0, 2, 4],
                npartitions=3,
            )

        out = read_sql_table("test", db, index_col="number", bytes_per_chunk=100)
        m = out.memory_usage_per_partition(deep=True, index=True).compute()
>       assert (50 < m).all() and (m < 200).all()
E       assert (True and False)
E        +  where True = <bound method NDFrame._add_numeric_operations.<locals>.all of 0    True\n1    True\n2    True\n3    True\n4    True\ndtype: bool>()
E        +    where <bound method NDFrame._add_numeric_operations.<locals>.all of 0    True\n1    True\n2    True\n3    True\n4    True\ndtype: bool> = 50 < 0     78\n1     76\n2     78\n3     77\n4    238\ndtype: int64.all
E        +  and   False = <bound method NDFrame._add_numeric_operations.<locals>.all of 0     True\n1     True\n2     True\n3     True\n4    False\ndtype: bool>()
E        +    where <bound method NDFrame._add_numeric_operations.<locals>.all of 0     True\n1     True\n2     True\n3     True\n4    False\ndtype: bool> = 0     78\n1     76\n2     78\n3     77\n4    238\ndtype: int64 < 200.all
../../../BUILDROOT/python-dask-2023.4.1-1.fc39~bootstrap.x86_64/usr/lib/python3.12/site-packages/dask/dataframe/io/tests/test_sql.py:290: AssertionError

The main point here is that the last element has a size of 238, which is over the expected maximum of 200. I'm not sure how these numbers came about; maybe the maximum could just be increased?
Minimal Complete Verifiable Example:
pytest

Anything else we need to know?:
Environment:

Dask version: 2023.7.0
Python version: 3.12.0b4
Operating System: Fedora Rawhide
Install method (conda, pip, source): source

Hi, thanks for your report. pandas 1.5.3 is not compatible with Python 3.12, same for NumPy 1.24, there is no guarantee that this works. We can't do much as long as there aren't any upstream wheels available for 3.12

Quick update on this: the test still fails in exactly this way with Python 3.12 when using Pandas 2.1.4 and NumPy 1.24.2.  (I haven't yet been able to test it with NumPy 1.26.x.)

@juliangilbey: Thanks for the update. Support for Python 3.12 was only added in NumPy 1.26, so updating your NumPy version should fix the issue. Please let us know if this works!

@juliangilbey: Thanks for the update. Support for Python 3.12 was only added in NumPy 1.26, so updating your NumPy version should fix the issue. Please let us know if this works!

Indeed; when Debian updates it's NumPy (probably after the ongoing Python 3.12 transition has finished), I will be able to retest this.

Given that this test passes on CI for Python 3.12 (https://github.com/dask/dask/actions/runs/7281247936/job/19841385444#step:10:9553), I'll close this issue. Feel free to reopen if you experience an issue on Debian once you've updated.

This is still failing on Arch with numpy 1.26.4

authors: QuLogic antonio-rojas hendrikmakait juliangilbey phofl

repo: dask/dask


text:
tuple index out of range error when assign a dask array to a dask DataFrame column

i was trying to assign a dask array to a dask dataframe but came across this IndexError: tuple index out of range. I also tried ts_df.assign(test=new_col) but still got the same IndexError.
import dask
import dask.dataframe as dd
import dask.array as da

# create a random dataframe like official documentation does
ts_df = dask.datasets.timeseries(
    '2000', '2003', freq='2H', partition_freq='2Y'
)

print(ts_df.head())
"""
                      id    name         x         y         z
timestamp
2000-12-31 00:00:00  962     Tim -0.562319 -0.331690 -0.894009
2000-12-31 02:00:00  995   Frank -0.952083 -0.725073 -1.677156
2000-12-31 04:00:00  932     Bob  0.579130  0.923146  1.502276
2000-12-31 06:00:00  997  Ursula  0.120449 -0.059130  0.061319
2000-12-31 08:00:00  975  Victor -0.947120  0.647456 -0.299664
"""

# assign a new column using dask Series works fine
ts_df['z'] = ts_df.x + ts_df.y

new_col = da.where(
    ((ts_df.x >= 0) | (ts_df.y >= 0)),
   'N/A',
    ts_df.x.astype(str) + 'x' + ts_df.y.astype(str)
)

# assign a new column using dask array runs into error
ts_df['test'] = new_col

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-224-392b07690d97> in <module>
      2     ((ts_df.x >= 0) | (ts_df.y >= 0)),
      3     'N/A',
----> 4     ts_df.x.astype(str) + 'x' + ts_df.y.astype(str)
      5 )

~/venvs/se/lib/python3.7/site-packages/dask/dataframe/core.py in __setitem__(self, key, value)
   3373             df = self.assign(**{k: value for k in key})
   3374         else:
-> 3375             df = self.assign(**{key: value})
   3376
   3377         self.dask = df.dask

~/venvs/se/lib/python3.7/site-packages/dask/dataframe/core.py in assign(self, **kwargs)
   3633                         )
   3634                     )
-> 3635                 kwargs[k] = from_dask_array(v, index=self.index)
   3636
   3637         pairs = list(sum(kwargs.items(), ()))

~/venvs/se/lib/python3.7/site-packages/dask/dataframe/io/io.py in from_dask_array(x, columns, index)
    416     dask.dataframe._Frame.to_records: Reverse conversion
    417     """
--> 418     meta = _meta_from_array(x, columns, index)
    419
    420     if x.ndim == 2 and len(x.chunks[1]) > 1:

~/venvs/se/lib/python3.7/site-packages/dask/dataframe/io/io.py in _meta_from_array(x, columns, index)
     57         )
     58     else:
---> 59         if np.isnan(x.shape[1]):
     60             raise ValueError("Shape along axis 1 must be known")
     61         if columns is None:

IndexError: tuple index out of range

I am using Python 3.7.6 and dask 2.9.2

@jerrytim interested in investigating why da.where(dask_series) has incorrect chunks?

@TomAugspurger yeah absolutely. I did try reshape / play with the chunk of the da.where output but no luck. any ideas?
import dask
import dask.dataframe as dd
import dask.array as da

# create a random dataframe like official documentation does
ts_df = dask.datasets.timeseries(
    '2000', '2003', freq='2H', partition_freq='2Y'
)

z = ts_df.x + ts_df.y
zz = ts_df.x * ts_df.y

cond = ((ts_df.x >= 0) | (ts_df.y >= 0))

# giving all dask Series as input doesn't work
# dask.array<where, shape=(), dtype=float64, chunksize=(), chunktype=numpy.ndarray>
 new_col = da.where(cond, z, zz)

# new_col = new_col.rechunk(chunks=(np.nan,)) doesn't work
# new_col = new_col[None,] doesn't work
# also see some inconsistency, new_col returns "NotImplemented"

# assign a new column using dask array runs into error
ts_df['test'] = new_col

I notice that da.where() works as expected on Series.values
In [23]: s = pd.Series(list(range(10)))

In [24]: ds = dd.from_pandas(s, 2)

In [25]: da.where(ds > 5, 0, 1).shape
Out[25]: ()

In [26]: da.where(ds.values > 5, 0, 1).shape
Out[26]: (nan,)

Perhaps da.where should automatically convert dask series to arrays?  Most of the work seems to be done in elemwise. It's not clear to me if always converting dask series to arrays there is appropriate or not, but it may be worth trying.

@TomAugspurger hmmm, that's very interesting. converting to dask array works like a charm. straightforward solution...I do think dask array function should accept dask series as input or do the convert automatically, that's more consistent with Pandas / Numpy API? Not sure would it considered to be a bug or improvement...thanks for the help

Stumbled upon this issue today!  Converting to array also solved this problem for me, but I noticed this issue was closed awhile ago and seems like no work was done to automatically convert the series to array?
@TomAugspurger - mind if I submit a PR to perform this conversion when performing da.where?

authors: TomAugspurger jerrytim the-matt-morris


text:
Add include_path_column to read_parquet

include_path_column has already been added to read_csv - excellent work!
Can we include a similar include_path_column option to the read_parquet method?

That seems like a nice addition. A PR adding this would be welcome. cc @dharhas who may find this useful

As this is not yet implemented, is there any workaround to add the filepath as colum to the resulting dataframe when reading parquet files?

@guidocioni - I am not aware of any workaround, but pinging @martindurant and @rjzamora as they might know.

As this is not yet implemented, is there any workaround to add the filepath as colum to the resulting dataframe when reading parquet files?

It may be tricky to add this feature to read_parquet. Can you use somethig like the following to capture what you need?
def reader(path):
    df = pd.read_parquet(path)
    df["path"] = path

ddf = dd.from_map(reader, ["file_0", ..., "file_N"])

def reader(path):
    df = pd.read_parquet(path)
    df["path"] = path

ddf = dd.from_map(reader, ["file_0", ..., "file_N"])

Isn't this going to load every single parquet into memory?

I found a solution using dask.delayed and assigning manually the filepath as a new column. I'll post the code here.
Unfortunately I started having another issue of type mismatching due (I believe) to the fact that Dask only checks some of the rows to infer the type...
When loading directly all the parquet files into pandas and concatenating I didn't have any issue, so I'm not sure exactly which row is causing the problem.
Anyway, as I really have no time to troubleshoot the issues in Dask, and as the parquet files are just barely fitting into memory, for now I'll just use pandas directly to load them so that I can also add the filepath column manually.

Please add this feature. Thank you

authors: MrPowers guidocioni jrbourbeau rjzamora vladjohnson


text:
Distributed Dask Fails with to_sql

When I run the below without client = Client('127.0.0.1:8786') it works.
from dask.distributed import Client
# client = Client('127.0.0.1:8786')

import dask.dataframe as dd

df = dd.read_csv('./propdata/pp-complete.csv')
uristring = "postgresql://postgres@127.0.0.1:5432/mydb?sslmode=disable"
df.to_sql('test', uristring)

When I run it with distributed by uncommenting client = Client('127.0.0.1:8786'), I get the following error:
Traceback (most recent call last):
  File "dask_prop_load_sql.py", line 32, in <module>
    df.to_sql('test', uristring)
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/dask/dataframe/core.py", line 1454, in to_sql
    parallel=parallel,
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/dask/dataframe/io/sql.py", line 414, in to_sql
    dask.compute(result)
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/dask/base.py", line 452, in compute
    results = schedule(dsk, keys, **kwargs)
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/distributed/client.py", line 2725, in get
    results = self.gather(packed, asynchronous=asynchronous, direct=direct)
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/distributed/client.py", line 1992, in gather
    asynchronous=asynchronous,
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/distributed/client.py", line 833, in sync
    self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/distributed/utils.py", line 340, in sync
    raise exc.with_traceback(tb)
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/distributed/utils.py", line 324, in f
    result[0] = yield future
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/distributed/client.py", line 1851, in _gather
    raise exception.with_traceback(traceback)
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/dask/dataframe/io/sql.py", line 378, in make_meta
    return meta.to_sql(**kwargs)
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/pandas/core/generic.py", line 2615, in to_sql
    method=method,
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/pandas/io/sql.py", line 598, in to_sql
    method=method,
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/pandas/io/sql.py", line 1827, in to_sql
    table.create()
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/pandas/io/sql.py", line 721, in create
    if self.exists():
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/pandas/io/sql.py", line 708, in exists
    return self.pd_sql.has_table(self.name, self.schema)
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/pandas/io/sql.py", line 1838, in has_table
    return len(self.execute(query, [name]).fetchall()) > 0
  File "/Users/saul/code/dask/venv/lib/python3.7/site-packages/pandas/io/sql.py", line 1679, in execute
    cur = self.con.cursor()

Environment:

Dask version: 2.30.0
Python version: 3.7
Operating System: Mac
Install method (conda, pip, source): pip

Thanks for opening this issue @saul-gush. In order to figure out what's going on, we'll need to have a reproducible example. In this case, we'll also what to know more about the scheduler that you have at '127.0.0.1:8786' is that just a LocalCluster?

@saul-gush I will close this issue, due to lack of information. If you are still having this problem? Please feel free to re-open with a minimal reproducible example as described here

authors: jsignell ncclementi saul-gush


text:
Update Readme file

Hello Dask team,
I am a newbie in Dask, I am learning Dask these days and want to start contributing with you team
so I want to start with the small contribution
the Readme file in this repository need more work wherefore I want to add some features into such as :

Dask Logo
What is Dask
some Code example
Contents, Quickstarts
Installation
References
Contribution Guide

these topics are important for the project
please if this is an acceptable contribution let me know to start working on it
Thanks

Thanks for raising an issue @Kareem-negm. Historically we've opted for a more minimal README in order to avoid replicating information that's already in the Dask docs and can get out of sync over time. See https://github.com/dask/dask/issues/2230 and https://github.com/dask/dask/pull/2231 for previous conversations on this topic.
That said, I'm sure there are ways we can still improve our README, but wanted to point out that there are other factors to consider. @jsignell @jacobtomlinson have been thinking about the Dask docs recently -- I wonder if they have thoughts on how the README might tie into some of the other work going on.

Hello @jrbourbeau , Thanks  a lot for your quick respond
I understand now why you do not make a README file to avoid replicating information in Docs but I think it will be better if we made a suitable README that can show for the reader what is Dask and not replicate information
I can work on a suitable README if you allow me to do that shown "what is the Dask is!! "
So that Mr.@jsignell and @jacobtomlinson will tell me about the points that I should take into consideration

I'm in two minds about this.

One option would be to add some more descriptive content to the README that should be fairly evergreen. Like the content on dask.org. Then add common headers to the README like installation, contributing, etc, but just include links to existing content in the docs.
The other option is to leave things as it is. Keeping things minimal and just having one link out to the main website seems reasonable.

@jsignell what do you think?

I kind of like the idea of putting more content on the README. I tend to agree with @jacobtomlinson that this content changes very infrequently and if we can improve discoverability of information at all that is a good thing. If we end up including contributing and installation sections we can always put a test to make sure that it stays exactly the same as that content in the docs.

@jrbourbeau ,  @jsignell and @jacobtomlinson, please  review if this is a good style and tell me your feedback

@Kareem-negm welcome and thank you for opening up this issue! It looks like it has generated some good ongoing discussion here on the best way to move forward. Might be good to let this conversation play out a bit before spending too much time on writing a whole readme.

Hello @scharlottej13 , Ok
I spent a little time writing a basic style for the README file that I think you will love it
you can see it here :
https://github.com/Kareem-negm/dask/tree/readme
This is a preliminary work that I did to get some modifications to the final visualization of the README structure
I am not moving forward until you and the team guide me

Hello, Happy new year!
there are any updates?

Hello, Happy new year!
there are any updates?

Closing as we already closed the companion PR https://github.com/dask/dask/pull/8488

authors: Kareem-negm jacobtomlinson jrbourbeau jsignell scharlottej13


text:
No matching distribution found for dask==2022.3.0 on RHEL 8

What happened:
I tried to install dask==2022.3.0 on RHEL 8, using Python 3.9.6, but got the following error:
Could not find a version that satisfies the requirement dask==2022.3.0 (from -r /home/ec2-user/requirements.txt (line 20)) (from versions: 0.7.4.linux-x86_64, 0.7.5.linux-x86_64, 0.7.6.linux-x86_64, 0.8.1.macosx-10.5-x86_64, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.2.6, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.10.0, 0.10.1, 0.10.2, 0.11.0, 0.11.1, 0.12.0, 0.13.0rc1, 0.13.0, 0.14.0, 0.14.1, 0.14.2, 0.14.3, 0.15.0, 0.15.1, 0.15.2, 0.15.3, 0.15.4, 0.16.0, 0.16.1, 0.17.0, 0.17.1, 0.17.2, 0.17.3, 0.17.4, 0.17.5, 0.18.0, 0.18.1, 0.18.2, 0.19.0, 0.19.1, 0.19.2, 0.19.3, 0.19.4, 0.20.0, 0.20.1, 0.20.2, 1.0.0, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.2.1, 1.2.2, 2.0.0, 2.1.0, 2.2.0, 2.3.0, 2.4.0, 2.5.0, 2.5.2, 2.6.0, 2.7.0, 2.8.0, 2.8.1, 2.9.0, 2.9.1, 2.9.2, 2.10.0, 2.10.1, 2.11.0, 2.12.0, 2.13.0, 2.14.0, 2.15.0, 2.16.0, 2.17.0, 2.17.1, 2.17.2, 2.18.0, 2.18.1, 2.19.0, 2.20.0, 2.21.0, 2.22.0, 2.23.0, 2.24.0, 2.25.0, 2.26.0, 2.27.0, 2.28.0, 2.29.0, 2.30.0, 2020.12.0, 2021.1.0, 2021.1.1, 2021.2.0, 2021.3.0)

No matching distribution found for dask==2022.3.0

What you expected to happen:
Pip to install the package as usual
Minimal Complete Verifiable Example:
# requirements.txt
dask==2022.3.0

pip3 install -r requirements.txt

# ..or ..

pip3 install dask=2022.3.0

Anything else we need to know?:
The installation has been attempted from the official RHEL8 AWS AMI ami-0ec23856b3bad62d3
Environment:

Dask version: 2022.3.0
Python version: 3.9.6
Operating System: RHEL 8
Install method (conda, pip, source): pip3

Cluster Dump State:

Are you using a private pypi mirror, or installing from pypi itself? The wheel for 2022.3.0 exists on pypi (https://pypi.org/project/dask/2022.3.0/#files) and seems to be valid for python 3.9.

It looks like I was using the wrong executable path to pip with Ansible. Using the right one fixed the problem. Thanks!

authors: jcrist walterdolce


text:
subs doesn't needlessly compare keys and values

Numpy throws a deprecation warning on bool(array == scalar), which would come up in subs periodically, polluting the terminal. This fix ensures comparison only happens if the key and task are the same type, removing these warnings.

+1

authors: jcrist mrocklin


text:
Bytes paths

For consistency in how paths are passed around with the backends.
This would require a little knock-on code in distributed.hdfs and gcsfs.dask_link .

Is it possible to do this in such a way that slightly older versions of dask.distributed will continue to work, although without the improved feature set?

(Should have gone in the main description)
Previous distributed.hdfs did not work as intended https://github.com/dask/dask/pull/2171 , which is where this came from

Understood.  I'm not asking for that bug to be retroactively fixed in dask.distributed, I'm asking that we ensure that these changes do not cause dask.distributed tests to break without changes of their own.  We should subsequently make those changes, I'm just trying to make sure that users are allowed some flexibility so that the libraries don't have to be updated in precise lock-step.

Yes, I can confirm that the tests in distributed/tests/test_hdfs pass with these changes in place.

I have tested this using:
import dask.dataframe as dd
df = dd.read_parquet('s3://bucket/dask_data/*')

and I have the following errors:
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-c743efed9868> in <module>()
----> 1 df = dd.read_parquet('s3://mr-meeseeks/testing/dask_data/*')

/home/ec2-user/coding/dask/dask/dataframe/io/parquet.py in read_parquet(path, columns, filters, categories, index, storage_options)
     75
     76     if len(paths) > 1:
---> 77         pf = fastparquet.ParquetFile(paths, open_with=myopen, sep=myopen.fs.sep)
     78     else:
     79         try:

/opt/anaconda/anaconda3.4.3.1/lib/python3.6/site-packages/fastparquet/api.py in __init__(self, fn, verify, open_with, sep)
     43                  sep=os.sep):
     44         try:
---> 45             fn2 = sep.join([fn, '_metadata'])
     46             self.fn = fn2
     47             with open_with(fn2, 'rb') as f:

TypeError: sequence item 0: expected str instance, list found

@DigitalPig , are you using the latest fastparquet master?

Gahh, fails on windows paths in LocalFileSystem

@mrocklin , this is a timing sensitive test?
dask/tests/test_threaded.py::test_interrupt FAILED

Yes, it has been breaking recently.  I think that @jcrist is aware of it but he's out this week.

https://github.com/dask/dask/issues/2192

Rerunning made it pass

+1 from me

authors: DigitalPig martindurant mrocklin


text:
Avoid threaded pa.Table.from_pandas calls

Note:  This is a simple solution to #7334, but it may be more harsh than necessary.

[x] Closes #7334
[ ] Tests added / passed
[x] Passes black dask / flake8 dask

Thanks Rick! 😄

authors: jakirkham rjzamora


text:
Update test_groupby.py to un-xfail tests

Resolves: https://github.com/dask/dask/pull/9074#issuecomment-1137176593
We had to xfail since associated cudf PR https://github.com/rapidsai/cudf/pull/10838 wasn't merged. Now that it is merged, it is safe to remove xfail

cc: @brandon-b-miller @quasiben


text:
Bug? The lib skip directory contents

Hello, I try to find out a reason of the behaviour. It looks like a bug
hdfs3 was installed from the conda-forge.
So with simple hadoop ls commands:

$ hadoop fs -ls / | grep smartdata
drwxr-xr-x      - hdfs  supergroup          0 2018-02-16 11:39 /smartdata
$ hadoop fs  -ls  /smartdata
drwxrwxrwx   - hdfs  supergroup          0 2018-02-02 11:35  /smartdata/hive
...
$ hadoop fs -ls  /smartdata/hive
drwxrwxrwx   - hdfs  supergroup          0 2018-04-26 09:32  /smartdata/hive/external
...

And this is with hdfs3 lib. It does not show listing of /smartdata, but does it for /smartdata/hive.

$ python -c 'from hdfs3 import HDFileSystem; print HDFileSystem().ls("/")'
[u'//smartdata', ...]
$ python -c 'from hdfs3 import HDFileSystem; print HDFileSystem().ls("/smartdata")'
[]
$ python -c 'from hdfs3 import HDFileSystem; print HDFileSystem().ls("/smartdata/hive")'
[u'/smartdata/hive/external', u'/smartdata/hive/hql', ...]

I saw similar issue here. But it seems it is not permission issue:
https://stackoverflow.com/questions/40405527/python-hdfs3-fails-to-list-non-owned-files

Hm, honestly I have no idea what might be going on here. Does walk fail to descend into the directories?

Hello, I have found the bug. hdfs3 does not work correctly with ACL.

$ python -c 'from hdfs3 import HDFileSystem; print HDFileSystem().ls("/user")'
[u'/user/cloudera', u'/user/history', u'/user/hive', u'/user/hue', u'/user/jenkins', u'/user/oozie', u'/user/root', u'/user/spark']
$ hadoop fs -setfacl -m user:jenkins:rwx /user/cloudera
$ python -c 'from hdfs3 import HDFileSystem; print HDFileSystem().ls("/user")'
[]

I found similar issue for libhdfs3-downstream
https://github.com/ContinuumIO/libhdfs3-downstream/issues/4
It is fixed for master  branch, but conda-forge goes to concat branch for installing hdfs3
https://github.com/conda-forge/libhdfs3-feedstock/blob/47aec11797cf29907738a02941690ef81de2fcfd/recipe/build.sh#L3

Yes, I agree that libhdfs3 should be rereleased, I can try to get to that soon.

authors: martindurant timoninmaxim


text:
Q: s3 key object created time ?

Hello,
s3fs looks very helpful and many thanks to all contributors.
I am trying to get key related time with info or ls method, but I cannot find it.
Can you help me how to get a key created time ?

s3.info('bucket/dir/file') gives output like:
{'Key': 'MDtemp/x.xz',
 'LastModified': datetime.datetime(2016, 8, 20, 13, 1, 18, tzinfo=tzutc()),
 'ETag': '"8dc5aea5b03dff8595f096f9e368e888"',
 'Size': 32,
 'StorageClass': 'STANDARD',
 'type': 'file',
 'size': 32,
 'name': 'MDtemp/x.xz'}

I do see a timestamp, is that not good enough?

Thanks  martindurant!.    I was confused with directory type which does not include detail info.


text:
refactor(index): create no longer calls createArchive if its a duplicate

Fixes #15

Codecov Report

Merging #16 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #16      +/-   ##
==========================================
+ Coverage   97.75%   97.77%   +0.02%
==========================================
  Files           1        1
  Lines          89       90       +1
==========================================
+ Hits           87       88       +1
  Misses          2        2

Impacted Files
Coverage Δ

index.js
97.77% <100%> (+0.02%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6b14d4...5b37158. Read the comment docs.

thoughts? cc @juliangruber @maxogden :upside_down_face:

authors: bplaster codecov-io jwerle

repo: dat-land/multidrive


text:
[Bug] Internal Server Error from data-infographics API

Describe the bug
when querying this news flash - news flash id=231352
When looking at newsflash API  street1_hebrew has \t in it.
I think that's that's one problem we need to fix, perhaps with strip func before adding yishuvs/streets to DB (but not all of it, I think there is another problem):
In addition, when using the search API for this street, we get the same Internal Server Error. Does this mean that this street has no data? https://anyway-infographics-staging.web.app/cityAndStreet/ערערה/אל דהראת
To Reproduce
API: https://www.anyway.co.il/api/infographics-data?lang=he&news_flash_id=231352&years_ago=5
Expected behavior
Return widgets if there are any for the newsflash location, and no widgets (empty dict as json) if there aren't any for the newsflash location
Screenshots

Environment
System: Mac
Additional context
Add any other context about the problem here.

@ziv17 can you take a look?

Hi @atalyaalon ,
What should be returned if a newsflash with the given ID is not found?

Hi @atalyaalon ,
Look at anyway/flask_app.py:1290. There is code there that specifically returns error if the returned widget list is empty. Should it be removed?

Hi @atalyaalon , What should be returned if a news_flash_id parameter is provided, and a newsflash with the provided ID is not found?

I don't think this is the case - since the current news flash id 231352  exists in production
However, if it's not found I guess we should return some kind of error, perhaps 404?

Hi @atalyaalon , Look at anyway/flask_app.py:1290. There is code there that specifically returns error if the returned widget list is empty. Should it be removed?

I think we need differ between 2 cases:

Location exists but no accidents are in our DB
Location does not exist

For the first case, which, I believe is our current case, I suggest that for now, sending an empty list is fine, and we might want to send in the future a special widget indicating that no accidents in DB for this location.
For the second case - I think that sending this error is OK
Is it easy to differ these two cases?

Hi @atalyaalon ,

I think we need differ between 2 cases:
1. Location exists but no accidents are in our DB

2. Location does not exist

Should we differ between location does not exist and given newsflash id does not exist? If yes, in what way? because currently our error message is "The requested URL was not found".

Hi @atalyaalon ,

I think we need differ between 2 cases:
1. Location exists but no accidents are in our DB

2. Location does not exist

Should we differ between location does not exist and given newsflash id does not exist?

Hi @atalyaalon ,

I think we need differ between 2 cases:
1. Location exists but no accidents are in our DB

2. Location does not exist

Should we differ between location does not exist and given newsflash id does not exist?

I guess it would be a best practice, either to differ or to have a debug log for that

Hi @atalyaalon ,
This newsflash location has issues:

It does not have numeric values for the location fields
The street name has a tab \t appended to it, and this is not recognized.

To summarize:

If newsflash not found, or address not found, we return 404, URL not found.
If we find the address, but no accidents, we return empty data.

repo: data-for-change/anyway


text:
Newly released Pydantic version has a breaking change in dbt-cloud-cli

Following the new version of Pydantic provides an breaking change in dbt-cloud-cli.
Running dbt-cloud job <arguments> provides the following traceback:
pydantic.errors.PydanticUserError: The fieldandconfigparameters are not available in Pydantic V2, please use theinfo parameter instead.

I also had this problem - it had worked a week earlier, but on a fresh pip install dbt-cloud-cli I came across this issue when pydantic 2.0.2 was installed automatically.
By first installing pydantic 1.10.9 before dbt-cloud-cli I was able to work around this issue
pip install pydantic==1.10.9
pip install dbt-cloud-cli

Fixed in #75

@runar-alvseike FYI https://github.com/data-mie/dbt-cloud-cli/releases/tag/0.7.6

authors: runar-alvseike stumelius wonkyto

repo: data-mie/dbt-cloud-cli


text:
Feature/arrow inputs

increments / decrements on ArrowUp and ArrowDown inputs.
I copied the switch statement style I found in dom.js.
tests passing on
Chrome 73.0.3683.86 (Official Build) (64-bit)
Firefox 66.0.3 (64-bit)
Opera 60.0.3255.70
Safari 12.0.3 (14606.4.5)

fixes https://github.com/dataarts/dat.gui/issues/195


text:
Use ethereum ics20 cli

Use ethereum-ics20-cli
https://github.com/datachainlab/ethereum-ics20-cli

I think there are reasonable choices.

You always print Alice's balance on ChainA and Bob's balance on ChainB.
You always print both users' balances on both chains (by calling show_balance four times each time).

https://github.com/datachainlab/yui-relayer-build/pull/9/commits/01f4c22c4fc48e40b8001d2d5a0f0cd81b812885
call show_balance four times each time

authors: dongrie siburu

repo: datachainlab/yui-relayer-build


text:
Correction for raw affiliation Adaptations Physiologiques à l'Exercice et Réadaptation à l'effort - UR UPJV 3300; Halmstad University

Correction needed for raw affiliation Adaptations Physiologiques à l'Exercice et Réadaptation à l'effort - UR UPJV 3300; Halmstad University
raw_affiliation_name: Adaptations Physiologiques à l'Exercice et Réadaptation à l'effort - UR UPJV 3300; Halmstad University
new_rors: 01gyxrk03;03h0qfp10
previous_rors: 05vb5nm66;03h0qfp10
works_examples: W3213584511
contact: 5a2f01743f34a46f51e7909362a9da70:180a3c84dd5236077be605c0577a9c42 @ u-picardie.fr

This issue was accepted and ingested by the OpenAlex team on 2024-10-10. The new affiliations should be visible within the next 7 days.

repo: dataesr/openalex-affiliations


text:
Correction for raw affiliation Centre de RMN à Très Hauts Champs de Lyon (UMR 5082 − CNRS, ENS Lyon, UCB Lyon 1), Université de Lyon, 5 Rue de la Doua, 69100 Villeurbanne, France

Correction needed for raw affiliation Centre de RMN à Très Hauts Champs de Lyon (UMR 5082 − CNRS, ENS Lyon, UCB Lyon 1), Université de Lyon, 5 Rue de la Doua, 69100 Villeurbanne, France
raw_affiliation_name: Centre de RMN à Très Hauts Champs de Lyon (UMR 5082 − CNRS, ENS Lyon, UCB Lyon 1), Université de Lyon, 5 Rue de la Doua, 69100 Villeurbanne, France
new_rors: 04zmssz18;029brtt94;02feahw73;004wefe19
previous_rors: 04zmssz18;029brtt94;02feahw73
works_examples: W4382359832
contact: adb53098d5f0f7399b8aff016f0cd956:8b7f95fc82722be120fa90575870 @ ens-lyon.fr

This issue was accepted and ingested by the OpenAlex team on 2024-10-10. The new affiliations should be visible within the next 7 days.


text:
Correction for raw affiliation Aeres University of Applied Sciences Almere, Stadhuisstraat 18, 1315 HC Almere, The Netherlands

Correction needed for raw affiliation Aeres University of Applied Sciences Almere, Stadhuisstraat 18, 1315 HC Almere, The Netherlands
raw_affiliation_name: Aeres University of Applied Sciences Almere, Stadhuisstraat 18, 1315 HC Almere, The Netherlands
new_rors: 03jnx2c74
previous_rors:
works_examples: W3012615648
contact: 9806c6fdd41a677db1723405957fa5b5:442132c549c93292d625 @ hotmail.com

This issue was accepted and ingested by the OpenAlex team on 2024-10-10. The new affiliations should be visible within the next 7 days.


text:
Paramètre iframe nécessaire dans l'URL de l'iframe nosgestesclimat

Il est utilisé pour gérer la hauteur de certains éléments et éviter des
boucles de redessin infinies.
Fixes https://github.com/datagir/nosgestesclimat-site/issues/595

@florianpanchout je me permets de merger ça sans ta revue, étant donné l'importance du bug et le peu d'implications pour le reste du site logiquement.

repo: datagir/datagir-site


text:
Migrate scripting files to TS

Currently, scripting file are plain Node.js. However, as they start to be complex, having type will facilitate their maintenance.

Tu peux décrire mieux le problème ? Merci :)

Non je pense qu'en effet, avoir une générique comme cela n'apporte pas grand chose.

authors: EmileRolley lbranaa

repo: datagir/nosgestesclimat-site


text:
Charts | Sunburst: doesn't respond on table switching

Charts 1.2.0

Charts 1.2.1

repo: datagrok-ai/public


text:
fix(ingest): pin great-expectations dependency to fix profiling error

Issue raised on Slack
Checklist

[x] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[x] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@hsheth2 : looks like this pin creates problems for pip?

authors: mayurinehate shirshanka

repo: datahub-project/datahub


text:
Support customization of text in connection forms from connectors or other packages

Instead of creating Translatable instances with a fixed i18n domain from within the form, we need to implement some kind of factory that would be accessible via SR and would create translatables with a custom domain given a translation key.
Let's call it TextProvider. It would look something like this:
from dl_i18n.localizer_base import Translatable

class TextProvider:
    def get_translatable(self, text_key: str, default_i18n_domain: str) -> Translatable:
        ...

This TextProvider instance can be supplied to form factory instances so that they can implement a method like this:
SomeFormFactory:
    text_provider: TextProvider
    default_i18n_domain: str

    def get_translatable(self, text_key: str) -> Translatable:
        return self.text_provider.get_translatable(text_key, default_i18n_domain=self.default_i18n_domain)

Packages provide custom texts in their own i18n domains, and a list of these custom domains should be provided via settings to the TextProvider instance.
Internal issue: BI-4909

Exceptions also make use of the translatable texts, so it would be logical to also extend this interface to wherever the main exception-generating places are (e.g. USM, executors/adapters).

repo: datalens-tech/datalens-backend


text:
preflight (exit status 1; not expected)

Hello!
Default setup with zitadel. I'm getting a lot of spam in the log:
datalens-us  | 2024-11-12 14:24:53,516 INFO exited: preflight (exit status 1; not expected)
datalens-us  | 2024-11-12 14:24:54,518 INFO spawned: 'preflight' with pid 241
datalens-us  | 2024-11-12 14:24:54,520 INFO success: preflight entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
datalens-us  | 2024-11-12 14:24:55,217 INFO exited: preflight (exit status 1; not expected)

When I run the command '/bin/sh scripts/preflight.sh' from inside the container, I get the following output (maybe this might be the cause of the problem):
....
[14:26:42 UTC] INFO: AppConfig details
    appName: "united-storage"
    appEnv: "prod"
    appInstallation: "opensource"
    appDevMode: true
[14:26:42 UTC] INFO: Core-db is up and running!
node:internal/modules/cjs/loader:1228
  throw err;
  ^

Error: Cannot find module 'source-map-support'
Require stack:
- /opt/app/dist/server/index.js
- /opt/app/dist/server/db/scripts/migrate.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1225:15)
    at Module._load (node:internal/modules/cjs/loader:1051:27)
    at Module.require (node:internal/modules/cjs/loader:1311:19)
    at require (node:internal/modules/helpers:179:18)
    at Object.<anonymous> (/opt/app/dist/server/index.js:25:5)
    at Module._compile (node:internal/modules/cjs/loader:1469:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
    at Module.load (node:internal/modules/cjs/loader:1288:32)
    at Module._load (node:internal/modules/cjs/loader:1104:12)
    at Module.require (node:internal/modules/cjs/loader:1311:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/opt/app/dist/server/index.js',
    '/opt/app/dist/server/db/scripts/migrate.js'
 ]
}

Node.js v20.18.0

As far as I can see from the pull requests, the issue might already be resolved in the latest builds.

repo: datalens-tech/datalens-us


text:
multigtfs df attribute row sort

          Also applies to summarise_trips, row and column sort should be consistent with that of GtfsInstance:

row sort on day is: ['friday', 'friday', 'monday', 'monday', 'saturday', 'saturday', 'sunday', 'sunday', 'thursday', 'thursday', 'tuesday', 'tuesday', 'wednesday', 'wednesday']
column sort is: ['day', 'route_type', 'route_count_max', 'route_count_mean', 'route_count_median', 'route_count_min']
GtfsInstance column sort is ['day', 'route_count_max', 'route_count_mean', 'route_count_median', 'route_count_min', 'route_type']
Originally posted by @r-leyshon in https://github.com/datasciencecampus/transport-network-performance/pull/207#discussion_r1410359868

weekday row sort has been updated. Sort on route_type and column sort requires implementing:
GTFS.summarise_routes()

GTFS.summarise_trips()

Column sort is inconsistent.


text:
Move setup check to src

Description

Moved test logic to src code in order to get the ball rolling on coverage stats.
Fixes #(issue_number)
No issue raised.
Motivation and Context

Coverage failing as no src code. Introduced a placeholder file but better to have the test logic in src code. Moved this across and asserted outputs of unit met expected file pattern.
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

Test configuration details:
Tested with pytest using --runsetup flag. Test does not run without flag.

OS: macos
Python version: 3.8
Java version: 11
Python management system: pip

Advice for reviewer

Checklist:

[X] My code follows the intended structure of this project
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[X] Any dependent changes have been merged and published in downstream modules

Additional comments

Codecov Report
Patch coverage: 17.64% and project coverage change: +17.64 :tada:

Comparison is base (bec9fdb) 0.00% compared to head (66b955b) 17.64%.

Additional details and impacted files
@@           Coverage Diff            @@
##           dev      #12       +/-   ##
========================================
+ Coverage     0   17.64%   +17.64%
========================================
  Files        0        1        +1
  Lines        0       17       +17
========================================
+ Hits         0        3        +3
- Misses       0       14       +14

Flag
Coverage Δ

unittests
17.64% <17.64%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/heimdall_transport/gtfs/r5_helpers.py
17.64% <17.64%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Closing due to strategy change: all set-up tests in test_setup.py to keep src folder as clean as possible for package building.

authors: codecov-commenter ethan-moss r-leyshon


text:
ProductResultSetMapper doesn't properly handle null fields

Deserializing non-Option types can erroneously result in a null value if a previous column has a null value.
case class DBFoo(bar: Option[Int], baz: String)
trait FooDao {
  @SqlQuery("SELECT bar, baz from foo limit 1")
  @SingleValueResult
  def getFoo(): Option[DBFoo]
}

"Jdbi-scala" should "properly deserialize non-options" in {
    handle.execute("CREATE TABLE `foo` (`bar` INT, `baz` varchar(128) NOT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8;")
    handle.execute("INSERT INTO foo (baz) VALUES (?)", "this isn't null")
    handle.commit()
    val dao = dbi.onDemand(classOf[FooDao])
    val foo: Option[DBFoo] = dao.getFoo()
    foo.get.baz should equal("this isn't null")
  }

This results in a failure:
null did not equal "this isn't null"
I believe this is due to the order in which ProductResultSetMapper is identifying null values:
override def map(index: Int,
                   rs: ResultSet,
                   ctx: StatementContext): A = {
...
      t match {
        // todo: do we need to explicitly match on the java variations of these types too?
        case _ if t.isAssignableFrom(classOf[Option[_]]) =>
          Option(rs.getObject(i))
        case _ if !t.isPrimitive && rs.wasNull =>
          null
...
        case _ if t.isAssignableFrom(classOf[String]) =>
          rs.getString(i)
...
      }
    }

The case _ if !t.isPrimitive && rs.wasNull => line is using rs.wasNull which refers to the last retrieved field value (not the current field).  So if a previous column was null (and deserialized to an Option), the current field will get mapped to a null - even if the value of the current field is not null.
From the ResultSet docs:
/**
Reports whether
     * the last column read had a value of SQL NULL.
     * Note that you must first call one of the getter methods
     * on a column to try to read its value and then call
     * the method wasNull to see if the value read was
     * SQL NULL.
*/
boolean wasNull() throws SQLException;

One possible workaround would be to call rs.getObject(i) just just prior to the call to rs.wasNull but this will result in retrieving every field twice.  Another option might be to not allow null values that are not mapped to an Option.

Thanks for spotting this and figuring out the issue!
I've fixed this in master, and it'll be in the next release.

authors: eschuchmann nicktelford

repo: datasift/dropwizard-scala


text:
[pulsar connector] Do not use DS driver core shaded

Fix #67
Connector is using com.datastax.oss:java-driver-core-shaded but it is not needed since the non-shaded version is already imported by another dependency

Closing for #70. If you use your fork branch the DSE tests can't run

repo: datastax/cdc-apache-cassandra


text:
Release request

Hello,
What is the schedule for releases? We'd like to use a previously patched commit but we're just waiting for the next release.
Thank you again for the continuous patches.
This is the patch we're after.
https://github.com/datastax/pulsar-helm-chart/pull/141
Thanks!

Looks to be in the previous release, all good.

repo: datastax/pulsar-helm-chart


text:
Rename "GC Pauses" to "GC Time" in Pulsar JVM dashboards

source metric is not about GC pauses.
the source metric is "jvm_gc_collection_seconds_sum" which is not the same as "GC pause"
The metric is defined in Prometheus Java client

"Time spent in a given JVM garbage collector in seconds."
source code is https://github.com/prometheus/client_java/blob/e68daf23336eb5de7856df406eb1d497f51ad3be/simpleclient_hotspot/src/main/java/io/prometheus/client/hotspot/GarbageCollectorExports.java#L53

calls GarbageCollectorMXBean.getCollectionTime()

Javadoc https://docs.oracle.com/en/java/javase/11/docs/api/java.management/java/lang/management/GarbageCollectorMXBean.html#getCollectionTime()
"Returns the approximate accumulated collection elapsed time in milliseconds."

Stackoverflow Q&A: https://stackoverflow.com/a/44686539

"GC Collection Time" is redundant. It should be "GC Time".

"GC Collection Time" is redundant. It should be "GC Time".

@dave2wave thanks for pointing that out. fixed.

There's also a metric for GC Pauses in Pulsar. The metric name is jvm_full_gc_pause.
Here's the implementation:
https://github.com/apache/pulsar/blob/52d8fe03160648f14dc196edaa87921064a1b2b2/pulsar-common/src/main/java/org/apache/pulsar/common/stats/JvmDefaultGCMetricsLogger.java#L93-L94
The implementation does raise some questions since it's not exposed directly from the source metric. Instead, the logic is attempting to calculate the duration of the most recent GC pause. There might be issues with such conversion.
The source metric HotspotRuntimeMBean.getTotalSafepointTime is described in https://stackoverflow.com/a/42515743

Thanks @lhotari. Is there a way to get an accurate GC pause metric?

Thanks @lhotari. Is there a way to get an accurate GC pause metric?

Brokers already have a separate metric for this, jvm_full_gc_pause .
https://github.com/apache/pulsar/blob/4dcb166e0bfcce7fc85fd8d59a25b881f6f9c6fa/pulsar-common/src/main/java/org/apache/pulsar/common/stats/JvmDefaultGCMetricsLogger.java#L93

authors: dave2wave lhotari michaeljmarshall


text:
[improvement][devex] Add the ability to compute primitive schema type

Today the compute step only allows modifications of fields in an avro schema or a select set of headers (like destination topics). Attempt to modify the value for example for primitive types will fail with:
Invalid compute field name: value. It should be prefixed with 'key.' or 'value.' or 'properties.' or be one of [destinationTopic, messageKey]

The idea is to enable users to perform simple stuff like schema conversion or concatenating string types like:
{
  "steps": [
    {
      "type": "compute",
      "fields": [
        {
          "name": "value", // e.g this could be a STRING type with value 1.3
          "expression" : "value"
          "type": "DOUBLE" // effectively does the casting
        }
      ]
    }
  ]
}

or
{
  "steps": [
    {
      "type": "compute",
      "fields": [
        {
          "name": "value",
          "expression" : "fn:(value, '!'")
          "type": "STRING"
        }
      ]
    }
  ]
}

Fixed by #39

authors: aymkhalil cbornet

repo: datastax/pulsar-transformations


text:
Lamport Clock support for SharedContext

To try it out:
python check.py```

Grrr. I wanted develop.

repo: datawire/mdk


text:
20210114 - meeting

:alarm_clock: Time: 14 Jan 2021, 19:00 GMT
Just an hour hanging out and talking about Dat/hyper://* stuff. Now that Beaker has turned 1.0 🎂, we've got new opportunities to invite and involve hackers of all skill sets and levels!
:point_right: Post a comment in this github issue with any topics you'd like to cover this week
:link: https://zoom.us/j/181036238 (Native)
:link: https://zoom.us/wc/join/181036238?pwd= (Web)

Timezone
Date/Time

California
Jan 14 Thu, 11 AM

Colorado
Jan 14 Thu, noon

New York
Jan 14 Thu, 02 PM

Buenos Aires
Jan 14 Thu, 04 PM

London
Jan 14 Thu, 07 PM or 19:00

Berlin
Jan 14 Thu, 08 PM or 20:00

I'd like to talk about RON (Replicated Object Notation).
That is CRDT related stuff. I believe there was interest at the last meeting...

I'd like to talk about RON (Replicated Object Notation).
That is CRDT related stuff. I believe there was interest at the last meeting...

P.S. This RON http://doc.replicated.cc/^Wiki/ron.sm

P.S. This RON http://doc.replicated.cc/^Wiki/ron.sm

Interesting. Have you had a look at the kappa-db stuff to compare it?

Interesting. Have you had a look at the kappa-db stuff to compare it?

@DougAnderson444 kappa-db is a lightweight database, RON is a notation. kappadb employs JSON as a notation. kappadb is last-write-wins, RON is CRDT and everything else. So, the only overlap is the event-sourced approach.
On the technical side, kappa db is node.js while PLS and DaRWiN (RON things) are C++.
Although, if PLS will work on Hypercore one day, that would be nice.

@DougAnderson444 kappa-db is a lightweight database, RON is a notation. kappadb employs JSON as a notation. kappadb is last-write-wins, RON is CRDT and everything else. So, the only overlap is the event-sourced approach.
On the technical side, kappa db is node.js while PLS and DaRWiN (RON things) are C++.
Although, if PLS will work on Hypercore one day, that would be nice.

I'm pumped!

I'm pumped!

Notes
https://hackmd.io/W_mL2jIYQLOmlGIDz_0p_Q

Notes
https://hackmd.io/W_mL2jIYQLOmlGIDz_0p_Q

authors: DougAnderson444 gritzko micahscopes ninabreznik zootella

repo: datproject/comm-comm


text:
pathlib isn't supported anymore

Looks like pathlib is not actively maintained anymore. They recommend an updated pathlib2 library
https://pypi.python.org/pypi/pathlib2/

Thanks @kain88-de! From this discussion, it looks like what we need is to just use pathlib2 as our import instead of pathlib. Since we want to support < python 3.4 for now still, we'll do this.


text:
Change war file naming / context path and command line options

The default name JasperReportsIntegration.war is quite long and sometimes difficult for people to use, specifically since this will result in the context path /JasperReportsIntegration.
Thus, in the future the war file will be called jri.war which makes it a lot easier to type.
Also, we will add command line options similar to the ords.war file, e.g.:
java -jar jri.war configdir
java -jar jri.war configdir <new configdir>
java -jar jri.war encryptpasswords

The context path is changed from /JasperReportsIntegration to /jri. But you can still copy or rename the file jri.war back to JasperReportsIntegration.war ... and it will pick up /JasperReportsIntegration as context path again. Should also work in Weblogic, I have removed the context-root from the WEB-INF/Weblogic.xml file.

repo: daust/JasperReportsIntegration


text:
Update sbt-sonatype to 3.9.6

Updates org.xerial.sbt:sbt-sonatype from 3.8.1 to 3.9.6.
GitHub Release Notes - Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.xerial.sbt", artifactId = "sbt-sonatype" } ]

labels: sbt-plugin-update, semver-minor

Superseded by #231.


text:
Update sbt to 1.5.2

Updates org.scala-sbt:sbt from 1.3.13 to 1.5.2.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt", artifactId = "sbt" } ]

labels: library-update, semver-minor

Superseded by #465.

repo: davenverse/cormorant


text:
release v0.2.4 is missing go.sum entries

Hello,
We run into an issue while trying to use the release v0.2.4 of vacuum with pre-commit.
[INFO] Installing environment for https://github.com/daveshanley/vacuum.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/opt/hostedtoolcache/go/1.20.6/x64/bin/go', 'install', './...')
return code: 1
stdout: (none)
stderr:
    go: downloading github.com/dustin/go-humanize v1.0.1
    go: downloading github.com/pterm/pterm v0.12.62
    go: downloading github.com/spf13/cobra v1.7.0
    go: downloading gopkg.in/yaml.v3 v3.0.1
    go: downloading github.com/gizak/termui/v3 v3.1.0
    go: downloading github.com/santhosh-tekuri/jsonschema/v5 v5.3.0
    go: downloading golang.org/x/exp v0.0.0-20220909182711-5c715a9e8561
    go: downloading github.com/alecthomas/chroma v0.10.0
    go: downloading github.com/mitchellh/mapstructure v1.5.0
    go: downloading github.com/ghodss/yaml v1.0.0
    go: downloading go.uber.org/zap v1.24.0
    go: downloading github.com/json-iterator/go v1.1.12
    go: downloading atomicgo.dev/cursor v0.1.1
    go: downloading atomicgo.dev/keyboard v0.2.9
    go: downloading atomicgo.dev/schedule v0.0.2
    go: downloading github.com/gookit/color v1.5.3
    go: downloading github.com/lithammer/fuzzysearch v1.1.8
    go: downloading github.com/mattn/go-runewidth v0.0.14
    go: downloading golang.org/x/term v0.8.0
    go: downloading golang.org/x/text v0.9.0
    go: downloading github.com/spf13/pflag v1.0.5
    go: downloading github.com/mitchellh/go-wordwrap v0.0.0-20150314170334-ad45545899c7
    go: downloading github.com/nsf/termbox-go v0.0.0-20190121233118-02980233997d
    go: downloading github.com/dlclark/regexp2 v1.4.0
    go: downloading gopkg.in/yaml.v2 v2.4.0
    go: downloading github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421
    go: downloading github.com/modern-go/reflect2 v1.0.2
    go: downloading go.uber.org/atomic v1.7.0
    go: downloading go.uber.org/multierr v1.6.0
    go: downloading github.com/containerd/console v1.0.3
    go: downloading github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e
    go: downloading github.com/rivo/uniseg v0.4.4
    go: downloading golang.org/x/sys v0.8.0
Error:     model/rules.go:7:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi (imported by github.com/daveshanley/vacuum/model); to add:
    	go get github.com/daveshanley/vacuum/model
Error:     parser/json_schema.go:11:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi-validator/errors (imported by github.com/daveshanley/vacuum/functions/core); to add:
    	go get github.com/daveshanley/vacuum/functions/core
Error:     parser/json_schema.go:12:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi-validator/schema_validation (imported by github.com/daveshanley/vacuum/functions/openapi); to add:
    	go get github.com/daveshanley/vacuum/functions/openapi
Error:     model/results.go:5:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi/datamodel (imported by github.com/daveshanley/vacuum/cmd); to add:
    	go get github.com/daveshanley/vacuum/cmd
Error:     parser/json_schema.go:13:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi/datamodel/high/base (imported by github.com/daveshanley/vacuum/functions/core); to add:
    	go get github.com/daveshanley/vacuum/functions/core
Error:     motor/rule_applicator.go:17:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi/datamodel/high/v2 (imported by github.com/daveshanley/vacuum/motor); to add:
    	go get github.com/daveshanley/vacuum/motor
Error:     motor/rule_applicator.go:18:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi/datamodel/high/v3 (imported by github.com/daveshanley/vacuum/motor); to add:
    	go get github.com/daveshanley/vacuum/motor
Error:     parser/json_schema.go:14:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi/datamodel/low (imported by github.com/daveshanley/vacuum/functions/core); to add:
    	go get github.com/daveshanley/vacuum/functions/core
Error:     parser/json_schema.go:15:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi/datamodel/low/base (imported by github.com/daveshanley/vacuum/functions/core); to add:
    	go get github.com/daveshanley/vacuum/functions/core
Error:     functions/openapi/operation_descriptions.go:9:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi/datamodel/low/v3 (imported by github.com/daveshanley/vacuum/functions/openapi); to add:
    	go get github.com/daveshanley/vacuum/functions/openapi
Error:     model/rules.go:9:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi/index (imported by github.com/daveshanley/vacuum/cmd); to add:
    	go get github.com/daveshanley/vacuum/cmd
Error:     motor/rule_applicator.go:20:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi/resolver (imported by github.com/daveshanley/vacuum/motor); to add:
    	go get github.com/daveshanley/vacuum/motor
Error:     html-report/build_report.go:18:2: missing go.sum entry for module providing package github.com/pb33f/libopenapi/utils (imported by github.com/daveshanley/vacuum/functions/core); to add:
    	go get github.com/daveshanley/vacuum/functions/core

Oh dear! I am sorry about this. Thank you for providing a fix.

authors: daveshanley gnuletik

repo: daveshanley/vacuum


text:
Enhancement ; Rules for detecting non recommended HTTP DELETE + Payload

implement a warning rules when a DELETE has a requestBody as even it it can be supported is not a recommended pattern
could be set to INFO or Warning

I have written a custom plugin rule that checks that delete operations return 204 and contain no response body. I can contribute it as a PR on Monday

@lobocv  as the rules will looks similar if you have the solution for DELETE can you also push a rules for GET + payload as warning
btw here is a conversation for the rules
https://stackoverflow.com/questions/978061/http-get-with-request-body

Thanks for the bump! Sorry, this fell off my radar. I will try and get a PR up today or tomorrow. I have a few other things on my plate right now I need to prepare for.
I can definitely do that too @LasneF!

🤔 interesting , by no content i was not looking for HTTP response code that is here subject to debate toward simplicity and 'preference'
i was more thinking about the presence of the requestBody

Ah I see. I misunderstood. Yes the HTTP DELETE and GET operations should not have a request payload. I was thinking of response payload. You can do this with the DSL easily:

  http-delete-no-request-body:
    id: delete-no-request-body
    description: HTTP DELETE operations should not accept a request body
    type: style
    severity: error
    given: "$.paths[*].delete"
    then:
      - field: 'requestBody'
        function: undefined

  http-get-no-request-body:
    id: delete-no-request-body
    description: HTTP GET operations should not accept a request body
    type: style
    severity: error
    given: "$.paths[*].get"
    then:
      - field: 'requestBody'
        function: undefined

I am just finishing up a PR to add these functions. It was pretty easy. I'll post it soon. It handles get and delete methods

authors: LasneF lobocv


text:
nextcloud - change images to variables

create image variables so that users can define/overwrite the defaults

What this PR does / why we need it:
Allow users to choose what images (versions basically) to use for NextCloud. Current default is 14 which is 10 major versions behind the current version. For compatibility sake to OG users the default can remain at v14 and they can then override this when they choose to upgrade. It's recommended that NEW ansible-nas users override the defaults and use the current versions. (Maybe this should be a comment with the variables...hmmmm)
Which issue (if any) this PR fixes:
Fixes #
Any other useful info:
I know this has been brought up in the past but for whatever reason it hasn't been done. (Maybe because there were other changes that were incorporated at the same time that didn't pass the mustard?) So I figured let's make this simple and get it done.

I think this would be good opportunity to flip the image to latest too?

Problem is that I run NC 24 and SQL 8.  So someone else needs to actually test the DB migration that is going to be required for a full frontal upgrade.
I've used this a couple times in the past for upgrades, but not sure if it will work from 14 -> 24:
docker exec -u 33 nextcloud /var/www/html/occ maintenance:mode --off
When using the latest tag for NC, newer versions of NC will auto-update. The 10 version jump is problem.
Someone needs to be a Guinea pig and test. Sorry, but I don't have the time to go through it again.

repo: davestephens/ansible-nas


text:
Scroll lags heavily on the website

There's some problem with http://davetron5000.github.io/methadone/
It's only there do I get so heavy lagging when wheel-scrolling. Very heavy lagging.
PS Thanks for a great gem.

What browser/OS?  It seems OK to me but I only have access to a macbook pro.

It's Ubuntu 16.04, Chrome. Still lags with all extensions disabled. But in Guest mode it's alright. Firefox is fine too. No idea why's that. Never ran into this before.

Can't repro this, plus it's been a few years.

Yep, probably was a problem on my side, sorry :man_shrugging:

authors: davetron5000 nicktime staring-frog


text:
Add option to disable or activate user not in expert mode

in myuser.yaml i had state: disabled by default, a choice  because i’d like that admin enable them one by one. Of course he can enabled them by editing the yaml in expert mode, i think it would be great to have a toggle button to enable  or disable them

Added in v2.2.0

@david-szabo97 thanks for this update


text:
Add Plugin repository links to docs

To Do
Add a GitHub repository link on each plugin page. Suggestion:

Solved with this commit.

repo: davidcetinkaya/embla-carousel


text:
Does not clear preRelease

When set preRelease and then coment the line on build.gradle the prerelease still is in the apk.
We have to ser prerelease = "", and then coment the prerelease line

@jrcacd, thanks, the file values overwrites the field in memory which is empty.
I will publish a new patch version for it.

authors: davideas jrcacd

repo: davideas/GrabVer


text:
Fix for issue #32

This commit solves issue #32 and passes all tests.

I didn't notice it fails with python 2.6. I'll try to fix it, but this is a necessary merge to comply JSON-RPC specification.


text:
feat: Add ability to read document contents

Adds a new method documents::contents() that allows you to read the
contents of the Firestore document without deserializing the JSON

Thanks, looks good

Thanks, looks good

authors: JadedEvan davidgraeff

repo: davidgraeff/firestore-db-and-auth-rs


text:
Separate "Timed Race" and "Timed Trial" game modes

Why are these two listed under the same abbreviation?
In Timed Trial (should be named "Time Challenge") you have to reach time gates to increase your remaining time and not get eliminated, whereas Timed Race is the same as a Speed Race but with a time limit. They are considerably different modes. If any should be grouped together it should be Timed Race and Speed Race.

I forgot why I typed it as “Timed Trial”, but I think you’re right, I should have separated those two modes.

Any preference for the abbreviations?
e.g.

Mode
Abbr.

Timed Race
Tr

Time Challenge
Tc

Ah, wait. Was just going through these and found a "Time Trial", so there are actually 3 different time-related game modes.
For reference:

Mode
First Chapter

Time Challenge
2-5

Timed Race
1-1

Time Trial
7-2

Are you able to confirm which of the Chinese game mode names match up to each?

Any preference for the abbreviations?

Yeah that looks good. As for Time trail, perhaps Tt?

Abbr.
En
Zh

Tc
Time Challenge
限時挑戰賽

Tr
Timed Race
限時賽

Tt
Time Trial
計時賽

Thanks.
Similar, yes. I suppose that is why you combined them before. Also, there are only 3 Time Trials.
Time Trial is the normal Time Trial mode; essentially Speed Race without any opponents. Timed Race on the other hand is a Speed Race but with a time limit high enough that you shouldn't ever fail; not sure why they made that one.

authors: FauxBlue davidhcefx


text:
To learn

[ ] Azure Virtual Desktop
[ ] Azure Front door
[ ] Azure Migrate
[ ] Strong experience in Azure database platforms or integration tools such as Azure Data Factory, Azure Databricks, Azure Synapse, and Data Lake Storage Gen2, etc.

Certificate
https://learn.microsoft.com/en-us/credentials/certifications/exams/dp-203/
https://learn.microsoft.com/en-us/credentials/certifications/exams/dp-100/

repo: davidkhala/azure-utils


text:
Hierarchical onExit Actions

Disclaimer: This might be against state machine convention.

Bug
Parent states don't expose the onExit actions of their nested children.
This means that any possible cleanup of the child needs to currently happen on the parent state.
Description:
Using the example of the pedestrian traffic light, on exit of the red state doesn't expose the exit actions of the current active pedestrian state.
(Bug) Expected result:
On exit of a root lightMachine state(red), all onExit actions of the active pedestrian state(stop) will be passed up the tree.
(Bug) Actual result:
On exit of a root lightMachine state(red), the active pedestrian state(stop) is unaware of the change.
(Bug) Potential fix:
Same method as grabbing the initial states might possibly apply to the on exits of a transition.

Hmm. I've tried reproducing this, but when I try a simple onExit on a child state it does fire.  Check out the debugger — click "parse" and then the "initial" button then "flick" three times.  It goes from "off.B" to "on" and triggers B's onExit.

Oh wow, thank you so much, I logged this on my team's behalf and probably find we're on an older version. Will close this ticket and ensure it is updated.

authors: mogsie shawnnosaurus

repo: davidkpiano/xstate


text:
@xstate/immer initial release

This PR adds the initial implementation of @xstate/immer
import { createMachine, interpret } from 'xstate';
import { assign, createUpdater } from '@xstate/immer';

const levelUpdater = createUpdater('UPDATE_LEVEL', (ctx, level) => {
  ctx.level = level;
});

const toggleMachine = createMachine({
  id: 'toggle',
  context: {
    count: 0,
    level: 0
  },
  initial: 'inactive',
  states: {
    inactive: {
      on: {
        TOGGLE: {
          target: 'active',
          // Immutably update context the same "mutable"
          // way as you would do with Immer!
          actions: assign((ctx) => ctx.count++)
        },
        // Use the updater for more convenience:
        [levelUpdater.type]: {
          actions: levelUpdater.action
        }
      }
    },
    active: {
      on: {
        TOGGLE: { target: 'inactive' }
      }
    }
  }
});

const toggleService = interpret(toggleMachine)
  .onTransition((state) => {
    console.log(state.context);
  })
  .start();

toggleService.send('TOGGLE');
// { count: 1, level: 0 }

toggleService.send(levelUpdater.update(9));
// { count: 1, level: 9 }

toggleService.send('TOGGLE');
// { count: 2, level: 9 }

toggleService.send(levelUpdater.update(-100));
// Notice how the level is not updated in 'inactive' state:
// { count: 2, level: 9 }

cc. @mweststrate Would love to get your brief thoughts on this!


text:
Another Crash!

So the crashes I had before ( #152 ) were fixed.  But now I get this every so often.
/root/MEGA/node_modules/agario-client/node_modules/ws/lib/WebSocket.js:426
event.type = 'error';
^
TypeError: Cannot assign to read only property 'type' of no handler for opcode 15
at WebSocket.onError (/root/MEGA/node_modules/agario-client/node_modules/ws/lib/WebSocket.js:426:16)
at emitTwo (events.js:100:13)
at WebSocket.emit (events.js:185:7)
at Receiver.onerror (/root/MEGA/node_modules/agario-client/node_modules/ws/lib/WebSocket.js:851:10)
at Receiver.error (/root/MEGA/node_modules/agario-client/node_modules/ws/lib/Receiver.js:321:8)
at Receiver.processPacket (/root/MEGA/node_modules/agario-client/node_modules/ws/lib/Receiver.js:234:43)
at Receiver.add (/root/MEGA/node_modules/agario-client/node_modules/ws/lib/Receiver.js:95:24)
at Socket.realHandler (/root/MEGA/node_modules/agario-client/node_modules/ws/lib/WebSocket.js:800:20)
at emitOne (events.js:90:13)
at Socket.emit (events.js:182:7)

I'm using node v0.12.9 (I'm using minimap too) and never an issue.
Try it and lets see if you have the same issue.

@WJKH link to that version? (p.s. im using latest version)

I'm not using last feeder version, I'll update soon and test it too.
Try it https://nodejs.org/download/release/v0.12.9/

lol just comment out the lines the error occur on, it works!

authors: MasonBurdette WJKH


text:
Update pub packages

The packages in pubspec.yaml are very out of date. I am using this in a project that uses built_redux. However, built_redux_thunk depends on a much lower built_value version. This results in the following output upon running pub get in my project:
Because built_redux_thunk >=0.5.1 depends on built_value >=4.1.0 <7.0.0 and myproject depends on built_value ^7.1.0, built_redux_thunk >=0.5.1 is forbidden.
So, because myproject depends on built_redux_thunk ^0.5.1, version solving failed.
Running "flutter pub get" in my_project...
pub get failed (1; So, because myproject depends on built_redux_thunk ^0.5.1, version solving failed.)

I initially was going to update only the built_value version. However, there were some other issues when compiling the tests (shown in attached file).
compilation-errors.txt
After spending some time in version conflict hell, the resulting pubspec.yaml has had nearly every package upgraded. The test now compiles and runs successfully.
Please let me know if there are any changes you'd like me to make. (I'm also considering another PR to add a Travis-CI build if that's okay with you. There's already a test which a CI pipeline could be used to validate a PR with. Let me know what your thoughts are on this.)

any progress on this ticket?

authors: IncPlusPlus jngbng


text:
Intention: to detect more number of  faces using the method bulk _detect _face

How to detect Bulk Faces in a image:
what  i did:
1.i have an image with 10 faces of different sizes.
2.i run mtcnn and call the detect face method from detectface.py file
3.i got 3 detected faces.
what i want:
1.i plan to run bulk detect face  method to find 10 faces.
2.what value should i assign for variable "detection_window_size_ratio"
3.how to run to get 10 faces in an image.

The normal face detector detects multiple faces. But in later stages in the alignment process the largest and most centered face are selected.

authors: davidsandberg harishank

repo: davidsandberg/facenet


text:
Pipeline interface

Hello,
I have been reading the code and been trying to make more elaborate examples. Currently do not understand how
to properly use the pipeline interface (or there is a bug).
I would like to get the hello_world example working again and Having read https://zipcpu.com/blog/2017/08/14/strategies-for-pipelining.html
At present it is not clear to me how the pipeline interface is supposed to work.
A few things I noticed

uart_in_ready is only valid when the state of the pipeline is CycleData
when setting uart_in_valid I do not know the state of the pipeline. it looks like I will need to keep uart_in_valid high for an unknown amount of time.

Would it be possible to explain how the interfaces is  supposed to be used?
While the example (uart/echo) is working I am wondering why
A good start for me would be a quick review of this code
https://github.com/keesj/tiny_usb_examples/blob/keesj/hello_world/src/top.v
but explanations would be greatly appreciated!

Ready Valid is at once incredibly simple and a total head-scratcher.   What could be simpler, than "offer" and "acceptance" or "give" and "take".  Things change hands when both signals are present at the same time.  And yet, for the beginner (and me) when thinking in terms of FPGA it can get really complex when thinking about what is being asserted and when.
Take a look at this Pipeline tutorial Design section for a little more detail.
https://davidthings.github.io/spokefpga/pipelines#design
As is noted in the Appendix there you are actually attempting the hardest kind of pipeline code - the middleman.
My suggestion is to start by separating the incoming pipe from the outgoing one.  Maybe set the pipe in ready signal to true, and letting the in valid blink an LED.  Then you know when you're getting an incoming char.  On the outgoing side, use a timer (one of the bits of a counter) to trigger a sequence that sets the pipe out valid and loads pipe out data with the next character.  When the pipe out ready goes high, you've had a transfer and you can go to the next char or finish.
What you're attempting is a head melting place to start.  Start simple.
It should be noted that tying the "in valid" signal to the trigger will next get you what I think you're trying to do.
Keep us posted with how you do.

Thank you very much for the info. it is much easier to implement when you know the design behind. I got the hello world working and will continue my endeavors.

From debugging it appears to me that uart_in_ready will only become 1 uart_in_valid is first set to 1.
my guess at this point is that if uart_in_valid is not set no data out is needed and the ep does not become ready. is the interface in need of an enable signal?
and does the following code look sane?
https://github.com/keesj/tiny_usb_examples/blob/keesj/hello_world/src/top.v#L118

Will check the code later, but in the mean time, yes... the uart_in_valid signal is used to signal to the usb system that there is data to send.  The usb system then creates the request to the host and allocates a buffer.  No characters are dropped and the contract of the ready-valid pair is honored.  However, all the ready valid code in here was written by a (then!) Verilog noobie.  Now I would do it differently (better, I hope).  I would signal ready, then when a character is valid, I'd consume and buffer it, and drop the ready signal until all the buffers were ready etc.

For your peace of mind, I will offer that I have used this code in perhaps 20 or more miniprojects including several that are aimed purely at getting 100% performance out of the interface.  It does work!

authors: davidthings keesj


text:
Segmentation fault (core dumped)

python3
import dlib
Segmentation fault (core dumped)

Same issue on lastest version.(v19.24)
My platform is NVIDIA Jetson AGX.
Problem solved after downgrading to v19.22 .

authors: Glory-Peng mr-cn

repo: davisking/dlib


text:
Segmentation fault (core dumped) error

when I run python test_gpus.py, I got error "Segmentation fault (core dumped) error" seems NCNN not able to find any GPU> Following is the ouptput of command vulkaninfo, my environment is : OS: Ubuntu 22.02 and Graphic card: RTX 4090
Any help would be highly appreciated.
VulkanInfo.txt

Can you try this?
https://github.com/Tencent/ncnn/wiki/FAQ-ncnn-vulkan
I'm running an Intel Arc a380 with Ubuntu Linux.
But my main processors are a Intel Arc a750 and Nvidia RTX a4000 on Windows.
It was a pain to get the Intel card work in Linux since it was so new with very little driver support in the Linux kernel..


text:
Added support for push notifications

Added support for:
https://mixpanel.com/help/reference/ios-push-notifications
https://mixpanel.com/help/reference/android-push-notifications

Possible to merge this @davodesign84 ?

Hello @davodesign84 ??

@skeie can you provide some documentation on how to set up the project on the JS side to capture the push notifications and read their data?

Indeed @dwilt, it lacks some methods to get it working on the JS side. For instance on iOS we should call [[UIApplication sharedApplication] registerForRemoteNotifications]; and we can't right now.

@Kerumen Thanks for the reply. So for now then, push notifications can't be used at all with this plugin until more work and documentation is provided, correct?

Yes. I'm not the original committer tho, I just realize that the work was half done.

For now we could probably just use https://github.com/zo0r/react-native-push-notification on the client side and use Mixpanel for the server-side push notification pushes correct?

Correct. But, as said before, you can't configure push notifications right now with this library. Hence you must stick with local notifications.

If I used the Mixpanel service to send a push notification and https://github.com/zo0r/react-native-push-notification to listen for push notifications, I shouldn't need this plugin at all correct (for push notifications)?

Oh right you can call PushNotification.configure() and in the onRegister callback call initPushHandling with the token generated. This should work.

@dwilt @Kerumen have either of you tried the  https://github.com/zo0r/react-native-push-notification workaround?

@alberthu16 I haven't no, but I might be coming to this point soon. I'll try posting any successes I make here.

@dwilt thanks :) it seems to me that push notifications has been implemented though.. I was running into issues while I was on v0.0.10 of this library, but updating to v0.0.14 seemed to fix it.

@alberthu16 can you share some of the code examples that you have to have gotten push notifications to work with mixpanel? I'd love to see how you did it.

@dwilt will do once I'm done with the process! Still in the middle of hooking things up.

@alberthu16 Awesome, thanks man.

authors: Kerumen alberthu16 dwilt skeie


text:
🛑 AdServer is down

In 3b2c07f, AdServer (https://ads.davecross.co.uk/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: AdServer is back up in 7b54af0 after 3 hours, 5 minutes.

repo: davorg/uptime


text:
speed up for neon intrinsics impl

Tested on my MI8 phone (SoC is QC845)
The previous implemention:
non-neon: 6ms
noen: 8ms
Updated implementation:
non-neon: 6ms
neon: 4ms~6ms (not very stable, but faster than previous)
Also fix the missing sum of remain (tail) part of array.

Hi zchrissirhcz,
Thanks for your interest in this repository. I merged, the pull request.
Regards,
DB

authors: dawidborycki zchrissirhcz

repo: dawidborycki/NeonIntrinsics-Android


text:
Adjust width

I just generated this image:

It'd be great if:

I could make the font size bigger (ref #46)
but also if I could shrink the width of the code block manually (or it adjust automatically based on longest line)

Hi, I think I can give it a try.

See #33 and #1 for discussion reference as well. I also believe @jakedex is working on font-size fix

I can create new setting next to Window controlls which will enables automatic width adjust based on longest line. IMHO manual width adjustment is not a bare necessary thing. What do you think?

Agreed, automatic would be optimal.

I'm totally fine with that. Automatic is a great start and we can leave #1 open in case someone wants to take a shot at manual.

repo: dawnlabs/carbon


text:
Creating a project from a commit URL fails if the commit is not from the default branch

          There seems to be an issue with creating a project from a commit URL (e.g. http://localhost:3000/test/test/commit/3aabec775f9b206bfe454563ef0d441a2181fa81). I get the following error:

Note: This commit is not from the main branch.
Originally posted by @Tpuljak in https://github.com/daytonaio/daytona/pull/689#discussion_r1645614508

The issue is cause by https://github.com/daytonaio/daytona/blob/main/pkg/git/service.go#L48
Because we clone only the default branch, the checkout fails because the commit is not present in the current git tree.
I suggest that we address this by first checking to which branch does the commit belong to and then cloning that branch first.

/bounty $100

This appears to be interesting. /attempt #701

Running the same command, daytona create http://localhost:3000/test/test/commit/3aabec775f9b206bfe454563ef0d441a2181fa81 -t local I got a different error:
FATA[0000] failed to get git provider for url: git provider not found
exit status 1

I could also generate other errors using github URLs, but not the error mentioned in the issue. Can you folks give me a hand in reproducing this?

Hi @ologbonowiwi, the URL provided in the issue was just an example.
You can test this properly by running daytona create https://github.com/daytonaio/daytona/commit/f7a4bf46417f0c7fc2e08eeb4f25c27235778a08
This should create the project on a branch and you'll notice that it creates it on main.

@Tpuljak  Is this happening with gitness or all other?

Not sure.
I've tried multiple github links (including the one provided on the comment above and some of my own repos) but I got a different error complaining about not being a tree (instead of this object not found), so technically I wasn't able to reproduce it.

@Tpuljak @ologbonowiwi Is this happening with gitness or all git providers ?

@the-johnwick it's happening for all git providers

I've tried multiple github links (including the one provided on the comment above and some of my own repos) but I got a different error complaining about not being a tree (instead of this object not found), so technically I wasn't able to reproduce it.

That's the issue 😄
Git providers might have different error messages but all of that should be solved as part of this issue.

@Tpuljak Oh, I got it now. I have completed the work for #763 please review it. Till then I will solve this issue.
/attempt

@the-johnwick no need to tag us for reviews. We see everything going on in the repo.
You are welcome to start working on this right away. We can not guarantee when we will review the AWS PR.

No problem. I thought you guys will not allow me to work on second issue since I have one pr opened. I will start working on it.

@the-johnwick this issue is a bit older and we refactored how the repo gets cloned in the meantime.
The clone function I mentioned there is now ignored because the project is cloned beforehand. You can check out the creation process here https://github.com/daytonaio/daytona/blob/main/pkg/docker/create.go.
In order to test changes made in the docker lib, you can refer to this PR that will add a README into the package directory.
If you have any trouble testing this, let me know.

@Tpuljak I am getting error setting up daytona-provider-docker on go mod tidy

@the-johnwick please rebase the docker provider and daytona with main

authors: Tpuljak idagelic ologbonowiwi the-johnwick

repo: daytonaio/daytona


text:
Upgrade prompt_toolkit to 0.52.

@eliangcs This is an upgrade of the prompt toolkit library to 0.52. I'd highly encourage you to test this PR thoroughly before merging.
Changelog for prompt_toolkit (https://github.com/jonathanslenders/python-prompt-toolkit/blob/master/CHANGELOG).
Here are the pending PRs for pgcli and mycli for this exact same change.
pgcli - https://github.com/dbcli/pgcli/pull/362
mycli - https://github.com/dbcli/mycli/pull/159

I had to disable the mouse support since it made it impossible to copy the output.

Don't merge this one. There is a bug in prompt_toolkit where the prompt has extra characters and the enter key seems broken in windows.

Sorry for late reply. Although you have discard the branch, I tested it using behave on Mac and it didn't pass.

Yes there was a bug in prompt_toolkit that was adding a few extra characters in the prompt.

authors: amjith eliangcs

repo: dbcli/vcli


text:
Don't sort xml attribute with auto format functionality.

Right now when working on xml string (xml data type) either through Value Panel using Auto Format or on editor (Shift + Enter) -> choose xml -> Ctrl-Shift-F for auto formatting, the xml attributes is also sorted alphabetically.
Need a way to preserve the actual data when auto formatting as there is a reason where the attribute is located in the string.
Thank you.

@mhandoko-fff Hello. Can you provide more detail? What DB are you using. It will be great if can provide an example to more specifically see what and when is being sorted.

The DB is MS SQL 2017 with xml type column. The xml stored in the table/column is basically a one long string.

If I format the string outside DBeaver, it will look like below (please notice the attribute next to container that it will go from containerid - depth - width - height - etc)

If I format the same sting either through value panel or xml editor, it will look like below as the xml attributes were sorted alphabetically

@mhandoko-fff Thank you for the report

Thank you for the getting it in for the next release.

Fixed.

Verified

authors: LonwoLonwo Matvey16 mhandoko-fff

repo: dbeaver/dbeaver


text:
Cannot see column names within tables/views or console

Hello,
I am new to DBeaver and have came across issue:  cannot view column names of tables or views.  Have tried various options to visualize them with no luck.  Is there a setting within the environment that will correct this; or a particular version of DBeaver?  I had the current version with this issue and did an uninstall and reinstall of prior version that an associate was using and was able to see the column names.  Both the current version and 21.1.5 did not correct my issues.  Please advise.
System information:

Edition	Windows 10 Enterprise
Version	21H1
Installed on	10/5/2021
OS build	19043.928
Experience	Windows Feature Experience Pack 120.2212.551.0

DBeaver tried both current version and 21.1.5 (another user had this previous version and column names were available)

using Redshift AWS

unable to see column names within tables or views, and within properties/columns console.
Steps to reproduce, if exist:  click drop down from table/view; 'loading' appears but column names do not.  clicking on view diagram/properties does not show views either.

I can recommend an Advanced connection view.

And again - screenshots can help us.

This would be an example of the script that is used creates a copy of
another schema/view and would be similar to:
'create or replace view CopyView.v_tableA as select * from
SourceView.tableA' with no schema binding'.
Does that make sense?
On Thu, Oct 21, 2021 at 2:03 PM Anastasiya @.***> wrote:

@kkoehleranalyst https://github.com/kkoehleranalyst
An example of table/view DDL (without columns) will be helpful. And info -
is the table in external schema or not.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dbeaver/dbeaver/issues/14331#issuecomment-948916807,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWFBAMV6EEEIIRFJHWBQCFDUIBPY7ANCNFSM5GOJCH2Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

SourceView.tableA DDL will also be useful.

there is no update on the issue for a long time. if it is still actual for you - feel free to reopen it with additional info

authors: LonwoLonwo kkoehleranalyst uslss


text:
Display line number in SQLServer error

If my query has an error in the SQL, when I run it in sqlcmd, I get:
Msg 207, Level 16, State 1, Server MYSERVER, Procedure MyProc, Line 40
Invalid column name 'Id'.

In DBeaver I don't get the line number:
Error occurred during SQL query execution

Reason:
SQL Error [207] [S0001]: Invalid column name 'Id'.

Hello @powelleric
Ok, maybe we can try do this in the ticket https://github.com/dbeaver/dbeaver/issues/15320
Closed as duplicate

authors: LonwoLonwo powelleric