text:
Add cloudpickle to dask.dataframe dependencies

This PR adds cloudpickle as a dependency for Dask DataFrame. In #5804 we added a lambda function that results in a non-deterministic key test failures when cloudpickle isn't installed (xref https://travis-ci.org/dask/dask/jobs/647052775#L833-L865).
Alternatively, we could just move the lambda to a top level function in dask/dataframe/core.py that can be serialized by pickle. However, I suspect adding cloudpickle as a dependency isn't too big of a deal and will help us avoid this type of thing in the future.
cc @TomAugspurger

[ ] Tests added / passed
[ ] Passes black dask / flake8 dask

I think I'm -0 on this.  My motivation here is that occasionally we do have to talk to potential downstream projects and convince them to depend on dask[foo].  Some of these projects are pretty sensitive (arguably too sensitive).  As an example, I think that Scikit-Image has depended on dask[array] in the past, and my guess is that seeing cloudpickle in the list of dependencies would be a turn-off.
The cloudpickle issue is also a little bit nice just to force the issue from a performance perspective, but I think that the benefits there are probably marginal.
https://github.com/dask/dask/pull/5901

Agreed with removing the lambda for now. It'd be nice to have some kind of
linter to catch this for those of us who are forgetful :)
On Fri, Feb 14, 2020 at 2:43 PM Matthew Rocklin notifications@github.com
wrote:

I think I'm -0 on this. My motivation here is that occasionally we do have
to talk to potential downstream projects and convince them to depend on
dask[foo]. Some of these projects are pretty sensitive (arguably too
sensitive). As an example, I think that Scikit-Image has depended on
dask[array] in the past, and my guess is that seeing cloudpickle in the
list of dependencies would be a turn-off.
The cloudpickle issue is also a little bit nice just to force the issue
from a performance perspective, but I think that the benefits there are
probably marginal.
#5901 https://github.com/dask/dask/pull/5901
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/pull/5900?email_source=notifications&email_token=AAKAOIUJXDWNLGPTCNYSAMDRC3653A5CNFSM4KVNT3OKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL2L6OA#issuecomment-586465080,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKAOIUIX5ZV7LBQOGOOJR3RC3653ANCNFSM4KVNT3OA
.

Closing in favor of #5901

authors: TomAugspurger jrbourbeau mrocklin


text:
Array optimization: cull before fuse_roots

Because fuse_roots materializes Blockwise layers, if you have a situation where you're sub-selecting a few items out of a large initial Blockwise array, culling the graph (cheap) before materializing it (slow) can be massively more performant. As more becomes Blockwise (https://github.com/dask/dask/pull/7417), this will become more relevant. For example:
a = da.from_zarr("s3://10-trillion-chunk-dataset/a")
b = da.from_zarr("s3://10-trillion-chunk-dataset/b")
metric = a / b
tiny_part = metric[:10]
# Without this PR, optimizing `tiny_part` could require materializing the 10-trillion-chunk
# `from_array` graphs, just to cull them down to nothing a moment later

I doubt this is exactly correct—maybe we need to cull again after fuse_roots? (Culling is cheap but not that cheap; I'd like to not do it twice if possible.)
Because it materializes, I also think fuse_roots should perhaps go after the if config.get("optimization.fuse.active") is False: return dsk bailout.
Frankly, I don't understand fuse_roots. The situation described in https://github.com/dask/dask/pull/5451#issuecomment-552983458, and the test for it, are already handled by plain Blockwise fusion. In fact, if I remove fuse_roots entirely, test_fuse_roots still passes!
cc @rjzamora @ian-r-rose

[x] Tests added / passed
[x] Passes pre-commit run --all-files

Although I’d like to remove HighLevelGraph.cull completely medium-to-long term (along the lines of #9216), I do think it makes sense to address the outstanding issue that: (1) fuse_roots materializes Blockwise layers, and (2) the specific cases that benefit from fuse_roots are poorly understood and not even captured by CI.
I spent some time exploring fuse_roots myself, and I think I agree that the optimization should probably come after the "optimization.fuse.active” bailout, because it is indeed materializing the graph for the purpose of low-level fusion. Also, most of the cases it is designed to target should be covered by Blockwise IO or general low-level fusion.
One case where fuse_roots is currently important is dask.dataframe’s  from_delayed.  With that said, I don’t see any reason why we couldn’t implement a simpler delayed-specific optimization for this (one that doesn’t materialize the target Blockwise Layer).

I don’t see any reason why we couldn’t implement a simpler delayed-specific optimization for this (one that doesn’t materialize the target Blockwise Layer)

I'm all in favor of removing fuse_roots entirely and doing this instead!

I'm all in favor of removing fuse_roots entirely and doing this instead!

I took a rough pass at something like this in https://github.com/dask/dask/pull/9273

source: gharchive/pull-request

repo: dask/dask

license_type: permissive

license_source: github-api


text:
google.auth.exceptions.RefreshError with excessive concurrent requests.

gcsfs propagates an google.auth.exceptions.RefreshError when executing many concurrent requests from a single node using the google_default credentials class. This is likely due to repeated, excessive number of requests to the internal metadata service. This is a known bug of the external library at GoogleCloudPlatform/google-auth-library-python#211.
Anecdotally, I've primarily observed this in dask.distributed workers and believe this might occur due to the way GCSFiles are distributed. This primarily occurs when a large number of small files are being read from storage and many worker threads are performing concurrent reads. I believe the GCSFiles serialized in dask tasks then each instantiate a separate GCSFilesystem, resolve credentials and open a session.
If this is the case it would be preferable to store a fixed set of AuthenticatedSession handles, ideally via cache on the GCSFilesystem class, and dispatch to an auth-method-specific session in the GCSFilesystem._connect_* connection functions.
As a more specific solution, google.auth.exceptions.RefreshError or its base class should be added to the retrying exception list in _call, however this may mask legitimate authentication errors. The credentials should probably be "tested" via some call that does not retry this error during session initialization. This may be as simple as calling session.credentials.refresh or performing a single authenticated request after session initialization.

When the scheduler farms out tasks to workers, they indeed all individually authenticate themselves (unless you explicitly pass a token, which is unsafe and generally still needs to be refreshed). This should only happen the first time, after which the existing instance should be reused. I am including this for information only.

Can you think of a case where RefreshError should not lead to a retry? I am thinking that this is not the error you get if the call is processed completely by the server, but the auth details are somehow invalid.

I haven't had a chance to read though the google.auth documentation or code deeply enough to understand if this error is thrown for invalid refresh requests or just due to temporary errors during token refresh. This would probably require some grep-ing.
@martindurant to your comment above, do you mean "this should only happen the first time" as in "this in the currently implemented behavior of the system" or "this behavior should be implemented"? I believe your comment in https://github.com/pangeo-data/pangeo/issues/112 using __setstate__ to detect whether a cached session can be used is the right starting place, though probably requires some logic updates to make sure the correct auth method is being used.

Is gcsfs thread-safe? A dask worker could be running multiple threads.

gcsfs is used routinely with Dask, but does not guarantee thread-safety. Specifically, if you have the same set of parameters when instantiating (which would be true for your example), you only create one instance and share it, so only one auth request is sent. However, the underlying library requests is almost, but not entirely thread-safe: apparently it is possible for connections to be dropped if a pool fills up; but this case would seem very unlikely in this kind of use.
Directory listings could also potentially fall out of sync, but the code aggressively purges the cache when writing, and in the dask scenario, listings are usually done just once in the client.

source: gharchive/issue

authors: asford martindurant skeller88

repo: dask/gcsfs


text:
Fixes pangeo-data/pangeo #176

Google apparently changed their error message when a key does not exist.

Thanks @martindurant for the quick fix!

@rabernat are you able to verify that this resolves the issue?
On Thu, Mar 22, 2018 at 3:10 PM, Martin Durant notifications@github.com
wrote:

Merged #92 https://github.com/dask/gcsfs/pull/92.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/gcsfs/pull/92#event-1536385666, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AASszKSVOOTvLwlG5Y-dGQXMKxWL4N-Nks5tg_clgaJpZM4S3fCA
.

If so then I'll push a new image up
On Thu, Mar 22, 2018 at 3:12 PM, Matthew Rocklin mrocklin@anaconda.com
wrote:

@rabernat are you able to verify that this resolves the issue?
On Thu, Mar 22, 2018 at 3:10 PM, Martin Durant notifications@github.com
wrote:

Merged #92 https://github.com/dask/gcsfs/pull/92.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/gcsfs/pull/92#event-1536385666, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AASszKSVOOTvLwlG5Y-dGQXMKxWL4N-Nks5tg_clgaJpZM4S3fCA
.

@rabernat are you able to verify that this resolves the issue?

Yes, it resolves the issue. 👍

authors: martindurant mrocklin rabernat


text:
Iterating over a column

In the last meeting, there was a section when we talked some default which, if pandas changed, would make things easier for other libraries.
One example which @kkraus14 gave, at about 0:43:30, was about iterating over a column in Python-land, value by value.
@shwina mentioned that polars doesn't allow this.
I may have misunderstood what Keith was referring to, but to me it looks like it is allowed?
In [77]: df = pl.DataFrame({'ts': pl.date_range(datetime(2020, 1, 1), datetime(2020, 1, 10))})

In [78]: for i in range(len(df)):
    ...:     print(f'value at position {i} is: {df["ts"][i]}')
    ...:
value at position 0 is: 2020-01-01 00:00:00
value at position 1 is: 2020-01-02 00:00:00
value at position 2 is: 2020-01-03 00:00:00
value at position 3 is: 2020-01-04 00:00:00
value at position 4 is: 2020-01-05 00:00:00
value at position 5 is: 2020-01-06 00:00:00
value at position 6 is: 2020-01-07 00:00:00
value at position 7 is: 2020-01-08 00:00:00
value at position 8 is: 2020-01-09 00:00:00
value at position 9 is: 2020-01-10 00:00:00

Could you please clarify what it is that polars doesn't allow, which you would like it if pandas disallowed?
EDIT: the above example requires
import polars as pl
from datetime import datetime

to run

@MarcoGorelli you're using a __getitem__ call with a scalar as opposed to an __iter__. That being said it does look like polars allowing iterating over a column. cuDF on the other hand explicitly poisons the method because there's no way to do it reasonably on the GPU.
import polars as pl
import cudf
from datetime import datetime

pl_df = pl.DataFrame({'ts': pl.date_range(datetime(2020, 1, 1), datetime(2020, 1, 10))})
cudf_df = cudf.DataFrame.from_arrow(pl_df.to_arrow())  # cudf supports everything above, I'm just lazy

# This works for polars
for val in pl_df['ts']:
    print(val)

# This loudly errors that Series object is not iterable.
for val in cudf_df['ts']:
    print(val)

There's also
In [104]: df = pl.DataFrame({'ts': pl.date_range(datetime(2020, 1, 1), datetime(2020, 1, 10))})

In [105]: iter_ = df.iter_rows()

In [106]: next(iter_)
Out[106]: (datetime.datetime(2020, 1, 1, 0, 0),)

In [107]: next(iter_)
Out[107]: (datetime.datetime(2020, 1, 2, 0, 0),)

In [108]: next(iter_)
Out[108]: (datetime.datetime(2020, 1, 3, 0, 0),)

There's also
In [104]: df = pl.DataFrame({'ts': pl.date_range(datetime(2020, 1, 1), datetime(2020, 1, 10))})

In [105]: iter_ = df.iter_rows()

In [106]: next(iter_)
Out[106]: (datetime.datetime(2020, 1, 1, 0, 0),)

In [107]: next(iter_)
Out[107]: (datetime.datetime(2020, 1, 2, 0, 0),)

In [108]: next(iter_)
Out[108]: (datetime.datetime(2020, 1, 3, 0, 0),)

cuDF poisons the iterrows() method as well

We discussed this today, and concluded that we should implement __iter__ in the standard and make it raise an exception, to avoid a construct that's easy to write but very slow. Just leaving it out isn't good enough, because Python will fall back to __getitem__ and make a for-loop work.
The rationale for raising is that there's almost always much better ways of achieving whatever the user needs to achieve, rather than for-loops. And if really needed, there's still the explicit __getitem__ usage within a for-loop.

Sorry, just coming back to this

because Python will fall back to getitem and make a for-loop work.

If the standard doesn't have getitem, then would poisoning __iter__ actually be necessary?

authors: MarcoGorelli kkraus14 rgommers

repo: data-apis/dataframe-api

license: MIT


text:
Pborne

This PR contains:

Updates to latest Cloud-vision plugin.
Example pipelines
Update to Output schema.

Known Issues:
https://issues.cask.co/browse/PLUGIN-120
Errors in pipeline run on Sandbox environment only.

@googlebot I signed it!

@pbo-cirus , a couple of suggestions, to make this PR easy to review:

Can you break this PR into multiple? At the very basic level, all the input data and sample pipelines can be a separate PR.
Can you please make sure that you are using the right intellij settings, so that you don't change the formatting of unchanged lines of code, which adds to the size of the PR? The default tab size should be 2 spaces.

Any other ways you can think of where you can submit logical chunks of code would also be helpful.

authors: bdmogal pbo-cirus

repo: data-integrations/cloud-vision

license_source: bigquery


text:
Error code 217t [Amended 2023 v1.3]

Note: Rule previously implemented from issue #632 but rule has been amended in updated guidance.  Changes highlighted with bold and strikethrough.
Error information
Error code
217t
Description to display
Placement type 'P2' is not a valid placement type from 28 October 2023
DfE Long-form description of error
Independent living is not a valid placement from 28 October 2023.
Any independent living placements must have an end date and must have ended on or before 27 28 October.
These young people may have been placed in regulated supported accommodation. If so then this accommodation should be recorded as 'H5' and the provider URN should be provided.
DfE suggested coding
Where <PL> = 'P2'; <DEC> must be present and must be <= 27/10/2023 28/10/2023
For further info, refer to the DfE validation document here

Aand DFE have now reverted the change to this validation rule following feedback from LAs!


text:
Make tox integration tests take additional arguments for pytest command.

Description
Makes tox integration tests take additional arguments for pytest command.
For example:
tox -e integration-databricks-sql-endpoint -- -k test_dbt_empty

I think this is just good to go. I’d merge this now.

repo: databricks/dbt-databricks


text:
close file handle  after to_csv

It looks like pandas to_csv doesn't close file handle and need close after to_csv

Sorry, but why is this an issue? The CLI process will close all its file handles when it exits, and Flask will close the stream on its own to respond to the request.

Sorry, but neither of these is an issue. The predict command exits immediately and Flask doesn't require you to close the stream there.

authors: mateiz xueyumusic

repo: databricks/mlflow


text:
databricks bundle init mlops-stacks // Error: variable "input_cloud" not defined

Hello!
Databricks CLI v0.211.0.
Python 3.10.0
When I trigger the initialization of the project with databricks bundle init mlops-stacks
I received the following message:
Welcome to MLOps Stacks. For detailed information on project generation, see the README at https://github.com/databricks/mlops-stacks/blob/main/README.md.
Error: variable "input_cloud" not defined

Hi @rjarpa! I'm having a bit of trouble reproducing this. Would you mind confirming using which databricks and databricks -v that you are using the Databricks CLI v0.211.0. as mentioned above?

Hi @arpitjasa-db!
I did the test in two different Windows 10 machines using bash terminal version GNU bash, version 4.4.23(2)-release (x86_64-pc-msys) inside of VS Code,  result when I run commands:
which databricks -> /c/Windows/databricks
databricks -v -> Databricks CLI v0.211.0
In both machine the same problem. Looks about the problem happened only in the bash terminal of VS Code.
I opened a Command Prompt in Windows and execute databricks bundle init mlops-stacks works fine showing Project Name [my-mlops-project]: prompt.
I tried to do the same in VS Code in MacOS and no problem the only different is bash version in MacOs is GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin23)
In that case I can move forward if I dont use bash GNU bash, version 4.4.23(2)-release (x86_64-pc-msys) inside of VS Code.

@rjarpa thank you for all of those details. Likely this makes me think there's some dependency issue or CLI issue rather than an MLOps Stacks issue. I'll pass this along to our CLI team to dig into deeper, but glad to hear that there is a workaround! I'll update this thread when I have more details.

Yeah looks something with CLI and not MLops Stacks. Thanks for you fast response @arpitjasa-db !

Thank you for reporting this issue @rjarpa. I tried out databricks bundle init mlops-stacks using GNU bash, version 5.2.15(2)-release (x86_64-pc-msys) on windows. It works for me on both vscode and the normal terminal.
Some context around the error: It seems like the CLI mistakenly thinks that the current terminal session is not interactive. I suspect updating to the latest msys2 distribution should solve this issue for you. Please try it out and late us know if it works for you :)

Hello @shreyas-goenka,
Thanks for your answer, extra info: the bash console I'm using is git bash that is distributed with git installer. I run git update to update git bash terminal to GNU bash, version 5.2.21(1)-release (x86_64-pc-msys)
Still issue arise anyway even outside of  VS Code the results between. below the differents screen between git bash and command prompt

I'm moving forward by now with Windows Command Prompt, but could useful for you this info :) .

authors: arpitjasa-db rjarpa shreyas-goenka

repo: databricks/mlops-stacks


text:
[Maps Tool] Enable Census Tracts in States

This PR enables the map tool to plot "Places in  of type " for all US states. The states California, Florida, New York, and Texas have >5,000 census tracts, which requires special handling to avoid server limits. This PR makes multiple ("batched") api calls as as a temporary workaround until caches can be updated.
Note that render times for California, Florida, New York, and Texas is VERY slow (map takes ~40 seconds to load).
More specifically, this PR:

Edit webserver endpoints to make multiple calls for census tracts in CA, FL, NY, and TX
Add helper functions for merging batched api call responses into a single response
Add tests for the new helper functions

Testing:
Verified changes don't break Place pages, disaster dashboard, sustainability pages, scatter plot explorer, timelines explorer

@shifucun @chejennifer (reviewers): For CA, FL, TX, NY, the page load time is very long (~40 secs). If we think this is too long, I can also just disable CensusTracts as an option for those states instead.

Converting to draft -- A cache update will supersede these changes. Will close once cache changes land.

Closing because changes are superceded by a cache update.

repo: datacommonsorg/website


text:
domains dataset, used in multiple notebooks

This PR models a solution to #509 by putting the known email domain categorization metadata in a datasets section of the bigbang module. This is now potentially documented in the autodocs that go to the RTD website. The same dataset submodule can then be used across multiple notebooks.
For this PR, some remaining work:

When I get the full IETF mailing list history loaded, will update the Full Archive Study and Classifying Email Domains notebooks
Add the autodocs for this dataset and further documentation in the appropriate part of the RTD section.

The idea is that this models what can be done with other data we package with BigBang, such as the organizational metadata collected by @nllz

See also #437

The new file datasets/domains/all_email_provider_domains.txt contains all email domains found in the IETF datatracker correct? I guess I should scan over them and add any that appear in 3GPP but are currently missing?

The new file datasets/domains/all_email_provider_domains.txt contains all email domains found in the IETF datatracker correct? I guess I should scan over them and add any that appear in 3GPP but are currently missing?

No, it does not contain all those domains.
It contains only domains that have been identified as generic email address domains.
These get combined with another list of domains of known categories (which is much shorter).
It's a good point that I should document this more carefully. I'll do that before merging.

authors: Christovis sbenthall

repo: datactive/bigbang

license: mit


text:
[proxima-direct-io-kafka] fix ValueAsBytesSerializer for non-wildcard…

… attributes

cc @dmvk

repo: datadrivencz/proxima-platform


text:
Correction for raw affiliation INRIA Lorraine, Campus Scientifique, F-54506, Vandoeuvre-les-Nancy, France

Correction needed for raw affiliation INRIA Lorraine, Campus Scientifique, F-54506, Vandoeuvre-les-Nancy, France
raw_affiliation_name: INRIA Lorraine, Campus Scientifique, F-54506, Vandoeuvre-les-Nancy, France
new_rors: 02kvxyf05
previous_rors: 02mn0vt57;02kvxyf05
works_examples: W2065347132
contact: 166a39e2a19e5a615ead36354c12026a:19a33dc6f6c2e42206180c49 @ inist.fr

This issue was accepted and ingested by the OpenAlex team on 2024-10-10. The new affiliations should be visible within the next 7 days.

repo: dataesr/openalex-affiliations


text:
Correction for raw affiliation Faculty of Pharmacy, Université de Montréal, Pavillon Jean-Coutu, 2940 chemin de Polytechnique, Montréal, QC, H3T 1J4, Canada

Correction needed for raw affiliation Faculty of Pharmacy, Université de Montréal, Pavillon Jean-Coutu, 2940 chemin de Polytechnique, Montréal, QC, H3T 1J4, Canada
raw_affiliation_name: Faculty of Pharmacy, Université de Montréal, Pavillon Jean-Coutu, 2940 chemin de Polytechnique, Montréal, QC, H3T 1J4, Canada
new_rors: 0161xgx34
previous_rors: 05f8d4e86;0161xgx34
works_examples: W3004945196;W3152798658
contact: 97189304ae9bfeea5ad5605aa91a3b27:7cdba9d029eca063c9e922 @ polymtl.ca

This issue was accepted and ingested by the OpenAlex team on 2024-11-01. The new affiliations should be visible within the next 7 days.


text:
Correction for raw affiliation Univ. Lyon, ENS de Lyon, CNRS UMR 5182, Université Claude Bernard Lyon 1

Correction needed for raw affiliation Univ. Lyon, ENS de Lyon, CNRS UMR 5182, Université Claude Bernard Lyon 1
raw_affiliation_name: Univ. Lyon, ENS de Lyon, CNRS UMR 5182, Université Claude Bernard Lyon 1
new_rors: 029brtt94;04zmssz18;02feahw73;02gaw1s20
previous_rors: 029brtt94;04zmssz18;02feahw73
works_examples: W4255289157
contact: c42b20685850a0e2f128c628dca10fd3:3174287f67b9f222992ad71eff4d @ ens-lyon.fr

This issue was accepted and ingested by the OpenAlex team on 2024-12-21. The new affiliations should be visible within the next 7 days.


text:
Correction for raw affiliation Centre Technique Interprofessionnel des Fruits et Légumes, Centre de Balandran, 751 Chemin de Balandran, Bellegarde, 30 127, France

Correction needed for raw affiliation Centre Technique Interprofessionnel des Fruits et Légumes, Centre de Balandran, 751 Chemin de Balandran, Bellegarde, 30 127, France
raw_affiliation_name: Centre Technique Interprofessionnel des Fruits et Légumes, Centre de Balandran, 751 Chemin de Balandran, Bellegarde, 30 127, France
new_rors: 02a63xw04
previous_rors: 01jvqnd16
works_examples: W2333528180
contact: db19e1e6521ba0fabec7b347d39bd24c:598cadae30c66862566113 @ ctifl.fr

This issue was accepted and ingested by the OpenAlex team on 2024-12-19. The new affiliations should be visible within the next 7 days.


text:
Migrated DateAnTime to TimeAndDate to support Java8 Time, and depreca…

…te Timestamps and java.util.Date code.
Should address https://github.com/datafaker-net/datafaker/issues/1183

Ugh, just one minor comment. Apparently I was half asleep and set approve anyway 🤷♂

Ugh, just one minor comment. Apparently I was half asleep and set approve anyway 🤷♂

Appreciate the approval, I fixed the naming, and a few other minor improvements. If you want to have another look, that would be great!

@kingthorin I've merged this PR, but I will take your feedback into account. Only reason I merged is so I can ask @asolntsev to use this faker instead in the other PR.

authors: bodiam kingthorin

repo: datafaker-net/datafaker


text:
Connection string support

Allow user to init storage backend like what we does for database:

fs:///path/to/dir
s3://s3.amazonaws.com/bucket/path/to/dir?aws_access_key_id=xxx&aws_secret_access_key=xxx&region=xxx

ping @BohuTANG for more feature requests.

Cool!
For databend, the s3://s3.amazonaws.com/bucket/path/to/dir URL will not be passed to OpenDAL directly.
And the CREDENTIALS will parse into a struct(Stage) in SQL layer, then call OpenDAL API like:
let operator = opendal::services::s3::Backend::build()
                    .bucket(bucket)  -- we know bucket
                    .credential(credential) -- we know the credential
                    .finish()
                    .await

All, the current higher priority for databend tasks are(These two are enough for now):
https://github.com/datafuselabs/opendal/issues/37 and https://github.com/datafuselabs/opendal/issues/12

authors: BohuTANG Xuanwo

repo: datafuselabs/opendal


text:
Scripts with more than 6 inputs fail at the start

Sample:
//name: test Core
//language: javascript
//input: double A1 = 1
//input: double A2 = 2
//input: double A3 = 3
//input: double A4 = 4
//input: double A5 = 5
//input: double A6 = 6
//input: double A7 = 7
//input: double A8 = 8
//input: double A9 = 9
//input: double A10 = 10
//input: double A11 = 11
//input: double A12 = 12
//input: double A13 = 13

Fixed!

repo: datagrok-ai/public


text:
Glossary Term: "Read More" in Term description/About section not working

Describe the bug
If a glossary term description/About section contains a very long string, the UI renders only certain # of characters as the rest of the string characters can be read by clicking “Read More” button. However, the current version of the datahub 8.24 Read More is not working as expected.
To Reproduce
Steps to reproduce the behavior:

Go to 'Glossary Term and add a term with a long description and save'
Click on 'Read More in About section of this term'
Should see no action as it won't expand the string.

Expected behavior
About section "Read More" should renter the rest of the description string characters.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version 0.8.24

Additional context
Add any other context about the problem here.

This is addressed in recent versions of DataHub. Closing the issue!
John

repo: datahub-project/datahub


text:
The extended data model was not recognized

I confirm the extended data model under metadata-models-custom package according to the information given in the document:

Create PDL file and write in entity-registry.yaml file

id: kbase-model
entities:
  - name: glossaryTerm
    keyAspect: glossaryTermKey
    aspects:
      - kbaseSchemaMetadata

Then package the project and get the zip in dist;
Because we are a docker deployment project, I manually unzip the zip package to get libs and entity-registry.yaml . which are placed in the /etc/datahub/plugins/models/kbase-model/0.0.1/directory mapped by docker.
The log also outputs the log about this aspect, but there is no query under /config

curl -s http://localhost:8080/config | jq .
{
  "supportsImpactAnalysis": true,
  "models": {},
  "versions": {
    "linkedin/datahub": {
      "version": "v0.8.28",
      "commit": "beb51ebf59a045de0aac9115f6eeecc8f7207d6b"
    }
  },
  "statefulIngestionCapable": true,
  "retention": "true",
  "noCode": "true"
}

OK, the above problems have been solved. The reason is that the version is updated to more than 0.8.30


text:
Attachment search: Exact phrase (in quotes) does not return expected results

Search for exact phrase using quotes isn't working. E.g., search for "Red Line" in screenshot below.

Closed via PR #325 !

repo: datamade/la-metro-councilmatic


text:
Make library creation more java friendly

This makes DataSonnet Library creation more java friendly with some small modifications to the contract to use java collections instead of scala ones, plus the creation of a utility function to abstract creation of scala functions that do not need to use the eval scope or file scope, the most common kind of simple function. A similar function should be added later that does pass the eval scope and file scope, since it is very common for DataSonnet code to use higher order functions. That may entail a "function calling wrapper" so it is possible to continue to use pure java.
An extremely simple all-Java library implementation could then be
package com.datasonnet;

import com.datasonnet.header.Header;
import com.datasonnet.spi.DataFormatService;
import com.datasonnet.spi.Library;
import scala.Option;
import sjsonnet.Val;

import java.util.Collections;
import java.util.Map;
import java.util.Set;

public class ExampleLibrary extends Library {
    @Override
    public String namespace() {
        return "example";
    }

    @Override
    public Map<String, Val.Func> functions(DataFormatService dataFormats, Header header) {
        return Collections.singletonMap("echo", makeSimpleFunc(Collections.singletonList("value"), vs -> vs.get(0)));
    }

    @Override
    public Map<String, Val.Obj> modules(DataFormatService dataFormats, Header header) {
        return Collections.emptyMap();
    }

    @Override
    public Set<String> libsonnets() {
        return Collections.emptySet();
    }
}

@jam01 once you give the thumbs up for the test implementations of the functions in the camel PR you mentioned you wanted to do in our call I'll give this a merge

@fugu13, @JakeMHughes and I were just looking at this. It does work! IntelliJ gives me some grief with casting to Val.Str but maven seems to compile it happily. So we'd just have ignore those errors... I don't imagine that'd be something the camel team would have a problem with since again, it seems to be an IDE issue.

repo: datasonnet/datasonnet-mapper


text:
xtuner 微调报错，求指点

微调的时候出现这个问题，请问大佬们如何解决
(base) root@91bd5febc58b:/data/nlp_translate# CUDA_VISIBLE_DEVICES=2,3 NPROC_PER_NODE=2   xtuner train /data/nlp_translate/train_for_internlm/internlm2_5_chat_7b_qlora.py  --deepspeed deepspeed_zero2
09/20 09:09:17 - mmengine - WARNING - Use random port: 29768
/opt/conda/lib/python3.8/site-packages/mmengine/optim/optimizer/zero_optimizer.py:11: DeprecationWarning: TorchScript support for functional optimizers is deprecated and will be removed in a future PyTorch release. Consider using the torch.compile optimizer instead.
from torch.distributed.optim import
[2024-09-20 09:09:22,245] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.4
[WARNING]  using untested triton version (3.0.0), only 1.0.0 is known to be compatible
/opt/conda/lib/python3.8/site-packages/deepspeed/runtime/zero/linear.py:49: FutureWarning: torch.cuda.amp.custom_fwd(args...) is deprecated. Please use torch.amp.custom_fwd(args..., device_type='cuda') instead.
def forward(ctx, input, weight, bias=None):
/opt/conda/lib/python3.8/site-packages/deepspeed/runtime/zero/linear.py:67: FutureWarning: torch.cuda.amp.custom_bwd(args...) is deprecated. Please use torch.amp.custom_bwd(args..., device_type='cuda') instead.
def backward(ctx, grad_output):
:219: RuntimeWarning: scipy._lib.messagestream.MessageStream size changed, may indicate binary incompatibility. Expected 56 from C header, got 64 from PyObject
09/20 09:09:26 - mmengine - WARNING - WARNING: command error: 'partially initialized module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline' (most likely due to a circular import)'!
09/20 09:09:26 - mmengine - WARNING -
Arguments received: ['xtuner', 'train', '/data/nlp_translate/train_for_internlm/internlm2_5_chat_7b_qlora.py', '--deepspeed', 'deepspeed_zero2']. xtuner commands use the following syntax:
    xtuner MODE MODE_ARGS ARGS

    Where   MODE (required) is one of ('list-cfg', 'copy-cfg', 'log-dataset', 'check-custom-dataset', 'train', 'test', 'chat', 'convert', 'preprocess', 'mmbench', 'eval_refcoco')
            MODE_ARG (optional) is the argument for specific mode
            ARGS (optional) are the arguments for specific command

Some usages for xtuner commands: (See more by using -h for specific command!)

    1. List all predefined configs:
        xtuner list-cfg
    2. Copy a predefined config to a given path:
        xtuner copy-cfg $CONFIG $SAVE_FILE
    3-1. Fine-tune LLMs by a single GPU:
        xtuner train $CONFIG
    3-2. Fine-tune LLMs by multiple GPUs:
        NPROC_PER_NODE=$NGPUS NNODES=$NNODES NODE_RANK=$NODE_RANK PORT=$PORT ADDR=$ADDR xtuner dist_train $CONFIG $GPUS
    4-1. Convert the pth model to HuggingFace's model:
        xtuner convert pth_to_hf $CONFIG $PATH_TO_PTH_MODEL $SAVE_PATH_TO_HF_MODEL
    4-2. Merge the HuggingFace's adapter to the pretrained base model:
        xtuner convert merge $LLM $ADAPTER $SAVE_PATH
        xtuner convert merge $CLIP $ADAPTER $SAVE_PATH --is-clip
    4-3. Split HuggingFace's LLM to the smallest sharded one:
        xtuner convert split $LLM $SAVE_PATH
    5-1. Chat with LLMs with HuggingFace's model and adapter:
        xtuner chat $LLM --adapter $ADAPTER --prompt-template $PROMPT_TEMPLATE --system-template $SYSTEM_TEMPLATE
    5-2. Chat with VLMs with HuggingFace's model and LLaVA:
        xtuner chat $LLM --llava $LLAVA --visual-encoder $VISUAL_ENCODER --image $IMAGE --prompt-template $PROMPT_TEMPLATE --system-template $SYSTEM_TEMPLATE
    6-1. Preprocess arxiv dataset:
        xtuner preprocess arxiv $SRC_FILE $DST_FILE --start-date $START_DATE --categories $CATEGORIES
    6-2. Preprocess refcoco dataset:
        xtuner preprocess refcoco --ann-path $RefCOCO_ANN_PATH --image-path $COCO_IMAGE_PATH --save-path $SAVE_PATH
    7-1. Log processed dataset:
        xtuner log-dataset $CONFIG
    7-2. Verify the correctness of the config file for the custom dataset:
        xtuner check-custom-dataset $CONFIG
    8. MMBench evaluation:
        xtuner mmbench $LLM --llava $LLAVA --visual-encoder $VISUAL_ENCODER --prompt-template $PROMPT_TEMPLATE --data-path $MMBENCH_DATA_PATH
    9. Refcoco evaluation:
        xtuner eval_refcoco $LLM --llava $LLAVA --visual-encoder $VISUAL_ENCODER --prompt-template $PROMPT_TEMPLATE --data-path $REFCOCO_DATA_PATH
    10. List all dataset formats which are supported in XTuner

Run special commands:

    xtuner help
    xtuner version

GitHub: https://github.com/InternLM/xtuner

可以去xtuner的官方仓库提交issue哦，哪里有负责解答~

repo: datawhalechina/self-llm


text:
jpeg instead of jpg

The last two or three stable releases store images as jpeg instead of jpg as it used to be before.
Just a heads up, not a real issue.

The extensions are not changed and are kept in the original format
If your in a situation where you really need them all to be the same do an overwrite for photos
https://of-scraper.gitbook.io/of-scraper/getting-started/config-options/advanced-config-options/mediatype-overwrites

En el config.json :
En "file_format" > .{custom.get(ext,ext)}
"code-execution": true,
"custom_values": {"jpeg": "jpg"},

Thanks!

@Puk0 Works like a charm, as soon as I found out, where to place it all. Thanks again.

Did some more research. The log shows the "original" filenames as OF seems to deliver them. It's some quite cryptic path with the filename being a combination of the dimensions and and an ID. At this point the extension is still "jpg". The renaming to whatever is set in "file_format" happens later on but is not logged (at least not at the level that's used by default). The change to "jpeg" must happen here. Re-downloading the files for one subscription with --force-all --no-cache gave me a bunch of duplicate files. Previously downloaded files with "jpg" and the new ones with "jpeg".

Re-downloading the files for one subscription with --force-all --no-cache

Solo tenias que renombrar con algún programa como "Renamer" y cambiar .jpeg por .jpg
Añadir > remplazar > jpeg jpg, y quitar >ignorar extensión,

Sure, that would work. And it's easily resolvable with your custom.get-hint. But it's still strange.

Sure, that would work. And it's easily resolvable with your custom.get-hint. But it's still strange.

What did you config look like after the changes in those particular areas?

authors: DonaldTPP El-tra Puk0 datawhores

repo: datawhores/OF-Scraper


text:
Enhance Apache Spark text extraction job to load data into Elasticsearch

The main repository for storing data will be Elasticsearch. Use the elasticsearch-hadoop plugin to load each text resume into Elasticsearch.

Successfully connected Apache Spark to the Elasticsearch database, resumes are stored using the unique file name as the doc id, and the full extracted text is stored in the source field.  Next ETL task will be to use OpenNLP to tag entities from the raw text and store those into ES.

authors: davidmezzetti gilberbe


text:
Unable to autogenerate migrations when using plugin prettier-plugin-tailwindcss

Hi,
Currently having issues trying to run the following
npx datocms migrations:new --autogenerate=feature:main 'test'
I have the plugin "prettier-plugin-tailwindcss": "^0.5.6", installed with "@datocms/cli": "^1.2.1" as devDependencies
I am getting the following error
Writing "migrations\1697721894_test.ts"... !
    Error: require() of ES Module C:\Users\_..omitted.._\node_modules\prettier-plugin-tailwindcss\dist\index.mjs not supported.
    Instead change the require of C:\Users\_..omitted.._\node_modules\prettier-plugin-tailwindcss\dist\index.mjs to a dynamic import() which is available in all
    CommonJS modules.
    Code: ERR_REQUIRE_ESM

@khaleelkhalifa can you please try to temporarily replace the content of your tsconfig with the following, and running the migrations again?
{
"compilerOptions": {
"declaration": true,
"importHelpers": true,
"module": "commonjs",
"strict": true,
"target": "es2019"
}
}

Hi @stefanoverna , Still the same error I am afraid

@khaleelkhalifa we don't use "prettier-plugin-tailwindcss" so it's quite strange. Can you provide a complete stack trace, if available?

@stefanoverna I am only getting that error back without any stack trace and can't see a flag to turn on verbose.
Do you know what I need to pass in to get one?

Were you able to replicate this issue? @stefanoverna

Nope @khaleelkhalifa.. can you share with us a repo with the problem? You can send it to support@datocms.com if you prefer.

Hi, Please see https://github.com/khaleelkhalifa/dato-migrations

@marcelofinamorvieira can you take a look at this?

Hi, Has there been any update on this? Keen to get migrations working on our tech stack

Hey @khaleelkhalifa sorry for the delay on the response!
It seems like all versions after v0.5 do not support ts files that are compiled to "commonjs" to include "import" as you can read right here, so it won't be compatible with our CLI:
https://github.com/tailwindlabs/prettier-plugin-tailwindcss/issues/207#issuecomment-1698071122
Since our migration files always start with
import { Client, SimpleSchemaTypes } from "@datocms/cli/lib/cma-client-node";
You get that error when trying to write the migration file if you run the generate migration command in a repository with that prettier plugin version.
An easy way to solve this would be to just generate and run the migrations in a different app folder, one without the prettier + prettier-tailwind plugin, with a clean package.json
Using a repo just for the migrations is usually better (as the migrations don't need to be in the same folder as your front-end, since they only interact with Dato directly)

I started experiencing this issue after upgrading @datocms/cli from version 2.0.14 to 2.0.15.

Mh, v2.0.15 solves a super-minor issue (migration generation crashed if field ID or label contained the word "export") so I don't think it could be the cause of the problem.
Can you share a repo reproducing the issue @luukschipperheyn so we can better investigate?

@stefanoverna here you go. I copied the dependencies and some config files from the project I'm working on: https://github.com/luukschipperheyn/datocms-cli-migration-issue
steps to reproduce

npm i
make a datcms project. add a simple schema to main
fork main, call it develop
make a schema change in develop
create .env and add DATOCMS_API_TOKEN=[your-token]
npx datocms migrations:new --autogenerate=develop:main test

This yields the following error for me:
Error: require() of ES Module /Users/luuk/development/datocms-cli-migration-issue/node_modules/prettier-plugin-tailwindcss/dist/index.mjs
not supported.
Instead change the require of /Users/luuk/development/datocms-cli-migration-issue/node_modules/prettier-plugin-tailwindcss/dist/index.mjs to
    a dynamic import() which is available in all CommonJS modules.
Code: ERR_REQUIRE_ESM

Downgrading @datocms/cli to 2.0.14 gets rid of the error:

npm install --save-dev @datocms/cli@2.0.14
npx datocms migrations:new --autogenerate=develop:main test

Thanks for the repro @luukschipperheyn! v2.0.16 should fix this issue.
— Basically, what happens is that the CLI formats the migration file with Prettier and tries to reuse the .prettierrc settings from the repository in which it is run (which makes sense). But things can go wrong. In your case, the Tailwind plugin only supports Prettier 3, while the CLI uses Prettier 2.
The fix has been to continue trying to use the .prettierrc.. but if it fails, fallback to pristine Prettier settings.

authors: khaleelkhalifa luukschipperheyn marcelofinamorvieira stefanoverna

repo: datocms/cli


text:
query.selectActive should return only Observable<SomeClass[]>

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report
[ ] Performance issue
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
[x] Other... Please describe:

Current behavior
The return of this.query.selectActive(); is typed as Observable<SomeModel[]> | Observable<SomeModel>
Expected behavior
I believe it should be typed as Observable<SomeModel[]> only, or at least Observable<SomeModel[] | SomeModel>.

Currently it is Observable<SomeModel[]> | Observable<SomeModel>

What is the motivation / use case for changing the behavior?
1 - Typescript don't allow to use the .pipe(map(x => ...))

Have to 'cast' to Observable<any> if want to use the map
(this.query.selectActive() as Observable<any>).pipe(map(item => item instanceof Array ? item : [item]));

2 - It's easier to get the first element of array (if only one is allowed)
3 - I believe the Observable is for sure the return type. The doubt here is whether it is an array or not. So, Observable<SomeModel[] | SomeModel> is better (if not Observable<SomeModel[]>).
Environment

Angular version: 7.3.0

Browser:
- [x] Chrome (desktop) version XX
- [ ] Chrome (Android) version XX
- [ ] Chrome (iOS) version XX
- [ ] Firefox version XX
- [ ] Safari (desktop) version XX
- [ ] Safari (iOS) version XX
- [ ] IE version XX
- [ ] Edge version XX

Others:

You need to implement the right interface - ActiveState for single and MultiActiveState for multiple. Please, read the docs. https://netbasal.gitbook.io/akita/entity-store/entity-store/active-state

Advanced comment: Only relevant if generically extending QueryEntity as a helper subclass.
I ran into a related issue when creating my own QueryEntityExtended class that I inherit from. I add various helper functions in here
for example: QueryEntityExtended<...> extends QueryEntity<...>
You can actually cast Observable<E[]> | Observable<E> to Observable<E[] | E> and then run the Array.isArray(...) check within your pipe(...)
So I ended up with:
(this.selectActive() as Observable<E | E[]>).pipe(switchMap(...))

Like I said this is only relevant AFAIK if you're extending the generic class - in which case it won't know what ActiveState you're using. So for most people finding this you probably just need to read the ActiveState docs.

@simeyla wow, I faced the exact problem as yours, and I named my helper class as ExtendedQueryEntity lol. so lucky to find your answer here, after several hours in the problem...

authors: NetanelBasal isaacqc jvitor83 simeyla

repo: datorama/akita


text:
travis: test on node 8

npm bundles with node 8 aborts installation if the browserify step fails, see https://github.com/datproject/dat-desktop/issues/436

Interesting. It builds just fine on node 8 o.O, but not on my mahine rips hair

repo: datproject/dat-desktop


text:
Not ARC-compliant

In CHLayoutManager.m, there is an @selector(dealloc), which is illegal. Overall, the custom -dealloc behavior should probably be removed. There is also manual block-copying and block-releasing in CHLayoutConstraint.m

Sorry, but as is mentioned in the readme, CHLayoutManager is no longer maintained. My recommendation would be to add the compiler flags to your project that would make these files be compiled without ARC.

Okay, I thought someone else might be willing to take that on.

authors: abbeycode davedelong

repo: davedelong/CHLayoutManager


text:
200 Ok with no sdp

Hi Dave,
We are using Drachtio server with drachtio-siprec-recording-server and Freeswitch. It used to work fine. We are seeing a weird issue that used to work fine before. Drachtio is sending 200 OK response without SDP for the INVITE from Freeswitch. Hence Freeswitch is terminating both the calls by sending BYE for the outbound call and 500 response for the inbound INVITE. I am attaching the log messages from Drachtio. Please check and let me know.
Thanks,
Ajay
drachtio_siprec_call_failure_200_no_sdp.txt

can you open this issue in the repo for the app (drachtio-siprec-recording-server)?

Sure. I will run it again with higher debugging and provide the logs there. Thanks Dave.

authors: ajay-sabat davehorton

repo: davehorton/drachtio-server


text:
Scroll should be on columns, not the whole window

I don't think there is anyway to do this nicely. There isn't a way to track scrolling in real time you would have to poll the view to see if it was scrolled then revert it and move the content of the column around to get the behavior you want.

One way to do it without having to move the content but revert the UI when scrolling would be to have another column with the custom UI only, then you could dedicate first column to UI and second column to output.
Something like:
========================================================
Debug panel | call stack | Debugger Console | Variables|  Column 1
========================================================
| call stack output.     | console output   |          |
|                        |                             |
|                        |                             |  Column 2
|                        |                             |
|                        |                             |
|======================================================|

This will disconnect the header from the content and therefore might cause confusion as to which row contains what information but it would fix the mentioned problem, not sure if that's a better compromise.

authors: CandyFace daveleroy kud


text:
Update cats-effect, cats-effect-kernel, ... to 3.0.0-RC3

Updates

org.typelevel:cats-effect
org.typelevel:cats-effect-kernel
org.typelevel:cats-effect-laws

from 3.0.0-RC2 to 3.0.0-RC3.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.typelevel" } ]

labels: library-update, semver-pre-release

Superseded by #182.

repo: davenverse/agitation


text:
Update cats-core to 2.3.1

Updates org.typelevel:cats-core from 2.1.1 to 2.3.1.
GitHub Release Notes - Changelog - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.typelevel", artifactId = "cats-core" } ]

Applied Migrations

github:typelevel/cats/Cats_v2_2_0?sha=v2.2.0

Documentation:

https://github.com/typelevel/cats/blob/v2.2.0/scalafix/README.md#migration-to-cats-v220

labels: library-update, semver-minor, scalafix-migrations

Superseded by #162.

repo: davenverse/github


text:
Maintain Todo Example

There have been a lot of changes to the starter kit recently, so the todo branch needs to be updated accordingly.

I'd give it a try, what's your preference ?

Hey thanks @agdolla! Honestly I don't have much of a preference, though it would probably be nice to stick as close to the Redux TodoMVC example as possible. There's currently a branch setup with the example, but it's quite a bit behind master so you're welcome to just start over if you're up for it. If not, no worries.

Sent a PR

Thanks so much! Just merged it, and am going to close this issue for now then.

authors: agdolla davezuko

repo: davezuko/react-redux-starter-kit


text:
Some referenced packages are showing as "not found"

Hi. Firstly, thank you for making Go Search. It's a useful tool.
I have noticed that some of my own packages are showing as "not found" on go-search.org, even though they are shown as imports of other packages. They begin "xi2.org/x/..."
For example, if you view the page http://go-search.org/view?id=github.com%2Fklauspost%2Fpassword and select my package "xi2.org/x/xz" which appears as an import at the bottom of that page you get the "not found" response.
I decided I would try to debug the issue, so I downloaded the source and compiled gcse myself. I ran it and limited the crawl to just a handful of packages including my own, and to my surprise it worked fine when I ran it locally! So I can't help debug it it would seem since there must be something different about my local environment :(
As far as I know the packages download fine with "go get" and they show up ok on godoc.org. Any ideas? Is there something weird about my web server?
It might seem quite conceited of me to be concerned about my own packages, but you never know, it might also be happening to someone else's too.
Best wishes

Checking the logs, here is the reason I'm guess so far:
Request to "xi2.org/x/xz" will be redirected to "github.com", but the crawler does not use autheticated requests, so it leads to a very low rate limit.
The fix needs some modification on the crawler package.

Thanks for replying. I hope the crawler modification is not too big a change.

repo: daviddengcn/gcse


text:
Replace Databricks with BigQuery resources

This would also help as we'll be able to ingest OSO data!

Done here:

https://github.com/davidgasquez/filecoin-data-portal/commit/9aade14b3210b25da25bb3b5a6b1c6d254fe7b5c
https://github.com/davidgasquez/filecoin-data-portal/commit/92a32b3072dfe9e60ef4360337a74fb89a5d49d2

repo: davidgasquez/filecoin-data-portal


text:
The target origin provided does not match the recipient window’s origin.

Isn't the whole point of this script to work cross origin?
Well, it doesn't seem to.
After installing the iframeResizer.contentWindow.min.js file in the page that I am accessing via the iframe (the iframe's content page - in the body), and installing iframeResizer.min.js before the <iframe id="myIframe"> and the iFrameResize({ log: true }, '#myIframe') after the </iframe> I am the lucky owner of a brand new error message in console:
Failed to execute ‘postMessage’ on ‘DOMWindow’: The target origin provided (‘https://iframe.content.source’) does not match the recipient window’s origin (‘https://page.with.iframe’).

My guess is I'm a complete idiot and there is a setting I need to discover by reading for countless hours.
Ain't nobody got time fo dat.

After moving the mouse around many times in the iframe and clicking on things I finally got this message:
[iFrameSizer][Host page: frame-one] IFrame (frame-one) height set to 150px

The funny part is that page loading in the iframe is a minimum of 525px...so...I guess the origin side script actually doesn't know the height of the content so it just guesses 150px? Or maybe it's still a CORS problem but the error only appears once at first load?
I wrote a script like this a very long time ago, like 7+ years ago that actually worked well back then, but it seems that with the CORS standards of today's browsers that old script of mine won't send the height data, but it does at least output the correct content height to console.
So why isn't your script calculating the correct content height?
How can I assure the content is completely loaded on the origin side before your script calculates the content height?

@AmtechInnovarch I ran into this same problem and think I found the solution in the troubleshooting documentation https://github.com/davidjbradshaw/iframe-resizer/blob/master/docs/troubleshooting.md#failed-to-execute-postmessage-on-domwindow

To add to my previous response, I was able to remove this error by making sure my iframe content was loaded before attempting to use this library. Here is a code snippet where this.iframe is the HTML iframe element:
this.iframe.addEventListener("load", () => {
  window['iFrameResize'](this.iframe);
});

authors: AmtechInnovarch akdarrah

repo: davidjbradshaw/iframe-resizer


text:
Problem with local transitions

If I have this statechart:
{
  initial: 'Hidden',
  states: {
    Hidden: {
      on: {
        PUBLISH_FAILURE: 'Failure',
      },
    },
    Failure: {
      on: {
        CLICKED_CLOSE: 'Hidden',
      },
    },
  },
}

When I'm in 'Failure' and the CLICKED_CLOSE event is fired, I properly go into the Hidden state. The problem is that I want to put the CLICKED_CLOSE event one level higher:
{
  initial: 'Hidden',
  on: {
    CLICKED_CLOSE: '.Hidden',
  },
  states: {
    Hidden: {
      on: {
        PUBLISH_FAILURE: 'Failure',
      },
    },
    Failure: {
    },
  },
}

When I do this, the transition doesn't happen. I even tried doing CLICKED_CLOSE: 'Alert.Hidden', -- where Alert is the name of the parent state.
Any idea what I'm doing wrong?

Ah yeah, this might be a bug. I'll look into it.

This seems to be a problem with events declared at the top level. I added a couple of test cases, one where the on CLICKED_CLOSED was declared at the top level, the other in an outer top level state; otherwise identical statecharts. Only the first one fails.  Digging to see if there's a general problem with top level states, or if it only has to do with local transitions.

Further testing seems to indicate that it's simply top level machine events that don't seem to work.  I moved the failing test to machine.test.ts instead.

It fails in the scxml branch too.

Hello, sorry to bother you guys, but the fix is going to be backported to the incoming 3.2 release too?
And by the way, is there any approximative release date for 3.2?
Thank you!

@AnimaLupi This will be fixed into 3.2, which I plan on releasing at React Finland (April 24-26).

@davidkpiano Thank you very much for the release info and fast reply! Nice work by the way!

authors: AnimaLupi davidkpiano lmatteis mogsie

repo: davidkpiano/xstate


text:
Implement update operators

Field Update Operators

[x] $inc
[x] $min
[x] $max
[x] $mul
[x] $rename
[x] $set
[x] $setOnInsert
[x] $unset
[x] $currentDate

Array Update Operators

[x] $
[x] $[]
[x] $[]
[x] $addToSet
[x] $pop
[x] $pull
[x] $push
[x] $pullAll

Modifiers

[x] $each
[x] $position
[x] $slice
[x] $sort

All implemented, at 0cf44c5

repo: davidlatwe/montydb


text:
Unable to access dynamic params from child component

Hey,
I'm using react-router-dom 4 with react-router-modal and I have a dynamic segment/parameter in my route like this:
<ModalRoute
  parentPath="/"
  path="/users/:userId"
  component={UserProfile}
/>

and my child component is as simple as follows:
const UserProfile = props => {
  return
{props.match.params.userId}

}

When I visit:
http://localhost/users/3234
The modal is being displayed as:

undefined

I expect to have the match with the parameter {userId: 3234} and path to be /users/3234.  However, the params object is empty, and the path is "/"
Am I doing something wrong?
How do I access match.params from UserProfile?

Check out the "Using match.params" example here:
https://davidmfoley.github.io/react-router-modal-examples
If you share the complete set of code you are working on (create a private gist if you don't want to share with the public), I will take a look and see if I can replicate the behavior. It sounds like you are doing everything correctly, so if there is a common error case that is not obvious, it would be good to detect it and warn the user.
Thanks,
Dave

Never mind.
It was a bad implementation from my side.
I did not notice that I was misusing withRouter while exporting UserProfile.

authors: cdes davidmfoley

repo: davidmfoley/react-router-modal


text:
Add support for ConversationBufferWindowMemory class

It keeps in memory the number of interactions specified.
E.g. k=1 only remembers the last input/output

https://python.langchain.com/en/latest/reference/modules/memory.html?highlight=ConversationBufferWindowMemory#langchain.memory.ConversationBufferWindowMemory

latest link: https://python.langchain.com/docs/modules/memory/how_to/buffer_window

@davidmigloz i can give it a try 🙋. Any thoughts about where to start?

@mauricepoirrier good to hear!
You can look at the original implementations:

Python: https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/memory/buffer_window.py
JS: https://github.com/hwchase17/langchainjs/blob/main/langchain/src/memory/buffer_window_memory.ts

And their tests:

Python: https://github.com/langchain-ai/langchain/blob/master/libs/langchain/tests/unit_tests/chains/test_memory.py
JS: https://github.com/hwchase17/langchainjs/blob/main/langchain/src/memory/tests/buffer_window_memory.test.ts

Also, you can check the current ConversationBufferMemory Dart implementation.

https://github.com/davidmigloz/langchain_dart/blob/main/packages/langchain/lib/src/memory/buffer.dart

And based on those you can create the Dart implementation.

Implementation: packages/langchain/lib/src/memory/buffer_window.dart
Tests: packages/langchain/test/memory/buffer_window.dart

Let me know if you have any questions 🙂

authors: davidmigloz dileep9490 mauricepoirrier


text:
Grid & Load Power values are incorrect after update to 24.12.06_2

Description
After updating to the latest release I needed re-add the integration according to the resolution stated in bug: https://github.com/davidrapan/ha-solarman/issues/283
When I did that the values of Load Power and Grid Power where multiplied by 10. In order to fix this I have updated the configuration of the my inverter HYBRID by Deye and decrease the scale for the 2 sensors from 10 to 1.
Can this be changed in future releases?
Reproduction Steps
Update to latest version and re-add integration.
Log
Not applicable.
Version
24.12.06_2

Yes I'm aware of that but thank you for your report. I will do something about it.
Could you please tell me exact model number of your device? (e.g. SUN-12K-SG04LP3)

Thanks for the quick response! My device is: Deye SUN-8K-SG01LP1-EU

Could you also do for me a screenshot like this one? https://github.com/davidrapan/ha-solarman/issues/281#issuecomment-2520017035

Hello, I've the same inverter : Deye SUN-8K-SG01LP1-EU with the same problem than @tinntbg and this is my screenshot.
In the screenshot the param Phases is 3 but I have only 1 Phase :/

In the screenshot the param Phases is 3 but I have only 1 Phase :/

Guess your inverter has it incorrectly written in the registers (Nothing to be really worried about as your device was correctly identified as Single Phase Hybrid Inverter, numeric value: 3).
v24.12.06_4:

feat: Add scaling modifier + Reverted Deye Hybrid scaling

s this phase problem due to the configuration of the inverter in the integration or is it directly in the inverter ?
In any case I tested the scale problem for consumption and everything is ok
Thanks

Is this phase problem due to the configuration of the inverter in the integration or is it directly in the inverter ?

That value is read directly from your device.

Thanks for the quick fix! I have tested as well and it all looks to be working now!
Should I close the ticket?

You can leave it for a while. 😉

Can please someone confirm if is working single phase detection is working in latest release? And I don't mean displayed value in the device sensor's attributes but in the amount of added sensors to the list (= sensors w/ L2 are not present).

Hi @davidrapan,
I have just deleted and re-added my inverter in home assistant and from what I can see there are a lot of sensors with L2 names.
So the short answer is that it's not working.
Thanks!

If I manually change the number of MPPTs and the number of phases then the values are not created, but the auto detection is not working with release 24.12.08_9

Could you please show me read of register number 8 using services? Thank you!

Is this what you are looking for or something else?

Mh I guess that's the culprit. So it's either not used or 0 also means single phase...

So what's the current state @tinntbg?

Hey @davidrapan,
Just updated to the latest version, deleted and re-added my inverter, but unfortunately, nothing has changed. I still see L2 values and it's still been detected as a 3 phase inverter.
Thanks!

Here are all the Auto discovered values:

Same problem on 12k sg01lp1.
Defined as 1 phase, but essentially 3 phase. Also, “scale” for “GRID Power” and “LOAD Power” does not work correctly, I had to manually change the values

I had to manually change the values

What do you mean by that? Where?

Same problem on 12k sg01lp1

This is not enough information.

I had to manually change the values

What do you mean by that? Where?

After a recent update of the Deye inverter, integer numbers in watts stopped being displayed, that is, it was 1043 watts, it became 104, judging by Deye Cloud - they removed this figure and multiplied the value by 10, their watts are always equal to 10.
In order to fix this I changed the value of "scale: 10" to "Grid Power" and "Load Power" in deye_hybrid.yaml

There is a switch for that in configuration. 😉

authors: Fennsy davidrapan ikeare tinntbg

repo: davidrapan/ha-solarman


text:
A few new ideas for the contribution!

Hallo David!
I would like to build an AI for pokemon battle ,and then I found your project!
As I am new in C++ and not exactly understand how your program works, I have a few ideas.

About Predicting the items
if the weather (sandstorm) lasts longer than 5 Turns : Sandstone (other weather stone can be detected in the same way)
if the opponent has damaged your team more than (Opponents Pokemon attack, , full IV, full EV, max Random ) 1.3 = life orb, more than 1.5 then it must be choice glass . it would not take too long to calculate. (because that is how human understand the way too high damage)
and then, about team prediction : garchomp Max ATK SPD and garchomp max DEF HP are almost two species ! we may use the existed data to make a correlation ,see which type garchomp  is now more popular and while type is strong correlated with which other pokemon.

Good ideas! The current usage stats are hard to parse, buggy, and leave out some important information. There has been a plan to update them for a while, but I might just have to write my own usage stat parser and contribute that to Smogon so that I can get the stats I need to be able to start working on some of the stuff like this. Obviously the "detect items from effects without messages" thing doesn't rely on that necessarily, but my thinking is that they would be connected because I could set the probability of having a certain item to 0 when I can rule it out.
Before I can do this effectively, however, I need to make another change I've been planning for a while. Right now, I have a single type, Team (technically it's Team<generation>, but that's not important for this). What I need to do is create two types: Team and SeenTeam (or maybe FoeTeam). SeenTeam can store as much information as it needs and it doesn't need to worry so much about efficiency, and then when I evaluate a position I predict a Team from a SeenTeam and use that. This gives me a place to put all the metadata I need for a prediction without slowing down the evaluation by making a Team larger for information that's not needed during evaluation.

authors: Halphas-Kreuz davidstone

repo: davidstone/technical-machine


text:
onchanged function ui library error

so theres a problem, my code when i try to use :onchanged it keeps triggering while on execution, so, i must assume its ui issue, correct me if im wrong

bozo + ratio + the curse of nile🦉(┬┬﹏┬┬)ᓚᘏᗢಥ_ಥಠ_ಠ༼ つ ◕_◕ ༽つ:-x^.^^^(#｀-_ゝ-)o(￣┰￣*)ゞ( *^-^)ρ(*╯^╰)（＞人＜；）o((>ω< ))o(～￣(OO)￣)ブo(≧口≦)o(╬▔皿▔)╯

authors: SmokeXDev uksfx

repo: dawid-scripts/Fluent


text:
GSSAPI operation failed with error - An invalid status code was supplied (Server not found in Kerberos database).

mssql-cli try only default_realm in /etc/krb5.conf
if mssql has another realm than default it will throw exception.
Error message: Cannot authenticate using Kerberos. Ensure Kerberos has been initialized on the client with 'kinit' and a Service Principal Name has been registered for the SQL Server to allow Kerberos authentication.
ErrorCode=InternalError, Exception=Interop+NetSecurityNative+GssApiException: GSSAPI operation failed with error - An invalid status code was supplied (Server not found in Kerberos database).
   at System.Net.Security.NegotiateStreamPal.GssInitSecurityContext(SafeGssContextHandle& context, SafeGssCredHandle credential, Boolean isNtlm, SafeGssNameHandle targetName, GssFlags inFlags, Byte[] buffer, Byte[]& outputBuffer, UInt32& outFlags, Int32& isNtlmUsed)
   at System.Net.Security.NegotiateStreamPal.EstablishSecurityContext(SafeFreeNegoCredentials credential, SafeDeleteContext& context, String targetName, ContextFlagsPal inFlags, SecurityBuffer inputBuffer, SecurityBuffer outputBuffer, ContextFlagsPal& outFlags)
   at System.Data.SqlClient.SNI.SNIProxy.GenSspiClientContext(SspiClientContextStatus sspiClientContextStatus, Byte[] receivedBuff, Byte[]& sendBuff, Byte[] serverName)
   at System.Data.SqlClient.SNI.TdsParserStateObjectManaged.GenerateSspiClientContext(Byte[] receivedBuff, UInt32 receivedLength, Byte[]& sendBuff, UInt32& sendLength, Byte[] _sniSpnBuffer)
   at System.Data.SqlClient.TdsParser.SNISSPIData(Byte[] receivedBuff, UInt32 receivedLength, Byte[]& sendBuff, UInt32& sendLength)

# sqlcmd work normal

same message and sqlcmd works too with kerberos auth
Error message: Cannot authenticate using Kerberos. Ensure Kerberos has been initialized on the client with 'kinit' and a Service Principal Name has been registered for the SQL Server to allow Kerberos authentication.
ErrorCode=InternalError, Exception=Interop+NetSecurityNative+GssApiException: GSSAPI operation failed with error - An invalid status code was supplied (Server not found in Kerberos database).
   at System.Net.Security.NegotiateStreamPal.GssInitSecurityContext(SafeGssContextHandle& context, SafeGssCredHandle credential, Boolean isNtlm, SafeGssNameHandle targetName, GssFlags inFlags, Byte[] buffer, Byte[]& outputBuffer, UInt32& outFlags, Int32& isNtlmUsed)
   at System.Net.Security.NegotiateStreamPal.EstablishSecurityContext(SafeFreeNegoCredentials credential, SafeDeleteContext& context, String targetName, ContextFlagsPal inFlags, SecurityBuffer inputBuffer, SecurityBuffer outputBuffer, ContextFlagsPal& outFlags)
   at System.Data.SqlClient.SNI.SNIProxy.GenSspiClientContext(SspiClientContextStatus sspiClientContextStatus, Byte[] receivedBuff, Byte[]& sendBuff, Byte[] serverName)
   at System.Data.SqlClient.SNI.TdsParserStateObjectManaged.GenerateSspiClientContext(Byte[] receivedBuff, UInt32 receivedLength, Byte[]& sendBuff, UInt32& sendLength, Byte[] _sniSpnBuffer)
   at System.Data.SqlClient.TdsParser.SNISSPIData(Byte[] receivedBuff, UInt32 receivedLength, Byte[]& sendBuff, UInt32& sendLength)

Any pointers?

Problem is with credential storing file in environment at. You can have one auth per environment.
If you do authentication before using client it should work.
Another problem should be that you have to use direct port. Port 1433 is not instance port.
All instances have own port. 1433 is used as location service to redirect to correct port.
#!/bin/bash
# myscript
KRB5CCNAME=/tmp/my-temp-creds
kinit hnelson@EXAMPLE.COM

#mssql-cli -S localhost -d DBTEST -U hnelson
#mssql-cli -S localhost:1433 -d DBTEST -U hnelson
mssql-cli -S localhost:49168 -d DBTEST -U hnelson

bash> ./myscript
I hope this will point you to right path.

Thank you but no luck on Linux. I've tried connecting to different ports but it does not look like it even accepts :port for me. It just treats it as a hostname with colon.
It does work on Windows with -E flag.

mssql-cli -S HOST,PORT
Not sure but look how to call specific port

authors: digash mertuarez

repo: dbcli/mssql-cli


text:
ResultSet tabs action "close tabs to the left" closes pinned tabs

Description
When multiple ResultSet tabs are open, the context menu has several options for closing multiple tabs:

Close All
Close All of Same Query
Close Other Tabs
Close Tabs to the Left

All of these leave pinned tabs open (as expected) except for "Close Tabs to the Left", which closes pinned tabs.
DBeaver Version
Community Version 22.3.2.202301060755
Operating System
MacOS 12.6.2
Database and driver
No response
Steps to reproduce

Execute a query and pin the resultset tab
Execute 2 more queries, each in a new tab
Open the context menu on the 3rd tab, and select "Close Tabs to the Left"
Tabs 1 and 2 will both be closed, even though tab 1 was pinned

Additional context
No response

Reproduced on Windows 10

Thank you for bug report.

repo: dbeaver/dbeaver


text:
I can't see the progress commands when backup or restore postgresql in ubuntu

Description
I upgrade to 23.2.1 in ubuntu and when I tried to run a backup I noticed that I can't see the progress commands. I tried to restore a backup and I noticed the same.
DBeaver Version
CE 23.2.1
Operating System
Ubuntu 22.04.3 LTS
Database and driver

Database: PostgreSQL 13.11
Driver: PostgreSQL JDBC Driver 42.2.20

Steps to reproduce

Connect to a postgresql database
Select a database
Go to tools/backup o tools/restore

The next step button is disabled and when I press proceed the process runs in background.

Additional context
No response

Thank you for bug report. It's a duplicate of #21302. We've already fixed it and re-released this version.


text:
Bug introduced in v24.1.3 affecting displaying of SQL functions

Description
In version 24.1.3, a bug was introduced that affects the display of SQL functions. Attached is an image showing a function definition in 24.1.2 (everything is fine) and in 24.1.3 (incorrect comments).
24.1.2

24.1.3

DBeaver Version
24.1.3 - 24.1.5
Operating System
No response
Database and driver
PostgreSQL
Steps to reproduce
Just open the description of any function
Additional context
No response

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Apparently this has just been fixed in #35202.

As a workaround, you can turn the setting Enable semantic analysis in Preferences->Editors->SQL Editor->Code Editor


text:
How do I run an sql script and export the result by running from command line?

I am trying to run a set of sql scripts for automation and I want to do something like this
C:\> dBeaver.exe -con "name=...|other connection parameters" -f myFileToRun.sql -o outputFileToWhereMyResultShouldBeExported.txt
I found this issue may be related to what I am asking for.
I looked through the Command line wiki but I was not able to find a particular example of what I am asking for.
Is this currently possible? Did I miss any documentation?

duplicates #2095

authors: kishvanchee uslss


text:
Sybase connections always require to redownload license.txt

System information:

Windows 10 v 10.0.17763.503
DBeaver 6.0.4 (although this issue also occurred with 6.0.0 and later versions)
Java version 1.8.0_201
No extensions

Connection specification:

Database name and version Sybase Adaptive Server Enterprise 12.5.4
Driver name: Sybase jConnect
Do you use tunnels or proxies (SSH, SOCKS, etc)?  No

Describe the problem you're observing:
Whenever I connect to a Sybase ASE database, DBeaver prompts me to download the file LICENSE.txt for the jConnect driver and refuses to connect to the server otherwise.

This behavior only disappears after I manually remove the LICENSE.txt file from the dialog, forcing the application to redownload the driver itself.
Steps to reproduce, if exist:
Attempt to connect to a Sybase ASE 12.5.4 database.

Thanks for the bug report

verified

authors: ALDamico serge-rider uslss


text:
Neo4j: not able to connect

Hello,
When trying to connect to a local instance of Neo4j running in docker, I get an unexpected error from DBeaver:

The error is Connection to the database terminated. This can happen due to network instabilities, or due to restarts of the database
Here is the simple docker-compose file I used to launch the Neo4j instance:
version: "3.7"

services:
  neo4j:
    image: neo4j:4.0.2
    container_name: neo4j
    environment:
      - NEO4J_AUTH=neo4j/root
    ports:
      - 7687:7687
      - 7474:7474
      - 7473:7473

If i use the Neo4j browser based navigator, I am able to connect using the credentials provided in the docker-compose file.
I tried connecting to all the available ports without any success.
Thank you for your help!

Ran into an identical issue but with Neo4j running locally instead of in Docker. Stack trace from DBeaver debug log shows the connection error was reported by the Neo4j driver, not DBeaver. Updating the Neo4j driver from 3.x to 4.x resolved this for me.
2020-05-11 21:43:31.719 - org.jkiss.dbeaver.model.exec.DBCConnectException: org.neo4j.driver.v1.exceptions.ServiceUnavailableException: Connection to the database terminated. This can happen due to network instabilities, or due to restarts of the database
	at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:196)
	at org.jkiss.dbeaver.ext.generic.model.GenericDataSource.openConnection(GenericDataSource.java:124)
	at org.jkiss.dbeaver.model.impl.jdbc.JDBCExecutionContext.connect(JDBCExecutionContext.java:91)
	at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.initializeMainContext(JDBCRemoteInstance.java:86)
	at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.<init>(JDBCRemoteInstance.java:52)
	at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.initializeRemoteInstance(JDBCDataSource.java:109)
	at org.jkiss.dbeaver.ext.generic.model.GenericDataSource.<init>(GenericDataSource.java:106)
	at org.jkiss.dbeaver.ext.generic.model.meta.GenericMetaModel.createDataSourceImpl(GenericMetaModel.java:72)
	at org.jkiss.dbeaver.ext.generic.GenericDataSourceProvider.openDataSource(GenericDataSourceProvider.java:95)
	at org.jkiss.dbeaver.registry.DataSourceDescriptor.connect(DataSourceDescriptor.java:801)
	at org.jkiss.dbeaver.registry.DataSourceDescriptor.reconnect(DataSourceDescriptor.java:1048)
	at org.jkiss.dbeaver.registry.DataSourceDescriptor.refreshObject(DataSourceDescriptor.java:596)
	at org.jkiss.dbeaver.model.navigator.DBNDatabaseNode.refreshNode(DBNDatabaseNode.java:324)
	at org.jkiss.dbeaver.model.navigator.DBNDataSource.refreshNode(DBNDataSource.java:272)
	at org.jkiss.dbeaver.ui.navigator.actions.NavigatorHandlerRefresh$1.run(NavigatorHandlerRefresh.java:154)
	at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:103)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: java.sql.SQLException: org.neo4j.driver.v1.exceptions.ServiceUnavailableException: Connection to the database terminated. This can happen due to network instabilities, or due to restarts of the database
	at org.neo4j.jdbc.bolt.impl.BoltNeo4jDriverImpl.connect(BoltNeo4jDriverImpl.java:97)
	at org.neo4j.jdbc.Driver.connect(Driver.java:61)
	at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.lambda$0(JDBCDataSource.java:157)
	at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:174)
	... 16 more
Caused by: org.neo4j.driver.v1.exceptions.ServiceUnavailableException: Connection to the database terminated. This can happen due to network instabilities, or due to restarts of the database
	at org.neo4j.driver.internal.util.ErrorUtil.newConnectionTerminatedError(ErrorUtil.java:44)
	at org.neo4j.driver.internal.async.HandshakeHandler.channelInactive(HandshakeHandler.java:84)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
	at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
	at org.neo4j.driver.internal.shaded.io.netty.handler.ssl.SslHandler.channelInactive(SslHandler.java:1010)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at org.neo4j.driver.internal.shaded.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
	at org.neo4j.driver.internal.shaded.io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at org.neo4j.driver.internal.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1409)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at org.neo4j.driver.internal.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:927)
	at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822)
	at org.neo4j.driver.internal.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at org.neo4j.driver.internal.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
	at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at org.neo4j.driver.internal.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
	at org.neo4j.driver.internal.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

unfortunately we've got no env to test.
Is issue still reproducible?

there is no update on the issue for a long time.
If it is still actual - feel free to reopen it

there is no update on the issue for a long time.
If it is still actual - feel free to reopen it


text:
Prepared the Traditional Chinese translation

For Traditional Chinese users

Wow, this looks great 👍
Thanks a lot for the contribution.
I'll review next week.

authors: ralic serge-rider


text:
Issue #4500

Filter Database Objects error

Please write a bit more info. In my test env it works fine with current filter.
Query "show databases like ?" is just fine.
What is the point to replace it with
show databases WHERE `Database` LIKE ?

?

authors: rubengonzalezasencio serge-rider ydq


text:
[Feature] unit testing: dbt should tell me why it couldn't get columns (the model doesn't yet exist)

Is this a new bug in dbt-core?

[X] I believe this is a new bug in dbt-core
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
I tried making my first unit test from scratch (no using the docs!) and got the below error.
It turns out that the issue was that the models hadn't been built yet, but the error didn't explain why it couldn’t get the columns.
(dbt-dev) joel@Joel-Labes integration_tests % dbt test -s test_type:unit
02:56:57  Running with dbt=1.8.0-b3
02:56:57  Registered adapter: snowflake=1.8.0-b3
02:56:58  Found 22 models, 2 analyses, 24 seeds, 19 data tests, 576 macros, 1 unit test
02:56:58
02:56:59  Concurrency: 8 threads (target='dev')
02:56:59
02:56:59  1 of 1 START unit_test unit_compare_queries::identical_records_compare_queries . [RUN]
02:57:01  1 of 1 ERROR unit_compare_queries::identical_records_compare_queries ........... [ERROR in 1.62s]
02:57:01
02:57:01  Finished running 1 unit test in 0 hours 0 minutes and 3.29 seconds (3.29s).
02:57:01
02:57:01  Completed with 1 error and 0 warnings:
02:57:01
02:57:01    Compilation Error in model unit_test_model_a (models/unit_test_wrappers/unit_tests.yml)
  Not able to get columns for unit test 'unit_test_model_a' from relation ANALYTICS_DEV.DBT_JLABES_INT_TESTS.unit_test_model_a

  > in macro get_fixture_sql (macros/unit_test_sql/get_fixture_sql.sql)
  > called by model unit_test_model_a (models/unit_test_wrappers/unit_tests.yml)
02:57:01
02:57:01  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1

Expected Behavior
Ignoring the fact that there could be other causes for this (ephemeral models? permissions issues? idk), the general thing that I felt was missing was an explanation of what I could do to solve it. Something like
Not able to get columns for unit test 'unit_test_model_a' from relation ANALYTICS_DEV.DBT_JLABES_INT_TESTS.unit_test_model_a because the relation doesn't exist.
would be plenty to make me go "oh of course I need to do a build (or defer to prod).
Steps To Reproduce

Add a new model
Add a unit test to that model but don't build the new model
Run dbt test -s test_type:unit

Relevant log output
No response
Environment
- OS:
- Python:
- dbt: dbt-core@2edd5b3335ddea4ca5e7dd8aaa3bfe3a5d30499a (using Core on main as of about 5 days ago, roughly equivalent to the 1.8 beta)

Which database adapter are you using with dbt?
No response
Additional Context
No response

I tried making my first unit test from scratch (no using the docs!) and got the below error.

That's awesome 🤩
Yeah, it would be nice to give more hints to our intrepid hikers on how to get back on the trail.
Re-labeling as a feature request and adding the refinement label to consider an alternative error message.

Hello! A few thoughts:

if you're using deferral, you should be fine here! (as long as the relation is built in the environment you're deferring to)
you can also use the --empty flag to build an empty relation, if you don't want to use less warehouse $$$

  Not able to get columns for unit test 'unit_test_model_a' from relation ANALYTICS_DEV.DBT_JLABES_INT_TESTS.unit_test_model_a

I agree this is a bit vague. You could not be able to get the columns from this relation because:

the relation doesn't exist (hasn't been built yet)
the relation doesn't exist (it never will, it's an ephemeral model)

I'm with updating to:
  Not able to get columns for unit test 'unit_test_model_a' from relation ANALYTICS_DEV.DBT_JLABES_INT_TESTS.unit_test_model_a because the relation doesn't exist.

Hey @graciegoheen @joellabes, I would like to take this issue to get started in the repository, can I take it ?

@AlejandroBlanco2001 That would be great!

@AlejandroBlanco2001 Nice detective work! I think this might be a bit simpler though — the compilation error is actually being raised directly by the get_fixture_sql macro in dbt-adapters. It sounds like the proposed resolution is to update the language in that exception message.

@jtcohen6 Oh, thanks for being even more specific about where the message is. Do you think we should update that error message directly, or should we raise the compilation error inside the other method to keep it more generic? I'm happy to open the PR after that.

@jtcohen6 @graciegoheen PR opened 🎉

repo: dbt-labs/dbt-core


text:
Macro resolver remove lazy loading

resolves #

Problem

Solution

Checklist

[ ] I have read the contributing guide and understand what's expected of me
[ ] I have run this code in development and it appears to resolve the stated issue
[ ] This PR includes tests, or tests are not required/relevant for this PR
[ ] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
[ ] This PR includes type annotations for new and modified functions

After this change:
x❯ python3 scripts/migrate-adapters.py core/dbt/adapters

core/dbt/adapters/protocol.py:
  - dbt.contracts.graph.model_config::BaseConfig
  - dbt.contracts.graph.manifest::Manifest

core/dbt/adapters/factory.py:
  - dbt.mp_context::get_mp_context

core/dbt/adapters/base/connections.py:
  - dbt.contracts.graph.manifest::Manifest

core/dbt/adapters/base/query_headers.py:
  - dbt.context.manifest::generate_query_header_context
  - dbt.contracts.graph.manifest::Manifest

core/dbt/adapters/base/impl.py:
  - dbt.contracts.graph.manifest::Manifest

migration progress: 97.07% of dbt imports are valid (from adapters or common)
remaining core imports: 7

Closing in favor of: https://github.com/dbt-labs/dbt-core/pull/9243


text:
Snowflake: When creating external tables, add an option to use database and schema

Describe the feature
Currently snowflake uses database and schema from the profile when it connects to the snowflake.
This prevents creation of external tables in a different database/schema than the default database/schema that snowflake connects to.
Describe alternatives you've considered
Manage multiple profiles for each database/schema neeed
Additional context
This is specific to snowflake
Who will this benefit?
Useful for creating external tables in multiple/different schemas at the same time

This is already possible, though I think I can understand your confusion. Ultimately, this package is just an extension of Sources. So this package (under-documented as it is) makes no mention of this feature.
Instead it's described as a feature of Sources (see Add Sources to your DAG: Declaring a source)
The constraint here is that you must use modify each combination of database & schema as a distinct source. In my opinion, that's a good thing, as it forces you to separate your concerns.
I'm closing for now, but feel free to open back up, if I'm mistaken here

authors: dataders samarpatel-ce

repo: dbt-labs/dbt-external-tables


text:
Add --quiet flag docs and print macro

Contributions

[X] I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates
https://github.com/dbt-labs/docs.getdbt.com/blob/current/website/docs/reference/global-configs.md
https://github.com/dbt-labs/docs.getdbt.com/blob/current/website/docs/reference/dbt-jinja-functions.md
What part(s) of the page would you like to see updated?
I'd like to update the documentation to add info about the new global config that PR-4701 will create. Additionally, I'd like to add documentation for the print Jinja function that the PR creates.
Additional information
No response

Making a note that this change will come out with dbt-core v1.1.0.

@ehmartens Hello! The beta for v1.1 will be cut next week for this. Did you want to draft docs for this? I can review and edit them. One caveat is that we are adding versioning so the docs would be encapsulated in versioning code like this:
<VersionBlock firstVersion="1.1">

New content for `-- quiet` flag

</VersionBlock>

cc: @jtcohen6

Hi @runleonarun ! I'm so sorry for the delay on this. I'm happy to do this tomorrow morning (Us Eastern time) if there is still a need. Let me know! I can have a PR done by Noon US Eastern Time.

@ehmartens Apologies from me while I try to figure out a process for features created from the dbt community 😄 Anything you could provide would be helpful, even if unpolished. I can clean it up and publish it.

authors: ehmartens emmyoop runleonarun


text:
devhub group for blog articles, not all of dx

Description & motivation
I want fewer emails

months later, joel's inbox can know peace

authors: gwenwindflower joellabes


text:
Cypress component test config

Code Task Complete ⚡
✨ View the Asana Task here ✨
Description of Changes
(A brief idea of what reviewers should be looking for, or the expected behavior of the new update.)
Description of the Issue:
This is the beginning of the Cypress Test Improvements.  This PR focus on configuration for component testing and a small example that component test.
Preview Links
https://deploy-preview-2827--docs-getdbt-com.netlify.app/
Additional Info
(Any additional comments here, if any)

Hey @wendyvasq, in its current state should I be able to run npx cypress open, click Component Testing, and be able to run the VersionBlock test?
I just went to try that and while I'm now seeing the Component Testing as Configured within the GUI, upon clicking I'm getting the error below.

yup john those are the steps.
Oh interesting I will get that fixed . Good looking out !

@john-rock  I pushed up supporting docs for component testing let me know if that fix the issue you saw.  using npx cypress open  -> navigate to component testing.

@john-rock could you please get back to Wendy on this? Thanks!

authors: lfracalossiFA wendyvasq


text:
to_posix_string and to_windows_string functions

Along the lines of #1 it would be useful to support POSIX and Windows path renderings for applications which may communicate across platforms.

I could use this as well.  Either as to_posix_string / to_windows_string or a way to treat a path implicitly as the compiled platform (default) or a chosen platform in most functions.  For instance reading in "C:\whatever\thing.exe" on a mac should be able to be flagged and treated as "C://whatever//thing.exe" and get the same filename/basename considerations as it would on windows.

authors: hcarty rseymour

repo: dbuenzli/fpath

license: ISC


text:
Consider using models like ORM?

Hi, I use coffeescript and I think that collection-helpers can use models, like an ORM. Right?

Hey @kidchenko there's other packages available that add orm-like features such as https://atmospherejs.com/emmerge/graviton it's not the intention with this package.

hmm. I guess it was not what I wanted

authors: dburles kidchenko

repo: dburles/meteor-collection-helpers


text:
DCOS-15287: Improve app tests

Improve the app system tests by adjusting and improving the test cases.

Add additional checks to verify that the Marathon response is as expected. To do so the test will open the service detail, hit edit and check the form content.
Introduce new test to check that Marathon throws an error in case an app with an existig ID already exists.
Adjust create simple app test to bail early in case of an error.

I tried
docker run -it --rm --ipc=host \
  -e CLUSTER_URL=$OPEN_CLUSTER_URL  \
  -v `pwd`:/dcos-ui \
  mesosphere/dcos-ui dcos-system-test-driver \
  -v ./system-tests/driver-config/dev.sh

and it doesn't work, I'm still behind how the whole system-test thingie should be working

Jobs tests are failing, unrelated to the changes though

Hm, which one exactly?

Like all of them

Though I might have tested it wrong. It is not always clear which magic shell command I need to run 😩

retest this please

Can be merged once the tests are all ✅

authors: juliangieseke nLight weblancaster

repo: dcos/dcos-ui


text:
DCOS-8109 Add spacing between icon and text

From

To

Reviewer: @MatApple

mesosphere_ci retest this please

@pierlo-upitup Pushed a commit, feel free to merge if you're cool with my changes.

Duh, of course! I blanked out ;-)
Thanks!!!

authors: jfurrow mesosphere-web pierlo-upitup


text:
Update cache headers to more appropriate values

High-level description
The UI configuration in Admin Router was modified in #5140. This was back-ported to 1.13 in #5281 with some additional changes based on further discussion. This PR forward-ports those additional changes back into master to keep the configurations aligned.
Corresponding DC/OS tickets (obligatory)
These DC/OS JIRA ticket(s) must be updated (ideally closed) in the moment this PR lands:

DCOS_OSS- Foo the Bar so it stops Bazzing.

Related tickets (optional)
Other tickets related to this change:

DCOS-5978 Admin Router: enable gzip compression for relevant MIME types

Checklist for all PRs

[ ] Added a comprehensible changelog entry to CHANGES.md or explain why this is not a user-facing change: Added in previous PR
[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[x] Read the DC/OS contributing guidelines
[x] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[ ] Change log from the last version integrated (this should be a link to commits for easy verification and review): example
[ ] Test Results: [link to CI job test results for component]
[ ] Code Coverage (if available): [link to code coverage report]

PLEASE FILL IN THE TEMPLATE ABOVE / DO NOT REMOVE ANY SECTIONS ABOVE THIS LINE
Instructions and review process
What is the review process and when will my changes land?
All PRs require 2 approvals using GitHub's pull request reviews.
Reviewers should be:

Developers who understand the code being modified.
Developers responsible for code that interacts with or depends on the code being modified.

It is best to proactively ask for 2 reviews by @mentioning the candidate reviewers in the PR comments area. The responsibility is on the developer submitting the PR to follow-up with reviewers and make sure a PR is reviewed in a timely manner. Once a PR has 2 ship-it's, no red reviews, and all tests are green it will be included in the next train.

@mesosphere-mergebot label Ready For Review

repo: dcos/dcos


text:
Enable Windows App tests.

High-level description
The Windows app test should pass because the windows- fault domains have been
removed.
Corresponding DC/OS tickets (required)

D2IQ-66235
D2IQ-65808

@mesosphere-mergebot request-review @jongiddy @janisz

@mesosphere-mergebot changelog-not-required internal change

@mesosphere-mergebot changelog-not-required @DominikDary

@mesosphere-mergebot request-review @DominikDary

@mesosphere-mergebot label Ship It

@mesosphere-mergebot sync

authors: janisz jeschkies


text:
Add GitHub Actions CI

Replace TravisCI.
Add rustdoc workflow.
Add build and test and lint workflow for Rust.
Add audit workflow with cargo-deny.
rustfmt the codebase on latest stable.
cargo clippy on latest stable.

oops sorry I meant to publish this to my fork.

repo: dcrewi/rust-mersenne-twister


text:
What's the use case of this language?

Is there a academic paper, or why this came to be?

There's no paper or real reason; I made Song just for fun. I liked the idea of a language that was heavily focussed on pattern matching, so that got me motivated to give it a go.
If you love programming, I highly recommend writing your own programming language. It's an amazing feeling to execute code in your own language!

authors: dcutting freesuraj

repo: dcutting/Song


text:
🛑 Danbs is down

In f235687, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 3473 ms

Resolved: Danbs is back up in 2b224ea after 332 days, 18 hours, 19 minutes.

repo: ddanbs/upptime


text:
🛑 Google is down

In 4df7554, Google (https://www.google.com) was down:

HTTP code: 429
Response time: 293 ms

Resolved: Google is back up in 519d053 after 6 minutes.