text:
Allow to omit type hint in GroupBy.transform, filter, apply

This PR proposes to allow to omit type hint.

If type hint is not given, Koalas collects data as pandas DataFrame.
1.1. If it returns less than 1000, it returns ks.DataFrame(pdf).
1.2. if it returns more than 1000, it just infers schema from pdf.
If type hint is found, uses that as return type directly.

When the schema is inferred without type hint, index is cleanly kept.

When the schema is given via type hint, index is lost - this is because pandas sometimes keeps the index and sometimes not. Koalas does not know without executing the given func once.
Please see the example below:
pdf = pd.DataFrame({"timestamp":[0.0, 0.5, 1.0, 0.0, 0.5], "car_id": ['A','A','A','B','B']})
print(pdf.groupby('car_id').apply(lambda _: pd.DataFrame({"column": [0.0]})))

          column
car_id
A      0     0.0
B      0     0.0

pdf = pd.DataFrame({"timestamp":[0.0, 0.5, 1.0, 0.0, 0.5], "car_id": ['A','A','A','B','B']})
print(pdf.groupby('car_id').apply(lambda x: x))

   timestamp car_id
0        0.0      A
1        0.5      A
2        1.0      A
3        0.0      B
4        0.5      B

Therefore, index information is lost in this case

Resolves #628
This PR also resolves 3. at https://github.com/databricks/koalas/issues/409#issuecomment-518762148

@patryk-oleniuk, with this PR and https://github.com/databricks/koalas/pull/633 PR, you will be able to proceed further. Hope this unblocks you.

Codecov Report

Merging #646 into master will decrease coverage by 2.34%.
The diff coverage is 62.5%.

@@            Coverage Diff             @@
##           master     #646      +/-   ##
==========================================
- Coverage   92.97%   90.63%   -2.35%
==========================================
  Files          31       31
  Lines        5085     5082       -3
==========================================
- Hits         4728     4606     -122
- Misses        357      476     +119

Impacted Files
Coverage Δ

databricks/koalas/groupby.py
85.75% <62.5%> (+4.19%)
:arrow_up:

databricks/koalas/usage_logging/__init__.py
23.14% <0%> (-74.08%)
:arrow_down:

databricks/koalas/usage_logging/usage_logger.py
50% <0%> (-50%)
:arrow_down:

databricks/koalas/__init__.py
76.31% <0%> (-7.9%)
:arrow_down:

databricks/conftest.py
90.69% <0%> (-6.98%)
:arrow_down:

databricks/koalas/utils.py
97.43% <0%> (-2.57%)
:arrow_down:

databricks/koalas/frame.py
92.81% <0%> (-1.84%)
:arrow_down:

databricks/koalas/namespace.py
87.6% <0%> (-1.61%)
:arrow_down:

databricks/koalas/plot.py
92.51% <0%> (-1.58%)
:arrow_down:

databricks/koalas/internal.py
98.63% <0%> (-0.69%)
:arrow_down:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51c0299...78c5ccc. Read the comment docs.

tests passed. codecov is due to missing coverage hits in Python worker side

authors: HyukjinKwon codecov-io

repo: databricks/koalas


text:
How to get started!

Would it be possible to add a Get Started section to the Read-me file.
I mean this is a scientific library, and scientists usually jump from one framework to another to find what best suits them. So it is better to add this section to make their life easier.
I have started using spark and scala couples of weeks ago and I want to use spark in a functional data analysis project. I passed by Spark TFOCS and I am not not sure how to get started (this library is not shipped with spark).
After some research about sbt I figured it out. I downloaded the source code and attached it to my sbt project. It was simple but if it was here in the first place it would have saved me some time.

Hi AM-DNEGEL, thank you for trying out TFOCS for Spark. Since this is a free open source library we invite you to document the installation procedure you followed and submit a PR, as a resource for other interested developers.

source: gharchive/issue

authors: AM-DNEGEL staple

repo: databricks/spark-tfocs

license_type: permissive

license_source: bigquery


text:
Rows with special characters getting dropped

I expect the below row to be parsed with default UTF-8, though it has some special characters, but end up getting dropped. any configurations that I'm missing or am I staring at an enhancement?
<artist><id>4800147</id><name>Ðœ. Ð¨ÑƒÑÑ‚ÐµÑ€</name><profile></profile><data_quality>Needs Major Changes</data_quality></artist>
WARN StaxXmlParser$: Dropping malformed row: <artist><id>4800147</id><name>╨£. ╨¿╤â╤?╤é╨╡╤Ç</name><profile></profile><data_quality>Needs Major Changes</data_quality></ar
tist>

Hm. I will take a look. I think it should work okay.

Here is the abt file I'm working with, also running on Windows 10
name := "dataframes"
version := "1.0"
scalaVersion := "2.11.8"
val sparkVersion="1.4.0"
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % sparkVersion % "provided",
"org.apache.spark" %% "spark-sql" % sparkVersion,
"com.databricks" %% "spark-xml" % "0.3.3"
)
resolvers ++= Seq(
"Apache HBase" at "https://repository.apache.org/content/repositories/releases",
"Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/"
)
resolvers += Resolver.mavenLocal

@somasundaramsekar Do you mind if I ask to try this with 0.3.4? It seems this was fixed somewhere.

@HyukjinKwon just tried, still the same, rows are being dropped

I am facing the same issue with CDATA in my XML that contains the Hex character denoted by the sequence c0 4d 57 . I am using spark 1.6.1, scala 2.10, spark-xml 0.4.1
Are there any unit tests around StaxXmlParser that shows this kind of UTF-8 data is handled correctly?

Are you sure it's not the newline in your closing tag? or is that just how it was pasted here?
It's read according to the charset you specify; are you sure it's UTF-8?

authors: HyukjinKwon bjacques somasundaramsekar srowen

repo: databricks/spark-xml

license_source: github-api


text:
Dockerized tests in order to run in Ubuntu 16.04 (required by tf >= 1.5)

Run tests in docker environment with Ubuntu 16.04.

Travis is under maintenance, I will check again later.

Thanks for the review @tjhunter!
I am not sure I understand your concerns. To clarify the situation:

What this PR does:

move Travis project build and test execution into docker with ubuntu 16.04 environment

I believe all of the code remained the same, e.g. the conda environment was used in
the same way before this change.

Why does it do it:

tf >= 1.5 requires Ubuntu >= 16.04, travis only supports 12.04 and 14.04

What is affected:

only Travis; everything is now run inside a docker container.

Are there other options:

the only other option I am aware of is to move the tests into different CI environment such as Jenkins.

Can we revert this change in the future

yes, when travis makes ubuntu 16.04 available, but it's not clear when that is gonna happen.

I chose the docker path cause it seemed to be the least possible change required to get the tests running again. The docker code can definitely be simplified. I did it this way partly cause I am not a master of docker and partly because this closely mimics the previous state - i.e. all the commands are there in the same order, the only difference is we now send them to the docker container.
Personally, I don't see a big problem with running tests inside Docker. It gives us greater control of the environment and I believe it is quite common for people to do that, including tests run inside Travis. We can spend some time to figure out how to do it properly (e.g. get the maven cache working again) but in the short term, at least this way we can run the tests successfully.
But I am really open to suggestions. Feel free to scrap this PR  or make changes to it if you have a better solution.

Ok,  will do. Thanks Tim!

@tomasatdatabricks I am merging. Thanks for the work!

source: gharchive/pull-request

authors: thunterdb tomasatdatabricks

repo: databricks/tensorframes


text:
updates to stat var explorer

includes

adds description and source url
adds examples to default page
updates number formats
removes toggle from tree widget
some style changes

For the cursory audience, could you please include a description of the updates, esp user-visible ones?

screenshots are also really useful for these. on a mac, take a screenshot to the clipboard, and paste it in the edit box in github.

screenshot

thanks for the review!

authors: beets n-h-diaz pradh

repo: datacommonsorg/website


text:
[library] Add highlight component

Adds the ability to render a highlight tile as a web component.
This PR:

Adds a highlight component that wraps the highlight tile
Adds the ability to specify a specific date in the highlight tile
Adds a "part" to the highlight tile containing div to allow for web component styling via CSS
Adds documentation for the new highlight component
Also fixes an error in the map web component documentation (date attribute should be under optional, not required)

Now the syntax
<datacommons-highlight
  description="Population of the United States"
  place="country/USA"
  variable="Count_Person"
  date="2020"
></datacommons-highlight>

Produces:

Thanks Julia! I think @pradh added something to the highlight tile the other day to show the unit-can you verify this works with the web component?

The unit will show in the tile when a unit is passed into the statvar spec. Added an optional attribute to set a "unit" string to display to the web component.


text:
Correction for raw affiliation ISCR - Institut des Sciences Chimiques de Rennes (Campus de Beaulieu - Bât. 10 Avenue du Général Leclerc 35042 Rennes Cedex - France); ScanMAT - Synthèse Caractérisation Analyse de la Matière (Université de Rennes 1, 263 Avenue du General Leclerc, 35042 RENNES CEDEX - France)

Correction needed for raw affiliation ISCR - Institut des Sciences Chimiques de Rennes (Campus de Beaulieu - Bât. 10 Avenue du Général Leclerc 35042 Rennes Cedex - France); ScanMAT - Synthèse Caractérisation Analyse de la Matière (Université de Rennes 1, 263 Avenue du General Leclerc, 35042 RENNES CEDEX - France)
raw_affiliation_name: ISCR - Institut des Sciences Chimiques de Rennes (Campus de Beaulieu - Bât. 10 Avenue du Général Leclerc 35042 Rennes Cedex - France); ScanMAT - Synthèse Caractérisation Analyse de la Matière (Université de Rennes 1, 263 Avenue du General Leclerc, 35042 RENNES CEDEX - France)
new_rors: 00adwkx90;015m7wh34;04fsxy317
previous_rors: 00adwkx90;015m7wh34
works_examples: W4309108670;W4380562639;W4364375975;W4386813120;W4385549104
contact: 61c23a692b7bed4bcd62ee17f32b0b99:a44bf4ab165a748e385b71f093b25d3f @ univ-rennes.fr

This issue was accepted and ingested by the OpenAlex team on 2024-10-10. The new affiliations should be visible within the next 7 days.

repo: dataesr/openalex-affiliations

license: MIT


text:
Correction for raw affiliation Université de Rennes, EHESP, CNRS, Inserm, Arènes–UMR 6051, RSMS (Research on Services, Management in Health), U 1309, Rennes, France

Correction needed for raw affiliation Université de Rennes, EHESP, CNRS, Inserm, Arènes–UMR 6051, RSMS (Research on Services, Management in Health), U 1309, Rennes, France
raw_affiliation_name: Université de Rennes, EHESP, CNRS, Inserm, Arènes–UMR 6051, RSMS (Research on Services, Management in Health), U 1309, Rennes, France
new_rors: 02vjkv261;01sc83v92;015m7wh34;02feahw73;00m4rxj18
previous_rors: 02vjkv261;01sc83v92;015m7wh34;02feahw73
works_examples: W4386530000
contact: 4554724d63c7a887aabe5cd48594b578:641c2392e75ccf2bb5b323cde2617dee @ univ-rennes.fr

This issue was accepted and ingested by the OpenAlex team on 2024-10-10. The new affiliations should be visible within the next 7 days.


text:
Correction for raw affiliation MISTIC

Correction needed for raw affiliation MISTIC
raw_affiliation_name: MISTIC
new_rors:
previous_rors: 01pd2sz18
works_examples: W4285762928;W3208860470;W4285761052;W4286970242
contact: 4a93f2d2f940dd89ad00e92e7e77a228:14db96f49763d899291527e6398aaf2859b8 @ inrae.fr

This issue was accepted and ingested by the OpenAlex team on 2024-12-19. The new affiliations should be visible within the next 7 days.


text:
WIP: AsRef

Changelog
WIP: AsRef

This change is

It looks like using AsRef<[ID]> won't work.
I guess it's because the compiler does not whether BTreeSet implements AsRef<[ID]> too.
  --> openraft/src/quorum/quorum_set_impl.rs:29:1
   |
6  | / impl<ID> QuorumSet<ID> for BTreeSet<ID>
7  | | where ID: PartialOrd + Ord + Copy + 'static
8  | | {
9  | |     fn is_quorum<'a, I: Iterator<Item = &'a ID> + Clone>(&self, ids: I) -> bool {
...  |
25 | |     }
26 | | }
   | |_- first implementation here
...
29 | / impl<ID, T> QuorumSet<ID> for T
30 | | where
31 | |     ID: PartialOrd + Ord + Copy + 'static,
32 | |     T: AsRef<[ID]>,
...  |
50 | |     }
51 | | }
   | |_^ conflicting implementation for `std::collections::BTreeSet<_>

@schreter

repo: datafuselabs/openraft


text:
UI exception after adding a data contract

After adding the data contract below via: datahub datacontract upsert -f mq.yaml, when navigating to the dataset in UI I get: No enum constant com.linkedin.datahub.graphql.generated.AssertionType.FRESHNESS (code 400)
mq.yaml
entity: urn:li:dataset:(urn:li:dataPlatform:postgres,ml_db.public.mq,PROD)
version: 1
data_quality:
  - type: unique

I would have expected the data contract to show up in the UI under the validation tab. At the moment, the dataset is not accessible anymore because of said error.
Desktop (please complete the following information):

OS: deployed datahub via the docker quickstart. OS: MacOS Sonoma
Browser: chrome
Version: v0.13.1

When deleting the datacontract via CLI, the issue seems to be persisting and now I have a broken entity I can't access anymore.

The same happens when in the contract you specify a schema. This time the exception is: No enum constant com.linkedin.datahub.graphql.generated.AssertionType.DATA_SCHEMA (code 400). Even after deleting the data contract, the issue persists. This makes these datasets unusable. Even deleting records in the metadata table in mariadb leads to nothing.

Same issue here, is anybody looking into this ?

The issue persists even after deleting the document related to the datacontract in elasticsearch.

Same issue here - it made the asset that i have attached the data contract to unusable even after removing the object.

Hey everyone, I've opened a PR with a possible fix here: https://github.com/datahub-project/datahub/pull/10534
I will attempt to repro soon, but if you're able to check out the branch and verify this fixes it for you let me know!

Thanks @jayacryl for addressing this!

authors: adriano-sportsbet darthale jayacryl

repo: datahub-project/datahub


text:
Breaking change in tableau ingestion

Describe the bug
After the changes introduced here we are using a field projectLuid that was introduced in Tableau Cloud June 2022 / Server 2022.3.
This is a breaking change, we should either make it explicit here or find a workaround.
Tableau releases
To Reproduce
Steps to reproduce the behavior:

Ingest from Tableau using a version previous to 2022.3
Ingestion fails with error Validation error of type FieldUndefined: Field 'projectLuid' in type 'Workbook' is undefined @ 'workbooksConnection/nodes/projectLuid'

Expected behavior
Ingestion should continue to work with versions > 2021.1.10.
Screenshots

@mohdsiddique this looks like a real regression / issue with https://github.com/datahub-project/datahub/pull/7483 or https://github.com/datahub-project/datahub/pull/7426.
Is there any way to avoid this, or do we need to make our requirements more explicit?

I'm also experiencing this issue with the projectLuid field being introduced in a later version of Tableau than what I'm currently using. It's definitely a breaking change that affects backward compatibility, and I agree that finding a workaround or making it explicit in the documentation would be helpful.
Has anyone found a solution or workaround for this yet? If not, let's work together to find a solution that works for everyone impacted.
Thanks for letting us know @maaaikoool

We are facing the same issue when trying to migrate from DataHub 0.9.6 to 0.10.1.
See this slack thread for some context
I would really appreciate a workaround as this prevents us from adopting a newer version than 0.9.6 (I would assume that for most teams/setups, upgrading Tableau is out of reach or not easily possible)

Folks while we work to figure this out please continue to use the v0.9.6 ingestion source for your PowerBI recipe :)


text:
fix: fix customer logo alignment and make responsive

Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

thanks @jeffmerrick - locally tested & going to merged it

authors: jeffmerrick yoonhyejin


text:
Not all bills have a bill type.

Not all bills have a bill type, so this distorts the results for the faceted search filter.

Can you give me an example of a bill without a bill type?

False alarm! I was doing some work with the Haystack indices, and I might have temporarily messed up something. Short story: everything is there now.

authors: fgregg reginafcompton

repo: datamade/la-metro-councilmatic

license: mit


text:
[HW] Pankaj Biradar

Name: Pankaj Biradar
Email: pankaj28988@gmail.com
Linkedin Profile: https://www.linkedin.com/in/pankaj-biradar
Attach the homework step I screenshots below:
Gitpod -

Netlify -

My Astra DB -

Attach the homework step II screenshots below:
Swagger UI to fetch data -

Final State of App in Production -

Well, done! Enjoy your badge and see you on the third episode of the series!

Hey Pankaj Biradar, great job! Congrats! Here is your badge! https://api.badgr.io/public/assertions/3LEelbltRt6r-op0fKl_gg

authors: HadesArchitect hemidactylus pankaj892


text:
[Bug] [登录页] 正常登录之后，重新访问url 的 / 路径，会自动跳转登录页

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
在登录之后，可以正常访问功能页面，接口请求也正常，但是如果直接访问URL 的根路径（比如 http://localhost:5600 ）,会默认跳转登录页，但是更改路径访问功能页面也是正常的，应该是页面拦截有些问题。
DataVines Version
1
DataVines Config
无

Running Command
无

Error Exception
无

Engine Type
No response
Java Version
No response
Screenshots
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

经本地调试，可在前端代码 src/App/index.ts 中的重定向那个地方添加一个本地会话判断，从而解决该问题：
原代码： <Redirect from="/" to="/login"} />
新代码： <Redirect from="/" to={shareData.sessionGet(DV_STORAGE_LOGIN)? '/main' : '/login'} />

repo: datavane/datavines


text:
Artist Sales

Currently, tez-tax classifies all token sales as capital gains/losses. In the event that an artist is selling their own work, this may be better classified as 'other income' instead.

This has now been addressed for most major marketplaces. There are known issues with early OBJKT sales of fxhash tokens where the minter was paid royalties instead of the artist. As a result, these tokens are classified as 'other income'.
c40a361658c2251f20c07897f483902c4a845f87

repo: datcsv/tez-tax


text:
Hub on another Flow

Hi how can i use the same hub on another flow? The port is busy.
"Error: listen EADDRINUSE :::80"
Unable to start on Port 80

You need to use only one hub and connect all to this. To connect to another flow, use link nodes.

That works for me thank you.

authors: SamNitro hifiberry

repo: datech/node-red-contrib-amazon-echo


text:
Windows related issues caused by random-access-file dependency

Rolling back to random-access-file@2.1.0 fixes #238. We can update this dependency once https://github.com/random-access-storage/random-access-file/pull/20 comes through.

Codecov Report

Merging #240 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #240   +/-   ##
=======================================
  Coverage   85.52%   85.52%
=======================================
  Files           7        7
  Lines         297      297
=======================================
  Hits          254      254
  Misses         43       43

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c5f3773...4c503d6. Read the comment docs.

‘random-access-file@2.1.2’ released, will reflect that in this pull request

since it is a patch release I don't think you'll need this, unless there's a package-lock.

authors: codecov-io karissa nharlow89

repo: datproject/dat-node


text:
Picture about loading 1000*30000

The author, Hello, I would like to ask the loading of 1000*30000 pictures can not be loaded, can we optimize it?

Sorry, I cannot help if you delete the issue template.

authors: LuckSiege davemorrissey


text:
Update cats-core to 2.4.0

Updates org.typelevel:cats-core from 2.1.1 to 2.4.0.
GitHub Release Notes - Changelog - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Applied Migrations

github:typelevel/cats/Cats_v2_2_0?sha=v2.2.0

Documentation:

https://github.com/typelevel/cats/blob/v2.2.0/scalafix/README.md#migration-to-cats-v220

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.typelevel", artifactId = "cats-core" } ]

labels: library-update, semver-minor, scalafix-migrations

Superseded by #214.


text:
Update scala-library to 2.13.5

Updates org.scala-lang:scala-library from 2.13.1 to 2.13.5.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-lang", artifactId = "scala-library" } ]

labels: library-update, semver-patch

Superseded by #191.


text:
refactor: use model.SeverityError throughout

Big but trivial/mechanic change, just something I ran into while looking into something else, and thought I'd refactor while at it.

Codecov Report

Merging #156 (223d46a) into main (fae7a82) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #156   +/-   ##
=======================================
  Coverage   85.64%   85.64%
=======================================
  Files          77       77
  Lines        3336     3336
=======================================
  Hits         2857     2857
  Misses        479      479

Flag
Coverage Δ

unittests
85.64% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

html-report/build_report.go
86.27% <ø> (ø)

model/rules.go
100.00% <ø> (ø)

motor/rule_applicator.go
97.14% <ø> (ø)

rulesets/ruleset_functions.go
100.00% <ø> (ø)

rulesets/rulesets.go
94.44% <ø> (ø)

cmd/lint.go
89.82% <100.00%> (ø)

cmd/shared_functions.go
55.88% <100.00%> (ø)

model/results.go
95.62% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

authors: codecov-commenter scop

repo: daveshanley/vacuum


text:
[chore] Add a blog to the Korean category blog section.

Add Blog name of Som's velog

Modified: blogs.json

There was a duplicate of Serena's Blog in this PR, but I removed it.
Thank you!

I apologize for being so focused on resolving the code conflicts that I neglected other aspects. It was my mistake. I will pay more attention to merging processes and ensure my pull requests are more carefully prepared in the future. I sincerely appreciate your willingness to take on the trouble to resolve this. Thank you very much.
@daveverwer

No problem at all @devKobe24. You continue to do amazing work. Please keep going 👍

repo: daveverwer/iOSDevDirectory


text:
Pluto Notebook Environments and @bind

Nice package, I could see this useful for writing journal articles. I was running into a few issues when converting Pluto.jl notebooks:

The Pluto.jl  @bind macro isn't in scope so notebooks with code lines using it throw:

notebooktolatex("MyNotebook.jl",template=:book)
ERROR: LoadError: UndefVarError: @bind not defined

Similarly the environment/package setup within a pluto notebook isn't activated, so these have to be manually added in the working julia environment. Seems like somewhere before this function call: https://github.com/davibarreira/NotebookToLaTeX.jl/blob/841a022840277c50be8fca0210cd8ddbe54d29bb/src/NotebookToLaTeX.jl#L56 you should be able to use Pluto.activate_notebook_environment("MyNotebook.jl") to achieve configuring the environment if Pluto is added as a dependency.

Thanks, I'll check. BTW, if you think you know how to solve this, feel free to send a PR.

authors: davibarreira stefanbringuier


text:
FileNotFoundError: Could not find folder `checkpoint'

Dear David Garcia,
I'm so sorry to bother you . When I run your code, It has such error:
D:\Users\aaron\Anaconda3\python.exe E:/Code/srez/srez_main.py
Traceback (most recent call last):
File "E:/Code/srez/srez_main.py", line 191, in
tf.app.run()
File "D:\Users\aaron\Anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "E:/Code/srez/srez_main.py", line 186, in main
_demo()
File "E:/Code/srez/srez_main.py", line 111, in _demo
raise FileNotFoundError("Could not find folder %s'" % (FLAGS.checkpoint_dir,)) FileNotFoundError: Could not find folder checkpoint'
It seems that I dont create checkpoint dir or train dir in a right way, but how can I solve it?
THX

It seems you didn't train the mode firstly.

I have run the "serz_train",but no checkpoint be output.

Have you solved it yet?

Let me know if this is solved, please

repo: david-gpu/srez


text:
64x64 Feature Based Support

I implemented support for 64x64 matrices using a 5th address pin named f. I did not like duplicating the type impl for the  PINS but did not immediately see a better way. Happy to make changes to make it more elegant or do additional testing.

LGTM! The duplicate pin impls aren't nice, but at first glance seem to be the best way.

authors: camrbuss david-sawatzke


text:
Adjusting the step to run the specs with each paginator

Just adjusting the way to run the specs with the proper paginator.

Oops! Thanks!

authors: davidcelis marlonandrade

repo: davidcelis/api-pagination


text:
Installation: form validation

Check AngularJS validation for password confirm (even if checked on server side).

Included in issue #2 as task, closing.

repo: davidefavia/nope


text:
Invariant Violation: requireNativeComponent: "RNArcGISMapView" was not found in the UIManager

Getting the error above in an expo managed workflow.
package.json
{
  "main": "node_modules/expo/AppEntry.js",
  "scripts": {
    "start": "expo start",
    "android": "expo start --android",
    "ios": "expo start --ios",
    "web": "expo start --web",
    "eject": "expo eject"
  },
  "dependencies": {
    "expo": "^35.0.0",
    "react": "16.8.3",
    "react-dom": "16.8.3",
    "react-native": "https://github.com/expo/react-native/archive/sdk-35.0.0.tar.gz",
    "react-native-arcgis-mapview": "^1.0.11",
    "react-native-web": "^0.11.7"
  },
  "devDependencies": {
    "babel-preset-expo": "^7.1.0"
  },
  "private": true
}

Usage
import React from 'react';
import ArcGISMapView from 'react-native-arcgis-mapview';
import { StyleSheet, Text, View } from 'react-native';

export default function App() {
    return (
        <View style={styles.container}>
            <ArcGISMapView ref={mapView => this.mapView = mapView} style={{width: 360, height: 500}}
                initialMapCenter={[{latitude: 34.055561, longitude: -117.182602}]}
                recenterIfGraphicTapped={true}
                rotationEnabled={false}
                onLayerPress={ this.onLayerPress }
                onRoutingStatusUpdate={this.onRoutingStatusUpdate}
                mapBasemap={ { type: 'normal' } }
                onSingleTap={this.onSingleTap}
            />
        </View>
    );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    backgroundColor: '#fff',
    alignItems: 'center',
    justifyContent: 'center',
  },
});

The error is thrown here in AGSMapView.js:
  render () {
    return <AGSMap {...this.props} ref={e => this.agsMapRef = e} />
  }

Not sure what the issue was but when I switched to a react-native cli everything worked...


text:
Governance and Catalog

I was thinking of building a "poor's man data platform" when I found datadex and it is almost everything in my wishlist.
But in a scenario where you may have many data-products, I miss a place to discover and document them. Have you considered adding a discovery and governance layer to the mix?  I am thinking about Datahub or Open-metadata.
P.S. Not sure if this was the right way to ask. ¯_(ツ)_/¯

This is definitely the right way to ask!
Glad to hear Datadex fits what you wanted to build and also, great questions.

in a scenario where you may have many data-products, I miss a place to discover and document them

Not entirely sure what you mean by data-products there but if you're talking about models/datasets. These are some of my thoughts:

Models can be documented both via dbt Docs or by manually creating a Notebook that loops over the datasets (example on one implementation of Datadex).
Quarto is super flexible and you can totally make a catalog or documentation with it.

These are very manual approaches. To be honest is not something I've spent much time thinking about. Would love to explore it together and learn about tools or approaches that improves this process!
Will try to spend some time with Datahub or Open Metadata and report back. Also, feel free to fork the repo and try anything that might fit your needs!

I will consider that... I actually like quarto very much and there is always an overhead when you add one more tool.  I have been also considering it to generate the pdf of the reports.  I recently have been using Typst.app (think of it like a LaTeX replacement) a lot and Quarto less and less. But I heard Quarto now can render Typst templates.

Quarto and Evidence also seem to have some overlap. I am new to Evidence, but I like it is built in Svelte which I can understand better than Quarto source code.

Nice! Been thinking about Typst recently too.

But I heard Quarto now can render Typst templates.

Yes! It's on their prerelease channel though.

Quarto and Evidence also seem to have some overlap

:100: I think Quarto is a bit more generic as it can render JS, SQL, Python, ... but Evidence looks better and has a better DX. I'm struggling between adding it or keeping it simple :see_no_evil:

Typst is now part of the official release and I already created a Typst template :-)

authors: davidgasquez fredguth

repo: davidgasquez/datadex


text:
How to use inPageLinks?

Any place in documentation about this functionality that I missed? Thanks! <3

Set the option to true, and then it will just work
On Tue, Jun 19, 2018 at 7:39 PM, Viktor Fonic notifications@github.com
wrote:

Any place in documentation about this functionality that I missed? Thanks!
<3
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/davidjbradshaw/iframe-resizer/issues/600, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAkHEhXNscw0KFpN7ZSOFRWON_Y24MYgks5t-UV_gaJpZM4UuCDD
.

--
David J. Bradshaw )'( dave@bradshaw.net
'The people we think are a little weird are the ones who change the world'

Simon Sinek

Thanks @davidjbradshaw! But...what does it do? 😄
I have couple of links on the parent page that I'd like to open/navigate to various pages within the iframe, instead of doing full page refresh on the parent page.
I'll just do plain ol' postMessage.
Thanks!

It works the other way around. Fromchi,d to parent
On Thu, 21 Jun 2018 at 09:50, Viktor Fonic notifications@github.com wrote:

Thanks @davidjbradshaw https://github.com/davidjbradshaw! But...what
does it do? 😄
I have couple of links on the parent page that I'd like to open/navigate
to various pages within the iframe, instead of doing full page refresh on
the parent page.
I'll just do plain ol' postMessage.
Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/davidjbradshaw/iframe-resizer/issues/600#issuecomment-399026382,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAkHEhdigPw4ulX8me52JRBLF_qvwQXEks5t-15qgaJpZM4UuCDD
.

--
Sent from my iPhone.

authors: davidjbradshaw vfonic

repo: davidjbradshaw/iframe-resizer


text:
Issue when i integrate jquery.maphilight.js

Hi,
I have implemented imageMapResizer and jquery.maphilight.js for highlighting area on hover or click. Both are working fine. When i resize the window, image is not responding to responsiveness. Area positions are changing and highlight is also working fine.
FYI: http://davidlynch.org/projects/maphilight/docs/
Please suggest...

If you provide a link to a simple test case, I'll take a quick look.

Ok here is the link: http://jsfiddle.net/sandeepgovada/5z7p2514/

Ok here is the link to download zip: http://bit.ly/1JdWZiB (POC i created)

You need to look at your use of CSS, the image can not resize if you you give it a set width and the contain it sits in a fixed width.

For image there are no FIXED width and height written in HTML and in CSS.

Look at your inline styles. You have a fixed sized container for the image.

They are adding on page load. If i open 320/420 screen. The fixed container changes its dimension. But image ONLY is not acting to responsive.

Here is the reference link FYI: https://github.com/kemayo/maphilight/issues/21#issuecomment-63390541

Having looked at this, the issue is with maphilight. It add's styles to the page that stop the image resizing.

Yes. Ur correct. I followed the steps provided and i can see a progress. Hope you will consider this as a issue and resolve in ur upcoming versions.
I still have a problem. When resizing window, i have to refresh the page. Which should be automatically happen without refresh.

authors: DesigningStudios davidjbradshaw

repo: davidjbradshaw/image-map-resizer


text:
Font generation

Hello, with which tool can you generate your own fonts or convert free fonts?

Hey @volthouse thanks for reaching out. Its a bit more involved.
First, you can use the the template https://github.com/davidkreidler/OpenCO2_Sensor/blob/main/font/bahn.indd to insert your own font in the right size (3 times).

Then export it as png and use https://github.com/ayoy/font2bytes to convert it to binary.
./f2b -h 70 -w 47 bahn.png > bahn_big.cpp
./f2b -h 44 -w 29 bahn2.png > bahn_mid.cpp
./f2b -h 22 -w 13 bahn3.png > bahn_sml.cpp
Then import the result into you Arduino\libraries folder esp32-waveshare-epd\src\bahn*.cpp (it needs to be tweaked slightly).
If you have any questions please feel free to reach out.

Thank you for your reply, I apologise for my late reply, I have only just started working on my project again.
I am not familiar with INDD files, is it correct that it is an Adobe InDesign file format? Do I need Adobe software for this?
Many thanks for your help!

Yes the format is Adobe InDesign. You can also use any other Grafik Programm to create the Images for your font in the right format for https://github.com/ayoy/font2bytes .

authors: davidkreidler volthouse


text:
test(e2e): remove test deps from Angular e2e apps

To avoid them from being maintained (specifically, taken into account by dep update bot)

Current dependencies on/for this PR:

main

PR #86   👈

This stack of pull requests is managed by Graphite.


text:
Get github readme content from url are blocked by github using langchain_dart

System Info
flutter latest
Related Components

[ ] doc-loaders
[ ] doc-transformers
[ ] prompts
[ ] llms
[ ] chat-models
[ ] output-parsers
[ ] chains
[ ] memory
[ ] stores
[ ] embeddings
[ ] retrievers
[ ] agents

Reproduction
I am using langchain dart WebLoader to load text from url.
for github url, the first attempt is ok, but after that, every time it was returns this:
 OpenAIChatCompletionChoiceMessageModel(role: OpenAIChatMessageRole.user, content: Skip to contentToggle navigationSign upProductActionsAutomate any workflowPackagesHost and manage packagesSecurityFind and fix vulnerabilitiesCodespacesInstant dev environmentsCopilotWrite better code with AICode reviewManage code changesIssuesPlan and track workDiscussionsCollaborate outside of codeExploreAll featuresDocumentationGitHub SkillsBlogSolutionsForEnterpriseTeamsStartupsEducationBy SolutionCI/CD & AutomationDevOpsDevSecOpsResourcesLearning PathwaysWhite papers, Ebooks, WebinarsCustomer StoriesPartnersOpen SourceGitHub SponsorsFund open source developersThe ReadME ProjectGitHub community articlesRepositoriesTopicsTrendingCollectionsPricingSearch or jump to...Search code, repositories, users, issues, pull requests...SearchClearSearch syntax tipsProvide feedbackWe read every piece of feedback, and take your input very seriously.Include my email address so I can be contactedCancelSubmit feedbackSaved searchesUse saved searches to filter your results more quicklyNameQueryTo see all available qualifiers, see ourdocumentation.CancelCreate saved searchSign inSign in to GitHubUsername or email addressPasswordForgot password?or sign in with a passkeySign upYou signed in with another tab or window.Reloadto refresh your session.You signed out in another tab or window.Reloadto refresh your session.You switched accounts on another tab or window.Reloadto refresh your session.Dismiss alert{{ message }}Find code, projects, and people on GitHub:SearchContact Support—GitHub Status—@githubstatusSubscribe toThe GitHub InsiderDiscover tips, technical guides, and best practices in our monthly newsletter for developers.SubscribeProductFeaturesEnterpriseCopilotSecurityPricingTeamResourcesRoadmapCompare GitHubPlatformDeveloper APIPartnersElectronGitHub DesktopSupportDocsCommunity ForumProfessional ServicesPremium SupportSkillsStatusContact GitHubCompanyAboutCustomer storiesBlogThe ReadME ProjectCareersPressInclusionSocial ImpactShopGitHub on XGitHub on FacebookGitHub on LinkedInGitHub on YouTubeGitHub on TwitchGitHub on TikTokGitHub’s organization on GitHub© 2023 GitHub, Inc.TermsPrivacy(Updated 08/2022)SitemapWhat is Git?You can’t perform that action at this time.

Expected behavior
this is the blank jump link for github, Please add some complicated rule so that github won't treat it as a scraper?

hey @lucasjinreal,
Thanks for reporting the issue.
I don't think there is anything I can do about it. If you want to load files from GitHub, you can create your custom document loader that uses the GitHub API instead. That way you can avoid the scraping protection (and improve the performance and accuracy).

authors: davidmigloz lucasjinreal


text:
Virus Detected

On windows this gets detected (still) as a fasle positive for a virus. Presumably from the UPX headers.

Just wondering if perhaps, for the sake of simplicity and the fact that it's small anyway the UPX compression could be removed or changedd in some way that doesn't make this happen?
@davidnewhall we discussed this a while back on discord and it appears it's still happening even with the latest release zipped up.

UPX: 3-4MB
no UPX: 8-10MB
I can't share non-UPX versions on Discord because they go over 8 MB. I don't think I'll be removing it because with 20+ released compiled binaries it adds up.

My understanding is that even without compression there are still false positives. Does this get caught? (no UPX, just ZIP)
unpackerr.amd64.exe.zip

Perhaps I can remove it for just Windows. Let me tinker on that idea.

No threat detected. The false positive is from the UPX header present. I see no benefit in 2021 to have a 3MB binary over an 8MB one. If you can't upload them to discord, paste a link to a release binary or pay for nitro tier 1?

I see no benefit

I'm just not you. ;)

while you're at it, let me know if that binary I sent over works. It has a whole new set of logic for extraction ordering. :)

while you're at it, let me know if that binary I sent over works. It has a whole new set of logic for extraction ordering. :)
re: golift/xtractr#5

It appears to work fine.

It hasn't extracted anything yet. :D Just keep an eye on it for a day or two. Thank you!

I'll cut a beta release in a bit so you can try that out. It'll have the correct version. I built that one before I pulled the current tag.

New release is out. Please let me know if you have any problems! https://github.com/davidnewhall/unpackerr/releases/tag/v0.9.2-beta1

authors: davidnewhall napei

repo: davidnewhall/unpackerr


text:
Don't duplicate stores in groceries store when resuming WS connection

This fixes a bug wherein, after the WebSocket connection was lost and then reestablished, stores (and their items) could/would be duplicated within the check-in interface.

Unfortunately, it's hard to write a test for this, because doing so would involve disconnecting and then reconnecting the WebSocket connection in a test, which I tried to do using various Chrome/Selenium APIs (emulateNetworkConditions, etc.), but I couldn't get that working, and I'm not sure that it's possible to get working right now; there are some bug reports for Chromium that make me think that this might be a bug / missing feature in Chromium.


text:
Fails on Live URL

Hi @davidtheclark, thanks for this library!
When I run this with the following MDX content...

  freeCodeCamp: A Beginner’s Guide to GraphQL

...it fails as in the error message below:
$ yarn remark index.md
index.md
  2:9-2:86  warning  Link to https://www.freecodecamp.org/news/a-beginners-guide-to-graphql-86f849ce1bec/" is dead  no-dead-urls  remark-lint

⚠ 1 warning

remark-lint-no-dead-urls appears to be taking the " at the end along with the link.
cc @wooorm

Ahh, sorry, just realized that this is an incompatibility between Markdown and MDX.
Markdown does not allow for that syntax (at least on https://markdownlivepreview.com/):

I guess that it wouldn't make sense for remark to support this.
I will see what I can do with gotOptions or something...
Closing.

Ah yeah, HTML like this is not supported by Markdown. See CommonMark. So the URL, in GFM, is seen as an actual URL.
If you want to ignore MDX, you can use the remark-mdx plugin. Although it won’t check href props

Right, thanks for the explanation 👍
I'm looking to lint all of my .mdx files for broken links, don't have any .md files at the moment...

Hmm... I guess I'll use patch-package to remove the " from the end of node.url. 😅

why not use remark-mdx?

Hm, maybe I misunderstood your comment above:

If you want to ignore MDX, you can use the remark-mdx plugin. Although it won’t check href props

I don't want to ignore MDX - I want to check it :)

Looking at remark-mdx, can't tell - does it work with remark-lint and remark-cli?
Edit: I guess it does - my package.json:
{
  "remarkConfig": {
    "plugins": [
      "remark-mdx",
      "remark-lint-no-dead-urls"
    ]
  }
}

I don't want to ignore MDX - I want to check it :)

Well, this plugin works on markdown syntax. remark-mdx makes sure MDX is understood, but it’s still a different syntax that isn’t markdown. I don’t think there exists a project like remark-lint-no-dead-links for MDX. Would <Icon src="..."> also be checked? Etc., many new problems!

Ah, understood. So the links in format of  are not recognized as a link with a node.url by remark-mdx, I suppose...

That is correct, remark/markdown does not understand HTML. And while MDX understands JSX, it does not map JSX to “equivalents” in Markdown.

authors: karlhorky wooorm

repo: davidtheclark/remark-lint-no-dead-urls


text:
Question & discussion on feature: How to see remote calendar events in future months

First of all, thank you davish for making this magnificent plugin to make event management inside obsidian possible. This plugin really makes my life easily with obsidian as I no longer need to open up the browser to see what event do I have
I would like to ask the loading of remote calendars ( CalDAV more specifically). I have my Nextcloud instance and I've connected all my calendars into full calendar. But the thing is that I can't really see the future events (e.g. in 2 months) from full calendar when I open it. I usually need to wait for sometimes in order to load events.
I also notice that the loading of remote calendar is much slower and I don't have any visual hint if the my remote calendars are loaded or failed. Without visual hint, I could well doubt that if I really don't have a event at the desired time slot or is it just not appearing. So sometimes I do have to either wait a bit to make sure my calendar is loaded 2) or open up the website to check the time slot there.
I know that dealing with the remote calendar is much difficult dependent on the network issue or other factors. But I would like to ask if it is possible to 1. have a visual hint on calendar UI about the availability of remote Calendars 2. to have a local copy of remote calendars such that the loading can be faster?  3. or to expose an option at setting to specify how many future month/day events do you want to load in advance (but this is potential bad for those who opens obsidian just to check their events.)
I think the above three are possible methods to deal with the scenario I mentioned. I would like to know how you deal with this problem.
Thanks for your great work and hope you a good day.

Hi @TangdizhiH, thank you for the kind words! All these requests are coming quite soon. The experience with remote ICS and CalDAV calendars will be improved upon in the next update, you can track #303 for progress on that. Remote calendars will be cached after they're accessed the first time, and refreshed from time to time. If any calendar fails to refresh, there will be a Notice in the top-right corner of the window. I'll close this issue as being related to #77, but know that your feedback is appreciated and it will be addressed soon!

authors: TangdizhiH davish

repo: davish/obsidian-full-calendar


text:
Add http protocol

Submit on Discussions

[X] I will submit my feature request as a Discussion.

Hi @davish
I have a few calendars that I access via gateways configured in http (localhost) via DavMail for example. I'd find it very useful to be able to integrate this protocol into FullCalendar too...
What do you think?


text:
OpenApi 3.1 schema validator does not handle variables in server urls

Attached file gives an error: value at /servers/0/url does not match format: uri-reference
While according to OpenApi docs it's a valid url with variable: https://spec.openapis.org/oas/v3.1.0#fixed-fields-3
OpenApi 3.0 validator does not seem to have this problem.
oas-amocrm.json

Interesting—OpenAPI validation uses their schema, which specficies uri-reference: https://github.com/OAI/OpenAPI-Specification/blob/6627c32921a341b2a7001fd1cf9f757b8b728fd1/schemas/v3.1/schema.json#L171
But I don't think using variables (in brackets) makes for a valid uri-reference (I also checked https://www.jsonschemavalidator.net/):
>> JSONSchemer.schema({ 'format': 'uri-reference' }).valid?('https://{subdomain}.amocrm.ru')
=> false

I'll dig into this more later, but here's what I've found so far:

Schema change: https://github.com/OAI/OpenAPI-Specification/pull/2586
Proposed fix: https://github.com/OAI/OpenAPI-Specification/pull/3235
Future definition: https://github.com/OAI/OpenAPI-Specification/issues/3256

I opened a pull request to change the format in the upstream schemas.

Temporary fix: https://github.com/davishmcclurg/json_schemer/pull/161

This also fixed a problem with IDNs. "uri-reference" doesn't not allow those, so at least "iri-reference" should have been used instead.

authors: davishmcclurg ekzobrain


text:
TypeError: a bytes-like object is required, not 'str'

Hi im fairly new to using ghp-import, i'm trying to use it to import a file called "output" from dev branch to master branch and i keep getting this error.
Can anyone help ?

I am unable to reproduce this issue on Windows 10 with conda 4.8.3 (see screenshot below) so I'll resolve this issue for now. Please feel free to re-open if you can provide more details on how to reproduce this. Thanks in advance!

authors: c-w jjsalomon

repo: davisp/ghp-import


text:
Publish newest changes to pip

The search API does not work in the latest version published in PIP. There was a fix merged to github master #31 which should resolve the issue.

I know this took forever :( but latest version is on pip.

authors: chris-catignani dawoudt

repo: dawoudt/JustWatchAPI


text:
feat: @day1co 패키지들을 internal 직전에 위치하도록 수정

PR 의 종류는 어떤 것인가요?

[x] 새로운 기능

수정이 필요하게된 이유가 무엇인가요? (Jira 이슈가 있다면 링크를 연결해주세요)
schema 수정중에 typeorm 전에 @day1co/types 가 위치하는 것을 발견해 eslintrc 레벨에서 수정하고자 합니다.
기존:
import {} from '@day1co/types'
import {} from 'typeorm'

개선:
import {} from 'typeorm'
import {} from '@day1co/types'

참고: https://github.com/import-js/eslint-plugin-import/blob/main/docs/rules/order.md
무엇을 어떻게 변경했나요?
코드 변경을 이해하기 위한 배경지식이 필요하다면 설명 해주세요.
디펜던시 변경이 있나요?
어떻게 테스트 하셨나요?
코드의 실행결과를 볼 수 있는 로그가 있다면 첨부해주세요.

p3: 마이너 버전 올려도 괜찮을 것 같어요

저도 그게 좋을것 같습니다. description 에 내용보충이 있으니 확인 부탁드립니다 :-)

확인했습니다
전사 강제 항목이긴 한데 백오피스만 해도 고칠게 한 가득이더라구요

authors: elegantcoder soomtong


text:
Regression: DeviceIoControl failed from libusb_open immediately after libusb_close on Windows 10

Our users started to get this error on Windows:
libusb_open returned: -99
First-chance exception at 0x00007FFDB1743C58 in Protonect.exe: Microsoft C++ exception: UsbDkDriverFileException at memory location 0x00000095014FDA30. Driver file operation error. DeviceIoControl failed (The system cannot find the file specified. Error code = 2)
First-chance exception at 0x00007FFEC5F83C58 in Protonect.exe: Microsoft C++ exception: UsbDkDriverFileException at memory location 0x000000900C59D3F0. Driver file operation error. DeviceIoControl failed (The system cannot find the file specified. Error code = 2)
https://github.com/OpenKinect/libfreenect2/issues/812#issuecomment-293617474
The error can be reproduced by libusb_open() immediately after libusb_close() and would go away if there is some time delay between the two calls.
The error is only reproduced with v1.0.17 and not v1.0.16.

This is by design. On libusb_close() UsbDk initiates device reset and client have to wait until reset completion in order to open it again.
Commit 848bb79ac2379c1ce0681088320f4ae504f2bce3 introduced between versions 1.0.16 and 1.0.17 increased amount of time client application has to wait for device re-appearance, therefore you hit this issue more frequently.
In order to resolve the issue properly, you should either wait for hotplug event related to the device in question or wait for the device in question to re-appear in libusb device list. After that, libusb_open() will work as expected.

libusb API provides:

Devices are reference counted with libusb_ref_device() and libusb_unref_device(), and are freed when the reference count reaches 0.

The bug happens when the reference count is 1.
libusb_get_device_descriptor, ref=1
libusb_open, ref=2
libusb_close, ref=1
libusb_open <-- bug

Presumably the device should not even be reset when  libusb_get_device_descriptor() still holds onto it.

UsbDk resets physical USB device on close, this does not affect libusb device structure, it remains valid.

authors: dmitryfleytman xlz

repo: daynix/UsbDk


text:
Feature: down all migrations one command and allow use db-migrate as module

I have many migrations that represents all my database, when I run tests I use the migrations to recreate a test database in a known state usualy I pass something like
var exec = require('child_process').exec;
var Promise = require('bluebird');

function execAsync(cmd) {
  return new Promise(function(resolve, reject) {
    exec(cmd, function(err, stdout, stderr) {
      if (err) { reject(err); } else { resolve({stdout: stdout, stderr: stderr}); }
    });
  }
}
module.exports = {
  down: function() {
    return execAsync('db-migrate down -c 100');
  },
  up: function(down) {
     return Promise.resolve()
       .then(function() {
         if (down) { return this.down() }  else {
           return Promise.resolve();
         }
       }.bind(this))
       .then(function() {
          return execAsync('db-migrate up');
       });
  }
}

would be great if I can pass something like --all to the down command and if I can use db-migrate as module
  var migrate = require('db-migrate');

  ...
  return migrate.down({all: true}).then(function() { return migrate.up(); });

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/23545036-feature-down-all-migrations-one-command-and-allow-use-db-migrate-as-module?utm_campaign=plugin&utm_content=tracker%2F73887&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F73887&utm_medium=issues&utm_source=github).

@rkmax I'm working on this :)
There is a reset method, that already does what you search for and the db-migrate as a module with a stable programable api is also part of the v0.10.x version.
I need to finish and test the last details of the seeder and of the internal structure of db-migrate (restructuring the driver architecture, promisifying and so on...), after this I will release a pre release of 0.10.x, while I will finish the docu and the missing tests.
Has worked for a while on this 0.10.x for now and I changed a lot of things, but I think we will be finally there shortly!

thanks @wzrdtales for all your hard work

refers to #214
Notes: reset commands already fullfills the described need.

authors: rkmax wzrdtales

repo: db-migrate/node-db-migrate


text:
In some cases, boolean column is not detected as column (Ctrl + mouse click doesn't work)

Description

use the code below
press and hold Ctrl, mouse hover over b1 column
it is expected that is is underlined (so that mouse click opens it) but it doesn't take place

create table tab (
    a1 int,
    b1 boolean
);

insert into tab (a1, b1)
values (1, true), (2, false);

select *
from tab
where a1 > 0 and b1 = true   --work
;
select *
from tab
where a1 > 0 and b1 is true  --doesn't work
;
select *
from tab
where a1 > 0 and b1 is not true  --doesn't work
;
select *
from tab
where a1 > 0 and b1  --doesn't work
;
select *
from tab
where a1 > 0 and not b1  --doesn't work
;

https://github.com/user-attachments/assets/450cdaad-52ef-4601-9d7e-480b687011a3
DBeaver Version
Enterprise Edition 24.2.3-202410032322
Operating System
Windows 10
Database and driver
PostgreSQL 16
PostgreSQL JDBC Driver 42.7.4
Steps to reproduce
No response
Additional context
No response

I don't remember exactly, but seems to be, along with postgres, it is also applicable to MySQL, SQLite and oracle, maybe some other database engines.

Thanks for the bug report

repo: dbeaver/dbeaver


text:
Column width if flushed after each query execution

After I ran query I change column width:

but after next execution its width is flushed:

Please keep column width

Yes, we should keep columns width on refresh.


text:
Exasol Driver not working

Hi, DBeaver 5.3.0 on Mac or Windows. Currently running Java 1.8.0.45 on Mac.
When connecting to an Exasol Instance, the driver manager opens and tries to download some libraries.
Dialog "Driver Settings": Can't resolve libraries
When clicking on Download:
Maven artifact 'maven:/com.exasol:exasol-jdbc:RELEASE' not found
Am I missing something or what's the reason for that? It stopped working before 5.3.0 i think.
Thank you

Are you behind some corporate firewall/proxy?
DBeaver downloads driver's jars from remote servers. You can configure proxy settings in DBeaver preferences or download Exasol driver jars manually and then add them in the driver using driver manager.

Hi, I'm currently not behind a proxy and there is no proxy configuration present. The Exasol JDBC-Driver vor Mac is an Installer currently. I didn't find any jar's do download. Is it possible that exasol changed their strategy and doesn't take care of the maven repo anymore?

I uninstalled dbeaver and deleted .dbeaver, .dbeaver-drivers reinstalled and added an exasol connection. here is how it looks like.

PostgreSQL works:

What could be wrong here...
Java version? java version "1.8.0_45"

I still think it is some network issue. Check debug log (https://github.com/dbeaver/dbeaver/wiki/Log-files) for details, there should be some IO errors.
Exasol repository differs from other drivers (like PostgreSQL) which are hosted in Maven Central.
Try to browse it: https://maven.exasol.com/artifactory/exasol-releases to test it.

You're right! It's a network issue. The cert of maven.exasol.com was renewed on August 8th. This is ca. the time when it stopped to work. I know this is not a dbeaver problem but a java problem.
However, any ideas how to handle it for others who run into this issue?
2018-12-21 21:12:09.407 - Open [https://maven.exasol.com/artifactory/exasol-releases/com/exasol/exasol-jdbc/maven-metadata.xml]
2018-12-21 21:12:09.543 - Open [https://maven.exasol.com/artifactory/exasol-releases/com/exasol/exasol-jdbc/]
2018-12-21 21:12:09.651 - Maven artifact 'com.exasol:exasol-jdbc:RELEASE' not found in repository 'https://maven.exasol.com/artifactory/exasol-releases/': sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
2018-12-21 21:12:09.656 - Open [file:/Applications/DBeaver.app/Contents/Eclipse/maven-local/com/exasol/exasol-jdbc/maven-metadata.xml]
2018-12-21 21:12:09.659 - Open [file:/Applications/DBeaver.app/Contents/Eclipse/maven-local/com/exasol/exasol-jdbc/]

I believe it is because we bundle a bit outdated JRE in DBeaver Windows installer. I'll update it in the next version, thanks for checking this.
Workaround: install Java 9 or Java 10 manually and use it instead of JRE shipped along with DBeaver.
BTW we are going to migrate to JRE 10 or JRE 11 before DBeaver 6.0 release.

authors: dizdi serge-rider


text:
Unsupported operation code: 97

System information:

Operating system (distribution) and version: Windows 10
DBeaver version- 6.1.5

Connection specification:

Firebird 3.0

Describe your question:
I have an application that connects to the Firebird database, I use an encryption plugin from the site below and I have not problems.
https://ib-aid.com/en/firebird-encryption-plugin-framework/
When I try to connect using DBBeaver I get  Unsupported operation code: 97

Receiving this error would mean that you're using Jaybird 3.0.3 or earlier. Upgrade to Jaybird 3.0.4 or higher (at this time, 3.0.8 is the latest available version).

So pressing the "Download/update" does not work or upgrade to jaybird 4.0.0
https://www.firebirdsql.org/en/jdbc-driver/

@FIFernandes That doesn't seem directly related to this issue. However, it could have to do with #8294.

authors: FIFernandes mrotteveel


text:
Add missing containerId prop to all MiradorMenuButton instances

This had the effect of making button popovers disappear in fullsscreen mode, since the popovers would be  rendered as a child of the body element, which was not in fullscreen scope.

Codecov Report

Merging #40 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #40   +/-   ##
=======================================
  Coverage   86.88%   86.88%
=======================================
  Files          15       15
  Lines         686      686
  Branches      217      217
=======================================
  Hits          596      596
  Misses         85       85
  Partials        5        5

Impacted Files
Coverage Δ

src/components/settings/ColorWidget.js
93.33% <ø> (ø)

src/components/settings/OverlaySettings.js
98.46% <ø> (ø)

src/index.js
0.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7899639...6347c5a. Read the comment docs.

authors: codecov-commenter jbaiter

repo: dbmdz/mirador-textoverlay


text:
Yolact++ to onnx？

Has anyone successfully converted a Yolact++ model to onnx?
    preds = net(batch)     dummy_input = Variable(torch.randn(1, 3, 550, 550))     torch.onnx.export(net, dummy_input, "yolact_0421_20000.onnx", verbose=True)
but got the errors, any suggestions would be appreciated.
RuntimeError: Tried to trace <__torch__.yolact.FPN object at 0x9c26df60> but it is not part of the active trace. Modules that are called during a trace must be registered as submodules of the thing being traced.

you need change the
class FPN(ScriptModuleWrapper):

to
class FPN(nn.Module):

this doesn't work for me...did it actually work for u?

您的邮件已收到 谢谢

Any update on this? @zhoujinhai

Any update on this? @zhoujinhai

Yes, It worked for me!

how to do this?

@MiaoRain make use_jit = false in Yolact.py. and  'no_jit': True in data/config.py
Then you will have to make effort to export DCN_v2. For that you can see my comment here

you need to set use_jit=False in yolact.py:
# As of March 10, 2019, Pytorch DataParallel still doesn't support JIT Script Modules use_jit = torch.cuda.device_count() <= 1 use_jit = False  # add this line if u get RuntimeError if not use_jit:     print('Multiple GPUs detected! Turning off JIT.')

you need to set use_jit=False in yolact.py:
use_jit = torch.cuda.device_count() <= 1
use_jit = False  # add this line if u get RuntimeError
if not use_jit:
print('Multiple GPUs detected! Turning off JIT.')

you need to set use_jit=False in yolact.py: use_jit = torch.cuda.device_count() <= 1 use_jit = False # add this line if u get RuntimeError if not use_jit: print('Multiple GPUs detected! Turning off JIT.')

you need to set use_jit=False in yolact.py:
'''use_jit = torch.cuda.device_count() <= 1
use_jit = False # add this line if u get RuntimeError
if not use_jit:
print('Multiple GPUs detected! Turning off JIT.')'''

you need to set use_jit=False in yolact.py:
‘’‘use_jit = torch.cuda.device_count() <= 1
use_jit = False # add this line if u get RuntimeError
if not use_jit:
print('Multiple GPUs detected! Turning off JIT.')’‘’

you need to set use_jit = False in yolact.py:
use_jit = torch.cuda.device_count() <= 1
use_jit = False  # add this line if u get RuntimeError
if not use_jit:
    print('Multiple GPUs detected! Turning off JIT.')

authors: LiaoChengkk MiaoRain abhiagwl4262 avickars sarmientoj24 zhoujinhai

repo: dbolya/yolact


text:
Toggles for operations

Added toggles for operations with Multiplication checked by default

Resolving: https://github.com/dbrentley/beltmatic-solver/issues/2

Visual:

repo: dbrentley/beltmatic-solver


text:
support variable arguments, for subcommands like "tldr git checkout"

tldr tldr shows tldr git checkout as an option:
➜  components git:(develop) ✗ tldr tldr

  Simplified man pages.

  Get typical usages of a command (hint: this is how you got here!):

      tldr command

  Show the tar tldr page for linux:

      tldr -p linux tar

  Get help for a git subcommand:

      tldr git checkout

➜  components git:(develop) ✗

tldr git checkout doesn't work:
➜  components git:(develop) ✗ tldr tealdeer
Page tealdeer not found in cache
Try updating with `tldr --update`, or submit a pull request to:
https://github.com/tldr-pages/tldr
➜  components git:(develop) ✗

Poking around, tldr git-checkout works:
➜  components git:(develop) ✗ tldr git-checkout

  Checkout a branch or paths to the working tree.
  Homepage: <https://git-scm.com/docs/git-checkout>.

  Create and switch to a new branch:

      git checkout -b branch_name

  Create and switch to a new branch based on a specific reference (branch, remote/branch, tag are examples of valid references):

      git checkout -b branch_name reference

  Switch to an existing local branch:

      git checkout branch_name

  Switch to an existing remote branch:

      git checkout --track remote_name/branch_name

  Discard all unstaged changes in the current directory (see `git reset` for more undo-like commands):

      git checkout .

  Discard unstaged changes to a given file:

      git checkout file_name

  Replace a file in the current directory with the version of it committed in a given branch:

      git checkout branch_name -- file_name

➜  components git:(develop) ✗

I think that perhaps if there are variable arguments they get joined together with a dash. It would be nice if everything tldr tldr worked with tealdeer :)

This was implemented a while ago in #75 (tealdeer 1.2.0).

This was implemented a while ago in #75 (tealdeer 1.2.0).

authors: benatkin dbrgn

repo: dbrgn/tealdeer


text:
Null pointer dereference at cli/wvunpack.c

Hi, I found a null pointer dereference at cli/wvunpack.c:911
Here's ASAN log:
AddressSanitizer:DEADLYSIGNAL
==84257==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x561b47a970c6 bp 0x7fff13952fb0 sp 0x7fff1394fca0 T0)
==84257==The signal is caused by a WRITE memory access.
==84257==Hint: address points to the zero page.
#0 0x561b47a970c5 in main cli/wvunpack.c:911
#1 0x7efc4f5c0082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082)
#2 0x561b47a945ed in _start (/usr/local/bin/wvunpack+0xa5ed)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV cli/wvunpack.c:911 in main
==84257==ABORTING
Steps to Reproduce
./configure --disable-shared CFLAGS="-fsanitize=address -ggdb" CXXFLAGS="-fsanitize=address -ggdb"
make & make install
/usr/local/bin/wvunpack -m poc.wv -o /
poc.ZIP

Hi, and thanks for reporting this. It's quite a catch!
I have pushed a fix for it. Please let me know if you run into any more.
BTW, I don't consider this to be particularly worrisome from a security standpoint. It requires the command-line program (which are not standard in any repo) and it requires both a crafted WavPack file and a crafted command line. Further, all it can cause is an exception (i.e., no code injection).
Nevertheless, I'm glad I was able to fix it before my imminent release!

authors: dbry xiaoxiaoafeifei

repo: dbry/WavPack


text:
[Maintainership] GitHub Action to switch between the awaiting_response and triage labels

Housekeeping

[X] I am a maintainer of dbt-adapters

Short description
GitHub Action to switch between the awaiting_response and triage labels
Acceptance criteria

When a new issue is opened, it automatically has the triage label
When an issue has the awaiting_response label and a user adds a comment, the awaiting_response label is removed and the triage label is added.

Suggested Tests
N/A
Impact to Other Teams
None.
Will backports be required?
No.
Context
No response

I copied the existing workflow from dbt-snowflake and updated the name to be less prescriptive about the implementation. We use a single label today, but perhaps this changes in the future. And updating action names doesn't always work as intended.
As for the first bullet (When a new issue is opened, it automatically has the triage label), this occurs via the issue templates for bugs, regressions, and feature requests. It will not happen for internally submitted tickets, though I suspect this is what we want.

Oh, and I don't think user docs is correct, is it? That's the label we use to create a docs issue on docs.getdbt.com. Do we need something related to this in that repo?

repo: dbt-labs/dbt-adapters


text:
[Bug] API Relation Table Used in SQL Header Not Recognized as Parent of Model

Is this a new bug in dbt-core?

[X] I believe this is a new bug in dbt-core
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
In one of my DBT models my_downstream_model, I am defining a table reference using api.relation.create like this:
{% set my_table = api.Relation.create(
    database=<my_database>,
    schema=<my_schema>,
    identifier=<my_identifier>
) -%}

This table is then used to leverage the declare variables functionality of BigQuery within a SQL header. I am doing this since the main table used in the model is a hive partitioned external table, so declaring the partitions as a variable greatly reduces the query cost compared to having the partitions be defined by the result of another query.
The SQL header call looks something like this:
{% call set_sql_header(config) %}
    DECLARE my_partition_list ARRAY<STRING> DEFAULT(ARRAY(select partition_field from {{ my_table }}));
{%- endcall %}

my_table is not used anywhere else in the main model query, and it is not recognized as a parent of my_downstream_model.
Is this the intended behavior? I also tried to set it as a dependency manually, doing -- depends_on: {{ my_table }} but that didn't get it to be recognized as a parent either.
Expected Behavior
My expectation was that my_table would be recognized as a parent of my_downstream_model, so that the DAG execution ordering would be correct.
Steps To Reproduce
Create a DBT SQL model with my_table defined as above, and use it in a SQL header call as I defined. I ran dbt ls --select=my_table+ and my_downstream_model was not in its list of dependencies
Relevant log output
No response
Environment
- OS: macOS 14.5 and Debian 11.4
- Python: 3.10.8
- dbt: 1.8.9

Which database adapter are you using with dbt?
bigquery
Additional Context
No response

Thanks for reaching out @aboomer07 !
What you described sounds like the expected behavior to me. This is because dbt will only recognize dependencies that come from a call to the ref or source functions. The dbt DAG does not support dynamic graph building -- all the dependencies must be specified up-front.
Is there a reason why you can't create a source and then utilize it like the following?

First, create a source with your database, schema, and identifier:

models/_sources.yml
sources:
  - name: <my_source_name>  # often same as <my_schema>, but you can name the source whatever you want
    database: <my_database>
    schema: <my_schema>
    tables:
      - name: <my_identifier>

Then, utilize it within a dbt model:

models/my_downstream_model.sql
-- ...

-- depends_on: {{ source("<my_source_name>", "<my_identifier>") }}

-- ...

I didn't try one way or another with placing the source reference within the sql_header, but the at the very least the -- depends_on:  approach should work.

Thanks for getting back!
I now see that wasn't quite enough information on my part, but in general that makes sense what you're saying about dynamic graph building.
So I have a couple more things going on here, I have both a DBT project (<my_project>) as well as a private external (to the project) package (<my_package>) I built that I am using in that project. <my_package> has some generalized DBT macros and models that can be used across multiple projects.
So <my_table> is a model within <my_project> and gets referenced with {{ ref }} in other parts of the project.
<my_downstream_table> is actually a generic model within <my_package>. The way the models within <my_package> are built is that the project using <my_package> gets to supply a variable in the project specifying the name of the tables required by <my_downstream_table>. The goal of this was to set up a package where the user would only need to supply two tables specific to their project, and then the package would be able to generate results from those.
So following the DAG I have <my_table> which is a model defined within and created by <my_project>. <my_downstream_table> is a generic model defined within <my_package>. <my_project> uses <my_downstream_table> and creates a realization of it dependent on <my_table>.
<my_downstream_table> is actually dependent on a few other tables either created / with their sources defined within <my_project> that are referenced within the main query, and the dependency graphing of those works fine!
Then <my_project> has some other tables that are dependent on <my_downstream_table>.
Hopefully this makes sense? I found a workaround for now, and if this isn't a good pattern to pursue then I'm fine with closing this issue!

Thank you for that additional context @aboomer07 💡

<my_downstream_table> is actually a generic model within <my_package>. The way the models within <my_package> are built is that the project using <my_package> gets to supply a variable in the project specifying the name of the tables required by <my_downstream_table>. The goal of this was to set up a package where the user would only need to supply the table names specific to their project (and containing the correct fields), and then the package would be able to generate results from those.

This would be a feature request rather than a bug report, so I'm going to update the title accordingly. The ability to pass in a list of table dependencies is not something we'd be interested in adopting especially since you have a workaround so I'm going to close this as "not planned".

repo: dbt-labs/dbt-core


text:
stage generation ignores value of FIELD_OPTIONALLY_ENCLOSED_BY

Describe the bug
DBT ignores the value of the optional parameter field_optionally_enclosed_by.
Steps to reproduce
          +materialized: from_external_stage
          +stage_file_format: 'file_format_csv'
          +field_del: ','
          +field_optionally_enclosed_by: '"'

Expected results
create or replace file format  LZ_CATALOGUE_PRODUCTSUP_FEED.file_format_csv
       type = 'CSV'
        RECORD_DELIMITER = '\n'
        FIELD_DELIMITER = ','
        FIELD_OPTIONALLY_ENCLOSED_BY='"'
        SKIP_HEADER = 1
        NULL_IF = ('null','')
        ESCAPE_UNENCLOSED_FIELD = '\r'
        EMPTY_FIELD_AS_NULL=true
        COMPRESSION=GZIP
        ENCODING =  'ISO 8859-1';

Actual results
create or replace file format  LZ_CATALOGUE_PRODUCTSUP_FEED.file_format_csv
       type = 'CSV'
        RECORD_DELIMITER = '\n'
        FIELD_DELIMITER = ','
        FIELD_OPTIONALLY_ENCLOSED_BY=none
        SKIP_HEADER = 1
        NULL_IF = ('null','')
        ESCAPE_UNENCLOSED_FIELD = '\r'
        EMPTY_FIELD_AS_NULL=true
        COMPRESSION=GZIP
        ENCODING =  'ISO 8859-1';

Screenshots and log output

System information
packages:
  - package: dbt-labs/dbt_utils
    version: 0.8.0
  - package: dbt-labs/dbt_external_tables
    version: 0.8.0

Which database are you using dbt with?

[ ] redshift
[X] snowflake
[ ] other (specify: ____________)

The output of dbt --version:
    version: 1.0.0

The operating system you're using:
The output of python --version:
Additional context

different problem not inside the makro

repo: dbt-labs/dbt-external-tables


text:
to_json crash on MIRToStakeCredentials

Any transaction that contains a MIR certificate crashes when calling to_json. This may be caused by our good old friend linked_hash_map
Crash log:
Err` value: JsError { msg: "to_json: key must be a string" }

Steps to reproduce
#[test]
    fn foo_bar() {
        let mut cred = MIRToStakeCredentials::new();
        cred.insert(
        &StakeCredential::from_keyhash(
            &Ed25519KeyHash::from_bytes(
                hex::decode("b861eeadde300385d88aaa98cad0f0ed1f95419bbb9971a0fb7c96fb").unwrap()
            ).unwrap()
        ),
        &Int::from_str("5").unwrap()
        );
        println!("{}", cred.to_json().unwrap());
    }

I believe the root cause is a missing implementation of serde::Serialize such as
impl serde::Serialize for TransactionMetadatum {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where S: serde::Serializer {
        let json_str = decode_metadatum_to_json_str(self, MetadataJsonSchema::DetailedSchema)
            .map_err(|e| serde::ser::Error::custom(&format!("{:?}", e)))?;
        serializer.serialize_str(&json_str)
    }
}

repo: dcSpark/cardano-multiplatform-lib


text:
Fetch deployments from Marathon

This PR is dependent on functionality introduced in #135
This PR introduces functionality to the MarathonStore. It now fetches data from the Marathon deployments endpoint (ref the docs) and holds it on the store, inline with similar functionality which fetches services data from Marathon.

Reviewer: @mlunoe

I addressed all feedback and rebased downstream branches.

authors: mesosphere-web philipnrmn

repo: dcos/dcos-ui


text:
DCOS-13386: Add Resume option for suspended services

This PR adds a "Resume" option for suspended services (meaning their instance count is 0).
When the modal appears, the user can choose how many instances they'd like to resume to. If the app label MARATHON_SINGLE_INSTANCE_APP is present, we do not present the instance count textbox, and instead scale the service back to a single instance.

Checklist

[x] Did you add a JIRA issue in a commit message or as part of the branch name?
[ ] Did you add new unit tests?
[x] Did you add new integration tests?
[ ] If this is a regression, did you write a test to catch this in the future?

@jfurrow could you please add an integration test!

@rcorral Yes you're right, tests incoming!

retest this please

ping @wavesoft I believe all of your comments have been addressed, could you please review again? Thank you!

Approved 😎

Thanks @MatApple, it should be good to go now!

Ping @MatApple

Looks great, works great! 💯  For some reason certain frameworks cannot be suspended which requires more investigation (e.g. Cassandra)

authors: MatApple jfurrow rcorral wavesoft


text:
Add helper text for configuring docker volumes and ports

Reviewer: @philipnrmn

did the text need to be links where once you clicked them, it switched tabs?

@kennyt they are links in Marathon but I'm going to go for simple here. Its a lot more work to make these linkable. Lets test the reaction

authors: MatApple kennyt mesosphere-web


text:
🛑 Danbs is down

In fbdd642, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 170 ms

Resolved: Danbs is back up in 3522d43 after 334 days, 1 hour, 5 minutes.

repo: ddanbs/upptime


text:
🛑 Danbs is down

In 104612d, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 152 ms

Resolved: Danbs is back up in 429c11d after 6 minutes.


text:
🛑 Danbs is down

In 4a8eb4a, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 139 ms

Resolved: Danbs is back up in d2f6d95 after 20 minutes.


text:
🛑 Danbs is down

In 4c65ab1, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 3214 ms

Resolved: Danbs is back up in ba61f64.


text:
🛑 Danbs is down

In 9f94a7f, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 2830 ms

Resolved: Danbs is back up in 998e9b9.


text:
Doesn't work on new GitHub Design

I face this problem too. GitHub new design is now released for everyone.
Can't back to old design anymore.

Hi,
thank you for creating an issue. There's possible fix for this issue #23

@dderevjanik Great!

authors: dderevjanik gluons kyle-seongwoo-jun

repo: dderevjanik/github-vscode-icons


text:
2023 review: Add more about contributor training

The Issue
@bmartinez287 pointed out the importance of contributor training in 2023.

It be great to hear more about the contributions Simon made. Maybe a highlight of the biggest items specially since we are mentioning Matt.

It would be great to hear more about the contributions Simon made

I could write a whole blog post just about that. I think it would be a distraction in the midst of this post though. I wish he were here so we could honor him with a post like that.


text:
ddev does not respect connections to external networks anymore

Is there an existing issue for this?

[X] I have searched the existing issues

Output of ddev debug test
Expand `ddev debug test` diagnostic information
Running bash [-c /tmp/test_ddev.sh]
======= Existing project config =========
These config files were loaded for project 726-feature-84358-testomize-php8-2: [/home/gitlab-runner/builds/mysysmex-mobifi/feature-84358-testomize-php8-2/.ddev/config.yaml /home/gitlab-runner/builds/mysysmex-mobifi/feature-84358-testomize-php8-2/.ddev/config.override.yaml]
name: 726-feature-84358-testomize-php8-2
type: typo3
docroot: public
php_version: 8.2
webserver_type: apache-fpm
webimage: ddev/ddev-webserver:v1.22.6
router_http_port: 8080
router_https_port: 4430
additional_fqdns: [www.mysysmex.vmdev testing.mysysmex.vmdev]
database: {mariadb 10.2}
hooks: map[pre-start:[map[exec-host:docker network inspect mysysmex >/dev/null 2>&1 || docker network create --driver bridge mysysmex]]]
project_tld: ddev.site
use_dns_when_possible: true
timezone: Europe/Berlin
composer_version: 2
nodejs_version: 16
default_container_timeout: 120
======= Creating dummy project named  tryddevproject-18316 in ../tryddevproject-18316 =========
OS Information: Linux ddev-runner-mysysmexmobifi 6.2.0-39-generic #40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
User information: uid=1001(gitlab-runner) gid=1004(gitlab-runner) groups=1004(gitlab-runner),1002(docker)
DDEV version:  ITEM             VALUE
 DDEV version     v1.22.6
 architecture     amd64
 db               ddev/ddev-dbserver-mariadb-10.4:v1.22.6
 ddev-ssh-agent   ddev/ddev-ssh-agent:v1.22.6
 docker           24.0.7
 docker-compose   v2.23.3
 docker-platform  linux-docker
 mutagen          0.17.2
 os               linux
 router           ddev/ddev-traefik-router:v1.22.6
 web              ddev/ddev-webserver:v1.22.6
PROXY settings: HTTP_PROXY='' HTTPS_PROXY='' http_proxy='' NO_PROXY=''
======= DDEV global info =========
Global configuration:
instrumentation-opt-in=false
omit-containers=[]
performance-mode=none
router-bind-all-interfaces=false
internet-detection-timeout-ms=3000
disable-http2=false
use-letsencrypt=false
letsencrypt-email=
table-style=default
simple-formatting=false
use-hardened-images=false
fail-on-hook-fail=false
required-docker-compose-version=v2.23.3
use-docker-compose-from-path=false
project-tld=
xdebug-ide-location=
no-bind-mounts=false
router=traefik
wsl2-no-windows-hosts-mgt=false
router-http-port=80
router-https-port=443
mailpit-http-port=8025
mailpit-https-port=8026
traefik-monitor-port=10999

======= DOCKER info =========
docker location: -rwxr-xr-x 1 root root 35939040 Oct 26 11:07 /usr/bin/docker
docker version:
Client: Docker Engine - Community
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.20.10
 Git commit:        afdd53b
 Built:             Thu Oct 26 09:07:41 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.10
  Git commit:       311b9ff
  Built:            Thu Oct 26 09:07:41 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.27
  GitCommit:        a1496014c916f9e62104b33d1bb5bd03b0858e59
 runc:
  Version:          1.1.11
  GitCommit:        v1.1.11-0-g4bccb38
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
DOCKER_DEFAULT_PLATFORM=notset
======= Mutagen Info =========
======= Docker Info =========
Docker platform: linux-docker
Using Docker context: default (unix:///var/run/docker.sock)
docker-compose: v2.23.3
Using DOCKER_HOST=unix:///var/run/docker.sock
Docker version: 24.0.7
Able to run simple container that mounts a volume.
Able to use internet inside container.
Docker disk space:
Filesystem                Size      Used Available Use% Mounted on
overlay                  18.7G     10.6G      7.3G  59% /

The ddev-ssh-agent container has been removed. When you start it again you will have to use 'ddev auth ssh' to provide key authentication again.
Network ddev_default removed
Existing docker containers:
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
Network ddev_default created
Starting tryddevproject-18316...
Network ddev-tryddevproject-18316_default created
 Container ddev-ssh-agent  Created
 Container ddev-ssh-agent  Started
ssh-agent container is running: If you want to add authentication to the ssh-agent container, run 'ddev auth ssh' to enable your keys.
Building project images...
Project images built in 0s.
 Container ddev-tryddevproject-18316-web  Created
 Container ddev-tryddevproject-18316-db  Created
 Container ddev-tryddevproject-18316-web  Started
 Container ddev-tryddevproject-18316-db  Started
Waiting for web/db containers to become ready: [web db]
Starting ddev-router if necessary...
 Container ddev-router  Created
 Container ddev-router  Started
Waiting for additional project containers to become ready...
All project containers are now ready.
Successfully started tryddevproject-18316
Project can be reached at https://tryddevproject-18316.ddev.site https://127.0.0.1:32776
======== Curl of site from inside container:
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 17 Jan 2024 17:22:19 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding

======== curl -I of http://tryddevproject-18316.ddev.site from outside:
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Date: Wed, 17 Jan 2024 17:22:19 GMT
Server: nginx
Vary: Accept-Encoding

======== full curl of http://tryddevproject-18316.ddev.site from outside:
Success accessing database... db via TCP/IP
ddev is working. You will want to delete this project with 'ddev delete -Oy tryddevproject-18316'
======== Project ownership on host:
drwxr-xr-x 4 gitlab-runner gitlab-runner 4096 Jan 17 18:21 ../tryddevproject-18316
======== Project ownership in container:
drwxr-xr-x 4 gitlab-runner gitlab-runner 4096 Jan 17 17:21 /var/www/html
======== In-container filesystem:
Filesystem     Type 1K-blocks     Used Available Use% Mounted on
/dev/sda1      ext4  19610420 11282144   7484256  61% /var/www/html
======== curl again of tryddevproject-18316 from host:
Success accessing database... db via TCP/IP
ddev is working. You will want to delete this project with 'ddev delete -Oy tryddevproject-18316'
Thanks for running the diagnostic. It was successful.
Please provide the output of this script in a new gist at gist.github.com
Running ddev launch in 5 seconds
/usr/bin/xdg-open: 882: www-browser: not found
/usr/bin/xdg-open: 882: links2: not found
/usr/bin/xdg-open: 882: elinks: not found
/usr/bin/xdg-open: 882: links: not found
/usr/bin/xdg-open: 882: lynx: not found
/usr/bin/xdg-open: 882: w3m: not found
xdg-open: no method available for opening 'https://tryddevproject-18316.ddev.site'
Failed to run launch ; error=exit status 3

Please run cleanup after debugging with 'ddev debug testcleanup'
Failed running test_ddev.sh: exit status 1
. You can run it manually with `curl -sL -O https://raw.githubusercontent.com/ddev/ddev/master/cmd/ddev/cmd/scripts/test_ddev.sh && bash test_ddev.sh`

Expected Behavior
We added an extended docker-compose network configuration in .ddev/docker-compose.testomize-frontproxy.yaml to add ddev container to an external traefik frontproxy which uses the docker network frontproxy:
version: '3.6'

services:

  web:
    networks:
      - default
      - frontproxy
    labels:
      - "traefik.enable=true"

      - "traefik.http.routers.router-726-feature-84358-testomize-php8-2-1.rule=Host(`feature-84358-testomize-php8-2.review.example.com`)"
      - "traefik.http.routers.router-726-feature-84358-testomize-php8-2-1.entrypoints=websecure"
      - "traefik.http.routers.router-726-feature-84358-testomize-php8-2-1.tls.certresolver=basic"
      - "traefik.http.routers.router-726-feature-84358-testomize-php8-2-1.tls.domains[0].main=review.example.com"
      - "traefik.http.routers.router-726-feature-84358-testomize-php8-2-1.tls.domains[0].sans=*.review.example.com"
      - "traefik.http.routers.router-726-feature-84358-testomize-php8-2-1.middlewares=auth-726-feature-84358-testomize-php8-2-1"
      - "traefik.http.routers.router-726-feature-84358-testomize-php8-2-1.service=service-726-feature-84358-testomize-php8-2-1"
      - "traefik.http.services.service-726-feature-84358-testomize-php8-2-1.loadbalancer.server.port=80"
networks:
    frontproxy:
        name: frontproxy
        external: true

After starting the ddev project all labels are added to the docker container but it is not added to the external network frontproxy as in earlier versions of ddev.
docker inspect ddev-726-feature-84358-testomize-php8-2-web shows only the default ddev networks ddev-726-feature-84358-testomize-php8-2_default and ddev_default but NOT frontproxy.
This results in a gateway timeout error of the external front proxy because it is not able to connect to the ddev webserver container.
I would expect that the ddev container is added to the external docker network frontproxy.
Actual Behavior
The ddev containers are not connected to the external networks anymore (it was working in older versions).
Steps To Reproduce
Add a docker network configuration with connection to an external docker network in .ddev/docker-compose-something.yaml, start the setup with ddev start and inspect the ddev web container with docker inspect CONTAINERNAME. Watch out for the networks section and if the external network is listed there.
Anything else?
No response

Hi @galoppi,
Thank you for the report.
The change that caused this problem:

#5533

I'll think about what I can do here.

Can you just...

run your external traefik in the ddev_default network, so it doesn't have to connect to another one?
Add configuration to DDEV's traefik to do whatever it is you want to do?

It's unlikely that we would fiddle with DDEV's setup to solve your unusual situation, which probably has lots of other possible solutions. We can talk about other solutions if you like. Yours sounds complex.

Thanks a lot for your help and support!
Indeed we have a complex setup: We create an instant ddev setup during our CI pipeline for each merge request on a cloud server with an "offical" dynamic domain name and a letsencrypt certificate. The hostnames are dynamic and created during the CI process. Therefore we add dynamic configuration, i.e. .ddev/docker-compose.testomize.yaml containing all the information for the dynamic hostnames as labels for traefik.
I'll give it a try using the ddev_default network.
Do you have an example for using the build in traefik in combination with letsencrypt?

We haven't explored Let's Encrypt with traefik yet, see https://ddev.readthedocs.io/en/latest/users/topics/hosting/
But the docs link to Traefik's technique, https://doc.traefik.io/traefik/https/acme/ - your exploration would be welcome!
My own hosting situation still uses the nginx-proxy router, which still works fine with Let's Encrypt.

It sounds like you already must be an expert with LE and Traefik so would love to have you explore it. https://ddev.readthedocs.io/en/latest/users/extend/traefik-router/#traefik-configuration

No response, closing, happy to help you explore traefik, and especially happy to see you explore LetEncrypt there.

Do you have an example for using the build in traefik in combination with letsencrypt?

It is close to working, was supposed to work in v1.23.4 but I broke it in another fix. It's now easy to get going, but will be easier in v1.23.5.
See

https://github.com/ddev/ddev/pull/6494

@galoppi if you respond in

https://github.com/ddev/ddev/pull/6501

and take a look at that PR it may have a chance to go in. Otherwise, that PR may not have a future.

authors: galoppi rfay stasadev

repo: ddev/ddev


text:
Same oauth_nonce, add a way to seed the getNonce function

Hi, my team is currently experiencing an unexpected behavior when bots try to index our page, this is because some bots always generate the same value when you call:
Math.random();
So when this library tries to generate an oauth_nonce It will always be the same because this line use the number generated by Math.random
       result += word_characters[parseInt(Math.random() * word_characters.length, 10)];
Maybe we should add a way to seed the randomness of this function? What do you think?

i wonder how Math.random() can return same result? did you modify Math.random?

No, that's because when the code run on the Google's servers (To be indexed in Google search), Math.random() is going to be deterministic and It's always going to return the same result, you can refer this for more info:
http://www.tomanthony.co.uk/blog/googlebot-javascript-random/

please test this random method
https://developer.mozilla.org/en-US/docs/Web/API/Crypto/getRandomValues
so i may try this one

Hi, the crypto.getRandomValues didn't work, It also created the same number each time, but I fixed it by adding a timestamp at the end of the oauth_nonce.
Thank you.

ty will try to improve the random function

for some clouds or serverless services Math.random could return the same fyi

for some clouds or serverless services Math.random could return the same fyi

So isn't good idea to use this package for AWS lambdas?

authors: LinsonAlfred MiguelCrespo ddo miguelcrespo


text:
Shelling out breaks various functionality

With the execution model change in baa7b76, a couple issues have arisen in my setup.

ElasticSearch no longer starts up, due the way it spawns processes. Playing with wrapper scripts and shell environments hasn't seemed to help.
Some processes no longer terminate correctly.

OS X Lion
ruby 1.9.3p0
foreman 0.32.0 (Back to 0.31.0 for now.)

I reverted the process management changes in 0.32.0 while I investigate. Re-released as 0.33.0

Can you give example of processes which can't be terminated?

When I opened this issue, the two processes that weren't terminated by Foreman were spawned by the commands postgres and rails server.

It uses setsid.
Probably due to the uninstall/reinstall, the behavior has changed, and ElasticSearch is now starting properly. (Sorry I can't reproduce the exact problem, this could just be ordinary OS X clutter.)
rails server still isn't terminating, but this is a much more minor issue. If there's any more information I could provide that would be helpful, please let me know.

Also, with the latest version of foreman, rails server terminates properly - can't explain that any better than the previous vanishing problem. Nothing more to report here, folks, but thanks for your help!

@bore Latest version works because it reverts to simple exec with its cons. So I want to get to the bottom of this, if we want to have a better alternative (which can be enabled via config flag may be for starters), and I need your help, can you tell me which web server do you use in your rails application? Not webrick, by chance?

OK, sorry about the confusion earlier. I completely spaced that the code substitution was remedying most of the trouble. All the issues I described are still reproducible in 0.32 before the substitution.
I am indeed using webrick as a development server, and it's still hanging on and refusing to be killed. The process appears to be breaking out of the user session, for what it's worth.
Let me know if there's anything I can do to help out.

I see the problem. Webrick has shown an underlying problem, since it ignores the SIGTERM (http://oldwiki.rubyonrails.org/rails/pages/WEBrick) and can be handled gracefully only with SIGINT.
So in current codebase (which uses simple exec) webrick is closed gracefully because TRAP('INT', 'IGNORE') does not work after exec and therefore SIGINT from foreman goes straight to webrick.
In codebase with Process.setsid/setpgrp SIGINT from foreman is not propagated to its children and so webrick is not closed gracefully. In theory foreman should send SIGKILL after waiting for children for 5 seconds, but it did not work because Process.detach borked waiting.
The simple solution is to remove Process.detach, but it won't help in case of forking in tracked processes, because Process.waitall will wait only for intermediate children. So to have an ideal bulletproof solution we need to monitor that all processes belonging to spawned process groups are killed. That will be pretty easy by using sys-proctable gem.
Something along the lines:
 Sys::ProcTable.ps.select {|it| running_processes.keys.include? it.pgid  }

So @ddollar what do you think? I can wrap up a prototype pretty quickly. And to ensure that it will work smoothly we can provide a config variable to manually enable this mod, so people can test it out and if they see problems can report.

I'd be willing to try a prototype. We can release it as a prerelease gem for testing. The sys-proctable gem has C extensions in it which makes it less than ideal (it's harder to package as a .deb, .tgz, etc when there are gem build requirements)

In my ideal world, every Procfile entry would spawn only one process (that itself may spawn processes that I don't care about)
The interface between foreman and each process would be std{in,out,err} and SIGTERM.

authors: bore brainopia ddollar

repo: ddollar/foreman