text:
🛑 Danbs is down

In 2328b6c, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 237 ms

Resolved: Danbs is back up in 79b7429 after 34 minutes.


text:
🛑 Danbs is down

In 238e131, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 245 ms

Resolved: Danbs is back up in 1824b95.

source: gharchive/issue

repo: ddanbs/upptime

license: MIT

license_type: permissive

license_source: github-api


text:
🛑 Danbs is down

In d84e225, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 183 ms

Resolved: Danbs is back up in 10154d1 after 14 minutes.


text:
🛑 Danbs is down

In 023415a, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 127 ms

Resolved: Danbs is back up in a1eec1f.


text:
🛑 Danbs is down

In 1f157f0, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 206 ms

Resolved: Danbs is back up in fc7b1b7.


text:
🛑 Danbs is down

In 18e0b32, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 113 ms

Resolved: Danbs is back up in 6e46c6f.


text:
🛑 Danbs is down

In 0d06b7d, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 182 ms

Resolved: Danbs is back up in cc10f53.


text:
🛑 Danbs is down

In 75209ce, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 193 ms

Resolved: Danbs is back up in ed59e5f.


text:
🛑 Danbs is down

In 39ad0f6, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 196 ms

Resolved: Danbs is back up in 33baee9.


text:
🛑 Danbs is down

In 14a91a0, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 1576 ms

Resolved: Danbs is back up in 2cd3962.


text:
🛑 Danbs is down

In 4dba16f, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 194 ms

Resolved: Danbs is back up in a633da8.


text:
🛑 Danbs is down

In 1554375, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 1238 ms

Resolved: Danbs is back up in fc9e69f.


text:
ADD ddev config should infer/detect PHP version from composer.json

Is there an existing issue for this?

[X] I have searched the existing issues

Is your feature request related to a problem?
I init new ddev projects daily.
going to edit the .ddev/config.yaml to change the actual php version from 8.0 (current global default I assume?) to whatever I actually need gets old very fast
OR
user doesn't init that many ddev projects
forgets that PHP version is not inferred from composer.json
inits a ddev project and wastes a million hours trying to figure out why everything is broken
Describe your solution
ddev config should search composer.json in cascading order for:

require-dev.php  (since we are most likely in dev, since we are using ddev)
require.php
config.platform.php
ddev global default php
current env php version

Describe alternatives
manual labour
Additional context
$ ddev version
 ITEM             VALUE
 DDEV version     v1.21.6
 architecture     amd64
 db               drud/ddev-dbserver-mariadb-10.4:v1.21.5
 dba              phpmyadmin:5
 ddev-ssh-agent   drud/ddev-ssh-agent:v1.21.5
 docker           20.10.17
 docker-compose   v2.15.1
 docker-platform  S4
 mutagen          0.16.0
 os               linux
 router           drud/ddev-router:v1.21.5
 web              drud/ddev-webserver:v1.21.5

Unfortunately, not all projects use composer, and not all composer.json properly suggest the php version, and composer.json is not always in a predictable location.
Why don't you start using ddev config --php-version=<whatever> instead of just ddev config ?

BTW, not sure if this affects you, but most teams check in the .ddev directory, so only the team lead (or someone doing a PR) will change the configuration.

I run lots of random PHP libraries and projects from github, where there is no .ddev directory
the PHP version itself differs all the time

So... ddev config --php-version=what-you-want

I imagine this as an optional param in the config.yaml file, telling DDEV whether it should respect the PHP version defined in composer.json file or not. This may be particularly helpful in teams where not all members are aware of DDEV configuration.

Some frameworks (Eg. Drupal) do not have PHP declaration in their composer.json.
There's movement towards supporting Javascript-based framworks that are not based on PHP (see #4898).
Personally, I configure DDEV once at the start of the project and commit the ./.ddev/config.yml.  People that clone the repo will then get the correct version of PHP when using DDEV.
If they are using something else (Eg. XAMPP), composer reports any PHP conlicts when trying to resolve the versions via composer.lock.

I don't expect DDEV to change PHP version based on composer.json, as all projects don't even have a composer.json, and it may not be in the root.
DDEV has extensive support for composer, and you can change the PHP version any time you want, ddev config --php-version=<version>
Remember that for many projects the composer.json does not even exist at the time of project creation.

I only want ddev to infer PHP version from composer.json when there is a composer.json in its root dir.
..in addition to ddev's other inferring logic, not instead :)

I understand that you want this. Have you considered creating a PR with appropriate (and very significant) tests, or having your organization support DDEV at a substantial level?
DDEV only does static PHP version inference (by project type). Otherwise it uses the default or what's configured.

ok, I'll see if I can do it myself
please reopen and assign to me :)

Hi @allanlaal,
It's a good idea, reopened it.
It reminds me how PhpStorm uses composer.json to determine what PHP features are available or not.

I only want ddev to infer PHP version from composer.json when there is a composer.json in its root dir.

No need to limit this to the project root. We always know the folder used for the composer inside the container, this new check should use the specified directory composer_root.
This feature should also respect disable_settings_management.
And, of course, we need a validation for the parsed PHP version.
Happy that you are interested in this!

Somebody else was asking about a feature like this the other day.

This is a reasonable idea, but it would completely change the traditional semantics of DDEV, so closing. People do fine with php_version

authors: allanlaal mrneatly rfay stasadev tyler36

repo: ddev/ddev


text:
Update scala-library to 2.13.13

About this PR
📦 Updates org.scala-lang:scala-library from 2.13.2 to 2.13.13
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

🔍 Files still referring to the old version number
The following files still refer to the old version number (2.13.2).
You might want to review and update them manually.
.travis.yml

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-lang", artifactId = "scala-library" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.scala-lang", artifactId = "scala-library" }
}]

labels: library-update, early-semver-patch, semver-spec-patch, old-version-remains, commit-count:1

Superseded by #54.

Superseded by #73.

source: gharchive/pull-request

repo: deal-engine/finagle-postgres


text:
DBZ-1241 - allow null values again for commitTime

@gunnarmorling Could you verify if this change doesn't break the DBZ-1082?

@jorkzijlstra Thanks, LGTM!
Could you please just extend io.debezium.connector.postgresql.SourceInfoTest with a test method that will call io.debezium.connector.postgresql.SourceInfo.update(Long, Long, Instant, Long, TableId, Long) with all parameters set to null so it is verified that it works?
Also thinking about it - in io.debezium.connector.postgresql.connection.wal2json.StreamingWal2JsonMessageDecoder.outOfOrderChunk(byte[]) the commitTime should be initilized to current time insted of null.

commitTime should be initilized to current time insted of null.

Interesting, wouldn't this be more confusing? If we don't have the metadata, why pretend we do? It seems the code must generally be prepared to work without having a value for the commit time?

I have added the test setting all null values.
As for the default value setting it to current time. I agree with @gunnarmorling that it would be more confusing.

@gunnarmorling I was working on my company laptop with github enterprise user. If fixed it now and my account has been properly connected. So my name is indeed Jork Zijlstra.

Thanks, @jorkzijlstra! LGTM now; will wait for CI to pass and then merge.

@gunnarmorling Thnx, although its a small change, thnx for all the help in getting it right :)

Applied. Thanks a lot for your contribution, @jorkzijlstra!

authors: gunnarmorling jorkzijlstra jpechane

repo: debezium/debezium

license_source: bigquery


text:
debug.enable() NOT dynamic

I am new to debug logger and I thank you for your awesome utility.
In my experience, using debug.enable('myProj:myModule:myFn') does NOT dynamically impact the logging filters in the current process (in either the browser or node).  It merely resets the DEBUG env var (or the browser's localStorage.debug).  The process must then be re-started for it to take effect.
I am using the latest 4.3.3 release.
I must be doing something wrong, as the README docs clearly state that enable() dynamically resets the filter.
The ability to dynamically change the logging filters (without restarting the process) is a crucial feature for production code, especially in a node/server environment.
I have searched both the debug project issues -and- stackoverflow with no luck.  I did find a number of enable() issues, but none that seemed to match my problem.
One thing I considered is I am creating my log instances in a "module scope", so my probes live a lifetime in the process.  Is this an incorrect usage?  Should I be creating log instances in a function scope, so they are re-created on each function invocation?  I tried a quick test of this, and it seemed the results were the same (still no dynamic behavior).
From an application perspective, this feature would seem to be very rudimentary ... so as I stated, I must be doing something wrong.
Any help would be greatly appreciated.

It might be the case, honestly. Using debug.enable() is very highly discouraged and hasn't worked "correctly" since it was written. I know that's probably not the answer you want to hear.
You should be using the DEBUG environment variable whenever possible. Setting individual debugger enabled properties should also work correctly. The top-level debug.enable() function is a mine field and has multiple documented issues.

Thanks for your quick response @Qix.
FYI: When I used the term "rudimentary", I was not implying that it was easy to implement, rather that it's usage was easy (not understanding how I messed it up :-)
Background
Certainly in a development environment it is feasible to use the DEBUG env var.  However I am working on a server administrative utility that provides the ability to dynamically alter the filters in a production deployment (without re-starting the server).  In this context, the debug.enable() feature is crucial in diagnosing unexpected conditions in production.
Regarding My Problem
For what it is worth, I just now partially resolved my problem.  I had a dream about this last night (I do my best work in my sleep :-).
I thought I had it fixed fix, but it is still very problematic (as you say a "mine field").
My project is a MonoRepo, where I am using the debug logging utility in multiple packages.  The way in which I am managing the MonoRepo, I ended up with multiple debug instances, which (I believe) manifested itself in the behavior I was seeing.
This would explain how debug.enable(...)

changed localStorage.debug ... because this is more of a global state

but did not dynamically change the filters of the logging probes that were created through a different debug instance!

My attempted fix was to promote a single debug instance from a central utility (promoted by ONE of my MonoRepo packages).
I was able to get it working correctly on my Svelte client, however had NO success on my Node server.  I suspect this problem is further compounded by the fact that my server dependency (Express) also uses debug.
Thoughts?
Mine Field Future?
Regarding the debug.enable() "mine field", are there plans to shore shore this up (possibly in a major release)?  As I mentioned, this is a crucial feature for projects that are retaining their logging probes in production.
Misc Questions
If you would be so kind, I would appreciate you insight on a couple of questions regarding performance ...
Question 1: Is the overhead of debug's "pure logging functions" fairly minimal when disabled?  Have you done any performance tests on this?  I'm using the term pure so as to omit any overhead incurred by the app-level message generation (I realize the app must add defensive measures to prevent that overhead when the log is disabled).  Performance is certainly relevant for projects that intend to leave their logging probes in-place for production.
Question 2: Speaking of performance ... there are cases where I want to use a distinct log namespace for a specific function (within a module) ... EX: "myApp:myModule:myFunction".  In this case, is it OK (from a performance perspective) to create the log in that function scope (where it would be re-created on each function invocation), or should I create all the logs at one time (in the module scope) and use the appropriate log in each function?
Question 3: Is there a way to obtain the current logging filter from the debug API?  I realize I can glean this (initially) from the env var, but I would prefer to get it from the horses mouth (i.e. the debug API).  I noticed that debug.disable() returns this information.  I suppose I could run disable() (to get the filter) and then run enable() with that same filter, but that seems rather hacky.  I may have missed it, but that simple feature would be nice one to add to the list of enhancements.
Thanks
Thanks in advance, for putting up with all my questions, thanks again for this awesome utility!

are there plans to shore shore this up

Yes, quite a few issues are open about it. Checked the pin roadmap issue #656 which mentions the various pieces of the .enable() puzzle.

authors: KevinAst Qix-

repo: debug-js/debug


text:
undertow update

current: 1.3.15
https://github.com/undertow-io/undertow/releases/tag/1.4.0.Final

Also, I updated Pippo SNAPSHOT on sonatype.

authors: decebals etemelkuran

repo: decebals/pippo


text:
Monk nodes seems to be stuck at block 500015

My 3 nodes (Windows GUI Wallet, and 2 CLI ubuntu nodes) are stuck at block 500015.
Decenomy explorer is also stuck at the same block.
Waiting for an update ;)

devs have frozen chain for a couple of hours to thwart hacker

repo: decenomy/MONK


text:
Setup Husky and imports order linting

Description
This PR installs husky and lint-staged as well as adds linting rules for import order.
husky is a tool for running commands/scripts on commit hooks (pre-commit, post-commit and many more). We're using husky to trigger lint-staged.
For additional details on husky check https://www.npmjs.com/package/husky
lint-staged is a tool that allows us to run any commands only on staged files, which means that you don't have to run whole codebase linting on every commit, but only changed files. Current configuration also contains --fix flag, that will automatically fix eslint problems (some of them might be non-fixable though, then your git commit will be abandoned.
Notes
IF YOU NEED TO COMMIT SOME SHIT WITHOUT LINTING THAT - use git commit --no-verify.
But hey - don't do that. I'm watching you 👀
Issue / Notion doc (if applicable)
Testing

Make some auto-fixable errors in any file under src folder. For instance - add extra space or move imports around.
Just run git commit -am "This is my important commit"
Enjoy linting with nice terminal output 😊

Screenshots (if applicable)

Ohhhh didn't realize that, LFG

repo: decent-dao/fractal-interface


text:
priority.priority.TooManyStreamsError

After using the same connection over and over, I eventually hit this error:
priority.priority.TooManyStreamsError: Refusing to insert 1001 streams into priority tree at once

The reason it seems is that self._priority.remove_Stream(stream_id) is never being called. Maybe and example usage would be https://twistedmatrix.com/trac/attachment/ticket/7460/7460_4.patch

Hello !
I am currently working on DoH traffic and using the doh-proxy tool. I'm facing the same problem mentioned by @chantra . Is there any way to patch this in aioh2 or do it manually in the doh-proxy tool?
Could you possibly help me solve this problem?
Thanks in advance.

authors: LoicCaudron chantra


text:
[BUG] Error in transport locks up entire scene

Issue Description:
I have a bug that may or may not be related to my code BUT feels like in part could be improved on the client side.  I cannot consistently reproduce it but once it happens the entire scene stops processing.  I can walk around but cannot interact with anything.  Point downs system, triggers etc. nothing that was working before continues to work.
I understand if my code does something to throw an error, but how is it is stopping processing of the scene as a whole.  I think a try/catch is missing somewhere in the client to allow it to recover.     I have yet been able to isolate it.  It maybe a race condition but when this error does appear,  nothing in my scene works again until I reload it.
"Error: Uncaught Error: [create] Component core::Transform for 131086 already exists
at Worker. (https://cdn.decentraland.org/@dcl/explorer/1.0.160612-20240226145831.commit-52cf906/index.js:63296:123523)
at Worker.gr (https://cdn.decentraland.org/@dcl/explorer-website/2.1.1/assets/index-lLmTeOCG.js:225:1961)
SDK:

[ ] SDK6
[x ] SDK7

Tool:

[ ] Editor
[ ] CLI

CLI Version:
Node Version:
Steps to reproduce:
I have yet to isolate a cause.
Expected behaviour:
Scene not to become unresponsive.
Current behaviour:
I can walk around but cannot interact with anything.  Point downs system, triggers etc. nothing that was working before continues to work.
Reproduction rate:

Intermittent.
Code Snippets:

I have checked my code for poor usage of Transfom.create.  I have not spotted one.  Maybe its still there but my point is the entire scene should not become unusable due to this.
Platforms:

[ ] Windows
[ ] Mac
[x ] Not platform specific

Browser:

[x ] Google Chrome
[ ] Microsoft Edge
[ ] Firefox
[ ] Desktop
[ ] Brave
[ ] Safari
[ ] Not Browser Specific

Environment:

[x ] production - web client
[ ] production - desktop client
[ ] local preview - Have not been able to reproduce here
[ ] web editor
[ ] desktop editor

Evidence:

Additional Notes:

Can be reproduced here https://decentraland.org/play/?position=148%2C60&DEBUG_SCENE_LOG=&realm=main&island=default5k at random.   Not sure exact steps.  Related to clicking relogin, collecting coins etc.

I solved this on the latest release. Can you try using @dcl/sdk@latest and see if still happens ?

@gonpombo8 I only get this on deployed scene.  Or are there easy steps to reproduce locally?  How can I test this live now?   Is it on the stage env?  If so what is the stage env url / if its open to public.
What is eta to release?

@nearnshaw  @gonpombo8  we deployed our scene maybe 24 hours ago "@dcl/js-runtime": "7.4.7",
"@dcl/sdk": "^7.4.7". Which version is it fixed on?
and I just got it again today.  whole scene becomes unresponsive when this happens

Issue is no longer a problem.  I can't recall the SDK version but at some point you fixed this error from crashing the entire scene.   So at this point I dont know if its still an error in the underpinnings or not and cannot tell because of the fix to not let the SDK crash the scene on the error it wont crash the scene anymore :) works for me :)

repo: decentraland/sdk


text:
Issue #99 -- Should Submission Requirements take precedence over Input Descriptors?

Address #99

@brentzundel @csuwildcat updated the language


text:
Merge WACI Spec History

Preserve the Bloom spec history via git:
https://stackoverflow.com/questions/6426247/merge-git-repository-in-subdirectory

@JaceHensley  I had to invite you, please accept invitation.


text:
Installation instruction on theta

Describe the bug
Deephyper version is 0.2.2 rather than 0.2.4 by following the installation instruction on theta: https://deephyper.readthedocs.io/en/latest/install/theta.html
To Reproduce
Steps to reproduce the behavior:
module load postgresql
module load miniconda-3
conda create -p dh-theta python=3.8 -y
conda activate dh-theta/
conda install gxx_linux-64 gcc_linux-64 -y
pip install deephyper[analytics,balsam]
conda install tensorflow -c intel -y

Expected behavior
Checked version after installation,

(miniconda-3/latest/base) jkoo@thetalogin5:/lus/theta-fs0/projects/OptADDN/hepnos/run/env21/run/exp_run_62> source activate /lus/theta-fs0/projects/OptADDN/hepnos/sw/dh-theta
(miniconda-3/latest//lus/theta-fs0/projects/OptADDN/hepnos/sw/dh-theta) jkoo@thetalogin5:/lus/theta-fs0/projects/OptADDN/hepnos/run/env21/run/exp_run_62>
(miniconda-3/latest//lus/theta-fs0/projects/OptADDN/hepnos/sw/dh-theta) jkoo@thetalogin5:/lus/theta-fs0/projects/OptADDN/hepnos/run/env21/run/exp_run_62> ipython

Python 3.8.8 (default, Apr 13 2021, 19:58:26)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.23.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import deephyper as dh

In [2]: print (dh.__version__)
0.2.2

Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. Ubuntu]
System: [Theta]
Python version [3.8]
DeepHyper Version [0.2.2]

Additional context
Ray with 1.3.0 is installed.

try pip install deephyper --upgrade

Thank you for your suggestion. pip install deephyper --upgrade leads to 0.2.4, though dependency conflicts for tenforflow. I think this comes from conda install tensorflow -c intel -y
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
intel-tensorflow 2.3.0 requires numpy<1.19.0,>=1.16.0, but you have numpy 1.19.4 which is incompatible.
intel-tensorflow 2.3.0 requires scipy==1.4.1, but you have scipy 1.6.3 which is incompatible.
intel-tensorflow 2.3.0 requires tensorflow-estimator<2.4.0,>=2.3.0, but you have tensorflow-estimator 2.4.0 which is incompatible.
Successfully installed deephyper-0.2.4 grpcio-1.32.0 numpy-1.19.4
(miniconda-3/latest//lus/theta-fs0/projects/OptADDN/hepnos/sw/dh-theta) jkoo@thetalogin6:/lus/theta-fs0/projects/OptADDN/hepnos> ipython
Python 3.8.8 (default, Apr 13 2021, 19:58:26)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.23.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import deephyper as dh
In [2]: print (dh.version)
0.2.4

repo: deephyper/deephyper


text:
Byte-Pair Encoding tokenizer

Description
I would like to use a model from huggingface that uses a BPE tokenizer. This model is flaubert/flaubert_base_uncased.
It can be found at: https://huggingface.co/flaubert/flaubert_base_uncased
It doesn't have a tokenizer.json, so the djl huggingface tokenizers won't work with it.

Is there any other way to build this kind of tokenizer with djl?
Do you think it would be possible to add some method to build the tokenizer file using the vocabulary.json and the merges.txt in the huggingface wrapper?

Will this change the current api? How?
Maybe it could be just a new method in the HuggingFaceTokenizer to create a tokenizer with the 2 files?
Who will benefit from this enhancement?
anyone using byte-pair encoding tokenizer
References

https://huggingface.co/docs/transformers/main/en/tokenizer_summary#byte-pair-encoding
https://github.com/huggingface/tokenizers/blob/main/tokenizers/src/models/bpe/model.rs

@larochef
Yes, we should be able to expose BPE api in our huggingface tokenizer extension.
For the mean time, you should be able to use our sentencepiece extension, it has BPE implementation.

I have pushed some MR for it, and here are a few points:

I didn't manage to get the builder work, I have faced some memory move issues the I didn't know how to fix properly. I guess it is somewhat linked to what is done in the cast_handle method that doesn't play nice with the builder.
For some reason, I can't get the jni link work in unit tests, there should be something I'm missing, but I don't really see what

Tell me what you think of it, and let's see how we can get it fully working!

https://github.com/mymagicpower/AIAS/blob/main/2_nlp_sdks/kits/tokenizer_sdk/src/main/java/me/aias/example/utils/ClipBPETokenizer.java

authors: frankfliu larochef mymagicpower

repo: deepjavalibrary/djl


text:
Gaussian blur cpu performance

I have been doing some experiments with PIX since it allows computing image augmentations in the GPU in contrast to torchvision which computes in the CPU and requires multiple workers to avoid bottlenecks. When performing some very simple timeit examples I observed a very high time when performing a gaussian blur in the CPU. I created a simple Colab notebook to demonstrate these experiments. I even tested transferring the image to CPU before performing the blur but it doesn't seem to make any difference. I was wondering if this is intended and I should not rely on CPU computations at all or if something is yet to be optimized for CPU computation.

Hi @mgoulao, thanks for reaching out! Yeah indeed I've also tested this and is not performing quite well on CPU. Transferring the image to CPU only helps a little, it's a gain of few us over a several ms operation. This is not technically intended, the goal we try to achieve with PIX is to have implementations that perform well on TPUs/GPUs, taking what we get as a result of this when running on CPUs. This doesn't mean, of course, that we don't want/have to improve CPU implementation as well 😄 Feel free to submit a PR with any optimisation for CPU!

Hi @mgoulao
I made a JAX package for stencil computation that can be used to calculate the gaussian blur.
I checked the performance of kernex vs dm_pix on Colab CPU  using the following code.
It seems that kernex backed convolution is faster on the CPU for this specific function.
Hope this helps
Best.
# !pip install dm_pix
# !pip install kernex

import jax
import jax.numpy as jnp
import kernex as kex
import dm_pix
import numpy.testing as npt

def gaussian_blur(image, sigma, kernel_size):
    x = jnp.linspace(-(kernel_size - 1) / 2.0, (kernel_size- 1) / 2.0, kernel_size)
    w = jnp.exp(-0.5 * jnp.square(x) * jax.lax.rsqrt(sigma))
    w = jnp.outer(w, w)
    w = w / w.sum()

    @kex.kmap(kernel_size=(kernel_size, kernel_size), padding="same")
    def conv(x):
        return jnp.sum(x * w)

    return conv(image)

sigma = 1.
kernel_size=5

gaussian_blur_pix = jax.jit(lambda x: dm_pix.gaussian_blur(x,sigma, kernel_size))
gaussian_blur_kex = jax.jit(lambda x: gaussian_blur(x, sigma, kernel_size))

x = jax.random.uniform(jax.random.PRNGKey(0), (512,512))
xx = jnp.expand_dims(x, axis=2)
npt.assert_allclose(gaussian_blur_pix(xx)[:,:,0], gaussian_blur_kex(x), atol=1e-5)

# warm up
gaussian_blur_pix(xx)
gaussian_blur_kex(x)

%timeit gaussian_blur_pix(xx).block_until_ready()
%timeit gaussian_blur_kex(x).block_until_ready()

111 ms ± 40 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
11.1 ms ± 3.61 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)

Noted  thanks,
I will try to contribute in the coming days.
Best.

Hey,
I implemented dm_pix.gaussian_blur with no extra dependencies in this colab
you can find testing and benchmarking against the depthwise-based implementation.
On colab CPU I'm getting the following speed up based on timeit average time for the jitted version of both implementations.
# average time ratio pix/kex for 3x3 kernel
# (64, 64, 1):	        12.17
# (128, 128, 1):	14.70
# (256, 256, 1):	17.38
# (512, 512, 1):	16.37
# (64, 64, 32):	62.64
# (128, 128, 32):	44.88
# (256, 256, 32):	36.19
# (512, 512, 32):	36.60
# (64, 64, 64):	42.34
# (128, 128, 64):	80.46
# (256, 256, 64):	57.42
# (512, 512, 64):	54.94

for GPU, the speed-up ratio is
# average time ratio pix/kex for 3x3 kernel
# (64, 64, 1):	        1.76
# (128, 128, 1):	1.87
# (256, 256, 1):	1.82
# (512, 512, 1):	1.98
# (64, 64, 32):	1.81
# (128, 128, 32):	2.67
# (256, 256, 32):	2.72
# (512, 512, 32):	5.24
# (64, 64, 64):	2.96
# (128, 128, 64):	1.78
# (256, 256, 64):	3.22
# (512, 512, 64):	8.81

Let me know if it's suitable for a PR
Best.

Thanks @ASEM000, I'll have a look at it as soon as I can, unfortunately that will probably be end of month 😭

Yes, you are right; sorry for the typo.

I'm finally back. I'll try to look into this asap!

Hello,
Any updates or feedback?
Additionally, I implemented a Gaussian filter based on  FFT depthwise convolution, which should be faster for large kernels.
https://github.com/ASEM000/serket/blob/main/serket/nn/blur.py
Let me know if you are interested, so I can provide no extra dependencies version

Hey @ASEM000, I have not forgotten about this 😄
I've been quite busy and should finally be back to normal work regime, I'll try to look at all this asap 🚀

authors: ASEM000 claudiofantacci mgoulao


text:
spike(ui) Navigate back to most recent chat thread

This spike is to research the possibility of navigating the user back to the most recent chat thread when they navigate back to the chat page of LFAI. If there is ever not a most recent chat thread to navigate to then a new chat thread will be the default.
Spend only a couple hours on this to see if it is feasible. If not, no big deal, we will then figure an alternate solution

closed by #615

authors: andrewrisse gregclark-defenseunicorns

repo: defenseunicorns/leapfrogai


text:
wait-for waits the full timeout with nonexistent or nonvalid kubeconfig

Environment
Device and OS: macOS 14.4
App version: v0.32.4
Kubernetes distro being used: none
Other:
Steps to reproduce

Set Kubeconfig to a nonexistent file or a nonvalid kubeconfig file
Run zarf tools wait-for app=test ready -n default

Expected result
Quick return with some error about an nonvalid kubeconfig
Actual Result
Waits the full timeout (default 5 minutes)
Visual Proof (screenshots, videos, text, etc)
nonexistent kubeconfig contrasting kubectl wait vs zarf tools wait-for:

nonvalid kubeconfig contrasting kubectl wait vs zarf tools wait-for:

This could be fixed if we chose to go forward with #2497

authors: michaelabix phillebaba

repo: defenseunicorns/zarf


text:
Hotfix: Change release names of resources uploaded to S3

Description
The name of the artifacts uploaded to the S3 bucket should match the name of the resources when they're built locally.

Funny you should bring this up.  @Madeline-UX and I chatted about this today and think that it would be cleaner to just use the GitHub release artifacts and only keep the s3 uploads as a backup for customers in case of issues with GitHub. Thoughts @YrrepNoj?

Funny you should bring this up. @Madeline-UX and I chatted about this today and think that it would be cleaner to just use the GitHub release artifacts and only keep the s3 uploads as a backup for customers in case of issues with GitHub. Thoughts @YrrepNoj?

That sounds good to me. Is that something we should be adding to the README with a description along the lines of 'mirrors of the release artifacts? If so, right now there's no way to explore the S3 bucket so unless you know the exact path you won't be able to actually use the bucket to download releases. I think we could make https://zarf-public.s3-us-gov-west-1.amazonaws.com/release/ public so that we could essentially 'explore' the different releases and get their downloads.

Tbh I don't think it matters.  If github is down (not super likely) then they wouldn't have the readme unless they cloned the repo already--in which case they could also just build the release themselves.  I think backup to GovCloud will matter to give more confidence to government customers, but a public link isn't really that critical in my mind.  An alternative might be to just cache the release URLs via a CDN or just state they are also backed up to GovCloud.

@jeff-mccoy  That makes sense to me. Updated.

authors: YrrepNoj jeff-mccoy


text:
Can't make it work

Hi, here's what I did:

setup *.localtunnel.function.fr CNAME function.fr.
ping localtunnel.function.fr works
ping dsadsa.localtunnel.function.fr works
using lt --host http://localtunnel.function.fr --port 8080 fails

It fails because maybe_bounce() does not find any client and so return true; and respond 502 to client.
I wonder how this could work, I know it does, but can you help on this? Everything is up and running if you want to test.
Also, I am setting this up because I get "Not running" errors most of the time while testing with zuul/saucelabs.

Would https://github.com/inconshreveable/ngrok be a more reliable solution to localtunneling with zuul? I really want my zuul testing flow to be reliable.
Either ngrok or sauceconnect would be good, maybe you know exactly what's wrong with the localtunnel issues..?

I removed the last comment, not relevant. I only really want to make localtunnel-server work.. Can you help @defunctzombie?

How does ping dsadsa.localtunnel.function.fr works work if no tunnel has yet been established?

Because there's a wildcard DNS set on *.function.fr.

Maybe we can chat on gitte quickly to solve this.

Sure. Link me.

Reading the code: https://github.com/defunctzombie/localtunnel-server/blob/master/server.js#L28
When doing a requets, it goes to maybe_bounce(), I have a hostname, I have a subdomain (localtunnel.function.fr is a subdomain), returns false.

AH! I know, is localtunnel compatible with being served on a subdomain? This is the case: I am trying to use --lt localtunnel.function.fr, it may break?

I think as long as the server code around identifying the "client id" (subdomain) is correct it should work. That code (using the module to do that) might be wrong or returning too much of the subdomain.

https://gitter.im/defunctzombie/localtunnel-server But you have to create it.

I suspect it is returning everything before function.fr instead of just the stuff before the first dot.

So the strategy is:
if no hostname return false (go to localtunnel app)
if no subdomain (clientID) then return false, but in my case, it's returning "localtunnel"

Yea. I think the code could be made to handle this better. It may be that auto-detecting the "base domain" part is not a great way to do it and we can just accept an ENV var for the basename.

I will try to host localtunnel on a top level TLD, it should work then

Yes, as a top level domain it should work.

So tried the toplevel, now it almost work:
> lt --host http://localtunnel.eu --port 8080
your url is: http://pgpnimhrzm.localtunnel.eu

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: connection refused: localtunnel.eu:46831 (check your firewall settings)
    at Socket.<anonymous> (/home/vvo/.nvm/v0.10.36/lib/node_modules/localtunnel/client.js:84:32)
    at Socket.emit (events.js:95:17)
    at net.js:441:14
    at process._tickCallback (node.js:442:13)

My server seems to not have any specific iptables rule set, strange

Make sure your local network is also not doing anything funky. Sometimes
outbound connections on high ports are blocked to protect against malware.
On Thursday, February 26, 2015, Vincent Voyer notifications@github.com
wrote:

My server seems to not have any specific iptables rule set, strange
—
Reply to this email directly or view it on GitHub
https://github.com/defunctzombie/localtunnel-server/issues/21#issuecomment-76277696
.

Sometimes it just works and I get a url but then going to the url, it says "no active client.."

I should note, using localtunnel.me always works

http://localtunnel.eu/ app is showing, at least something works :D

rebooting, because, well..

now it works (sometimes), but all I can get is "localtunnel error: no active client for 'xxxxx'", omg

specifying a subdomain will always work,untill I shutdown/restart the server and try to reuse the same domain, then it fails with connection refused, what is different from my server from yours localtunnel.me. that's magic

mhh might be a DNS problem, seems like *.localtunnel.eu does not resolves to the good IP..

Ok now it works but I still have the sort of caching issue that when restarting the localtunnel server, it will fail from the client side with "connection refused".
Maybe DNS configuration/caching involved or some unclosed connection somewhere

Ok I think Ill use sauce connect for my needs, thanks for help

authors: defunctzombie vvo

repo: defunctzombie/localtunnel-server

license: mit


text:
feat(wip): add deposit machine

Declares state changes for direct deposits on the NEAR network and deposits via the POA bridge (generating an address).
Created the foundational structure for the deposit service, including the main interfaces.

Note: Unit tests are not included in this PR since the core concepts of the state machine are still under development.

:tada: This PR is included in version 1.4.0 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:

authors: amdefuse jobotics

repo: defuse-protocol/defuse-sdk


text:
chore(requirements): update idna to 2.4

See https://github.com/kjd/idna/blob/master/HISTORY.rst#24-2017-03-01

Codecov Report

Merging #1253 into master will decrease coverage by -0.08%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1253      +/-   ##
==========================================
- Coverage   87.29%   87.21%   -0.08%
==========================================
  Files          44       44
  Lines        3873     3873
  Branches      673      673
==========================================
- Hits         3381     3378       -3
- Misses        324      326       +2
- Partials      168      169       +1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 934d68d...ec342d3. Read the comment docs.

authors: codecov-io mboersma

repo: deis/controller


text:
login and register hangs

Hi guys.
I've built new cluster of 3 bare metal machines living in Hetzner. CoreOS beta (607), Deis 1.4.1.
Immediately after setup I was able to register, but few hours later both deis register and deis login hangs after entering all values. If I interrupt it with C-c, I see
^CTraceback (most recent call last):
  File "<string>", line 2113, in <module>
  File "<string>", line 2109, in main
  File "<string>", line 2052, in _dispatch_cmd
  File "<string>", line 801, in auth_login
  File "/home/jenkins/workspace/build-deis-cli-installer-linux/src/github.com/deis/deis/client/build/deis/out00-PYZ.pyz/requests.sessions", line 504, in post
  File "<string>", line 158, in request
  File "/home/jenkins/workspace/build-deis-cli-installer-linux/src/github.com/deis/deis/client/build/deis/out00-PYZ.pyz/requests.sessions", line 461, in request
  File "/home/jenkins/workspace/build-deis-cli-installer-linux/src/github.com/deis/deis/client/build/deis/out00-PYZ.pyz/requests.sessions", line 573, in send
  File "/home/jenkins/workspace/build-deis-cli-installer-linux/src/github.com/deis/deis/client/build/deis/out00-PYZ.pyz/requests.adapters", line 370, in send
  File "/home/jenkins/workspace/build-deis-cli-installer-linux/src/github.com/deis/deis/client/build/deis/out00-PYZ.pyz/requests.packages.urllib3.connectionpool", line 518, in urlopen
  File "/home/jenkins/workspace/build-deis-cli-installer-linux/src/github.com/deis/deis/client/build/deis/out00-PYZ.pyz/requests.packages.urllib3.connectionpool", line 353, in _make_request
  File "/home/jenkins/workspace/build-deis-cli-installer-linux/src/github.com/deis/deis/client/build/deis/out00-PYZ.pyz/httplib", line 1045, in getresponse
  File "/home/jenkins/workspace/build-deis-cli-installer-linux/src/github.com/deis/deis/client/build/deis/out00-PYZ.pyz/httplib", line 409, in begin
  File "/home/jenkins/workspace/build-deis-cli-installer-linux/src/github.com/deis/deis/client/build/deis/out00-PYZ.pyz/httplib", line 365, in _read_status
  File "/home/jenkins/workspace/build-deis-cli-installer-linux/src/github.com/deis/deis/client/build/deis/out00-PYZ.pyz/socket", line 476, in readline
KeyboardInterrupt

I've checked etcd logs on all servers, two are fine, one emits errors:
Mar 13 13:43:24 staging-coreos-2 etcd[625]: [etcd] Mar 13 13:43:24.551 INFO      | a6efd505a1c5420f8ab1c11af55571b3: warning: heartbeat time out peer="2fdc33aa2bb347aaa1da0d742d41190c" missed=1 backoff="2s"
Mar 13 13:44:08 staging-coreos-2 etcd[625]: [etcd] Mar 13 13:44:08.551 INFO      | a6efd505a1c5420f8ab1c11af55571b3: warning: heartbeat time out peer="2fdc33aa2bb347aaa1da0d742d41190c" missed=1 backoff="2s"
Mar 13 13:44:24 staging-coreos-2 etcd[625]: [etcd] Mar 13 13:44:24.761 INFO      | a6efd505a1c5420f8ab1c11af55571b3: warning: heartbeat time out peer="2fdc33aa2bb347aaa1da0d742d41190c" missed=1 backoff="2s"
Mar 13 13:44:34 staging-coreos-2 etcd[625]: [etcd] Mar 13 13:44:34.051 INFO      | a6efd505a1c5420f8ab1c11af55571b3: warning: heartbeat time out peer="2fdc33aa2bb347aaa1da0d742d41190c" missed=1 backoff="2s"

Time is the same on all three servers (up to seconds), though ceph -s reports
     health HEALTH_WARN clock skew detected on mon.staging-coreos-2, mon.staging-coreos-3

Though maybe ceph requires more precise time. Ntpd is running on all servers.
What can I do to further investigate the issue? Thank you.

deis-controller logs:
Mar 13 14:06:55 staging-coreos-1 sh[23009]: 2015-03-13T14:06:55Z b338afdb5f45 confd[84]: ERROR Config check failed: exit status 1
Mar 13 14:09:01 staging-coreos-1 sh[23009]: [2015-03-13 14:09:00 +0000] [90] [CRITICAL] WORKER TIMEOUT (pid:119)
Mar 13 14:09:01 staging-coreos-1 sh[23009]: [2015-03-13 14:09:00 +0000] [119] [INFO] Worker exiting (pid: 119)
Mar 13 14:09:01 staging-coreos-1 sh[23009]: [2015-03-13 14:09:01 +0000] [3049] [INFO] Booting worker with pid: 3049
Mar 13 14:09:01 staging-coreos-1 sh[23009]: 2015-03-13T14:09:01Z b338afdb5f45 confd[84]: ERROR Config check failed: exit status 1
Mar 13 14:10:29 staging-coreos-1 sh[23009]: 2015-03-13T14:10:29Z b338afdb5f45 confd[84]: ERROR Config check failed: exit status 1
Mar 13 14:13:21 staging-coreos-1 sh[23009]: Error:  501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
Mar 13 14:13:23 staging-coreos-1 sh[23009]: Error:  501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
Mar 13 14:13:23 staging-coreos-1 sh[23009]: 2015-03-13T14:13:22Z b338afdb5f45 confd[84]: ERROR 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
Mar 13 14:13:27 staging-coreos-1 sh[23009]: 2015-03-13T14:13:27Z b338afdb5f45 confd[84]: ERROR Config check failed: exit status 1

Finally it worked. I had to restart controller and, more strange, routers.

that does sound odd. Those confd errors from the controller are etcd connection errors (could be network latency, bad connection etc.) but nothing else stands out to me.
As for the clock skew, would you mind checking to see if https://github.com/deis/deis/pull/3222 works for you? Thanks!

@bacongobbler
Time shift is not really that much for me. Please see how it differs:
core@staging-coreos-1 ~ $ sudo systemctl stop ntpd && sudo ntpdate pool.ntp.org && sudo systemctl start ntpd
13 Mar 15:47:22 ntpdate[13973]: adjust time server 176.9.40.142 offset -0.125435 sec

core@staging-coreos-2 ~ $ sudo systemctl stop ntpd && sudo ntpdate pool.ntp.org && sudo systemctl start ntpd
13 Mar 15:47:41 ntpdate[5138]: adjust time server 5.9.49.12 offset 0.002371 sec

core@staging-coreos-3 ~ $ sudo systemctl stop ntpd && sudo ntpdate pool.ntp.org && sudo systemctl start ntpd
13 Mar 15:47:51 ntpdate[22860]: adjust time server 46.4.28.205 offset 0.000287 sec

So nothing is more than 0.12 second. Is it too much for Ceph?

that should be completely fine. I guess then the main issue would be the network latency with etcd that occurred. If you get logs, I'd check with upstream to see if there's a potential issue there.

That's weird as servers are connected via switch in the same rack, so no possible routing latency. Which logs should I try to gather?

I'd gather etcd logs from all hosts, traffic logs, maybe monitoring data if you have it available

@ineu the threshold to show the warning is 0.05 seconds
http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-mon/#clock-skews

@aledbf
Thank you, good to know. ATM ceph health is ok, so no time difference.
@bacongobbler
etcd is fine now, no errors, but still 503. nginx config returns 503 because of https://github.com/deis/deis/blob/master/router/image/templates/nginx.conf#L91.
For some reason etcd doesn't containt 'host' key:
core@staging-coreos-1 ~ $ etcdctl ls --recursive /deis/controller
/deis/controller/webEnabled
/deis/controller/builderKey
/deis/controller/protocol
/deis/controller/registrationEnabled
/deis/controller/secretKey
/deis/controller/unitHostname

though deis-controller is running:
core@staging-coreos-1 ~ $ fleetctl status deis-controller
● deis-controller.service - deis-controller
   Loaded: loaded (/run/fleet/units/deis-controller.service; linked-runtime; vendor preset: disabled)
   Active: active (running) since Mon 2015-03-16 13:40:12 UTC; 2h 2min ago
  Process: 18973 ExecStartPre=/bin/sh -c docker inspect deis-controller >/dev/null && docker rm -f deis-controller || true (code=exited, status=0/SUCCESS)
  Process: 18949 ExecStartPre=/bin/sh -c IMAGE=`/run/deis/bin/get_image /deis/controller` && docker history $IMAGE >/dev/null || docker pull $IMAGE (code=exited, status=0/SUCCESS)
 Main PID: 18982 (sh)
   CGroup: /system.slice/deis-controller.service
           ├─18982 /bin/sh -c IMAGE=`/run/deis/bin/get_image /deis/controller` && docker run --name deis-controller --rm -p 8000:8000 -e EXTERNAL_PORT=8000 -e HOST=$COREOS_PRIVATE_IPV4 -v /var/run/fleet.sock:/var/run/fleet.sock -v /var/lib/deis/store:/data $IMAGE
           └─18999 docker run --name deis-controller --rm -p 8000:8000 -e EXTERNAL_PORT=8000 -e HOST=192.168.0.1 -v /var/run/fleet.sock:/var/run/fleet.sock -v /var/lib/deis/store:/data deis/controller:v1.4.1

Mar 16 15:42:21 staging-coreos-1 sh[18982]: controller: waiting for confd to write initial templates...
Mar 16 15:42:26 staging-coreos-1 sh[18982]: controller: waiting for confd to write initial templates...
Mar 16 15:42:31 staging-coreos-1 sh[18982]: controller: waiting for confd to write initial templates...

Don't know if this 'waiting for confd' lines is normal.

those lines are not normal. What's happening here is that the controller's waiting on some other component to come up. Could you please dump etcdctl ls / --recursive for me?

@bacongobbler Here it is: https://gist.github.com/ineu/22ad40dcf4d1ece1deb7 (skipped users and services sections).

@ineu from your output the database is not running (deis-database)
/deis/database
/deis/database/adminUser
/deis/database/engine
/deis/database/adminPass
/deis/database/name
/deis/database/password
/deis/database/user
/deis/database/bucketName

(host and port are missing)
you can restart the component running fleetctl stop deis-database && fleetctl start database from any node in the cluster.

well, it is running:
core@staging-coreos-1 ~ $ fleetctl list-units | grep database
deis-database.service						2fdc33aa.../192.168.0.1	active	running

core@staging-coreos-1 ~ $ systemctl status deis-database
● deis-database.service - deis-database
   Loaded: loaded (/run/fleet/units/deis-database.service; linked-runtime; vendor preset: disabled)
   Active: active (running) since Fri 2015-03-13 07:55:40 UTC; 4 days ago
  Process: 1708 ExecStartPre=/bin/sh -c docker inspect deis-database >/dev/null 2>&1 && docker rm -f deis-database >/dev/null 2>&1 || true (code=exited, status=0/SUCCESS)
  Process: 1679 ExecStartPre=/bin/sh -c IMAGE=`/run/deis/bin/get_image /deis/database` && docker history $IMAGE >/dev/null || docker pull $IMAGE (code=exited, status=0/SUCCESS)
  Process: 1672 ExecStartPre=/bin/sh -c docker inspect deis-database-data >/dev/null 2>&1 || docker run --name deis-database-data -v /var/lib/postgresql ubuntu-debootstrap:14.04 /bin/true (code=exited, status=0/SUCCESS)
 Main PID: 1721 (sh)
   CGroup: /system.slice/deis-database.service
           ├─1721 /bin/sh -c IMAGE=`/run/deis/bin/get_image /deis/database` && docker run --name deis-database --volumes-from=deis-database-data -p 5432:5432 -e EXTERNAL_PORT=5432 -e HOST...
           └─1739 docker run --name deis-database --volumes-from=deis-database-data -p 5432:5432 -e EXTERNAL_PORT=5432 -e HOST=192.168.0.1 deis/database:v1.4.1

But there are errors in the journal indeed:
Mar 17 12:35:49 staging-coreos-1 sh[1721]: wal_e.retries WARNING  MSG: retrying after encountering exception
Mar 17 12:35:49 staging-coreos-1 sh[1721]: DETAIL: Exception information dump:
Mar 17 12:35:49 staging-coreos-1 sh[1721]: Traceback (most recent call last):
Mar 17 12:35:49 staging-coreos-1 sh[1721]: File "/usr/local/lib/python2.7/dist-packages/wal_e/retries.py", line 62, in shim
Mar 17 12:35:49 staging-coreos-1 sh[1721]: return f(*args, **kwargs)
Mar 17 12:35:49 staging-coreos-1 sh[1721]: File "/usr/local/lib/python2.7/dist-packages/wal_e/worker/s3/s3_deleter.py", line 27, in _delete_batch
Mar 17 12:35:49 staging-coreos-1 sh[1721]: bucket.delete_keys([key.name for key in page])
Mar 17 12:35:49 staging-coreos-1 sh[1721]: File "/usr/local/lib/python2.7/dist-packages/boto/s3/bucket.py", line 726, in delete_keys
Mar 17 12:35:49 staging-coreos-1 sh[1721]: while delete_keys2(headers):
Mar 17 12:35:49 staging-coreos-1 sh[1721]: File "/usr/local/lib/python2.7/dist-packages/boto/s3/bucket.py", line 714, in delete_keys2
Mar 17 12:35:49 staging-coreos-1 sh[1721]: data=data)
Mar 17 12:35:49 staging-coreos-1 sh[1721]: File "/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py", line 664, in make_request
Mar 17 12:35:49 staging-coreos-1 sh[1721]: retry_handler=retry_handler
Mar 17 12:35:49 staging-coreos-1 sh[1721]: File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 1070, in make_request
Mar 17 12:35:49 staging-coreos-1 sh[1721]: retry_handler=retry_handler)
Mar 17 12:35:49 staging-coreos-1 sh[1721]: File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 1029, in _mexe
Mar 17 12:35:49 staging-coreos-1 sh[1721]: raise ex
Mar 17 12:35:49 staging-coreos-1 sh[1721]: timeout: timed out

ceph -s shows HEALTH OK

Restart helped indeed. Any idea why it could be failing and what should I do to prevent it in the future?

@ineu you saw this issue https://github.com/deis/deis/issues/2741

@aledbf Ok, I see. Any way to avoid it before #2701 is merged?

Ok, I see. Any way to avoid it before #2701 is merged?

The root here is that the database component can no longer communicate with store-gateway. However, there are a few different causes of gateway troubles. Can you get the gateway logs and ceph -s output the next time this happens?

@carmstrong I've posted logs here: https://github.com/deis/deis/issues/2741#issuecomment-83932022
ceph -s and journalctl -u etcd are fine.

As a workaround I've implemented simple guard and wrapped it into systemd unit:
#!/bin/bash

while true; do
    if ! etcdctl get /deis/database/host >/dev/null 2>&1; then
        echo "Key not found, restarting deis-database"
        fleetctl stop deis-database
        sleep 5
        fleetctl start deis-database
    fi
    sleep 60
done

And looking at its logs for today:
Mar 27 06:59:48 staging-coreos-3 systemd[1]: Started Simple guard for deis-database.
Mar 27 10:32:54 staging-coreos-3 deis-database-guard.sh[26972]: Key not found, restarting deis-database
Mar 27 10:32:55 staging-coreos-3 deis-database-guard.sh[26972]: Unit deis-database.service loaded on 2fdc33aa.../192.168.0.1
Mar 27 10:33:02 staging-coreos-3 deis-database-guard.sh[26972]: Unit deis-database.service launched on 2fdc33aa.../192.168.0.1
Mar 27 14:11:08 staging-coreos-3 deis-database-guard.sh[26972]: Key not found, restarting deis-database
Mar 27 14:11:10 staging-coreos-3 deis-database-guard.sh[26972]: Unit deis-database.service loaded on 2fdc33aa.../192.168.0.1
Mar 27 14:11:17 staging-coreos-3 deis-database-guard.sh[26972]: Unit deis-database.service launched on 2fdc33aa.../192.168.0.1
Mar 27 17:50:30 staging-coreos-3 deis-database-guard.sh[26972]: Key not found, restarting deis-database
Mar 27 17:50:32 staging-coreos-3 deis-database-guard.sh[26972]: Unit deis-database.service loaded on 2fdc33aa.../192.168.0.1
Mar 27 17:50:39 staging-coreos-3 deis-database-guard.sh[26972]: Unit deis-database.service launched on 2fdc33aa.../192.168.0.1

So this problem appeared three times for me today.
Output of journalctl -u etcd on every machine in the cluster is ok, no network splits, no quorum losses.
Time difference on servers is tiny:
> $ pssh -i -H 'coreos-1 coreos-2 coreos-3' ntpdate -q 0.coreos.pool.ntp.org | grep adjust
27 Mar 18:35:54 ntpdate[4808]: adjust time server 5.9.122.148 offset 0.001519 sec
27 Mar 18:35:54 ntpdate[6859]: adjust time server 46.4.39.238 offset -0.000423 sec
27 Mar 18:35:55 ntpdate[15398]: adjust time server 85.10.246.226 offset 0.000806 sec

Output of journalctl -u deis-store-gateway@1 --since today
-- Logs begin at Thu 2015-03-19 06:04:17 UTC, end at Fri 2015-03-27 18:25:48 UTC. --
Mar 27 01:31:58 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [24121533]
Mar 27 01:35:17 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [24125633]
Mar 27 02:30:33 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [24194551]
Mar 27 02:42:20 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [24209201]
Mar 27 04:30:45 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [24344258]
Mar 27 04:58:00 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [24378136]
Mar 27 06:32:33 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [24496002]
Mar 27 06:39:36 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [24504737]
Mar 27 08:18:52 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [24631109]
Mar 27 11:44:43 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [24892189]
Mar 27 13:48:41 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25048751]
Mar 27 14:10:50 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25077019]
Mar 27 14:11:52 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25078328]
Mar 27 14:25:42 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25095924]
Mar 27 14:43:25 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25119153]
Mar 27 14:47:20 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25124349]
Mar 27 14:54:36 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25133927]
Mar 27 15:01:57 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25143589]
Mar 27 15:04:09 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25146526]
Mar 27 15:23:57 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25172637]
Mar 27 16:17:51 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25243469]
Mar 27 16:26:34 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25254503]
Mar 27 16:42:52 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25275051]
Mar 27 17:52:30 staging-coreos-2 sh[2577]: Error:  100: Key not found (/deis/store/gateway/masterLock) [25363857]

Ceph is ok:
root@30e039d6b63f:/app# ceph -s
    cluster 3964dde2-a9c2-49b2-b490-dea0655da56b
     health HEALTH_OK

@ineu thanks for the detailed debug output.  Extremely helpful.
I have a strong suspicion the issue is caused by the database publishing logic.  The /deis/database/host key is written with a TTL inside a loop that can potentially be interrupted by periodic backups.  If backups take longer than the TTL, the keys can disappear from etcd.  I suspect that's what is happening here.
To resolve this, we need to decouple publishing from period backups.  The best way to do this is to rewrite the database init in Go and use goroutines for background tasks, as we do elsewhere.

We got hit by this issue today.

Closing - we're tracking the root issue in #2741.

authors: aledbf bacongobbler carmstrong gabrtv ineu ngbinh

repo: deis/deis


text:
Cannot deploy using Heroku buildpack and private repositories.

Using version 1.7.3 I am unable to push to a fresh Vagrant Deis (3 node) cluster. Using the latest Heroku buildpack specified here:
https://github.com/heroku/heroku-buildpack-nodejs
I have logged into the deis-builder and confirmed that the container has access to GitHub with the correct known_hosts and private/public keys. However when doing a git push to Deis (Node.js v0.12.5 and npm 2.7.x) I get the following...
   $ git push -f deis testing/deis
   Counting objects: 427, done.
   Delta compression using up to 8 threads.
   Compressing objects: 100% (200/200), done.
   Writing objects: 100% (427/427), 45.25 KiB | 0 bytes/s, done.
   Total 427 (delta 251), reused 366 (delta 218)
   -----> Node.js app detected

   -----> Reading application state
          package.json...
          build directory...
          cache directory...
          environment variables...

          Node engine:         0.12.x
          Npm engine:          2.7.x
          Start mechanism:     npm start
          node_modules source: package.json
          node_modules cached: false

          NPM_CONFIG_PRODUCTION=true
          NODE_MODULES_CACHE=true

   -----> Installing binaries
          Resolving node version 0.12.x via semver.io...
          Downloading and installing node 0.12.5...
          Resolving npm version 2.7.x via semver.io...
          Downloading and installing npm 2.7.6 (replacing version 2.11.2)...

          ...

          npm ERR! Host key verification failed.
          npm ERR! fatal: Could not read from remote repository.
          npm ERR!
          npm ERR! Please make sure you have the correct access rights
          npm ERR! and the repository exists.
          npm ERR!
          npm ERR!
          npm ERR! If you need help, you may report this error at:
          npm ERR!     <https://github.com/npm/npm/issues>

          npm ERR! Please include the following file with any support request:
          npm ERR!     /tmp/build/npm-debug.log

   -----> Build failed

If I then force the npm version to 2.6.x I get a different build error as follows...
   $ git push -f deis testing/deis:master
   Counting objects: 424, done.
   Delta compression using up to 8 threads.
   Compressing objects: 100% (197/197), done.
   Writing objects: 100% (424/424), 45.00 KiB | 0 bytes/s, done.
   Total 424 (delta 249), reused 366 (delta 218)
   -----> Node.js app detected

   -----> Reading application state
          package.json...
          build directory...
          cache directory...
          environment variables...

          Node engine:         0.12.x
          Npm engine:          2.6.x
          Start mechanism:     npm start
          node_modules source: package.json
          node_modules cached: false

          NPM_CONFIG_PRODUCTION=true
          NODE_MODULES_CACHE=true

   -----> Installing binaries
          Resolving node version 0.12.x via semver.io...
          Downloading and installing node 0.12.5...
          Resolving npm version 2.6.x via semver.io...
          Downloading and installing npm 2.6.1 (replacing version 2.11.2)...

          ...

   npm ERR! ssh: connect to host github.com port 22: No route to host
   npm ERR! fatal: Could not read from remote repository.
   npm ERR!
   npm ERR! Please make sure you have the correct access rights
   npm ERR! and the repository exists.

ssh having issues getting through to GitHub on port 22. Yet inside the deis-builder I had connectivity. Very odd that a simple change of npm version could result in two different build errors.

@Crispy1975 why are you downgrading npm to an older version than the provided by 0.12.5?

@aledbf simple as the issue here shows a problem with npm 2.7.x and the buildpack, so tried the earlier version to see if it helped. While it did not, it was a different issue. Here is where I sourced the details...
https://github.com/heroku/heroku-buildpack-nodejs/issues/208

@Crispy1975 ok, but you are replacing the npm version provided in node.js 0.12.5 (2.11.2) with an older release (https://github.com/npm/npm/compare/v2.7.6...master)
If don't specify the npm engine version the deploy works?

@aledbf no, the first code block is done with the associated npm version and not forcing it to an earlier version. So the package.json contains only the following to force the version of Node.js
"engines": {
	"node": "0.12.x"
}

The npm version is naturally chosen and not specified in this case.

@Crispy1975 ok, the error that you see is "Host key verification failed." if you the default version?
If that's the case are you using npm dependencies behind a private registry or git repository that requires authentication?

@aledbf yes, the reference to another issue that seems similar is on this thread above. I have logged into the deis-builder container and confirmed known_hosts file and keys are present, and that I can authenticate against GitHub with git -T git@github.com. But when pushing I get the failed build based on the host error.

This does not seem specifically related to the Heroku buildpacks from what I can tell now. I have converted a number of our apps to use Dockerfiles over buildpacks and get the same "Host key verification failed." error.
npm ERR! Cloning into bare repository '/root/.npm/_git-remotes/ssh-git-github-com-api-plugin-git-be643e25'...
npm ERR! Host key verification failed.
npm ERR! fatal: Could not read from remote repository.
npm ERR!
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR!
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /tmp/npm-debug.log

We eventually got around this by switching from buildpacks to using Dockerfiles for building and deploying. In essence we pull in our custom Docker image, make sure cURL is installed along with a few other things, then pull the private repo key from a private S3 bucket (doesn't have to be S3, could be any location), the make sure that is in the correct location for the build process. Here is a sanitised version of the Dockerfile I am now using...
# Set the base image to Debian Jessie with Node.js
FROM crispy1975/jessie-nodejs:latest

# File Author / Maintainer
MAINTAINER Crispy1975

# Install Node.js and other dependencies
RUN apt-get update \
	&& apt-get -y install git-core \
	&& apt-get -y install curl \
	&& apt-get -y install openssh-client \
	&& apt-get -y install python \
	&& apt-get -y install build-essential \
	&& apt-get -y upgrade

# Copy repo access keys over
RUN mkdir -p /root/.ssh \
	&& curl -SLO https://s3-eu-west-1.amazonaws.com/my-bucket/my-key \
	&& mv my-key /root/.ssh/id_rsa \
	&& chmod 600 /root/.ssh/id_rsa \
	&& ssh-keyscan github.com >> /root/.ssh/known_hosts

# Provide a cached layer for node_modules
COPY package.json /tmp/package.json
RUN cd /tmp && npm install
RUN mkdir -p /src && cp -a /tmp/node_modules /src/

# Remove key for security
RUN rm -R /root/.ssh

# Define working directory
WORKDIR /src
COPY . /src

# Move into the src folder
RUN cd /src

# Expose port (override this elsewhere)
EXPOSE  80

# Run the app
ENTRYPOINT ["npm", "start"]

Hope that helps somebody having problems with this. Would also love to hear if there is a better way.

Coming back to this I feel I should also mention that you should make sure you restrict the access the the key location by machine IP addresses, secutiry groups if on AWS etc. We now have a custom script that we use to download private files into the Docker build process ... ssl certs, private keys etc. etc.

@Crispy1975 mind giving this a shot on v2? :)

Yup, will do for sure. We're gonna try it in Google Cloud and not AWS and see how we get on. :)

closing this issue since I believe this is now fixed, but please re-open if you find different results. Thanks!

Is there an actual solution to this? On Dokku, we have to use the host keys plugin to get around this. It uses the same buildpack as Heroku, so Heroku must provide a similar mechanism to use private github repos as npm dependencies. Does it?

Yes. The solution is to use deis config:set SSH_KEY=.... https://github.com/deis/deis/pull/3305#issuecomment-81807358 implemented this feature and the docs also describe the usage in https://deis.com/docs/workflow/applications/using-buildpacks/#using-private-repositories.

@bacongobbler I deleted my comment immediately because I realized it was in the wrong repo. It was a heroku question. Thanks though.

authors: Crispy1975 aledbf bacongobbler cmmartin


text:
chore: Add contribution ladder

Signed-off-by: Sajay Antony sajaya@microsoft.com
Description
What this PR does / why we need it:
This add the contribution ladder template for the Ratify project and is based of https://github.com/cncf/project-template/blob/main/CONTRIBUTING.md
The current template outlines only a subset of roles.
/cc @bridgetkromhout @akashsinghal @toddysm @susanshi @binbin-li

This PR might sit in draft form for a while since folks are ironing out definitions in related projects.

repo: deislabs/ratify


text:
Possible to jump to a line?

I really wish I could back up two lines and re-execute them.  Does byebug support setting the program counter, similar to gdb's jump command?
Alas, I just don't see a way to do it.
Thanks for a stable and super pleasant debugging tool!

+1

+1

I've continued to work on one approach to this, now extracted out into a pry plugin at https://github.com/awwaiid/pry-timetravel . Recently I've figured out how to write a test suite for it, and am working on auto-snapshotting. It was easier and more general-purpose to work my approach into a pry plugin.

Cool, @awwaiid. Will check it out!

Cool, I'll follow your work!

This is also useful to skip a line of code in the forward direction.

+1

Well as a minor unhelpful update, I ran into a significant issue with the fork-based approach when working on anything that has threads .... which includes Rails which is the main app that I personally would use it on. I haven't gone back and research this in a number of years though. The approach is pretty extreme, so maybe there is a less-insane version of jumping.

authors: Edderic MaxPleaner awwaiid bronson deivid-rodriguez georgemillo marcello-telles

repo: deivid-rodriguez/byebug


text:
Change Language

This plugin is great, I just wanto to know how to change default language, because my website is in spanish and some album names are in english.
Thanks.

ok, so if I understand correctly, you'd like some of the album names which are in english to appear in spanish?? if that is the case then your best approach might be to change the names of the albums from english into spanish. That way they will appear in spanish on the site when the plugin pulls the albums. Its not the plugin's job to give you a way to change the language. If you have trouble just google how to change the name: http://lmgtfy.com/?q=changing+name+of+album+on+facebook
Once you change the album name into spanish then when the plugin pulls it onto your site it will be in spanish.

Sorry, I did not explain correctly, I meant the default albums: Timeline Photos, Cover Photos, etc.
I tried with line 630 in the js file: js.src = "//connect.facebook.net/en_US/sdk.js#xfbml=1&version=v2.3&appId=775908159169504"; changing en_US by es_LA, but it didn't work.
Anyway thanks for your time, and your plugin.

I'm not the author of the plugin, I'm just helping where I can.
I'm really not sure how to help you for this. Sorry man

authors: uropheniks zsavicreative

repo: dejanstojanovic/Facebook-Album-Browser


text:
[Feature request] Basic search (filter) of displayed todo's with page display option

Firstly, this is a superb plugin - thank you for developing this!
Thinking about usage as an actual GTD system - having the ability to search (filter) the todo pane items would be incredibly helpful. Perhaps a search box in the pane header (between the tag and the count) that when typing filters down the list to only show matches. With a standard "x" to clear search. And finally, maybe a different css class for the header if the list has been filtered - so it's obvious to the user they're looking at a partial list of todo's, not their entire list. Something like this maybe?

Somewhat related: if there was a setting to optionally display the page for which a task belongs, that would be very helpful for displaying tasks filtered by tag. As an example, let's say user is grouping by tag and the tag they're using is #gtd. And further suppose all their tasks fall into one of two tags: #gtd/waiting and #gtd/next. Then when viewing that list of next tasks, for example, it would be very useful to quickly see which page (ie project) each task belongs to. Perhaps the task with optional project (page) display could look something like:

Anyway, not sure how much work I'm proposing, but those two components (search + page display) would make for a robust GTD implementation w/in Obsidian.
Thank you again for the work you've put into this!

Thanks for the suggestions, I think these are good ideas and I will try to get them in a future release

authors: apg-dev delashum

repo: delashum/obsidian-checklist-plugin


text:
'freeze_bn' not recognized

Hi there... thanks for putting together this repo.
I was trying to run the coco training code and ran into the following error:
Traceback (most recent call last):
File "examples/train_coco.py", line 60, in
model = create_model(weights=args.weights)
File "examples/train_coco.py", line 38, in create_model
return ResNet50RetinaNet(image, num_classes=90, weights=weights)
File "build/bdist.linux-x86_64/egg/keras_retinanet/models/resnet.py", line 36, in ResNet50RetinaNet
File "build/bdist.linux-x86_64/egg/keras_resnet/models/_2d.py", line 178, in ResNet50
File "build/bdist.linux-x86_64/egg/keras_resnet/models/_2d.py", line 82, in ResNet
File "build/bdist.linux-x86_64/egg/keras/legacy/interfaces.py", line 87, in wrapper
TypeError: init() got an unexpected keyword argument 'freeze_bn'
Any thoughts on what might be wrong with my setup?
Thanks.

I'm guessing you don't have the latest version (master even) of keras-resnet installed. I'll update the README.md with a guide on how to install keras-retinanet.

Thanks. When I ran the setup script, this is what was installed:
Installed /usr/local/lib/python2.7/dist-packages/keras_retinanet-0.0.1-py2.7.egg
Processing dependencies for keras-retinanet==0.0.1
Searching for keras-resnet
Reading https://pypi.python.org/simple/keras-resnet/
Downloading https://pypi.python.org/packages/30/19/a348c06440276c575a578ae41a953e81ba7a643bb548afda19fad5c89775/keras-resnet-0.0.8.tar.gz#md5=07f8e0f6462df78a4662c77f0da02f9d
Best match: keras-resnet 0.0.8
Processing keras-resnet-0.0.8.tar.gz
Writing /tmp/easy_install-EjGzMF/keras-resnet-0.0.8/setup.cfg
Running keras-resnet-0.0.8/setup.py -q bdist_egg --dist-dir /tmp/easy_install-EjGzMF/keras-resnet-0.0.8/egg-dist-tmp-bYQGtV
zip_safe flag not set; analyzing archive contents...
Moving keras_resnet-0.0.8-py2.7.egg to /usr/local/lib/python2.7/dist-packages
Adding keras-resnet 0.0.8 to easy-install.pth file
Installed /usr/local/lib/python2.7/dist-packages/keras_resnet-0.0.8-py2.7.egg
Does that look like an older version?

check closed issue for solution: https://github.com/delftrobotics/keras-retinanet/issues/16.

This commit should address this issue. Feedback on this guide is welcome, I couldn't test it fully due to that tensorflow-gpu on pypi seems to use CUDA 8 while I have CUDA 9 installed.
If it doesn't work and you need to take extra steps, I'd be happy to hear them.
Also if this resolves your issue, please close this issue.

Thanks for the update and the quick response to this issue. I was able to get it to work by cloning keras-resnet repo and installing it (which looks to be what you suggest in the updated readme). Seems to be working now.

authors: hgaiser jwnsu mhwilder

repo: delftrobotics/keras-retinanet


text:
Fixed launch crash

Fixed launch screen crash by updating UserDataProvider link (fix HTTP 301 error -> changes page to be requested from “about” to “pages/about”

Also updated some gradle files per Android Studio's request.

Thank you for the pull request, @auzbuzzard!
I'm afraid I'm no longer able to maintain this repo. It'd be great if you could keep the fork and release the APK yourself. You'd need to use your own signing key, and the users of this app would need to uninstall my version and install yours. After that, you'd be able to release subsequent versions yourself and perhaps even pick a better distribution channel like F-Droid.
If you're interested, I can put a link to your fork in the repo description and README

@deliciousblackink Sure! I can keep maintain and perhaps continue development on my current fork, and if you're ok with linking it in your readme.

@auzbuzzard, sorry for my late reply, I've been rather busy these past few months, and I only saw your fork today. Great job fixing the issue with user lists, by the way! I'm afraid I can't offer you any help besides putting a link to your page in the README (efbff596697209c2f4df576f25819dc07112ba8b), though.
Have fun with it!

authors: auzbuzzard deliciousblackink

repo: deliciousblackink/Derpibooru


text:
[BUG]: Update CSM Observability release note with github issue link and support matrix

Bug Description
The features in CSM Observability release note don't have github issue links, and support matrix should be updated.
Logs
n/a
Screenshots
No response
Additional Environment Information
No response
Steps to Reproduce
n/a
Expected Behavior
Features should have corresponding github issue links.
Support matrix should be updated.
CSM Driver(s)
n/a
Installation Type
No response
Container Storage Modules Enabled
No response
Container Orchestrator
n/a
Operating System
n/a

Close as PR has been merged.

repo: dell/csm


text:
Python filesystems rewrite?

Description
:warning: This is a rough draft for now, since I want to make sure this new approach makes sense.
This PR would deprecate the existing DeltaStorageHandler that subclasses pyarrow.FileSystem in favor of a DeltaFileSystem that has a to_pyarrow() method to get the equivalent PyArrow filesystem. The idea is that for reading and writing data files we should use the PyArrow ones, while the delta log is read by the delta-rs one.
The other approach I looked at initially is just filling in the methods
Some notes on each filesystem:

S3: PyArrow's S3 filesystem is more optimized than what we'd be able to write here and is already complete, so for S3 it's a clear win.
GCS: PyArrow doesn't have GCS bindings yet, but the C++ implementation is done and the Python bindings are in progress. So we should expect that soon 🤞.
ADLS: ADLS is held up in PyArrow because the C++ library uses C++ 14 while Arrow is strictly C++ 11, at least for another year or so 😞 . However, there is a community-supported fsspec implementation of ADLS, which could be combined with PyArrow's fsspec handler.

So if we went with this, we'd soon have 2 of 3 file systems already supported in the writer, and one that could be supported if another library is added as an optional dependency.
Related Issue(s)
For example:

closes #570

Documentation

Alternatively, maybe what we should do is create a FileSystem Rust object that can wrap any pyarrow.FileSystem object, and impl StorageBackend for that. You could even support any arbitrary fsspec filesystem through that too, and that's likely what Python users would want. Only hard thing there is making sure we don't get deadlocks with the GIL 😬

I think the interface we'd like to aim for is having filesystem parameters that can accept any pyarrow.FileSystem and fsspec filesystem. We don't want to use these as is, at least by default, since they don't support rename_obj_noreplace and that's required for concurrent writers. (Not all users may need this, but I think there's a strong case this should be supported by default.)
I think there's three different solutions we can implement:

For the main pyarrow.FileSystem ones (S3 and soon GCS), we should automatically map them internally to their associated StorageBackend, and use that for delta log interaction. The other benefit of using these two separate filesystems is that they can each operate without the Python GIL and thus use multiple threads.
For any other fsspec filesystem, we write a SafeDeltaFileSystem abstract mix-in that users could implement to add a rename_noreplace method and create a StorageBackend implementation that can wrap classes that use that mixin. This allows concurrent writers, but would be less performant than (1) since the GIL would preclude multi-threaded reads of the delta-log (though I honestly have no idea how noticeable that would be).
For users that don't care about concurrent writers, we could also implement a wrapper around fsspec filesystems that just used it's move method for rename_noreplace. We would obviously want to document the caveat, but I imagine there are use cases where users would be fine with that.

Personally I think your approach makes a lot of sense.
There may be however an opportunity to support files systems not supported by pyarrow out of the box without the need to have users provide a custom implementation. Once #573 lands  we should be able to write RecordBatches to the delta table using the storage backends from this crate. Obviously this will initially only support simple updates, but there are some experiments about (#523) using the datafusion query engine that could be extended to also support spark-like delta SQL syntax for more complex updates.
Haven't really thought about it yet, if this could be fully transparent to the python users, but thanks to arrow moving record batches across language boundaries should be very cheap :).

Alternatively, maybe what we should do is create a FileSystem Rust object that can wrap any pyarrow.FileSystem object, and impl StorageBackend for that.

This would require ser/de between rust bytes and python bytes across language runtime on every write isn't it? The current storage backend abstraction in Rust core is not very well designed because it requires reading the whole object into memory before passing the data down to the caller. Ideally, the abstraction should support streaming read so we can parse json/parquet in a streaming fashion. So this is another thing to consider in the future if we want to go this route.

Ideally, the abstraction should support streaming read so we can parse json/parquet in a streaming fashion.

@houqp I'm now thinking it might be better to simply work on improving the delta-rs filesystems and continue wrapping those in the Python module. For streaming reads and writes, is there a particular trait we should be implementing?
It's unclear to me what the parquet crate accepts. datafusion has an ObjectStore and ObjectReader traits, but that create is also an optional dependency.

For streaming reads and writes, is there a particular trait we should be implementing?

I think the ObjectStore trait in datafusion is a good one to adopt. I have been pushing datafusion to move the object store abstraction out of the core so we can reuse it in delta-rs ;)
async read/write has been added to the parquet2 create a long time ago. async read has been added to parquet-rs recently, see https://github.com/apache/arrow-rs/pull/1154. I have been working on parquet2 migration on and off at https://github.com/delta-io/delta-rs/pull/465. We likely will need to support both parquet crates with a feature flag for some time until we are confident that parquet2 can cover all our use-cases. It's supposed to be the fastest parquet implementation out there, so I am hoping that we will get a decent performance boost from the migration.

authors: houqp roeap wjones127

repo: delta-io/delta-rs


text:
PR submitters cannot label issues (Title changed for testing purposes)

People who submit PRs but do not have write access to the repository cannot set labels.
We want people to be able to add and remove labels even without write access.
Approach: install a webhook that scans incoming PRs, Issues, and comments on these for labelling commands in the form of [+LabelName] and [-LabelName] inside the message (or title).

Test comment added.

Test close #2

repo: delta-project-org/main


text:
[FeatureRequest] Pull files to be obfuscated by "last modified" attribute

Feature Description / 功能描述
Add an option to make Amarok obfuscate file names in the order they were last modified.
Reason / 理由
This would ensure files being able to be displayed in the order they were created or modified.
Especially useful for document / download / etc. folders where the most recent files are mostly those being looked for.
Currently Amarok will shuffle the affected folders and makes it hard to find files.
Suggest Solution / 实现方案
Maybe implement a function to pull files via FileTime lastModifiedTime() and obfuscate from oldest to newest per affected folder.

File obfuscation now preserves the original last modified time by default since v0.8.4a1.
Thank you for your feedback.

authors: TheRetenor deltazefiro

repo: deltazefiro/Amarok-Hider


text:
Link to all tool docs pages from docs.hyperdrive.box

agent0
hyperdrive-rs needs to be generated
sdk

is there another page for this??

any others?

There's a Developer Resources page in the docs. Could you all please help confirm that the links point where we want them to point to? @ryangoree @dpaiton @jalextowle
Also feel free to comment about any other tools we want to mention here, or suggest any changes to how we list/present these.
Thanks!

can you change the agent0 link to https://agent0.readthedocs.io/en/latest/index.html? that's our docs homepage

Done! You may not see it live for a bit due to (I think) caching issues, but the change was merged into Gitbook

I just discovered this page:
https://hyperdrive.box/build
It has boxes for each build library, maybe we can make those clickable? This is probably redundant with the suggestion above. I would suggest getting rid of /build and having docs.hyperdrive.box behave as above

authors: MazyGio dpaiton

repo: delvtech/hyperdrive


text:
Issue with "VirusTotal - Private API" in @site/docs/reference/integrations/virus-total---private-api.md

Describe the problem

Page: VirusTotal - Private API
Source: https://github.com/demisto/content/blob/master//Packs/VirusTotal-Private_API/Integrations/VirusTotal-Private_API/README.md

The parameter " Do not use by default " is not documented.

Screenshots

Environment

OS: [e.g. Windows]
Browser: [e.g. chrome, safari, firefox..]
Browser Version:

Suggested fix

Please explain the purpose of this setting and its effect(s).

Hi @sylvainmeras, Thanks for reporting this.
The parameter is generic for all Integrations. When checked it excludes the integration instance when running a generic command that uses all available integrations such as !file.
We have this documented in our main documentation site under Feed Integrations:
https://docs.paloaltonetworks.com/cortex/cortex-xsoar/6-0/cortex-xsoar-threat-intel-management-guide/threat-intelligence-feeds/feed-integrations
We will evaluate how to improve this information for all integrations.
@kirbles19 FYI

authors: glicht sylvainmeras

repo: demisto/content-docs


text:
🛑 StatPing is down

In 2ce8939, StatPing (https://stat.eka.monster) was down:

HTTP code: 0
Response time: 0 ms

Resolved: StatPing is back up in fce51f6 after 5 minutes.

repo: demonsteru/up


text:
I uh kind of turned your project into a template

First, I really want to say this is an amazing project. When I first saw your post on reddit my mind was completely blown. I had always wanted to write really rich terminal-user-interface apps but found the amount of state required really daunting. Seeing your combination of reagent, react, and blessed really lit a lightbulb in my mind 💡 This has to be made into a template!
After a few months of work I am nearing full release:
https://github.com/eccentric-j/cljs-tui-template
All that’s left is filling out the documentation.
Anyway, I wanted to make sure this is cool with you since it is very heavily based on your stack. I have credited you in both the README and the demo app it generates. I also made sure to gut all your application specific code so as not to duplicate what floki offers.
Features

Supports lein-figwheel, figwheel-main, or shadow-cljs
Provides a test runner and sample test
Supports nREPL ClojureScript REPL
All npm packages are managed directly through package.json

Please let me know if you feel this project impedes upon your work, we can discuss the next steps to make this right. Alternatively, I could setup some tests for floki having struggled to figure it out with this stack if it would be helpful to you.

Nice work! 👍
Thanks for the contribution!

You're welcome. Thanks again for getting it all started!
One last thing came to mind:

(defonce tty-fd
  (fs/openSync "/dev/tty" "r+"))

(defonce program
  (blessed/program #js {:input  (tty/ReadStream tty-fd)
                        :output (tty/WriteStream tty-fd)}))

(defonce screen
  (doto
    (blessed/screen #js {:program     program
                         :autoPadding true
                         :smartCSR    true
                         :title       "Floki"})
    keys/setup))

I noticed if you remove the program attribute from the screen it still works, but uses the current tty session. The advantage of this is that your program will automatically resize when the terminal is resized. Is there something I'm missing that would make creating the program worth it?

I think both of your work is based on my gist:
https://gist.github.com/polymeris/5e117676b79a505fe777df17f181ca2e
I think it's amazing what you have done with it -- but would you mind crediting me?

authors: denisidoro eccentric-j polymeris

repo: denisidoro/floki


text:
feat: Remove deprecated methods from FeatureChecker

Closes https://github.com/denoland/deno_core/issues/901

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81.50%. Comparing base (0c7f83e) to head (9151622).
Report is 120 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #905      +/-   ##
==========================================
+ Coverage   81.43%   81.50%   +0.06%
==========================================
  Files          97       98       +1
  Lines       23877    24788     +911
==========================================
+ Hits        19445    20203     +758
- Misses       4432     4585     +153

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

authors: bartlomieju codecov-commenter


text:
Relative import path "@std/fmt/colors" not prefixed (v1.13.0)

Our deployment pipeline started failing since v1.13.0 with the following error:
error: Relative import path "@std/fmt/colors" not prefixed with / or ./ or ../
    at https://deno.land/x/deploy@1.13.0/deployctl.ts:9:33

This error started when upgrading deployctl from v1.12.0 to v1.13.0.
Here's the command we use to deploy:
deno run -A https://deno.land/x/deploy/deployctl.ts deploy --project=[PROJECT_ID] --include='server/,dist/' server/entry.deno.js

We're using Deno 2.0.3
Note: using Deno 2.0.3 with deployctl@1.12.0 works as expected.

i was able to fix this by including deno.json

authors: azohra ianlet

repo: denoland/deployctl


text:
Make tslib optional

Currently, dnt adds tslib as a dependency to all packages even if the compiled code doesn't use it. I understand that determining if tslib is needed might be complicated, but maybe dnt could provide a way to opt out for the authors?

Thanks for the suggestion! This is released now and off by default to align with the default in the ts compiler.

authors: dsherret zandaqo

repo: denoland/dnt


text:
Error: Automatic mode Islands throwing error in the browser.

Hy fresh team, I'm using fresh@1.1.0 and using the new automatic mode. After this bump, following this documentation (remove all pragmas and add compilerOptions at deno.json), I noticed the islands are breaking in the browser.
The error in the browser:

The builded island file:

My import_map.json
{
  "imports": {
    ...
    "$fresh/": "https://deno.land/x/fresh@1.1.0/",
    "preact": "https://esm.sh/preact@10.10.6",
    "preact/": "https://esm.sh/preact@10.10.6/",
    "preact-render-to-string": "https://esm.sh/preact-render-to-string@5.2.3?deps=preact@10.10.6",
    "twind": "https://esm.sh/twind@0.16.17",
    "twind/": "https://esm.sh/twind@0.16.17/",
  }
}

deno.json
{
  "compilerOptions": {
    "jsx": "react-jsx",
    "jsxImportSource": "preact"
  },
  "importMap": "./import_map.json"
}

I got the problem. I'm using a custom manifest, that i build myself, and this manifest was missing this:
import config from "./deno.json" assert { type: "json" };
const manifest = {
  // ...
  config,
}

repo: denoland/fresh


text:
Dockerized setup for development

As discussed with @Vinzent03, this PR contains the setup that can be used by new developers to quickly get into this plugin - as all they need to have is Docker. Simply running scripts/restart-dev.sh will install all dependencies and run the compiler in watch-mode.
I've extracted this code from #288.
This PR is a draft, since it was decided to not merge this. Hence, this is here primarily for future reference.
If you want, we can move this into an issue instead.

I don't know docker that well and feel like nodejs is installed on almost every dev machine, so I don't really see the need as well.

repo: denolehov/obsidian-git