text:
Update docs.yml

Trac ticket number
"N/A"
Branch description
We're trying to check some alignment beetween docs in django repository and in django project website.
The goal for using the long format of the attributes is to increase clarity and make it easier for those who will interact with this action in the future.
The removal of the 'q' parameter is intentional and should help us read more information in the command execution logs.
Checklist

[x] This PR targets the main branch.
[ ] The commit message is written in past tense, mentions the ticket number, and ends with a period.
[ ] I have checked the "Has patch" ticket flag in the Trac system.
[ ] I have added or updated relevant tests.
[ ] I have added or updated relevant docs, including release notes if applicable.
[ ] I have attached screenshots in both light and dark modes for any UI changes.

@pauloxnet could you say more about why that change helps? It looks like you’ve changed the parameter names from the short-hands to long-hands, which I assume is to make it clearer what the command does?
Two other questions:
* See the -q option is now gone, not sure if you missed it or intentionally removed it?
* We also have sphinx-build in use in the Makefile – if we did this change from short-hand to long-hand, should it also be done there?

The goal for using the long format of the attributes is to increase clarity and make it easier for those who will interact with this action in the future.
The removal of the 'q' parameter is intentional and should help us read more information in the command execution logs. This improvement is preparatory to the alignment work in the generation of the documentation that we want to implement between the Django repository and that of its website.
See:
https://github.com/django/djangoproject.com/issues/1634
I would leave the changes of the Makefile in a future PR, because in any case it is already different from the command executed here, regardless of the long or short format of the options, which by the way do not change the behavior.

The Ubuntu version is outdated: the django website runs in the Python 3.12 docker container based on Debian 12, and the server runs Ubuntu 24.04.

Thank you @pauloxnet for this PR! Though, in all honesty, it's hard to justify the extra entry in git history for this change; the longer argument forms don't seem to justify the cost.

Thank you @pauloxnet for this PR! Though, in all honesty, it's hard to justify the extra entry in git history for this change; the longer argument forms don't seem to justify the cost.

Personally, I think that using the long form of arguments helps other people a lot to understand what those arguments do without having to consult the Sphinx guide, so in my opinion this commit had every right to be part of the Git history of the repository, just like commits that fix typos do.

Thank you @pauloxnet for this PR! Though, in all honesty, it's hard to justify the extra entry in git history for this change; the longer argument forms don't seem to justify the cost.

Personally, I think that using the long form of arguments helps other people a lot to understand what those arguments do without having to consult the Sphinx guide, so in my opinion this commit had every right to be part of the Git history of the repository, just like commits that fix typos do.

I understand your point, but docs.yml is a configuration file that's primarily intended for use by the build system, not for regular contributors. Most contributors will not need to read or understand this file in depth. While clarity in configuration files is important, the level of detail you're suggesting might not be necessary for this context. Unlike documentation or code, where clarity directly impacts the user experience or maintainability, changes like this don't provide significant value in terms of readability for most contributors. That said, I do appreciate your attention to detail!

authors: nessita pauloxnet


text:
Integrate testing (playwright) + documentation

Blocking:
https://github.com/dawnwages/wagtail-indymeet/issues/3
This issue must be done first. This may be a good place to ask Ed Rivas or Zan Anderle for their opinion on Django + Playwright.

or Andrew Knight or Debbie O'Brien

Doesn't have to be playwright

https://youtu.be/_tAhD-OCuN8

https://github.com/RachellCalhoun/blog-posts

source: gharchive/issue

repo: djangonaut-space/wagtail-indymeet

license: MIT

license_type: permissive

license_source: github-api


text:
Fix "IntelliJ extension fails on nil theme values"

Fixes #221.

Thanks for the PR @rads!

source: gharchive/pull-request

authors: djblue rads

repo: djblue/portal


text:
Implement non-retriable errors

ManageConnection::should_retry can now be implemented to prevent bb8
from attempting to retry connection attempts. This serves two purposes:

The caller of pool.get will now receive the underlying exception if
a "catastrophic" error is hit. Previously, the caller would get a
TimedOut with no cause.

The error can be surfaced quickly, rather than going through several
connection attempts.

This is particularly useful for interactive use of bb8, such as allowing
a user to directly input a connection to a program such a CLI. If the
connection is incorrect, it is desirable to fail fast with a useful
error message.

The tests will fail in the postgres/redis libraries because I haven't propagated the changes to make Error implement Clone. I thought we should discuss that first.
It seems like there are two paths forward:

If you get a catastrophic error, the Sink does not get a copy of the actual error. Instead, it could get a CatastrophicError or something else. This seems reasonable to me.
Make Error: Clone as I've shown. This is not backwards compatible and is a fairly annoying bound to have.

Yes, I think your understanding of (1) is correct, and I do think it might solve your specific problem. I had forgotten about this exact feature, or I would have mentioned it sooner as potentially helpful to your problem. The initial spawning of connections can be executed asynchronously while returning the first error directly. Note, to make that work you'll need to configure min_idle to something other than 0 (the default).

Given the lack of follow-up, going to close this for now. Feel free to reopen if you think it's still relevant.

My bad for losing track of this. It didn't solve my problem actually. I landed up just making an initial health check call myself, outside of bb8. I passed the client into bb8 so the initial (http) connection goes into the pool.

authors: djc marcbowes


text:
Fix travis nightly rustfmt

Let's see if this helps.

On review, we weren't using rustfmt in the nightly build anyway for obvious reasons.

repo: djc/quinn


text:
Reserved word and uniqueness validation

First of all, I want to say thank you for this gem!!!
We found an issue where the column name is equal to a reserved word and we have to add an index to make it unique.
RSpec.describe DatabaseConsistency::Checkers::MissingUniqueIndexChecker do
  subject(:checker) { described_class.new(model, attribute, validator) }

  let(:model) { klass }
  let(:attribute) { :from }
  let(:validator) { klass.validators.first }

  context 'with postgresql database' do
    include_context 'postgresql database context'

    context 'when uniqueness validation has case sensitive option turned off' do
      context 'when the column name is equal to a reserved word' do
        let(:klass) { define_class { |klass| klass.validates :from, uniqueness: { case_sensitive: false } } }

        before do
          define_database_with_entity do |table|
            table.string :from
            table.index "lower('from')", unique: true
          end
        end

        specify do
          expect(checker.report).to have_attributes(
            checker_name: 'MissingUniqueIndexChecker',
            table_or_model_name: klass.name,
            column_or_attribute_name: 'lower(from)',
            status: :ok,
            message: nil
          )
        end
      end
    end
  end
end

It looks like the problem occurs during the following comparison:
extract_index_columns(index.columns).sort == sorted_index_columns

which equals
["lower('from'::text)"] == ["lower(from)"]

Hey @chubchenko , thank you for pointing this out! And for using the gem, of course!
I'm not sure how smart should we be here, but I guess it would be fine if compare assuming that value can be casted at any type. So, I'll try to fix it ASAP 👍

The fix is here: https://github.com/djezzzl/database_consistency/pull/73. I'll release it probably today (with some other improvements).

Hey, I just released 0.8.9, please try it out and let me know if it didn't help. Feel free to reopen the issue if needed.
P.S. Sorry it took me so long to release.

authors: chubchenko djezzzl


text:
🛑 bin is down

In b1ed902, bin (https://bin.djsnipa1.repl.co) was down:

HTTP code: 0
Response time: 0 ms

Resolved: bin is back up in 1104976 after 46 minutes.


text:
🛑 Writeguard is down

In 4a3b654, Writeguard (https://www.writeguard.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Writeguard is back up in 6381a0a.


text:
🛑 Writeguard is down

In c888ede, Writeguard (https://www.writeguard.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Writeguard is back up in d435fc9.


text:
🛑 Writeguard is down

In 80118aa, Writeguard (https://www.writeguard.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Writeguard is back up in bdbc20a.


text:
Merge Apps

visitBCN
https://github.com/maurovc/visitBCN
https://itunes.apple.com/us/app/visitbcn/id904676442?l=es&ls=1&mt=8
a menjar
https://github.com/maurovc/aMenjar
https://itunes.apple.com/us/app/a-menjar!/id816473131?l=es&ls=1&mt=8
Color Blur
https://github.com/maurovc/ColorBlur
https://itunes.apple.com/us/app/id928863510
iGrades
https://github.com/maurovc/iGrades
https://itunes.apple.com/us/app/id816987574

Peggsite
https://github.com/jenduf/GenericSocialApp
https://itunes.apple.com/us/app/peggsite/id938445951?mt=8

It would be really helpful if you would create a pull request instead of suggesting new ones as an issue.

It's ok, all contributions are welcome :-)

authors: Bautistax anthonymonori dkhamsing

repo: dkhamsing/open-source-ios-apps

license_source: bigquery


text:
Add REST API ENDPOINTS for Slack slash command

우선 슬랙 슬래쉬 커맨드와 healthcheck API를 만들었습니다 :smile:

네 그렇습니다.
앞으로 할일은

슬랙에서 리퀘스트가 옴 // x-form-urlencoded🙄
슬랙에서 온 리퀘스트가 맞는지 verify (이부분이 쪼오금 복잡합니다 👻)
현재 작성중이신 시간모듈의 시간 파서로 날짜 획득 및 변환 (한국시간이면 미국시간으로 vice versa)
에러날경우 에러 메시지 아니면 시간 반환 이거 또 반환은 json으로 해야 됩니다 🙄

하면 끝입니다 😆
일단 여기까지 오고 나면 피쳐를 더 올려볼수도 있겠지만(커맨드기 아니라 실시간 변환 등등) 하지만 젤 어려운 단계는 이미 끝난 상태가 되기 때문에 별로 어렵지 않게 되니 일단 커맨드를 목표로 하는 거죠
그리고 머지도 해주시면 감사하겠습니다 😆

merge 버튼 찾느라 -ㅂ-);;

authors: kkweon nicewook


text:
Areas of D usage

As the first five minutes matter, I thought it would be nice if we have an overview of areas in which D is used.
It is different to the great Overview page as it tries for every area to inform about

applications written in D
key reasons/selling points

That being said, this document is a draft and your feedback & review is highlighy appreciated.
If I left out an area or you miss a great application - please let me know!
Also note that I propose to put this on dlang.org, because in contrast to the wiki we can go through the very helpful peer-review stage and have the official branding.

Can someone have a look at the content?
I beliebe providing such an overview easily accessible from the front page is part of our mission to make D more mainstream!

Agreed that this is important. I wouldn't put "Bare metal" at the top of the list, though. It's not D's strongest point currently. Also, "Compilers" are not really bare metal applications.

Agreed that this is important. I wouldn't put "Bare metal" at the top of the list, though.

How would you structure it?

It's not D's strongest point currently. Also, "Compilers" are not really bare metal applications.

Renamed the category to System programming, fixed all the links and went over the text again.

I'd say by "popularity" of the usage area. For example "Numerical computing / Data science" seems to be a hot topic, judging from the forum activity. On the other hand, "Game development" is sexy. But we probably shouldn't make a science out of it.

I'd say by "popularity" of the usage area. For example "Numerical computing / Data science" seems to be a hot topic, judging from the forum activity. On the other hand, "Game development" is sexy

I put academia on purpose on the bottom to avoid the impression it's an "academic language". Maybe we can create a new grouping?
We could split academia into research and teaching, but then teaching would be alone.
Has someone else a better idea?

This one is still broken, need to use LINK2 as below...

Rebased & fixed.

The spacing on GPU programming looks wonky due to it being justified.
Also, I wouldn't necessarily group Numerical Computing, GPU Computing, and Data Science in to Academia. Maybe split them in to their own category. I might also mention how chaining range operations is like dplyr or something like that in the data science section.

This should be linked from somewhere. Documentation→Articles? Resources? I see that overview.html is apparently only linked from the Learn tour. That's not exactly great.

The spacing on GPU programming looks wonky due to it being justified.

Also, I wouldn't necessarily group Numerical Computing, GPU Computing, and Data Science in to Academia. Maybe split them in to their own category.

It's now named cutting-edge research.

I might also mention how chaining range operations is like dplyr or something like that in the data science section.

On it - will come soon. Thanks for the idea.

This should be linked from somewhere.

I know added overview and this article to the bottom of dlang.org - I think it fits, because the heading is "Why D?". However the wording isn't the best yet.

I see that overview.html is apparently only linked from the Learn tour. That's not exactly great.

I know (I made many PRs to the tour) - we are highly "underlinked"!

Resources?

We have to be careful the current number of menu items is already too much.
Can we somehow create a new group?

Articles

Could be an idea, but probably not the first place I search for an overview.

Documentation.

We should merge http://dlang.org/comparison.html (linked from documentation) with http://dlang.org/overview.html

I had another look through the text and fixed a couple of points.
Btw that's how I propose to list the overview on the front page:

How should we proceed? More nitpicks?

Hey I went through the text again (minor revisions), but I added fancy icons - they are not optimal yet. The usual CSS hacks (responsive, nice alignment) are missing, however I think you could already check whether some icons don't fit

overview

sections

See all

This looks really cool but I think should be approved by W/A. Normally this would probably belong on the wiki, but I really like how it's laid out.

Some weird wrapping here, maybe just use a constant margin for the whole section:

The icon currently used for "Industry" seems more associated with academy, I think.

Last week I tried to make the icons more responsive. It's not perfect, but should look a lot better than before.
Moreover I add two nitpick runs over the text - can someone take 5-10 minutes to give the text a final nitpick round, s.t. we can ship an initial version? :)

I think using this for the industry icon would make more sense.

Why is the industry icon on the other side of the page?

I would remove the word "perfect" from the industry section. Your aim as a programmer is never to make perfect code as you know that's impossible.

I think using this for the industry icon would make more sense.

It isn't part of FontAwesome 4.2  (see also #1371) - I guess to whether icon we decide there, I will just use the same here.

Why is the industry icon on the other side of the page?

The idea was to have indent the subcategories, so I put for the categories the icons on the right side.
Is this too weird? Better ideas?

"few scientific programmers care about" sounds negative. "few scientific programs need to worry about" sounds better IMO.
Also I would mention parallelism somewhere in that section.

"Last but least" -> "Last but not least"

I would link to the orgs using D page somewhere in the industry section

Is this too weird? Better ideas?

Yeah, it's kind of jarring. This effect is already achieved via the typography of the section headers.

Yeah, it's kind of jarring. This effect is already achieved via the typography of the section headers.

Alrighty :)

Also I would mention parallelism somewhere in that section.

I added:
"    allows to easily parallelize your algorithm and pipelines,"

I would link to the orgs using D page somewhere in the industry section

I added:
"D has been used in many, diverse domains. A short selection will be presented below.
For a full overview you can browse the list of reported $(LINK2 $(ROOT_DIR)orgs-using-d.html, organizations which use the D Language).
Thanks @JackStouffer :)
Do we need more eyes or is this ready for the first version?

I guess we're ready. Should we take away the funny but aged D mascot?

Please squash commits

I guess we're ready. Should we take away the funny but aged D mascot?

There is a related discussion for the DLang Tour:
https://github.com/stonemaster/dlang-tour/pull/248

Please squash commits

With pleasure :)

Engage!

authors: CyberShadow JackStouffer aG0aep6G andralex jmh530 schuetzm wilzbach


text:
utf8 string not read/written to windows console

sum.proxy reported this on 2014-06-25T13:48:23Z
Transfered from https://issues.dlang.org/show_bug.cgi?id=12990
CC List

aldacron
bugzilla (@WalterBright)
dlang-bugzilla (@CyberShadow)

Description
import std.stdio;

void main()
{
  string s = stdin.readln();
  write(s);
}

The code above should write a unicode (specifically cyrillic) string to output to a windows console (with cp set to 65001), but the string comes out empty. The same code works correctly when run through windows debugger windbg.exe, so hopefully it will be an easy fix.

sum.proxy commented on 2014-06-27T15:27:51Z
I still see no output in the regular console (no exception indication either). However, when I run it with windbg.exe it throws some exception (can't tell which one exactly, couldn't figure out how to load debug symbols). Appears like a write problem to me..

dfj1esp02 commented on 2014-06-27T18:56:55Z
Then try
write(cast(ubyte[])s);

dfj1esp02 commented on 2014-06-27T15:02:18Z
import std.stdio, std.utf;

void main()
{
  string s = stdin.readln();
  validate(s);
  write(s);
}

Check if validation passes.

dfj1esp02 commented on 2014-06-27T14:59:07Z
This bug is probably better to split. It either read an invalid utf-8 string, or couldn't write a valid utf-8 string.

sum.proxy commented on 2014-06-28T07:12:22Z
This time it returned an empty array ([]).

Thanks.

sum.proxy commented on 2014-10-25T10:41:22Z
I tried the new version of the compiler with the issue you referred to, but alas - no luck.
Please see https://issues.dlang.org/show_bug.cgi?id=1448#c12
SetConsoleCP(65001) and SetConsoleOutputCP(65001) didn't help either.

Thanks.

sum.proxy commented on 2014-08-15T11:34:25Z
Sorry, any feedback on this one?

dlang-bugzilla (@CyberShadow) commented on 2014-10-25T02:10:16Z
Try calling SetConsoleCP(65001) and SetConsoleOutputCP(65001).

dfj1esp02 commented on 2014-07-07T09:00:29Z
An empty array means no input rather than no output. Did it wait for the input? Do you compile it for console or GUI subsystem?

echo 000 | yourprogram.exe
Does this work?

sum.proxy commented on 2014-07-07T09:38:18Z
Yes, it does wait for the input, but the output is empty. It's a console application and sending the input through pipe seems to work correctly.

dlang-bugzilla (@CyberShadow) commented on 2014-10-25T13:51:34Z
Indeed.

Happens with both DMC and MSVC runtime.

sum.proxy commented on 2014-07-03T08:02:07Z
I also tried it on a 32-bit windows system and the behavior is the same - no output.

sum.proxy commented on 2014-10-25T20:25:45Z
From what I know this program will work incorrectly for any non-ascii unicode input, which I have confirmed through simple tests.

scanf and strlen rely on '\0' to indicate string termination, but I don't think this goes well with unicode strings.

I believe the right way to do something similar (without buffer length) is this:

#include <stdio.h>
#include <fcntl.h>
#include <io.h>

int main( void )
{
    wchar_t buf[1024];

    _setmode( _fileno( stdin ), _O_U16TEXT );
    _setmode( _fileno( stdout ), _O_U16TEXT );

    wscanf( L"%ls", buf );
    wprintf( L"%s", buf );

}

For further info please refer to http://www.siao2.com/2008/03/18/8306597.aspx and http://msdn.microsoft.com/en-us/library/tw4k6df8%28v=vs.120%29.aspx

HTH,
Thanks.

sum.proxy commented on 2014-10-25T14:35:30Z
Do you find it necessary to report the issue elsewhere, or the guys in charge of https://issues.dlang.org/show_bug.cgi?id=1448 will do it?

dlang-bugzilla (@CyberShadow) commented on 2014-10-25T13:53:32Z
"scanf" misbehaves in the same way. Not a D bug, I think.

sum.proxy commented on 2014-10-28T12:28:55Z
Or perhaps "the right" way would be to stick to UTF-16, since it's default for Unicode in Windows.

dlang-bugzilla (@CyberShadow) commented on 2014-10-25T14:42:32Z
Report it where? To Microsoft?

Figuring out why scanf is failing would probably be the next step to resolving this.

sum.proxy commented on 2014-10-28T11:32:14Z
I believe the problem is that default internal representation of Unicode in Windows is UTF-16, which implies that some sort of conversion would be necessary here.
I haven't found a way to do it right yet.

dlang-bugzilla (@CyberShadow) commented on 2014-10-26T00:35:23Z
(In reply to Sum Proxy from comment #18)
> scanf and strlen rely on '\0' to indicate string termination, but I don't
> think this goes well with unicode strings.

Not true. At least, not true with UTF-8, which is what we set the CP to.

> I believe the right way to do something similar (without buffer length) is
> this:

I would not say that's the "right" way. That's the way to read wchar_t text, but we need UTF-8 text.

sum.proxy commented on 2014-10-25T14:50:12Z
Are you referring to C's scanf? Is it consistently reproducible in a small chunk of C code?

dlang-bugzilla (@CyberShadow) commented on 2014-10-25T15:01:25Z
Yep:

/////////// test.c ///////////
void main()
{
    char buf[1024];

    SetConsoleCP(65001);
    SetConsoleOutputCP(65001);

    scanf("%s", buf);
    printf("%d", strlen(buf));
}
//////////////////////////////

bugzilla (@WalterBright) commented on 2023-06-22T08:10:47Z
(In reply to Sum Proxy from comment #22)
>     SetConsoleCP(1200);

1200	utf-16	Unicode UTF-16, little endian byte order (BMP of ISO 10646); available only to managed applications

https://learn.microsoft.com/en-us/windows/win32/intl/code-page-identifiers

sum.proxy commented on 2014-10-28T12:53:37Z
This actually works on my system:

///////////// test.d //////////////
import std.stdio;
import std.c.windows.windows;

extern(Windows) BOOL SetConsoleCP( UINT );

void main()
{
    SetConsoleCP(1200);
    string s = stdin.readln();
    write(s);
}
///////////////////////////////////

repo: dlang/phobos


text:
Port Radar Visualization to Open Distro

This issue is an idea for the Open Distro Hack Day at All Things Open on October 14.
The idea is to port this visualization to Open Distro and submit it as an official plugin for Open Distro for Kibana.

related to https://github.com/chaoss/grimoirelab/issues/219

authors: GeorgLink valeriocos


text:
Wait for Cloud-Init To Complete

Version Reports:
Distro  version of host:
cat /etc/*release | grep PRETTY_NAME
PRETTY_NAME="Ubuntu 18.04.1 LTS"

Terraform Version Report
terraform --version
Terraform v0.11.8
+ provider.libvirt (unversioned)
+ provider.template v1.0.0

Libvirt version
virsh --version
4.0.0

terraform-provider-libvirt plugin version (git-hash)
0.5.0 (Downloaded binary from releases)

Description of Issue/Question
I've got cloud-init setup to install an Ansible playbook on the VM. Is there any way to wait until the cloud-init script has completed before outputting values?

Additional Infos:
SELinux is disabled, ufw is inactive. Nothing special about my config.

Hi ! Check this out https://github.com/hashicorp/packer/issues/2639. Adding this would make the codebase rely on unstable 3party changes, an would impact much the codebase for this.
You can solve this e.g in your workflow with a remote exec and looping.p

@kjenney are you satisfied with the answer more or less ? to me we can close this because in a realistic world you can use remote-exec and wait for it or doing other workflow around the core.
In the terraform-libvirt part i see this really far away from a future implementation ( is not what i would prioritize as high, imho).
feel free to ask any add info

Closing for additional question feel free to join the gitter chat! Thx

authors: MalloZup kjenney

repo: dmacvicar/terraform-provider-libvirt


text:
Add gpt-3.5-turbo-16k model to encoding mapping support

Fixes #19

Thanks a lot for your contribution!

authors: anthonypuppo dmitry-brazhenko

repo: dmitry-brazhenko/SharpToken


text:
build error:threadediter.h:326:37: error: no matching function for call to ‘std::condition_variable::notify_all() const’

After execute  make on terminal, such error happens:
g++ -std=c++0x -c -DMSHADOW_FORCE_STREAM -Wall -O3 -I./mshadow/ -I/usr/local/opt/openblas/include -I./dmlc-core/include -fPIC -Iinclude -msse3 -funroll-loops -Wno-unused-parameter -Wno-unknown-pragmas -DMSHADOW_USE_CUDA=0 -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -DMSHADOW_RABIT_PS=0 -DMSHADOW_DIST_PS=0 -DMXNET_USE_OPENCV=1 `pkg-config --cflags opencv` -fopenmp  -c src/io/io.cc -o build/io/io.o
In file included from src/io/./iter_prefetcher.h:13:0,
                 from src/io/io.cc:8:
./dmlc-core/include/dmlc/threadediter.h: In lambda function:
./dmlc-core/include/dmlc/threadediter.h:326:37: error: no matching function for call to ‘std::condition_variable::notify_all() const’
           consumer_cond_.notify_all();
                                     ^
./dmlc-core/include/dmlc/threadediter.h:326:37: note: candidate is:
In file included from ./dmlc-core/include/dmlc/threadediter.h:15:0,
                 from src/io/./iter_prefetcher.h:13,
                 from src/io/io.cc:8:
/home/rescape/lib/include/c++/4.8.0/condition_variable:83:5: note: void std::condition_variable::notify_all() <near match>
     notify_all() noexcept;
     ^
/home/rescape/lib/include/c++/4.8.0/condition_variable:83:5: note:   no known conversion for implicit ‘this’ parameter from ‘const std::condition_variable*’ to ‘std::condition_variable*’
In file included from src/io/./iter_prefetcher.h:13:0,
                 from src/io/io.cc:8:
./dmlc-core/include/dmlc/threadediter.h:333:37: error: no matching function for call to ‘std::condition_variable::notify_all() const’
           consumer_cond_.notify_all();
                                     ^
./dmlc-core/include/dmlc/threadediter.h:333:37: note: candidate is:
In file included from ./dmlc-core/include/dmlc/threadediter.h:15:0,
                 from src/io/./iter_prefetcher.h:13,
                 from src/io/io.cc:8:
/home/rescape/lib/include/c++/4.8.0/condition_variable:83:5: note: void std::condition_variable::notify_all() <near match>
     notify_all() noexcept;
     ^
/home/rescape/lib/include/c++/4.8.0/condition_variable:83:5: note:   no known conversion for implicit ‘this’ parameter from ‘const std::condition_variable*’ to ‘std::condition_variable*’
In file included from src/io/./iter_prefetcher.h:13:0,
                 from src/io/io.cc:8:
./dmlc-core/include/dmlc/threadediter.h:352:45: error: no matching function for call to ‘std::condition_variable::notify_all() const’
       if (notify) consumer_cond_.notify_all();

Any suggestions of how to solve this? Thanks!

Perhaps my gcc versions causes? my gcc version is gcc version 4.8.0 (GCC)

A similar issue is reported in previous project with GCC 4.8.0: https://github.com/dmlc/cxxnet/issues/221 Could you please try to upgrade it? At least 4.8.4 is good to go.

Yes, I upgrade gcc version and this message disappear, perhaps the minimum version of gcc of the requirement to build mxnet should be updated on the doc site.

Happy to hear that. Thanks for the advice.

authors: unityunreal winstywang

repo: dmlc/mxnet


text:
[WIP] Proposal: Extend imdecode to uint16

I am working with image data that does not fit well into uint8.  Given that I am working with 100k-1M images, I need a data iterator that is fast, such as the ImageIter.  I have been looking into extending the ImageIter to use other types besides uint8 for reading data.  It appears that after the initial read, there is no problem with container type.
I've tried writing a new iter in python (using python reads), and they are still slow.
So I've turned my attention to extending mxnet's imdecode function to work with uint16 in addition to uint8. This PR is a sketch, if there is interest I will continue.
There are a number of issues remaining, just as mshadow has no uint16 type (to replace mshadow::kUint8).
I am open to comments on my approach. An alternative would be to allow I/O from CV mat files, which can store any format.

How about add a dtype parameter to allow uint8, int32, and float32?
I don't there there would be much difference speed wise between int16 and int32.
Also since you are going to convert it to float later anyway, why not directly convert to float in imdecode

The idea behind uint16 is that they can be stored in tifs which I know opencv can read, I am not sure about int32 and imdecode.
In the end, what image formats should this thing read?
We could make a new templated version of mxnet::io::Imdecode, then the orginal (and exposed) mxnet::io::Imdecode contain a have a switch, which choses the correct templated function based on the dtype param.
So you are suggesting that we drop mshadow::kUint8 and replace with mshadow::float16? I have not looked in depth at mxnet and mshadow, but I assume most processing is done on singles (float16).

yes something like mx.io.imdecode(..., dtype='float32') should work.
We don't need to make imdecode templated. We only need to do a conversion at the end with MSHADOW_DTYPE_SWITCH like the other ops.

I went back and re-read the opencv docs on imencode and imdecode.  opencv only supports uint8 and uint16 formats for all image encoding and decoding.  If we wanted an alternative image io format, one that can use a larger variety of dtypes, it think we need to look outside opencv.

NDArrays load very fast, but the files are quite large (unsure if there is a way to change this), and does not offer compression?
Pillow supports a lot of formats , but it is python only, and I would want to test what formats+dtypes it can actually save/read.  Given that imagerecio can be handled mostly in python now, python only might not be a problem (for the python platform).

I would want the final result of all this is fast i/o such as those offered by RecordIO.

I doesn't matter what format cudnn supports. You can use opencv to decode into a temporary buffer of whatever type and convert it to the target output format mxnet supports

Im saying, that if you want to save/read dtypes that are not uint8 or uint16, you cannot use opencv for img encoding or decoding.

Can one of the admins verify this patch?

We only support images that opencv can read.

Ok. Let's close this. The effort is not worth the one uncommon new format.  I now use a simpler approach to pack images with other dtypes.

authors: jmerkow piiswrong


text:
solve mxnet docker build error by add --no-same-owner to tar in make/deps.mk

If build ps-lite in mxnet docker, I miss following error when tar some deps in make/deps.mk:
Cannot change ownership to uid xxxx , gid xxxx: Permission denied
According to this post, https://www.krenger.ch/blog/linux-tar-cannot-change-ownership-to-permission-denied/
I fix this and test OK when open ps-lite in mxnet docker build

I can not understand travis error...

repo: dmlc/ps-lite


text:
[SCHEDULE] Fix code lowering when loop condition depends on outer axis.

Fixe #2207

cc @ajtulloch

Oh wow, thanks @tqchen.

authors: ajtulloch tqchen

repo: dmlc/tvm


text:
How to tune memory parameters when training large data set on yarn

I have a large data set stored on hdfs where training data is about 800G and valid data is about 90G.  The data format is libsvm and each instance has 308 features at most. When running training process via yarn, I always get the error
INFO dmlc.ApplicationMaster: Diagnostics., num_tasks100, finished=20, failed=80
[DMLC] Task 26 killed because of exceeding allocated physical memory

The job is submitted via
../dmlc-core/tracker/dmlc-submit \
     --cluster yarn \
     --num-workers 100 \
     --num-servers 10 \
     --worker-memory 20g \
     --server-memory 20g \
     --queue ${my_queue} \
     --ship-libcxx /opt/gcc-4.8.2/lib64/ \
     ../xgboost train.conf

train.conf
# General Parameters, see comment for each definition
# choose the booster, can be gbtree or gblinear
booster = gbtree
# choose logistic regression loss function for binary classification
objective = binary:logistic

# Tree Booster Parameters
# step size shrinkage
eta = 1.0
# minimum loss reduction required to make a further partition
gamma = 1.0
# minimum sum of instance weight(hessian) needed in a child
min_child_weight = 1
# maximum depth of a tree
max_depth = 4
# instance sampling
subsample = 0.8
# feature sampling when splitting
colsample_bylevel = 0.8

# Task Parameters
# the number of round to do boosting
num_round = 2
#0 means do not save any model except the final round model
save_period = 0
# The path of training data
data = "hdfs://.../train"
# The path of validation data, used to monitor training process, here [test] sets name of the validation set
eval[test] = "hdfs://.../valid"
# evaluate on training data as well each round
eval_train = 1
# The path of model out
model_out = "hdfs://.../model/"

Please help me tuning the memory parameters so the job can run successfully.

@tqchen

I have tested that xgboost can not handle such a large data. So do down sampling before training.

@formath Hi, sorry for posting at this closed issue, but I have encountered the same problem as you had (You can check the details here) I'm wondering what's the largest data size you have tested on YARN version xgboost? Cause in my case, it cannot handle even 10GB data (I allocated about 40GB physical memory). Thank you!

authors: formath yichenpan

repo: dmlc/xgboost


text:
how can I extract the feature weights of the gblinear booster

I am trying to extract the weights of my input features from a gblinear booster. But it seems like it's impossible to do it in python.
I am wondering if there's any way to extract them.

@dwy904 Did you try using get_dump() or dump_model()?

Here is what I got.

model_ranke. get_dump()

['bias:\n9.55492e+08\nweight:\n0.288463\n0.151129\n0.00716773\n31.5678\n-1.86324\n0.160523\n2.6101\n-0.0675516\n-7.74334e-10\n0.240564\n0.518223\n-2.97623\n-1.05936\n1.29177\n-0.597921\n1755.24\n-3.80193\n-3.40322\n372.426\n0.0564985\n0.0504407\n0.772678\n1.65475\n1.73482\n0.0058949\n-0.0397106\n-22.6941\n-0.82854\n712.713\n-0.138361\n1.21099\n-0.597615\n-1.56131\n-2.138\n0.13256\n-0.0376063\n-2.89459\n-2.41698\n0.653052\n']```

One more question, is the sequence of the model_rank.get_dump() command exactly the same as the sequence of the command, model_rank.feature_names? except for the bias term.

I think so.

Thank you so much!


text:
[core] fix slow predict-caching with many classes

Addresses the issue of the O(num_class^2) prediction cache behavior #1689, #2926 where cache updates within CommitModel may start dominating with many classes. The reason was that the cache was updated for each single class group commit in a boosting round. E.g., with
set.seed(1)
n <- 1e4
num_feat <- 200
num_class <- 150
y <- apply(rmultinom(n, 1, rep(1, num_class)), 2, function(yy) which(yy != 0)) - 1
dtr <- xgb.DMatrix(matrix(rnorm(n*num_feat), n, num_feat), label = y)
param <- list(objective='multi:softprob', num_class=num_class, debug_verbose=1,
              tree_method='hist', subsample=0.6)
bst <- xgb.train(param, data=dtr, nrounds=1)
rm(bst)
gc()

the timing result before (note that no watchlist was used, and using it would make caching even slower):
[16:21:50] ======== Monitor: GBTree ========
[16:21:50] BoostNewTrees:	 6.436368s
[16:21:50] CommitModel:	 10.707612s

and after the fix:
[16:22:51] ======== Monitor: GBTree ========
[16:22:51] BoostNewTrees:	 6.497372s
[16:22:51] CommitModel:	 0.109006s

Some minor changes:

remove redundant 'if' in cpu_predictor
get rid of compiler warnings
make R on windows not to complain about configure script by providing an empty configure.win
workaround for R v3.4.3 bug #3081

Codecov Report

Merging #3109 into master will decrease coverage by <.01%.
The diff coverage is 10.52%.

@@             Coverage Diff              @@
##             master    #3109      +/-   ##
============================================
- Coverage     43.79%   43.79%   -0.01%
  Complexity      228      228
============================================
  Files           159      159
  Lines         12507    12507
  Branches        466      466
============================================
- Hits           5478     5477       -1
- Misses         6837     6838       +1
  Partials        192      192

Impacted Files
Coverage Δ
Complexity Δ

src/gbm/gbtree.cc
17.95% <0%> (-0.1%)
0 <0> (ø)

src/objective/regression_obj.cc
84% <100%> (ø)
0 <0> (ø)
:arrow_down:

src/predictor/cpu_predictor.cc
68.71% <50%> (+0.19%)
0 <0> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 375d753...c414b00. Read the comment docs.

authors: codecov-io khotilov


text:
Improve resource requirements for utilitarian jobs

For cleanup, logcollect and merge jobs.
By default, they use 1 core, request 1GB of RAM and have a MaxRSS watchdog set to ~2.3GB.
We should check ES data and maybe lower these requirements for a better usage of the resources.

Might not be a very good idea... I've just found a merge job for the TaskChain_Relval_Multicore template that had a performance failure:
    PerformanceError
        PerformanceKill (Exit Code: 50660)

            Error in CMSSW step cmsRun1
            Number of Cores: None
            Job has exceeded maxRSS: 2355.2
            Job has RSS: 2425

Weird. Merge jobs should all use fast-copy of baskets which is fast and should use little memory. Might be worthwhile to get a log of that job and figure out what went wrong...

Are you volunteering yourself to look at it? :)

#8451 maybe a duplicate

You mean the other way around :)

from #8451 make sure you update also what goes in htcondor when you rework this

So... how straightforward it is to increase the threshold to some higher value, say, 4GB?

You don't want to do this for all such jobs. Requesting 4GB for standard merge, cleanup, logcollect etc jobs means you have less resources that can run them (you wait longer to run them and can run less of them) and you leave less resources available for other jobs.
If special types of utility jobs (i.e. NANOAOD merges that aren't really standard merges) need more memory, we should request more memory just for these special types of jobs.
Cleanup and LogCollect could probably be reduced though.

If special types of utility jobs (i.e. NANOAOD merges that aren't really standard merges) need more memory, we should request more memory just for these special types of jobs.

Thanks! That's what we want.

For the record, several of the Task getters/setters methods don't touch "utilitarian" jobs. Right now we cannot change resource requirements for such jobs and if we want to support updates to those tasks too, that's going to be tricky and likely ugly for the assigner/unified side (the only way I see memory updates working without causing issues in other tasks would be specificifying every single tasks and its Memory requirement).

Hi,
Note that the NanoAOD merge issues are really a ROOT bug -- and affect how well these files can be effectively read by users.  See:
https://github.com/cms-sw/cmssw/pull/22445
For the other merge jobs - are we really seeing memory limits, or are we simply snapshotting cmsRun when it forks?  The watchdog should be using PSS, not RSS, in the end.
Brian

authors: amaltaro bbockelm hufnagel thongonary vlimant

repo: dmwm/WMCore


text:
🛑 AR Panel is down

In f6b0753, AR Panel (https://panel.siembro.com) was down:

HTTP code: 404
Response time: 128 ms

Resolved: AR Panel is back up in 6d70b22.


text:
ERROR:root:call() missing 1 required positional argument: 'value'

When I ran %reload_ext lab_black in my Jupyterlab, I got an error message:
ERROR:root:__call__() missing 1 required positional argument: 'value'
Traceback (most recent call last):
  File "/opt/anaconda3/lib/python3.7/site-packages/lab_black.py", line 218, in format_cell
    formatted_code = _format_code(cell)
  File "/opt/anaconda3/lib/python3.7/site-packages/lab_black.py", line 29, in _format_code
    return format_str(src_contents=code, mode=FileMode())
TypeError: __call__() missing 1 required positional argument: 'value'

Here's line 29:

Here's line 218:

Which function misses what argument? Also, I just updated to Jupyterlab to 2.1.0 so I don't know if there are weird incompatibility issues. Did anyone encounter this? Thanks!!

I have been receiving the same error as well.  I doubt if it has anything to do with compatibility of the version.  Anyway currently I am using JupyterLab version 1.2.16
Any suggestions on this would be helpful.

This is caused by having an old version of black. pip install -U black fixed it for me.

authors: Yuan-Meng ashwinidathatri terhorst


text:
"See More Results" when searching produces an error when searching in Firefox

Description
While on Firefox, if the user tries to search and use "See More Results" at the bottom of the search results, it produces an error and does not open the results page.
Expected result
Jump to search result page.
This works correctly on google chrome.
Current result
Error in console:
TypeError: access to strict mode caller function is censored
Affected version

[x] 9.2

Fixed by #2039
Please close @dnnsoftware/tag is you have no further comment

license: mit


text:
Cannot create new user with email address as username

Description
Going to Persona Bar > Security > Member Accounts tab > Registration Settings, and marking Use Email Address as Username, will generate an error upon login, displaying message "A critical error has occurred. Please check the Event Viewer for further details."
Steps to reproduce

Set up a clean DNN 9.2 install.
Go to Persona Bar > Manage > Users.
Click the "Add User" button.
Fill details for new user, setting an email address as the username.
Click "Save" button.
An error message is displayed saying: "The username specified is invalid. Please specify a valid username."

Current result
After upgrading to DNN 9.2, customer is no longer able to add user with their email addresses as username. The following Error Message is displayed: "The username specified is invalid. Please specify a valid username."
Expected result
Being able to add email address as a valid user name in the default user creation form.
Affected version

[x] 9.2

This has been fixed in PR #2070 and has been caused by #1972

There is also a companion PR dnnsoftware/Dnn.AdminExperience.Extensions#537 which applies the same fix in Persona Bar and removes some code duplication

Still occurs in 9.3.2

@nickcrisp I cannot reproduce in 9.3.2 using the above steps to reproduce...

I can not reproduce either. @valadas Close?

Actually, this one IS closed :) @nickcrisp can you please open a new issue with new steps to reproduce if you can still make it happen on 9.3.2 Thanks.

authors: Tychodewaard nickcrisp tpluscode valadas


text:
Failed to acquire CRAS

Please paste the output of the following command here: sudo edit-chroot -all

chronos@localhost / $ sudo edit-chroot -all
name: precise
encrypted: no
Entering /mnt/stateful_partition/crouton/chroots/precise...
crouton: version 1-20170228132702~master:21cb695b
release: precise
architecture: amd64
targets: kde
host: version 9000.91.0 (Official Build) stable-channel swanky
kernel: Linux localhost 3.10.18 #1 SMP Wed Feb 22 23:32:43 PST 2017 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Unmounting /mnt/stateful_partition/crouton/chroots/precise...
Please describe your issue:
Cannot complete chroot setup due to failing fetching CRAS.
If known, describe the steps to reproduce the issue:
I was longing to install Ubuntu on my Toshiba CB35, and after a really long list of something, the system started to acquire CRAS.
"Connecting to chromium.googlesource.com (chromium.googlesource.com)|2404:6800:4008:c06::52|:443... failed: Network is unreachable."
and for unknown reason,the process was terminated and all the effort seemed gone.
I typed sudo startkde as a hail Mary. Unlike previous attempts, the system came back with"the chroot may not be fully set up, would you like to finish it?", it liked a ray of hope that broke the darkness.
With joy and excitement I typed yes. Yet the system still cannot acquire CRAS persistently. I copied the link and found it's accessible in Chrome, and the file has been successfully downloaded.
Anyhow, I tried to boot the system again but this time it came back with "UID 1000 not found in precise",as my last ray of hope faded away.
I have established a proxy link in System Setting due to the censorship in China mainland. However it doesn't solve my issue.
Thank you for your reading and I hope God be with you.

I found a modification that requires a mandatory SSL link in order to ensure proxy involved on Chinese website.
I tried with that and it came back with "Proxy tunneling failed: Proxy Authentication Required Unable to establish SSL connection."
I have deployed a Shadowsocks on another computer and used as a proxy server link via LAN, it works fine until now. On other website no security issue discovered. That computer using AES-256 to establish link with some server (you guys call it VPS right?)

I just missed an option -P for assign a proxy. Sorry for being a noob.
Have a nice day guys!

repo: dnschneid/crouton


text:
Is there a way to make a crouton chroot use Chrome OS VPN (that is setup by an Android Play Store app)?

name: stretch
encrypted: no
Entering /mnt/stateful_partition/crouton/chroots/stretch...
crouton: version 1-20190403182822~master:174af0eb
release: stretch
architecture: amd64
xmethod: xorg
targets: xorg,gnome,xiwi,keyboard,core,extension
host: version 12105.42.0 (Official Build) beta-channel cave
kernel: Linux localhost 3.18.0-19339-g3e4e496860da #1 SMP PREEMPT Fri May 17 02:20:05 PDT 2019 x86_64 GNU/Linux
freon: yes

Please describe your issue:
I have a VPN running on Chrome OS that is setup by an Android Play Store app, https://play.google.com/store/apps/details?id=com.fast.free.unblock.thunder.vpn&hl=en_US, as an example.
The chroot does not use this VPN connection.  (A curl request inside the chroot to http://api.ipify.org, shows that it is using the original, non-VPN IP address).
If known, describe the steps to reproduce the issue:
I have seen https://github.com/dnschneid/crouton/wiki/VPNC.  I do not have the 'settings' for the setup/config of vpnc.conf, and ideally would like to be able to use any store VPN app.  In the wiki I read:

some VPNs requires client features and configurations that are not available. For those networks, it is possible to establish a VPN connection from a chroot that is usable from both within the chroot and Chromium OS.

Which made me hopeful when I first started reading it.  However, at configuration, it asks for settings, which I do not have access to since it is an android app.
If it is not possible, I get it.  If it is possible, or there is some way that it could be possible, if anyone could point me in the right direction, or offer any help at all, it would be much appreciated!
I have read, and seen the solutions (on Reddit) for using OpenVPN or Private Internet Access VPN (You would have knowledge of actual settings to use to be able to setup).  I am wanting/trying to make this work with any Play Store VPN that sets up a VPN in ChromeOS.

I was actually kind of surprised when I first checked the IP that it was NOT going through the VPN.  I figured that "that" would be a problem people may have been trying to figure out (how to get chroot to NOT go through ChromeOS VPN).
Anyways...

Thank you.

I think this is much more a function of the VPN application creating the tunnel. Some VPN applications seem to configure the tunnel such that everything goes through it...others not...it may also be a function of chromeos version or even hardware. Google searching suggests this is certainly a varying topic.
For me anyway, running the TorGuard VPN android app on my samsung pro routes Android, ChromeOS, and crouton traffic through the VPN.

also...this might help...mine's been set to "default" for a while now though...
within chrome://flags, there's Enable ARC VPN integration. ("Allow Android VPN clients to tunnel Chrome traffic. – Chrome OS")


text:
Kiwi: Send keycodes instead of keysyms

This is a partial fix for #1275 (but still much better than current situation, so it can be merged as-is).
The protocol now sends X11 key codes instead of key symbols. Keycodes (mostly) correspond to physical keyboard keys, so that the chroot can translate it, depending on the keyboard layout set in crouton.
For that purpose, I had to update protocol version from VF1 to VF2. The extension is backward compatible, though.
What works:

Setting US keyboard in Chromium OS (or any keyboard layout without dead keys), any keyboard layout in crouton should work fine.
Extension can connect to old chroot (VF1), in that case keysyms are sent.

What still does not work:

Setting a keyboard layout in Chromium OS with dead keys (e.g. US Intl) leads to lost key events (I think it's fixable, as the key code appears when the key is lifted).
Basic framework for reverting Search+key mapping is there, if the user wants Search+Left to mean Super+Left and not Home (see #1324). Not implemented yet (we need another configuration check box).

Other things to be implemented in other PR:

Warn the user that it is connecting to an old chroot.
Eventually, drop support for VF1 (this requires extensive rework of error handling in kiwi)

What I don't think can be fixed:

Switching Ctrl/Search/Alt in Chromium OS also results in switched positions in crouton.
Search+numbers appears the same as Search+fn keys (F1-F10). I don't think we can distinguish between the 2, to support Super+numbers (see #1324).

(untested on freon, my peppy refuses to switch to dev channel...)

This works well on Freon.  I'm inclined to merge it as-is, since my comments are only nits.

I was having trouble testing this with the current extension. Does the new extension get rolled out automatically?

@stsquad : The extension auto-updates, when no chroot is running (version in chrome://extensions should now be 2.1.0). You also need to update your chroot.

@drinkcat \o/ with the updated extension I just tested master and I have ~# and `¬ working ;-)

\o/ with the updated extension I just tested master and I have ~# and `¬ working ;-)

Good to hear!

See https://code.google.com/p/chromium/issues/detail?id=425156#c16.
Both "What still does not work", and "What I don't think can be fixed" are magically fixed with freon! Now there is still a slight bug as "OSLeft"/Super_L event are sometimes discarded. Happy not to have to implement another horrible hack...

Awesome!  I think freon will be generally a good thing for crouton.

Just out of curiosity, is there any reason to still keep the fix in this commit when using Freon? It seems to be the culprit of #1558.

@cribalik can you confirm that #1275 doesn't re-emerge if you revert this?

@dnschneid I confirmed it with Swedish, Dvorak US and Spanish keyboards in Chrome; the keys recieved from X11 were the same regardless of the layout in Chrome. Switching around the Alt/Search/Ctrl keys also did not have any impact on X11 keycodes. Search + LeftArrow is received as  OSLeft + LeftArrow, and not OSLeft + Home (which is what the fix was for I believe).
This seems to be Freon's doing that suddenly the correct keycodes are sent. Do we have to keep backwards compatability with non-Freon users? If not, I propose reverting the aforementioned fix.

Many users are still on non-freon systems, so a straight-out revert is not an option (yet).  We'll need to either have two code paths based on whether Freon is present or not (which probably means having the chroot tell the extension one way or the other), or wait until Freon is in stable on all devices to revert.

My device (2012 Samsung ARM / stable channel) isn't on Freon yet, but is very stable with precise, and I wouldn't mind losing the GUI for awhile. I've got a blog platform underneath that is very stable (open ssh, mysql, apache2, php5, myphpadmin, wordpress, etc.). I'm willing to move to trusty at some point, but precise has been extremely stable the past few months on the ARM. I mostly run the chroot with enter-chroot and utilize the server processes.

authors: cribalik dnschneid drinkcat stsquad tedm


text:
🛑 Site Histovec is down

In d7c7083, Site Histovec (https://histovec.interieur.gouv.fr/histovec) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Site Histovec is back up in 121dc68.

repo: dnum-mi/stats-sites-api


text:
Better error handling

With better error handling, I can help people resolve issues easily. Needs to keep aligned with the privacy statement on the README.

Added in recent slew of commits → https://github.com/doamatto/southnode/compare/1f4703cf154d...90e23efb57dd

repo: doamatto/southnode


text:
[feature] Java 11 Builds

Any plans to add Java 11 build tests to Travis?

Update Travis build file is easy.
Making Achilles work for Java 11 is another story:

massive update of all maven plugins because of the new module system introduced in Java 9
maybe some internal JDK classes used by Achilles to circumvent some defects of APT are no longer available/have been moved

You are welcomed to contribute to this migration

I found the Cassandra JIRA that Java 11 would be targetted for 4.0, so I assume changes here would be dependent on that

repo: doanduyhai/Achilles


text:
Build mod_proxy_*.so by default

I think it would be useful if mod_proxy and friends were available by default.

+1 (I asked for this some time ago in #6, for Apache 2.2)

authors: carletes oyvindio

repo: docker-library/httpd


text:
Added initContainers

Added initContainers (with extraVolumes and extraVolumeMounts fields). Also closes: #114.
These fields are need if for example one uses custom CA authority (Step Certificates) and needs to set a custom certificate that needs to be stored in /etc/ssl/certs/. Using an init container, one can mount the certificates to the given folder and run the update-ca-certificates command.

For extra volumes, could we just use the already existing persistence key?

Should we update the persistence key to work for this use case? I'm still not loving the idea of having two ways of defining volumes.
Also would it be worth splitting this into two MRs - one for init containers and then a separate one for the updated volume support?

Yes, I think it might work. The difference between the two is that in PR #117 the extraVolume(s/Mounts) are in the .Values, whereas in this PR they are in the .Values.deployment like in other charts (e.g. argo or cert-manager). There are also charts that use them directly from .Values (prometheus-adapter) it they know that there will be only one deployment).

I see. Thanks!

authors: cfis johnstarxx


text:
Nimble has moved

File: engine/extend/legacy_plugins.md
Can we please link the Nimble DVP to https://scod.hpedev.io/docker_volume_plugins/hpe_nimble_storage/index.html

Created a new issue in the upstream repo that contains this file. Closing this issue in favor of the new one.
https://github.com/docker/cli/issues/3589

authors: craig-osterhout datamattsson


text:
Docker daemon unresponsive and gigabytes of memory when full BGP table added to system (650k routes)

Description of problem:
We have a server which is connected to full BGP feed. It means that system has 650k routes in route table which in fact is quite normal. After starting docker daemon it allocates gigabytes of memory and never stops. All docker commands (eg. "docker ps") hangs. After removing routes everything is going back to normal.
Reproduced on docker vesions 1.6.2 and 1.7.1
docker version:
Client version: 1.7.1
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 786b29d
OS/Arch (client): linux/amd64
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

docker info:
Containers: 0
Images: 9
Storage Driver: devicemapper
 Pool Name: docker-9:1-1184122-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: extfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 493.9 MB
 Data Space Total: 107.4 GB
 Data Space Available: 12.15 GB
 Metadata Space Used: 1.102 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.146 GB
 Udev Sync Supported: false
 Deferred Removal Enabled: false
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.82-git (2013-10-04)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.18.11
Operating System: Debian GNU/Linux 7 (wheezy)
CPUs: 32
Total Memory: 125.9 GiB
Name: host
ID: DYIE:LMBJ:V5FV:FYTY:3WTQ:5ZLW:CPZI:RVHK:VFOO:XKXR:H2EP:PVUS
WARNING: No swap limit support

uname -a:
Linux host 3.18.11 #1 SMP Thu Jun 11 12:40:05 CEST 2015 x86_64 GNU/Linux

Environment details (AWS, VirtualBox, physical, etc.):
Physical server with full BGP route table and 128GB of ram. Can be reproduced on virtual servers too.
How reproducible:
Insert thousands of routes into routing table and restart docker daemon.
Steps to Reproduce:

add dummy interface:
ip link add name dummy0 type dummy && ip link set up dummy0
add multiple routes into the system (it takes few minutes):

for a in {2..10}; do
  for b in {1..253}; do
   for c in {1..253};
    do ip route add 1.$a.$b.$c/32 dev dummy0;
   done ;
  done;
done

start docker daemon

docker -D -d
DEBU[0000] Registering OPTIONS,
DEBU[0000] Registering GET, /events
DEBU[0000] Registering GET, /images/json
DEBU[0000] Registering GET, /containers/json
DEBU[0000] Registering GET, /containers/{name:.*}/export
DEBU[0000] Registering GET, /containers/{name:.*}/logs
DEBU[0000] Registering GET, /containers/{name:.*}/stats
DEBU[0000] Registering GET, /containers/{name:.*}/attach/ws
DEBU[0000] Registering GET, /info
DEBU[0000] Registering GET, /version
DEBU[0000] Registering GET, /images/search
DEBU[0000] Registering GET, /images/get
DEBU[0000] Registering GET, /images/{name:.*}/json
DEBU[0000] Registering GET, /containers/{name:.*}/top
DEBU[0000] Registering GET, /containers/ps
DEBU[0000] Registering GET, /containers/{name:.*}/changes
DEBU[0000] Registering GET, /exec/{id:.*}/json
DEBU[0000] Registering GET, /_ping
DEBU[0000] Registering GET, /images/{name:.*}/get
DEBU[0000] Registering GET, /images/{name:.*}/history
DEBU[0000] Registering GET, /containers/{name:.*}/json
DEBU[0000] Registering POST, /containers/{name:.*}/start
DEBU[0000] Registering POST, /containers/{name:.*}/stop
DEBU[0000] Registering POST, /exec/{name:.*}/start
DEBU[0000] Registering POST, /images/create
DEBU[0000] Registering POST, /images/load
DEBU[0000] Registering POST, /containers/{name:.*}/kill
DEBU[0000] Registering POST, /containers/{name:.*}/unpause
DEBU[0000] Registering POST, /exec/{name:.*}/resize
DEBU[0000] Registering POST, /containers/{name:.*}/rename
DEBU[0000] Registering POST, /auth
DEBU[0000] Registering POST, /containers/create
DEBU[0000] Registering POST, /containers/{name:.*}/wait
DEBU[0000] Registering POST, /containers/{name:.*}/resize
DEBU[0000] Registering POST, /containers/{name:.*}/exec
DEBU[0000] Registering POST, /build
DEBU[0000] Registering POST, /images/{name:.*}/tag
DEBU[0000] Registering POST, /containers/{name:.*}/restart
DEBU[0000] Registering POST, /containers/{name:.*}/attach
DEBU[0000] Registering POST, /commit
DEBU[0000] Registering POST, /images/{name:.*}/push
DEBU[0000] Registering POST, /containers/{name:.*}/pause
DEBU[0000] Registering POST, /containers/{name:.*}/copy
DEBU[0000] Registering DELETE, /containers/{name:.*}
DEBU[0000] Registering DELETE, /images/{name:.*}
DEBU[0000] docker group found. gid: 999
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)

Actual Results:
ip route show | wc -l
576088

ps aux --sort=-vsz,-rss | grep docker
root     17145  100  8.8 11867512 11623548 pts/2 Sl+ 14:17   1:21 docker -D -d

Expected Results:
Docker daemon can start and be responsive with full BGP route table.

I'll look into this because I've investigate some related code with similar problems in the past.

ping

authors: sanmai-NL sysopcorner unclejack

repo: docker/docker


text:
Can ping but not connect to container running on the same network

If I create a network and put a mysql server on it:
$ docker network create net
$ docker run -itd --net=net --name mysql -e MYSQL_ROOT_PASSWORD=password mysql:5.6.25

I can ping this container,
$ docker run --net=net mysql:5.6.25 ping mysql
PING mysql (172.19.0.2): 48 data bytes
56 bytes from 172.19.0.2: icmp_seq=0 ttl=64 time=0.144 ms
...

but not connect to the server,
$ docker run --net=net mysql:5.6.25 mysql -h mysql --password=password
#hangs indefinitely...

The logs show the server is running, and in fact I can exec into the mysql container and connect to the server from its own container, so that's all fine.
An important point is I've got two machines with identical versions of Docker (docker version and docker info are identical) but this happens on only one of them, so it must be some other settings on this machine interacting with Docker. Here's the info for the one where the problem happens:
Info
Linux darkenergy 3.19.0-31-generic #36~14.04.1-Ubuntu SMP Thu Oct 8 10:21:08 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 13:12:04 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 13:12:04 UTC 2015
 OS/Arch:      linux/amd64

Containers: 7
Images: 316
Server Version: 1.9.1
Storage Driver: aufs
 Root Dir: /home/boincadm/docker/aufs
 Backing Filesystem: extfs
 Dirs: 334
 Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.19.0-31-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 32
Total Memory: 251.9 GiB
Name: darkenergy
ID: HFOU:WMAG:6J3D:JLDB:LXBS:62C4:A7PY:RL5A:CP3V:ARO3:BGVC:DNXX
WARNING: No swap limit support

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

I've seen this behaviour when ufw was running.

@marius311 can you check if the problem disappears if ufw is disabled?

This is an old issue.  I will close it as stale.

authors: GordonTheTurtle marius311 sam-thibault teetrinkers thaJeztah


text:
Docker 1.13 overwrite/remove named volume when stack deploy/update

Currently there is no option when updating a stack (docker stack deploy) to overwrite a named volume created by this stack. In a same way, there is no option (such -v) to remove named volume associated with a stack when you remove the stack, other than removing the volumes with docker volume rm.
Here is a usecase :
Given this file:
version: '3'

services:
  config:
    image: test/config-hoster:1.3
    volumes:
      - volume_config:/datas
    deploy:
      mode: global
      restart_policy:
        condition: none
  nginx:
    image: nginx
    volumes:
      - volume_config:/datas:ro
    depends_on:
      - config

volumes:
  volume_config:

where config-hoster is a from scratch image, exposing a volume with the config files in it, and nginx a service used just for testing the correct mapping.
When I first deploy this stack, everything is fine, the volume is created and filled with the config files of the config service. But if I change the version (tag) of the config service, when I update the stack the volume is not deleted, so datas inside are obviously not overwrited. Same behavior if I rm and redeploy the stack.
Since this is a named volume, not directly bounded to an host directory, and not created externaly, I see no problem to erase or overwrite this volume when the stack is removed or updated.

Hmm, this would actually be a pretty big problem if implemented.
The reason it's doing this is the volume exists already and is populated with data when you update the config.
I think the right thing here is to actually support config objects (like we do secrets).

I've just checked the secret objects in docker. This is great, and actually solve a problem we were facing with ssh keys. Indeed supporting config objects in a same way will perfecly match the previous usecase.
I'm not sure to understand why and what would be the problem of deleting volumes created inside a stack, if you're sure of what you're doing (hence the -v option). Could you clarify this for me please ?

@BastienAr Volumes are meant for persistent storage. If you are telling a service to use a named volume, this is making a statement on the retention of this volume.
For instance if you use an anonymous volume (don't specify a name), this would be cleaned up.

I'm using NFS as volume storage backend
and.... I can't find any way to change its options except stop containers, remove volumes and deploy again.

@cpuguy83 Ok, I get the philosophy. So we definitively need a way to start a service requiring a file that could be modified (like config files) on any swarm node. Currently if you have to change configuration, either you rebuild the image with correct config (dirty) or you bind volume and become host dependent. NFS is also a solution, but I find it cumbersome for this purpose.

@BastienAr I feel like volumes are not a good use for injecting configuration, even though it may be the only way to do it right now.
One option that is currently available is to inject the config as a secret. It's a little hacky but works well in the short-term.

Yeah, that could be a trick, but it's still inconvenient due to the fixed path of secrets (/run/secret/<secret_name>). And the encryption of the file could add unnecessary process (config files rarely need encryption)

And the encryption of the file could add unnecessary process (config files rarely need encryption)

I think that overhead can be neglected; the encryption/decryption is part of the raft store, and should not cause a noticeable performance issue.
w.r.t. the paths (defaulting to /run/secret/<name>), the <name> part can be overridden when using the secret (--secret src=my-little-secret,target=config.json), but the path inside the container currently not (we kept this possibility open for a future enhancement, but needs additional discussion). You can create a symlink inside the container if needed.

Okay, this solution is quite simple and operational. Still I think this could be cleaner if shared file that are not secret would be separated from the term secret, even if under the hood this is the exact same process.
Bty is there any chance we could talk about this subject at the next Dockercon ? Some guys from our company (including me) will be there.

resolved in 17.06 by the introduction of Docker configs. I close the issue


text:
docs: change events --since to fit RFC3339Nano

PR6931 changed time format to RFC3339Nano.
But the example in cli.md does not changed.
Signed-off-by: Chen Hanxiao chenhanxiao@cn.fujitsu.com

@tiborvass

LGTM - @fredlf @jamtur01

authors: SvenDowideit chenhanxiao


text:
Fix docker start help message

Signed-off-by: Lei Jitang leijitang@huawei.com
Docker start can start multiple containers, but the help messages show
Restart a stopped container which is not correct.

ping @estesp  @jfrazelle

LGTM

ping

LGTM but would like a doc maintainer to make sure we didn't miss any docs.
ping @moxiegirl @SvenDowideit @fredlf

LGTM


text:
Enable validate-lint as part of CI

Yes, I'm ashamed.
Yes, I hope it passes.
Don't judge me.

lol LGTM, if it passes

Also had to fix pkg/chrootarchive/diff_windows.go because of #14862.

:stuck_out_tongue_closed_eyes:

authors: icecrime jfrazelle vdemeester


text:
Use sync.Pool for io.Copy buffers

Small ioutils.Copy function uses buffers from sync.Pool instead of allocating them on each io.Copy. The size of buffer chosen as a default size in io.Copy.
I used it only in overlay copy function because it was major memory eater.

I think I'll wait with this change and will use bufio.Reader.


text:
Support parsing SCTP port mapping

please see https://github.com/moby/moby/pull/33922

Signed-off-by: Wataru Ishida ishida.wataru@lab.ntt.co.jp

CI failure after rebase is unrelated

opened https://github.com/docker/go-connections/pull/47

rebased

authors: AkihiroSuda ishidawataru

repo: docker/go-connections


text:
Error installing tensorflow - Mac OSX 10.11.3

Error while pulling image: Get https://index.docker.io/v1/repositories/drunkar/anaconda-tensorflow-gpu/images: dial tcp: lookup index.docker.io on 127.0.0.54:53: read udp 127.0.0.1:40707->127.0.0.54:53: read: connection refused
Downloading has been active for > 1hr.  No network issue but download completion status not visible.

When you get error like this, does docker-machine restart fix it?

Where is your DNS resolver pointing? e.g. is it going through proxy

The restart should fix this, feel free to add more comments here if this doesn't solve it.

authors: FrenchBen arvind114 nathanleclaire

repo: docker/kitematic


text:
Unable to pull

docker pull microsoft/aspnet:1.0.0-rc1-update1-core
Expected behavior
The image "microsoft/aspnet:1.0.0-rc1-update1-core" gets installed correctly
Actual behavior
I am getting the following error:
Network timed out while trying to connect to https://index.docker.io/v1/repositories/raduporumb/aspnetcore-rc1-update1/images. You may want to check your internet connection or if you are behind a proxy.
Information about the Issue
docker pull microsoft/aspnet:1.0.0-rc1-update1-core
Steps to reproduce the behavior

...
...

Please make sure that you're on a stable connection and use a solid DNS such as Google DNS.

authors: FrenchBen praveenprabharavindran


text:
Default listener is writing directly to stdout

In https://github.com/docker/libcompose/blob/master/project/listener.go#L71, the default listener writes directly to stdout (tested on windows and ubuntu).
I am writing a cli app that uses libcompose to talk with the docker daemon and the output from the listener pollutes the output of my cli app.
Is it possible to redirect the output of the listener into memory so that I can write the output of the listener to stdout on my terms?

Hi @F21, thanks for the report. It's on its way, there is some refactoring to come in this area :stuck_out_tongue_closed_eyes:.

@vdemeester Is there anyway I can help get the ball rolling on this one? Maybe the adding a Listener field to the context struct passed to NewProject(). If there is a listener passed in, it uses it, otherwise it creates a DefaultListener. Let me know what you think!

I just notice NewDefaultListener() requires an instance of project, which causes a chicken and egg problem.

repo: docker/libcompose


text:
Removed printfs

Changed some prints into proper logging, also
was missing the \n at the end
Signed-off-by: Flavio Crisciani flavio.crisciani@docker.com

Not a maintainer, but this looks good to me.

LGTM

authors: aaronlehmann fcrisciani mavenugo

repo: docker/libnetwork


text:
Fixed typo

Fixed Minor typo.
Signed-off-by: Ian Lee IanLee1521@gmail.com]

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "patch-1" git@github.com:IanLee1521/machine.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

LGTM
Thanks!

repo: docker/machine


text:
force cgo resolver for name resolution

By default, the pure Go resolver is used which makes direct DNS
requests first to resolve a hostname before checking /etc/hosts.
If a host on the network has the same name as the linked container,
the host on the network will be used instead of the linked container.
I had a machine on the network with hostname mysql and bringing up the containers with docker-compose had the server and signer linked containers trying to connect to to it...
$ sudo docker-compose up
...
mysql_1  | 2016-05-19 19:33:29 140622360905664 [Note] mysqld: ready for connections.
mysql_1  | Version: '10.1.10-MariaDB-1~jessie'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
server_1 | waiting for notarymysql to come up.
signer_1 | waiting for notarymysql to come up.
server_1 | waiting for notarymysql to come up.
signer_1 | waiting for notarymysql to come up.
server_1 | waiting for notarymysql to come up.
...

Forcing cgo resolver will use C library routines that will honor
values in /etc/hosts first and then fall back on DNS.
From the docs: https://golang.org/pkg/net/#hdr-Name_Resolution
Signed-off-by: Andrew Hsu andrewhsu@acm.org (github: andrewhsu)

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "compose-resolv" git@github.com:andrewhsu/notary.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "compose-resolv" git@github.com:andrewhsu/notary.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Can one of the admins verify this patch?

Sorry about the multiple force pushes...trying to get signature requirement squared away. I think it's good to go now.

jenkins, test this please

@cyli umm...i think jenkins is a bot. that's all he can say: https://github.com/docker-jenkins?tab=activity

@andrewhsu Yes it is :). We gate PRs on code coverage results (which wait for results from Jenkins and CircleCI) as well as CI results - this Jenkins server runs our yubikey tests, but doesn't automatically run them for PRs from authors not in the org.  The comment I left basically lets it know that it's ok to run tests for this PR, else the codecov check would never finish.
We were wondering if converting the compose file to v2 and using a network definition would be sufficient?  That would not put anything in /etc/hosts, and the DNS resolution would hopefully work correctly.

@andrewhsu it's our jenkins bot and @cyli's comment is what we use to instruct it to run its tests. It will only pay attention to the maintainers on this project :-)

@cyli I tried as you suggested, to convert the docker compose files to v2 but it still had the same issue. If I add the environment variable GODEBUG=netdns=cgo then everything works. So, I created #755 as an alternative to this PR with the docker compose v2 format change.

@andrewhsu I may be misunderstanding the docs but they don't appear to indicate the behaviour you describe. We think what you might be encountering is a compose issue (previously described here: https://github.com/docker/distribution/issues/1362 ) and while it can be solved with the cgo change, we really don't want to go that route of hacking every other project to work around an issue elsewhere. If you think it's the same issue could you file a ticket on the docker-compose repository?

@endophage I understand. I've adjusted #755 to remove the netdns workaround. I'll close out this PR.

@andrewhsu Just checking as a possible reason for the issue - did you happen to have more than 2 DNS servers on your host when the other machine took precedence over the linked mysql container?

@cyli no i did not have 2 dns servers on the host at the time. Outside of docker containers on the host I had entries something like this in /etc/resolv.conf:
search internal.example.com
nameserver 10.0.0.1
nameserver 10.0.0.2

And running our own DNS servers resolves hostnames like this:
bash$ host mysql
mysql.internal.example.com has address 10.10.10.5

In any case, I tried to replicate the issue again after moving my entire development environment from Kitematic to the new Docker for Mac OS X 1.12.0-rc2-beta16 and I was not able to get it to fail like before so I'm not going to spend any more time on it. Stuff works.

Ah ok, thanks for clarifying and trying to replicate!
On Thursday, July 7, 2016, Andrew Hsu notifications@github.com wrote:

@cyli https://github.com/cyli no i did not have 2 dns servers on the
host at the time. Outside of docker containers on the host I had entries
something like this in /etc/resolv.conf:
search internal.example.com
nameserver 10.0.0.1
nameserver 10.0.0.2
And running our own DNS servers resolves hostnames like this:
bash$ host mysqlmysql.internal.example.com has address 10.10.10.5
In any case, I tried to replicate the issue again after moving my entire development environment from Kitematic to the new Docker for Mac OS X 1.12.0-rc2-beta16 and I was not able to get it to fail like before so I'm not going to spend any more time on it. Stuff works.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/docker/notary/pull/753#issuecomment-231110375, or mute
the thread
https://github.com/notifications/unsubscribe/AANn4PBau7RBlOc9LdGAHhpg1_Rh7Ms_ks5qTRjygaJpZM4IimJO
.

authors: GordonTheTurtle andrewhsu cyli docker-jenkins endophage

repo: docker/notary


text:
Small cleanup of Cluster.createContainer

Signed-off-by: Andrea Luzzardi aluzzardi@gmail.com

/cc @jimmyxian @vieux

@aluzzardi This cleanup will save the soft-image-affinity in ContainerConfig

4/ Retry with a soft-affinity (but don't store it in the ContainerConfig)

@jimmyxian You are right :)

authors: aluzzardi jimmyxian

repo: docker/swarm


text:
[18.03] backport reaper fixes

Backports for 18.03 of;

https://github.com/docker/swarmkit/pull/2526 [manager/orchestrator/task_reaper] Fix task reaper test to also set the desired state on tasks to prevent reconciliation races
https://github.com/docker/swarmkit/pull/2591 [manager/orchestrator/taskreaper] Move task_reaper_test to orchestrator/taskreaper
https://github.com/docker/swarmkit/pull/2666 [orchestrator/task reaper] Clean up tasks in dirty list for which the service has been deleted
https://github.com/docker/swarmkit/pull/2675 [manager/orchestrator/reaper] Clean out the task reaper dirty set at the end of tick()
https://github.com/docker/swarmkit/pull/2669 Fix task reaper batching

git cherry-pick -s -S -x a388cad309edddb9880899fe8927afbe4717a18e
git cherry-pick -s -S -x 8cfb337920a6658b302643f27074ca3d669176ec
git cherry-pick -s -S -x 592e8eddfa43ec5fbd6e34da5ad6890dfa9313fb
git cherry-pick -s -S -x 1a43a3b612d8c775db8a44c8399844e1f7e4aed2
git cherry-pick -s -S -x 5291c7a7b45773a4fe18720a54485ee2dde0af3d

Conflict when cherry-picking https://github.com/docker/swarmkit/commit/8cfb337920a6658b302643f27074ca3d669176ec, likely because things were cherry-picked out of order;
On branch 18.03-backport_reaper_2
You are currently cherry-picking commit 8cfb3379.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:

	modified:   manager/orchestrator/taskreaper/task_reaper_test.go

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)

	deleted by them: manager/orchestrator/replicated/task_reaper_test.go

Resolved by git rm manager/orchestrator/replicated/task_reaper_test.go to mark resolution
To verify everything looked ok after merging, I compared the directories with master;
git diff master  manager/orchestrator/taskreaper/
git diff master  manager/orchestrator/

Which produced an empty diff, so the master/orchestrator directory is fully up to date with master

This supersedes https://github.com/docker/swarmkit/pull/2668
ping @anshulpundir @dperny @cyli PTAL if this LGTY

LGTM, let's do it.

authors: dperny thaJeztah

repo: docker/swarmkit


text:
WDL parsing check for recursive imports catches cases that are not recursive

See https://github.com/gevro/gatk4-exome-analysis-pipeline-flat
If you try and register it with the new dockstore WDL 1.0 parsing code, it will say there might be recursive imports. I checked the import structure and this is not true, it is a flaw with our code.
Our current WDL parsing code cannot parse these 1.0 files either, so users cannot publish their valid 1.0 workflows.
From
https://discuss.dockstore.org/t/unable-to-get-publish-button-to-become-active/1972
┆Issue is synchronized with this Jira Story
┆Issue Type: Story
┆Fix Versions: Dockstore 1.7
┆Sprint: Seabright Sprint 17 Raft
┆Issue Number: DOCK-911

@agduncan94 taking a look, let me know if you have code for this already

@denis-yuen I have the code, it is in https://github.com/dockstore/dockstore/tree/feature/2766/recursive-imports
I still have to write tests, was going to use the one from the discourse discussion.

Ah, too late

I can now register https://raw.githubusercontent.com/gevro/gatk4-exome-analysis-pipeline-flat/master/ExomeGermlineSingleSample.wdl

authors: agduncan94 denis-yuen garyluu

repo: dockstore/dockstore


text:
Dockstore-based workflows with registered tools

Feature Request
Is it possible to create Workflow using Tools registered in Dockstore? I noticed that many available workflows in Dockstore provide all tools (CWL files) locally. I want to write Workflow files (CWL) using tools that I previously registered in Dockstore. Any ideia?
Desired behaviour
A CWL Workflow example:
# ...
steps:
  quality_report:
    run:  registry.hub.docker.com/welliton/rqc # latest version
    in: {}
    out: {}
  quality_control:
    run: registry.hub.docker.com/welliton/trimgalore:v0.4.4 # with tag
    in: {}
    out: {}
# ...

┆Issue is synchronized with this Jira Story
┆Fix Versions: Dockstore 2.X
┆Issue Number: DOCK-490
┆Sprint: Backlog
┆Issue Type: Story

Hi,
I think the approach that comes to mind to try is:
Multiple tools are built from multiple Dockerfiles+CWL files in one repo. These are then added to Dockstore
Subsequently, a workflow is added to the same repo that uses those tools.
We haven't tried sharing multiple tools between multiple repos yet, however. It might depend on how to reference a CWL step in a different repo in CWL.

This is something we'd also like to be able to do.  To some extent we're also thinking about 'custom' CWL which isn't in a repo but uses registered tools to allow experimentation or user centric plug-and-play.

One thought I had is I have the impression you may be able to import workflow steps via URL http://www.commonwl.org/v1.0/SchemaSalad.html#Import
This is something that I haven't tried myself yet, but if it works, it might be nice to work out a pattern based on this and support it in Dockstore explicitly.

Thank you @denis-yuen for your suggestion. It worked! I used the URL for raw Dockstore.cwl file from GithHub. See example below.
cwlVersion: v1.0
class: Workflow

inputs:
  files: File[]
  groups: string[]

outputs:
  qc_report:
    type: File
    outputSource: qc_raw/qc_report

steps:
  qc_raw:
    run: https://raw.githubusercontent.com/labbcb/dockstore-rqc/v3.5/Dockstore.cwl
    in:
      files: files
      groups: groups
    out: [qc_report]

Input example:
files:
  - class: File
    path: ERR127302_1_subset.fastq.gz
  - class: File
    path: ERR127302_2_subset.fastq.gz
groups: [None, None]

I tested using cwltool. Another way is to dowload CWL files via dockstore tool cwl. For example:
dockstore tool cwl --entry registry.hub.docker.com/welliton/rqc:v3.5 > Rqc.cwl

@denis-yuen the answer to this is no, but it could be. Do you want @aofarrel to make a PR to include this in the current documentation? If so, Ash can target develop.

I think that would be nice. There's a bit of a start of a description of different kinds of imports at https://docs.dockstore.org/en/stable/end-user-topics/language-support.html#converting-file-path-based-imports-to-public-http-s-based-imports-for-wdl and a new section could be added for CWL

authors: Welliton309 bethsheets denis-yuen keiranmraine wdesouza


text:
Docopt parser get all words in doc

Hi,

Before starting, here is my configuration:
docopt==0.6.2
Python 3.5.2

So, why when i run this command like this example right here:
python naval_fate.py ship Guardian move 100 150 --speed=15
With this script:
"""Naval Fate.
Usage:
  naval_fate.py ship new <name>...
  naval_fate.py ship <name> move <x> <y> [--speed=<kn>]
  naval_fate.py ship shoot <x> <y>
  naval_fate.py mine (set|remove) <x> <y> [--moored|--drifting]
  naval_fate.py -h | --help
  naval_fate.py --version
Options:
  -h --help     Show this screen.
  --version     Show version.
  --speed=<kn>  Speed in knots [default: 10].
  --moored      Moored (anchored) mine.
  --drifting    Drifting mine.
"""
from docopt import docopt

if __name__ == '__main__':
    arguments = docopt(__doc__, version='Naval Fate 2.0')
    print(arguments)

Results are:
{'--drifting': False,      '--help': 0,
 '--moored': False,        '--speed': '15',
 '--version': 0,           '.': False,
 '10': False,              '<name>': ['Guardian'],
 '<x>': '100',             '<y>': '150',
 'Drifting': False,        'Moored': False,
 'Options:': False,        'Show': 0,
 'Speed': False,           'anchored': False,
 'default:': False,        'in': False,
 'knots': False,           'mine': False,
 'mine.': 0,               'move': True,
 'new': False,             'remove': False,
 'screen.': False,         'set': False,
 'ship': True,             'shoot': False,
 'this': False,            'version.': False}

and not like this example right here:
{'--drifting': False,    'mine': False,
 '--help': False,        'move': True,
 '--moored': False,      'new': False,
 '--speed': '15',        'remove': False,
 '--version': False,     'set': False,
 '<name>': ['Guardian'], 'ship': True,
 '<x>': '100',           'shoot': False,
 '<y>': '150'}

Did I make a mistake or did the docopt parser have a problem?
Sorry for my english.

Oh, i just forgot to add an empty line between "Usage" and "Options". It works!

repo: docopt/docopt


text:
Inverse-side PersistentCollections should be immutable

This issue came out of #1086, where we discussed several edge cases when working with inverse-side PersistentCollections that have been modified.

@jmikola: Should we consider documenting that inverse collections are read-only, or perhaps enforce that with a special PersistentCollection sub-class? AFAIK, changes to such a collection would not be reflected in the changeset (so they are already read-only from ODM's perspective). If users actually want to modify the collection (so their data model doesn't have to care), I suppose they can manually unwrap it in a lifecycle event and work with an ArrayCollection.
@alcaeus: I agree with having an ImmutablePersistentCollection class which forbids any write operations (add(), clear(), remove(), removeKey(), set()) in order to preserve the intended behavior. Only thing is, while it might be that we're just strictly enforcing something that was an unwritten rule before, we're technically breaking BC here.

I suppose they can manually unwrap it in a lifecycle event and work with an ArrayCollection.

What about having additional parameter in mapping to unwrap it automatically? Lifecycle event seems to be too much compared to @ReferenceMany(..., immutable="false")
Anyway I still have in mind introducing custom collections so I'd say create a (really) Big Red Warning in the docs (not current notice, a Big Red Warning) to not complicate things in future?

For the record from IRC:

@alcaeus: @Ocramius had some input on that one as well
basically, the recommendation at the time was to have PersistentCollection be mutable to allow people to add items to inverse collections before persisting to the database - well knowing that some counts may be off if they do
ORM has the same issue we do. personally, i don't like removing functionality from a class, which is why i could live with leaving PersistentCollection fully mutable as long as we document this somehow

+1 for flexibility and freedom of immutable="false" !
I guess in bi-directional relations like Task <-> Tag where tags are reusable among the Tasks is ok, but in (for example) Task <-> Comment, I would like to delete comments from inside comments Task collection.
As you were discussing, its quite confusing, the fact that you can call remove() on a PersistentCollection and it is not persisted, because is the inverse-side of the relation.
Anyway, I hope I understood everything well ! It took me a while and I feel I don't know even a half ! :)
Thanks !!

Custom collections are implemented for a while now, whoever wants immutability for inverse side of references can employ them :)

authors: dossorio jmikola malarzm

repo: doctrine/mongodb-odm


text:
envelopesApi.update: Error: INVALID_REQUEST_BODY

In an attempt to resend a document using the node api client, the code below throws error status 400 - INVALID REQUEST BODY.
await envelopesApi.update(dsJwtAuth.accountId, envelopeId, {
    resendEnvelope: true
})

However, doing the following works:
await envelopesApi.update(dsJwtAuth.accountId, envelopeId, {
    resendEnvelope: true,
    envelope: {}
})

Fix suggestions:

Update node-client-api documentation for update(accountId, envelopeId, opts, callback) to say opts.envelope is required

OR

Update api/EnvelopesApi.js, line 3861 to be var postBody = opts['envelope'] || {} ;

Thank you for the bug report. I have filed internal report DCM-3369.

Hi @by12380 This issue has been resolved starting in version 4.10.1 and 5.8.1

Can confirm that this is working, thanks! might be nice to have this as an example on examples project

repo: docusign/docusign-node-client


text:
Option to dim / disable display brightness

As a user I want to be able to dim / disable the display brightness because in some situation it is too bright for the room

Agree. Would also like to turn it off at night, say between 11 PM and 5 AM.

authors: AronGahagan japaweiss

repo: doidotech/TBM


text:
86148ec0: "Implement DXVK pieces required for DX11 DLSS support" Breaks Origin

Software information
Origin crashes using current master of DXVK. 1.9.2 works. After bisecting I found that "86148ec070628f5a89fbb0a91603bae2ce89529a:  Implement DXVK pieces required for DX11 DLSS support" is the offending commit. Reverting it fixes the issue
System information

GPU: Nvidia RTX 3090
Driver: 470.74
Wine version:  All wine versions after 6.6
DXVK version: Current master

Apitrace file(s)
Origin.4.trace.tar.gz
Log files

d3d9.log: Origin_d3d9.log
d3d11.log: Origin_d3d11.log
dxgi.log: Origin_dxgi.log

Do other 32-bit apps work or is it only Origin?

Battle.Net works. Not sure if Ubisoft Connect is 32-bit or not, but it works as well. I know Battle.Net is 32-bit, though, and it works fine.

Fixed on master.

repo: doitsujin/dxvk

license: Zlib


text:
Ghost Recon Advanced Warfighter 2 does not render shadows at all

Ghost Recon Advanced Warfighter 2 and its predecessor (GRAW) do not render shadows with DXVK or WineD3D. There is an old wine bug that could be relevant, https://bugs.winehq.org/show_bug.cgi?id=38015 something to do with cascaded shadows?
Software information
Ghost Recon Advanced Warfighter 2 downloaded from Amazon Games since it's no longer on another online store. I've maxed out the settings but I've also tried disabling Dynamic Shadows or enabling them at low, medium, and high.
System information

GPU: AMD Raven Ridge Vega 10
Driver: Mesa 21.3.2
Wine version: Tested on 6.21-devel and 7.0-rc2
DXVK version: Tested on 1.9.2 and latest master (as of two days ago)
Fedora 35

Apitrace file(s)
Apitrace
Log files

d3d9.log: https://drive.google.com/file/d/1CIp1-ECqnLKarAl59Ka_RON9mvaTMNZI/view?usp=sharing
d3d11.log: N/A
dxgi.log:https://drive.google.com/file/d/1L-Sok3HzlLzLmVHJiwU50ozKUmiwKf81/view?usp=sharing

Looks like all your google drive links require a password.

Looks like all your google drive links require a password.

My apologies, I just changed the permissions. Can you try now?

I got around to installing Windows 11 and the shadows do render when I play. When I drop in the DXVK DLLs next to the game exe, the shadows no longer render.

@Alexithymia2014 Can i get you to test this issue again?
Your trace is weird when i replay it with either wined3d or dxvk. The world doesn't seem to load in at all with both once you load the game.

I'll try to recreate the trace for you
On Mon, Jul 18, 2022, 1:30 PM Blisto91 @.***> wrote:

@Alexithymia2014 https://github.com/Alexithymia2014 Can i get you to
test this issue again?
Your trace is weird when i replay it with either wined3d or dxvk. The
world doesn't seem to load in at all with both once you load the game in
your trace.
—
Reply to this email directly, view it on GitHub
https://github.com/doitsujin/dxvk/issues/2410#issuecomment-1187870102,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AH52BUZSCBCHL3OYVYQ5S43VUWIEVANCNFSM5KTEWXSQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi @Blisto91, the shadows work now in 1.10.2! Out of curiosity, was there a specific fix for this?

Not as far as i know. But there have been some general dx9 fixes which might affect a bunch of games.
Glad to hear  it's working! 🙂


text:
Update shouldSubmit to correctly handle descriptorPoolOverallocation

Currently shouldSubmit will force the dxvk context to be flushed when too many descriptor pools have been allocated. This heuristic does not work when VK_NV_descriptor_pool_overallocation is in use because there will only ever be a single pool.
This change updates the heuristic to use the number of allocated sets when VK_NV_descriptor_pool_overallocation is in use.
Resolves the issue described in https://github.com/ValveSoftware/Proton/issues/7862

For the bug I included in the description the application ends up repeatedly calling vkAllocateDescriptorSets so the driver will eventually hit an OOM if the pool isn't reset. The game doesn't use d3d for presentation so in this case the descriptor pool ends up growing indefinitely.

alright, I was kind of expecting pool memory to still be limited in some way, good to know.

authors: doitsujin esullivan-nvidia


text:
fix(sozo): ensure warnings don't stop tests build

Closes DOJ-252, #1646.

hmm im still getting the reported error even with with this change

hmm im still getting the reported error even with with this change

With which project did you try? Do you have only warnings?
I'll add tests tomorrow, that's a good point.
But I've tested on spawn-and-move and I have the expected output:

hmm im still getting the reported error even with with this change

With which project did you try? Do you have only warnings? I'll add tests tomorrow, that's a good point.
But I've tested on spawn-and-move and I have the expected output:

ah nvm my mistake.. its working lol

As discussed this morning, the rework of sozo ops and commands will address related tests.

authors: glihm kariy

repo: dojoengine/dojo


text:
Add support for material 3

Can't use this library on material 3 currently
In my app I am using material 3 only. I don't want to mix both material 3 and 2.

Thanks for the suggestion, will ship an M3 artifact when I have time.

Great. I'm willing to help in this

v0.6.0 is out with Material 3 support: https://github.com/dokar3/ChipTextField/releases/tag/v0.6.0


text:
THANK YOU TEAM! This worked perfectly for me!

This worked perfectly for me! I purchased the IOGEAR Bluetooth 4.0 USB Micro Adapter (GBU521) hopes to utilize Apple's handoff and airdrop features. I have the MacBook Pro (15-inch, Mid 2010) running OS X Yosemite. I installed 'Continuity Activation Tool'. It was seamless. There were no hiccups and all is working like a charm!
THANK YOU TEAM!

Thank you for your support @mbain108 !

repo: dokterdok/Continuity-Activation-Tool