text:
Allow build args to be overridden from the command line

In addition to the static set of args within the dobi file, it would be helpful to allow build args to be overridden from the command line.
One of the scenarios this would support is where a build arg is used within a Dockerfile  but is known only at runtime. For example, an arg might form part of the base image name/tag from which the image extends.

Setting anything on the command line is difficult. dobi was designed so that any parameters are encoded in the config. make works in a similar way. The only way to specify parameters is through environment variables. #91 seems related, I don't have a good solution for that.
There are a couple ways to set dynamic values from the config. By setting an env variable on the command like you can read it using {env.VARIABLENAME} (http://dnephin.github.io/dobi/variables.html).
If the value is only available as output from some command than you can use job:capture (http://dnephin.github.io/dobi/tasks.html#job-tasks) to run the command and capture stdout to a variable, which you can then use with the same {env.VARNAME} syntax.

Thanks @dnephin; I'd missed the capture task. That might do the job, cheers!

source: gharchive/issue

authors: dnephin outofcoffee

repo: dnephin/dobi

license_type: permissive

license_source: bigquery


text:
blog.kablamo.org?

Hello, is Perly-Bot invite only?  Any objection to adding blog.kablamo.org?

Thanks for adding your blog, this looks great!

source: gharchive/pull-request

authors: dnmfarrell kablamo

repo: dnmfarrell/Perly-Bot


text:
Bump Visual Studio Requirement to 2017 or 2019

Description of bug
After merging #2774 and #649 we can no longer compile the Platform or AdminExperience in Visual Studio 2015.
The current requirements for the Platform are VS2015 and for Extensions 2013 or 2015.
https://www.dnnsoftware.com/docs/developers/requirements.html
https://dnndocs.com/content/administrators/requirements/index.html#dnn-requirements
Suggested Solution
Upgrade our required version of Visual Studio to 2017 or 2019 with the Platform v9.4 release.

@ohine I agree that #2774 made the problem visible, I believe #2758 caused the problem as that enforced all platform code to compile under C# 7.0. There were many projects in the platform that were still targeting C# 6.0. #2774 used some features that are not available in C# 6.0.
I think we need to create a issue in the Admin Experience project to get everything on C# 7.0 if it isn't already there

Good catch @ahoefling, I totally forgot about the C# 7.0 bump.
As for AE, yes we need to get it on C#7 and .net 4.7.2. It would be nice to do the same with CDF and CK if someone has the time to submit those PRs before Friday.

PR Submitted dnnsoftware/Dnn.AdminExperience#948

Duplicates #2786

authors: ahoefling ohine valadas

license: mit


text:
Winding back to RC5 and redoing earlier changes

Winding back to RC5 and redoing earlier changes to fix this build

This is not the right way


text:
I want to reuse a WebRequest object so that cookies and session would be saved for later request to the server. Below is my code. If i use Post function twice on the second time at  request.ContentLength = byteArray.Length; it will throw an exception  This property cannot be set after writing has started! But as you can see  dataStream.Close(); Should close the writing process! Anybody knows what's going on?

public void sendSingleSMS(String username, String password, String senderid, String mobileNo, String message)
{
        CookieContainer cookieJar = new CookieContainer();
        //request.ContentLength = 0;
        String smsservicetype = "singlemsg"; //For single message.

        String query = "username=" + HttpUtility.UrlEncode(username) + "&password=" + HttpUtility.UrlEncode(password) + "&smsservicetype=" + HttpUtility.UrlEncode(smsservicetype) + "&content=" + HttpUtility.UrlEncode(message) + "&mobileno=" + HttpUtility.UrlEncode(mobileNo) + "&senderid=" + HttpUtility.UrlEncode(senderid);
        request.CookieContainer = cookieJar;
        byte[] byteArray = Encoding.UTF8.GetBytes(query);
        request.ContentType = "application/x-www-form-urlencoded";

        request.ContentLength = byteArray.Length;
        dataStream = request.GetRequestStream();
        dataStream.Write(byteArray, 0, byteArray.Length);
        dataStream.Close();
        WebResponse response = request.GetResponse();
        String Status = ((HttpWebResponse)response).StatusDescription;

        dataStream = response.GetResponseStream();
        StreamReader reader = new StreamReader(dataStream);
        string responseFromServer = reader.ReadToEnd();
        reader.Close();
        //byteArray.clear(0);
        //byteArray.length = 0;
        dataStream.Close();
        response.Close();
        //request.ContentType = null;

        request.Abort();
    }

Is this a crouton issue?

nice title

Same problem as in #262

authors: DennisLfromGA Lightsockie dnschneid santosh770

repo: dnschneid/crouton

license_source: github-api


text:
Link user keychain to session keychain

This prevents odd "mount(2)" errors with mount-chroot.

I could not mount my encrypted chroot on a Lenovo Thinkpad 13 running Version 59.0.3071.71 (Official Build) beta (64-bit) without this fix.

I signed it!

Is this overkill, with potentially bad side-effects?
Would it be better to just make sure the decrypt key ends up in the appropriate keyring to begin with?

I don't really know about these keyrings, but linking the user keyring in the session keyring seems ok.
On samus-release/R58-9334.72.0, I see this right after login:
$ sudo keyctl list @s
1 key in keyring:
 92994838: --alswrv     0 65534 keyring: _uid.0
$ sudo keyctl list @u
2 keys in keyring:
286101033: --alswrv     0     0 user: a5aeddcf6261f807
905769048: --alswrv     0     0 user: d224878099ec22d9

While on samus-release/R61-9647.0.0 (ext4 crypto) the link is not setup (there is a dircrypt keyring instead):
$ sudo keyctl list @u
keyring is empty
$ sudo keyctl list @s
1 key in keyring:
308501995: --alswrv     0     0 keyring: dircrypt

But after linking it looks like this, which seems to make sense.
$ sudo keyctl list @s
2 keys in keyring:
991469042: ---ls-rv     0     0 keyring: dircrypt
259439550: ---lswrv     0 65534 keyring: _uid.0

My concern is that they may be unlinked for a reason (ARC++ or multi-profile is usually a good guess).
What shows up if you print out the keyrings right after unwrapping the keys for the chroot?  Can we somehow only make those keys accessible where they need to be?

Without this patch:
$ sudo keyctl list @u
2 keys in keyring:
890533281: --alswrv     0     0 user: 8c7159f15c183071
772187242: --alswrv     0     0 user: d3f79ffd75f42358
$ sudo keyctl list @s
1 key in keyring:
308501995: --alswrv     0     0 keyring: dircrypt

(then mount fails and keys are immediately cleared from @u)
With this patch:
$ sudo keyctl list @u
2 keys in keyring:
338803567: --alswrv     0     0 user: 8c7159f15c183071
1044269498: --alswrv     0     0 user: d3f79ffd75f42358
$ sudo keyctl list @s
2 keys in keyring:
308501995: --alswrv     0     0 keyring: dircrypt
346063147: --alswrv     0 65534 keyring: _uid.0

Then, on umount, @u is cleared, but the link in @s stays.

Thank you for the patch!
To use ext4 encryption on Chrome OS, we are applying a local patch to upstart to share a single session keyring among all user processes.
https://chromium-review.googlesource.com/c/331340/
This means, linking the root's user keyring to the session keyring results in exposing it to non-root users too, and I think it's unsafe.
So, instead of linking the root's user keyring to the current session keyring, how about setting up a new session keyring?
keyctl session  # Set up a new session keyring
keyctl link @u @s  # Link the user keyring to the new session keyring (this is needed because ecryptfs util functions add keys to the user keyring)
keyctl link  @s # Link the original session keyring to the new session keyring (this is needed to access user's data on new devices if it's encrypted with ext4 encryption instead of ecryptfs)
The resulting keyring will look like this:
keyctl show
Session Keyring
860524043 --alswrv      0     0  keyring: _ses
1065162552 --alswrv      0 65534   _ keyring: _uid.0
798736045 --alswrv      0     0   _ keyring: _ses
220661875 --alswrv      0     0       _ keyring: dircrypt

@hshmt Thanks for the insight!
From offline discussion, we don't even need to link the old _ses keyring in the new session (mount-chroot just needs to have the keys for ecryptfs). Also, keyctl new_session is what we want here, as keyctl session spawns a new subshell.

@smerrill thanks for the initial investigation and fix!  Continuing in #3278

@drinkcat

From offline discussion, we don't even need to link the old _ses keyring in the new session (mount-chroot just needs to have the keys for ecryptfs). Also, keyctl new_session is what we want here, as keyctl session spawns a new subshell.

I am not sure that this solution is the right one for the reasons given below:

keyctl new_session does link the old _ses keyring in the new session. Thus the crouton chroot does have access to the old keyring (including ext4 encryption keys) as can be seen from the output of keyctl show @s.From offline discussion, we don't even need to link the old _ses keyring in the new session (mount-chroot just needs to have the keys for ecryptfs). Also, keyctl new_session is what we want here, as keyctl session spawns a new subshell.
Since we are not in a new session, it is more complicated for the user of the chroot to add keys to the keyring. For this reason if (say) a user tries to create an encryptfs directory on an external device, then that user needs to jump through hoops as explained in my 21 Nov update to my message on 19 Oct in #3461.

I will try to find a "better" fix once I find time to debug the process!
Kapil.

I'm really glad this all makes sense to someone here.  Thanks for looking into it!

authors: dnschneid drinkcat hshmt kapilhp smerrill


text:
🛑 Site Histovec is down

In c51aee5, Site Histovec (https://histovec.interieur.gouv.fr/histovec) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Site Histovec is back up in 007b571.

repo: dnum-mi/stats-sites-api

license: MIT


text:
Run with "secured" podman fail

I tried to give minimal permission to postgres using podman:
podman run --name mydatabase \
    --uts=private --hostname mydatabase \
    --pod mypod \
    --pull missing --detach \
    --cap-drop=all --read-only \
    --mount "type=volume,src=mypgdata,dst=/var/lib/postgresql/data" \
    --mount "type=volume,src=mypgrun,dst=/var/run/postgresql" \
    -e "POSTGRES_DB=mydb" \
    -e "POSTGRES_USER=mydba" \
    -e "POSTGRES_PASSWORD=supersecret" \
    -e "PGDATA=/var/lib/postgresql/data/pgdata" \
    postgres:15.2-bullseye

It fail because of:
chown: changing ownership of '/var/lib/postgresql/data/pgdata': Operation not permitted

How can I avoid this chown behavior ?

If you run your container as non-root (--user), it will not try to chown (see the "Running as an arbitrary user" section of the image description on Docker Hub).

authors: emidevops tianon

repo: docker-library/postgres


text:
Avoid panicing if cmd.Start() from container.createCmd() failed

Signed-off-by: Kenfe-Mickael Laventure mickael.laventure@gmail.com
--
Required to fix docker/docker#25955

Sgtm, I'll retest integration tests in docker once containerd is revendored there.
Totally unsure why process is nil there though

@runcom I'm thinking we hit this branch: https://github.com/docker/containerd/blob/6dd2f1c422d63a105a7d275d93bdcd0b790dad23/runtime/container.go#L473

LGTM

authors: crosbymichael mlaventure runcom

repo: docker/containerd


text:
Add ContainerService.Info to get info about a single container

For clients which only want to know about one container this is simpler than
searching the result of execution.List.
Signed-off-by: Ian Campbell ian.campbell@docker.com
In docker/swarmkit#1965 I currently have:
func (c *containerAdapter) inspect(ctx context.Context) (container.Container, error) {
	// XXX inefficient.
	rsp, err := c.client.List(ctx, &execution.ListRequest{})
	if err != nil {
		return container.Container{}, err
	}

	for _, candidate := range rsp.Containers {
		if candidate.ID == naming.Task(c.task) {
			return *candidate, nil
		}
	}
	return container.Container{}, errors.Errorf("Container not found")
}

I'd prefer to have something like:
func (c *containerAdapter) inspect(ctx context.Context) (container.Container, error) {
	id := naming.Task(c.task)
	rsp, err := c.client.Info(ctx, &execution.InfoRequest{ID: id})
	if err != nil {
		return container.Container{}, err
	}
	return *rsp, nil
}

Codecov Report

Merging #565 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #565   +/-   ##
=======================================
  Coverage   62.12%   62.12%
=======================================
  Files           3        3
  Lines         404      404
=======================================
  Hits          251      251
  Misses         93       93
  Partials       60       60

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 660783c...cc253b0. Read the comment docs.

LGTM

authors: codecov-io ijc25 stevvooe


text:
Add support for Debian Buster

Signed-off-by: Micha Felle micha@dietpi.com

Raspberry Pi users with the new official Raspbian Buster image, required for Raspberry Pi 4, will still run into an error since there is no Raspbian Buster branch yet in the Docker repo: https://download.docker.com/linux/raspbian/dists/
A new build for Raspbian would be good anyway since there are reported issues with the latest current available package: https://github.com/MichaIng/DietPi/issues/2282
But that's another topic 😉.

repo: docker/docker-install


text:
Binaries?

Not sure if this is the right place to ask, but is there any chance of getting updates binaries into the various package managers?  Centos has 0.6.8 I believe.  I've built the registry before on Centos 6.x, but it's quite a pain on the 7.x image thanks to systemd/container funkiness (https://registry.hub.docker.com/_/centos/).  You can't install openssl-devel for example on the library/centos:7 image as is.
My end goal is a container with apache fronting the registry with SSL.  I know the docker way of dealing with this is two containers, but I'd really prefer to package a single image for my users.

Hello Matt,
We don't have any specific relations with distros as far as packaging this is concerned.
Furthermore, we strongly recommend users to use the docker image and not the distros packages that we don't support.
Now, if you want to have a custom image, I would recommend you start your own with:
FROM registry as a starting point (if I were to do it I'd probably go with supervisor to launch both apache and the registry).
Finally, most of the effort these days is focused on https://github.com/docker/distribution the new golang version of the registry - v1 is in maintenance now.

repo: docker/docker-registry


text:
Typo in ubuntulinux.md

Proposed changes

Unreleased project version (optional)

Related issues (optional)

Thanks @alberto :)

authors: alberto johndmulhausen


text:
removed ubuntu wily 15.10 references in install docs

@thaJeztah @mlaventure @tianon @justincormack
Submitting PR for docs changes related to https://github.com/docker/docker/pull/27042/files (Remove Ubuntu 15.10 (Wily Werewolf) from build-deb targets)
Docs source has moved to new consolidated repository, and docs.docker.com will begin serving out of this repo today. You all can file docs PR's on this repo. When you have code changes that require docs changes, please x-ref/link the PR's across the two repos.
(For now you can view the generated / updated docs at https://docker.github.io/ preview site, once this PR is merged.)
Signed-off-by: Victoria Bialas victoria.bialas@docker.com

LGTM, merging

Thanks @londoncalling

@londoncalling oops looks like this should have gone into the vnext-engine branch, as this change is for 1.13

Opened https://github.com/docker/docker.github.io/issues/503 on the vnext branch, and  https://github.com/docker/docker.github.io/issues/505 to revert on master

authors: londoncalling thaJeztah


text:
publish to docs.docker.com

Proposed changes

Unreleased project version (optional)

Related issues (optional)

Deploy preview for docsdocker ready!
Built with commit 09c8a2d726ab2b60d9b0b34a3c9af96e7154c9f2
https://deploy-preview-9657--docsdocker.netlify.com

authors: DawnWood-Docker GordonTheTurtle


text:
/var/run/docker.sock mounting /run/docker.sock instead

Description of problem:
When I run docker run -it --name registrator2 -v /var/run/docker.sock:/tmp/docker.sock -h ubuntu-14 gliderlabs/registrator:v5 consul://127.0.0.1:8500, I see the following volumes entry after running docker inspect registrator2:
"Volumes": {
        "/tmp/docker.sock": "/run/docker.sock"
    },

despite having this entry further up:
"HostConfig": {
        "Binds": [
            "/var/run/docker.sock:/tmp/docker.sock"
        ],

docker version:
Docker version 1.6.2, build 7c8fca2
docker info:
Containers: 11
Images: 305
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 327
 Dirperm1 Supported: false
Execution Driver: native-0.2
Kernel Version: 3.13.0-24-generic
Operating System: Ubuntu 14.04 LTS
CPUs: 2
Total Memory: 993.9 MiB
Name: ubuntu-14
ID: TAKT:FKDW:PND3:BUCN:TGPE:AUMN:KDA3:EBWQ:BWLV:SBBC:YXRP:JZ55

uname -a:
Linux ubuntu-14 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Environment details (AWS, VirtualBox, physical, etc.):
VirtualBox (4.3.26r98988)
How reproducible:
Consistently
Steps to Reproduce:

run docker run -it --name registrator2 -v /var/run/docker.sock:/tmp/docker.sock -h ubuntu-14 gliderlabs/registrator:v5 consul://127.0.0.1:8500
run docker inspect registrator2

Actual Results:
"Volumes": {
        "/tmp/docker.sock": "/run/docker.sock"
    },

Expected Results:
"Volumes": {
        "/tmp/docker.sock": "/var/run/docker.sock"
    },

The path pointed to should have /var/ in front of it
Additional info:

I think this is because of FollowSymlink. Isn't it working?

Ah! That would make sense. I was not aware that volumes reported on the symlink destination rather than the source.
Initially I thought it wasn't working, but it turns out it was due to Registrator not reporting errors when failing to connect to Consul which caused Docker events to not get picked up.

@jeffcharles Thanks for check!

repo: docker/docker


text:
docker daemon on Windows should support longer path than 260 characters

Description of problem:
It seems that the Windows docker client and/or daemon needs to support longer file names than 260 characters.
docker version:
Client:
 Version:      1.9.0-dev
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   4376380
 Built:        Wed Aug 19 14:59:24 UTC 2015
 OS/Arch:      windows/amd64

Server:
 Version:      1.9.0-dev
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   4376380
 Built:        Wed Aug 19 14:59:24 UTC 2015
 OS/Arch:      windows/amd64

docker info:
Containers: 37
Images: 89
Storage Driver: windowsfilter
 Windows:
Execution Driver: Windows 1854 1.9.0-dev 4376380
Logging Driver: json-file
Kernel Version: 10.0 10514 (10514.0.amd64fre.th2_release.150808-1529)
Operating System: Windows Server 2016 Technical Preview 3
CPUs: 2
Total Memory: 4 GiB
Name: VAGRANT-2016
ID: 3A6C:S32I:LEC6:YQJB:7PIZ:OK6F:5DEX:X4EI:4SQV:U6MJ:FOTF:W2KN
Debug mode (server): true
File Descriptors: -1
Goroutines: 14
System Time: 2015-08-22T12:59:06.9061205-07:00
EventsListeners: 0
Init SHA1:
Init Path: C:\Windows\System32\docker.exe
Docker Root Dir: C:\ProgramData\docker

uname -a:
MSYS_NT-10.0 vagrant-2016 2.1.0(0.288/5/3) 2015-06-28 18:57 x86_64 Msys

Environment details (AWS, VirtualBox, physical, etc.):
Windows 2016 TP3 running in VMware Fusion 7.1.2 on a Mac OSX 10.10.4
How reproducible:
Steps to Reproduce:

create sub directories until absolute path length is longer than 260 characters
docker build -t test .

Actual Results:
Error message Error checking context: 'can't stat ...
Expected Results:
Docker should add all sub directories even if the absolute path length is longer than 260 characters. This is needed for Node.js/io.js apps running in a Windows container.
Additional info:
I try to build a Windows Docker image with an io.js application on a Windows 2016 TP3.
I've created a local iojs:3.1.0 Docker image with this Dockerfile.
Then I tried to add a Node.js/io.js application with a deep nested node_modules file tree with this Dockerfile:
FROM iojs:3.1.0

ADD . /code/

CMD "node /code/bin/app.js"

But there is an error adding the directory tree:
PS C:\Users\vagrant\Documents\example-iojs-app> docker build -t example-iojs-app .
Error checking context: 'can't stat 'C:\Users\vagrant\Documents\example-iojs-app\node_modules\cust-log\node_modules\flaschenpost\node_modules\update-notifier\node_modules\latest-version\node_modules\package-json\node_modules\got\node_modules\duplexify\node_modules\end-of-stream\node_modules\once''.

The io.js application works fine running on the Win2016TP3 host itself with io.js 3.1.0 as io.js supports long file names https://github.com/nodejs/node/issues/1990
Is there a way to help improve the Docker daemon for Windows? Is there a description how to build the docker.exe that has to be replaced in the TP3 installation?

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

I hope Go supports long pathes on Windows, just found this one https://github.com/golang/go/issues/3358

Thanks for reporting, @StefanScherer
/cc @jhowardmsft @swernli

Thanks for the heads up!  This can indeed be fixed on the docker side by pre-pending the appropriate paths with "\?" as was used in https://github.com/docker/docker/pull/15491.

@swernli Thank you! Ah, that seems promising.
I found another issue in a combination of WORKDIR and  the ONBUILD COPY while writing a little blog post about your amazing work of porting Docker to Windows. I have seen in #15491 that CI is not up and running for the Windows Docker daemon yet. Should I wait until CI is available or should I file an issue right now? Perhaps this is covered in some other tests.

We are still catching up with our tests for the daemon side, but better to file the issue now (and include myself and @jhowardmsft on it) so that we can track it and fix it sooner rather than later.

@swernli Thanks for the PR, I've just tested it with latest nightly build
Server:
 Version:      1.9.0-dev
 API version:  1.22
 Go version:   go1.5.1
 Git commit:   9657062
 Built:        Thu Oct 29 02:23:58 UTC 2015
 OS/Arch:      windows/amd64

And adding a deep node_modules tree works fine!
I'll close this issue.

I had a similar error about "Cannot stat ." Docker build on windows seems to have problems with junctions. The fix was to move to a directory for docker build that has no junctions underneath it.

authors: GordonTheTurtle StefanScherer blaskovicz swernli thaJeztah


text:
docker 1.9.0 docker build caching ignores changed file contents.

Description of problem
As of docker 1.9.0, docker build incorrectly reuses a cached layer even when the content of a file which should be part of that layer has modified contents.
This is important regression, because it results in broken images; versions 1.8.3 and earlier of docker-engine do not suffer from the issue.
Context
docker version: Docker version 1.9.0, build 76d6bc9
docker info:
Containers: 2
Images: 2286
Server Version: 1.9.0
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: xfs
 Dirs: 2290
 Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.19.0-32-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 8
Total Memory: 15.58 GiB
Name: jp440p
ID: CB5Z:P7RF:RIET:BEGL:XC55:566P:2VWO:GL6X:GU75:5ZZN:66WM:SOAH
Http Proxy: http://10.0.3.84:3128
Https Proxy: http://10.0.3.84:3128
No Proxy: localhost,127.0.0.1,192.168.33.1,192.168.122.1,192.168.33.1,192.168.33.10.xip.io,10.0.2.15,/var/run/docker.sock
WARNING: No swap limit support

uname -a: Linux jp440p 3.19.0-32-generic #37~14.04.1-Ubuntu SMP Thu Oct 22 09:41:40 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Environment details (AWS, VirtualBox, physical, etc.): physical x86/64 workstation running Ubuntu 14.04.3 (with kernel 3.19)
Reproducing the issue
How reproducible: 100% (under docker 1.9.0; 1.8.3 and earlier versions do not have the problem)
Steps to Reproduce:

create a test directory and cd to it
create the docker-build-caching-bug.sh file with the contents below and make it chmod +x
run docker-build-caching-bug.sh orig # with orig as argument
run docker-build-caching-bug.sh updated # with updated as argument

#!/bin/sh

set -e

SUFFIX=${1:-orig} # pass any other value on 2nd invocation

cat <<EOF > Dockerfile
FROM busybox:ubuntu-14.04
ADD / /data
CMD ["/bin/cat", "/data/volatilefile.txt"]
EOF

cat <<EOF > volatilefile.txt
volatile-${SUFFIX}
EOF

docker build -t test .

# now the test. should work on docker < 1.9.0 and break with docker = 1.9.0
output=`docker run --rm test || true`

if [ "${output}" != "volatile-${SUFFIX}" ]
then
  echo "BUG: output was '${output}', expected 'volatile-${SUFFIX}'"
  exit 1
else
  echo "OK: output was '${output}', same as expected"
  exit 0
fi

Actual Results: the result displayed at the end of step 4 is BUG: output was 'volatile-orig', expected 'volatile-updated'.
Expected Results: the result displayed at the end of step 4 should be OK: output was 'volatile-updated', same as expected.
Additional info:

I have tested docker 1.8.3 and docker 1.7.1 which do NOT suffer from this problem.

This is not happening on latest master:
runcom @ fedora koding/docker/test ‹test*› » ./test.sh orig
Sending build context to Docker daemon 4.608 kB
Step 1 : FROM busybox:ubuntu-14.04
 ---> 1356831708f4
Step 2 : ADD / /data
 ---> Using cache
 ---> b3225073d237
Step 3 : CMD /bin/cat /data/volatilefile.txt
 ---> Using cache
 ---> f6a96c21d1f5
Successfully built f6a96c21d1f5
OK: output was 'volatile-orig', same as expected
runcom @ fedora koding/docker/test ‹test*› » ./test.sh updated
Sending build context to Docker daemon 4.608 kB
Step 1 : FROM busybox:ubuntu-14.04
 ---> 1356831708f4
Step 2 : ADD / /data
 ---> Using cache
 ---> a3a101097dad
Step 3 : CMD /bin/cat /data/volatilefile.txt
 ---> Using cache
 ---> 980db6ff861b
Successfully built 980db6ff861b
OK: output was 'volatile-updated', same as expected
runcom @ fedora koding/docker/test ‹test*› »

Looking for what fixed it though

I'm leaving kind/bug cause maybe the fix could go into 1.9.1 (I'm testing it btw)

Can you confirm that the issue reproduces on your 1.9.0 @runcom and that it is not just the docker on my workstation?

Hmm, I think this is because you are doing ADD / /data instead of ADD . /data

I'm not saying that's correct... but if I change to . it's fine.

I am experiencing this issue as well.  Interestingly, it only is happening on one of my two workstations, even though they're mostly identical (same version of Mac OS X, same version of Docker Toolbox, same version of Virtualbox, building the same dockerfiles.)

@jperville's shows a failure on both machine (including the working one).
@cpuguy83's patch shows a success on both machines (including the broken one).  Not sure why the test script is working when my actual projects are not.

I was able to reproduce it by creating a Python virtualenv (virtualenv env) in the docker build context.  Not sure what about a virtualenv is breaking it.

There's a known issue with symlinks, which is fixed on master and should be in 1.9.1 once it's out.

Ah, that would be it.
Making sure symlinks aren't in the build context fixes my builds.  :+1:

I'm also having this issue.
$ docker info
Containers: 0
Images: 0
Server Version: 1.9.0
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.19.0-32-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 1
Total Memory: 993.2 MiB
Name: vagrant-ubuntu-trusty-64
ID: XRJB:2P54:OO2B:UWYB:I5L2:POAV:HQXJ:4SOP:H7FW:ARPV:X5QL:S7XJ
Debug mode (server): true
 File Descriptors: 17
 Goroutines: 21
 System Time: 2015-11-10T04:09:58.510165056Z
 EventsListeners: 0
 Init SHA1: 787da34bb103f11fd6114d8782217bc4f7032e3e
 Init Path: /usr/lib/docker/dockerinit
 Docker Root Dir: /var/lib/docker
WARNING: No swap limit support

Dockerfile:
FROM scratch
COPY /  /some_folder

Simple test:
$ echo "foo" > some_file && \
> docker build -t test . && \
> echo "bar" > some_file && \
> docker build -t test .

Sending build context to Docker daemon 3.072 kB
Step 1 : FROM scratch
 --->
Step 2 : COPY / /some_folder
 ---> ab8ec20b08d4
Removing intermediate container d5433605c363
Successfully built ab8ec20b08d4
Sending build context to Docker daemon 3.072 kB
Step 1 : FROM scratch
 --->
Step 2 : COPY / /some_folder
 ---> Using cache
 ---> ab8ec20b08d4
Successfully built ab8ec20b08d4

You can see the 2nd build uses the cache even though some_file has changed.
If I update the Dockerfile to use COPY . /some_folder, it does not use cache, and builds a new image.

authors: cpuguy83 jperville luhn pbitty runcom


text:
Docker daemon crashes/logs full of network not found

BUG REPORT INFORMATION (may be related to #19988)
Output of docker version:
Docker version 1.9.0, build 76d6bc9
# and also
Docker version 1.10.2, build c3959b1

Output of docker info:
# none, I can't reanimate the daemon anymore and all my 230+ containers are basically down

Provide additional environment details (AWS, VirtualBox, physical, etc.):
OpenStack VM, 4 cores, 8 GB mem, Ubuntu 14.04.4 LTS, Linux 2b320682-1875-4548-a1e2-29d15b11b5a8 3.19.0-49-generic #55~14.04.1hf1533043v20160201b1-Ubuntu SMP Mon Feb 1 20:41:00 UT x86_64 x86_64 x86_64 GNU/Linux.
List the steps to reproduce the issue:
Started by script since months without problems, but now failing. Also fails after Docker update and OS update. Here the launch command we use (230+ containers all started with restart policy = always):
docker daemon --api-cors-header=false --debug=false --disable-legacy-registry=false --group vcap --graph /var/vcap/store/docker/docker --host unix:///var/vcap/sys/run/docker/docker.sock --icc=true --ip-forward=true --ip-masq=true --iptables=true --ipv6=false --log-level=info --pidfile /var/vcap/sys/run/docker/docker.pid --selinux-enabled=false --host tcp://0.0.0.0:4243 --tls=false --userland-proxy=true

Describe the results you received:
INFO[0000] API listen on /var/vcap/sys/run/docker/docker.sock
INFO[0000] API listen on [::]:4243
INFO[0000] [graphdriver] using prior storage driver "aufs"
ERRO[0000] getNetworkFromStore for nid d48a6206205146ee2e1ca9ae0ce6f5ea1c6b2b1a64d739f771bccfd0ac5970e1 failed while trying to build sandbox for cleanup: network d48a6206205146ee2e1ca9ae0ce6f5ea1c6b2b1a64d739f771bccfd0ac5970e1 not found
ERRO[0000] getEndpointFromStore for eid 8fc41dcb6bf31938e79b193d01ce84acd7f660d8ae82afb6f31c8bf191e1663f failed while trying to build sandbox for cleanup: could not find endpoint 8fc41dcb6bf31938e79b193d01ce84acd7f660d8ae82afb6f31c8bf191e1663f: Key not found in store
WARN[0000] Failed detaching sandbox 569c2a0eefb84fb14b3e3c6cbb554163b480331b996e2ef9f7b2fa9a4e033672 from endpoint c71f3a95931e93968f28f0ab27bef203d10aa1d48b2c18b36ca56eaf023fcc10: failed to get network from store during leave: network d48a6206205146ee2e1ca9ae0ce6f5ea1c6b2b1a64d739f771bccfd0ac5970e1 not found

WARN[0000] Failed deleting endpoint c71f3a95931e93968f28f0ab27bef203d10aa1d48b2c18b36ca56eaf023fcc10: failed to get network during Delete: network d48a6206205146ee2e1ca9ae0ce6f5ea1c6b2b1a64d739f771bccfd0ac5970e1 not found

WARN[0000] Failed detaching sandbox 569c2a0eefb84fb14b3e3c6cbb554163b480331b996e2ef9f7b2fa9a4e033672 from endpoint 8fc41dcb6bf31938e79b193d01ce84acd7f660d8ae82afb6f31c8bf191e1663f: failed to get endpoint from store during leave: could not find endpoint 8fc41dcb6bf31938e79b193d01ce84acd7f660d8ae82afb6f31c8bf19
1e1663f: Key not found in store

WARN[0000] Failed deleting endpoint 8fc41dcb6bf31938e79b193d01ce84acd7f660d8ae82afb6f31c8bf191e1663f: failed to get endpoint from store during Delete: could not find endpoint 8fc41dcb6bf31938e79b193d01ce84acd7f660d8ae82afb6f31c8bf191e1663f: Key not found in store

ERRO[0000] failed to delete sandbox 569c2a0eefb84fb14b3e3c6cbb554163b480331b996e2ef9f7b2fa9a4e033672 while trying to cleanup: could not cleanup all the endpoints in container 27f42f1c8f2792639d4c96f621408b1551a9121c184f2721b6aa17444c96177f / sandbox 569c2a0eefb84fb14b3e3c6cbb554163b480331b996e2ef9f7b2fa9a4e03
3672

...
ERRO[0025] getNetworkFromStore for nid d48a6206205146ee2e1ca9ae0ce6f5ea1c6b2b1a64d739f771bccfd0ac5970e1 failed while trying to build sandbox for cleanup: network d48a6206205146ee2e1ca9ae0ce6f5ea1c6b2b1a64d739f771bccfd0ac5970e1 not found                                                                 [18/6605]
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x168 pc=0x87f3d9]

goroutine 1 [running]:
github.com/docker/libnetwork.epHeap.Less(0xc20864d570, 0x2, 0x2, 0x1, 0x0, 0xc208033a70)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/sandbox.go:968 +0x159
github.com/docker/libnetwork.(*epHeap).Less(0xc208854630, 0x1, 0x0, 0x2)
        <autogenerated>:50 +0xb5
container/heap.up(0x7f835b7a4438, 0xc208854630, 0x1)
        /usr/local/go/src/container/heap/heap.go:93 +0x5f
container/heap.Push(0x7f835b7a4438, 0xc208854630, 0x13ce9e0, 0xc208033a70)
        /usr/local/go/src/container/heap/heap.go:54 +0x7e
github.com/docker/libnetwork.(*controller).sandboxCleanup(0xc2080a0370)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/sandbox_store.go:226 +0xc06
github.com/docker/libnetwork.New(0xc208765bc0, 0x5, 0x8, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/controller.go:200 +0x6cb
github.com/docker/docker/daemon.(*Daemon).initNetworkController(0xc208071040, 0xc2080eb180, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/daemon/daemon_unix.go:364 +0xfd
github.com/docker/docker/daemon.NewDaemon(0xc2080eb180, 0xc20803a430, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/daemon/daemon.go:770 +0x2149
main.(*DaemonCli).CmdDaemon(0xc2084f5240, 0xc20800a020, 0x16, 0x16, 0x0, 0x0)
        /go/src/github.com/docker/docker/docker/daemon.go:269 +0x1722
reflect.callMethod(0xc2084d7f50, 0xc2084dbce8)
        /usr/local/go/src/reflect/value.go:605 +0x179
reflect.methodValueCall(0xc20800a020, 0x16, 0x16, 0x1, 0xc2084d7f50, 0x0, 0x0, 0xc2084d7f50, 0x0, 0x4dcb9f, ...)
        /usr/local/go/src/reflect/asm_amd64.s:29 +0x36
github.com/docker/docker/cli.(*Cli).Run(0xc2084d7e90, 0xc20800a010, 0x17, 0x17, 0x0, 0x0)
        /go/src/github.com/docker/docker/cli/cli.go:89 +0x38e
main.main()
        /go/src/github.com/docker/docker/docker/docker.go:65 +0x418

goroutine 5 [syscall]:
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
        /usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 12 [chan receive]:
github.com/docker/docker/pkg/listenbuffer.(*defaultListener).Accept(0xc208504380, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/pkg/listenbuffer/buffer.go:73 +0xb6
net/http.(*Server).Serve(0xc2080428a0, 0x7f835b79a568, 0xc208504380, 0x0, 0x0)
        /usr/local/go/src/net/http/server.go:1728 +0x92
github.com/docker/docker/api/server.(*HTTPServer).Serve(0xc208504460, 0x0, 0x0)
        /go/src/github.com/docker/docker/api/server/server.go:113 +0x4d
github.com/docker/docker/api/server.func·006(0xc208504460)
        /go/src/github.com/docker/docker/api/server/server.go:86 +0x163
created by github.com/docker/docker/api/server.(*Server).ServeAPI
        /go/src/github.com/docker/docker/api/server/server.go:90 +0x144

goroutine 10 [chan receive]:
github.com/docker/docker/api/server.(*Server).ServeAPI(0xc208037900, 0x0, 0x0)
        /go/src/github.com/docker/docker/api/server/server.go:94 +0x1b6
main.func·007()
        /go/src/github.com/docker/docker/docker/daemon.go:255 +0x3b
created by main.(*DaemonCli).CmdDaemon
        /go/src/github.com/docker/docker/docker/daemon.go:261 +0x1571

goroutine 11 [chan receive]:
github.com/docker/docker/daemon.func·030()
        /go/src/github.com/docker/docker/daemon/debugtrap_unix.go:17 +0x5c
created by github.com/docker/docker/daemon.setupDumpStackTrap
        /go/src/github.com/docker/docker/daemon/debugtrap_unix.go:20 +0x18e

goroutine 13 [chan receive]:
github.com/docker/docker/pkg/listenbuffer.(*defaultListener).Accept(0xc2085044a0, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/pkg/listenbuffer/buffer.go:73 +0xb6
net/http.(*Server).Serve(0xc208042a20, 0x7f835b79a568, 0xc2085044a0, 0x0, 0x0)
        /usr/local/go/src/net/http/server.go:1728 +0x92
github.com/docker/docker/api/server.(*HTTPServer).Serve(0xc208504580, 0x0, 0x0)
        /go/src/github.com/docker/docker/api/server/server.go:113 +0x4d
github.com/docker/docker/api/server.func·006(0xc208504580)
        /go/src/github.com/docker/docker/api/server/server.go:86 +0x163
created by github.com/docker/docker/api/server.(*Server).ServeAPI
        /go/src/github.com/docker/docker/api/server/server.go:90 +0x144

goroutine 14 [select]:
github.com/docker/libnetwork.(*controller).watchLoop(0xc2080a0370)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/store.go:399 +0x13f
created by github.com/docker/libnetwork.(*controller).startWatch
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/store.go:413 +0xf0

goroutine 39 [select]:
github.com/docker/libnetwork/osl.removeUnusedPaths()
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/osl/namespace_linux.go:73 +0x48b
created by github.com/docker/libnetwork/osl.createBasePath
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/osl/namespace_linux.go:58 +0xb1

Describe the results you expected:
Daemon starts, all containers start.
Provide additional info you think is important:
We haven't restarted the Docker daemon for two weeks. Maybe there are too many containers deployed now? But we had already daemons with 300+ containers running.
Maybe something went wrong during shutdown/the VM was terminated? The VM gets automatically recreated/reprovisioned and the former volume gets reattached. Could there by some issues with a hard kill?
In any case, the crash when relaunching the daemon doesn't look too good/like a real issue.
Do you have short term remedy proposals? Is there a way to force a cleanup? Or some way to start the daemon, but not all the containers (all were started with restart policy = always)?
In short, we see tons of:
* getNetworkFromStore for nid failed while trying to build sandbox for cleanup: network not found
* failed getting network for ep during sandbox delete: network not found
* failed to delete sandbox while trying to cleanup: could not cleanup all the endpoints in container / sandbox

... and then it crashes in:

github.com/docker/libnetwork.epHeap.Less (Docker v1.9) or
github.com/docker/libnetwork.(*sandbox).clearNetworkResources (Docker 1.10)

@vlerenc it seems like the local db (/var/lib/docker/network/files/local-kv.db) is corrupted. Could you pls take a backup of that file (and share it with us ?). once its backed up, pls remove this file and restart the daemon.
We have to identify how and when the file is corrupted.

Yes, it seems so. After removing the file, I could restart the daemon again.
You don't know how much you saved my day. Thank you very much!
What's the consequences of the removed file? Does it impact anything?
The second question is why it happened out of the sudden? The VMs were regularly updated, i.e. the daemon was shutdown every couple of weeks. Usually, with that many containers, the daemon kills most of the containers,right? Most likely, the VM updater even killed the daemon itself. Docker was (for the past couple of months) on the same 1.9 release, so no change there.
Besides the question on the impact of the now deleted network DB file, how can I make the corrupted file available to you?
And thank you again for your prompt help!

What's the consequences of the removed file? Does it impact anything?

If you are using custom bridge networks (other than the default docker0 network), those needs to be recreated manually. But it seems in your case, that is not the case. So, you wont see any functionality issues. Other than that, this file holds the resources to be cleaned up if it was not cleaned up properly when the daemon is brought down ungracefully (which seems to be your case).

The second question is why it happened out of the sudden?

Its a file corruption due to some event (could be timing related). The best way to know that would be to look at the daemon logs (preferably with debugs enabled with -D flag) Thanks to the config reload feature in 1.10, one can enable the daemon debug flags for a running daemon. PTAL : https://github.com/docker/docker/blob/master/docs/reference/commandline/daemon.md#configuration-reloading & https://github.com/docker/docker/blob/master/docs/reference/commandline/daemon.md#daemon-configuration-file

You don't know how much you saved my day. Thank you very much!

You don't know how much you are helping us by reporting the issue so that we can resolve it in next release. Thank you very much. OSS FTW !

@mavenugo @vlerenc shall we close this issue, or do you want to keep this opened for tracking? (I think it's already on your radar, but just to be sure)

@vlerenc

how can I make the corrupted file available to you?

Easiest would be to put it in dropbox/box or other file sharing site and share the link here (or) send the link to my email madhu@docker.com

@thaJeztah this is already in the radar yes. but we don't have a solid use-case to reproduce it yet. We can close this but lets dupe to the other open issue appropriately.

@mavenugo I sent you the link to your e-mail address. Thank you for looking into it!

@vlerenc can you pls answer this question as well

Also, can you please confirm if you have created a custom docker network using docker network create and using it in your containers ?

No, we haven't.

@vlerenc thanks for sharing the corrupted local-kv.db. I analyzed it and found that there are/were bunch of containers run in overlay network. Now that you have replaced the local-kv.db, you may not see that. But, I see a bunch of sandboxes attached to an overlay network and a corresponding KV-Store.
All the network d48a6206205146ee2e1ca9ae0ce6f5ea1c6b2b1a64d739f771bccfd0ac5970e1 not found
errors are expected if the daemon doesnt have connectivity to external KVStore you might have once used. The panic is a different story though.
Atleast, it helps answer my question :
Also, can you please confirm if you have created a custom docker network using docker network create and using it in your containers ?
and the answer is yes. Can you please confirm if there is a disconnect ?

@vlerenc can you pls attach the traceback for
github.com/docker/libnetwork.(*sandbox).clearNetworkResources (Docker 1.10) ?
Easiest thing to do is to

stop the daemon.
take a backup of your /var/lib/docker/network/files/local-kv.db.
Replace it with the corrupted one that you sent me
restart the daemon & capture the tracelog.
replace the backed up local-kv.db
restart the daemon.

@vlerenc never mind. found the issue and am fixing it now.
For confirmation, this is not a file corruption issue. But, this is one such scenario of using a global kv-store to try out multi-host networking followed by an unclean daemon shutdown and restarting the daemon with the kv-store down/not configured.
Thanks for reporting and sharing all the requested information.

@mavenugo Glad you found it! As for your question, I am sorry. We are using https://github.com/cloudfoundry-community/cf-containers-broker which is using https://github.com/swipely/docker-api and when checking I didn't see any "special network" code in https://github.com/cloudfoundry-community/cf-containers-broker/blob/master/app/models/docker_manager.rb. So I am actually puzzled by your finding. Anyhow, we have hundreds of users and containers and after deletion of the DB and restart, nobody complained/the existing and now restarted containers run just fine. They all run on their own reached only through the exposed port by the applications that use them.

@vlerenc not related to this issue, but could you send me an e-mail at sebastiaan@docker.com ? Wondering if you'd be interested to be added to our "testers" mailinglist

authors: mavenugo thaJeztah vlerenc


text:
Build endpoint abruptly closes connection when tag param is invalid

Originally reported in https://github.com/docker/docker-py/issues/978
Expected result: HTTP status 400 "Bad request", (ideally) with explicit error message about invalid parameter.
Observed result: Connection aborted - Connection reset by peer.

If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
For more information about reporting issues, see CONTRIBUTING.md.
You don't have to include this information if this is a feature request
(This is an automated, informational response)

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

/cc @duglin for builder

I tried myself on 1.10.3 and current master, it seems that this has already been addressed:
$ python dockerbuilder.py
Traceback (most recent call last):
  File "dockerbuilder.py", line 10, in <module>
    rm=True)]
  File "/usr/local/lib/python2.7/dist-packages/docker/client.py", line 230, in _stream_helper
    yield self._result(response)
  File "/usr/local/lib/python2.7/dist-packages/docker/client.py", line 150, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python2.7/dist-packages/docker/client.py", line 146, in _raise_for_status
    raise errors.APIError(e, response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("Error parsing reference: "ESSAI" is not a valid repository/tag")

Notice the last line: Error parsing reference: "ESSAI" is not a valid repository/tag

Thanks @yongtang
@shin- can you confirm it's fixed, then we can close this

Yes, looks like this is fixed in 1.11.0rc1 - Thank you!

authors: GordonTheTurtle shin- thaJeztah yongtang


text:
Docker NIC adapter shall not register DNS name with DNS server

Expected behavior
Checkbox on HNS Internal NIC for DNS registration shall be unchecked
Actual behavior
Box is checked
Information
HNS adapter shall have automatic DNS registration disabled. This setting registers private docker IP with Active Directory DNS and causing problems for internal communications
Steps to reproduce the behavior

ping @msabansal PTAL

authors: artisticcheese thaJeztah


text:
Incorrect warning about IP address binding

If I bind docker to 127.0.0.1, I get no warning, as expected:
$ sudo /usr/bin/docker -d -H tcp://127.0.0.1:2375                                            [13:53:42]
2014/11/08 13:53:48 docker daemon: 1.3.1 4e9bbfa; execdriver: native; graphdriver:
[d0ce11ba] +job serveapi(tcp://127.0.0.1:2375)
[info] Listening for HTTP on tcp (127.0.0.1:2375)
[d0ce11ba] +job init_networkdriver()
[d0ce11ba] -job init_networkdriver() = OK (0)
2014/11/08 13:53:48 WARNING: Your kernel does not support cgroup swap limit.
[info] Loading containers:
...........[info] : done.
[d0ce11ba] +job acceptconnections()
[d0ce11ba] -job acceptconnections() = OK (0)

However, if I say localhost, it gives me a big scary warning:
$ sudo /usr/bin/docker -d -H tcp://localhost:2375
2014/11/08 13:54:00 docker daemon: 1.3.1 4e9bbfa; execdriver: native; graphdriver:
[847b1621] +job serveapi(tcp://localhost:2375)
[info] Listening for HTTP on tcp (localhost:2375)
[info] /!\ DON'T BIND ON ANOTHER IP ADDRESS THAN 127.0.0.1 IF YOU DON'T KNOW WHAT YOU'RE DOING /!\
[847b1621] +job init_networkdriver()
[847b1621] -job init_networkdriver() = OK (0)
2014/11/08 13:54:01 WARNING: Your kernel does not support cgroup swap limit.
[info] Loading containers:
...........[info] : done.
[847b1621] +job acceptconnections()
[847b1621] -job acceptconnections() = OK (0)

I believe this is incorrect -- it should only warn if the hostname does not resolve to the 127.0.0.1 IP address.

the warning is there for that reason, closing

Nonetheless, shouldn't it be consistent between 'localhost' and '127.0.0.1'?

@jfrazelle oh maybe we should resolve and check if the IP is in 127/8, like we do for the registry?

authors: Wilfred jfrazelle tiborvass


text:
Allow to change the path where the plugins are loaded from.

This patch allows to set the plugins root at the deamon level.
I'm not super happy about the implementation, so all your feedback will be very welcome.
Signed-off-by: David Calavera david.calavera@gmail.com

oh man... I'd punt on this. Don't see why anyone would really need to customize this other than "look ma, I can customize things".

@cpuguy83 I totally agree with you, but I'm sure if we don't do it someone else will. To be clear, I'm okay with closing this PR and forget about it.

@cpuguy83 I just thought about a case where we'll want to change this, windows :wink:
Moving forward with this after a rebase.

@calavera In that case it's just a constant set in an _windows.go file :)

that's very true.

Not sure about this either; is there an actual use case already? If not, then I wonder if we should add it just because it may become handy.
Just my 0.02

Okay, I'm sold, closing this PR.

Reopening this because of #13859

Maybe we should make /var/lib/docker default? :)

For my normal testing, I start the Docker daemon with docker -d -D -g /data/docker - which is really its own mounted partition.
rather than hard coding the default to /var/lib/docker, perhaps it would be useful to default to {graphdir}/plugins.
however, I then note another pain point - the plugin API does not communicate daemon settings to it - so it doesn't know that in this case, it should probably make the the mounts in /data/docker.
tbh, I'm hoping the daemon can pass on some more client oriented info too, but that may be pushing it :)

@SvenDowideit it cannot go into {graphdir}/plugins. We already had that discussion, as @tianon pointed out, the drivers expect a defined structure.

Also, keep in mind that /var/lib/docker is meant to be ephemeral. You can always wipe out the directory, run docker run container and everything works. If we put the plugins directory there, that directory stops being ephemeral, because you'll want to save the plugins configuration.

Based on the plugins docs it seems that the plugins are either a unix socket or a spec file pointing to a unix socket (why not a symlink?). The accepted pattern for IPC unix sockets is to put them into /run/. So, I would suggest that the plugins be a combination of three locations:
/run/docker/plugins
/etc/docker/plugins
/usr/lib/docker/plugins
And have an inheritance order where /run overrides /etc and /etc overrides /usr on name collision. I would also suggest that both /etc and /usr only be allowed to contain spec files.

cc @icecrime

Sounds legit to me.
I don't like configurable plugin paths at all.

Sounds like a good idea to me too, I'll make the changes here and update the title to reflect that.

Closing. I'm going to start over in a new pull request.

authors: LK4D4 SvenDowideit calavera cpuguy83 philips thaJeztah


text:
Move networking api types to the api/types/networking package.

I believe this resolves the TODO that I removed. It moves the network types that are used as as part of the API to the appropriate package.
Settings is left in the old package, because it does not appear to be used by the api.
cc @calavera

awesome, thank you!
LGTM

LGTM

authors: calavera dnephin icecrime


text:
Optimize slow bottleneck test of TestRestartWithPolicyUserDefinedNetwork (#19425).

This fix tries to improve the slow bottleneck test of TestRestartWithPolicyUserDefinedNetwork in #19425.
In TestDockerNetworkInternalMode, the majority of the time are spent on ping test cases. In this fix IPv4 (-4) option is used explicitly to reduce the execution time. The result is almost ~90% of the improvement.
Signed-off-by: Yong Tang yong.tang.github@outlook.com

@thaJeztah I also updated this pull request with adding IPv4 (-4) option only. Let me know if there are any other issues.

Let me close this as well, per https://github.com/docker/docker/pull/21200#issuecomment-196930730. We've got https://github.com/docker/docker/issues/21224 tracking the IPv6 performance issue

authors: thaJeztah yongtang


text:
move func getNetwork into helper.go to keep consistent

I found that when cluster is getting resources like swarm, node, service, task via getSwarm, getNode, getService, and getTask, this functions are all in helper.go.
While in swarm cluster, getNetwork is also a kind of resource fetching. I think it is better to get all these functions together to keep consistent.
- What I did
1.move func getNetwork into helper.go to keep consistent
- How I did it
- How to verify it
- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)
Signed-off-by: allencloud allen.sun@daocloud.io

LGTM

authors: allencloud vieux


text:
Log tapper

Adds ability for tests to tap into the compose logs during the test run and check for expected output on the tested services.    Adds a new binary to test runner to tapping logs and listens to a unix socket for tap requests.
Closes #47

Probably need docs for this one since its use will not be self explanatory. Here is an example of how it could be used...
golem_tapper compose > /tmp/tap.out &
docker pull localregistry:5440/someimage
grep "something" /tmp/tap.out

ping @aaronlehmann

Looks good in general. I imagine the vast majority of use cases will involve looking for the presence or absence of a specific pattern in the log. It might be useful to allow the client to specify a regexp that can be evaluated on the server side, rather than sending the full log output for the client to evaluate. However, this may be a premature optimization.

I do agree about the patterns, I will create an feature issue for it when this gets merged.

authors: aaronlehmann dmcgowan

repo: docker/golem


text:
Network issues while doing in kitemaic as well as terminal

Hi,
am facing issue while installing spark from bigdata university through kitmatic as well as terminal.
Network timed out while trying to connect to https://index.docker.io/v1/repositories/bigdatauniversity/spark/images. You may want to check your internet connection or if you are behind a proxy.

Clicking the above link I get the proper json data. Are you behind a proxy? You may need to configure your VM for proper connectivity with the Proxy.
https://github.com/docker/kitematic/wiki/Common-Proxy-Issues-&-Fixes

Closing this issue due to lack of update/activity - Feel free to re-open if appropriate.

repo: docker/kitematic


text:
docker won't start

Expected behavior
docker gui via kitematic
Actual behavior
HTTP code is 404 which indicates error: undefined - {"message":"manifest for williwombat1066/joseph_williams:latest not found"}
Information about the Issue
Steps to reproduce the behavior

...
...

Go to https://hub.docker.com/r/williwombat1066/joseph_williams/tags/
Observe no latest tags were created for that image
Select a different tag in Kitematic (if one exists)
🎉  enjoy container running.

Ideally you open an issue with the repo owner to let them know how frustrating it is that they don't have a latest image tagged

authors: FrenchBen jtimothywilliams


text:
Virtual Box Setup gives "We're Sorry!"

Expected behavior
stops with errors after seconds
Actual behavior
Says needet to use VirtualBox - Setup. Then it runs to 99% says ""We're Sorry!" and stops with the erros bolow
Information about the Issue
Command failed: C:\Program Files\Docker Toolbox\docker-machine.exe -D create -d virtualbox --virtualbox-memory 2048 default,Docker Machine Version: 0.13.0, build 9ba6da9,Found binary path at C:\Program Files\Docker Toolbox\docker-machine.exe,Launching plugin server for driver virtualbox,Plugin server listening at address 127.0.0.1:52453,() Calling .GetVersion,Using API Version 1,() Calling .SetConfigRaw,() Calling .GetMachineName,(flag-lookup) Calling .GetMachineName,(flag-lookup) Calling .DriverName,(flag-lookup) Calling .GetCreateFlags,Found binary path at C:\Program Files\Docker Toolbox\docker-machine.exe,Launching plugin server for driver virtualbox,Plugin server listening at address 127.0.0.1:52457,() Calling .GetVersion,Using API Version 1,() Calling .SetConfigRaw,() Calling .GetMachineName,(default) Calling .GetMachineName,(default) Calling .DriverName,(default) Calling .GetCreateFlags,(default) Calling .SetConfigFromFlags,(default) Calling .PreCreateCheck,(default) DBG | COMMAND: C:\Program Files\Oracle\VirtualBox\VBoxManage.exe --version,(default) DBG | STDOUT:,(default) DBG | {,(default) DBG | 5.2.6r120293,(default) DBG | },(default) DBG | STDERR:,(default) DBG | {,(default) DBG | },(default) DBG | COMMAND: wmic computersystem get hypervisorpresent,(default) DBG | STDOUT:,(default) DBG | {,(default) DBG | HypervisorPresent ,(default) DBG | TRUE ,(default) DBG | ,(default) DBG | },Error with pre-create check: "This computer is running Hyper-V. VirtualBox won't boot a 64bits VM when Hyper-V is activated. Either use Hyper-V as a driver, or disable the Hyper-V hypervisor. (To skip this check, use --virtualbox-no-vtx-check)",open C:\Users\Administrator.docker\machine\machines\default\default\Logs\VBox.log: Das System kann den angegebenen Pfad nicht finden.,notifying bugsnag: [Error with pre-create check: "This computer is running Hyper-V. VirtualBox won't boot a 64bits VM when Hyper-V is activated. Either use Hyper-V as a driver, or disable the Hyper-V hypervisor. (To skip this check, use --virtualbox-no-vtx-check)"],

I had a similar error. This error page should be improved:

The error messages are logically organized in lines; still, they are displayed as an unreadable blob.
To make them display nicer, replace ,(?! ) with ,\n.

Since this is an "unexpected error", there should be a "report this error" button.

The last line tells you exactly what the error is:
This computer is running Hyper-V. VirtualBox won't boot a 64bits VM when Hyper-V is activated. Either use Hyper-V as a driver, or disable the Hyper-V hypervisor. (To skip this check, use --virtualbox-no-vtx-check)

authors: FrenchBen rillig sl5net


text:
something was wrong in this case, sql it dosnt run and some container have a problem like quickstart plz reply

Expected behavior
Actual behavior
Information about the Issue
Steps to reproduce the behavior

...
...

something was wrong in this case, sql it dosnt run and some container have a problem like quickstart plz reply

Can you please share your error

when i create a container sql in kitematic (ubuntu-18), this container is not turn on i don't know why

hello again,  when the container sql is created is turn off and return this problem(error: database is uninitialized and password option is not specified
You need to specify one of MYSQL_ROOT_PASSWORD, MYSQL_ALLOW_EMPTY_PASSWORD and MYSQL_RANDOM_ROOT_PASSWORD)

Please go through documentation for mysql docker. Follow the proper command to create a container accordingly for docker or docker compose
https://hub.docker.com/_/mysql/

ok,sql is run and he have a 2 port and 2 local host but the both not access in google chrome ?
plz reply

in the container mysql in kitematic i have this word (0 No logs for this container.) and i have a localhost and port but he didnt access in google chroma plz a i want a solution to this problem .
plz reply

Do tell me command that you use to run container

in cmd i create (docker run mysql) or in kitematic (new container ->i click to mysql and put the in selected tag the version a try more than one and click to create .. just this

You have to assign the port to access it from the outside. Try following command.
docker container run -p 3306:3306 -e MYSQL_ROOT_PASSWORD=my-secret-pw  mysql

sorry, failed the port in google chrome not open...!!!
what i can do !!

After running this command, run docker ps command on console. Send me output that you see

container id                image                          command                          created
status                           ports                              names
just this

please send details of this screen, everthing. A screenshot will be helpful


text:
Remove --lines flag for logs command

This flag isn't actually implemented, so it's probably confusing to users when it's shown in help. The name is wrong as well, since Docker Compose uses --tail for this.
Signed-off-by: Josh Curl josh@curl.me

@joshwget oh boy, I did add it without implementing it… my bad :sweat_smile:

No problem! I think I'll implement tail pretty soon :smile:

authors: joshwget vdemeester

repo: docker/libcompose


text:
Upgrade is broken

These lines should probably be removed: https://github.com/docker/machine/blob/master/libmachine/provision/ubuntu_systemd.go#L76-L93 as they are causing upgrade to fail due to the removed sources.list file.

Duplicate of #3009 ?

Seems to be so, yes.

OK, so, anyone who wants a fast fix to this issue, here is a script / sequence of commands that you can run which will put you back on track.
In the following block, make sure to replace mcn_name, lsb_dist, and dist_version with the correct values for your situation (e.g. change lsb_dist to debian and dist_version to jessie if on Debian Jessie).
The following was written for a machine I have running Ubuntu 15.10.
mcn_name='nathanleclaire-swarm-playground'
apt_url='https://apt.dockerproject.org'
lsb_dist='ubuntu'
dist_version='wily'
arch=$(docker-machine ssh ${mcn_name} dpkg --print-architecture)
repo='main'
docker-machine ssh ${mcn_name} "echo deb [arch=${arch}] ${apt_url}/repo ${lsb_dist}-${dist_version} ${repo} | sudo tee /etc/apt/sources.list.d/docker.list"
docker-machine ssh ${mcn_name} sudo apt-get update
docker-machine ssh ${mcn_name} sudo apt-get install -y docker-engine

I'm not too keen on introducing code directly into Machine itself to fix this issue, since a workaround for upgrade coded directly into the Machine source was what caused the issue in the first place.
I will be submitting a patch to remove the offending bits of code.  I'll have to confer with the other maintainers on whether or not we'll do a 0.7.1 release to include the fix that removes this destructive bug.  Thanks all for your patience in the meantime.

https://github.com/docker/machine/pull/3312

ok it works for digital ocean, thanks

👍

@nathanleclaire FWIW -- this also effected our generically provisioned debian jessie machines. my work around was to shell in and evaluate the get.docker.com script; and to manually fix the systemd unit file ( https://github.com/docker/machine/issues/3009#issuecomment-190778447 ). It's not limited to ubuntu.

@briceburg Yep, thanks -- the fix above should work for Jessie if the variables are changed as specified.

You can simple do:
docker-machine ssh <machine name> sudo mv /etc/apt/sources.list.d/docker.list /etc/apt/sources.list.d/docker-engine.list
and everything will be fine.

@nathanleclaire thank you, I tried your script but it seems to be not working for me or did I missed something?, sorry for the late reply. I pasted the script in a small utils_script
upgrade_machine () {
    mcn_name=${mcn_name:-$H}
    apt_url='https://apt.dockerproject.org'
    lsb_dist='ubuntu'
    dist_version='wily'
    arch=$(docker-machine ssh ${mcn_name} dpkg --print-architecture)
    repo='main'
    docker-machine ssh ${mcn_name} "echo deb [arch=${arch}] ${apt_url}/repo ${lsb_dist}-${dist_version} ${repo} | sudo tee /etc/apt/sources.list.d/docker.list"
    docker-machine ssh ${mcn_name} sudo apt-get update
    docker-machine ssh ${mcn_name} sudo apt-get install -y docker-engine
}

+ source /Users/jeanepaul/Developer/docker/docker-machine/utils_scripts.sh
++ set -e
+ upgrade_machine
+ mcn_name=dev02
+ apt_url=https://apt.dockerproject.org
+ lsb_dist=ubuntu
+ dist_version=wily
++ docker-machine ssh dev02 dpkg --print-architecture
+ arch=amd64
+ repo=main
+ docker-machine ssh dev02 'echo deb [arch=amd64] https://apt.dockerproject.org/repo ubuntu-wily main | sudo tee /etc/apt/sources.list.d/docker.list'
deb [arch=amd64] https://apt.dockerproject.org/repo ubuntu-wily main
+ docker-machine ssh dev02 sudo apt-get update
Ign http://asia-east1.gce.archive.ubuntu.com trusty InRelease
Hit http://asia-east1.gce.archive.ubuntu.com trusty-updates InRelease
Hit http://asia-east1.gce.archive.ubuntu.com trusty Release.gpg
Hit http://asia-east1.gce.archive.ubuntu.com trusty Release
Hit http://asia-east1.gce.archive.ubuntu.com trusty-updates/main Sources
Hit http://asia-east1.gce.archive.ubuntu.com trusty-updates/universe Sources
Hit http://asia-east1.gce.archive.ubuntu.com trusty-updates/main amd64 Packages
Hit http://asia-east1.gce.archive.ubuntu.com trusty-updates/universe amd64 Packages
Hit http://asia-east1.gce.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://asia-east1.gce.archive.ubuntu.com trusty-updates/universe Translation-en
Hit http://asia-east1.gce.archive.ubuntu.com trusty/main Sources
Hit http://asia-east1.gce.archive.ubuntu.com trusty/universe Sources
Hit http://asia-east1.gce.archive.ubuntu.com trusty/main amd64 Packages
Hit http://asia-east1.gce.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://asia-east1.gce.archive.ubuntu.com trusty/main Translation-en
Hit http://asia-east1.gce.archive.ubuntu.com trusty/universe Translation-en
Ign http://asia-east1.gce.archive.ubuntu.com trusty/main Translation-en_US
Ign http://asia-east1.gce.archive.ubuntu.com trusty/universe Translation-en_US
Get:1 https://apt.dockerproject.org ubuntu-wily InRelease
Ign https://apt.dockerproject.org ubuntu-wily InRelease
Hit https://apt.dockerproject.org ubuntu-wily Release.gpg
Hit https://apt.dockerproject.org ubuntu-wily Release
Hit https://apt.dockerproject.org ubuntu-wily/main amd64 Packages
Ign http://archive.canonical.com trusty InRelease
Get:2 https://apt.dockerproject.org ubuntu-wily/main Translation-en_US
Hit http://security.ubuntu.com trusty-security InRelease
Ign https://apt.dockerproject.org ubuntu-wily/main Translation-en_US
Hit http://archive.canonical.com trusty Release.gpg
Ign https://apt.dockerproject.org ubuntu-wily/main Translation-en
Hit http://security.ubuntu.com trusty-security/main Sources
Hit http://archive.canonical.com trusty Release
Hit http://security.ubuntu.com trusty-security/universe Sources
Hit http://archive.canonical.com trusty/partner amd64 Packages
Hit http://security.ubuntu.com trusty-security/main amd64 Packages
Hit http://archive.canonical.com trusty/partner Translation-en
Hit http://security.ubuntu.com trusty-security/universe amd64 Packages
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Reading package lists...
+ docker-machine ssh dev02 sudo apt-get install -y docker-engine
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  linux-headers-3.16.0-49 linux-headers-3.16.0-49-generic
  linux-headers-3.16.0-67 linux-headers-3.16.0-67-generic
  linux-headers-3.16.0-69 linux-headers-3.16.0-69-generic
  linux-image-3.16.0-49-generic linux-image-3.16.0-67-generic
  linux-image-3.16.0-69-generic linux-image-3.19.0-56-generic
  linux-image-extra-3.16.0-49-generic linux-image-extra-3.16.0-67-generic
  linux-image-extra-3.16.0-69-generic linux-image-extra-3.19.0-56-generic
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
  docker-engine
Recommended packages:
  yubico-piv-tool
0 upgraded, 0 newly installed, 0 to remove and 86 not upgraded.
➜  azure git:(master) ✗

this is how I have created the machine btw
docker-machine -D create \
    --driver google \
    --google-project $PROJECT_ID \
    --google-zone $MACHINE_ZONE \
    --google-machine-type $MACHINE_TYPE \
    --google-disk-size $MACHINE_DISK_SIZE $NAME

@Hokutosei Did you read this bit?

In the following block, make sure to replace mcn_name, lsb_dist, and dist_version with the correct values for your situation (e.g. change lsb_dist to debian and dist_version to jessie if on Debian Jessie).

It looks like your version of Ubuntu may be trusty, not wily.  Did you set dist_version=trusty?

@nathanleclaire damn I did not set that! it worked! really sorry for the trouble. script work perfectly thanks you!

this should make it general
Just save the following to a file (i.e.: fixit.sh).
Then execute it with the name of your machine as a parameter.
i.e: $ sh fixit.sh my_machine
Based on @nathanleclaire code.
Modifications:

made the machine name a parameter.
distribution name is captured from lsb_release output.
dist_version is captured from lsb_release output.

dm_name=$1
apt_url='https://apt.dockerproject.org'
lsb_dist=$(docker-machine ssh ${dm_name} "lsb_release -a 2>/dev/null | grep Distributor | cut -f2 | tr '[:upper:]' '[:lower:]'")

dist_version=$(docker-machine ssh ${dm_name} "lsb_release -a 2>/dev/null | grep Codename | cut -f2")
arch=$(docker-machine ssh ${dm_name} dpkg --print-architecture)
repo='main'

echo deb [arch=${arch}] ${apt_url}/repo ${lsb_dist}-${dist_version} ${repo}
exit

docker-machine ssh ${dm_name} "echo deb [arch=${arch}] ${apt_url}/repo ${lsb_dist}-${dist_version} ${repo} | sudo tee /etc/apt/sources.list.d/docker.list"
docker-machine ssh ${dm_name} sudo apt-get update
docker-machine ssh ${dm_name} sudo apt-get install -y docker-engine

@ctroncoso Script looks pretty good, but are the echo and exit lines in the middle intentional?

Oooops!... @nathanleclaire. Pasted the debug version. Fixed the post.

Experience same issue with softlayer machines. The @ctroncoso's script fixed.

Just a quick warning, the above worked for me, but upon running docker-compose up -d I got:
ERROR: for db  Unable to remove filesystem for 985ac64bb671214d61335a609a3605f254b52ca823c47c37002bb96242027a20: remove /var/lib/docker/containers/985ac64bb671214d61335a609a3605f254b52ca823c47c37002bb96242027a20/shm: device or resource busy
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "compose/cli/main.py", line 63, in main
AttributeError: 'ProjectError' object has no attribute 'msg'
docker-compose returned -1

Still investigating, I'll post more info as I figure it out.

Found my way here because I'm trying to upgrade 1.10.3 to 1.11.1b but the docker-machine is centos image running on GCE so script above won't work and upgrade fails

@RobMaskell docker-machine upgrade does not work?  What is the output for docker-machine --debug upgrade machinename ?

The referenced bug seems to be Ubuntu/CentOS specific, so it may not be the same issue.
However, it's possible that the upstream packages / install method changed for RH-based distros.

@nathanleclaire had an issue for ages on 1.10.3 that docker on the GCE image would't start up correctly (suspected timing issue) and as it was annoying me I figured it might be fixed in later version so installed the Docker toolbox onto my mac, which upgraded my default virtualbox instance fine. Ran the upgrade on the first of my GCE docker machines and it failed. Looked at a few issues on here and they all seemed to link back to this one. I can't provide the output you ask as I deleted the whole machine and recreated it at 1.11.1b. Good news is it seems to startup ok now.

Greetings,
upgraded my local docker to docker mac, and then upgrade docker-machines, with docker-machine upgrade
➜  accounting git:(develop) dm upgrade dev02
Waiting for SSH to be available...
Detecting the provisioner...
Upgrading docker...
Restarting docker...
➜  accounting git:(develop) dm ls
NAME      ACTIVE   DRIVER       STATE     URL                          SWARM   DOCKER    ERRORS
azure01   -        azure        Running   tcp://xxx.xxx.xx.xxx:2376            v1.11.2
dev       -        virtualbox   Running   tcp://xxx.xxx.xx.xxx:2376            v1.11.0
dev01     -        google                                                      Unknown
dev02     -        google       Running   tcp://xxx.xxx.xx.xxx:2376            v1.11.2

tried the above script again,
    mcn_name=${mcn_name:-$H}
    apt_url='https://apt.dockerproject.org'
    lsb_dist='ubuntu'
    dist_version='trusty'
    arch=$(docker-machine ssh ${mcn_name} dpkg --print-architecture)
    repo='main'
    docker-machine ssh ${mcn_name} "echo deb [arch=${arch}] ${apt_url}/repo ${lsb_dist}-${dist_version} ${repo} | sudo tee /etc/apt/sources.list.d/docker.list"
    docker-machine ssh ${mcn_name} sudo apt-get update
    docker-machine ssh ${mcn_name} sudo apt-get install -y docker-engine

unfortunately, no, not working..
➜  accounting git:(develop) docker version
Client:
 Version:      1.12.0-rc2
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   906eacd
 Built:        Fri Jun 17 20:35:33 2016
 OS/Arch:      darwin/amd64
 Experimental: true

Server:
 Version:      1.12.0-rc2
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   a7119de
 Built:        Fri Jun 17 22:09:20 2016
 OS/Arch:      linux/amd64
 Experimental: true

➜  accounting git:(develop) docker-machine version
docker-machine version 0.7.0, build a650a40

can't even do docker-machine ls without doing unset DOCKER_TLS_VERIFY. with set, It give me  Are you trying to connect to a TLS-enabled daemon without TLS? might do this on different post

@Hokutosei What's the output of this script? I'd also make sure that you're using Bash, on the off chance there are incompatibilities.
dm_name='dev02'
apt_url='https://apt.dockerproject.org'
lsb_dist=$(docker-machine ssh ${dm_name} "lsb_release -a 2>/dev/null | grep Distributor | cut -f2 | tr '[:upper:]' '[:lower:]'")

dist_version=$(docker-machine ssh ${dm_name} "lsb_release -a 2>/dev/null | grep Codename | cut -f2")
arch=$(docker-machine ssh ${dm_name} dpkg --print-architecture)
repo='main'

docker-machine ssh ${dm_name} "echo deb [arch=${arch}] ${apt_url}/repo ${lsb_dist}-${dist_version} ${repo} | sudo tee /etc/apt/sources.list.d/docker.list"
docker-machine ssh ${dm_name} sudo apt-get update
docker-machine ssh ${dm_name} sudo apt-get install -y docker-engine

@christianbundy sorry, I could not retrieve what are the logs, was on different machines. after an hour and half with frustrations  and it was 3am, I gave up and manually install the latest version from this post.

@christianbundy
thanks for the help..tried your script, but it seems to be not working..
+ dm_name=azure01
+ apt_url=https://apt.dockerproject.org
++ docker-machine ssh azure01 'lsb_release -a 2>/dev/null | grep Distributor | cut -f2 | tr '\''[:upper:]'\'' '\''[:lower:]'\'''
+ lsb_dist=ubuntu
++ docker-machine ssh azure01 'lsb_release -a 2>/dev/null | grep Codename | cut -f2'
+ dist_version=wily
++ docker-machine ssh azure01 dpkg --print-architecture
+ arch=amd64
+ repo=main
+ docker-machine ssh azure01 'echo deb [arch=amd64] https://apt.dockerproject.org/repo ubuntu-wily main | sudo tee /etc/apt/sources.list.d/docker.list'
deb [arch=amd64] https://apt.dockerproject.org/repo ubuntu-wily main
+ docker-machine ssh azure01 sudo apt-get update
Hit https://apt.dockerproject.org ubuntu-wily InRelease
Hit https://apt.dockerproject.org ubuntu-wily/main amd64 Packages
Hit http://azure.archive.ubuntu.com wily InRelease
Get:1 https://apt.dockerproject.org ubuntu-wily/main Translation-en_US [454 B]
Get:2 https://apt.dockerproject.org ubuntu-wily/main Translation-en [454 B]
Get:3 https://apt.dockerproject.org ubuntu-wily/main Translation-en_US [454 B]
Get:4 https://apt.dockerproject.org ubuntu-wily/main Translation-en [454 B]
Get:5 https://apt.dockerproject.org ubuntu-wily/main Translation-en_US [454 B]
Get:6 https://apt.dockerproject.org ubuntu-wily/main Translation-en [454 B]
Get:7 https://apt.dockerproject.org ubuntu-wily/main Translation-en_US [454 B]
Get:8 https://apt.dockerproject.org ubuntu-wily/main Translation-en [454 B]
Get:9 https://apt.dockerproject.org ubuntu-wily/main Translation-en_US [454 B]
Ign https://apt.dockerproject.org ubuntu-wily/main Translation-en_US
Hit http://security.ubuntu.com wily-security InRelease
Get:10 https://apt.dockerproject.org ubuntu-wily/main Translation-en [454 B]
Ign https://apt.dockerproject.org ubuntu-wily/main Translation-en
Hit http://azure.archive.ubuntu.com wily-updates InRelease
Hit http://azure.archive.ubuntu.com wily-backports InRelease
Hit http://azure.archive.ubuntu.com wily/main Sources
Hit http://security.ubuntu.com wily-security/main Sources
Hit http://azure.archive.ubuntu.com wily/restricted Sources
Hit http://azure.archive.ubuntu.com wily/universe Sources
Hit http://security.ubuntu.com wily-security/restricted Sources
Hit http://azure.archive.ubuntu.com wily/multiverse Sources
Hit http://azure.archive.ubuntu.com wily/main amd64 Packages
Hit http://security.ubuntu.com wily-security/universe Sources
Hit http://azure.archive.ubuntu.com wily/restricted amd64 Packages
Hit http://azure.archive.ubuntu.com wily/universe amd64 Packages
Hit http://azure.archive.ubuntu.com wily/multiverse amd64 Packages
Hit http://security.ubuntu.com wily-security/multiverse Sources
Hit http://azure.archive.ubuntu.com wily/main Translation-en
Hit http://azure.archive.ubuntu.com wily/multiverse Translation-en
Hit http://security.ubuntu.com wily-security/main amd64 Packages
Hit http://azure.archive.ubuntu.com wily/restricted Translation-en
.........
Reading package lists...
+ docker-machine ssh azure01 sudo apt-get install -y docker-engine
Reading package lists...
Building dependency tree...
Reading state information...
docker-engine is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 32 not upgraded.

bash-3.2$ docker-machine ssh azure01
docker-user@azure01:~$ sudo -i
root@azure01:~# docker version
Client:
 Version:      1.11.2
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   b9f10c9
 Built:        Wed Jun  1 21:54:25 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.11.2
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   b9f10c9
 Built:        Wed Jun  1 21:54:25 2016
 OS/Arch:      linux/amd64

bash-3.2$ docker-machine ls
NAME      ACTIVE   DRIVER       STATE     URL                          SWARM   DOCKER        ERRORS
azure01   -        azure        Running   tcp://xxx.xx.xxx.xxx:2376            v1.11.2

pardon me, am I missing something?

@Hokutosei Did you manually install v1.12.0-rc2? Running docker-machine upgrade just installed the most recent stable version of docker-engine from here, and since v1.12.0 hasn't been released it isn't an install candidate.

@christianbundy installed it from here and prior to 1.12.0-rc2 versions,  docker-machine upgrade seems to be not working to me at all.
and yeah, not even RC yet, so until then, its just wait and see... can't downgrade too, lots of work that needs to be done today. it just sucks, upgrading my dev machine for everyday job is a bad idea.

@Hokutosei What version are you expecting when you run docker-machine upgrade? It looks like you're already on the most stable version. If you want to upgrade azure01 to v1.12.0-rc2 you can do that by running docker-machine ssh azure01 and then running curl -fsSL https://test.docker.com/ | sh. If you want to be a little more risky you can run curl -fsSL https://experimental.docker.com/ | sh for experimental features, but I don't know how deep into the rabbit hole you want to go.

@christianbundy I was expecting it to be the latest, if not atleast don't give me an error, if not. from my dev machine, doing something to docker-machines (e.g azure01)
Error response from daemon: client is newer than server (client API version: 1.24, server API version: 1.23)

anyway, I am manually upgrading just to fix that error, because I need it to work asap.
thank you, btw.

@Hokutosei You can also set DOCKER_API_VERSION to work around that.

@schmunk42 sick! tried it on my old dev virtBox(v1.11.0) it worked like magic  😄
➜  docker-machine git:(master) ✗ dm ls
NAME      ACTIVE   DRIVER       STATE     URL                          SWARM   DOCKER        ERRORS
azure01   -        azure        Running   tcp://xxx.xx.xxx.xxx:2376            v1.12.0-rc2
dev       *        virtualbox   Running   tcp://xxx.xx.xxx.xxx:2376            v1.11.0

➜  docker-machine git:(master) ✗ DOCKER_API_VERSION=1.23 HOST=dev do remote ps -a
+ docker -H tcp://192.168.99.100:2376 ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                   PORTS               NAMES
d5ea64cf83d2        b-eee/nginx-lb      "nginx -g 'daemon off"   4 weeks ago         Exited (0) 2 weeks ago                       tiny_hugle
e6eb9fbbd0fa        b-eee/nginx-lb      "nginx"                  4 weeks ago         Exited (0) 4 weeks ago                       suspicious_goldstine

@schmunk42 's  export DOCKER_API_VERSION=1.xx is the simplest and most elegant way to solve this problem.

I am trying to deploy a stack but I am getting an issue -:
Command -:
docker  -H ${SWARM_SERVER_NAME} stack deploy --prune -c mgmt-compose.yml build
Error -:
docker stack deploy requires API version 1.25, but the Docker daemon API version is
docker version
Client:
Version:       17.12.1-ce
API version:   1.35
Go version:    go1.9.4
Git commit:    7390fc6
Built: Tue Feb 27 22:17:53 2018
OS/Arch:       linux/amd64
Server:
Engine:
Version:      17.12.1-ce
API version:  1.35 (minimum version 1.12)
Go version:   go1.9.4
Git commit:   7390fc6
Built:        Tue Feb 27 22:16:25 2018
OS/Arch:      linux/amd64
Experimental: true
OS -: Ubuntu 17.10:latest
Any solution or suggestion will be highly appreciable.
Thanks
Uttam

authors: AstraSerg Hokutosei RobMaskell brettdh briceburg christianbundy ctroncoso gittycat mxab nathanleclaire s17t schmunk42 uttamanand

repo: docker/machine


text:
Wait for Docker daemon on host start

cc @docker/machine-maintainers
Closes https://github.com/docker/machine/issues/2175
(some of the instances of the problem, at least)
You should be able to duplicate this bug by doing something like this with the most recent Machine RC:
$ docker-machine stop
$ docker-machine start; docker-machine env
Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.99.100:2376": dial tcp 192.168.99.100:2376: getsockopt: connection refused
You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
Be advised that this will trigger a Docker daemon restart which will stop running containers.

This patch fixes the issue by waiting for the Docker daemon as part of the Host Start.
Signed-off-by: Nathan LeClaire nathan.leclaire@gmail.com

@vikstrous FYI

@nathanleclaire I'm going to take a look this morning

LGTM

Will follow up with something similar on Restart()

authors: dgageot nathanleclaire


text:
Fix small docs typo

Signed-off-by: Nathan LeClaire nathan.leclaire@gmail.com

cc @ehazlett

LGTM

authors: ehazlett nathanleclaire


text:
Fix a typo in version.go

Signed-off-by: Yuan Sun sunyuan3@huawei.com

Thanks!  LGTM


text:
Feature/4229/file tree

For dockstore/dockstore#4229
Added a file tree dialog that's shown when the number of files exceeds 10 in a workflow version
The core issue however, is not quite fixed because it's very much possible for a workflow with under 10 files to have two files with the same relative path. This could be easily resolved by switching to absolute path but that would result in a very long, unreadable file path.
Also removed the light blue border of input text focus. We don't need it if using material design. If we notice there is a place that uses it, we should convert that to material instead.

Actually tried this locally, unlike the draft PR. I don't like how the dialog keeps resizing as the tree expands/collapses. Could it fixed size, or is the problem you don't know what the fixed size should be?

That's odd. it's supposed to be fixed size. Wonder if the search bar affected things

Edit: you were talking about vertical (not horizontal). We use variable size vertical for every modal/dialog so far. If you feel strongly about not using variable size, make an issue, discuss with Kim, and we can convert all modals (organization description, edit version, etc) to non-variable size vertical at once.

This one seems of a different magnitude than the others -- one keypress can radically change the size. But I will log separately.

Edit: you were talking about vertical (not horizontal). We use variable size vertical for every modal/dialog so far. If you feel strongly about not using variable size, make an issue, discuss with Kim, and we can convert all modals (organization description, edit version, etc) to non-variable size vertical at once.

This one seems of a different magnitude than the others -- one keypress can radically change the size. But I will log separately.

Like I said, the modals are all variable in size and this issue would be present in all past and future dialogs that have changing (or rapidly changing content). So no, the dialog behavior is not different (though this may have made it more pronounced). So when you log an issue for this. The expected behaviour needs to work for everything.

authors: coverbeck garyluu


text:
Hook up filter for #2868

n/t

 Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- dockstore-webservice/src/main/java/io/openapi/api/impl/ToolsApiServiceImpl.java  4

See the complete overview on Codacy

Commit references issue but not PR, not sure if that's intentional

Commit references issue but not PR, not sure if that's intentional

Either/or shrug

Commit references issue but not PR, not sure if that's intentional

Either/or shrug

Looks like it's less obvious that it's merged when looking at the issue

authors: denis-yuen garyluu

repo: dockstore/dockstore


text:
Adjust cron task for Shopware 6.5.0

From Shopware 6.5.0, the cron task call must look like this (failed|async), otherwise it will not be running due to decision dialog.

Possibly requires to clear the cache to have the new schema

aaaaaa
super, thank you for this
but quick question...the commited file only has comments?
so i guess i should integrate it, so that its automatically done like this in all versions >= 6.5?

Yes, I noticed there isn't any versioning of the cron file yet, and I'm not familiar how the image building in Dockware works

hi there
thanks ive added it now
i had to convert to a twig and do an if condition, so i need to close this PR
but i've added you to the changelog with a thanks contribution entry :)

great 🚀

authors: boxblinkracer btxtiger

repo: dockware/dockware


text:
Avoid static init problem in insufficient_procs() (MPI)

Description
Reverses an incidental change that came along with #623. That PR gives a proper initial value for world_size_before_init. Incidentally, it also adds an evaluation of this variable possibly prior to its initialization. (The function insufficient_procs() is called before entering main()). For me it caused all MPI_TEST_CASE to be marked as skipped, even when the actual process count was sufficient. The present PR fixes this problem.

@philbucher Could you take a look when you have the time?

Hi,
Thanks for noticing the bug! I tested your PR on my side and I see no regression. So I am OK to merge it.

@philbucher Sorry to bother you again, but do you think you could give a last lookover to this and the other PR? I'd really like to merge them before 2.4.9 and I'd really like to release that update in the near future!

Sorry for the delay, I posted one more comment. I am not sure if relevant, please go ahead if necessary, it was only about a detail anyway
Thanks!

Thanks for your contribution! :)

authors: BerengerBerthoul Saalvage philbucher starintheuniverse

repo: doctest/doctest


text:
[RFC] Added support for OneToMany with orphanRemoval.

Replacing entire collection now deletes the replaced collection (scheduled for deletion). No event handling is done as it happens at DBAL level.
Fixes DDC-3644

I was not able to break this, despite effort. :+1:

I don't agree with this approach: we had it before and it was reverted before 2.5 because a OneToMany association is always the inverse side of an association. I actually think we should keep it like that: the inverse side causes no DB-level changes.
My previous implementation was also simply using the entity persisters for the associations to apply the deletes, which IMO is cleaner and simpler.
What is the use-case that brought you here?

@Ocramius I don't necessarily agree. This is indeed a valid case.
Whenever you configure a *ToMany, or *ToOne, you have the ability to configure orphanRemoval. However, on ManyToMany, orphanRemoval only happens on owning side.
Unfortunately, swapping collections should trigger the orphanRemoval normally. In ManyToMany cases, this will always happen (as we only delete the join table records), but for OneToMany things gets hairy. There's no way to delete the weak entities and we can easily put UoW in an inconsistent state. Look at this example:
$entityManager->persist($developerGroup = new Group('developer'));
$entityManager->persist($managerGroup = new Group('manager'));
$entityManager->persist($executiveGroup = new Group('executive'));

$user->setGroups(new ArrayCollection([$developerGroup, $managerGroup]));

$entityManager->persist($user);
$entityManager->flush();

$user->setGroups(new ArrayCollection([$executiveGroup]));

$entityManager->persist($user);
$entityManager->flush();

echo count($user->getGroups()); // prints 1

$entityManager->clear();
$user = $entityManager->find('User', $user->getId());

echo count($user->getGroups()); // prints 3

What happened? We basically replaced the collection with a new one. Conceptually, we should take into consideration 2 execution paths:

with orphanRemoval: we should exclude the weak entities (exactly what this PR does)
without orphanRemoval: we should set FK to null (Hibernate takes this approach always, and we agreed that it is not the unique solution for orphanRemoval case)

What you presented above (prints 1 vs prints 3) is an unrelated quirk related to the fact that we don't track swapped collections.
I don't think it has anything to do with the proposed change: it's a different bug that requires UoW changes rather than collection persister changes.

@Ocramius I think you're not getting the purpose of this PR.
We do have tracking for swapped collections in UoW and it does that in ManyToMany associations. The OneToMany tracking culminates in a CollectionPersister call to delete(), but there's no implementation for this specific scenario in the method. All I've done was add the implementation for this case (which is valid as per unit test before/after).

As per discussion with @guilhermeblanco, I misunderstood the problem. Will need to review again.

This issue bit me today.  Any updates on this getting merged?

Merged into master after manual rebase via doctrine/doctrine2@2383ccfba583c9434baa447a97434256863f8f53

Thanks guys!

authors: Ocramius guilhermeblanco richardfullmer zeroedin-bill


text:
Added deprecation warnings for 2.x

Based on UPGRADE document for 3.0.
Deprecations added for:

EntityManager::copy() method
EntityManager::merge() method
EntityManager::detach() method
EntityManager::flush() calls with arguments
YAML mapping driver
Version class
Proxy interface

I have not added any notes about mapping/metadata changes here because it's still pretty unstable.
Tests marked as @legacy use any of the deprecated methods.
Added explicit deprecation tests for deprecated methods.
I'd like to get these into 2.6, there is no reason to delay it.

We just need to keep in mind that our plan is to release 2.6 WITHOUT the deprecation notices.

Well, the sooner we ship them, the better for users, but if there are other reasons, 2.7 is fine by me.

Updated, added deprecations for codegen (6870).

I've also added deprecatons for:

Configuration::getProxy*()
ProxyFactory
proxy Autoloader

Also partially imported UPGRADE notes from develop and reworded them (no mapping-related changes since it's unfinished).

I think deprecations for named queries should be added as well.
https://github.com/doctrine/doctrine2/blob/master/UPGRADE.md#bc-break-removed-support-for-named-queries-and-named-native-queries


text:
Identifier getters in proxies are non-lazy if identifier properties are in traits

When generating proxies from documents: if a document has its Doctrine\ODM\MongoDB\Mapping\Annotations\Id annotation set on the $id property it is getting from a trait, the proxy gets generated and works, but in the proxy class the getId() field is non-lazy like any other property like so:
/**
 * {@inheritDoc}
 */
public function getId()
{

    $this->__initializer__ && $this->__initializer__->__invoke($this, 'getId', []);

    return parent::getId();
}

If I move the $id property to the document either directly or to any of its ancestor classes indirectly, then in the generated proxy the getId() method gets generated to be lazy as expected:
/**
 * {@inheritDoc}
 */
public function getId()
{
    if ($this->__isInitialized__ === false) {
        return  parent::getId();
    }

    $this->__initializer__ && $this->__initializer__->__invoke($this, 'getId', []);

    return parent::getId();
}

In short: IDs in traits are non-lazy

@amcsi Proxy generation is handled entirely by doctrine/common - would you mind reposting it there?

Okay, closing in favor of doctrine/common#800

authors: amcsi malarzm

repo: doctrine/mongodb-odm