text:
Update HTcondor python bindings to 10.0.1

Impact of the new feature
The following feature will update the version of condor we use to submit via python (the HTCondor python bindings0
Is your feature request related to a problem? Please describe.
This is related to #11402. The current version we use for the bindings (9.2.0) is compatible with the HTCondor schedd 10.0.2. However, we should move eventually to the current stable release v10.
Describe the solution you'd like
Update the cmsdist recipe with HTCondor 10.0.1
https://github.com/cms-sw/cmsdist/blob/comp_gcc630/condorpy3.spec
Describe alternatives you've considered
Stay with 9.2.0

@khurtado Kenyi, can you please reach out to Farrukh and someone from CERN SI to understand if this upgrade has to be considered for this quarter?

Given the known bug (already fixed) in HTCondor reported in this message:
https://github.com/dmwm/WMCore/issues/11402#issuecomment-1553227571
I think we should consider this as high priority for the next quarter.

I just managed to build HTCondor - still via the RPM system - on version 10.2.3, see:
https://github.com/cms-sw/cmsdist/pull/8582
this version should fix the ServerTime attribute retrieved from the schedd, which makes debugging and some agent operations a bit harder. Still to be tested though.

HTCondor python bindings version 10.2.3 has been provided in
https://github.com/cms-sw/cmsdist/pull/8582
and basic tests were successful.
@germanfgv @LinaresToine once you start working on a new T0 agent release, make sure the t0 spec gets migrated to this py3-htcondor.spec spec (instead of the condorpy3 one).

authors: amaltaro khurtado

repo: dmwm/WMCore


text:
Changing priority in assignment-approved fails

https://cmsweb.cern.ch/reqmgr2/fetch?rid=pdmvserv_task_PPD-RunIISummer17PrePremix-00006__v1_T_180212_090017_6410 trying to set up to 111000 gives people
Server error message
Invalid spec parameter value: There are invalid arguments for no-status update: {u'SoftTimeout': 129600, u'BlockCloseMaxEvents': 25000000, u'MaxMergeEvents': 100000000, u'CustodialSites': u'', u'Ma...
please advise, and prioritize a fix

Were you trying to change just priority without changing the status to assignment?

Yes, without changing the request status. I just reproduced this error (I think I put in this bug in my latest changes to that code...)
Jean-Roch, changing the priority via REST API still works. We'll have this fixed by the next production deployment.

And this is why it fails via web UI, because there is a stricter input arg validation now:
https://github.com/dmwm/WMCore/pull/8419/files#diff-c4879fa72848db5e9f2c06c100bdd3c0R382
and when we GET a request via reqmgr2 UI, it loads most of the assignment parameters, which is posted again when one pushes the submit button to change "only" the priority.
There are two ways to fix it:

hard way: when one post arguments for a request in assignment-approved, we first cross-check which arguments changed and we pass only those upstream
easier way: when there is a "no status update", we only make changes to RequestPriority or global workqueue stats, completely disregarding any other parameters.

I'm of course going to walk the easier path :)

source: gharchive/issue

authors: amaltaro ticoann vlimant

license_type: permissive

license_source: github-api


text:
Investigate listDatasetFileDetails DBSClient API

During testing of WMCore DBS3Reader I found that both Py/Go server takes significant amount of time on the following DBS3Reader API:
     dataset="/VBF1Parked/Run2012D-v1/RAW"
     res = reader.listDatasetFileDetails(dataset)

This API call by itself has several for (nested loops) and received data from
       fileDetails = self.getFileListByDataset(dataset=datasetPath, validFileOnly=validFileOnly, det    ail=True
        blocks = set()  # the set of blocks of the dataset
        # Iterate over the files and prepare the set of blocks and a dict where the keys are the files
        files = {}
        for f in fileDetails:
            blocks.add(f['block_name'])
            files[f['logical_file_name']] = remapDBS3Keys(f, stringify=True)
            files[f['logical_file_name']]['ValidFile'] = f['is_file_valid']
            files[f['logical_file_name']]['Lumis'] = {}
            files[f['logical_file_name']]['Parents'] = []
        # Iterate over the blocks and get parents and lumis
        for blockName in blocks:
            # get the parents
            if getParents:
                parents = self.dbs.listFileParents(block_name=blockName)
                for p in parents:
                    if p['logical_file_name'] in files:  # invalid files are not there if validFileOnly=1
                        files[p['logical_file_name']]['Parents'].extend(p['parent_logical_file_name'])

            if getLumis:
                # get the lumis
                file_lumis = self.dbs.listFileLumis(block_name=blockName)
                for f in file_lumis:
                    if f['logical_file_name'] in files:  # invalid files are not there if validFileOnly=1
                        if f['run_num'] in files[f['logical_file_name']]['Lumis']:
                            files[f['logical_file_name']]['Lumis'][f['run_num']].extend(f['lumi_section_num'])
                        else:
                            files[f['logical_file_name']]['Lumis'][f['run_num']] = f['lumi_section_num']

        return files

I suggest to provide individual APIs to better understand timing spend in them and within internal (nested) for loops of DBS3Reader API call.

this is done now, closing the ticket.

license: MIT


text:
More options for content block preview (redirect)

User Story
As a CMS user, I want to be able to access content block options from a block, so that I can do this while reviewing the block at a page level.
Acceptance Criteria

[ ] Content blocks have a clickable ‘more options’ icon, as per the wireframe: https://projects.invisionapp.com/share/ZRL4T2HQAGB#/screens/301923101
[ ] Clicking the icon presents a popup with the following (clickable) options:
[ ] Clicking 'Content' takes user to existing content page tab
[ ] Clicking ‘History’ takes user to existing History page tab
[ ] Clicking ‘Settings’ takes user to existing Settings page tab
[ ] Any other tabs that are top level are also included and link to existing functionality, e.g link tracking
[ ] Mouse hover styling is present for options in popup
[ ] More options icon is still available when the block is expanded (as per https://github.com/silverstripe/silverstripe-elemental-blocks/issues/43)

Notes

Further options for versioning a block are captured here: https://github.com/dnadesign/silverstripe-elemental/issues/290

We are going to take a little bit of a performance hit by having to load all the CMS form fields for every block when you load a page in the CMS.
Relevant comment from @chillu on recent work for a similar feature: https://github.com/symbiote/silverstripe-gridfieldextensions/pull/225#issuecomment-347404105

Thanks for flagging this Robbie! Can we please measure before-after state on a non-trivial page (e.g. using ten instances of five different blocks)? In general, we should move this to a client-side cache which gets refreshed in the background, for the (usually small) chance that the form fields on this particular record are different from the ones we've cached. In which case React can re-render the fields. We've split out structure from data in the form schema to allow for this, right? There's potential for edge cases, where you started editing a form field which is then removed in the re-render (because of conditional logic in getCMSFields().
Ideally this should happen via GraphQL, which comes built-in with powerful client caching (and invalidation). Which means getting form schemas through GraphQL, and the "form registry" we're using for AssetAdmin.
Again, outlining the long term vision here, let's try to get as close as possible to this. OSSers team is happy to help with any GraphQL form schema work if that's required as a baseline.

So I've been thinking a little about this. The performance hit for only 3 blocks is pretty severe - about 20% of the request is spent on getCMSFields. This would be way higher for a full page of blocks - and that didn't include more complex blocks like elemental userforms (although this isn't intended to be edited in-line). I don't think we can realistically load tab names for each block on the page without a significant hit to performance.
There could be a compromise though - we can load and cache tab names per block type. This would give a list of options that's accurate most of the time. In the rare case that the state of an element affects the tabs that it shows, there is a hook that we can use in FormBuilderLoader to update the options - the downside being that the user would have to trigger in-line editing by either expanding the block or clicking one of the existing options.
End result:

Performance not affected when loading tab names from a cache
List of tab options may not be complete until the form is loaded for a block
Once the form is loaded the block will maintain an updated list of tab options
If the user manages to remove tab options from modifying block data, the options should automatically update.
Eliminates the need for an additional request when a block is added

@clarkepaul wasn't keen on just having some of the tabs as options, so hopefully this is an acceptable compromise.
There is one gotcha here. The way that "many" relations add tabs (eg. Link Tracking) means that we need an existing object to fetch form fields for, or we can do this hacky thing:
$object = self::singleton();
$object->ID = 1;
$fields = $object->getCMSFields();

This works and gives the right list of tabs though!

there is a hook that we can use in FormBuilderLoader to update the options - the downside being that the user would have to trigger in-line editing by either expanding the block or clicking one of the existing options

AFAIK you will need to expand the block in order to edit it, it'll show a preview by default.

The way that "many" relations add tabs (eg. Link Tracking) means that we need an existing object to fetch form fields for, or we can do this hacky thing:

Note that the presence of these two tabs has been raised as a core bug: https://github.com/silverstripe/silverstripe-cms/issues/2227

AFAIK you will need to expand the block in order to edit it, it'll show a preview by default.

This is true, but it appears the edit options still appear in the list even when you don't have the form loaded. If the A/C means the form has to be loaded before we get those options in the dropdown then that's ideal.

I don't think that sounds like it makes sense =(

The tabs don't show as tabs with this, the tab content is shown when the section is chosen form the dropdown. The dropdown is rendered regardless of whether the form is rendered. I assume when you click the tab option (eg. Content) the form will be loaded and that tab content will be shown. Maybe @clarkepaul can clarify.

The designs answered the question:

We should have the tab options before we load the form.

I have split this into a few issues: #323, #324 and #325

authors: ScopeyNZ brynwhyman chillu robbieaverill

repo: dnadesign/silverstripe-elemental


text:
🛑 MX Auth is down

In 7766015, MX Auth (https://auth.siembro.mx) was down:

HTTP code: 500
Response time: 242 ms

Resolved: MX Auth is back up in 7fc2938.


text:
⚠ Site ANTS has degraded performance

In d166e80, Site ANTS (https://ants.gouv.fr) experienced degraded performance:

HTTP code: 200
Response time: 11996 ms

Resolved: Site ANTS performance has improved in a5c6dd6.

repo: dnum-mi/stats-sites-api


text:
🛑 Site SCAE RGA is down

In 073893e, Site SCAE RGA (https://www.sca-rga.interieur.gouv.fr/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Site SCAE RGA is back up in ff1645b.


text:
mac idea 2021.1上带泛型的参数的通用返回结果无法正常解析?

我用的mac idea 2021.1 生成接口文档后如果我的返回里带有泛型参数,给的结果是一堆莫名其妙的东西..
是哪里有问题?

我用的mac idea 2021.1 生成接口文档后如果我的返回里带有泛型参数,给的结果是一堆莫名其妙的东西.. 是哪里有问题?

@masterpopy 嗯, 麻烦提供下 demo, 我好复现下看看. 目前再复杂的泛型和嵌套, 理论都是支持的; 如果不支持, 那就是代码不够完善......

public class GenericResult extends AbstractResult {
private static final long serialVersionUID = 1L;
public static final GenericResult SUCCESS = new GenericResult() {
private static final long serialVersionUID = -3815215167127265311L;
    public void setSuccess(boolean success) {
    }

    public void setCode(String code) {
    }

    public void setMsg(String msg) {
    }

    public void setRetry(boolean retry) {
    }

    public void setArgs(Object[] args) {
    }

    public void setData(Void data) {
    }
};
private T data;

}
似乎是因为这个SUCCESS对象造成的.最后生成的api文档里会有近1w行的类似这种内容:
"data": {
"TYPE": {
"ANNOTATION": 44,
"ENUM": 66,
"SYNTHETIC": 48,
"cachedConstructor": {
"ACCESS_PERMISSION": {
"name": "翠埃沂秤嫡掷贡焊侗残"
},
"override": true,
"reflectionFactory": {
"initted": true,
"reflectionFactoryAccessPerm": {},
"langReflectAccess": {},
"hasStaticInitializerMethod": {
"ACCESS_PERMISSION": {},
"override": true,
"reflectionFactory": {},
"securityCheckCache": {},
"hasRealParameterData": false,
"parameters": [
{
....省略约1w行.
当然我是去不掉这个SUCCESS的.这是公司的框架.
我自己单独写了一个测试类测试一下不带这个SUCCESS的类的似乎没问题.

public class GenericResult extends AbstractResult {
private static final long serialVersionUID = 1L;
public static final GenericResult SUCCESS = new GenericResult() {
private static final long serialVersionUID = -3815215167127265311L;
    public void setSuccess(boolean success) {
    }

    public void setCode(String code) {
    }

    public void setMsg(String msg) {
    }

    public void setRetry(boolean retry) {
    }

    public void setArgs(Object[] args) {
    }

    public void setData(Void data) {
    }
};
private T data;

}
似乎是因为这个SUCCESS对象造成的.最后生成的api文档里会有近1w行的类似这种内容:
"data": {
"TYPE": {
"ANNOTATION": 44,
"ENUM": 66,
"SYNTHETIC": 48,
"cachedConstructor": {
"ACCESS_PERMISSION": {
"name": "翠埃沂秤嫡掷贡焊侗残"
},
"override": true,
"reflectionFactory": {
"initted": true,
"reflectionFactoryAccessPerm": {},
"langReflectAccess": {},
"hasStaticInitializerMethod": {
"ACCESS_PERMISSION": {},
"override": true,
"reflectionFactory": {},
"securityCheckCache": {},
"hasRealParameterData": false,
"parameters": [
{
....省略约1w行.
当然我是去不掉这个SUCCESS的.这是公司的框架.
我自己单独写了一个测试类测试一下不带这个SUCCESS的类的似乎没问题.

希望可以做一个功能可以过滤某些字段?

希望可以做一个功能可以过滤某些字段?

@masterpopy 有的, 添加注释即可! 如:

/**
 * 新增一个用户
 * @param addUserRequest 用户信息
 * @return 操作是否成功
 * #hiddenResponse SUCCESS
 */
public ResultBean<Boolean> addUser(AddUserRequest addUserRequest) {
    // .....
}

ResultBean.java
@Data
public class ResultBean<T> {

    private static final ResultBean SUCCESS = new ResultBean();

    private Integer code;

    private String msg;

    private T data;

    ........

}

你先试试啊... 针对这种情况, 我看看能不能改进下......

@masterpopy  另外, 使用 #hiddenRequest 可以过滤参数, 参数key以springMVC的规则为准, 比如 上段实例代码中, 我要过滤 AddUserRequest 中的某个字段, 只需要 #hiddenRequest xxx 即可, 而非 addUserRequest.xxx 这样多个参数名称作为前缀.

@masterpopy 最后提一句, 注释中的 # 可以替换为 @ 之所以用 # 是因为 idea 针对 @ 会有黄色警告... 不是每个人都会关闭警告, 所以两种都支持!

@masterpopy 我是说, 有没有可能, 你用的不是最新版..... 我好像没能复现你的情况......   也可能是你提供的代码太模糊了... 结构不够清晰...  可以的话, 提供清晰的结构给我, 不用真实代码, 你脱敏下你的代码, 结构是一样的就行

这个TYPE应该是来自于某些基本类型里的静态TYPE属性.

我多测试了一下添加
#hiddenResponse data.TYPE
后生成的文档显示
| data     | Void    |   |   |
data的类型为Void,而我的泛型为一个对象. Void里有TYPE这个属性,所以导致了这个问题.
可能是idea的bug,我用的是插件是最新的,idea是2021.1 apple silicon版本

我多测试了一下添加 #hiddenResponse data.TYPE 后生成的文档显示 | data | Void | | | data的类型为Void,而我的泛型为一个对象. Void里有TYPE这个属性,所以导致了这个问题. 可能是idea的bug,我用的是插件是最新的,idea是2021.1 apple silicon版本

(⊙o⊙)…
问题解决了吗? 我刚发布了一个版本(还在审核中), 这个版本过滤了 transient 和static修饰的字段
压缩包提前发你吧
docer-savior-plugin-2.0.2.zip

用你的最新的压缩包解决问题了,谢谢支持


text:
Debian image openssl vulnerability?

It seems that a Dockerfile used to build an image FROM python:3.5 i.e. python:latest will be using a debian image with a vulnerable openssl.
I don't understand precisely why, as an image I built yesterday FROM debian:jessie uses the more recent openssl but an image I built today FROM python:3.5 seems to use an older one.
python:3.5 -> buildpack-deps:jessie -> buildpack-deps:jessie-scm -> buildpack-deps:jessie-curl -> debian:jessie
Is an outdated upstream debian:jessie image being inadvertently cached perhaps?
This arose from a vulnerability warning I received from a python:3.5-based  image I pushed to my registry at IBM Bluemix Containers.

Security Notice: DSA-3673-2
Package        : openssl
Debian Bug     : 838652 838659
It was discovered that the original patch applied for CVE-2016-2182 in
DSA-3673-1 was incomplete, causing a regression when parsing
certificates. Updated packages are now available to address this
problem.
For the stable distribution (jessie), this problem has been fixed in
version 1.0.1t-1+deb8u5.
We recommend that you upgrade your openssl packages.

This was fixed with https://github.com/docker-library/official-images/pull/2179.  Ran a quick check and python:3.5 does contain the fixed version:
docker pull python:3.5
3.5: Pulling from library/python

43c265008fae: Pull complete
af36d2c7a148: Pull complete
143e9d501644: Pull complete
f6a5aab6cd0c: Pull complete
132c9d275259: Pull complete
e5dbf2ccb0ad: Pull complete
606e98dfd3f6: Pull complete
Digest: sha256:223993aa0c52de64dbf9c7acd0b8712145d12f18466c7a3d738a7e76e4bd1c92
Status: Downloaded newer image for python:3.5
$ docker run -it --rm python:3.5 bash
root@253a43173a3c:/# dpkg --status openssl
Package: openssl
Status: install ok installed
Priority: optional
Section: utils
Installed-Size: 1093
Maintainer: Debian OpenSSL Team <pkg-openssl-devel@lists.alioth.debian.org>
Architecture: amd64
Version: 1.0.1t-1+deb8u5
Depends: libc6 (>= 2.15), libssl1.0.0 (>= 1.0.1k-3+deb8u3)
Suggests: ca-certificates
Conffiles:
 /etc/ssl/openssl.cnf 7df26c55291b33344dc15e3935dabaf3
Description: Secure Sockets Layer toolkit - cryptographic utility
 This package is part of the OpenSSL project's implementation of the SSL
 and TLS cryptographic protocols for secure communication over the
 Internet.
 .
 It contains the general-purpose command line binary /usr/bin/openssl,
 useful for cryptographic operations such as:
  * creating RSA, DH, and DSA key parameters;
  * creating X.509 certificates, CSRs, and CRLs;
  * calculating message digests;
  * encrypting and decrypting with ciphers;
  * testing SSL/TLS clients and servers;
  * handling S/MIME signed or encrypted mail.
root@253a43173a3c:/#

Perhaps you have an older python:3.5 image and need to pull a new one?  Just pulling a new debian:jessie won't update your images that are from it.

Oh good, that's reassuring that the official python is up-to-date .... a bit worrying, though, that it is so easy for me  to have stale basis images.

authors: iainhouston yosifkit

repo: docker-library/python


text:
CVE-2022-1292 openssl vulnerability in Python 3.8-slim-bullseye

Hello,
We detected this new vulnerability in the latest image of Python3.8-slim-bullseye. CVE-2022-1292
Here is the response from docker scan:
Image layer: Introduced by your base image (python:3.8.13-slim-bullseye)
✗ Critical severity vulnerability found in openssl/libssl1.1
  Description: OS Command Injection
  Info: https://snyk.io/vuln/SNYK-DEBIAN11-OPENSSL-2807596
  Introduced through: openssl/libssl1.1@1.1.1n-0+deb11u1, ca-certificates@20210119, krb5/libgssapi-krb5-2@1.18.3-6+deb11u1
  From: openssl/libssl1.1@1.1.1n-0+deb11u1
  From: ca-certificates@20210119 > openssl@1.1.1n-0+deb11u1 > openssl/libssl1.1@1.1.1n-0+deb11u1
  From: krb5/libgssapi-krb5-2@1.18.3-6+deb11u1 > krb5/libkrb5-3@1.18.3-6+deb11u1 > openssl/libssl1.1@1.1.1n-0+deb11u1
  and 1 more...
  Image layer: '/bin/sh -c set -eux;    apt-get update;         apt-get install -y --no-install-recommends              ca-certificates                 netbase              tzdata  ;       rm -rf /var/lib/apt/lists/*'

https://security-tracker.debian.org/tracker/CVE-2022-1292
It's unfixed in Debian stable so there's nothing actionable for us to do
It's centered specifically around the c_rehash script which is considered obsolete and superseded by the CLI tool https://www.openssl.org/docs/manmaster/man1/c_rehash.html
Background:

Tags in the [official-images] library file[s] are only built through an update to that library file or as a result of its base image being updated (ie, an image FROM debian:buster would be rebuilt when debian:buster is built).
-https://github.com/docker-library/official-images/tree/2f086314307c04e1de77f0a515f20671e60d40bb#library-definition-files

Official Images FAQ:

Though not every CVE is removed from the images, we take CVEs seriously and try to ensure that images contain the most up-to-date packages available within a reasonable time frame
- https://github.com/docker-library/faq/tree/6138d05aabf61563606d86f98d0ccbd99f162b33#why-does-my-security-scanner-show-that-an-image-has-cves

Since our build system makes heavy use of Docker build cache, just rebuilding the all of the Dockerfiles won't cause any change. So we rely on periodic base image updates.

We strive to publish updated images at least monthly for Debian and Ubuntu. We also rebuild earlier if there is a critical security need. Many Official Images are maintained by the community or their respective upstream projects, like Alpine and Oracle Linux, and are subject to their own maintenance schedule.
- from the same FAQ link

Good news:
https://security-tracker.debian.org/tracker/CVE-2022-1292
bullseye (security) | 1.1.1n-0+deb11u2 | fixed

so there is a fix in debian stable (11 - bullseye)

Any work-around our Prisma scans label this as Critical and our CI/CD platform won't let us deploy our docker images with Critical CVEs?

You can make a derivative Dockerfile to get the updated version in the meantime
$ docker build -t python:test - << EOF
FROM python
RUN apt update && apt upgrade -y && rm -rf /var/lib/apt/lists/*
EOF

. . .

$ docker run --rm python:test apt list | grep -i openssl

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libcurl4-openssl-dev/now 7.74.0-1.3+deb11u1 amd64 [installed,local]
libevent-openssl-2.1-7/now 2.1.12-stable-1 amd64 [installed,local]
openssl/now 1.1.1n-0+deb11u2 amd64 [installed,local]

Thanks @wglambert  my Dockerfile looks as follows now
FROM python:3.8-slim
.
.
.
RUN apt-get update && apt-get install openssl -y && rm -rf /var/lib/apt/lists/*
.

when my CI/CD runs it provides the following output
2022-05-19T18:25:56.278Z] #10 [6/7] RUN apt-get install openssl -y && rm -rf /var/lib/apt/lists/*

[2022-05-19T18:25:56.278Z] #10       digest: sha256:e936df1390e11aeeb87877f251b72fd92d5d1ee1c9dc82a71e25b3ed5d11b986

[2022-05-19T18:25:56.278Z] #10         name: "[6/7] RUN apt-get install openssl -y && rm -rf /var/lib/apt/lists/*"
[2022-05-19T18:25:56.278Z] #10      started: 2022-05-19 18:25:55.988222233 +0000 UTC
[2022-05-19T18:25:56.845Z] #10 0.079 Reading package lists...
[2022-05-19T18:25:56.845Z] #10 0.593 Building dependency tree...
[2022-05-19T18:25:56.845Z] #10 0.688 Reading state information...
[2022-05-19T18:25:56.845Z] #10 0.813 The following packages will be upgraded:
[2022-05-19T18:25:57.103Z] #10 0.813   openssl
[2022-05-19T18:25:57.103Z] #10 0.828 1 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
[2022-05-19T18:25:57.103Z] #10 0.828 Need to get 852 kB of archives.
[2022-05-19T18:25:57.103Z] #10 0.828 After this operation, 0 B of additional disk space will be used.
[2022-05-19T18:25:57.103Z] #10 0.828 Get:1 http://security.debian.org/debian-security bullseye-security/main amd64 openssl amd64 1.1.1n-0+deb11u2 [852 kB]
[2022-05-19T18:25:57.103Z] #10 0.926 debconf: delaying package configuration, since apt-utils is not installed
[2022-05-19T18:25:57.103Z] #10 0.953 Fetched 852 kB in 0s (54.1 MB/s)
[2022-05-19T18:25:57.104Z] #10 0.969 (Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 7023 files and directories currently installed.)
[2022-05-19T18:25:57.104Z] #10 0.974 Preparing to unpack .../openssl_1.1.1n-0+deb11u2_amd64.deb ...
[2022-05-19T18:25:57.104Z] #10 0.981 Unpacking openssl (1.1.1n-0+deb11u2) over (1.1.1n-0+deb11u1) ...
[2022-05-19T18:25:57.362Z] #10 1.068 Setting up openssl (1.1.1n-0+deb11u2) ...
[2022-05-19T18:25:57.362Z] #10    completed: 2022-05-19 18:25:57.127417945 +0000 UTC
[2022-05-19T18:25:57.362Z] #10     duration: 1.139195712s

So looks like it's not getting the fixed version? 1.1.1o?

Thanks @wglambert yeah looks like you are correct - that's interesting as the Prisma scan of this image is still barfing :-(
Probably something I am doing wrong.
[2022-05-19T18:26:13.337Z] Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
[2022-05-19T18:26:13.337Z] 		 Affected argument(s) used the following variable(s): [PRISMA_KEY_USR, PRISMA_KEY_PSW]
[2022-05-19T18:26:13.337Z] 		 See https://jenkins.io/redirect/groovy-string-interpolation for details.
[2022-05-19T18:26:13.840Z] + /home/jenkins/twistcli images scan --address https://us-east1.cloud.twistlock.com/us-2-158284691 --user **** --password **** --details --docker-address tcp://localhost:2375 docker.cogitocorp.us/cogito-locust:0.0.1-b3-912cef0
[2022-05-19T18:26:17.124Z]
[2022-05-19T18:26:17.124Z] Scan results for: image docker.cogitocorp.us/cogito-locust:0.0.1-b3-912cef0 sha256:6cf48673ac7b29fd5f7469792b3aaa112dda21949ab836b74f6cdf67d12ff909
[2022-05-19T18:26:17.124Z] Vulnerabilities
[2022-05-19T18:26:17.124Z] +------------------+----------+------+------------+------------------+---------------------------+------------+------------+----------------------------------------------------+-------------------+

[2022-05-19T18:26:17.124Z] |       CVE        | SEVERITY | CVSS |  PACKAGE   |     VERSION      |          STATUS           | PUBLISHED  | DISCOVERED |                    DESCRIPTION                     | TRIGGERED FAILURE |

[2022-05-19T18:26:17.124Z] +------------------+----------+------+------------+------------------+---------------------------+------------+------------+----------------------------------------------------+-------------------+
[2022-05-19T18:26:17.124Z] | CVE-2022-1292    | critical | 9.80 | openssl    | 1.1.1n-0+deb11u1 | fixed in 1.1.1n-0+deb11u2 | 16 days    | < 1 hour   | The c_rehash script does not properly sanitise     | Yes               |
[2022-05-19T18:26:17.124Z] |                  |          |      |            |                  | 16 days ago               |            |            | shell metacharacters to prevent command injection. |                   |
[2022-05-19T18:26:17.124Z] |                  |          |      |            |                  |                           |            |            | This script is distributed by some operating       |                   |
[2022-05-19T18:26:17.124Z] |                  |          |      |            |                  |                           |            |            | systems...                                         |                   |

Yeah odd, it's detecting the version as 1.1.1n-0+deb11u1. Maybe it's cached results or the old image.
You can also docker tag to overwrite the existing image:tag if that makes things easier for deployment/testing
$ docker tag python:test python:latest
$ docker run --rm python:latest apt list | grep -i openssl

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libcurl4-openssl-dev/now 7.74.0-1.3+deb11u1 amd64 [installed,local]
libevent-openssl-2.1-7/now 2.1.12-stable-1 amd64 [installed,local]
openssl/now 1.1.1n-0+deb11u2 amd64 [installed,local]

Digging into the prisma analysis do I need to update libssl as well? It appears so?

Yes this makes the world "green" for me
RUN apt-get update &&  apt-get --only-upgrade install openssl libssl1.1 -y && rm -rf /var/lib/apt/lists/*

@wglambert when can we expect the next base image update?

https://github.com/docker-library/official-images/pull/12531 :+1:

authors: acs frankjkelly lukasz-mitka meher404 tianon wglambert


text:
docker registry: image deletion by name and tag

Hi,
this is a feature request.
Currently it is possible to delete images by using the REST API of a private registry, only by specifying the digest (<reference>)
DELETE /v2/<name>/manifests/<reference>

This is inconvenient, since it will delete all the images with the same digest, whilst the expected behavior is to delete just one specific image (i.e. "un-tag" it) and to perform the actual deletion only if it is the last tag referring to that digest.
This is also the behavior implemented by the docker CLI interface when managing local images. So this is also a issue of uniformity.
So I propose to add another API, to implement the deletion via image name and tag. For example something like this:
DELETE /v2/<name>/manifests/<tag>

In case of an image without a tag (i.e. in cases where the docker images command displays "none" as a tag for an image), it should be possible to specify a "special none tag". For example something like this:
DELETE /v2/<name>/manifests/_

Will you consider this feature?
Regards

Can you open an issue on https://github.com/distribution/distribution, please?
This is the repo for Docker image only.

Hi,
sure, thank you.
Regards.

Hi @milosgajdos is-it possible to reopen this ticket, since the issue in OCI Distribution has been fixed in v1.0.0 (https://github.com/distribution/distribution/pull/2169)? The latest release of docker image registry is v2.7.1 and it does not contain this fix. It would be great to have a v2.7.2 with this fix, and thus I suggest reopening this ticket! Thanks!
As of today, the only reliable way to remove a tag without removing the whole manifest, is for us, to backup the whole repository elsewhere with a whitelist, then remove the whole repository and restoring it. It's a long inefficient process!

@antoinetran this repository is what turns releases from https://github.com/distribution/distribution into image updates at https://hub.docker.com/_/registry, so what needs to happen first is a new release on https://github.com/distribution/distribution which includes https://github.com/distribution/distribution/pull/3427

authors: FStefanni antoinetran milosgajdos tianon

repo: docker/distribution-library-image


text:
Swift as backend with docker registry

Hope it is not a duplicate issue , i tried checking the issues to check if its not duplicate.
I am using
Docker version 1.8.3, build f4bf5c7 trying to configure it with swift as a storage backend
My docker config.yml file looks like
version: 0.1
log:
level: debug
formatter: text
fields:
service: registry
environment: staging
storage:
swift:
username: docker-registry
password: xxxxxx
authurl: http://www.example.com:5000/v2
tenant: docker-registry-v2.0
tenantid: 9707a0b6786743ffa292cc3f0cbb7865
insecureskipverify: true
region: nova
container: docker-registry-v2.0
rootdirectory: /swift/object/name/prefix
Then running
docker run -d -p 5000:5000 --restart=always --name registry-2 -v pwd/config.yml:/etc/docker/registry/config.yml registry:2
Running it in debug mode and tailing the logs only show
docker logs --follow --tail='all' registry-2
http://paste.ubuntu.com/13088275/
Can someone suggest

@riteshnanda09 : what exact error are you experiencing?

No errors , it just gives trace
6c6f3 trace.line=123 version=v2.1.1
time="2015-11-03T02:00:59.831949214Z" level=debug msg="swift.List("/")" environment=staging instance.id=3bbb41e1-bcb8-4db2-a811-3ed51e02fb22 service=registry trace.duration=32.437937ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=7aba4a43-3bed-4fc4-b1ba-f8426f1e1901 trace.line=123 version=v2.1.1
time="2015-11-03T02:01:09.81400514Z" level=debug msg="swift.List("/")" environment=staging instance.id=3bbb41e1-bcb8-4db2-a811-3ed51e02fb22 service=registry trace.duration=14.448663ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=40710464-90d5-4e05-95fd-4bb26d76c1d0 trace.line=123 version=v2.1.1
time="2015-11-03T02:01:19.814997289Z" level=debug msg="swift.List("/")" environment=staging instance.id=3bbb41e1-bcb8-4db2-a811-3ed51e02fb22 service=registry trace.duration=15.490087ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=69f7c531-9e8d-415c-ab91-1398b58aeccc trace.line=123 version=v2.1.1
time="2015-11-03T02:01:29.817140245Z" level=debug msg="swift.List("/")" environment=staging instance.id=3bbb41e1-bcb8-4db2-a811-3ed51e02fb22 service=registry trace.duration=17.619116ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=3f44fea8-9a2b-4422-80d3-146975b72015 trace.line=123 version=v2.1.1
time="2015-11-03T02:01:39.817034083Z" level=debug msg="swift.List("/")" environment=staging instance.id=3bbb41e1-bcb8-4db2-a811-3ed51e02fb22 service=registry trace.duration=17.508003ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=666af36b-62ac-4232-a229-401f7efe8388 trace.line=123 version=v2.1.1
time="2015-11-03T02:01:49.816746989Z" level=debug msg="swift.List("/")" environment=staging instance.id=3bbb41e1-bcb8-4db2-a811-3ed51e02fb22 service=registry trace.duration=17.195128ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=08412d2a-5736-44e2-b870-f8de265597da trace.line=123 version=v2.1.1

What is the problem you are trying to solve?

i am trying running docker registry with swift as a backend.

It seems you have successfully done that.  Why have you submitted a github issue about this?

So if i try to push a image
docker push 10.24.48.220:5000/ubuntu
The push refers to a repository [10.24.48.220:5000/ubuntu] (len: 1)
Sending image list
Put http://10.24.48.220:5000/v1/repositories/ubuntu/: dial tcp 10.24.48.220:5000: connection refused

For some reason, the daemon cannot talk to the registry.  That is why there is nothing in the registry logs and that is why you see connection refused from the daemon.  There is something wrong with your setup.
Post the command line you use to start the daemon and the logs it produces.

This is the command i use to run registry container
docker run -d -p 5000:5000 --restart=always --name registry-2 -v pwd/config.yml:/etc/docker/registry/config.yml registry:2
it is reading a config.yml file
version: 0.1
log:
level: debug
formatter: text
fields:
service: registry
environment: staging
storage:
swift:
username: docker-registry
password: docker-registry
authurl: https://os-identity.vip.lvs03.paypalc3.com:5443/v2.0
tenant: docker-registry-v2.0
tenantid: 9707a0b6786743ffa292cc3f0cbb7865
insecureskipverify: true
region: lvs03
container: docker-registry-v2.0
rootdirectory: /swift/object/name/prefix
this is /var/log/upstart/docker.log
INFO[0966] POST /v1.20/containers/create?name=registry-2
INFO[0967] POST /v1.20/containers/e068a3a37686f38990b14f32617fbac5b4c9a8e49049f0f4b2901d3c701a8697/start
INFO[0971] GET /v1.20/containers/registry-2/json
INFO[0971] GET /v1.20/containers/registry-2/logs?follow=1&stderr=1&stdout=1&tail=all
INFO[1121] GET /v1.20/containers/registry-2/json
INFO[1121] GET /v1.20/containers/registry-2/logs?follow=1&stderr=1&stdout=1&tail=all
INFO[1258] GET /v1.20/containers/registry-2/json
INFO[1258] GET /v1.20/containers/registry-2/logs?follow=1&stderr=1&stdout=1&tail=all
INFO[1466] GET /v1.20/containers/registry-2/json
INFO[1466] GET /v1.20/containers/registry-2/logs?follow=1&stderr=1&stdout=1&tail=all
INFO[1494] GET /v1.20/containers/registry-2/json
INFO[1494] GET /v1.20/containers/registry-2/logs?follow=1&stderr=1&stdout=1&tail=all
INFO[1549] GET /v1.20/containers/registry-2/json
INFO[1549] GET /v1.20/containers/registry-2/logs?follow=1&stderr=1&stdout=1&tail=all
INFO[1983] GET /v1.20/containers/registry-2/json
INFO[1983] GET /v1.20/containers/registry-2/logs?follow=1&stderr=1&stdout=1&tail=all
INFO[2752] GET /v1.20/containers/registry-2/json
INFO[2752] GET /v1.20/containers/registry-2/logs?follow=1&stderr=1&stdout=1&tail=all
INFO[3053] POST /v1.20/images/create?fromImage=ubuntu%3Alatest
INFO[3093] POST /v1.20/images/ubuntu/tag?repo=10.24.48.220%3A5000%2Fubuntu&tag=
INFO[3140] POST /v1.20/images/10.24.48.220:5000/ubuntu/push?tag=
INFO[3194] GET /v1.20/containers/registry-2/json
INFO[3194] GET /v1.20/containers/registry-2/logs?follow=1&stderr=1&stdout=1&tail=all

i wil close this as i got working , it did'nt work for me through config.yml , but it worked using docker run with environment variables.
docker run -d --name registry -p 5000:5000 -e "REGISTRY_STORAGE=swift" -e "REGISTRY_STORAGE_SWIFT_USERNAME=xxxx" -e "REGISTRY_STORAGE_SWIFT_PASSWORD=xxxx" -e "REGISTRY_STORAGE_SWIFT_AUTHURL=xxxx/v2/tokens" -e "REGISTRY_STORAGE_SWIFT_CONTAINER=xxx" -e "REGISTRY_STORAGE_SWIFT_REGION=xxx" -e "REGISTRY_STORAGE_SWIFT_TENANT=xxxx" -e "REGISTRY_STORAGE_SWIFT_TENANTID=xxx" registry:2

repo: docker/distribution


text:
Add signal handler to re-read Registry configuration

It would be a nice feature if Registry supported handling of "kill -HUP" to force configuration reload. Such feature could be used for switching Registry between readonly "enable/disable" modes without a need to completely restart Registry. Something like "docker exec registry_conainer_name kill -HUP 1"

hi @sergeyfd
Graceful restart adds a lot of complexity for relatively little benefit.   In-flight requests would need to be drained and new ones refused.  The registry starts almost instantly and the docker engine now has retry built-in.

Makes sense. Thanks.

authors: RichardScothern sergeyfd


text:
Private registry push fail: 500 error. "s3aws: AuthorizationHeaderMalformed: The authorization header is malformed;"

I am configuring a docker registry to run behind a NGINX proxy. I was able to authenticate. But a problem occurs when I try to push a image to the registry.
docker push registry.mydomain.com.br/busybox

I get this error message:
time="2016-11-01T10:51:50.960847082Z" level=error msg="response completed with error" err.code=unknown err.detail="s3aws: AuthorizationHeaderMalformed: The authorization header is malformed; the Credential is mal-formed; expecting \"<YOUR-AKID>/YYYYMMDD/REGION/SERVICE/aws4_request\".\n\tstatus code: 400, request id: A1896A22345B3CD8" err.message="unknown error" go.version=go1.6.3 http.request.host=registry.mydomain.com.br http.request.id=e90033d4-d377-4dc1-87f4-5f13e9f0ad9e http.request.method=HEAD http.request.remoteaddr=52.55.236.208 http.request.uri="/v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" http.request.useragent="docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=32.585103ms http.response.status=500 http.response.written=117 instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry vars.digest="sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" vars.name=busybox version=v2.5.1

But apparently I have everything configured right. It is like is not passing some auth info to the s3. But I have confirmed my key, secret and region for my s3 are valid.
/data/registry/config.yaml
version: 0.1
log:
  level: debug
  fields:
    service: registry
http:
  addr: :5000
  host: https://registry.mydomain.com.br
storage:
  cache:
    layerinfo: inmemory
  s3:
    accesskey: myawskey
    secretkey: myawssecret
    region: us-east-1
    bucket: my-s3-bucket
    encrypt: true
    secure: true
    v4auth: true
    chunksize: 5242880
    rootdirectory: /

Running
docker run --rm --name registry \
       -v /data/registry/config.yaml:/etc/docker/registry/config.yml \
       -p 5000 registry:2

Registry log
time="2016-11-01T10:51:19.90279961Z" level=warning msg="No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable." go.version=go1.6.3 instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry version=v2.5.1
time="2016-11-01T10:51:19.90286376Z" level=info msg="redis not configured" go.version=go1.6.3 instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry version=v2.5.1
time="2016-11-01T10:51:19.903420589Z" level=info msg="Starting upload purge in 1m0s" go.version=go1.6.3 instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry version=v2.5.1
time="2016-11-01T10:51:19.935437384Z" level=info msg="using inmemory blob descriptor cache" go.version=go1.6.3 instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry version=v2.5.1
time="2016-11-01T10:51:19.936112542Z" level=info msg="listening on [::]:5000" go.version=go1.6.3 instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry version=v2.5.1
time="2016-11-01T10:51:45.889450622Z" level=debug msg="authorizing request" go.version=go1.6.3 http.request.host=registry.mydomain.com.br http.request.id=c7d15dea-2bd1-4802-8a69-60fcaa266e62 http.request.method=HEAD http.request.remoteaddr=52.55.236.208 http.request.uri="/v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" http.request.useragent="docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64" instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry vars.digest="sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" vars.name=busybox version=v2.5.1
time="2016-11-01T10:51:45.889560532Z" level=debug msg=GetBlob go.version=go1.6.3 http.request.host=registry.mydomain.com.br http.request.id=c7d15dea-2bd1-4802-8a69-60fcaa266e62 http.request.method=HEAD http.request.remoteaddr=52.55.236.208 http.request.uri="/v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" http.request.useragent="docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64" instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry vars.digest="sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" vars.name=busybox version=v2.5.1
time="2016-11-01T10:51:45.917756023Z" level=debug msg="s3aws.GetContent(\"/docker/registry/v2/repositories/busybox/_layers/sha256/56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190/link\")" go.version=go1.6.3 http.request.host=registry.mydomain.com.br http.request.id=c7d15dea-2bd1-4802-8a69-60fcaa266e62 http.request.method=HEAD http.request.remoteaddr=52.55.236.208 http.request.uri="/v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" http.request.useragent="docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64" instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry trace.duration=28.117459ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=003b53e0-7b7d-417a-940c-972404241b42 trace.line=82 vars.digest="sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" vars.name=busybox version=v2.5.1
time="2016-11-01T10:51:45.91792986Z" level=error msg="response completed with error" err.code=unknown err.detail="s3aws: AuthorizationHeaderMalformed: The authorization header is malformed; the Credential is mal-formed; expecting \"<YOUR-AKID>/YYYYMMDD/REGION/SERVICE/aws4_request\".\n\tstatus code: 400, request id: DFE2A7C70E32C322" err.message="unknown error" go.version=go1.6.3 http.request.host=registry.mydomain.com.br http.request.id=c7d15dea-2bd1-4802-8a69-60fcaa266e62 http.request.method=HEAD http.request.remoteaddr=52.55.236.208 http.request.uri="/v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" http.request.useragent="docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=29.855287ms http.response.status=500 http.response.written=117 instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry vars.digest="sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" vars.name=busybox version=v2.5.1
172.17.0.3 - - [01/Nov/2016:10:51:45 +0000] "HEAD /v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190 HTTP/1.1" 500 117 "" "docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64"
time="2016-11-01T10:51:50.929522596Z" level=debug msg="authorizing request" go.version=go1.6.3 http.request.host=registry.mydomain.com.br http.request.id=e90033d4-d377-4dc1-87f4-5f13e9f0ad9e http.request.method=HEAD http.request.remoteaddr=52.55.236.208 http.request.uri="/v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" http.request.useragent="docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64" instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry vars.digest="sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" vars.name=busybox version=v2.5.1
time="2016-11-01T10:51:50.929609491Z" level=debug msg=GetBlob go.version=go1.6.3 http.request.host=registry.mydomain.com.br http.request.id=e90033d4-d377-4dc1-87f4-5f13e9f0ad9e http.request.method=HEAD http.request.remoteaddr=52.55.236.208 http.request.uri="/v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" http.request.useragent="docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64" instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry vars.digest="sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" vars.name=busybox version=v2.5.1
time="2016-11-01T10:51:50.96071846Z" level=debug msg="s3aws.GetContent(\"/docker/registry/v2/repositories/busybox/_layers/sha256/56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190/link\")" go.version=go1.6.3 http.request.host=registry.mydomain.com.br http.request.id=e90033d4-d377-4dc1-87f4-5f13e9f0ad9e http.request.method=HEAD http.request.remoteaddr=52.55.236.208 http.request.uri="/v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" http.request.useragent="docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64" instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry trace.duration=31.035061ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=594b98b9-4e1c-4ebb-8bbd-c2526253b1cc trace.line=82 vars.digest="sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" vars.name=busybox version=v2.5.1
time="2016-11-01T10:51:50.960847082Z" level=error msg="response completed with error" err.code=unknown err.detail="s3aws: AuthorizationHeaderMalformed: The authorization header is malformed; the Credential is mal-formed; expecting \"<YOUR-AKID>/YYYYMMDD/REGION/SERVICE/aws4_request\".\n\tstatus code: 400, request id: A1896A22345B3CD8" err.message="unknown error" go.version=go1.6.3 http.request.host=registry.mydomain.com.br http.request.id=e90033d4-d377-4dc1-87f4-5f13e9f0ad9e http.request.method=HEAD http.request.remoteaddr=52.55.236.208 http.request.uri="/v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" http.request.useragent="docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=32.585103ms http.response.status=500 http.response.written=117 instance.id=2dbcb6bd-1e3a-43b7-a1a5-42c97d4290f8 service=registry vars.digest="sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190" vars.name=busybox version=v2.5.1
172.17.0.3 - - [01/Nov/2016:10:51:50 +0000] "HEAD /v2/busybox/blobs/sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190 HTTP/1.1" 500 117 "" "docker/1.10.3 go/go1.5.4 git-commit/1f8f545 kernel/4.7.0-coreos-r1 os/linux arch/amd64"

Deamon log
Nov 01 11:10:08 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:08.831384762Z" level=debug msg="Calling GET /_ping"
Nov 01 11:10:08 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:08.831422196Z" level=debug msg="GET /_ping"
Nov 01 11:10:10 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:10.725027043Z" level=debug msg="Calling POST /v1.22/images/registry.mydomain.com.br/busybox/push"
Nov 01 11:10:10 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:10.725064365Z" level=debug msg="POST /v1.22/images/registry.mydomain.com.br/busybox/push?tag="
Nov 01 11:10:10 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:10.726115465Z" level=debug msg="hostDir: /etc/docker/certs.d/registry.mydomain.com.br"
Nov 01 11:10:10 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:10.726565277Z" level=debug msg="hostDir: /etc/docker/certs.d/registry.mydomain.com.br"
Nov 01 11:10:10 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:10.726625385Z" level=debug msg="Trying to push registry.mydomain.com.br/busybox to https://registry.mydomain.com.br v2"
Nov 01 11:10:10 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:10.737121910Z" level=debug msg="Pushing repository: registry.mydomain.com.br/busybox:latest"
Nov 01 11:10:10 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:10.762947677Z" level=error msg="Upload failed, retrying: Received unexpected HTTP status: 500 Internal Server Error"
Nov 01 11:10:15 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:15.887727168Z" level=error msg="Upload failed, retrying: Received unexpected HTTP status: 500 Internal Server Error"
Nov 01 11:10:17 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:17.888059572Z" level=info msg="Pull session cancelled"
Nov 01 11:10:17 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:17.888118184Z" level=debug msg="Not continuing with error: context canceled"
Nov 01 11:10:18 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:18.832602646Z" level=debug msg="Calling GET /_ping"
Nov 01 11:10:18 ip-10-1-14-30.ec2.internal dockerd[1647]: time="2016-11-01T11:10:18.832640958Z" level=debug msg="GET /_ping"

Registry Version
docker run --rm registry:2 --version
registry github.com/docker/distribution v2.5.1

Docker version
Client:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.4
 Git commit:   1f8f545
 Built:
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.4
 Git commit:   1f8f545
 Built:
 OS/Arch:      linux/amd64

Docker info
Containers: 2
 Running: 2
 Paused: 0
 Stopped: 0
Images: 4
Server Version: 1.10.3
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
 Volume: local
 Network: null host bridge
Kernel Version: 4.7.0-coreos-r1
Operating System: CoreOS 1122.3.0 (MoreOS)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.862 GiB
Name: ip-10-1-14-30.ec2.internal
ID: NNS5:WQJG:3WGS:XMME:XQBE:UEN3:CSIM:X52U:K7UJ:HPNU:X6EC:CECC
Debug mode (server): true
 File Descriptors: 36
 Goroutines: 54
 System Time: 2016-11-01T11:06:43.936196975Z
 EventsListeners: 1
 Init SHA1: 2877c6642ddf0db27b616fc3271258333ef9f1ee
 Init Path: /usr/libexec/docker/dockerinit
 Docker Root Dir: /var/lib/docker

Are you sure your environment is clean?  (you're not overriding any config entries with environment variables)
Does this happen without nginx?
Can you see what the requests look like on the wire (with wireshark, mitm proxy etc)?
Are you sure your environment

The env variables looks fine:
docker exec -ti registry env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=6ec6f09eeb2e
VIRTUAL_HOST=registry.mydomain.com.br
VIRTUAL_PORT=5000
HOME=/root

I will try use some sniffer tool to find out how the requests looks like.

@RichardScothern I found the problem. I had switched the accesskey with the secretkey. So when the s3 sdk was building the request credential it would include the secretekey instead of the accesskey in the Authorization header. And my secretkey has a slash in the middle. Which as causing the aws think I was sending the wrong number of args in the credential value. Since it uses slash to separate each arg.

authors: RichardScothern maxcnunes


text:
Fixes error codes in the API spec template

Uses UPPER_UNDERSCORE_CASE instead of the nice error message format
added in #911

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "fixErrorCodesInAPITemplate" git@github.com:BrianBland/distribution.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Ran make docs/spec/api.md and then API spec file didn't change :+1:

@BrianBland The "pretty" code is really only for error messages printed from a go process.  The actual error code value should be VERY_BIG_AND_LOUD.
Did #911 break that?

Yeah, before this change the template was automatically using the .Error() method on the error codes

@BrianBland Yuck. Just realized that. Looks like .Error() is favored over .String().
LGTM.

GCS driver tests seem to be broken right now due to the work on test env variables, but can we ignore that for this PR?

Temporarily disabled.  LGTM

Current coverage is 58.22%

Merging #1345 into master will increase coverage by +0.29% as of ac03070

@@            master   #1345   diff @@
======================================
  Files          123     123
  Stmts        11094   11094
  Branches       772     772
  Methods          0       0
======================================
+ Hit           6427    6460    +33
  Partial        772     772
+ Missed        3895    3862    -33

Review entire Coverage Diff as of ac03070

Uncovered Suggestions

+0.29% via ...ge/driver/gcs/gcs.go#185...216
+0.22% via ...ge/driver/gcs/gcs.go#474...497
+0.21% via ...ge/driver/gcs/gcs.go#124...146
See 7 more...

Powered by Codecov. Updated on successful CI builds.

source: gharchive/pull-request

authors: BrianBland GordonTheTurtle RichardScothern codecov-io stevvooe


text:
Can't seem to capture the output in a log file

Hey all,
When running the docker container I would like to save the output into a log file. According to the readme: "If the docker container is used then the log files will be created inside the container in location /usr/local/bin/log/". However when I bind the /usr/local/bin/log to my /tmp/ folder it doesn't seem to create a log file there. Here is the docker run command we use:
docker run -d --rm --net host --pid host --userns host --cap-add audit_control \     -e DOCKER_CONTENT_TRUST=$DOCKER_CONTENT_TRUST \     -v /etc:/etc:ro \     -v /usr/lib/systemd/system:/lib/systemd/system:ro \     -v /usr/bin/containerd:/usr/bin/containerd:ro \     -v /usr/bin/runc:/usr/bin/runc:ro \     -v /usr/lib/systemd:/usr/lib/systemd:ro \     -v /var/lib:/var/lib:ro \     -v /var/run/docker.sock:/var/run/docker.sock:ro \     -v /tmp/docker-security-bench:/usr/local/bin/log/ \     --label docker_bench_security \     docker/docker-bench-security
What am I doing wrong here?
I was thinking that maybe I should pass the option -l FILE. Can I pass this without adjusting the dockerfile/rebuilding the image?

I am facing the same issue . Any solution for this ?

Hi @poencho and @Vdsouza22 and so sorry for the delay.
I can't reproduce this.
Are you building the docker image by yourself or using the public one (ref https://github.com/docker/docker-bench-security/issues/405)?
~$ docker build --no-cache -t docker/docker-bench-security .
~$ docker run -d --rm --net host --pid host --userns host --cap-add audit_control  -e DOCKER_CONTENT_TRUST=$DOCKER_CONTENT_TRUST -v /etc:/etc:ro  -v /usr/lib/systemd/system:/lib/systemd/system:ro -v /    usr/bin/containerd:/usr/bin/containerd:ro -v /usr/bin/runc:/usr/bin/runc:ro -v /usr/lib/systemd:/usr/lib/systemd:ro -v /var/lib:/var/lib:ro -v /var/run/docker.sock:/var/run/docker.sock:ro -v /tmp/dock    er-security-bench:/usr/local/bin/log/ --label docker_bench_security docker/docker-bench-security
  1 aa213ce1951630d2e322c4900caa669aca0ebcb7eb5fa940d02f47e9a5e72b74
~$ jq '.dockerbenchsecurity' /tmp/docker-security-bench/docker-bench-security.log.json
"1.3.6"

@konstruktoid I am using the public image: docker/docker-bench-security
The way I launch the container the log file should appear in my tmp folder when using this volume mount correct? -v /tmp/docker-security-bench:/usr/local/bin/log/

Yeah, the public image is broken in all sorts of ways, see https://github.com/docker/docker-bench-security/issues/405.
I recommend you build your own.

Thanks!

You weren't lazy and the public image should be up-to-date and working, sorry to say that isn't the case.

Closing due to inactivity.

authors: Vdsouza22 konstruktoid poencho

repo: docker/docker-bench-security

license_source: bigquery


text:
Network on laptop does not work (Windows 10 1909)

Problem description
Transparent network can be created, but the container does not get the IP I want to assign.
Also, with the default settings, the container does not get a generated IP and cannot be accessed.
On other stations both cases work well.
Problem location
I have uploaded the diagnostics with ID:
872172A6-2C37-4AF5-805E-2F13638A9F17/20200327063916
Project version(s) affected
Happened on 1903 and also 1909.
Suggestions for a fix
I do not see any possible fix.
I am not able to see any error in any console, the network just does not work.

Thanks for sharing the issue. If you need support, you can try the Docker Community Forums. If you believe this is a bug, please create an issue in the application repo. If there is an error in the documentation, please let us know.

authors: craig-osterhout danzaharescu


text:
Paid through Docker needs to be removed

File: docker-hub/publish/index.md
This has been deprecated for some time, all partners with such content have been notified.. has been removed from partner doc's since December. Can we please ensure these doc's are also updated?

Issue moved to docker/docs-private #1160 via ZenHub


text:
docker desktop for windows is a bad link

File: docker-for-windows/release-notes.md
This is an incorrect file !
https://download.docker.com/win/stable/Docker for Windows Installer.exe
size: 875349208
It should be Docker Desktop Community 2.1.0.1 instead of 2.1.0.3
https://download.docker.com/win/stable/37199/Docker Desktop Installer.exe

Hi @gclove, the most recest version of Docker Desktop Community is v2.1.0.3 though. Can you provide more information (links, screenshots etc.)?
I'm on Ubuntu and can't really reproduce the error. Would love to help though :)

@lkreimann
You can check it by HEAD method.
$ curl -i -X HEAD "https://download.docker.com/win/stable/Docker%20for%20Windows%20Installer.exe"

HTTP/1.1 200 OK
Content-Type: application/x-msdownload
Content-Length: 875349208
Connection: keep-alive
Last-Modified: Thu, 08 Aug 2019 09:57:55 GMT
x-amz-version-id: 3FsmIvlR242Omelg8KkD2kkGa1TxBJtL
Server: AmazonS3
Date: Tue, 08 Oct 2019 07:28:54 GMT
ETag: "fe2b344dc62f2bbba266e450eb8c5091-105"
X-Cache: Hit from cloudfront
Via: 1.1 d51ceda436f155dcdc6b24ba6dcf73cd.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: NRT57-C4
X-Amz-Cf-Id: GtkRTyUH1Oio7JZyrWdml3VD8Yozi09df89vXypRtZ2wfq8bUNalWw==
Age: 191

$ curl -i -X HEAD "https://download.docker.com/win/stable/37199/Docker%20Desktop%20Installer.exe"

HTTP/1.1 200 OK
Content-Length: 875349208
Connection: keep-alive
Date: Tue, 08 Oct 2019 07:33:46 GMT
Last-Modified: Wed, 07 Aug 2019 16:45:47 GMT
x-amz-version-id: 89_8qDOBLXDq1G1BfC5vpLUsAi.aJHxu
ETag: "65f31ae1d241585706d75443aee07b8c-167"
Server: AmazonS3
X-Cache: Miss from cloudfront
Via: 1.1 c8cb49c2b391bae10c56b1a57df509d0.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: NRT57-C4
X-Amz-Cf-Id: WjjUZ0OcvuZrDxwup7a5pWJsNZ5jjhyXkAB_db8rEafo7dlRUw-t2A==

Their Content-Length: 875349208 are the same, indicating that they are the same file.
37199 is the v2.1.0.1
38240 should be v2.1.0.3
$ curl -i -X HEAD "https://download.docker.com/win/stable/38240/Docker%20Desktop%20Installer.exe"

HTTP/1.1 200 OK
Content-Length: 877449944
Connection: keep-alive
Date: Thu, 03 Oct 2019 01:20:58 GMT
Last-Modified: Fri, 13 Sep 2019 09:03:58 GMT
x-amz-version-id: PVJi1ljAzVzTW7A9vkHjhCUYEb5Gmjh8
ETag: "33d49ed8cb39be7cb509cb0132cad9bf-168"
Server: AmazonS3
X-Cache: Hit from cloudfront
Via: 1.1 ece495703bac6f634e6e16b4037affae.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: NRT57-C4
X-Amz-Cf-Id: yU_IRn8NYIgyQfilmDI5O9arRCy9v9zSDmDkkV40LC73zFFtU9q2MA==
Age: 189

authors: gclove lkreimann


text:
Some tweaks to the API styling

Update the logo
Make fonts and colors a bit more consistent with the rest of the docs
Don't uppercase TOC menu items
Update _fetch_upstream_resources script to make the logo-URL relative
to the current website.

Before:

After:

@usha-mandya ptal


text:
extensions-sdk reorg (rebase)

this is a rebase of https://github.com/docker/docker.github.io/pull/14917, which had merge conflicts
What this does is;

squashed all the commits from https://github.com/docker/docker.github.io/pull/14917
added a separate commit that moves some of the files, so that the second commit can be used to review the changes that were made
note that we should review if any of the changes made in these files should be restored; I think those were from https://github.com/docker/docker.github.io/commit/e1a8d3bbde62f144efd6e7c73dfc2d349146767f

Oh; before I fail to mention; if we merge this PR, we should not squash, and keep the first commit separate to make sure that git preserves history (otherwise it won't correlate the old file with the new one)

Discussed with @aevesdocker to revert the dev -> dev-tools move, as it's not bringing too much advantages, but will require adding a lot of redirects; temporarily moved this back to draft, but will have a look at that

I reverted the dev -> dev-tools move, and moved the API reference changes to https://github.com/docker/docker.github.io/pull/14988 and opened an internal PR to update the docs generator (https://github.com/docker/pinata/pull/18357)

@thaJeztah I think the toc needs to be updated to reflect the dev-tools to dev move as I'm getting broken links on the preview

@aevesdocker thanks! Yes, I just noticed that, and pushed
Also wondering if we should change api/overview.md to api/index.md, but we can look at that in a follow-up

arf; more to fix; looks like there's a redirect loop now

authors: aevesdocker thaJeztah


text:
device-mapper: reload ioctl on docker-***-pool failed: Invalid argument

Daemon won't start.
I'm messing around on a 13.10 x64 digitalocean droplet, and I can potentially provide access for debugging if desired, though I may destroy it before too long (find my contact in my github profile).
I've not done anything to the machine other than apt-get update and installing according to the instructions here: http://docs.docker.io/en/latest/installation/ubuntulinux/#ubuntu-raring-saucy.
Client version: 0.8.1
Go version (client): go1.2
Git commit (client): a1598d1

$ docker -D -d
[/var/lib/docker|e60d3fbc] +job serveapi(unix:///var/run/docker.sock)
[/var/lib/docker|e60d3fbc] +job initserver()
[/var/lib/docker|e60d3fbc.initserver()] Creating server
2014/03/07 18:17:36 Listening for HTTP on unix (/var/run/docker.sock)
[debug] api.go:1084 Registering GET, /info
[debug] api.go:1084 Registering GET, /version
[debug] api.go:1084 Registering GET, /images/viz
[debug] api.go:1084 Registering GET, /containers/ps
[debug] api.go:1084 Registering GET, /containers/{name:.*}/top
[debug] api.go:1084 Registering GET, /events
[debug] api.go:1084 Registering GET, /images/json
[debug] api.go:1084 Registering GET, /images/{name:.*}/get
[debug] api.go:1084 Registering GET, /images/{name:.*}/history
[debug] api.go:1084 Registering GET, /images/{name:.*}/json
[debug] api.go:1084 Registering GET, /containers/{name:.*}/changes
[debug] api.go:1084 Registering GET, /images/search
[debug] api.go:1084 Registering GET, /containers/json
[debug] api.go:1084 Registering GET, /containers/{name:.*}/export
[debug] api.go:1084 Registering GET, /containers/{name:.*}/json
[debug] api.go:1084 Registering GET, /containers/{name:.*}/attach/ws
[debug] api.go:1084 Registering POST, /commit
[debug] api.go:1084 Registering POST, /images/{name:.*}/tag
[debug] api.go:1084 Registering POST, /containers/create
[debug] driver.go:79 Error loading driver aufs: AUFS was not found in /proc/filesystems
[debug] deviceset.go:500 Generated prefix: docker-253:0-398374
[debug] deviceset.go:503 Checking for existence of the pool 'docker-253:0-398374-pool'
[debug] api.go:1084 Registering POST, /containers/{name:.*}/kill
[debug] api.go:1084 Registering POST, /containers/{name:.*}/stop
[debug] api.go:1084 Registering POST, /containers/{name:.*}/resize
[debug] api.go:1084 Registering POST, /auth
[debug] api.go:1084 Registering POST, /images/{name:.*}/insert
[debug] api.go:1084 Registering POST, /images/create
[debug] api.go:1084 Registering POST, /images/load
[debug] api.go:1084 Registering POST, /images/{name:.*}/push
[debug] api.go:1084 Registering POST, /containers/{name:.*}/start
[debug] api.go:1084 Registering POST, /build
[debug] api.go:1084 Registering POST, /containers/{name:.*}/restart
[debug] api.go:1084 Registering POST, /containers/{name:.*}/wait
[debug] api.go:1084 Registering POST, /containers/{name:.*}/attach
[debug] api.go:1084 Registering POST, /containers/{name:.*}/copy
[debug] api.go:1084 Registering DELETE, /containers/{name:.*}
[debug] api.go:1084 Registering DELETE, /images/{name:.*}
[debug] api.go:1084 Registering OPTIONS,
[debug] api.go:1194 docker group found. gid: 999
[debug] deviceset.go:517 Pool doesn't exist. Creating it.
[debug] deviceset.go:380 libdevmapper(3): ioctl/libdm-iface.c:1768 (-1) device-mapper: reload ioctl on docker-253:0-398374-pool failed: Invalid argument
[debug] deviceset.go:534
--->Err: Error running DeviceCreate (createPool)

[debug] driver.go:79 Error loading driver devicemapper: Error running DeviceCreate (createPool)
[debug] runtime.go:625 Using graph driver vfs
[debug] runtime.go:640 Creating images graph
[debug] graph.go:62 Restored 0 elements
[debug] runtime.go:652 Creating volumes graph
[debug] graph.go:62 Restored 0 elements
[debug] runtime.go:657 Creating repository list
[/var/lib/docker|e60d3fbc] +job init_networkdriver()
[/var/lib/docker|e60d3fbc.init_networkdriver()] creating new bridge for docker0
[debug] driver.go:259 Creating bridge docker0 with network 172.17.42.1/16
Error creating bridge: package not installed[/var/lib/docker|e60d3fbc] -job init_networkdriver() = ERR (1)
init_networkdriver: Error creating bridge: package not installed[/var/lib/docker|e60d3fbc] -job initserver() = ERR (1)
2014/03/07 18:17:36 initserver: init_networkdriver: Error creating bridge: package not installed

@dallas' solution worked like a charm. Thank you. <3

authors: liquid pwaller

repo: docker/docker


text:
Fix docker cli silently does nothing when provided a slash-prefixed container name

Signed-off-by: Lei Jitang leijitang@huawei.com
closed #16563
When provided a slash-prefixed container name, the docker cli silently does nothing and thing going worse for cli docker attach and docker start -ai, it will block.
[lei@fedora ~]$ docker start -ai /test

and there is no any response whatever you type.

I think this solution is a bit too specific. There are a several related cases this won't catch:
docker attach //foo
docker attach foo/
docker attach %2Ffoo

My first thoughts were to run the field through url.QueryEscape. But from some tests with nc, this doesn't appear to solve it.
Perhaps we should just validate that the argument contains only [a-zA-Z0-9_.-].

@phemmer  My first thought about this is that the name such as //foo foo/ %2Ffoo is not a valid name,  you can't create a container with these name, but /foo is a valid name, so this PR is to ensure the valid name work normally.
I agree with you that we could just validate the argument.
And what confuse me is that docker inspect /test can work normally, after some investigating,
I found the CLI which call GET API can work normally with /foo such as docker top docker diff docker logs, but the CLI call POST API can't work with /foo such as docker start docker attach docker stop  Do you have any idea about this?

@coolljt0725 looks like it has to do with how the URL router is handling route cleanup differently between GET and POST.  GET is very lenient on slashes.. I can start a container named "bob", and any of the following will return valid inspect data json:
GET /containers/bob/json
GET /containers//bob/json
GET /containers///bob/json
GET /containers///bob/////json

In the case of POST, the daemon (or more specifically Gorilla muxer) responds with a 301, but our client code doesn't handle redirects, so it does nothing useful:
POST /containers//bob/stop?t=10

# response:
HTTP/1.1 301 Moved Permanently
Location: /v1.21/containers/bob/stop?t=10
Date: Fri, 25 Sep 2015 14:53:56 GMT
Content-Length: 0
Content-Type: text/plain; charset=utf-8

If I use an external HTTP client which handles redirects and call the bare API, it actually works perfectly even with slashes.

@estesp  I'll investigate on how URL router handling route cleanup differently between GET and POST and see if we can make POST work perfectly just like GET tomorrow, it's midnight in my place  :-)
Should our client handle the redirect? I can work on this if we need

@estesp  After investigating, there seems nothing to do with the route cleanup. Both GET and POST will response the http.StatusMovedPermanently (https://github.com/gorilla/mux/blob/master/mux.go#L71). But the http client(https://golang.org/src/net/http/client.go?s=6009:6070#L163 used by docker client at https://github.com/docker/docker/blob/master/api/client/utils.go#L94) handle automatically redirecting differently between GET and POST
For GET  when the statusCode is StatusMovedPermanently, StatusFound, StatusSeeOther, StatusTemporaryRedirect, it will automatically redirect
For POST when the statusCode is StatusFound, StatusSeeOther, it will automatically redirect

design makes sense for me
ping @docker/core-maintainers

Just wondering... why are we tying to clean up the /'s ? Why aren't we treating them as part of the name and escaping them in the URL?  Then let the daemon complain about bad /'s in the name.  Seems like we should be in total control over the shape of the URL, not let the user's input influence it. And we shouldn't try to "clean up" the user's data like this, if they gave us "////bob" then we should complain that its not a valid name, not convert it to "/bob" and be happy with it (assuming /bob is ok).

@duglin What I think is /foo is a valid name, docker will add a prefix slash to name by default
$ docker create --name=foo busybox
8befdf87e4db2b237eeeac88dca13a61ee63ce4421e2d32a840054828101b275
$ docker inspect -f {{.Name}} foo
/foo

but docker cli with slash-prefix name will cause a redirect by http server and daemon has no chance to handle it. we should make docker cli with valid name work properly

@coolljt0725 I probably wasn't clear. I'm not suggesting that we change what is valid or invalid values, what I'm suggesting is that we don't try to change the name that the user provided to us. For example, if they give us "////bob" and that's valid then that's fine. But if its invalid I think it would be wrong to convert it to a valid value, "bob", and then use that because that is not what the user asked for  Additionally, if we just escaped the value (e.g. "/bob" -> "%25bob" ) in the URL then we wouldn't need to worry about redirects.
Now, the fact that "bob" might be convert to "/bob" due to our internal processing is a different issue.

@duglin @estesp Where are we on this one? I personally agree with https://github.com/docker/docker/pull/16577#issuecomment-152725943, I.e. produce an error instead of trying to automagically fix it, but I'm not sure that works in this case (see https://github.com/docker/docker/pull/16577#issuecomment-143239607)

@coolljt0725 Could you explain why client didn't fail exactly?

@coolljt0725 where are we on this one?

@LK4D4 sorry for the late response.  https://github.com/docker/docker/pull/16577#issuecomment-143719501 has explained why client didn't fail exactly. The name with slash prefixed will cause a redirect, but our client code does not handle the redirect and the GET method will automatically handle the redirect. so  docker inspect /foo work normally but docker start /foo silently does nothing.
@thaJeztah I think why we should not produce an error instead is that the docker cli with GET method will work normally and docker cli with POST method does not.They should have the same behavior.
@duglin  I think the proper fix may handle the redirect in the client side.  The name with slash prefix will  cause a redirect, but our client does nothing with redirect response and just ignore it.

@coolljt0725 but my point is that nothing the user gives us should cause a redirect. In other words, any slashes in the name should be escaped so that we have total control over the URL and the slashes are given to the daemon "as is" and we let it either accept or reject the name based on those slashes.

@duglin  The gorilla/mux will cause a redirect. If we has prefix slash in the name like docker inspect /bob , the API would like this GET /containers//bob/json and then gorilla/mux will return a rediect see https://github.com/gorilla/mux/blob/master/mux.go#L71 . My englist is not good, I don't know if I explain it correctly -:)

@coolljt0725 we should never send /containers//bob/json, it should be /containers/%2Fbob/json then there won't be a redirect.

However, I do agree that we should handle redirects on the CLI in general, but not as part of this issue since, to me, that's a different topic/issue.

@duglin  So, for this issue, we should validate the input name?

How about fixing this by changing
serverResp, err := cli.call("GET", "/containers/"+cleanName+"/json", nil, nil)

to
serverResp, err := cli.call("GET", filepath.Join("/containers", name, "json"), nil, nil)

@coolljt0725 no, I don't think we should validate the name on the CLI (unless we really want to just so we "fail fast"). I think the name should be escaped (convert things like / into %2f ) and then let the daemon un-escape it and do the checking there.  This way we catch the CLI and REST API flows.
@crosbymichael I don't think that will solve the issue because I think its a mistake to "fix" the user's data for them. Meaning, if they asked for "///bob" then we should either give them "///bob" or generate an error. Converting it to just "bob" isn't what they asked for.

just using url.QueryEscape(name) before we stick it in the URL should do it, i think.

@duglin  I have try that, it didn't work..

Did you run the name thru url.QueryUnescape() on the daemon side?  What didn't work?

This may be a golang problem - see the trail starting with https://github.com/golang/go/issues/6658
Still digging though....

ok, its an issue with gorilla/mux - if we change https://github.com/docker/docker/blob/master/vendor/src/github.com/gorilla/mux/mux.go#L69 to use URL.RawPath instead of URL.Path then the %2f will be passed to the daemon (as the 'name' variable) and we'll see an error like:
Error response from daemon: no such id: ///foo

Checking into gorilla/mux's issue: https://github.com/gorilla/mux/issues/132

I think we should wait for the fix to the https://github.com/gorilla/mux/issues/132 issue as I think that's the more correct solution.

I agree with @duglin. Should we use EscapePath() instead? I don't think we should be worried about dropping old go versions support, since we don't need this for a patch release very urgently :smile:

So what's the status here; should we wait for https://github.com/gorilla/mux/issues/132, or can we use EscapePath() now (independent of the gorilla/mux issue?)

I'm not sure how we would use EscapedPath() since I think its the mux code that would need to be changed not Docker, so we may need to wait for https://github.com/gorilla/mux/issues/132 . But @calavera is more familiar with the http processing logic than I am so perhaps there's a way to trick-out the mux code w/o modifying it.

ping @calavera can you have one more look? If this depends on https://github.com/gorilla/mux/issues/132, I suggest to close until that's resolved

@thaJeztah I think that is sane, then we dont have to bother @coolljt0725 with rebases in the meantime and can wait on the dependent change :)

@coolljt0725 https://github.com/gorilla/mux/issues/132 is now fixed, did you want to re-open this one and see if it can be fixed?

@dugin I bumped the gorilla/mux to the latest commit and rebuilt the docker, but  issues seems still exists.
[lei@fedora docker]$ docker run -tid busybox
a4ccbdb7f0c8852cfc6d77185fafe166bd0d557c1c31077311b21434450ae544
[lei@fedora docker]$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
a4ccbdb7f0c8        busybox             "sh"                11 seconds ago      Up 9 seconds                            pensive_shannon
[lei@fedora docker]$ docker attach /pensive_shannon

the attach still blocked.

@coolljt0725 did you escape the outgoing URL (e.g. use EscapePath()) on the name so that the / in the name is converted to %25 and then won't be treated as a path separator?

@duglin  oh, sorry, I forgot the EscapePath(), I'll  try again

@duglin  I escape the outgoing URL (use url.QueryEscape) on the name, the docker attach /pensive_shannon  failed with.
No such container: %2Fpensive_shannon
If I escape the outgoing URL on client and un-escape the url on daemon side, it works
[lei@fedora docker]$ docker ps -l
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                      PORTS               NAMES
a4ccbdb7f0c8        busybox             "sh"                5 hours ago         Exited (0) 32 seconds ago                       pensive_shannon
[lei@fedora docker]$ docker start pensive_shannon

pensive_shannon
[lei@fedora docker]$
[lei@fedora docker]$ docker attach /pensive_shannon

/ # [lei@fedora docker]$
[lei@fedora docker]$
[lei@fedora docker]$ docker attach ////pensive_shannon
No such container: ////pensive_shannon
persistent connection closed
[lei@fedora docker]$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
a4ccbdb7f0c8        busybox             "sh"                5 hours ago         Up 50 seconds                           pensive_shannon

with a simple patch below
diff --git a/api/server/router/container/container_routes.go b/api/server/router/container/container_routes.go
index a03ef95..e7aa0b6 100644
--- a/api/server/router/container/container_routes.go
+++ b/api/server/router/container/container_routes.go
@@ -5,6 +5,7 @@ import (
        "fmt"
        "io"
        "net/http"
+       "net/url"
        "strconv"
        "strings"
        "syscall"
@@ -416,7 +417,7 @@ func (s *containerRouter) postContainersAttach(ctx context.Context, w http.Respo
        if err != nil {
                return err
        }
-       containerName := vars["name"]
+       containerName, _ := url.QueryUnescape(vars["name"])

        _, upgrade := r.Header["Upgrade"]
        detachKeys := r.FormValue("detachKeys")
diff --git a/vendor/src/github.com/docker/engine-api/client/container_attach.go b/vendor/src/github.com/docker/engine-api/client/container_attach.go
index 1b616bf..cca7873 100644
--- a/vendor/src/github.com/docker/engine-api/client/container_attach.go
+++ b/vendor/src/github.com/docker/engine-api/client/container_attach.go
@@ -30,5 +30,5 @@ func (cli *Client) ContainerAttach(ctx context.Context, container string, option
        }

        headers := map[string][]string{"Content-Type": {"text/plain"}}
-       return cli.postHijacked(ctx, "/containers/"+container+"/attach", query, nil, headers)
+       return cli.postHijacked(ctx, "/containers/"+url.QueryEscape(container)+"/attach", query, nil, headers)

With this patch, with and without bumping the mux to latest commit both work.

That's interesting that it works even w/o the fix to mux.  Perhaps some other change over the last few months helped.  Did you want to reopen this PR with your change?

@duglin  Maybe the changes of golang helped. I'll work on this later

authors: LK4D4 calavera coolljt0725 crosbymichael duglin estesp jfrazelle phemmer thaJeztah


text:
Hash some build time args when necessary

- What I did
Hash some build time args in cache and history.
- How I did it
Use sha256 to hash certain build time args when building a Dockerfile.
- How to verify it
Use command
docker build --build-arg http_proxy=http://account:password@server:port . to build a image imgName.
Before this PR, you'll see the whole http_proxy info (account and password) from docker history --no-trunc imgName.
After this PR, you'll see http_proxy=7c87f39bb182... only hash number in arg value.
- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)
Some environments depend on proxys to connect to internet,
so we use --build-arg to pass http_proxy and
https_proxy envs to container when building a Dockerfile.
And it'll make images made in this way undistributable
because it contains proxy accounts and passwords in
docker history.
This patch hashes certain build args (currently only
http_proxy, https_proxy and ftp_proxy), so we can
keep these information private and don't have side
effects as the approach in #18702.
Signed-off-by: Qiang Huang h.huangqiang@huawei.com

This was suggested on the original PR, but decided against, so I'm not sure this will be accepted
ping @diogomonica @NathanMcCauley

@thaJeztah I don't see any object decision in the original PR about hashing the secret info, is there any threads that I can follow?

It may have been the one before the PR that was merged; originally it hashed the args, but it was not cryptically secure enough, so for that reason hashing was removed to not give a false sense of security

The issue is not specific to http_proxy, https_proxy, and ftp_proxy; we should cover AWS_ACCESS_KEY and so on, but it is impossible to cover all the things.
So I think something like docker build --hide-build-arg http_proxy,https_proxy,ftp_proxy is better.
For encryption, I think we should better do stretching (i.e. repeating applying the hash function for number of times)

Encryption would be too complicated for this. And I don't think repeating applying hash function would help because hashing is not secure because it can be reverted in theory, increase reverting time won't make community think it's secure, which is why I use sha256 instead of sha512, if they think sha256 is not secure, sha512 won't either.
A new option like --hide-build-arg looks a bit harmful to me, it'll make a image unreproducible.

@thaJeztah Thanks for clarifying, I know it's not ideal, hope we can get some inputs from @diogomonica @NathanMcCauley , this is a big security issue to us and exists for a long time.

What you want is build-time secrets, correct? I think we should handle this as a build-time option called --secrets, that receives a path to a tar with secrets and creates a tmpfs volume that is mounted in into the containers doing the actual building under /secrets/.
/cc @crosbymichael

@hqhq
What we were thinking of doing is creating a tmpfs mount inside the container so that only that specific container an see the secrets provided to it.  Then in the daemon or a hook process, we setns into the containers mount namespace and extract the contents that were provided in the tar.
It should work the same across both build and any container run.  What do you think? I think the runtime implementation would be very simple to do.

@diogomonica in this case, it's not really about "build time secrets", but people behind the GFW, that need to use a proxy so that they can actually use docker; unfortunately those proxy settings (with credentials) are baked in the image history

@diogomonica @crosbymichael Thanks for replying, as @thaJeztah said, it's not only about something we don't want it stored in the image, but also can't been seen in image history. Currently build time envs (including some secrets) is not part of image, but baked in the image history.

IMO, it's better to just hash the account and passwd of the proxy, so user can still see which proxy server was used during  build from docker history.

hashing is not making something a secret
if we only hash specific vars then an attacker knows that a hashed field is likely to contain a secret and it will be much simpler to brute force it

OK, looks like hashing is not a acceptable approach, I'll close this PR, thanks all.

@thaJeztah Is there any issues tracing this kind of problem? If not, I can create one.

@hqhq there's the "generic" https://github.com/docker/docker/issues/13490, but possibly there's another issue
ping @ehazlett do you know? ^^

@thaJeztah  yes that is currently the issue we are using to track these

@thaJeztah @ehazlett Thanks.

authors: AkihiroSuda coolljt0725 cpuguy83 crosbymichael diogomonica ehazlett hqhq thaJeztah


text:
Fix the broken link

Fix the broken link which should be updated when modify the link topic.
cc @londoncalling
Signed-off-by: yuexiao-wang wang.yuexiao@zte.com.cn

@yuexiao-wang @thaJeztah That was my fault. I changed the topic title and forgot to change the link. :-/ Thanks you guys!!

authors: londoncalling yuexiao-wang


text:
refactor cluster/helper.go and split cluster.go into multiple files

Signed-off-by: allencloud allen.sun@daocloud.io
This PR did two things:

split cluster/cluster.go into multiple files services.go, nodes.go, networks.go, swarm.go, tasks.go
refactor cluster/helper.go to make it more efficient.

I found that when docker is in Swarm Mode, all the handlers about swarm cluster is located in cluster/cluster.go except secret handlers. We could split cluster/cluster.go into several parts like secrets.go, services.go, swarm.go, nodes.go, network.go, task.go and so on. This explains the purpose of my first part of work.
For second part of this pr's work, I refactor the cluster/helper.go, I think I did not change the original logic, while I changed to a more reasonable way, like return fast, like avoid a long block and so on.
- What I did
- How I did it
- How to verify it
- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

I agree with @vdemeester, I'd prefer to see cluster.go getting split into multiple files.

Thanks for your feedback. @vieux @vdemeester
I agree with splitting cluster.go into multiple files. Now I updated the PR to finish the split. PTAL.

/cc @aaronlehmann

Unfortunately this is very hard to review. I think it would be helpful to split the code movement into separate commits from actual changes to the functions.

Thanks for your feedback, @aaronlehmann . The original way is indeed hard to review. And I try to split this PR into 6 commits which makes review a little bit easy. PTAL, thanks a lot.

LGTM
Thanks, this is a worthy cleanup.
@tonistiigi: Are you okay with the reorganization?

LGTM

authors: aaronlehmann allencloud tonistiigi vdemeester vieux


text:
dependency failed to start: container genai-stack-database-1 exited (1)

How to solve the following problem?
OS:Win10
After docker compose up, I got these:
Attaching to api-1, bot-1, database-1, front-end-1, loader-1, pdf_bot-1, pull-model-1
database-1    |
database-1    | Folder /data is not accessible for user: 7474 or group 7474. This is commonly a file permissions issue on the mounted folder.
database-1    |
database-1    | Hints to solve the issue:
database-1    | 1) Make sure the folder exists before mounting it. Docker will create the folder using root permissions before starting the Neo4j container. The root permissions disallow Neo4j from writing to the mounted folder.
database-1    | 2) Pass the folder owner's user ID and group ID to docker run, so that docker runs as that user.
database-1    | If the folder is owned by the current user, this can be done by adding this flag to your docker run command:
database-1    |   --user=$(id -u):$(id -g)
database-1    |
pull-model-1  | pulling ollama model llama2 using http://host.docker.internal:11434
database-1 exited with code 1
Gracefully stopping... (press Ctrl+C again to force)
dependency failed to start: container genai-stack-database-1 exited (1)
.env file I used:
#*****************************************************************
LLM and Embedding Model
#*****************************************************************
LLM=qwen2.5:3b #or any Ollama model tag, gpt-4 (o or turbo), gpt-3.5, or any bedrock model
EMBEDDING_MODEL=mxbai-embed-large:latest #or google-genai-embedding-001 openai, ollama, or aws
#*****************************************************************
Neo4j
#*****************************************************************
NEO4J_URI=neo4j://database:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=12345678
#*****************************************************************
Langchain
#*****************************************************************
Optional for enabling Langchain Smith API
LANGCHAIN_TRACING_V2=true # false
LANGCHAIN_ENDPOINT="https://api.smith.langchain.com"
LANGCHAIN_PROJECT="" #your-project-name
LANGCHAIN_API_KEY="" #your-api-key ls_...
#*****************************************************************
Ollama
#*****************************************************************
OLLAMA_BASE_URL=http://host.docker.internal:11434

It seems that .env has no effect, ollama model llama2 was pulled instead of qwen2.5:3b

getting same issue in my project setup as well

To resolve this issue open docker-compose.yaml file then edit database service by changing the volumes section as below
volumes:
      - ./data:/data

To resolve this issue open docker-compose.yaml file then edit database service by changing the volumes section as below
volumes:
      - ./data:/data

Here this solution didn't work

authors: AswiniOmsakthi GeoZhanglw snepraj2709 thecodergus

repo: docker/genai-stack


text:
No Contributing guidlines

Requesting to add CONTRIBUTION.md file with guidlines for new contributors to be followed.
I would like to work on this issue .

@oskarhane is this going to be implemented?

sure @oskarhane or if you can provide me the docs i could also contribute in that way probably?


text:
(HTTP code 404) no such container - oci runtime error: exec: "sh": executable file not found in $PATH

Expected behavior
Actual behavior
Information about the Issue
(HTTP code 404) no such container - oci runtime error: exec: "sh": executable file not found in $PATH
Steps to reproduce the behavior

save container
docker save x/centos-base > x_centos-base.tar
import container
cat x_centos-base.tar | sudo docker import - x/centos-base:latest
create container from "My Images"
run container and receive the following error:

"(HTTP code 404) no such container - oci runtime error: exec: "sh": executable file not found in $PATH"

Used load instead of import.

repo: docker/kitematic


text:
(HTTP code 404) unexpected

Expected behavior
Actual behavior
Information about the Issue
Steps to reproduce the behavior

...
...

No information provided or steps to replicate - Closing 😞


text:
Add exceptions for checkMountDest

We still should prevent mount in /sys for security, special
destionations like /sys/fs/cgroup we can treated as exceptions,
these can be unknown from users for now.
Signed-off-by: Qiang Huang h.huangqiang@huawei.com

ping @ewindisch can you provide some feedback here?

:+1:  I absolutely prefer to whitelist than blacklist. If there are other paths we should whitelist, it would be good to get them in together before the next release.

LGTM
Dunno what other paths though.
ping @crosbymichael

authors: LK4D4 crosbymichael ewindisch hqhq

repo: docker/libcontainer


text:
Issue #184 - Improve Build Image in Makefile

Signed-off-by: Sotiris Salloumis sotiris.salloumis@gmail.com

Sorry learning the environment I guess commenting on the issue should be enough to see the patch. If not please let me know what actions I need to do.

Thanks for the contribution @codergr, LGTM

LGTM
/cc @aboch @mrjana @mavenugo

@codergr
When you pushed this PR, code was still quickly evolving. Because of that and other priorities, we did not all allocate time to review your change. Sorry for that.
Given the code has change considerably since then, I will go on and close this one for now.
But please do not esitate to open a new PR, if you see room for improvement in current code.
It will get more attention this time.

authors: aboch codergr dave-tucker

repo: docker/libnetwork


text:
Fix the daemon panic on consul server restart

Fixes #671
Signed-off-by: Santhosh Manohar santhosh@docker.com

LGTM

LGTM

authors: mavenugo mrjana sanimej