text:
pygrambank check output different

The check command used to output a col called "selected", it doesn't seem to do so anymore. Yesterday, @chrzyki helped me run pygrambank. We had to reinstall various things
It may be that the command I ran is somehow still an outdated version.
For example, HJH_mort1237 and MH_mort1237 both contain lots of errors, but the sheet that was actually selected for this language was coded by @d97hah and has no automatic errors.
I've made some changes to the r-script so that it can take the new check output, but since there's not information on selected I can't run it yet and update the relevant lists. The current version of the script is here:
https://github.com/glottobank/Grambank/compare/script_to_update?expand=1

Passing --report report.tsv to the check command should give you what you want:
$ grambank check -husage: grambank check [-h] [--report REPORT]

Check original_sheets/ for correctness.

optional arguments:
  -h, --help       show this help message and exit
  --report REPORT  Path of TSV file, to which results will be written.
                   (default: None)

Right okay. I didn't know there had been a change. What's the easiest way to stay up to date with changes of this kind?

Switching to a new way of organizing the cli will not happen again any time soon. So changes of this kind are rare. But whenever you update a package (which you did with pygrambank), and then experience differences in behaviour, you'd check the version number:
Python 3.5.2 (default, Oct  8 2019, 13:06:37)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygrambank
>>> pygrambank.__version__
'1.0.1.dev0'

and if the major version changed, differences are to be expected (i.e. you'd have to relearn the behaviour, e.g. reading the cli help); if not, you'd file an issue.

Okay, right.
So far, I've only ever needed one command in pygrambank - check. This change seems to be a reversion to a previous state of the command rather than a change to something new, and the cli help documentation didn't specify that the command line output was no longer the same, and that the report file was different from what's shown on the console. If there are changes like this in other commands in these projects, chances are I'll trial and error it and if that doesn't work I'll be filing issues. If that's alright, then it'll be fine.

authors: HedvigS xrotwang

repo: glottobank/pygrambank


text:
Support CTAS(Create Table As Select ~) statement

🚧 This PR is a draft yet
Support CTAS(Create Table As Select ~) statement(#404)

[x] Add a test case: src/tests/alter/create_table.rs
[x] Add source to translate if there is query statements: src/translate/mod.rs
[x] Get source from ast: src/ast/mod.rs
[x] Pass source to executor: src/executor/execute.rs
[ ] ⏳ Implement CTAS

[x] Get source table's schema
[x] Create new table with the source schema
[x] Insert data from source to new table
[x] Clear redundant and repetitive code
[ ] Should we add payload?

[x] Add more test cases

Codecov Report

Merging #414 (6c6a02b) into main (6add0fa) will decrease coverage by 0.10%.
The diff coverage is 74.07%.

@@            Coverage Diff             @@
##             main     #414      +/-   ##
==========================================
- Coverage   91.01%   90.90%   -0.11%
==========================================
  Files         144      144
  Lines        8432     8478      +46
==========================================
+ Hits         7674     7707      +33
- Misses        758      771      +13

Impacted Files
Coverage Δ

src/executor/execute.rs
67.87% <0.00%> (-0.42%)
:arrow_down:

src/executor/alter/table.rs
78.66% <72.09%> (-11.08%)
:arrow_down:

src/tests/alter/create_table.rs
100.00% <100.00%> (ø)

src/translate/mod.rs
98.21% <100.00%> (+0.10%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6add0fa...6c6a02b. Read the comment docs.

source: gharchive/pull-request

authors: codecov-commenter devgony

repo: gluesql/gluesql

license_type: permissive

license_source: github-api


text:
Some times vagrant up is failing

Vagrant up is failing with below error, it's not reproducible every time. Here my observation is that, with out  running prepare.sh on base deploy directory.  Copy data from deploy directory to new directory, run prepare.sh on new directory then if we perform "vagrant up" , at the end vagrant up is failing below error.
==> kube3: An error occurred. The error will be shown after all tasks complete.
An error occurred while executing multiple actions in parallel.
Any errors that occurred are shown below.

An error occurred while executing the action on the 'kube3'
machine. Please handle this error then try again:

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

@rmadaka you need to add complete logs from vagrant up.

@Madhu-1 TASK [kubernetes-apps/ansible : Kubernetes Apps | Wait for kube-apiserver] *****
Tuesday 23 October 2018  23:58:02 +0530 (0:00:00.347)       0:11:49.520 *******
ok: [kube1]
TASK [kubernetes-apps/ansible : Kubernetes Apps | Delete old CoreDNS resources] ***
Tuesday 23 October 2018  23:58:03 +0530 (0:00:01.284)       0:11:50.804 *******
ok: [kube1] => (item=deploy)
ok: [kube1] => (item=configmap)
ok: [kube1] => (item=svc)
TASK [kubernetes-apps/ansible : Kubernetes Apps | Delete kubeadm CoreDNS] ******
Tuesday 23 October 2018  23:58:05 +0530 (0:00:01.966)       0:11:52.771 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Delete old KubeDNS resources] ***
Tuesday 23 October 2018  23:58:05 +0530 (0:00:00.119)       0:11:52.891 *******
ok: [kube1] => (item=deploy)
ok: [kube1] => (item=svc)
TASK [kubernetes-apps/ansible : Kubernetes Apps | Delete kubeadm KubeDNS] ******
Tuesday 23 October 2018  23:58:07 +0530 (0:00:01.476)       0:11:54.367 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down CoreDNS Template] ***
Tuesday 23 October 2018  23:58:07 +0530 (0:00:00.162)       0:11:54.529 *******
changed: [kube1] => (item={u'type': u'configmap', u'name': u'coredns', u'file': u'coredns-config.yml'})
changed: [kube1] => (item={u'type': u'sa', u'name': u'coredns', u'file': u'coredns-sa.yml'})
changed: [kube1] => (item={u'type': u'deployment', u'name': u'coredns', u'file': u'coredns-deployment.yml'})
changed: [kube1] => (item={u'type': u'svc', u'name': u'coredns', u'file': u'coredns-svc.yml'})
changed: [kube1] => (item={u'type': u'clusterrole', u'name': u'coredns', u'file': u'coredns-clusterrole.yml'})
changed: [kube1] => (item={u'type': u'clusterrolebinding', u'name': u'coredns', u'file': u'coredns-clusterrolebinding.yml'})
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down Secondary CoreDNS Template] ***
Tuesday 23 October 2018  23:58:15 +0530 (0:00:07.520)       0:12:02.050 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down KubeDNS Template] ***
Tuesday 23 October 2018  23:58:15 +0530 (0:00:00.152)       0:12:02.203 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Patch system:kube-dns ClusterRole] ***
Tuesday 23 October 2018  23:58:15 +0530 (0:00:00.217)       0:12:02.421 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Start Resources] *************
Tuesday 23 October 2018  23:58:15 +0530 (0:00:00.111)       0:12:02.532 *******
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'4f11cac252dda1de9730138b24727619', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-config.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 459, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319287.78-197647888928355/source', u'group': u'root', '_ansible_item_label': {u'type': u'configmap', u'name': u'coredns', u'file': u'coredns-config.yml'}, 'item': {u'type': u'configmap', u'name': u'coredns', u'file': u'coredns-config.yml'}, u'checksum': u'7768ca5c0ca93fd5df09b0082224a21e64a442c4', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-config.yml', u'selevel': None, u'_original_basename': u'coredns-config.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319287.78-197647888928355/source', u'group': None, u'unsafe_writes': None, u'checksum': u'7768ca5c0ca93fd5df09b0082224a21e64a442c4', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'bfec6ec350bf47d850c62497fed4a879', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-sa.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 190, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319289.8-225830423288078/source', u'group': u'root', '_ansible_item_label': {u'type': u'sa', u'name': u'coredns', u'file': u'coredns-sa.yml'}, 'item': {u'type': u'sa', u'name': u'coredns', u'file': u'coredns-sa.yml'}, u'checksum': u'120a7b2f1f1e14e639ff7b5daa4ed21a980d80f7', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-sa.yml', u'selevel': None, u'_original_basename': u'coredns-sa.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319289.8-225830423288078/source', u'group': None, u'unsafe_writes': None, u'checksum': u'120a7b2f1f1e14e639ff7b5daa4ed21a980d80f7', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'7d464669b6fc51fac4ea9fa69401540b', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-deployment.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 2727, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319290.72-274529291592300/source', u'group': u'root', '_ansible_item_label': {u'type': u'deployment', u'name': u'coredns', u'file': u'coredns-deployment.yml'}, 'item': {u'type': u'deployment', u'name': u'coredns', u'file': u'coredns-deployment.yml'}, u'checksum': u'883ff5b38ca1c86279e01717924be70385439e5e', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-deployment.yml', u'selevel': None, u'_original_basename': u'coredns-deployment.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319290.72-274529291592300/source', u'group': None, u'unsafe_writes': None, u'checksum': u'883ff5b38ca1c86279e01717924be70385439e5e', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'c304c8275b2325fac1097c6456db3db2', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-svc.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 585, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319292.02-197766345158336/source', u'group': u'root', '_ansible_item_label': {u'type': u'svc', u'name': u'coredns', u'file': u'coredns-svc.yml'}, 'item': {u'type': u'svc', u'name': u'coredns', u'file': u'coredns-svc.yml'}, u'checksum': u'89c95955f6fea7fc14283bee6594911f171d6012', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-svc.yml', u'selevel': None, u'_original_basename': u'coredns-svc.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319292.02-197766345158336/source', u'group': None, u'unsafe_writes': None, u'checksum': u'89c95955f6fea7fc14283bee6594911f171d6012', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'40cdcba30a60aa92bdb8acba416e4b95', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-clusterrole.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 324, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319293.08-70805323886615/source', u'group': u'root', '_ansible_item_label': {u'type': u'clusterrole', u'name': u'coredns', u'file': u'coredns-clusterrole.yml'}, 'item': {u'type': u'clusterrole', u'name': u'coredns', u'file': u'coredns-clusterrole.yml'}, u'checksum': u'd62b1b25b430df8254746340e5032b98df625491', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-clusterrole.yml', u'selevel': None, u'_original_basename': u'coredns-clusterrole.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319293.08-70805323886615/source', u'group': None, u'unsafe_writes': None, u'checksum': u'd62b1b25b430df8254746340e5032b98df625491', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'8547087bed620deaa317d38926f92ba0', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-clusterrolebinding.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 450, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319294.07-6148148231673/source', u'group': u'root', '_ansible_item_label': {u'type': u'clusterrolebinding', u'name': u'coredns', u'file': u'coredns-clusterrolebinding.yml'}, 'item': {u'type': u'clusterrolebinding', u'name': u'coredns', u'file': u'coredns-clusterrolebinding.yml'}, u'checksum': u'680a19b34a6814eeff494c4b89c050803a9c0c53', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-clusterrolebinding.yml', u'selevel': None, u'_original_basename': u'coredns-clusterrolebinding.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319294.07-6148148231673/source', u'group': None, u'unsafe_writes': None, u'checksum': u'680a19b34a6814eeff494c4b89c050803a9c0c53', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
TASK [kubernetes-apps/ansible : Kubernetes Apps | Check if netchecker-server manifest already exists] ***
Tuesday 23 October 2018  23:58:22 +0530 (0:00:06.979)       0:12:09.511 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Apply netchecker-server manifest to update annotations] ***
Tuesday 23 October 2018  23:58:22 +0530 (0:00:00.107)       0:12:09.619 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Netchecker Templates list] ***
Tuesday 23 October 2018  23:58:22 +0530 (0:00:00.106)       0:12:09.726 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Append extra templates to Netchecker Templates list for PodSecurityPolicy] ***
Tuesday 23 October 2018  23:58:22 +0530 (0:00:00.140)       0:12:09.866 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down Netchecker Template] ***
Tuesday 23 October 2018  23:58:23 +0530 (0:00:00.120)       0:12:09.987 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Purge old Netchecker server] ***
Tuesday 23 October 2018  23:58:23 +0530 (0:00:00.112)       0:12:10.099 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Start Netchecker Resources] ***
Tuesday 23 October 2018  23:58:23 +0530 (0:00:00.117)       0:12:10.217 *******
TASK [kubernetes-apps/ansible : Kubernetes Apps | Delete old kubernetes-dashboard resources] ***
Tuesday 23 October 2018  23:58:23 +0530 (0:00:00.126)       0:12:10.343 *******
ok: [kube1] => (item=ClusterRoleBinding)
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay down dashboard template] ***
Tuesday 23 October 2018  23:58:24 +0530 (0:00:01.072)       0:12:11.416 *******
changed: [kube1] => (item={u'type': u'deploy', u'name': u'kubernetes-dashboard', u'file': u'dashboard.yml'})
TASK [kubernetes-apps/ansible : Kubernetes Apps | Start dashboard] *************
Tuesday 23 October 2018  23:58:25 +0530 (0:00:01.301)       0:12:12.718 *******
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'e8b24dcba4936f16f3e0c0a8e0288706', u'uid': 0, u'dest': u'/etc/kubernetes/dashboard.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 6001, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319304.75-243873575723566/source', u'group': u'root', '_ansible_item_label': {u'type': u'deploy', u'name': u'kubernetes-dashboard', u'file': u'dashboard.yml'}, 'item': {u'type': u'deploy', u'name': u'kubernetes-dashboard', u'file': u'dashboard.yml'}, u'checksum': u'd3af91fa4447cbfeb57d03f0304fc127eebaca3d', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/dashboard.yml', u'selevel': None, u'_original_basename': u'dashboard.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1540319304.75-243873575723566/source', u'group': None, u'unsafe_writes': None, u'checksum': u'd3af91fa4447cbfeb57d03f0304fc127eebaca3d', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
TASK [kubernetes-apps/helm : Helm | Make sure HELM_HOME directory exists] ******
Tuesday 23 October 2018  23:58:27 +0530 (0:00:01.418)       0:12:14.137 *******
TASK [kubernetes-apps/helm : Helm | Set up helm launcher] **********************
Tuesday 23 October 2018  23:58:27 +0530 (0:00:00.109)       0:12:14.246 *******
TASK [kubernetes-apps/helm : Helm | Lay Down Helm Manifests (RBAC)] ************
Tuesday 23 October 2018  23:58:27 +0530 (0:00:00.111)       0:12:14.357 *******
TASK [kubernetes-apps/helm : Helm | Apply Helm Manifests (RBAC)] ***************
Tuesday 23 October 2018  23:58:27 +0530 (0:00:00.151)       0:12:14.509 *******
TASK [kubernetes-apps/helm : Helm | Set up TLS] ********************************
Tuesday 23 October 2018  23:58:27 +0530 (0:00:00.142)       0:12:14.651 *******
TASK [kubernetes-apps/helm : Helm | Install/upgrade helm] **********************
Tuesday 23 October 2018  23:58:27 +0530 (0:00:00.106)       0:12:14.758 *******
TASK [kubernetes-apps/helm : Helm | Force apply tiller overrides if necessary] ***
Tuesday 23 October 2018  23:58:27 +0530 (0:00:00.117)       0:12:14.876 *******
TASK [kubernetes-apps/helm : Helm | Set up bash completion] ********************
Tuesday 23 October 2018  23:58:28 +0530 (0:00:00.123)       0:12:14.999 *******
TASK [kubernetes-apps/registry : Registry | Create addon dir] ******************
Tuesday 23 October 2018  23:58:28 +0530 (0:00:00.104)       0:12:15.104 *******
TASK [kubernetes-apps/registry : Registry | Templates list] ********************
Tuesday 23 October 2018  23:58:28 +0530 (0:00:00.108)       0:12:15.213 *******
TASK [kubernetes-apps/registry : Registry | Append extra templates to Registry Templates list for PodSecurityPolicy] ***
Tuesday 23 October 2018  23:58:28 +0530 (0:00:00.101)       0:12:15.314 *******
TASK [kubernetes-apps/registry : Registry | Create manifests] ******************
Tuesday 23 October 2018  23:58:28 +0530 (0:00:00.102)       0:12:15.417 *******
TASK [kubernetes-apps/registry : Registry | Apply manifests] *******************
Tuesday 23 October 2018  23:58:28 +0530 (0:00:00.103)       0:12:15.521 *******
TASK [kubernetes-apps/registry : Registry | Create PVC manifests] **************
Tuesday 23 October 2018  23:58:28 +0530 (0:00:00.108)       0:12:15.629 *******
TASK [kubernetes-apps/registry : Registry | Apply PVC manifests] ***************
Tuesday 23 October 2018  23:58:28 +0530 (0:00:00.109)       0:12:15.739 *******
TASK [kubernetes-apps/persistent_volumes/openstack : Kubernetes Persistent Volumes | Lay down OpenStack Cinder Storage Class template] ***
Tuesday 23 October 2018  23:58:28 +0530 (0:00:00.117)       0:12:15.857 *******
TASK [kubernetes-apps/persistent_volumes/openstack : Kubernetes Persistent Volumes | Add OpenStack Cinder Storage Class] ***
Tuesday 23 October 2018  23:58:29 +0530 (0:00:00.109)       0:12:15.967 *******
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU| gather os specific variables] ***
Tuesday 23 October 2018  23:58:29 +0530 (0:00:00.110)       0:12:16.077 *******
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Set fact of download url Tesla] ***
Tuesday 23 October 2018  23:58:29 +0530 (0:00:00.128)       0:12:16.206 *******
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Set fact of download url GTX] ***
Tuesday 23 October 2018  23:58:29 +0530 (0:00:00.101)       0:12:16.308 *******
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Create addon dir] ***
Tuesday 23 October 2018  23:58:29 +0530 (0:00:00.111)       0:12:16.419 *******
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Create manifests for nvidia accelerators] ***
Tuesday 23 October 2018  23:58:29 +0530 (0:00:00.106)       0:12:16.526 *******
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Apply manifests for nvidia accelerators] ***
Tuesday 23 October 2018  23:58:29 +0530 (0:00:00.132)       0:12:16.659 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_private_key] ***
Tuesday 23 October 2018  23:58:29 +0530 (0:00:00.127)       0:12:16.786 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_region_id] ***
Tuesday 23 October 2018  23:58:29 +0530 (0:00:00.101)       0:12:16.888 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_tenancy_id] ***
Tuesday 23 October 2018  23:58:30 +0530 (0:00:00.136)       0:12:17.024 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_user_id] ***
Tuesday 23 October 2018  23:58:30 +0530 (0:00:00.117)       0:12:17.142 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_user_fingerprint] ***
Tuesday 23 October 2018  23:58:30 +0530 (0:00:00.108)       0:12:17.251 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_compartment_id] ***
Tuesday 23 October 2018  23:58:30 +0530 (0:00:00.114)       0:12:17.365 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_vnc_id] ***
Tuesday 23 October 2018  23:58:30 +0530 (0:00:00.120)       0:12:17.486 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_subnet1_id] ***
Tuesday 23 October 2018  23:58:30 +0530 (0:00:00.107)       0:12:17.594 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_subnet2_id] ***
Tuesday 23 October 2018  23:58:30 +0530 (0:00:00.107)       0:12:17.701 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_security_list_management] ***
Tuesday 23 October 2018  23:58:30 +0530 (0:00:00.106)       0:12:17.807 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Generate Configuration] ***
Tuesday 23 October 2018  23:58:30 +0530 (0:00:00.108)       0:12:17.916 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Encode Configuration] ***
Tuesday 23 October 2018  23:58:31 +0530 (0:00:00.114)       0:12:18.031 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Apply Configuration To Secret] ***
Tuesday 23 October 2018  23:58:31 +0530 (0:00:00.114)       0:12:18.145 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Apply Configuration] ***
Tuesday 23 October 2018  23:58:31 +0530 (0:00:00.111)       0:12:18.257 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Download Controller Manifest] ***
Tuesday 23 October 2018  23:58:31 +0530 (0:00:00.110)       0:12:18.367 *******
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Apply Controller Manifest] ***
Tuesday 23 October 2018  23:58:31 +0530 (0:00:00.138)       0:12:18.505 *******
PLAY [Deploy GCS] **************************************************************
TASK [GCS Pre | Cluster ID | Generate a UUID] **********************************
Tuesday 23 October 2018  23:58:31 +0530 (0:00:00.121)       0:12:18.627 *******
changed: [kube1]
TASK [GCS Pre | Cluster ID | Set gcs_gd2_clusterid fact] ***********************
Tuesday 23 October 2018  23:58:32 +0530 (0:00:00.686)       0:12:19.314 *******
ok: [kube1]
TASK [GCS Pre | Manifests directory | Create a temporary directory] ************
Tuesday 23 October 2018  23:58:32 +0530 (0:00:00.467)       0:12:19.782 *******
changed: [kube1]
TASK [GCS Pre | Manifests directory | Set manifests_dir fact] ******************
Tuesday 23 October 2018  23:58:33 +0530 (0:00:00.789)       0:12:20.571 *******
ok: [kube1]
TASK [GCS Pre | Manifests | Sync GCS manifests] ********************************
Tuesday 23 October 2018  23:58:33 +0530 (0:00:00.187)       0:12:20.758 *******
changed: [kube1] => (item=gcs-namespace.yml)
changed: [kube1] => (item=gcs-etcd-operator.yml)
changed: [kube1] => (item=gcs-etcd-cluster.yml)
changed: [kube1] => (item=gcs-gd2-services.yml)
changed: [kube1] => (item=gcs-csi.yml)
TASK [GCS Pre | Manifests | Create GD2 manifests] ******************************
Tuesday 23 October 2018  23:58:38 +0530 (0:00:05.039)       0:12:25.798 *******
included: /root/gcs-8/rajesh-75/tasks/create-gd2-manifests.yml for kube1
included: /root/gcs-8/rajesh-75/tasks/create-gd2-manifests.yml for kube1
included: /root/gcs-8/rajesh-75/tasks/create-gd2-manifests.yml for kube1
TASK [GCS Pre | Manifests | Create GD2 manifests for kube1 | Set fact kube_hostname] ***
Tuesday 23 October 2018  23:58:38 +0530 (0:00:00.127)       0:12:25.925 *******
ok: [kube1]
TASK [GCS Pre | Manifests | Create GD2 manifests for kube1 | Create gcs-gd2-kube1.yml] ***
Tuesday 23 October 2018  23:58:39 +0530 (0:00:00.184)       0:12:26.110 *******
changed: [kube1]
TASK [GCS Pre | Manifests | Create GD2 manifests for kube2 | Set fact kube_hostname] ***
Tuesday 23 October 2018  23:58:40 +0530 (0:00:01.151)       0:12:27.262 *******
ok: [kube1]
TASK [GCS Pre | Manifests | Create GD2 manifests for kube2 | Create gcs-gd2-kube2.yml] ***
Tuesday 23 October 2018  23:58:40 +0530 (0:00:00.214)       0:12:27.476 *******
changed: [kube1]
TASK [GCS Pre | Manifests | Create GD2 manifests for kube3 | Set fact kube_hostname] ***
Tuesday 23 October 2018  23:58:41 +0530 (0:00:01.164)       0:12:28.640 *******
ok: [kube1]
TASK [GCS Pre | Manifests | Create GD2 manifests for kube3 | Create gcs-gd2-kube3.yml] ***
Tuesday 23 October 2018  23:58:41 +0530 (0:00:00.181)       0:12:28.822 *******
changed: [kube1]
TASK [GCS | Namespace | Create GCS namespace] **********************************
Tuesday 23 October 2018  23:58:42 +0530 (0:00:01.057)       0:12:29.880 *******
ok: [kube1]
TASK [GCS | ETCD Operator | Deploy etcd-operator] ******************************
Tuesday 23 October 2018  23:58:43 +0530 (0:00:00.904)       0:12:30.784 *******
ok: [kube1]
TASK [GCS | ETCD Operator | Wait for etcd-operator to be available] ************
Tuesday 23 October 2018  23:58:44 +0530 (0:00:00.999)       0:12:31.784 *******
FAILED - RETRYING: GCS | ETCD Operator | Wait for etcd-operator to be available (50 retries left).
FAILED - RETRYING: GCS | ETCD Operator | Wait for etcd-operator to be available (49 retries left).
FAILED - RETRYING: GCS | ETCD Operator | Wait for etcd-operator to be available (48 retries left).
changed: [kube1]
TASK [GCS | ETCD Cluster | Deploy etcd-cluster] ********************************
Tuesday 23 October 2018  23:59:17 +0530 (0:00:32.682)       0:13:04.467 *******
ok: [kube1]
TASK [GCS | ETCD Cluster | Get etcd-client service] ****************************
Tuesday 23 October 2018  23:59:18 +0530 (0:00:01.118)       0:13:05.586 *******
FAILED - RETRYING: GCS | ETCD Cluster | Get etcd-client service (5 retries left).
changed: [kube1]
TASK [GCS | ETCD Cluster | Set etcd_client_endpoint] ***************************
Tuesday 23 October 2018  23:59:25 +0530 (0:00:06.395)       0:13:11.981 *******
ok: [kube1]
TASK [GCS | ETCD Cluster | Wait for etcd-cluster to become ready] **************
Tuesday 23 October 2018  23:59:25 +0530 (0:00:00.204)       0:13:12.185 *******
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (50 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (49 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (48 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (47 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (46 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (45 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (44 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (43 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (42 retries left).
ok: [kube1]
TASK [GCS | GD2 Cluster | Deploy GD2 services] *********************************
Wednesday 24 October 2018  00:01:04 +0530 (0:01:39.729)       0:14:51.915 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Deploy GD2] ******************************************
Wednesday 24 October 2018  00:01:06 +0530 (0:00:01.044)       0:14:52.959 *****
included: /root/gcs-8/rajesh-75/tasks/deploy-gd2.yml for kube1
included: /root/gcs-8/rajesh-75/tasks/deploy-gd2.yml for kube1
included: /root/gcs-8/rajesh-75/tasks/deploy-gd2.yml for kube1
TASK [GCS | GD2 Cluster | Set fact kube_hostname] ******************************
Wednesday 24 October 2018  00:01:06 +0530 (0:00:00.117)       0:14:53.076 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Deploy glusterd2 on kube1] ***************************
Wednesday 24 October 2018  00:01:06 +0530 (0:00:00.193)       0:14:53.269 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Set fact kube_hostname] ******************************
Wednesday 24 October 2018  00:01:07 +0530 (0:00:00.956)       0:14:54.226 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Deploy glusterd2 on kube2] ***************************
Wednesday 24 October 2018  00:01:07 +0530 (0:00:00.191)       0:14:54.417 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Set fact kube_hostname] ******************************
Wednesday 24 October 2018  00:01:08 +0530 (0:00:01.024)       0:14:55.442 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Deploy glusterd2 on kube3] ***************************
Wednesday 24 October 2018  00:01:08 +0530 (0:00:00.192)       0:14:55.635 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Get glusterd2-client service] ************************
Wednesday 24 October 2018  00:01:09 +0530 (0:00:01.139)       0:14:56.774 *****
changed: [kube1]
TASK [GCS | GD2 Cluster | Set gd2_client_endpoint] *****************************
Wednesday 24 October 2018  00:01:10 +0530 (0:00:00.794)       0:14:57.568 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready] **********
Wednesday 24 October 2018  00:01:10 +0530 (0:00:00.190)       0:14:57.758 *****
FAILED - RETRYING: GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready (50 retries left).
FAILED - RETRYING: GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready (49 retries left).
FAILED - RETRYING: GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready (48 retries left).
FAILED - RETRYING: GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready (47 retries left).
ok: [kube1]
TASK [GCS | GD2 Cluster | Get peers in cluster] ********************************
Wednesday 24 October 2018  00:01:58 +0530 (0:00:47.875)       0:15:45.634 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Add devices] *****************************************
Wednesday 24 October 2018  00:01:59 +0530 (0:00:01.085)       0:15:46.720 *****
included: /root/gcs-8/rajesh-75/tasks/add-devices-to-peer.yml for kube1
included: /root/gcs-8/rajesh-75/tasks/add-devices-to-peer.yml for kube1
included: /root/gcs-8/rajesh-75/tasks/add-devices-to-peer.yml for kube1
TASK [GCS | GD2 Cluster | Add devices | Set facts] *****************************
Wednesday 24 October 2018  00:01:59 +0530 (0:00:00.124)       0:15:46.844 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Add devices | Add devices for kube1] *****************
Wednesday 24 October 2018  00:02:00 +0530 (0:00:00.221)       0:15:47.065 *****
failed: [kube1] (item=/dev/vdb) => {"changed": false, "content": "", "disk": "/dev/vdb", "msg": "Status code was -1 and not [200]: Request failed: <urlopen error [Errno 111] Connection refused>", "redirected": false, "status": -1, "url": "http://10.233.4.195:24007/v1/devices/8169309d-b845-4478-abd4-2c7232dcd5f6"}
failed: [kube1] (item=/dev/vdc) => {"changed": false, "content": "", "disk": "/dev/vdc", "msg": "Status code was -1 and not [200]: Request failed: <urlopen error [Errno 111] Connection refused>", "redirected": false, "status": -1, "url": "http://10.233.4.195:24007/v1/devices/8169309d-b845-4478-abd4-2c7232dcd5f6"}
ok: [kube1] => (item=/dev/vdd)
to retry, use: --limit @/root/gcs-8/rajesh-75/vagrant-playbook.retry
PLAY RECAP *********************************************************************
kube1                      : ok=379  changed=110  unreachable=0    failed=1
kube2                      : ok=288  changed=88   unreachable=0    failed=0
kube3                      : ok=249  changed=68   unreachable=0    failed=0
Wednesday 24 October 2018  00:02:03 +0530 (0:00:02.930)       0:15:49.996 *****
GCS | ETCD Cluster | Wait for etcd-cluster to become ready ------------- 99.73s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 48.83s
GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready --------- 47.88s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 45.42s
download : file_download | Download item ------------------------------- 40.13s
kubernetes/master : Master | wait for the apiserver to be running ------ 33.09s
GCS | ETCD Operator | Wait for etcd-operator to be available ----------- 32.68s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 24.44s
etcd : Gen_certs | Write etcd master certs ----------------------------- 21.33s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 18.99s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 17.05s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 11.07s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 10.88s
docker : Docker | pause while Docker restarts -------------------------- 10.24s
etcd : reload etcd ------------------------------------------------------ 8.88s
etcd : Gen_certs | Gather etcd master certs ----------------------------- 8.43s
kubernetes-apps/external_provisioner/local_volume_provisioner : Local Volume Provisioner | Apply manifests --- 8.12s
kubernetes-apps/ansible : Kubernetes Apps | Lay Down CoreDNS Template --- 7.52s
download : container_download | Download containers if pull is required or told to always pull (all nodes) --- 7.48s
kubernetes-apps/network_plugin/flannel : Flannel | Wait for flannel subnet.env file presence --- 7.05s
==> kube3: An error occurred. The error will be shown after all tasks complete.
An error occurred while executing multiple actions in parallel.
Any errors that occurred are shown below.
An error occurred while executing the action on the 'kube3'
machine. Please handle this error then try again:
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

Most of the times vagrant up failing with latest build, if we try for 10 times, out of 10 times gcs deployment succeeded  for one or two times.
console logs:
TASK [kubernetes-apps/ansible : Kubernetes Apps | Delete kubeadm KubeDNS] ******
Thursday 08 November 2018  14:57:40 +0530 (0:00:01.408)       0:13:20.379 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down CoreDNS Template] ***
Thursday 08 November 2018  14:57:40 +0530 (0:00:00.149)       0:13:20.529 *****
changed: [kube1] => (item={u'type': u'configmap', u'name': u'coredns', u'file': u'coredns-config.yml'})
changed: [kube1] => (item={u'type': u'sa', u'name': u'coredns', u'file': u'coredns-sa.yml'})
changed: [kube1] => (item={u'type': u'deployment', u'name': u'coredns', u'file': u'coredns-deployment.yml'})
changed: [kube1] => (item={u'type': u'svc', u'name': u'coredns', u'file': u'coredns-svc.yml'})
changed: [kube1] => (item={u'type': u'clusterrole', u'name': u'coredns', u'file': u'coredns-clusterrole.yml'})
changed: [kube1] => (item={u'type': u'clusterrolebinding', u'name': u'coredns', u'file': u'coredns-clusterrolebinding.yml'})
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down Secondary CoreDNS Template] ***
Thursday 08 November 2018  14:57:46 +0530 (0:00:06.404)       0:13:26.934 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down KubeDNS Template] ***
Thursday 08 November 2018  14:57:46 +0530 (0:00:00.131)       0:13:27.065 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Patch system:kube-dns ClusterRole] ***
Thursday 08 November 2018  14:57:46 +0530 (0:00:00.208)       0:13:27.274 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Start Resources] *************
Thursday 08 November 2018  14:57:47 +0530 (0:00:00.118)       0:13:27.393 *****
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'4f11cac252dda1de9730138b24727619', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-config.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 459, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669260.45-164355608419030/source', u'group': u'root', '_ansible_item_label': {u'type': u'configmap', u'name': u'coredns', u'file': u'coredns-config.yml'}, 'item': {u'type': u'configmap', u'name': u'coredns', u'file': u'coredns-config.yml'}, u'checksum': u'7768ca5c0ca93fd5df09b0082224a21e64a442c4', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-config.yml', u'selevel': None, u'_original_basename': u'coredns-config.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669260.45-164355608419030/source', u'group': None, u'unsafe_writes': None, u'checksum': u'7768ca5c0ca93fd5df09b0082224a21e64a442c4', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'bfec6ec350bf47d850c62497fed4a879', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-sa.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 190, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669261.59-100154422686682/source', u'group': u'root', '_ansible_item_label': {u'type': u'sa', u'name': u'coredns', u'file': u'coredns-sa.yml'}, 'item': {u'type': u'sa', u'name': u'coredns', u'file': u'coredns-sa.yml'}, u'checksum': u'120a7b2f1f1e14e639ff7b5daa4ed21a980d80f7', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-sa.yml', u'selevel': None, u'_original_basename': u'coredns-sa.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669261.59-100154422686682/source', u'group': None, u'unsafe_writes': None, u'checksum': u'120a7b2f1f1e14e639ff7b5daa4ed21a980d80f7', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'7d464669b6fc51fac4ea9fa69401540b', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-deployment.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 2727, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669262.53-140567871485103/source', u'group': u'root', '_ansible_item_label': {u'type': u'deployment', u'name': u'coredns', u'file': u'coredns-deployment.yml'}, 'item': {u'type': u'deployment', u'name': u'coredns', u'file': u'coredns-deployment.yml'}, u'checksum': u'883ff5b38ca1c86279e01717924be70385439e5e', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-deployment.yml', u'selevel': None, u'_original_basename': u'coredns-deployment.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669262.53-140567871485103/source', u'group': None, u'unsafe_writes': None, u'checksum': u'883ff5b38ca1c86279e01717924be70385439e5e', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'c304c8275b2325fac1097c6456db3db2', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-svc.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 585, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669263.58-231860519155254/source', u'group': u'root', '_ansible_item_label': {u'type': u'svc', u'name': u'coredns', u'file': u'coredns-svc.yml'}, 'item': {u'type': u'svc', u'name': u'coredns', u'file': u'coredns-svc.yml'}, u'checksum': u'89c95955f6fea7fc14283bee6594911f171d6012', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-svc.yml', u'selevel': None, u'_original_basename': u'coredns-svc.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669263.58-231860519155254/source', u'group': None, u'unsafe_writes': None, u'checksum': u'89c95955f6fea7fc14283bee6594911f171d6012', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'40cdcba30a60aa92bdb8acba416e4b95', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-clusterrole.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 324, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669264.63-79959839198539/source', u'group': u'root', '_ansible_item_label': {u'type': u'clusterrole', u'name': u'coredns', u'file': u'coredns-clusterrole.yml'}, 'item': {u'type': u'clusterrole', u'name': u'coredns', u'file': u'coredns-clusterrole.yml'}, u'checksum': u'd62b1b25b430df8254746340e5032b98df625491', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-clusterrole.yml', u'selevel': None, u'_original_basename': u'coredns-clusterrole.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669264.63-79959839198539/source', u'group': None, u'unsafe_writes': None, u'checksum': u'd62b1b25b430df8254746340e5032b98df625491', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'8547087bed620deaa317d38926f92ba0', u'uid': 0, u'dest': u'/etc/kubernetes/coredns-clusterrolebinding.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 450, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669265.58-211155287060143/source', u'group': u'root', '_ansible_item_label': {u'type': u'clusterrolebinding', u'name': u'coredns', u'file': u'coredns-clusterrolebinding.yml'}, 'item': {u'type': u'clusterrolebinding', u'name': u'coredns', u'file': u'coredns-clusterrolebinding.yml'}, u'checksum': u'680a19b34a6814eeff494c4b89c050803a9c0c53', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/coredns-clusterrolebinding.yml', u'selevel': None, u'_original_basename': u'coredns-clusterrolebinding.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669265.58-211155287060143/source', u'group': None, u'unsafe_writes': None, u'checksum': u'680a19b34a6814eeff494c4b89c050803a9c0c53', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
TASK [kubernetes-apps/ansible : Kubernetes Apps | Check if netchecker-server manifest already exists] ***
Thursday 08 November 2018  14:57:52 +0530 (0:00:05.375)       0:13:32.769 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Apply netchecker-server manifest to update annotations] ***
Thursday 08 November 2018  14:57:52 +0530 (0:00:00.120)       0:13:32.889 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Netchecker Templates list] ***
Thursday 08 November 2018  14:57:52 +0530 (0:00:00.124)       0:13:33.013 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Append extra templates to Netchecker Templates list for PodSecurityPolicy] ***
Thursday 08 November 2018  14:57:52 +0530 (0:00:00.129)       0:13:33.143 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay Down Netchecker Template] ***
Thursday 08 November 2018  14:57:52 +0530 (0:00:00.123)       0:13:33.267 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Purge old Netchecker server] ***
Thursday 08 November 2018  14:57:53 +0530 (0:00:00.112)       0:13:33.379 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Start Netchecker Resources] ***
Thursday 08 November 2018  14:57:53 +0530 (0:00:00.120)       0:13:33.500 *****
TASK [kubernetes-apps/ansible : Kubernetes Apps | Delete old kubernetes-dashboard resources] ***
Thursday 08 November 2018  14:57:53 +0530 (0:00:00.126)       0:13:33.627 *****
ok: [kube1] => (item=ClusterRoleBinding)
TASK [kubernetes-apps/ansible : Kubernetes Apps | Lay down dashboard template] ***
Thursday 08 November 2018  14:57:54 +0530 (0:00:00.816)       0:13:34.443 *****
changed: [kube1] => (item={u'type': u'deploy', u'name': u'kubernetes-dashboard', u'file': u'dashboard.yml'})
TASK [kubernetes-apps/ansible : Kubernetes Apps | Start dashboard] *************
Thursday 08 November 2018  14:57:55 +0530 (0:00:01.242)       0:13:35.686 *****
ok: [kube1] => (item={'_ansible_parsed': True, u'md5sum': u'e8b24dcba4936f16f3e0c0a8e0288706', u'uid': 0, u'dest': u'/etc/kubernetes/dashboard.yml', '_ansible_item_result': True, '_ansible_no_log': False, u'owner': u'root', 'diff': [], u'size': 6001, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669274.34-172019028884446/source', u'group': u'root', '_ansible_item_label': {u'type': u'deploy', u'name': u'kubernetes-dashboard', u'file': u'dashboard.yml'}, 'item': {u'type': u'deploy', u'name': u'kubernetes-dashboard', u'file': u'dashboard.yml'}, u'checksum': u'd3af91fa4447cbfeb57d03f0304fc127eebaca3d', u'changed': True, 'failed': False, u'state': u'file', u'gid': 0, u'secontext': u'system_u:object_r:etc_t:s0', u'mode': u'0644', u'invocation': {u'module_args': {u'directory_mode': None, u'force': True, u'remote_src': None, u'dest': u'/etc/kubernetes/dashboard.yml', u'selevel': None, u'_original_basename': u'dashboard.yml.j2', u'delimiter': None, u'regexp': None, u'owner': None, u'follow': False, u'validate': None, u'local_follow': None, u'src': u'/home/vagrant/.ansible/tmp/ansible-tmp-1541669274.34-172019028884446/source', u'group': None, u'unsafe_writes': None, u'checksum': u'd3af91fa4447cbfeb57d03f0304fc127eebaca3d', u'seuser': None, u'serole': None, u'content': None, u'setype': None, u'mode': None, u'attributes': None, u'backup': False}}, '_ansible_ignore_errors': None})
TASK [kubernetes-apps/helm : Helm | Make sure HELM_HOME directory exists] ******
Thursday 08 November 2018  14:57:56 +0530 (0:00:01.505)       0:13:37.191 *****
TASK [kubernetes-apps/helm : Helm | Set up helm launcher] **********************
Thursday 08 November 2018  14:57:57 +0530 (0:00:00.116)       0:13:37.308 *****
TASK [kubernetes-apps/helm : Helm | Lay Down Helm Manifests (RBAC)] ************
Thursday 08 November 2018  14:57:57 +0530 (0:00:00.110)       0:13:37.419 *****
TASK [kubernetes-apps/helm : Helm | Apply Helm Manifests (RBAC)] ***************
Thursday 08 November 2018  14:57:57 +0530 (0:00:00.158)       0:13:37.578 *****
TASK [kubernetes-apps/helm : Helm | Set up TLS] ********************************
Thursday 08 November 2018  14:57:57 +0530 (0:00:00.142)       0:13:37.720 *****
TASK [kubernetes-apps/helm : Helm | Install/upgrade helm] **********************
Thursday 08 November 2018  14:57:57 +0530 (0:00:00.108)       0:13:37.828 *****
TASK [kubernetes-apps/helm : Helm | Force apply tiller overrides if necessary] ***
Thursday 08 November 2018  14:57:57 +0530 (0:00:00.117)       0:13:37.946 *****
TASK [kubernetes-apps/helm : Helm | Set up bash completion] ********************
Thursday 08 November 2018  14:57:57 +0530 (0:00:00.121)       0:13:38.067 *****
TASK [kubernetes-apps/registry : Registry | Create addon dir] ******************
Thursday 08 November 2018  14:57:57 +0530 (0:00:00.115)       0:13:38.182 *****
TASK [kubernetes-apps/registry : Registry | Templates list] ********************
Thursday 08 November 2018  14:57:58 +0530 (0:00:00.132)       0:13:38.315 *****
TASK [kubernetes-apps/registry : Registry | Append extra templates to Registry Templates list for PodSecurityPolicy] ***
Thursday 08 November 2018  14:57:58 +0530 (0:00:00.121)       0:13:38.436 *****
TASK [kubernetes-apps/registry : Registry | Create manifests] ******************
Thursday 08 November 2018  14:57:58 +0530 (0:00:00.117)       0:13:38.554 *****
TASK [kubernetes-apps/registry : Registry | Apply manifests] *******************
Thursday 08 November 2018  14:57:58 +0530 (0:00:00.117)       0:13:38.671 *****
TASK [kubernetes-apps/registry : Registry | Create PVC manifests] **************
Thursday 08 November 2018  14:57:58 +0530 (0:00:00.115)       0:13:38.787 *****
TASK [kubernetes-apps/registry : Registry | Apply PVC manifests] ***************
Thursday 08 November 2018  14:57:58 +0530 (0:00:00.121)       0:13:38.909 *****
TASK [kubernetes-apps/persistent_volumes/openstack : Kubernetes Persistent Volumes | Lay down OpenStack Cinder Storage Class template] ***
Thursday 08 November 2018  14:57:58 +0530 (0:00:00.152)       0:13:39.061 *****
TASK [kubernetes-apps/persistent_volumes/openstack : Kubernetes Persistent Volumes | Add OpenStack Cinder Storage Class] ***
Thursday 08 November 2018  14:57:58 +0530 (0:00:00.135)       0:13:39.197 *****
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU| gather os specific variables] ***
Thursday 08 November 2018  14:57:59 +0530 (0:00:00.111)       0:13:39.308 *****
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Set fact of download url Tesla] ***
Thursday 08 November 2018  14:57:59 +0530 (0:00:00.138)       0:13:39.447 *****
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Set fact of download url GTX] ***
Thursday 08 November 2018  14:57:59 +0530 (0:00:00.109)       0:13:39.557 *****
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Create addon dir] ***
Thursday 08 November 2018  14:57:59 +0530 (0:00:00.121)       0:13:39.678 *****
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Create manifests for nvidia accelerators] ***
Thursday 08 November 2018  14:57:59 +0530 (0:00:00.110)       0:13:39.788 *****
TASK [kubernetes-apps/container_engine_accelerator/nvidia_gpu : Container Engine Acceleration Nvidia GPU | Apply manifests for nvidia accelerators] ***
Thursday 08 November 2018  14:57:59 +0530 (0:00:00.133)       0:13:39.922 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_private_key] ***
Thursday 08 November 2018  14:57:59 +0530 (0:00:00.146)       0:13:40.069 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_region_id] ***
Thursday 08 November 2018  14:57:59 +0530 (0:00:00.117)       0:13:40.187 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_tenancy_id] ***
Thursday 08 November 2018  14:58:00 +0530 (0:00:00.130)       0:13:40.318 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_user_id] ***
Thursday 08 November 2018  14:58:00 +0530 (0:00:00.112)       0:13:40.431 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_user_fingerprint] ***
Thursday 08 November 2018  14:58:00 +0530 (0:00:00.119)       0:13:40.550 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_compartment_id] ***
Thursday 08 November 2018  14:58:00 +0530 (0:00:00.121)       0:13:40.672 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_vnc_id] ***
Thursday 08 November 2018  14:58:00 +0530 (0:00:00.111)       0:13:40.783 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_subnet1_id] ***
Thursday 08 November 2018  14:58:00 +0530 (0:00:00.114)       0:13:40.898 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_subnet2_id] ***
Thursday 08 November 2018  14:58:00 +0530 (0:00:00.105)       0:13:41.003 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Credentials Check | oci_security_list_management] ***
Thursday 08 November 2018  14:58:00 +0530 (0:00:00.120)       0:13:41.124 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Generate Configuration] ***
Thursday 08 November 2018  14:58:00 +0530 (0:00:00.105)       0:13:41.229 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Encode Configuration] ***
Thursday 08 November 2018  14:58:01 +0530 (0:00:00.120)       0:13:41.350 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Apply Configuration To Secret] ***
Thursday 08 November 2018  14:58:01 +0530 (0:00:00.126)       0:13:41.476 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Apply Configuration] ***
Thursday 08 November 2018  14:58:01 +0530 (0:00:00.115)       0:13:41.592 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Download Controller Manifest] ***
Thursday 08 November 2018  14:58:01 +0530 (0:00:00.111)       0:13:41.704 *****
TASK [kubernetes-apps/cloud_controller/oci : OCI Cloud Controller | Apply Controller Manifest] ***
Thursday 08 November 2018  14:58:01 +0530 (0:00:00.125)       0:13:41.829 *****
PLAY [Deploy GCS] **************************************************************
TASK [GCS Pre | Cluster ID | Generate a UUID] **********************************
Thursday 08 November 2018  14:58:01 +0530 (0:00:00.130)       0:13:41.960 *****
changed: [kube1]
TASK [GCS Pre | Cluster ID | Set gcs_gd2_clusterid fact] ***********************
Thursday 08 November 2018  14:58:02 +0530 (0:00:00.595)       0:13:42.555 *****
ok: [kube1]
TASK [GCS Pre | Manifests directory | Create a temporary directory] ************
Thursday 08 November 2018  14:58:02 +0530 (0:00:00.200)       0:13:42.756 *****
changed: [kube1]
TASK [GCS Pre | Manifests directory | Set manifests_dir fact] ******************
Thursday 08 November 2018  14:58:03 +0530 (0:00:00.866)       0:13:43.622 *****
ok: [kube1]
TASK [GCS Pre | Manifests | Sync GCS manifests] ********************************
Thursday 08 November 2018  14:58:03 +0530 (0:00:00.181)       0:13:43.804 *****
changed: [kube1] => (item=gcs-namespace.yml)
changed: [kube1] => (item=gcs-etcd-operator.yml)
changed: [kube1] => (item=gcs-etcd-cluster.yml)
changed: [kube1] => (item=gcs-gd2-services.yml)
changed: [kube1] => (item=gcs-csi.yml)
TASK [GCS Pre | Manifests | Create GD2 manifests] ******************************
Thursday 08 November 2018  14:58:08 +0530 (0:00:04.939)       0:13:48.744 *****
included: /root/gcs-latest2/rajesh-latest5/tasks/create-gd2-manifests.yml for kube1
included: /root/gcs-latest2/rajesh-latest5/tasks/create-gd2-manifests.yml for kube1
included: /root/gcs-latest2/rajesh-latest5/tasks/create-gd2-manifests.yml for kube1
TASK [GCS Pre | Manifests | Create GD2 manifests for kube1 | Set fact kube_hostname] ***
Thursday 08 November 2018  14:58:08 +0530 (0:00:00.129)       0:13:48.873 *****
ok: [kube1]
TASK [GCS Pre | Manifests | Create GD2 manifests for kube1 | Create gcs-gd2-kube1.yml] ***
Thursday 08 November 2018  14:58:08 +0530 (0:00:00.204)       0:13:49.078 *****
changed: [kube1]
TASK [GCS Pre | Manifests | Create GD2 manifests for kube2 | Set fact kube_hostname] ***
Thursday 08 November 2018  14:58:09 +0530 (0:00:01.072)       0:13:50.150 *****
ok: [kube1]
TASK [GCS Pre | Manifests | Create GD2 manifests for kube2 | Create gcs-gd2-kube2.yml] ***
Thursday 08 November 2018  14:58:10 +0530 (0:00:00.195)       0:13:50.346 *****
changed: [kube1]
TASK [GCS Pre | Manifests | Create GD2 manifests for kube3 | Set fact kube_hostname] ***
Thursday 08 November 2018  14:58:11 +0530 (0:00:01.052)       0:13:51.399 *****
ok: [kube1]
TASK [GCS Pre | Manifests | Create GD2 manifests for kube3 | Create gcs-gd2-kube3.yml] ***
Thursday 08 November 2018  14:58:11 +0530 (0:00:00.181)       0:13:51.581 *****
changed: [kube1]
TASK [GCS | Namespace | Create GCS namespace] **********************************
Thursday 08 November 2018  14:58:12 +0530 (0:00:01.100)       0:13:52.682 *****
ok: [kube1]
TASK [GCS | ETCD Operator | Deploy etcd-operator] ******************************
Thursday 08 November 2018  14:58:13 +0530 (0:00:01.053)       0:13:53.735 *****
ok: [kube1]
TASK [GCS | ETCD Operator | Wait for etcd-operator to be available] ************
Thursday 08 November 2018  14:58:14 +0530 (0:00:00.977)       0:13:54.713 *****
FAILED - RETRYING: GCS | ETCD Operator | Wait for etcd-operator to be available (50 retries left).
FAILED - RETRYING: GCS | ETCD Operator | Wait for etcd-operator to be available (49 retries left).
FAILED - RETRYING: GCS | ETCD Operator | Wait for etcd-operator to be available (48 retries left).
FAILED - RETRYING: GCS | ETCD Operator | Wait for etcd-operator to be available (47 retries left).
changed: [kube1]
TASK [GCS | ETCD Cluster | Deploy etcd-cluster] ********************************
Thursday 08 November 2018  14:58:57 +0530 (0:00:43.242)       0:14:37.955 *****
ok: [kube1]
TASK [GCS | ETCD Cluster | Get etcd-client service] ****************************
Thursday 08 November 2018  14:58:58 +0530 (0:00:01.140)       0:14:39.095 *****
changed: [kube1]
TASK [GCS | ETCD Cluster | Set etcd_client_endpoint] ***************************
Thursday 08 November 2018  14:58:59 +0530 (0:00:00.788)       0:14:39.883 *****
ok: [kube1]
TASK [GCS | ETCD Cluster | Wait for etcd-cluster to become ready] **************
Thursday 08 November 2018  14:58:59 +0530 (0:00:00.207)       0:14:40.091 *****
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (50 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (49 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (48 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (47 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (46 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (45 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (44 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (43 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (42 retries left).
FAILED - RETRYING: GCS | ETCD Cluster | Wait for etcd-cluster to become ready (41 retries left).
ok: [kube1]
TASK [GCS | GD2 Cluster | Deploy GD2 services] *********************************
Thursday 08 November 2018  15:00:51 +0530 (0:01:51.921)       0:16:32.012 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Deploy GD2] ******************************************
Thursday 08 November 2018  15:00:52 +0530 (0:00:01.191)       0:16:33.204 *****
included: /root/gcs-latest2/rajesh-latest5/tasks/deploy-gd2.yml for kube1
included: /root/gcs-latest2/rajesh-latest5/tasks/deploy-gd2.yml for kube1
included: /root/gcs-latest2/rajesh-latest5/tasks/deploy-gd2.yml for kube1
TASK [GCS | GD2 Cluster | Set fact kube_hostname] ******************************
Thursday 08 November 2018  15:00:53 +0530 (0:00:00.121)       0:16:33.326 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Deploy glusterd2 on kube1] ***************************
Thursday 08 November 2018  15:00:53 +0530 (0:00:00.193)       0:16:33.519 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Set fact kube_hostname] ******************************
Thursday 08 November 2018  15:00:54 +0530 (0:00:01.135)       0:16:34.655 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Deploy glusterd2 on kube2] ***************************
Thursday 08 November 2018  15:00:54 +0530 (0:00:00.198)       0:16:34.853 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Set fact kube_hostname] ******************************
Thursday 08 November 2018  15:00:55 +0530 (0:00:01.085)       0:16:35.939 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Deploy glusterd2 on kube3] ***************************
Thursday 08 November 2018  15:00:56 +0530 (0:00:00.366)       0:16:36.306 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Get glusterd2-client service] ************************
Thursday 08 November 2018  15:00:56 +0530 (0:00:00.932)       0:16:37.238 *****
changed: [kube1]
TASK [GCS | GD2 Cluster | Set gd2_client_endpoint] *****************************
Thursday 08 November 2018  15:00:57 +0530 (0:00:00.782)       0:16:38.021 *****
ok: [kube1]
TASK [GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready] **********
Thursday 08 November 2018  15:00:58 +0530 (0:00:00.311)       0:16:38.332 *****
FAILED - RETRYING: GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready (50 retries left).
FAILED - RETRYING: GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready (49 retries left).
FAILED - RETRYING: GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready (48 retries left).
ok: [kube1]
TASK [GCS | GD2 Cluster | Get peers in cluster] ********************************
Thursday 08 November 2018  15:01:34 +0530 (0:00:36.770)       0:17:15.103 *****
fatal: [kube1]: FAILED! => {"changed": false, "content": "", "msg": "Status code was -1 and not [200]: Request failed: <urlopen error [Errno 111] Connection refused>", "redirected": false, "status": -1, "url": "http://10.233.59.18:24007/v1/peers"}
to retry, use: --limit @/root/gcs-latest2/rajesh-latest5/vagrant-playbook.retry
PLAY RECAP *********************************************************************
kube1                      : ok=375  changed=111  unreachable=0    failed=1
kube2                      : ok=289  changed=89   unreachable=0    failed=0
kube3                      : ok=250  changed=69   unreachable=0    failed=0
Thursday 08 November 2018  15:01:35 +0530 (0:00:01.147)       0:17:16.251 *****
GCS | ETCD Cluster | Wait for etcd-cluster to become ready ------------ 111.92s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 66.38s
download : file_download | Download item ------------------------------- 66.14s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 46.78s
GCS | ETCD Operator | Wait for etcd-operator to be available ----------- 43.24s
GCS | GD2 Cluster | Wait for glusterd2-cluster to become ready --------- 36.77s
kubernetes/master : Master | wait for the apiserver to be running ------ 33.56s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 26.93s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 24.47s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 21.32s
etcd : Gen_certs | Write etcd master certs ----------------------------- 20.35s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 14.68s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 12.89s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 12.26s
download : container_download | Download containers if pull is required or told to always pull (all nodes) -- 11.15s
etcd : reload etcd ----------------------------------------------------- 10.92s
docker : Docker | pause while Docker restarts -------------------------- 10.26s
etcd : wait for etcd up ------------------------------------------------- 8.91s
etcd : Gen_certs | Gather etcd master certs ----------------------------- 8.59s
kubernetes/master : Master | wait for kube-controller-manager ----------- 7.84s
==> kube3: An error occurred. The error will be shown after all tasks complete.
An error occurred while executing multiple actions in parallel.
Any errors that occurred are shown below.
An error occurred while executing the action on the 'kube3'
machine. Please handle this error then try again:
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

@kshlm this could be the issue I discussed with you last time
currently we are not checking all ETCD pods in the cluster are in running state or not. so because of that gd2 is not responding. can you please check

@rmadaka Can you please give it a try with the latest GCS head?

Probably fixed through #65

@rmadaka please verify this issue

@rmadaka Have we come across this again with the latest bits?

@atinmu  With latest build, till now i didn't see any issue, i will try some more times and will close this issue.

First timer here (on gcs), failed with:
==> kube3: Running provisioner: ansible...
Vagrant has automatically selected the compatibility mode '2.0'
according to the Ansible version installed (2.7.2).

Alternatively, the compatibility mode can be specified in your Vagrantfile:
https://www.vagrantup.com/docs/provisioning/ansible_common.html#compatibility_mode

==> kube3: Vagrant has detected a host range pattern in the `groups` option.
==> kube3: Vagrant doesn't fully check the validity of these parameters!
==> kube3:
==> kube3: Please check https://docs.ansible.com/ansible/intro_inventory.html#hosts-and-groups
==> kube3: for more information.
    kube3: Running ansible-playbook...
ERROR! Unable to retrieve file contents
Could not find or access '/home/ykaul/github/gcs/deploy/kubespray/cluster.yml' on the Ansible Controller.
If you are using a module and expect the file to exist on the remote, see the remote_src option
==> kube3: An error occurred. The error will be shown after all tasks complete.
An error occurred while executing multiple actions in parallel.
Any errors that occurred are shown below.

An error occurred while executing the action on the 'kube3'
machine. Please handle this error then try again:

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

Fedora 29, vagrant version: 2.1.2, Ansible 2.7.2

@Madhu-1 @atinmu , Today , i have deployed 3 more times, didn't find any issues

then we can close this issue, and reopen if we face any issue

@Madhu-1 - all I ran was 'vagrant up'. Was I supposed to do anything else?

@mykaul  need to do couple of steps before running vagrant up readme link here

@Madhu-1 - OK, seems to be running on my laptop. Perhaps we can integrate this prepare.sh to the Vagrantfile . I'll take a look.
(This issue can be closed from my perspective).

source: gharchive/issue

authors: Madhu-1 atinmu mykaul rmadaka

repo: gluster/gcs


text:
動画情報取得

サムネイル
投稿日時

sm入力→スクレイピングで取得とかできたら良いけど負荷的にどうなんだろ…

repo: glyzinieh/nicovideo-rekari-list

license: MIT


text:
Add ignore files feature

I don't want to stylize build_runner generated files, because when they are generated again they are unnecessarily counted as modified files by git.
Would you consider adding a flag or config file to list files and globs to ignore?

Would you consider adding a flag or config file to list files and globs to ignore?

That sounds like a nice feature to add. Do you have a preference or a favorite glob style or favorite command you would like to emulate? I ask because there are an unlimited number of ways that this could be implemented, and if you are familiar with something it might be best to start there.

I would say that using the same syntax used in analysis_options.yaml would be good because flutter developers would be already accustomed to it.
This is what they use: https://pub.dev/packages/glob#syntax
Maybe creating something like flutter-stylezer.yaml would be even better, with something like this inside:
exclude:
  - "**/*.g.dart"
  - "**/*.gr.dart"
  - "**/*.freezed.dart"

@Maatteogekko - please try out the v0.1.4 release and see if this implements what you asked for:
https://github.com/gmlewis/go-flutter-stylizer/releases/tag/v0.1.4

Works like a charm! 💯
Thank you for the nice addition in so little time. Keep up the great work! 💪🏻

authors: Maatteogekko gmlewis

repo: gmlewis/go-flutter-stylizer


text:
🛑 Seasia is down

In 01ecf22, Seasia (https://seasia.co) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Seasia is back up in 33e0a7c.

repo: gnfi/status


text:
Shut down Polygon and BSC legacy instances of CGW

See https://github.com/gnosis/safe-client-gateway/issues/706
ETA
February 28th

Done

authors: jpalvarezl rmeissner

repo: gnosis/safe-client-gateway


text:
🛑 Beacon RPC is down

In 8d529b0, Beacon RPC (https://rpc-gbc.chiadochain.net/eth/v1/beacon/headers) was down:

HTTP code: 503
Response time: 178 ms

Resolved: Beacon RPC is back up in cafc495.

repo: gnosischain/chiado-uptime


text:
🛑 Blockscout Explorer is down

In ca1c4d6, Blockscout Explorer (https://blockscout.com/xdai/mainnet) was down:

HTTP code: 503
Response time: 20247 ms

Resolved: Blockscout Explorer is back up in adb35b4.

repo: gnosischain/gnosischain-uptime


text:
🛑 RPC is down

In 0be2f88, RPC (https://rpc.gnosischain.com/) was down:

HTTP code: 502
Response time: 5153 ms

Resolved: RPC is back up in 057aa04 after 5 hours, 42 minutes.


text:
Panic and recoverer middleware

Should I put this middleware before panic? There is not much about it.

See https://github.com/go-chi/httplog/blob/master/httplog.go#L47C27-L55

repo: go-chi/httplog


text:
Potential problem with session

Gitea version (or commit ref): 1.14.0
Git version: 2.29.2
Operating system: Debian 10.9

This gist shows detail how we setup our gitea server:
https://gist.github.com/ndaidong/2fad09f49db27d115f6d47eba7e91cb8

Database (use [x]):

[x] PostgreSQL
[ ] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[x] No

Log gist:

Activity log
Access log

Description
We've setup gitea for our company. After migrating from Bitbucket, here is what we have now:

There is often about 40 users online at any time.
Everything work well, thank you so much for your effort to build a great platform as Gitea.
However, sometimes we found that the session suddenly closes and we have to login again. It occurs with admin accounts and normal users too.
Could you give us any idea, or conjecture about this odd phenomenon?

Just a quick note about your config, you have fully copied the app.example.ini which is for documentation purposes and not meant to be copied in full (you'll note: https://gist.github.com/ndaidong/802c8157df3771ce097a42c3c9a01b70#file-app-ini-L441 log.x doesn't actually do anything), and so you should only set settings you are confident of and leave default settings up to Gitea.

@techknowlogick yes I just modify what I need and understand. Do you mean I should remove the default parts from that .ini?

Is this folder exist with correct file permission? /storage/runtime/sessions

@lunny yes, the whole /storage directory is owned by git:git.
For more info, I separate to several parts, in which:

/storage/gitea: persistent data to backup
/storage/runtime:  data generated in runtime, not saving to NAS

sysad@gitserver /storage $ tree -L 2
.
├── gitea
│   ├── repositories
│   ├── ssh
│   └── uploads
├── lfs
├── lost+found [error opening dir]
├── runtime
│   ├── indexers
│   ├── logs
│   └── sessions
└── tmp
    └── local-repo

default sessions is stored into files which should be persistent. Once the files deleted, the session will be invalid and users will be prompted to login.
Also default indexers will use bleve which will store as files. Once the files deleted, and Gitea will regenerated them when restart.

@lunny certainly, these files are persistent. And I can ensure that nothing affect to them while the system is working. Any suggestion to check?

Have you tried to upgrade to v1.14.1?

@lunny not yet. But I have plan to try. Because at least it resolves the issue superfluous response.WriteHeader .

@techknowlogick yes I just modify what I need and understand. Do you mean I should remove the default parts from that .ini?

the file you are basing your current app.ini on is documentation and provides examples, but should NOT be use as a base with just a few things updated. You should start with an empty app.ini and only add the things you need to it.

@techknowlogick I'm upgrading to v1.14.1 and refactoring  the app.ini file. Hopefully that it becomes more stable. I will update the situation after few hours of monitoring.

@lunny @techknowlogick hello, I've upgraded successfully to v1.14.1 and refactored app.ini file. Now the config file is only 4.5 kb.
It seems the session is more stable than early but the error "superfluous response.WriteHeader call" is still happening, and there is another issue.

@lunny Please look into this issue carefully.  I believe that we are facing a serious problem here.
Some people in my company and I found that sometimes we get into other session. Let's says a user A login Gitea last night, and today morning, when A comes back, he found that the account he is using is B. In almost cases, he is being redirected to login page, but in the worst cases, he can even use B's account to navigate arround and change personal settings.
The sessions have been stored into hard drive, as below:
[session]
;PROVIDER          = file
;PROVIDER_CONFIG   = /storage/runtime/sessions
COOKIE_NAME       = i_dont_like_cookie
COOKIE_SECURE     = true
SAME_SITE         = lax

Now we've changed to redis:
[session]
PROVIDER          = redis
PROVIDER_CONFIG   = network=tcp,addr=127.0.0.1:6379,db=0,pool_size=100,idle_timeout=180
COOKIE_NAME       = i_dont_like_cookie
COOKIE_SECURE     = true
SAME_SITE         = lax

Have you use a reverse proxy for your Gitea instance?

@lunny yes, we use nginx. And let's ensrypt for SSL certificate.

@lunny @techknowlogick hello, this problem is still happening with us. Today at least 2 users reported: they have seen that they have been using the account of another user (same account, let's say X).

case 1: after weekend, A opens the homepage and clicks on the signin buton --> redirect to X dashboard without login
case 2: B clicks on a link to an issue shared by C via Skype --> open that page under X account

IMO, the cause may not relate to the session, but SSO mechanish. I'm investigating the modules at /auth/sso, please share to me the diagrams here if any.

Which SSO system do you use?

@techknowlogick no, I mean gitea's SSO (modules/auth/sso),  I don't use any SSO system outside.

We support reverse proxy sso, see https://github.com/go-gitea/gitea/blob/master/modules/auth/sso/reverseproxy.go

@lunny I notice that today this problem occurs again for many accounts in our system.
(Last week I've switched to postgres db, and it worked as well until now.)
At 05/May/2021:08:53:53,  administor suddenly sees himself as phutv:
10.1.0.25:47790 - administor [05/May/2021:08:53:53 +0700] "GET /phutv HTTP/1.1" 200 25272 "\" \"Mozilla/5.0 (X11; Linux x86_64; rv:88.0) Gecko/20100101 Firefox/88.0"

At this time, system logs produce many superfluous errors:
2021/05/05 08:53:18 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:22 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:22 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:22 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:22 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:22 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:53 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:53 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:53 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:53 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:53 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:57 ...les/public/public.go:165:handle() [I] [Static] Serving /js/clipboard.js
2021/05/05 08:53:57 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:57 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:57 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:57 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/05 08:53:57 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)

I'm not sure that Gitea generates the log with the user requesting: 10.1.0.25:47790 - administor [05/May/2021:08:53:53 +0700] "GET /phutv HTTP/1.1" 200 25272 "\" \"Mozilla/5.0 (X11; Linux x86_64; rv:88.0) Gecko/20100101 Firefox/88.0", are you sure you are not using a reverse proxy, or SSO of any kind? Where did you get that log from? That log also shows the user just accessing the profile page for that user, not that the user has switched or anything. You can see in the top right of the page the user that is requesting the page.

@techknowlogick we are using nginx as reverse proxy, no SSO of any kind. The architecture is quite simple:

@techknowlogick @lunny another problem may relate to this issue: non-member user can access organization/repo!
One of organization owners reported to me today. He just added 6 people to the organization eoffice-qn, but he has been seeing non-members push code to the repos there. As I've checked, this non-member can access the repos via SSH/HTTP and web GUI too.
This is people in the organization:

And this is the view by non-member user:

I'm trying to replicate this locally, however using the config you've provided I can't. Are you able to start a fresh install, and be able to provide repeatable step by step instructions on how to trigger this?

@techknowlogick I could not reproduce the issue. I have already a fresh install for mirror system. The problem does not happen here, with 3 or 4 people in the manager team.
In the past I've used gitea for several small projects, and never seen the issues as today when I deploy it to a company with more than 100 users. I can provide you a regular account  for test if needed, sometimes you may see you as system admin.
Anyway to downgrate to v1.3? I would like to check if it relates to the replacement of go-chi/session.

Are these people using the same browser?
It would be useful to check their cookies and in particular the path for the cookies.
One of the things we did in 1.14 was actually formalize and rationalize our cookie paths. So if you have old cookies you may find that they're not logging out properly - delete them and they'll never come back and it will be fine. (Or even just change the name of the cookies that gitea is using.)
Could you check your config paths for cookies on the server too?

@zeripath These people each one work on their own computer, use their own account. The first version I've setup is v1.14, then upgraded to v1.14.1 to fix this issue but it's still happening.
Here is the cookie on my browser.

@techknowlogick - they've set up the access logger - so it's possible that's that's coming from there:
[log]
MODE                = console, file
LEVEL               = info
ROOT_PATH           = /storage/runtime/logs
ENABLE_ACCESS_LOG   = true
ACCESS_LOG_TEMPLATE = {{.Ctx.RemoteAddr}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}\" \"{{.Ctx.Req.UserAgent}}"

[log.file]
LEVEL               = info
FILE_NAME           = /storage/runtime/logs/activity.log

@ndaidong  Sorry that this is taking so long to sort out - this is a complex problem and we still don't have enough information to work out what is happening let alone how to solve it. I think we need to be more systematic instead of just guessing at things now.
So first I'm going to review the app.ini:

I'm a little concerned by:

[server]
DOMAIN           = localhost

HTTP_ADDR        = 0.0.0.0
HTTP_PORT        = 8081
ROOT_URL         = http://localhost:8081/

DISABLE_SSH      = false

I don't think that this can be true at all. What is the value of PROTOCOL and please tell me that your ROOT_URL is not http://localhost:8081/?
I also see that you're explicitly setting the cookie names - which is definitely in keeping with your use of app.example.ini in the past which may suggest that you've had the paths etc for these wrong.
I see you have the [cache] set-up which could be a difference with other users - but I'm not certain how that could cause a problem - it's just something to bear in mind.
What I'm not seeing are things like REVERSE_PROXY headers which is a good thing. I also note that you don't have BASIC authentication disabled - (don't change this yet.)

But there's no gigantic smoking gun there (Apart from that localhost in ROOT_URL and lack of PROTOCOL)

Now let's think about how to solve or investigate this further.
To me it looks like we need some more logging. We're not aware of how people are getting their IDs changed and why.
This is comment 30 on this issue and we still haven't seen any DEBUG logs - which is kinda ridiculous tbh - however looking at 1.14 and main I'm not sure there's much logging that would be gained by this - meaning that we will need to add some patches to add some logging. (I had PRs that were significantly working on rationalising this stuff but they've totally stagnated due to other pressures and lack of reviews... #15304 #15303 fyi - there is a 3rd PR that's going to refactor by moving modules/auth/sso to services/auth and then add all of this logging and but with the lack of reviews for the previous two - progress is totally stalled.)
So are you able to compile Gitea with a few patches to attempt to add at least some TRACE logging to modules/auth/sso and elsewhere? Once those patches are in it would be possible to add a very restricted trace level logger to gitea using a simple additional config:
Additional Config
[log]
MODE                = console, file, tracesso
LEVEL               = info
ROOT_PATH           = /storage/runtime/logs
ENABLE_ACCESS_LOG   = true
ACCESS_LOG_TEMPLATE = {{.Ctx.RemoteAddr}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}\" \"{{.Ctx.Req.UserAgent}}"

[log.file]
LEVEL               = info
FILE_NAME           = /storage/runtime/logs/activity.log

[log.tracesso]
LEVEL=trace
EXPRESSION=modules/auth/sso; This may need to be adjusted depending on how these patches turn out - this is just an example.
MODE=file
FILE_NAME           = /storage/runtime/logs/tracesso.log

Another option - just to rule out any weirdness with cookies is simply to change all of the names of the cookies - in case there are some weird issues there.
One other thing - are people using BASIC authentication to log in to Gitea using the swagger API on these machines or something else? Assuming that you're only pushing with SSH you could actually try disabling basic authentication. However, I think that would be seriously detrimental to most people's workflows.
I know that we're only creating session IDs within 16 byte character space which isn't huge - Are you having lots of sessions created and destroyed? Maybe you're genuinely hitting collisions with sessions?!

All in all I think we need more logging as we're totally in the dark. Unfortunately due to the lack of trace logging in these functions  we're gonna need to have some patches compiled in.
If you would like me to write these and/or compile things I would be happy to discuss them further - drop me an email or message on discord. I'll pop up a PR to Gitea to actually add some trace logging to main/v1.15.x however, I need other maintainers to review and approve. Backporting from that PR to 1.14 would then be easier. (And paradoxically easier because of the stalled progress on  #15304 & #15303 and their subsequent PRs.)

@zeripath thank you so much. I understand that the infomation is quite vague. That's not easy to guess the root cause.

For the config, I'm sorry for the missed things in the app.ini; it has been modified a few times. I've just updated it with the latest content from our server.

https://gist.github.com/ndaidong/802c8157df3771ce097a42c3c9a01b70
The ROOT_URL  is actually set to https://source.greenglobal.vn.

Regarding the cookie, I've used custom name for all of them.
In my company almost people use basic authentication, so I have to enable both HTTP/SSH. However, the swagger has been disabled since first installation.
Totally, there are only about 150 - 180 sessions.

I will try to re-compile using the pr #15304 & #15303 to gather more meaningful logs first. If that doesn't help, I will ask you on discord for more steps.
Once again, thank you for your advice.

#15304 & #15303 won't necessarily help this. I still need to pop up the logging PR and will refer to this issue once it's up.
#15303 would rule out Basic Authentication as the cause for weird shifts so is not completely useless.

#15803 is the trace logging PR

@zeripath hello, after upgrading to v1.5.x, I see nothing new in the activity logs. Does it require updated config?
Current, it looks like below:
[log]
MODE                = console, file
LEVEL               = info
ROOT_PATH           = /storage/runtime/logs
ENABLE_ACCESS_LOG   = true
ACCESS_LOG_TEMPLATE = {{.Ctx.RemoteAddr}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}\" \"{{.Ctx.Req.UserAgent}}"

[log.file]
LEVEL               = info
FILE_NAME           = /storage/runtime/logs/activity.log

And the output:
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:27 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:31 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:31 ...les/public/public.go:164:handle() [I] [Static] Serving /vendor/plugins/codemirror/mode/meta.js
2021/05/12 14:48:31 ...les/public/public.go:164:handle() [I] [Static] Serving /js/easymde.js
2021/05/12 14:48:31 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:31 ...les/public/public.go:164:handle() [I] [Static] Serving /vendor/plugins/codemirror/addon/mode/loadmode.js
2021/05/12 14:48:31 ...les/public/public.go:164:handle() [I] [Static] Serving /css/easymde.css
2021/05/12 14:48:31 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:31 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:31 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)
2021/05/12 14:48:31 ...les/public/public.go:164:handle() [I] [Static] Serving /fonts/icons.woff2
2021/05/12 14:48:31 ...les/public/public.go:164:handle() [I] [Static] Serving /js/dropzone.js
2021/05/12 14:48:31 ...les/public/public.go:164:handle() [I] [Static] Serving /css/dropzone.css
2021/05/12 14:48:31 ...les/public/public.go:164:handle() [I] [Static] Serving /js/tribute.js
2021/05/12 14:48:31 ...les/public/public.go:164:handle() [I] [Static] Serving /js/clipboard.js

Hi!
As I wrote above you'd need to add some additional config to your app.ini to add a trace logger to modules/auth/sso.
Change your app.ini log sections to
[log]
MODE                = console, file, tracesso
LEVEL               = info
ROOT_PATH           = /storage/runtime/logs
ENABLE_ACCESS_LOG   = true

[log.file]
LEVEL               = info
FILE_NAME           = /storage/runtime/logs/activity.log

[log.tracesso]
LEVEL=trace
EXPRESSION=modules/auth/sso
MODE=file
FILE_NAME           = /storage/runtime/logs/tracesso.log

It looks like your ACCESS_LOG_TEMPLATE is just the default so you don't need to set that.

What version exactly did you upgrade to?  These log lines:
2021/05/12 14:48:31 ...c/net/http/server.go:3139:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)

Shouldn't be present in main - so I'm confused as to how you're getting them.

@zeripath thank you, I've updated config.

authors: lunny ndaidong techknowlogick zeripath

repo: go-gitea/gitea