text:
WIP: Validate profile by building the seccomp filter

What type of PR is this?
/kind feature
What this PR does / why we need it:
The new containers/common ValidateProfile() API can be used to build
a libseccomp filter without loading it into the kernel. This can be now
used by the operator to have a more in-depth profile validation. This
now requires the seccomp build tag and a (statically) compiled version
of libseccomp.
Which issue(s) this PR fixes:

Fixes #119
Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

- Changed profile validation to build the whole filter without loading it into the kernel

Now it gets interesting:
2s          Warning   InvalidSeccompProfile     configmap/test-profile   build seccomp filter: create filter for default action Action: Log system call: could not create filter

Locally reproducible, but if I write a unit tests to init the seccomp filter with ACT_LOG, then it works…scary.

It's a permission problem, sure.

Looks like that this is the first time that the seccomp profile of the operator blocks a necessary syscall for the LOG action. @pjbgf do you have any hint which syscall we have to add here?

It was the seccomp(2) syscall which had to be added.

It was the seccomp(2) syscall which had to be added. Ready for review.

If this change requires the seccomp syscall, that means we are not only "building" the profile, but we are also applying it. Going through upstream code it does look like that:
https://github.com/containers/common/blob/e8021ed05dc2fb314b2e0dd26699de1d01d7cb19/pkg/seccomp/filter.go#L82
On that note, as part of calling upstream we are also setting the filter's NoNewPrivs bit to false. I am not sure we would want to do that as a side effect of "building" the profile. What are your thoughts around this?
https://github.com/containers/common/blob/e8021ed05dc2fb314b2e0dd26699de1d01d7cb19/pkg/seccomp/filter.go#L72

It was the seccomp(2) syscall which had to be added. Ready for review.

If this change requires the seccomp syscall, that means we are not only "building" the profile, but we are also applying it. Going through upstream code it does look like that:
https://github.com/containers/common/blob/e8021ed05dc2fb314b2e0dd26699de1d01d7cb19/pkg/seccomp/filter.go#L82

The funny thing is that it was only required by reconciling the profile with the contents of { "defaultAction": "SCMP_ACT_LOG" }. I'm not exactly sure which libseccomp internals triggered this requirement.

On that note, as part of calling upstream we are also setting the filter's NoNewPrivs bit to false. I am not sure we would want to do that as a side effect of "building" the profile. What are your thoughts around this?
https://github.com/containers/common/blob/e8021ed05dc2fb314b2e0dd26699de1d01d7cb19/pkg/seccomp/filter.go#L72

We only set the bit to the new filter. Do you think that this already modifies some state of the currently running process? :thinking:

The funny thing is that it was only required by reconciling the profile with the contents of { "defaultAction": "SCMP_ACT_LOG" }, tiggered by the e2e tests. I'm not exactly sure which libseccomp internals made this requirement.

My take is that by using SCMP_ACT_LOG in this context, we are logging but still allowing all the things we haven't allowed on the operator's seccomp. Reason being that a process can have multiple filters, and we are effectively bypassing ours.
I think this is the flow that is making the seccomp syscall:
matchSyscall -> AddRule (or AddRuleConditional) -> [libseccomp-golang] f.addRuleGeneric -> AddRuleWrapper -> C.seccomp_rule_add_...
Would you be opposed to stop the process here and return the filter before executing it? We would not have syscall names validated then, so we would have to extend that to cover syscall names.

We only set the bit to the new filter. Do you think that this already modifies some state of the currently running process? 🤔

Well this is a very good question. 😆  My understanding is that once set, you can't unset it, so feels like that code is a bit redundant. Here's from the kernel documentation:
Once the bit is set, it is inherited across fork, clone, and execve and cannot be unset.
https://www.kernel.org/doc/html/latest/userspace-api/no_new_privs.html

@pjbgf I switched to a modified version of containers/common and it looks like the ACT_LOG still complains that it would like to run the seccomp() syscall (see e2e test result)
I think it will happen there:
https://github.com/containers/common/pull/303/files#diff-570d2b26b4e989d337a2ecdd2fff07afR65
:thinking:

I think it will happen there:
https://github.com/containers/common/pull/303/files#diff-570d2b26b4e989d337a2ecdd2fff07afR65

you are right, the seccomp_init is called within that method. I am assuming this call may also be filtered by the seccomp syscall filter.
Given the implications, I think we better off keeping the existing validation and then start moving towards the CRD.
What are your thoughts?

I think it will happen there:
https://github.com/containers/common/pull/303/files#diff-570d2b26b4e989d337a2ecdd2fff07afR65

you are right, the seccomp_init is called within that method. I am assuming this call may also be filtered by the seccomp syscall filter.
Given the implications, I think we better off keeping the existing validation and then start moving towards the CRD.
What are your thoughts?

I agree :+1:

authors: pjbgf saschagrunert

repo: kubernetes-sigs/seccomp-operator


text:
Fix Kubernetes install on Ubuntu VM

What type of PR is this?
/kind failing-test
What this PR does / why we need it:
See: https://github.com/kubernetes/release/issues/2862#issuecomment-1556508707
Which issue(s) this PR fixes:

None
Does this PR have test?
None

Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

None

Codecov Report

Merging #1697 (f70cd2d) into main (3ae34cf) will decrease coverage by 0.05%.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1697      +/-   ##
==========================================
- Coverage   49.35%   49.30%   -0.05%
==========================================
  Files          71       71
  Lines        6930     6930
==========================================
- Hits         3420     3417       -3
- Misses       3380     3382       +2
- Partials      130      131       +1

@kubernetes-sigs/security-profiles-operator-maintainers PTAL

source: gharchive/pull-request

authors: codecov-commenter saschagrunert

license_type: permissive

license_source: github-api


text:
Updating ParametersRef docs and examples to be cluster-scoped

This matches the scoping of the GatewayClass and matches the intention of parameters being configured with implementation specific custom resources.
Fixes https://github.com/kubernetes-sigs/service-apis/issues/144
/cc @jpeach @hbagdi

Question: If an implementation doesn't want to use a CustomResource for this purpose and wants to resort to a ConfigMap, that is not possible? If yes, why do we want to impose such a limitation?

@hbagdi, that's a great question. I did not add enough context here. I think I gave a better summary as I was closing my previous PR related to this (https://github.com/kubernetes-sigs/service-apis/pull/183#issuecomment-673109175)

https://github.com/kubernetes-sigs/service-apis/pull/183#issuecomment-673109175

I somehow missed that activity in my notifications.

Secrets are potentially interesting but I don't think I've seen any use cases for GatewayClass parameters that would benefit from Secrets.

You could have an operator running in the cluster that reconciles Gateways associated with a specific GatewayClass by spinning up infra inside and outside the k8s cluster. To do this, it requires a set of credentials, stored in Secrets. Now, one can argue that the secrets then should be part of the deployment of the operator and not the GatewayClass.
I think it is safer to start with cluster-scoped references and then re-evaluate this if the above hypothetical case becomes a reality.

You could have an operator running in the cluster that reconciles Gateways associated with a specific GatewayClass by spinning up infra inside and outside the k8s cluster. To do this, it requires a set of credentials, stored in Secrets. Now, one can argue that the secrets then should be part of the deployment of the operator and not the GatewayClass.

I think it is safer to start with cluster-scoped references and then re-evaluate this if the above hypothetical case becomes a reality.

I agree with all of that. If we want to support namespace-scoped params here in the future, I think the starting point is convincing the broader sig-network community that IngressClass params should be namespace-scoped. If we can build consensus there, I think it's a no brainer to add the same capability here.

@robscott thanks for addressing https://github.com/kubernetes-sigs/service-apis/pull/286#discussion_r486641898, LGTM.

@robscott Needs another rebase.
/lgtm

@hbagdi just rebased this PR, please take another look when you have time.

@robscott Needs more rebasing.

@hbagdi Rebased once more, don't think there are any more PRs open that will conflict with this.

LGTM

/lgtm

authors: danehans hbagdi robscott

repo: kubernetes-sigs/service-apis


text:
[incubator/zookeeper]StatefullSet Desired=3 Current=1 and Pod status stays "Pending"

Is this a request for help?:
YES
Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG REPORT
Version of Helm and Kubernetes:
Helm v2.8.2
Kubernetes v1.9.2  - Docker for Mac with Kubernetes
Which chart:
incubator/zookeeper
What happened:
I installed incubator/zookeeper without error , but the pod status stays "Pending" and the number of stateful set does not increase.
$> kubectl get po
NAME                     READY     STATUS    RESTARTS   AGE
myzk-zookeeper-0         0/1       Pending   0          23m
$> kubectl get sts
NAME             DESIRED   CURRENT   AGE
myzk-zookeeper   3         1         25m
$>kubectl logs myzk-zookeeper-0
$>kubectl logs sts/myzk-zookeeper
$>

What you expected to happen:
ZooKeeper starting up correctly
How to reproduce it (as minimally and precisely as possible):
helm install incubator/zookeepe
Anything else we need to know:

What does kubectl describe show for the pod?

kubectl describe po XXXX showed below , and changed configuration.
Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  19h (x1114 over 1d)  default-scheduler  0/1 nodes are available: 1 Insufficient memory.

Now it is working perfectly.
Thank you!

source: gharchive/issue

authors: ebabani tesujiro

repo: kubernetes/charts


text:
[stable/magento] Update to the recommended pvc patterns.

This is a split-off of PR #408 to just contain the chart magento.
If there's any commonality in changes nessary between this PR and the others that are being submitted at once, please mention them in #408.
Some things to note:

This initial PR will not be updating the Chart.yaml's version string, as I feel it should be done manually per-pr once each chart's otherwise acceptable.
An automated script created the splits and the PRs, on a per-folder basis under stable, after copying the relevent files from a backup of my local copy of #408

@k8s-bot ok to test

authors: AmandaCameron viglesiasce


text:
[stable/owncloud] Add icon

PR in the context of normalizing/improving metadata in the official charts. Refs https://github.com/kubernetes/charts/issues/124 and https://github.com/helm/monocular/issues/93
NOTE: I have not bumped the chartVersion attribute in order to avoid race conditions/conflicts with existing PRs. Maintainers should be able to change this value before the merge. Llet me know if you prefer that I update it instead.

@k8s-bot ok to test

/lgtm

authors: mgoodness migmartri


text:
[occm] Workaround with hostname ip.nip.io does not work with IPv6

Is this a BUG REPORT or FEATURE REQUEST?:

/kind feature

What happened:
We are using the settings enable-ingress-hostname=true, because we are using proxy-protocoll and therefore have to route all Traffic over the loadbalancer (as described in the docs https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/openstack-cloud-controller-manager/expose-applications-using-loadbalancer-type-service.md#use-proxy-protocol-to-preserve-client-ip).
But unfortunately this workaround does not work with IPv6, since XXX:XXX:XXX.nip.io is not a valid dns name and the controller failed to patch the service
E0513 08:39:48.529201       1 controller.go:310] error processing service xxx (will retry): failed to update load balancer status: Service "xxx" is invalid: status.loadBalancer.ingress[0].hostname: Invalid value: XXX:XXX:XXX.nip.io": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')

What you expected to happen:
Honestly, I think the hostname workaround with ip.nip.io does not work properly I would prefer a proper solution with an annotation service.beta.kubernetes.io/do-loadbalancer-hostname, like already described in https://github.com/kubernetes/cloud-provider-openstack/issues/1287#issuecomment-716618469.
I know this is a temporary workaround, but with the current solution, I do not know how to properly use IPv6.
How to reproduce it:
Use the enable-ingress-hostname=true for an IPv6 loadbalancer.
Anything else we need to know?:
We are also using the workaround described in #1086.

As a note I had to edit the issue, because I messed up the references to other issues.

Too bad, I am still very interested in this issue and would also be willing to help here.
/remove-lifecycle stale

repo: kubernetes/cloud-provider-openstack

license_source: bigquery


text:
[csi-manila] Manila client moved into a separate package

What this PR does / why we need it:
Part of https://github.com/kubernetes/cloud-provider-openstack/issues/714
This PR:

moves the Manila client into a separate package. It implements manilaclient.Interface interface, making it possible to create other implementations of the client - fake Manila client in particular - a prerequisite for unit/sanity tests,
processing of cmd args was broken: changes introduced by #641 caused a bunch of cmd flags to not be recognized anymore. We're now using the Cobra library which solves the issue.

NONE

/ok-to-test

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.15

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.13

PTAL @tsmetana @gouthampacha @tombarron

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.13

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.15

/test cloud-provider-openstack-acceptance-test-e2e-conformance

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.14

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.14

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.15

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.13

/test cloud-provider-openstack-acceptance-test-e2e-conformance

/test cloud-provider-openstack-acceptance-test-lb-octavia

The CI failures don't look to be related to the code and seem quite random. I think it's OK to merge now.
/lgtm

@tsmetana thanks! @adisky PTAL, thank you!

Also @ramineni, could you please take a look if this seems ok?

/test cloud-provider-openstack-acceptance-test-lb-octavia

/test cloud-provider-openstack-acceptance-test-lb-octavia

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.14

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.14

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.13

/test cloud-provider-openstack-acceptance-test-e2e-conformance

/test cloud-provider-openstack-acceptance-test-e2e-conformance

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.13

/test cloud-provider-openstack-acceptance-test-lb-octavia

authors: adisky gman0 gouthampacha ramineni tsmetana


text:
IPvS Option for KubeProxy

Feature Description

One-line feature description (can be used as a release note):
Primary contact (assignee):
Responsible SIGs:
Design proposal link (community repo): https://github.com/kubernetes/community/issues/429
Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
Approver (likely from SIG/area to which feature belongs):
Feature target (which target equals to which milestone):

Alpha release target (x.y)
Beta release target (x.y)
Stable release target (x.y)

############### Outdated template below ###########
Description
IPvS or LVS is a kernel feature that can proxy requests in 3 different ways, Direct Routing model is the preferred model.
Progress Tracker

[ ] Alpha

[ ] Write and maintain draft quality doc

[ ] During development keep a doc up-to-date about the desired experience of the feature and how someone can try the feature in its current state. Think of it as the README of your new feature and a skeleton for the docs to be written before the Kubernetes release. Paste link to Google Doc: DOC-LINK

[ ] Design Approval

[x] Design Proposal.  https://github.com/kubernetes/community/issues/429
[ ] Decide which repo this feature's code will be checked into. Not everything needs to land in the core kubernetes repo. REPO-NAME
[ ] Initial API review (if API).  Maybe same PR as design doc. PR-NUMBER

Any code that changes an API (/pkg/apis/...)
cc @kubernetes/api

[ ] Identify shepherd (your SIG lead and/or kubernetes-pm@googlegroups.com will be able to help you). My Shepherd is: replace.me@replaceme.com (and/or GH Handle)

A shepherd is an individual who will help acquaint you with the process of getting your feature into the repo, identify reviewers and provide feedback on the feature. They are not (necessarily) the code reviewer of the feature, or tech lead for the area.
The shepherd is not responsible for showing up to Kubernetes-PM meetings and/or communicating if the feature is on-track to make the release goals. That is still your responsibility.

[ ] Identify secondary/backup contact point. My Secondary Contact Point is: replace.me@replaceme.com (and/or GH Handle)

[ ] Write (code + tests + docs) then get them merged.  ALL-PR-NUMBERS

[ ] Code needs to be disabled by default.   Verified by code OWNERS
[ ] Minimal testing
[ ] Minimal docs

cc @kubernetes/docs on docs PR
cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
New apis: Glossary Section Item in the docs repo: kubernetes/kubernetes.github.io

[ ] Update release notes

[ ] Beta

[ ] Testing is sufficient for beta
[ ] User docs with tutorials
- Updated walkthrough / tutorial in the docs repo: kubernetes/kubernetes.github.io
- cc @kubernetes/docs on docs PR
- cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
[ ] Thorough API review

cc @kubernetes/api

[ ] Stable

[ ] docs/proposals/foo.md moved to docs/design/foo.md
- cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
[ ] Soak, load testing
[ ] detailed user docs and examples

cc @kubernetes/docs
cc @kubernetes/feature-reviewers on this issue to get approval before checking this off

FEATURE_STATUS is used for feature tracking and to be updated by @kubernetes/feature-reviewers.
FEATURE_STATUS: IN_DEVELOPMENT
More advice:
Design

Once you get LGTM from a @kubernetes/feature-reviewers member, you can check this checkbox, and the reviewer will apply the "design-complete" label.

Coding

Use as many PRs as you need.  Write tests in the same or different PRs, as is convenient for you.
As each PR is merged, add a comment to this issue referencing the PRs.  Code goes in the http://github.com/kubernetes/kubernetes repository,
and sometimes http://github.com/kubernetes/contrib, or other repos.
When you are done with the code, apply the "code-complete" label.
When the feature has user docs, please add a comment mentioning @kubernetes/feature-reviewers and they will
check that the code matches the proposed feature and design, and that everything is done, and that there is adequate
testing.  They won't do detailed code review: that already happened when your PRs were reviewed.
When that is done, you can check this box and the reviewer will apply the "code-complete" label.

Docs

[ ] Write user docs and get them merged in.
User docs go into http://github.com/kubernetes/kubernetes.github.io.
When the feature has user docs, please add a comment mentioning @kubernetes/docs.
When you get LGTM, you can check this checkbox, and the reviewer will apply the "docs-complete" label.

We have a tested implementation of IPVS kubeproxy in https://github.com/kubernetes/kubernetes/issues/44063
Busy tying the various related issues and PR's together now.

@boynux  I've updated the feature description to fit the new template. Please, fill the empty fields in the new template (their actual state was unclear).

@idvoretskyi I think this can be closed as a duplicate of https://github.com/kubernetes/features/issues/265, which is the issue being tracked by sig-network for 1.8.

authors: boynux caseydavenport idvoretskyi quinton-hoole

repo: kubernetes/features


text:
Release 0.3 of frakti

Updated the frakti version
Updated CHANGELOG

This change is

repo: kubernetes/frakti


text:
Rate metrics converter

Fixes #984
Needs unit tests and manual verification.
cc: @piosz @timstclair

Unit test added

Added couple other changes required for this to work:

do not aggregate container cumulative metrics
print scrape time in logsink

LGTM

authors: mwielgus piosz

repo: kubernetes/heapster


text:
Suggestion: include hint about helm repo update when charts cannot be located

I recently tried to install seomthing from stable/*, and the chart failed to install. It turns out I just needed to update the local cache, but until this point I didn't event know there was a local cache. It may help people like me to include the name of the command helm repo update in the error message so we can immediately see  it.
This is a dumb PoC:
diff --git a/cmd/helm/install.go b/cmd/helm/install.go
index da1ee52d..6d8d68c6 100644
--- a/cmd/helm/install.go
+++ b/cmd/helm/install.go
@@ -443,7 +443,7 @@ func locateChartPath(repoURL, name, version string, verify bool, keyring,
                return filename, err
        }

-       return filename, fmt.Errorf("failed to download %q", name)
+       return filename, fmt.Errorf("failed to download %q (hint: running `helm repo update` may help)", name)
 }

 func generateName(nameTemplate string) (string, error) {

I think that "dumb" PoC might actually be a great feature to have. More actionable errors in pkg cmd is good! If you want to PR that, we would be very appreciative!

@bacongobbler Thanks! Made the PR #3716.

authors: bacongobbler lestrrat

repo: kubernetes/helm


text:
add unit test cases for core.pkg.k8s

Add unit test cases for core.pkg.k8s.GetNodeIP &core.pkg.k8s.GetPodDetails,
and change the wrong word 'retuned' to the correct value 'returned' in the previous test cases, thank you!

This change is

Coverage increased (+0.7%) to 43.088% when pulling 5b313737e0ed7affc643965750024a465885472d on chentao1596:add-unit-test-for-k8s into f1520a1232a8690b2a4138f14da73a4b94936ef3 on kubernetes:master.

@chentao1596 thanks!

/lgtm

repo: kubernetes/ingress


text:
Minimalistic Machines API proposal.

This is a proposal to add a new API for managing Nodes in a declarative way: Machines.
It is part of the overall Cluster API effort.

This change is

Hey,
we are working on a very similar concept:
https://github.com/kube-node/nodeset
Basic idea is to rely on the already existing node resources.
We defined 2 extra resources:

nodeset (Replicaset for nodes)
nodeclass (provisioning information for a node) https://github.com/kube-node/kube-machine/blob/master/examples/NodeClass_do.yaml

Flow is:

nodeset-controller creates node resources
node-controller provisions machines at the cloud-provider based upon the node resource and the assigned nodeclass

We have a example node-controller (using docker-machine): https://github.com/kube-node/kube-machine
Example nodeset controller (using archon): https://github.com/kube-node/archon-nodeset
We are working right now on a generic nodeset controller & a gke-nodeset controller.
How can we align and possible collaborate on this topic?
Contributors (not sorted by anything):
@sttts @ledzep2 @s-urbaniak @adieu @scheeles @realfake @chaosaffe @guusvw @metalmatze

@mrIncompetent Your nodeset looks similar to what we're trying to do, but I'm wondering if there's a write-up somewhere of the explicit goals of the project? It's not clear to me from reading the code where its boundaries are, or what user problems it's trying to solve.
For instance, I don't see any notions of software versions in the nodeset, so it doesn't seem like you're targeting being able to upgrade nodes like we are. Also, only having a "set" concept for nodes without being able to address specific ones seems like it would give you no control over which nodes to scale down when you have excess capacity (which could be important, if you want to target the most idle nodes for deletion). I could be wrong, though, if you're intending for the usecase to be handled by deleting Node objects themselves.
That said, it would be great to have any/all of its contributors join us in our ongoing Cluster/Machines API discussions. We're meeting weekly on Wednesdays at 11:00 PST via Zoom (https://zoom.us/j/166836624). If you want to get the invite on your calendar, you can join the SIG Cluster Lifecycle mailing list, where we can also start a thread to discuss this before our next Zoom meeting.

@kubernetes/sig-cluster-lifecycle-pr-reviews

@pipejakob Just wanted to share some thoughts while we were working on a project called Archon with similar ideas.
The basic idea is the same. Using declarative resources in Kubernetes to define the node machines and delegating real work to controllers. We chose Instance and InstanceGroup as the resource names but they could be easily mapped to Machine and MachineSet.
Here is the definition for Instance:
type Instance struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata"`
	Spec              InstanceSpec       `json:"spec,omitempty"`
	Status            InstanceStatus     `json:"status,omitempty"`
	Dependency        InstanceDependency `json:"-"`
}

type InstanceSpec struct {
	OS                  string                 `json:"os,omitempty"`
	Image               string                 `json:"image,omitempty"`
	InstanceType        string                 `json:"instanceType,omitempty"`
	NetworkName         string                 `json:"networkName,omitempty"`
	ReclaimPolicy       InstanceReclaimPolicy  `json:"reclaimPolicy,omitempty"`
	Files               []FileSpec             `json:"files,omitempty"`
	Secrets             []LocalObjectReference `json:"secrets,omitempty"`
	Configs             []ConfigSpec           `json:"configs,omitempty"`
	Users               []LocalObjectReference `json:"users,omitempty"`
	Hostname            string                 `json:"hostname,omitempty"`
	ReservedInstanceRef *LocalObjectReference  `json:"reservedInstanceRef,omitempty"`
}

type InstanceStatus struct {
	Phase      InstancePhase       `json:"phase,omitempty"`
	Conditions []InstanceCondition `json:"conditions,omitempty"`
	// TODO: allow multiple ips
	PrivateIP         string      `json:"privateIP,omitempty"`
	PublicIP          string      `json:"publicIP,omitempty"`
	InstanceID        string      `json:"instanceID,omitempty"`
	CreationTimestamp metav1.Time `json:"creationTimestamp,omitempty" protobuf:"bytes,8,opt,name=creationTimestamp"`
}

type FileSpec struct {
	Name               string `json:"name,omitempty" yaml:"name,omitempty"`
	Encoding           string `json:"encoding,omitempty" yaml:"encoding,omitempty" valid:"^(base64|b64|gz|gzip|gz\\+base64|gzip\\+base64|gz\\+b64|gzip\\+b64)$"`
	Content            string `json:"content,omitempty" yaml:"content,omitempty"`
	Template           string `json:"template,omitempty" yaml:"template,omitempty"`
	Owner              string `json:"owner,omitempty" yaml:"owner,omitempty"`
	UserID             int    `json:"userID,omitempty" yaml:"userID,omitempty"`
	GroupID            int    `json:"groupID,omitempty" yaml:"groupID,omitempty"`
	Filesystem         string `json:"filesystem,omitempty" yaml:"filesystem,omitempty"`
	Path               string `json:"path,omitempty" yaml:"path,omitempty"`
	RawFilePermissions string `json:"permissions,omitempty" yaml:"permissions,omitempty" valid:"^0?[0-7]{3,4}$"`
}

We put lots of information in InstanceSpec because we think Instance should contain all the information needed to create the machine and later we could introduce InstanceGroup, InstaceDeployment using Instance as a base. It's just like the relationship between Pod, ReplicaSet and Deployment which could be easily adopted by Kubernetes users.
To adapt for different cloud controllers, we put common fields like OS, Image, InstanceType in InstanceSpec and left cloud specific configs in annotations and Files. A File is like a unix file. We use File to inject files directly into the target machine and controllers could watch for specific path to retrieve additional configuration information just like using /proc/ files. There should be an agreement between the controller author and controller user on the path and format for the configuration files but it should be hidden from the generic view as it's kind of implementation detail. The concept behind the special File idea and your ProviderConfig idea is the same. We should left some extension point to the controllers.
There are two minor differences:
We chose to treat Instance as a read only resource just like Pod. One could only modify the machine by delete and recreate the Instance resource. In order to reuse some existing machines, we introduced a new concept called ReservedInstance. We found this approach easier to implement and reason about. However if we went down the mutable approach, controller authors could still stay in the immutable approach if they like. So there's no real problem here.
Another difference is that we target Archon as a general purpose computing resource management tool instead of a Kubernetes specific one. The fundamental design could be used to  build an etcd cluster or any distributed system but we have Kubernetes support builtin. I can understand that kube-deploy is a Kubernetes project and it's main focus is on Kubernetes, but in real world sysadmins manage a bunch of other servers besides the Kubernetes cluster. Many of them had their own tools for server bootstrapping and configuration. It will be very hard to persuade them to adopt a new tool which could only be used to manage Kubernetes clusters. If we could support generic server management, adapting tools could be made to support existing tools. terraform-provider-archon is a adapting tool we made for existing terraform users.
Whether the Machine resource represents only a Kubernetes node or a more generic server is an important design decision. Maybe we should hear from more people about the pros and cons. Personally I would support the generic server model, because it's easy to add a higher level abstraction for Kubernetes but not vise versa. There are use cases like dedicated etcd cluster and storage cluster which could be covered in this model. In order to be more generic, we have resources like Network and User in Archon and controllers to manage VPCs and certificates.
I just wanted to raise another question here. Is the Machine resource more like an Ingress resource or a Pod resource? The answer will influence the controller design. If it's like Pod then we probably have a central controller and something like cri for different backends. If it's like Ingress, then we would not have a master controller and all the controllers would consume the resource definition by themselves. We could leave this question aside until we begin to implement the controller but I think the answer to the question will influent the design of the Machine resource.
Working with @mrIncompetent and others, we introduced concepts like NodeSet and NodeClass. Like what you said in your comments, it's another dimension. We made archon-nodeset to consume the NodeSet resources and translate them into Archon InstanceGroup resources. Hope the work we had done in the kube-node project could be used as a reference for your Machine, MachineClass and MachineTemplate design.
BTW, we use jsonnet to build the final InstanceGroup definition in modules and we bundled the jsonnet files into one single executable file to improve user experience. I think they might be useful for your Machine resource too. I'll share more information on this topic in another thread.
I'm really looking forward to see a common resource shared by all the Kubernetes bootstrapping tools got defined. My thoughts on this topic might not be correct nor optimal. I just wanted to bring some discussions on the design decisions to be made. Hope we could attract more people interested in this idea and polish the resource definition together.

@munnerz @simonswine

@adieu Yes, Archon is one of the many projects we looked at when starting to work on the Cluster API project. I do think there's a lot of overlap conceptually, but that the two efforts have fundamentally different goals.
As you say yourself, Archon's Instances are deliberately general purpose and Kubernetes-unaware. I believe that this makes configuring a Kubernetes cluster much closer to the Kubernetes The Hard Way experience than the ease of other installers. Rather than saying conceptually "I would like to use the 1.8.1 version of the control plane," one must explicitly model the entire static pod manifest of every component, including every flag to pass them, the liveness probe to use, the volumes to mount, etc. This offers infinite flexibility, but because you give this much power to the enduser by only abstracting away the concept of files to place on disk, I believe it actually becomes much more tedious to configure a functioning cluster from scratch, with no guarantee that two clusters created by two different users actually look the same. They could choose to put their static pod manifests in different directories, or even run all of their control plane components via systemd instead of kubelet. This flexibility is very powerful, but makes it very difficult to operate on those clusters in a generic way.
One of the most important usecases we're targeting with the Cluster and Machines APIs is for developers to be able to write operational tooling on top of these APIs that is completely agnostic of the cluster's environment, the cloud that it's running in, and even the deployment mechanism used to provision the cluster. With the current proposal of having Kubernetes concepts be first-class citizens of these APIs, we will enable tooling to be written like generic cluster upgraders that only have to update the value of a single field on an object, and have the right thing happen with little room for shooting oneself in the foot.
In the Archon world, I don't see how a tool could generically upgrade a single Instance without understanding whether that Instance is supposed to be running the control plane, or just run a kubelet that registers with a cluster master. Further, the tool will have to understand which Files refer to which control plane components in order to understand how to even upgrade them. Flag names can change between different Kubernetes versions, so the upgrade tool would need to know about what name transformations, deprecations, and additions to make to the flags passed to each component separately. Please correct me if I'm wrong, but I think any tooling written on top of Archon's Instances would need to deeply inspect every object and have many switch statements to know how to safely upgrade or downgrade Instances or the components running on them, or else have these sections maintained by hand by the cluster admin.  Also, if an Instance is not Kubernetes-aware, then would tooling that deletes an Instance need to handle safely evicting workloads from that Instance first?
I understand the desire for creating a completely generic abstraction of hosts, but I think that direction is off the spectrum of what would be useable for the usecases we're targeting to support with this particular project.
However, one way that I can see these two projects potentially collaborating is by having a Machine -> Instance adapter that allows us to take advantage of all of the existing work in Archon in order to jumpstart the number of providers supported by the Machines API, and to keep taking advantage of any new providers implemented in the Archon project.
As for your question of whether a Machine object is modeled more like Ingress than a Pod, I would say Ingress (based on my understanding of the intent of your question). It is fully expected that your cluster should have a cloud- or environment-specific controller handling your Machine objects.  However, most of the Machine's spec is generic, so the power comes from these objects being operated on in an generic way. You don't need to know what cloud a Machine is in in order to change its kubelet version, but a cloud-specific controller will handle reconciling the real world with that new declared spec.

@pipejakob Couldn't the abstraction still be done with archon like language with some kind of helper on the host and annotations of some kind on the node? Say user sets k8s version previous+1 annotation. node itself could pull the annotation and tweak things like performing a kubelet upgrade. Maybe some of the logic your talking about belongs in kubeadm? Being a ndoe level thing has the advantage of being very agnostic to the tool (could be driven in the image by go, ansible, chef, etc). Could map annotations to chef roles for example.
It would be really nice if the both use cases could be handled by the same object with some extra fields somehow. For an operator, sometimes the distinction between a fully k8s node, and oh, I need to override this one thing on the host is a very fine line. being able to mix both use cases together would really be helpful I think.

Do we really need a new object that presents Node? We can just use the existing Noe object. I think using a separate object type will create another point of reconciliation.
Especially for cloud providers like AWS/GCE where the actual nodes are created via AutoScaler (hence thew names are generated randomly), this will require sync among 3 things - Cloud providers IntanceGroup , []Machine and []Node.
In case of appscode/pharmer, we defined a NodeGroup object https://github.com/appscode/pharmer/blob/dd266cded7e686bdbdc037496351b947bf8081eb/apis/v1alpha1/node.go#L16
This gets translated to appropriate group concept depending for cloud providers that support them (AWs & GCE). For simple VPS provider this just becomes a simple loop over Node creation. To maintain the sync, we pass the NodeGroup name via kubelet's --node-labels flag.

@pipejakob IMHO the two projects are trying to tackle the same problem from different ends. Cluster API project is more about forming a standard which all cluster management tools could be built on whereas Archon is more about cloud controller implementation details.
To make the whole Cluster API project useful for end users. We need a commonly adopted resource definition and controllers which support it. The structure is like this:
┌─────────────────────────────────────────────────────┐
│                     Kubernetes                      │
│                                                     │
│   ┌─────────────┐ ┌─────────────┐ ┌─────────────┐   │
│   │ AutoScaler  │ │ AutoUpdater │ │ Other Tool  │   │
│   └─────────────┘ └─────────────┘ └─────────────┘   │
│   ┌─────────────────────────────────────────────┐   │
│   │              Machine Resource               │   │
│   └─────────────────────────────────────────────┘   │
│   ┌─────────────┐ ┌─────────────┐ ┌─────────────┐   │
│   │     GCP     │ │     AWS     │ │    Other    │   │
│   │ Controller  │ │ Controller  │ │ Controller  │   │
│   └─────────────┘ └─────────────┘ └─────────────┘   │
│                                                     │
└─────────────────────────────────────────────────────┘

In reality, companies have different infrastructure design and their own preference on the OS they use and they might want to customize different parts of the cluster (eg. whether to use AutoScaler provided by the cloud; different certificates management strategy). So we can imagine more controllers would be built. The structure might be like this:
┌───────────────────────────────────────────────────────────────┐
│                          Kubernetes                           │
│                                                               │
│  ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐  │
│  │   Autoscaler    │ │   AutoUpdater   │ │   Other Tool    │  │
│  └─────────────────┘ └─────────────────┘ └─────────────────┘  │
│  ┌─────────────────────────────────────────────────────────┐  │
│  │                    Machine Resource                     │  │
│  └─────────────────────────────────────────────────────────┘  │
│  ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐  │
│  │  GCP + CoreOS   │ │   AWS + kops    │ │    HA Master    │  │
│  │   Controller    │ │   Controller    │ │   Controller    │  │
│  └─────────────────┘ └─────────────────┘ └─────────────────┘  │
│  ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐  │
│  │  GCP + CentOS   │ │  AWS + kubeadm  │ │  External ETCD  │  │
│  │   Controller    │ │   Controller    │ │   Controller    │  │
│  └─────────────────┘ └─────────────────┘ └─────────────────┘  │
│                                                               │
└───────────────────────────────────────────────────────────────┘

I guess most of the controllers would be implemented in Go. Each one has it's own server bootstrapping process embedded in the Go binary. Those controllers might or might not share code but we can see that there are some common parts in them. All of them have to deal with the Machine spec, and they all have to talk to the cloud provider they're supporting. That's a lot of code could be shared. And that's what Archon is built for: a meta controller which support different cloud providers and customizable server bootstrapping process defined by plain text files. Archon extracted the business logic from Go source code and put them in the server definition resources like this:
┌─────────────────────────────────────────────────────────────────┐
│                            Kubernetes                           │
│                                                                 │
│ ┌───────────────────┐┌───────────────────┐┌───────────────────┐ │
│ │    AutoScaler     ││    AutoUpdater    ││    Other Tool     │ │
│ └───────────────────┘└───────────────────┘└───────────────────┘ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │                      Machine Resource                       │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────┐ ┌────────────┐ │
│ │              Archon Controller               │ │            │ │
│ │                                              │ │            │ │
│ │ ┌────────────┐ ┌────────────┐ ┌────────────┐ │ │            │ │
│ │ │   CoreOS   │ │   CentOS   │ │   Other    │ │ │            │ │
│ │ │  Cluster   │ │  Cluster   │ │  Cluster   │ │ │   Other    │ │
│ │ │ Definition │ │ Definition │ │ Definition │ │ │ Controller │ │
│ │ └────────────┘ └────────────┘ └────────────┘ │ │            │ │
│ │ ┌────────────┐ ┌────────────┐ ┌────────────┐ │ │            │ │
│ │ │    GCP     │ │    AWS     │ │   Aliyun   │ │ │            │ │
│ │ └────────────┘ └────────────┘ └────────────┘ │ │            │ │
│ │                                              │ │            │ │
│ └──────────────────────────────────────────────┘ └────────────┘ │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

There are some unsolved problems in Archon like the ones you had mentioned in your comment. To stay focus on the Cluster API project, I'll leave them for another issue maybe in the Archon repo. The core idea is that it's important to bring both programmers and sysadmins onboard. It will be ideal that programmers could collaborate on a shared codebase and sysadmins could collaborate on the server bootstrapping definitions. It's what Archon was trying to achieve just like Helm did for app packaging.
As a controller author, my suggestion to Cluster API project is to leave enough extension points for controllers. ProviderConfig should be enough for the first version. After controllers begin to adopt the Machine resource definition, maybe we could have more discussion on this topic.

hmmm..... Maybe we're mixing use cases too much here and should step back and well define the components.
I see two use cases here..... cluster api is about the user being able to manage the k8s cluster versioning/lifecycle via an api?
The other use case is managing vm's/baremetal as first class resources in k8s (machine api)?
I see these as being disjoint but partially overlapping:
https://upload.wikimedia.org/wikipedia/commons/5/56/Venn-diagram-AB.png
The cluster api (should this really be called k8s lifecycle api?) could layer on top of a machine api, but that is an implementation detail. It could also have 0 overlap too.
Say k8s is self hosting, for example, bootkube was being used on the cluster. There could be a bootkube cluster controller that mapped cluster api to k8s deployment tweaks for the self hosted apiserver, controller-manager, and the node labels. In this case, the machine api might not even be used at all. (some operators will prefer this)
But for kopts and other systems, it could load a kopts controller that mapped one to one to the machines api to accomplish managing the cluster. (other operators will prefer this)
The user in either case sees a consistent api, which I think is very good.
The machine api itself should probably have no notion then of whats driving it. As well as not too much idea of what it is provisioning. It should be able to provision nodes and hand over some data to software running on the node in order for the node itself to deal more with lifecycle things (what packages to install, which version of kubelet to install, settings, etc). This allows the machine objects to be used for use cases that are much more broad then just the cluster api. We didn't make a specialized version of Pod just for kubernetes controller components. We shouldn't for machines that only work as cluster components.
I think we probably should seriously consider breaking up the cluster api and machine api to two separate complimentary discussions/proposals?
Once split, I also think we should evaluate Archon for a good starting point for the Machine api side of things. I think it already checks most of the boxes of requirements?

@pipejakob @kfox1111 We wrote an proposal  https://github.com/kube-node/nodeset/blob/master/proposal.md about the idea and the concept about kube-node and NodeSet.
In general the goal is to have a higher abstraction layer for managed k8s nodes and to be able to integrate Kubernetes with different providers in a generic way. It is similar to you idea and we would like to work with you to find a general approach.
We already have parts in production and using it for our customer.

@pipejakob  Is the plan to move this design into the community repo, or are we planning on keeping it here?

@chrislovecnm We hemmed and hawed about where this code and proposals should live. I don't know if the community repo is the right answer, given that we're not targeting new functionality in Kubernetes core, but I do think it's pretty safe to assume that kube-deploy is a temporary home. When the project gains traction, it will likely warrant its own repository and incubator project.

Do we really need a new object that presents Node? We can just use the existing Noe object. I think using a separate object type will create another point of reconciliation.

@tamalsaha There were a few different factors that brought us here:

Strictly additive functionality on top of core Kubernetes. We wanted the new concepts to play well with core, but not modify any of its code or break compatibility with core. I think a good smoke test would be whether or not an existing vanilla Kubernetes cluster could adopt the Machines API temporarily, and then go back to being a vanilla Kubernetes cluster without ill effects.
We wanted to introduce new declarative attributes that aren't present in the existing Node object. Right now, I believe the only attributes missing are the control plane version and providerConfig, but the idea was to evolve the Machine definition to keep pulling in the implementation-specific fields from providerConfig into first-class fields when it seemed appropriate. For instance, if every cloud implementation finds it useful to give users a field to specify the disk size to use, then this is probably a generic (and generally-applicable) enough concept to pull out of the providerConfig into a top-level attribute in the Machine.Spec. That means that over time, we'll likely have more and more fields that don't exist in the core Node object.
Don't abuse annotations for things that belong in a spec. Multiple people called this out as an anti-pattern used in Ingress objects that have caused a lot of confusion and pain for customers. Annotations aren't type-safe, you can't get a signal back if the controller you're running doesn't support or recognize a particular annotation, etc.

Creating our own new concept of Machines gives us full control over the shape and evolution of the object without being tethered to the current definition of Node in core, and without having to resort to annotations.
Since we're creating a new Machine object, with new controllers to handle them, that makes it very easy to remove Cluster API support from a cluster: just remove the new controllers and the Machine objects. Any Nodes that were created as the result of Machines will still remain, and be fully compatible with core Kubernetes.

@pipejakob I hear yah. Sub projects - what the heck do we do now. My concern is that this impact a lot of installers. The demo at Kubecon will help with the messaging, but communication of what is going on cross community is always a challenge. We need to shout it off the mountain tops a bit, and figure out an appropriate home for that. When you feel it is appropriate I have some ideas. Always looking to improve what we do.

I've updated the types based on more discussion/feedback from our breakout sessions:

Added:

Machine.Spec.Taints
Machine.Status.Versions (especially for ControlPlane version)
consts for MachineRoles: MasterRole, NodeRole

Removed:

Machine.Status.Ready

Clarified that Machine.Status.Error* fields are only for terminal errors, and transient errors should be logged or become events.

Still needs resolution:

Should kubelet version be a struct? If so, what should it look like?
Should control plane version be striped horizontally rather than vertically? e.g. Rather than every Machine having a single control plane version, should we instead have a single control plane version per component so that you can first upgrade kube-apiserver, then kube-controller-manager, etc.? Should we have both dimensions (per-component and per-Machine)?

Should kubelet version be a struct? If so, what should it look like?

A string could work, if everyone's cool parsing it. Might be the simpler API, since providers might want their own version formats for custom patches. You should require that vanilla version formats like v1.8.1 imply the Kubelet is a vanilla OSS Kubelet. Providers that deploy custom kubelets should have to do something like what GKE does with the -gke.1 versions.
If you want a struct, a base struct for semver (all ints) and an additional string or int for the provider version might suffice. Another thought that crossed my mind is a semver struct and a map[string]int to handle provider versions.

hey @pipejakob can you address the conflict?

On it, but it looks like the upstream build of gcp-cluster-api is broken, so I can't fully test my rebase yet. I'll wait for https://github.com/kubernetes/kube-deploy/pull/561 or a similar patch to fix the build.

https://github.com/kubernetes/kube-deploy/pull/568 is the fix for the compilation error, which is being reviewed.
cc @karan @jessicaochen

Okay, rebased and tested end-to-end via cluster-api-gcp. Ready to ship?

/lgtm

authors: adieu chrislovecnm k8s-reviewable kfox1111 kris-nova luxas mattbates mrIncompetent mtaufen pipejakob roberthbailey rsdcastro scheeles tamalsaha

repo: kubernetes/kube-deploy


text:
kubectl apply -k unknown shorthand flag: 'k' in -k

What happened:
kubectl apply -k "github.com/aws/eks-charts/stable/aws-load-balancer-controller//crds?ref=master"
Error: unknown shorthand flag: 'k' in -k
What you expected to happen:
apply the crds
How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:
Environment:

Kubernetes client and server versions (use kubectl version):
kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.0", GitCommit:"0ed33881dc4355495f623c6f22e7dd0b7632b7c0", GitTreeState:"clean", BuildDate:"2018-09-27T17:05:32Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.9-eks-d1db3c", GitCommit:"d1db3c46e55f95d6a7d3e5578689371318f95ff9", GitTreeState:"clean", BuildDate:"2020-10-20T22:18:07Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
Cloud provider or hardware configuration:
OS (e.g: cat /etc/os-release):
mac os mojave
kustomize version
kustomize version
{Version:kustomize/v3.9.2 GitCommit:e98eada7365fc564c9aba392e954f306a9cbf1dd BuildDate:2021-01-21T14:51:07+00:00 GoOs:darwin GoArch:amd64}

@modevops

Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.0", GitCommit:"0ed33881dc4355495f623c6f22e7dd0b7632b7c0", GitTreeState:"clean", BuildDate:"2018-09-27T17:05:32Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.9-eks-d1db3c", GitCommit:"d1db3c46e55f95d6a7d3e5578689371318f95ff9", GitTreeState:"clean", BuildDate:"2020-10-20T22:18:07Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

Your kubectl version seems too old. 1.12.0 is currently not supported. Install the latest and try again. ref https://kubernetes.io/docs/tasks/tools/install-kubectl/

Please reopen if you still encounter this using a supported version.
/close

authors: dty1er eddiezane modevops

repo: kubernetes/kubectl


text:
Enable kubectl to complete kubectl get pod/...

What would you like to be added:
Testing in PowerShell, kubectl can complete the pod name if you use a space:
kubectl get pod <pod-name>

However if you use a forward-slash the completion doesn't work:
kubectl get pod/<pod-name>

Why is this needed:
It would be nice if the completions supported both formats since kubectl itself accepts both formats.

This is possible and Marc may be working on it.
/triage accepted
/priority backlog
/assign @marckhouzam

PR now available: https://github.com/kubernetes/kubernetes/pull/108493
Feedback welcomed.

/assign @marckhouzam

authors: dhirschfeld eddiezane marckhouzam


text:
'fieldpath' parsing fails with error 'field specified in var not found in corresponding resource'

See below the yaml files to reproduce the issue. (A full version of the deployment can be found here - https://github.com/hmanikkothu/ibmmq-jmsspringboot-kubernetes/tree/master/kubernetes )

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mqspringboot-deployment
  labels:
    app: mqspringboot
spec:
  replicas: 2
  selector:
    matchLabels:
      app: mqspringboot
  template:
    metadata:
      labels:
        app: mqspringboot
    spec:
      containers:
        - name: mqspringboot
          image: hmanikkothu/mqspringboot:latest
          ports:
            - containerPort: 8080

mqsvc.yaml

kind: Service
apiVersion: v1
metadata:
  name: ibmmqsvc
spec:
  ports:
    - port: 1414
      targetPort: 1414

kustomization.yaml

resources:
  - mqsvc.yaml
  - deployment.yaml
patchesStrategicMerge:
  - patch.yaml

vars:
  - name: MQ_SVC_PORT
    objref:
      kind: Service
      name: ibmmqsvc
      apiVersion: v1
    fieldref:
      fieldpath: spec.ports[0].port

patch.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mqspringboot-deployment
spec:
  template:
    spec:
      containers:
        - name: mqspringboot
          env:
            - name: MQ_PORT
              value: $(MQ_SVC_PORT)

kustomize version - 3.5.4 (latest)
kustomize version
{Version:3.5.4 GitCommit:3af514fa9f85430f0c1557c4a0291e62112ab026 BuildDate:2020-01-11T06:57:29+00:00 GoOs:darwin GoArch:amd64}

kubectl version - 1.17 (latest)
kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2019-12-13T11:52:32Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.8", GitCommit:"211047e9a1922595eaa3a1127ed365e9299a6c23", GitTreeState:"clean", BuildDate:"2019-10-15T12:02:12Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

Reproduce
Step 1
Run kustomize build . , which correctly finds the port from service definition and merges in the deployment
$ kustomize build .

apiVersion: v1
kind: Service
metadata:
  name: ibmmqsvc
spec:
  ports:
  - port: 1414
    targetPort: 1414
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: mqspringboot
  name: mqspringboot-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: mqspringboot
  template:
    metadata:
      labels:
        app: mqspringboot
    spec:
      containers:
      - env:
        - name: MQ_PORT
          value: **1414**
        image: hmanikkothu/mqspringboot:latest
        name: mqspringboot
        ports:
        - containerPort: 8080

Step 2
Use kubectl apply to deploy the yaml configs to the cluster, this throws an error.
$ kubectl apply -k .

**error: field specified in var '{MQ_SVC_PORT ~G_v1_Service {spec.ports[0].port}}' not found in corresponding resource**

Expected result: Step 2 should successfully generated the final deployment and mqsvc yaml files and applies to the cluster.
Actual result: Step 2 throws the error 'field specified in var '{MQ_SVC_PORT ~G_v1_Service {spec.ports[0].port}}' not found in corresponding resource'

I believe this is a kustomize bug. The reason you don't get error when using kustomize build is because your kustomize version is the newest which is different from kubectl 's kustomize version. I tested kustomize build using version 2.0.2 will get the same error.


text:
Issue with /docs/tutorials/kubernetes-basics/update-interactive/

This is a...

[ ] Feature Request
[x] Bug Report

Problem:
The last paragraph says:
We see that the deployment is using a stable version of the app (v1). The Rollback was successful.
I think there is a typo. Shouldn't it be saying v2.
Proposed Solution:
We see that the deployment is using a stable version of the app (v2). The Rollback was successful.
Page to Update:

http://kubernetes.io/docs/tutorials/kubernetes-basics/update-interactive/
Step 3 of 3

Thanks for the feedback. This has been fixed.

This was a duplicate issue and has been resolved. Thx @BenHall

authors: BenHall araj7 chenopis


text:
WIP: Fix travis failure

This change is

@jeffmendoza @pwittrock

LGTM, thanks Janet.

The issue seems related to golang. Bumping go version to 1.7.3 solves it (chose 1.7.3 because we're using 1.7.3 in kubernetes as well https://github.com/kubernetes/kubernetes/pull/35716)

authors: janetkuo jeffmendoza steveperry-53


text:
Move Guide topics: Logging

This change is

@steveperry-53 Please cc owners of the page next time you're changing it. Thanks!

@Crassirostris, Will do.


text:
consistent-list-from-cache tests are broken

What happened:
consistent-list-from-cache tests are broken.
What you expected to happen:
Them passing
How to reproduce it (as minimally and precisely as possible):
See on dashboard https://testgrid.k8s.io/sig-scalability-experiments#Summary
Anything else we need to know?:
No

/help

Fixed

repo: kubernetes/perf-tests


text:
Log response body when there is an error from Prometheus

/assign @mm4tt
ref https://github.com/kubernetes/kubernetes/issues/86082

/lgtm


text:
Issue with k8s.io/docs/tutorials/kubernetes-basics/deploy-app/deploy-interactive/

This is a Bug Report

Problem:
Misleading explanation in the text compared to the command stated in the command line box. The test says to use the command kubectl run, but the command in the command line box uses kubectl create deployment
Proposed Solution:
Align the explanation in the text with the command to be executed.
Page to Update:
https://kubernetes.io/...

Thanks! I must have missed this when updating the example.
Alternatively, you can raise a PR at https://github.com/katacoda-scenarios/kubernetes-bootcamp-scenarios

/kind bug
/priority important-longterm

Think it's appropriate to flag this as a
/good-first-issue
Mention me (@sftim) if you are looking at this and are not sure where to start.

/assign

PR katacoda-scenarios/kubernetes-bootcamp-scenarios#16 will fix this.

@amalts Sorry for the delay, I didn't see the notification until now. This has been merged.

/close

Thanks @BenHall.
This can be closed @andreaskring

authors: BenHall amalts andreaskring sftim

repo: kubernetes/website


text:
Outdated files in the dev-1.18-ko.5 branch

This is a Bug Report
Problem
Outdated files in the upstream/dev-1.18-ko.5 branch.
24 files to be modified

[ ] content/en/docs/concepts/architecture/cloud-controller.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/architecture/nodes.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/cluster-administration/cloud-providers.md 1 file changed, 2 insertions(+), 2 deletions(-)
[ ] content/en/docs/concepts/configuration/overview.md 1 file changed, 2 insertions(+), 3 deletions(-)
[ ] content/en/docs/concepts/containers/images.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md 1 file changed, 4 insertions(+), 2 deletions(-)
[ ] content/en/docs/concepts/overview/working-with-objects/namespaces.md 1 file changed, 5 insertions(+), 5 deletions(-)
[ ] content/en/docs/concepts/policy/pod-security-policy.md 1 file changed, 4 insertions(+)
[ ] content/en/docs/concepts/scheduling-eviction/taint-and-toleration.md 1 file changed, 2 insertions(+), 13 deletions(-)
[ ] content/en/docs/concepts/services-networking/connect-applications-service.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/services-networking/dns-pod-service.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/workloads/controllers/daemonset.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/workloads/controllers/deployment.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/workloads/controllers/jobs-run-to-completion.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/workloads/controllers/statefulset.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/workloads/pods/init-containers.md 1 file changed, 2 insertions(+), 2 deletions(-)
[ ] content/en/docs/contribute/new-content/overview.md 1 file changed, 3 insertions(+)
[ ] content/en/docs/home/_index.md 1 file changed, 3 insertions(+), 3 deletions(-)
[ ] content/en/docs/setup/production-environment/tools/kubeadm/ha-topology.md 1 file changed, 5 insertions(+)
[ ] content/en/docs/tasks/access-application-cluster/web-ui-dashboard.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/tasks/manage-kubernetes-objects/kustomization.md 1 file changed, 12 insertions(+), 13 deletions(-)
[ ] content/en/docs/tutorials/hello-minikube.md 1 file changed, 6 insertions(+), 2 deletions(-)
[ ] content/en/docs/tutorials/stateful-application/zookeeper.md 1 file changed, 1 deletion(-)
[ ] content/en/training/_index.html 1 file changed, 1 insertion(+), 1 deletion(-)

0 files to be renamed
0 files to be deleted
Proposed Solution
Use git diff to check what is changed in the upstream. And apply the upstream changes manually
to the content/ko of upstream/dev-1.18-ko.5 branch.
For example:
# checkout `upstream/dev-1.18-ko.5`
...
# check what is updated in the upstream
git diff upstream/dev-1.18-ko.4 upstream/dev-1.18-ko.5 -- content/en/docs/concepts/architecture/cloud-controller.md
# apply changes to content/ko
vi content/ko/docs/concepts/architecture/cloud-controller.md
...
# commit and push
...
# make PR to `upstream/dev-1.18-ko.5`

Pages to Update
/language ko

/assign

/close

authors: gochist ysyukr


text:
Interactive Tutorial - Configuring a Java Microservice

This tutorial is too complex to be delivered in a tiny little Katakoda window, you guys should simplify it when you get the chance

/retitle Simplify Interactive Tutorial - Configuring a Java Microservice
/kind feature
/language en
I think you're also able to follow this tutorial on your own cluster.
/priority awaiting-more-evidence

/retitle Simplify Interactive Tutorial - Configuring a Java Microservice
/kind feature
/language en
I think you're also able to follow this tutorial on your own cluster.
/priority awaiting-more-evidence

authors: iiian sftim


text:
Documenting & defining the Issue Wrangler Role

Per #38861, SIG Docs is introducing the Issue Wrangler Role. After issuing a call for inputs, we are ready to move on to the next step of documenting + defining the role formally.
This issue has been opened to track the below efforts

[ ] Document the Issue Wrangler role in our contribution guide per information provided in #38861 . This will need to be done by a PR to k/website and can be done by a contributor
[ ] Add Wiki entry for the Issue Wrangler role.
[ ] Issue call out for nominations

cc: @natalisucks @reylejano

/triage accepted

@octonawish-akcodes : The details of the role have been mentioned in #38861 .
You may structure it similar to the PR Wranglers page in our contribution guide & place it under the Participate in SIG Docs section.

Got it :)

@octonawish-akcodes - Please could you advise if you have commenced work on this PR? It has been three weeks, which is why the check-in. If you're unable to work on it due to bandwidth issues, that is completely fine. Please let us know. Failing any response by Friday EOD, we will be assigning this to someone internally so that they can work on it & we can issue a call for nominations within the next couple of weeks.

/assign

@haripriya9647 : Any update on this, please? We're looking at wrapping this entire thing up by September.

Hello @divya-mohan0209 yes I am working on this, actually, I had some issues with my laptop due to which I couldn't send you an update

Should I take this?
I am personally waiting for this issue wrangler role to be implemented. Hopefully the time to triage issues would be improved by a lot.

Hello @Affan-7 i have already created a pr for this issue.

The issue wrangler role has been documented and defined. Any follow-up improvements to the role/documentation can be made via subsequent issues and PRs
/close


text:
[hi] Localize concepts/Storage page

What would you like to be added
Localize concepts/Storage page in Hindi.
Why is this needed
There is no Hindi translation for this page

/area localization
/language hi

related page -> https://kubernetes.io/docs/concepts/storage/

/help

There seems to be a large section of concept/stroage to be localized ⇾

[ ] Volumes
[ ] Persistent Volumes
[ ] Projected Volumes
[ ] Ephemeral Volumes
[ ] Storage Classes
[ ] Volume Attributes Classes
[ ] Dynamic Volume Provisioning
[ ] Volume Snapshots
[ ] Volume Snapshot Classes
[ ] CSI Volume Cloning
[ ] Storage Capacity
[ ] Node-specific Volume Limits
[ ] Volume Health Monitoring
[ ] Windows Storage

As there are large number of files to be localized, a single user must split the PR or multiple assignees can assign the pages they want to localize.


text:
clean-up minikube toc

clean-up minikube toc
clean-up localkube link

Note: verify that localkube has been deprecated

/assign @cody-clark

/lgtm
/approve

authors: cody-clark kbhawkey


text:
Update nl.toml

Update i18n/nl.toml
see #20009

/lgtm
/approve

authors: jimangel wwgfhf


text:
ID localization for access-application-cluster list-image

This PR translates /docs/tasks/access-application-cluster/list-all-running-container-images/ as mentioned #22296

Thanks for the translation! I've left some reviews to improve clarity @ariscahyadi

Thanks, @phanama for your review. I have addressed them all. Please check!

/lgtm
/approve

authors: ariscahyadi girikuncoro


text:
Fix missing command in helm search statement

helm search requires either hub or repo command after seach.
kejones@jetson-nano-1:~/Kubernetes$ helm version
version.BuildInfo{Version:"v3.1.0", GitCommit:"b29d20baf09943e134c2fa5e1e1cab3bf93315fa", GitTreeState:"clean", GoVersion:"go1.13.7"}

kejones@jetson-nano-1:~/Kubernetes$ helm search service-catalog

Search provides the ability to search for Helm charts in the various places
they can be stored including the Helm Hub and repositories you have added. Use
search subcommands to search different locations for charts.

Usage:
  helm search [command]

Available Commands:
  hub         search for charts in the Helm Hub or an instance of Monocular
  repo        search repositories for a keyword in charts

kejones@jetson-nano-1:~/Kubernetes$ helm search repo service-catalog
NAME                	CHART VERSION	APP VERSION	DESCRIPTION
svc-cat/catalog     	0.3.0        	           	service-catalog webhook server and controller-m...
svc-cat/catalog-v0.2	0.2.3        	           	service-catalog API server and controller-manag...

You will need to sign the CLA @kjw3

CLA signed

/approve

authors: kjw3 markyjackson-taulia sftim


text:
Update ko/docs/concepts/overview/working-with-objects/common-labels/

From #23939
/language ko

Direct link to the change: https://deploy-preview-24370--k8s-dev-ko.netlify.app/ko/docs/concepts/overview/working-with-objects/common-labels/#애플리케이션과-애플리케이션-인스턴스

/assign

@pjhwa 리뷰 감사합니다!
의견 주신 대로 반영하였습니다.

@pjhwa 리뷰 감사합니다!
의견 주신 대로 반영하였습니다.

감사합니다!
커밋 스쿼시 작업을 부탁드립니다.

@pjhwa 커밋 스쿼시 작업 완료하였습니다!

/lgtm

PTAL: @kubernetes/sig-docs-ko-owners
(저는 컨트리뷰터와 같은 기관 소속이므로 다른 Approver 께서 승인해주시면 좋겠습니다. ^^)

@jihoon-seo 감사합니다!
/approve

authors: gochist jihoon-seo pjhwa seokho-son


text:
es: Add content/es/docs/concepts/policy/limit-range.md; Add content/es/docs/concepts/policy/_index.md

/language es
/sig-docs
/assign @raelga
Created _index.md for es/docs/concept/policy.
Created limit-range.md, content-type:concept for es/docs/concepts/policy

Required: Content update ([es]do not exists): Nombres de subdominio DNS
Suggested: Task assignation, content does not exists: es/ Managing Resource of Containers.

Edit: I had a better understanding after checking the LimitRanger layout document. So take a close look and let me know what do you think about it. Thanks a lot

Note: A lot of outputs complaining about lost "pt" definition like this:

WARN 2021/03/03 08:13:59 Failed to get translated string for language "pt" and ID "ui_pager_prev": %!s(<nil>)
WARN 2021/03/03 08:13:59 Failed to get translated string for language "pt" and ID "ui_pager_next": %!s(<nil>)
WARN 2021/03/03 08:13:59 Failed to get translated string for language "pt" and ID "post_last_mod": %!s(<nil>)
WARN 2021/03/03 08:13:59 Failed to get translated string for language "pt" and ID "post_edit_this": %!s(<nil>)
WARN 2021/03/03 08:13:59 Failed to get translated string for language "pt" and ID "post_create_issue": %!s(<nil>)
WARN 2021/03/03 08:13:59 Failed to get translated string for language "pt" and ID "post_last_mod": %!s(<nil>)
WARN 2021/03/03 08:13:59 Failed to get translated string for language "pt" and ID "post_edit_this": %!s(<nil>)
WARN 2021/03/03 08:13:59 Failed to get translated string for language "pt" and ID "post_create_issue": %!s(<nil>)
...

Server runing:

EN
ZH
KO
JA
FR
IT
DE
ES
PT
ID
RU
VI
PL
UK

Pages
1221
811
421
312
250
57
104
181
68
282
118
63
52
64

Paginator pages
36
7
0
0
0
0
0
0
0
0
0
0
0
0

Non-page files
469
346
237
244
66
20
15
29
20
107
25
8
6
24

Static files
774
774
774
774
774
774
774
774
774
774
774
774
774
774

Processed images
1
1
0
0
0
0
0
0
0
0
0
0
0
0

Aliases
8
1
2
0
0
1
0
0
1
2
0
0
0
0

Sitemaps
2
1
1
1
1
1
1
1
1
1
1
1
1
1

Cleaned
0
0
0
0
0
0
0
0
0
0
0
0
0
0

Built in 92290 ms
Watching for changes in /src/{archetypes,assets,content,data,i18n,layouts,package.json,postcss.config.js,static,themes}
Watching for config changes in /src/config.toml, /src/themes/docsy/config.toml, /src/go.mod
Environment: "development"
Serving pages from /tmp/hugo
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at...

Side note: ZH had been randomly generating 810 / 811 pages (at differents make container-serve executions I was intrigued)

/uncc @alexbrand
/cc @electrocucaracha

@Karmavil Do you mind to move the README-es.md update to it's own PR?

@raelga give me a minute I'm on it.. I would like to add fast fix too a typo (I missed a word)

@raelga . It's done.

/uncc @rajeshdeshpande02
/retitle [es] Add content/es/docs/concepts/policy/limit-range.md
Part of #13948

@raelga I'm so sorry. I screw up. Do not approve without check this message.
I checkout the remote not the origin undoing my changes.. and I just checked and there was a commit recently.
I saw the file changed disappeared but in fact what I've done is upload changes that wasn't mine.
So it's easy for merge but is not wright.
Let me know if I can fix that

Oh ok forget it was looking at my own changes, my repo. The remote (origin) do not have recent changes

@Karmavil don't need to be sorry at all. Thanks for being that enthusiastic, happy to have you on the team!
Let's review the PR and get it published as soon as posible so Spanish folks can enjoy your content!
/label tide/merge-method-squash

Muy buenas observaciones.
Dejé uno de los cambios sugeridos para re-evaluar contigo porque la traducción de LimitRanges fue incómoda en todo el documento.
Sería "Recursos límite"  o mucho mejor "Límites de recursos". Para mí entender, "rangos de límites" es dificil de interpretar. La idea es diferencia lo que es la voluntad de poner límites, y otra distinta son los verdaderos límites de un recurso. Por lo que tu sugerencia no esta mal ya que el LimitResource establece valores por defecto y máximos y mínimos..
Voy a aceptar esa sugerencia, sin embargo sería genial si @nelvadas puede corregir ese typo (una palabrita extra que se pasó) hice referencia en uno de los mensajes del commit changes

A propósito, la utilización de "container" desde el glossary no es posible porque hay que actualizar ese contenido. Está tirando un error para el merge por esa razón (lo había olvidado). Podríamos utilizar la definición del inglés o esperar a que se actualize el contenido en español agregando la definición

Listo @electrocucaracha  podrías revisar esa sección nuevamente y dar tu parecer?
Gracias!

Gracias por la PR!
Algunos términos propios de k8s, como Namespace no los localizaría ya que puede llevar a confusión. Sin embargo, al proporcionar en cada referencia el tooltip, para una persona no anglosajona queda claro su significado en castellano.
Cualquier cosa lo comentamos por slack o por aquí.
:)

No sé como resolver esta sugerencia, Namespace, Pods y Containers muchas veces se una más de una vez por párrafo. Y es realmente una moléstia tener el tooltip apareciendo sobre el texto por todos lados.
Limité el tooltip a una aparición por párrafo. En la página el tooltip se repite varias veces pero no tanto como para tener todos los párrafos subrayados innecesariamente. El lector puede inferirr adonde están los tooltip fácilmente cuando no hay demasiados de ellos presentes

Sobre los temas que comentas, lo de los tooltips ya lo discutiremos y si aplicaremos cambios (si toca) en una futura PR. Tal y como está el contenido, yo lo veo perfecto.
Referente a lo de Rangos Límite, ahora mismo el título es Rangos de límites (Limit Ranges), que yo creo que queda bastante bien, pero también me suena bien Rangos Límite (Limit Ranges). A ver que opina @electrocucaracha.
/lgtm

No estaría entendiendo cual es el Review que está faltando aprobar.
Alguna sugerencia?

/approve

No estaría entendiendo cual es el "Changes requested" que está faltando aprobar.
Alguna sugerencia?
Edit: Ok ya entendí.. el Approve como tag de un approver

Aun no se ha mandado el PR para agregarme a la lista de approvers, habra que esperar a @raelga

Publicado 🎉  ✨
Gracias @Karmavil por el contenido y @electrocucaracha por la revisión!

Gracias por el tiempo dedicado. Encantado de colaborar. Aprendí un par de cositas muy útiles.

authors: Karmavil electrocucaracha raelga


text:
[wip] Validate merge from main in to dev-1.22-hi.1

For discussion in https://github.com/kubernetes/website/issues/35749
Potential alternative to https://github.com/kubernetes/website/pull/35752

Not needed.


text:
Typo Update index.md

typo change

IMO that is not a typo and it does not need a fix.

/retitle Rewrite word in Spotlight on SIG Multicluster quote
/hold
We prefer not to edit quotes from what people have said.

I'm closing this PR as this blog post is an interview with SIG Multicluster leads
/close

authors: jihoon-seo kundan2707 reylejano sftim


text:
[zh-cn] sync blog: 2023-08-21-job-update-post.md

/lgtm

/lgtm


text:
blog: add blog post on the affect of containerization on databases

This is a blog talking about containerization and its influence on databases.

/hold
pending assignment of publication date


text:
[zh-cn] sync sidecar-containers.md

content/zh-cn/docs/concepts/workloads/pods/sidecar-containers.md

/lgtm


text:
NPM module doesn't have license

vscode-kubescape uses an NPM module called buffers, which was published to GitHub over 10 years ago and was never updated. It does not contain a license.  I imagine this is an oversight, but I have been unable to contact the author and the original repository has been deleted from GitHub.
FOSSA report
I've raised an issue with the vscode-test repo also.

This has been fixed upstream: future releases of vscode-test won't fail in this way.  Marking closed.

This is now released in @vscode/test-electron@2.3.0

Thanks @connor4312 !

repo: kubescape/vscode-kubescape


text:
Add list of Fleets to Dashboard

For each fleet show:

name
namespace

Details pane containing 3 tabs:

the CRD for the fleet
a list of APIs associated with the fleet
a list of StaticRoutes associates with the fleet

PR - https://github.com/kubeshop/kusk-gateway-dashboard/pull/19

authors: olensmar topliceanurazvan

repo: kubeshop/kusk-gateway

license: MIT


text:
change how kusk manifests are applied to the cluster

Signed-off-by: jasmingacic jasmin.gacic@gmail.com
This PR fixes how kusk manifests are installed
For some reason, unknown to me, invoking (sample below) fails without returning  error. For the same reason I switched from creating bundled manifests for the dashboard and API Server into separate files .
instCmd := NewKubectlCmd()
instCmd.SetArgs([]string{"apply", fmt.Sprintf("-f=%s", filename)})

return instCmd.Execute()

Tested it vigorously with:
 minikube delete && minikube start && go run main.go cluster install

kind delete cluster && kind create cluster && go run main.go cluster install

 k3d cluster delete && k3d cluster create && go run main.go cluster install

 k3s-uninstall.sh && curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server --no-deploy traefik" sh && go run main.go cluster install

To test it you must run VERSION=v1.2.5 make build and ./kusk cluster install  or cd cmd/kusk && go run main.go cluster install

@mbana there are instructions on how to build and how it works under-the-hood here -> https://github.com/kubeshop/kusk-gateway/pull/771
As for go-bindata in the Makefile, could you please help me add it to it.

@mbana there are instructions on how to build and how it works under-the-hood here -> https://github.com/kubeshop/kusk-gateway/blob/main/cmd/kusk/howtobuild.md
As for go-bindata in the Makefile, could you please help me add it to it.

It's probably a better idea to fix that go-bindata issue in another branch.

authors: jasmingacic mbana


text:
change filter icon when a filter is active

Sometimes I don't see any resources because I have a filter active which I have forgotten to reset - would be nice if the filter-icon somehow shows that it is currently active - perhaps with a number showing how many filters that are set? or just a highlight?

I agree that we should highlight the icon and/or show the number of filters.
We can also display a text in the navigator that says something along the lines "No resources found for the current filters"

authors: catalinhoha olensmar

repo: kubeshop/monokle


text:
feature: Create edgeManifest CRD

Feature Description
we need a way to store, via concatenation, all individual yaml content from individual workload items dropped off in the WDS.  Once in an edgeManifest object in the KubeStellar namespace, the KubeStellar syncer can pull and apply contents of edgeManifest.  This reduces complexity of the internal workflow for KubeStellar.  This also eliminates the MBS.  One fewer workspace to manage and maintain.  Removing the MBS and using the KubeStellar namespace is also a more scalable solution.
Proposed Solution
create the edgeManifest CRD which is capable of containing all individual yaml definitions for a workload that is destined for a specific location(s).
Want to contribute?

[ ] I would like to work on this issue.

Additional Context
No response

Is this intended to preserve the option for 1:many distribution? If so, how does this reduce complexity of the internal workflow for KubeStellar?

Another feature of the current design is supporting many:1 distribution. An example use case would be where an organization has a base layer of functionality they want distributed to every WEC, plus some more specialized applications that each goes to a subset of the WECs. That can be served by one EdgePlacement that directs the base layer to all the WECs, and an EdgePlacement for each specialized application.
Is the design revision here intended to keep that functionality?

We already limit the privileges of each syncer in the core, to its mailbox space. It is easy to limit privileges according to space, by namespace, and to individually configured object (changing that configuration is done by changing RBAC objects typically but there are other authorization customization techniques available such as webhooks).
If we preserve 1:many distribution and many:1 distribution then it might be easiest for the placement translator to produce one EdgeManifest object per (EdgePlacement,WEC). That would involve multiple EdgeManifest objects per WEC, that is multiple per syncer, in general --- with the collection varying dynamically. The easiest way to get the privileges limited per syncer would to put each syncer's collection in a distinct namespace.

Changing name of CRD to edgeDeployment to map back to the mental model of what a standard Kube deployment does.  This helps the user understand why this mechanism exists and how it might operate.  Our doc will satisfy details on top of that existing mental model.
As for your comments @MikeSpreitzer.  It is fine to suggest multiple namespaces. That sounds more scalable and secure.  great suggestion

@yana1205 this is what we talked about over lunch.  This would be the kubestellar equivalent to manifestwork.  The edgeDeployment CRD would hold any and all objects that are matched by the edgeplacement object

@nirrozenbaum this is the KubeStellar Syncer interface for the new pluggable placement translator you are working on.  Please work with @yana1205 to include his definition for the edgeDeployment CRD as 1 type of output that PT will support.  The other output is manifestWork.

added pluggable-transport label since this should be aligned with the pluggable transport and is very much related.

authors: MikeSpreitzer clubanderson nirrozenbaum

repo: kubestellar/kubestellar


text:
Example of storybook components usage and decription

See: #414
Example code that demonstrates how to use Storybook for different components (Button, Page, Header) and an introduction page with tests and links.
THIS IS EXAMPLE CODE - do not merge

example code, closing without merge for future use as example and documentation

authors: sgratch yaacov


text:
[Addon] Upgrade flink-kubernetes-operator version

Signed-off-by: yanghua yanghua1127@gmail.com

Description of your changes

How has this code been tested?

Checklist

I have:

[ ] Title of the PR starts with type (e.g. [Addon] , [example] or [Doc]).
[ ] Updated/Added any relevant documentation and examples.
[ ] New addon should be put in experimental.
[ ] Update addon should modify the version in metadata.yaml to generate a new version.

Verified Addon promotion rules
If this pr wants to promote an experimental addon to verified, you must check whether meet these conditions too:

[ ] This addon must be tested by addon's e2e-test to guarantee this addon can be enabled successfully.
This addon must have some basic but necessary information.

[ ] An accessible icon url and source url defined in addon's metadata.yaml.
[ ] A detail introduction include a basic example about how to use and what's the benefit of this addon in README.md.
[ ] Also provide an introduction in KubeVela documentation.
[ ] It's more likely to be accepted if useful examples provided in example dir.

@wangwang @wonderflow Please take a look when you have time.

repo: kubevela/catalog


text:
center hero block

resolve https://github.com/kubevela/kubevela.io/issues/668

cc @wonderflow


text:
CNV-51418: disable delete on running vm

📝 Description
We don't want to enable delete action on running vms
🎥 Demo

/retest

repo: kubevirt-ui/kubevirt-plugin


text:
Fixing the conditions column in vm

Signed-off-by: Dana Orr dorr@redhat.com

📝 Description
VM conditions column had wrong value.
I changed the reason for type.
And I added in the pophover if there is no message that will appear the reason.
🎥 Demo
Before:

After:

can you fix the description ?
what is the problem ?
why it happen ?
how you fixed it ?
screenshots:
"the list before fix, see the condition in red triangle has ... "
"this is the list after fix, see ..."

/approve

/lgtm

a - the after image still show the bug
b - the bug was that we show " is  false/true" instead of " is false/true" so users could not know what is wrong, for example instead of saying "ready = false" letting the user the VM is not ready, we showed "pod fail = false" , so users may thing "ahh, pod is not failing" all is OK, but what we needed to say is "vm is not ready"
c - fix is to show " is false/true" instead of " is  false/true"

/retest

/retitle Bug 2078709: VM conditions column have wrong key/values

authors: DanaOrr avivtur yaacov


text:
virtualmachine-controller  Error creating pod: Pod ,spec.affinity.nodeAffinity error

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:
/kind bug
/kind enhancement

What happened:
when i create vmi
root@miaoyun-node-4:~# kubectl get vmi
NAME       AGE   PHASE     IP    NODENAME
vm-win10   19m   Pending
root@miaoyun-node-4:~# kubectl describe vmi/vm-win10
Name:         vm-win10
Events:
Type     Reason        Age               From                       Message

Warning  FailedCreate  19m               virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-b8lgk" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  19m               virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-qvkvh" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  19m               virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-ljjzn" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  19m               virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-kdjsm" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  19m               virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-7qzpx" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  19m               virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-xtkc7" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  19m               virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-skrts" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  19m               virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-94wcc" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  19m               virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-x68ft" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  9m (x9 over 19m)  virtualmachine-controller  (combined from similar events): Error creating pod: Pod "virt-launcher-vm-win10-2wncp" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  104s              virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-tbjtn" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
Warning  FailedCreate  104s              virtualmachine-controller  Error creating pod: Pod "virt-launcher-vm-win10-svx66" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

KubeVirt version (use virtctl version):
./virtctl version
Client Version: version.Info{GitVersion:"v0.28.0", GitCommit:"340a0d45524b2861c14b276ab18bd5fb9b4ed83c", GitTreeState:"clean", BuildDate:"2020-04-02T07:18:44Z", GoVersion:"go1.12.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{GitVersion:"{gitVersion}", GitCommit:"{gitCommit}", GitTreeState:"{gitTreeState}", BuildDate:"{buildDate}", GoVersion:"go1.13.14", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes version (use kubectl version):
kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:48:36Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

VM or VMI specifications:
apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
labels:
kubevirt.io/vm: vm-win10
name: vm-win10
spec:
runStrategy: Manual
template:
metadata:
labels:
kubevirt.io/vm: vm-win10
spec:
domain:
resources:
requests:
memory: 4096M
devices:
disks:
- name: containerdisk
disk:
bus: virtio
- name: emptydisk
disk:
bus: virtio
terminationGracePeriodSeconds: 60
volumes:
- name: containerdisk
containerDisk:
image: 10.221.101.125:5000/vmdisks/win10:latest
- name: emptydisk
emptyDisk:
capacity: "2Gi"

Cloud provider or hardware configuration:

OS (e.g. from /etc/os-release):
NAME="Ubuntu"
VERSION="16.04.6 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.6 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

Kernel (e.g. uname -a):
Linux miaoyun-node-4 4.15.0-74-generic #83~16.04.1-Ubuntu SMP Wed Dec 18 04:56:23 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Install tools:

Others:

virt-controller  logs
{"component":"virt-controller","level":"info","msg":"reenqueuing VirtualMachineInstance default/vm-win10","pos":"vmi.go:204","reason":"failed to create virtual machine pod: Pod "virt-launcher-vm-win10-fwp8v" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'","service":"http","timestamp":"2020-11-06T03:04:01.747330Z"}
{"component":"virt-controller","level":"info","msg":"reenqueuing VirtualMachineInstance default/vm-win10","pos":"vmi.go:204","reason":"failed to create virtual machine pod: Pod "virt-launcher-vm-win10-2kffh" is invalid: spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].values: Required value: must be specified when operator is 'In' or 'NotIn'","service":"http","timestamp":"2020-11-06T03:20:41.849628Z"}

repo: kubevirt/kubevirt


text:
"RerunOnFailure" VM will stay stopped after eviction

What happened:
For a VM that is using "RerunOnFailure", if the virt-launcher pod get evicted (e.g. node drain), the VM will get shutdown but never come back online.
This also happens when explicitly delete the virt-launcher pod.
What you expected to happen:
tl;dr: The VM should stay running after Pod eviction.
From the doc: A VirtualMachineInstance will be respawned if the previous instance failed in an error state. It will not be re-created if the guest stopped successfully (e.g. shut down from inside guest).
IMO this feature is very useful to support "user explicitly want the VM guest to be down" comparing to "Always". However, we should get an exception for Pod eviction case which the guest stop is not initiated by user.
How to reproduce it (as minimally and precisely as possible):

Set VM runStrategy  as "RerunOnFailure"
Delete VM virt-launcher pod
Observe the VMI is in "Succeeded" phase under VMI.Status.Phase
Observe virt-launcher pod get never recreated

Additional context:
This issue doesn't seem to apply when using live migration as eviction strategy.
My understanding of what is happening after virt-launcher pod get evicted or deleted:

SIGTERM send inside to virt-launcher
Pod maybe get graceful or ungraceful shutdown.
VMI.Status.Phase will report to be "Succeeded": https://github.com/kubevirt/kubevirt/blob/cbe6f662786b5c5bfd872762019310d75c4fab19/pkg/virt-handler/vm.go#L3144
VMI controller will NOT recreate the virt-launcher pod: https://github.com/kubevirt/kubevirt/blob/9d68c4bdfcffb1d2d7015530ad2d341664d60364/pkg/virt-controller/watch/vmi.go#L1153
VM controller will not recreate VMI because of "RerunOnFailure": https://github.com/kubevirt/kubevirt/blob/9d68c4bdfcffb1d2d7015530ad2d341664d60364/pkg/virt-controller/watch/vm.go#L741

cc. @rmohr

Yes, I think that makes sense.
We can probably re-use the live-migration eviction status notification to indicate also for non-live-migrate cases that the VMI changed the state because of an eviction and let the VM controller react accordingly.

The issue should still be there.


text:
Kubevirt ui metrics gaps ci debug

DONTMERGE - troubleshoot helper for https://github.com/kubevirt/kubevirt/pull/2077
will close once we properly figure out what's going on

ci test please

Turns out it was all caused by a silly mistake I made: https://github.com/kubevirt/kubevirt/pull/2077/commits/522c6f6804bbfaf8b183f2c2bf325dcf3c5077f4