text:
[typescript/admin] eslintの更新

該当イシュー
なし
外部仕様の変更点
(フロントエンド の人に説明する必要がある場合)
内部仕様の変更点

eslintrcへのルールの追加

コードの書き方の統一

@typescript-eslint/array-type ... 配列の型の書き方をT[]の形式に統一(Array<T>は禁止)
use-isnan ... NaNのチェックにはisNaN()を用いる
curly ... 制御文では中括弧を用いる(if (foo) foo++;のような書き方は禁止)
comma-spacing ... カンマの直前のスペースは禁止し、また直後にはスペースをつけるようにする

❌[1 ,2,3]
⭕[1, 2, 3]

コードの読みやすさ、安全性などの改善

@typescript-eslint/explicit-function-return-type ... 関数の戻り値の型を明示的に記述させる
@typescript-eslint/no-explicit-any ... anyを禁止(代替としてはunknownの使用を推奨)
github/array-foreach ... forEach()の使用禁止(代替としてはfor...ofの使用を推奨)
no-fallthrough ... case構文でのフォールスルーを禁止(各case内では必ず最後にreturnかbreakを使用する)
no-warning-comments ... 先頭に"todo"などとついているコメントは残さないようにする(一時的には利用するコメントなので、"error"ではなく"warn"設定)
max-statements ... 1つの関数内で記述できる構文の数を10以内に制限

要検討(現状は"warn"で設定)
1つの関数が肥大化するのを避けるために活用できそう

no-negated-condition ... else句を持つif文での否定構文を禁止(if(!condition) { ... } else { ... }は、if文で否定構文を使っているので禁止)

動作を保証するためのテストケースの詳細

rspecのテストケース
エンドツーエンドテストケース

@minminzemi
それぞれルールについてコメントします！

@typescript-eslint/array-type ... 配列の型の書き方をT[]の形式に統一(Arrayは禁止)
完全に宗教の域なんですが、個人的にはArrayの書き方がそこそこ好きでちょいちょい使ってるんですけど、理性的に考えるとやはり統一したほうがいいですよね、、。

use-isnan ... NaNのチェックにはisNaN()を用いる
agree

curly ... 制御文では中括弧を用いる(if (foo) foo++;のような書き方は禁止)
関数内で例外処理などを記述する際、いくつかの条件で早期 return したいときに

func() {
    if (arg) return
    if (arg2) return
}

とかけるのが見易さ的にメリットが大きいなと思っていたのですが、デメリットがどの辺にあるか考えがあればコメントください！

comma-spacing ... カンマの直前のスペースは禁止し、また直後にはスペースをつけるようにする
agree

@typescript-eslint/explicit-function-return-type ... 関数の戻り値の型を明示的に記述させる
@typescript-eslint/no-explicit-any ... anyを禁止(代替としてはunknownの使用を推奨)
agree

github/array-foreach ... forEach()の使用禁止(代替としてはfor...ofの使用を推奨)
僕もforEachあまり使わないし、いいとは思うのですが、これって今非推奨なんでしたっけ？

no-warning-comments ... 先頭に"todo"などとついているコメントは残さないようにする(一時的には利用するコメントなので、"error"ではなく"warn"設定)
max-statements ... 1つの関数内で記述できる構文の数を10以内に制限

要検討(現状は"warn"で設定)
1つの関数が肥大化するのを避けるために活用できそう
no-negated-condition ... else句を持つif文での否定構文を禁止(if(!condition) { ... } else { ... }は、if文で否定構文を使っているので禁止)
agree

@kouhaa
各ルールに対して丁寧にコメントありがとうございます！

完全に宗教の域なんですが、個人的にはArrayの書き方がそこそこ好きでちょいちょい使ってるんですけど、理性的に考えるとやはり統一したほうがいいですよね、、。

これに関しては宗教の問題なのは確かにそうですね、、:thinking:
自分としてはどちらでも良いのですが、とにかく統一はしたいなという気持ちがあるので、フロントエンドエンジニア間で議論して決めても良いですし多数決でもなんでも良いかなと思っています！(現状"[]"に統一という形にしようとしているのは単純に自分がこれまでその書き方だったからですね)
以下、Array<T>とT[]の書き方それぞれに対する個人的な考えです(参考までに)

Array

「型引数に渡したもの」の配列を表すので、Tがnumber | nullなどのようなUnion型だった場合、T[]の場合と違い"()"が不要で、そのままnumber | nullを渡せる

"T[]"の形式だと、(number | null)[]と書く必要がある

多重配列を表す場合Array<Array<T>>となり、T[][]よりは直感的に「配列の配列」

T[]

単純に記述する文字数が少ない
直感的に配列を表しているとわかる

Arrayの場合、他のジェネリクス型と書き方が同じなので"Array"という文字列に注目しないと配列のジェネリクスであることがわからない

@kouhaa
各ルールに対して丁寧にコメントありがとうございます！

完全に宗教の域なんですが、個人的にはArrayの書き方がそこそこ好きでちょいちょい使ってるんですけど、理性的に考えるとやはり統一したほうがいいですよね、、。

これに関しては宗教の問題なのは確かにそうですね、、:thinking:
どちらにしても、可読性の意味も込みでとにかく統一はしたいなという気持ちがあるので、フロントエンドエンジニア間で議論して決めても良いですし多数決でもなんでも良いかなと思っています！
以下、Array<T>とT[]の書き方それぞれに対する個人的な考えです(参考までに)

Array

「型引数に渡したもの」の配列を表すので、Tがnumber | nullなどのようなUnion型だった場合、T[]の場合と違い"()"が不要で、そのままnumber | nullを渡せる

"T[]"の形式だと、(number | null)[]と書く必要がある

T[]

省略記法である故に、単純に記述する文字数が少ない
直感的に配列を表しているとわかる

Arrayの場合、他のジェネリクス型と書き方が同じなので"Array"という文字列に注目しないと配列のジェネリクスであることがわからない

curlyについて

確かに、例外処理などでreturnだけするための制御文を書くことが自分も多々あるので、その一点においては括弧を省略して1行で書く書き方は、手間としても読みやすさとしても良いとは思いましたが、同時に以下のようなコードも許可してしまうことになり、それらは読みづらくバグの原因にもなり得ると思っているので禁止にしたいなと思いました:bow:
if (foo) foo++;
else doSomething();

if (foo) foo++; else doSomething();

if (foo)
  foo++;

forEach()について

これまでにみてきたforEach()を使っている場面では大抵、map()、reduce()、find()、filter()が有効かつこちら方が適切であり、そうでなくともfor...ofの方が機能的に優れていると思うからですね
for...ofでは、break、continue、returnの制御ができますし、forEach()はArrayに生えているメソッドで配列に対してしか使用できないのに対し、for...ofではイテラブルなものであれば何でも使用できるので、よりフレキシブルな方へ統一する意味も込めて、こちらに合わせたいなと思いました！
(正直、以前偶然見つけたこちらの記事に多分に影響された結果でもあります笑)

@minminzemi
Arrayの整理ありがとうございます！
確かにそう見ると、それぞれメリットがありますね、。
個人的にはどちらかに統一するなら genericなんですけど、調べてみたらいいとこ取りの設定が一応できそうですね。
https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/docs/rules/array-type.md
array-simple どうですかね？

curly

なるほど弾けないパターンがあるんですね。
個人的にはこれ
if (foo) foo++;
else doSomething();

は大丈夫で、
これ
if (foo)
  foo++;

は絶対バグるから嫌って考えてるんですけど(これだけ弾く設定は確かできます。)、
これ
if (foo) foo++; else doSomething();

超気持ち悪いですね。これ弾く方法なさそうなので、やっぱり curly: errorでいきましょう、。

forEach()について

完全に合理性が優ってますね。この設定でいきましょう！

@kouhaa

array-simple

これめっちゃいいですね！array-simpleで設定しようと思います！！
表現の統一はできませんが、eslint側で決められたルールに則って使い分けるので、無秩序というわけでもなく良い感じかなと思います！

curlyについて

意見ありがとうございます！
確かに一部の変な書き方を弾くルールはあるのですが、やはり抜け穴があって完全とは言えなさそうなので一旦"error"の設定にしておきます:bow:

authors: kouhaa minminzemi

repo: ispec-inc/monorepo


text:
I just can't get it to work...

I installed the toolchain, went to the directory containing esp-ginx and entered 'make'
And this is what I got: http://pastebin.com/fmnqZrC4
Scroll to the bottom - I have highlighted the errors - please be gentle, I'm a novice.
On Mac OS X Yosemite.

You gcc is treating this cast warnings as errors. Odd cause here it's just fine.
Anyway my last commit should fix it. Please let me know.

Looks like it worked: http://pastebin.com/Qu9draix

Now all I need to know is what do I do with those 4 bin files?

You just need to exec 'make flash' now.
Basically you will upload the 0x00000.bin and 0x10000.bin files to the ESP.
If you change something in the html files, exec "make html" before "make" and then "make flash"

source: gharchive/issue

repo: israellot/esp-ginx

license: mit

license_type: permissive

license_source: bigquery


text:
build: try to reproduce Windows issue

see if we can reproduce Windows issue.

Coverage remained the same at 97.845% when pulling c78e7565885ccb0617596173c9c132e4828b5a7e on windows-issue into fc1bbbf490f6ab0272359ce10ceb4987d1716256 on master.

source: gharchive/pull-request

authors: bcoe coveralls

repo: istanbuljs/nyc

license: isc


text:
Changing lat/lng from using CSS top/left to using CSS transforms

I've substituted top/left with CSS transforms in my fork of this project and it seems to work well, except for the moment the user unclicks their mouse after clicking and dragging the map to move it around. It seems there's a rerender happening on all markers at that moment which has them snap to a position that's related to where the map was dragged to, then they assume the correct position. Because I'm using CSS transitions too though, this movement after the initial snap is animated, which is not desired.
What could explain this behavior? How might I go about fixing this undesirable snapping and subsequent adjustment to the correct position for my markers? Because this snapping doesn't happen when I disable CSS transitions, I think this brief position issue is actually present in the master branch that I forked from, it just isn't noticeable because of how quickly it occurs using top/left and the only reason I'm noticing it is because I'm using CSS transitions on the CSS transforms I'm using instead of top/left, which together slows down and animates the movement so that it's noticeable.
For clarity, here are the changes I made to accomplish this so far (it didn't take much):

https://github.com/jamespizzurro/google-map-react/commit/5b19333106174956b69af4ad91a1ad67164b06c0
https://github.com/jamespizzurro/google-map-react/commit/4b89ca34210513ff8b2b38245251e415281d90ea
https://github.com/jamespizzurro/google-map-react/commit/8d5cf4af9e86d84b5f129cdea2d8d5b8bd18c0fb

Thanks in advance for any help.

Update: the initial position snapping does not occur until after the map has stopped moving after a click and drag, so if there's some left over movement from a "swipe" (a click and a drag that results in the map still moving after the unclick), the snap happens after the map position stops changing.

I've substituted top/left with CSS transforms in my fork of this project and it seems to work wel

Why?

To animate my markers, all of which are subject to moving around based on their current GPS position, which changes every few seconds. CSS seems like the way to go for this.

Every few seconds is not a real time animation, you will see no performance benefit using css transform over left right, and will get a lot of issues and browsers bugs using translate.
I'm pretty sure all the performance problem you have is not based on left top change.
IMO if you need a big amount of realtime animated markers you need something like this https://github.com/istarkov/google-map-thousands-markers
And all that css transform gives a little performance benefit.

Gotcha, thanks, I'll definitely check out that project. Just to clarify though, this isn't a performance problem: the "snapping" appears to occur because, for a brief moment, the wrong position is being fed to each marker, which is then subsequently corrected. I was hoping to get some clarity on this mechanism and maybe some ideas of why this might be happening. If you don't know, that works too. :p

All this project is a big hack over the black box (google map api) it can happen for enormous amount of reasons. Some hacks does not solve the problem of unsynchronised content - just reduce the probability to see it - like double raf inside the source ;-)
Two years ago when I wrote it I think I had knowledge about everything you see, but now not ;-)

Totally understand, haha.
I did manage to get this working with these two additional commits:

https://github.com/jamespizzurro/google-map-react/commit/301c1d9b39c1768543dbd813482cba252529730c
https://github.com/jamespizzurro/google-map-react/commit/40b6012ce3979b14bcc0108b58c5c53dd7fd0dc9

There's probably a performance hit to these changes or something bad, but it works. :100:

can you help me to re-position the marker? I have found something in the documentation about it? But the link is not valid
https://github.com/istarkov/google-map-react/blob/master/DOC.md
The section "Whats wrong with "Simple example" above" says about it but the link is invalid?

@thientran1707 I am taking care of the DOC links in #549, sorry about the inconvenience.
@jamespizzurro You can just add transform: translate(-50%, -50%); in css to your marker component and that would do the trick. Let me know if that helps.

authors: istarkov itsmichaeldiego jamespizzurro thientran1707

repo: istarkov/google-map-react

license: MIT

license_source: github-api


text:
Consider elevating "Installing Istio"  in /docs

Issue with _docs/index.md
hackathon feedback
Installing istio is on a page linked via 2nd bullet (tasks) on the main page.
User needs to look around to find it.

strongly +1 this one - "Tasks" isn't an intuitive way to get started

is good enough for now

authors: ldemailly mandarjog


text:
Multicluster setup issue

Bug description
I have two EKS clusters in private subnets in EKS. I am trying to connect them via steps mentioned in https://istio.io/docs/setup/kubernetes/install/multicluster/gateways/. Installation looks ok but when i try to connect cross cluster httpbin service i get following
kubectl exec --context=$CTX_CLUSTER1 $SLEEP_POD -n foo -c sleep -- curl -I httpbin.bar.global:8000/headers
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (52) Empty reply from server
command terminated with exit code 52

This is seen on istiocoredns
2019-07-29T22:28:49.022180Z	info	Have 1 service entries
2019-07-29T22:28:49.022222Z	info	adding DNS mapping: httpbin.bar.global.->[127.255.0.3]
2019-07-29T22:28:53.612113Z	info	DNS query  ;; opcode: QUERY, status: NOERROR, id: 61096
;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;httpbin.bar.global.global.	IN	 AAAA
2019-07-29T22:28:53.612195Z	info	Could not find the service requested
2019-07-29T22:28:53.612232Z	info	DNS query  ;; opcode: QUERY, status: NOERROR, id: 60888
;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;httpbin.bar.global.global.	IN	 A
2019-07-29T22:28:53.612254Z	info	Query A record: httpbin.bar.global.global.->{httpbin.bar.global.global. 1 1}
2019-07-29T22:28:53.612264Z	info	Could not find the service requested
2019-07-29T22:28:53.613557Z	info	DNS query  ;; opcode: QUERY, status: NOERROR, id: 10366
;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;httpbin.bar.global.	IN	 AAAA
2019-07-29T22:28:53.613589Z	info	Could not find the service requested
2019-07-29T22:28:53.613655Z	info	DNS query  ;; opcode: QUERY, status: NOERROR, id: 10225
;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;httpbin.bar.global.	IN	 A
2019-07-29T22:28:53.613704Z	info	Query A record: httpbin.bar.global.->{httpbin.bar.global. 1 1}
2019-07-29T22:28:53.613711Z	info	Found httpbin.bar.global.->[127.255.0.3]
Any pointers what i could be doing wrong or what logs should i be looking into.
Affected product area (please put an X in all that apply)
[ ] Configuration Infrastructure
[ ] Docs
[X] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Expected behavior
Cross cluster connection should be established.
Steps to reproduce the bug
Version (include the output of istioctl version --remote and kubectl version)
istioctl version --remote
client version: 1.2.2
citadel version: master-20190725-09-15
egressgateway version: master-20190725-09-15
galley version: master-20190725-09-15
ingressgateway version: master-20190725-09-15
pilot version: master-20190725-09-15
policy version: master-20190725-09-15
sidecar-injector version: master-20190725-09-15
telemetry version: master-20190725-09-15
kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-20T04:49:16Z", GoVersion:"go1.12.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.7-eks-c57ff8", GitCommit:"c57ff8e35590932c652433fab07988da79265d5b", GitTreeState:"clean", BuildDate:"2019-06-07T20:43:03Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
How was Istio installed?
Istio installed via steps mentioned in https://istio.io/docs/setup/kubernetes/install/multicluster/gateways/
Environment where bug was observed (cloud vendor, OS, etc)
EKS
Additionally, please consider attaching a cluster state archive by attaching
the dump file to this issue.

@rajeshneo, are you still seeing this problem? If so, can you re-post on discuss.istio.io with the latest details?

authors: ayj rajeshneo

repo: istio/istio


text:
Istio 1.2.4: Response bytes size changes between istio ingress and upstream service

Bug description
Hey all, I'm stuck with one issue, and I hope you guys can help me with figuring out this issue. We have an API endpoint that returns PDF as steam response, and we have observed that response byte size occasionally gets smaller between istio ingress and upstream service. This is corrupting PDF. It only happens sometime for the same API endpoint. Here is the diagram that summarizes the issue.

You can also confirm this from these logs.
Logs from istio-proxy in istio ingress

Logs from istio-proxy in upstream service pod

As you can see, istio proxy ingress receives the different response size even though upstream service receives the same response size from the further upstream service.
Here is my gateway configuration and service virtual service and destination rule configuration.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mygateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 80
      name: http2
      protocol: HTTP2
    hosts:
    - app.test.com
    tls:
      httpsRedirect: true
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-certs/tls.key
    hosts:
    - app.test.com

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: test
spec:
  hosts:
  - app.test.com
  gateways:
  - mygateway
  http:
  - route:
    - destination:
        port:
          number: 80
        host: test.default.svc.cluster.local
    websocketUpgrade: true

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
 name: test
spec:
 host: test.default.svc.cluster.local
 trafficPolicy:
   loadBalancer:
     consistentHash:
       useSourceIp: true

Affected product area (please put an X in all that apply)
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[x ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Expected behavior
Steps to reproduce the bug
Version (include the output of istioctl version --remote and kubectl version)
1.2.4
How was Istio installed?
Using helm chart
Environment where bug was observed (cloud vendor, OS, etc)
Google Cloud Kubernetes Engine 1.13.7-gke.8

Hi, mkinoshi -- Do you have an idea of how often bytes are being dropped?

@oaktowner We have observed that it happens around once in every 5 API request to the same endpoint.

authors: mkinoshi oaktowner


text:
httpsRedirect breaks cert-manager acme challenge, which was working in Istio 1.6

Bug description
After upgrading from 1.6.2 to 1.7.2 then cert-manager acme challenges failed to complete. I finally worked out that if I deleted the istio gateway resource for that host then the certificate request completed.
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-istio
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: hostmaster@example.com
    privateKeySecretRef:
      name: letsencrypt-istio
    solvers:
    - selector: {}
      http01:
        ingress:
          class: istio

The order, challenge and ingress are all created and the acme pod starts but it just never completes.
[ ] Docs
[ ] Installation
[X] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[X] User Experience
[ ] Developer Infrastructure
Expected behavior
Certificate requests complete successfully as they did in version 1.6.2
Version (include the output of istioctl version --remote and kubectl version --short and helm version if you used Helm)
$ istioctl version --remote
client version: 1.7.2
control plane version: 1.7.2
data plane version: 1.7.2 (57 proxies)
$ kubectl version --short
Client Version: v1.17.3
Server Version: v1.17.3
How was Istio installed?
istioctl upgrade -f istio-operator.yaml
Environment where bug was observed (cloud vendor, OS, etc)
Kubernetes 1.17.3 on-premise on Centos 7

Update
I downgraded to version 1.6.11 and the letsencrypt challenges started succeeding again.
We also use istio multi cluster for certificates from a central vault pod and this wasn't working in version 1.17 either. I didn't mention it before as I thought it was a vault issue but as soon as I downgraded istio then it started working again.
Thanks,
Sarah

@sazzle2611 can you possibly post the Gateway configuration you have that is conflicting? We do tests this combination in general (cert-manager acme challenge) so it must be specific to the config you are using. Would be good to get a config dump from the gateway as well to avoid back and forth: https://github.com/istio/istio/wiki/Troubleshooting-Istio. Thanks!

Hi @howardjohn
This is the config yaml that we use for one of our k8s clusters (the istio-operator.yaml file that we use to install and update istio)
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  addonComponents:
    istiocoredns:
      enabled: true
      k8s:
        strategy:
          type: RollingUpdate
          rollingUpdate:
            maxSurge: 1
            maxUnavailable: 1
        hpaSpec:
          minReplicas: 2
          maxReplicas: 3
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - istiocoredns
              topologyKey: kubernetes.io/hostname
            preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - istiocoredns
                topologyKey: failure-domain.beta.kubernetes.io/zone
    prometheus:
      enabled: false
    grafana:
      enabled: false
    kiali:
      enabled: false
    tracing:
      enabled: true

  components:
    pilot:
      k8s:
        strategy:
          type: RollingUpdate
          rollingUpdate:
            maxSurge: 1
            maxUnavailable: 1
        hpaSpec:
          minReplicas: 2
          maxReplicas: 3
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - istiod
              topologyKey: kubernetes.io/hostname
            preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - istiod
                topologyKey: failure-domain.beta.kubernetes.io/zone
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
      k8s:
        strategy:
          type: RollingUpdate
          rollingUpdate:
            maxSurge: 1
            maxUnavailable: 1
        hpaSpec:
          minReplicas: 2
          maxReplicas: 3
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - istio-ingressgateway
              topologyKey: kubernetes.io/hostname
            preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - istio-ingressgateway
                topologyKey: failure-domain.beta.kubernetes.io/zone
        service:
          ports:
          - name: status-port
            port: 15021
            targetPort: 15021
          - name: http2
            port: 80
            targetPort: 8080
          - name: https
            port: 443
            targetPort: 8443
          - name: tls
            port: 15443
            targetPort: 15443
          - name: tcp-redis
            port: 6379
            targetPort: 6379
          loadBalancerIP: 192.168.28.123
    - name: wx5-ingressgateway
      enabled: true
      label:
        app: wx5-ingressgateway
      k8s:
        strategy:
          type: RollingUpdate
          rollingUpdate:
            maxSurge: 1
            maxUnavailable: 1
        hpaSpec:
          minReplicas: 2
          maxReplicas: 3
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - wx5-ingressgateway
              topologyKey: kubernetes.io/hostname
            preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - wx5-ingressgateway
                topologyKey: failure-domain.beta.kubernetes.io/zone
            - weight: 10
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - wx9-ingressgateway
                    - wx10-ingressgateway
                topologyKey: kubernetes.io/hostname
        service:
          ports:
          - name: status-port
            port: 15021
            targetPort: 15021
          - name: http2
            port: 80
            targetPort: 8080
          - name: https
            port: 443
            targetPort: 8443
          - name: tls
            port: 15443
            targetPort: 15443
          - name: tcp-sftp
            port: 51051
            protocol: TCP
            targetPort: 51051
          loadBalancerIP: 192.168.28.120
          externalTrafficPolicy: Local
    - name: wx9-ingressgateway
      enabled: true
      label:
        app: wx9-ingressgateway
      k8s:
        strategy:
          type: RollingUpdate
          rollingUpdate:
            maxSurge: 1
            maxUnavailable: 1
        hpaSpec:
          minReplicas: 2
          maxReplicas: 3
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - wx9-ingressgateway
              topologyKey: kubernetes.io/hostname
            preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - wx9-ingressgateway
                topologyKey: failure-domain.beta.kubernetes.io/zone
            - weight: 10
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - wx5-ingressgateway
                    - wx10-ingressgateway
                topologyKey: kubernetes.io/hostname
        service:
          ports:
          - name: status-port
            port: 15021
            targetPort: 15021
          - name: http2
            port: 80
            targetPort: 8080
          - name: https
            port: 443
            targetPort: 8443
          - name: tls
            port: 15443
            targetPort: 15443
          - name: tcp-sftp
            port: 51051
            protocol: TCP
            targetPort: 51051
          - name: tcp-sftp-mfi
            port: 51052
            protocol: TCP
            targetPort: 51052
          - name: tcp-gflistener
            port: 10052
            protocol: TCP
            targetPort: 10052
          loadBalancerIP: 192.168.28.128
          externalTrafficPolicy: Local
    - name: wx10-ingressgateway
      enabled: true
      label:
        app: wx10-ingressgateway
      k8s:
        strategy:
          type: RollingUpdate
          rollingUpdate:
            maxSurge: 1
            maxUnavailable: 1
        hpaSpec:
          minReplicas: 2
          maxReplicas: 6
        resources:
          limits:
            cpu: "2"
            memory: 1Gi
          requests:
            cpu: 200m
            memory: 156Mi
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - wx10-ingressgateway
              topologyKey: kubernetes.io/hostname
            preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - wx10-ingressgateway
                topologyKey: failure-domain.beta.kubernetes.io/zone
        service:
          ports:
          - name: status-port
            port: 15021
            targetPort: 15021
          - name: http2
            port: 80
            targetPort: 8080
          - name: https
            port: 443
            targetPort: 8443
          - name: tls
            port: 15443
            targetPort: 15443
          - name: tcp-sftp
            port: 51051
            protocol: TCP
            targetPort: 51051
          loadBalancerIP: 192.168.28.122
    egressGateways:
    - name: istio-egressgateway
      enabled: true
      k8s:
        strategy:
          type: RollingUpdate
          rollingUpdate:
            maxSurge: 1
            maxUnavailable: 1
        hpaSpec:
          minReplicas: 2
          maxReplicas: 3
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - istio-egressgateway
              topologyKey: kubernetes.io/hostname
            preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - istio-egressgateway
                topologyKey: failure-domain.beta.kubernetes.io/zone
        service:
          ports:
          - name: http2
            port: 80
            targetPort: 8080
          - name: https
            port: 443
            targetPort: 8443
          - name: tls
            port: 15443
            targetPort: 15443

  values:
    global:
      prometheusNamespace: monitoring
      # Provides dns resolution for global services
      podDNSSearchNamespaces:
        - global
      multiCluster:
        enabled: true
      controlPlaneSecurityEnabled: true
    meshConfig:
      accessLogFile: '/dev/stdout'
    sidecarInjectorWebhook:
      neverInjectSelector:
      - matchExpressions:
        - key: app
          operator: In
          values:
          - csi-rbdplugin-provisioner
          - rook-ceph-crashcollector
          - rook-ceph-detect
          - rook-ceph-detect-version
          - rook-ceph-mgr
          - rook-ceph-mon
          - rook-ceph-operator
          - rook-ceph-osd
          - rook-ceph-osd-prepare
          - rook-ceph-tools
          - rook-detect
          - rook-discover
    gateways:
      istio-ingressgateway:
        sds:
          enabled: true
      istio-egressgateway:
        env:
          # Needed to route traffic via egress gateway if desired.
          ISTIO_META_REQUESTED_NETWORK_VIEW: "external"

I'm not totally sure what config dump you are asking for. Is it for one our gateways or for the istio-ingressgateway. Also we have downgraded all our clusters back to 1.6 where letsencypt works so are you wanting a dump of a 1.7 gateway?
We have been intending to set up a new k8s cluster so I'm happy to do that this weekend and give you any info that you need if that helps?
Thanks,
Sarah

I finally worked out that if I deleted the istio gateway resource for that host then the certificate request completed.
I am looking for the Gateway that you needed to delete

Ah sorry, this is one of them, from a different k8s cluster but they were all the same
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: tpcoding
  namespace: istio-system
spec:
  selector:
    app: wx11-ingressgateway
    istio: ingressgateway
  servers:
  - hosts:
    - tpcoding.com
    - www.tpcoding.com
    port:
      name: http
      number: 80
      protocol: HTTP
    tls:
      httpsRedirect: true
  - hosts:
    - tpcoding.com
    - www.tpcoding.com
    port:
      name: https
      number: 443
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: tpcoding-tls

Sorry to bump an old issue but this seems to be the same issue I'm having. On 1.6 cert-manager was able to connect to the ACME challenge pod and on 1.7 and 1.8 it is not.
What I have learned is this seems to come down to httpsRedirect being enabled. When I have that enabled and attempt to hit the "well-known" ingress:
$ curl -v -HHost:www.mydomain.com "192.168.1.250/.well-known/acme-challenge/21RZvQxYINfJcl8HBOnU92IpALjh05XrRs357F71LhQ"
*   Trying 192.168.1.250...
* TCP_NODELAY set
* Connected to 192.168.1.250 (192.168.1.250) port 80 (#0)
> GET /.well-known/acme-challenge/21RZvQxYINfJcl8HBOnU92IpALjh05XrRs357F71LhQ HTTP/1.1
> Host:www.mydomain.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< location: https://www.mydomain.com/.well-known/acme-challenge/21RZvQxYINfJcl8HBOnU92IpALjh05XrRs357F71LhQ
< date: Sun, 03 Jan 2021 00:09:25 GMT
< server: istio-envoy
< content-length: 0

Once I update the gateway spec I get this:
$ curl -v -HHost:www.mydomain.com "192.168.1.250/.well-known/acme-challenge/21RZvQxYINfJcl8HBOnU92IpALjh05XrRs357F71LhQ"
*   Trying 192.168.1.250...
* TCP_NODELAY set
* Connected to 192.168.1.250 (192.168.1.250) port 80 (#0)
> GET /.well-known/acme-challenge/21RZvQxYINfJcl8HBOnU92IpALjh05XrRs357F71LhQ HTTP/1.1
> Host:www.mydomain.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< cache-control: no-cache, no-store, must-revalidate
< date: Sun, 03 Jan 2021 00:11:31 GMT
< content-length: 87
< content-type: text/plain; charset=utf-8
< x-envoy-upstream-service-time: 1
< server: istio-envoy

As a workaround I have been toggling that flag on all my gateways when I need to renew. Given I have a lot of domains this has been a pretty serious regression for me.

Sorry to bump an old issue but this seems to be the same issue I'm having. On 1.6 cert-manager was able to connect to the ACME challenge pod and on 1.7 and 1.8 it is not.
What I have learned is this seems to come down to httpsRedirect being enabled. When I have that enabled and attempt to hit the "well-known" ingress:
$ curl -v -HHost:www.mydomain.com "192.168.1.250/.well-known/acme-challenge/21RZvQxYINfJcl8HBOnU92IpALjh05XrRs357F71LhQ"
*   Trying 192.168.1.250...
* TCP_NODELAY set
* Connected to 192.168.1.250 (192.168.1.250) port 80 (#0)
> GET /.well-known/acme-challenge/21RZvQxYINfJcl8HBOnU92IpALjh05XrRs357F71LhQ HTTP/1.1
> Host:www.mydomain.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< location: https://www.mydomain.com/.well-known/acme-challenge/21RZvQxYINfJcl8HBOnU92IpALjh05XrRs357F71LhQ
< date: Sun, 03 Jan 2021 00:09:25 GMT
< server: istio-envoy
< content-length: 0

Once I update the gateway spec I get this:
$ curl -v -HHost:www.mydomain.com "192.168.1.250/.well-known/acme-challenge/21RZvQxYINfJcl8HBOnU92IpALjh05XrRs357F71LhQ"
*   Trying 192.168.1.250...
* TCP_NODELAY set
* Connected to 192.168.1.250 (192.168.1.250) port 80 (#0)
> GET /.well-known/acme-challenge/21RZvQxYINfJcl8HBOnU92IpALjh05XrRs357F71LhQ HTTP/1.1
> Host:www.mydomain.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< cache-control: no-cache, no-store, must-revalidate
< date: Sun, 03 Jan 2021 00:11:31 GMT
< content-length: 87
< content-type: text/plain; charset=utf-8
< x-envoy-upstream-service-time: 1
< server: istio-envoy

As a workaround I have been toggling that flag on all my gateways when I need to renew. Given I have a lot of domains this has been a pretty serious regression for me.

@SeanWallace what Gateway configuration do you have? Can you post the exact thing? Working on a fix for this and want to make sure i fully understand the problem

@SeanWallace what Gateway configuration do you have? Can you post the exact thing? Working on a fix for this and want to make sure i fully understand the problem

My understanding is in 1.6 it was actually broken such that when cert-manager created the Ingress for the challenge, httpsRedirect actually stopped working for all requests. In 1.7, this was "fixed", and now it requires TLS on all requests to www.mydomain.com. I think the distinction here is we would want the redirect only on things other than .well-known/acme-challenge/... but I am not sure Envoy supports that currently

My understanding is in 1.6 it was actually broken such that when cert-manager created the Ingress for the challenge, httpsRedirect actually stopped working for all requests. In 1.7, this was "fixed", and now it requires TLS on all requests to www.mydomain.com. I think the distinction here is we would want the redirect only on things other than .well-known/acme-challenge/... but I am not sure Envoy supports that currently

I think to do it we would need to stop using RequireTLS in envoy, and instead use route_action.https_redirect and a match on :scheme. This is not only a big change, but its broken in envoy currently: https://github.com/envoyproxy/envoy/issues/14587. So would likely be a while until fixed.

I think to do it we would need to stop using RequireTLS in envoy, and instead use route_action.https_redirect and a match on :scheme. This is not only a big change, but its broken in envoy currently: https://github.com/envoyproxy/envoy/issues/14587. So would likely be a while until fixed.

Your theory sounds correct to me just based on my observation.
Here is my fully Gateway config:
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: website-mydomain-com
  namespace: mine
spec:
  selector:
    istio: istio-ingressgateway-mine
  servers:
  - hosts:
    - mydomain.com
    - www.mydomain.com
    port:
      name: http
      number: 80
      protocol: HTTP
    tls:
      httpsRedirect: true
  - hosts:
    - mydomain.com
    - www.mydomain.com
    port:
      name: https
      number: 443
      protocol: HTTPS
    tls:
      credentialName: website-mydomain-com-cert
      minProtocolVersion: TLSV1_2
      mode: SIMPLE
      privateKey: sds
      serverCertificate: sds

And in case it's useful the ingress that was generated by cert-manager:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: istio
    nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0,::/0
  creationTimestamp: "2021-01-04T02:08:56Z"
  generateName: cm-acme-http-solver-
  generation: 1
  labels:
    acme.cert-manager.io/http-domain: "2332887661"
    acme.cert-manager.io/http-token: "964497066"
    acme.cert-manager.io/http01-solver: "true"
spec:
  rules:
  - host: mydomain.com
    http:
      paths:
      - backend:
          service:
            name: cm-acme-http-solver-pslzv
            port:
              number: 8089
        path: /.well-known/acme-challenge/pUB07WsWON9TLYcACp-jpLwLnxBKNQZmvIywbfjTC8U
        pathType: ImplementationSpecific
status:
  loadBalancer:
    ingress:
    - ip: 192.168.1.250

Thank you for your attention on this, it is much appreciated!

Your theory sounds correct to me just based on my observation.
Here is my fully Gateway config:
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: website-mydomain-com
  namespace: mine
spec:
  selector:
    istio: istio-ingressgateway-mine
  servers:
  - hosts:
    - mydomain.com
    - www.mydomain.com
    port:
      name: http
      number: 80
      protocol: HTTP
    tls:
      httpsRedirect: true
  - hosts:
    - mydomain.com
    - www.mydomain.com
    port:
      name: https
      number: 443
      protocol: HTTPS
    tls:
      credentialName: website-mydomain-com-cert
      minProtocolVersion: TLSV1_2
      mode: SIMPLE
      privateKey: sds
      serverCertificate: sds

And in case it's useful the ingress that was generated by cert-manager:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: istio
    nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0,::/0
  creationTimestamp: "2021-01-04T02:08:56Z"
  generateName: cm-acme-http-solver-
  generation: 1
  labels:
    acme.cert-manager.io/http-domain: "2332887661"
    acme.cert-manager.io/http-token: "964497066"
    acme.cert-manager.io/http01-solver: "true"
spec:
  rules:
  - host: mydomain.com
    http:
      paths:
      - backend:
          service:
            name: cm-acme-http-solver-pslzv
            port:
              number: 8089
        path: /.well-known/acme-challenge/pUB07WsWON9TLYcACp-jpLwLnxBKNQZmvIywbfjTC8U
        pathType: ImplementationSpecific
status:
  loadBalancer:
    ingress:
    - ip: 192.168.1.250

Thank you for your attention on this, it is much appreciated!

I know this issue is old and likely not high priority but can it be kept open? Seems as though this issue was reported again not too long ago.

Definitely. This is actually not low priority in my opinion, but its really hard to fix unfortunately

For anyone looking for a solution, here might be one.
This has also be mentioned in https://discuss.istio.io/t/how-to-redirect-to-https-except-for-well-known-acme-challenge/9011/4.
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: my-cert
  namespace: istio-system
  annotations:
    cert-manager.io/issue-temporary-certificate: "true"

Enabling cert-manager.io/issue-temporary-certificate generates a self-signed certificate until a valid one is available.
Let's Encrypt states that they follow HTTPS redirects without checking the certificate.

Our implementation of the HTTP-01 challenge follows redirects, up to 10 redirects deep. It only accepts redirects to “http:” or “https:”, and only to ports 80 or 443. It does not accept redirects to IP addresses. When redirected to an HTTPS URL, it does not validate certificates (since this challenge is intended to bootstrap valid certificates, it may encounter self-signed or expired certificates along the way).
Source: https://cert-manager.io/docs/usage/certificate/#temporary-certificates-whilst-issuing

@simonherbert I was under the impression that this is a different problem: that httpsRedirect used to have a special case to allow cert-manager traffic to its special verification pods, but no longer does?

@camelpunch They are related. Issuing a temporary self-signed certificate makes the https port available before a certificate could be issued. This might not work with all certificate authorities, but at least Let's Encrypt states that they accept that.
The special case for .well-known is the real solution.

For anyone looking for a solution, here might be one.
This has also be mentioned in https://discuss.istio.io/t/how-to-redirect-to-https-except-for-well-known-acme-challenge/9011/4.
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
name: my-cert
namespace: istio-system
annotations:
cert-manager.io/issue-temporary-certificate: "true"
Enabling cert-manager.io/issue-temporary-certificate generates a self-signed certificate until a valid one is available.
Let's Encrypt states that they follow HTTPS redirects without checking the certificate.

I tried this solution, unfortunately Let's Encrypt or Istio (not exactly sure which one) didn't replace the temporary certificate with a valid one. This of course means the browser still displays a warning that the served certificate is self signed and will block the connection by default.
Is there another option/configuration I have to set to get this solution to work?

With https://github.com/istio/api/pull/2088 you can do this if/when its merged. Not as good of a UX though:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: http-redirect
  namespace: default
spec:
  gateways:
  - https-redirect
  hosts:
  - 'mydomain.com'
  http:
  - redirect:
      scheme: https
      redirectCode: 302
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: https-redirect
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "mydomain.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: https
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    hosts:
    - "mydomain.com"
    tls:
      credentialName: sds-credential
      mode: SIMPLE
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: echo
spec:
  hosts:
  - "mydomain.com"
  gateways:
  - https
  http:
  - route:
    - destination:
        host: echo
        port:
          number: 80

@howardjohn Is there any updates on this issue? I would rather not split VS and gateway into four resources as you show above if update is on its way.
Thanks.

Just ran into this problem myself. My work-around was to stop using
tls: {
    httpsRedirect: true
},

in my Istio Gateway spec, and instead use a VirtualService matcher to do the redirection.
    {
      // This rule exists because of the Istio issue on the httpsRedirect flag above
      match: [
        {
          port: 80,
          uri: {
            prefix: "/"
          },
        },
      ],
      redirect: {
        scheme: "https"
      },
    },
...

Doing this allowed cert-manager to insert its own HTTP routes, and when those get cleaned up automatically, revert back to the standard redirect.

@coldguy101
Thank you for your post.  It didn't quite work for me, but it got me to this solution which is working for me.  Here is the complete declaration.
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: kiali
  namespace: monitoring
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - kiali.sandbox.acceleratelearning.com
    - port:
        number: 443
        name: https
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: kiali-certificate
      hosts:
        - kiali.sandbox.acceleratelearning.com
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: kiali
  namespace: monitoring
spec:
  hosts:
    - "*"
  gateways:
    - kiali
  http:
    # The redirect for https has to happen in the virtual service,
    # otherwise it will prevent the cert-manager http01 challenge from working
    - match:
        - scheme:
            exact: http
      redirect:
        scheme: https
    - route:
        - destination:
            host: kiali
            port:
              number: 20001

@RobCannon you saved my life
Just had to upgrade my Istio control plane from 1.11 to 1.13.4, otherwise the redirect.scheme field was broken

@RobCannon you saved my life
Just had to upgrade my Istio control plane from 1.11 to 1.13.4, otherwise the redirect.scheme field was broken

--- BELOW A POTENTIAL SIMPLE HACKY SOLUTION :) ---
Every 3 months I'm like struggling hard with Istio and CertManager and I'm so sad after years it's still a real pain even for really simple use cases (using httpsRedirect: true for example). Hope a proper solution will come out one day!
My solution:

As stated above, we have to do the HTTPS redirection on the VirtualService level since the gateway cannot specify rules of matching
As I specified 3 months ago https://github.com/istio/istio/issues/27643#issuecomment-1095306328 the idea is to do the HTTPS redirection on all URIs except for /.well-known/acme-challenge/ (due to limitation of the RE2 regex It's a bit hacky but working!)

gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: main-gateway
  namespace: mynamespace
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http-ingress
        protocol: HTTP2
      hosts:
        - "mynamespace/www.mydomain.com"
      tls:
        httpsRedirect: false # The HTTPS redirections will be handled in the VirtualService layer to not conflict with certificates renewing...
    - port:
        number: 443
        name: https-ingress
        protocol: HTTPS
      tls:
        mode: SIMPLE
        privateKey: sds
        serverCertificate: sds
        credentialName: ingress-cert
      hosts:
        - "mynamespace/www.mydomain.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: workaround-to-renew-certs
  namespace: mynamespace
spec:
  hosts:
    - "*"
  gateways:
    - main-gateway
  http:
    # The redirect for https has to happen in the virtual service,
    # otherwise it will prevent the cert-manager http01 challenge from working
    - match:
        - scheme:
            exact: http
          uri:
            # This is not a perfect regex but RE2 does not allow negations so we have to hack a it
            # Note: this pattern will redirect to HTTPS all URLs NOT starting with `/.well-` (which should be the best case since your business URLs are unlikely to start with that)
            # this to let pass requests to renew certs like `/.well-known/acme-challenge/...`
            regex: "^/(([^\\.].*)|(\\.[^w].*)|(\\.w[^e].*)|(\\.we[^l].*)|(\\.wel[^l].*)|(\\.well[^\\-].*))"
      redirect:
        scheme: https
        redirectCode: 302 # 302 to not mess with risky permanent redirections since a workaround

🏎💨

instead of negation can you just match everything? Then when cert manager creates a rule it will be more specific and take precedence. I haven't tested but I think it could work

That has been my experience.  The cert-manager rule is inserted dynamically and takes precedence.  But, I tried to build a cluster this morning and the cert-manager challenge is not working, so maybe something has changed somewhere since yesterday.

@howardjohn I looked at precedence from the Kubernetes perspective... there is no explicit priority way when dealing with distinct CR, also I only found one guy mentioning it was relying on creationTimestamp order...
Same for Istio.
From what I understand you mean there is a logic somewhere that detects rules and say "if stricter than others, top priority". Can you clarify on what's it's based? Is it a logic inside match: where exact (high priority) > prefix (medium) > regex (low)?
Note: I know multiple rules in a VirtualService are from top to bottom... but since CertManager manages another file... it does not apply here

I had this issue plus the fact that I have several ingress gateway. This two point was resolved using the HTTP-01 Gateway API solver https://cert-manager.io/docs/configuration/acme/http01/#configuring-the-http-01-gateway-api-solver.  The switch from
istio Gateway,VirtualService to API Gateway and HTTPRoute is easy on our case as we don't have complex VS.

To supplement @sneko 's answer above, we can do a better match for http01 requests.
FROM:

  http:
    # The redirect for https has to happen in the virtual service,
    # otherwise it will prevent the cert-manager http01 challenge from working
    - match:
        - scheme:
            exact: http
          uri:
            # This is not a perfect regex but RE2 does not allow negations so we have to hack a it
            # Note: this pattern will redirect to HTTPS all URIs NOT starting with `/.well-` (which should be the best case since your business URIs are unlikely to start with that)
            # this to let pass requests to renew certs like `/.well-known/acme-challenge/...`
            regex: "^/(([^\\.].*)|(\\.[^w].*)|(\\.w[^e].*)|(\\.we[^l].*)|(\\.wel[^l].*)|(\\.well[^\\-].*))"

TO:
  http:
    - match:
        - withoutHeaders:
            :path:
              prefix: /.well-known/acme-challenge/

Is there any progress here?
Is the above workaround working correctly?

This worked for us, we changed tls httpsRedirect to false (or commented it out) in the gateway and then added this as the first rule under http in the virtual service
  - match:
      - scheme:
          exact: http
    redirect:
      scheme: https

We have so far changed about 50 configs to this in the last 6 months and not had a problem yet

This is the correct solution. I've written a bit more in details about why and how it works: https://gruchalski.com/posts/2023-07-09-istio-cert-manager-lets-encrypt-and-https-redirect/.

does anyone know how the solution with the virtual service could be implemented in a setup including OAuth2-proxy? The issue right now is that the authorization check comes before the virtual service so there is no redirect to https when the authorization page is displayed. Only once the user has passed the authorization, the redirect happens correctly

  http:
    - match:
        - withoutHeaders:
            :path:
              prefix: /.well-known/acme-challenge/

Sadly that works no longer with the latest release.
istio is complaining:
VirtualService/istio-ingress/workaround-to-renew-certs dry-run failed: admission webhook "validation.istio.io" denied the request: configuration is invalid: header name :path is not a valid header name

@fritterhoff I don't use Istio currently so I can't check, but does replacing :path with uri work?

Let me check...

So replacing path with uri does not work directly.
❯ curl -v http://pki.hm.edu/.well-known/acme-challenge/test
* Host pki.hm.edu:80 was resolved.
* IPv6: 2001:4ca0:6800:3::6
* IPv4: 141.39.241.102
*   Trying [2001:4ca0:6800:3::6]:80...
* Connected to pki.hm.edu (2001:4ca0:6800:3::6) port 80
> GET /.well-known/acme-challenge/test HTTP/1.1
> Host: pki.hm.edu
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 302 Found
< location: https://pki.hm.edu/.well-known/acme-challenge/test
< content-security-policy: frame-ancestors 'none';
< x-frame-options: deny
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
< referrer-policy: no-referrer

instead of negation can you just match everything? Then when cert manager creates a rule it will be more specific and take precedence. I haven't tested but I think it could work

This worked for us, we changed tls httpsRedirect to false (or commented it out) in the gateway and then added this as the first rule under http in the virtual service
  - match:
      - scheme:
          exact: http
    redirect:
      scheme: https

We have so far changed about 50 configs to this in the last 6 months and not had a problem yet

I hope @sazzle2611 's answer still works. It's likely I'll run into this issue again if I deploy issue for a future project

I'll check that later.

Seems like that could work if I add that fragment into every service...

authors: RobCannon SeanWallace camelpunch coldguy101 dahovey fritterhoff howardjohn melkishengue mick1627 mway-niels radekg romanlum sazzle2611 simonherbert sneko sudomann yekibud yekinabud


text:
proxy.istio.io/config annotation - override does not work

Bug description
Related documentation:
https://istio.io/latest/docs/tasks/observability/distributed-tracing/configurability/#create-a-meshconfig-with-trace-settings
The value specified in the deployment (sampling 20) does not override the global value (sampling 100).
With the specified configuration, 100 percent of requests always arrive at the collector.
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    defaultConfig:
      tracing:
        sampling: 100
        zipkin:
          address: simplest-collector.observability:9411

Deployment metadata:
  template:
    metadata:
      labels:
        version: v1
      annotations:
        proxy.istio.io/config: |
          tracing:
            sampling: 20
            zipkin:
              address: simplest-collector.observability:9411

[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[X ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
Expected behavior
The meshConfig sampling value (in this case 100) should be overridden by using proxy.istio.io/config annotation (20) and tracing data should be collected by jaeger (only 20 percent).
Steps to reproduce the bug
Version (include the output of istioctl version --remote and kubectl version --short and helm version --short if you used Helm)
istioctl version --remote
client version: 1.9.0
control plane version: 1.9.0
data plane version: 1.9.0 (3 proxies)

kubectl version --short
Client Version: v1.19.4
Server Version: v1.19.3

How was Istio installed?
istioctl  install
Environment where the bug was observed (cloud vendor, OS, etc)
Ubuntu 18.04 baremetal

This worked for me copying your config. Can you post the first ~300 lines of  logs of the proxy?

@howardjohn Hello! Thanks for response.
I attach the log of the istio-proxy container.
I sent 15 requests and all 15 requests came to jaeger.
logs-from-istio-proxy-in-ngx-frontend-red-77c75df567-lss9w.txt

From the logs looks like it applied
can you look at the config_dump? https://github.com/istio/istio/wiki/Troubleshooting-Istio
There should be a part about the trace sampling there.

kubectl -n test3 exec ngx-frontend-red-77c75df567-lss9w -c istio-proxy -- curl 'localhost:15000/config_dump' > config_dump.json

config_dump.json.gz

config looks right:
             "client_sampling": {
              "value": 100
             },
             "random_sampling": {
              "value": 20
             },
             "overall_sampling": {
              "value": 100
             },

I do not understand. In the main config (via IstioOperator), the value is 100. The deployment config indicates the value 20.
I make 20 requests and see 20 requests in jaeger :(

I don't know either, sorry. Envoy is getting configured properlywhich is the extent on my knowledge. need someone in @istio/wg-policies-and-telemetry-maintainers to weigh in

@dgpat are you making 20 requests directly to the target service, or do they go through another proxy somewhere (like an ingressgateway)?

@douglas-reid Hello! Thanks for response.
I am making requests via VirtualService.

I made a check.
kubectl -n test3 get svc
NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
ngx-frontend-red   ClusterIP   10.99.65.253     <none>        80/TCP    3d8

Checking from the kubernetes server.
grep testz /etc/hosts
10.99.65.253 red.testz.test.local

for i in {1..100};  do curl https://red.testz.test.local; done

If you do this, then everything is fine.
I make 100 requests, in jaeger I will see 20 requests
So what is the problem with using VirtualService?

I don't think the problem is with the VirtualService so much as that you are going through a gateway. The gateway is itself a proxy and, critically, it makes a sampling decision for the entire request.
With your setup, I believe you have 100% sampling at the gateway. The request hits the gateway, which makes the sampling decision (essentially sample=true), then routes the request to your target service. The proxy in front of your target service sees that a sampling decision has already been made, and honors it.

Ok. And what do I need to use to get normal behavior?

By the way, I tried doing the opposite.
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    defaultConfig:
      tracing:
        sampling: 1
        zipkin:
          address: simplest-collector.observability:9411

Deployment metadata:
  template:
    metadata:
      labels:
        version: v1
      annotations:
        proxy.istio.io/config: |
          tracing:
            sampling: 100
            zipkin:
              address: simplest-collector.observability:9411

If you do this, then I see only 1% of requests in jaeger

@howardjohn please tell me, did you use VirtaulService for check this case?

By the way, I tried doing the opposite.
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    defaultConfig:
      tracing:
        sampling: 1
        zipkin:
          address: simplest-collector.observability:9411

Deployment metadata:
  template:
    metadata:
      labels:
        version: v1
      annotations:
        proxy.istio.io/config: |
          tracing:
            sampling: 100
            zipkin:
              address: simplest-collector.observability:9411

If I do this, then I see only 1% of requests in jaeger

The key thing is that: if your request first goes through a gateway or another proxy, the only sampling rate that matters is the one that is applied there. Envoy supports per route tracing decisions, but there is not support in the Istio APIs for that level of control. You can try to use EnvoyFilters to achieve your goal for ingress.

I'm sorry, I did not understand.
The documentation does not indicate what type (Ingress, VirtualService or something else) of connection should be used.
This documentation doesn't say there are sampling restrictions.
And this documentation says to use Gateway+VirtualService.
Do I need to stop using Gateway+VirtualService and switch to Ingress?

Sampling decisions for tracing are made once per request. The sampling decision is propagated on each child request once it has been made.
If your request is flowing through a proxy, such as an ingress gateway, that makes sampling decisions, you will need to change the sampling rate at that proxy and not at the downstream proxy.
Here's is my lame attempt at diagramming what I think is happening in your scenario:
client --> Istio IngressGateway (makes sampling decision; random sample rate of 100%) -->  Sidecar proxy (respects original sampling decision; random sampling rate here is irrelevant) --> backend
The Istio Networking APIs aren't what is relevant. It is the fact that the sampling rate at your gateway is 100%.
To get 20% sampling on requests going to your backend, you need to set the sampling rate at the gateway to be 20%. That can be blunt, and the workarounds for greater precision are a bit complex, but that is the general idea.
Does that make sense?
If you apply a 20% sampling rate at the sidecar proxy for your backend, that will only apply to traffic for which no prior sampling decision has been made.

Ok.
Then the task "collect from all request (to all namespace) 20%, and from a separate namespace 100%" - has no solution?
That is, if I want to collect 100% from one separate namespace, do I need to turn off sampling in the gateway, and use only annotation in the deployment?

Ok.
Then the task "collect from all request (to all namespace) 20%, and from a separate namespace 100%" - has no solution?
That is, if I want to collect 100% from one separate namespace, do I need to turn off sampling in the gateway, and use only annotation in the deployment?

It has a solution, but it is more involved. It requires, at this time, some EnvoyFilter manipulation for per-route configuration. Or different gateways for different backends.

authors: dgpat douglas-reid howardjohn


text:
plese provide security CVE's and Release details to github's release page

Describe the feature request
the easiest way to track new releases in a normalized way for opensource products is to leverage github's RSS feeds for releases, like this:
https://github.com/istio/istio/releases.atom
However, the istio releases only contain a link to a blog post discussing the release, and morever it sometimes requires even further clickthrough's to find CVE's or assorted security bulletins related to releases.
Please include changelog and highlights for isitio releases in the git released tag.
Describe alternatives you've considered
I've tried looking through the istio.io repo to see if it's easily tracked for this information, but unfortunately due to the structure of the site, it's not feasible looking.
Affected product area (please put an X in all that apply)
[X] Docs
[X] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[X] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

We do publish security builtin on https://github.com/istio/istio/security/advisories now. These can be watched, although not sure about a RSS feed.
One thing I was not clear on is why clicking a link in the description is an issue? This is common in many open source projects.

I'm trying to setup a common notification feed of my most critical assets and am leveraging RSS feeds for github releases as a  means of a quick single source of data to take action.
for example, check out grafanan's releases:
https://github.com/grafana/grafana/releases
if you are using slack, you can easily subscribe to new releases and quickly see if there are any security concerns

We do report them on https://github.com/istio/istio/security/advisories. Is that not accessible through any feeds?

authors: howardjohn nobletrout


text:
Locality affinity support

Edit: outcome of discussion in slack:
In EDS code, we will maintain a per zone EDS, where all endpoints in local zone will get 75% traffic and those in remote zones will get rest 25%.
Configuration considerations:
EDS Configuration

Supplying of core.Locality in LocalityLbEndpoints.
Setting of load_balancing_weight field

CDS Common LB Configuration

Locality_weighted_lb_config enabled

I don't think it ever worked fully in v1 - but for multi-cluster it'll be important to get it right.
Few notes:

existing code to get the AZ from node annotations seems ok, but need to reduce logging ( some clusters don't support this, no need to be so verbose )
for clusters without node annotations, in a multi-cluster env, it should be possible to get the zone from the cluster config, and maybe default to the custer name

In eds.go we'll need to group the endpoints by zone, and return each list separately, and set the zone
in each set.

+sdake - if you have some bandwidth for the multi-cluster parts ( or the entire thing ).

Locality LB endpoints are somewhat orthogonal to cluster zones in Envoy.
PTAL
https://github.com/envoyproxy/data-plane-api/blob/master/envoy/api/v2/endpoint/endpoint.proto#L6
3
On Thu, Mar 22, 2018 at 1:59 PM Costin Manolache notifications@github.com
wrote:

+sdake - if you have some bandwidth for the multi-cluster parts ( or the
entire thing ).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/issues/4481#issuecomment-375455563, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AJGIxmgg6GDm8KRoF-20kl5b5rRJT0UUks5thBCqgaJpZM4S3yK3
.

None of this will work until we sort out how to update the service zone of the proxy with mTLS enabled. We need to restart Envoy with the appropriate service zone since we don’t have access to the pod’s az during startup. Alternatively, we invest time in making that piece dynamic in Envoy.

Regarding AZ-on-startup - it's orthogonal, I have some other work in progress that will cover this (with agent making an explicit MTLS + gRPC call to pilot, it can get AZ and other data ).
This issue is specifically for fixing the EDS result - need to group the endpoints by locality, and pass
the right locality.

Also: it is already possible to configure the sidecars with an explicit zone, without having to get it from pilot.
It works in many cases where you control how you deploy.

Also: it is already possible to configure the sidecars with an explicit zone, without having to get it from pilot.

That is true, but how would a user go about configuring an explicit zone?

This issue is specifically for fixing the EDS result - need to group the endpoints by locality, and pass
the right locality.

I'm not sure I understand correctly. You wan't to do some additional explicit locality filtering in pilot based on the endpoint region/zone?

For explicit zone config: the CI/CD + kube-inject or webhook can add it using a custom template for injection.In a multi-cluster mode each cluster will have a separate webhook and may have a separate template.
But this bug is not about filtering or fancier config: it's about how we generate the EDS config, and checking it works. Envoy expects to see a list of LocalityLbEndpoints, each with a locality and a list of endpoints in that  locality. Right now we return a list of LocailtyLbEndpoints each with a single endpoint, and with the zone repeated in each. To fix this we need to sort/group the endpoints - should be relatively simple. Testing it - with some form of zone config - is the tricky part.

I understand now. Just reading through the docs I think this actually may be instead of setting --service-zone?

zone
(string) Defines the local service zone where Envoy is running. Though optional, it should be set if discovery service routing is used and the discovery service exposes zone data, either in this message or via --service-zone. The meaning of zone is context dependent, e.g. Availability Zone (AZ) on AWS, Zone on GCP, etc.

https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/core/base.proto#envoy-api-msg-core-locality

@costinm apologies I missed the assignment of this issue (~3 weeks ago).  Overflowing in github email :(  I will have some time to work on this in the coming few weeks.
@liamawhite want to join forces, or going to handle on your own?
Cheers
-steve

Happy to join forces. Let me know when you have some time to work on it.

@liamawhite I suspect I will be consumed until 0.8 is branched doing what I can to get 0.8 ready to go, after which I will have time available.  I think the branch is at end of week.
Do you have any specific task list in mind?
Cheers
-steve

I think for 0.8 we need to confirm envoy is ok with what we send - i.e. the
repeated list of
one locality + one endpoint, instead of the grouped format.
And to test it actually works - for that we need 2 clusters with a service
with at least 2 endpoints
in each cluster.  I would be happy with a manual test for 0.8 - setting up
the environment is tricky
and still in flux.
We can optimize/refine after 0.8.
On Mon, Apr 9, 2018 at 1:04 PM, Steven Dake notifications@github.com
wrote:

@liamawhite https://github.com/liamawhite I suspect I will be consumed
until 0.8 is branched doing what I can to get 0.8 ready to go, after which
I will have time available. I think the branch is at end of week.
Do you have any specific task list in mind?
Cheers
-steve
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/issues/4481#issuecomment-379876842, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFI6lMt51BmbFC5_DIG19Z0B8h-Ibspks5tm77bgaJpZM4S3yK3
.

Anyone has time for this in the next 3 days ? I guess testing may be more time consuming then
actually grouping the endpoints by zone and implementing the optimization.
If nobody else volunteers I can take it (probably on Fri)

Design Doc is here. Discussions ongoing.

Setting it for 1.0 - I think it's required.

@liamawhite I am interested in reading the design doc but google tells me it's not publicly available, do I need to belong to some mailing list to read it or it's really not for public consumption....

In EDS code, we will maintain a per zone EDS, where all endpoints in local zone will get 75% traffic and those in remote zones will get rest 25%.

I wonder if the 75/25 split is configurable? We would like to adjust these parameters to better suit our orgs requirement.
Thanks!

@munnerz There's no reason we can't do this, we just need to work out the best place to expose it to users. It will probably initially be a mesh wide configuration variable though.
@bjhaid Not sure why you can't see it, let me do some investigation

What's the status of this? I went through some issues, documentation and discussions and is not clear if we can achieve Locality Weighted Load Balancing or something similar somehow.
btw I'm also interested in reading the design doc above and meeting notes but have no access.

This has not yet been prioritised, but I will take a look if more people are interested in this. It will not be in by 1.1 though.

Thank you @liamawhite. Is there a recommended way to preferably route traffic within the same availability zone? I've scratched the code base and saw that ServiceInstances have AvailabilityZone but it was unclear if it can be used to achieve this.
I know we can workaround this with DestinationRules but was wondering if there is a better approach.

The availability zone fields are not configured properly in Envoy yet so won't do it for you.
I think you could probably do this with DestinationRules but that would be pretty horrible, I can't think of a better approach right now though.

Hey! I have an use-case enabling zone-aware load-balancing towards hosts outside of the mesh. Would this issue relate to that, too?
My initial guess is that I'd need to add a kind of zone field to the ServiceEntry.Location type, consuming the zone value from it, configure Envoy accordingly via EDS.

Hey
Just wanted to rekindle this issue a bit, we're super interested in zone aware routing both from a performance and cost optimisation perspective.
It'd be great to see some traction?

Discussions are currently underway on how we can implement this efficiently. Could everyone who is interested please go into a bit more detail on their use cases? This will allow us to better understand what things need to be made configurable.

Happily,
So we run in GKE/GCP are for availability reasons run our cluster across three availability zones.    We make use of affinity rules to ensure latency sensitive applications are co-located (node, and zonal) but we also then distribute those instances across AZ's for BCP reasons.  So for example;
Zone A
  - App 1
  - App 2

Zone B
  - App 1
  - App 2

etc....
We quite simply want to keep the traffic between App1 and App2 within the same zone where possible, unless the zonal instance of an app was offline or a circuit breaker was firing.
This:

Reduces latency
Reduces cost (zonal egress charges)

My use case doesn't really require much configuration, just simple prefer the same zone, where possible.

We have a similar use case described above. However, we still also need to prioritize traffic within the same availability zone for things running outside Kubernetes, such as databases. We have an in-house solution for this ATM that integrates with our former 'service mesh' solution but deprecating this custom stuff would be great.
Also, we have concerns of unbalanced deployments across the cluster potentially overload some service instances more than others if the weight criteria are purely AZ-based. Having the weight defined by node, AZ and something else like load(# of requests) could be one way to go, and if is possible to add such metrics to the equation, latency could also be an alternative and ~my guess~ would make things easier for this particular issue - It would try to prioritize traffic to the shortest distance, but when some service instance on the shortest network route is taking long to respond (in case of being overloaded) it would shift traffic to others responding faster.
Well, in any case, I really appreciate you guys are starting to talk about this again 💯

What's the status here? It looks like a PR may have been merged then reverted?

@dwradcliffe Hey! My understanding is that this is mostly coming to v1.1.
The only missing feature would be Targeted Locality Weighting (Locality Weighted Load Balancing) for cluster-internal traffic. For egress traffic, it is already implemented in #11381.
See @liamawhite's comment at https://github.com/istio/istio/pull/11381#issuecomment-468397227 for more context.

This will be experimental in 1.1 and off by default behind an env var. See https://github.com/istio/istio.io/pull/3593 for more details.
All features should be there but have not been extensively tested. The aim is to do this over the course of 1.1.x releases and then turn it on by default.

This has been release in 1.1. If there are any issues please open a new issue.
See https://istio.io/help/ops/traffic-management/locality-load-balancing/ for docs.

authors: Stono bjhaid cainelli costinm dwradcliffe howardjohn kyessenov liamawhite mumoshu munnerz rshriram sdake


text:
Istioctl Analyze not consistently respecting Exports for ServiceEntry used by VirtualService

Is this the right place to submit this?

[X] This is not a security vulnerability or a crashing bug
[X] This is not a question about how to use Istio

Bug Description
I have noticed that when running istioctl analyze -A it erroneously reports that my host cannot be found. My basic setup is:

A ServiceEntry in workspace foo, which is exported to ns bar
A VirtualService in namespace bar, which is not exported exported to any other namespace.
The VS routes traffic to the host of the SE.

When I run istioctl analyze -A I get the error:
$ istioctl Analyze -A
Error [IST0101] (VirtualService bar/foo-route) Referenced host not found: "foobar.staging.mesh"
Error: Analyzers found issues when analyzing all namespaces.
See https://istio.io/v1.17/docs/reference/config/analysis for more information about causes and resolutions.

These are the two resources needed to reproduce this:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-svc-virtualdestination
  namespace: foo
spec:
  hosts:
  - foobar.staging.mesh
  location: MESH_INTERNAL
  ports:
  - number: 80
    name: https
    protocol: HTTP
    targetPort: 80
  resolution: DNS
  exportTo:
    - "bar"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: foo-route
  namespace: bar
spec:
  hosts:
  - foo.foo.svc.cluster.local
  http:
  - name: "foo-route"
    match:
    - uri:
        prefix: "/"
    route:
    - destination:
        host: foobar.staging.mesh
        port:
          number: 80

What's interesting is that if I change the SE to export to all "*" instead of explicitly to the VS NS "bar", then analyze has no issues:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-svc-virtualdestination
  namespace: foo
spec:
  hosts:
  - foobar.staging.mesh
  location: MESH_INTERNAL
  ports:
  - number: 80
    name: https
    protocol: HTTP
    targetPort: 80
  resolution: DNS
  exportTo:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: foo-route
  namespace: bar
spec:
  hosts:
  - foo.foo.svc.cluster.local
  http:
  - name: "foo-route"
    match:
    - uri:
        prefix: "/"
    route:
    - destination:
        host: foobar.staging.mesh
        port:
          number: 80

$ istioctl analyze -A
✔ No validation issues found when analyzing all namespaces.

Also interesting is that if the VirtualService is exported to the ServiceEntry's namespace, then this also makes analyze happy:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-svc-virtualdestination
  namespace: foo
spec:
  hosts:
  - foobar.staging.mesh
  location: MESH_INTERNAL
  ports:
  - number: 80
    name: https
    protocol: HTTP
    targetPort: 80
  resolution: DNS
  exportTo:
    - "bar"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: foo-route
  namespace: bar
spec:
  exportTo:
  - "foo"
  hosts:
  - foo.foo.svc.cluster.local
  http:
  - name: "foo-route"
    match:
    - uri:
        prefix: "/"
    route:
    - destination:
        host: foobar.staging.mesh
        port:
          number: 80

$ istioctl analyze -A

✔ No validation issues found when analyzing all namespaces.

Version
Istio 1.17.2

Additional Information
Other than the VirtualService and ServiceEntry described above, my environment is a fresh kind cluster with a vanilla istioctl install -y run on it, with two namespaces foo and bar, both of which are injected and each ns running a sample httpbin pod.
Affected product area

[ ] Ambient
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[X] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[X] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Control Plane Revisions

I can probably take a shot at fixing this myself actually


text:
Istio Grafa not able to access

So i am running istio on aws ec2.
I am trying to depoy the grafana dashboard via this
kubectl -n istio-system port-forward $(kubectl -n istio-system get pod -l app=grafana -o jsonpath='{.items[0].metadata.name}') 3000:3000
However, when i try to access masterip:3000, nothing happens.
it slistening on port 3000 when i did a lsof
lsof -i:3000
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
kubectl 12267  opc    6u  IPv4  45964      0t0  TCP localhost:hbci (LISTEN)
kubectl 12267  opc    7u  IPv6  45965      0t0  TCP localhost:hbci (LISTEN)
Whats happening. on local host this work however

port-forward only binds to 127.0.0.1 IOW, it will not bind to your master IP.  port-forward will not do what you want. In 1.0.0, there are config options called telemetry-ingress however that feature was DOA in 1.0.0 sadly.  1.0.2 has resolved this problem with commit: https://github.com/istio/istio/commit/40143b05d92e5192df5a4629da7ece36111aed17.  I'd encourage you to try 1.0.2
Use the helm commands documented with --set telemetry-gateway.grafanaEnabled and access port 15031 of your load balancer external IP.  Note there is no authentication!
Can you point out the documentation which mentions to use port-forward so it can be corrected?
Cheers
-steve

@leeadh and @sdake this is probably better discussed on a mailing-list.
Can we please move this to: istio-users@ googlegroups.com ?

@leeadh can you report this question on the mailing list please.
Thanks!
-steve

https://istio.io/docs/tasks/telemetry/using-istio-dashboard/
@sdake.  An using istio 1.1.0

authors: douglas-reid leeadh sdake


text:
Add MCP stress test suite

This PR adds a stress test suite for the MCP source/sink stack. Test*() functions test the functional correctness multiple clients in common scenarios. Most parameters are also exposed as flags so that additional manual testing can be performed, e.g adjust NACK %, fake update delays.
Some sample benchmarks are also included for the purposes of generating pprof cpu/memory profiles.
Other minor changes include:

fixes a typo in source.CollectionOptions (s/CollectionsOptions/Collections). The MCP packages are referenced by external projects. Fixing the exported package API now before 1.1 official ships.

increases pkg/mcp/internal/queue.go test coverage to 100%

add additional unit tests for incremental MCP

fixes https://github.com/istio/istio/issues/11489

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: ayj
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

galley/OWNERS [ayj]
pkg/mcp/OWNERS [ayj]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
0521a198f8e90ae163099429caa466085925127b
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
a96877cc1c9114dcd252ee29a8948a5fae92c440
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
a96877cc1c9114dcd252ee29a8948a5fae92c440
link
/test istio-unit-tests

prow/istio-pilot-multicluster-e2e.sh
a96877cc1c9114dcd252ee29a8948a5fae92c440
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
66798ed441bc7fca2849417e56cf4589d1c64423
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
66798ed441bc7fca2849417e56cf4589d1c64423
link
/test istio-pilot-multicluster-e2e

prow/istio-unit-tests.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio-unit-tests

prow/istio-pilot-multicluster-e2e.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/home/prow/go/src/istio.io/istio/vendor/google.golang.org/grpc/server.go:522 +0x748
/home/prow/go/src/istio.io/istio/galley/pkg/server/server.go:242 +0xfb
/home/prow/go/src/istio.io/istio/galley/pkg/server/server.go:233 +0x5c

/test istio-unit-tests

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio-unit-tests

prow/istio-pilot-multicluster-e2e.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio-pilot-multicluster-e2e

prow/e2e_pilotv2_auth_sds.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio_auth_sds_e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio-pilot-multicluster-e2e

prow/e2e_pilotv2_auth_sds.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio_auth_sds_e2e

prow/istio-unit-tests.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio_auth_sds_e2e

prow/istio-pilot-multicluster-e2e.sh
25a31999a22daee82a52358745f45ec7d3837673
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

--- FAIL: TestLRUExpiration (0.01s)
    cache_test.go:201: Got no value, expected LATER to still be present
    cache_test.go:205: Got 2 evictions, expecting 1

re-running ci/circleci: test

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
793974284d6a6d474ff2bccbf96b5b5d9e11f5c6
link
/test istio_auth_sds_e2e

prow/istio-pilot-multicluster-e2e.sh
25a31999a22daee82a52358745f45ec7d3837673
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
25a31999a22daee82a52358745f45ec7d3837673
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ozevren @jeffmendoza - this is ready for final review

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
25a31999a22daee82a52358745f45ec7d3837673
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
25a31999a22daee82a52358745f45ec7d3837673
link
/test istio-integ-k8s-tests

prow/istio-unit-tests.sh
b71753176ae63590cbf06f041d675b37caa91dc2
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
25a31999a22daee82a52358745f45ec7d3837673
link
/test istio-integ-k8s-tests

prow/istio-unit-tests.sh
b71753176ae63590cbf06f041d675b37caa91dc2
link
/test istio-unit-tests

prow/istio-pilot-multicluster-e2e.sh
b71753176ae63590cbf06f041d675b37caa91dc2
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
25a31999a22daee82a52358745f45ec7d3837673
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-integ-k8s-tests

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-pilot-e2e-envoyv2-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test istio-pilot-e2e-envoyv2-v1alpha3

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-integ-k8s-tests

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-pilot-e2e-envoyv2-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test istio-pilot-e2e-envoyv2-v1alpha3

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-integ-k8s-tests

prow/release-test.sh
a7cd653987203fcae8113468ea33651559936394
link
/test release-test

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
1d8b574cd5752d5c37950b14ad69c2bc69c97caf
link
/test istio-integ-k8s-tests

prow/release-test.sh
a7cd653987203fcae8113468ea33651559936394
link
/test release-test

prow/istio-pilot-multicluster-e2e.sh
a7cd653987203fcae8113468ea33651559936394
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/lgtm
/approve

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: ayj, ozevren
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

galley/OWNERS [ayj,ozevren]
pkg/mcp/OWNERS [ayj,ozevren]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

error pulling image configuration: received unexpected HTTP status: 502 Bad Gateway

/test release-test

cc @wenchenglu, can you help merge? This PR adds additional stress tests and benchmarks for MCP.

authors: ayj istio-testing ozevren


text:
Fix syntax error in virtual-service-ratings-test-delay.yaml

While going through the bookinfo fault injection tutorial, I noticed the following problem when attempting to apply the demonstration file virtual-service-ratings-test-delay.yaml.
$ kubectl apply -f samples/bookinfo/networking/virtual-service-ratings-test-delay.yaml
Error from server: error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"networking.istio.io/v1alpha3\",\"kind\":\"VirtualService\",\"metadata\":{\"annotations\":{},\"name\":\"ratings\",\"namespace\":\"mhite\"},\"spec\":{\"hosts\":[\"ratings\"],\"http\":[{\"fault\":{\"delay\":{\"fixedDelay\":\"7s\",\"percentage\":{\"value\":100}}},\"match\":[{\"headers\":{\"end-user\":{\"exact\":\"jason\"}}}],\"route\":[{\"destination\":{\"host\":\"ratings\",\"subset\":\"v1\"}}]},{\"route\":[{\"destination\":{\"host\":\"ratings\",\"subset\":\"v1\"}}]}]}}\n"}},"spec":{"http":[{"fault":{"delay":{"fixedDelay":"7s","percentage":{"value":100}}},"match":[{"headers":{"end-user":{"exact":"jason"}}}],"route":[{"destination":{"host":"ratings","subset":"v1"}}]},{"route":[{"destination":{"host":"ratings","subset":"v1"}}]}]}}
to:
&{0xc420274900 0xc420437500 mhite ratings samples/bookinfo/networking/virtual-service-ratings-test-delay.yaml 0xc42000c008 4047459 false}
for: "samples/bookinfo/networking/virtual-service-ratings-test-delay.yaml": admission webhook "pilot.validation.istio.io" denied the request: error decoding configuration: YAML decoding error: hosts:
- ratings
http:
- fault:
    delay:
      fixedDelay: 7s
      percentage:
        value: 100
  match:
  - headers:
      end-user:
        exact: jason
  route:
  - destination:
      host: ratings
      subset: v1
- route:
  - destination:
      host: ratings
      subset: v1
 unknown field "percentage" in v1alpha3.HTTPFaultInjection_Delay

This PR fixes the demo file.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: mhite
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: rshriram
If they are not already assigned, you can assign the PR to them by writing /assign @rshriram in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

samples/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @mhite. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

I signed it!

I signed it!

See this comment on #11354.

Makes sense! My mistake. I should have switched to the 1.0.5 release branch before attempting to follow the web site tutorial.

authors: istio-testing mhite