profile
viewpoint
John Howard howardjohn Google Sunnyvale, CA Istio @ Google

fortio/fortio 1350

Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

howardjohn/kubectl-resources 11

Plugin to access Kubernetes resource requests, limits, and usage.

howardjohn/Chromium-Downloader 1

Downloads the latest chromium nightly build automatically

howardjohn/file-based-istio 1

Capture XDS responses from Pilot and generate a static Envoy deployment

howardjohn/istio 1

Connect, secure, control, and observe services.

howardjohn/api 0

API, config and standard vocabulary definitions for the Istio project

howardjohn/AWScala 0

Using AWS SDK on the Scala REPL

howardjohn/charts 0

The IBM/charts repository provides helm charts for IBM and Third Party middleware.

howardjohn/cni 0

Istio CNI to setup kubernetes pod namespaces to redirect traffic to sidecar proxy.

Pull request review commentistio/istio

Log XDS client rejections due to SPIFFE

 func (v *PeerCertVerifier) AddMappings(certMap map[string][]*x509.Certificate) { // VerifyPeerCert is an implementation of tls.Config.VerifyPeerCertificate. // It verifies the peer certificate using the root certificates associated with its trust domain. func (v *PeerCertVerifier) VerifyPeerCert(rawCerts [][]byte, _ [][]*x509.Certificate) error {+	spiffeLog.Debugf("Verifying %d peer certificates", len(rawCerts)) 	if len(rawCerts) == 0 { 		// Peer doesn't present a certificate. Just skip. Other authn methods may be used.+		spiffeLog.Infof("Peer didn't present certificate")

This log is going to be highly confusing in a real deployment. Which peer? Where did it not present cert? Is it intended (because it's using jwt) or an error?

I think we are logging at the wrong level most likely. I also suspect this will log on every CSR request, as those will of course not have certs?

esnible

comment created time in 31 minutes

pull request commentistio/istio

ads refactor

Also yes the test client (adsc) is different than envoy behavior, at the very least it doesn't include resource names in ack

On Fri, Jul 3, 2020, 8:05 AM John Howard howardjohn@google.com wrote:

No let's merge this one first if it works - mine doesn't work. But i think this may move us to consistently wrong... Our entire xds handling doesn't really match the expectation of envoy. But we should probably be consistent then fix things up

On Fri, Jul 3, 2020, 8:03 AM Rama Chavali notifications@github.com wrote:

PTAL when you get chance- if there is an overlap with the other PR, we can wait till that is merged and see if this is needed or we can decide what is the best way to move forward based on what you think. My main intention was to refactor the protocol handling to a separate function so that the inconsistencies can be avoided

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/istio/istio/pull/25192#issuecomment-653587287, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEYGXPTOSAABNIOZJYEGBDRZXXLLANCNFSM4OPWLXIQ .

ramaraochavali

comment created time in an hour

pull request commentistio/istio

ads refactor

No let's merge this one first if it works - mine doesn't work. But i think this may move us to consistently wrong... Our entire xds handling doesn't really match the expectation of envoy. But we should probably be consistent then fix things up

On Fri, Jul 3, 2020, 8:03 AM Rama Chavali notifications@github.com wrote:

PTAL when you get chance- if there is an overlap with the other PR, we can wait till that is merged and see if this is needed or we can decide what is the best way to move forward based on what you think. My main intention was to refactor the protocol handling to a separate function so that the inconsistencies can be avoided

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/istio/istio/pull/25192#issuecomment-653587287, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEYGXPTOSAABNIOZJYEGBDRZXXLLANCNFSM4OPWLXIQ .

ramaraochavali

comment created time in 2 hours

pull request commentistio/istio

ads refactor

By the way in my I tried to refactor and modify the logic - I think this is just refactorings (?)

On Fri, Jul 3, 2020, 7:15 AM Istio Automation notifications@github.com wrote:

@ramaraochavali https://github.com/ramaraochavali: The following tests failed, say /retest to rerun all failed tests: Test name Commit Details Rerun command unit-tests_istio 6d6a509 https://github.com/istio/istio/commit/6d6a509de94063268ccc918ad27095ae9ba07720 link https://prow.istio.io/view/gcs/istio-prow/pr-logs/pull/istio_istio/25192/unit-tests_istio/15794 /test unit-tests_istio integ-pilot-k8s-tests_istio 6d6a509 https://github.com/istio/istio/commit/6d6a509de94063268ccc918ad27095ae9ba07720 link https://prow.istio.io/view/gcs/istio-prow/pr-logs/pull/istio_istio/25192/integ-pilot-k8s-tests_istio/15350 /test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here https://git.k8s.io/community/contributors/guide/pull-requests.md. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra https://github.com/kubernetes/test-infra/issues/new?title=Prow issue: repository. I understand the commands that are listed here https://go.k8s.io/bot-commands.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/istio/istio/pull/25192#issuecomment-653568535, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEYGXKIBRZBKYDZGAKZLKTRZXRX5ANCNFSM4OPWLXIQ .

ramaraochavali

comment created time in 2 hours

issue commentistio/istio

The resources of EnvoyFilter works globally only within namespace istio-system

https://istio.io/latest/docs/reference/config/istio.mesh.v1alpha1/ see rootNamespace

shaoyifeng1013

comment created time in 3 hours

issue commentistio/istio

For istio 1.5.7 `istioctl manifest generate` creates a yaml file that is not applicable on on a clean system

We actually do have sorting logic.. We must not be applying it correctly is my guess? Because k8s will apply in order afaik although it's eventually consistent so you may still have issues but it's rare

mhenke1

comment created time in 3 hours

pull request commentistio/istio

ads refactor

But I am happy to see this!

ramaraochavali

comment created time in 3 hours

pull request commentistio/istio

pilot: incremental EDS requests

any newly requested resources

it's a bit ambiguous but to me this seems like what we are doing now - only the newly request ones

howardjohn

comment created time in 3 hours

Pull request review commentistio/istio

Make proxyv2 image building operations universal

 COPY pilot-agent /usr/local/bin/pilot-agent  COPY envoy_policy.yaml.tmpl /var/lib/istio/envoy/envoy_policy.yaml.tmpl -COPY stats-filter.wasm /etc/istio/extensions/stats-filter.wasm-COPY metadata-exchange-filter.wasm /etc/istio/extensions/metadata-exchange-filter.wasm+COPY . *.wasm /etc/istio/extensions/

You are copying the entire docker context to the extensions folder though rather than nothing, I think.

GLYASAI

comment created time in 13 hours

Pull request review commentistio/istio

Make proxyv2 image building operations universal

 COPY pilot-agent /usr/local/bin/pilot-agent  COPY envoy_policy.yaml.tmpl /var/lib/istio/envoy/envoy_policy.yaml.tmpl -COPY stats-filter.wasm /etc/istio/extensions/stats-filter.wasm-COPY metadata-exchange-filter.wasm /etc/istio/extensions/metadata-exchange-filter.wasm+COPY . *.wasm /etc/istio/extensions/

isn't this copying . which is everything?

GLYASAI

comment created time in 14 hours

pull request commentistio/istio

Make proxyv2 image building operations universal

Do we need to change all the variable names? MOSN can still override the variables even if it's named "envoy". There are so many CI pipelines using these variables that will be painful to update for no real reason.

Also is it so bad to have the wasm bundles added? They are quite small... If you want to customize the image more I highly encourage building your own images

GLYASAI

comment created time in 14 hours

Pull request review commentistio/istio

eds: look in informer store before going to k8s

 func (e *endpointsController) onEvent(curr interface{}, event model.Event) error func (e *endpointsController) buildIstioEndpoints(endpoint interface{}, host host.Name) []*model.IstioEndpoint { 	endpoints := make([]*model.IstioEndpoint, 0) 	ep := endpoint.(*v1.Endpoints)+	key := kube.KeyFunc(ep.Name, ep.Namespace) 	for _, ss := range ep.Subsets { 		for _, ea := range ss.Addresses { 			pod := e.c.pods.getPodByIP(ea.IP) 			if pod == nil {-				// This means, the endpoint event has arrived before pod event. This might happen because-				// PodCache is eventually consistent. We should try to get the pod from kube-api server. 				if ea.TargetRef != nil && ea.TargetRef.Kind == "Pod" {-					pod = e.c.pods.getPod(ea.TargetRef.Name, ea.TargetRef.Namespace)+					podkey := kube.KeyFunc(ea.TargetRef.Name, ea.TargetRef.Namespace)

Won't the if ! exist part trigger?

I think it's very clear we need to add some extensive testing around this area so we don't have to guess

ramaraochavali

comment created time in a day

Pull request review commentistio/istio

eds: look in informer store before going to k8s

 func (e *endpointsController) onEvent(curr interface{}, event model.Event) error func (e *endpointsController) buildIstioEndpoints(endpoint interface{}, host host.Name) []*model.IstioEndpoint { 	endpoints := make([]*model.IstioEndpoint, 0) 	ep := endpoint.(*v1.Endpoints)+	key := kube.KeyFunc(ep.Name, ep.Namespace) 	for _, ss := range ep.Subsets { 		for _, ea := range ss.Addresses { 			pod := e.c.pods.getPodByIP(ea.IP) 			if pod == nil {-				// This means, the endpoint event has arrived before pod event. This might happen because-				// PodCache is eventually consistent. We should try to get the pod from kube-api server. 				if ea.TargetRef != nil && ea.TargetRef.Kind == "Pod" {-					pod = e.c.pods.getPod(ea.TargetRef.Name, ea.TargetRef.Namespace)+					podkey := kube.KeyFunc(ea.TargetRef.Name, ea.TargetRef.Namespace)

I think it's useful, the first check is the pod cache which must make its way through the informer and the queue, so if it's out of order in the queue this may he useful

ramaraochavali

comment created time in a day

issue commentistio/istio

Exclusion ports are not working causing readiness and liveness probes to fail

"The kubelet sends the probe to the pod’s IP address, unless the address is overridden by the optional host field in httpGet."

I think we need this logic in the rewrite probe if we don't already

vvavepacket

comment created time in a day

issue commentistio/istio

Exclusion ports are not working causing readiness and liveness probes to fail

0.0.0.0 or 127.0.0.1

@incfly is this expected that we require this? shouldn't we send to the same destination that kubelet does?

vvavepacket

comment created time in a day

issue commentistio/istio

Exclusion ports are not working causing readiness and liveness probes to fail

Does /ready listen on the pod ip? maybe that is why request to localhost fails. I don't think the problem is going through envoy

vvavepacket

comment created time in a day

issue commentistio/istio

Update to Kubernetes 1.19 client libraries

Yes but it's not a big deal - we just direct API call today so it's just a code cleanup we aren't missing any functionality

howardjohn

comment created time in a day

Pull request review commentistio/istio

Extend security test for VMs

 func TestReachability(t *testing.T) { 							return false 						} +						// Exclude calls from naked->VM since VM is not injected with sidecar

lei, I see what you mean in theory but that's not how it works in practice. The pod sends a curl vm request, which kube-proxy routes to the correct endpoint. However kube proxy doesn't have any endpoints, because it doesn't read workload entries it only reads endpoints.

with envoy it does work because we program envoy to have the addresses in the workload entry

ZhengzheYang

comment created time in 2 days

Pull request review commentistio/istio

Extend security test for VMs

 func (rc *Context) Run(testCases []TestCase) { 			ctx.Logf("[%s] [%v] Finish waiting. Continue testing.", testName, time.Now())  			for _, src := range []echo.Instance{rc.A, rc.B, rc.Headless, rc.Naked} {-				for _, dest := range []echo.Instance{rc.A, rc.B, rc.Headless, rc.Multiversion, rc.Naked} {+				for _, dest := range []echo.Instance{rc.A, rc.B, rc.Headless, rc.Multiversion, rc.Naked, rc.VM} {

Agreed, but let's make sure we do add it when DNS is ready

ZhengzheYang

comment created time in 2 days

Pull request review commentistio/istio

Extend security test for VMs

 func TestReachability(t *testing.T) { 							return false 						} +						// Exclude calls from naked->VM since VM is not injected with sidecar

Naked means no envoy, so kubernetes does all of the traffic routing. Kubernetes knows absolutely nothing about workload entry, so the fact one exists with the address is not relevant

ZhengzheYang

comment created time in 2 days

Pull request review commentistio/istio

Extend security test for VMs

 func TestReachability(t *testing.T) { 							return false 						} +						// Exclude calls from naked->VM since VM is not injected with sidecar

Very close to correct - there is a k8s service but there is no associated endpoints (instead there is workload entry only). So right now naked to VM won't work

ZhengzheYang

comment created time in 2 days

Pull request review commentistio/community

Updating Steering Committee charter

 # Istio Steering Committee  The Istio Steering Committee was formed to oversee the administrative aspects of the project, including governance, branding, and marketing.-Steering was created to allow the Technical Oversight Committee to exclusively focus on the technical aspects of the project. +Steering was created to allow the Technical Oversight Committee to exclusively focus on the technical aspects of the project.  * [Charter](#charter)+* [Membership and Voting](#membership-and-voting) * [Committee meetings](#committee-meetings) * [Committee members](#committee-members) * [General questions](#general-questions) * [Getting in touch](#getting-in-touch)  ## Charter -1. Define, evolve, and defend the vision, values, mission, and scope of the project - to establish and maintain the soul of Istio.-2. Define and evolve project governance structures and policies, including how contributors become committers/maintainers, approvers, reviewers, members, code of conduct, etc.-3. Control access to, establish processes regarding, and provide a final escalation path for any Istio repository.-4. Control and delegate access to and establish processes regarding other project resources and assets, including artifact repositories, build and test infrastructure, web sites and their domains, blogs, social-media accounts, etc.-5. Manage the Istio brand to decide which things can be called “Istio” and how that mark can be used in relation to other efforts or vendors.-6. Resolve any dispute from the Technical Oversight Committee.+The Steering Committee’s responsibilities include:++1. Establishing rules of governance for the Istio project, including creation+and ratification of bylaws for the Steering Committee and Technical Oversight+Committee.+1. Establishing and enforcing principles to guide the Istio project.+1. Fostering an environment for a healthy and happy community of developers,+contributors, and users.+1. Defining, evolving, and defending a+[Code of Conduct](CONTRIBUTING.md#code-of-conduct).+1. Advising the trademark owner on issues relating to the Istio trademark and+logo.+1. Setting marketing and advocacy direction for the project; establishing a+publishing schedule and vetting content, encouraging and assisting in project+community members, fostering an ecosystem of vendors, creating content for+conferences, etc.+1. Controlling and delegating access to, and establishing processes regarding,+project resources/assets, including but not limited to artifact repositories,+build and test infrastructure, web sites and their domains, blogs, and social+media accounts.+1. Providing neutral mediation as appropriate to try to resolve non-technical+disputes that arise as part of the project.++## Membership and voting+1.  The Steering Committee is structured to allow companies who are most+invested in the success of the Istio project to participate in business and+non-technical decision-making.+1.  All members should abide by the project Code of Conduct.+1.  There are two types of seats on the Steering Committee: Contributor Seats+and Community Seats.+1.  Both Contributor and Community Seats will be appointed beginning in+July 2020. The appointments for the Contribution and Community seat types will+expire on staggered dates. After the initial term for both seats, all seats+will have an annual term.+1.  Contributor Seat terms expire on January 1 and Community Seat terms expire+on July 1. If necessary, a company holding a Contributor Seat may change the+appointed individual at any time during the term.+1.  No Contributing Company can have more than 5 seats in total on the Steering+Committee.+    1.  Subsidiaries, an entity in which another company owns more than 50%+    of the voting or membership interests; and related companies, companies+    which are controlled by a common third party entity; and affiliate+    companies, will be treated together as a single entity, referred to as a+    Contributing Company.+1.  There shall be nine **Contributor Seats**. Their allocation is determined+by the approximate effort and expenditure on the Istio project, as approximated+by the number of merged pull requests on GitHub over the one year period prior+to the Contributor Seat assignments:+    1.  The top three contributing companies to Istio are eligible for+    Contributor Seats, proportional to their contribution.+        1.  Each company is allocated one seat;+        1.  The remaining six seats are allocated based on percentage project+        contribution, with no Contributing Company exceeding 5 seats in total+        as outlined in section 6.+1.  There shall be four **Community Seats.**+    1.  Two **Community Seats** will be elected by the Istio contributors and+    community, beginning in July 2020.+        1.  Any [project member](ROLES.md#member) can self-nominate for the+        election, or nominate another project member with their consent.+        1.  Elections use time-limited, Condorcet voting.+        1.  The following are eligible to vote for Community Seats:+            1.  Project [Members](ROLES.md#member) who have had a merged PR in+            the 12 months prior to the election; and+            1.  People who have submitted a voting exception form to the+            Steering Committee, demonstrating contribution to the Istio project+            that is of a non-code nature in the 12 months prior to the election,+            and are granted a vote for the election by a simple majority vote of+            the Steering Committee.+        1.  Community Seats are maintained for the term by the individual, even+        if they change their company affiliation.+            1.  If an individual changes company affiliation mid-term in a way+            that is incompatible with the company representation policies in+            this Charter, the individual may keep their seat for the duration of+            the term.+    1.  Two Community Seats will be appointed through a vote of the Steering+    Committee after the community election.+        1.  These seats are intended to diversify perspectives and expertise on+        the Committee.+    1.  Because the goal of Community Seats is to increase the perspectives on

That definitely does! I think that wasn't there when I commented it, or I missed it

oaktowner

comment created time in 2 days

push eventhowardjohn/istio

John Howard

commit sha 7030634a37b6a0bc40e49f313862ea1fab0581ad

lint

view details

push time in 2 days

push eventhowardjohn/istio

John Howard

commit sha f3ddba4bb02d1414798c78505bee551539de8350

fix build

view details

push time in 2 days

pull request commentistio/istio

Make TypeUrl handling generic in ADS

@costinm to avoid making either part of the code worse, for now I just split them. We can converge in the future

howardjohn

comment created time in 2 days

push eventhowardjohn/istio

Steven Landow

commit sha f7a2d3355aeb661c0f172cb6bf3209773c37060c

use local registry for kind (#24957) * use local registry for kind * fix * remove leftpadding for multicluster kind cfg * add leftpad to containerdConfigPatches * use kind.x-k8s.io/v1alpha4 * kind registry uses same internal/external port * remove containerdConfigPatches from trustworthy-*.yaml * cleanup old kind yamls * measure test setup time * capture all images * cleanup kind_push_image(s); don't use local when SKIP_BUILD * fix hub initial assignment

view details

Costin Manolache

commit sha 9529c229ea2510dcfe31f70af1b70a64214f7f44

Remove UDP port (#25008) * Remove UDP port * make gen Co-authored-by: Costin Manolache <costin@google.com>

view details

Pengyuan Bian

commit sha d21ab33479af75d9386f6d4d067c4bd82971a7cb

Raise default protocol detection timeout (#24763) * raise default protocol detection timeout * increate time out

view details

John Howard

commit sha 587c51b9730f7e221bd5c2494e26a9355bd69120

Log framework scope even without CI flag (#25012) This way users still see some level of logging. They can always disable it explicitly if, for whatever reason, they want no logs. The context here its impossible to run with framework:info currently, which is my preferred log level

view details

Istio Automation

commit sha f4a14f53726a8c182aba7d3c753cd4d56de159ee

Automator: update istio/api@master dependency in istio/istio@master (#25004)

view details

Steven Dake

commit sha 2ccef0c2c01a73f207497109ba39b1cd20e1daf6

Lock down the operator process security context (#24963) * Lock down the operator process security context 1. Ensure the filesystem is readonly 2. Disable any privilege escalation 3. Drop all capabilities * Address reviewer comments

view details

John Howard

commit sha c4242d6dc783616a6f3c3254826e8c9c5cbc8204

Fix wrong error log in operator (#25017)

view details

John Howard

commit sha 3451d9d4b48913dc6d2cd0414dedc185551c1f70

Drop requests from 100 -> 50 for locality tests (#25011) Sending 100 requests at once seems to overwhelm CI sometimes: https://github.com/istio/istio/issues/25010 Trying to drop it down a bit

view details

Robert Panzer

commit sha 81d53479aa9daa14ee9f809a34b6ed0afe198a51

Fixes 24969. Fix eds endpoint selection for subsets with no or empty subset labels (#25020)

view details

Navraj Singh Chhina

commit sha 017f49f86e4babe8cebf36965453902884a13482

refactor applyUpstreamTLSSettings (#24822) * refactor upstreamTLS * add todo and change comment * split mutual and simple

view details

Ed Snible

commit sha 2c4e69a32577769e47cc40b44d575919318526da

Don't consider it a problem if no destination rule is found (#24955)

view details

John Howard

commit sha dc4e59d7d6696f4b491a35eaa351b2af355c0a76

Add back deleted tests (#25016) In a recent commit these somehow got deleted, and our test decided a missing input file is a t.Skip() instead of t.Fatal(). This rolls back those files, and makes it an error if they are missing

view details

Shamsher Ansari

commit sha ab13f9d73fe37367e7c77964c0baa97bde55a3f2

Update go to 1.14 in go.mod (#25022)

view details

Rama Chavali

commit sha 0dc31434d2e75475e17f6e47350c7add240e78e5

move xds implementation to xds package (#24837) * move xds implementation to xds package Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * revert unnecessary change Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * lint Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * remove redundant type defs Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * move to pkg/xds Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * update codecov threshold Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * lint Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * move inside pilot Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix import Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

view details

Ed Snible

commit sha 62e4fabd1d256c0619123e416603026cb5289a2f

'dashboard' should block and prefer remote port. 'proxy-config' portforward close log fix (#24945)

view details

Shamsher Ansari

commit sha c5c4ee41f5da2f40996019fca0eb62dc5025fbb6

Update go version for tcp-echo sample and rpm build (#25025) * Update go version for tcp-echo sample and rpm build * Update go version to 1.14.2

view details

Navraj Singh Chhina

commit sha dd9704e2a6ddd27d8fa62e6a0ae94970d9351bdc

TLS origination by Egress Gateway integration tests (#24786) * cleanup mess and start afresh * rebase and squash more files to be removed test if all traffic is going through egress gw * retry traffic * retry traffic 2 * retry traffic 3 * retry traffic 4 * add new config format * change 504 -> 400 * add test for ill-configured root cert cleanup mtls test case and a lot of comments add strict mtls on simple tls origination rename destination to server test another case add mutual test add mTLS test case and remove portName handle comments change deleteconfig remove root cert remove ctx cleanup fix quotes skip mtls tests patch egress gw with certs add fake root cert test make gen and cleanup make gen and cleanup2 * fix lint line break * put dr tls tests back in whitelist * use retry instead of timeout wait * green CI please * move comments to tc * mount secrets directly * remove camount clutter

view details

John Howard

commit sha 25bf46237382f5e6de7c3b8f1cfa8958ec0f9f28

Align helm and istioctl settings (#25026)

view details

John Howard

commit sha badfdb3b734c77e7fbd9e486a76dd8b0717cce5a

Update service-apis (#25014) * Update service-apis * Yaml library updated * Update licenses

view details

Istio Automation

commit sha 9d054bda2655299f38036c2c399e4af2d5dafe01

Automator: update istio/api@master dependency in istio/istio@master (#25032)

view details

push time in 2 days

pull request commentistio/istio

Fix ingress crash on shutdown

/retest

howardjohn

comment created time in 2 days

issue commentistio/istio

Istio proxy fais with JWT token missing JWT token

Are you sure nothing is modifying the istio-sidecar-injector configmap or mutating webhook somehow? I don't see how else this could occur...

dawn-chu

comment created time in 2 days

PR opened istio/istio

kube controller: remove metadata informer

This is no longer used now, we have the full node informer so we might as well just use that.

+28 -49

0 comment

3 changed files

pr created time in 2 days

create barnchhowardjohn/istio

branch : pilot/remove-metadata-informer

created branch time in 2 days

Pull request review commentistio/istio

Extend security test for VMs

 type Context struct { 	Multiversion echo.Instance 	Headless     echo.Instance 	Naked        echo.Instance+	VM           echo.Instance }  // CreateContext creates and initializes reachability context.-func CreateContext(ctx framework.TestContext, p pilot.Instance) Context {+func CreateContext(ctx framework.TestContext, p pilot.Instance, buildVM bool) Context {

Please don't make a new install... meshExpansion being enabled doesn't make all those tests you mentioned less valid... just so we are on the same page - the flag simply opens a few more ports on the ingress gateway. I don't see how that can impact the other tests and give us less test coverage, at least not worth an extra 10 min of testing

ZhengzheYang

comment created time in 2 days

issue commentistio/istio

Istio proxy fais with JWT token missing JWT token

They must be getting injected by a different instance of the sidecar injector. Do you somehow have an old and new istio running concurrently?

On Wed, Jul 1, 2020 at 1:48 PM pxzbwdk notifications@github.com wrote:

I can see that the pods who keep restarting are missing the annotation of "istio-token" and "istiod-ca-cert". they instead has "istio-certs". they are also missing JWT_POLICY, istio-token. Do you know how can those pods missing those, there is no different setting that we did

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/istio/istio/issues/25138#issuecomment-652638850, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEYGXJHQKKPFL7C5RW562LRZOOJFANCNFSM4OM3Y5DQ .

dawn-chu

comment created time in 2 days

create barnchhowardjohn/istio

branch : pilot/kube-queue-by-name

created branch time in 2 days

pull request commentistio/istio

Add prometheus parsing to metric merge test

@istio/wg-networking-maintainers ptal

howardjohn

comment created time in 2 days

push eventhowardjohn/istio

John Howard

commit sha 86ea3f280ae8c69b02105959c1dd26f92d64cd81

fix registration

view details

push time in 2 days

pull request commentistio/istio

Incremental RDS requests

/test all

howardjohn

comment created time in 2 days

PR opened istio/istio

Incremental RDS requests

Based upon https://github.com/istio/istio/pull/25143

+28 -12

0 comment

3 changed files

pr created time in 2 days

create barnchhowardjohn/istio

branch : pilot/incremental-rds

created branch time in 2 days

push eventhowardjohn/istio

John Howard

commit sha 1e4255a5d49c9907372313919eea1dfb5b361291

cleanup dead code

view details

push time in 2 days

pull request commentistio/istio

Update proxy sha

diff --git a/tools/packaging/common/envoy_bootstrap.json b/tools/packaging/common/envoy_bootstrap.json
index 6d39f1e1fd..492f461b0a 100644
--- a/tools/packaging/common/envoy_bootstrap.json
+++ b/tools/packaging/common/envoy_bootstrap.json
@@ -21,6 +21,20 @@
     },
     "metadata": {{ .meta_json_str }}
   },
+  "layered_runtime": {
+      "layers": [
+          {
+              "name": "deprecation",
+              "static_layer": {
+                  "envoy.deprecated_features:envoy.config.listener.v3.Listener.hidden_envoy_deprecated_use_original_dst": true
+              }
+          },
+          {
+              "name": "admin",
+              "admin_layer": {}
+          }
+      ]
+  },
   "stats_config": {
     "use_all_default_tags": false,
     "stats_tags": 
bianpengyuan

comment created time in 2 days

pull request commentistio/istio

Update proxy sha

oh maybe that was an old log. Anyhow I am looking into it

bianpengyuan

comment created time in 2 days

pull request commentistio/istio

Update proxy sha

https://github.com/envoyproxy/envoy/pull/11160 is the breaker

@bianpengyuan looks like stats may also be not working though?

virtualInbound: Unable to parse JSON as proto (INVALID_ARGUMENT:(config.configuration): invalid value "{
  "debug": "false",
  "stat_prefix": "istio"
}
" for type type.googleapis.com/google.protobuf.Any): {"config":{"root_id":"stats_inbound","configuration":"{\n  \"debug\": \"false\",\n  \"stat_prefix\": \"istio\"\n}\n","vm_config":{"vm_id":"tcp_stats_inbound","code":{"local":{"inline_string":"envoy.wasm.stats"}},"runtime":"envoy.wasm.runtime.null"}}}
bianpengyuan

comment created time in 2 days

push eventhowardjohn/istio

John Howard

commit sha 953222208669cd3463db6a8b13994db5fafbb59a

Add comment

view details

push time in 2 days

PR opened istio/istio

kube: remove legacy client in Istiod

Please provide a description for what this PR is for.

And to help us figure out who should review this PR, please put an X in all the areas that this PR affects.

[ ] Configuration Infrastructure [ ] Docs [ ] Installation [ ] Networking [ ] Performance and Scalability [ ] Policies and Telemetry [ ] Security [ ] Test and Release [ ] User Experience [ ] Developer Infrastructure

+11 -16

0 comment

3 changed files

pr created time in 2 days

PR closed istio/istio

Move injector to shared kube client size/XS

This one is almost a NOP, as injection has a watch on a single object. I think the best thing to do here is to create a new informer?

[ ] Configuration Infrastructure [ ] Docs [ ] Installation [ ] Networking [ ] Performance and Scalability [ ] Policies and Telemetry [ ] Security [ ] Test and Release [ ] User Experience [ ] Developer Infrastructure

+5 -4

1 comment

2 changed files

howardjohn

pr closed time in 2 days

create barnchhowardjohn/istio

branch : kube/remove-legacy-client

created branch time in 2 days

PR opened istio/istio

Move injector to shared kube client

This one is almost a NOP, as injection has a watch on a single object. I think the best thing to do here is to create a new informer?

Please provide a description for what this PR is for.

And to help us figure out who should review this PR, please put an X in all the areas that this PR affects.

[ ] Configuration Infrastructure [ ] Docs [ ] Installation [ ] Networking [ ] Performance and Scalability [ ] Policies and Telemetry [ ] Security [ ] Test and Release [ ] User Experience [ ] Developer Infrastructure

+5 -4

0 comment

2 changed files

pr created time in 2 days

create barnchhowardjohn/istio

branch : kube/share-informer-injection

created branch time in 2 days

issue commentistio/istio

Modify control plane logging levels with istioctl

curl localhost:9876/scopej/ads -X PUT -d '{"output_level":"debug"}' is the api

howardjohn

comment created time in 2 days

pull request commentistio/istio

pilot: incremental EDS requests

/test all

howardjohn

comment created time in 2 days

PR opened istio/istio

Fix ingress crash on shutdown

Currently this will panic when pilot exits; we should not run the informers ourselves anymore, as they can only call .Run() one time and they are shared. This is already called later at the top level

Please provide a description for what this PR is for.

And to help us figure out who should review this PR, please put an X in all the areas that this PR affects.

[ ] Configuration Infrastructure [ ] Docs [ ] Installation [ ] Networking [ ] Performance and Scalability [ ] Policies and Telemetry [ ] Security [ ] Test and Release [ ] User Experience [ ] Developer Infrastructure

+0 -5

0 comment

1 changed file

pr created time in 2 days

create barnchhowardjohn/istio

branch : pilot/ingress-no-crash

created branch time in 2 days

push eventhowardjohn/istio

John Howard

commit sha d488df8cc49d91e78cbc13a29afa879a6b810077

Only filter if we explicitly set the map

view details

push time in 2 days

issue commentistio/istio

client-go: unable to use AddToScheme for creating a controller based on istio CRDs

I think you need the k8s 1.18 client libraries

harpratap

comment created time in 2 days

issue commentistio/istio

Failed to exec command with istioctl compiled by myself

You need to either run make gen-charts before building, or pass the flag --charts manifests/ , as the error indicates

thefuckingcode

comment created time in 2 days

pull request commentistio/istio

Trigger endpoint update when pod comes in

/test all

howardjohn

comment created time in 2 days

PR opened istio/istio

Trigger endpoint update when pod comes in

@ramaraochavali @nrjpoddar what do you think? The basic idea is we record any pods we are missing. When those pods in later, we re-add the endpoint to the queue.

TODO: there is still a memory leak if the pod never comes

+65 -10

0 comment

4 changed files

pr created time in 2 days

create barnchhowardjohn/istio

branch : pilot/eds-async

created branch time in 2 days

pull request commentistio/istio

xds: move pilot cli to v3

/retest

ramaraochavali

comment created time in 2 days

pull request commentistio/istio

Update proxy sha

https://storage.googleapis.com/istio-prow/pr-logs/pull/istio_istio/25159/integ-operator-controller-tests_istio/3601/artifacts/operator-573ee727e6d346cb91aa15/TestController/default-1-80943-state145871668/client-v1-86748ccc8f-hg5pj-istio-proxy.log

wasm code changed?

bianpengyuan

comment created time in 2 days

push eventhowardjohn/istio

Martin Ostrowski

commit sha 4883d70091bb15dd86e53c1c8ef4d201de925243

Deprecate --charts flag in favor or --manifests (#24749) * Deprecate --charts flag in favor or --manifests * Fix bad auto replace

view details

Costin Manolache

commit sha be7004a7c619bc7baf725942953e3d043364c216

Security changes for testing (#25090) * Security changes for testing * More chanages need to be moved from original PR * More changes * Added the test file change to start secure grpc * Review and format

view details

John Howard

commit sha 6b87d56f492c291f704f47d39d3acb8ecceed7ed

ingress controller: use shared informers (#25126) * ingress controller: use shared informers * Update tests * Fix lint * fix import

view details

Steven Landow

commit sha 99d1200d0de09252ff10a4de05373746cd421358

fix: get resource from scope traverses parents (#25134) * fix: get resource from scope traverses parents * copyright banner

view details

John Howard

commit sha 9d7cf2d63af42de63ec807e95224066b03fe55e8

Don't initialize multiple default sidecars for ns (#25118) Currently, we initalize a default sidecar scope for each *service* - but the scopes only differ by namespace. Instead, we can just create one per namespace required. name old time/op new time/op InitPushContext/gateways-8 5.90ms ± 0% 3.17ms ± 0% InitPushContext/empty-8 268ns ± 0% 245ns ± 0% InitPushContext/telemetry-8 273ns ± 0% 247ns ± 0% InitPushContext/virtualservice-8 163ns ± 0% 149ns ± 0% I often see this codepath using a large chunk of Pilot CPU in real world tests, so I think the change is significant in these cases.

view details

Costin Manolache

commit sha 8d45bb66ae89bd6ae99edb2c87551d9e58e165a3

Move init connection to receive thread (#25124) * Security changes for testing * More chanages need to be moved from original PR * More changes * Added the test file change to start secure grpc * Review and format * Move init to the receive thread * Moving processing the request to receive thread as well * Move back process request to main thread, add comments * Make fmt

view details

Ed Snible

commit sha 210f8118b66c2138ff75ecef89e9b2f478990075

Send envoy.config.core.v3.ControlPlane proto with experimental XDS (#25042) * Send envoy.config.core.v3.ControlPlane proto with experimental/gen2 XDS responses * Use JSON format for control plane identifier * Don't regenerate ControlPlane on every message * Use a struct rather than a map to define config.core.v3.ControlPlane.identifier * Use 'istiod' instead of 'pilot' for identity

view details

John Howard

commit sha 841e3441360e5bc9e0524c746cde7c6428c5db01

Fix Ingress that is missing secretName (#24780) * Fix Ingress that is missing secretName Fixes https://github.com/istio/istio/issues/24758 * Update conversion_test.go

view details

Rama Chavali

commit sha 030c4c01aebaceb7ceeb839297bc5739873105c5

fix incorrect comment (#25150) Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

view details

Rama Chavali

commit sha b900d31394d8ea29ae21dd356950c8e59cd66777

fix xds updater interface docs (#25152) Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

view details

John Howard

commit sha bdac0787e932c747154e996cfb2aabef6c820b88

Improvements to XDS benchmarks (#25117) * Improvements to XDS benchmarks * Add new test for gateways. This is motivated by https://github.com/istio/istio/issues/25116 * Make the tests more flexible/explicit. All config is now in the test files. This does make the test a bit larger, as we define the Services and service instance in each test, but its more flexible and explicit so I think its worth it * Add more debugability. Tests now report the XDS size(in bytes and # of resources), and with a flag can do a configdump. * Add a test for push context initialization. This can help us make decisions about what makes sense to put in push context and what doesn't, as push context is essentially a cache. * format * Add different gateway type * add istio version

view details

John Howard

commit sha 5675f95d7b1b06c787b98c4e8c40eaea8d7d7553

Store virtual services by gateway for perf

view details

push time in 2 days

pull request commentistio/istio

xds: move pilot cli to v3

/retest

ramaraochavali

comment created time in 2 days

push eventhowardjohn/istio

Brian Cheung

commit sha 7d6670ed88b872929c3df8fb50c81e123fef7f72

Fix CNI installation/configuration race condition and churn (#25088) * Wait until CNI config file exists to install Istio CNI as a chained CNI plugin * Rename environment variables for consistency * Clean up variable expansion for consistency * Refactor file cleanup; Use mktemp for temp files * Add unit tests for waiting for valid CNI config file * Restart script (loop) on invalid config instead of exiting with error * Use .conf instead if valid config and .conflist missing * Cleanup: add function keyword, make variables local, add comment * Fix lint errors and spacing * Add output comments and removed unecessary logs * Add helper call for auxiliary test functions * Change NONE to empty string for magic value * Change find CNI config sleep interval to user defined var * Move log statement out of while loop * Speed up unit tests with polling and timeouts instead of sleep * Fix ticker in unit tests

view details

Istio Automation

commit sha e57173ea95c1ec7531f6a63d688e6348efa7008a

Automator: update istio/api@master dependency in istio/istio@master (#25125)

view details

Istio Automation

commit sha 9f808522010e6c047ae11507f42ca663fd183efa

Automator: update common-files@master in istio/istio@master (#25122)

view details

Martin Ostrowski

commit sha 4883d70091bb15dd86e53c1c8ef4d201de925243

Deprecate --charts flag in favor or --manifests (#24749) * Deprecate --charts flag in favor or --manifests * Fix bad auto replace

view details

Costin Manolache

commit sha be7004a7c619bc7baf725942953e3d043364c216

Security changes for testing (#25090) * Security changes for testing * More chanages need to be moved from original PR * More changes * Added the test file change to start secure grpc * Review and format

view details

John Howard

commit sha 6b87d56f492c291f704f47d39d3acb8ecceed7ed

ingress controller: use shared informers (#25126) * ingress controller: use shared informers * Update tests * Fix lint * fix import

view details

Steven Landow

commit sha 99d1200d0de09252ff10a4de05373746cd421358

fix: get resource from scope traverses parents (#25134) * fix: get resource from scope traverses parents * copyright banner

view details

John Howard

commit sha 9d7cf2d63af42de63ec807e95224066b03fe55e8

Don't initialize multiple default sidecars for ns (#25118) Currently, we initalize a default sidecar scope for each *service* - but the scopes only differ by namespace. Instead, we can just create one per namespace required. name old time/op new time/op InitPushContext/gateways-8 5.90ms ± 0% 3.17ms ± 0% InitPushContext/empty-8 268ns ± 0% 245ns ± 0% InitPushContext/telemetry-8 273ns ± 0% 247ns ± 0% InitPushContext/virtualservice-8 163ns ± 0% 149ns ± 0% I often see this codepath using a large chunk of Pilot CPU in real world tests, so I think the change is significant in these cases.

view details

Costin Manolache

commit sha 8d45bb66ae89bd6ae99edb2c87551d9e58e165a3

Move init connection to receive thread (#25124) * Security changes for testing * More chanages need to be moved from original PR * More changes * Added the test file change to start secure grpc * Review and format * Move init to the receive thread * Moving processing the request to receive thread as well * Move back process request to main thread, add comments * Make fmt

view details

Ed Snible

commit sha 210f8118b66c2138ff75ecef89e9b2f478990075

Send envoy.config.core.v3.ControlPlane proto with experimental XDS (#25042) * Send envoy.config.core.v3.ControlPlane proto with experimental/gen2 XDS responses * Use JSON format for control plane identifier * Don't regenerate ControlPlane on every message * Use a struct rather than a map to define config.core.v3.ControlPlane.identifier * Use 'istiod' instead of 'pilot' for identity

view details

John Howard

commit sha 841e3441360e5bc9e0524c746cde7c6428c5db01

Fix Ingress that is missing secretName (#24780) * Fix Ingress that is missing secretName Fixes https://github.com/istio/istio/issues/24758 * Update conversion_test.go

view details

Rama Chavali

commit sha 030c4c01aebaceb7ceeb839297bc5739873105c5

fix incorrect comment (#25150) Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

view details

Rama Chavali

commit sha b900d31394d8ea29ae21dd356950c8e59cd66777

fix xds updater interface docs (#25152) Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

view details

John Howard

commit sha bdac0787e932c747154e996cfb2aabef6c820b88

Improvements to XDS benchmarks (#25117) * Improvements to XDS benchmarks * Add new test for gateways. This is motivated by https://github.com/istio/istio/issues/25116 * Make the tests more flexible/explicit. All config is now in the test files. This does make the test a bit larger, as we define the Services and service instance in each test, but its more flexible and explicit so I think its worth it * Add more debugability. Tests now report the XDS size(in bytes and # of resources), and with a flag can do a configdump. * Add a test for push context initialization. This can help us make decisions about what makes sense to put in push context and what doesn't, as push context is essentially a cache. * format * Add different gateway type * add istio version

view details

John Howard

commit sha 7378afb2ce744843906bd163befa92c74df12c2c

Update namespace controller to use shared informer

view details

John Howard

commit sha 78afee610119359ae929176f327cc1f37235bdb2

Fix leader election

view details

push time in 2 days

push eventhowardjohn/istio

Nathan Mittler

commit sha 365a94720d5a5290fafeaee85d49ad83f144be1e

Merge kube Accessor and client (#25075) This is a pretty large change with a lot of cleanup. Overview of the changes: - Moved many of the Accessor methods to utilities. - Separated out the function of writing yaml content to files from the kube.Client API. The suite/test contexts now handle that work on behalf of the clusters. The ConfigManager interface is no longer implemented by Cluster. - Simplified the kube.clientFactory so that it just uses an in-memory discovery client. - Removed the kube.Deployment. It was only used in 2 places and didn't add much value.

view details

Navraj Singh Chhina

commit sha f6c18c9a84f2db1a9b7f177cf24d3cb3e31a0754

Bug Fix missing HTTP2 ALPN for UpstreamTLSContext (#25089) * bug fix h2 for simple TLS mode * lint

view details

Daniel Grimm

commit sha 32e7071abe3e96af09267106b1b9d6ecd5d36b98

manifests: enable mounting configVolumes into gateways (#25114)

view details

John Howard

commit sha d89c6a624cfd0962e2ed25238ec671baebda0ca5

Use a common kubeclient/shared informer throughout Istiod (#25031) * Add initial struct * Address comments * fix test * fix test * No clientFactory hacks * Change constructor

view details

Brian Cheung

commit sha 7d6670ed88b872929c3df8fb50c81e123fef7f72

Fix CNI installation/configuration race condition and churn (#25088) * Wait until CNI config file exists to install Istio CNI as a chained CNI plugin * Rename environment variables for consistency * Clean up variable expansion for consistency * Refactor file cleanup; Use mktemp for temp files * Add unit tests for waiting for valid CNI config file * Restart script (loop) on invalid config instead of exiting with error * Use .conf instead if valid config and .conflist missing * Cleanup: add function keyword, make variables local, add comment * Fix lint errors and spacing * Add output comments and removed unecessary logs * Add helper call for auxiliary test functions * Change NONE to empty string for magic value * Change find CNI config sleep interval to user defined var * Move log statement out of while loop * Speed up unit tests with polling and timeouts instead of sleep * Fix ticker in unit tests

view details

Istio Automation

commit sha e57173ea95c1ec7531f6a63d688e6348efa7008a

Automator: update istio/api@master dependency in istio/istio@master (#25125)

view details

Istio Automation

commit sha 9f808522010e6c047ae11507f42ca663fd183efa

Automator: update common-files@master in istio/istio@master (#25122)

view details

Martin Ostrowski

commit sha 4883d70091bb15dd86e53c1c8ef4d201de925243

Deprecate --charts flag in favor or --manifests (#24749) * Deprecate --charts flag in favor or --manifests * Fix bad auto replace

view details

Costin Manolache

commit sha be7004a7c619bc7baf725942953e3d043364c216

Security changes for testing (#25090) * Security changes for testing * More chanages need to be moved from original PR * More changes * Added the test file change to start secure grpc * Review and format

view details

John Howard

commit sha 6b87d56f492c291f704f47d39d3acb8ecceed7ed

ingress controller: use shared informers (#25126) * ingress controller: use shared informers * Update tests * Fix lint * fix import

view details

Steven Landow

commit sha 99d1200d0de09252ff10a4de05373746cd421358

fix: get resource from scope traverses parents (#25134) * fix: get resource from scope traverses parents * copyright banner

view details

John Howard

commit sha 9d7cf2d63af42de63ec807e95224066b03fe55e8

Don't initialize multiple default sidecars for ns (#25118) Currently, we initalize a default sidecar scope for each *service* - but the scopes only differ by namespace. Instead, we can just create one per namespace required. name old time/op new time/op InitPushContext/gateways-8 5.90ms ± 0% 3.17ms ± 0% InitPushContext/empty-8 268ns ± 0% 245ns ± 0% InitPushContext/telemetry-8 273ns ± 0% 247ns ± 0% InitPushContext/virtualservice-8 163ns ± 0% 149ns ± 0% I often see this codepath using a large chunk of Pilot CPU in real world tests, so I think the change is significant in these cases.

view details

Costin Manolache

commit sha 8d45bb66ae89bd6ae99edb2c87551d9e58e165a3

Move init connection to receive thread (#25124) * Security changes for testing * More chanages need to be moved from original PR * More changes * Added the test file change to start secure grpc * Review and format * Move init to the receive thread * Moving processing the request to receive thread as well * Move back process request to main thread, add comments * Make fmt

view details

Ed Snible

commit sha 210f8118b66c2138ff75ecef89e9b2f478990075

Send envoy.config.core.v3.ControlPlane proto with experimental XDS (#25042) * Send envoy.config.core.v3.ControlPlane proto with experimental/gen2 XDS responses * Use JSON format for control plane identifier * Don't regenerate ControlPlane on every message * Use a struct rather than a map to define config.core.v3.ControlPlane.identifier * Use 'istiod' instead of 'pilot' for identity

view details

John Howard

commit sha 841e3441360e5bc9e0524c746cde7c6428c5db01

Fix Ingress that is missing secretName (#24780) * Fix Ingress that is missing secretName Fixes https://github.com/istio/istio/issues/24758 * Update conversion_test.go

view details

Rama Chavali

commit sha 030c4c01aebaceb7ceeb839297bc5739873105c5

fix incorrect comment (#25150) Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

view details

Rama Chavali

commit sha b900d31394d8ea29ae21dd356950c8e59cd66777

fix xds updater interface docs (#25152) Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

view details

John Howard

commit sha bdac0787e932c747154e996cfb2aabef6c820b88

Improvements to XDS benchmarks (#25117) * Improvements to XDS benchmarks * Add new test for gateways. This is motivated by https://github.com/istio/istio/issues/25116 * Make the tests more flexible/explicit. All config is now in the test files. This does make the test a bit larger, as we define the Services and service instance in each test, but its more flexible and explicit so I think its worth it * Add more debugability. Tests now report the XDS size(in bytes and # of resources), and with a flag can do a configdump. * Add a test for push context initialization. This can help us make decisions about what makes sense to put in push context and what doesn't, as push context is essentially a cache. * format * Add different gateway type * add istio version

view details

push time in 2 days

push eventhowardjohn/pilot-load

John Howard

commit sha daf4120d12cf2d09011569046a5eac5c63af5520

Add jitter

view details

push time in 2 days

push eventhowardjohn/pilot-load

John Howard

commit sha dd43950a64c133cfb9fe6e113a2bed2668371cd7

Updated etcd

view details

push time in 2 days

pull request commentistio/istio

eds: look in informer store before going to k8s

I am a bit concerned about any approach that is not guaranteed to never directly call the API server. It seems to me basically its a ticking time bomb - if we every hit this code we will never recover, and there will be a mesh wide outage. This does make it much more rare but I am not sure its enough. I'll try playing around with it today

@hzxuzhonghu may have opinions as well

ramaraochavali

comment created time in 2 days

pull request commentistio/istio

record reverseStatus as hashset, due to high cpu utilization

2020-01-07_08-40-51

before and after your PR. Ship it!

therealmitchconnors

comment created time in 2 days

pull request commentistio/istio

Initial XDS-based (experimental) version subcommand

/test all

esnible

comment created time in 2 days

issue openedistio/istio

Spammy logs: Distribution Event Queue overwhelmed, status will be invalid.

If we trigger this condition (I did) we get an absurd amount of logs. Note that we can only trigger this condition by hitting 100,000 pending requests - then we log this every time. So we are in the order of O(100k) log messages. Pilot CPU in this case actually spend ~10% of its time on these log messages and my logging bill will probably hurt.

We should deduplicate these, only log once or something similar?

https://github.com/istio/istio/pull/24923/files might help make it less frequent but we still should never have this many logs

created time in 2 days

issue openedistio/istio

Overhead of metrics in pilot

pprof.pilot-discovery.samples.cpu.179.pb.gz

10% of total CPU 6% == worker.start which is addSample the bulk of it. I think this is just global recording overhead 2.5% == recordPushTriggers 1% == updateEdsStats

created time in 2 days

pull request commentistio/istio

xds: move pilot cli to v3

/test all

ramaraochavali

comment created time in 2 days

push eventhowardjohn/istio

Daniel Grimm

commit sha 32e7071abe3e96af09267106b1b9d6ecd5d36b98

manifests: enable mounting configVolumes into gateways (#25114)

view details

John Howard

commit sha d89c6a624cfd0962e2ed25238ec671baebda0ca5

Use a common kubeclient/shared informer throughout Istiod (#25031) * Add initial struct * Address comments * fix test * fix test * No clientFactory hacks * Change constructor

view details

Brian Cheung

commit sha 7d6670ed88b872929c3df8fb50c81e123fef7f72

Fix CNI installation/configuration race condition and churn (#25088) * Wait until CNI config file exists to install Istio CNI as a chained CNI plugin * Rename environment variables for consistency * Clean up variable expansion for consistency * Refactor file cleanup; Use mktemp for temp files * Add unit tests for waiting for valid CNI config file * Restart script (loop) on invalid config instead of exiting with error * Use .conf instead if valid config and .conflist missing * Cleanup: add function keyword, make variables local, add comment * Fix lint errors and spacing * Add output comments and removed unecessary logs * Add helper call for auxiliary test functions * Change NONE to empty string for magic value * Change find CNI config sleep interval to user defined var * Move log statement out of while loop * Speed up unit tests with polling and timeouts instead of sleep * Fix ticker in unit tests

view details

Istio Automation

commit sha e57173ea95c1ec7531f6a63d688e6348efa7008a

Automator: update istio/api@master dependency in istio/istio@master (#25125)

view details

Istio Automation

commit sha 9f808522010e6c047ae11507f42ca663fd183efa

Automator: update common-files@master in istio/istio@master (#25122)

view details

Martin Ostrowski

commit sha 4883d70091bb15dd86e53c1c8ef4d201de925243

Deprecate --charts flag in favor or --manifests (#24749) * Deprecate --charts flag in favor or --manifests * Fix bad auto replace

view details

Costin Manolache

commit sha be7004a7c619bc7baf725942953e3d043364c216

Security changes for testing (#25090) * Security changes for testing * More chanages need to be moved from original PR * More changes * Added the test file change to start secure grpc * Review and format

view details

John Howard

commit sha 6b87d56f492c291f704f47d39d3acb8ecceed7ed

ingress controller: use shared informers (#25126) * ingress controller: use shared informers * Update tests * Fix lint * fix import

view details

Steven Landow

commit sha 99d1200d0de09252ff10a4de05373746cd421358

fix: get resource from scope traverses parents (#25134) * fix: get resource from scope traverses parents * copyright banner

view details

John Howard

commit sha 9d7cf2d63af42de63ec807e95224066b03fe55e8

Don't initialize multiple default sidecars for ns (#25118) Currently, we initalize a default sidecar scope for each *service* - but the scopes only differ by namespace. Instead, we can just create one per namespace required. name old time/op new time/op InitPushContext/gateways-8 5.90ms ± 0% 3.17ms ± 0% InitPushContext/empty-8 268ns ± 0% 245ns ± 0% InitPushContext/telemetry-8 273ns ± 0% 247ns ± 0% InitPushContext/virtualservice-8 163ns ± 0% 149ns ± 0% I often see this codepath using a large chunk of Pilot CPU in real world tests, so I think the change is significant in these cases.

view details

Costin Manolache

commit sha 8d45bb66ae89bd6ae99edb2c87551d9e58e165a3

Move init connection to receive thread (#25124) * Security changes for testing * More chanages need to be moved from original PR * More changes * Added the test file change to start secure grpc * Review and format * Move init to the receive thread * Moving processing the request to receive thread as well * Move back process request to main thread, add comments * Make fmt

view details

Ed Snible

commit sha 210f8118b66c2138ff75ecef89e9b2f478990075

Send envoy.config.core.v3.ControlPlane proto with experimental XDS (#25042) * Send envoy.config.core.v3.ControlPlane proto with experimental/gen2 XDS responses * Use JSON format for control plane identifier * Don't regenerate ControlPlane on every message * Use a struct rather than a map to define config.core.v3.ControlPlane.identifier * Use 'istiod' instead of 'pilot' for identity

view details

John Howard

commit sha 841e3441360e5bc9e0524c746cde7c6428c5db01

Fix Ingress that is missing secretName (#24780) * Fix Ingress that is missing secretName Fixes https://github.com/istio/istio/issues/24758 * Update conversion_test.go

view details

Rama Chavali

commit sha 030c4c01aebaceb7ceeb839297bc5739873105c5

fix incorrect comment (#25150) Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

view details

Rama Chavali

commit sha b900d31394d8ea29ae21dd356950c8e59cd66777

fix xds updater interface docs (#25152) Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

view details

John Howard

commit sha 71a7074a2f01da54d526d6d755f41b24733d6599

Improvements to XDS benchmarks * Add new test for gateways. This is motivated by https://github.com/istio/istio/issues/25116 * Make the tests more flexible/explicit. All config is now in the test files. This does make the test a bit larger, as we define the Services and service instance in each test, but its more flexible and explicit so I think its worth it * Add more debugability. Tests now report the XDS size(in bytes and # of resources), and with a flag can do a configdump. * Add a test for push context initialization. This can help us make decisions about what makes sense to put in push context and what doesn't, as push context is essentially a cache.

view details

John Howard

commit sha 1bccf1927609cb844698a43e50dffca8106bb546

format

view details

John Howard

commit sha e81e14f5e1817987d98214c8c6fef12f5423173d

Add different gateway type

view details

John Howard

commit sha c283094cef4b77bea3c64f48ff63cec14cb4368b

add istio version

view details

push time in 2 days

pull request commentistio/istio

Make proxyv2 image building operations universal

Not strongly against this right now, just worried about the future for both projects - I don't want MOSN to be slowed down by Istio and also don't want Istio to take on too much complexity (in the future - this PR is simple).

I am also slightly worried about the variable rename, as I know a number of build pipelines setting these currently.

GLYASAI

comment created time in 2 days

Pull request review commentistio/community

Updating Steering Committee charter

 # Istio Steering Committee  The Istio Steering Committee was formed to oversee the administrative aspects of the project, including governance, branding, and marketing.-Steering was created to allow the Technical Oversight Committee to exclusively focus on the technical aspects of the project. +Steering was created to allow the Technical Oversight Committee to exclusively focus on the technical aspects of the project.  * [Charter](#charter)+* [Membership and Voting](#membership-and-voting) * [Committee meetings](#committee-meetings) * [Committee members](#committee-members) * [General questions](#general-questions) * [Getting in touch](#getting-in-touch)  ## Charter -1. Define, evolve, and defend the vision, values, mission, and scope of the project - to establish and maintain the soul of Istio.-2. Define and evolve project governance structures and policies, including how contributors become committers/maintainers, approvers, reviewers, members, code of conduct, etc.-3. Control access to, establish processes regarding, and provide a final escalation path for any Istio repository.-4. Control and delegate access to and establish processes regarding other project resources and assets, including artifact repositories, build and test infrastructure, web sites and their domains, blogs, social-media accounts, etc.-5. Manage the Istio brand to decide which things can be called “Istio” and how that mark can be used in relation to other efforts or vendors.-6. Resolve any dispute from the Technical Oversight Committee.+The Steering Committee’s responsibilities include:++1. Establishing rules of governance for the Istio project, including creation+and ratification of bylaws for the Steering Committee and Technical Oversight+Committee.+1. Establishing and enforcing principles to guide the Istio project.+1. Fostering an environment for a healthy and happy community of developers,+contributors, and users.+1. Defining, evolving, and defending a+[Code of Conduct](CONTRIBUTING.md#code-of-conduct).+1. Advising the trademark owner on issues relating to the Istio trademark and+logo.+1. Setting marketing and advocacy direction for the project; establishing a+publishing schedule and vetting content, encouraging and assisting in project+community members, fostering an ecosystem of vendors, creating content for+conferences, etc.+1. Controlling and delegating access to, and establishing processes regarding,+project resources/assets, including but not limited to artifact repositories,+build and test infrastructure, web sites and their domains, blogs, and social+media accounts.+1. Providing neutral mediation as appropriate to try to resolve non-technical+disputes that arise as part of the project.++## Membership and voting+1.  The Steering Committee is structured to allow companies who are most+invested in the success of the Istio project to participate in business and+non-technical decision-making.+1.  All members should abide by the project Code of Conduct.+1.  There are two types of seats on the Steering Committee: Contributor Seats+and Community Seats.+1.  Both Contributor and Community Seats will be appointed beginning in+July 2020. The appointments for the Contribution and Community seat types will+expire on staggered dates. After the initial term for both seats, all seats+will have an annual term.+1.  Contributor Seat terms expire on January 1 and Community Seat terms expire+on July 1. If necessary, a company holding a Contributor Seat may change the+appointed individual at any time during the term.+1.  No Contributing Company can have more than 5 seats in total on the Steering+Committee.+    1.  Subsidiaries, an entity in which another company owns more than 50%+    of the voting or membership interests; and related companies, companies+    which are controlled by a common third party entity; and affiliate+    companies, will be treated together as a single entity, referred to as a+    Contributing Company.+1.  There shall be nine **Contributor Seats**. Their allocation is determined+by the approximate effort and expenditure on the Istio project, as approximated+by the number of merged pull requests on GitHub over the one year period prior+to the Contributor Seat assignments:+    1.  The top three contributing companies to Istio are eligible for+    Contributor Seats, proportional to their contribution.+        1.  Each company is allocated one seat;+        1.  The remaining six seats are allocated based on percentage project+        contribution, with no Contributing Company exceeding 5 seats in total+        as outlined in section 6.+1.  There shall be four **Community Seats.**+    1.  Two **Community Seats** will be elected by the Istio contributors and+    community, beginning in July 2020.+        1.  Any [project member](ROLES.md#member) can self-nominate for the+        election, or nominate another project member with their consent.+        1.  Elections use time-limited, Condorcet voting.+        1.  The following are eligible to vote for Community Seats:+            1.  Project [Members](ROLES.md#member) who have had a merged PR in+            the 12 months prior to the election; and+            1.  People who have submitted a voting exception form to the+            Steering Committee, demonstrating contribution to the Istio project+            that is of a non-code nature in the 12 months prior to the election,+            and are granted a vote for the election by a simple majority vote of+            the Steering Committee.+        1.  Community Seats are maintained for the term by the individual, even+        if they change their company affiliation.+            1.  If an individual changes company affiliation mid-term in a way+            that is incompatible with the company representation policies in+            this Charter, the individual may keep their seat for the duration of+            the term.+    1.  Two Community Seats will be appointed through a vote of the Steering+    Committee after the community election.+        1.  These seats are intended to diversify perspectives and expertise on+        the Committee.+    1.  Because the goal of Community Seats is to increase the perspectives on

what if they change companies?

oaktowner

comment created time in 2 days

pull request commentistio/istio

Make proxyv2 image building operations universal

How many other customizations will MOSN need in the future? It seems to me like it may be more practical for MOSN to have its own dockerfile?

GLYASAI

comment created time in 2 days

issue commentistio/istio

Stale endpoints may cause istio-pilot to be stuck

Do not understand why would pilot will run in to loop

I don't think its an infinite loop, its just stuck looping over a single endpoint's pod IPs and doing look ups. I think in there case they have 1k pod scale, so a loop over alll the endpoints at 5qps is going to be very very slow. And the next event around - probably the same

phsiao

comment created time in 2 days

issue commentistio/istio

Istio proxy fais with JWT token missing JWT token

There should be a volume like

- name: istio-token
    projected:
      sources:
      - serviceAccountToken:
          audience: istio-ca
          expirationSeconds: 43200
          path: istio-token

and env var

    env:
    - name: JWT_POLICY
      value: third-party-jwt

The template unconditionally adds the env var, so I don't see how its missing:

        env:
        - name: JWT_POLICY
          value: {{ .Values.global.jwtPolicy }}

Can you run kubectl get cm istio-sidecar-injector -oyaml -n istio-system?

dawn-chu

comment created time in 2 days

Pull request review commentistio/istio

eds: look in informer store before going to k8s

 func (pc *PodCache) getPodByIP(addr string) *v1.Pod { 	return item.(*v1.Pod) } -// getPod loads the pod from k8s.+// getPod loads will try to load the pod from informer store and if it is not available, load it from k8s. func (pc *PodCache) getPod(name string, namespace string) *v1.Pod {+	key := kube.KeyFunc(name, namespace)+	// Try loading it from informer cache by key.

This also won't fix scale up quickly issues if we have a miss - basically once we hit the Pods() call more than a couple times it's all over, we will likely never recover if there is a large churn rate

ramaraochavali

comment created time in 2 days

push eventhowardjohn/pilot-load

John Howard

commit sha fbbfd6c2ae76d7162c7973a4ac14f78bc72cd5e6

Add injection

view details

push time in 2 days

push eventhowardjohn/pilot-load

John Howard

commit sha 30826c9f9ed3df318723ded04c3fb21b314b637c

Add gateway

view details

push time in 3 days

pull request commentistio/istio

update MutatingWebhook apiVersion to v1 and other minor fixes

We currently just list the current version support ,not past/future. https://github.com/istio/istio.io/pull/7154 adds it but its been in limbo (mostly my fault)

On Tue, Jun 30, 2020 at 8:52 PM Tariq Ibrahim notifications@github.com wrote:

@howardjohn https://github.com/howardjohn Side note, can we have this doc https://docs.google.com/document/d/14Xfe9V0vv8d7W75c21d-cY2XyK6xvRH0_Kg8j_zfh4Y/edit#bookmark=id.3gi4mzlz3xnp in istio.io docs? This would be a useful reference for Istio users.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/istio/istio/pull/24723#issuecomment-652173314, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEYGXOD525Z44C2B73MVDLRZKXJTANCNFSM4OABCTUQ .

tariq1890

comment created time in 3 days

push eventhowardjohn/pilot-load

John Howard

commit sha cefdca05887affb9310a87b1ad768bdd610d325b

Add other pod types

view details

push time in 3 days

pull request commentistio/istio

update MutatingWebhook apiVersion to v1 and other minor fixes

Istio 1.7 will support 1.16-1.18: https://docs.google.com/document/d/14Xfe9V0vv8d7W75c21d-cY2XyK6xvRH0_Kg8j_zfh4Y/edit#bookmark=id.3gi4mzlz3xnp

On Tue, Jun 30, 2020 at 8:33 PM Zhonghu Xu notifications@github.com wrote:

I would suggest postponing this pr unless we don't want to support k8s 1.15 anymore.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/istio/istio/pull/24723#issuecomment-652167803, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEYGXJXOPRWWUZYYHS4B4LRZKVBHANCNFSM4OABCTUQ .

tariq1890

comment created time in 3 days

issue commentkubernetes/kubernetes

StreamWatcher memory leak

We have proven this is an Istio issue, see https://github.com/istio/istio/issues/25112 for details. tl;dr don't block the handler for the informer

/close

vikramk7

comment created time in 3 days

issue commentistio/istio

Stale endpoints may cause istio-pilot to be stuck

@phsiao I reproduced in 1.6, I think your hypothesis is correct. great find!

phsiao

comment created time in 3 days

issue commentistio/istio

TestCEXLCompatibility test flake

https://prow.istio.io/view/gcs/istio-prow/pr-logs/pull/istio_istio/25042/unit-tests_istio/15701

howardjohn

comment created time in 3 days

pull request commentistio/istio

Move istio CRD reader to shared informers

/retest

howardjohn

comment created time in 3 days

pull request commentistio/istio

Stop writing root to istio-security configmap

something like istio-ca-root-cert or similar? You wrote the code

On Tue, Jun 30, 2020, 7:13 PM lei-tang notifications@github.com wrote:

Regarding to "Now we write to all namespaces in a different configmap", what is the name of the new configmap?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/istio/istio/pull/25131#issuecomment-652145261, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEYGXNK76PZSFFAL6AMKL3RZKLVHANCNFSM4OMWAMPQ .

howardjohn

comment created time in 3 days

Pull request review commentistio/istio

fix test issues that cause analyze to fail

 const ( // To use a particular kubeconfig (other than ~/.kube/config), set: // --istio.test.kube.config=<path> func TestIngressLoadBalancing(t *testing.T) {

@nmittler should we remove this test? its not actually run

stevenctl

comment created time in 3 days

pull request commentistio/istio

Extend security test for VMs

/test integ-security-k8s-tests_istio

LGTM

ZhengzheYang

comment created time in 3 days

pull request commentistio/istio

pilot: fix n^2 scaling behavior with gateways

/test all

howardjohn

comment created time in 3 days

pull request commentistio/istio

pilot: incremental EDS requests

/test all

howardjohn

comment created time in 3 days

PR opened istio/istio

pilot: incremental EDS requests

WIP

+17 -8

0 comment

2 changed files

pr created time in 3 days

create barnchhowardjohn/istio

branch : pilot/incremental-eds-requests

created branch time in 3 days

push eventhowardjohn/istio

John Howard

commit sha a1f6720fcd8da877278b66c0238a655b5e35e230

fixes and test

view details

push time in 3 days

issue commentistio/istio

Istio proxy fais with JWT token missing JWT token

Can you get the output of kubectl get pod -oyaml for the pod that is failing?

dawn-chu

comment created time in 3 days

Pull request review commentistio/istio

Move init connection to receive thread

 func isExpectedGRPCError(err error) bool { 	return false } -func receiveThread(con *Connection, reqChannel chan *discovery.DiscoveryRequest, errP *error) {+func (s *DiscoveryServer) receiveThread(con *Connection, reqChannel chan *discovery.DiscoveryRequest, errP *error) { 	defer close(reqChannel) // indicates close of the remote side.+	firstReq := true 	for { 		req, err := con.stream.Recv() 		con.mu.RLock() 		cid := con.ConID 		con.mu.RUnlock() 		if err != nil {+			cid := con.ConID 			if isExpectedGRPCError(err) {-				con.mu.RLock()

was this here for PeerAddr? is there still a race there? I guess probably no because line 176 doesn't have it

costinm

comment created time in 3 days

Pull request review commentistio/istio

Move init connection to receive thread

 func (s *DiscoveryServer) StreamAggregatedResources(stream discovery.AggregatedD 	// This also detects close. 	var receiveError error 	reqChannel := make(chan *discovery.DiscoveryRequest, 1)-	go receiveThread(con, reqChannel, &receiveError)+	go s.receiveThread(con, reqChannel, &receiveError)  	for { 		// Block until either a request is received or a push is triggered.+		// We need 2 go routines because 'read' blocks in Recv().+		//+		// To avoid 2 routines, we tried to have Recv() in StreamAggregateResource - and the push+		// on different short-lived go routines started when the push is happening. This would cut in 1/2+		// the number of long-running go routines, since push is throttled. The main problem is with+		// closing - the current gRPC library didn't allow closing the stream. 		select {-		case discReq, ok := <-reqChannel:+		case req, ok := <-reqChannel: 			if !ok {-				// Remote side closed connection.+				// Remote side closed connection or error processing the request. 				return receiveError 			}-			// This should be only set for the first request. The node id may not be set - for example malicious clients.-			if con.node == nil {-				if discReq.Node == nil {-					return errors.New("missing node ID")-				}-				// TODO: We should validate that the namespace in the cert matches the claimed namespace in metadata.-				if err := s.initConnection(discReq.Node, con); err != nil {-					return err-				}-				defer func() {-					s.removeCon(con.ConID)-					if s.InternalGen != nil {-						s.InternalGen.OnDisconnect(con)-					}-				}()-			}-			if s.StatusReporter != nil {-				s.StatusReporter.RegisterEvent(con.ConID, TypeURLToEventType(discReq.TypeUrl), discReq.ResponseNonce)-			}--			// Based on node metadata a different generator was selected, use it instead of the default-			// behavior.-			if con.node.XdsResourceGenerator != nil {-				// Endpoints are special - will use the optimized code path.-				err = s.handleCustomGenerator(con, discReq)-				if err != nil {-					return err-				}-				continue-			}--			switch discReq.TypeUrl {-			case v2.ClusterType, v3.ClusterType:-				if err := s.handleTypeURL(discReq.TypeUrl, &con.node.RequestedTypes.CDS); err != nil {-					return err-				}-				if err := s.handleCds(con, discReq); err != nil {-					return err-				}-			case v2.ListenerType, v3.ListenerType:-				if err := s.handleTypeURL(discReq.TypeUrl, &con.node.RequestedTypes.LDS); err != nil {-					return err-				}-				if err := s.handleLds(con, discReq); err != nil {-					return err-				}-			case v2.RouteType, v3.RouteType:-				if err := s.handleTypeURL(discReq.TypeUrl, &con.node.RequestedTypes.RDS); err != nil {-					return err-				}-				if err := s.handleRds(con, discReq); err != nil {-					return err-				}-			case v2.EndpointType, v3.EndpointType:-				if err := s.handleTypeURL(discReq.TypeUrl, &con.node.RequestedTypes.EDS); err != nil {-					return err-				}-				if err := s.handleEds(con, discReq); err != nil {-					return err-				}-			default:-				adsLog.Warnf("ADS: Unknown watched resources %s", discReq.String())+			// processRequest is calling pushXXX, accessing common structs with pushConnection.+			// Adding sync is the second issue to be resolved if we want to save 1/2 of the threads.git di

nit: random git comand is in the end of the comment

costinm

comment created time in 3 days

issue commentistio/istio

Stale endpoints may cause istio-pilot to be stuck

Based on your hypothesis, which seems very plausible, the issue is still present in 1.6 and just masked by the QPS. We can verify by dropping the QPS on 1.6 and reproducing I guess? As 1.4 is EOL we would not release a fix for this so ideally we shift focus to 1.6

phsiao

comment created time in 3 days

more