Skip to content

Commit 649bda2

Browse files
danwinshipdanwinship
authored
Update docs for deprecation of Endpoints API (#49831)
* KEP-4974: Update documentation for Endpoints deprecation * Bring EndpointSlice docs up to date - FQDN is not a supported AddressType - Update documentation around `serving` vs `ready` to just explain how it works now, without the historical context. (In particular, describe `serving` first, and then explain `ready` as being almost the same thing as `serving` rather than vice versa). - Fix feature-state version macro for `terminating` - Remove a note about some deprecated v1beta1 EndpointSlice fields that is just confusing if you didn't know that v1beta1 had a different topology API. - Remove the "Comparison with Endpoints"/"Motivation" section. * Fix up examples that still used `kubectl get endpoints` * Replace various passing references to Endpoints with EndpointSlices * Update main headings for Endpoints and Endpoint Slice Mirroring * Update glossary for Endpoints deprecation
1 parent 9e160e8 commit 649bda2

File tree

13 files changed

+117
-147
lines changed

13 files changed

+117
-147
lines changed

‎content/en/docs/concepts/scheduling-eviction/api-eviction.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -92,8 +92,7 @@ If the API server allows the eviction, the Pod is deleted as follows:
9292
resource is marked for termination and starts to gracefully shut down the
9393
local Pod.
9494
1. While the kubelet is shutting the Pod down, the control plane removes the Pod
95-
from {{<glossary_tooltip term_id="endpoint" text="Endpoint">}} and
96-
{{<glossary_tooltip term_id="endpoint-slice" text="EndpointSlice">}}
95+
from {{<glossary_tooltip term_id="endpoint-slice" text="EndpointSlice">}}
9796
objects. As a result, controllers no longer consider the Pod as a valid object.
9897
1. After the grace period for the Pod expires, the kubelet forcefully terminates
9998
the local Pod.

‎content/en/docs/concepts/services-networking/endpoint-slices.md

Lines changed: 56 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,7 @@ description: >-
1919
{{< feature-state for_k8s_version="v1.21" state="stable" >}}
2020

2121
Kubernetes' _EndpointSlice_ API provides a way to track network endpoints
22-
within a Kubernetes cluster. EndpointSlices offer a more scalable and extensible
23-
alternative to [Endpoints](/docs/concepts/services-networking/service/#endpoints).
22+
within a Kubernetes cluster.
2423

2524
<!-- body -->
2625

@@ -31,8 +30,8 @@ endpoints. The control plane automatically creates EndpointSlices
3130
for any Kubernetes Service that has a {{< glossary_tooltip text="selector"
3231
term_id="selector" >}} specified. These EndpointSlices include
3332
references to all the Pods that match the Service selector. EndpointSlices group
34-
network endpoints together by unique combinations of protocol, port number, and
35-
Service name.
33+
network endpoints together by unique combinations of IP family, protocol,
34+
port number, and Service name.
3635
The name of a EndpointSlice object must be a valid
3736
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
3837

@@ -67,17 +66,16 @@ more than 100 endpoints each. You can configure this with the
6766
{{< glossary_tooltip text="kube-controller-manager" term_id="kube-controller-manager" >}}
6867
flag, up to a maximum of 1000.
6968

70-
EndpointSlices can act as the source of truth for
69+
EndpointSlices act as the source of truth for
7170
{{< glossary_tooltip term_id="kube-proxy" text="kube-proxy" >}} when it comes to
7271
how to route internal traffic.
7372

7473
### Address types
7574

76-
EndpointSlices support three address types:
75+
EndpointSlices support two address types:
7776

7877
* IPv4
7978
* IPv6
80-
* FQDN (Fully Qualified Domain Name)
8179

8280
Each `EndpointSlice` object represents a specific IP address type. If you have
8381
a Service that is available via IPv4 and IPv6, there will be at least two
@@ -86,42 +84,37 @@ a Service that is available via IPv4 and IPv6, there will be at least two
8684
### Conditions
8785

8886
The EndpointSlice API stores conditions about endpoints that may be useful for consumers.
89-
The three conditions are `ready`, `serving`, and `terminating`.
90-
91-
#### Ready
92-
93-
`ready` is a condition that maps to a Pod's `Ready` condition. A running Pod with the `Ready`
94-
condition set to `True` should have this EndpointSlice condition also set to `true`. For
95-
compatibility reasons, `ready` is NEVER `true` when a Pod is terminating. Consumers should refer
96-
to the `serving` condition to inspect the readiness of terminating Pods. The only exception to
97-
this rule is for Services with `spec.publishNotReadyAddresses` set to `true`. Endpoints for these
98-
Services will always have the `ready` condition set to `true`.
87+
The three conditions are `serving`, `terminating`, and `ready`.
9988

10089
#### Serving
10190

10291
{{< feature-state for_k8s_version="v1.26" state="stable" >}}
10392

104-
The `serving` condition is almost identical to the `ready` condition. The difference is that
105-
consumers of the EndpointSlice API should check the `serving` condition if they care about pod readiness while
106-
the pod is also terminating.
93+
The `serving` condition indicates that the endpoint is currently serving responses, and
94+
so it should be used as a target for Service traffic. For endpoints backed by a Pod, this
95+
maps to the Pod's `Ready` condition.
10796

108-
{{< note >}}
97+
#### Terminating
10998

110-
Although `serving` is almost identical to `ready`, it was added to prevent breaking the existing meaning
111-
of `ready`. It may be unexpected for existing clients if `ready` could be `true` for terminating
112-
endpoints, since historically terminating endpoints were never included in the Endpoints or
113-
EndpointSlice API to begin with. For this reason, `ready` is _always_ `false` for terminating
114-
endpoints, and a new condition `serving` was added in v1.20 so that clients can track readiness
115-
for terminating pods independent of the existing semantics for `ready`.
99+
{{< feature-state for_k8s_version="v1.26" state="stable" >}}
116100

117-
{{< /note >}}
101+
The `terminating` condition indicates that the endpoint is
102+
terminating. For endpoints backed by a Pod, this condition is set when
103+
the Pod is first deleted (that is, when it receives a deletion
104+
timestamp, but most likely before the Pod's containers exit).
118105

119-
#### Terminating
106+
Service proxies will normally ignore endpoints that are `terminating`,
107+
but they may route traffic to endpoints that are both `serving` and
108+
`terminating` if all available endpoints are `terminating`. (This
109+
helps to ensure that no Service traffic is lost during rolling updates
110+
of the underlying Pods.)
120111

121-
{{< feature-state for_k8s_version="v1.22" state="beta" >}}
112+
#### Ready
122113

123-
`Terminating` is a condition that indicates whether an endpoint is terminating.
124-
For pods, this is any pod that has a deletion timestamp set.
114+
The `ready` condition is essentially a shortcut for checking
115+
"`serving` and not `terminating`" (though it will also always be
116+
`true` for Services with `spec.publishNotReadyAddresses` set to
117+
`true`).
125118

126119
### Topology information {#topology}
127120

@@ -133,18 +126,6 @@ per endpoint fields on EndpointSlices:
133126
* `nodeName` - The name of the Node this endpoint is on.
134127
* `zone` - The zone this endpoint is in.
135128

136-
{{< note >}}
137-
In the v1 API, the per endpoint `topology` was effectively removed in favor of
138-
the dedicated fields `nodeName` and `zone`.
139-
140-
Setting arbitrary topology fields on the `endpoint` field of an `EndpointSlice`
141-
resource has been deprecated and is not supported in the v1 API.
142-
Instead, the v1 API supports setting individual `nodeName` and `zone` fields.
143-
These fields are automatically translated between API versions. For example, the
144-
value of the `"topology.kubernetes.io/zone"` key in the `topology` field in
145-
the v1beta1 API is accessible as the `zone` field in the v1 API.
146-
{{< /note >}}
147-
148129
### Management
149130

150131
Most often, the control plane (specifically, the endpoint slice
@@ -169,34 +150,12 @@ slice object tracks endpoints for. This ownership is indicated by an owner
169150
reference on each EndpointSlice as well as a `kubernetes.io/service-name`
170151
label that enables simple lookups of all EndpointSlices belonging to a Service.
171152

172-
### EndpointSlice mirroring
173-
174-
In some cases, applications create custom Endpoints resources. To ensure that
175-
these applications do not need to concurrently write to both Endpoints and
176-
EndpointSlice resources, the cluster's control plane mirrors most Endpoints
177-
resources to corresponding EndpointSlices.
178-
179-
The control plane mirrors Endpoints resources unless:
180-
181-
* the Endpoints resource has a `endpointslice.kubernetes.io/skip-mirror` label
182-
set to `true`.
183-
* the Endpoints resource has a `control-plane.alpha.kubernetes.io/leader`
184-
annotation.
185-
* the corresponding Service resource does not exist.
186-
* the corresponding Service resource has a non-nil selector.
187-
188-
Individual Endpoints resources may translate into multiple EndpointSlices. This
189-
will occur if an Endpoints resource has multiple subsets or includes endpoints
190-
with multiple IP families (IPv4 and IPv6). A maximum of 1000 addresses per
191-
subset will be mirrored to EndpointSlices.
192-
193153
### Distribution of EndpointSlices
194154

195155
Each EndpointSlice has a set of ports that applies to all endpoints within the
196156
resource. When named ports are used for a Service, Pods may end up with
197157
different target port numbers for the same named port, requiring different
198-
EndpointSlices. This is similar to the logic behind how subsets are grouped
199-
with Endpoints.
158+
EndpointSlices.
200159

201160
The control plane tries to fill EndpointSlices as full as possible, but does not
202161
actively rebalance them. The logic is fairly straightforward:
@@ -244,34 +203,36 @@ You can find a reference implementation for how to perform this endpoint aggrega
244203
and deduplication as part of the `EndpointSliceCache` code within `kube-proxy`.
245204
{{< /note >}}
246205

247-
## Comparison with Endpoints {#motivation}
248-
249-
The original Endpoints API provided a simple and straightforward way of
250-
tracking network endpoints in Kubernetes. As Kubernetes clusters
251-
and {{< glossary_tooltip text="Services" term_id="service" >}} grew to handle
252-
more traffic and to send more traffic to more backend Pods, the
253-
limitations of that original API became more visible.
254-
Most notably, those included challenges with scaling to larger numbers of
255-
network endpoints.
256-
257-
Since all network endpoints for a Service were stored in a single Endpoints
258-
object, those Endpoints objects could get quite large. For Services that stayed
259-
stable (the same set of endpoints over a long period of time) the impact was
260-
less noticeable; even then, some use cases of Kubernetes weren't well served.
261-
262-
When a Service had a lot of backend endpoints and the workload was either
263-
scaling frequently, or rolling out new changes frequently, each update to
264-
the single Endpoints object for that Service meant a lot of traffic between
265-
Kubernetes cluster components (within the control plane, and also between
266-
nodes and the API server). This extra traffic also had a cost in terms of
267-
CPU use.
268-
269-
With EndpointSlices, adding or removing a single Pod triggers the same _number_
270-
of updates to clients that are watching for changes, but the size of those
271-
update message is much smaller at large scale.
272-
273-
EndpointSlices also enabled innovation around new features such dual-stack
274-
networking and topology-aware routing.
206+
### EndpointSlice mirroring
207+
208+
{{< feature-state for_k8s_version="v1.33" state="deprecated" >}}
209+
210+
The EndpointSlice API is a replacement for the older Endpoints API. To
211+
preserve compatibility with older controllers and user workloads that
212+
expect {{<glossary_tooltip term_id="kube-proxy" text="kube-proxy">}}
213+
to route traffic based on Endpoints resources, the cluster's control
214+
plane mirrors most user-created Endpoints resources to corresponding
215+
EndpointSlices.
216+
217+
(However, this feature, like the rest of the Endpoints API, is
218+
deprecated. Users who manually specify endpoints for selectorless
219+
Services should do so by creating EndpointSlice resources directly,
220+
rather than by creating Endpoints resources and allowing them to be
221+
mirrored.)
222+
223+
The control plane mirrors Endpoints resources unless:
224+
225+
* the Endpoints resource has a `endpointslice.kubernetes.io/skip-mirror` label
226+
set to `true`.
227+
* the Endpoints resource has a `control-plane.alpha.kubernetes.io/leader`
228+
annotation.
229+
* the corresponding Service resource does not exist.
230+
* the corresponding Service resource has a non-nil selector.
231+
232+
Individual Endpoints resources may translate into multiple EndpointSlices. This
233+
will occur if an Endpoints resource has multiple subsets or includes endpoints
234+
with multiple IP families (IPv4 and IPv6). A maximum of 1000 addresses per
235+
subset will be mirrored to EndpointSlices.
275236

276237
## {{% heading "whatsnext" %}}
277238

‎content/en/docs/concepts/services-networking/gateway.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ reference for a full definition of this API kind.
118118

119119
The HTTPRoute kind specifies routing behavior of HTTP requests from a Gateway listener to backend network
120120
endpoints. For a Service backend, an implementation may represent the backend network endpoint as a Service
121-
IP or the backing Endpoints of the Service. An HTTPRoute represents configuration that is applied to the
121+
IP or the backing EndpointSlices of the Service. An HTTPRoute represents configuration that is applied to the
122122
underlying Gateway implementation. For example, defining a new HTTPRoute may result in configuring additional
123123
traffic routes in a cloud load balancer or in-cluster proxy server.
124124

‎content/en/docs/concepts/services-networking/service.md

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -213,8 +213,8 @@ spec:
213213
targetPort: 9376
214214
```
215215

216-
Because this Service has no selector, the corresponding EndpointSlice (and
217-
legacy Endpoints) objects are not created automatically. You can map the Service
216+
Because this Service has no selector, the corresponding EndpointSlice
217+
objects are not created automatically. You can map the Service
218218
to the network address and port where it's running, by adding an EndpointSlice
219219
object manually. For example:
220220

@@ -307,14 +307,22 @@ until an extra endpoint needs to be added.
307307
See [EndpointSlices](/docs/concepts/services-networking/endpoint-slices/) for more
308308
information about this API.
309309

310-
### Endpoints
310+
### Endpoints (deprecated) {#endpoints}
311311

312-
In the Kubernetes API, an
312+
{{< feature-state for_k8s_version="v1.33" state="deprecated" >}}
313+
314+
The EndpointSlice API is the evolution of the older
313315
[Endpoints](/docs/reference/kubernetes-api/service-resources/endpoints-v1/)
314-
(the resource kind is plural) defines a list of network endpoints, typically
315-
referenced by a Service to define which Pods the traffic can be sent to.
316+
API. The deprecated Endpoints API has several problems relative to
317+
EndpointSlice:
318+
319+
- It does not support dual-stack clusters.
320+
- It does not contain information needed to support newer features, such as
321+
[trafficDistribution](/docs/concepts/services-networking/service/#traffic-distribution).
322+
- It will truncate the list of endpoints if it is too long to fit in a single object.
316323

317-
The EndpointSlice API is the recommended replacement for Endpoints.
324+
Because of this, it is recommended that all clients use the
325+
EndpointSlice API rather than Endpoints.
318326

319327
#### Over-capacity endpoints
320328

‎content/en/docs/concepts/workloads/pods/pod-lifecycle.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -511,8 +511,8 @@ containers:
511511

512512
`readinessProbe`
513513
: Indicates whether the container is ready to respond to requests.
514-
If the readiness probe fails, the endpoints controller removes the Pod's IP
515-
address from the endpoints of all Services that match the Pod. The default
514+
If the readiness probe fails, the EndpointSlice controller removes the Pod's IP
515+
address from the EndpointSlices of all Services that match the Pod. The default
516516
state of readiness before the initial delay is `Failure`. If a container does
517517
not provide a readiness probe, the default state is `Success`.
518518

@@ -677,7 +677,7 @@ Pod termination flow, illustrated with an example:
677677
to synchronize (or switch to using sidecar containers).
678678

679679
1. At the same time as the kubelet is starting graceful shutdown of the Pod, the control plane
680-
evaluates whether to remove that shutting-down Pod from EndpointSlice (and Endpoints) objects,
680+
evaluates whether to remove that shutting-down Pod from EndpointSlice objects,
681681
where those objects represent a {{< glossary_tooltip term_id="service" text="Service" >}}
682682
with a configured {{< glossary_tooltip text="selector" term_id="selector" >}}.
683683
{{< glossary_tooltip text="ReplicaSets" term_id="replica-set" >}} and other workload resources
@@ -690,7 +690,7 @@ Pod termination flow, illustrated with an example:
690690

691691
Any endpoints that represent the terminating Pods are not immediately removed from
692692
EndpointSlices, and a status indicating [terminating state](/docs/concepts/services-networking/endpoint-slices/#conditions)
693-
is exposed from the EndpointSlice API (and the legacy Endpoints API).
693+
is exposed from the EndpointSlice API.
694694
Terminating endpoints always have their `ready` status as `false` (for backward compatibility
695695
with versions before 1.26), so load balancers will not use it for regular traffic.
696696

‎content/en/docs/reference/access-authn-authz/rbac.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -655,9 +655,9 @@ When used in a <b>RoleBinding</b>, it gives full control over every resource in
655655
If used in a <b>RoleBinding</b>, allows read/write access to most resources in a namespace,
656656
including the ability to create roles and role bindings within the namespace.
657657
This role does not allow write access to resource quota or to the namespace itself.
658-
This role also does not allow write access to EndpointSlices (or Endpoints) in clusters created
658+
This role also does not allow write access to EndpointSlices in clusters created
659659
using Kubernetes v1.22+. More information is available in the
660-
["Write Access for EndpointSlices and Endpoints" section](#write-access-for-endpoints).</td>
660+
["Write Access for EndpointSlices" section](#write-access-for-endpoints).</td>
661661
</tr>
662662
<tr>
663663
<td><b>edit</b></td>
@@ -667,9 +667,9 @@ using Kubernetes v1.22+. More information is available in the
667667
This role does not allow viewing or modifying roles or role bindings.
668668
However, this role allows accessing Secrets and running Pods as any ServiceAccount in
669669
the namespace, so it can be used to gain the API access levels of any ServiceAccount in
670-
the namespace. This role also does not allow write access to EndpointSlices (or Endpoints) in
670+
the namespace. This role also does not allow write access to EndpointSlices in
671671
clusters created using Kubernetes v1.22+. More information is available in the
672-
["Write Access for EndpointSlices and Endpoints" section](#write-access-for-endpoints).</td>
672+
["Write Access for EndpointSlices" section](#write-access-for-endpoints).</td>
673673
</tr>
674674
<tr>
675675
<td><b>view</b></td>
@@ -1162,10 +1162,10 @@ In order from most secure to least secure, the approaches are:
11621162
--group=system:serviceaccounts
11631163
```
11641164

1165-
## Write access for EndpointSlices and Endpoints {#write-access-for-endpoints}
1165+
## Write access for EndpointSlices {#write-access-for-endpoints}
11661166

11671167
Kubernetes clusters created before Kubernetes v1.22 include write access to
1168-
EndpointSlices (and Endpoints) in the aggregated "edit" and "admin" roles.
1168+
EndpointSlices (and the now-deprecated Endpoints API) in the aggregated "edit" and "admin" roles.
11691169
As a mitigation for [CVE-2021-25740](https://github.com/kubernetes/kubernetes/issues/103675),
11701170
this access is not part of the aggregated roles in clusters that you create using
11711171
Kubernetes v1.22 or later.

‎content/en/docs/reference/glossary/endpoint-slice.md

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,16 +4,13 @@ id: endpoint-slice
44
date: 2018-04-12
55
full_link: /docs/concepts/services-networking/endpoint-slices/
66
short_description: >
7-
A way to group network endpoints together with Kubernetes resources.
7+
EndpointSlices track the IP addresses of Pods with matching Service selectors.
88
99
aka:
1010
tags:
1111
- networking
1212
---
13-
A way to group network endpoints together with Kubernetes resources.
13+
EndpointSlices track the IP addresses of Pods with matching {{< glossary_tooltip text="selectors" term_id="selector" >}}.
1414

1515
<!--more-->
16-
17-
A scalable and extensible way to group network endpoints together. These can be
18-
used by {{< glossary_tooltip text="kube-proxy" term_id="kube-proxy" >}} to
19-
establish network routes on each {{< glossary_tooltip text="node" term_id="node" >}}.
16+
EndpointSlices can be configured manually for {{< glossary_tooltip text="Services" term_id="service" >}} without selectors specified.

‎content/en/docs/reference/glossary/endpoint.md

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,19 @@ id: endpoints
44
date: 2020-04-23
55
full_link:
66
short_description: >
7-
Endpoints track the IP addresses of Pods with matching Service selectors.
7+
An endpoint of a Service is one of the Pods (or external servers) that implements the Service.
88
99
aka:
1010
tags:
1111
- networking
1212
---
13-
Endpoints track the IP addresses of Pods with matching {{< glossary_tooltip text="selectors" term_id="selector" >}}.
13+
An endpoint of a {{< glossary_tooltip text="Service" term_id="service" >}} is one of the {{< glossary_tooltip text="Pods" term_id="pod" >}} (or external servers) that implements the Service.
1414

1515
<!--more-->
16-
Endpoints can be configured manually for {{< glossary_tooltip text="Services" term_id="service" >}} without selectors specified.
17-
The {{< glossary_tooltip text="EndpointSlice" term_id="endpoint-slice" >}} resource provides a scalable and extensible alternative to Endpoints.
16+
For Services with {{< glossary_tooltip text="selectors" term_id="selector" >}},
17+
the EndpointSlice controller will automatically create one or more {{<
18+
glossary_tooltip text="EndpointSlices" term_id="endpoint-slice" >}} giving the
19+
IP addresses of the selected endpoint Pods.
20+
21+
EndpointSlices can also be created manually to indicate the endpoints of
22+
Services that have no selector specified.

0 commit comments

Comments
 (0)