Elasticsearch CA certificates are rejected by Python 3.13 #117769
Description
Elasticsearch Version
8.16.1
Installed Plugins
No response
Java Version
bundled
OS Version
Darwin arm64
Problem Description
Python 3.13 (released in October 2024) enabled by default the VERIFY_X509_STRICT flag for improved RFC 5280 compliance. This setting maps to the X509_V_FLAG_X509_STRICT OpenSSL flag documented as:
The
X509_V_FLAG_X509_STRICTflag disables workarounds for some broken certificates and makes the verification strictly applyX509rules.
The CA certificates generated by Elasticsearch (either by default on startup) or by elasticsearch-certutil are not compliant, at least because they're missing the key usage extension.
Steps to Reproduce
Run Elasticsearch:
$ docker run --name es01 -p 9200:9200 -it -m 1GB docker.elastic.co/elasticsearch/elasticsearch:8.16.1
... wait for startup
$ docker container cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt .
Try connecting to it using the Elasticsearch Python client:
from elasticsearch import Elasticsearch
client = Elasticsearch(
"https://localhost:9200",
ca_certs="http_ca.crt",
basic_auth=("elastic", "...")
)
print(client.info())This fails with:
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: CA cert does not include key usage extension (_ssl.c:1020)Alternatively, you can inspect the http_ca.crt file:
$ openssl x509 -in http_ca.crt -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
a4:50:12:ea:89:c9:78:fe:9e:9a:4b:7c:64:18:e0:13:04:d6:fb:58
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=Elasticsearch security auto-configuration HTTP CA
Validity
Not Before: Nov 29 14:07:47 2024 GMT
Not After : Nov 29 14:07:47 2027 GMT
Subject: CN=Elasticsearch security auto-configuration HTTP CA
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (4096 bit)
Modulus:
[...]]
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Key Identifier:
F4:BC:A2:F5:ED:8B:FD:93:F2:AE:76:82:A2:58:9E:EE:58:82:B9:BC
X509v3 Authority Key Identifier:
F4:BC:A2:F5:ED:8B:FD:93:F2:AE:76:82:A2:58:9E:EE:58:82:B9:BC
X509v3 Basic Constraints: critical
CA:TRUE
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
[...]In X509v3 extensions, you can see that the Key Usage extension is missing. If by contrast, I'm looking at the test CA certificate generated by trustme and used in the Python client:
$ openssl x509 -in .buildkite/certs/ca.crt -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
21:80:a5:61:65:e2:4e:c0:7c:68:ca:c4:10:ca:f3:76:b9:39:ac:eb
Signature Algorithm: ecdsa-with-SHA256
Issuer: O=trustme v1.2.0, OU=Testing CA #biw1Wc10lpqCFQL5
Validity
Not Before: Jan 1 00:00:00 2000 GMT
Not After : Jan 1 00:00:00 3000 GMT
Subject: O=trustme v1.2.0, OU=Testing CA #biw1Wc10lpqCFQL5
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
pub:
[...]
ASN1 OID: prime256v1
NIST CURVE: P-256
X509v3 extensions:
X509v3 Subject Key Identifier:
2A:C6:19:C3:BD:BF:45:00:59:2B:03:F7:73:FF:C7:63:13:36:22:5B
X509v3 Basic Constraints: critical
CA:TRUE, pathlen:9
X509v3 Key Usage: critical
Digital Signature, Certificate Sign, CRL Sign
Signature Algorithm: ecdsa-with-SHA256
Signature Value:
[...]
You can see that Key Usage is included, and indeed connections to Python 3.13 work.
Logs (if relevant)
No response