fix(websocket): verify the server certificate on wss dials (#1550)

[yashksaini-coder]

What was wrong?

Issue #1550

Fixes #1550

A wss dial with no explicit TLS client configuration built its context with
verify_mode = CERT_NONE and check_hostname = False, so
new_host(enable_websocket=True) accepted any certificate from any server.
go-libp2p uses a zero tls.Config and js-libp2p uses the platform TLS stack; both
verify against the system roots and check the hostname.

libp2p's own handshake runs inside the WebSocket, so this was never a stream
confidentiality or integrity hole. What an unverified outer TLS allowed was sitting
on the path unnoticed — and it is not what wss:// implies.

How was it fixed?

The dial-time fallback moved into _default_client_ssl_context() and returns an
unmodified ssl.create_default_context(). The old behaviour stays reachable
explicitly, with a warning logged:

config = WebsocketConfig(insecure_skip_verify=True)

The flag name matches the one WebSocketTLSConfig already uses. An explicit
tls_client_config still wins, and the proxy dial path gets the same context as the
direct one.

Stacked on #1549, included below as the first commit: while the transport
resolved a name to an IP before dialing, hostname verification could not have
succeeded against a certificate issued for that name — most likely why verification
was off. Please merge #1549 first; I will rebase to drop that commit, leaving a
single-commit diff.

@aojea — your issue and your ordering, so say the word if you would rather take it
and I will close this.

Tests

Three unit tests (default verifies, insecure_skip_verify opts out, explicit
tls_client_config returned untouched) and two real-TLS tests against a server
holding a fresh self-signed certificate: the default context refuses it with
ssl.SSLCertVerificationError, the opt-out completes the handshake.

The TLS tests use plain sockets in a thread deliberately — a trio SSLStream
server's do_handshake blocks rather than raising when the client refuses the
certificate, while stdlib reports the decision precisely on both sides.

Confirmed the verification tests fail against the old default and pass after
(2 failed, 2 passed → 4 passed). On this branch, rebased onto main @ d9dd9fb:

pytest tests/core/transport/websocket/   ->  125 passed, 1 skipped
make lint                                ->  12/12 hooks (mypy, pyrefly)
make build-docs                          ->  exit 0

make pr also shows 4 failures in tests/core/kad_dht/ and
tests/examples/test_dht_chat.py. They fail on a clean checkout too and pass when
rerun serially — a pre-existing race under -n auto, unrelated to this change.

Scope

/sni/<name> is still ignored on dial (#1551) and left for that issue. While
testing I filed #1554: a wss dial that fails verification reports a handshake
timeout instead of the certificate error and leaks the socket, because the
handshake runs in the Swarm's background nursery — separate fix, but it makes this
new behaviour harder to diagnose than it should be.

To-Do

• Clean up commit history
• Add or update documentation related to these changes — docs/examples.websocket.rst gained a "Dialing WSS" section
• Add entry to the release notes — filed as breaking, since this changes behaviour for anyone relying on self-signed wss endpoints

Cute Animal Picture

[yashksaini-coder]

@aojea — this is your #1550, and it is stacked on your #1549 because verification cannot go on before name-based dials do: while a /dns4 name is resolved to an IP first, hostname checking could never pass against a certificate issued for the name. #1549 needs to merge first; I will then rebase and this drops to a single commit. If you would rather carry #1550 yourself as you suggested in the issue, say so and I will close this — no hard feelings either way.

cc @acul71 for review (you have the most history in transport/websocket/ after me, and reviewed my last few).

@seetadev — flagging one maintainer decision: this changes a default, so it is filed as a breaking newsfragment. Anyone currently dialing a self-signed wss endpoint without configuring trust will start seeing a verification failure and will need WebsocketConfig(insecure_skip_verify=True) or their own tls_client_config. Worth a line in the next release notes given 0.8.0 just went out.

CI is green (38/38). Local: tests/core/transport/websocket/ 125 passed, make lint 12/12, make build-docs clean. The two verification tests were confirmed to fail against the old default and pass after.

[aojea]

awesome, thanks folks for iterating on this , looking forward for the next release

[yashksaini-coder]

Thanks @aojea 🙏

To be clear about the ordering for whoever picks this up: this one is blocked on #1549, not on review. #1549 is included here as its first commit because certificate verification cannot be turned on while names are still resolved to an IP before dialing. So the useful next step is merging #1549; I will then rebase and this collapses to a single commit.

@acul71 @sumanjeet0012 — could one of you take a look at #1549 first? Both are small. CI here is green (38/38) and the branch is current with main.

@seetadev one maintainer call on this one: it changes a default (an unverified wss dial starts failing), so it is filed as a breaking newsfragment and will want a visible line in the next release notes.