Use the chosen file info, not a zero size, to tell whether a candidate
was chosen, so smallest_size keeps an empty file and largest_size can
choose one.
When a handle_response status code did not evaluate to an integer, the
handler returned a 500 without closing the upstream response body,
leaking the connection.
* caddyauth: Support third-party http.authentication.hashes modules
basic_auth and hash-password only accepted the built-in bcrypt and
argon2id algorithms, even though the http.authentication.hashes
namespace is documented as extensible. Look up any other algorithm
name among the installed hash modules instead, and list the available
ones when the name is not recognized.
The built-in algorithms keep their own hash-password flags; other
modules are provisioned and must implement Hasher to generate hashes.
Closes#6670
* caddyauth: Base64-encode non-MCF hashes from hash-password modules
hash-password printed module hashes as raw bytes, but basic_auth
base64-decodes any password that isn't in Modular Crypt Format, so the
output couldn't be used as-is. Encode non-MCF hashes as base64 and add
a round-trip test that provisions HTTPBasicAuth with the command output.
* caddyauth: always base64-encode third-party hash output
* reverseproxy: Test that WebSockets + unix-sockets works
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Close temp file before reusing its path for the socket
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Strengthen unix socket WebSocket test, reuse tempSocketName helper
Check that upstream bytes sent with the 101 are delivered, echo a payload larger than any buffer, and verify the upstream connection closes when the client disconnects. Reuse the existing tempSocketName helper for all unix socket tests, which also closes the temp file descriptor the older tests leaked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* caddyhttp: let handler read deadlines cap the idle read timeout
IdleTimeoutReader sets the read deadline before every body read, so a
deadline set by a handler with http.ResponseController.SetReadDeadline()
was replaced before it could take effect. Since #7913, handlers can't
bound a stalled request body more tightly than read_body.
A handler's read deadline is now routed to the IdleTimeoutReader, which
applies it right away and keeps it as a ceiling for every idle reset and
for the deadline put back between HTTP/2 reads. A handler can make the
idle timeout stricter but can't loosen it, and a zero deadline removes
the ceiling, as #8145 does for write deadlines.
* caddyhttp: keep the hard read deadline and the drain deadline with handler deadlines
SetReadDeadline installed the handler's deadline as is, so a later or zero
deadline lifted read_timeout. It now installs the earlier of the two.
HandlerDone cleared a deadline set by the handler before arming the HTTP/1
drain deadline, leaving the post-handler drain unbounded.
* caddyhttp: keep the idle deadline of a read in flight on handler deadlines
SetReadDeadline replaced the idle deadline armed for a blocked read with the
handler's ceiling, so a zero or later deadline lifted the idle timeout for
that read. It now installs the earlier of the two until the read returns.
* caddyhttp: keep the expired drain deadline of a timed-out body on handler deadlines
After a body read timed out on HTTP/1, a zero or later deadline set by the
handler replaced the expired one kept to bound net/http's post-handler drain.
* perf: bypass header key normalization for canonical literals
http.Header's Get, Set, Add and Del run textproto.CanonicalMIMEHeaderKey
on every call. When the key is a canonical literal the result is the key
itself, so the scan is pure overhead. Access the map directly instead.
Only hot, allocation-free paths are converted, and only where the key is
a canonical literal. Call sites taking a key from config, a template or
the request itself still go through the normalizing methods, since they
need it for correctness.
allHeaderValues and lastHeaderValue dropped their http.CanonicalHeaderKey
call for the same reason: all callers pass canonical literals.
benchstat, Apple M5 Pro, go1.27.1:
PrepareRequest-15 659.4n ± 2% -> 457.7n ± 2% -30.58% (p=0.000 n=12)
AcceptedEncodings-15 330.8n ± 5% -> 309.4n ± 7% -6.44% (p=0.019 n=10)
ResponseWriterWriteHeader/event-stream-15 1007.5n ± 3% -> 732.5n ± 1% -27.30%
ResponseWriterWriteHeader/informational-15 60.0n ± 3% -> 48.0n ± 2% -20.01%
ResponseWriterWriteHeader/not-modified-15 214.3n ± 6% -> 175.3n ± 4% -18.20%
ResponseWriterWriteHeader/connect-15 60.2n ± 3% -> 51.9n ± 4% -13.85%
ResponseWriterWriteHeader/html-15 206.5n ± 4% -> 197.9n ± 2% -4.19%
geomean -14.38%
Allocations are unchanged.
* tests: verify http.Header map keys are canonical
Indexing http.Header directly skips the canonicalization Get/Set/Add/Del
perform, which also puts the key outside what CI checks. canonicalheader
only inspects method calls. staticcheck's SA1008 covers reads, but stops
descending as soon as an assignment writes to a header, so it misses the
write, the delete builtin and the read-modify-write append. A key that is
not canonical in those positions is a silent lookup miss, not a compile
error.
Walk the loaded packages and verify every string constant used as a key in
an http.Header map operation matches http.CanonicalHeaderKey. A helper that
takes the key from its caller would hide the literal again, so a parameter
named with a canonical prefix opts its arguments into the same check.
The WebSocket headers Caddy deliberately writes with RFC 6455 casing are
allowed explicitly; they are only ever written out, never looked up against
a parsed header map.
Covers 64 operations, runs in under a second, and promotes golang.org/x/tools
from an indirect to a direct dependency at the version already in the module
graph.
* headercheck: resolve aliases of http.Header
* headercheck: check http.Header composite literal keys
* headercheck: note overlap with canonicalheader
* headercheck: drop parameter-name check
Helpers now take the indexed values, so every key sits in an index expression the analyzer and the linter already check.
* headercheck: scope RFC 6455 exceptions to their call sites
A key-based allow-list also accepted reads such as h["Sec-WebSocket-Key"], which miss because parsed headers are canonical. Honor //nolint:canonicalheader on the line instead, which the linter will use too.
* encode: read every Accept-Encoding field line
AcceptedEncodings read only the first field line, so a client that split
the list across several lines had the rest ignored. RFC 9110 section 5.3
treats repeated list-field lines as one comma-joined list. Iterate over
the items of every line instead of joining them, and skip empty items,
which used to yield an empty encoding. file_server's precompressed
selection uses the same function and is fixed with it.
Fixes https://github.com/caddyserver/caddy/issues/8272
Co-authored-by: Kağan Küçük <kaankck@gmail.com>
* encode: read every Cache-Control and Content-Encoding field line
Only the first field line was read, but RFC 9110 section 5.3 makes
multiple lines of a list-based field equivalent to one comma-joined
value:
- Cache-Control: no-transform on a later line was ignored, so the
request or response was compressed anyway.
- Content-Encoding: an empty first line followed by a real coding let
the handler encode the body a second time.
* encode: parse Cache-Control no-transform per RFC 9111
The substring match missed No-Transform, though RFC 9111 section 5.2 compares directive names case-insensitively, and fired on text that is not the directive: longer tokens such as x-no-transform-ext, quoted arguments such as community="no-transform", and semicolon-separated values, which are not valid Cache-Control syntax. Split the field on commas outside quoted strings and match only a well-formed no-transform element.
* encode: keep accepted encoding preferences on the stack
AcceptedEncodings counted commas to size the preference slice, and the variable capacity put it on the heap. A constant capacity covers typical Accept-Encoding headers and lets the compiler keep the slice on the stack; append still moves it to the heap for longer ones. BenchmarkAcceptedEncodings: 151.8ns -> 120.5ns (-21%), 192 -> 64 B/op, 2 -> 1 allocs/op (n=8).
* encode: build the encoded ETag without fmt
fmt.Sprintf boxed both arguments and parsed the format on every encoded response with a strong ETag. Concatenation: 47ns -> 16.5ns, 56 -> 24 B, 3 -> 1 allocs.
* reverseproxy: access response Server and Via headers directly
finalizeResponse runs on every proxied response; skip the key normalization as prepareRequest already does on the request side.
* encode: access headers directly in Write, WriteHeader and Close
These run once per response; convert the remaining literal-key Get, Set and Add calls like the rest of the handler.
* encode: match Cache-Control no-transform with httpguts
httpguts.HeaderValuesContainsToken already compares list elements case-insensitively across field lines, so the hand-written list parser is unnecessary. A no-transform inside a comma-separated quoted argument now also disables encoding, which only errs toward not transforming.
* encode: inline the no-transform check
isEncodeAllowed only wrapped one httpguts call. Its unit test now drives the response directive through ServeHTTP instead.
---------
Co-authored-by: Kağan Küçük <kaankck@gmail.com>
* caddytls: keep ACME issuer network proxy across re-provisioning
LoadModule clears NetworkProxyRaw, so when the HTTP app provisions an
ACME issuer a second time (fillInACMEIssuer), the rebuilt template has
no HTTPProxy. Since #7510 that template is used via SetConfig, and ACME
requests bypass the configured proxy.
Keep the loaded proxy func on the issuer, like the DNS solver.
Fixes#8283
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 2366825f2380383dacd8adeb1508e34c3e25634d)
* caddytls: test that ACME requests use the proxy after RebuildCertMagic
Replace the provision-twice check with a regression for the path the
HTTP app takes: Provision, re-provision in fillInACMEIssuer, then
RebuildCertMagic. The test issues against a local proxy and asserts the
ACME request goes through it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: k4lleanka <257361691+k4lleanka@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* reverseproxy: strip hop-by-hop headers when copying response headers
copy_response_headers copied the upstream header map before
finalizeResponse could strip hop-by-hop fields. Clone and strip
first, so include and exclude cannot forward them, and leave the
upstream response unchanged for later handlers in the route.
Co-authored-by: Grok 4.7 <noreply@x.ai>
* replacer: cache file lookups and cap them
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* replacer: cache repeated file placeholder lookups
Memoize file placeholder results per replacement call, using an 8 MiB LRU for contents. Preserve empty and failed-read results without changing the output for unknown placeholders.
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
---------
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* reverseproxy: close backend body and hijacked socket on upgrade setup failures
When an upstream response indicates an HTTP upgrade (101 Switching Protocols),
handleUpgradeResponse takes ownership of res.Body and subsequently hijacks
the client connection.
However, if any setup check fails prior to starting handleUpgradeTunnel
(such as mismatched upgrade headers, non-writable body assertion failure,
hijack unsupported, or handshake buffer flush errors), the function returned
early without closing res.Body or the hijacked client socket.
Because calling Hijack() permanently disables net/http's automatic connection
cleanup, and raw TCP sockets are not closed by the garbage collector, these
early failure exits leaked file descriptors into CLOSE_WAIT/ESTABLISHED.
Fix this by:
1. Adding a deferred cleanup guard that closes res.Body and any hijacked
front connection unless the tunnel successfully takes ownership.
2. Wrapping connections with onceCloseConn to ensure idempotent closure
while preserving optional CloseWrite (TCP half-close) support.
3. Adding TestUpgradeResponseCleanupOnFailure to verify cleanup.
* reverseproxy: test post-hijack cleanup and remove once-close wrappers
Cover handshake flush and buffered backend-write failures after successful hijacking, checking that both connections close. Keep setup-failure cleanup while using the original connections directly.
Code and tests were AI-assisted using an OpenAI coding assistant in Pi.
* caddyhttp: add CEL expression support to the tls matcher
MatchTLS was the only core HTTP matcher without a CELLibrary() implementation,
so it could not be used inside an expression matcher. Every sibling matcher
(host, path, path_regexp, method, query, header, header_regexp, protocol,
remote_ip, client_ip, url_pattern, vars, vars_regexp, and the file-server's
file matcher) already has this.
Adds CELLibrary() to MatchTLS, exposing tls_handshake_complete('true') /
tls_handshake_complete('false') in CEL expressions. Mirrors MatchProtocol's
existing string-argument-macro pattern: CELMatcherImpl with cel.StringType,
parsing the argument with strconv.ParseBool and constructing a
MatchTLS{HandshakeComplete: &b} that reuses the matcher's own
MatchWithError. Adds the corresponding CELLibraryProducer interface guard.
Adds 6 cases to celmatcher_test.go's existing table-driven
TestMatchExpressionMatch/BenchmarkMatchExpressionMatch: match on
'true'/'false' against a complete/incomplete handshake, and three
invocation-error cases (no args, wrong arg type, non-boolean string).
Required adding a tlsHandshakeComplete *bool field to the shared test table
and wiring it into req.TLS.HandshakeComplete in both the test and benchmark
request-construction helpers.
Fixes#8087
* address review: use a native bool arg, rename to match the matcher's own name
francislavoie's review on #8088:
- "A boolean as a string argument is very strange." CELMatcherImpl only
supported string/list(string)/map argument types, so the original
implementation parsed a string literal with strconv.ParseBool to work
around that. Instead, extends CELMatcherImpl itself with a cel.BoolType
case (celMatcherBoolMacroExpander + isCELBoolExpr, mirroring the existing
string equivalents), so a matcher can accept a real CEL bool literal.
This is shared infrastructure, so any other matcher needing a bool
argument in the future gets it for free.
- "I'm not sure why tls_handshake_complete was chosen when the matcher is
named tls." Renamed the CEL function to tls(...), matching every other
matcher's naming convention (the function name mirrors the matcher's own
directive/module name: host(), path(), method(), protocol(), etc.).
Usage is now 'expression tls(true)' / 'expression tls(false)' instead of
'expression tls_handshake_complete(\'true\')'.
Updated the 6 existing test cases accordingly; still covers match/no-match
on both true/false and the two invocation-error cases (no args, wrong arg
type -- now a string instead of a bool, confirming the new type check).
* address review: cover tls() against a plain non-TLS request
steadytao asked for a test proving tls(false) doesn't treat the absence
of TLS as an incomplete handshake. MatchWithError already handles this
correctly (r.TLS == nil returns false regardless of HandshakeComplete),
this just adds the missing coverage for both tls(true) and tls(false)
against a plain http:// request.
Confirmed these tests catch a real regression: removing the r.TLS == nil
guard causes a nil pointer dereference panic, not just a wrong result.
* address review: redesign tls() CEL matcher to preserve all three Caddyfile states
The Caddyfile tls matcher has three states: bare `tls` (any TLS connection,
regardless of handshake completion), `tls early_data` (handshake not yet
complete), and the implicit "handshake complete" case. The previous
tls(true)/tls(false) design could only express the latter two, with no way
to express "any TLS, don't care about handshake state" from CEL.
Redesigns the macro to:
tls() -- any TLS connection (mirrors bare `tls`)
tls('complete') -- handshake complete
tls('early_data') -- handshake not yet complete (mirrors `tls early_data`)
tls() is implemented as sugar for tls('any'): a zero-arg macro that expands
to the same 2-arg call the one-arg string macro produces, just with the
literal injected by the macro itself. Both macros share one function
registration and one matcher factory, so there's a single source of truth
for the string-to-MatchTLS mapping.
This replaces the native-bool design from the prior commit, so also removes
the generic bool-argument support (celMatcherBoolMacroExpander,
isCELBoolExpr, and the CELMatcherImpl case for cel.BoolType) added for it,
since nothing else in the codebase used it and it would otherwise ship as
unreviewed, unused surface area.
Rewrites the existing tls(...) test cases for the new string-based calls,
and adds coverage for the new tls() form: matches regardless of handshake
state (both complete and incomplete), and still correctly fails to match a
plain non-TLS request. Also covers the new invocation-error cases: a
boolean argument is no longer valid, and an unrecognized string (anything
other than 'complete'/'early_data') is now a Provision-time error instead
of silently doing the wrong thing.
Verified: full caddyhttp test suite passes, and confirmed each new/changed
test fails when its corresponding implementation piece is reverted
(the zero-arg macro, and the complete/early_data boolean mapping).
* address review: document and test tls('any') as a supported literal form
steadytao pointed out that the zero-arg tls() macro expands internally to
the same tls_request_string(req, "any") call that the one-arg form uses,
which means tls('any') was already silently reachable as a literal string
even though it was never documented or tested as part of the public API.
Rather than special-casing the macro to bypass the shared string factory
(which would need a second function/overload just for the zero-arg case),
document and test tls('any') as a real, supported third spelling alongside
tls() -- they're already fully equivalent, so there's nothing to hide.
* reverseproxy: emit OpenTelemetry client spans for upstream requests
When the request carries a valid span context (set by the tracing
handler), each upstream round trip, including retries, is recorded as a
client span and that span's context is propagated to the upstream.
Requests that are not traced use the transport directly.
The transport is wrapped at the call site rather than replacing
h.Transport, so transport interface checks and health checks are
unaffected. The tracer provider comes from the span in the request
context, so no global provider is needed.
Fixes#7223
* reverseproxy: assert client span URL, upstream address and status
Use a provisioned HTTPTransport so the test fails if roundTrip stops
setting the scheme before the traced transport records url.full.
* reverseproxy: add license header to tracing test
When a forward proxy is configured (network_proxy, forward_proxy_url or
HTTP_PROXY), net/http wraps dial errors in a *net.OpError with Op
"proxyconnect". tryAgain used a type assertion to detect DialError, so it
missed the wrapped error and treated it as a failure after the request
was sent. Non-GET requests were then not retried, even though no
connection to the proxy was ever established.
Use errors.As so a DialError is recognized anywhere in the error chain.
Errors that do not contain a DialError are unaffected, so requests that
may have reached the proxy or upstream are still not retried by default.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
WithBindCapability assumed the kernel's privileged port range ends at
1024, but it is set by net.ipv4.ip_unprivileged_port_start, which can
be anywhere from 0 (Docker's default) to 65535. With a value above
1024, a non-root bind to e.g. :1500 failed even though the capability
was permitted.
Now listen is called normally first, and only if it fails with EACCES
on a TCP/UDP port, and CAP_NET_BIND_SERVICE is permitted but not
effective, the capability is raised and the bind retried. This also
skips raising it when the range starts lower than the port.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The formatter checked every token starting with "<<" against the heredoc
marker regexp. In a libFuzzer build each string comparison copies both
strings, and the regexp compares the remaining input on every step, so a
long marker-shaped token made the fuzz-format target time out.
A byte loop accepts the same markers in linear time. The lexer uses it
too, so there is one definition of a valid marker.
* headers: Check each ? default on its own and keep the response matcher
Several ? defaults in one header block shared one require, so a single
field already set on the response blocked all of them. A match in the
block was also dropped for the ? fields, so they were set on every
response.
* headers: Reject a response matcher on a field that also has a ? default
A ? default already requires its field to be absent, so a match on the
same field in that block could only be dropped or never be true. It is
now a Caddyfile error.
* headers: Test that the last ? default for a field wins
* headers: Allow an absence matcher on a field with a ? default
`match header !X-Test` asks for what `?X-Test` already requires, so it
loads again. Only a matcher that needs a value on that field is an error.
A password piped with a Windows line ending was hashed with the trailing
\r, so basic_auth rejected the real password, and input without a final
newline failed with EOF.
D'Ali Ben Macha
Nguyen Thanh Dat
Kévin Dunglas
Krish Singh Huraandsteadytao
EINIER FREYRE CORONA
a
Kartik K
hsdfat
Francis LavoieandClaude Opus 5.5
jasper
tannevaledandZen Dodd
bwisn
dependabot[bot]
sam-kota
Akinyemi Toluwalase
Sulaiman Dauda
tomholfordandGrok 4.7
Mohammed Al Sahaf
Rob Martin
Aradhya Jain
Adarsh KumarandClaude Opus 5.5
Mohamed A. Abdel Fattah
Jay Hemnani
awss