Repository navigation
Conversation
aryanmehrotra
left a comment
There was a problem hiding this comment.
Thanks @AbhiPra24 — I ran this against a real server rather than reading the diff, and both bugs are real and both of your fixes work. There is one regression that blocks it, and it is not visible from the diff or from the unit tests.
Both issues reproduce on development
#3853 POST /only-get -> 404, no Allow header (should be 405)
#3855 GET /static/evil.txt -> 200, body "TOPSECRET" (file outside the served root)
Your fixes do what they claim
| probe | development |
this PR |
|---|---|---|
POST /only-get |
404 | 405, Allow: GET |
DELETE /ping |
404 | 405, Allow: GET, POST |
GET /static/evil.txt |
200 TOPSECRET |
404 |
GET /static/evildir/passwords.txt |
200 | 404 |
dotdot-inside.txt (relative .. landing back inside) |
200 | 200 — correctly still served |
Same results under GOFR_ROUTER=trie, so the two matchers stay consistent. The Allow values are right, and the symlink containment does not over-reject.
Blocking: CORS preflight now returns 405
OPTIONS /only-get Origin: http://x.test Access-Control-Request-Method: GET
development: HTTP/1.1 200 OK
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Origin: *
this PR: HTTP/1.1 405 Method Not Allowed
Allow: GET
(no CORS headers at all)
Every cross-origin browser call to a GoFr app breaks — the preflight fails, so the browser never sends the real request.
Why, from the pinned gorilla/mux v1.8.1 (mux.go:138-168): the middleware chain is applied only in the matched-route branch, guarded by match.MatchErr == nil:
for _, route := range r.routes {
if route.Match(req, match) {
if match.MatchErr == nil { // <- middleware only here
for i := len(r.middlewares) - 1; i >= 0; i-- {
match.Handler = r.middlewares[i].Middleware(match.Handler)
}
}
return true
}
}
if match.MatchErr == ErrMethodMismatch {
if r.MethodNotAllowedHandler != nil {
match.Handler = r.MethodNotAllowedHandler // <- no middleware
return true
}
...
}
if r.NotFoundHandler != nil {
match.Handler = r.NotFoundHandler // <- no middleware
...
}middleware.CORS is registered through router.Use(...) (pkg/gofr/http_server.go:67-70, alongside Tracer, Logging and Metrics), so anything served by those two handler fields bypasses all four.
Blocking: 404 and 405 disappear from logs and metrics
Same root cause, and it also hits 404 — which previously went through PathPrefix("/"), a real route, and therefore had the full chain. Measured on the same app:
development |
this PR | |
|---|---|---|
app_http_response_count{path="/nope",status="404"} |
1 | absent |
app_http_response_count{path="/only-get"} |
1 | absent |
access-log lines for /nope |
3 | 0 |
access-log lines for /only-get |
2 | 0 |
Only the /ping 200 is still counted. A scanner walking unknown paths, or a client using the wrong verb, becomes completely invisible to logs, metrics and traces.
A shape that keeps the middleware
Rather than moving to mux's handler fields, keep the catch-all as a real route and decide inside it:
a.httpServer.router.PathPrefix("/").Handler(handler{
function: catchAllHandler, // 405 + Allow when AllowedMethods(r) is non-empty, else 404
container: a.container,
})AllowedMethods already skips routes with no methods, so the catch-all does not report itself. Because the catch-all matches, match.MatchErr stays nil, the whole chain runs, and CORS still intercepts OPTIONS before the handler is reached. Your AllowedMethods, ErrorMethodNotAllowed and the Allow formatting all carry over unchanged — this only changes where the decision is made.
If you go that way, worth re-checking whether the Path(...).Methods(...) reorder in Router.Add is still needed, since ErrMethodMismatch would no longer be what drives the 405.
Smaller things
An unrelated line was deleted. gofr.go drops:
if a.container.Logger != nil {
a.container.Logger.Infof("Registered HTTP server on port: %d", a.httpServer.port)
}Confirmed by running both: 1 such line on development, 0 here. It sits right where the catch-all was replaced, so it looks accidental.
Two new lint failures, both in the new test:
pkg/gofr/http/router_test.go:1007:21 G306: Expect WriteFile permissions to be 0600 or less (gosec)
pkg/gofr/http/router_test.go:1011:21 G306: (same)
Allow omits HEAD and OPTIONS. Allow: GET while OPTIONS /only-get is answered 405. RFC 9110 §10.2.1 has Allow list what the resource supports; fixing the CORS path will change this anyway, so it is worth settling both together.
On #3855, this departs from what the issue specified. The issue rules out this approach by name:
Adding
filepath.EvalSymlinksbefore the comparison would close the common case but reintroduces the TOCTOU windowos.Rootexists to eliminate — the link can be repointed between the resolve and the open.
Being fair to your version: the residual window needs write access inside the served tree, which is a far higher bar than the original bug, and EvalSymlinks avoids the breaking change os.Root brings (an absolute symlink pointing inside the root keeps working, where os.Root would reject it). That may well be the better trade — but it is a deliberate divergence from the design the issue landed on, and it should be argued in the PR description rather than left implicit.
Two unrelated issues in one PR. #3853 is self-contained and, once the middleware regression is fixed, ready. #3855 has an open design question. Splitting them would let the 405 work merge without waiting on that discussion.
What I ran
| check | result |
|---|---|
go build ./... |
clean |
go test ./pkg/gofr/http/... |
pass |
go test ./pkg/gofr |
fails identically on development (metrics-port FATAL) — not this PR |
gofmt -l |
2 dirty files, both pre-existing |
golangci-lint --new-from-rev |
2 new, listed above |
The unit tests all pass, which is worth saying plainly: nothing in them exercises CORS, logging or metrics on a 404/405 path, which is why an end-to-end run was the only thing that surfaced it. A test asserting that a 404 still carries Access-Control-Allow-Origin would have caught it and would be worth adding alongside the fix.
Good work on both root causes — the symlink reproduction in particular is a real vulnerability and the containment logic you wrote handles the ..-back-inside case correctly, which is the part that is easy to get wrong.
…inks - Route registered methods with Path before Methods to ensure Gorilla Mux correctly sets ErrMethodMismatch - Configure NotFoundHandler and MethodNotAllowedHandler on the HTTP router - Add AllowedMethods on Router to populate RFC 9110 Allow header on 405 Method Not Allowed responses - Add ErrorMethodNotAllowed error type and methodNotAllowedHandler - Use filepath.EvalSymlinks in AddStaticFiles and validateFile to ensure static file serving does not follow symlinks outside the root served directory - Add comprehensive tests for 405/404 routing and symlink containment Fixes gofr-dev#3853 Fixes gofr-dev#3855 Signed-off-by: Abhinav Prakash <abhinavprakash616@gmail.com>
…ic files with os.Root Addresses the changes requested on the previous head. 405 (gofr-dev#3853) - The catch-all stays a real PathPrefix("/") route and decides 404 vs 405 itself. Served from mux's NotFoundHandler/MethodNotAllowedHandler, both responses skipped Tracer, Logging, CORS and Metrics (gorilla/mux v1.8.1 applies the chain only when MatchErr == nil), so a CORS preflight to a GET-only path became a 405 with no CORS headers and every unknown path vanished from logs and metrics. - Allow lists OPTIONS, which the CORS middleware answers on every path. - Route registration order is back to Methods().Path(): with the catch-all deciding, nothing depends on ErrMethodMismatch, and the trie router's parity comments describe that order. - The "Registered HTTP server on port" log line is kept. - AllowedMethods matches each route once against the request as it came in; mux reports "every matcher but the method" as ErrMethodMismatch. One RouteMatch, reset per route, and GetMethods only for a route that matched: a 404 over 50 routes costs 42 allocs, against 39 before and 97 for a match-per-method walk. Static files (gofr-dev#3855) - Replace the string-prefix containment with os.Root, as the issue specifies. A path that leaves the served directory answers 404, indistinguishable from a missing file; os does not export the escape error, so it is captured from a root at registration. - The root is opened per request. A root held from registration pins the directory it was opened on, so a repointed release link (current -> releases/43) or a directory replaced in place would keep serving the old files until a restart. - Each file is opened once and checked from the open file's FileInfo: os.Root resolves one component at a time, so Stat-then-Open walked the path twice. O_NONBLOCK keeps a fifo from hanging the open. - The custom 404.html is read through the root too; on development a 404.html symlinked outside the folder is served. - Breaking: an absolute symlink inside the served folder is no longer followed; symlinks there must be relative. Documented in serving-static-files. Tests - App-level 404/405 test asserts CORS, X-Correlation-ID and the access-log line on every row, including the preflight. - Symlink matrix from gofr-dev#3855, lexical escape below the router, custom 404 through the root, redeploy (repointed link, replaced directory, removed directory), AllowedMethods with regexp parameters and route order. - gosec G306 in the new tests fixed.
cb559cd to
6daf48a
Compare
Umang01-hash
left a comment
There was a problem hiding this comment.
Nice fixes — 405+Allow and the os.Root symlink containment both check out (verified live: escaping abs/rel symlinks 404, 405 flows through the metrics middleware). One thing to address before merge, plus a few small notes.
MEDIUM: catchAll runs AllowedMethods on every unmatched request, including pure 404s. mux already walked all routes to reach the catch-all, so this is a second full O(routes) walk — 404 cost ~doubles (measured ~32µs/req at 1000 routes, all attacker-reachable). Please collect the Allow list without a second walk, or skip AllowedMethods when it isn't a method mismatch, so a 404 flood doesn't pay twice.
| // route is matched once, against the request as it arrived: gorilla/mux reports a request that | ||
| // satisfies every matcher except the method as ErrMethodMismatch (route.go, after the matcher | ||
| // loop), which is exactly "this path is registered, for other methods". | ||
| func (rou *Router) AllowedMethods(r *http.Request) []string { |
There was a problem hiding this comment.
NIT: AllowedMethods is new permanent exported API on http.Router (forced by the cross-package call from catchAll). Additive/non-breaking, just flagging that it widens the public surface — fine if intended.
There was a problem hiding this comment.
Intended, and yes it is permanent surface. catchAll lives in pkg/gofr and the router in pkg/gofr/http, so it has to be exported to be called; the alternative was moving the 404/405 decision into the http package, which would have put handler construction (it needs the container) on the wrong side of that boundary. Leaving this thread open for you to close if you are fine with it.
# Conflicts: # pkg/gofr/http/router.go
…d scan The catch-all called AllowedMethods on every request no route matched, and AllowedMethods walked every route. mux had already walked them all to reach the catch-all, so an unknown path cost two full scans, and any client can request unknown paths at will. AllowedMethods now narrows the request path through the route index and matches only the routes registered under it. The index is the one the trie matcher already builds; with the mux matcher it is built on the first unmatched request and used for this lookup only. Reading the mismatch mux records during its own scan is not an option: mux clears ErrMethodMismatch as soon as any matcher of a later route succeeds (gorilla/mux v1.8.1 route.go), so it is gone before the catch-all runs. At 1000 routes, an unregistered path: before 16031 ns/op 48 B/op 1 allocs/op after 187 ns/op 48 B/op 1 allocs/op
… of directories A path registered for OPTIONS on the router already carries it in the list AllowedMethods returns, and the catch-all appended a second one: "Allow: GET, OPTIONS, OPTIONS". os.Root opens the served directory, and each directory on the way to a file, for reading, where os.Open on the joined path only had to search them. A directory the process can traverse but not list now answers 403 for the files beneath it, and a served directory in that state is not registered. That is the cost of containment and it was not written down; it is now, in the docs and next to the code, with a test pinning it. Also documents ErrorMethodNotAllowed and the 405 alongside the other predefined errors, and corrects the comment on the lexical-escape test, which credited the root with a refusal rootRelativeName makes first.
|
Pushed three commits on top of
|
| Request | development |
this PR |
|---|---|---|
file under a 0311 subdirectory |
200 | 403 |
served root made 0311 after registration |
200 | 403 |
served root 0311 at registration |
200 | endpoint not registered, error logged, 404 |
Measured on darwin; os/root_unix.go is shared with Linux but I have not run it there. Test_StaticFileServing_UnlistableDirectory pins the subdirectory row, and the static-files page now says read permission is required.
Known difference left in place: trie matcher, method-restricted subrouter with its own NotFoundHandler
sub := r.PathPrefix("/nf").Methods("GET").Subrouter()
sub.NotFoundHandler = custom
sub.Path("/x").Handler(h)POST /nf/unknown answers 404 from custom with the mux matcher, and 405 Allow: GET, OPTIONS with GOFR_ROUTER=trie. The two matchers already disagreed on this request before the PR (404 from custom against 404 route not registered); the PR turns a body difference into a status difference. It needs a route built on the embedded mux.Router rather than through App, and the opt-in matcher. Not fixed here.
Local test coverage
pkg/gofr/http/... passes with -race, and with GOFR_ROUTER=trie. In pkg/gofr, 337 top-level tests pass in each matcher; 14 were skipped on my machine because another process holds ports 2121 and 8000 (TestInitMetricsServer_DefaultPort and the TestGraphQL_* / TestApp_setupGraphQL* family). Those are covered by CI only.
Static file serving returns to what development does. This PR now carries only the 405 fix for gofr-dev#3853. The containment asked for in gofr-dev#3855 was never reported by anyone, and following a symlink out of the served directory is what http.FileServer, nginx and Apache do by default. Building it showed the cost: a link from the served directory to storage outside it stops being served, an absolute link is refused even when its target is inside, and every served directory has to be readable rather than only traversable. Those break deployments that work today and there was no way to opt out. gofr-dev#3855 is closed as not planned. router.go and router_test.go are development's files with the 405 changes applied, not a hand-edit of the previous commit, so nothing of the removed half is left behind.
|
Scope change in That supersedes the What is left is #3853 only: the 405 with |
214435c to
c4182ae
Compare
|
Force-pushed to correct the author on my four commits (a stray local git identity). No content change: the tree at the new head SHAs quoted in the review threads map as follows:
|
Umang01-hash
left a comment
There was a problem hiding this comment.
Verified locally at head (622e2a6): wrong method → 405 with a correct Allow, unknown path → 404, bare and preflight OPTIONS still answered by CORS, and the full middleware chain (CORS/logging/tracer/metrics) runs for the catch-all. AllowedMethods is race-clean under 200 concurrent calls in both matcher modes, and the encoded-path union doesn't over-produce false 405s. Nicely done, and the tests are excellent.
One question, not blocking: HEAD on a GET-only route now returns 405 (Allow: GET, OPTIONS) where it was 404 before, and Allow omits HEAD since GoFr doesn't auto-register it for GET. Is 405-without-HEAD the intended answer, or should HEAD ride along with GET? Fine either way — just flagging the behavior change.
Umang01-hash
left a comment
There was a problem hiding this comment.
LGTM, nicely done. I verified the load-bearing gorilla/mux claims against the v1.8.1 source myself: middleware builds only on a matched route (mux.go:141), and a later route clears ErrMethodMismatch (route.go:76/87) — so using a real catch-all route (not mux's handlers) and re-scanning via AllowedMethods is the right call. Real-server E2E: POST /thing -> 405 Allow: GET, OPTIONS with the envelope, GET /nope -> 404, and both show up in the structured logs with a trace_id (the whole point — 404/405 now run the full chain). Revert-to-red confirms the 405 tests fail without the fix; -race clean in both router modes; AllowedMethods returns the exact method set across params/regex/host/header/subrouter/multi-method.
One thing to make sure reaches the release notes: 404 -> 405 for wrong-method-on-registered-path is a client-visible status change (RFC-correct, and #3853 asks for a minor release). One tiny inline nit.
| methods = append(methods, http.MethodOptions) | ||
| } | ||
|
|
||
| w.Header().Set("Allow", strings.Join(methods, ", ")) |
There was a problem hiding this comment.
Nit: AllowedMethods sorts internally, but appending OPTIONS after the sort leaves the header unsorted — a PATCH-only path returns Allow: "PATCH, OPTIONS" (reproduced on a real server). RFC 9110 treats Allow as an unordered set so it's cosmetic, but it undoes the sort.Strings intent. Easy fix: append OPTIONS before sorting, or sort after.
Fixes #3853.
A request for a path that is registered, but not for the method used, now answers 405 with an
Allowheader instead of 404.How
PathPrefix("/")catch-all stays a real route and decides 404 vs 405 itself, using the newRouter.AllowedMethods.NotFoundHandler/MethodNotAllowedHandler: gorilla/mux v1.8.1 applies the middleware chain only to a matched route (MatchErr == nil), so responses served from those fields skip Tracer, Logging, CORS and Metrics. The CORS preflight to a GET-only path came back 405 with no CORS headers, and every 404/405 vanished from logs, metrics and traces.AllowlistsOPTIONSbecause the CORS middleware answers it on every path. It is listed once even when a route registers it.AllowedMethodsdoes not walk the routes. mux has already scanned all of them to reach the catch-all, so it narrows the request path through the route index and matches only the routes registered under it. At 1000 routes an unregistered path costs 187 ns in that lookup, against 16031 ns for a second scan.Methods().Path()).Behaviour to know about
ErrorMethodNotAllowedis a new predefined error, documented with the others.GOFR_ROUTER=trie, a method-restricted subrouter that carries its ownNotFoundHandleranswers 405 where the mux matcher answers 404 from that handler. The two matchers already differed on that request; it needs a route built on the embeddedmux.Router.Not in this PR
The
os.Rootstatic-file containment that earlier revisions carried for #3855 has been taken out, and #3855 is closed as not planned. Static file serving is unchanged fromdevelopment.@AbhiPra24's original commit is kept as the first commit.