Understanding Go URL handling

Hi.

A couple of years ago I “took inspiration” for a HTTP reverse proxy in Go from Stack Overflow without putting too much thought into it, and this week it bit me back. A co-worker found out that it was normalising some URLs (/something//else will 301-redirect to /something/else) against their will. So I decided to take the opportunity and understand better how net/http handles URLs, and here are my findings.

Minimal Go HTTP Server

1
2
3
4
5
6
7
package main

import "net/http"

func main() {
	http.ListenAndServe(":8080", nil)
}

This simplest HTTP server you can write in Go. Given we’re providing the handler argument with nil, it won’t do much other than open the TCP port and return 404 for any URL you request. While that’s not particularly useful, is enough to make sure we’re able to compile, run and make requests.

However, it’s more likely that you’ll want the server to show some content, so for that you could write:

1
2
3
4
5
6
7
8
9
10
11
package main

import "net/http"

func main() {
	http.HandleFunc("/", func(writer http.ResponseWriter, request *http.Request) {
		writer.Write([]byte("Hello, world"))
	})

	http.ListenAndServe(":8080", nil)
}

And sure enough, it works! Requesting / returns Hello, world. But so does requesting /something. Hum. Why is that?

The first argument in HandleFunc is pattern, which will start matching at the beginning of the URL. So http.HandleFunc("/test/", ...) will return 200 for /test/ and /test/another/, but 404 for /another/test/. However, http.HandleFunc("/test", ...) (without the trailing slash) will only match /test, and will return 404 for both /test/ and /test/another. So ending the pattern with / has some special handling.

So how do I match / and only /? Let’s try with an empty string for http.HandleFunc:

1
2
3
4
5
6
7
8
9
10
11
12
13
$ go run main.go
panic: http: invalid pattern

goroutine 1 [running]:
net/http.(*ServeMux).Handle(0x14622e0, {0x0, 0x0}, {0x12e7c80?, 0x12a8380})
	/usr/local/Cellar/go/1.20.2/libexec/src/net/http/server.go:2510 +0x25f
net/http.(*ServeMux).HandleFunc(...)
	/usr/local/Cellar/go/1.20.2/libexec/src/net/http/server.go:2553
net/http.HandleFunc(...)
	/usr/local/Cellar/go/1.20.2/libexec/src/net/http/server.go:2565
main.main()
	/Users/kassner/workspace/test/main.go:6 +0x33
exit status 2

It doesn’t compile. Digging into it I found out that it’s actually not possible using the default behaviour. But as it was pointed out in the thread:

The ServeMux isn’t fundamental to the net/http package. If you don’t like its behavior, you can write your own mux.

Terminology is hard

So what is a ServeMux? Quoting from Go’s source:

ServeMux is an HTTP request multiplexer. It matches the URL of each incoming request against a list of registered patterns and calls the handler for the pattern that most closely matches the URL.

I have known that as a router from some Web Frameworks (Express, Spring, Symfony). All it does is to match a particular URL to an arbitrary piece of code that I declared. Sounds what I need, right?

Turns out the term ServeMux only confused me. What I needed is a Handler, and ServeMux is only one implementation of a handler that comes baked into Go. Of course, in some situations ServeMux might be enough, I just wish it was clearer to me it was a DefaultHandler.

It also didn’t help me that Go has both http.HandlerFunc and http.HandleFunc. The former is used to wrap a func into Handler, which is what we need to create our own Handler, and the latter does something similar, but registering the function with a pattern matching against the DefaultServeMux. A simple typo can cause you a good amount of head-scratching.

I have a preference for verbose and explicit intent, so I find this more comprehensive to read:

1
2
3
4
5
6
7
8
9
10
11
package main

import "net/http"

func main() {
	http.DefaultServeMux.HandleFunc("/", func(writer http.ResponseWriter, request *http.Request) {
		writer.Write([]byte("Hello, world"))
	})

	http.ListenAndServe(":8080", nil)
}

That way it’s explicit that I’ll be using the DefaultHandlerServeMux provided by Go, with all the behaviours that come out of the box.

Crafting a Handler

So, if a Handler is all we need, how do we rewrite our server using it?

1
2
3
4
5
6
7
8
9
10
11
package main

import "net/http"

func main() {
	handler := http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
		writer.Write([]byte("Hello, world"))
	})

	http.ListenAndServe(":8080", handler)
}

As we didn’t add any logic to it, any URL will return Hello, world. More importantly though, a request with double-slashes, which was my original problem, will not be normalized, which confirms we’re overriding the default behaviour.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ curl -v http://localhost:8080/something//else
*   Trying 127.0.0.1:8080...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /something//else HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 24 Mar 2023 06:01:38 GMT
< Content-Length: 12
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host localhost left intact
Hello, world

And if I want to match just /:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
package main

import "net/http"

func main() {
	handler := http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
		if request.URL.Path == "/" {
			writer.Write([]byte("Hello, world"))
		}

		http.NotFound(writer, request)
	})

	http.ListenAndServe(":8080", handler)
}

Conclusions

My main takeaway is that I need to be more careful with assumptions and read the source/manual more often, instead of just assume things with the same name work similarly across languages. Going through the issue also led me to read Go’s source code, which I honestly expected to be a lot harder than my previous experiences, mainly because it’s written in the same language. Lastly, it also showed me how a developer could take some shortcuts that would not please my explicit intent preferences, so I can be aware of those gotchas in the future. I’m glad I’ve done this.

Thank you.