DEV Community

Bar Dror
Bar Dror

Posted on Originally published at codebeneath.com

HTTP Trailer Headers and Chunked Encoding: Sending Metadata After Body

Originally published on Code Beneath.

Most developers treat HTTP headers as a fixed contract: send them before the body, never after. But HTTP/1.1 defines a lesser-known feature called trailer headers that let you send metadata after the response body completes, and they work specifically with chunked transfer encoding. This becomes critical when you are streaming large files and need to compute a checksum only after reading all bytes, or when you want to signal an error condition or attach signing information to the payload. Understanding HTTP trailer headers chunked encoding example implementation means grasping how trailers fit into the chunked protocol, what the wire format looks like, and how to build both server and client code that actually uses them without breaking on intermediaries that strip them.

Why Trailers Exist and When to Use Them

In HTTP/1.0 and early HTTP/1.1, all headers had to be sent before the body began. That made sense for static responses, but streaming introduces a problem: if you are streaming a large file and you want to attach a SHA256 hash of the content, you have to either compute the hash upfront (expensive for video or large objects) or store the entire body in memory (wasteful). Trailers solve this by allowing a second set of headers to arrive after the body ends, within the same connection.

The HTTP/1.1 specification (RFC 7230, Section 4.1.2) says trailers are headers that appear after the message body, inside the last chunk of a chunked transfer. HTTP/2 and HTTP/3 also support trailers, though the mechanism is different. The key constraint: trailers only work with chunked transfer encoding (Transfer-Encoding: chunked), because without chunks, there is no defined boundary between body and trailers.

Common use cases for trailers include:

  • Content-MD5 or Digest headers computed over the streaming body
  • X-Checksum-SHA256 for data integrity verification
  • X-Error or X-Warning to signal problems detected during streaming
  • X-Signature for cryptographic signing of the response payload
  • X-Trailer-* custom metadata that only became known after processing the stream

The Chunked Encoding Format and Trailers

Chunked transfer encoding breaks the body into chunks, each prefixed with its size in hexadecimal. The final chunk is always size 0, followed by a CRLF (carriage return and line feed: \r\n). After that final chunk, the trailers appear as plain headers, then a final CRLF. Here is the exact wire format:

Transfer-Encoding: chunked

[chunk size in hex]\r\n
[chunk data]\r\n
[next chunk size in hex]\r\n
[next chunk data]\r\n
...
0\r\n
[trailer-name]: [trailer-value]\r\n
[another-trailer]: [value]\r\n
\r\n
Enter fullscreen mode Exit fullscreen mode

If you are sending trailers, you must include a Trailer header in the response telling the client which trailers to expect. Some proxies and intermediaries strip unknown or unsupported trailers, so declaring them upfront helps.

Server Implementation: Streaming a Digest Trailer in Go

Here is a complete, runnable Go server that streams a large file and appends a SHA256 digest trailer:

package main

import (
    "crypto/sha256"
    "fmt"
    "io"
    "net/http"
    "strings"
)

func streamWithTrailer(w http.ResponseWriter, r *http.Request) {
    // Declare that we will send a Digest trailer
    w.Header().Set("Trailer", "Digest")
    w.Header().Set("Content-Type", "text/plain")
    w.Header().Set("Transfer-Encoding", "chunked")

    // Create a hasher to compute digest as we write
    hasher := sha256.New()

    // Use MultiWriter to write to both the response and the hasher
    mw := io.MultiWriter(w, hasher)

    // Simulate a large streaming response
    chunks := []string{
        "First chunk of data\n",
        "Second chunk of data\n",
        "Third chunk of data\n",
        "Fourth chunk of data\n",
    }

    for _, chunk := range chunks {
        if _, err := io.WriteString(mw, chunk); err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }
    }

    // Compute the digest in base64
    digest := fmt.Sprintf("SHA-256=%s", fmt.Sprintf("%x", hasher.Sum(nil)))

    // Set the trailer header after body is written
    w.Header().Set("Digest", digest)
}

func main() {
    http.HandleFunc("/stream", streamWithTrailer)
    fmt.Println("Server listening on :8080")
    http.ListenAndServe(":8080", nil)
}
Enter fullscreen mode Exit fullscreen mode

The key points here: the Trailer header is declared before any body data, the ResponseWriter automatically handles chunked encoding, and you set trailer headers via w.Header().Set() after writing the body but before the connection closes. The http.ResponseWriter in Go is smart enough to hold those headers and emit them in the final chunk boundary.

Client Implementation: Reading and Verifying Trailers

Now a client that reads the response body and then checks the trailer:

package main

import (
    "crypto/sha256"
    "fmt"
    "io"
    "net/http"
    "strings"
)

func main() {
    resp, err := http.Get("http://localhost:8080/stream")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    fmt.Printf("Status: %d\n", resp.StatusCode)
    fmt.Printf("Content-Type: %s\n", resp.Header.Get("Content-Type"))
    fmt.Printf("Transfer-Encoding: %s\n", resp.Header.Get("Transfer-Encoding"))
    fmt.Printf("Trailer: %s\n\n", resp.Header.Get("Trailer"))

    // Read the body and compute our own digest to verify
    hasher := sha256.New()
    body, err := io.ReadAll(io.TeeReader(resp.Body, hasher))
    if err != nil {
        panic(err)
    }

    fmt.Printf("Body (%d bytes):\n%s\n", len(body), string(body))

    // At this point, trailers are available in resp.Trailer
    digest := resp.Trailer.Get("Digest")
    fmt.Printf("\nDigest trailer: %s\n", digest)

    // Verify the digest matches our computation
    computedDigest := fmt.Sprintf("SHA-256=%x", hasher.Sum(nil))
    fmt.Printf("Computed digest: %s\n", computedDigest)

    if digest == computedDigest {
        fmt.Println("Digest verified successfully!")
    } else {
        fmt.Println("ERROR: Digest mismatch!")
    }
}
Enter fullscreen mode Exit fullscreen mode

The critical detail: in Go's http package, trailers appear in the response.Trailer map, but only after you have finished reading the body. The client must call io.ReadAll (or equivalent) on the response body to consume all chunks and trigger trailer parsing. If you try to access resp.Trailer before reading the body, you will get an empty map.

Running the Complete Example

Save the server code to server.go and the client code to client.go. Run them:

# Terminal 1
go run server.go

# Terminal 2
go run client.go
Enter fullscreen mode Exit fullscreen mode

You will see the client read the body, then verify the trailer digest matches. The chunked encoding and trailers happen transparently over the wire.

Wire Format Inspection with nc

To see the actual chunked format and trailers on the wire, use netcat to make a raw request:

(echo -e 'GET /stream HTTP/1.1\r\nHost: localhost:8080\r\nConnection: close\r\n\r\n'; sleep 1) | nc localhost 8080
Enter fullscreen mode Exit fullscreen mode

This outputs the raw HTTP response, including the chunked chunks and the trailer headers. You will see something like:

HTTP/1.1 200 OK
Content-Type: text/plain
Trailer: Digest
Transfer-Encoding: chunked

14
First chunk of data
15
Second chunk of data
15
Third chunk of data
15
Fourth chunk of data
0
Digest: SHA-256=abc123...
Enter fullscreen mode Exit fullscreen mode

The hex values (14, 15) are chunk sizes. The 0 marks the end. Then the trailer Digest header appears before the final CRLF.

Trailer Headers vs. Other Approaches

Approach Pros Cons When to Use
Trailer Headers Metadata after body; works with streaming; keeps HTTP semantic intact Requires chunked encoding; some proxies strip trailers; client must read full body first Digest, signatures, post-body errors on streaming responses
Content-Length with upfront headers All metadata known upfront; simpler; universal proxy support Must compute metadata before sending body; no streaming metadata Static files, pre-computed responses
Query param or separate endpoint Works everywhere; no encoding constraints Extra request; metadata not tied to specific response; coupling issues Polling for async status, optional metadata
Chunked body with metadata in footer chunk Streaming; custom format Not standard HTTP; requires client parsing logic; breaks caching Non-HTTP protocols; custom streaming formats

Critical Gotchas and Real-World Issues

The most common gotcha: you send trailers, but the client never sees them because an intermediary proxy stripped them. The HTTP/1.1 spec allows proxies to remove trailers, especially if they are not declared in the Trailer header. Always declare expected trailers upfront, and do not assume every trailer will reach the client.

Second gotcha: if you forget to set Transfer-Encoding: chunked, the server may send trailers but they will be ignored or misinterpreted as the start of a new request. Go's http.ResponseWriter handles this automatically if you use SetTrailer() early enough, but other frameworks may not.

Third gotcha: some load balancers and CDNs (Cloudflare, AWS ALB, Nginx without specific config) do not forward trailers to clients. If you rely on trailers for security (e.g., a signature), you must test with your actual deployment infrastructure.

Fourth gotcha: the Trailer header value should list which trailers you will send. If you declare "Trailer: Digest" but then send "Trailer: X-Custom", some strict clients will ignore the second one. Keep the declaration accurate.

Go http.ResponseWriter and SetTrailer

Go provides http.ResponseWriter.SetTrailer() to register which trailers you plan to send. Here is a more explicit version:

package main

import (
    "crypto/sha256"
    "fmt"
    "io"
    "net/http"
)

func streamWithExplicitTrailer(w http.ResponseWriter, r *http.Request) {
    // Register trailers before writing any body
    w.Header().Set("Trailer", "Digest, X-Error-Count")

    // Explicit call to SetTrailer (optional in Go, but explicit is better)
    w.Header().Add("Trailer", "Digest")
    w.Header().Add("Trailer", "X-Error-Count")

    w.Header().Set("Content-Type", "application/octet-stream")

    hasher := sha256.New()
    mw := io.MultiWriter(w, hasher)

    data := "Streaming data here"
    io.WriteString(mw, data)

    // Now set trailers
    w.Header().Set("Digest", fmt.Sprintf("sha256=%x", hasher.Sum(nil)))
    w.Header().Set("X-Error-Count", "0")
}

func main() {
    http.HandleFunc("/explicit", streamWithExplicitTrailer)
    http.ListenAndServe(":8080", nil)
}
Enter fullscreen mode Exit fullscreen mode

Trailers in HTTP/2 and HTTP/3

HTTP/2 and HTTP/3 do not use chunked encoding in the same way, but they still support trailers through their frame-based protocols. In HTTP/2, trailers are sent as a HEADERS frame after the DATA frames. In HTTP/3, they are in a HEADERS frame after the final DATA frame. The semantics are the same: metadata after the body. Go's http.ResponseWriter abstracts this, so the same trailer code works across HTTP/1.1, HTTP/2, and HTTP/3 if the server supports all versions.

Performance Implications

Trailers add minimal overhead if you are already streaming. The memory cost is negligible (a few header values). The latency impact is zero: the client does not see trailers until the body is complete anyway. However, trailers can delay the final ACK in some TCP scenarios because the server must hold the connection open until trailers are sent. For tiny trailers (a digest or count), this is irrelevant. For large metadata, consider whether you should send it upfront instead.

Security Considerations

If you are using trailers for cryptographic signatures, ensure the signature is computed over the entire body. Computing a signature only over the first N bytes and sending it as a trailer defeats the purpose. Use a streaming hash algorithm (like sha256.Hash in Go) to accumulate all bytes as they are written, then sign the final digest in the trailer.

Be aware that intermediaries may strip trailers, so do not rely on trailers as your only authentication or integrity mechanism if data must pass through untrusted proxies. Use a combination: send a preliminary digest in a regular header (if possible) and confirm it with a trailer signature. Or use TLS to prevent stripping.

Common Mistakes

Mistake 1: Setting trailers before reading the body on the client side. Trailers only populate after the body is consumed. Always call io.ReadAll or equivalent first.

Mistake 2: Forgetting the Trailer header declaration. Declaring "Trailer: Digest" tells the client and intermediaries what trailers to expect. Without it, proxies may strip them.

Mistake 3: Sending trailers without chunked encoding. The chunked format is mandatory for trailers in HTTP/1.1. If you set Content-Length instead, trailers are invalid and will be ignored.

Mistake 4: Assuming all proxy layers support trailers. They do not. Test through your CDN and load balancer before relying on trailers in production.

Mistake 5: Sending a large volume of trailer data. Trailers should be small (checksums, counts, brief status). If you have megabytes of metadata, include it in the body, not trailers.

Relationship to Chunked Transfer Encoding

Trailers are an extension of chunked transfer encoding. For more details on the chunked encoding itself, including how to decode chunks manually and handle chunk extensions, see the guide on HTTP chunked transfer encoding. Trailers sit at the end of the chunk sequence, after the zero-length terminating chunk.

FAQ

Do trailers work in HTTP/1.0?

No. HTTP/1.0 does not support chunked encoding or trailers. HTTP/1.1 made them optional, and HTTP/2 and HTTP/3 support them natively (though with different framing).

Can I send trailers without chunked encoding?

No. The HTTP/1.1 specification ties trailers to chunked transfer encoding. Without chunks, there is no defined end-of-body boundary, so trailers are invalid.

What happens if a proxy strips my trailers?

The client will not see them. If trailers are critical to your application, you must either use a direct connection (no proxies), verify support with your infrastructure team, or use an alternative mechanism (regular headers, separate request).

Can I send trailers conditionally?

Yes. Declare "Trailer: Digest" in the response header, but only actually set w.Header().Set("Digest", ...) if the condition is met. If the trailer is missing, clients that see the Trailer declaration will not expect it. Be consistent and explicit.

How large can a trailer be?

There is no hard limit in the spec, but typical web servers and clients have limits around 8 KB for all headers combined (regular plus trailers). Keep trailers small: a digest, a count, a brief status code.

Can I use trailers for error messages from a streaming endpoint?

Yes. This is a common use case. If an error occurs partway through streaming, you can send trailers like "X-Stream-Error: Database connection lost". The client reads the partial body and the trailer, and understands that the stream was incomplete.

Do trailers work with gzip compression?

Yes, but carefully. The trailers should be sent after decompression on the client side, so a gzip-compressed stream still ends with trailers. However, if you are computing a digest of the compressed bytes, declare that in a trailer (e.g., "X-Gzip-Digest"). Digest of the original uncompressed data should go in a regular header before compression, not in a trailer.

How do I debug trailers?

Use curl with -v flag: "curl -v http://localhost:8080/stream". Curl will print trailers if they are present. For raw inspection, use netcat as shown above. In Go, enable httputil.DumpResponse or log resp.Trailer after reading the body.

Top comments (0)