Content Negotiation in APIs: Accept Headers, Versioning, and Beyond

Your API returns JSON. Your mobile client wants Protobuf for bandwidth savings. Your enterprise customer demands XML. A legacy integration needs CSV. Content negotiation is the HTTP mechanism that lets a single endpoint serve all of them — and it goes far beyond just switching serialization formats.

How Content Negotiation Works#

Content negotiation uses HTTP headers to let the client express preferences and the server select the best representation.

The Accept Header#

The Accept header tells the server which media types the client can handle, optionally with quality weights:

Accept: application/json, application/xml;q=0.9, */*;q=0.1

This reads as: "I prefer JSON (implicit q=1.0), XML is acceptable (q=0.9), and anything else is a last resort (q=0.1)."

Server-Driven Negotiation#

The server examines the Accept header and selects the best match from its supported representations. The response includes a Content-Type header indicating what was chosen:

Request:
GET /api/products/42 HTTP/1.1
Accept: application/json

Response:
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept

The Vary: Accept header tells caches that the response varies based on the Accept header — critical for CDN correctness.

406 Not Acceptable#

If the server cannot produce any representation the client accepts, it should return 406 Not Acceptable with a list of available formats:

HTTP/1.1 406 Not Acceptable
Content-Type: application/json

{
  "error": "Not Acceptable",
  "supported_types": [
    "application/json",
    "application/xml",
    "application/protobuf"
  ]
}

Format Support#

JSON#

The default for most modern APIs. Strengths: human-readable, universal tooling, flexible schema. Weaknesses: verbose, no native binary support, parsing overhead.

Content-Type: application/json

XML#

Still prevalent in enterprise, finance, and government integrations. Strengths: schema validation (XSD), namespaces, digital signatures (XML-DSIG). Weaknesses: verbose, complex parsing, declining ecosystem.

Content-Type: application/xml

Protocol Buffers (Protobuf)#

A binary serialization format from Google. Strengths: compact (3-10x smaller than JSON), fast serialization, strict schema via .proto files, backward-compatible evolution. Weaknesses: not human-readable, requires compiled stubs.

Content-Type: application/protobuf

Other Formats#

Implementation Pattern#

from flask import Flask, request, jsonify
import xmltodict
import proto_module_pb2

app = Flask(__name__)

SERIALIZERS = {
    'application/json': lambda data: (jsonify(data), 'application/json'),
    'application/xml': lambda data: (
        xmltodict.unparse({'product': data}),
        'application/xml'
    ),
    'application/protobuf': lambda data: (
        proto_module_pb2.Product(**data).SerializeToString(),
        'application/protobuf'
    ),
}

@app.route('/api/products/<int:product_id>')
def get_product(product_id):
    product = fetch_product(product_id)
    best_match = request.accept_mimetypes.best_match(SERIALIZERS.keys())

    if best_match is None:
        return {'error': 'Not Acceptable'}, 406

    body, content_type = SERIALIZERS[best_match](product)
    return body, 200, {'Content-Type': content_type, 'Vary': 'Accept'}

API Versioning via Content Type#

Instead of version numbers in URLs (/v1/products) or query parameters (?version=1), you can version through content negotiation using vendor media types.

Vendor Media Type Pattern#

Accept: application/vnd.myapi.v2+json

The structure: application/vnd.{vendor}.{version}+{format}

How It Works#

# Client requesting v1
GET /api/products/42 HTTP/1.1
Accept: application/vnd.myapi.v1+json

# Client requesting v2
GET /api/products/42 HTTP/1.1
Accept: application/vnd.myapi.v2+json

The server routes to the appropriate handler based on the version in the Accept header. The URL stays the same across versions.

Comparing Versioning Strategies#

Content-type versioning is favored by API purists for its adherence to HTTP semantics, but URL-path versioning remains the most practical choice for most teams due to its simplicity and tooling support.

Compression#

Content negotiation also handles compression through the Accept-Encoding header.

Client Request#

GET /api/products HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, br, zstd

Server Response#

HTTP/1.1 200 OK
Content-Type: application/json
Content-Encoding: br
Vary: Accept, Accept-Encoding

Compression Algorithms#

When to Compress#

• Always for text-based formats (JSON, XML, HTML, CSV) — typically 70-90 % size reduction.
• Skip for already-compressed formats (Protobuf, images, video) — compression adds CPU cost with negligible size benefit.
• Set a minimum size threshold — compressing a 50-byte response wastes CPU. A common threshold is 1 KB.

Server Configuration (Nginx)#

gzip on;
gzip_types application/json application/xml text/csv;
gzip_min_length 1024;
gzip_vary on;

# Brotli (requires ngx_brotli module)
brotli on;
brotli_types application/json application/xml text/csv;
brotli_min_length 1024;

Pagination Headers#

While not strictly content negotiation, pagination headers follow the same pattern of client-server communication through HTTP headers.

Link Header (RFC 8288)#

The Link header provides navigation URIs for paginated responses:

HTTP/1.1 200 OK
Content-Type: application/json
Link: </api/products?page=3>; rel="next",
      </api/products?page=1>; rel="prev",
      </api/products?page=1>; rel="first",
      </api/products?page=42>; rel="last"
X-Total-Count: 1042
X-Page-Size: 25

Content-Range for Offset Pagination#

Borrowing from HTTP range requests:

Content-Range: items 50-74/1042

This tells the client: "You are viewing items 50 through 74 out of 1,042 total."

Accept Header for Page Size#

Some APIs allow the client to request a page size via a custom Accept parameter:

Accept: application/json; page-size=50

Cursor-Based Pagination#

For large datasets, cursor-based pagination avoids the performance pitfalls of OFFSET:

GET /api/products?after=eyJpZCI6MTAwfQ&limit=25 HTTP/1.1
Accept: application/json

HTTP/1.1 200 OK
Content-Type: application/json
Link: </api/products?after=eyJpZCI6MTI1fQ&limit=25>; rel="next"

The cursor (after) is an opaque token (often a base64-encoded identifier) that points to the last item in the current page.

Language and Character Negotiation#

Accept-Language#

Accept-Language: en-US, en;q=0.9, fr;q=0.5

The server returns localized content and indicates the language in the response:

Content-Language: en-US
Vary: Accept-Language

Accept-Charset#

Largely obsolete — modern APIs use UTF-8 universally. But the mechanism exists:

Accept-Charset: utf-8, iso-8859-1;q=0.5

Best Practices#

1. Default to JSON — If no Accept header is provided, respond with JSON. Do not return 406.
2. Always include Vary — List every header that influences the response (Accept, Accept-Encoding, Accept-Language).
3. Use quality weights for defaults — When the client sends Accept: */*, select JSON as the highest-quality default.
4. Document supported types — List available content types in your API documentation and in 406 responses.
5. Version deliberately — Pick one versioning strategy and apply it consistently. Do not mix URL-path and content-type versioning in the same API.
6. Compress aggressively — Enable Brotli or Zstandard for text formats. The bandwidth savings compound at scale.
7. Paginate from day one — Unbounded list endpoints become performance nightmares. Use cursor-based pagination for large or frequently-updated datasets.
8. Test with explicit headers — Use curl -H "Accept: application/xml" to verify negotiation works for every supported type.
9. Cache correctly — Incorrect Vary headers cause CDNs to serve the wrong format to the wrong client.

Content negotiation transforms a rigid, single-format API into a flexible contract between client and server. It is built into HTTP — you are not adding complexity, you are using the protocol as it was designed.

---

This is article #290 on Codelit.io — leveling up your engineering knowledge, one deep dive at a time. Explore more at codelit.io.