Integrating qs_codec for Complex Query Encoding in HTTPX

[techouse]

Background: Complex Query Parameters in HTTP APIs

HTTPX handles basic query parameters well (flat key–value pairs and simple lists). However, many web APIs (e.g., headless CMSs, search endpoints) use nested or structured query parameters to express filters, arrays, or deep object structures. In the Node.js ecosystem, the widely used qs library enables this with bracket and dot notation (foo[bar]=baz, a.b=c) and is commonly used by frameworks such as Express.

Current situation: HTTPX does not natively expand nested dictionaries into bracket/dot notation and only supports simple lists by repeating a key. More complex structures require manual encoding. This makes interacting with APIs that expect qs-style query strings tedious and error-prone.

What is qs_codec?

qs_codec is a faithful Python port of Node’s qs library. It brings to Python the same semantics for encoding and decoding nested query strings:

• Zero external dependencies
• 800+ unit tests (parity-focused) covering encode/decode behavior
• Python 3.9+
• Fully typed

Features and Advantages of qs_codec

• Nested objects and arrays: 'foo[bar][baz]=qux' ⇄ {'foo': {'bar': {'baz': 'qux'}}}.
• Multiple array formats: indexed (arr[0]=x&arr[1]=y), brackets (arr[]=x&arr[]=y), repeated keys (arr=x&arr=y), comma-separated (arr=x,y) with optional comma round-trip on decode.
• Dot notation: a.b=c ⇄ {'a': {'b': 'c'}}, with options to encode literal dots when needed.
• Character encoding: UTF-8 (default) and Latin-1; optional charset sentinel (utf8=✓) to auto-detect encodings.
• Hooks and controls: pluggable encoder/decoder, sorting and filtering, control over percent-encoding (keys-only, values-only), alternate delimiters (string or re.Pattern), ignore leading ?.
• Nulls and empties: strict_null_handling, skip_nulls, and support for empty lists.
• Dates: serialize_date for ISO 8601 or custom (e.g., UNIX timestamp).
• Safety limits: configurable max depth, parameter count, and list index; optional strict-depth errors; duplicate-key strategies (combine/first/last).

Usage Example: qs_codec in Practice (Strapi Client + HTTPX)

The Python package strapi-client uses HTTPX and employs qs_codec to serialize complex, nested queries compatible with Strapi (which commonly expects qs-style parameters). See the implementation in their repo: api_parameters.py#L62.

A simplified example of how developers can pair HTTPX with qs_codec today:

import httpx
import qs_codec as qs

# Build a nested query payload
query = {
    "filters": {
        "author": {"name": {"$eq": "Alice"}},
        "$or": [
            {"views": {"$gt": 1000}},
            {"title": {"$contains": "Python"}}
        ]
    },
    "populate": ["author", "tags"],
    "pagination": {"page": 2, "pageSize": 25},
}

# Encode into a single, correct query string
qs_str = qs.encode(query)  # percent-encoded by default (RFC 3986)

# Use with HTTPX
url = f"https://cms.example.com/api/articles?{qs_str}"
resp = httpx.get(url, timeout=10)
resp.raise_for_status()
data = resp.json()

Decoding is equally straightforward (e.g., when inspecting or transforming an incoming query string):

params = qs.decode("filters[author][name][$eq]=Alice&pagination[page]=2")
# {'filters': {'author': {'name': {'$eq': 'Alice'}}}, 'pagination': {'page': '2'}}

Proposal: Integrate or Recommend qs_codec in HTTPX

Two possible directions:

1. Native integration (optional/flagged): HTTPX could delegate nested query encoding to qs_codec when a nested structure is detected or when a specific flag is set (e.g., params_style="qs"). Because qs_codec has zero external dependencies and comprehensive tests, this would bring robust qs-style semantics directly into HTTPX.

2. Documentation recommendation: If integration is out of scope, the HTTPX docs could recommend qs_codec for advanced/nested query parameters. HTTPX already accepts a prebuilt query string (e.g., client.get(url, params="a[b]=c&d[0]=e")), so pointing users to qs_codec provides a clear, supported path without changing core behavior.

Why this benefits HTTPX users

• Matches de-facto patterns in the Node/Express world (and APIs like Strapi).
• Eliminates ad-hoc string manipulation for complex queries.
• Reduces bug surface around edge cases (space encoding modes, percent handling, nested arrays/objects).
• Encourages a consistent, battle-tested approach (parity with Node qs).

Links

• qs_codec (Python): https://github.com/techouse/qs_codec
• Node qs: https://www.npmjs.com/package/qs
• strapi-client on PyPI: https://pypi.org/project/strapi-client/
• Usage in strapi-client: https://github.com/Roslovets-Inc/strapi-client/blob/5bebcdad131c134fe47faca9b6e02eba0b100df1/src/strapi_client/models/api_parameters.py#L62

[lovelydinosaur]

Thanks. Appreciate the work, though wouldn't want this built in.

[lovelydinosaur]

Would you be open to a small docs addition

Perhaps.

I'm looking towards httpx 1.0 currently... the docs are somewhat terse right now, tho do at least have some proper URL docs.

Eg you could use the following patten...

def qurl(url: str, nestedparams: dict) -> httpx.URL:
    return httpx.URL(url, query=qs_codec.encode(nestedparams))

Notably the client implementation is so much simpler in httpx 1.0 pre-release, that custom clients are simple to define. See the example in the client docs and it'll be clear enough how to amend that without the indirection of custom transport classes.

[techouse]

Oh, cool, that makes it 99% easier to write extensions 😅 Where can I find the docs for httpx v1.0? All I see here are the current docs, so I've added a new httpx v0-style document here #3643

[techouse]

@karpetrosyan @lovelydinosaur I've publised a package that does the above https://pypi.org/project/httpx-qs/

[karpetrosyan]

Loved qs-codec! Super polished and very well tested. Awesome job!

[techouse]

Thanx 😇