Skip to content

API Gateway v2 (HTTP API) collapses repeated query parameters into one comma-joined value #1472

Description

@alexalligator

Context

First off — thanks for Zappa, and for the recent API Gateway v2 work. Moving our project to "apigateway_version": "v2" was almost entirely painless, so I hope this report is useful rather than a nuisance.

We hit one snag: with v2, a request containing repeated query parameters (?id=18&id=19&id=20) arrives at the application as a single parameter with the value "18,19,20". The same request under v1 works correctly.

This is because payload format 2.0 doesn't include the multiValueQueryStringParameters field that format 1.0 and ALB provide. Instead
it gives you two things:

  • queryStringParameters — repeats flattened into a comma-joined value, i.e. {"id": "18,19,20"}
  • rawQueryString — the query string verbatim and percent-encoded, i.e. "id=18&id=19&id=20"

process_lambda_payload_v2() currently reads the first one:

query = event_info.get("queryStringParameters", {})
query_string = urlencode(query) if query else ""

urlencode({"id": "18,19,20"}) produces id=18%2C19%2C20, which frameworks parse back as a single value. The v1 path handles this correctly just above, via urlencode(query, doseq=True) over multiValueQueryStringParameters.

A couple of related notes that might be worth folding into the same fix:

  • ASGI is affected too — zappa/asgi.py calls the same function, so a single change covers both.
  • Lambda Function URLs are affected as well, since they also use payload format 2.0 regardless of the apigateway_version setting.
  • Even for a parameter that isn't repeated, queryStringParameters can't tell ?tags=a,b (one value with a comma) from ?tags=a&tags=b whereas rawQueryStringcan.

API Gateway v1 and ALB are unaffected.

Expected Behavior

Repeated parameters reach the application intact, identically under v1 and v2:

request.GET.getlist("id")    # ['18', '19', '20']

Actual Behavior

Under v2 the application receives one value:

request.GET.getlist("id")    # ['18,19,20']

In our case this surfaced as a 500 from a Wagtail multiple snippet chooser, which sends selected IDs as repeated id parameters and passes them to a Django __in filter:

ValueError at /admin/snippets/choose/tabs/licenceclass/chosen-multiple/
Field 'id' expected a number but got '18,19,20,26,10,21,28,23,11,22'.

The same would apply to request.args.getlist() in Flask, django-filter, DRF filter backends, and anything else built on repeated parameters.

Possible Fix

Build QUERY_STRING from rawQueryString when that key is present, passing it through as-is — API Gateway already delivers it percent-encoded, so re-encoding would double-escape it and unquoting would break a literal & or = inside a value. Falling back to queryStringParameters when the key is absent keeps non-API-Gateway invokers working.

I have a PR ready with this change plus regression tests, and I'm happy to adjust the approach if you'd prefer to solve it differently.

One small thing that may explain how this slipped through: every v2 event fixture in the test suite currently uses "rawQueryString": "", and the only v2 assertion on QUERY_STRING checks that it equals "" — so it passes trivially.
There's no existing test that sends a v2 event with a non-empty query string.

Steps to Reproduce

No AWS account needed — this reproduces against the installed package, using the event shape API Gateway v2 actually sends.

  1. Save as repro.py:
from urllib.parse import parse_qs

from zappa.wsgi import create_wsgi_request

# What API Gateway v2 sends for GET /thing?id=18&id=19&id=20
v2_event = {
    "version": "2.0",
    "routeKey": "ANY /{proxy+}",
    "rawPath": "/thing",
    "rawQueryString": "id=18&id=19&id=20",
    "queryStringParameters": {"id": "18,19,20"},  # <- AWS flattens repeats here
    "headers": {"host": "example.com"},
    "requestContext": {"http": {"method": "GET", "path": "/thing"}, "stage": "dev"},
    "isBase64Encoded": False,
}

# The equivalent v1 event
v1_event = {
    "httpMethod": "GET",
    "path": "/thing",
    "headers": {"Host": "example.com"},
    "multiValueQueryStringParameters": {"id": ["18", "19", "20"]},
    "requestContext": {},
    "body": None,
}

for name, event in (("v2", v2_event), ("v1", v1_event)):
    qs = create_wsgi_request(event)["QUERY_STRING"]
    print(f"{name}: QUERY_STRING={qs!r}  parse_qs={parse_qs(qs)}")
  1. Run python repro.py

  2. Output:

v2: QUERY_STRING='id=18%2C19%2C20'   parse_qs={'id': ['18,19,20']}
v1: QUERY_STRING='id=18&id=19&id=20' parse_qs={'id': ['18', '19', '20']}

The v1 line is what both should produce.

Your Environment

  • Zappa version used: latest (main branch)
  • Operating System and Python version: Python 3.12 Docker based Lambda
  • Your zappa_settings.json:
{
    "production": {
        "apigateway_version": "v2"
    }
}

Thanks again for maintaining this — happy to test any fix against our deployment.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions