Administrator
Published on 2026-11-21 / 3 Visits
0
0

The Search API Succeeded. Why Did the Agent Save Empty Related Terms?

The search request returned ten results. JSON decoding succeeded. Yet the related terms saved for the next agent were [{}, {}, {}]. Was the provider failing, or was there simply nothing related to search for?

In this case, the loss happened locally. The provider returned each term under query, while the archive adapter's allowed-field list omitted that key. A useful record became an empty object without raising an exception. This was an actual defect found while improving this blog's pipeline; the example below reduces it to public fields without credentials.

Find the last place where the term still exists

This sanitized fragment reproduces the relevant shape:

{"relatedSearches": [{"query": "MCP vs CLI vs API"}]}

An adapter retaining only title, link, snippet, question, and searchTerm will not crash on this input. It will produce an empty dictionary for the row.

Compare the same field at three points: immediately after the response, after normalization, and after saving and loading the archive. If the term disappears at the second point, inspect the adapter before changing providers. If it survives normalization but not the archive, inspect persistence instead.

Serper exposes structured search results, with different result blocks visible in its public example. The query shape described here was observed in this run; it is not a promise that every provider or endpoint uses the same field.Serper

Successful decoding does not prove useful output

JSON decoding establishes that text can become an object. It does not establish that a transformation retains the fields its consumer needs.Python JSON documentation

Nor does a valid main-results list protect the related-search branch. The earlier regression fixtures exercised the request and primary result processing but did not assert that relatedSearches[].query survived normalization. Those checks could pass while this branch lost information.

The missing assertion is small: given a related-search record containing a term, the normalized output must still contain that term. Checking only that the list is nonempty is insufficient: [{}] is nonempty too.

Keep four different situations distinguishable

A response without relatedSearches did not supply that block. An explicit relatedSearches: [] supplied it without entries. Neither response measures total market demand for the subject.

A block that changes from an array to an object, or an array containing records without a valid query, raises a different question: has the response shape or our interpretation changed? Silently filtering every such case into [] can hide the next compatibility problem.

The production repair made two bounded changes: preserve the public query field and discard objects left empty by field filtering. It fixes the observed loss; it does not expose a complete four-state business interface. The teaching example below is stricter: missing data returns None, an explicit empty array returns [], and unexpected types or records raise an error.

Test the transformation with a fixture

def related_queries(payload):
    if not isinstance(payload, dict):
        raise ValueError('response must be an object')
    if 'relatedSearches' not in payload:
        return None
    rows = payload['relatedSearches']
    if not isinstance(rows, list):
        raise ValueError('relatedSearches must be an array')
    result = []
    for row in rows:
        if not isinstance(row, dict) or not isinstance(row.get('query'), str) or not row['query'].strip():
            raise ValueError('related search needs a nonempty query')
        result.append({'query': row['query'].strip()})
    return result

Test that a normal term survives and unrelated fields are excluded. Then test the missing block and empty array separately. Finally, test that wrong containers and bad rows fail visibly. A standard-library unittest fixture can exercise these branches without a live API credential.Python unittest documentation

Four test methods were run for this stricter example, covering those cases. They establish behavior for the supplied fixtures, not search relevance, every possible upstream response, or traffic uplift.

For another search service, start with its own sanitized response and adjust the mapping. When an agent next reports that no related questions were found, ask where that conclusion originated: did the provider return no entries, or did our transformation remove them?

Related reading: why structured output still needs semantic validation. That article checks whether formatted output satisfies business conditions. This case checks an earlier dependency: whether the adapter preserved the information at all.


Comment