While dense vector search has transformed information retrieval, relying exclusively on semantic embeddings often introduces blind spots. Vector search excels at capturing conceptual intent, but frequently stumbles on exact part numbers, acronyms, and specialized nomenclature. Conversely, traditional full-text search handles exact lexical matches with precision but lacks contextual intuition. By pairing PostgreSQL’s pgvector and native tsvector engines within a unified Go service, you can combine the best of both paradigms without introducing the operational overhead of a dedicated external vector database.
Concurrent Query Execution in Go
The primary concern when executing two distinct query patterns against a relational database is latency. Running lexical and semantic queries sequentially doubles your round-trip time. Fortunately, Go’s lightweight concurrency primitives make it straightforward to execute both queries concurrently. By utilizing an errgroup.Group, your application can dispatch the semantic embedding match (utilizing an HNSW index) and the full-text search (backed by a GIN index) across separate pooled connections simultaneously. Your total search latency becomes bounded by the slower of the two queries rather than their sum.
The Normalization Problem and Reciprocal Rank Fusion
Once both result sets return, the fundamental challenge is reconciling their scores. Comparing cosine distance (typically bounded between 0 and 1) against full-text relevance scores like ts_rank is an apples-to-oranges comparison that yields unpredictable weights. Reciprocal Rank Fusion (RRF) resolves this problem by discarding raw score values entirely and evaluating candidates solely by their relative rank positions within each list.
The RRF formula scores each candidate document d as follows:
- RRF Score: The sum of
1 / (k + rank)across both search mechanisms. - Constant k: A smoothing parameter (conventionally set to 60) that prevents top-ranked outliers from completely dominating the aggregated score.
- Zero-Overhead Merging: Documents appearing in both result sets accumulate higher scores, while documents present in only one set are still considered fairly based on their internal rank.
In-Memory Fusion vs. Complex SQL CTEs
While it is possible to write complex Common Table Expressions (CTEs) to calculate RRF directly inside PostgreSQL, offloading the rank fusion to Go provides significant performance and architectural benefits. Fetching the top 50 to 100 candidate IDs from each pipeline requires minimal network transfer. Merging these slices in Go using a simple hash map avoids putting analytical compute pressure on your primary database, reduces connection hold times, and executes in single-digit microseconds.
By unifying pgvector and tsvector with Go-driven parallel execution and Reciprocal Rank Fusion, you achieve state-of-the-art search precision while preserving sub-50ms response times and maintaining an elegantly simple infrastructure footprint.
— Muntasir Ashif
