Ranking and relevance scoring for search results.
Supports two scoring strategies:
:tf— Simple Term Frequency scoring:tfidf— TF-IDF (Term Frequency × Inverse Document Frequency):bm25— BM25 (Okapi BM25) probabilistic relevance scoring
BM25 Formula
score(D, Q) = Σ IDF(qi) × (tf(qi, D) × (k1 + 1)) / (tf(qi, D) + k1 × (1 − b + b × (|D| / avgdl)))Where:
- tf(qi, D) = term frequency of term qi in document D
- |D| = document length (sum of all TF in document)
- avgdl = average document length across the collection
- k1 = term frequency saturation (default 1.2)
- b = length normalization (default 0.75)
Summary
Functions
Computes the IDF (Inverse Document Frequency) for a term.
Ranks results using the specified strategy.
Types
@type result() :: {String.t(), float(), [{String.t(), non_neg_integer()}]}
Functions
@spec idf(non_neg_integer(), non_neg_integer()) :: float()
Computes the IDF (Inverse Document Frequency) for a term.
IDF(t) = log(1 + (N − df(t) + 0.5) / (df(t) + 0.5))Where N is the total number of documents and df(t) is the number of documents containing the term.
@spec rank( [{String.t(), [{String.t(), non_neg_integer()}]}], keyword() ) :: [result()]
Ranks results using the specified strategy.
Returns a list of {post_id, score, term_scores} tuples sorted by
descending score.
Options
:strategy—:tf(default),:tfidf, or:bm25:k1— BM25 k1 parameter (default 1.2):b— BM25 b parameter (default 0.75):total_docs— total document count (for IDF, required for TF-IDF/BM25):doc_freqs— %{term => doc_freq} map (for IDF, required for TF-IDF/BM25):avg_doc_length— average document length (for BM25)