AshScylla.Search.Storage (AshScylla v1.6.1)

Copy Markdown View Source

CQL schema definitions for the inverted index tables used by the search engine.

Provides functions to create and drop the required tables for the inverted index. The tables are:

  • search_post_terms — maps terms to post IDs with term frequency (TF)
  • search_post_fields — stores the analyzed field text for update/delete diffing

Usage

AshScylla.Search.Storage.create_tables(MyApp.Repo, "my_keyspace")
AshScylla.Search.Storage.drop_tables(MyApp.Repo, "my_keyspace")

Summary

Functions

Returns the CQL statement to create the search_post_fields table.

Returns the CQL statement to create the search_post_terms table.

Creates all search engine tables in the given keyspace.

Drops all search engine tables from the given keyspace.

Computes the shard number for a given term.

Functions

create_post_fields_cql(keyspace)

@spec create_post_fields_cql(String.t()) :: String.t()

Returns the CQL statement to create the search_post_fields table.

Stores the raw analyzed text per field for each post. Used during updates to diff old vs new terms and compute additions/removals.

create_post_terms_cql(keyspace)

@spec create_post_terms_cql(String.t()) :: String.t()

Returns the CQL statement to create the search_post_terms table.

This is the primary inverted index table. Each row maps a term to a post_id with term frequency for ranking.

Partition key: (term, shard) to avoid hotspot partitions for common terms. Clustering key: post_id for ordered retrieval.

create_tables(repo, keyspace)

@spec create_tables(module(), String.t()) :: :ok | {:error, term()}

Creates all search engine tables in the given keyspace.

Returns :ok on success or {:error, reason} on failure.

drop_tables(repo, keyspace)

@spec drop_tables(module(), String.t()) :: :ok | {:error, term()}

Drops all search engine tables from the given keyspace.

Returns :ok on success or {:error, reason} on failure.

shard_for(term, num_shards \\ 16)

@spec shard_for(String.t(), non_neg_integer()) :: non_neg_integer()

Computes the shard number for a given term.

Uses :erlang.phash2/2 to distribute terms across the configured number of shards. This prevents hotspot partitions for high-frequency terms.