Refget Module API Reference

Type stubs and documentation for the gtars.refget module.

This file serves two purposes:

1. Type Hints: Provides type annotations for IDE autocomplete and static type checking tools like mypy.

2. Documentation: Contains Google-style docstrings that mkdocstrings uses to generate the API reference documentation website.

Note: The actual implementation is in Rust (gtars-python/src/refget/mod.rs) and compiled via PyO3. This stub file provides the Python interface definition and structured documentation that tools can parse properly.

AlphabetType

class AlphabetType

Bases: Enum

Represents the type of alphabet for a sequence.

Attributes

• Dna2bit (int)
• Dna3bit (int)
• DnaIupac (int)
• Protein (int)
• Ascii (int)
• Unknown (int)

Methods

__str__

def __str__() -> str

FaiMetadata

class FaiMetadata

FASTA index (FAI) metadata for a sequence.

Contains the information needed to quickly seek to a sequence in a FASTA file, compatible with samtools faidx format.

Attributes:

• offset (int): Byte offset of the first base in the FASTA file.
• line_bases (int): Number of bases per line.
• line_bytes (int): Number of bytes per line (including newline).

Attributes

• offset (int)
• line_bases (int)
• line_bytes (int)

Methods

__repr__

def __repr__() -> str

__str__

def __str__() -> str

FaiRecord

class FaiRecord

A FASTA index record for a single sequence.

Represents one line of a .fai index file with sequence name, length, and FAI metadata for random access.

Attributes:

• name (str): Sequence name.
• length (int): Sequence length in bases.
• fai (Optional[FaiMetadata]): FAI metadata (None for gzipped files).

Attributes

• name (str)
• length (int)
• fai (Optional[FaiMetadata])

Methods

__repr__

def __repr__() -> str

__str__

def __str__() -> str

SequenceMetadata

class SequenceMetadata

Metadata for a biological sequence.

Contains identifying information and computed digests for a sequence, without the actual sequence data.

Attributes:

• name (str): Sequence name (first word of FASTA header).
• description (Optional[str]): Description from FASTA header (text after first whitespace).
• length (int): Length of the sequence in bases.
• sha512t24u (str): GA4GH SHA-512/24u digest (32-char base64url).
• md5 (str): MD5 digest (32-char hex string).
• alphabet (AlphabetType): Detected alphabet type (DNA, protein, etc.).
• fai (Optional[FaiMetadata]): FASTA index metadata if available.

Attributes

• name (str)
• description (Optional[str])
• length (int)
• sha512t24u (str)
• md5 (str)
• alphabet (AlphabetType)
• fai (Optional[FaiMetadata])

Methods

__repr__

def __repr__() -> str

__str__

def __str__() -> str

SequenceRecord

class SequenceRecord

A record representing a biological sequence, including its metadata and optional data.

SequenceRecord can be either a "stub" (metadata only) or "full" (metadata + data). Stubs are used for lazy-loading where sequence data is fetched on demand.

Attributes:

• metadata (SequenceMetadata): Sequence metadata (name, length, digests).
• sequence (Optional[bytes]): Raw sequence data if loaded, None for stubs.
• is_loaded (bool): Whether sequence data is loaded (True) or just metadata (False).

Attributes

• metadata (SequenceMetadata)
• sequence (Optional[bytes])

Properties

is_loaded

@property
def is_loaded -> bool

Whether sequence data is loaded (true) or just metadata (false).

Methods

decode

def decode() -> Optional[str]

Decode and return the sequence data as a string.

For Full records with sequence data, returns the decoded sequence. For Stub records without sequence data, returns None.

Returns:

• Optional[str]: Decoded sequence string if data is available, None otherwise.

__repr__

def __repr__() -> str

__str__

def __str__() -> str

SeqColDigestLvl1

class SeqColDigestLvl1

Level 1 digests for a sequence collection.

Attributes

• sequences_digest (str)
• names_digest (str)
• lengths_digest (str)

Methods

__repr__

def __repr__() -> str

__str__

def __str__() -> str

SequenceCollectionMetadata

class SequenceCollectionMetadata

Metadata for a sequence collection.

Contains the collection digest and level 1 digests for names, sequences, and lengths. This is a lightweight representation of a collection without the actual sequence list.

Attributes:

• digest (str): The collection's SHA-512/24u digest.
• n_sequences (int): Number of sequences in the collection.
• names_digest (str): Level 1 digest of the names array.
• sequences_digest (str): Level 1 digest of the sequences array.
• lengths_digest (str): Level 1 digest of the lengths array.
• name_length_pairs_digest (Optional[str]): Ancillary digest (if computed).
• sorted_name_length_pairs_digest (Optional[str]): Ancillary digest (if computed).
• sorted_sequences_digest (Optional[str]): Ancillary digest (if computed).

Attributes

• digest (str)
• n_sequences (int)
• names_digest (str)
• sequences_digest (str)
• lengths_digest (str)
• name_length_pairs_digest (Optional[str])
• sorted_name_length_pairs_digest (Optional[str])
• sorted_sequences_digest (Optional[str])

Methods

__repr__

def __repr__() -> str

__str__

def __str__() -> str

ImportReport

class ImportReport

Result of importing one or more FASTA files.

These are genuine per-run counters describing what the import actually did, unlike RefgetStore.stats, which is a snapshot of current RAM residency.

n_sequences_written + n_sequences_deduped equals the total number of sequence records seen across all processed files. Files short-circuited as already-present collections (before their FASTA is ever opened) contribute nothing to either counter.

Attributes:

• collections (List[tuple[SequenceCollectionMetadata, bool]]): Per-file (SequenceCollectionMetadata, was_new) results in expanded-input order.
• n_sequences_written (int): Sequences whose bytes were written this run (genuinely new content).
• n_sequences_deduped (int): Sequences already present by content digest, so no bytes were written.
• n_collections_new (int): Number of collections newly added this run.

Attributes

• collections (List[tuple[SequenceCollectionMetadata, bool]])
• n_sequences_written (int)
• n_sequences_deduped (int)
• n_collections_new (int)

Methods

__repr__

def __repr__() -> str

__len__

def __len__() -> int

VerifyFailure

class VerifyFailure

One sequence that failed RefgetStore.verify / ReadonlyRefgetStore.verify.

Attributes:

• digest (str): Stored sha512t24u digest of the sequence.
• name (str): Sequence name.
• alphabet (AlphabetType): Detected alphabet.
• length (int): Stored sequence length in bases.
• kind (str): One of "digest_mismatch", "md5_mismatch", "length_mismatch", "read_error". Precedence when a sequence has multiple problems: read_error > length_mismatch > digest_mismatch > md5_mismatch (only one failure is ever reported per sequence).
• bases_read (int): Number of bytes actually streamed/hashed before the check stopped.
• computed_sha512t24u (Optional[str]): Recomputed digest. None only for a "read_error" where zero bytes were read.
• stored_md5 (str): The md5 recorded in the store (may be empty).
• computed_md5 (Optional[str]): Recomputed md5. None when md5 was not checked, or for a "read_error".
• error (Optional[str]): The underlying I/O error, set for "read_error".

Attributes

• digest (str)
• name (str)
• alphabet (AlphabetType)
• length (int)
• kind (str)
• bases_read (int)
• computed_sha512t24u (Optional[str])
• stored_md5 (str)
• computed_md5 (Optional[str])
• error (Optional[str])

Methods

__repr__

def __repr__() -> str

VerifyReport

class VerifyReport

Result of RefgetStore.verify / ReadonlyRefgetStore.verify.

Attributes:

• n_checked (int): Number of sequences verified.
• n_ok (int): Number that verified clean.
• n_failed (int): Number that failed (equal to len(failures)).
• by_alphabet (Dict[str, tuple[int, int]]): Per-alphabet (checked, failed) counts, keyed by the alphabet's string form (e.g. "dna2bit", "dnaio", "protein", "ASCII").
• failures (List[VerifyFailure]): One entry per failing sequence, sorted by digest.

Attributes

• n_checked (int)
• n_ok (int)
• n_failed (int)
• by_alphabet (Dict[str, tuple[int, int]])
• failures (List[VerifyFailure])

Properties

ok

@property
def ok -> bool

True when no sequence failed verification.

Methods

to_dict

def to_dict() -> dict[str, Any]

Convert to a plain (JSON-serializable) dict.

__repr__

def __repr__() -> str

__len__

def __len__() -> int

Number of failing sequences.

__bool__

def __bool__() -> bool

True when no sequence failed verification (same as .ok).

SequenceCollection

class SequenceCollection

A collection of biological sequences (e.g., a genome assembly).

SequenceCollection represents a set of sequences with collection-level digests following the GA4GH seqcol specification. Supports iteration, indexing, and len().

Attributes:

• sequences (List[SequenceRecord]): List of sequence records.
• digest (str): Collection-level SHA-512/24u digest (Level 2).
• lvl1 (SeqColDigestLvl1): Level 1 digests for names, lengths, sequences.
• file_path (Optional[str]): Source file path if loaded from FASTA.

Examples:

Iterate over sequences:

for seq in collection:
    print(f"{seq.metadata.name}: {seq.metadata.length} bp")

Access by index:

first_seq = collection[0]
last_seq = collection[-1]

Get length:

n = len(collection)

Attributes

• sequences (List[SequenceRecord])
• digest (str)
• lvl1 (SeqColDigestLvl1)
• file_path (Optional[str])

Methods

write_fasta

def write_fasta(file_path: str, line_width: Optional[int] = None) -> None

Write the collection to a FASTA file.

Parameters:

• file_path (str): Path to the output FASTA file.
• line_width (Optional[int]): Number of bases per line (default: 70). (default: None)

Raises:

• IOError: If any sequence doesn't have data loaded.

Example:

collection = load_fasta("genome.fa")
collection.write_fasta("output.fa")
collection.write_fasta("output.fa", line_width=60)

__len__

def __len__() -> int

__getitem__

def __getitem__(idx: int) -> SequenceRecord

__iter__

def __iter__() -> Iterator[SequenceRecord]

__repr__

def __repr__() -> str

__str__

def __str__() -> str

RetrievedSequence

class RetrievedSequence(sequence: str, chrom_name: str, start: int, end: int)

Represents a retrieved sequence segment with its metadata.

Returned by methods that extract subsequences from specific regions, such as substrings_from_regions().

Attributes:

• sequence (str): The extracted sequence string.
• chrom_name (str): Chromosome/sequence name (e.g., "chr1").
• start (int): Start position (0-based, inclusive).
• end (int): End position (0-based, exclusive).

Attributes

• sequence (str)
• chrom_name (str)
• start (int)
• end (int)

Methods

__init__

def __init__(sequence: str, chrom_name: str, start: int, end: int) -> None

__repr__

def __repr__() -> str

__str__

def __str__() -> str

SequenceStream

class SequenceStream

Lazy, O(1)-memory stream over a (sub)sequence's decoded ASCII bases.

Produced by RefgetStore.stream_sequence() / ReadonlyRefgetStore.stream_sequence() (flow 2). Iterating yields str chunks of decoded bases as they are read from the backing source (resident bytes, a local .seq, or a remote HTTP byte-range), so peak memory stays bounded regardless of region length.

Example:

stream = store.stream_sequence("chr1_digest", 0, 1_000_000)
total = 0
for chunk in stream:
    total += len(chunk)

Methods

__iter__

def __iter__() -> SequenceStream

__next__

def __next__() -> str

read_all

def read_all() -> str

Read and return all remaining bases as a single string.

Exhausts the stream; defeats the O(1)-memory guarantee for the size of the result.

StorageMode

class StorageMode

Bases: Enum

Defines how sequence data is stored in the Refget store.

Variants

Raw: Store sequences as raw bytes (1 byte per base). Encoded: Store sequences with 2-bit encoding (4 bases per byte). Zstd: Each record compressed whole with zstd. Reported for stores built by other tools; cannot be set from Python.

Attributes

• Raw (int)
• Encoded (int)
• Zstd (int)

FhrMetadata

class FhrMetadata(**kwargs: Any)

FAIR Headers Reference genome metadata for a sequence collection.

Fields match the FHR 1.0 specification. All fields are optional. Note: schema_version is a number (int or float) per spec, passed as a Python numeric type and stored as serde_json::Number internally.

Attributes

• genome (Optional[str])
• version (Optional[str])
• masking (Optional[str])
• genome_synonym (Optional[list[str]])
• voucher_specimen (Optional[str])
• documentation (Optional[str])
• identifier (Optional[list[str]])
• scholarly_article (Optional[str])
• funding (Optional[str])

Methods

__init__

def __init__(**kwargs: Any) -> None

from_json

@staticmethod
def from_json(path: str) -> FhrMetadata

to_dict

def to_dict() -> dict[str, Any]

to_json

def to_json(path: str) -> None

__repr__

def __repr__() -> str

RefgetStore

class RefgetStore

A global store for GA4GH refget sequences with lazy-loading support.

RefgetStore provides content-addressable storage for reference genome sequences following the GA4GH refget specification. Supports both local and remote stores with on-demand sequence loading.

Attributes:

• cache_path (Optional[str]): Local directory path where the store is located or cached. None for in-memory stores.
• remote_url (Optional[str]): Remote URL of the store if loaded remotely, None otherwise.
• quiet (bool): Whether the store suppresses progress output.
• storage_mode (StorageMode): Current storage mode (Raw, Encoded, or Zstd).

Note

Boolean evaluation: RefgetStore follows Python container semantics, meaning bool(store) is False for empty stores (like list, dict, etc.). To check if a store variable is initialized (not None), use if store is not None: rather than if store:.

Example:

store = RefgetStore.in_memory()  # Empty store
bool(store)  # False (empty container)
len(store)   # 0

# Wrong: checks emptiness, not initialization
if store:
    process(store)

# Right: checks if variable is set
if store is not None:
    process(store)

Examples:

Create a new store and import sequences:

from gtars.refget import RefgetStore
store = RefgetStore.in_memory()
store.add_sequence_collection_from_fasta("genome.fa")

Open an existing local store:

store = RefgetStore.open_local("/data/hg38")
seq = store.get_substring("chr1_digest", 0, 1000)

Open a remote store with caching:

store = RefgetStore.open_remote(
    "/local/cache",
    "https://example.com/hg38"
)

Attributes

• cache_path (Optional[str])
• remote_url (Optional[str])

Properties

quiet

@property
def quiet -> bool

Whether the store is in quiet mode.

storage_mode

@property
def storage_mode -> StorageMode

Current storage mode (Raw, Encoded, or Zstd).

is_persisting

@property
def is_persisting -> bool

Whether the store is currently persisting to disk.

Example:

store = RefgetStore.in_memory()
print(store.is_persisting)  # False
store.enable_persistence("/data/store")
print(store.is_persisting)  # True

Class methods

in_memory

@classmethod
def in_memory() -> RefgetStore

Create a new in-memory RefgetStore.

Creates a store that keeps all sequences in memory. Use this for temporary processing or when you don't need disk persistence.

Returns:

• RefgetStore: New empty RefgetStore with Encoded storage mode.

Example:

store = RefgetStore.in_memory()
store.add_sequence_collection_from_fasta("genome.fa")

store_exists

@classmethod
def store_exists(path: Union[str, PathLike]) -> bool

Check whether a valid RefgetStore exists at the given path.

Returns True if the path contains a store manifest file, indicating the store has been initialized. Returns False if the path does not exist or does not contain a store.

This avoids hardcoding knowledge of the store's internal file format in calling code.

Parameters:

• path (Union[str, PathLike]): Path to the store directory.

Returns:

• bool: True if a store exists at the path, False otherwise.

Example:

from gtars.refget import RefgetStore
RefgetStore.store_exists("/data/hg38_store")  # True
RefgetStore.store_exists("/tmp/empty")  # False

on_disk

@classmethod
def on_disk(cache_path: Union[str, PathLike]) -> RefgetStore

Create or load a disk-backed RefgetStore.

If the directory contains an existing store (rgstore.json), loads it. Otherwise creates a new store with Encoded mode.

Parameters:

• cache_path (Union[str, PathLike]): Directory path for the store. Created if it doesn't exist.

Returns:

• RefgetStore: RefgetStore (new or loaded from disk).

Example:

store = RefgetStore.on_disk("/data/my_store")
store.add_sequence_collection_from_fasta("genome.fa")
# Store is automatically persisted to disk

open_local

@classmethod
def open_local(path: Union[str, PathLike]) -> RefgetStore

Open a local RefgetStore from a directory.

Loads only lightweight metadata and stubs. Collections and sequences remain as stubs until explicitly accessed with get_collection()/get_sequence().

Requires rgstore.json. Reads sequences.rgsi and collections.rgci (or collections/*.rgsi) if present.

Parameters:

• path (Union[str, PathLike]): Local directory containing the refget store.

Returns:

• RefgetStore: RefgetStore with metadata loaded, sequences lazy-loaded.

Raises:

• IOError: If the store directory or index files cannot be read.

Example:

store = RefgetStore.open_local("/data/hg38_store")
seq = store.get_substring("chr1_digest", 0, 1000)

open_remote

@classmethod
def open_remote(cache_path: Union[str, PathLike], remote_url: str) -> RefgetStore

Open a remote RefgetStore with local caching.

Loads only lightweight metadata and stubs from the remote URL: the store manifest (rgstore.json) and collection index (collections.rgci). The sequence index (sequences.rgsi) is fetched on first sequence access, and sequence data (.seq files) on demand, cached locally.

By default, persistence is enabled (sequences are cached to disk). Call disable_persistence() after loading to keep only in memory.

Parameters:

• cache_path (Union[str, PathLike]): Local directory to cache downloaded metadata and sequences. Created if it doesn't exist.
• remote_url (str): Base URL of the remote refget store (e.g., "https://example.com/hg38" or "s3://bucket/hg38").

Returns:

• RefgetStore: RefgetStore with metadata loaded, sequences fetched on-demand.

Raises:

• IOError: If remote metadata cannot be fetched or cache cannot be written.

Example:

store = RefgetStore.open_remote(
    "/data/cache/hg38",
    "https://refget-server.com/hg38"
)
# First access fetches from remote and caches
seq = store.get_substring("chr1_digest", 0, 1000)
# Second access uses cache
seq2 = store.get_substring("chr1_digest", 1000, 2000)

Methods

set_encoding_mode

def set_encoding_mode(mode: StorageMode) -> None

Change the storage mode, re-encoding/decoding existing sequences as needed.

When switching from Raw to Encoded, all Full sequences in memory are encoded (2-bit packed). When switching from Encoded to Raw, all Full sequences in memory are decoded back to raw bytes.

Parameters:

• mode (StorageMode): The storage mode to switch to (StorageMode.Raw or StorageMode.Encoded).

Raises:

• ValueError: If a sequence cannot be encoded with its stored alphabet. The store's alphabet metadata is wrong and the store must be re-imported. The store is left unchanged.
• ValueError: If mode is StorageMode.Zstd, which cannot be set from Python.

Example:

store = RefgetStore.in_memory()
store.set_encoding_mode(StorageMode.Raw)

enable_encoding

def enable_encoding() -> None

Enable 2-bit encoding for space efficiency.

Re-encodes any existing Raw sequences in memory.

Raises:

• ValueError: If a sequence cannot be encoded with its stored alphabet (see set_encoding_mode).

Example:

store = RefgetStore.in_memory()
store.disable_encoding()  # Switch to Raw
store.enable_encoding()   # Back to Encoded

disable_encoding

def disable_encoding() -> None

Disable encoding, use raw byte storage.

Decodes any existing Encoded sequences in memory.

Example:

store = RefgetStore.in_memory()
store.disable_encoding()  # Switch to Raw mode

set_quiet

def set_quiet(quiet: bool) -> None

Set whether to suppress progress output.

When quiet is True, operations like add_sequence_collection_from_fasta will not print progress messages.

Parameters:

• quiet (bool): Whether to suppress progress output.

Example:

store = RefgetStore.in_memory()
store.set_quiet(True)
store.add_sequence_collection_from_fasta("genome.fa")  # No output

enable_persistence

def enable_persistence(path: Union[str, PathLike]) -> None

Enable disk persistence for this store.

Sets up the store to write sequences to disk. Any in-memory Full sequences are flushed to disk and converted to Stubs.

Parameters:

• path (Union[str, PathLike]): Directory for storing sequences and metadata.

Raises:

• IOError: If the directory cannot be created or written to.

Example:

store = RefgetStore.in_memory()
store.add_sequence_collection_from_fasta("genome.fa")
store.enable_persistence("/data/store")  # Flush to disk

disable_persistence

def disable_persistence() -> None

Disable disk persistence for this store.

New sequences will be kept in memory only. Existing Stub sequences can still be loaded from disk if local_path is set.

Example:

store = RefgetStore.open_remote("/cache", "https://example.com")
store.disable_persistence()  # Stop caching new sequences

add_sequence_collection_from_fasta

def add_sequence_collection_from_fasta(file_path: Union[str, PathLike], force: bool = False, namespaces: Optional[List[str]] = None, collection_alias: Optional[str] = None) -> tuple[SequenceCollectionMetadata, bool]

Add a sequence collection from a FASTA file.

Reads a FASTA file, digests the sequences, creates a SequenceCollection, and adds it to the store along with all its sequences.

Parameters:

• file_path (Union[str, PathLike]): Path to the FASTA file to import.
• force (bool): If True, overwrite existing collections/sequences. If False (default), skip duplicates. (default: False)
• namespaces (Optional[List[str]]): Optional list of namespace prefixes to extract aliases from FASTA headers. For example, ["ncbi", "refseq"] will scan headers for tokens like ncbi:NC_000001.11 and register them as aliases. (default: None)
• collection_alias (Optional[str]): Register the imported collection under this collection alias, as "NAMESPACE:ALIAS" (e.g. "ucsc:hg38"). This names the collection as a whole and is distinct from namespaces. Errors if the alias already names a different collection, unless force is set. (default: None)

Returns:

• tuple[SequenceCollectionMetadata, bool]: A tuple containing:
  • SequenceCollectionMetadata: Metadata for the collection.
  • bool: True if the collection was newly added, False if it already existed.

Raises:

• IOError: If the file cannot be read or processed.

Example:

store = RefgetStore.in_memory()
metadata, was_new = store.add_sequence_collection_from_fasta("genome.fa")
print(f"{'Added' if was_new else 'Skipped'}: {metadata.digest}")

# Extract aliases from FASTA headers
metadata, was_new = store.add_sequence_collection_from_fasta(
    "genome.fa", namespaces=["ncbi", "refseq"]
)

add_sequence_collections_from_fastas

def add_sequence_collections_from_fastas(fastas: Union[str, PathLike, List[Union[str, PathLike]]], file_list: Optional[Union[str, PathLike]] = None, jobs: int = 0, force: bool = False, namespaces: Optional[List[str]] = None) -> ImportReport

Import multiple FASTA files into the store.

fastas accepts a single path, a glob pattern (e.g. "fasta/*.fa.gz"), a directory of FASTAs, or a list mixing any of these. file_list points at a file-of-filenames (one path per line; blank lines and #-comments ignored; may itself contain globs/directories). Inputs are expanded by gtars into a de-duplicated, deterministic order (explicit paths first, then file_list lines; directory/glob matches sorted lexicographically). jobs is the number of files imported concurrently (0 = auto, 1 = serial); it does not affect ordering or the resulting store.

Parameters:

• fastas (Union[str, PathLike, List[Union[str, PathLike]]]): A path, glob, directory, or list of any of these.
• file_list (Optional[Union[str, PathLike]]): Optional path to a file-of-filenames. (default: None)
• jobs (int): Number of files imported concurrently (0 = auto, 1 = serial). (default: 0)
• force (bool): If True, overwrite existing collections/sequences. (default: False)
• namespaces (Optional[List[str]]): Optional namespace prefixes to extract aliases from headers. (default: None)

Returns:

• An (ImportReport): class:ImportReport. report.collections is a list of
• ImportReport: (SequenceCollectionMetadata, bool) tuples in expanded-input
• ImportReport: order (the bool is True if newly added);
• ImportReport: report.n_sequences_written, report.n_sequences_deduped, and
• ImportReport: report.n_collections_new are per-run ingest counters. Use these
• ImportReport: — not store.stats() — to report what an import actually added.

Raises:

• ValueError: If the inputs cannot be expanded (e.g. glob matches nothing).
• IOError: If a file cannot be read or processed.

Example:

store = RefgetStore.in_memory()
report = store.add_sequence_collections_from_fastas("data/*.fa.gz", jobs=4)
print(report.n_collections_new, report.n_sequences_written)

add_sequence_collection

def add_sequence_collection(collection: SequenceCollection, force: bool = False) -> None

Add a pre-built SequenceCollection to the store.

Adds a SequenceCollection (created via digest_fasta() or programmatically) directly to the store without reading from a FASTA file.

Parameters:

• collection (SequenceCollection): A SequenceCollection to add.
• force (bool): If True, overwrite existing collections/sequences. If False (default), skip duplicates. (default: False)

Raises:

• IOError: If the collection cannot be stored.

Example:

from gtars.refget import RefgetStore, digest_fasta
store = RefgetStore.in_memory()
collection = digest_fasta("genome.fa")
store.add_sequence_collection(collection)

add_sequence

def add_sequence(sequence: SequenceRecord, force: bool = False) -> None

Add a sequence to the store without collection association.

The sequence can be created using digest_sequence() and later retrieved by its digest via get_sequence().

Parameters:

• sequence (SequenceRecord): A SequenceRecord created by digest_sequence().
• force (bool): If True, overwrite existing. If False (default), skip duplicates. (default: False)

Raises:

• IOError: If the sequence cannot be stored.

Example:

from gtars.refget import RefgetStore, digest_sequence
store = RefgetStore.in_memory()
seq = digest_sequence(b"ACGTACGT")
store.add_sequence(seq)
retrieved = store.get_sequence(seq.metadata.sha512t24u)

list_collections

def list_collections(page: int = 0, page_size: int = 100, filters: Optional[Dict[str, str]] = None) -> Dict[str, Any]

List collections with pagination and optional attribute filtering.

Parameters:

• page (int): 0-indexed page number. (default: 0)
• page_size (int): Number of results per page. (default: 100)
• filters (Optional[Dict[str, str]]): Optional attribute filters (AND logic). Keys are attribute names (names, lengths, sequences, name_length_pairs, sorted_name_length_pairs, sorted_sequences), values are digests. (default: None)

Returns:

• Dict[str, Any]: Dict with "results" (list of SequenceCollectionMetadata) and
• Dict[str, Any]: "pagination" (dict with page, page_size, total).

Example:

# Get first page of all collections
result = store.list_collections()
for meta in result["results"]:
    print(f"{meta.digest}: {meta.n_sequences} sequences")
print(f"Total: {result['pagination']['total']}")

# Filter by names digest
result = store.list_collections(filters={"names": "abc123"})

remove_collection

def remove_collection(digest: str, remove_orphan_sequences: bool = False) -> bool

Remove a collection from the store.

Parameters:

• digest (str): The collection's SHA-512/24u digest string.
• remove_orphan_sequences (bool): If True, also remove sequences no longer referenced by any remaining collection. Default: False. (default: False)

Returns:

• bool: True if the collection was found and removed, False if not found.

get_collection_metadata

def get_collection_metadata(collection_digest: str) -> Optional[SequenceCollectionMetadata]

Get metadata for a collection by digest.

Returns lightweight metadata without loading the full collection. Use this for quick lookups of collection information.

Parameters:

• collection_digest (str): The collection's SHA-512/24u digest.

Returns:

• Optional[SequenceCollectionMetadata]: Collection metadata if found, None otherwise.

Example:

meta = store.get_collection_metadata("uC_UorBNf3YUu1YIDainBhI94CedlNeH")
if meta:
    print(f"Collection has {meta.n_sequences} sequences")

get_collection

def get_collection(collection_digest: str) -> SequenceCollection

Get a collection by digest with all sequences loaded.

Loads the collection and all its sequence data into memory. Use this when you need full access to sequence content.

Parameters:

• collection_digest (str): The collection's SHA-512/24u digest.

Returns:

• SequenceCollection: The collection with all sequence data loaded.

Raises:

• IOError: If the collection cannot be loaded.

Example:

collection = store.get_collection("uC_UorBNf3YUu1YIDainBhI94CedlNeH")
for seq in collection.sequences:
    print(f"{seq.metadata.name}: {seq.decode()[:20]}...")

iter_collections

def iter_collections() -> List[SequenceCollection]

Iterate over all collections with their sequences loaded.

This loads all collection data upfront and returns a list of SequenceCollection objects with full sequence data.

For browsing without loading data, use list_collections() instead.

Returns:

• List[SequenceCollection]: List of all collections with loaded sequences.

Example:

for coll in store.iter_collections():
    print(f"{coll.digest}: {len(coll.sequences)} sequences")

is_collection_loaded

def is_collection_loaded(collection_digest: str) -> bool

Check if a collection is fully loaded.

Returns True if the collection's sequence list is loaded in memory, False if it's only metadata (stub).

Parameters:

• collection_digest (str): The collection's SHA-512/24u digest.

Returns:

• bool: True if loaded, False otherwise.

list_sequences

def list_sequences() -> List[SequenceMetadata]

List all sequence metadata in the store.

Returns metadata for all sequences without loading sequence data. Use this for browsing/inventory operations.

Returns:

• List[SequenceMetadata]: List of metadata for all sequences in the store.

Example:

for meta in store.list_sequences():
    print(f"{meta.name}: {meta.length} bp")

get_sequence_metadata

def get_sequence_metadata(seq_digest: str) -> Optional[SequenceMetadata]

Get metadata for a sequence by digest (no data loaded).

Use this for lightweight lookups when you don't need the actual sequence. Automatically strips "SQ." prefix from digest if present.

Parameters:

• seq_digest (str): The sequence's SHA-512/24u digest, optionally with "SQ." prefix.

Returns:

• Optional[SequenceMetadata]: Sequence metadata if found, None otherwise.

get_sequence

def get_sequence(digest: str) -> SequenceRecord

Retrieve a sequence record by its digest (SHA-512/24u or MD5).

Loads the sequence data if not already in memory. Supports lookup by either SHA-512/24u (preferred) or MD5 digest. Automatically strips "SQ." prefix if present (case-insensitive).

Parameters:

• digest (str): Sequence digest (SHA-512/24u base64url or MD5 hex string), optionally with "SQ." prefix.

Returns:

• SequenceRecord: The sequence record with data.

Raises:

• KeyError: If the sequence is not found.

Example:

record = store.get_sequence("aKF498dAxcJAqme6QYQ7EZ07-fiw8Kw2")
print(f"Found: {record.metadata.name}")
# Also works with SQ. prefix
record = store.get_sequence("SQ.aKF498dAxcJAqme6QYQ7EZ07-fiw8Kw2")

get_sequence_by_name

def get_sequence_by_name(collection_digest: str, sequence_name: str) -> SequenceRecord

Retrieve a sequence by collection digest and sequence name.

Looks up a sequence within a specific collection using its name (e.g., "chr1", "chrM"). Loads the sequence data if needed. Automatically strips "SQ." prefix from collection digest if present.

Parameters:

• collection_digest (str): The collection's SHA-512/24u digest, optionally with "SQ." prefix.
• sequence_name (str): Name of the sequence within that collection.

Returns:

• SequenceRecord: The sequence record with data.

Raises:

• KeyError: If the sequence is not found.

Example:

record = store.get_sequence_by_name(
    "uC_UorBNf3YUu1YIDainBhI94CedlNeH",
    "chr1"
)
print(f"Sequence: {record.decode()[:50]}...")

iter_sequences

def iter_sequences() -> List[SequenceRecord]

Iterate over all sequences with their data loaded.

This ensures all sequence data is loaded and returns a list of SequenceRecord objects with full sequence data.

For browsing without loading data, use list_sequences() instead.

Returns:

• List[SequenceRecord]: List of all sequences with loaded data.

Example:

for seq in store.iter_sequences():
    print(f"{seq.metadata.name}: {seq.decode()[:20]}...")

get_substring

def get_substring(seq_digest: str, start: int, end: int) -> str

Extract a substring from a sequence (flow 1: lean partial read).

Retrieves a specific region using 0-based, half-open coordinates [start, end), reading only the bytes that cover the region. Source resolution is resident -> local .seq -> remote HTTP byte-range, so a remote-backed store is served directly with no manual preload and without downloading or caching the whole sequence. For repeat-heavy access call load_sequence() (flow 3) first. Automatically strips "SQ." prefix from digest if present.

Parameters:

• seq_digest (str): Sequence digest (SHA-512/24u), optionally with "SQ." prefix.
• start (int): Start position (0-based, inclusive).
• end (int): End position (0-based, exclusive).

Returns:

• str: The substring sequence.

Raises:

• KeyError: If the sequence is not found.

Example:

# Get first 1000 bases of chr1
seq = store.get_substring("chr1_digest", 0, 1000)
print(f"First 50bp: {seq[:50]}")

stream_sequence

def stream_sequence(seq_digest: str, start: Optional[int] = None, end: Optional[int] = None, chunk_size: int = 65536) -> SequenceStream

Stream a (sub)sequence as decoded bases without loading it (flow 2).

Returns a SequenceStream that yields str chunks lazily, reading from resident bytes, a local .seq, or a remote HTTP byte-range as needed. Peak memory is O(1) in the region length. Omit start/end to stream the whole sequence.

Parameters:

• seq_digest (str): Sequence digest (SHA-512/24u), optionally with "SQ." prefix.
• start (Optional[int]): Start position (0-based, inclusive). Defaults to 0. (default: None)
• end (Optional[int]): End position (0-based, exclusive). Defaults to the sequence length. (default: None)
• chunk_size (int): Bytes read per chunk (default 64 KiB). (default: 65536)

Returns:

• SequenceStream: A SequenceStream iterator of decoded base chunks.

stats

def stats() -> dict

Returns statistics about the store.

Returns:

• dict: dict with keys:
  • 'n_sequences': Total number of sequences (Stub + Full)
  • 'n_sequences_in_memory': Number of sequences whose bytes are currently held in RAM (Full)
  • 'n_collections': Total number of collections (Stub + Full)
  • 'n_collections_in_memory': Number of collections whose sequence list is currently held in RAM (Full)
  • 'storage_mode': Storage mode ('Raw' or 'Encoded')

Note

These are a snapshot of CURRENT RAM residency, not a record of what an import run did. n_sequences_in_memory is structurally always 0 for a disk-backed store: importing writes bytes to disk and keeps only a stub in memory. n_collections_in_memory only reflects collections fully loaded in memory; it resets on process start and also counts collections merely touched by a read, so it is not a count of collections ingested. For remote stores, collections are loaded on-demand when accessed. For per-run ingest counts, use the ImportReport returned by add_sequence_collections_from_fastas.

Example:

stats = store.stats()
print(f"Store has {stats['n_sequences']} sequences")
print(f"Collections: {stats['n_collections']} total, {stats['n_collections_in_memory']} in memory")

write

def write() -> None

Write the store using its configured paths.

Convenience method for disk-backed stores. Uses the store's own local_path and seqdata_path_template.

Raises:

• IOError: If the store cannot be written.

write_store_to_dir

def write_store_to_dir(root_path: Union[str, PathLike], seqdata_path_template: Optional[str] = None) -> None

Write the store to a directory on disk.

Persists the store with all sequences and metadata to disk using the RefgetStore directory format.

Parameters:

• root_path (Union[str, PathLike]): Directory path to write the store to.
• seqdata_path_template (Optional[str]): Optional path template for sequence files (e.g., "sequences/%s2/%s.seq" where %s2 = first 2 chars of digest, %s = full digest). Uses default if not specified. (default: None)

Example:

store.write_store_to_dir("/data/my_store")
store.write_store_to_dir("/data/my_store", "sequences/%s2/%s.seq")

get_collection_level1

def get_collection_level1(digest: str) -> dict

Get level 1 representation (attribute digests) for a collection.

Parameters:

• digest (str): Collection digest.

Returns:

• dict: dict with spec-compliant field names (names, lengths, sequences,
• dict: plus optional name_length_pairs, sorted_name_length_pairs, sorted_sequences).

get_collection_level2

def get_collection_level2(digest: str) -> dict

Get level 2 representation (full arrays, spec format) for a collection.

Parameters:

• digest (str): Collection digest.

Returns:

• dict: dict with names (list[str]), lengths (list[int]), sequences (list[str]).

match_sequence_names

def match_sequence_names(digest_a: str, digest_b: str) -> dict[str, Any]

Match collection-local names by sequence content.

Returns:

• dict[str, Any]: A mapping with collection_a, collection_b, matches, a_only, and
• dict[str, Any]: b_only. Each row has digest, length, names_a, and names_b; repeated
• dict[str, Any]: content may have many names.

Raises:

• KeyError: If either collection digest is unknown.

compare

def compare(digest_a: str, digest_b: str) -> dict

Compare two collections by digest.

Parameters:

• digest_a (str): First collection digest.
• digest_b (str): Second collection digest.

Returns:

• dict: dict with keys: digests, attributes, array_elements.

find_collections_by_attribute

def find_collections_by_attribute(attr_name: str, attr_digest: str) -> List[str]

Find collections by attribute digest.

Parameters:

• attr_name (str): Attribute name (names, lengths, sequences, name_length_pairs, sorted_name_length_pairs, sorted_sequences).
• attr_digest (str): The digest to search for.

Returns:

• List[str]: List of collection digests that have the matching attribute.

get_attribute

def get_attribute(attr_name: str, attr_digest: str) -> Optional[list]

Get attribute array by digest.

Parameters:

• attr_name (str): Attribute name (names, lengths, or sequences).
• attr_digest (str): The digest to search for.

Returns:

• Optional[list]: The attribute array, or None if not found.

enable_ancillary_digests

def enable_ancillary_digests() -> None

Enable computation of ancillary digests.

disable_ancillary_digests

def disable_ancillary_digests() -> None

Disable computation of ancillary digests.

has_ancillary_digests

def has_ancillary_digests() -> bool

Returns whether ancillary digests are enabled.

has_attribute_index

def has_attribute_index() -> bool

Returns whether the on-disk attribute index is enabled.

enable_attribute_index

def enable_attribute_index() -> None

Enable indexed attribute lookup (not yet implemented).

disable_attribute_index

def disable_attribute_index() -> None

Disable indexed attribute lookup, using brute-force scan instead.

export_fasta_from_regions

def export_fasta_from_regions(collection_digest: str, bed_file_path: Union[str, PathLike], output_file_path: Union[str, PathLike]) -> None

Export sequences from BED file regions to a FASTA file.

Reads a BED file defining genomic regions and exports the sequences for those regions to a FASTA file.

Parameters:

• collection_digest (str): The collection's SHA-512/24u digest.
• bed_file_path (Union[str, PathLike]): Path to BED file defining regions.
• output_file_path (Union[str, PathLike]): Path to write the output FASTA file.

Raises:

• IOError: If files cannot be read/written or sequences not found.

Example:

store.export_fasta_from_regions(
    "uC_UorBNf3YUu1YIDainBhI94CedlNeH",
    "regions.bed",
    "output.fa"
)

substrings_from_regions

def substrings_from_regions(collection_digest: str, bed_file_path: Union[str, PathLike]) -> List[RetrievedSequence]

Get substrings for BED file regions as a list.

Reads a BED file and returns a list of sequences for each region.

Parameters:

• collection_digest (str): The collection's SHA-512/24u digest.
• bed_file_path (Union[str, PathLike]): Path to BED file defining regions.

Returns:

• List[RetrievedSequence]: List of retrieved sequence segments.

Raises:

• IOError: If files cannot be read or sequences not found.

Example:

sequences = store.substrings_from_regions(
    "uC_UorBNf3YUu1YIDainBhI94CedlNeH",
    "regions.bed"
)
for seq in sequences:
    print(f"{seq.chrom_name}:{seq.start}-{seq.end}")

export_fasta

def export_fasta(collection_digest: str, output_path: Union[str, PathLike], sequence_names: Optional[List[str]] = None, line_width: Optional[int] = None) -> None

Export sequences from a collection to a FASTA file.

Parameters:

• collection_digest (str): Collection to export from.
• output_path (Union[str, PathLike]): Path to write FASTA file.
• sequence_names (Optional[List[str]]): Optional list of sequence names to export. If None, exports all sequences in the collection. (default: None)
• line_width (Optional[int]): Optional line width for wrapping sequences. If None, uses default of 80. (default: None)

export_fasta_by_digests

def export_fasta_by_digests(seq_digests: List[str], output_path: Union[str, PathLike], line_width: Optional[int] = None) -> None

Export sequences by their digests to a FASTA file.

Parameters:

• seq_digests (List[str]): List of sequence digests to export.
• output_path (Union[str, PathLike]): Path to write FASTA file.
• line_width (Optional[int]): Optional line width for wrapping sequences. If None, uses default of 80. (default: None)

add_sequence_alias

def add_sequence_alias(namespace: str, alias: str, digest: str) -> None

Add a sequence alias: namespace/alias maps to sequence digest.

get_sequence_metadata_by_alias

def get_sequence_metadata_by_alias(namespace: str, alias: str) -> Optional[SequenceMetadata]

Resolve a sequence alias to sequence metadata (no data loading).

get_sequence_by_alias

def get_sequence_by_alias(namespace: str, alias: str) -> Optional[SequenceRecord]

Resolve a sequence alias and return the loaded sequence record.

Returns None if the alias is not found.

get_aliases_for_sequence

def get_aliases_for_sequence(digest: str) -> list[tuple[str, str]]

Reverse lookup: find all (namespace, alias) pairs pointing to this sequence digest.

list_sequence_alias_namespaces

def list_sequence_alias_namespaces() -> list[str]

List all sequence alias namespaces.

list_sequence_aliases

def list_sequence_aliases(namespace: str) -> Optional[list[str]]

List all aliases in a sequence alias namespace.

remove_sequence_alias

def remove_sequence_alias(namespace: str, alias: str) -> bool

Remove a single sequence alias. Returns True if it existed.

load_sequence_aliases

def load_sequence_aliases(namespace: str, path: str) -> int

Load sequence aliases from a TSV file (alias\tdigest per line).

add_collection_alias

def add_collection_alias(namespace: str, alias: str, digest: str) -> None

Add a collection alias: namespace/alias maps to collection digest.

get_collection_metadata_by_alias

def get_collection_metadata_by_alias(namespace: str, alias: str) -> Optional[SequenceCollectionMetadata]

Resolve a collection alias to collection metadata (no data loading).

get_collection_by_alias

def get_collection_by_alias(namespace: str, alias: str) -> Optional[SequenceCollection]

Resolve a collection alias and return the loaded collection.

Returns None if the alias is not found.

get_aliases_for_collection

def get_aliases_for_collection(digest: str) -> list[tuple[str, str]]

Reverse lookup: find all (namespace, alias) pairs pointing to this collection digest.

list_collection_alias_namespaces

def list_collection_alias_namespaces() -> list[str]

List all collection alias namespaces.

list_collection_aliases

def list_collection_aliases(namespace: str) -> Optional[list[str]]

List all aliases in a collection alias namespace.

remove_collection_alias

def remove_collection_alias(namespace: str, alias: str) -> bool

Remove a single collection alias. Returns True if it existed.

load_collection_aliases

def load_collection_aliases(namespace: str, path: str) -> int

Load collection aliases from a TSV file (alias\tdigest per line).

set_fhr_metadata

def set_fhr_metadata(collection_digest: str, metadata: FhrMetadata) -> None

Set FHR metadata for a collection.

get_fhr_metadata

def get_fhr_metadata(collection_digest: str) -> Optional[FhrMetadata]

Get FHR metadata for a collection. Returns None if missing.

remove_fhr_metadata

def remove_fhr_metadata(collection_digest: str) -> bool

Remove FHR metadata for a collection.

list_fhr_metadata

def list_fhr_metadata() -> list[str]

List all collection digests that have FHR metadata.

load_fhr_metadata

def load_fhr_metadata(collection_digest: str, path: str) -> None

Load FHR metadata from a JSON file and attach it to a collection.

verify

def verify(digests: Optional[List[str]] = None, collection: Optional[str] = None, jobs: int = 0, check_md5: bool = True) -> VerifyReport

Verify sequences against their stored digests.

Streams each sequence's decoded bytes through the normal decode path and recomputes its sha512t24u (and md5, when one is stored) to check that the stored bytes still hash to the digest they are stored under.

Parameters:

• digests (Optional[List[str]]): Only verify these sequence digests (sha512t24u or md5, "SQ." prefix optional). Mutually exclusive with collection. Default (None) with collection also None: verify every sequence in the store. (default: None)
• collection (Optional[str]): Only verify the sequences belonging to this collection: a digest, or "NAMESPACE:ALIAS". Mutually exclusive with digests. (default: None)
• jobs (int): Worker threads. 0 (default) = auto, 1 = serial. (default: 0)
• check_md5 (bool): Also recompute and compare md5 when the store has one recorded (default True). (default: True)

Returns:

• VerifyReport: Summary counts plus one VerifyFailure per bad
• VerifyReport: sequence.

Raises:

• ValueError: If both digests and collection are given.
• KeyError: If a requested digest, alias, or collection is not found.

Example:

report = store.verify()
if not report.ok:
    for f in report.failures:
        print(f.digest, f.kind)

into_readonly

def into_readonly() -> ReadonlyRefgetStore

Convert to a ReadonlyRefgetStore for concurrent read access.

Consumes this store (replacing it with an empty in-memory store) and returns a ReadonlyRefgetStore whose read methods all use &self (no mutable borrow), making it suitable for Arc<ReadonlyRefgetStore> in servers.

Call load_all_collections() or load_collection() before converting, since ReadonlyRefgetStore cannot lazy-load.

Returns:

• ReadonlyRefgetStore: An immutable store suitable for concurrent access.

Example:

store = RefgetStore.open_remote("/cache", "https://example.com")
store.load_all_collections()
readonly = store.into_readonly()
coll = readonly.get_collection("abc123")

compute_vrs_ids_parallel

def compute_vrs_ids_parallel(collection_digest: str, vcf_path: str, output_path: str, threads: Optional[int] = None) -> int

Compute GA4GH VRS Allele identifiers in parallel and stream them to a TSV.

Makes the collection's sequences resident (2-bit encoded, decoded on the fly), then runs worker threads with the GIL released. BGZF input uses the block-parallel reader; plain or gzip input uses a single reader. Rows are written in VCF order, identical to compute_vrs_ids.

Parameters:

• collection_digest (str): Digest of the sequence collection (genome assembly).
• vcf_path (str): Path to a VCF file (plain, gzip, or BGZF).
• output_path (str): Destination TSV. A header row chrom pos ref alt vrs_id is written first.
• threads (Optional[int]): Worker threads. Defaults to available cores. (default: None)

Returns:

• int: Number of VRS results written.

__len__

def __len__() -> int

__iter__

def __iter__() -> Iterator[SequenceMetadata]

__str__

def __str__() -> str

__repr__

def __repr__() -> str

ReadonlyRefgetStore

class ReadonlyRefgetStore

An immutable RefgetStore for concurrent read access.

All read methods use immutable references, making this suitable for concurrent access patterns (e.g., shared across threads in a server).

This type has NO write methods and NO constructors -- it is only obtainable via RefgetStore.into_readonly().

Read methods that require preloaded data (e.g., get_collection()) will error if the data was not loaded before conversion.

Attributes:

• cache_path (Optional[str]): Local directory path where the store is located or cached. None for in-memory stores.
• remote_url (Optional[str]): Remote URL of the store if loaded remotely, None otherwise.
• storage_mode (StorageMode): Current storage mode (Raw, Encoded, or Zstd).

Example:

store = RefgetStore.open_remote("/cache", "https://example.com")
store.load_all_collections()
readonly = store.into_readonly()
coll = readonly.get_collection("abc123")

Attributes

• cache_path (Optional[str])
• remote_url (Optional[str])

Properties

storage_mode

@property
def storage_mode -> StorageMode

Current storage mode (Raw, Encoded, or Zstd).

Methods

list_collections

def list_collections(page: int = 0, page_size: int = 100, filters: Optional[Dict[str, str]] = None) -> Dict[str, Any]

List collections with pagination and optional attribute filtering.

get_collection_metadata

def get_collection_metadata(collection_digest: str) -> Optional[SequenceCollectionMetadata]

Get metadata for a collection by digest.

get_collection

def get_collection(collection_digest: str) -> SequenceCollection

Get a collection by digest with all sequences loaded.

Requires that the collection was preloaded before conversion.

Parameters:

• collection_digest (str): The collection's SHA-512/24u digest.

Returns:

• SequenceCollection: The collection with all sequence data loaded.

Raises:

• IOError: If the collection was not preloaded.

is_collection_loaded

def is_collection_loaded(collection_digest: str) -> bool

Check if a collection is fully loaded.

get_collection_level1

def get_collection_level1(digest: str) -> dict

Get level 1 representation (attribute digests) for a collection.

get_collection_level2

def get_collection_level2(digest: str) -> dict

Get level 2 representation (full arrays) for a collection.

match_sequence_names

def match_sequence_names(digest_a: str, digest_b: str) -> dict[str, Any]

Match collection-local names by sequence content.

Each result row has digest, length, names_a, and names_b.

Raises:

• KeyError: If either collection digest is unknown.

compare

def compare(digest_a: str, digest_b: str) -> dict

Compare two collections by digest.

find_collections_by_attribute

def find_collections_by_attribute(attr_name: str, attr_digest: str) -> List[str]

Find collections by attribute digest.

get_attribute

def get_attribute(attr_name: str, attr_digest: str) -> Optional[list]

Get attribute array by digest.

has_ancillary_digests

def has_ancillary_digests() -> bool

Returns whether ancillary digests are enabled.

has_attribute_index

def has_attribute_index() -> bool

Returns whether the on-disk attribute index is enabled.

list_sequences

def list_sequences() -> List[SequenceMetadata]

List all sequence metadata in the store.

get_sequence_metadata

def get_sequence_metadata(seq_digest: str) -> Optional[SequenceMetadata]

Get metadata for a sequence by digest.

get_sequence

def get_sequence(digest: str) -> SequenceRecord

Retrieve a sequence record by its digest.

Parameters:

• digest (str): Sequence digest (SHA-512/24u or MD5).

Returns:

• SequenceRecord: The sequence record with data.

Raises:

• KeyError: If the sequence is not found.

get_sequence_by_name

def get_sequence_by_name(collection_digest: str, sequence_name: str) -> SequenceRecord

Retrieve a sequence by collection digest and sequence name.

Parameters:

• collection_digest (str): The collection's SHA-512/24u digest.
• sequence_name (str): Name of the sequence within that collection.

Returns:

• SequenceRecord: The sequence record with data.

Raises:

• KeyError: If the sequence is not found.

get_substring

def get_substring(seq_digest: str, start: int, end: int) -> str

Extract a substring from a sequence (flow 1: lean partial read).

Source resolution is resident -> local .seq -> remote HTTP byte-range; remote stores are served with no preload and without downloading whole sequences.

Parameters:

• seq_digest (str): Sequence digest (SHA-512/24u).
• start (int): Start position (0-based, inclusive).
• end (int): End position (0-based, exclusive).

Returns:

• str: The substring sequence.

Raises:

• KeyError: If the sequence is not found.

stream_sequence

def stream_sequence(seq_digest: str, start: Optional[int] = None, end: Optional[int] = None, chunk_size: int = 65536) -> SequenceStream

Stream a (sub)sequence as decoded bases without loading it (flow 2).

Returns a SequenceStream yielding str chunks lazily from resident bytes, a local .seq, or a remote HTTP byte-range. Peak memory is O(1) in the region length. Omit start/end to stream the whole sequence.

stats

def stats() -> dict

Returns statistics about the store.

get_sequence_metadata_by_alias

def get_sequence_metadata_by_alias(namespace: str, alias: str) -> Optional[SequenceMetadata]

Resolve a sequence alias to sequence metadata.

get_sequence_by_alias

def get_sequence_by_alias(namespace: str, alias: str) -> Optional[SequenceRecord]

Resolve a sequence alias and return the loaded sequence record.

get_aliases_for_sequence

def get_aliases_for_sequence(digest: str) -> list[tuple[str, str]]

Reverse lookup: find all (namespace, alias) pairs for this sequence.

list_sequence_alias_namespaces

def list_sequence_alias_namespaces() -> list[str]

List all sequence alias namespaces.

list_sequence_aliases

def list_sequence_aliases(namespace: str) -> Optional[list[str]]

List all aliases in a sequence alias namespace.

get_collection_metadata_by_alias

def get_collection_metadata_by_alias(namespace: str, alias: str) -> Optional[SequenceCollectionMetadata]

Resolve a collection alias to collection metadata.

get_collection_by_alias

def get_collection_by_alias(namespace: str, alias: str) -> Optional[SequenceCollection]

Resolve a collection alias and return the loaded collection.

get_aliases_for_collection

def get_aliases_for_collection(digest: str) -> list[tuple[str, str]]

Reverse lookup: find all (namespace, alias) pairs for this collection.

list_collection_alias_namespaces

def list_collection_alias_namespaces() -> list[str]

List all collection alias namespaces.

list_collection_aliases

def list_collection_aliases(namespace: str) -> Optional[list[str]]

List all aliases in a collection alias namespace.

get_fhr_metadata

def get_fhr_metadata(collection_digest: str) -> Optional[FhrMetadata]

Get FHR metadata for a collection.

list_fhr_metadata

def list_fhr_metadata() -> list[str]

List all collection digests that have FHR metadata.

verify

def verify(digests: Optional[List[str]] = None, jobs: int = 0, check_md5: bool = True) -> VerifyReport

Verify sequences against their stored digests.

See RefgetStore.verify for what this checks. This requires no preloading: it works directly off the store's sequence index.

Parameters:

• digests (Optional[List[str]]): Only verify these sequence digests (sha512t24u or md5, "SQ." prefix optional). Default (None): verify every sequence in the store. (default: None)
• jobs (int): Worker threads. 0 (default) = auto, 1 = serial. (default: 0)
• check_md5 (bool): Also recompute and compare md5 when the store has one recorded (default True). (default: True)

Returns:

• VerifyReport: Summary counts plus one VerifyFailure per bad
• VerifyReport: sequence.

Raises:

• KeyError: If a requested digest is not found.

__len__

def __len__() -> int

__str__

def __str__() -> str

__repr__

def __repr__() -> str

sha512t24u_digest

def sha512t24u_digest(readable: Union[str, bytes]) -> str

Compute the GA4GH SHA-512/24u digest for a sequence.

This function computes the GA4GH refget standard digest (truncated SHA-512, base64url encoded) for a given sequence string or bytes.

Parameters:

• readable (Union[str, bytes]): Input sequence as str or bytes.

Returns:

• str: The SHA-512/24u digest (32 character base64url string).

Raises:

• TypeError: If input is not str or bytes.

Example:

from gtars.refget import sha512t24u_digest
digest = sha512t24u_digest("ACGT")
print(digest)
# Output: 'aKF498dAxcJAqme6QYQ7EZ07-fiw8Kw2'

md5_digest

def md5_digest(readable: Union[str, bytes]) -> str

Compute the MD5 digest for a sequence.

This function computes the MD5 hash for a given sequence string or bytes. MD5 is supported for backward compatibility with legacy systems.

Parameters:

• readable (Union[str, bytes]): Input sequence as str or bytes.

Returns:

• str: The MD5 digest (32 character hexadecimal string).

Raises:

• TypeError: If input is not str or bytes.

Example:

from gtars.refget import md5_digest
digest = md5_digest("ACGT")
print(digest)
# Output: 'f1f8f4bf413b16ad135722aa4591043e'

digest_fasta

def digest_fasta(fasta: Union[str, PathLike]) -> SequenceCollection

Digest all sequences in a FASTA file and compute collection-level digests.

This function reads a FASTA file and computes GA4GH-compliant digests for each sequence, as well as collection-level digests (Level 1 and Level 2) following the GA4GH refget specification.

Parameters:

• fasta (Union[str, PathLike]): Path to FASTA file (str or PathLike).

Returns:

• SequenceCollection: Collection containing all sequences with their metadata and computed digests.

Raises:

• IOError: If the FASTA file cannot be read or parsed.

Example:

from gtars.refget import digest_fasta
collection = digest_fasta("genome.fa")
print(f"Collection digest: {collection.digest}")
print(f"Number of sequences: {len(collection)}")

compute_fai

def compute_fai(fasta: Union[str, PathLike]) -> List[FaiRecord]

Compute FASTA index (FAI) metadata for all sequences in a FASTA file.

This function computes the FAI index metadata (offset, line_bases, line_bytes) for each sequence in a FASTA file, compatible with samtools faidx format. Only works with uncompressed FASTA files.

Parameters:

• fasta (Union[str, PathLike]): Path to FASTA file (str or PathLike). Must be uncompressed.

Returns:

• List[FaiRecord]: List of FAI records, one per sequence, containing name, length,
• List[FaiRecord]: and FAI metadata (offset, line_bases, line_bytes).

Raises:

• IOError: If the FASTA file cannot be read or is compressed.

Example:

from gtars.refget import compute_fai
fai_records = compute_fai("genome.fa")
for record in fai_records:
    print(f"{record.name}: {record.length} bp")

load_fasta

def load_fasta(fasta: Union[str, PathLike]) -> SequenceCollection

Load a FASTA file with sequence data into a SequenceCollection.

This function reads a FASTA file and loads all sequences with their data into memory. Unlike digest_fasta(), this includes the actual sequence data, not just metadata.

Parameters:

• fasta (Union[str, PathLike]): Path to FASTA file (str or PathLike).

Returns:

• SequenceCollection: Collection containing all sequences with their metadata and sequence data loaded.

Raises:

• IOError: If the FASTA file cannot be read or parsed.

Example:

from gtars.refget import load_fasta
collection = load_fasta("genome.fa")
first_seq = collection[0]
print(f"Sequence: {first_seq.decode()[:50]}...")

digest_sequence

def digest_sequence(data: bytes, name: Optional[str] = None, description: Optional[str] = None) -> SequenceRecord

Create a SequenceRecord from raw data, computing all metadata.

This is the sequence-level parallel to digest_fasta() for collections. It computes the GA4GH sha512t24u digest, MD5 digest, detects the alphabet, and returns a SequenceRecord with computed metadata and the original data.

The input data is automatically uppercased to ensure consistent digest computation (matching FASTA processing behavior).

Parameters:

• data (bytes): The raw sequence bytes (e.g., b"ACGTACGT").
• name (Optional[str]): Optional sequence name (e.g., "chr1"). Defaults to "" if not provided. (default: None)
• description (Optional[str]): Optional description text for the sequence. (default: None)

Returns:

• SequenceRecord: A SequenceRecord with computed metadata and the original data (uppercased).

Example:

from gtars.refget import digest_sequence
seq = digest_sequence(b"ACGTACGT")
print(seq.metadata.length)
# Output: 8

seq = digest_sequence(b"ACGT", name="chr1")
print(seq.metadata.name, seq.metadata.length)
# Output: chr1 4

# With description
seq2 = digest_sequence(b"ACGT", name="chr1", description="Chromosome 1")
print(seq2.metadata.description)
# Output: Chromosome 1