Skip to content
BEDbase

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

Section titled “[Unreleased]”

RefgetStore (gtars-refget)

Section titled “RefgetStore (gtars-refget)”
  • Fix residues that did not round-trip through StorageMode::Encoded:
    • D and H in the IUPAC alphabet decoded as H and V. Fixed in the decoder, so existing stores read correctly without re-importing.
    • U in the IUPAC alphabet was stored as T. U now has its own code.
    • U (selenocysteine) and B in proteins were stored as A. The protein alphabet now holds U, B, Z, O and J.
  • Choose the smallest alphabet that holds every character in a sequence, instead of assuming each alphabet contains the one before it
  • Encoding a character that the chosen alphabet cannot hold is now an error instead of silently storing it as another residue (encode_sequence, SequenceEncoder::finalize and set_encoding_mode now return Result)
  • The 3-bit alphabet stores U in place of X, so RNA fits in 3 bits. Sequences with X now use the protein alphabet.
  • Add verify to check that every stored sequence still hashes to its digest (Rust, CLI gtars refget verify, Python store.verify(), R verify_store())
  • Faster alphabet detection and encoding

Stores built with 0.11.0 or earlier that contain U or B in proteins, or U in nucleotide sequences, must be re-imported. Run gtars refget verify <store> to list affected sequences.

[0.11.0] -- 2026-09-22

Section titled “[0.11.0] -- 2026-09-22”

RefgetStore (gtars-refget)

Section titled “RefgetStore (gtars-refget)”
  • Add StorageMode::Zstd for compressed novel/unplaced sequence storage
  • Lazy-load sequence index on first read (list_sequences() now returns Result)
  • Add content-based sequence name matching
  • Add resident overlay for RAM-cached sequence reads
  • Detect gzip by magic bytes instead of file extension
  • Fix remote listing, export, and WASM build
  • Fix encoded sequence ingestion

Python bindings (gtars PyPI package)

Section titled “Python bindings (gtars PyPI package)”
  • Add RefgetStore docstrings visible in Python (fixes #272)
  • Polish refget Python docstrings

[0.10.0] -- 2026-09-05

Section titled “[0.10.0] -- 2026-09-05”
  • refget: writers now take an exclusive store lock and commit only their own changes; concurrent imports no longer lose collections, and a stale handle no longer resurrects removed rows
  • refget: all index, manifest, alias, and sequence files are written atomically (write-temp then rename)
  • refget: fix cleanup after a failed import deleting pre-existing sequences on a reopened store
  • refget: orphan-sequence GC is linear instead of quadratic; large removals finish in seconds instead of hours
  • refget: export_fasta and get_sequence_by_name use the collection's own names and descriptions #270
  • refget: add ImportReport with per-run counters (n_sequences_written, n_sequences_deduped, n_collections_new); add_sequence_collections_from_fastas now returns it
  • refget: add import-time collection aliases (--collection-alias ns:alias, FastaImportOptions::collection_alias, Python collection_alias=)
  • refget: add logical_sequence_bytes to the store manifest; rename total_disk_size() to logical_sequence_bytes()
  • refget: rename StoreStats fields n_sequences_loaded/n_collections_loaded to n_sequences_in_memory/n_collections_in_memory across Rust, Python, R, and Node
  • refget: encoded stores accept pre-packed sequences at insert time
  • cli: add gtars refget export (-c <digest|ns:alias>, -w 0 for unwrapped FASTA, .gz output)
  • overlaprs: remove unused MultiChromOverlapperError
  • wasm: fix encodedByteRange return type in typings

[0.9.1] -- 2026-06-17

Section titled “[0.9.1] -- 2026-06-17”

Point release of refget 0.9.1, python 0.9.2, R 0.9.2, node 0.7.1, wasm 0.9.1 #268.

  • refget: unify remote on-demand sequence retrieval into three flows: get_substring (byte-range read), stream_sequence (streaming reader), load_sequence (download and cache)
  • refget: bulk substring reads on a remote-only sequence download and cache the .seq once instead of one request per range
  • python, R, node: expose stream_sequence, load_sequence, and load_all_sequences; node no longer auto-downloads whole chromosomes
  • wasm: add RemoteRefgetStore JS layer (HTTP byte-range + OPFS caching) via @databio/gtars/remote

[0.9.0] -- 2026-06-15

Section titled “[0.9.0] -- 2026-06-15”
  • add gtars-vrs crate: GA4GH VRS allele identifiers from VCF/HGVS, with an HGVS parser, allele normalization, and transcript-anchored mapping
  • add Node.js bindings (gtars-node)
  • refget: on-disk sequence store overhaul; add binary transcript store and coordinate mapper
  • genomicdist: add binary FASTA (.fab) format with zero-copy mmap access; add stranded region-set operations
  • uniwig: add BAM QC tooling
  • overlaprs: overlap-engine rewrite
  • expanded WASM and Python bindings
  • correctness fixes across vrs, refget, bamqc, and the Python HGVS AST

[0.8.0] -- 2026-03-19

Section titled “[0.8.0] -- 2026-03-19”
  • genomicdist: port of genomic distributions #231, bug fixes #244
  • uniwig: add streaming implementation alongside the batch parallel one #236
  • refget: RefgetStore updates #237
  • add gtars-lola crate; IGD supports an in-memory database #242
  • expose more functionality via Python #241

[0.7.0] -- 2026-02-24

Section titled “[0.7.0] -- 2026-02-24”
  • update all crates to Rust edition 2024
  • refget: add sequence and collection aliases, ancillary digests, seqcol comparison, and FHR metadata; sequences can exist independently of a collection
  • refget: add streaming FASTA digest computation in WASM
  • uniwig: fix off-by-one errors; add variable-step format
  • refactor R bindings; RefgetStore uses ExternalPtr so R garbage collection works
  • RegionSet returns an error instead of panicking on bad input

[0.6.0] -- 2026-01-26

Section titled “[0.6.0] -- 2026-01-26”
  • refget: add FAI (FASTA index) computation and metadata storage
  • refget: add remote-access RefgetStore with lazy loading and caching; SequenceRecord is now an enum (Stub vs Full)
  • refget: add on_disk(), in_memory(), load_remote() constructors and export methods

[0.5.3] -- 2026-01-21

Section titled “[0.5.3] -- 2026-01-21”
  • remove reqwest dependency from gtars-core
  • merge the gdrs package into gtars
  • genomicdist: add bed classifier

[0.5.2] -- 2025-11-04

Section titled “[0.5.2] -- 2025-11-04”
  • switch overlap operations to MultiChromOverlapper
  • genomicdist: more tests, fix region distribution plot, fix function naming in wasm bindings
  • load/write functions accept AsRef<Path>

[0.5.1] -- 2025-10-08

Section titled “[0.5.1] -- 2025-10-08”
  • introduce web assembly bindings
  • introduce R bindings

[0.5.0] -- 2025-09-25

Section titled “[0.5.0] -- 2025-09-25”
  • converted the codebase into a workspace with multiple crates

[0.4.0] -- 2025-09-10

Section titled “[0.4.0] -- 2025-09-10”
  • added get_max_end_per_chr and nucleotides_length methods to RegionSet

[0.3.0] -- 2025-07-30

Section titled “[0.3.0] -- 2025-07-30”
  • move digests functionality to refget
  • add RefgetStore to refget and its associated python bindings
  • integrate support for bits + backend types for tokenizers (AIList or BITS)
  • reworked the tokenization CLI to support the new bits and backend options

[0.2.5] -- 2025-04-06

Section titled “[0.2.5] -- 2025-04-06”
  • Rework tokenizer API to be more consistent with the HuggingFace tokenizers API.
  • Updates to RegionSet to improve performance and usability.
  • Added file_digest function to RegionSet struct
  • Fixed reqwest error in R bindings
  • Fixed #107

[0.2.4] -- 2025-03-05

Section titled “[0.2.4] -- 2025-03-05”
  • Attempt to fix failing python bindings in CI linux #104

[0.2.3] -- 2025-03-05

Section titled “[0.2.3] -- 2025-03-05”
  • Improved RegionSet, by adding a multiple new methods: to_bed, to_bed_gz, to_bigbed, identifier(), and others.
  • Fixed allowed fasta_digest to accept Path or bytes #93

[0.2.2] -- 2025-02-18

Section titled “[0.2.2] -- 2025-02-18”

[0.2.1] -- 2025-02-11

Section titled “[0.2.1] -- 2025-02-11”
  • allow comments at the beginning of fragment files
  • bump bigtools to 0.5.5, fixing #74 and #77

[0.2.0] -- 2025-01-13

Section titled “[0.2.0] -- 2025-01-13”
  • add position shift workflow for bam to bw (was previously added for bam to bed)
  • add scaling argument for bam to bw workflow #53
  • fix accumulation issue for bam workflow #56
  • fix wiggle file (core) beginning at 0 #43
  • fix npy file (end) using start instead of end #61
  • force zoom to 1 for bed/narrowPeak to bw #34
  • fix IGD overlap issue #45
  • add ga4gh refget digest functionality #58
  • fix wig and npy inconsistency #64
  • fix narrowPeak to bw zoom #34
  • fix bed to bw fileheader consistency issue #52
  • change npy metadata file structure #65

[0.1.2]

Section titled “[0.1.2]”
  • add position shift workflow for bam to bw (was previously added for bam to bed)
  • add scaling argument for bam to bw workflow #53
  • fix accumulation issue for bam workflow #56
  • fix wiggle file (core) beginning at 0 #43
  • fix npy file (end) using start instead of end #61
  • force zoom to 1 for bed/narrowPeak to bw #34
  • fix IGD overlap issue #45
  • add ga4gh refget digest functionality #58

[0.1.1] -- 2024-12-03

Section titled “[0.1.1] -- 2024-12-03”
  • hot fix for broken python bindings; remove IGD from the python bindings for now

[0.1.0] -- 2024-12-03

Section titled “[0.1.0] -- 2024-12-03”
  • Rust implementation of uniwig that expands on the C++ version
    • Uniwig now accepts a single sorted .bed file, .narrowPeak file, or .bam file.
    • Outputs now include .wig, .npy, .bedGraph, and .bw
    • Accumulations can now be counted via .narrowPeak scoring
  • Rust implementation of igd ported from the C version (experimental).
  • Region scoring matrix calculation for region clustering
  • Fragment file splitter for pseudobulking

[0.0.15] -- 2024-07-29

Section titled “[0.0.15] -- 2024-07-29”
  • added meta tokenization tools and a new MetaTokenizer struct that can be used to tokenize regions using the meta-token strategy.
  • added some annotations to the pyo3 #[pyclass] and #[pymethods] attributes to make the python bindings more readable.

[0.0.14] -- 2024-06-11

Section titled “[0.0.14] -- 2024-06-11”
  • renamed repository to gtars to better reflect the project's goals.

[0.0.13] -- 2024-06-03

Section titled “[0.0.13] -- 2024-06-03”
  • implemented a fragment file tokenizer that will generate .gtok files directly from fragments.tsv.gz files.
  • fix an off-by-one error in the region-to-id maps in the Universe structs. This was leading to critical bugs in our models.

[0.0.12] -- 2024-05-28

Section titled “[0.0.12] -- 2024-05-28”
  • optimize creation of PyRegionSet to reduce expensive cloning of Universe structs.

[0.0.11] -- 2024-05-22

Section titled “[0.0.11] -- 2024-05-22”
  • redesigned API for the tokenizers to better emulate the huggingface tokenizers API.
  • implemented new traits for tokenizers to allow for more flexibility when creating new tokenizers.
  • bumped the version pyo3 to 0.21.0
  • added rust-numpy dependency to the python bindings for exporting tokenized regions as numpy arrays.
  • overall stability improvements to the tokenizers and the python bindings.

[0.0.10] -- 2024-01-24

Section titled “[0.0.10] -- 2024-01-24”
  • update file format specifications

[0.0.9] -- 2024-01-22

Section titled “[0.0.9] -- 2024-01-22”
  • start working on the concept of a .gtok file-format to store tokenized regions
  • added basic readers and writers for this format

[0.0.8] -- 2024-01-17

Section titled “[0.0.8] -- 2024-01-17”
  • add a new ids_as_strs getter to the TokenizedRegionSet struct so that we can get the ids as strings quickly, this is meant mostly for interface with geniml.

[0.0.7] -- 2023-11-30

Section titled “[0.0.7] -- 2023-11-30”
  • move things around based on rust club feedback

[0.0.6] -- 2024-02-20

Section titled “[0.0.6] -- 2024-02-20”

[0.0.5] -- 2024-02-19

Section titled “[0.0.5] -- 2024-02-19”
  • add many "core utils"
  • move gtokenizers into this package inside gtars::tokenizers
  • create tokenize cli
  • add tests for core utils and tokenizers
  • RegionSet is now backed by a polars DataFrame
  • new python bindings for core utils and tokenizers

[0.0.4] -- 2023-11-06

Section titled “[0.0.4] -- 2023-11-06”
  • add type annotations to the python bindings

[0.0.3] -- 2023-11-06

Section titled “[0.0.3] -- 2023-11-06”
  • work on python bindings initialization

[0.0.2] -- 2023-09-20

Section titled “[0.0.2] -- 2023-09-20”
  • prepare for first release

[0.0.1] -- 2023-08-15

Section titled “[0.0.1] -- 2023-08-15”
  • initial setup of repository
  • two main wrappers: 1) wrapper binary crate, and 2) wrapper library crate
  • gtars can be used as a library crate. or as a command line tool