# Geneformer Module

<a id="geniml.geneformer"></a>

<a id="geniml.geneformer.GeneformerConfig"></a>

## `GeneformerConfig`

```python
class GeneformerConfig(vocab_size: int = 20275, hidden_size: int = 512, intermediate_size: int = 1024, num_attention_heads: int = 8, num_hidden_layers: int = 12, attention_probs_dropout_prob: float = 0.02, hidden_act: str = 'relu', hidden_dropout_prob: float = 0.02, initializer_range: float = 0.02, layer_norm_eps: float = 1e-12, max_position_embeddings: int = 4096, pad_token_id: int = 0, classifier_dropout: float = None, **kwargs)
```

Bases: `BertConfig`

Configuration for Geneformer model, a BERT-like transformer for gene tokens.

### Attributes

- <a id="geniml.geneformer.GeneformerConfig.model_type"></a>**`model_type`** = `'geneformer'`

<a id="geniml.geneformer.GeneformerModel"></a>

## `GeneformerModel`

```python
class GeneformerModel(config: GeneformerConfig)
```

Bases: `BertForMaskedLM`

Geneformer Model with a masked language modeling head.

### Attributes

- <a id="geniml.geneformer.GeneformerModel.config_class"></a>**`config_class`** = `GeneformerConfig`
- <a id="geniml.geneformer.GeneformerModel.base_model_prefix"></a>**`base_model_prefix`** = `'geneformer'`

### Methods

<a id="geniml.geneformer.GeneformerModel.get_input_embeddings"></a>

#### `get_input_embeddings`

```python
def get_input_embeddings()
```

Returns the input embeddings of the model.

<a id="geniml.geneformer.GeneformerModel.set_input_embeddings"></a>

#### `set_input_embeddings`

```python
def set_input_embeddings(value)
```

Sets the input embeddings of the model.

<a id="geniml.geneformer.GeneformerModel.forward"></a>

#### `forward`

```python
def forward(*args, **kwargs)
```

<a id="geniml.geneformer.TranscriptomeTokenizer"></a>

## `TranscriptomeTokenizer`

```python
class TranscriptomeTokenizer(custom_attr_name_dict=None, nproc=1, chunk_size=512, model_input_size=4096, special_token=True, collapse_gene_ids=True, gene_median_file=None, token_dictionary_file=None, gene_mapping_file=None)
```

Initialize tokenizer.

**Parameters:**

custom_attr_name_dict : None, dict
    | Dictionary of custom attributes to be added to the dataset.
    | Keys are the names of the attributes in the loom file.
    | Values are the names of the attributes in the dataset.
nproc : int
    | Number of processes to use for dataset mapping.
chunk_size : int = 512
    | Chunk size for anndata tokenizer.
model_input_size : int = 4096
    | Max input size of model to truncate input to.
    | For the 30M model series, should be 2048. For the 95M model series, should be 4096.
special_token : bool = True
    | Adds CLS token before and EOS token after rank value encoding.
    | For the 30M model series, should be False. For the 95M model series, should be True.
collapse_gene_ids : bool = True
    | Whether to collapse gene IDs based on gene mapping dictionary.
gene_median_file : Path
    | Path to pickle file containing dictionary of non-zero median
    | gene expression values across Genecorpus-30M.
token_dictionary_file : Path
    | Path to pickle file containing token dictionary (Ensembl IDs:token).
gene_mapping_file : None, Path
    | Path to pickle file containing dictionary for collapsing gene IDs.

### Attributes

- <a id="geniml.geneformer.TranscriptomeTokenizer.custom_attr_name_dict"></a>**`custom_attr_name_dict`**
- <a id="geniml.geneformer.TranscriptomeTokenizer.nproc"></a>**`nproc`**
- <a id="geniml.geneformer.TranscriptomeTokenizer.chunk_size"></a>**`chunk_size`**
- <a id="geniml.geneformer.TranscriptomeTokenizer.model_input_size"></a>**`model_input_size`**
- <a id="geniml.geneformer.TranscriptomeTokenizer.special_token"></a>**`special_token`**
- <a id="geniml.geneformer.TranscriptomeTokenizer.gene_median_dict"></a>**`gene_median_dict`**
- <a id="geniml.geneformer.TranscriptomeTokenizer.gene_token_dict"></a>**`gene_token_dict`**
- <a id="geniml.geneformer.TranscriptomeTokenizer.collapse_gene_ids"></a>**`collapse_gene_ids`**
- <a id="geniml.geneformer.TranscriptomeTokenizer.gene_keys"></a>**`gene_keys`**
- <a id="geniml.geneformer.TranscriptomeTokenizer.gene_mapping_dict"></a>**`gene_mapping_dict`**
- <a id="geniml.geneformer.TranscriptomeTokenizer.genelist_dict"></a>**`genelist_dict`**

### Methods

<a id="geniml.geneformer.TranscriptomeTokenizer.tokenize_data"></a>

#### `tokenize_data`

```python
def tokenize_data(data_directory: Path | str, output_directory: Path | str, output_prefix: str, file_format: Literal['loom', 'h5ad'] = 'loom', use_generator: bool = False)
```

Tokenize .loom files in data_directory and save as tokenized .dataset in output_directory.

**Parameters:**

data_directory : Path
    | Path to directory containing loom files or anndata files
output_directory : Path
    | Path to directory where tokenized data will be saved as .dataset
output_prefix : str
    | Prefix for output .dataset
file_format : str
    | Format of input files. Can be "loom" or "h5ad".
use_generator : bool
    | Whether to use generator or dict for tokenization.

<a id="geniml.geneformer.TranscriptomeTokenizer.tokenize_files"></a>

#### `tokenize_files`

```python
def tokenize_files(data_directory, file_format: Literal['loom', 'h5ad'] = 'loom')
```

<a id="geniml.geneformer.TranscriptomeTokenizer.tokenize_anndata"></a>

#### `tokenize_anndata`

```python
def tokenize_anndata(adata_file_path, target_sum=10000)
```

<a id="geniml.geneformer.TranscriptomeTokenizer.tokenize_loom"></a>

#### `tokenize_loom`

```python
def tokenize_loom(loom_file_path, target_sum=10000)
```

<a id="geniml.geneformer.TranscriptomeTokenizer.create_dataset"></a>

#### `create_dataset`

```python
def create_dataset(tokenized_cells, cell_metadata, use_generator=False, keep_uncropped_input_ids=False)
```