BEDspace Module

main

def main(data_path: str, metadata: str, universe: str, output: str, labels: str)

Main function for the preprocess pipeline.

Args: data_path (str): Path to bed files. metadata (str): Path to metadata file. universe (str): Path to universe file. output (str): Path to output folder. labels (str): Labels string (cell_type,target).

main

def main(path_to_starspace: str, input: str, output: str, num_epochs: str = DEFAULT_NUM_EPOCHS, dim: str = DEFAULT_DIM, learning_rate: str = DEFAULT_LEARNING_RATE)

Train a starspace model.

Args: path_to_starspace (str): Path to starspace folder (must be prebuilt). input (str): Path to input file generated by preprocess. num_epochs (str): Number of epochs. dim (str): Dimension of embeddings. learning_rate (str): Learning rate. output (str): Path to output folder.

main

def main(input: str, path_to_starspace: str, metadata_test: str, metadata_train: str, universe: str, project_name: str, output: str, labels: str, files: str, threshold: float = DEFAULT_THRESHOLD)

Main function for the distance pipeline.

Args: input (str): Path to the trained starSpace model. path_to_starspace (str): Path to starspace folder (must be prebuilt). metadata_test (str): Path to test metadata. metadata_train (str): Path to train metadata. universe (str): Path to universe file. project_name (str): Name of the project. output (str): Path to output folder to save the distance files. labels (str): Labels string (cell_type,target). files (str): Path to files to be embedded (the data). threshold (float): Threshold for filtering.

run_scenario1

def run_scenario1(query: str, distances: str, output: str, num_results: int = DEFAULT_NUM_SEARCH_RESULTS)

Run the search command for scenario 1: Give me a label, I'll return region sets.

Args: query (str): The query string (a label). distances (str): The path to the distances file. num_results (int): The number of results to return. output (str): The path to save the barplots.

run_scenario2

def run_scenario2(query: str, distances: str, output: str, num_results: int = DEFAULT_NUM_SEARCH_RESULTS)

Run the search command for scenario 2: Give me a region set, I'll return labels.

Args: query (str): The query string (a path to a file). distances (str): The path to the distances file. num_results (int): The number of results to return. output (str): The path to save the barplots.

run_scenario3

def run_scenario3(query: str, distances: str, output: str, num_results: int = DEFAULT_NUM_SEARCH_RESULTS)

Run the search command for scenario 3: Give me a region set, I'll return region sets.

Args: query (str): The query string (a path to a file). distances (str): The path to the distances file. num_results (int): The number of results to return. output (str): The path to save the barplots.

meta_preprocessing

def meta_preprocessing(meta_path, labels, input_path, mode, chunksize=10000)

Process the metadata file in chunks, combining file paths with associated labels.

Parameters:

• meta_path (str): Path to the metadata CSV containing file names and labels.
• labels (list): List of column names representing labels to concatenate.
• input_path (str): Directory path to prepend to each file name.
• chunksize (int): Number of rows to process at a time (default: 10000).

Returns:

• file_list (df): List of combined file paths and labels

data_preparation

def data_preparation(path_file_label: str, univ, mode: str)

Convert input region set data (BED files) into a StarSpace acceptable format.

Parameters:

• path_file_label (str): path to the BED file.
• univ(TreeTokenizer obj): universe to intersect the BED file with.
• mode (str): either "train" or "test".

Returns:

• (list): A list containing the file path and the formatted region set data (with or without labels).

get_label_embedding

def get_label_embedding(path_word_embedding, label_prefix)

Extract label embeddings from the trained model's word embeddings.

Parameters:

• path_word_embedding (str): Path to the word embedding file.
• label_prefix (str): Prefix used to identify label embeddings.

Returns:

• label_vectors (list): List of label embedding vectors.
• labels (list): List of labels without the prefix.

bed2vec

def bed2vec(file_list, univ, model, docs, doc_embed, path_to_starsapce)

Predict sample (region set) embedding using StarSpace. Write output to a file.

Parameters:

• file_list (df): List of BED files to process.
• univ (TreeTokenizer obj): universe for intersecting with regions.
• model (str): Path to the trained StarSpace model.
• docs (str): Path to save the context documents for embedding.
• doc_embed (str): Path to save the predicted document embeddings.
• path_to_starspace (str): Directory where StarSpace is located.

get_embedding_matrix

def get_embedding_matrix(path_embeded_document)

Create sample and label embedding matrix from the given document embedding file.

Parameters:

• path_embeded_document (str): Path to the document embedding file (CSV format).

Returns:

• Xs (list): A list of embedding vectors (samples) in the form of floats.

calculate_distance

def calculate_distance(X_files, X_labels, y_files, y_labels, mode)

Calculate the cosine distance matrix between two sets of file embeddings and return a melted DataFrame with the distances and associated labels.

Parameters:

• X_files (array-like): test embeddings.
• X_labels (array-like): train embeddings.
• y_files (list): test file name.
• y_labels (list): trained label or training file names.
• mode (str): Specifies the processing type; "rl" or "rr".

Returns:

• file_distance (df): A melted DataFrame containing distances and labels.

label_preprocessing

def label_preprocessing(path_label_embedding, label_prefix, common_labels=[])

UMAP_plot

def UMAP_plot(data_X, y, title='', nn=100, metric='cosine', filename='', plottitle='', output_folder='')

retrieve_meta_test

def retrieve_meta_test()

Scenario1

def Scenario1(path_simfile)

Scenario2

def Scenario2(path_simfile)

Attributes

• path_model = '../outputs/bedembed_output/starspace_model_hg19.tsv'
• label_prefix = '__label__'
• nn = 5
• ump = umap.UMAP(a=None, angular_rp_forest=False, b=None, force_approximation_algorithm=False, learning_rate=0.9, local_connectivity=1.0, low_memory=False, metric='cosine', metric_kwds=None, min_dist=0.1, n_components=2, n_epochs=1000, n_neighbors=nn, negative_sample_rate=3, output_metric='euclidean', output_metric_kwds=None, random_state=3, repulsion_strength=1.0, set_op_mix_ratio=1.0, spread=1.0, target_metric='categorical', target_metric_kwds=None, target_n_neighbors=(-1), target_weight=0.5, transform_queue_size=4.0, transform_seed=3, unique=False, verbose=False)
• ump_data = pd.DataFrame(ump.transform(label_vectors))
• Z = hierarchy.linkage(ump_data, 'weighted')
• target = 'target'
• common_labels = []
• fig = UMAP_plot(label_vectors, labels, title='label_embedding', metric='cosine', nn=nn, filename='', plottitle='Label Embedding', output_folder='./')
• path_meta = '../'
• meta_data = pd.read_csv(path_meta + 'tests/test_file_meta.csv')
• cell_types = list(set(meta_data['cell_type']))
• targets = [(str(t).lower()) for t in targets]
• path_simfile = '../outputs/bedembed_output/similarity_score_hg19.csv'
• search = 'target'
• meta_test = pd.read_csv(path_meta + 'tests/test_file_meta.csv')[['file_name', 'cell_type', 'target']]
• meta_train = pd.read_csv(path_meta + 'tests/test_file_meta.csv')[['file_name', 'cell_type', 'target']]
• file_name = '../outputs/bedembed_output/query_db_similarity_score_hg19.csv'
• sim = sim.merge(meta_test, left_on='test_name', right_on='file_name').merge(meta_train, left_on='train_name', right_on='file_name', suffixes=('_test', '_train'))
• nof = len(df[df.original_label_test == df.original_label_train])
• df = df.sort_values(by=['score'])