syconn.proc package

Submodules

• syconn.proc.general module

• syconn.proc.graphs module

• syconn.proc.image module

• syconn.proc.mapping module

• syconn.proc.meshes module

• syconn.proc.rendering module

• syconn.proc.sd_proc module

• syconn.proc.skel_based_classifier module

• syconn.proc.skel_based_classifier_helper module

• syconn.proc.ssd_proc module

• syconn.proc.stats module

syconn.proc.general module

syconn.proc.general.crop_bool_array(arr)

Crops a 3D boolean array to its True region. The function finds the indices of the True region and returns a cropped array along with the offset.

Parameters

arr (np.array) – A 3D boolean array to be cropped.

Returns

A tuple containing the cropped array and the offset.

Return type

Tuple[np.array, List[int]]

syconn.proc.general.cut_array_in_one_dim(array, start, end, dim)

Cuts a numpy array along a specified dimension from a start index to an end index. The function handles the case where only the last elements should be retrieved.

Parameters

• array (np.array) – The input numpy array to be cut.

• start (int) – The start index for the cut.

• end (int) – The end index for the cut.

• dim (int) – The dimension along which the array is to be cut.

Returns

The cut array.

Return type

np.array

syconn.proc.general.dense_matrix(sv, edge_size)

Returns a dense matrix representation of coordinates. The function creates a 3D matrix of zeros with dimensions equal to the cube of the edge size. It then sets the value of the matrix at the indices specified by the input array to 1.

Parameters

• sv (np.array) – A 2D numpy array where each row represents the 3D coordinates of a point.

• edge_size (int) – The edge size of the 3D matrix to be created.

Returns

A 3D numpy array representing the dense matrix.

Return type

np.array

syconn.proc.general.timeit(func)

A decorator function that measures the execution time of the decorated function. It prints the total execution time and the average time per sample.

Parameters

func (function) – The function to be decorated.

Returns

The decorated function.

Return type

function

syconn.proc.graphs module

syconn.proc.graphs.bfs_smoothing(vertices, vertex_labels, max_edge_length=120, n_voting=40)

This function smooths vertex labels by applying a majority vote on a BFS subset of nodes for every node in the graph.

Parameters

• vertices (np.array) – An array of shape N, 3 representing the vertices.

• vertex_labels (np.array) – An array of shape N, 1 representing the vertex labels.

• max_edge_length (float) – The maximum distance between vertices to consider them connected in the graph. Default is 120.

• n_voting (int) – The number of collected nodes during BFS used for majority vote. Default is 40.

Returns

An array representing the smoothed vertex labels.

Return type

np.array

syconn.proc.graphs.chunkify_contiguous(l, n)

This function yields successive n-sized chunks from a list.

Parameters

• l (list) – The input list.

• n (int) – The size of the chunks.

Yields

list – The next n-sized chunk from the list.

Reference:

https://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks

syconn.proc.graphs.coordpath2anno(coords, scaling=None)

Creates a skeleton from scaled coordinates. Assumes coordinates are in order for edge creation.

Parameters

• coords (np.array) – Array of coordinates.

• scaling (np.ndarray, optional) – Scaling factor for the skeleton. If not provided, the default scaling from the global configuration is used.

Returns

Skeleton annotation created from the coordinates.

Return type

SkeletonAnnotation

syconn.proc.graphs.create_ccsize_dict(g, bbs, is_connected_components=False)

This function calculates the bounding box size of connected components.

Parameters

• g (nx.Graph) – The supervoxel graph.

• bbs (dict) – A dictionary representing the bounding boxes in physical

• units. –

• is_connected_components (bool) – If True, the graph g is already

• False (connected components. If) –

• is (nx.connected_components) –

• False. (applied. Default is) –

Returns

A look-up dictionary which stores the connected component bounding box for every single node in the input Graph g.

Return type

dict

syconn.proc.graphs.create_graph_from_coords(coords, max_dist=6000, force_single_cc=True, mst=False)

Generates a skeleton from sample locations by adding edges between points within a maximum distance and then pruning the skeleton using a minimum spanning tree (MST). Nodes will have a ‘position’ attribute.

Parameters

• coords (np.ndarray) – Array of coordinates.

• max_dist (float, optional) – Maximum distance between two nodes to consider them connected. Defaults to 6000.

• force_single_cc (bool, optional) – If True, forces the tree generated from coordinates to be a single connected component. Defaults to True.

• mst (bool, optional) – If True, computes the minimum spanning tree. Defaults to False.

Returns

Networkx graph with edges between nodes (coordinate indices) using the ordering of

coordinates. For example, the edge (1, 2) connects coordinate coord[1] and coord[2].

Return type

nx.Graph

syconn.proc.graphs.draw_glia_graph(G, dest_path, min_sv_size=0, ext_glia=None, iterations=150, seed=0, glia_key='glia_probas', node_size_cap=inf, mcmp=None, pos=None)

Draws a graph with nodes colored in red (glia) and blue depending on their class. Writes the drawing to the destination path.

Parameters

• G (nx.Graph) – Graph to be drawn.

• dest_path (str) – Path to the destination file.

• min_sv_size (int, optional) – Minimum size of the supervoxel. Defaults to 0.

• ext_glia (dict, optional) – Dictionary with node in G as keys and class number as values.

• iterations (int, optional) – Number of iterations for layout generation. Defaults to 150.

• seed (int, optional) – Random seed for layout generation. Defaults to 0.

• glia_key (str, optional) – Key to access glia probabilities. Defaults to “glia_probas”.

• node_size_cap (int, optional) – Maximum node size. Defaults to infinity.

• mcmp (color palette, optional) – Color palette for the graph. If not provided, a default palette is used.

• pos (dict, optional) – Positions of nodes. If not provided, a spring layout is used.

Returns

None

syconn.proc.graphs.eucl_dist(a, b)

Calculates the Euclidean distance between two points.

Parameters

• a (np.array) – First point.

• b (np.array) – Second point.

Returns

Euclidean distance between the two points.

Return type

float

syconn.proc.graphs.get_glia_paths(g, glia_dict, node2ccsize_dict, min_cc_size_neuron, node2ccsize_dict_glia, min_cc_size_glia)

Finds paths between neuron type nodes in a graph that contain glia nodes.

Parameters

• g (nx.Graph) – Input graph.

• glia_dict (dict) – Dictionary mapping nodes to their glia predictions.

• node2ccsize_dict (dict) – Dictionary mapping neuron nodes to their sizes.

• min_cc_size_neuron (int) – Minimum size for a neuron connected component.

• node2ccsize_dict_glia (dict) – Dictionary mapping glia nodes to their sizes.

• min_cc_size_glia (int) – Minimum size for a glia connected component.

Returns

List of paths that contain glia nodes.

Return type

list

syconn.proc.graphs.get_glianess_dict(seg_objs, thresh, glia_key, nb_cpus=1, use_sv_volume=False, verbose=False)

Generates a dictionary of glia predictions and sizes for a list of SegmentationObjects.

Parameters

• seg_objs (list) – List of SegmentationObjects.

• thresh (float) – Threshold for glia prediction.

• glia_key (str) – Key to access glia predictions in the attribute dictionary of SegmentationObjects.

• nb_cpus (int, optional) – Number of CPUs to use for multiprocessing. Defaults to 1.

• use_sv_volume (bool, optional) – If True, use the volume of the supervoxel for size. Otherwise, use the bounding box. Defaults to False.

• verbose (bool, optional) – If True, print progress information. Defaults to False.

Returns

Two dictionaries, the first mapping SegmentationObjects to their glia predictions,

and the second mapping SegmentationObjects to their sizes.

Return type

tuple

syconn.proc.graphs.glia_loader_helper(args)

Helper function for loading glia predictions and sizes for a single SegmentationObject.

Parameters

args (tuple) – A tuple containing a SegmentationObject, a glia key, a threshold, and a boolean indicating whether to use supervoxel volume for size.

Returns

A tuple containing the glia prediction and size for the SegmentationObject.

Return type

tuple

syconn.proc.graphs.glia_path_length(glia_path, glia_dict, write_paths=None)

Calculates the shortest path length through a glia path. This function assumes a single connected glia component within the path. It uses the mesh property of each SegmentationObject to build a graph from all vertices to find the shortest path through (or more precise: along the surface of) glia. Edges between non-glia vertices have negligible distance (0.0001) to ensure shortest path along non-glia surfaces.

Parameters

• glia_path (list) – List of SegmentationObjects forming a path.

• glia_dict (dict) – Dictionary mapping SegmentationObjects to their glia predictions.

• write_paths (bool, optional) – If True, write the shortest path to a skeleton file. Defaults to None.

Returns

Shortest path length in nanometers.

Return type

float

syconn.proc.graphs.merge_nodes(G, nodes, new_node)

This function merges nodes in an unweighted, undirected graph.

Parameters

• G (Graph) – The input graph.

• nodes (list) – The nodes to be merged.

• new_node (any) – The new node that will replace the merged nodes.

syconn.proc.graphs.nxGraph2kzip(g, coords, kzip_path)

Writes a networkx graph to a kzip file. The representative coordinate of a node is used as the corresponding node location.

Parameters

• g (nx.Graph) – Networkx graph to be written.

• coords (np.ndarray) – Array of coordinates.

• kzip_path (str) – Path to the destination kzip file.

Returns

None

syconn.proc.graphs.remove_glia_nodes(g, size_dict, glia_dict, return_removed_nodes=False)

Removes glia nodes from a graph based on glia and size vertex properties, and calculates distance weights for shortest path analysis or similar.

Parameters

• g (nx.Graph) – Input graph.

• size_dict (dict) – Dictionary mapping nodes to their sizes.

• glia_dict (dict) – Dictionary mapping nodes to their glia predictions.

• return_removed_nodes (bool, optional) – If True, return the removed nodes. Defaults to False.

Returns

List of connected components of type neuron. If return_removed_nodes is True,

also returns a list of removed nodes.

Return type

list

syconn.proc.graphs.split_glia(sso, thresh, clahe=False, pred_key_appendix='')

This function splits a SuperSegmentationObject into glia and non-glia SegmentationObjects.

Parameters

• sso (SuperSegmentationObject) – The SuperSegmentationObject to be split.

• thresh (float) – The threshold for splitting.

• clahe (bool) – If True, the function will use CLAHE (Contrast Limited Adaptive

• False. (Histogram Equalization) Default is) –

• pred_key_appendix (str) – The appendix for the prediction key. Default is an

• string. (empty) –

Returns

Two lists representing the neuron and glia nodes, respectively.

Return type

list, list

syconn.proc.graphs.split_glia_graph(nx_g, thresh, clahe=False, nb_cpus=1, pred_key_appendix='')

This function splits a graph into glia and non-glia connected components.

Parameters

• nx_g (nx.Graph) – The input graph.

• thresh (float) – The threshold for splitting.

• clahe (bool) – If True, the function will use CLAHE (Contrast Limited Adaptive Histogram Equalization). Default is False.

• nb_cpus (int) – The number of CPUs to use. Default is 1.

• pred_key_appendix (str) – The appendix for the prediction key. Default is an empty string.

• verbose (bool) – No description provided in the new docstring.

Returns

Two lists representing the neuron and glia connected

components, respectively.

Return type

list, list

syconn.proc.graphs.split_subcc(g, max_nb, verbose=False, start_nodes=None)

This function creates a subgraph for each node consisting of nodes until the maximum number of nodes is reached.

Parameters

• g (Graph) – The input graph.

• max_nb (int) – The maximum number of nodes.

• verbose (bool) – If True, the function will display a progress bar. Default is False.

• start_nodes (iterable) – An iterable containing node IDs. Default is None.

Returns

A dictionary where the keys are the nodes and the values are the

subgraphs.

Return type

dict

syconn.proc.graphs.split_subcc_join(g, subgraph_size, lo_first_n=1)

This function creates a subgraph for each node consisting of nodes until the maximum number of nodes is reached.

Parameters

• g (nx.Graph) – The supervoxel graph.

• subgraph_size (int) – The size of subgraphs. The difference between subgraph_size and lo_first_n defines the supervoxel overlap.

• lo_first_n (int) – Leave out first n nodes. Will collect subgraph_size nodes starting from the center node and then omit the first lo_first_n nodes, i.e. not use them as new starting nodes. Default is 1.

Returns

A list of lists, where each inner list represents a subgraph with context.

Return type

list

syconn.proc.graphs.stitch_skel_nx(skel_nx, n_jobs=1)

Stitches connected components within a graph by recursively adding edges between the closest components.

Parameters

• skel_nx (nx.Graph) – Networkx graph. Nodes require a ‘position’ attribute.

• n_jobs (int, optional) – Number of jobs used for query of cKDTree. Defaults to 1.

Returns

Single connected component graph.

Return type

nx.Graph

syconn.proc.graphs.svgraph2kzip(ssv, kzip_path)

Writes the supervoxel (SV) graph stored in the SuperSegmentationObject to a kzip file. The representative coordinate of a SV is used as the corresponding node location.

Parameters

• ssv (SuperSegmentationObject) – Cell reconstruction object.

• kzip_path (str) – Path to the output kzip file.

Returns

None

syconn.proc.graphs.write_sopath2skeleton(so_path, dest_path, scaling=None, comment=None)

Writes a simple skeleton to a file where each node represents the center of mass of a SegmentationObject (SV), and edges are created in the order of the list.

Parameters

• so_path (list) – List of SegmentationObjects.

• dest_path (str) – Path to the destination file.

• scaling (np.ndarray or tuple, optional) – Scaling factor for the skeleton. If not provided, the default scaling from the global configuration is used.

• comment (str, optional) – Comment to be added to the skeleton.

Returns

None

syconn.proc.image module

syconn.proc.image.apply_clahe(arr, clipLimit=4.0, tileGridSize=(8, 8), ret_normalized=True)

This function applies the Contrast Limited Adaptive Histogram Equalization (CLAHE) filter on an array if the OpenCV library is available. It first checks if the array is two-dimensional and adds an extra dimension if necessary. The function then normalizes the array to have a maximum value of 255 and applies the CLAHE filter.

Parameters

• arr (np.array) – An array representing the input data.

• clipLimit (float, optional) – The threshold for contrast limiting. Defaults to 4.0.

• tileGridSize (tuple, optional) – The size of the grid for the CLAHE algorithm. Defaults to (8, 8).

• ret_normalized (bool, optional) – If True, return the normalized array. Defaults to True.

Returns

An array representing the data after applying the CLAHE filter.

Return type

np.array

syconn.proc.image.apply_clahe_plain(arr, clipLimit, tileGridSize)

syconn.proc.image.apply_equalhist(arr)

This function applies histogram normalization on an array if the OpenCV library is available. It first checks if the array is two-dimensional and adds an extra dimension if necessary. The function then normalizes the array to have a maximum value of 255 and applies histogram equalization.

Parameters

arr (np.array) – An array representing the input data.

Returns

An array representing the data after histogram normalization.

Return type

np.array

syconn.proc.image.apply_morphological_operations(vol, morph_ops, mop_kwargs=None)

Applies a series of morphological operations on the input volume. The operations are specified by a list of string identifiers that must match scipy.ndimage functions. The operations are applied in the order they appear in the list.

Parameters

• vol (np.ndarray) – The input 3D array on which the morphological operations are to be applied.

• morph_ops (List[str]) – A list of string identifiers specifying the morphological operations to be applied.

• mop_kwargs (Optional[dict]) – A dictionary of keyword arguments to be passed to the morphological operations.

Returns

The input volume after the specified morphological operations have been applied.

Return type

np.ndarray

syconn.proc.image.apply_pca(sv, pca=None)

This function applies Principal Component Analysis (PCA) to a supervoxel and returns the supervoxel coordinates rotated in the principal component system. If a PCA object is provided, the function uses it to transform the supervoxel. If not, the function creates a new PCA object and fits it to the supervoxel.

Parameters

• sv (np.array) – An array of coordinates representing the supervoxel.

• pca (PCA, optional) – A pre-fitted PCA object. Defaults to None.

Returns

An array of coordinates representing the supervoxel rotated in the principal component system.

Return type

np.array

syconn.proc.image.conn_comp(sv, max_dist)

This function computes the number of connected components and their labels in a supervoxel. It uses a pairwise distance function to compute the Euclidean distance between all pairs of points in the supervoxel. The function then creates a sparse matrix from the distance matrix and uses a connected components algorithm to find the connected components.

Parameters

• sv (np.array) – An array of coordinates representing the supervoxel.

• max_dist (int) – The maximum distance for two points to be considered connected.

Returns

A tuple containing the number of connected components and an array of labels.

Return type

tuple

syconn.proc.image.createCLAHE([clipLimit[, tileGridSize]]) → retval

. @brief Creates a smart pointer to a cv::CLAHE class and initializes it. . . @param clipLimit Threshold for contrast limiting. . @param tileGridSize Size of grid for histogram equalization. Input image will be divided into . equally sized rectangular tiles. tileGridSize defines the number of tiles in row and column.

syconn.proc.image.equalizeHist(src[, dst]) → dst

. @brief Equalizes the histogram of a grayscale image. . . The function equalizes the histogram of the input image using the following algorithm: . . - Calculate the histogram f$Hf$ for src . . - Normalize the histogram so that the sum of histogram bins is 255. . - Compute the integral of the histogram: . f[H’_i = sum _{0 le j < i} H(j)f] . - Transform the image using f$H’f$ as a look-up table: f$texttt{dst}(x,y) = H’(texttt{src}(x,y))f$ . . The algorithm normalizes the brightness and increases the contrast of the image. . . @param src Source 8-bit single channel image. . @param dst Destination image of the same size and type as src .

syconn.proc.image.fast_check_sing_comp(sv, max_dist=5)

This function performs a fast check to determine if a supervoxel is a single connected component by subsampling. It uses a pairwise distance function to compute the Euclidean distance between all pairs of points in the supervoxel. If the number of connected components is one, the function returns True.

Parameters

• sv (np.array) – An array of coordinates representing the supervoxel.

• max_dist (int, optional) – The maximum distance for two points to be

• 5. (considered connected. Defaults to) –

Returns

True if the supervoxel is a single connected component, False otherwise.

Return type

bool

syconn.proc.image.find_contactsite(coords_a, coords_b, max_hull_dist=1)

This function computes the contact sites between two sets of supervoxels and returns the coordinates of the contact site voxels. It uses a k-dimensional tree for efficient nearest neighbor queries. The function iterates over a range of distances until it finds at least one neighbor.

Parameters

• coords_a (np.array) – An array of coordinates representing the first set

• supervoxels. (of) –

• coords_b (np.array) – An array of coordinates representing the second set

• supervoxels. –

• max_hull_dist (int, optional) – The maximum distance between voxels in

• 1. (coords_a and coords_b. Defaults to) –

Returns

An array of coordinates representing the contact site voxels.

Return type

np.array

syconn.proc.image.get_aniso_struct(scaling)

Generates a kernel for morphological operations. The kernel is cross-like with anisotropic dilations in the xy plane. The size of the kernel is determined by the provided scaling factor.

Parameters

scaling (Union[tuple, np.ndarray]) – A tuple or array specifying the voxel size in nm.

Returns

A kernel for morphological operations, taking into account the voxel size.

Return type

Kernel

syconn.proc.image.multi_dilation(overlay, n_dilations, use_find_objects=False, background_only=True)

A wrapper function for dilation, which is a morphological operation that expands or thickens objects in a binary image.

Parameters

• overlay – The input image.

• n_dilations – The number of times dilation is applied.

• use_find_objects (bool, optional) – If True, uses ‘find_objects’ from scipy.ndimage to reduce the processed volume and apply the operation per object. Defaults to False.

• background_only (bool, optional) – If True, changes only apply to the background (0). Defaults to True.

Returns

The dilated image.

syconn.proc.image.multi_dilation_backgroundonly(overlay, n_dilations, mop_kwargs=None)

Similar to the multi_dilation function, but processes each object in the overlay independently. Changes only apply to the background (0), meaning objects will not dilate into other objects.

Parameters

• overlay – The input 3D volume of type uint.

• n_dilations – The number of times dilation is applied.

• mop_kwargs (dict, optional) – Additional keyword arguments passed to the morphological operation function. Defaults to None.

Returns

The dilated image.

syconn.proc.image.multi_mop(mop_func, overlay, n_iters, use_find_objects=False, mop_kwargs=None, verbose=False)

A generic function for binary morphological image operations with multi-label content. Currently supported operations include binary dilation, binary erosion, binary closing, and binary fill holes.

Parameters

• mop_func – The morphological operation function to be applied.

• overlay – The input image.

• n_iters – The number of times the morphological operation is applied.

• use_find_objects (bool, optional) – If True, uses ‘find_objects’ from scipy.ndimage to reduce the processed volume and apply the operation per object. Defaults to False.

• mop_kwargs (dict, optional) – Additional keyword arguments passed to the morphological operation function. Defaults to None.

• verbose (bool, optional) – If True, displays a progress bar. Defaults to False.

Returns

The image after applying the morphological operation.

syconn.proc.image.multi_mop_backgroundonly(mop_func, overlay, iterations, mop_kwargs=None)

Similar to the multi_mop function, but processes each object in the overlay independently. Changes only apply to the background (0), meaning objects will not dilate into other objects. The original regions/segmentation is only affected in case of erosion.

Notes

• For binary_closing it is advised to pass structure=np.ones((2, 2, 2)) in order to fill gaps at the array boundaries. See https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.ndimage.morphology.binary_closing.html for an example.

Parameters

• mop_func – The morphological operation function to be applied. One of binary_closing, binary_dilation, binary_erosion, binary_fill_holes (see scipy.ndimage).

• overlay – The input image. 3D volume of type uint.

• iterations – The number of times the morphological operation is applied.

• mop_kwargs (dict, optional) – Additional keyword arguments passed to the morphological operation function. Defaults to None.

Returns

The image after applying the morphological operation.

syconn.proc.image.normalize_img(img, max_val=255)

This function normalizes an image to have a maximum value specified by the user. It first converts the image to floating point values. The function then subtracts the minimum value of the image and divides by the maximum value. Finally, the function multiplies the image by the maximum value specified by the user.

Parameters

• img (np.array) – An array representing the input image.

• max_val (int or float, optional) – The maximum value for the normalized image. Defaults to 255.

Returns

An array representing the normalized image.

Return type

np.array

syconn.proc.image.normalize_vol(sv, edge_size, center_coord)

Returns a cube with a given edge size and a supervoxel centered at the center coordinate.

Parameters

• sv (np.array) – A 3D array representing the coordinates of voxels in the supervoxel.

• edge_size (int) – The edge size of the returned cube.

• center_coord (np.array) – The center coordinate for the supervoxel.

Returns

A cube with the given edge size and the supervoxel centered at the center coordinate.

Return type

np.array

syconn.proc.image.remove_outlier(sv, edge_size)

Removes outliers from a given array that are beyond the specified edge size. Outliers are defined as elements that fall outside the range [0, edge_size).

Parameters

• sv (np.array) – The input array from which outliers will be removed.

• edge_size (int) – The upper limit for determining outliers in the array.

Returns

The input array with outliers removed.

Return type

np.array

syconn.proc.image.rgb2gray(rgb)

syconn.proc.image.single_conn_comp(sv, max_dist=2, ref_coord=None, return_bool=False)

This function returns the single connected component of a set of coordinates. It first checks if the supervoxel is a single connected component. If it is, the function returns the supervoxel. If not, the function computes the connected components and their labels. The function then returns the largest connected component or the component closest to a reference coordinate.

Parameters

• sv (np.array) – An array of coordinates representing the supervoxel.

• max_dist (int, optional) – The maximum distance for two points to be considered connected. Defaults to 2.

• ref_coord (np.array, optional) – A reference coordinate. Defaults to None.

• return_bool (bool, optional) – If True, return a boolean array where True values correspond to the connected component. Defaults to False.

Returns

An array of coordinates representing the single connected component.

Return type

np.array

syconn.proc.image.single_conn_comp_img(img, background=1.0)

This function returns the connected component in an image that is located at the center. It first labels the connected components in the image. The function then creates a new image with the same shape as the input image and sets the pixels corresponding to the center component to their original values. TODO: add ‘max component’ option

Parameters

• img (np.array) – An array representing the input image.

• background (float, optional) – The value representing the background in the image. Defaults to 1.0.

Returns

An array representing the image with only the center connected component.

Return type

np.array

syconn.proc.mapping module

syconn.proc.mapping.crop_box_to_bndry(offset, box_size, bndry)

Restricts box_size and offset to valid values, i.e., within an upper limit (bndry) and a lower limit (0, 0, 0).

Parameters

• offset (np.array) – The offset of the box.

• box_size (np.array | list) – The size of the box.

• bndry (np.array) – The boundary of the box.

Returns

The valid box size and offset.

Return type

np.array

syconn.proc.mapping.get_glia_coverage(seg, neuron_ids, glia_ids, max_dist, scale)

Computes the glia coverage of neurons in a segmentation volume. Neurons and glia are treated as two classes and coverage is defined as neuron boundary voxels close (within max_dist) to the glia boundary.

Parameters

• seg (np.array) – The segmentation volume.

• neuron_ids (list) – The list of neuron IDs.

• glia_ids (list) – The list of glia IDs.

• max_dist (int|float) – The maximum distance for a voxel to be considered close.

• scale (np.array) – The scaling factor for the segmentation volume.

Returns

The number and fraction of neuron boundary voxels close to the glia boundary.

Return type

int | float

syconn.proc.mapping.map_glia_fraction(so, box_size=None, min_frag_size=10, overwrite=True)

Maps glia properties within a subvolume to a SegmentationObject (cs). This function requires the attribute ‘neuron_partners’.

Parameters

• so (SegmentationObject) – The SegmentationObject to which glia properties are mapped.

• box_size (np.array, optional) – The size in voxels (XYZ). Defaults to (500, 500, 250).

• min_frag_size (int, optional) – The minimum fragment size. Defaults to 10.

• overwrite (bool, optional) – If True, overwrites existing data. Defaults to True.

Returns

This function doesn’t return anything; it modifies the SegmentationObject in-place.

Return type

None

syconn.proc.meshes module

class syconn.proc.meshes.MeshObject(object_type, indices, vertices, normals=None, color=None, bounding_box=None)

Bases: object

property bounding_box

Returns the bounding box of the object.

Returns

The center and maximum distance of the object.

Return type

list

property colors

Returns the colors of the vertices. If no external color is provided, it returns an array of 0.5.

Returns

The colors of the vertices.

Return type

np.ndarray

property normals

Returns the normals of the vertices. If no normals are provided, it calculates the normals.

Returns

The normals of the vertices.

Return type

np.ndarray

property normals_resh

Reshapes the normals into a 2D array.

Returns

The reshaped normals.

Return type

np.ndarray

perform_pca_rotation()

Rotates the vertices into the principal component coordinate system.

renormalize_vertices(bounding_box=None)

Renormalizes the vertices using either the center and maximum distance from self.vertices or given from the bounding_box.

Parameters

• bounding_box (tuple, optional) – The center and scale, applied as

• follows – self.vert_resh / scale. Defaults to None.

retransform_external_coords(coords)

Retransforms the given coordinates according to the center and maximum distance of the object.

Parameters

coords (np.ndarray) – The coordinates to be retransformed.

Returns

The retransformed coordinates.

Return type

np.ndarray

transform_external_coords(coords)

Transforms the given coordinates according to the center and maximum distance of the object.

Parameters

coords (np.ndarray) – The coordinates to be transformed.

Returns

The transformed coordinates.

Return type

np.ndarray

property vert_resh

Reshapes the vertices into a 2D array.

Returns

The reshaped vertices.

Return type

np.ndarray

property vertices_scaled

Returns the scaled vertices.

Returns

The scaled vertices.

Return type

np.ndarray

syconn.proc.meshes.calc_contact_syn_mesh(segobj, voxel_dc=None, **gen_kwgs)

Calculates the mesh of a contact synapse segmentation object.

Parameters

• segobj ('segmentation.SegmentationObject') – The segmentation object to calculate the mesh for.

• voxel_dc (Optional[Union[VoxelStorageLazyLoading, VoxelStorageDyn]], optional) – The voxel data container. Defaults to None.

• **gen_kwgs – Additional keyword arguments for the gen_mesh_voxelmask function.

Returns

The mesh of the segmentation object.

Return type

Union[List[np.ndarray], List[List[np.ndarray]]]

syconn.proc.meshes.calc_rot_matrices(coords, vertices, edge_length, nb_cpus=1)

Fits a PCA to local sub-volumes in order to rotate them according to its main process (e.g. x-axis will be parallel to the long axis of a tube).

Parameters

• coords (np.ndarray) – Center coordinates [M x 3].

• vertices (np.ndarray) – Vertices [N x 3].

• edge_length (Union[float, int]) – Spatial extent of box used for querying vertices

• fit (for the PCA) –

• nb_cpus (int, optional) – Number of CPUs to use. Defaults to 1.

Returns

Flattened OpenGL rotation matrix (Fortran ordering).

Return type

np.ndarray

syconn.proc.meshes.compartmentalize_mesh(ssv, pred_key_appendix='')

Splits a SuperSegmentationObject mesh into axon, dendrite and soma based on axoness prediction of SV’s contained in SuperSuperVoxel ssv.

Parameters

• ssv (SuperSegmentationObject) – The SuperSegmentationObject to be split.

• pred_key_appendix (str, optional) – Specific version of axoness prediction.

• "". (Defaults to) –

Returns

Majority label of each face / triangle in mesh indices; Triangle faces are assumed. If majority class has n=1, majority label is set to -1. A dictionary containing the compartmentalized meshes with keys as ‘axon’, ‘dendrite’ and ‘soma’.

Return type

np.array

syconn.proc.meshes.find_meshes(chunk, offset, pad=0, ds=None, scaling=None, meshing_props=None)

Find meshes within a segmented cube. The offset is given in voxels. Mesh vertices are scaled according to global_params.config[‘scaling’].

Parameters

• chunk (np.ndarray) – Cube which is processed.

• offset (np.ndarray) – Offset of the cube in voxels.

• pad (int, optional) – Pad chunk array with mode ‘edge’. Defaults to 0.

• ds (Optional[Union[list, tuple, np.ndarray]], optional) – Downsampling array in xyz. Defaults to None.

• scaling (Optional[Union[tuple, list, np.ndarray]], optional) – Voxel size. Defaults to None.

• meshing_props (Optional[dict], optional) – Keyword arguments used in zmesh.Mesher.get_mesh. Defaults to None.

Returns

The mesh of each segmentation ID in the input chunk. Vertices are in nm.

Return type

Dict[int, List[np.ndarray]]

syconn.proc.meshes.gen_mesh_voxelmask(voxel_iter, scale, vertex_size=10, boundary_struct=None, depth=10, compute_connected_components=True, voxel_size_simplify=None, min_vert_num=200, overlap=1, verbose=False, nb_neighbors=20, std_ratio=2.0)

Generates a mesh from a voxel mask. The voxel mask is provided as an iterator over 3D cubes and their offsets. The mesh is simplified and optionally split into connected components. The function also allows for statistical outlier removal based on the distance between points.

Parameters

• voxel_iter (Iterator[Tuple[np.ndarray, np.ndarray]]) – Iterator of binary voxel mask (3D cube) and cube offset (in voxels).

• scale (np.ndarray) – Size of voxels in mask_list in nm (x, y, z).

• vertex_size (float, optional) – In nm. Resolution used to simplify mesh. Defaults to 10 nm.

• boundary_struct (np.ndarray, optional) – Connectivity of kernel used to determine boundary. Defaults to None.

• depth (int, optional) – Depth of the octree used for the surface reconstruction. An important parameter that defines the resolution of the resulting triangle mesh. A higher depth value means a mesh with more details. Defaults to 10.

• compute_connected_components (bool, optional) – Compute connected components of mesh. Return list of meshes. Defaults to True.

• voxel_size_simplify (float, optional) – Voxel size in nm when applying simplify_vertex_clustering. Defaults to vertex_size.

• min_vert_num (int, optional) – Minimum number of vertices of the connected component meshes (only applied if compute_connected_components=True). Defaults to 200.

• overlap (int, optional) – Overlap between adjacent masks in mask_list. Defaults to 1.

• verbose (bool, optional) – Extra stdout output. Defaults to False.

• nb_neighbors (int, optional) – Number of neighbors used to calculate distance mean and standard deviation. Defaults to 20.

• std_ratio (float, optional) – Standard deviation of distance between points used as threshold for filtering. Defaults to 2.0.

Notes: Use voxel_iter with cubes that have 1-voxel-overlap to guarantee that segmentation instance boundaries

that align with the 3D array border are identified correctly.

Returns

Flat Index/triangle, vertex and normals array of the mesh.

List[ind, vert, norm] if compute_connected_components=True.

Return type

Union[List[np.ndarray], List[List[np.ndarray]]]

syconn.proc.meshes.get_object_mesh(obj, ds, mesher_kwargs=None)

This function generates a mesh for a given SegmentationObject using the marching cubes algorithm. Boundary artifacts are minimized by using a single 3D mask array of the object. The function returns an empty mesh if the object is smaller than a predefined minimum size.

Notes

This method is not suited for large objects as it creates a single 3D binary mask of the object.

Parameters

• obj (SegmentationObject) – The object for which the mesh is to be generated.

• ds (Union[tuple, list, np.ndarray]) – The magnitude of downsampling for each axis.

• mesher_kwargs (Optional[dict]) – Additional keyword arguments for the ‘find_meshes’ method.

Returns

A list containing the indices, vertices, and normals of the generated mesh.

Return type

list

syconn.proc.meshes.get_random_centered_coords(pts, nb, r)

This function returns the coordinates of randomly located center of masses in pts.

Parameters

• pts (np.array) – A numpy array representing the coordinates.

• nb (int) – The number of center of masses to be returned.

• r (int) – The radius of query_ball_point in order to get the list of

• mass. (points for the center of) –

Returns

A numpy array representing the coordinates of randomly located center of masses in pts.

Return type

np.array

syconn.proc.meshes.merge_meshes(ind_lst, vert_lst, nb_simplices=3)

This function combines several meshes into a single one. It takes in a list of indices, a list of vertices, and the number of simplices. It returns a numpy array of indices and vertices.

Parameters

• ind_lst (list) – A list of numpy arrays, each of shape [N, 1], representing the indices of the meshes.

• vert_lst (list) – A list of numpy arrays, each of shape [N, 1], representing the vertices of the meshes.

• nb_simplices (int) – The number of simplices. For example, for triangles, nb_simplices=3.

Returns

A tuple containing two numpy arrays. The first array contains the indices of the merged mesh, and the second array contains the vertices of the merged mesh.

Return type

tuple

syconn.proc.meshes.merge_meshes_incl_norm(ind_lst, vert_lst, norm_lst, nb_simplices=3)

This function combines several meshes, including their normals, into a single one. It takes in a list of indices, a list of vertices, a list of normals, and the number of simplices. It returns a list of numpy arrays of indices, vertices, and normals.

Parameters

• ind_lst (list) – A list of numpy arrays, each of shape [M, 1], representing the indices of the meshes.

• vert_lst (list) – A list of numpy arrays, each of shape [N, 1], representing the vertices of the meshes.

• norm_lst (list) – A list of numpy arrays, each of shape [N, 1], representing the normals of the meshes.

• nb_simplices (int) – The number of simplices. For example, for triangles, nb_simplices=3.

Returns

A list containing three numpy arrays. The first array contains the indices of the merged mesh, the second array contains the vertices of the merged mesh, and the third array contains the normals of the merged mesh.

Return type

list

syconn.proc.meshes.merge_someshes(sos, nb_simplices=3, nb_cpus=1, color_vals=None, cmap=None, alpha=1.0, use_new_subfold=True)

This function merges the meshes of a list of SegmentationObjects. It also caches the SegmentationObjects. It takes in a list of SegmentationObjects, the number of simplices, the number of CPUs, color values for every mesh, a matplotlib colormap, an alpha value, and a boolean indicating whether to use a new subfolder. It returns a numpy array of indices, vertices, and optionally colors.

Parameters

• sos (Iterable[SegmentationObject]) – A list of SegmentationObjects whose meshes are to be merged.

• nb_simplices (int) – The number of simplices. For example, for triangles, nb_simplices=3.

• nb_cpus (int) – The number of CPUs to use for the operation.

• color_vals (Optional[Iterable[float]]) – Color values for every mesh, in the form of a list of floats representing RGBA values. No normalization is performed.

• cmap (Optional[Iterable[tuple]]) – A matplotlib colormap to use for the operation.

• alpha (float) – An alpha value to use for the operation.

• use_new_subfold (bool) – A boolean indicating whether to use a new subfolder.

Returns

A tuple containing two or three numpy arrays. The first array contains the indices of the merged mesh, the second array contains the vertices of the merged mesh, and the third array (if present) contains the colors of the merged mesh.

Return type

tuple

syconn.proc.meshes.mesh2obj_file(dest_path, mesh, color=None, center=None, scale=None)

This function writes a given mesh to a .obj file. The function allows for optional transformations such as centering and scaling of the mesh.

Parameters

• dest_path (str) – The path to the destination file.

• mesh (List[np.ndarray]) – A list containing the indices, vertices, and normals of the mesh.

• color (Optional[Union[int, np.ndarray]]) – The color of the mesh as an int or numpy array (rgba).

• center (Optional[np.ndarray]) – The center of the mesh. If provided, the center is subtracted

• locations. (from the original vertex) –

• scale (Optional[float]) – The scale of the mesh. If provided, the vertex locations are multiplied

• centering. (after) –

Returns

None

syconn.proc.meshes.mesh_area_calc(mesh)

This function calculates the surface area of a given mesh.

Parameters

mesh – The mesh for which the surface area is to be calculated.

Returns

The surface area of the mesh in um^2.

Return type

float

syconn.proc.meshes.mesh_chunk(args)

This function generates a mesh for a given object type within a chunk of data. The mesh is created using the marching cubes algorithm and stored in a MeshStorage object. The function skips objects that are smaller than a predefined minimum size.

Parameters

args (tuple) – A tuple containing the directory of the attribute dictionary and the object type.

Returns

None

syconn.proc.meshes.mesh_creator_sso(ssv, segobjs=('sv', 'mi', 'sj', 'vc'))

Cache meshes of specified SegmentationObjects.

Parameters

• ssv (SuperSegmentationObject) – The SuperSegmentationObject whose meshes are to be cached.

• segobjs (Iterable[str], optional) – Types of SegmentationObjects. Defaults to

• ('sv' –

• 'mi' –

• 'sj' –

• 'vc') –

Returns

None

syconn.proc.meshes.write_mesh2kzip(k_path, ind, vert, norm, color, ply_fname, force_overwrite=False, invert_vertex_order=False)

Writes a mesh as .ply’s to a k.zip file.

Parameters

• k_path (str) – The path to the zip file.

• ind (np.array) – The indices of the vertices.

• vert (np.array) – The vertices of the mesh.

• norm (np.array) – The normals of the vertices.

• color (tuple or np.array) – The color of the vertices in RGBA format, values between 0 and 255.

• ply_fname (str) – The filename of the ply file.

• force_overwrite (bool, optional) – If True, the existing file will be overwritten. Defaults to False.

• invert_vertex_order (bool, optional) – If True, the order of the vertices is inverted. Defaults to False.

syconn.proc.meshes.write_meshes2kzip(k_path, inds, verts, norms, colors, ply_fnames, force_overwrite=True, verbose=True, invert_vertex_order=False)

Writes multiple meshes as .ply’s to a k.zip file.

Parameters

• k_path (str) – The path to the zip file.

• inds (list of np.array) – The list of indices of the vertices for each mesh.

• verts (list of np.array) – The list of vertices for each mesh.

• norms (list of np.array) – The list of normals for each mesh.

• colors (list of tuple or np.array) – The list of colors for each mesh in RGBA format, between 0 and 255.

• ply_fnames (list of str) – The list of filenames for the ply files.

• force_overwrite (bool, optional) – If True, the existing files will be overwritten. Defaults to True.

• verbose (bool, optional) – If True, progress information will be printed. Defaults to True.

• invert_vertex_order (bool, optional) – If True, the order of the vertices is inverted. Defaults to False.

syconn.proc.rendering module

syconn.proc.rendering.get_sso_view_dc(sso, verbose=False)

Extracts views from sampled positions in a SuperSegmentationObject (SSO) for each SegmentationObject (SV).

Parameters

• sso (SuperSegmentationObject) – The SSO from which to extract views.

• verbose (bool, optional) – If True, additional information is logged during

• False. (execution. Defaults to) –

Returns

A dictionary where the key is the SSO id and the value is a lz4 compressed view array.

Return type

dict

syconn.proc.rendering.load_rendering_func(func_name)

Loads the rendering function based on the specified function name and the platform specified in the global parameters. The platform can be either ‘egl’ or ‘osmesa’. If the platform is ‘egl’ but the requirements could not be imported, it switches to ‘osmesa’ platform.

Parameters

func_name (str) – Name of the rendering function to be loaded.

Returns

The specified rendering function.

Return type

Callable

Raises

• NotImplementedError – If the platform specified in the global parameters

• is not 'egl' or 'osmesa'. –

syconn.proc.rendering.render_mesh(mo, **kwargs)

Renders super voxel raw views located at randomly chosen center of masses in the vertex cloud. The rendering is done by calling the ‘multi_view_mesh’ function.

Parameters

• mo (MeshObject) – The mesh object to be rendered.

• **kwargs – Keyword arguments passed to the ‘multi_view_mesh’ function.

Returns

The rendered view array.

Return type

np.ndarray

syconn.proc.rendering.render_mesh_coords(coords, ind, vert, **kwargs)

Renders raw views located at given coordinates in the mesh. If ‘dest_dir’ is None, it returns a list of ViewContainer, otherwise, it writes the views to ‘dest_path’.

Parameters

• coords (np.ndarray) – The rendering locations.

• ind (np.ndarray) – The mesh indices/faces [N, 1].

• vert (np.ndarray) – The mesh vertices [M, 1].

• **kwargs – Keyword arguments passed to the ‘_render_mesh_coords’ function.

Returns

The rendered views at each coordinate.

Return type

np.ndarray

syconn.proc.rendering.render_sampled_sso(sso, ws=None, verbose=False, woglia=True, return_rot_mat=False, overwrite=True, add_cellobjects=None, index_views=False, return_views=False, cellobjects_only=False, rot_mat=None, view_key=None)

Renders views at sampled locations for each supervoxel (SV) from the combined mesh of all SVs. The number of sampled locations is dependent on the SV mesh size with scaling factor.

Parameters

• sso ('SuperSegmentationObject') – The SuperSegmentationObject to be rendered.

• ws (Optional[Tuple[int, int]]) – Window size in pixels (y, x). Default is specified in the config.yml or custom configs in the working directory.

• verbose (bool) – If True, logs additional information.

• woglia (bool) – If True, stores views with “without glia” identifier, i.e. flags the views as being created after the glia separation.

• return_rot_mat (bool) – If True, returns rotation matrices.

• add_cellobjects (Optional[Union[bool, Iterable[str]]]) – Default is (‘sj’, ‘vc’, ‘mi’). This ordering determines the channel order of the view array.

• overwrite (bool) – If True, does not skip existing views.

• index_views (bool) – If True, also renders index views.

• return_views (bool) – If True, returns view arrays.

• cellobjects_only (bool) – If True, only renders cell objects.

• rot_mat (Optional[np.ndarray]) – Rotation matrix array for every rendering location [N, 4, 4].

• view_key (Optional[str]) – String identifier for storing view arrays. Only needed if ‘return_views’ is False.

Returns

Depending on ‘return_views’ and ‘return_rot_mat’, it returns None, the view array, the view array and rotation matrices, or the rotation matrices.

Return type

Union[np.ndarray, Tuple[np.ndarray, np.ndarray], None]

syconn.proc.rendering.render_sso_coords(sso, coords, add_cellobjects=None, verbose=False, clahe=False, ws=None, cellobjects_only=False, wire_frame=False, nb_views=None, comp_window=None, rot_mat=None, return_rot_mat=False)

This function renders views of a SuperSegmentationObject at given coordinates. It can be used to visualize the 3D structure of the object and its subcellular components. The function supports various rendering options including the use of contrast limited adaptive histogram equalization (CLAHE), wireframe rendering, and the option to only render cell objects. The function also supports the return of rotation matrices which can be useful for further processing or analysis.

Parameters

• sso (SuperSegmentationObject) – The SuperSegmentationObject to be rendered.

• coords (np.ndarray) – An array specifying the 3D coordinates at which to render views of the object.

• add_cellobjects (Optional[Union[bool, Iterable[str]]]) – Specifies the subcellular structures to be included in the rendering. Default is None, which includes all structures.

• verbose (bool) – If True, additional log information will be printed. Default is False.

• clahe (bool) – If True, contrast limited adaptive histogram equalization (CLAHE) will be used to enhance the contrast of the rendered views. Default is False.

• ws (Optional[Tuple[int]]) – Specifies the window size in pixels for the rendered views. Default is None, which uses the window size specified in the global configuration.

• cellobjects_only (bool) – If True, only cell objects will be rendered. Default is False.

• wire_frame (bool) – If True, the object will be rendered as a wireframe. Default is False.

• nb_views (Optional[int]) – Specifies the number of views to render. Default is None, which uses the number of views specified in the global configuration.

• comp_window (Optional[float]) – Specifies the size of the window for the rendered views in nanometers. Default is None, which uses the window size specified in the global configuration.

• rot_mat (Optional[np.ndarray]) – An array specifying the rotation matrices for the rendered views. Default is None, which calculates the rotation matrices based on the object and coordinates.

• return_rot_mat (bool) – If True, the function will return the rotation matrices used for the rendered views. Default is False.

Returns

If return_rot_mat is False, the function returns an array

containing the rendered views. If return_rot_mat is True, the function returns a tuple containing the rendered views and the rotation matrices used.

Return type

np.ndarray or Tuple[np.ndarray, np.ndarray]

syconn.proc.rendering.render_sso_coords_index_views(sso, coords, verbose=False, ws=None, rot_mat=None, nb_views=None, comp_window=None, return_rot_matrices=False)

This function renders index views of a SuperSegmentationObject at given coordinates. Index views are color-coded views where each color corresponds to a different face of the object. This is useful for calculating the surface coverage captured by the views. The function supports various rendering options including the return of rotation matrices which can be useful for further processing or analysis.

Parameters

• sso (SuperSegmentationObject) – The SuperSegmentationObject to be rendered.

• coords (np.ndarray) – An array specifying the 3D coordinates at which to render index views of the object.

• verbose (bool) – If True, additional log information will be printed. Default is False.

• ws (Optional[Tuple[int, int]]) – Specifies the window size in pixels for the rendered views. Default is None, which uses the window size specified in the global configuration.

• rot_mat (Optional[np.ndarray]) – An array specifying the rotation matrices for the rendered views. Default is None, which calculates the rotation matrices based on the object and coordinates.

• nb_views (Optional[int]) – Specifies the number of views to render. Default is None, which uses the number of views specified in the global configuration.

• comp_window (Optional[float]) – Specifies the size of the window for the rendered views in nanometers. Default is None, which uses the window size specified in the global configuration.

• return_rot_matrices (bool) – If True, the function will return the rotation matrices used for the rendered views. Default is False.

Returns

If return_rot_matrices is False, the function returns an array

containing the rendered index views. If return_rot_matrices is True, the function returns a tuple containing the rendered index views and the rotation matrices used.

Return type

np.ndarray or Tuple[np.ndarray, np.ndarray]

syconn.proc.rendering.render_sso_coords_label_views(sso, vertex_labels, coords, verbose=False, ws=None, rot_mat=None, nb_views=None, comp_window=None, return_rot_matrices=False)

This function renders label views of a SuperSegmentationObject at given coordinates. Label views are color-coded views where each color corresponds to a different label assigned to the vertices of the object. This is useful for visualizing different labeled regions of the object. The function supports various rendering options including the return of rotation matrices which can be useful for further processing or analysis.

Parameters

• sso (SuperSegmentationObject) – The SuperSegmentationObject to be rendered.

• vertex_labels (np.ndarray) – An array specifying the labels assigned to the vertices of the object. The length and ordering of this array must match the vertex array of the object.

• coords (np.ndarray) – An array specifying the 3D coordinates at which to render label views of the object.

• verbose (bool) – If True, additional log information will be printed. Default is False.

• ws (Optional[Tuple[int, int]]) – Specifies the window size in pixels for the rendered views. Default is None, which uses the window size specified in the global configuration.

• rot_mat (Optional[np.ndarray]) – An array specifying the rotation matrices for the rendered views. Default is None, which calculates the rotation matrices based on the object and coordinates.

• nb_views (Optional[int]) – Specifies the number of views to render. Default is None, which uses the number of views specified in the global configuration.

• comp_window (Optional[float]) – Specifies the size of the window for the rendered views in nanometers. Default is None, which uses the window size specified in the global configuration.

• return_rot_matrices (bool) – If True, the function will return the rotation matrices used for the rendered views. Default is False.

Returns

If return_rot_matrices is False, the function returns an array

containing the rendered label views. If return_rot_matrices is True, the function returns a tuple containing the rendered label views and the rotation matrices used.

Return type

np.ndarray or Tuple[np.ndarray, np.ndarray]

syconn.proc.rendering.render_sso_coords_multiprocessing(ssv, n_jobs, rendering_locations=None, verbose=False, render_kwargs=None, view_key=None, render_indexviews=True, return_views=True)

Renders views of a SuperSegmentationObject (SSO) at specified coordinates using multiple processes.

Parameters

• ssv (SuperSegmentationObject) – The SSO to render.

• n_jobs (int) – The number of parallel jobs to run on the same node of the cluster.

• rendering_locations (np.ndarray, optional) – The locations to render. If not provided, locations are retrieved from the SSO’s SVs and results are stored at SV locations.

• verbose (bool, optional) – If True, additional information is logged during execution. Defaults to False.

• render_kwargs (dict, optional) – Additional keyword arguments for the rendering function.

• view_key (str, optional) – The key to use when storing the view arrays. Only needed if return_views is False.

• render_indexviews (bool, optional) – If True, index views are also rendered. Defaults to True.

• return_views (bool, optional) – If False and rendering_locations is None, views are saved at the supervoxel level. Defaults to True.

Returns

If return_views is True, returns an array of views after rendering.

Otherwise, returns None.

Return type

Union[None, np.ndarray]

syconn.proc.rendering.write_sv_views_chunked(svs, views, part_views, view_kwargs, disable_locking=False)

Writes the views of a list of SegmentationObjects (SVs) in chunks.

Parameters

• svs (List[SegmentationObject]) – The list of SVs for which to write views.

• views (np.ndarray) – The array of views to write.

• part_views (np.ndarray) – The cumulative number of views, used to determine the start and end indices of SV views in the views array.

• view_kwargs (dict) – Additional keyword arguments for the view writing function.

• disable_locking (bool, optional) – If True, disables locking. This is usually required as SVs are stored in chunks and rendered distributed on many processes. Defaults to False.

syconn.proc.sd_proc module

syconn.proc.sd_proc.convert_nvox2ratio_mapdict(map_dc)

Converts the number of overlapping voxels of each subcellular structure object inside the mapping dictionaries to each cell SV (subcell ID -> cell ID -> number overlap vxs) to fraction.

Parameters

map_dc (dict) – A dictionary mapping subcellular structure objects to cell SVs and the number of overlapping voxels.

syconn.proc.sd_proc.dataset_analysis(sd, recompute=True, n_jobs=None, compute_meshprops=False)

Analyzes a SegmentationDataset and extracts and caches SegmentationObjects attributes as numpy arrays. This function only recognizes dict/storage entries of type int for object attribute collection.

Parameters

• sd (SegmentationDataset) – The SegmentationDataset to analyze. This is typically a set of cell supervoxels (‘sv’).

• recompute (bool, optional) – A flag indicating whether to recompute key information of each object (representative coordinate, bounding box, size). Defaults to True.

• n_jobs (int, optional) – The number of jobs to run in parallel. Defaults to None.

• compute_meshprops (bool, optional) – A flag indicating whether to compute mesh properties. If set to True, it will also calculate meshes (sparsely) if not available. Defaults to False.

syconn.proc.sd_proc.init_sos(sos_dict)

Initializes a list of SegmentationObjects from a dictionary.

Parameters

sos_dict (dict) – A dictionary containing parameters for initializing SegmentationObjects.

Returns

A list of initialized SegmentationObjects.

Return type

list

syconn.proc.sd_proc.invert_mdc(mapping_dict)

Inverts a mapping dictionary to: cell ID -> subcell ID -> value (ratio or voxel count).

Parameters

mapping_dict (dict) – A dictionary mapping subcellular structure objects to cell SVs and the number of overlapping voxels or the ratio of overlapping voxels.

Returns

The inverted mapping dictionary.

Return type

dict

syconn.proc.sd_proc.map_subcell_extract_props(kd_seg_path, kd_organelle_paths, n_folders_fs=1000, n_folders_fs_sc=1000, n_chunk_jobs=None, n_cores=1, cube_of_interest_bb=None, chunk_size=None, log=None, overwrite=False)

Extracts and caches segmentation properties for each Supervoxel (SV) in cell and subcellular segmentation. This function requires KNOSSOSDATASETs (KDs) at kd_seg_path and kd_organelle_paths.

The process involves two main steps: 1. Extract properties (representative coordinate, bounding box, size) and overlap voxels

locally (organelles <-> cell segmentation).

2. Write out combined results for each SV object.

Parameters

• kd_seg_path (str) – Path to the KNOSSOSDATASET (KD) of the cell segmentation.

• kd_organelle_paths (dict) – Dictionary of paths to the KDs of the subcellular segmentations.

• n_folders_fs (int, optional) – Number of folders in the file system for the cell segmentation. Defaults to 1000.

• n_folders_fs_sc (int, optional) – Number of folders in the file system for the subcellular segmentation. Defaults to 1000.

• n_chunk_jobs (Optional[int], optional) – Number of chunks to be processed in parallel. Defaults to None.

• n_cores (int, optional) – Number of cores to be used for processing. Defaults to 1.

• cube_of_interest_bb (Optional[tuple], optional) – Bounding box of the cube of interest. Defaults to None.

• chunk_size (Optional[Union[tuple, np.ndarray]], optional) – Size of the chunks to be processed. Defaults to None.

• log (Logger, optional) – Logger for logging the process. Defaults to None.

• overwrite (bool, optional) – If True, overwrite existing results. Defaults to False.

syconn.proc.sd_proc.merge_map_dicts(map_dicts)

Merges map dictionaries in-place. Values will be stored in the first dictionary.

Parameters

map_dicts (List[dict]) – A list of map dictionaries to be merged.

syconn.proc.sd_proc.merge_meshes_dict(m_storage, tmp_dict)

This function merges mesh dictionaries. It iterates over the object IDs in the temporary dictionary and merges the meshes for each object ID.

Parameters

• m_storage (dict) – A dictionary where the key is the object ID and the value is a list of faces, vertices, and

• mesh. (and normals of the) –

• tmp_dict (dict) – A temporary dictionary where the key is the object ID and the value is a list of faces,

• vertices –

• mesh. –

Returns

None

syconn.proc.sd_proc.merge_meshes_single(m_storage, obj_id, tmp_dict)

Merges mesh dictionaries for a single object. This function takes in a MeshStorage object, an object id, and a temporary dictionary, and merges the temporary dictionary into the MeshStorage object.

Parameters

• m_storage (MeshStorage) – A MeshStorage object where the merged data will be stored.

• obj_id (int) – The id of the object whose mesh data is being merged.

• tmp_dict (dict) – A temporary dictionary containing mesh data to be merged into the MeshStorage object.

syconn.proc.sd_proc.merge_prop_dicts(prop_dicts, offset=None)

Merges property dictionaries in-place. All values will be stored in the first dictionary. If an offset is provided, it is added to the representative coordinates and bounding boxes of the objects.

Parameters

• prop_dicts (List[List[dict]]) – A list of property dictionaries to be merged.

• offset (Optional[np.ndarray]) – An optional offset to be added to the representative coordinates and bounding boxes of the objects.

syconn.proc.sd_proc.mesh_proc_chunked(working_dir, obj_type, nb_cpus=None)

Caches the meshes for all SegmentationObjects within the SegmentationDataset with a given object type.

Parameters

• working_dir (str) – The working directory for the SegmentationDataset.

• obj_type (str) – The type of the SegmentationObjects whose meshes are to be cached. Object type identifier, like ‘sj’, ‘vc’ or ‘mi’.

• nb_cpus (int, optional) – The number of CPUs to use for caching. Defaults to the number of cores per node in the global parameters. Default is 20.

syconn.proc.sd_proc.multi_probas_saver(args)

Saves the probabilities of predictions for a list of SegmentationObjects.

Parameters

args (tuple) – A tuple containing a SegmentationObject, the probabilities of its predictions, and the key to use for storing the predictions.

syconn.proc.sd_proc.predict_sos_views(model, sos, pred_key, nb_cpus=1, woglia=True, verbose=False, raw_only=False, single_cc_only=False, return_proba=False)

Predicts the views of a list of SegmentationObjects using a given model.

Parameters

• model (nn.Model) – The model to use for prediction.

• sos (list) – A list of SegmentationObjects whose views are to be predicted.

• pred_key (str) – The key to use for storing the predictions.

• nb_cpus (int, optional) – The number of CPUs to use for prediction. Defaults to 1.

• woglia (bool, optional) – Whether to exclude glia from the prediction. Defaults to True.

• verbose (bool, optional) – Whether to print verbose output. Defaults to False.

• raw_only (bool, optional) – Whether to use only raw data for prediction. Defaults to False.

• single_cc_only (bool, optional) – Whether to use only single connected components for

• False. (Defaults to) –

• return_proba (bool, optional) – Whether to return the probabilities of the predictions.

• False. –

Returns

The predicted views if return_proba is True, otherwise None.

Return type

np.ndarray

syconn.proc.sd_proc.predict_views(model, views, ch, pred_key, single_cc_only=False, verbose=False, return_proba=False, nb_cpus=1)

Predicts the views of a list of SegmentationObjects using a given model. The predictions are not written to disk if return_proba is True.

Parameters

• model (nn.Model) – The model to use for prediction.

• views (np.array) – An array of views to be predicted.

• ch (List[SegmentationObject]) – A list of SegmentationObjects whose views are to be predicted.

• pred_key (str) – The key to use for storing the predictions.

• single_cc_only (bool, optional) – Whether to use only single connected components for prediction. Defaults to False.

• verbose (bool, optional) – Whether to print verbose output. Defaults to False.

• return_proba (bool, optional) – Whether to return the probabilities of the predictions. Defaults to False.

• nb_cpus (int, optional) – The number of CPUs to use for prediction. Defaults to 1.

Returns

The predicted views if return_proba is True, otherwise None.

Return type

Optional[List[np.ndarray]]

syconn.proc.sd_proc.sos_dict_fact(svixs, version=None, scaling=None, obj_type='sv', working_dir=None, create=False)

Creates a dictionary with parameters for initializing SegmentationObjects.

Parameters

• svixs (list) – A list of segmentation object indices.

• version (str, optional) – The version of the SegmentationObjects. Defaults to None.

• scaling (list, optional) – The scaling factors for the SegmentationObjects. Defaults to None.

• obj_type (str, optional) – The type of the SegmentationObjects. Defaults to “sv”.

• working_dir (str, optional) – The working directory for the SegmentationObjects. Defaults to None.

• create (bool, optional) – Whether to create the SegmentationObjects if they do not exist. Defaults to False.

Returns

A dictionary with parameters for initializing SegmentationObjects.

Return type

dict

syconn.proc.skel_based_classifier module

syconn.proc.skel_based_classifier_helper module

syconn.proc.ssd_assembly module

syconn.proc.ssd_assembly.init_ssd_from_kzips(dir_path)

Placeholder function for initializing a SuperSegmentationDataset (SSD) from multiple k.zip files.

Parameters

dir_path (str) – Directory path containing the k.zip files.

Returns

None

syconn.proc.ssd_assembly.init_sso_from_kzip(path, load_as_tmp=True, sso_id=None)

Initializes a SuperSegmentationObject (SSO) from a k.zip file. The k.zip file should contain mesh files (‘sv.ply’, ‘mi.ply’, ‘sj.ply’, ‘vc.ply’), a meta dictionary (‘meta.pkl’), and optionally rendering locations (‘sample_locations.pkl’), a supervoxel graph (‘rag.bz2’), a skeleton representation (‘skeleton.pkl’), and an attribute dictionary (‘attr_dict.pkl’). Note: Currently broken to use .npy for rendering locations, fixed in python 3.7.

Parameters

• path (str) – Path to the k.zip file containing SSO data.

• load_as_tmp (bool, optional) – If True, the ‘working_dir’ and ‘version_dict’ in the meta.pkl dictionary are not passed to the SSO constructor. Instead, all versions will be set to ‘tmp’ and the working directory will be None. This is used to process the SSO independent of the working directory. Defaults to True.

• sso_id (int, optional) – ID of the SSO. If not provided, the function looks for the first scalar occurrence in path. Defaults to None.

Returns

Initialized SSO.

Return type

SuperSegmentationObject

Raises

ValueError – If ‘load_as_tmp’ is False and no SSO ID is found in the meta dictionary.

syconn.proc.ssd_proc module

syconn.proc.ssd_proc.aggregate_segmentation_object_mappings(ssd, obj_types, n_jobs=None, nb_cpus=None)

Populates the attributes of SuperSegmentationDataset (SSD) objects with mapping information for each object type specified in obj_types. Each object type must exist in the version dictionary of the SSD. This function is essential for mapping cellular structures to cells in the SyConn toolkit.

Parameters

• ssd (SuperSegmentationDataset) – The SSD whose attributes are to be populated.

• obj_types (List[str]) – List of object identifiers to be mapped to the cells. For example, [‘mi’, ‘sj’, ‘vc’].

• n_jobs (Optional[int]) – Number of jobs to be run in parallel. If not provided, it defaults to twice the total number of cores available.

• nb_cpus (Optional[int]) – Number of cores to be used per job when using BatchJob. If single node multiprocessing is used, this represents the total number of jobs.

syconn.proc.ssd_proc.apply_mapping_decisions(ssd, obj_types, n_jobs=None, nb_cpus=None)

Populates the attributes of SuperSegmentationDataset (SSD) objects with the specified object types. Each object type must exist in the version dictionary of the SSD and in the cell objects configuration of the SSD. This function is essential for applying mapping decisions to cells in the SyConn toolkit. It requires the prior execution of the aggregate_segmentation_object_mappings function.

Parameters

• ssd (SuperSegmentationDataset) – The SSD whose attributes are to be populated.

• obj_types (List[str]) – List of object identifiers to be mapped to the cells. For example, [‘mi’, ‘sj’, ‘vc’].

• n_jobs (Optional[int]) – Number of jobs to be run in parallel. If not provided, it defaults to twice the total number of cores available.

• nb_cpus (Optional[int]) – Number of cores to be used per job when using BatchJob. If single node multiprocessing is used, this represents the total number of jobs.

syconn.proc.ssd_proc.get_available_ssv_ids(ssd, n=2)

Generates available SuperSegmentationObject IDs. The function iterates through the range of the maximum SuperSegmentationObject ID plus the provided number ‘n’, and yields the IDs that are not already in use. The function stops when ‘n’ IDs have been generated.

Parameters

• ssd – The SuperSegmentationDataset to check for available IDs.

• n (int, optional) – The number of IDs to generate. Defaults to 2.

Yields

int – The next available SuperSegmentationObject ID.

syconn.proc.ssd_proc.init_ssv(ssv_id, sv_ids, ssd)

Initializes a SuperSegmentationObject and caches all relevant data. This function requires that the cell organelles and supervoxel SegmentationDatasets are already initialized. The function creates a new SuperSegmentationObject with the provided ID and supervoxel IDs, preprocesses it, and returns it.

Parameters

• ssv_id (int) – The ID for the new SuperSegmentationObject.

• sv_ids (List[int]) – The supervoxel IDs for the new SuperSegmentationObject.

• ssd (SuperSegmentationDataset) – The SuperSegmentationDataset that the new

• to. (SuperSegmentationObject belongs) –

Returns

The newly initialized SuperSegmentationObject.

Return type

SuperSegmentationObject

syconn.proc.ssd_proc.map_synssv_objects(synssv_version=None, log=None, nb_cpus=None, n_jobs=None, syn_threshold=None)

This function maps syn_ssv objects and merges their meshes for all SuperSegmentationObjects (SSOs) in a SuperSegmentationDataset (SSD). It stores the meshes with keys: ‘syn_ssv’, ‘syn_ssv_sym’, and ‘syn_ssv_asym’ (if synapse type is available). This operation may take a while.

Parameters

• synssv_version (str, optional) – String identifier for the syn_ssv

• None. (during the mapping of syn_ssv objects. Defaults to) –

• n_jobs (int, optional) – Number of jobs. Defaults to None.

• log (Logger, optional) – Logger for logging information. Defaults to None.

• nb_cpus (int, optional) – Number of CPUs for local multi-processing.

• None. –

• syn_threshold (float, optional) – Probability threshold applied

• None. –

syconn.proc.ssd_proc.map_synssv_objects_thread(args)

This function is a multi-threaded version of the ‘map_synssv_objects’ function. It maps syn_ssv objects and merges their meshes for a subset of SuperSegmentationObjects (SSOs) in a SuperSegmentationDataset (SSD).

Parameters

args (list) – A list containing the following elements: - ssv_obj_ids (list): List of SSO ids. - version (str): Version of the SSD. - version_dict (dict): Dictionary containing versions of the SSD. - working_dir (str): Working directory of the SSD. - ssd_type (str): Type of the SSD. - synssv_version (str): Version of the syn_ssv objects. - syn_threshold (float): Probability threshold applied during the mapping of syn_ssv objects.

syconn.proc.ssd_proc.mesh_proc_ssv(working_dir, version=None, ssd_type='ssv', nb_cpus=None)

This function caches the meshes of SuperSegmentationObjects (SSOs) in a SuperSegmentationDataset (SSD) locally with a specified number of CPUs in parallel.

Parameters

• working_dir (str) – Path to the working directory.

• version (str, optional) – Version identifier, like ‘spgt’ for spine ground truth SSD. Defaults to the SSD of the cellular SSVs. Defaults to None.

• ssd_type (str) – Type of the SSD. Default is ‘ssv’.

• nb_cpus (int, optional) – Number of CPUs for local multi-processing. Default is the total number of CPUs.

syconn.proc.ssd_proc.split_ssv(ssv, splitted_sv_ids)

Splits a SuperSegmentationObject into two separate SuperSegmentationObjects. The split is based on the provided supervoxel IDs. The function checks if the SuperSegmentationObject’s dataset is defined and if the provided supervoxel IDs are part of the original SuperSegmentationObject. If these conditions are met, the function generates two new SuperSegmentationObjects with new IDs and returns them.

Parameters

• ssv (SuperSegmentationObject) – The SuperSegmentationObject to be split.

• splitted_sv_ids (Iterable[int]) – The supervoxel IDs used to split the

• SuperSegmentationObject. –

Returns

The two new SuperSegmentationObjects resulting from the split.

Return type

Tuple[SuperSegmentationObject, SuperSegmentationObject]

syconn.proc.stats module

class syconn.proc.stats.FileTimer(working_dir, overwrite=False, add_detail_vols=False)

Bases: object

This class is a context decorator for timing. It stores the results as a dictionary in a pickle file. The class is used to track the execution time of several major steps of the analysis in the SyConn toolkit. The results are written as a dictionary to the file ‘.timing.pkl’ in the working directory. The timing data can be accessed after the run by initializing FileTimer with the output file.

Examples

ft = FileTimer(path_to_timings_pkl) # this is a dict with the step names as keys and the timings in seconds as values print(ft.timings)

Parameters

• working_dir (str) – The working directory where the timing file is stored.

• overwrite (bool) – If True, overwrites the existing timing file. Default is False.

• add_detail_vols (bool) – If True, adds detailed volume information. Default is False.

property dataset_mm3: float

Returns the size of the dataset in cubic millimeters. If the dataset size is not already calculated, it calls the prepare_vol_info method to calculate it.

Returns

The size of the dataset in cubic millimeters.

Return type

float

property dataset_nvoxels: float

Returns the size of the dataset in giga voxels. If the dataset size is not already calculated, it calls the prepare_vol_info method to calculate it.

Returns

The size of the dataset in giga voxels.

Return type

float

property dataset_shape: float

Returns the shape of the dataset in giga voxels. If the dataset shape is not already calculated, it calls the prepare_vol_info method to calculate it.

Returns

The shape of the dataset in giga voxels, equivalent to the data set size.

Return type

float

prepare_report()

Prepares a report of the timing data. It calculates the total time taken for the analysis and the time taken for each step. The report is returned as a string.

Returns

The report of the timing data.

Return type

str

prepare_vol_info()

Prepares the volume information of the dataset. It calculates the shape, number of voxels, and size in cubic millimeters of the dataset.

start(step_name)

Starts the timer for a specific step in the analysis.

Parameters

step_name (str) – The name of the step for which the timer is started.

stop()

Stops the timer for the current step in the analysis, calculates the time interval, and saves the timing data to the pickle file.

syconn.proc.stats.array2xls(dest_p, arr)

Writes a 2D array to an Excel file.

Parameters

• dest_p (str) – Path to the output Excel file.

• arr (list) – 2D list or array to write to the Excel file.

syconn.proc.stats.cluster_summary(train_d, train_l, valid_d, valid_l, fold, prefix='', pca=None, return_valid_pred=False)

This function creates a clustering summary and saves the results to a specified folder. The function performs classification using kNN and RFC on the PCA latent space. It also generates precision-recall curves and density plots in the PCA latent space.

Parameters

• train_d (np.array) – Training data in feature space.

• train_l (np.array) – Sparse labels for the training data.

• valid_d (np.array) – Validation data in feature space.

• valid_l (np.array) – Sparse labels for the validation data.

• fold (str) – Destination folder where the results will be saved.

• prefix (str, optional) – Prefix for the data. Default is an empty string.

• pca (PCA, optional) – Pre-fitted PCA object to use to project data of train_d. Default is None.

• return_valid_pred (bool, optional) – If True, returns the predictions for the validation data. Default is False.

Returns

Predictions for the validation data if return_valid_pred is True. Otherwise, None.

Return type

np.array

syconn.proc.stats.fscore(rec, prec, beta=1.0)

Calculates f-score with beta value

Parameters

• rec (np.array) – recall

• prec (np.array) – precision

• beta (float) – weighting of precision

Returns

f-score

Return type

np.array

syconn.proc.stats.hist(vals, labels=None, dest_path=None, axis_labels=None, x_lim=None, y_lim=None, y_log_scale=False, ls=22, color=None, **kwargs)

Plots a histogram of the given values.

Parameters

• vals (list) – Values to plot.

• labels (list, optional) – Labels for the values. Defaults to None.

• dest_path (str, optional) – Path to save the plot. Defaults to None.

• axis_labels (list, optional) – Labels for the x and y axes. Defaults to None.

• x_lim (tuple, optional) – Limits for the x-axis. Defaults to None.

• y_lim (tuple, optional) – Limits for the y-axis. Defaults to None.

• y_log_scale (bool, optional) – If True, uses a log scale for the y-axis. Defaults to False.

• ls (int, optional) – Label size. Defaults to 22.

• color (str, optional) – Color for the histogram. Defaults to None.

• **kwargs – Additional keyword arguments for seaborn’s distplot function.

syconn.proc.stats.model_performance(proba, labels, model_dir=None, prefix='', n_labels=3, fscore_beta=1, target_names=None, add_text='')

Evaluates the performance of a model and generates a precision-recall plot. The function also logs the classification report and accuracy score.

Parameters

• proba (np.array) – Model probabilities.

• labels (np.array) – True labels.

• model_dir (str, optional) – Directory to save the precision-recall plot and performance metrics.

• prefix (str, optional) – Prefix for the saved files.

• n_labels (int, optional) – Number of labels in the data. Defaults to 3.

• fscore_beta (int, optional) – Beta parameter for F-score calculation. Defaults to 1.

• target_names (list, optional) – Names of the target classes. Defaults to None.

• add_text (str, optional) – Additional text to add to the saved performance metrics file. Defaults to ‘’.

syconn.proc.stats.model_performance_predonly(y_pred, y_true, model_dir=None, prefix='', target_names=None, labels=None)

Evaluates the performance of a model using only predictions and true labels. The function logs the classification report and accuracy score.

Parameters

• y_pred (np.array) – Model predictions.

• y_true (np.array) – True labels.

• model_dir (str, optional) – Directory to save the performance metrics.

• prefix (str, optional) – Prefix for the saved files.

• target_names (list, optional) – Names of the target classes. Defaults to None.

• labels (np.array, optional) – Array of unique labels. Defaults to None.

syconn.proc.stats.plot_pr(precision, recall, title='', r=[0.67, 1.01], legend_labels=None, save_path=None, nbins=5, colorVals=None, xlabel='Recall', ylabel='Precision', l_pos='lower left', legend=True, r_x=[0.67, 1.01], ls=22, xtick_labels=())

Plots the precision-recall curve.

Parameters

• precision (list) – Precision values.

• recall (list) – Recall values.

• title (str, optional) – Title for the plot. Defaults to ‘’.

• r (list, optional) – Range for the y-axis. Defaults to [0.67, 1.01].

• legend_labels (list, optional) – Labels for the legend. Defaults to None.

• save_path (str, optional) – Path to save the plot. Defaults to None.

• nbins (int, optional) – Number of bins for the histogram. Defaults to 5.

• colorVals (list, optional) – Colors for the plot lines. Defaults to None.

• xlabel (str, optional) – Label for the x-axis. Defaults to ‘Recall’.

• ylabel (str, optional) – Label for the y-axis. Defaults to ‘Precision’.

• l_pos (str, optional) – Position of the legend. Defaults to ‘lower left’.

• legend (bool, optional) – If True, includes a legend. Defaults to True.

• r_x (list, optional) – Range for the x-axis. Defaults to [0.67, 1.01].

• ls (int, optional) – Label size. Defaults to 22.

• xtick_labels (tuple, optional) – Labels for the x-axis ticks. Defaults to ().

syconn.proc.stats.projection_pca(ds_d, ds_l, dest_path, pca=None, colors=None, do_3d=True, target_names=None)

This function performs PCA on the given dataset and generates a density plot of the first two principal components. If do_3d is True, it also generates density plots of the first and third, and second and third principal components.

Parameters

• ds_d (np.array) – Data in feature space, e.g. (#data, #feature).

• ds_l (np.array) – Sparse labels, i.e. (#data, 1).

• dest_path (str) – File name of the plot.

• pca (PCA, optional) – Pre-fitted PCA object to use to project data of ds_d. Default is None.

• colors (list, optional) – List of colors to use for the plot. Default is None.

• do_3d (bool, optional) – If True, generates 3D plots. Default is True.

• target_names (list, optional) – List of target names. Default is None.

syconn.proc.stats.projection_tSNE(ds_d, ds_l, dest_path, colors=None, target_names=None, do_3d=False, cmap_ident='prism', **tsne_kwargs)

Performs t-SNE visualization on the given data and saves the resulting plot to the specified path. It supports both 2D and 3D visualization. The function also handles memory errors by downsampling the data.

Parameters

• ds_d (np.array) – Data in feature space, shape (#data, #feature).

• ds_l (np.array) – Sparse labels, shape (#data, 1).

• dest_path (str) – File path where the plot will be saved.

• pca (PCA) – Prefitted PCA object to use to project data of ds_d.

• colors (list, optional) – List of colors to use for different labels. If not provided, colors

• generated. (are) –

• target_names (list, optional) – List of target names corresponding to labels. If not provided,

• used. (labels are) –

• do_3d (bool, optional) – If True, performs 3D visualization. Default is False.

• cmap_ident (str, optional) – Identifier for the colormap to use. Default is ‘prism’.

• **tsne_kwargs – Arbitrary keyword arguments for the t-SNE model.

Returns

Fitted t-SNE model.

Return type

tsne (TSNE_sc)