syconn.reps package

Submodules

syconn.reps.super_segmentation_dataset module
syconn.reps.super_segmentation_object module
syconn.reps.super_segmentation_helper module
syconn.reps.segmentation module
syconn.reps.segmentation_helper module
syconn.reps.super_segmentation module
syconn.reps.rep_helper module

syconn.reps.super_segmentation_dataset module

class syconn.reps.super_segmentation_dataset.SuperSegmentationDataset(working_dir=None, version=None, ssd_type='ssv', version_dict=None, sv_mapping=None, scaling=None, config=None, sso_caching=False, sso_locking=False, create=False, sd_lookup=None, cache_properties=None, overwrite=False)[source]

Bases: syconn.reps.rep_helper.SegmentationBase

Represents a set of agglomerated supervoxels, which are represented by SegmentationObject instances, and provides methods for accessing and manipulating their data.

Examples

After initializing with run_create_neuron_ssd and subsequent analysis, SSV properties can be loaded via load_cached_data with keys corresponding to ssv_ids:

‘id’: ID array, identical to ssv_ids.

‘bounding_box’: Bounding box of every SSV.

‘size’: Number of voxels of each SSV.

‘rep_coord’: Representative coordinates for each SSV.

‘sv’: Supervoxel IDs for every SSV.

‘sample_locations’: Rendering locations for each SSV.

‘celltype_cnn_e3’: Celltype classifications based on elektronn3 CMN.

‘celltype_cnn_e3_probas’: Celltype logits as an array (M, C).

‘syn_ssv’: Synapse IDs assigned to each SSV.

‘syn_sign_ratio’: Area-weighted ratio of symmetric synapses.

‘sj’: Synaptic junction object IDs mapped to each SSV.

‘mapping_sj_ids’: Synaptic junction objects overlapping with SSVs.

‘mapping_sj_ratios’: Overlap ratio of synaptic junctions.

‘vc’: Vesicle clouds mapped to each SSV.

‘mapping_vc_ids’: Vesicle cloud objects overlapping with SSVs.

‘mapping_vc_ratios’: Overlap ratio of vesicle clouds.

‘mi’: Mitochondria mapped to each SSV.

‘mapping_mi_ids’: Mitochondria objects overlapping with SSVs.

‘mapping_mi_ratios’: Overlap ratio of mitochondria.

Initialize SuperSegmentationDataset and explore attributes:

import numpy as np from syconn.reps.super_segmentation import * ssd = SuperSegmentationDataset(working_dir=’~/SyConn/example_cube1/’) n_synapses = [len(ssv.syn_ssv) for ssv in ssd.ssvs] path_length = [ssv.total_edge_length() for ssv in ssd.ssvs] syn_densities = np.array(n_synapses) / np.array(path_length) print(np.mean(syn_densities), np.std(syn_densities))

Obtain total number of synapses per cell type:

celltypes = ssd.load_numpy_data(‘celltype_cnn_e3’) n_synapses = np.array([len(el) for el in ssd.load_numpy_data(‘syn_ssv’)]) n_synapes_per_type = {ct: np.sum(n_synapses[celltypes==ct])

for ct in range(np.max(celltypes))}

print(n_synapes_per_type)

sso_caching

Enables caching mechanisms in SuperSegmentationObjects returned via get_super_segmentation_object.

Type: bool

sso_locking

If True, locking is enabled for SSV files.

Type: bool

Parameters

working_dir (Optional[str]) – Path to the working directory.
version (Optional[str]) – Indicates the version of the dataset, e.g. ‘0’, ‘groundtruth’ etc.
ssd_type (str) – Changes the directory prefix the dataset is stored in.
version_dict (Optional[Dict[str, str]]) – Dictionary with versions of other dataset types sharing the same working directory.
sv_mapping (Optional[Union[Dict[int, int], str]]) – Dictionary mapping supervoxel IDs to their super-supervoxel ID.
scaling (Optional[Union[List, Tuple, np.ndarray]]) – Array defining the voxel size in XYZ. Default is from config.yml.
config (DynConfig) – Config object, see ~syconn.handler.config.DynConfig. Will be copied and fixed by setting ~syconn.handler.config.DynConfig.fix_config to True.
create (bool) – Create folder if True.
sd_lookup (Optional[Dict[str, SegmentationDataset]]) – Lookup dict for SegmentationDataset, enabling usage of property cache arrays for all attributes specified in property_cache during init. of SegmentationDataset.
cache_properties (Optional[List[str]]) – Use numpy cache arrays to populate specified object properties when initializing SuperSegmentationObject via get_super_segmentation_object.
overwrite (bool) – Overwrite existing data if True.

apply_mergelist(sv_mapping)[source]

Applies a supervoxel agglomeration to the dataset.

Parameters: sv_mapping (Union[Dict[int, int], str]) – Agglomeration mapping or path to the mapping file.

property config: syconn.handler.config.DynConfig

Retrieves the configuration object containing dataset-specific parameters, ensuring all relevant details from the previous documentation are retained.

Returns: An object with all dataset-specific parameters.
Return type: DynConfig

create_mapping_lookup_reverse()[source]: Creates a data structure for efficient look-ups from supervoxel ID to cell ID, see syconn.backend.storage.BinarySearchStore.

enable_property_cache(property_keys)[source]

Enables caching for specified properties.

Parameters: property_keys (List[str]) – List of property keys to cache. Numpy cache arrays must exist.

get_segmentationdataset(obj_type)[source]

Retrieves the SegmentationDataset for a given object type.

Parameters: obj_type (str) – Object type identifier.
Returns: Corresponding SegmentationDataset.
Return type: SegmentationDataset

get_super_segmentation_object(obj_id, new_mapping=False, caching=None, create=False)[source]

Factory method for creating SuperSegmentationObject instances.

Parameters

obj_id (Union[int, Iterable[int]]) – ID(s) of the super-supervoxel(s) to instantiate. Can also be an iterable.
new_mapping (bool) – If True, uses the latest supervoxel agglomeration. Returns SuperSegmentationObject based on mapping_dict.
caching (Optional[bool], default False) – Enables caching of various attributes.
create (bool) – If True, creates the directory structure for the super-supervoxel in the dataset folder structure.

Notes

caching parameter’s default value updated to False, as per PS 20Feb2019.

Returns: The requested SuperSegmentationObject instance(s) corresponding to the given obj_id (int or Iterable[int]).
Return type: Union[SuperSegmentationObject, List[SuperSegmentationObject]]

load_mapping_dict()[source]: Loads the mapping dictionary from the specified .pkl file.

load_numpy_data(prop_name, allow_nonexisting=True, suppress_warning=False)[source]

Loads a numpy array of a cached property.

Parameters

prop_name (str) – Identifier for the requested cache array. Ordering of the array is the same as ssv_ids.
allow_nonexisting (bool) – If False, will raise an error for missing numpy files.
suppress_warning (bool) – If True, suppresses the warning if the property does not exist.

Returns

Numpy array of the cached property, or None if not: found.

Return type

Optional[np.ndarray]

load_version_dict()[source]: Loads the version dictionary from the specified .pkl file.

property mapping_dict: Dict[int, numpy.ndarray]

Retrieves the dictionary containing the supervoxel IDs for each super-supervoxel.

Returns: A dictionary where each key is the super-supervoxel identifier and the corresponding value is an array of supervoxel IDs.
Return type: Dict[int, np.ndarray]

property mapping_dict_exists: bool

Checks if the mapping dictionary exists (upper-supervoxel ID to supervoxel IDs).

Returns: True if the mapping dictionary exists, False otherwise.
Return type: bool

property mapping_dict_path: str

Retrieves the path to the mapping dictionary file.

Returns: Path to the mapping dictionary .pkl file.
Return type: str

property mapping_lookup_reverse: syconn.backend.storage.BinarySearchStore

Retrieves the BinarySearchStore for efficient look-ups from supervoxel ID to cell ID.

Returns: Lookup store object.
Return type: BinarySearchStore

property mapping_lookup_reverse_path: str

Retrieves the path to the data structure storing the lookup from supervoxel ID to cell ID.

Returns

Path to the data structure that stores the lookup from supervoxel ID: to cell ID.

Return type

str

property path: str

Retrieves the full path to the dataset directory.

Returns

Full path to the dataset directory ensuring to include all relevant: information and resolve any conflicts with the old docstring.

Return type

str

save_dataset_deep(extract_only=False, attr_keys=(), n_jobs=None, nb_cpus=None, use_batchjob=True, new_mapping=True)[source]

Saves attributes of all SSVs within the given SSD and computes properties like size and representative coordinate. The order of ssv_ids may change each run. Populates the sv_ids attribute of all SSVs. The behavior and order of operations might differ from run to run. See save_dataset_deep().

Parameters

extract_only (bool) – Only cache attributes attr_keys from attribute dict. If True, adds suffix ‘_sel’ to the cache array file names, updates will not apply to the load_cached_data() method.
attr_keys (Iterable[str]) – Attributes to cache, only used if extract_only is True.
n_jobs (Optional[int]) – Enables batch job system if set. Requires any string to enable batch job system, will be replaced by a global flag soon.
nb_cpus (Optional[int]) – Number of CPUs per worker.
use_batchjob (bool) – If True, uses batchjob processing instead of local multiprocessing.
new_mapping (bool) – If True, applies new supervoxel agglomeration.

save_dataset_shallow(overwrite=False)[source]

Saves version_dict, mapping_dict.

Parameters

overwrite (bool) – If True, allows overwriting existing files. Do not replace
files. (existing) –

save_mapping_dict()[source]: Save the mapping dictionary to a .pkl file.

save_version_dict()[source]: Saves the version dictionary to a .pkl file.

property scaling: numpy.ndarray

Retrieves the voxel size in nanometers (XYZ).

The default values are obtained from the config.yml file and can be accessed through the config.

Returns: Voxel size in nanometers as a numpy array.
Return type: np.ndarray

property ssv_ids: numpy.ndarray

Retrieves the super-supervoxel IDs that are part of this SuperSegmentationDataset object.

Returns: Array of super-supervoxel IDs.
Return type: np.ndarray

property ssvs: Generator[syconn.reps.super_segmentation_object.SuperSegmentationObject, None, None]

Generator of SuperSegmentationObject objects which are part of this SuperSegmentationDataset object.

Yields: SuperSegmentationObject – SuperSegmentationObject instances.
Return type: Generator[SuperSegmentationObject, None, None]

store_path_densities_seg_objs(obj_type, compartments_of_interest=None, ax_pred_key='axoness_avg10000', overwrite=False, nb_cpus=None)[source]

Stores path densities of all cells in the dataset for a given sub-cellular structure. The order corresponds to ssv_ids.

Parameters

obj_type (str) – Key to any available sub-cellular structure.
compartments_of_interest (Optional[list]) – Compartments to include in the calculation. Specify axon: 1, dendrite: 0, soma: 2 for filtering.
ax_pred_key (Optional[str]) – Key of compartment prediction stored in the skeleton. Defaults to ‘axoness_avg10000’ if compartments_of_interest is set.
overwrite (Optional[bool]) – If True, overwrites the existing file. Defaults to False.
nb_cpus (Optional[int]) – Number of CPUs to use for the computation. Defaults to None.

store_total_edge_lengths(ax_pred_key='axoness_avg10000', overwrite=False, nb_cpus=None)[source]

Stores total edge lengths of all the cells in this dataset in nanometers. Same ordering as ssv_ids.

Parameters

ax_pred_key (Optional[str]) – Key of compartment prediction stored in the skeleton attribute.
overwrite (Optional[bool]) – If True, overwrites the existing total_edge_lengths.npy file. Defaults to False.
nb_cpus (Optional[int]) – Number of CPUs to use for the computation. Defaults to None.

sv2ssv_ids(ids, nb_cpus=1)[source]

Queries the cell ID for a given array of supervoxel IDs using the mapping_lookup_reverse.

Parameters

ids (np.ndarray) – Array of unique IDs to find the corresponding cell IDs.
nb_cpus (int) – Number of CPUs to use for the query. IDs that are not in
dictionary. (sv_ids will not be added to the output) –

Returns

Dictionary with supervoxel ID as key and cell ID as value.

Return type

Dict[int, int]

property sv_ids: numpy.ndarray

Retrieves a flat array of supervoxel IDs that are part of the cells (ssv_ids) in this SuperSegmentationDataset object.

Returns: Array of supervoxel IDs.
Return type: np.ndarray

property type: str

Retrieves the type of the underlying supervoxel objects.

This method returns the class type of the supervoxel objects used in the SuperSegmentationObject.

Returns: The type of the supervoxel objects as represented in the SuperSegmentationObject.
Return type: str

syconn.reps.super_segmentation_object module

class syconn.reps.super_segmentation_object.SuperSegmentationObject(ssv_id, version=None, version_dict=None, working_dir=None, create=False, sv_ids=None, scaling=None, object_caching=True, voxel_caching=True, mesh_caching=True, view_caching=False, config=None, nb_cpus=1, enable_locking=False, enable_locking_so=False, ssd_type=None, ssd=None, sv_graph=None)[source]

Bases: syconn.reps.rep_helper.SegmentationBase

Class instances represent individual neuron reconstructions, defined by a list of agglomerated supervoxels (see SegmentationObject).

This class is used to create a cell reconstruction object after successful execution of run_create_neuron_ssd. It can be instantiated directly with an SSV ID or via the SuperSegmentationDataset. The class provides methods to access and manipulate various attributes and properties related to the neuron reconstruction, such as meshes, skeletons, and segmentation objects for different cellular components.

Examples

This class can be used to create a cell reconstruction object after successful executing run_create_neuron_ssd() as follows:

from syconn import global_params
# import SuperSegmentationObject and SuperSegmentationDataset
from syconn.reps.super_segmentation import *
# set the current working directory SyConn-wide
global_params.wd = '~/SyConn/example_cube1/'

ssd = SuperSegmentationDataset()
cell_reconstr_ids = ssd.ssv_ids
# call constructor explicitly ...
cell = SuperSegmentationObject(cell_reconstr_ids[0])
# ... or via the SuperSegmentationDataset - both contain the same data
cell = ssd.get_super_segmentation_object(cell_reconstr_ids[0])
# inspect existing attributes
cell.load_attr_dict()
print(cell.attr_dict.keys())

To cache SegmentationObject attributes use the cache_properties argument during initialization of the SegmentationDataset and pass it on to the SuperSegmentationDataset instantiation:

sd_mi = SegmentationDataset(obj_type=’mi’, cache_properties=[‘rep_coord’]) ssd = SuperSegmentationDataset(sd_lookup=dict(mi=sd_mi)) ssv = ssd.get_super_segmentation_object(ssd.ssv_ids[0]) # SegmentationObject from mis don’t require loading rep_coord # from its storage file. for mi in ssv.mis:

rc = mi.rep_coord # normally this requires to load the attribute dict storage file.

Subsequent analysis steps (see the SyConn/scripts/example_run/start.py) augment the cell reconstruction with more properties:

# to iterate over all cell reconstructions use the generator:
for cell in ssd.ssvs:
    # e.g. collect some analysis results
    cell.load_attr_dict()
    n_synapses = len(cell.syn_ssv)
    celltype = cell.attr_dict["celltype_cnn_e3"]
    ...
    # write out cell mesh
    cell.mesh2kzip('~/cell{}_mesh.k.zip'.format(cell.id))
    # write out cell mesh and meshes of all existing cell organelles
    cell.meshes2kzip('~/cell{}_meshes.k.zip'.format(cell.id))
    # color the cell mesh according to a semantic prediction
    cell.semseg2mesh(semseg_key='spiness', dest_path='~/cell{}_spines.k.zip'.format(cell.id))

See also SyConn/docs/api.md (WIP).

attr_dict

Attribute dictionary which serves as a general-purpose container. Accessed via the AttributeDict interface. After successfully executing syconn.exec.exec_init.run_create_neuron_ssd() and subsequent analysis steps (see the SyConn/scripts/example_run/start.py) the following keys are present in attr_dict:

‘id’: ID array, identical to ssv_ids. All other properties have the same ordering as this array, i.e. if SSV with ID 1234 has index 42 in the ‘id’-array you will find its properties at index 42 in all other cache-arrays.

‘bounding_box’: Bounding box of every SSV.

‘size’: Number voxels of each SSV.

‘rep_coord’: Representative coordinates for each SSV.

‘sv’: Supervoxel IDs for every SSV.

‘sample_locations’: Lists of rendering locations for each SSV. Each entry is a list (length corresponds to the number of supervoxels) of coordinate arrays for the corresponding SSV.

‘celltype_cnn_e3’: Celltype classifications based on the elektronn3 CMN.

‘celltype_cnn_e3_probas’: Celltype logits for the different types as an array of shape (M, C; M: Number of predicted random multi-view sets, C: Number of classes). In the example run there are currently 9 predicted classes: STN=0, DA=1, MSN=2, LMAN=3, HVC=4, GP=5, FS=6, TAN=7, INT=8.

‘syn_ssv’: Synapse IDs assigned to each SSV.

‘syn_sign_ratio’: Area-weighted atio of symmetric synapses, see syn_sign_ratio().

‘sj’: Synaptic junction object IDs which were mapped to each SSV. These are used for view rendering and also to generate the ‘syn_ssv’ objects in combination with contact sites (see corresponding section in the documentation).

‘mapping_sj_ids’: Synaptic junction objects which overlap with the respective SSVs.

‘mapping_sj_ratios’: Overlap ratio of the synaptic junctions.

‘vc’: Vesicle clouds mapped to each SSV.

‘mapping_vc_ids’: Vesicle cloud objects which overlap with the respective SSVs.

‘mapping_vc_ratios’: Overlap ratio of the vesicle clouds.

‘mi’: Mitochondria mapped to each SSV.

‘mapping_mi_ids’: Mitochondria objects which overlap with the respective SSVs.

‘mapping_mi_ratios’: Overlap ratio of the mitochondria.

skeleton

The skeleton representation of this super-supervoxel. Keys which are currently in use:

‘nodes’: Array of the node coordinates (in nanometers).

‘edges’: Edges between nodes.

‘diameters’: Estimated cell diameter at every node.

various node properties, e.g. ‘axoness’ and ‘axoness_avg10000’. Check the

available keys sso.skeleton.keys() of an initialized SuperSegmentationObject object sso after loading the skeleton (sso.load_skeleton()).

enable_locking_so: Locking flag for all syconn.reps.segmentation.SegmentationObject assigned to this object (e.g. SV, mitochondria, vesicle clouds, …)

nb_cpus: Number of cpus for parallel jobs. will only be used in some processing steps.

view_dict: A dictionary for caching 2D projection views. Those are stored as a numpy array of shape (M, N, CH, x, y). M: Length of sample_locations and has the same ordering; N: Number of views per location; CH: Number of channels (1 for glia prediction containing only the cell shape and 4 for neuron analysis containing cell and cell organelle shapes. Stored at view_path and accessed via the CompressedStorage interface.

version_dict: A dictionary which contains the versions of other dataset types which share the same working directory. Defaults to the Versions entry in the config.yml file.

aggregate_segmentation_object_mappings(obj_types, save=False)[source]

Aggregates mapping information of cellular organelles from the supervoxels of the SSV. After this step, apply_mapping_decision() can be called to apply final assignments.

Examples

A mitochondrion can extend over multiple supervoxels, so it will overlap with all of them partially. Here, the overlap information of all supervoxels assigned to this SSV will be aggregated.

Parameters

obj_types (List[str]) – A list of cell organelle types to process.
save (bool) – If True, saves the attribute dictionary at the end.

Returns

None. This method aggregates the mapping information.

apply_mapping_decision(obj_type, correct_for_background=True, lower_ratio=None, upper_ratio=None, sizethreshold=None, save=True)[source]

Applies mapping decisions of cellular organelles to the SSV object based on overlap ratios. A SegmentationObject in question is assigned to this SuperSegmentationObject if they share the highest overlap. For more details see SyConn/docs/object_mapping.md. Default parameters for the mapping will be taken from the config.yml file.

Parameters

obj_type (str) – Type of SegmentationObject which are to be mapped.
correct_for_background (bool) – If True, ignores the background ID during mapping.
lower_ratio (Optional[float]) – The minimum overlap ratio required for objects to be mapped.
upper_ratio (Optional[float]) – The maximum overlap ratio allowed for objects to be mapped.
sizethreshold (Optional[float]) – The minimum voxel size of an object to be considered for mapping, objects below will be ignored.
save (bool) – If True, saves the attribute dictionary after mapping.

Returns: None. Applies the mapping decisions directly to the SSV.

property attr_dict_exists: bool

Checks if an attribute dictionary file exists at attr_dict_path for this SuperSegmentationObject.

Returns: True if the attribute dictionary file exists, False otherwise.
Return type: bool

property attr_dict_path: str

Path to the attribute storage. attr_dict can be loaded from here.

Return type: str

attr_exists(attr_key)[source]

Checks if a specified attribute exists for this SuperSegmentationObject.

Parameters: attr_key (str) – The attribute key to check.
Returns: True if the attribute exists in attr_dict, False otherwise.
Return type: bool

attr_for_coords(coords, attr_keys, radius_nm=None, k=1)[source]

Queries skeleton node attributes at specified coordinates. Any attribute stored in self.skeleton is supported. If a radius_nm is provided, it will assign the majority attribute value.

Parameters

coords (np.array) – Unscaled voxel coordinates, format - [N, 3].
radius_nm (Optional[float]) – If None, only the attribute of the nearest node is used, otherwise, the majority attribute value is assigned.
attr_keys (List[str]) – Identifier for the attribute.
k (int) – Number of nearest neighbors, applicable only if radius_nm is None.

Returns

Returns a list of the same length as coords. For each coordinate in coords, it: returns the majority label within radius_nm or [-1] if the Key does not exist.

Return type

list

average_node_axoness_views(**kwargs)[source]

Applies a sliding window averaging along the axon predictions stored at the nodes of the skeleton. See _average_node_axoness_views() for more details. This will call save_skeleton().

Parameters: **kwargs – Key word arguments used for

:param _average_node_axoness_views().:

axoness2mesh(dest_path, k=1, pred_key_appendix='')[source]

Deprecated. Writes the per-location CMN axon predictions to a kzip file. See semseg2mesh().

Parameters

dest_path (str) – Path to the kzip file.
k (int) – Number of nearest neighbors used for the majority vote.
pred_key_appendix (str) – Key to load specific predictions.

axoness_for_coords(coords, radius_nm=4000, pred_type='axoness')[source]

Returns the majority label for given coordinates within a specified radius. This method is not limited to axoness, it supports any attribute stored in self.skeleton.

Parameters

coords (np.ndarray) – Voxel coordinates. These should be unscaled.
radius_nm (float) – The search radius specified in nanometers.
pred_type (str) – The type of prediction to retrieve.

Returns

An array of the same length as coords. Each element corresponds to the majority label within the specified radius for the corresponding coordinate.

Return type

np.ndarray

property bounding_box: List[numpy.ndarray]

Retrieves the bounding box of this SuperSegmentationObject.

Returns: The bounding box represented by two numpy arrays.
Return type: List[np.ndarray]

calculate_bounding_box()[source]: Calculates the bounding box and size of this SuperSegmentationObject (refer bounding_box and size).

calculate_size()[source]: Calculates the size (number of voxels) of this SuperSegmentationObject, referenced as size.

calculate_skeleton(force=False, **kwargs)[source]

Merges existing supervoxel skeletons or calculates them from scratch using create_sso_skeletons_wrapper() if allow_ssv_skel_gen=True. Skeleton will be saved at skeleton_path.

Parameters

force (bool) – If True, skips load_skeleton() and forces the calculation of
data. (the skeleton regardless of existing) –

celltype(key=None)[source]

Retrieves the cell type classification result. If key is specified, it returns the corresponding value loaded by lookup_in_attribute_dict(). Otherwise, the default CMN model is used.

Parameters: key (Optional[str]) – The key where the classification result is stored. If None, the default key is used.
Return type: int
Returns: The cell type classification result.

certainty_celltype(pred_key=None)[source]

Estimates the certainty of the celltype prediction.

If is_logit is True, generates pseudo-probabilities from the input using softmax.

Sums the evidence per class and (re-)normalizes.

Computes the entropy, scales it with the maximum entropy (equal probabilities) and subtracts it from 1.

Note

See certainty_estimate()

Parameters: pred_key (Optional[str]) – Key of classification results (one C-class probability vector for every multi-view sample). pred_key + '_probas' must exist in attr_dict.
Returns: Certainty measure based on the entropy of the cell type logits.
Return type: float

clear_cache()[source]

Clears the following cached data related to the SSV:

voxels
voxels_xy_downsampled
sample_locations
_objects
_views
skeleton
_meshes

Returns: None. The method clears various cached attributes of the SSV.

cnn_axoness2skel(**kwargs)[source]

Maps CNN axoness predictions to the skeleton.

Parameters: **kwargs – Additional keyword arguments.

property compartment_meshes: dict

Retrieves the compartment mesh storage for this SuperSegmentationObject.

Returns: A dictionary representing the meshes of each compartment in the SuperSegmentationObject.
Return type: dict

property config: syconn.handler.config.DynConfig

Retrieves the configuration object containing all dataset-specific parameters. Refer to DynConfig for more details.

Returns: The configuration object.
Return type: DynConfig

copy2dir(dest_dir, safe=True)[source]

Copies the content at ssv_dir to another directory.

Example

To copy the data of this SSV object (ssv_orig) to another, yet not existing SSV (ssv_target), call ssv_orig.copy2dir(ssv_target.ssv_dir). All files contained in the directory py:attr:~ssv_dir of ssv_orig will be copied to ssv_target.ssv_dir.

Parameters

dest_dir (str) – The destination directory where the SSV data will be copied.
safe (bool) – If True, does not overwrite existing data.

Returns

None. The method copies the SSV data to the specified directory.

property dense_kzip_ids: Dict[str, int]

?

Return type: Dict[str, int]

property edgelist_path: str

Identifier of SSV graph

Return type: str

export2kzip(dest_path, attr_keys=('skeleton',), rag=None, sv_color=None, individual_sv_meshes=True, object_meshes=None, synssv_instead_sj=True)[source]

Writes the SuperSegmentationObject to a KNOSSOS loadable k.zip file. This includes the mergelist, its meshes, dataset specific information and extra data (attr_keys). The range of values for the exported object lies within 0 to 255. The saved SSO can also be reloaded as an SSO instance.

Notes

Will not invoke load_attr_dict().

Parameters

dest_path (str) – The destination path for the k.zip file.
attr_keys (Iterable[str]) – Currently allowed: ‘sample_locations’, ‘skeleton’, ‘attr_dict’, ‘rag’.
rag (Optional[nx.Graph]) – The region adjacency graph of the SuperSegmentationObject.
sv_color (Optional[np.ndarray]) – Supervoxel colors. Array with RGBA (0…255) values or None to use default values.
individual_sv_meshes (bool) – If True, exports individual supervoxel meshes.
object_meshes (Optional[tuple]) – Default to subcellular organelles defined in config.yml (‘process_cell_organelles’).
synssv_instead_sj (bool) – If True, uses ‘syn_ssv’ objects instead of ‘sj’.

Returns

None

get_seg_dataset(obj_type)[source]

Factory method for the SegmentationDataset of a specified obj_type.

Parameters: obj_type (str) – Type of the requested SegmentationDataset.
Returns: The SegmentationDataset of type obj_type, sharing the same working directory as this SSV object.
Return type: SegmentationDataset

get_seg_obj(obj_type, obj_id)[source]

Factory method for creating a SegmentationObject of a specific type and ID.

Parameters: obj_type (str) – Type of the requested SegmentationObject from

:param SegmentationObject.: :type obj_id: int :param obj_id: ID of the requested object. :type obj_id: int

Returns: The created SegmentationObject of type obj_type, sharing the same working directory as this SSV object.
Return type: SegmentationObject

get_seg_objects(obj_type)[source]

Factory method for creating SegmentationObject`s of a specified `obj_type.

Parameters: obj_type (str) – Type of requested

:param :class:`~syconn.reps.segmentation.SegmentationObject`s.:

Returns: A list of SegmentationObject`s of the specified `obj_type, sharing the same working directory as this SSV object.
Return type: List[SegmentationObject]

get_spine_compartments(semseg_key='spiness', k=1, min_spine_cc_size=None, dest_folder=None)[source]

Retrieves connected components of vertex spine predictions using a specified semantic segmentation key and k number of nearest neighbors for majority label vote. If a connected component has a minimum number of vertices specified by min_spine_cc_size, it’s considered valid. If dest_folder is specified, the mean location and size of the head and neck components are stored as numpy array files in this folder.

Parameters

semseg_key (str) – The key of the semantic segmentation used for spine prediction.
k (int) – Number of nearest neighbors for smoothing of classification results.
min_spine_cc_size (Optional[int]) – Minimum number of vertices to consider a connected component a valid object.
dest_folder (Optional[str]) – Default is None, else provide a path (str) to a folder. The mean location and size of the head and neck connected components will be stored as numpy array file (npy).

Return type

Tuple[ndarray, ndarray, ndarray, ndarray]

Returns

A tuple containing arrays of neck locations, neck sizes, head locations, and head sizes. Location and size arrays have the same ordering.

gliapred2mergelist(dest_path=None, thresh=None, pred_key_appendix='')[source]

Writes glia predictions to a mergelist within a k.zip file.

Parameters

dest_path (Optional[str]) – The destination path for the k.zip file.
thresh (Optional[float]) – The threshold for classifying glia.
pred_key_appendix (str) – An appendix to the prediction key.

Returns

None

gliapred2mesh(dest_path=None, thresh=None, pred_key_appendix='')[source]

Writes glia predictions as a mesh to a k.zip file.

Parameters

dest_path (Optional[str]) – The destination path for the k.zip file.
thresh (Optional[float]) – The threshold for classifying glia.
pred_key_appendix (str) – An appendix to the prediction key.

Returns

None

gliaprobas2mesh(dest_path=None, pred_key_appendix='')[source]

Writes glia probabilities as a mesh to a k.zip file.

Parameters

dest_path (Optional[str]) – The destination path for the k.zip file.
pred_key_appendix (str) – An appendix to the prediction key.

Returns

None

gliasplit(recompute=False, thresh=None, verbose=False, pred_key_appendix='')[source]

Splits the supervoxels into glia and non-glia based on predictions.

Parameters

recompute (bool) – If True, recomputes the split even if it already exists.
thresh (Optional[float]) – The threshold for classifying glia.
verbose (bool) – If True, logs additional information.
pred_key_appendix (str) – An appendix to the prediction key.

Returns

None

gliasplit2mesh(dest_path=None, pred_key_appendix='')[source]

Writes the result of glia splitting to meshes in a k.zip file.

Parameters

dest_path (Optional[str]) – The destination path for the k.zip file.
pred_key_appendix (str) – An appendix to the prediction key.

Returns

None

property id: int

Default value is the smalles SV ID which is part of this cell reconstruction.

Return type: int
Returns: Globally unique identifier of this object.

property identifier: str

Identifier used to create the folder name of the :class: ~syconn.reps.super_segmentation_dataset.SuperSegmentationDataset this object belongs to.

Return type: str

label_dict(data_type='vertex')[source]

Retrieves the compressed storage of labels for a specific data type like ‘vertex’. This dictionary stores various predictions, with keys and the associated labels corresponding to the mesh vertices. The ordering is consistent with the vertex order in self.mesh[1].

Uses the CompressedStorage interface.

Parameters: data_type (str) – The key for the stored labels.
Returns: The storage object containing the requested labels.
Return type: CompressedStorage
Raises: ValueError – If the requested data type is not supported.

load_attr_dict()[source]

Loads the attribute dictionary of the SuperSegmentationObject stored at ssv_dir from disk.

Returns: Status code (0 for success, -1 for failure).
Return type: int

load_mesh(mesh_type)[source]

Load mesh of a specific type, e.g. ‘mi’, ‘sv’ (cell supervoxel), ‘sj’ (connected components of the original synaptic junction predictions), ‘syn_ssv’ (overlap of ‘sj’ with cell contact sites), ‘syn_ssv_sym’ and ‘syn_ssv_asym’ (only if syn-type predictions are available).

Parameters: mesh_type – Type of SegmentationObject used for mesh retrieval.
Returns: indices, vertices, normals
Return type: Three flat arrays

Raises: ValueError: If mesh_type does not exist in _meshes.

load_skeleton()[source]

Loads the skeleton from file or computes it if it does not exist and generation is allowed (requires allow_ssv_skel_gen=True).

Return type: bool
Returns: A boolean indicating whether the skeleton was successfully loaded or generated.

load_so_attributes(obj_type, attr_keys)[source]

Collects attributes from instances of the SegmentationObject class of a specified type. The attribute value ordering for each key is the same as the svs attribute.

Parameters

obj_type (str) – The type of SegmentationObject to collect attributes from.
attr_keys (List[str]) – A list of attribute keys to collect; these keys must exist
obj_type. (for the requested) –

Returns

A list of attribute values for each key in attr_keys.

Return type

List[Any]

load_sv_edgelist()[source]

Loads the edge list of the supervoxel graph.

Return type: List[Tuple[int, int]]
Returns: Edge list representing the supervoxel graph.

load_sv_graph()[source]

Loads the supervoxel graph for this SuperSegmentationObject, where node objects are instance of SegmentationObject. The graph is derived from the supervoxel ID graph stored in _sv_graph or the edge list stored at edgelist_path.

Returns: The supervoxel graph with SegmentationObject nodes.
Return type: nx.Graph

load_views(view_key=None, woglia=True, raw_only=False, force_reload=False, nb_cpus=None, ignore_missing=False, index_views=False)[source]

Loads views for the SuperSegmentationObject (SSV) either from the view dictionary, view_dict, or from the view storage, view_path, given the key view_key. If the specified key does not exist at the SSV level or is None, it tries to load the views from the underlying SegmentationObjects. If views are cached and the key is present, they are returned directly.

Parameters

view_key (Optional[str]) – The key used for the view retrieval.
woglia (bool) – If True, loads views rendered from the glia-free agglomeration.
raw_only (bool) – If True, only returns the cell shape channel in the views.
force_reload (bool) – If True, forces the views to reload from the storage.
nb_cpus (Optional[int]) – The number of CPUs to use for loading views in parallel.
ignore_missing (bool) – If True, it will not raise KeyError if SegmentationObject is missing.
index_views (bool) – If True, loads views that contain the indices of the vertices at the respective pixels, used for semantic label mapping onto mesh vertices.

Return type

ndarray

Returns

An array containing concatenated views for each SegmentationObject in self.svs with shape [N_LOCS, N_CH, N_VIEWS, X, Y].

load_voxels_downsampled(downsampling=(2, 2, 1), nb_threads=10)[source]

Loads all voxels of this SuperSegmentationObject.

Parameters

downsampling (tuple) – The downsampling factors for each axis.
nb_threads (int) – The number of threads to use for loading.

Returns

An array of downsampled voxel coordinates.

Return type

np.ndarray

lookup_in_attribute_dict(attr_key)[source]

Retrieves the value associated with a specified attribute key from the attribute dictionary stored in attr_dict.

Parameters: attr_key (str) – Attribute key to look up.
Returns: The value to the key attr_key, or None if the key is not existent.
Return type: Optional[Any]

majority_vote(prop_key, max_dist)[source]

Smooths property prediction in annotation using a sliding window of 2 times max_dist and a majority vote.

Parameters

prop_key (str) – Property to average.
max_dist (float) – Maximum distance (in nm) for the sliding window used in majority voting.

Returns

Smoothed property predictions.

Return type

np.ndarray

mergelist2kzip(dest_path=None)[source]

Writes the mergelist to a text file within a k.zip file.

Parameters: dest_path (Optional[str]) – The destination path for the k.zip file. If None, the default skeleton k.zip path is used.
Returns: None

property mesh: Optional[Union[Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray], List[numpy.ndarray], Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray]]]

Mesh of all cell supervoxels.

Return type: Union[Tuple[ndarray, ndarray, ndarray], List[ndarray], Tuple[ndarray, ndarray, ndarray, ndarray], None]

mesh2file(dest_path=None, center=None, color=None, scale=None, obj_type='sv')[source]

Writes a mesh to a file (e.g. .ply, .stl, .obj) using the ‘openmesh’ library. If possible, the function writes it in binary format.

Parameters

dest_path (str) – The destination path for the mesh file.
center (Optional[np.ndarray]) – Scaled center coordinates in nanometers.
color (Optional[np.ndarray]) – Either a single color (1D; will be applied to all vertices) or a per-vertex color array (2D).
scale (Optional[float]) – A scaling factor to apply to vertex locations after centering.
obj_type (str) – Defines the object type used for loading the mesh with load_mesh().

Returns

None

mesh2kzip(dest_path=None, obj_type='sv', ext_color=None, **kwargs)[source]

Writes the mesh of a specified object type to a k.zip file as a .ply file.

Parameters

dest_path (Optional[str]) – The destination path for the k.zip file. If None, the default skeleton k.zip path is used.
obj_type (str) – The type of object to retrieve the mesh for. Options include ‘sv’ for cell surface, ‘mi’ for mitochondria, ‘vc’ for vesicle clouds, and ‘sj’ for synaptic junctions.
ext_color (Optional[np.ndarray]) – An optional color array to apply to the mesh. If scalar, it must be an integer between 0 and 255. If array, it must be of type uint/int and of shape (N, 4) where N is the number of vertices of the SSV cell surface mesh: N = len(self.mesh[1].reshape((-1, 3))).

Returns

None

property mesh_caching: bool

If True, mesh data is cached.

Return type: bool

property mesh_dc_path: str

Identifier of mesh storage

Return type: str

mesh_exists(obj_type)[source]

Checks if the mesh of SegmentationObject`s of type `obj_type is stored in the MeshStorage at mesh_dc_path.

Parameters: obj_type (str) – The type of requested SegmentationObject.
Returns: True if the mesh exists, False otherwise.
Return type: bool

meshes2kzip(dest_path=None, sv_color=None, synssv_instead_sj=True, object_types=None, **kwargs)[source]

Writes multiple object meshes including SV, mito, vesicle cloud, and synaptic junction to a k.zip file.

Parameters

dest_path (Optional[str]) – The destination path for the k.zip file. If not provided, the default path is used.
sv_color (Optional[np.ndarray]) – An optional color array with RGBA values to apply to the supervoxel mesh. If not provided, default values are used (see mesh2kzip()).
synssv_instead_sj (bool) – If True, uses ‘syn_ssv’ objects instead of ‘sj’ for synaptic junctions.
object_types (Optional[List[str]]) – A list of object types to export. Defaults to [‘sj’, ‘vc’, ‘mi’, ‘sv’].

Returns

None

property mi_ids: numpy.ndarray

All mitochondria (mi) supervoxel IDs which are assigned to this cell reconstruction.

Return type: ndarray

property mi_mesh: Optional[Union[Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray], List[numpy.ndarray], Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray]]]

Mesh of all mitochondria (mi) supervoxels.

Return type: Union[Tuple[ndarray, ndarray, ndarray], List[ndarray], Tuple[ndarray, ndarray, ndarray, ndarray], None]

property mis: List[syconn.reps.segmentation.SegmentationObject]

All mitochondria (mi) SegmentationObjects objects which are assigned to this cell reconstruction.

Return type: List[SegmentationObject]

morphembed2mesh(dest_path, pred_key='latent_morph', whiten=True)[source]

Writes a morphology embedding as RGB to a k.zip file.

Parameters

dest_path (str) – The destination path for the k.zip file.
pred_key (str) – The key for the morphology embedding.
whiten (bool) – If True, applies whitening to the embedding before coloring.

Returns

None

property object_caching: bool

If True, :class:`~syconn.reps.segmentation.SegmentationObject`s which are part of this cell reconstruction are cached.

Return type: bool

property objects_dense_kzip_path: str

Identifier of cell organell overlays

Return type: str

partition_cc(max_nb_sv=None, lo_first_n=None)[source]

Splits the supervoxel graph of this SSV into subgraphs. These subgraphs are defined based on the specified parameters, with default values generated from config.

Parameters

max_nb_sv (Optional[int]) – The maximum number of supervoxels per subgraph. This defines the sub-graph context.
lo_first_n (Optional[int]) – The number of first traversed nodes to exclude from new BFS traversals. This allows the partitioning of the original supervoxel graph of size N into N//lo_first_n sub-graphs.

Return type

List[List[Any]]

Returns

A list of lists, each sublist representing a partitioned subgraph.

path_density_seg_obj(obj_type, compartments_of_interest=None, ax_pred_key='axoness_avg10000')[source]

Calculates the average volume per path length of a segmented object.

Parameters

obj_type (str) – Key to any available sub-cellular structure.
compartments_of_interest (Optional[List[int]]) – Which compartments to take into account
axon (for calculation.) – 1, dendrite: 0, soma: 2
ax_pred_key (str) – Key of compartment prediction stored in the skeleton, only used if
set. (compartments_of_interest was) –

Returns

Average volume per path length (um^3 / um).

Return type

float

pklskel2kzip()[source]: Converts the skeleton stored in a pickle file to a KNOSSOS compatible k.zip file.

predict_cell_morphology_pts(**kwargs)[source]

Stores local cell morphology with key ‘latent_morph’ (+ pred_key_appendix) in the SSV skeleton.

Parameters: **kwargs – Additional keyword arguments.

predict_celltype_multiview(model, pred_key_appendix, model_tnet=None, view_props=None, onthefly_views=False, overwrite=True, model_props=None, verbose=False, save_to_attr_dict=True)[source]

Infers celltype classification using model, optionally includes cell morphology embedding via model_tnet. Stores results as celltype_cnn_e3 and celltype_cnn_e3_probas in the attr_dict.

Parameters

model (nn.Module) – The prediction model for celltype classification.
pred_key_appendix (str) – Additional key to append to the prediction key.
model_tnet (Optional[nn.Module]) – Optional model for cell morphology embedding. The resulting embedding is stored as latent_morph_ct.
view_props (Optional[dict]) – Dictionary containing view properties. If None, defaults defined in config will be used.
onthefly_views (bool) – If True, views will be rendered on-the-fly.
overwrite (bool) – If True, overwrites existing predictions.
model_props (Optional[dict]) – Model properties as defined in config.yml.
verbose (bool) – If True, prints additional log information.
save_to_attr_dict (bool) – If True, saves predictions in the attribute dictionary.

predict_semseg(m, semseg_key, nb_views=None, verbose=False, raw_view_key=None, save=False, ws=None, comp_window=None, add_cellobjects=True, bs=10)[source]

Generates label views based on an input model and stores it under the ‘semseg_key’. If ‘nb_views’ or ‘raw_view_key’ are provided, it requires pre-rendered raw views. Otherwise, it uses default views stored at the SSV’s SegmentationObjects.

Parameters

semseg_key (str) – Key under which the semantic segmentation results are stored.
nb_views (Optional[int]) – Number of views used for prediction (if non-default).
k (int) – An unspecified parameter.
verbose (bool) – If True, logs additional information.
raw_view_key (str) – Key used for storing raw view arrays. Default: ‘raw{}’.format(nb_views). If key does not exist, views will be re-rendered with properties defined in config or as given in the kwargs ws, nb_views, and comp_window (if non-default).
save (bool) – If True, saves the predicted label views.
ws (tuple[int]) – Window size in pixels [y, x] for rendering (if non-default).
comp_window (float) – Physical extent in nm of the view-window along the y-axis (if non-default).
add_cellobjects (Union[bool, Iterable]) – Adds cell objects to views during rendering. Either bool or a list of structures used for rendering. Only used if raw_view_key or nb_views is None.
bs (int) – Batch size used during inference.

Returns

None

predict_views_axoness(model, verbose=False, pred_key_appendix='')[source]

Predicts axoness and stores resulting labels at vertex dictionary.

Parameters

model – The prediction model to be used.
verbose (bool) – If True, additional log information will be printed.
pred_key_appendix (str) – Additional key to be appended to the prediction key.

predict_views_embedding(model, pred_key_appendix='', view_key=None)[source]

Saves a latent vector capturing a local morphology fingerprint for every skeleton node location based on the nearest rendering location. This method requires existing views. For on the fly view rendering, use view_embedding_of_sso_nocache.

Parameters

model – The prediction model to be used.
pred_key_appendix (str) – Additional key to be appended to the prediction key.
view_key (str) – View identifier, e.g. if views have been pre-rendered and are stored in self.view_dict.

predict_views_gliaSV(model, verbose=True, pred_key_appendix='')[source]

Predicts glia views for supervoxels using a given model.

Parameters

model – The model used for prediction.
verbose (bool) – If True, logs additional information.
pred_key_appendix (str) – An appendix to the prediction key.

Returns

None

preprocess()[source]

Processes object mapping (requires prior assignment of object candidates), caches object meshes, and calculates the SSV skeleton.

Returns: None. This method performs preprocessing steps on the SSV.

property rag: networkx.classes.graph.Graph

Retrieves the region adjacency graph (defining the supervoxel graph) of this SuperSegmentationObject.

Returns: The supervoxel graph with nodes of type SegmentationObject, as defined in the syconn.reps.segmentation class.
Return type: nx.Graph

render_indexviews(nb_views=2, save=True, force_recompute=False, verbose=False, view_key=None, ws=None, comp_window=None)[source]

Renders SSV raw views with a non-default number of views and stores them in the SSV view dictionary. Default raw/index/prediction views are stored decentralized in corresponding SVs. If the views already exist and recompute is not forced, they are returned directly. Otherwise, they are rendered and optionally saved.

Parameters

nb_views (int) – The number of views to render.
save (bool) – If True, saves the rendered views.
force_recompute (bool) – If True, forces re-rendering of the views.
verbose (bool) – If True, logs additional information.
view_key (Optional[str]) – The key used to store the view array. Default: ‘index{}’.format(nb_views)
ws (Tuple[int]) – The window size in pixels for rendering.
comp_window (float) – The physical extent in nm of the view-window along the y-axis (see ws to infer pixel size).

Returns

An array of the rendered views if save is False, otherwise None.

Return type

np.array

render_ortho_views_vis(dest_folder=None, colors=None, ws=(2048, 2048), obj_to_render=('sv',))[source]

Renders orthogonal views for visualization.

Parameters

dest_folder (Optional[str]) – Destination folder for saving images.
colors (Optional[dict]) – Dictionary specifying colors for rendering.
ws (tuple) – Window size for rendering views.
obj_to_render (tuple) – Objects to render.

render_views(add_cellobjects=False, verbose=False, overwrite=True, cellobjects_only=False, woglia=True, skip_indexviews=False)[source]

Renders views for each SegmentationObject based on the context of the SSV and stores them at the SV level. Typically used for initial glia or axoness prediction, the results are distributed to each SegmentationObject of this object. Views are not cached in the view_dict or view_path of the SSV and are used during initial glia, compartment and cell type predictions. Refer to _render_rawviews for storing views in the SSV storage, which is used during GT generation.

Parameters

add_cellobjects (bool) – If True, adds cellular organelle channels to the 2D projection views.
verbose (bool) – If True, logs additional information.
overwrite (bool) – If True, re-renders views at all rendering locations.
cellobjects_only (bool) – If True, renders only cellular organelle channels (currently not used).
woglia (bool) – If True, loads views from the glia-free agglomeration.
skip_indexviews (bool) – If True, does not generate index views. Used for initial SSV glia-removal rendering.

Returns

None

property rep_coord: numpy.ndarray

Retrieves the representative coordinate of this SuperSegmentationObject, which is the representative coordinate of the first supervoxel in ~svs.

Returns: A 1D array of the representative coordinate (XYZ).
Return type: np.ndarray

sample_locations(force=False, cache=True, verbose=False, ds_factor=None)[source]

Samples coordinates for rendering views for each SegmentationObject in the SSV. Optionally forces resampling and caches the locations.

Parameters

force (bool) – If True, forces resampling of locations.
cache (bool) – If True, saves the sampled locations in the SSV’s attribute dictionary.
verbose (bool) – If True, logs additional information.
ds_factor (float) – The downscaling factor used to generate locations.

Returns

Contains sampled coordinates for each SegmentationObject in the SSV.

Return type

list of array

save_attr_dict()[source]: Saves the SuperSegmentationObject’s attribute dictionary to disk.

save_attributes(attr_keys, attr_values)[source]

Writes attributes to the attribute dictionary on the file system. Does not care about self.attr_dict.

Parameters

attr_keys (List[str]) – A list of attribute keys to save.
attr_values (List[Any]) – A list of corresponding attribute values to save.

save_objects_to_kzip_dense(obj_types, dest_path=None)[source]

Exports cellular organelles as coordinates with size, shape, and overlap properties in a KNOSSOS compatible format.

Parameters

obj_types (List[str]) – Type identifiers of the supervoxel objects which are exported.
dest_path (Optional[str]) – Path to the destination file. If None, result will be stored at the objects_dense_kzip_path.

Returns

None. The function saves a KNOSSOS compatible k.zip file with the specified cellular organelles.

save_objects_to_kzip_sparse(obj_types=None, dest_path=None)[source]

Exports cellular organelles as coordinates with size, shape, and overlap properties in a KNOSSOS compatible format to a k.zip file.

Parameters

obj_types (Optional[Iterable[str]]) – Type identifiers of the supervoxel objects which are exported.
dest_path (Optional[str]) – Path to the destination file. If None, results will be stored at the skeleton k.zip path.

Returns

None. The method saves a KNOSSOS compatible k.zip file with the specific cellular organelles.

save_skeleton(to_kzip=False, to_object=True)[source]

Saves the skeleton to default locations as .pkl and optionally as .k.zip.

Parameters

to_kzip (bool) – If True, stores the skeleton as a KNOSSOS compatible XML inside a k.zip file.
to_object (bool) – If True, stores the skeleton as a dictionary in a pickle file.

Returns

None. The skeleton is saved to the specified formats and locations.

save_skeleton_to_kzip(dest_path=None, name='skeleton', additional_keys=None, comments=None)[source]

Saves the skeleton representation of the super-supervoxel to a KNOSSOS compatible k.zip file, including additional properties and comments.

Parameters

dest_path (Optional[str]) – Destination path for k.zip file. If None, the default skeleton k.zip path is used.
name (str) – Identifier or name of saved skeleton that appears in KNOSSOS.
additional_keys (Optional[List[str]]) – Additional skeleton keys to be converted into KNOSSOS skeleton node properties. Always attempts to write out the keys ‘axoness’, ‘cell_type’, and ‘meta’.
comments (Union[ndarray, List[str], None]) – np.ndarray of strings or list of strings of length N where N equals the number of skeleton nodes. These comments will be converted into KNOSSOS skeleton node comments.

Returns

None. The method saves a KNOSSOS compatible k.zip file containing the super-supervoxel skeleton and its node properties.

save_views(views, view_key='views')[source]

Saves the view array to the SSV’s view storage under the specified view key. However, this will only save views on SSV level and not for each individual SV. If the SSV version is ‘tmp’, a warning is logged and views are not saved to disk.

Parameters

views (ndarray) – The view array.
view_key (str) – The key used for the look-up.

Returns

None

property scaling: numpy.ndarray

Voxel size in nanometers (XYZ). Default is taken from the config.yml file and accessible via config.

Return type: ndarray

semseg2mesh(semseg_key, dest_path=None, nb_views=None, k=1, force_recompute=False, index_view_key=None)[source]

Generates vertex labels from semantic segmentation results and stores them in the SSV’s label storage. Optionally, stores the labeled mesh as a .ply file in a k.zip at the given path.

Examples

Default situation:: semseg_key = 'spiness', nb_views=None This will load the index and label views stored at the SSV’s SVs.
Non-default:: semseg_key = 'spiness4', nb_views=4 This requires to run self._render_rawviews(nb_views=4), self.render_indexviews(nb_views=4) and predict_semseg(MODEL, 'spiness4', nb_views=4). This method then has to be called like: self.semseg2mesh('spiness4', nb_views=4)

Parameters

semseg_key (str) – The key used to retrieve semantic segmentation results.
dest_path (Optional[str]) – The path where the labeled mesh will be stored as a .ply in a k.zip.
nb_views (Optional[int]) – The number of views used for generating the segmentation (if non-default).
k (int) – The number of nearest vertices to average over for smoothing the segmentation. If k=0 unpredicted vertices will be treated as ‘unpredicted’ class.
force_recompute (bool) – If True, forces re-computation of the vertex labels.
index_view_key (Optional[str]) – The key used to retrieve index views (if non-default).

Returns

None

semseg_for_coords(coords, semseg_key, k=5, ds_vertices=20, ignore_labels=None)[source]

Retrieves the semantic segmentation with key semseg_key from the k nearest vertices at every coordinate in coords.

Parameters

coords (ndarray) – np.array Voxel coordinates, unscaled! [N, 3]
semseg_key (str) – str The key of the semantic segmentation to use.
k (int) – int Number of nearest neighbors (NN) during k-NN classification.
ds_vertices (int) – int Striding factor for vertices. Uses max(1, ds_vertices // 10) if len(vertices) < 5e6.
ignore_labels (Optional[Iterable[int]]) – List[int] Vertices with labels in ignore_labels will be ignored during majority vote, e.g., used to exclude unpredicted vertices.

Returns: np.array: An array of the same length as coords. For every coordinate in coords returns the majority label based on its k-nearest neighbors.

property shape: numpy.ndarray

Retrieves the XYZ extent of this SuperSegmentationObject in voxels.

Returns: The shape/extent of this SSV object in voxels (XYZ).
Return type: np.ndarray

shortestpath2soma(coordinates, axoness_key=None)[source]

Computes the shortest path to the soma along the skeleton. Cell compartment predictions must exist in the skeleton’s ‘axoness_avg10000’, as per the ‘syconn.exec.exec_inference.run_semsegaxoness_mapping’ function. A populated skeleton is required, e.g., via the ‘load_skeleton’ function.

Parameters

coordinates (np.ndarray) – Starting coordinates in voxel coordinates with shape (N, 3).
axoness_key (Optional[str]) – Key to axon prediction stored in the skeleton.

Raises

KeyError – If axon prediction does not exist.

Examples

To get the shortest paths between all synapses and the soma, use the following code:

from syconn.reps.super_segmentation import * from syconn import global_params

global_params.wd = ‘~/SyConn/example_cube1/’ ssd = SuperSegmentationDataset() # get any cell reconstruction ssv = ssd.get_super_segmentation_object(ssd.ssv_ids[0]) # get synapse coordinates in voxels. syns = np.array([syn.rep_coord for syn in ssv.syn_ssv]) shortest_paths = ssv.shortestpath2soma(syns)

Returns: The shortest path in nanometers for each start coordinate.
Return type: List[float]

property size: int

Retrieves the number of voxels associated with this SuperSegmentationObject.

Returns: The number of voxels associated with this SuperSegmentationObject.
Return type: int

property sj_ids: numpy.ndarray

All synaptic junction (sj) supervoxel IDs which are assigned to this cell reconstruction.

Return type: ndarray

property sj_mesh: Optional[Union[Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray], List[numpy.ndarray], Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray]]]

Mesh of all synaptic junction (sj) supervoxels. These objects are based on the original synapse prediction and might contain merger.

Return type: Union[Tuple[ndarray, ndarray, ndarray], List[ndarray], Tuple[ndarray, ndarray, ndarray, ndarray], None]

property sjs: List[syconn.reps.segmentation.SegmentationObject]

All synaptic junction (sj) SegmentationObjects objects which are assigned to this cell reconstruction. These objects are based on the initial synapse predictions and may contain synapse-synapse merger. See syn_ssv for merger-free inter-neuron synapses.

Return type: List[SegmentationObject]

property skeleton_kzip_path: str

Path to the skeleton storage.

Return type: str

property skeleton_kzip_path_views: str

Path to the skeleton storage.

Return type: str

property skeleton_path: str

Identifier of SSV skeleton

Return type: str

skelproperty2mesh(key, dest_path=None, k=1)[source]

Writes a skeleton property to a mesh in a k.zip file.

Parameters

key (str) – The key of the skeleton property to write.
dest_path (Optional[str]) – The destination path for the k.zip file.
k (int) – The number of nearest neighbors for averaging predictions.

Returns

None

property ssd_dir: str

Path to the SuperSegmentationDataset directory this object belongs to.

Return type: str

property ssd_kwargs: dict

Return type: dict

property ssv_dir: str

Path to the folder where the data of this super-supervoxel is stored.

Return type: str

property ssv_kwargs: dict

Return type: dict

property sv_graph_uint: networkx.classes.graph.Graph

Retrieves the supervoxel graph with unsigned integer node IDs.

Returns: The supervoxel graph with uint node IDs.
Return type: nx.Graph

property sv_ids: numpy.ndarray

All cell supervoxel IDs which are assigned to this cell reconstruction.

Return type: ndarray

svprobas2mergelist(key='glia_probas', dest_path=None)[source]

Writes the supervoxel probabilities to a mergelist within a k.zip file.

Parameters

key (str) – The key of the supervoxel probabilities to write.
dest_path (Optional[str]) – The destination path for the k.zip file.

Returns

None

property svs: List[syconn.reps.segmentation.SegmentationObject]

All cell SegmentationObjects objects which are assigned to this cell reconstruction.

Return type: List[SegmentationObject]

syn_sign_ratio(weighted=True, recompute=True, comp_types=None, comp_types_partner=None)[source]

Computes the ratio of symmetric synapses between specified functional compartments. The ratio ranges between 0 and 1, and -1 is returned if no synapse objects exist. The predicted boutons are converted into axon labels; 3 (en-passant) changes to 1 and 4 (terminal) to 1.

Parameters

weighted (bool) – If True, computes an area-weighted ratio of symmetric synapses.
recompute (bool) – If True, disregards any existing value and recomputes the ratio.
comp_types (list) – The functional compartment types on this cell to consider for the ratio. Default is [1,].
comp_types_partner (list) – The types on the partner cell to consider. Default is [0,].

Returns

The (area-weighted) ratio of symmetric synapses, or -1 if no synapses are present.

Return type

float

property syn_ssv: List[syconn.reps.segmentation.SegmentationObject]

All synaptic junctions SegmentationObject objects which are between super-supervoxels (syn_ssv) and assigned to this cell reconstruction. These objects are generated as an agglomeration of ‘syn’ objects, which themselves have been generation as a combination of synaptic junction (sj) and contact site (cs) objects to remove merges in the sj objects.

Return type: List[SegmentationObject]

property syn_ssv_mesh: Optional[Union[Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray], List[numpy.ndarray], Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray]]]

Mesh of all inter-neuron synapses junction (syn_ssv) supervoxels. These objects are generated as a combination of contact sites and synaptic junctions (sj).

Return type: Union[Tuple[ndarray, ndarray, ndarray], List[ndarray], Tuple[ndarray, ndarray, ndarray, ndarray], None]

total_edge_length(compartments_of_interest=None, ax_pred_key='axoness_avg10000')[source]

Calculates the total edge length of the super-supervoxel skeleton in nanometers.

Parameters

compartments_of_interest (Optional[List[int]]) – A list specifying which compartments to include in the calculation. Options include axon (1), dendrite (0), and soma (2).
ax_pred_key (str) – The key of the compartment prediction stored in the skeleton. This is used only if compartments_of_interest is set.

Return type

Union[ndarray, float]

Returns

The total sum of all edge lengths (L2 norm) in the skeleton, measured in nanometers.

property type: str

‘ssv’.

Return type: str
Returns: String identifier of the object type.
Type: The type of this super-sueprvoxel. Default

typedsyns2mesh(dest_path=None, rewrite=False)[source]

Generates typed meshes for ‘syn_ssv’ and saves them at mesh_dc_path (keys: 'syn_ssv_sym' and 'syn_ssv_asym'), and writes them to an optional dest_path (if provided). The meshes can be accessed with the respective keys via load_mesh.

Synapse types are identified in the ‘syn_ssv’ AttributeDicts and handled as follows:

excitatory / asymmetric: 1
inhibitory / symmetric: -1

Parameters

dest_path (Optional[str]) – An optional output path for the synapse meshes.
rewrite (bool) – If set to True, disregards existing meshes in

:param _meshes or at mesh_dc_path and overwrites them.:

Returns: None

property vc_ids: numpy.ndarray

All vesicle cloud (vc) supervoxel IDs which are assigned to this cell reconstruction.

Return type: ndarray

property vc_mesh: Optional[Union[Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray], List[numpy.ndarray], Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray]]]

Mesh of all vesicle clouds (vc) supervoxels.

Return type: Union[Tuple[ndarray, ndarray, ndarray], List[ndarray], Tuple[ndarray, ndarray, ndarray, ndarray], None]

property vcs: List[syconn.reps.segmentation.SegmentationObject]

All vesicle cloud (vc) SegmentationObjects objects which are assigned to this cell reconstruction.

Return type: List[SegmentationObject]

property version: str

Version of the ~SuperSegmentationDataset this object belongs to. Can be any character or string like ‘0’ or ‘axongroundtruth’.

Return type: str
Returns: String identifier of the object’s version.

property view_caching: bool

If True, view data is cached.

Return type: bool

view_existence(woglia=True, index_views=False, view_key=None)[source]

Checks whether a specific set of views exists for this object.

Parameters

woglia (bool) – If True, will load the views render from the glia-free agglomeration.
index_views (bool) – Views which contain the indices of the vertices at the respective pixels. Used as look-up to map the predicted semantic labels onto the mesh vertices.
view_key (Optional[str]) – The key used for the look-up.

Return type

List[bool]

Returns

True if the specified views exist.

property view_path: str

Identifier of view storage

Return type: str

property vlabel_dc_path: str

Identifier of vertex label storage

Return type: str

property voxel_caching: bool

If True, voxel data is cached.

Return type: bool

property voxels: Optional[numpy.ndarray]

Retrieves the voxels associated with the SuperSegmentationObject.

Returns: A 3D binary array indicating voxel locations. If no voxels are present, returns None.
Return type: Optional[np.ndarray]

property voxels_xy_downsampled: Optional[numpy.ndarray]

Retrieves the XY-downsampled voxels associated with this SuperSegmentationObject.

Returns: A 3D binary array indicating downsampled voxel locations, or None if no voxels are present.
Return type: Optional[np.ndarray]

weighted_graph(add_node_attr=())[source]

Creates a Euclidean distance (in nanometers) weighted graph representation of the skeleton of this SSV object. The node IDs represent the index in the node array part of the skeleton. Weights are stored as ‘weight’ in the graph, which allows usage of methods like nx.single_source_dijkstra_path(..).

Parameters

add_node_attr (Iterable[str]) – An iterable of node attributes to be added to the graph.
skeleton. (These must exist in the) –

Return type

Graph

Returns

A networkx graph representing the SSV object’s skeleton with Euclidean distance (in nanometers) weights.

property working_dir: str

Working directory.

Return type: str

write_axpred_rfc(dest_path=None, k=1)[source]

Writes axon predictions from an RFC to a k.zip file.

Parameters

dest_path (Optional[str]) – The destination path for the k.zip file.
k (int) – The number of nearest neighbors for averaging predictions.

Returns

None

write_gliapred_cnn(dest_path=None)[source]

Writes glia predictions from a CNN to a k.zip file.

Parameters: dest_path (Optional[str]) – The destination path for the k.zip file.
Returns: None

write_locations2kzip(dest_path=None)[source]

Writes the sampled locations to a KNOSSOS annotation file within a k.zip file.

Parameters: dest_path (Optional[str]) – The destination path for the k.zip file. If None, the default skeleton k.zip path is used. If provided without ‘.k.zip’ extension, it is appended.
Returns: None

write_svmeshes2kzip(dest_path=None, **kwargs)[source]

Writes individual cell supervoxel (‘sv’) meshes in ply format to a k.zip file.

Parameters: dest_path (Optional[str]) – Target file name for the k.zip file.
Returns: None

syconn.reps.super_segmentation_object.celltype_predictor(args)[source]

Predicts the cell type for a set of super segmentation objects (SSOs) using a specified model.

Parameters: args – A tuple containing the SSO IDs, number of CPUs, and model properties.
Return type: Iterable
Returns: An iterable of missing or failed SSO IDs.

syconn.reps.super_segmentation_object.render_sampled_sos_cc(sos, ws=(256, 128), verbose=False, woglia=True, render_first_only=0, add_cellobjects=True, overwrite=False, cellobjects_only=False, index_views=False, enable_locking=True)[source]

Renders views at various sampled locations from the combined mesh of all super voxels (SVs). The number of sampled locations depends on the size of the SV mesh, with a scaling factor considered.

Parameters

sos – A list of SegmentationObject instances representing super voxels.
ws – A tuple indicating the window size for rendering views.
verbose – A bool, if True enables verbose output.
woglia – A bool, if True renders view without glia components.
render_first_only – An int, indicating the number of initial SVs to render. If 0, all are rendered.
add_cellobjects – A bool, if True adds cellular objects to the views.
overwrite – A bool, if True overwrites existing views.
cellobjects_only – A bool, if True renders only cellular object channels.
index_views – A bool, if True enables rendering of index views.
enable_locking – A bool, if True enables system locking when writing views.

Returns

None

syconn.reps.super_segmentation_object.render_so(so, ws=(256, 128), add_cellobjects=True, verbose=False)[source]

Renders super voxel views at given locations without writing views to so.views_path.

Parameters

so – A SegmentationObject instance representing a super voxel ID.
ws – A tuple indicating the rendering window size.
add_cellobjects – A boolean flag indicating whether to include cellular objects in the views.
verbose – A boolean flag for verbose output.

Returns

A numpy array containing the rendered views.

syconn.reps.super_segmentation_object.semsegaxoness2skel(sso, map_properties, pred_key, max_dist)[source]

This function populates the skeleton of the provided SuperSegmentationObject (SSO) with axoness predictions and executes a majority vote smoothing operation. It adds two keys to the SSO’s skeleton dictionary: - “{}_avg{}”.format(pred_key, max_dist): This holds the raw axoness predictions. - “{}_avg{}_comp_maj”.format(pred_key, max_dist): This contains the smoothed axoness predictions after the majority vote.

Parameters

sso (SuperSegmentationObject) – The SuperSegmentationObject whose skeleton will be populated with axoness predictions.
map_properties (dict) – The properties that map the vertex predictions to the skeleton nodes.
pred_key (str) – The key used for retrieving vertex labels and storing the mapped node labels in the skeleton.
max_dist (int) – The distance used in the majority vote for the smoothing operation.

Notes

If there are no mesh vertices available or there are no nodes, the node predictions will be zero.
The skeleton must be loaded for this function to work. If it’s not, the function will attempt to load it.
If the skeleton has no nodes or the SSO doesn’t have any mesh vertices, warnings will be logged and the skeleton keys will be filled with zeros.

Returns: This function modifies the skeleton of the SSO in-place.
Return type: None

syconn.reps.super_segmentation_object.semsegaxoness_predictor(args)[source]

Predicts axoness for a set of super segmentation objects (SSOs) and stores the results in the vertex dictionary.

Parameters: args – A tuple containing the SSO IDs, view properties, number of CPUs, mapping properties, prediction key, and maximum distance for mapping.
Return type: List[int]
Returns: A list of missing or failed SSO IDs.

syconn.reps.super_segmentation_object.semsegspiness_predictor(args)[source]

Predicts spiness for a list of SuperSegmentationObjects and stores the resulting labels in their vertex dictionaries.

Parameters: args (tuple) – A tuple containing the following elements: - ssv_ids (List[int]): List of SuperSegmentationObject IDs to process. - view_props (dict): Properties for the view rendering. - nb_cpus (int): Number of CPUs to use for parallel processing. - kwargs_semseg2mesh (dict): Keyword arguments for the semseg2mesh function. - kwargs_semsegforcoords (dict): Keyword arguments for the semseg_for_coords function.
Returns: List of IDs of SuperSegmentationObjects where prediction has failed.
Return type: List[int]

syconn.reps.super_segmentation_helper module

syconn.reps.super_segmentation_helper.assemble_from_mergelist(ssd, mergelist)[source]

Creates a mapping dictionary and saves the dataset shallowly based on a mergelist.

This function will overwrite existing mapping dict, id changer, and version files.

Parameters

ssd (SuperSegmentationDataset) – The dataset to be updated with the new mapping.
mergelist (Union[Dict[int, int], str]) – Supervoxel agglomeration provided either as a dictionary or as a file path
mergelist. (to a previously generated) –

Returns

None

syconn.reps.super_segmentation_helper.average_node_axoness_views(sso, pred_key=None, pred_key_appendix='', max_dist=10000, return_res=False, use_cache=False)[source]

Averages axoness prediction along skeleton with maximum path length of ‘max_dist’. The majority prediction of neighboring nodes within this distance is assigned to each node in the skeleton of the SSO.

Parameters

sso (super_segmentation.SuperSegmentationObject) – The SuperSegmentationObject whose skeleton nodes are evaluated.
pred_key (Optional[str]) – The key for retrieving stored supervoxel predictions. If None, a default key is used: "axoness_preds_cnn%s" % pred_key_appendix.
pred_key_appendix (str) – Appended to default prediction key if pred_key is None, formatting into a complete key.
max_dist (int) – The path length over which to average the predictions.
return_res (bool) – When True, a list of average predictions for each node is returned rather than modifying the SSO.
use_cache (bool) – If enabled, caches intermediate supervoxel predictions in the SSO attribute dictionary and can save to disk.

Returns

If return_res is True, it returns a dictionary with the averaged predictions for each skeleton node. Otherwise, the results are integrated into the SSO and accessible via the generated key "%s_views_avg%d" % (pred_key, max_dist) and no value is returned. The method does not call sso.save_skeleton().

syconn.reps.super_segmentation_helper.celltype_of_sso_nocache(sso, model, ws, nb_views, comp_window, nb_views_model=20, pred_key_appendix='', verbose=False, overwrite=True, use_syntype=True, da_equals_tan=True, n_classes=7, save_to_attr_dict=True)[source]

Predicts the cell type of a SuperSegmentationObject without using file system caching.

This function renders raw views at rendering locations determined by comp_window and following the given view properties. These views are then predicted with the provided model. By default, the resulting predictions and probabilities are stored as ‘celltype_cnn_e3’ and ‘celltype_cnn_e3_probas’ in the attribute dictionary.

Parameters

sso – SuperSegmentationObject to be processed.
model – A machine learning model used for predictions.
ws – Tuple[int, int], window size in pixels [y, x], determines the size of each view.
nb_views – int, number of views rendered at each location. Views are not stored on disk.
comp_window – float, physical extent in nm of the view-window along y.
nb_views_model (int) – int, bootstrap sample size of view locations for model prediction.
pred_key_appendix (str) – str, appendix for the prediction key in the attribute dictionary.
verbose (bool) – bool, if True, adds progress bars for view generation.
overwrite (bool) – bool, if True, overwrites existing views in temporary view dictionary.
use_syntype (bool) – bool, if True, uses the type of presynaptic synapses for prediction.
da_equals_tan (bool) – bool, if True, merges DA and TAN classes, requiring n_classes to be 7.
n_classes (int) – int, number of output classes of the model, must be 7 if da_equals_tan is True.
save_to_attr_dict (bool) – bool, if True, saves the prediction in the attribute dictionary.

Returns

None

syconn.reps.super_segmentation_helper.cnn_axoness2skel(sso, pred_key_appendix='', k=1, force_reload=False, save_skel=True, use_cache=False)[source]

Generates axoness predictions and probabilities for a given SuperSegmentationObject (SSO) and saves them to the ‘axoness_preds_cnn’ attribute in the SSV attribute dict as skeleton attributes. It maps the predictions from the supervoxel views to the skeleton nodes using nearest neighbor assignment.

Parameters

sso (super_segmentation.SuperSegmentationObject) – The SuperSegmentationObject for which axoness predictions and probabilities are generated.
pred_key_appendix (str) – A string appended to the prediction keys to differentiate between different prediction sets.
k (int) – Deprecated. Previously used to define the number of nearest neighbors for prediction assignment.
force_reload (bool) – If True, forces the reloading of predictions even if they already exist. Reload SV predictions.
save_skel (bool) – If True, saves the skeleton with the new prediction attributes, saving the SSV skeleton with prediction attributes “axoness” and “axoness_probas”.
use_cache (bool) – If True, caches the intermediate supervoxel predictions in the SSO attribute dictionary on disk. Write intermediate SV predictions in SSV attribute dict to disk.

Returns

None

syconn.reps.super_segmentation_helper.compartments_graph(ssv, axoness_key)[source]

Creates graphs for axon, dendrite, and soma compartments based on skeleton node predictions.

Parameters

ssv (super_segmentation.SuperSegmentationObject) – Cell reconstruction object. Its skeleton must exist and must contain keys 'edges', 'nodes' and axoness_key.
axoness_key (str) – str, key for axon predictions in ssv.skeleton (0: dendrite, 1: axon, 2: soma). Convert labels 3 (en-passant bouton) and 4 (terminal bouton) to 1 (axon).

Returns

Graphs for dendrite, axon, and soma compartments, respectively.

Return type

Tuple[nx.Graph, nx.Graph, nx.Graph]

syconn.reps.super_segmentation_helper.convert_coord(coord_list, scal)[source]

Converts a list of coordinates using a scaling factor.

Parameters

coord_list – list or np.ndarray The list of coordinates to convert.
scal – numeric or np.ndarray The scaling factor to be applied to each coordinate in coord_list.

Returns

The scaled coordinates, with the same shape as coord_list.

Return type

np.ndarray

syconn.reps.super_segmentation_helper.create_new_skeleton(sv_id, sso)[source]

Creates a new skeleton for a supervoxel within a SuperSegmentationObject.

Parameters

sv_id (int) – The ID of the supervoxel.
sso (SuperSegmentationObject) – The SuperSegmentationObject containing the supervoxel.

Returns

Arrays representing the nodes, diameters, and edges of the new skeleton.

Return type

Tuple[np.ndarray, np.ndarray, np.ndarray]

syconn.reps.super_segmentation_helper.create_new_skeleton_sv_fast(args)[source]

Generates a sparse skeleton for a super-segmentation object (SSO). This method creates edges between supervoxels based on connectivity, suitable for creating sparse representations of object skeletons. Note that performance varies based on connectivity rules.

Parameters

sso – The Super Segmentation Object to generate the skeleton for.
pruning_thresh (int) – Pruning threshold value for branch length, with shorter branches being removed.
sparsify (bool) – Controls whether the generated skeleton should be sparsified.
max_dist_thresh (float) – Maximum permissible distance between edges for node pruning during initial sparsification.
dot_prod_thresh (float) – Dot product threshold for edge adjacencies influencing further pruning post-sparsification.
max_dist_thresh_iter2 (float) – Secondary distance threshold for additional pruning steps after the initial round of sparsifying and pruning.

Returns

A tuple consisting of arrays that represent the nodes, diameters, and edges of the sparse skeleton, alongside cell reconstruction featuring a minimum spanning tree (MST) with estimated radii measures.

Notes: - If the use of multi-processing is desired, set ssv.nb_cpus to a value greater than 1. - The initial sparsening algorithm can be found at skeleton_optimization(). - Further sparsification and pruning steps are detailed at

sparsify_skeleton_fast().

syconn.reps.super_segmentation_helper.create_sso_skeleton_fast(sso, pruning_thresh=800, sparsify=True, max_dist_thresh=600, dot_prod_thresh=0.0, max_dist_thresh_iter2=600)[source]

Creates a sparse skeleton for a super-segmentation object (SSO). This method connects supervoxels in the supervoxel graph, potentially using multi-processing (set ssv.nb_cpus > 1 for this). It offers optional pruning and sparsification steps.

Parameters

sso – The SuperSegmentationObject to process.
pruning_thresh – Threshold for pruning short branches. Short branches are removed below this path length, expressed in NM, as per prune_stub_branches.
sparsify – If True, the skeleton is sparsified; otherwise, it is not.
max_dist_thresh – Initial threshold for node pruning based on the maximum distance in NM between two adjacent edges.
dot_prod_thresh – Threshold for pruning nodes based on the dot product value of two adjacent edges. Nodes are pruned when their connecting edges’ dot product is above this value.
max_dist_thresh_iter2 – A secondary distance threshold in NM for additional node pruning between two adjacent edges after initial sparsening and pruning.

Returns

The SuperSegmentationObject with an updated sparse skeleton, including minimal spanning tree (MST) and radius estimates.

syconn.reps.super_segmentation_helper.create_sso_skeletons_wrapper(ssvs, dest_paths=None, nb_cpus=None, map_myelin=False, save=True)[source]

Generates skeleton representations for a list of SuperSegmentationObjects. If global_params.config.allow_ssv_skel_gen = True, skeletons are created via surface sampling which may result in skeletons partially outside cell segmentation, but close to the cell surface. Conversely, if global_params.config.allow_ssv_skel_gen = False, existing supervoxel skeletons are pruned, stitched, and diameter estimates are performed. Skeletons are saved using ssv.save_skeleton and accessible through ssv.skeleton.

Parameters

ssvs (List[ForwardRef]) – List[SuperSegmentationObject] | An iterable of cell reconstruction objects.
dest_paths (Optional[str]) – Optional[str] | Paths to kzips for each object in ssvs.
nb_cpus (Optional[int]) – Optional[int] | Number of CPUs used for every ssv in ssvs.
map_myelin (bool) – bool | If True, uses map_myelin2coords to map myelin predictions to ssv.skeleton[“nodes”], storing the result in ssv.skeleton[“myelin”]. Predictions are smoothed via majority vote with 10 micrometers traversal.
save (bool) – bool | If True, writes the generated skeleton to disk.

Returns

None

syconn.reps.super_segmentation_helper.extract_spinehead_volume_mesh(sso, ctx_vol=(200, 200, 100))[source]

Calculate the volume of spine heads using a watershed approach on cell segmentation.

This method applies a watershed procedure to the cell segmentation to determine the volume of spine heads. The process begins with predictions on the cell mesh, then refines these predictions by mapping them to voxels within a specified bounding box around synapses. The watershed seeds are derived from local maxima of the cell mask’s distance transform and are labeled based on the majority vote among their k-nearest vertices.

Results are stored in the SuperSegmentationObject.attr_dict with the key ‘spinehead_vol’.

Notes: - The calculated ‘spine_headvol’ is in micrometers cubed (µm^3). - The segmentation mask is downsampled to match the z voxel size. - The predicted cell mesh must have ‘spiness’ in label_dict(‘vertex’)[‘spiness’]. - To store results, invoke sso.save_attr_dict().

Parameters

sso (super_segmentation.SuperSegmentationObject) – The SuperSegmentationObject to be processed. It requires a predicted cell mesh, i.e. ‘spiness’ must be present in label_dict(‘vertex’)[‘spiness’].
ctx_vol – A tuple representing the additional volume around the spine head synapse representative coordinate used for volume estimation. The inspected volume is 2*ctx_vol + synapse_bounding_box.

Returns

None

syconn.reps.super_segmentation_helper.find_incomplete_ssv_skeletons(ssd, n_cores=None)[source]

Identifies SuperSegmentationObjects within a SuperSegmentationDataset that have incomplete skeletons.

Parameters

ssd – The SuperSegmentationDataset to search within.
n_cores (Optional[int]) – The number of cores to use for processing. If None, the default number of cores specified in the global configuration is used.

Returns

A list of IDs of SuperSegmentationObjects with incomplete skeletons.

syconn.reps.super_segmentation_helper.find_incomplete_ssv_views(ssd, woglia, n_cores=None)[source]

Identifies SuperSegmentationObjects within a SuperSegmentationDataset that have incomplete views.

Parameters

ssd (SuperSegmentationDataset) – The SuperSegmentationDataset to search within.
woglia (bool) – A boolean indicating whether to consider views with glia removal.
n_cores (Optional[int]) – The number of cores to use for processing. If None, the default number of cores specified in the global configuration is used.

Returns

A list of IDs of SuperSegmentationObjects with incomplete views.

syconn.reps.super_segmentation_helper.find_missing_sv_attributes_in_ssv(ssd, attr_key, n_cores=None)[source]

Identifies SuperSegmentationObjects within a SuperSegmentationDataset that are missing specified attributes.

Parameters

ssd – The SuperSegmentationDataset to search within.
attr_key – The key of the attribute to check for.
n_cores (Optional[int]) – The number of cores to use for processing. If None, the default number of cores specified in the global configuration is used.

Returns

A list of IDs of SuperSegmentationObjects missing the specified attribute.

syconn.reps.super_segmentation_helper.from_netkx_to_arr(skel_nx)[source]

Converts a networkx graph representing a skeleton into arrays of nodes, diameters, and edges.

Parameters: skel_nx (Graph) – The networkx graph representing the skeleton.
Returns: Arrays representing the nodes, diameters, and edges of the skeleton.
Return type: Tuple[np.ndarray, np.ndarray, np.ndarray]

syconn.reps.super_segmentation_helper.from_netkx_to_sso(sso, skel_nx)[source]

Converts a networkx graph representation of a skeleton into a SuperSegmentationObject’s skeleton.

Parameters

sso – SuperSegmentationObject The SuperSegmentationObject to update with the new skeleton.
skel_nx – networkx.Graph The networkx graph representing the skeleton.

Returns

The updated SSO with the new skeleton.

Return type

SuperSegmentationObject

syconn.reps.super_segmentation_helper.from_sso_to_netkx_fast(sso, sparsify=True, max_edge_length=1500.0)[source]

Create a sparse supervoxel skeleton from an existing initial skeleton. This method acts as a multi-process helper similar to create_new_skeleton, often used in conjunction with functions like from_sso_to_netkx_fast.

Parameters

sso – The SuperSegmentationObject to process (equivalent to Supervoxel ID for skeleton creation).
sparsify – Flag indicating whether to sparsify the skeleton (equates to the sparse flag).
max_edge_length – Optional; defines the maximum length of edges in the resulting skeleton. If not provided, defaults to the procedure used in the initial skeletonization.

Returns

Node coordinates, expressed in voxels,
An estimation of the diameter for each node,
The list of edges connecting the nodes to represent the skeleton as a graph.

Return type

A tuple of three elements

This encompasses both the original and additional information, ensuring consistency and completeness while adhering to the maximum character limit.

syconn.reps.super_segmentation_helper.get_pca_view_hists(sso, t_net, pca)[source]

syconn.reps.super_segmentation_helper.get_sso_axoness_from_coord(sso, coord, k=5)[source]

Determines the majority axoness class of the k nearest neighbor nodes within an SSO skeleton.

Parameters

sso – SuperSegmentationObject The SuperSegmentationObject containing the skeleton.
coord – np.array The unscaled coordinate to query.
k – int The number of nearest neighbors to consider for the majority vote.

Returns

The majority class of the nodes (0 for dendrite, 1 for axon, or 2 for soma).

Return type

int

syconn.reps.super_segmentation_helper.glia_pred_exists(so)[source]

syconn.reps.super_segmentation_helper.gliapred_sso_nocache(sso, model, verbose=True)[source]

Performs a multi-view based astrocyte inference on a SuperSegmentationObject without using cached views. The result is stored as ‘glia_probas’ in the attribute dictionaries of every supervoxel within the SuperSegmentationObject. Access the probabilities of a supervoxel via sso.svs[idx].attr_dict[‘glia_probas’].

Parameters

sso (SuperSegmentationObject) – The SuperSegmentationObject to process.
model – A PyTorch model used for astrocyte inference.
verbose (bool) – If True, additional output is printed during the process.

Returns

None. The probabilities are stored in the attribute dictionaries of the supervoxels within ‘sso’.

syconn.reps.super_segmentation_helper.label_array_for_sso_skel(sso, comment_converter)[source]

Converts skeleton node comments to a label array matching the node order.

Parameters

sso – SuperSegmentationObject, contains the skeleton extracted from sso.skeleton_kzip_path (see SkeletonAnnotation from knossos utils).
comment_converter – dict, maps node comments to integer labels. Unspecified comments receive label -1.

Returns

np.array, an array of labels corresponding to the order of nodes in sso.skeleton[“nodes”].

syconn.reps.super_segmentation_helper.load_voxels_downsampled(sso, downsampling=(2, 2, 1), nb_threads=10)[source]

Loads the voxels of a SuperSegmentationObject with downsampling.

Parameters

sso – SuperSegmentationObject The SuperSegmentationObject to load voxels for.
downsampling – tuple The downsampling factors for each dimension (z, y, x).
nb_threads – int The number of threads to use for parallel loading.

Returns

A downsampled boolean array representing the voxels of the SSO.

Return type

np.ndarray

syconn.reps.super_segmentation_helper.majority_vote(anno, prop, max_dist)[source]

Performs smoothing of property prediction in an annotation using a sliding window and majority voting, leaving somata untouched for axoness property.

Parameters

anno – SkeletonAnnotation The annotation object containing the skeleton.
prop – str The property to average, e.g., ‘axoness’.
max_dist – int The maximum distance (in nm) for the sliding window used in majority voting.

Returns

None

syconn.reps.super_segmentation_helper.majority_vote_compartments(sso, ax_pred_key='axoness')[source]

Determines the majority compartment prediction for each connected component of the skeleton graph of an SSO, excluding soma nodes. The majority prediction is used to relabel the nodes of each connected component. By default, will save new skeleton attribute with key ax_pred_key + “_comp_maj”. Will not call sso.save_skeleton().

Parameters

sso (SuperSegmentationObject) – The SuperSegmentationObject whose skeleton compartments are being processed.
ax_pred_key (str) – The key for accessing axoness predictions stored in the SSO’s skeleton.

Returns

None

syconn.reps.super_segmentation_helper.majorityvote_skeleton_property(sso, prop_key, max_dist=10000, return_res=False)[source]

Applies a sliding window majority vote along the skeleton of a SuperSegmentationObject (SSO), utilizing the specified property key. This vote determines the prevailing value over a path length for each skeletal node.

Parameters

sso (super_segmentation.SuperSegmentationObject) – The SuperSegmentationObject (SSO) whose skeleton property is being processed. This is the cell reconstruction object.
prop_key (str) – The key identifier of the property to be processed.
max_dist (int) – The maximum path length permitted for the sliding window along the L2-distance weighted skeleton graph.
return_res (bool) – If True, the function returns the resulting majority vote values for the skeleton nodes.

Return type

ndarray

Returns

If return_res is True, a numpy array with the majority vote result for each skeleton node is returned. When return_res is False, the function does not return a value; instead, it directly modifies the SSO without invoking sso.save_skeleton().

Note

The function will not call sso.save_skeleton() post-processing. To persist modifications, this call must be executed manually after function execution if necessary.

syconn.reps.super_segmentation_helper.map_myelin2coords(coords, cube_edge_avg=array([11, 11, 5]), thresh_proba=127, thresh_majority=0.5, mag=4)[source]

Retrieves a myelin prediction at every location in coords. The classification is the majority label within a cube of size cube_edge_avg around the respective location. Voxels are classified as myelinated by thresholding the probability using thresh_proba. A ratio thresh_majority decides the label.

Examples

The entire myelin prediction for a single cell reconstruction including a smoothing via majorityvote_skeleton_property() is implemented as:

from syconn import global_params from syconn.reps.super_segmentation import * from syconn.reps.super_segmentation_helper import map_myelin2coords, majorityvote_skeleton_property

# init. example data set global_params.wd = ‘~/SyConn/example_cube1/’

# initialize example cell reconstruction ssd = SuperSegmentationDataset() ssv = list(ssd.ssvs)[0] ssv.load_skeleton()

# get myelin predictions myelinated = map_myelin2coords(ssv.skeleton[“nodes”], mag=4) ssv.skeleton[“myelin”] = myelinated # this will generate a smoothed version at ssv.skeleton[“myelin_avg10000”] majorityvote_skeleton_property(ssv, “myelin”) # store results as a KNOSSOS readable k.zip file ssv.save_skeleton_to_kzip(dest_path=’~/{}_myelin.k.zip’.format(ssv.id),

additional_keys=[‘myelin’, ‘myelin_avg10000’])

Parameters

coords (ndarray) – Coordinates used to retrieve myelin predictions. In voxel coordinates (mag=1).
cube_edge_avg (ndarray) – Cube size used for averaging myelin predictions for each location. The loaded data cube will always have the extent given by cube_edge_avg, regardless of the value of mag.
thresh_proba (float) – Classification threshold in uint8 values (0..255).
thresh_majority (float) – Majority ratio for myelin (between 0..1), i.e. thresh_majority=0.1 means that 10% myelin voxels within cube_edge_avg will flag the corresponding locations as myelinated.
mag (int) – Data magnification level used to retrieve the prediction results.

Returns

no myelin, 1: myelinated neuron) for each coordinate.

Return type

An array of myelin predictions (0

syconn.reps.super_segmentation_helper.nodes_in_pathlength(anno, max_path_len)[source]

Identifies nodes reachable within a specified path length from each source node in an annotation.

Parameters

anno – AnnotationObject The annotation object containing the nodes.
max_path_len – float The maximum distance from the source node.

Returns

A list of lists, each containing nodes reachable within max_path_len. The outer list has a length equal to the number of nodes in anno, obtained by len(anno.getNodes()).

Return type

List[List[SkeletonNode]]

syconn.reps.super_segmentation_helper.pred_sv_chunk_semseg(args)[source]

Helper method for predicting the 2D projections of supervoxels in chunks.

Parameters: args – A tuple containing paths to the supervoxel storages to be processed, the model, and supervoxel and prediction parameters.
Returns: None. The predictions are stored in the supervoxel storages.

syconn.reps.super_segmentation_helper.pred_svs_semseg(model, views, pred_key=None, svs=None, return_pred=False, nb_cpus=1, verbose=False, bs=10)[source]

Predicts semantic segmentation for views of a list of Supervoxels (SVs) and optionally saves them using SV.save_views. This is an efficient helper function designed for chunked predictions and requires pre-loaded views.

Parameters

model – The model used for semantic segmentation prediction.
views – List[np.array] N_SV each with np.array of shape [N_LOCS, N_CH, N_VIEWS, X, Y] as uint8 scaled from 0 to 255, representing views of each supervoxel.
pred_key – The key under which predictions will be saved in SV view storages.
svs – Optional[list[SegmentationObject]] List of SegmentationObject instances corresponding to SVs. If not provided, SVs must be pre-loaded.
return_pred – Optional[bool] If True, returns the predicted label views instead of saving them.
nb_cpus – int The number of CPUs to use for saving SV views.
verbose – bool If True, prints additional output during the prediction process.
bs (int) – int Batch size used during inference.

Returns: list[np.array]: If ‘return_pred=True’, returns the label views of the supervoxels as numpy arrays. Otherwise, the predictions are saved without returning any value.

syconn.reps.super_segmentation_helper.predict_sso_celltype(sso, model, nb_views_model=20, use_syntype=True, overwrite=False, pred_key_appendix='', da_equals_tan=True, n_classes=7, save_to_attr_dict=True)[source]

Predicts the cell type of a SuperSegmentationObject based on local views and synapse type ratio feature, using cached views. This method uses precomputed views, also used for axon and spine prediction. To generate predictions without using cached views, refer to celltype_of_sso_nocache. The random view subsets used for prediction are prepared by the function sso_views_to_modelinput(). The final cell type prediction is determined by the majority vote across all subset predictions.

Parameters

sso (super_segmentation.SuperSegmentationObject) – SuperSegmentationObject The SuperSegmentationObject to predict the cell type for.
model (Any) – nn.Module The prediction model, typically a neural network module.
nb_views_model (int) – int The number of views to use in the model for prediction.
use_syntype – bool Whether to consider the synapse type in the prediction. n_classes must be 7 if da_equals_tan is True.
overwrite (bool) – bool If True, overwrite existing predictions. Use this option with caution to ensure that previous predictions are not unintentionally lost.
pred_key_appendix – str An appendix to the prediction key used for result storage. This allows differentiation between different prediction iterations or parameters.
da_equals_tan (bool) – bool If True, merge DA and TAN classes into one category. This parameter should be set to True only when n_classes is 7, to maintain consistency in class representation across predictions.
n_classes (int) – int The number of output classes in the model. It has to be set to 7 if da_equals_tan is True in order to align with the merged class definitions.
save_to_attr_dict (bool) – bool If True, persist the prediction result in the SuperSegmentationObject’s attribute dictionary for subsequent retrieval and analysis.

Returns

None

syconn.reps.super_segmentation_helper.predict_views_semseg(views, model, batch_size=10, verbose=False)[source]

Predicts semantic segmentation for a given array of views using a specified model. It processes the array in batches and can provide verbose output.

Parameters

views – A numpy array of shape [N_LOCS, N_CH, N_VIEWS, X, Y], where N_LOCS is the number of locations, N_CH is the number of channels (e.g., shape of cell, mitochondria, synaptic junctions, and vesicle clouds), N_VIEWS is the number of views per location, and X, Y are the spatial dimensions of each view. The array should be uint8 scaled from 0 to 255.
model – A PyTorch model used for prediction.
batch_size – The batch size to use during prediction.
verbose – If True, additional output is printed during the process.

Returns

A numpy array of predicted views with the same shape as the input ‘views’.

syconn.reps.super_segmentation_helper.prune_stub_branches(sso=None, nx_g=None, scal=None, len_thres=1000, preserve_annotations=True)[source]

Removes short stub branches from a skeleton graph, maintaining the true morphology.

Parameters

sso – Optional[SuperSegmentationObject] The SuperSegmentationObject containing the skeleton. If None, no SuperSegmentationObject will be returned.
nx_g – networkx.Graph The graph representing the skeleton.
scal – Optional[np.array of size 3] The scaling factor for the coordinates. Defaults to the original scale.
len_thres – int The threshold for the length below which branches will be pruned.
preserve_annotations – bool If True, annotations are preserved during pruning.

Returns

The pruned SuperSegmentationObject (if provided) and the pruned skeleton graph. If sso is None, only the pruned graph is returned.

Return type

Tuple[Optional[SuperSegmentationObject], networkx.Graph]

syconn.reps.super_segmentation_helper.radius_correction_found_vertices(sso, plump_factor=1, num_found_vertices=10)[source]

Estimates the diameters of skeleton nodes by finding the median distance to the nearest mesh vertices.

Parameters

sso (super_segmentation.SuperSegmentationObject) – SuperSegmentationObject The SuperSegmentationObject containing the skeleton and mesh.
plump_factor (int) – int A factor to adjust the estimated radius. This is a multiplication factor for the radius.
num_found_vertices (int) – int The number of closest vertices to query for each node.

Returns

The updated skeleton with diameters estimated.

syconn.reps.super_segmentation_helper.save_view_pca_proj(sso, t_net, pca, dest_dir, ls=20, s=6.0, special_points=(), special_markers=(), special_kwargs=())[source]

syconn.reps.super_segmentation_helper.semseg2mesh(sso, semseg_key, nb_views=None, dest_path=None, k=1, colors=None, force_recompute=False, index_view_key=None)[source]

Maps semantic segmentation predictions to the mesh of a SuperSegmentationObject (SSO) and optionally saves the colored mesh to a file.

Parameters

sso – The SuperSegmentationObject whose mesh will be colored based on semantic segmentation predictions.
semseg_key – The key identifying the views containing the semantic segmentation results.
index_view_key – The key identifying the views containing the vertex indices. If set, nb_views is ignored.
nb_views – The number of views used for the prediction, required if index_view_key is not set.
dest_path – If provided, the colored mesh will be written to a k.zip file at this path.
k – The number of nearest vertices to average over when mapping predictions to the mesh. If k=0, unpredicted vertices will be treated as ‘unpredicted’ class.
colors – An array mapping labels to colors. If None, the majority label is returned instead. Note to add a color for unpredicted vertices if k==0; here illustrated with by the spine prediction example: if k=0: [neck, head, shaft, other, background, unpredicted] else: [neck, head, shaft, other, background].
force_recompute – If True, forces re-mapping of the predicted labels to the mesh vertices.

Notes

k>0 should only be used if a prediction for all vertices is absolutely required. Filtering of background and unpredicted vertices should be favored if time complexity is critical.

Returns: If dest_path is None, returns a tuple containing the mesh indices, vertices, normals, and colors. Otherwise, the function has no return value and the colored mesh is saved to the specified path.

syconn.reps.super_segmentation_helper.semseg2mesh_counter(index_arr, label_arr, bg_label, count_arr)[source]

Counts the occurrence of labels in ‘label_arr’ for each vertex ID in ‘index_arr’ and accumulates the counts in ‘count_arr’.

Parameters

index_arr (ndarray) – A flat array of contiguous vertex IDs, corresponding to the order in ‘label_arr’.
label_arr (ndarray) – A flat array of semantic segmentation prediction results, corresponding to the order in ‘index_arr’. The maximum value must be below ‘bg_label’.
bg_label (int) – The label used to represent the background, which will not be counted.
count_arr (ndarray) – A zero-initialized array to store the per-vertex counted labels from ‘label_arr’. It must have the shape (M, bg_label), where M is the number of vertices of the underlying mesh.

Return type

ndarray

Returns

An array filled with the per-vertex label counts.

syconn.reps.super_segmentation_helper.semseg_of_sso_nocache(sso, model, semseg_key, ws, nb_views, comp_window, k=1, dest_path=None, verbose=False, add_cellobjects=True, bs=10)[source]

Renders raw and index views at rendering locations determined by comp_window and according to given view properties without storing them on the file system. Views will be predicted with the given model and maps prediction results onto mesh. Vertex labels are stored on file system and can be accessed via sso.label_dict(‘vertex’)[semseg_key]. If sso._sample_locations is None, generate_rendering_locs(verts, comp_window / 3) will be called to generate rendering locations.

Examples

Given a cell reconstruction exported as kzip (see ) at cell_kzip_fn the compartment prediction (axon boutons, dendrite, soma) can be started via the following script:

# set working directory to obtain models
global_params.wd = '~/SyConn/example_cube1/'

# get model for compartment detection
m = get_semseg_axon_model()
view_props = global_params.config['compartments']['view_properties_semsegax']
view_props["verbose"] = True

# load SSO instance from k.zip file
sso = init_sso_from_kzip(cell_kzip_fn, sso_id=1)

# run prediction and store result in new kzip
cell_kzip_fn_axon = cell_kzip_fn[:-6] + '_axon.k.zip'
semseg_of_sso_nocache(sso, dest_path=cell_kzip_fn_axon, model=m,
                      **view_props)

syconn.reps.segmentation module

class syconn.reps.segmentation.SegmentationDataset(obj_type, version=None, working_dir=None, scaling=None, version_dict=None, create=False, config=None, n_folders_fs=None, cache_properties=None)[source]

Bases: syconn.reps.rep_helper.SegmentationBase

Represents a set of supervoxel objects within connectomics data. Each supervoxel corresponds to a distinct anatomical or functional region. This class provides utilities for managing and accessing these objects, their properties, and their relationships.

_type

Defines the type of supervoxel objects in the dataset.

Type: str

_n_folders_fs

Number of folders in the dataset directory tree.

Type: int

_sizes

Array of all supervoxel sizes in the dataset.

Type: np.ndarray

_ids

Array of unique identifiers for all supervoxels.

Type: np.ndarray

_rep_coords

Array of representative coordinates for all supervoxels.

Type: np.ndarray

_config

Configuration object with dataset-specific parameters.

Type: DynConfig

_soid2ix

Maps supervoxel IDs to their index in the dataset.

Type: dict

_property_cache

Cache for quick access to supervoxel properties.

Type: dict

_version

Version identifier for the dataset.

Type: str

_scaling

Voxel size in nanometers (XYZ).

Type: np.ndarray

_working_dir

Path to the dataset working directory.

Type: str

Examples

To initialize the SegmentationDataset for cell supervoxels:: sd_cell = SegmentationDataset(‘sv’)
After running dataset analysis, load properties from numpy arrays:: sd_cell.load_numpy_data(‘size’)

Note: More detailed information about supervoxel properties and additional functionality can be found in the original docstring.

property config: syconn.handler.config.DynConfig

Retrieves the configuration object which contains all dataset-specific parameters. See DynConfig.

Return type: DynConfig
Returns: The configuration object.

enable_property_cache(property_keys)[source]

Adds properties to the cache for faster access.

Parameters: property_keys (Iterable[str]) – Property keys. Numpy cache arrays must exist.

property exists: bool

Verifies if the dataset directory, referenced by path, exists.

Return type: bool
Returns: True if the dataset directory exists, False otherwise.

get_segmentation_object(obj_id, create=False, **kwargs)[source]

Factory method for retrieving SegmentationObject instances which are part of this dataset.

Parameters

obj_id (Union[int, List[int]]) – A single ID or a list of IDs for the supervoxel(s) to retrieve.
create (bool) – If True, creates the folder hierarchy for the requested supervoxel(s).

Return type

Union[SegmentationObject, List[SegmentationObject]]

Returns

The requested SegmentationObject object or a list of SegmentationObject instances.

get_segmentationdataset(obj_type)[source]

Factory method to retrieve a SegmentationDataset of a specified supervoxel type.

Parameters: obj_type (str) – Dataset of supervoxels with type obj_type.
Return type: SegmentationDataset
Returns: The requested SegmentationDataset object containing the specified supervoxel type.

get_volume(source='total')[source]

Calculates the total volume of the region adjacency graph (RAG).

Parameters

source (str) – Allowed sources: ‘total’ (all SVs contained in
SegmentationDataset ('sv')), 'neuron' (use glia-free RAG) –
'glia' (use glia RAG) –
calculation. (in the volume) –

Return type

float

Returns

The volume of the RAG in cubic millimeters.

property ids: numpy.ndarray

Retrieves the array of IDs for all supervoxels that are part of this dataset.

Return type: ndarray
Returns: An array of all supervoxel IDs which are part of this dataset.

iter_so_dir_paths()[source]

Iterates over all possible SegmentationObject storage base directories. This iterator may return paths to storages that do not exist if no object fell into its ID bucket.

Return type: Iterator[str]
Returns: An iterator yielding paths to ID storage base folders.

load_numpy_data(prop_name, allow_nonexisting=True)[source]

Loads a cached array of supervoxel properties. The ordering of the returned array will correspond to ids.

Parameters

prop_name – Identifier of the requested cache array.
allow_nonexisting (bool) – If False, raises an error for missing numpy files.

Return type

ndarray

Returns

A numpy array of the requested property prop_name.

load_version_dict()[source]: Loads the version dictionary from a .pkl file.

property n_folders_fs: int

Retrieves the number of folders in the SegmentationDataset directory tree.

Return type: int
Returns: The number of folders in this directory tree as an integer.

property path: str

Retrieves the full path to the SegmentationDataset.

Return type: str
Returns: The path to this SegmentationDataset as a string.

property path_ids: str

Retrieves the path to the cached array of the object IDs.

Return type: str
Returns: The path to the ids.npy file as a string.

property path_rep_coords: str

Retrieves the path to the cached array of the object representative coordinates.

Return type: str
Returns: The path to the rep_coords.npy file as a string.

property path_sizes: str

Retrieves the path to the cached array of the object voxel sizes.

Return type: str
Returns: The path to the sizes.npy file as a string.

property rep_coords: numpy.ndarray

Retrieves the array of representative coordinates for all supervoxels part of this dataset. The ordering corresponds to ids.

Return type: ndarray
Returns: An array of representative coordinates of all supervoxel.

save_version_dict()[source]: Saves the version dictionary to a .pkl pickle file.

property scaling: numpy.ndarray

Retrieves the voxel size in nanometers (XYZ).

Return type: ndarray
Returns: Voxel size in nanometers (XYZ).

property sizes: numpy.ndarray

Retrieves the array of sizes for all supervoxels in the dataset.

Return type: ndarray
Returns: A size array of all supervoxel which are part of this dataset. The ordering of the returned array will correspond to ids.

property so_dir_paths: List[str]

Retrieves a sorted list of paths to all supervoxel object directories in the directory tree from the so_storage_path.

Return type: List[str]
Returns: A sorted list of paths to supervoxel object directories.

property so_storage_path: str

Provides the path to the root folder for supervoxel storage.

Returns: The path to the root directory.
Return type: str

property so_storage_path_base: str

Retrieves the base name (‘so_storage’) of the root folder for supervoxel storage.

Return type: str
Returns: The base name of the root folder as a string.

property soid2ix

Retrieves or creates a mapping from supervoxel IDs to their index in the dataset.

Returns: A dictionary mapping supervoxel IDs to indices.

property sos: Generator[syconn.reps.segmentation.SegmentationObject, None, None]

Generator that yields all SegmentationObject instances associated with this dataset.

Yields: SegmentationObject
Return type: Generator[SegmentationObject, None, None]

property type: str

Retrieves the type of SegmentationDataset.

Return type: str
Returns: String identifier of the object type.

property version: str

Retrieves the version identifier of the dataset.

Return type: str
Returns: String identifier of the version.

property version_dict_exists: bool

Checks whether the version dictionary file exists, referred to by version_dict_path.

Return type: bool
Returns: True if the version_dict.pkl file exists, False otherwise.

property version_dict_path: str

Retrieves the path to the version dictionary pickle file.

Return type: str
Returns: The path to the version_dict.pkl file as a string.

property working_dir: str

Retrieves the working directory of the SegmentationDataset instance.

Return type: str
Returns: The working directory of this SegmentationDataset as a string.

class syconn.reps.segmentation.SegmentationObject(obj_id, obj_type='sv', version=None, working_dir=None, rep_coord=None, size=None, scaling=None, create=False, voxel_caching=True, mesh_caching=False, view_caching=False, config=None, n_folders_fs=None, enable_locking=True, skeleton_caching=True, mesh=None)[source]

Bases: syconn.reps.rep_helper.SegmentationBase

Represents individual supervoxels. Used for cell shape (‘sv’), cell organelles, e.g. mitochondria (‘mi’), vesicle clouds (‘vc’) and synaptic junctions (‘sj’).

Examples

Can be used to initialized single SegmentationObject object of a specific type, is also returned by get_segmentation_object():

from syconn.reps.segmentation import SegmentationObject, SegmentationDataset
cell_sv = SegmentationObject(obj_id=.., obj_type='sv', working_dir='..')
cell_sv.load_attr_dict()  # populates `cell_sv.attr_dict`

cell_sd = SegmentationDataset(obj_type='sv', working_dir='..')
cell_sv_from_sd = cell_sd.get_segmentation_object(obj_id=cell_sv.id)
cell_sv_from_sd.load_attr_dict()

keys1 = set(cell_sv.attr_dict.keys())
keys2 = set(cell_sv_from_sd.attr_dict.keys())
print(keys1 == keys2)

attr_dict: Attribute dictionary which serves as a general-purpose container. Accessed via the AttributeDict interface.

enable_locking: If True, enables file locking.

property attr_dict_exists: bool

Checks if an attribute dictionary file exists at attr_dict_path for the SegmentationObject.

Return type: bool
Returns: True if the attribute dictionary file exists, False otherwise.

property attr_dict_path: str

Retrieves the path to the attribute storage.

Returns: The path to the attribute storage.
Return type: str

attr_exists(attr_key)[source]

Checks if attr_key exists in either attr_dict or at attr_dict_path.

Parameters: attr_key (str) – The attribute key to check.
Returns: True if the attribute exists, False otherwise.
Return type: bool

axoness_preds(pred_key_appendix='')[source]

Predicts the axoness of this supervoxel at every sample location based on img2scalar CMN.

Parameters

pred_key_appendix (str) – Identifier for specific axon predictions. Used only during development.

Returns

The axon prediction of this supervoxel at each sample location: (0: dendrite, 1: axon, 2: soma).

Return type

np.ndarray

axoness_probas(pred_key_appendix='')[source]

Retrieves the axon probabilities (0: dendrite, 1: axon, 2: soma) based on img2scalar CMN of this supervoxel at every sample location. The probabilities underlie the attribute axoness_preds and are only valid if type is sv.

Parameters

pred_key_appendix (str) – Identifier for specific axon predictions. Only used during
development. –

Returns

The axon probabilities of this supervoxel at every sample_locations.

Return type

np.ndarray

property bounding_box: numpy.ndarray

Retrieves the bounding box of the SegmentationObject.

Return type: ndarray
Returns: An array representing the bounding box (XYZ).

calculate_bounding_box(voxel_dc=None)[source]

Calculates the supervoxel bounding_box.

Parameters: voxel_dc (Optional[Dict[int, ndarray]]) – Pre-loaded dictionary which contains the voxel data of this object.

calculate_rep_coord(voxel_dc=None)[source]

Calculates or loads the supervoxel representative coordinate.

Parameters: voxel_dc (Optional[Dict[int, ndarray]]) – Pre-loaded dictionary which contains the voxel data of this object.

calculate_size(voxel_dc=None)[source]

Calculates the size of the supervoxel object size.

Parameters: voxel_dc (Union[VoxelStorageDyn, VoxelStorage, None]) – Pre-loaded dictionary which contains the voxel data of this object.

clear_cache()[source]

Clears the cached data for the following:

voxels
voxel lists
views
skeletons

Note: It doesn’t clear cached data for meshes as indicated in the previous version.

property config: syconn.handler.config.DynConfig

Retrieves the configuration object containing all dataset-specific parameters.

Return type: DynConfig
Returns: The configuration object.

copy2dir(dest_dir, safe=True)[source]

Copies all files from the supervoxel object’s directory to a specified destination directory.

Parameters

dest_dir (str) – Destination directory where all files contained in py:attr:~segobj_dir will be copied to.
safe (bool) – If True, existing files at the destination will not be overwritten. Defaults to True.

Examples

To copy the content of this supervoxel object (sv_orig) to the destination of another (e.g., not yet existing) supervoxel object (sv_target), use: sv_orig.copy2dir(sv_target.segobj_dir).

property cs_partner: Optional[List[int]]

Retrieves the IDs of the two supervoxels that are part of the contact site specific attribute.

Return type: Optional[List[int]]
Returns: A list of two IDs if the object type is ‘cs’, otherwise None.

property dataset: syconn.reps.segmentation.SegmentationDataset

Factory method to create a ~syconn.reps.segmentation.SegmentationDataset instance to which this object belongs.

Returns: An instance of SegmentationDataset.
Return type: ~syconn.reps.segmentation.SegmentationDataset

glia_pred(thresh, pred_key_appendix='')[source]

Predicts if the supervoxel is a glia or neuron. Only valid if type is sv.

Parameters

thresh (float) – The classification threshold.
pred_key_appendix (str) – An identifier for specific glia predictions. Only used during development.

Returns

The glia prediction for this supervoxel (0: neuron, 1: glia).

Return type

int

glia_proba(pred_key_appendix='')[source]

Retrieves the glia probability of this supervoxel. This is only valid if type is sv.

Parameters

pred_key_appendix (str) – Identifier for specific glia predictions. This is only used
development. (during) –

Returns

The glia prediction of this supervoxel (0: neuron, 1: glia).

Return type

float

property id: int

Retrieves the globally unique identifier of the SegmentationObject.

Return type: int
Returns: Globally unique identifier of this object.

property identifier: str

Retrieves the identifier used to form the folder name of the ~syconn.reps.segmentation.SegmentationDataset.

Return type: str
Returns: The identifier as a string.

load_attr_dict()[source]

Loads the attr_dict.

Returns: 0 if successful, -1 if attribute dictionary storage does not exist.
Return type: int

load_attributes(attr_keys)[source]

Reads attributes from attribute storage, ignoring self.attr_dict and always pulling them from storage. It does not throw KeyError but returns None for missing keys.

Parameters: attr_keys (List[str]) – List of attribute keys to be loaded from attribute storage.
Returns: Attribute values corresponding to attr_keys, returns None for missing keys.
Return type: List[Any]

load_skeleton(recompute=False)[source]

Loads the skeleton representation of this supervoxel.

Parameters: recompute (bool) – Recompute the skeleton. Currently not implemented.
Returns: Dictionary containing flat arrays of indices, vertices, diameters, and attributes.
Return type: dict

load_views(woglia=True, raw_only=False, ignore_missing=False, index_views=False, view_key=None)[source]

Loads views with specified properties.

Parameters

woglia (bool) – If True, looks for views without glia, i.e. after astrocyte separation.
raw_only (bool) – If True, ignores cell organelles projections.
ignore_missing (bool) – If True, will not throw ValueError if views do not exist.
index_views (bool) – If True, refers to index views.
view_key (Optional[str]) – Identifier of the requested views.

Returns

Views with the requested properties.

Return type

np.ndarray

load_voxel_list()[source]

Loader method of voxel_list. Loads a sparse, 2-dimensional array of voxel coordinates.

Returns: Sparse, 2-dimensional array of voxel coordinates.
Return type: np.ndarray

load_voxel_list_downsampled(downsampling=(2, 2, 1))[source]

Loads a downsampled list of voxel coordinates.

Parameters: downsampling – Tuple specifying the downsampling factor for each axis.
Returns: Downsampled list of voxel coordinates.
Return type: np.ndarray

load_voxel_list_downsampled_adapt(downsampling=(2, 2, 1))[source]

Loads a downsampled list of voxel coordinates with adaptive scaling.

Parameters: downsampling – Tuple specifying the downsampling factor for each axis.
Returns: Downsampled list of voxel coordinates with adaptive scaling.
Return type: np.ndarray

load_voxels(voxel_dc=None)[source]

Loads the voxels associated with this supervoxel.

Parameters: voxel_dc (Union[VoxelStorageDyn, VoxelStorage, None]) – Pre-loaded dictionary which contains the voxel data of this object.
Returns: 3D array of all voxels which belong to this supervoxel.
Return type: np.ndarray

load_voxels_downsampled(downsampling=(2, 2, 1))[source]

Loads downsampled voxels.

Parameters: downsampling – Tuple specifying the downsampling factor for each axis.
Returns: Downsampled voxel data.
Return type: np.ndarray

property locations_path: str

Retrieves and returns the path to the rendering location storage.

Return type: str

lookup_in_attribute_dict(attr_key)[source]

Looks up a value in the attribute dictionary.

Parameters: attr_key (str) – Attribute key to look for.
Returns: The value of attr_key in attr_dict or None if it does not exist. If key does not exist in attr_dict, tries to load from attr_dict_path.
Return type: Any

mergelist2kzip(dest_path)[source]

Writes the supervoxel agglomeration to a KNOSSOS compatible format.

Parameters: dest_path (str) – Path to the k.zip file.

property mesh: Union[Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray], List[numpy.ndarray], Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray]]

Retrieves the mesh data of this object.

Returns: indices, vertices, normals.
Return type: A tuple of three flat arrays

mesh2kzip(dest_path, ext_color=None, ply_name='')[source]

Writes the mesh to a k.zip file.

Parameters

dest_path (str) – Path to the k.zip file which contains the mesh.
ext_color (Union[Tuple[int, int, int, int], List, ndarray, None]) – Optional RGBA color tuple. If set to 0, no color will be written out, use to adapt color in Knossos.
ply_name (str) – Name of the ply file within the k.zip archive, must not end with .ply.

property mesh_area: float

Retrieves the mesh surface area of the supervoxel.

Returns: Mesh surface area in um^2.
Return type: float

property mesh_bb: numpy.ndarray

Retrieves the bounding box of the object meshes in nanometers. This is approximately the same as the scaled ‘bounding_box’. Returns an array representing the bounding box of the meshes.

Return type: ndarray

property mesh_caching: bool

Indicates if mesh data is cached.

Return type: bool
Returns: True if mesh data is cached, False otherwise.

property mesh_exists: bool

Checks if mesh data exists for the SegmentationObject.

Return type: bool
Returns: True if mesh data exists, False otherwise.

mesh_from_scratch(ds=None, **kwargs)[source]

Calculates the mesh based on get_object_mesh().

Parameters

ds (Optional[Tuple[int, int, int]]) – Downsampling of the object’s voxel data.
**kwargs – Additional keyword arguments passed to triangulation().

Returns

Mesh data including indices, vertices, and normals.

Return type

List[np.ndarray]

property mesh_path: str

Retrieves the path to the mesh storage.

Return type: str
Returns: The path to where the mesh data is stored.

property mesh_size: float

Retrieves the length of the bounding box diagonal (BBD) of the object meshes.

Return type: float
Returns: The diagonal (BBD) length of the mesh bounding box in nanometers.

property n_folders_fs: int

Retrieves the number of folders used to store the data of SegmentationObjects. This value defines the hierarchy of the folder structure organized by SegmentationDataset.

Return type: int
Returns: The number of leaf folders used for storing supervoxel data.

property rep_coord: numpy.ndarray

Retrieves the representative coordinate of this SegmentationObject, which will be the ‘rep_coord’ of the first supervoxel in ‘svs’.

Return type: ndarray
Returns: A 1D array representing the representative coordinate (XYZ).

sample_locations(force=False, save=True, ds_factor=None)[source]

Retrieves the rendering locations of this supervoxel. This is only valid for cell fragments, i.e., type must be sv.

Parameters

force – If True, overwrites existing data.
save – If True, saves the result at locations_path. Utilizes
CompressedStorage. –
ds_factor – Down sampling factor used to generate the rendering locations.

Returns

Array of rendering locations (XYZ) with shape (N, 3) in nanometers.

Return type

np.ndarray

property sample_locations_exist: bool

Checks if rendering locations have been stored.

Returns: True if rendering locations have been stored at locations_path.
Return type: bool

save_attr_dict()[source]: Saves the attribute dictionary to the attribute dictionary path. Any existing dictionary will be updated.

save_attributes(attr_keys, attr_values)[source]

Writes attributes to attribute storage. Ignores attr_dict. Values have to be serializable and will be written via the AttributeDict interface.

Parameters

attr_keys (List[str]) – List of attribute keys which will be written to attr_dict_path.
attr_values (List[Any]) – List of attribute values which will be written to attr_dict_path.

save_kzip(path, kd=None, write_id=None)[source]

Writes the supervoxel segmentation to a k.zip file.

Parameters

path (str) – The file path where the k.zip file will be saved.
kd (Optional[knossosdataset.KnossosDataset], optional) – The KnossosDataset object. If None, it will be loaded from the configuration. Defaults to None.
write_id (Optional[int], optional) – The supervoxel ID. If None, the ID of the current supervoxel object will be used. Defaults to None.

save_skeleton(overwrite=False)[source]

Saves the skeleton data of this supervoxel.

Parameters: overwrite (bool) – Overwrite existing skeleton entry.
Returns: Flat arrays of indices, vertices, and normals.
Return type: Tuple[np.ndarray, np.ndarray, np.ndarray]

save_views(views, woglia=True, cellobjects_only=False, index_views=False, view_key=None, enable_locking=None)[source]

Saves views according to its properties. If a particular view_key is provided, it must correspond to a specific type of view, like spine predictions, and all other kwargs should be set to their default values to avoid errors.

Parameters

views (ndarray) – Array of views to be saved.
woglia (bool) – If True, saves views that do not contain glia, i.e., after astrocyte separation.
cellobjects_only (bool) – If True, only cell organelle views are saved (deprecated).
index_views (bool) – If True, saves index views.
view_key (Optional[str]) – Identifier for the specific views to be saved.
enable_locking (Optional[bool]) – If True, activates file locking during the save operation.

property scaling

Retrieves the voxel size in nanometers (XYZ). Default value is taken from the config.yml file and accessible via self.config.

Returns: An array representing the voxel size in nanometers.

property segds_dir: str

Retrieves the path to the ~syconn.reps.segmentation.SegmentationDataset directory.

Return type: str
Returns: The path to the ~syconn.reps.segmentation.SegmentationDataset directory.

property segobj_dir: str

Retrieves the path to the folder where the data of this supervoxel is stored.

Returns: The path to the supervoxel data folder.
Return type: str

property shape: numpy.ndarray

Retrieves the XYZ extent of this SSV object in voxels.

Return type: ndarray
Returns: An array representing the shape/extent of this SSV object in voxels (XYZ).

property size: int

Retrieves the number of voxels in the SegmentationObject.

Return type: int
Returns: The number of voxels, represented as an integer, in the SegmentationObject.

property skeleton: dict

Retrieves the skeleton representation of this supervoxel.

Returns: “nodes”, estimated node “diameters”, and “edges”.
Return type: A dictionary containing at least three numpy arrays

property skeleton_caching

Indicates if skeleton data is cached.

Returns: True if skeleton data is cached, False otherwise.

property skeleton_dict_path: str

Retrieves the path to the skeleton storage.

Returns: The file path to the skeleton storage.
Return type: str

property skeleton_exists: bool

Checks if skeleton data exists for the SegmentationObject.

Return type: bool
Returns: True if skeleton exists, False otherwise.

property skeleton_path: str

Retrieves the path to the skeleton storage.

Returns: The path to the skeleton storage.
Return type: str

property so_storage_path: str

Retrieves the path to the entry folder of the directory tree containing all supervoxel data of the corresponding ~syconn.reps.segmentation.SegmentationDataset.

Return type: str
Returns: The path to the supervoxel data storage.

property so_storage_path_base: str

Retrieves the base folder name for the SegmentationObject storage.

Return type: str
Returns: The base folder name as a string.

split_component(dist, new_sd, new_id)[source]

Splits the supervoxel into components based on a distance threshold and assigns them new IDs.

Parameters

dist (float) – The distance threshold for splitting the supervoxel.
new_sd ('SegmentationDataset') – The SegmentationDataset to which the new components will belong.
new_id (int) – The starting ID for the new components.

total_edge_length()[source]

Calculates the total edge length of the supervoxel skeleton in nanometers.

Returns: Sum of all edge lengths (L2 norm) in skeleton.
Return type: float

property type: str

Retrieves the type of the supervoxel. This can be used to initialize a single SegmentationObject of a specific type or the corresponding dataset collection handled with the SegmentationDataset class.

Keys used include: * ‘mi’: Mitochondria. * ‘vc’: Vesicle clouds. * ‘sj’: Synaptic junction. * ‘syn_ssv’: Synapses between two. * ‘syn’: Synapse fragment between two SegmentationObject objects. * ‘cs’: Contact site.

Example usage:

from syconn.reps.segmentation import SegmentationObject, SegmentationDataset cell_sv = SegmentationObject(obj_id=.., obj_type=’sv’, working_dir=’..’) cell_sv.load_attr_dict() # populates cell_sv.attr_dict

cell_sd = SegmentationDataset(obj_type=’sv’, working_dir=’..’) cell_sv_from_sd = cell_sd.get_segmentation_object(obj_id=cell_sv.id) cell_sv_from_sd.load_attr_dict()

keys1 = set(cell_sv.attr_dict.keys()) keys2 = set(cell_sv_from_sd.attr_dict.keys()) print(keys1 == keys2)

Return type: str
Returns: The type of the supervoxel as a string identifier.

property version: str

Retrieves the version of the SegmentationDataset this object belongs to.

Return type: str
Returns: String identifier of the object’s version.

property view_caching

Indicates if view data is cached. If True, view data is stored for future use.

Returns: True if view data is cached, False otherwise.

view_path(woglia=True, index_views=False, view_key=None)[source]

Retrieves the path to the view storage.

Parameters

woglia – If True, looks for views without glia, i.e., after astrocyte separation.
index_views – If True, refers to index views.
view_key – Identifier of the requested views.

Return type

str

Returns

Path to the view storage.

views(woglia, index_views=False, view_key=None)[source]

Retrieves the views of this supervoxel. Valid only for cell fragments, i.e. type must be sv.

Parameters

woglia (bool) – If True, retrieves views without glia, i.e., after astrocyte separation.
index_views (bool) – If True, retrieves index views.
view_key (Optional[str]) – Identifier of the requested views.

Returns

The requested view array or -1 if it does not exist.

Return type

Union[np.ndarray, int]

views_exist(woglia, index_views=False, view_key=None)[source]

Determines if rendering locations have been stored at the specified view path.

Parameters

woglia (bool) – If True, checks for views without glia, i.e. after astrocyte separation.
index_views (bool) – If True, checks for index views.
view_key (Optional[str]) – Identifier of the requested views.

Returns

True if rendering locations have been stored.

Return type

bool

property voxel_caching: bool

Specifies whether the voxel data should be cached after loading. If True, voxel data is retained in cache post load process.

Return type: bool
Returns: A boolean indicating whether voxel data is cached (True) or not (False) after loading.

property voxel_list: numpy.ndarray

Retrieves the voxels associated with the SSV object.

Return type: ndarray
Returns: A 2D array with sparse voxel coordinates.

property voxel_path: str

Retrieves the path to the voxel storage. See VoxelStorageDyn for more details.

Return type: str
Returns: The path to the voxel storage.

property voxels: numpy.ndarray

Retrieves the voxels associated with the SSV object.

Return type: ndarray
Returns: A 3D binary array indicating voxel locations.

property voxels_exist: bool

Checks if voxel data exists for the SegmentationObject.

Return type: bool
Returns: True if voxel data exists, False otherwise.

property working_dir: str

Retrieves the working directory of the SegmentationObject.

Return type: str
Returns: The working directory path as a string. This working directory corresponds to the SegmentationObject’s operating environment.

syconn.reps.segmentation_helper module

syconn.reps.segmentation_helper.acquire_obj_ids(sd)[source]

Retrieves all object IDs present in the SegmentationDataset. If an id array is available, it loads it to assemble the id list; otherwise, it iterates over all voxel / attribute dicts which is slower.

Parameters: sd (SegmentationDataset) – The SegmentationDataset from which to acquire object IDs.
Returns: The object IDs are stored within the SegmentationDataset instance.
Return type: None

syconn.reps.segmentation_helper.calc_center_of_mass(point_arr)[source]

Calculates the closest point to the center of mass from a given array of points. This function identifies a representative point nearest to the geometric center of a collection of points. It assumes uniform point distribution in isotropic units.

Parameters: point_arr (ndarray) – A numpy array of points, where each point is represented by its coordinates (x, y, z). The points should conform to isotropic units preferably in nanometers.
Return type: ndarray
Returns: A numpy array representing the single point that is closest to the center of mass of the input array of points.

syconn.reps.segmentation_helper.find_missing_sv_attributes(sd, attr_key, n_cores=20)[source]

Identifies missing attributes for supervoxels in a SegmentationDataset.

Parameters

sd (SegmentationDataset) – The SegmentationDataset to check for missing attributes.
attr_key (str) – The attribute key to check for in each supervoxel.
n_cores (int) – The number of CPU cores to use for parallel processing.

Returns

The result is not returned but may be used internally for processing.

Return type

None

syconn.reps.segmentation_helper.find_missing_sv_skeletons(svs, n_cores=20)[source]

syconn.reps.segmentation_helper.find_missing_sv_views(sd, woglia, n_cores=20)[source]

syconn.reps.segmentation_helper.generate_skeleton_sv(so)[source]

Generates a simplified skeleton representation for a given SegmentationObject. This function is a quick solution for creating a skeleton, primarily used for glia predictions. The skeleton consists of nodes, edges, and uniform diameters.

Parameters: so (SegmentationObject) – The SegmentationObject instance for which the skeleton is to be generated.
Return type: Dict[str, ndarray]
Returns: A dictionary containing the keys “nodes”, “edges”, and “diameters”. Each key maps to a value of 1, indicating a basic scaffold structure without any specific measurements or attributes.

syconn.reps.segmentation_helper.get_sd_load_distribution(sd, use_vxsize=True)[source]

Calculates the load distribution, which is the number of objects per storage, for the given SegmentationDataset’s AttributeDicts. This is useful for understanding the distribution of data across different storage units and can help in optimizing data access patterns.

Parameters

sd (SegmentationDataset) – An instance of SegmentationDataset for which the load distribution is to be calculated.
use_vxsize (bool) – A boolean flag indicating whether to use voxel size for calculating the load distribution (default is False). If True, the load is calculated based on the voxel size of each object.

Return type

ndarray

Returns

A numpy array representing the load distribution across the storage units of the SegmentationDataset. Each element in the array corresponds to the number of objects in a storage unit.

syconn.reps.segmentation_helper.glia_pred_so(so, thresh, pred_key_appendix)[source]

Classifies a cell supervoxel as neuron or glia based on the provided threshold.

Parameters

so (SegmentationObject) – The cell supervoxel object to classify.
thresh (float) – The probability threshold above which the supervoxel is classified as glia.
pred_key_appendix (str) – A string appended to the prediction key used to retrieve glia probabilities.

Returns

The classification result, where 0 indicates neuron and 1 indicates glia.

Return type

int

syconn.reps.segmentation_helper.glia_proba_so(so, pred_key_appendix)[source]

Calculates the mean glia probability for a cell supervoxel.

Parameters

so (SegmentationObject) – The cell supervoxel object for which to calculate the probability.
pred_key_appendix (str) – A string appended to the prediction key used to retrieve glia probabilities.

Returns

The mean glia probability of a cell supervoxel with 0 indicating neuron and: 1 indicating glia.

Return type

float

syconn.reps.segmentation_helper.load_mesh(so, recompute=False)[source]

Loads the mesh representation of a SegmentationObject.

Parameters

so (SegmentationObject) – The SegmentationObject from which to load the mesh.
recompute (bool) – If True, the mesh will be recomputed instead of loaded from storage. If False, the mesh is loaded without re-computation.

Returns

A tuple containing indices, vertices, and normals of the mesh,: all flattened. TODO: Currently ignores potential color/label array.

Return type

MeshType

syconn.reps.segmentation_helper.load_skeleton(so, recompute=False)[source]

Loads the skeleton representation of a SegmentationObject.

Parameters

so (SegmentationObject) – The SegmentationObject from which to load the skeleton.
recompute (bool) – If True, the skeleton will be recomputed and not stored in SkeletonStorage. (default=False)

Returns

A dictionary containing “nodes”, “diameters”, and “edges” of: the skeleton.

Return type

dict

syconn.reps.segmentation_helper.load_so_attr_bulk(sos, attr_keys, use_new_subfold=True, allow_missing=False)[source]

Bulk loader for SegmentationObject (SO) meshes. This method minimizes IO by loading IDs from the same storage at the same time and organizes them into a dictionary. If only one attr_key is provided, a single dictionary is returned; otherwise, a dictionary of dictionaries for multiple keys is returned. Existing attributes in the object’s attr_dict are checked, leveraging cache_properties for efficiency with large datasets.

Parameters

sos (List[ForwardRef]) – A list of SegmentationObjects for which to load attributes.
attr_keys (Union[str, Iterable[str]]) – A single attribute key or an iterable of keys to load.
use_new_subfold (bool) – If True, uses the new sub-folder structure for storage paths, beneficial for managing storage.
allow_missing (bool) – If True, sets missing attribute values to None; otherwise, raises a KeyError.

Returns

A dictionary or a dict of dicts with object IDs as keys and attribute values as values, based on the provided attr_keys.

Return type

Union[Dict[str, Dict[int, Any]], Dict[int, Any]]

syconn.reps.segmentation_helper.load_so_meshes_bulk(sos, use_new_subfold=True, cache_decomp=True)[source]

Bulk loader for SegmentationObject (SO) meshes. Minimizes IO by loading IDs from the same storage at the same time. This will not assign the _mesh attribute!

Parameters

sos (Union[List[ForwardRef], Iterable[ForwardRef]]) – SegmentationObjects or an iterable of SegmentationObjects for which to load meshes.
use_new_subfold (bool) – Use new sub-folder structure for storage if True.
cache_decomp – Cache decompressed meshes.

Returns

ID, value: mesh data.

Return type

Dictionary, key

syconn.reps.segmentation_helper.load_so_voxels_bulk(sos, use_new_subfold=True, cache_decomp=True)[source]

Parameters

sos (List[ForwardRef]) –
use_new_subfold (bool) –
cache_decomp –

Returns:

syconn.reps.segmentation_helper.load_voxel_list(so)[source]

Loads a list of voxel coordinates for a SegmentationObject.

Parameters: so (SegmentationObject) – SegmentationObject. The SegmentationObject from which to load voxel coordinates.
Returns: 2D array of coordinates to all voxels in the SegmentationObject.
Return type: np.ndarray

syconn.reps.segmentation_helper.load_voxel_list_downsampled(so, downsampling=(2, 2, 1))[source]

Loads a downsampled list of voxel coordinates for a SegmentationObject.

Parameters

so – The SegmentationObject from which to load downsampled voxel coordinates.
downsampling – A tuple indicating the downsampling factors along each axis.

Returns

A downsampled list of voxel coordinates.

Return type

np.ndarray

TODO: refactor, probably more efficient implementation possible.

syconn.reps.segmentation_helper.load_voxel_list_downsampled_adapt(so, downsampling=(2, 2, 1))[source]

Loads a downsampled list of voxel coordinates for a SegmentationObject, adapting the downsampling if necessary.

Parameters

so – The SegmentationObject from which to load downsampled voxel coordinates.
downsampling – A tuple indicating the initial downsampling factors along each axis.

Returns

An adaptively downsampled list of voxel coordinates.

Return type

np.ndarray

syconn.reps.segmentation_helper.load_voxels_depr(so, voxel_dc=None)[source]

Helper function to load voxels of a SegmentationObject as 3D array. Also calculates size and bounding box and assigns it to so._size and so._bounding_box respectively (deprecated function).

Parameters

so (SegmentationObject) – The SegmentationObject from which to load voxels.
voxel_dc (Optional[VoxelStorage]) – An optional VoxelStorage instance to use for loading.

Returns: np.array: 3D binary mask array, 0: background, 1: supervoxel locations.

Return type: ndarray

syconn.reps.segmentation_helper.load_voxels_downsampled(so, ds=(2, 2, 1))[source]

Loads downsampled voxels of a SegmentationObject.

Parameters

so (SegmentationObject) – The SegmentationObject from which to load downsampled voxels.
ds (Tuple[int, int, int]) – A tuple indicating the downsampling factors along each axis.

Returns

A downsampled 3D binary mask or an empty list if no voxels are present.

Return type

Union[np.ndarray, List]

syconn.reps.segmentation_helper.prepare_so_attr_cache(sd, so_ids, attr_keys)[source]

Prepares a cache of attributes for a subset of SegmentationObjects within a SegmentationDataset.

Parameters

sd (SegmentationDataset) – The SegmentationDataset containing the SegmentationObjects.
so_ids (ndarray) – An array of SegmentationObject IDs for which to collect attributes.
attr_keys (List[str]) – A list of attribute keys to collect. The corresponding numpy arrays must exist.

Returns

A dictionary with attr_keys as keys and dictionaries of attribute values for the IDs so_ids. For example, attr_cache[attr_keys[0]] [so_ids[0]] will return the attribute value of type attr_keys[0] for the first SegmentationObject in so_ids.

Return type

Dict[str, dict]

syconn.reps.segmentation_helper.save_skeleton(so, overwrite=False)[source]

Saves the skeleton data for a SegmentationObject.

Parameters

so (SegmentationObject) – The SegmentationObject whose skeleton data is to be saved.
overwrite (bool) – If True, existing skeleton data will be overwritten.

Returns

The skeleton data is saved to the storage associated with the SegmentationObject.

Return type

None

syconn.reps.segmentation_helper.save_voxels(so, bin_arr, offset, overwrite=False)[source]

Helper function to save SegmentationObject voxels.

Parameters

so (SegmentationObject) – SegmentationObject The SegmentationObject to which the voxel data belongs.
bin_arr (ndarray) – np.array A binary mask array indicating supervoxel locations. 0: background, 1: supervoxel locations.
offset (ndarray) – np.array The offset of the binary mask array within the full dataset.
overwrite (bool) – bool If True, existing voxel data will be overwritten.

Returns

The voxel data is saved to the storage associated with the SegmentationObject.

Return type

None

syconn.reps.segmentation_helper.sv_attr_exists(args)[source]

syconn.reps.segmentation_helper.sv_skeleton_missing(sv)[source]

syconn.reps.segmentation_helper.sv_view_exists(args)[source]

syconn.reps.super_segmentation module

syconn.reps.rep_helper module

class syconn.reps.rep_helper.SegmentationBase[source]

Bases: object

Base class for segmentation-related operations in SyConn.

_scaling

Voxel size in nm.

Type: Optional[np.ndarray]

_working_dir

Working directory for file operations.

Type: Optional[str]

_config

Configuration object.

Type: Optional[DynConfig]

syconn.reps.rep_helper.assign_rep_values(target_coords, rep_coords, rep_values, nb_cpus=- 1, return_ixs=False)[source]

Assigns values corresponding to representative coordinates to every target coordinate.

Parameters

target_coords (np.array) – An array with shape [N, 3], where N is the number of target coordinates.
rep_coords (np.array) – An array with shape [M, 3], where M is the number of representative coordinates.
rep_values (np.array) – An array with shape [M, Z] containing any type of values for each representative coordinate, where Z is the dimensionality.
nb_cpus (int) – Number of CPUs to use for the computation.
return_ixs (bool) – If true, returns indices of k-closest representative coordinates for each target coordinate.

Returns

An array where each entry has the representation values: assigned for each target coordinate based on the representative coordinates.

Return type

np.array [N, Z]

syconn.reps.rep_helper.colorcode_vertices(vertices, rep_coords, rep_values, colors=None, nb_cpus=- 1, k=1, return_color=True)[source]

Assigns all vertices the kNN majority label from rep_coords/rep_values and if return_color is True, assigns those a color. Colors vertices based on the nearest representative coordinates and values.

Parameters

vertices (np.ndarray) – Array of vertices to be color-coded, shape [N, 3].
rep_coords (np.ndarray) – Representative coordinates, shape [M, 3].
rep_values (np.ndarray) – Int values to be color coded for each vertex; used as indices for colors, shape [M, 1].
colors (Optional[np.ndarray]) – Color for each rep_value, used if return_color is True.
nb_cpus (int) – Number of CPUs to use for the computation.
k (int) – Number of nearest neighbors to consider for majority voting and/or color assignment.
return_color (bool) – If False, returns the majority vote for each index; if True, returns the color values.

Returns

If return_color is True, RGBA values for every vertex from 0 to 255, shape [N, 4]. If return_color is False, indices of the representative values, shape [N, 1].

Return type

np.ndarray

syconn.reps.rep_helper.get_unique_subfold_ixs(n_folders)[source]

Generates unique IDs each associated with a unique storage dictionary.

Parameters: n_folders (int) – Total number of folders.
Returns: Array of unique IDs.
Return type: np.ndarray

syconn.reps.rep_helper.ix_from_subfold(subfold, n_folders)[source]

Determines the index from the subfolder path and number of folders.

Parameters

subfold (str) – Subfolder path.
n_folders (int) – Total number of folders.

Returns

Index of the object.

Return type

int

syconn.reps.rep_helper.ix_from_subfold_OLD(subfold, n_folders)[source]

Determines the index from the subfolder path and number of folders using the old method.

Parameters: subfold (str) – The subfolder path provided as a string to calculate the index.
Returns: Index of the object determined from the provided subfolder path.
Return type: int

syconn.reps.rep_helper.ix_from_subfold_new(subfold, n_folders)[source]

_Determines the index from the subfolder path and number of folders using the new method._

Parameters: subfold (str) – Subfolder path.
Returns: Index of the object.
Return type: int

syconn.reps.rep_helper.knossos_ml_from_ccs(cc_ixs, ccs, coords=None, comments=None)[source]

Converts list of connected components (i.e. list of SV IDs) into KNOSSOS merge list string.

Parameters

cc_ixs (Union[List[int], np.ndarray]) – Connected component IDs, i.e. super-supervoxel IDs.
ccs (List[List[int]]) – Supervoxel IDs for every connected component.
coords (Optional[np.ndarray]) – Coordinates to each connected component (in voxels).
comments (Optional[List[str]]) – Comments for each connected component.

Returns

A KNOSSOS compatible merge list in string representation.

Return type

str

syconn.reps.rep_helper.knossos_ml_from_sso(sso, comment=None)[source]

Converts the supervoxels which are part of the sso into a KNOSSOS compatible merge list string.

Parameters

sso (SuperSegmentationObject) – The SuperSegmentationObject to be converted.
comment (Optional[str]) – Comment to be included in the merge list.

Returns

A KNOSSOS compatible merge list in string representation.

Return type

str

syconn.reps.rep_helper.knossos_ml_from_svixs(sv_ixs, coords=None, comments=None)[source]

Generate a KNOSSOS merge list of an array of supervoxels with optional coordinates and comments.

Parameters

sv_ixs (Union[np.ndarray, List]) – Supervoxel IDs.
coords (Optional[Union[np.ndarray, List[np.ndarray]]]) – Representative coordinates of each supervoxel (in voxels).
comments (Optional[Union[List[str], np.ndarray]]) – Comments for each supervoxel.

Returns

A KNOSSOS compatible merge list in string representation.

Return type

str

syconn.reps.rep_helper.subfold_from_ix(ix, n_folders, old_version=False)[source]

Determines the subfolder path based on the index and number of folders.

Parameters

ix (int) – Index of the object.
n_folders (int) – Total number of folders.
old_version (bool) – Flag to use the old version of the function.

Returns

Subfolder path for the given index.

Return type

str

syconn.reps.rep_helper.subfold_from_ix_OLD(ix, n_folders, old_version=False)[source]

Determines the subfolder path based on the index and number of folders using the old method.

Parameters

ix (int) – Index of the object.
n_folders (int) – Total number of folders.
old_version (bool) – Flag to use the old version of the function. # TODO: remove ‘old_version’ as soon as possible, currently there is one usage

Returns

Subfolder path for the given index.

Return type

str

syconn.reps.rep_helper.subfold_from_ix_SSO(ix)[source]

Determines the subfolder path for a SuperSegmentationObject based on its index.

Parameters: ix (int) – Index of the SuperSegmentationObject.
Returns: Subfolder path for the SuperSegmentationObject.
Return type: str

syconn.reps.rep_helper.subfold_from_ix_new(ix, n_folders)[source]

Determines the subfolder path based on the index and number of folders using the new method.

Parameters

ix (int) – Index of the object.
n_folders (int) – Total number of folders.

Returns

Subfolder path for the given index.

Return type

str

syconn.reps.rep_helper.surface_samples(coords, bin_sizes=(2000, 2000, 2000), max_nb_samples=5000, r=1000)[source]

Samples locations from a density grid based on coordinates and bin sizes. Collects coordinates within the given radius at each grid center to calculate the center of mass which yields the sample location.

Parameters

coords (np.ndarray) – Array of coordinates.
bin_sizes (Tuple[int, int, int]) – Sizes of bins used for creating the density grid.
max_nb_samples (int | None) – Maximum number of samples or None for no limit.
r (int) – Radius within which to collect coordinates for center of mass calculation.

Returns

Array of sampled locations.

Return type

np.ndarray