syconn.backend package

Submodules

• syconn.backend.base module

• syconn.backend.storage module

syconn.backend.base module

class syconn.backend.base.BTBase(identifier, cache_decomp=False, read_only=True, disable_locking=False)

Bases: syconn.backend.base.StorageBase

class syconn.backend.base.FSBase(inp_p, cache_decomp=False, read_only=True, max_delay=100, timeout=1000, disable_locking=True, max_nb_attempts=100)

Bases: syconn.backend.base.StorageBase

Customized dictionary to store compressed numpy arrays, but with a intuitive user interface, i.e. compression will happen in background. kwarg ‘cache_decomp’ can be enabled to cache decompressed arrays additionally (save decompressing time when accessing items frequently).

copy() → a shallow copy of D

items() → a set-like object providing a view on D's items

keys() → a set-like object providing a view on D's keys

pull(source=None)

Fetches data from source.

Parameters

source (Optional[str]) – Source location

push(dest=None)

Pushes data to destination.

Parameters

dest (Optional[str]) – storage destination

update([E, ]**F) → None.  Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → an object providing a view on D's values

syconn.backend.storage module

class syconn.backend.storage.AttributeDict(inp_p, **kwargs)

Bases: syconn.backend.base.FSBase

General purpose dictionary class inherited from syconn.backend.base.StorageClass.

copy_intern()

update([E, ]**F) → None.  Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

class syconn.backend.storage.BinarySearchStore(fname, id_array=None, attr_arrays=None, overwrite=False, n_shards=None, rdcc_nbytes=5242880)

Bases: object

get_attributes(obj_ids, attr_key)

Query attributes of given obj_ids. Note that this will not raise an Exception if a ID does not exist in the store, as the lookup uses binary search.

Parameters

• obj_ids (ndarray) – Object IDs to query.

• attr_key (str) – Value type obtained from the store.

Return type

ndarray

Returns

Value array.

property id_array: numpy.ndarray

Returns: Flat ID array.

Return type

ndarray

property n_shards: int

Number of shards/chunks the ID and attribute arrays are split into. Returns:

Return type

int

class syconn.backend.storage.CompressedStorage(inp, **kwargs)

Bases: syconn.backend.base.FSBase

Customized dictionary to store compressed numpy arrays, but with a intuitive user interface, i.e. compression will happen in background. kwarg ‘cache_decomp’ can be enabled to cache decompressed arrays additionally (save decompressing time when accessing items frequently).

class syconn.backend.storage.MeshStorage(inp, load_colarr=False, compress=False, **kwargs)

Bases: syconn.backend.base.FSBase

Customized dictionary to store compressed numpy arrays, but with a intuitive user interface, i.e. compression will happen in background. kwarg ‘cache_decomp’ can be enabled to cache decompressed arrays additionally (save decompressing time).

class syconn.backend.storage.SkeletonStorage(inp, **kwargs)

Bases: syconn.backend.base.FSBase

Stores skeleton dictionaries (keys: “nodes”, “diameters”, “edges”) as compressed numpy arrays.

syconn.backend.storage.VoxelStorage(inp, **kwargs)

Alias for VoxelStorageDyn.

Parameters

• inp –

• **kwargs –

Returns:

class syconn.backend.storage.VoxelStorageClass(inp, **kwargs)

Bases: syconn.backend.storage.VoxelStorageL

Customized dictionary to store compressed numpy arrays, but with a intuitive user interface, i.e. compression will happen in background. kwarg ‘cache_decomp’ can be enabled to cache decompressed arrays additionally (save decompressing time).

No locking provided in this class!

class syconn.backend.storage.VoxelStorageDyn(inp, voxel_mode=True, voxeldata_path=None, **kwargs)

Bases: syconn.backend.storage.CompressedStorage

Similar to VoxelStorageL but does not store the voxels explicitly, but the information necessary to query the voxels of an object.

If voxel_mode = True getter method will operate on underlying data set to retrieve voxels of an object. __setitem__ throws RuntimeError. __getitem__ will return a list of 3D binary cubes with ones at the object’s locations (key: object ID). Note: The item ID has to match the object ID in the segmentation.

Otherwise (voxel_mode = False) __getitem__ and __setitem__ allow manipulation of the object’s bounding boxes. In this case voxeldata_path has to be given or already be existent in loaded dictionary. Expects the source path of a KnossoDataset (see knossos_utils), like:

kd = KnossoDataset() kd.initialize_from_knossos_path(SOURCE_PATH)

or

kd = kd_factory(SOURCE_PATH)

__setitem__ requires the object ID as key and an 3 dimensional array with

all bounding boxes defining the object (N, 2, 3). Those BBs are then used to query the object voxels. The bounding box is expected to be two 3D coordinates which define the lower and the upper limits.

get_boundingdata(item)

Get the object bounding boxes. :type item: int :param item: Object ID.

Return type

List[ndarray]

Returns

List of bounding boxes (in voxels; xyz).

get_voxel_cache(key)

Voxels corresponding to item key must have been added to store via set_voxel_cache(). This implementation operates independent of get_voxeldata().

Parameters

key (int) – Segment ID.

Returns

Voxel coordinates.

get_voxel_data_cubed(item)

Get the object binary mask as dense 3D array (xyz).

Parameters

item (int) – Object ID.

Return type

Tuple[ndarray, ndarray]

Returns

3D mask, cube offset in voxels (xyz).

get_voxeldata(item)

Get the object binary mask as list of 3D cubes with the respective offsets (in voxels). All in xyz.

Parameters

item (int) – Object ID.

Return type

Tuple[List[ndarray], List[ndarray]]

Returns

List of 3D binary masks and offsets (in voxels; xyz).

get_voxelmask_offset(item, overlap=0)

increase_object_size(item, value)

iter_voxelmask_offset(item, overlap=0)

Return type

Iterator[Tuple[ndarray, ndarray]]

keys() → a set-like object providing a view on D's keys

object_repcoord(item)

object_size(item)

push()

Pushes data to destination.

Parameters

dest – storage destination

set_object_repcoord(item, value)

set_voxel_cache(key, voxel_coords)

This is only used to store the voxels during the synapse extraction step. This method operates independent of __setitem__().

Parameters

• key (int) – Segment ID.

• voxel_coords (ndarray) – Voxel coordinates.

class syconn.backend.storage.VoxelStorageL(inp, **kwargs)

Bases: syconn.backend.base.FSBase

Customized dictionary to store compressed numpy arrays, but with a intuitive user interface, i.e. compression will happen in background. kwarg ‘cache_decomp’ can be enabled to cache decompressed arrays additionally (save decompressing time).

append(key, voxel_mask, offset)

class syconn.backend.storage.VoxelStorageLazyLoading(path, overwrite=False)

Bases: object

Similar to VoxelStorage but uses lazy loading via numpy npz files.

Notes

• Once written, npz storages will not support modification via __setitem__.

• Key of types other than int are not supported. Internally, keys are converted to string, as required by npz, and then always converted to int for “external” use (e.g. keys).

• Call close when opening an existing npz file.

close()

keys()

pull()

push()

syconn.backend.storage.bss_get_attr_helper(args)

Helper function to query attributes from a BinarySearchStore instance.

Parameters

args – BinarySearchStore, query_ids, attribute key.

Returns

Query result.