svkcore.common

svkcore.common.common

This module provides common utility functions for simple vision kit. It includes functions for computing md5 hash, aligning file paths, grouping lists, finding files recursively, collecting examples, and encoding/decoding images using base64.

svkcore.common.common.align_paths(paths0, paths1, *args, sort=False, key_fn=None)[source]

Align paths base on its name head This function will delete dis-matched paths with no prompt.

Parameters:

  • paths0 (list) – the first list of file paths

  • paths1 (list) – the second list of file paths

  • args – the others list of file paths

  • sort (bool) – sorted output by key

  • key_fn (callable) – extract align key function, default is basename_head()

Returns:

align paths in list

Return type:

list

svkcore.common.common.b64decode_image(data: bytes) → Image[source]

Decode bytes data of encoded image to an instance of PIL.Image.Image

Parameters:

data (bytes) – base64 encoded bytes image data

Returns:

An instance of Image.Image represents the decode image

Return type:

PIL.Image.Image

svkcore.common.common.b64encode_image(image: Image, format: str = 'JPEG') → bytes[source]

Convert PIL.Image.Image object to bytes data use base64 encode

Parameters:

  • image (PIL.Image.Image) – an instance of PIL.Image.Image

  • format (str) – a string represents image encoding format. could be “JPEG” or “PNG”

Returns:

base64 encoded image data

Return type:

bytes

svkcore.common.common.basename_head(path, sep='.', align_left=False)[source]

Get basename head of a path

Parameters:

  • path (str) – a path

  • sep (str) – a separator str to split path’s basename. Default is ".".

  • align_left (bool) – split basename with separator from left or not. Default is False. And this function will split basename from right.

Returns:

the head part of basename of path

Return type:

str

svkcore.common.common.bsn_head(path, sep='.', align_left=False)

Get basename head of a path

Parameters:

  • path (str) – a path

  • sep (str) – a separator str to split path’s basename. Default is ".".

  • align_left (bool) – split basename with separator from left or not. Default is False. And this function will split basename from right.

Returns:

the head part of basename of path

Return type:

str

svkcore.common.common.collect_examples(directory, suffixes_list, ignore_case=False, sort=False, key_fn=None)[source]

Collect examples from one directory base on given suffixes list

Parameters:

  • directory (str) – Root directory of examples

  • suffixes_list (list) – A list of suffix list for example’s each part

  • ignore_case (bool) – ignore case when match file name suffix. Default is False.

  • sort (bool) – sort the result list by key. Default is False.

  • key_fn (callable) – a function use to extract align key, default is basename_head()

Returns:

all matched file paths list

Return type:

list

svkcore.common.common.collect_pascal_data(directory)[source]

Collect pascal format dataset

Parameters:

directory (str) – the root directory of a pascal dataset

Returns:

two list of images and annotations of the the pascal dataset

Return type:

list

svkcore.common.common.find_file_recursive(directory, suffixes, ignore_case=False)[source]

Find all files with provided suffixes

Parameters:

  • directory (str) – the target directory

  • suffixes (Union[str, list]) – a suffix or a list of suffixes of file to be find

  • ignore_case (bool) – match file suffix in case ignore mode

Returns:

a list which returns the path to the file which meets the postfix

Return type:

list

svkcore.common.common.find_files(directory, suffixes, ignore_case=False)

Find all files with provided suffixes

Parameters:

  • directory (str) – the target directory

  • suffixes (Union[str, list]) – a suffix or a list of suffixes of file to be find

  • ignore_case (bool) – match file suffix in case ignore mode

Returns:

a list which returns the path to the file which meets the postfix

Return type:

list

svkcore.common.common.get_default_font(size: int = 24) → ImageFont[source]

Get a default PIL.ImageFont.ImageFont instance for show label name on image which could deal with both English and Chinese

Parameters:

size (int) – font size

Returns:

a font object

Return type:

PIL.ImageFont.ImageFont

svkcore.common.common.group(lst, key, value=None) → Dict[object, list][source]

Group list by key and return a dict. Each value of the result dict is a list. And each list contains all values with same key.

Parameters:

  • lst (list) – a list of objects to be group

  • key (Union[list, callable]) – a list of key objects or a callable function map each item in lst to its key

  • value (Union[list, callable]) – a list of value objects or a callable function map each item in lst to its value. Default is None and group will use lst as value.

Returns:

grouped result

Return type:

Dict[object, list]

svkcore.common.common.group_map(_group: dict, func: Callable, with_key: bool = False)[source]

Do map on a group result

Parameters:

  • _group (Dict[object, list]) – a result dict of group

  • func (Callable) – a function used to process value or key and value

  • with_key (bool) – func process with key and value or only value. Default is False, only process value.

Returns:

map result dict

Return type:

dict

svkcore.common.common.ifind_file_recursive(directory, suffixes, ignore_case=False)[source]

Find all files with provided suffixes

Parameters:

  • directory (str) – the target directory

  • suffixes (Union[str, list]) – a suffix or a list of suffixes of file to be find

  • ignore_case (bool) – match file suffix in case ignore mode

Returns:

a generator which returns the path to the file which meets the postfix

Return type:

a generator

svkcore.common.common.image_md5(image: Image) → str[source]

Compute a image md5 value

Parameters:

image (PIL.Image.Image) – pillow image object to be hashed by md5

Returns:

md5 hash string

Return type:

str

svkcore.common.common.str_md5(_bytes: bytes) → str[source]

Compute string md5 value

Parameters:

_bytes (bytes) – data in bytes format to be hashed by md5

Returns:

md5 hash string

Return type:

str

svkcore.common.fileio

This module provides functions for file input/output operations, including copying files, loading and saving json, pickle, csv files, and reading and writing images using OpenCV.

svkcore.common.fileio.copy_files(paths, dst_dir, src_dir=None)[source]

Copy files in list paths from src_dir to dst_dir

Parameters:

  • paths (list) – a list of paths

  • dst_dir (str) – destination directory

  • src_dir (str) – root directory of all paths

svkcore.common.fileio.cv2imread(path)[source]

Read image from file path using OpenCV

Parameters:

path (str) – image file path

Returns:

loaded image as numpy array

Return type:

numpy.ndarray

svkcore.common.fileio.cv2imwrite(path, img)[source]

This function is a compatible version of cv2.imwrite. It writes an image to the specified file path in .jpg format.

Parameters:

  • path (str) – output image file path

  • img (numpy.ndarray) – output image data

svkcore.common.fileio.load_csv(path, *, with_header=True, encoding=None)[source]

This function is used to load csv format file. It reads the csv file from the given path and returns the rows as a list of lists. If with_header is True, the first row is considered as the header and returned separately. If encoding is provided, it is used to decode the csv file.

Parameters:

  • path (str) – csv file path

  • with_header (bool) – return result with data header

  • encoding (str) – csv file encoding like utf-8, etc.

Returns:

Return type:

list

svkcore.common.fileio.load_json(path, *, encoding=None, **kwargs)[source]

Load json format file

Parameters:

  • path (str) – json file path

  • encoding (str) – json file encoding like utf-8, etc.

  • kwargs – other options for json.load

Returns:

loaded json object

Type:

object

svkcore.common.fileio.load_pickle(path)[source]

Load object from pickle format

Parameters:

path (str) – pickle file path

Returns:

loaded object

Return type:

object

svkcore.common.fileio.save_csv(rows, path, *, header=None, encoding=None)[source]

This function is used to save data into csv format file. If header is provided, header will place at the first row of the output csv file. If encoding is provided, it is used to decode the csv file.

Parameters:

  • rows (list) – a list of data row

  • path (str) – csv file path

  • header (list) – data header

  • encoding (str) – csv file encoding like utf-8, etc.

svkcore.common.fileio.save_json(obj, path, indent=2, ensure_ascii=False, *args, **kwargs)[source]

Save object as json format

Parameters:

  • obj (object) – object to be saved as json format

  • path (str) – json file path

  • indent (int) – number of spaces for indentation

  • ensure_ascii (bool) – whether to ensure ascii encoding

  • args – other options for json.dump

  • kwargs – other options for json.dump

svkcore.common.fileio.save_pickle(obj, path)[source]

Save object as pickle format

Parameters:

  • obj (object) – object to be saved as pickle format

  • path (str) – pickle file path

svkcore.common.np_ops

Some common numpy operations

svkcore.common.np_ops.circle_kernel(diameter, dtype=<class 'numpy.int32'>)[source]

Create circle kernel

Parameters:

  • diameter (int) – diameter of the circle

  • dtype (numpy.dtype, optional) – data type of the kernel

Returns:

circle kernel

Return type:

numpy.ndarray

svkcore.common.np_ops.ellipse_kernel(ksize, dtype=<class 'numpy.int32'>)[source]

Create ellipse kernel

Parameters:

  • ksize (tuple) – kernel size, a tuple of (height, width)

  • dtype (numpy.dtype, optional) – data type of kernel

Returns:

ellipse kernel

Return type:

numpy.ndarray

svkcore.common.np_ops.generate_grid(panel_size, grid_size=None, grid_num=None, overlap_size=None, overlap_ratio=None, allow_cross_boundary=False)[source]

Generate grid for crop image patches

Parameters:

  • panel_size (bool) – a tuple of image size (height, width)

  • grid_size – a tuple of grid size (grid_height, grid_width)

  • grid_num – a tuple of grid num (grid_rows, grid_column)

  • overlap_size – a tuple of overlap size

  • overlap_ratio – a tuple of grid overlap ratio

  • allow_cross_boundary – allow the last row or column position cross panel or not

Returns:

grids [rows, columns, 4], each grid consists (ymin, xmin, ymax, xmax)

Return type:

np.ndarray

svkcore.common.np_ops.ndarray_index(shape)[source]

Create np.ndarray index

Parameters:

shape (Union[list, tuple]) – index array shape

Returns:

np.ndarray index

Return type:

np.ndarray

svkcore.common.np_ops.nms_mask(seg, ksize=3, dtype=None)[source]

Non-maximum suppression mask for segmentation

Parameters:

  • seg (np.ndarray) – segmentation result, probability map between [0.0, 1.0]

  • ksize (int) – kernel size for max pooling

  • dtype (np.dtype) – data type of output mask

Returns:

non-maximum suppression mask

Return type:

np.ndarray

svkcore.common.np_ops.points_distance(points0, points1, weights=(1.0, 1.0))[source]

Calculate distances between two point array

Parameters:

  • points0 (np.ndarray) – a numpy array of shape (n, 2) representing the first set of points

  • points1 (np.ndarray) – a numpy array of shape (m, 2) representing the second set of points

  • weights (tuple) – a tuple of two floats representing the weights for the distance calculation along the vertical and horizontal axes

Returns:

a numpy array of shape (n, m) representing the distances between each pair of points from points0 and points1

Return type:

np.ndarray

svkcore.common.np_ops.seg2line(seg, fb_threshold=0.5, smooth_width=3, partition_width=20, partition_height=30)[source]

Find all valid lines for segmentation

Parameters:

  • seg (np.ndarray) – point object segmentation result, probability map between [0.0, 1.0]

  • fb_threshold (float) – foreground vs background threshold

  • smooth_width (float) – bin width for line x-coordinate smooth

  • partition_width (float) – line will split when x-coordinate interval greater than partition_width

  • partition_height (float) – line will split when y-coordinate interval greater than partition_height

Returns:

a list of lines

Return type:

list

svkcore.common.np_ops.seg2point(seg, max_diameter: int, min_distance, fb_threshold: float = 0.5, min_fore_count: int = 1, max_fore_count: int = -1, avg_fore_score: float = 0.55, distance_weights=(1.0, 1.0))[source]

Find all valid points from segmentation

Parameters:

  • seg (np.ndarray) – point object segmentation result, probability map between [0.0, 1.0]

  • max_diameter (int) – max_radius for a point

  • min_distance (float) – min distance between two point center

  • fb_threshold (float) – foreground vs background threshold

  • min_fore_count (int) – The minimum count of foreground pixel

  • max_fore_count (int) – The maximum count of foreground pixel

  • avg_fore_score (float) – The minimum average fore score of foreground

  • distance_weights (tuple) – distance value’s weights of vertical and horizontal

Returns:

A list of point

Return type:

np.ndarray