Utils & Preproc

Welcome to the documentation about the utilities.

Hereby, you find one table that list the preprocess parameters in relation with the JSON config file. They are used to transform the image before applying the detector or the tracker.

All functions are defined in details further down the page.

When a parameter is not specified, the transformation is simply not applied to the image.

List of preproc parameters and their usage

Parameter

Sub-parameter

Description

border

centered (bool)

Whether black borders are placed so that the image is always centered.

resize

width, height (int)

The width/height of the image after resizing, in pixels.

roi*

coords (str)

The set of the polygon coords that defines the Region of Interest (the white pixels). It must be of the following format: "(0, 0), (450, 0), (450, 200), (0, 200)"

path (str)

The path to a binary mask where white pixels represent the Region of Interest (ROI) and the black pixels represent the regions to be ignored.

* For the Region of Interest (ROI), either choose coords or path, not both!

NB: Note that applying a ROI to the image could lead to weird results if the model has never seen the mask before. If you wish to apply a ROI, it is advised to apply it as a post-process parameters (see dedicated section), as it will suppress objects outside the ROI without altering the image.

image_helper

Copyright (c) 2021-2022 UCLouvain, ICTEAM Licensed under GPL-3.0 [see LICENSE for details] Written by Jonathan Samelson (2021-2022)

get_cv2_img_from_str(path: str, flags=cv2.IMREAD_COLOR) → numpy.array[source]

Decodes an image from a path and returns a np.array containing the image, under cv2 format. If installed and if it is a .jpg or .jpeg image, uses TurboJPEG instead for faster reading.

Parameters

  • path (str) – Path to the image to be decoded

  • flags (int) – A cv2 flag indicating the reading mode. By default, it uses cv2.IMREAD_COLOR.

Returns

The image that was read from the input path.

Return type

np.array

get_cv2_img_from_url(url, flags=cv2.IMREAD_COLOR) → numpy.array[source]

Decodes an image from an URL and returns a np.array containing the image, under cv2 format.

Parameters

  • url (str) – A string containing the URL where the image should be fetched.

  • flags (int) – A cv2 flag indicating the reading mode. By default, it uses cv2.IMREAD_COLOR.

Returns

The image that was read from the URL.

Return type

np.array

_get_cv2_img_from_buffer(buffer, flags=cv2.IMREAD_COLOR)[source]
resize(image: numpy.array, width: int, height: int) → numpy.array[source]

Applies a resizing method on the image with cv2.INTER_AREA for the interpolation.

Parameters

  • image – The image to be resized.

  • width – The width of the image after resizing, in pixels.

  • height – The height of the image after resizing, in pixels.

Returns

A new image resized to the required dimension.

Return type

np.array

get_roi_file(roi_path: str)[source]

Gets the Region of Interest (ROI) from a path to an image file.

Parameters

roi_path (str) – The path to the image.

Returns

A binary mask where white pixels represent the Region of Interest (ROI) and the black pixels represent the regions to be ignored.

Return type

np.array

get_roi_coords(image_shape: tuple, roi_coords: str) → numpy.array[source]

Gets the Region of Interest (ROI) from a set of polygon coords.

Parameters

  • image_shape (tuple) – The dimension of the binary mask (the image) that will be returned

  • roi_coords (np.array) – The set of the polygon coords that defines the Region of Interest (the white pixels). It must be of the following format: “(0, 0), (450, 0), (450, 200), (0, 200)”

Returns

A binary mask where white pixels represent the Region of Interest (ROI) and the black pixels represent the regions to be ignored.

Return type

np.array

apply_roi(image: numpy.array, roi: numpy.array) → numpy.array[source]

Applies the Region of Interest (ROI), which is a binary mask, onto the image.

Parameters

  • image (np.array) – The original image on which the mask will be applied.

  • roi (np.array) – The mask to be applied on the image

Returns

A new image where black pixels of the mask are applied on the image.

Return type

np.array

add_borders(image: numpy.array, centered=False) → Tuple[numpy.array, numpy.array][source]

Adds black border to ‘image’ to keep the aspect ratio. returns the frame in letterbox format and the number of black pixels on each side.

Parameters

  • image (np.array) – The image to apply the transformation.

  • centered (bool) – Whether black borders are placed so that the image is always centered.

Returns

A tuple containing

  • frame (np.array): The image in letterbox format.

  • border_px (np.array): The border applied on each side [right, left, bottom, top] in pixels.

transformation

Copyright (c) 2021-2022 UCLouvain, ICTEAM Licensed under GPL-3.0 [see LICENSE for details] Written by Jonathan Samelson (2021-2022)

pre_process(preprocess_parameters: dict, image: numpy.ndarray, prev_roi: Optional[numpy.ndarray] = None, detection: Optional[pytb.output.detection.Detection] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray], Optional[numpy.ndarray], Optional[pytb.output.detection.Detection]][source]

Applies the preprocess parameters onto the image and optionally a detection. It consists of image transformation method (apply a region of interest, add a border, resize the image).

Parameters

  • preprocess_parameters (dict) – A dictionary containing the operations to be applied on the image. It consists of key-value pairs that should be validated against the validator module.

  • image (np.ndarray) – The image on which pre-process modification must be applied.

  • prev_roi (np.ndarray) – An image defining a binary mask to apply the ROI. This is optional and only needed to avoid repeated readings of the ROI file if the same ROI is used along a sequence of image.

  • detection (Detection) – If provided, returns the detection transformed to take into account the image modification (such as the modification of the dimensions).

Returns

A tuple containing

  • image (np.ndarray): The modified detection, if provided as an input to take into account the image modification.

  • roi (Optional[np.ndarray]): An optional frame of the ROI that was read, to be reused for next calls.

  • border_px (Optional[np.ndarray]): If “border” is to be applied, the number of pixels that was added on each side of the frame in the following order: [right, left, bottom, top].

  • detection (Optional[Detection]): The detection transformed in accordance with the image modification.

post_process(postprocess_parameters: dict, detection: pytb.output.detection.Detection, prev_roi: Optional[numpy.ndarray] = None) → Tuple[pytb.output.detection.Detection, Optional[numpy.ndarray]][source]

Applies the postprocess parameters onto the detection. It mainly consists of filtering method that removes a set of bounding boxes based on a set of thresholds. Learn more about those methods in the output classes (e.g. BBoxes2D) where those methods are implemented. In this function, the methods are called in a specific order that should provide the best results (yet, it is not guaranteed and one could change the order to obtain better results as the order matters).

Parameters

  • postprocess_parameters (dict) – A dictionary containing the operations to be applied on the detection. It consists of key-value pairs that should be validated against the validator module.

  • detection (Detection) – The detection on which post-process operations must be applied. The operations may vary depending on the type of the detection (only BBoxes2D & BBoxes2DTrack are supported at the moment).

  • prev_roi (np.ndarray) – An image defining a binary mask to apply the ROI. This is optional and only needed to avoid repeated readings of the ROI file if the same ROI is used along a sequence of image.

Returns

A tuple containing

  • detection (Detection): The modified detection, after applying the post-process operation.

  • roi (Optional[np.ndarray]): An optional frame of the ROI that was read, to be reused for next calls.

_get_roi(roi_params: dict, image_shape: tuple) → numpy.ndarray[source]
Parameters

  • roi_params (dict) – The parameters to apply the region of interest (ROI) as part of the preproc or postproc parameters.

  • image_shape (tuple) – The shape of the image for which it should be resized if the parameter “path” is chosen. Otherwise, in case of a polygon coords, it is assumed it is provided in the correct dimensions.

Returns

The binary mask of the ROI, either from the polygon coords or the image path.

Return type

np.ndarray

util

iou(bbox1: numpy.array, bbox2: numpy.array) → float[source]

Calculates the intersection-over-union of two bounding boxes.

Parameters

  • bbox1 (numpy.array, list of floats) – bounding box in format x1,y1,x2,y2.

  • bbox2 (numpy.array, list of floats) – bounding box in format x1,y1,x2,y2.

Returns

intersection-over-onion of bbox1, bbox2

Return type

float

validator

Copyright (c) 2021-2022 UCLouvain, ICTEAM Licensed under GPL-3.0 [see LICENSE for details] Written by Jonathan Samelson (2021-2022), Arthur Pisvin (2023)

validate_preprocess_parameters(pre_params: dict) → bool[source]

Check validity and compatibility between provided preprocess parameters.

Parameters

pre_params (dict) – the dictionary containing preprocess parameters.

Returns

whether it is a valid configuration.

Return type

bool

validate_postprocess_parameters(post_params: dict) → bool[source]

Check validity and compatibility between provided postprocess parameters.

Parameters

post_params (dict) – the dictionary containing postprocess parameters.

Returns

whether it is a valid configuration.

Return type

bool

_validate_roi_parameters(roi_params: dict, preproc: bool) → bool[source]
validate_detector_parameters(det_params: dict) → bool[source]

Check validity and compatibility between provided detector parameters.

Parameters

det_params (dict) – the dictionary containing detector parameters.

Returns

whether it is a valid configuration

Return type

bool

_validate_yolo4_parameters(det_params: dict) → bool[source]
_validate_yolo5_parameters(det_params: dict) → bool[source]
_validate_yolo8_parameters(det_params: dict) → bool[source]
_validate_backgroundsubtractor_parameters(det_params: dict) → bool[source]
_validate_detectron2_parameters(det_params: dict) → bool[source]
_validate_maskrcnn_parameters(det_params: dict) → bool[source]
_validate_fasterrcnn_parameters(det_params: dict) → bool[source]
validate_tracker_parameters(track_params: dict) → bool[source]

Check validity and compatibility between provided detector parameters.

Parameters

track_params (dict) – the dictionary containing tracker parameters.

Returns

whether it is a valid configuration

Return type

bool

_validate_sort_parameters(track_params: dict) → bool[source]
_validate_deepsort_parameters(track_params: dict) → bool[source]
_validate_centroid_parameters(track_params: dict) → bool[source]
_validate_iou_parameters(track_params: dict) → bool[source]

video_capture_async

Copyright (c) 2021-2022 UCLouvain, ICTEAM Licensed under GPL-3.0 [see LICENSE for details] From https://github.com/LeonLok/Deep-SORT-YOLOv4/blob/master/tensorflow2.0/deep-sort-yolov4/videocaptureasync.py Edited by Jonathan Samelson (2021-2022)

class VideoCaptureAsync(file_path: str)[source]

Bases: object

__init__(file_path: str)[source]

This class allows to read a video file asynchronously, meaning that the next frame is fetched between the call to the read() function. It is used the same way as cv2.VideoCapture, except that it is created using cap = VideoCaptureAsync(video_path) and that cap.stop() should be called before calling cap.release()

Parameters

file_path (str) – The path to the video file to read

set(var1, var2)[source]
start()[source]
update()[source]
read()[source]
is_opened()[source]
stop()[source]
release()[source]
get(x)[source]