Outputs & Postproc

Welcome to the documentation about the outputs.

All functions are defined in details further down the page.

detection

class Detection(number_objects: int)[source]

Bases: abc.ABC

__init__(number_objects: int)[source]

An abstract class representing a Detection. It stores the number of detected objects, but also the preprocessing, processing and postprocessing time that are filled by the DetectionManager

Parameters: number_objects (int) – The number of detected objects.

bboxes_2d

Herebelow, you can find a table of postproc parameters in relation with the JSON config file. They are used to filter out the detection in the case of bounding boxes 2D. When a parameter is not specified, the filter is simply not applied to the set of detections.

List of postproc parameters and their usage
Parameter	Sub-parameter	Description
`nms`	`pref_implem` (str)	Either `"cv2"` or `"Malisiewicz"`. Apply a Non-Max Suppression algorithm. Based on a threshold, it removes bounding boxes that overlap and only keeps one entity based on a selection criteria. This criteria depends on the implementation.
	`nms_thresh` (float)	The threshold to apply the Non-Max Suppression ranging from 0 to 1.
`min_conf` (float)		Keeps only the detections that have a confidence score above the threshold.
`max_height` (int)		Keeps only the detections whose the height is below the provided percentage of the frame.
`min_height` (int)		Keeps only the detections whose the height is above the provided percentage of the frame.
`max_width` (int)		Keeps only the detections whose the width is below the provided percentage of the frame.
`min_width` (int)		Keeps only the detections whose the width is above the provided percentage of the frame.
`min_area` (int)		Keeps only the detections whose the area is above the minimum area threshold. More specifically, if the detection’s width multiplied by the detection’s height is not (strictly) above the threshold, it is filtered out.
`top_k` (int)		Keeps only the K most confident prediction. If a choice has to be made between two detections that have the same confidence score, this choice is arbitrary.
`coi` (str)		Format is a string of a list of int (e.g. “[0, 2]”). Keeps only the detections that are included in the set of classes of interest.
`roi`	`path` (str)	The path to a binary mask where white pixels represent the Region of Interest (ROI) and the black pixels represent the regions to be ignored.
	`max_outside_roi_thresh` (float)	The threshold of area percentage above which a bounding box has to be removed ranging from 0 to 1.
`resize_results`	`width`, `height` (int)	Scales up/down the dimensions of the bounding boxes in accordance to the provided `width` and `height`.

class BBoxes2D(detection_time: float, bboxes: numpy.array, class_IDs: numpy.array, det_confs: numpy.array, dim_width: int, dim_height: int, bboxes_format: str = 'xt_yt_w_h')[source]

Bases: pytb.output.detection.Detection

__init__(detection_time: float, bboxes: numpy.array, class_IDs: numpy.array, det_confs: numpy.array, dim_width: int, dim_height: int, bboxes_format: str = 'xt_yt_w_h')[source]

A class that encompasses the result of a Bounding Box 2D Detector (such as YOLO). It also includes utility methods to filter the results or to apply transformation.

Parameters

detection_time (float) – The time elapsed in detection (e.g. forward method of a DNN)
bboxes (np.array) – The bounding boxes, including one np.array for each bounding box (i.e. it is a 2-dimensional array). It is either formatted as [[x_top, y_top, width, height], […], […], …] (default) or [[x_top_left, y_top_left, x_bottom_right, y_bottom_right], […], […], …] depending on the value of bboxes_format
class_IDs (np.array) – The ID of the class of each detected object. It follows the order of ‘bboxes’, but it is a 1-dimensional array.
det_confs (np.array) – The confidence of each detected object, typically ranging from 0 to 1. It follows the order of ‘bboxes’, but it is a 1-dimensional array.
dim_width (int) – The image width on which the ‘bboxes’ have been detected.
dim_height (int) – The image height on which the ‘bboxes’ have been detected.
bboxes_format (str) – Either “xt_yt_w_h” (default) or “x1_y1_x2_y2” depending on the format of ‘bboxes’.

nms_filter(pref_implem: str, nms_thresh: float)[source]

Apply a Non-Max Suppression algorithm. Based on a threshold, it removes bounding boxes that overlap and only keeps one entity based on a selection criteria. This criteria depends on the implementation.

This method does not return the object instance, it filters out directly on the instance’s attributes.

Parameters

pref_implem (str) – At the moment, either “cv2” or “Malisiewicz”. The former is the implementation of OpenCV, the latter is a custom implementation published in pyimagesearch (see more information below in _nms_malisiewicz documentation)
nms_thresh (float) – The threshold to apply the Non-Max Suppression ranging from 0 to 1.

top_k(k: int)[source]

Keeps only the K most confident prediction. If a choice has to be made between two detections that have the same confidence score, this choice is arbitrary.

This method does not return the object instance, it filters out directly on the instance’s attributes.

Parameters: k (int) – The number of detections to keep (i.e. including the Kth most confident detection).

confidence_filter(threshold: float)[source]

Keeps only the detections that have a confidence score above the threshold.

This method does not return the object instance, it filters out directly on the instance’s attributes.

Parameters: threshold (float) – The threshold above which the detection is kept.

class_filter(classes_of_interest: set)[source]

Keeps only the detections that are included in the set of classes of interest.

This method does not return the object instance, it filters out directly on the instance’s attributes.

Parameters: classes_of_interest (set) – The set of classes that will be kept.

height_filter(threshold: float, max_filter: bool)[source]

Keeps only the detections whose the height is above/below the percentage of the height of the frame on which the detection has been found (represented by dim_height attribute).

This method does not return the object instance, it filters out directly on the instance’s attributes.

Parameters

threshold (float) – The percentage of the image height.
max_filter (bool) – Whether to apply a max or min filter. True means it keeps only detections strictly below the threshold, False means it keeps only detections strictly above the threshold.

width_filter(threshold: float, max_filter: bool)[source]

Keeps only the detections whose the width is above/below the percentage of the width of the frame on which the detection has been found (represented by dim_width attribute).

This method does not return the object instance, it filters out directly on the instance’s attributes.

Parameters

threshold (float) – The percentage of the image width.
max_filter (bool) – Whether to apply a max or min filter. True means it keeps only detections strictly below the threshold, False means it keeps only detections strictly above the threshold.

min_area_filter(threshold: int)[source]

Keeps only the detections whose the area is above the minimum area threshold. More specifically, if the detection’s width multiplied by the detection’s height is not (strictly) above the threshold, it is filtered out.

This method does not return the object instance, it filters out directly on the instance’s attributes.

Parameters: threshold (float) – The threshold in square pixels (px²). It should be provided in accordance with dim_width and dim_height since the width and height of the bounding boxes are scaled according to those attributes.

cv2_filter(nms_thresh: float, conf_thresh: float, eta: float = 1.0, top_k: int = 0)[source]

Filters the detections using OpenCV NMSBoxes. In addition to Non Max Suppression (NMS), it can be used to directly remove the bounding boxes that are below a confidence threshold or to keep only K detections.

This method does not return the object instance, it filters out directly on the instance’s attributes.

Parameters

nms_thresh (float) – The threshold for the non-max suppression (NMS).
conf_thresh (float) – The confidence threshold. Only detections above this threshold are kept.
eta (float) – A coefficient in adaptive threshold formula : nms_thresh i+1 = eta . nms_thresh i
top_k (int) – The maximum numbers of bounding boxes to keep.

roi_filter(roi: numpy.ndarray, max_outside_roi_thresh: float)[source]

Removes the bounding boxes that are outside the Region Of Interest (ROI) based on a threshold.

This method does not return the object instance, it filters out directly the instance’s attributes.

Parameters

roi (np.ndarray) – The binary image mask. If the provided image has more than 1 channel, it will be automatically converted to a 1-channel image. The mask must match dim_width and dim_height attributes.
max_outside_roi_thresh (float) – The threshold of area percentage above which a bounding box has to be removed ranging from 0 to 1.

to_x1_y1_x2_y2()[source]

Changes the format of the ‘bboxes’ attribute. The bounding boxes array is converted from [x_top, y_top, width, height] to [x_left_top, y_left_top, x_right_bottom, y_right_bottom].

This method does not return the object instance, it modifies directly the instance’s attributes.

to_xt_yt_w_h()[source]

Changes the format of the ‘bboxes’ attribute. The bounding boxes array is converted from [x_left_top, y_left_top, x_right_bottom, y_right_bottom] to [x_top, y_top, width, height].

This method does not return the object instance, it modifies directly the instance’s attributes.

change_dims(new_width: int, new_height: int)[source]

It sets the dim_width and dim_height to the provided values. Scales up/down the dimensions of the bounding boxes in accordance to the new values such that the dimensions of the bounding boxes are like if they were detected on an image whose the dimension is dim_width x dim_height.

This method does not return the object instance, it modifies directly the instance’s attributes.

Parameters

new_width (int) – The new value of dim_width.
new_height (int) – The new value of dim_height.

remove_borders(borders_px: numpy.array)[source]

Adjusts the bounding boxes sizes to take into account the removal of black borders around the image.

This method does not return the object instance, it modifies directly the instance’s attributes.

Parameters

borders_px (np.array) – The number of pixels that was added on each side of the frame
order (in the following) – [right, left, bottom, top].

add_borders(borders_px)[source]

Adjusts the bounding boxes sizes to take into account the addition of black borders around the image.

This method does not return the object instance, it modifies directly the instance’s attributes.

Parameters

borders_px (np.array) – The number of pixels that was added on each side of the frame
order (in the following) – [right, left, bottom, top].

remove_idx(indices: numpy.array)[source]

Removes the bounding boxes together with their confidence and class ID given a list of indices.

This method does not return the object instance, it modifies directly the instance’s attributes.

Parameters: indices (np.array) – The indices of the detected objects to remove.

_select_indices(indices: numpy.array)[source]

Selects the bounding boxes to keep together with their confidence and class ID given a list of indices.

This method does not return the object instance, it modifies directly the instance’s attributes.

Parameters: indices (np.array) – The indices of the detected objects to keep.

_nms_malisiewicz(nms_thresh)[source]

An implementation of Non-max Suppression (NMS) by Dr. Tomasz Malisiewicz.

Original code from 1 that has been adapted to include confidence score.

1: http://www.pyimagesearch.com/2015/02/16/faster-non-maximum-suppression-python/

Parameters: nms_thresh (float) – The threshold to apply the Non-Max Suppression ranging from 0 to 1.

bboxes_2d_track

class BBoxes2DTrack(detection_time: float, bboxes: numpy.array, class_IDs: numpy.array, det_confs: numpy.array, dim_width: int, dim_height: int, tracking_time: float, global_IDs: numpy.array, bboxes_format: str = 'xt_yt_w_h')[source]

Bases: pytb.output.bboxes_2d.BBoxes2D

__init__(detection_time: float, bboxes: numpy.array, class_IDs: numpy.array, det_confs: numpy.array, dim_width: int, dim_height: int, tracking_time: float, global_IDs: numpy.array, bboxes_format: str = 'xt_yt_w_h')[source]

A class that encompasses the result of a Bounding Box 2D Tracker (such as SORT). It extends the BBoxes2D class and adds a np.array for the ID of the object and a tracking time.

Parameters

tracking_time (float) – The time elapsed in the tracking
global_IDs (np.array) – A 1-dimensional array that follows the same order as ‘bboxes’. It contains the IDs of the detected object that should be consistent between successive frames.

remove_idx(indices: list)[source]

Removes the bounding boxes together with their confidence, class ID and global ID given a list of indices.

This method does not return the object instance, it modifies directly the instance’s attributes.

Parameters: indices (list) – The indices of the detected objects to remove.

_select_indices(indices: numpy.array)[source]

Selects the bounding boxes to keep together with their confidence, class ID and global ID given a list of indices.

This method does not return the object instance, it modifies directly the instance’s attributes.

Parameters: indices (np.array) – The indices of the detected objects to keep.