Skip to content

Detection

Built-in pipelines for preprocessing, agumentating and predicting.

[source]

AugmentBoxes

paz.pipelines.detection.AugmentBoxes(mean=(104, 117, 123))

Perform data augmentation with bounding boxes.

Arguments

  • mean: List of three elements used to fill empty image spaces.

[source]

AugmentDetection

paz.pipelines.detection.AugmentDetection(prior_boxes, split=0, num_classes=21, size=300, mean=(104, 117, 123), IOU=0.5, variances=[0.1, 0.1, 0.2, 0.2])

Augment boxes and images for object detection.

Arguments

  • prior_boxes: Numpy array of shape [num_boxes, 4] containing prior/default bounding boxes.
  • split: Flag from paz.processors.TRAIN, paz.processors.VAL or paz.processors.TEST. Certain transformations would take place depending on the flag.
  • num_classes: Int.
  • size: Int. Image size.
  • mean: List of three elements indicating the per channel mean.
  • IOU: Float. Intersection over union used to match boxes.
  • variances: List of two floats indicating variances to be encoded for encoding bounding boxes.

[source]

PreprocessBoxes

paz.pipelines.detection.PreprocessBoxes(num_classes, prior_boxes, IOU, variances)

Preprocess bounding boxes

Arguments

  • num_classes: Int.
  • prior_boxes: Numpy array of shape [num_boxes, 4] containing prior/default bounding boxes.
  • IOU: Float. Intersection over union used to match boxes.
  • variances: List of two floats indicating variances to be encoded for encoding bounding boxes.

[source]

PostprocessBoxes2D

paz.pipelines.detection.PostprocessBoxes2D(offsets, valid_names=None)

Filters, squares and offsets 2D bounding boxes

Arguments

  • valid_names: List of strings containing class names to keep.
  • offsets: List of length two containing floats e.g. (x_scale, y_scale)

[source]

DetectSingleShot

paz.pipelines.detection.DetectSingleShot(model, class_names, score_thresh, nms_thresh, preprocess=None, postprocess=None, variances=[0.1, 0.1, 0.2, 0.2], draw=True)

Single-shot object detection prediction.

Arguments

  • model: Keras model.
  • class_names: List of strings indicating the class names.
  • preprocess: Callable, pre-processing pipeline.
  • postprocess: Callable, post-processing pipeline.
  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • variances: List, of floats.
  • draw: Boolean. If True prediction are drawn in the returned image.

[source]

DetectHaarCascade

paz.pipelines.detection.DetectHaarCascade(detector, class_names=None, colors=None, draw=True)

HaarCascade prediction pipeline/function from RGB-image.

Arguments

  • detector: An instantiated HaarCascadeDetector model.
  • offsets: List of two elements. Each element must be between [0, 1].
  • class_names: List of strings.
  • draw: Boolean. If True prediction are drawn in the returned image.

Returns

A function for predicting bounding box detections.

[source]

SSD512HandDetection

paz.pipelines.detection.SSD512HandDetection(score_thresh=0.4, nms_thresh=0.45, draw=True)

Minimal hand detection with SSD512Custom trained on OPenImageV6.

Arguments

  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • draw: Boolean. If True prediction are drawn in the returned image.

Example

from paz.pipelines import SSD512HandDetection

detect = SSD512HandDetection()

# apply directly to an image (numpy-array)
inferences = detect(image)

Returns

A function that takes an RGB image and outputs the predictions as a dictionary with keys: image and boxes2D. The corresponding values of these keys contain the image with the drawn inferences and a list of paz.abstract.messages.Boxes2D.

Reference

[source]

SSD512MinimalHandPose

paz.pipelines.detection.SSD512MinimalHandPose(right_hand=False, offsets=[0.25, 0.25])

Hand detection and minimal hand pose estimation pipeline.

Arguments

  • right_hand: Boolean. True for right hand inference.
  • offsets: List of two elements. Each element must be between [0, 1].

Example

from paz.pipelines import SSD512MinimalHandPose

detect = SSD512MinimalHandPose()

# apply directly to an image (numpy-array)
inferences = detect(image)

Returns

A function that takes an RGB image and outputs the predictions as a dictionary with keys: image, boxes2D, Keypoints2D, Keypoints3D. The corresponding values of these keys contain the image with the drawn inferences.

[source]

SSDPreprocess

paz.pipelines.detection.SSDPreprocess(model, mean=(104, 117, 123), color_space=4)

Preprocessing pipeline for SSD.

Arguments

  • model: Keras model.
  • mean: List, of three elements indicating the per channel mean.
  • color_space: Int, specifying the color space to transform.

[source]

SSDPostprocess

paz.pipelines.detection.SSDPostprocess(model, class_names, score_thresh, nms_thresh, variances=[0.1, 0.1, 0.2, 0.2], class_arg=0, box_method=0)

Postprocessing pipeline for SSD.

Arguments

  • model: Keras model.
  • class_names: List, of strings indicating the class names.
  • score_thresh: Float, between [0, 1]
  • nms_thresh: Float, between [0, 1].
  • variances: List, of floats.
  • class_arg: Int, index of class to be removed.
  • box_method: Int, type of boxes to boxes2D conversion method.

[source]

DetectSingleShotEfficientDet

paz.pipelines.detection.DetectSingleShotEfficientDet(model, class_names, score_thresh, nms_thresh, preprocess=None, postprocess=None, draw=True)

Single-shot object detection prediction for EfficientDet models.

Arguments

  • model: Keras model.
  • class_names: List of strings indicating class names.
  • preprocess: Callable, preprocessing pipeline.
  • postprocess: Callable, postprocessing pipeline.
  • draw: Bool. If True prediction are drawn on the returned image.

Properties

  • model: Keras model.
  • draw: Bool.
  • preprocess: Callable.
  • postprocess: Callable.
  • draw_boxes2D: Callable.
  • wrap: Callable.

Methods

call()

[source]

EfficientDetPreprocess

paz.pipelines.detection.EfficientDetPreprocess(model, mean=(123, 117, 104), standard_deviation=(58.4, 57.1, 57.3))

Preprocessing pipeline for EfficientDet.

Arguments

  • model: Keras model.
  • mean: Tuple, containing mean per channel on ImageNet.
  • standard_deviation: Tuple, containing standard deviations per channel on ImageNet.

[source]

EfficientDetPostprocess

paz.pipelines.detection.EfficientDetPostprocess(model, class_names, score_thresh, nms_thresh, variances=[1.0, 1.0, 1.0, 1.0], class_arg=None)

Postprocessing pipeline for EfficientDet.

Arguments

  • model: Keras model.
  • class_names: List of strings indicating class names.
  • score_thresh: Float between [0, 1].
  • nms_thresh: Float between [0, 1].
  • variances: List of float values.
  • class_arg: Int, index of the class to be removed.
  • renormalize: Bool, if true scores are renormalized.
  • method: Int, method to convert boxes to Boxes2D.

[source]

EFFICIENTDETD0COCO

paz.pipelines.detection.EFFICIENTDETD0COCO(score_thresh=0.6, nms_thresh=0.45, draw=True)

Single-shot inference pipeline with EFFICIENTDETD0 trained on COCO.

Arguments

  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • draw: Boolean. If True prediction are drawn in the returned image.

References

Google AutoML repository implementation of EfficientDet

[source]

EFFICIENTDETD1COCO

paz.pipelines.detection.EFFICIENTDETD1COCO(score_thresh=0.6, nms_thresh=0.45, draw=True)

Single-shot inference pipeline with EFFICIENTDETD1 trained on COCO.

Arguments

  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • draw: Boolean. If True prediction are drawn in the returned image.

References

Google AutoML repository implementation of EfficientDet

[source]

EFFICIENTDETD2COCO

paz.pipelines.detection.EFFICIENTDETD2COCO(score_thresh=0.6, nms_thresh=0.45, draw=True)

Single-shot inference pipeline with EFFICIENTDETD2 trained on COCO.

Arguments

  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • draw: Boolean. If True prediction are drawn in the returned image.

References

Google AutoML repository implementation of EfficientDet

[source]

EFFICIENTDETD3COCO

paz.pipelines.detection.EFFICIENTDETD3COCO(score_thresh=0.6, nms_thresh=0.45, draw=True)

Single-shot inference pipeline with EFFICIENTDETD3 trained on COCO.

Arguments

  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • draw: Boolean. If True prediction are drawn in the returned image.

References

Google AutoML repository implementation of EfficientDet

[source]

EFFICIENTDETD4COCO

paz.pipelines.detection.EFFICIENTDETD4COCO(score_thresh=0.6, nms_thresh=0.45, draw=True)

Single-shot inference pipeline with EFFICIENTDETD4 trained on COCO.

Arguments

  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • draw: Boolean. If True prediction are drawn in the returned image.

References

Google AutoML repository implementation of EfficientDet

[source]

EFFICIENTDETD5COCO

paz.pipelines.detection.EFFICIENTDETD5COCO(score_thresh=0.6, nms_thresh=0.45, draw=True)

Single-shot inference pipeline with EFFICIENTDETD5 trained on COCO.

Arguments

  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • draw: Boolean. If True prediction are drawn in the returned image.

References

Google AutoML repository implementation of EfficientDet

[source]

EFFICIENTDETD6COCO

paz.pipelines.detection.EFFICIENTDETD6COCO(score_thresh=0.6, nms_thresh=0.45, draw=True)

Single-shot inference pipeline with EFFICIENTDETD6 trained on COCO.

Arguments

  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • draw: Boolean. If True prediction are drawn in the returned image.

References

Google AutoML repository implementation of EfficientDet

[source]

EFFICIENTDETD7COCO

paz.pipelines.detection.EFFICIENTDETD7COCO(score_thresh=0.6, nms_thresh=0.45, draw=True)

Single-shot inference pipeline with EFFICIENTDETD7 trained on COCO.

Arguments

  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • draw: Boolean. If True prediction are drawn in the returned image.

References

Google AutoML repository implementation of EfficientDet

[source]

EFFICIENTDETD0VOC

paz.pipelines.detection.EFFICIENTDETD0VOC(score_thresh=0.6, nms_thresh=0.45, draw=True)

Single-shot inference pipeline with EFFICIENTDETD0 trained on VOC.

Arguments

  • score_thresh: Float between [0, 1]
  • nms_thresh: Float between [0, 1].
  • draw: Boolean. If True prediction are drawn in the returned image.

References

Google AutoML repository implementation of EfficientDet

[source]

DetectVVAD

paz.pipelines.detection.DetectVVAD(architecture='CNN2Plus1D_Light', stride=10, averaging_window_size=6, average_type='weighted', offsets=[0, 0], colors=[[0, 255, 0], [255, 0, 0]])

Visual Voice Activity Detection classification and detection pipeline.

Example

from paz.backend.camera import VideoPlayer, Camera
import paz.pipelines.detection as dt

detect = DetectVVAD()

pipeline = dt.DetectVVAD()
# To input multiple images, use a camera or a prerecorded video
camera = Camera(args.camera_id)
player = VideoPlayer((640, 480), pipeline, camera)
player.run()

Returns

A function that takes an RGB image and outputs the predictions as a dictionary with keys: image and boxes2D. The corresponding values of these keys contain the image with the drawn inferences and a list of paz.abstract.messages.Boxes2D. Note multiple images are needed to produce a prediction.

Returns

A function that takes an RGB image and outputs the predictions as a dictionary with keys: image and boxes2D. The corresponding values of these keys contain the image with the drawn inferences and a list of paz.abstract.messages.Boxes2D. Note multiple images are needed to produce a prediction.

Arguments

  • architecture: String. Name of the architecture to use. Currently supported: 'VVAD-LRS3-LSTM', 'CNN2Plus1D', 'CNN2Plus1D_Filters' and 'CNN2Plus1D_Light'
  • stride: Integer. How many frames are between the predictions (computational expansive (low stride) vs high latency (high stride))
  • averaging_window_size: Integer. How many predictions are averaged. Set to 1 to disable averaging
  • average_type: String. 'mean' or 'weighted'. How the predictions are averaged. Set averaging_window_size to 1 to disable averaging