Skip to content

Pose

Built-in pipelines for preprocessing, agumentating and predicting.

[source]

EstimatePoseKeypoints

paz.pipelines.pose.EstimatePoseKeypoints(detect, estimate_keypoints, camera, offsets, model_points, class_to_dimensions, radius=3, thickness=1)

[source]

HeadPoseKeypointNet2D32

paz.pipelines.pose.HeadPoseKeypointNet2D32(camera, offsets=[0, 0], radius=5, thickness=2)

Head pose estimation pipeline using a HaarCascade face detector and a pre-trained KeypointNet2D estimation model.

Arguments

  • camera: Instance of paz.backend.camera.Camera with camera intrinsics.
  • offsets: List of floats indicating the scaled offset to be added to the Box2D coordinates.
  • radius: Int. radius of keypoint to be drawn.

Example

from paz.pipelines import HeadPoseKeypointNet2D32

estimate_pose = HeadPoseKeypointNet2D32()

# apply directly to an image (numpy-array)
inferences = estimate_pose(image)

Returns

A function that takes an RGB image and outputs the following inferences as keys of a dictionary: image, boxes2D, keypoints and poses6D.

[source]

SingleInstancePIX2POSE6D

paz.pipelines.pose.SingleInstancePIX2POSE6D(model, object_sizes, camera, epsilon=0.15, resize=False, class_name=None, draw=True)

Predicts a single pose6D from an image. Optionally if a box2D message is given it translates the predicted points2D to new origin located at box2D top-left corner.

Arguments

  • model: Keras segmentation model.
  • object_sizes: Array (3) determining the (width, height, depth)
  • camera: PAZ Camera with intrinsic matrix.
  • epsilon: Float. Values below this value would be replaced by 0.
  • resize: Boolean. If True RGB mask is resized before computing PnP.
  • class_name: Str indicating object name.
  • draw: Boolean. If True drawing functions are applied to output image.

Returns

Dictionary with inferred points2D, points3D, pose6D and image.

[source]

MultiInstancePIX2POSE6D

paz.pipelines.pose.MultiInstancePIX2POSE6D(estimate_pose, offsets, camera=None, draw=True)

Predicts poses6D of multiple instances the same object from an image.

Arguments

  • estimate_pose: Function that takes as input an image and outputs a dictionary with points2D, points3D and pose6D messages e.g SingleInstancePIX2POSE6D
  • offsets: List of length two containing floats e.g. (x_scale, y_scale)
  • camera: PAZ Camera with intrinsic matrix.
  • draw: Boolean. If True drawing functions are applied to output image.

Returns

Dictionary with inferred boxes2D, poses6D and image.

[source]

MultiInstanceMultiClassPIX2POSE6D

paz.pipelines.pose.MultiInstanceMultiClassPIX2POSE6D(detect, name_to_model, name_to_size, camera, offsets, epsilon=0.15, resize=False, draw=True)

Predicts poses6D of multiple instances of multiple objects from an image

Arguments

  • detect: Function that takes as input an image and outputs a dictionary containing Boxes2D messages.
  • name_to_model: Dictionary with class name as key and as value a Keras segmentation model.
  • name_to_size: Dictionary with class name as key and as value the object sizes.
  • camera: PAZ Camera with intrinsic matrix.
  • offsets: List of length two containing floats e.g. (x_scale, y_scale)
  • epsilon: Float. Values below this value would be replaced by 0.
  • resize: Boolean. If True RGB mask is resized before computing PnP.
  • draw: Boolean. If True drawing functions are applied to output image.

Returns

Dictionary with inferred boxes2D, poses6D and image.

[source]

AugmentColor

paz.pipelines.pose.AugmentColor()

[source]

AugmentEfficientPose

paz.pipelines.pose.AugmentEfficientPose(model, mean, camera_matrix, split=0, num_classes=8, size=512, IOU=0.5, variances=[0.1, 0.1, 0.2, 0.2], probability=0.5, num_pose_dims=3)

Augment images, boxes and poses for pose estimation.

Arguments

  • model: Keras model.
  • split: Flag from paz.processors.TRAIN, paz.processors.VAL or paz.processors.TEST. Certain transformations would take place depending on the flag.
  • num_classes: Int, specifying the number of classes to train.
  • size: Int. Image size.
  • mean: List of three elements indicating the per channel mean.
  • camera_matrix: Array with camera matrix of shape (3, 3).
  • IOU: Float. Intersection over union used to match boxes.
  • variances: List of two floats indicating variances to be encoded for encoding bounding boxes.
  • probability: Float indicating the probability of data augmentation.
  • num_pose_dims: Int, number of dimensions for pose.

[source]

EfficientDetPreprocess

paz.pipelines.detection.EfficientDetPreprocess(model, mean=(123, 117, 104), standard_deviation=(58.4, 57.1, 57.3))

Preprocessing pipeline for EfficientDet.

Arguments

  • model: Keras model.
  • mean: Tuple, containing mean per channel on ImageNet.
  • standard_deviation: Tuple, containing standard deviations per channel on ImageNet.

[source]

EfficientDetPostprocess

paz.pipelines.detection.EfficientDetPostprocess(model, class_names, score_thresh, nms_thresh, variances=[1.0, 1.0, 1.0, 1.0], class_arg=None)

Postprocessing pipeline for EfficientDet.

Arguments

  • model: Keras model.
  • class_names: List of strings indicating class names.
  • score_thresh: Float between [0, 1].
  • nms_thresh: Float between [0, 1].
  • variances: List of float values.
  • class_arg: Int, index of the class to be removed.
  • renormalize: Bool, if true scores are renormalized.
  • method: Int, method to convert boxes to Boxes2D.

[source]

EstimateEfficientPose

paz.pipelines.pose.EstimateEfficientPose(model, class_names, score_thresh, nms_thresh, object_sizes, mean, camera_matrix, regress_translation, compute_tx_ty_tz, preprocess=None, postprocess=None, variances=[0.1, 0.1, 0.2, 0.2], show_boxes2D=False, show_poses6D=True)

Object detection and pose estimation for EfficientPose models.

Arguments

  • model: Keras model.
  • class_names: List of strings indicating class names.
  • score_thresh: Float between [0, 1].
  • nms_thresh: Float between [0, 1].
  • camera_matrix: Array of shape (3, 3) camera matrix.
  • object_sizes: Dict, dataset object sizes.
  • preprocess: Callable, preprocessing pipeline.
  • postprocess: Callable, postprocessing pipeline.
  • variances: List of float values.
  • show_boxes2D: Boolean. If True prediction are drawn in the returned image.
  • show_poses6D: Boolean. If True estimated poses are drawn in the returned image.

Properties

  • model: Keras model.
  • class_names: List.
  • score_thresh: Float.
  • nms_thresh: Float.
  • camera_matrix: Array.
  • variances: List.
  • class_to_sizes: Dict.
  • colors: List.
  • show_boxes2D: Bool.
  • show_poses6D: Bool.
  • preprocess: Callable.
  • postprocess: Callable.
  • draw_boxes2D: Callable.
  • wrap: Callable.

Methods

_build_draw_pose6D() call()