Skip to content

Introduction

PAZ is a hierarchical perception library in Python.

Selected examples:

PAZ is used in the following examples (links to real-time demos and training scripts):

Probabilistic 2D keypoints 6D head-pose estimation Object detection
Emotion classifier 2D keypoint estimation Mask-RCNN (in-progress)
3D keypoint discovery Haar Cascade detector 6D pose estimation
Implicit orientation Attention (STNs)

All models can be re-trained with your own data (except for Mask-RCNN, we are working on it here).

Hierarchical APIs

PAZ can be used with three diferent API levels which are there to be helpful for the user's specific application.

High-level

Easy out-of-the-box prediction. For example, for detecting objects we can call the following pipeline:

from paz.pipelines import SSD512COCO

detect = SSD512COCO()

# apply directly to an image (numpy-array)
inferences = detect(image)

There are multiple high-level functions a.k.a. pipelines already implemented in PAZ here. Those functions are build using our mid-level API described now below.

Mid-level

While the high-level API is useful for quick applications, it might not be flexible enough for your specific purporse. Therefore, in PAZ we can build high-level functions using our a mid-level API.

Mid-level: Sequential

If your function is sequential you can construct a sequential function using SequentialProcessor. In the example below we create a data-augmentation pipeline:

from paz.abstract import SequentialProcessor
from paz import processors as pr

augment = SequentialProcessor()
augment.add(pr.RandomContrast())
augment.add(pr.RandomBrightness())
augment.add(pr.RandomSaturation())
augment.add(pr.RandomHue())

# you can now use this now as a normal function
image = augment(image)

You can also add any function not only those found in processors. For example we can pass a numpy function to our original data-augmentation pipeline:

augment.add(np.mean)

There are multiple functions a.k.a. Processors already implemented in PAZ here.

Using these processors we can build more complex pipelines e.g. data augmentation for object detection: pr.AugmentDetection

Mid-level: Explicit

Non-sequential pipelines can be also build by abstracting Processor. In the example below we build a emotion classifier from scratch using our high-level and mid-level functions.

from paz.applications import HaarCascadeFrontalFace, MiniXceptionFER
import paz.processors as pr

class EmotionDetector(pr.Processor):
    def __init__(self):
        super(EmotionDetector, self).__init__()
        self.detect = HaarCascadeFrontalFace(draw=False)
        self.crop = pr.CropBoxes2D()
        self.classify = MiniXceptionFER()
        self.draw = pr.DrawBoxes2D(self.classify.class_names)

    def call(self, image):
        boxes2D = self.detect(image)['boxes2D']
        cropped_images = self.crop(image, boxes2D)
        for cropped_image, box2D in zip(cropped_images, boxes2D):
            box2D.class_name = self.classify(cropped_image)['class_name']
        return self.draw(image, boxes2D)

detect = EmotionDetector()
# you can now apply it to an image (numpy array)
predictions = detect(image)

Processors allow us to easily compose, compress and extract away parameters of functions. However, most processors are build using our low-level API (backend) shown next.

Low-level

Mid-level processors are mostly built from small backend functions found in: boxes, cameras, images, keypoints and quaternions.

These functions can found in paz.backend:

from paz.backend import boxes, camera, image, keypoints, quaternion

For example, you can use them in your scripts to load or show images:

from paz.backend.image import load_image, show_image

image = load_image('my_image.png')
show_image(image)

Additional functionality