Skip to content

Keypoints

Models for 2D keypoint estimation

[source]

KeypointNet

paz.models.keypoint.keypointnet.KeypointNet(input_shape, num_keypoints, depth=0.2, filters=64, alpha=0.1)

Keypointnet model for discovering keypoint locations in 3D space

Arguments

  • input_shape: List of integers indicating [H, W, num_channels).
  • num_keypoints: Int. Number of keypoints to discover.
  • depth: Float. Prior depth (centimeters) of keypoints.
  • filters: Int. Number of filters used in convolutional layers.
  • alpha: Float. Alpha parameter of leaky relu.

Returns

Keras/tensorflow model

References

[source]

KeypointNet2D

paz.models.keypoint.keypointnet.KeypointNet2D(input_shape, num_keypoints, filters=64, alpha=0.1)

Model for discovering keypoint locations in 2D space, modified from

Arguments

  • input_shape: List of integers indicating [H, W, num_channels].
  • num_keypoints: Int. Number of keypoints to discover.
  • filters: Int. Number of filters used in convolutional layers.
  • alpha: Float. Alpha parameter of leaky relu.

Returns

Keras/tensorflow model

References

[source]

Projector

paz.models.keypoint.projector.Projector(self, focal_length, use_numpy=False)

Projects keypoints from image coordinates to 3D space and viceversa. This model uses the camera focal length and the depth estimation of a point to project it to image coordinates. It works with numpy matrices or tensorflow values. See use_numpy.

Arguments

  • focal_length: Float. Focal length of camera used to generate keypoints.
  • use_numpy: Boolean. If True both unproject and project functions take numpy arrays as inputs. If False takes tf.tensors as inputs.

[source]

DetNet

paz.models.keypoint.detnet.DetNet(input_shape=(128, 128, 3), num_keypoints=21)

DetNet: Estimate 3D keypoint positions of minimal hand from input color image.

Arguments

  • input_shape: Shape for 128x128 RGB image of left hand. List of integers. Input shape to the model including only spatial and channel resolution e.g. (128, 128, 3).
  • num_keypoints: Int. Number of keypoints.

Returns

Tensorflow-Keras model. - xyz: Numpy array [num_keypoints, 3]. Normalized 3D keypoint locations. - uv: Numpy array [num_keypoints, 2]. The uv coordinates of the keypoints on the heat map, whose resolution is 32x32.

Reference

-Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data

[source]

IKNet

paz.models.keypoint.iknet.IKNet(input_shape=(84, 3), num_keypoints=21, depth=6, width=1024)

IKNet: Estimate absolute joint angle for the minimal hand keypoints.

Arguments

  • input_shape: [num_keypoint x 4, 3]. Contains 3D keypoints, bone orientation, refrence keypoint, refrence bone orientation.
  • num_keypoints: Int. Number of keypoints.

Returns

Tensorflow-Keras model. absolute joint angle in quaternion representation.

Reference

[source]

SimpleBaseline

paz.models.keypoint.simplebaselines.SimpleBaseline(input_shape=(32,), num_keypoints=16, keypoints_dim=3, hidden_dim=1024, num_layers=2, rate=1, weights='human36m')

Model that predicts 3D keypoints from 2D keypoints Arguments

  • num_keypoints: numer of kepoints
  • keypoints_dim: dimension of keypoints
  • hidden_dim: size of hidden layers
  • input_shape: size of the input
  • num_layers: number of layers
  • rate: dropout drop rate

Returns

keypoints3D estimation model