1. class
  2. Biigle
  3. \
  4. Jobs
  5. \
  6. ProcessAnnotatedVideo

ProcessAnnotatedVideo

class ProcessAnnotatedVideo extends ProcessAnnotatedFile

Traits

SerializesModels
InteractsWithQueue
Queueable
Dispatchable

Constants

DINO_PATCH_SIZE

Size of a square input patch for generating feature vectors with DINO.

Properties

int $maxRedispatchTries

The maximum number of times the job may be redispatched.

 from  ProcessAnnotatedFile
int $tries

The number of times the job may be attempted.

 from  ProcessAnnotatedFile
protected bool $deleteWhenMissingModels

Ignore this job if the annotation does not exist any more.

 from  ProcessAnnotatedFile

Methods

array
getAnnotationBoundingBox(array $points, Shape $shape, int $pointPadding = self::DINO_PATCH_SIZE / 2, int $boxPadding = 0, int $minSize = 32)

Get the bounding box of an annotation

from  GenerateFeatureVectors
array
ensureBoxAspectRatio(array $box, int $aspectWidth, int $aspectHeight)

Modify a bounding box so it adheres to the aspect ratio given by width and height.

from  GenerateFeatureVectors
makeBoxContained(array $box, int|null $maxWidth, int|null $maxHeight)

Adjust the position and size of the box so it is contained in a box with the given dimensions.

from  GenerateFeatureVectors
array
getPointBoundingBox(array $points, int $padding)

Get the bounding box of a point annotation.

from  GenerateFeatureVectors
array
getCircleBoundingBox(array $points)

Get the bounding box of a circle annotation.

from  GenerateFeatureVectors
array
getPolygonBoundingBox(array $points)

Get the bounding box of an annotation that is no point, circle or whole frame.

from  GenerateFeatureVectors
array
makeBoxIntegers(array $box)

Round and cast box values to int.

from  GenerateFeatureVectors
array
generateAnnotationBoxes(VolumeFile $file, Collection $annotations)

No description

from  GenerateFeatureVectors
string
getCropBufferForPyworker(Image $image, array $box)

Get the byte string of the cropped and resized patch for the Python worker.

from  GenerateFeatureVectors
array
sendPyworkerRequest(string $buffer)

Send the PNG image crop to the Python worker and return the feature vector array.

from  GenerateFeatureVectors
__construct(VolumeFile $file, array $only = [], bool $skipPatches = false, bool $skipFeatureVectors = false, bool $skipSvgs = false, string|null $targetDisk = null, int $redispatchTries = 0)

Create a new job instance.

from  ProcessAnnotatedFile
static string
getTargetPath(Annotation $annotation, string|null $format = null)

Assemble the target path for an annotation patch.

from  ProcessAnnotatedFile
void
handle()

Execute the job.

from  ProcessAnnotatedFile
handleFile(VolumeFile $file, string $path)

Handle a single file.

bool
shouldGiveUpAfterException(Exception $e)

Determine if the job should give up trying because the error will likely not be fixed the next time.

from  ProcessAnnotatedFile
bool
shouldRetryAfterException(Exception $e)

Determine if this job should retry instead of fail after an exception

from  ProcessAnnotatedFile
void
createSvgs()

Generate and upload annotation SVGs for the file.

from  ProcessAnnotatedFile
void
createSvg(Annotation $annotation)

Generate and upload an SVG for the annotation.

from  ProcessAnnotatedFile
string
getAnnotationPatch(Image $image, array $points, Shape $shape)

Get the annotation patch as buffer.

from  ProcessAnnotatedFile
void
generateFeatureVectors(Collection $annotations, array|string $filePath)

Generates feature vectors for the specified annotations belonging to the file of this job. This method either creates new feature vector models or updates the existing ones for the annotations.

from  ProcessAnnotatedFile
void
updateOrCreateFeatureVectors(Collection $annotations, Generator $output)

Create the feature vectors based on the Python script output.

Builder
getAnnotationQuery(VolumeFile $file)

Get the query builder for the annotations (maybe filtered by IDs).

SVGNodeContainer
getSVGAnnotation(array $points, Shape $shape)

Draw annotation as SVG

from  ProcessAnnotatedFile
SVGRect
getRectangleSvgAnnotation(array $tuples)

Get an SVG rectangle element.

from  ProcessAnnotatedFile
SVGEllipse
getEllipseSvgAnnotation(array $tuples)

Get an SVG ellipse element.

from  ProcessAnnotatedFile
float
computeRotationAngle(array $v, array $u)

Computes angle between two vectors

from  ProcessAnnotatedFile
array
getOrientedCoordinates(array $tuples, Shape $shape)

Determines position of coordinate

from  ProcessAnnotatedFile
void
redispatch()

Dispatch a copy of this job with a delay.

from  ProcessAnnotatedFile
getVipsImageForPyworker(string $path, array $options = [])

Get the vips image instance for submission to the Python worker for feature vectors.

from  ProcessAnnotatedFile
void
processAnnotationChunk(Collection $annotations, Video $video)

Process a chunk of annotations of this job's file.

getVideo(string $path)

Get the FFMpeg video instance.

Image
getVideoFrame(Video $video, float $time, int $trySeek = 60)

Get a video frame from a specific time as VipsImage object.

Details

in GenerateFeatureVectors at line 29
array getAnnotationBoundingBox(array $points, Shape $shape, int $pointPadding = self::DINO_PATCH_SIZE / 2, int $boxPadding = 0, int $minSize = 32)

Get the bounding box of an annotation

Parameters

array $points
Shape $shape
int $pointPadding

The default is half the patch size of 224 that is expected by DINO.
int $boxPadding
int $minSize

Each side of the box should have at least this number of pixels. Must be divisible by 2.

Return Value

array

in GenerateFeatureVectors at line 78
array ensureBoxAspectRatio(array $box, int $aspectWidth, int $aspectHeight)

Modify a bounding box so it adheres to the aspect ratio given by width and height.

Parameters

array $box
int $aspectWidth
int $aspectHeight

Return Value

array

in GenerateFeatureVectors at line 116
makeBoxContained(array $box, int|null $maxWidth, int|null $maxHeight)

Adjust the position and size of the box so it is contained in a box with the given dimensions.

Parameters

array $box
int|null $maxWidth
int|null $maxHeight

in GenerateFeatureVectors at line 142
protected array getPointBoundingBox(array $points, int $padding)

Get the bounding box of a point annotation.

Parameters

array $points
int $padding

Return Value

array

in GenerateFeatureVectors at line 155
protected array getCircleBoundingBox(array $points)

Get the bounding box of a circle annotation.

Parameters

array $points

Return Value

array

in GenerateFeatureVectors at line 168
protected array getPolygonBoundingBox(array $points)

Get the bounding box of an annotation that is no point, circle or whole frame.

Parameters

array $points

Return Value

array

in GenerateFeatureVectors at line 188
protected array makeBoxIntegers(array $box)

Round and cast box values to int.

Parameters

array $box

Return Value

array

in GenerateFeatureVectors at line 193
protected array generateAnnotationBoxes(VolumeFile $file, Collection $annotations)

No description

Parameters

VolumeFile $file
Collection $annotations

Return Value

array

in GenerateFeatureVectors at line 219
protected string getCropBufferForPyworker(Image $image, array $box)

Get the byte string of the cropped and resized patch for the Python worker.

Parameters

Image $image
array $box

Return Value

string

in GenerateFeatureVectors at line 230
protected array sendPyworkerRequest(string $buffer)

Send the PNG image crop to the Python worker and return the feature vector array.

Parameters

string $buffer

Return Value

array

in ProcessAnnotatedFile at line 79
__construct(VolumeFile $file, array $only = [], bool $skipPatches = false, bool $skipFeatureVectors = false, bool $skipSvgs = false, string|null $targetDisk = null, int $redispatchTries = 0)

Create a new job instance.

Parameters

VolumeFile $file

The file to process.
array $only

If filled with annotation IDs belonging to the file, only the annotations will be processed.
bool $skipPatches

Disable generation of annotation patches.
bool $skipFeatureVectors

Disable generation of annotation feature vectors.
bool $skipSvgs

Disable generation of annotation SVGs.
string|null $targetDisk

The storage disk to store annotation patches to ( default is the configured largo.patch_storage_disk).
int $redispatchTries

Counter to track the number of attempted redispatches.

in ProcessAnnotatedFile at line 94
static string getTargetPath(Annotation $annotation, string|null $format = null)

Assemble the target path for an annotation patch.

Parameters

Annotation $annotation
string|null $format

Return Value

string

in ProcessAnnotatedFile at line 114
void handle()

Execute the job.

Return Value

void

at line 24
handleFile(VolumeFile $file, string $path)

Handle a single file.

Parameters

VolumeFile $file
string $path

Path to the cached file.

in ProcessAnnotatedFile at line 166
protected bool shouldGiveUpAfterException(Exception $e)

Determine if the job should give up trying because the error will likely not be fixed the next time.

Parameters

Exception $e

Return Value

bool

in ProcessAnnotatedFile at line 197
protected bool shouldRetryAfterException(Exception $e)

Determine if this job should retry instead of fail after an exception

Parameters

Exception $e

Return Value

bool

in ProcessAnnotatedFile at line 217
void createSvgs()

Generate and upload annotation SVGs for the file.

Return Value

void

in ProcessAnnotatedFile at line 228
protected void createSvg(Annotation $annotation)

Generate and upload an SVG for the annotation.

Parameters

Annotation $annotation

Return Value

void

in ProcessAnnotatedFile at line 268
protected string getAnnotationPatch(Image $image, array $points, Shape $shape)

Get the annotation patch as buffer.

Parameters

Image $image
array $points
Shape $shape

Return Value

string

in ProcessAnnotatedFile at line 300
protected void generateFeatureVectors(Collection $annotations, array|string $filePath)

Generates feature vectors for the specified annotations belonging to the file of this job. This method either creates new feature vector models or updates the existing ones for the annotations.

Parameters

Collection $annotations
array|string $filePath

If a string, a file path to the local image to use for feature vector generation. If an array, a map of annotation IDs to a local image file path.

Return Value

void

at line 98
protected void updateOrCreateFeatureVectors(Collection $annotations, Generator $output)

Create the feature vectors based on the Python script output.

Parameters

Collection $annotations
Generator $output

Return Value

void

at line 163
protected Builder getAnnotationQuery(VolumeFile $file)

Get the query builder for the annotations (maybe filtered by IDs).

Parameters

VolumeFile $file

Return Value

Builder

in ProcessAnnotatedFile at line 397
protected SVGNodeContainer getSVGAnnotation(array $points, Shape $shape)

Draw annotation as SVG

Parameters

array $points

one dimensional array filled with coordinates of annotation
Shape $shape

shape of given annotation

Return Value

SVGNodeContainer

annotation as SVG

in ProcessAnnotatedFile at line 452
protected SVGRect getRectangleSvgAnnotation(array $tuples)

Get an SVG rectangle element.

Parameters

array $tuples

Return Value

SVGRect

in ProcessAnnotatedFile at line 473
protected SVGEllipse getEllipseSvgAnnotation(array $tuples)

Get an SVG ellipse element.

Parameters

array $tuples

Return Value

SVGEllipse

in ProcessAnnotatedFile at line 501
protected float computeRotationAngle(array $v, array $u)

Computes angle between two vectors

Parameters

array $v

first vector
array $u

second vector

Return Value

float

rotation angle in degree *

in ProcessAnnotatedFile at line 532
protected array getOrientedCoordinates(array $tuples, Shape $shape)

Determines position of coordinate

Parameters

array $tuples

coordinates array
Shape $shape

shapeId of given annotation

Return Value

array

with coordinates assigned to their position on the plane

in ProcessAnnotatedFile at line 558
protected void redispatch()

Dispatch a copy of this job with a delay.

Return Value

void

in ProcessAnnotatedFile at line 575
protected getVipsImageForPyworker(string $path, array $options = [])

Get the vips image instance for submission to the Python worker for feature vectors.

Parameters

string $path
array $options

at line 42
protected void processAnnotationChunk(Collection $annotations, Video $video)

Process a chunk of annotations of this job's file.

Parameters

Collection $annotations
Video $video

Return Value

void

at line 124
protected getVideo(string $path)

Get the FFMpeg video instance.

Parameters

string $path

at line 138
protected Image getVideoFrame(Video $video, float $time, int $trySeek = 60)

Get a video frame from a specific time as VipsImage object.

Parameters

Video $video
float $time
int $trySeek

Return Value

Image