1. class
  2. Biigle
  3. \
  4. FileCache
  5. \
  6. FileCache

FileCache

class FileCache implements FileCache

The file cache.

Constants

LOCK_DIR

Name of the subdirectory in the cache path that holds the per-process lock files.
REFS_DIR

Name of the subdirectory in the cache path that holds the reference links.
MAX_RETRIEVE_ATTEMPTS

Maximum number of times retrieve() retries when the cached file keeps disappearing or is left empty by a failing writer. Bounds the retries so a persistently failing file cannot loop forever.

Properties

protected int $linkCount

Counter for reference links used to generate unique names per instance.
protected array $config

File cache configuration.
protected Filesystem $files

The Filesytem instance to use
protected FilesystemManager $storage

File FilesystemManager instance to use
protected ClientInterface $client

Guzzle HTTP client to use
protected resource|null $lockHandle

Open file handle holding the exclusive lock that signals this process is alive. Created lazily and held for the lifetime of the process.
protected string|null $lockToken

Unique token identifying this process. Embedded in every reference link name and used as the name of this process' lock file.

Methods

__construct(array $config = [], Filesystem $files = null, FilesystemManager $storage = null, $client = null)

Create an instance.

__destruct()

Release the process lock when the instance is destroyed. If this does not run (e.g. on a crash) the kernel releases the lock anyway and the orphan lock file is reaped by the pruner.

bool
exists(File $file)

Check if a file exists.

mixed
get(File $file, callable $callback, bool $throwOnLock = false)

Perform a callback with the path of a cached file. This takes care of shared locks on the cached file file so it is not corrupted due to concurrent write operations.

mixed
getOnce(File $file, callable $callback, bool $throwOnLock = false)

Like get but deletes the cached file afterwards (if it is not used somewhere else).

resource
getStream(File $file)

Get a stream resource for an file. If the file is cached, the resource points to the cached file instead. This will not cache uncached files. Make sure to close the streams!

mixed
batch(array $files, callable $callback, bool $throwOnLock = false)

Perform a callback with the paths of many cached files. Use this to prevent pruning of the files while they are processed.

mixed
batchOnce(array $files, callable $callback, bool $throwOnLock = false)

Like batch but deletes the cached files afterwards (if they are not used somewhere else).

mixed
runBatch(array $files, callable $callback, bool $throwOnLock, callable $cleanup)

Retrieve the files, run the callback with their paths and clean up the reference links afterwards.

prune()

Remove cached files that are too old or exceed the maximum cache size.

clear()

Delete all unused cached files.

bool
existsRemote(File $file)

Check for existence of a remte file.

bool
existsDisk(File $file)

Check for existence of a file from a storage disk.

assertMimeTypeAllowed(string $type)

Throw if the given MIME type is not in the configured allow list.

assertSizeAllowed(int $size)

Throw if the given file size exceeds the configured max_file_size. A negative max_file_size disables the check.

bool
delete(SplFileInfo $file)

Delete a cached file it it is not used.

array
retrieve(File $file, bool $throwOnLock = false, int $attempt = 0)

Cache a remote or cloud storage file if it is not cached and get the path to the cached file. If the file is local, nothing will be done and the path to the local file will be returned.

array
retrieveExistingFile(string $cachedPath, resource $handle)

Get path and reference link for a file that exists in the cache.

array
retrieveNewFile(File $file, string $cachedPath, resource $handle)

Get path and reference link for a file that does not yet exist in the cache.

string
createReferenceLink(string $cachedPath, resource $handle)

Create the reference link that signals the cached file is in use and release the lock handle.

Finder
canonicalFiles()

Get a Finder for the canonical cache files, excluding reference links and lock files.

string
referenceLinkToken(string $filename)

Extract the owning process' lock token from a reference link filename.

ensureProcessLock()

Acquire the lock that signals this process is alive, holding it for the lifetime of the process.

bool
isProcessAlive(string $token)

Determine whether the process owning the given lock token is still alive.

pruneStaleReferences()

Remove reference links left behind by crashed workers.

int
pruneFilesByMaxAge()

Prune canonical files that are older than max_age.

pruneFilesByMaxSize(int $currentSize)

Prune oldest canonical files that exceed the max_size.

pruneOrphanLockFiles()

Remove lock files of processes that have terminated.

string
lockDir()

Get the directory in which process lock files are stored.

string
refsDir()

Get the directory in which reference links are stored.

string
lockPath(string $token)

Get the path to the lock file for the given process token.

string
getDiskFile(File $file, resource $target)

Cache an file from a storage disk and get the path to the cached file. Files from local disks are not cached.

string
cacheFromResource(File $file, resource $source, resource $target)

Store the file from the given resource to a cached file.

ensureDirExists(string $dir)

Creates the directory if it doesn't exist yet.

string
getCachedPath(File $file)

Get the path to the cached file file.

resource
getFileStream(string $url)

Get the stream resource for an file.

bool
isRemote(File $file)

Determine if an file is remote, i.e. served by a public webserver.

Filesystem
getDisk(File $file)

Get the storage disk on which a file is stored.

ClientInterface
makeHttpClient()

Create a new Guzzle HTTP client.

Details

at line 105
__construct(array $config = [], Filesystem $files = null, FilesystemManager $storage = null, $client = null)

Create an instance.

Parameters

array $config

Optional custom configuration.
Filesystem $files
FilesystemManager $storage
$client

at line 118
__destruct()

Release the process lock when the instance is destroyed. If this does not run (e.g. on a crash) the kernel releases the lock anyway and the orphan lock file is reaped by the pruner.

at line 132
bool exists(File $file)

Check if a file exists.

Parameters

File $file

Return Value

bool

Whether the file exists or not.

at line 144
mixed get(File $file, callable $callback, bool $throwOnLock = false)

Perform a callback with the path of a cached file. This takes care of shared locks on the cached file file so it is not corrupted due to concurrent write operations.

Parameters

File $file
callable $callback

Gets the file object and the path to the cached file file as arguments.
bool $throwOnLock

Whether to throw an exception if a file is currently locked (i.e. written to). Otherwise the method will wait until the lock is released.

Return Value

mixed

Result of the callback.

at line 154
mixed getOnce(File $file, callable $callback, bool $throwOnLock = false)

Like get but deletes the cached file afterwards (if it is not used somewhere else).

Parameters

File $file
callable $callback

Gets the file object and the path to the cached file file as arguments.
bool $throwOnLock

Whether to throw an exception if a file is currently locked (i.e. written to). Otherwise the method will wait until the lock is released.

Return Value

mixed

Result of the callback.

at line 164
resource getStream(File $file)

Get a stream resource for an file. If the file is cached, the resource points to the cached file instead. This will not cache uncached files. Make sure to close the streams!

Parameters

File $file

Return Value

resource

Exceptions

Exception

at line 200
mixed batch(array $files, callable $callback, bool $throwOnLock = false)

Perform a callback with the paths of many cached files. Use this to prevent pruning of the files while they are processed.

Parameters

array $files
callable $callback

Gets the array of file objects and the array of paths to the cached file files (in the same ordering) as arguments.
bool $throwOnLock

Whether to throw an exception if a file is currently locked (i.e. written to). Otherwise the method will wait until the lock is released.

Return Value

mixed

Result of the callback.

at line 213
mixed batchOnce(array $files, callable $callback, bool $throwOnLock = false)

Like batch but deletes the cached files afterwards (if they are not used somewhere else).

Parameters

array $files
callable $callback

Gets the array of file objects and the array of paths to the cached file files (in the same ordering) as arguments.
bool $throwOnLock

Whether to throw an exception if a file is currently locked (i.e. written to). Otherwise the method will wait until the lock is released.

Return Value

mixed

Result of the callback.

at line 233
protected mixed runBatch(array $files, callable $callback, bool $throwOnLock, callable $cleanup)

Retrieve the files, run the callback with their paths and clean up the reference links afterwards.

Parameters

array $files
callable $callback
bool $throwOnLock
callable $cleanup

Cleanup applied to each retrieved file that has a reference link (i.e. is not locally stored).

Return Value

mixed

The return value of the callback.

at line 267
prune()

Remove cached files that are too old or exceed the maximum cache size.

at line 286
clear()

Delete all unused cached files.

at line 311
protected bool existsRemote(File $file)

Check for existence of a remte file.

Parameters

File $file

Return Value

bool

at line 357
protected bool existsDisk(File $file)

Check for existence of a file from a storage disk.

Parameters

File $file

Return Value

bool

at line 386
protected assertMimeTypeAllowed(string $type)

Throw if the given MIME type is not in the configured allow list.

Parameters

string $type

Exceptions

Exception

at line 401
protected assertSizeAllowed(int $size)

Throw if the given file size exceeds the configured max_file_size. A negative max_file_size disables the check.

Parameters

int $size

Size in bytes.

Exceptions

Exception

at line 417
protected bool delete(SplFileInfo $file)

Delete a cached file it it is not used.

Parameters

SplFileInfo $file

Return Value

bool

If the file has been deleted.

at line 472
protected array retrieve(File $file, bool $throwOnLock = false, int $attempt = 0)

Cache a remote or cloud storage file if it is not cached and get the path to the cached file. If the file is local, nothing will be done and the path to the local file will be returned.

Parameters

File $file

File to get the path for
bool $throwOnLock

Whether to throw an exception if a file is currently locked (i.e. written to). Otherwise the method will wait until the lock is released.
int $attempt

Current retry attempt. Used internally to bound the recursion.

Return Value

array

Containing the 'path' to the file and the reference 'link' (or null for locally stored files). Unlink the reference link when finished.

Exceptions

Exception
FileLockedException

at line 567
protected array retrieveExistingFile(string $cachedPath, resource $handle)

Get path and reference link for a file that exists in the cache.

Parameters

string $cachedPath
resource $handle

Return Value

array

at line 588
protected array retrieveNewFile(File $file, string $cachedPath, resource $handle)

Get path and reference link for a file that does not yet exist in the cache.

Parameters

File $file
string $cachedPath
resource $handle

Return Value

array

Create the reference link that signals the cached file is in use and release the lock handle.

Parameters

string $cachedPath
resource $handle

Return Value

string

Path to the reference link.

at line 681
protected Finder canonicalFiles()

Get a Finder for the canonical cache files, excluding reference links and lock files.

Return Value

Finder

at line 698
protected string referenceLinkToken(string $filename)

Extract the owning process' lock token from a reference link filename.

Parameters

string $filename

Return Value

string

at line 713
protected ensureProcessLock()

Acquire the lock that signals this process is alive, holding it for the lifetime of the process.

The lock is created lazily the first time a reference link is needed. The kernel releases it when the process exits (including crashes), which is how the pruner detects that this worker's reference links are stale. flock also works across (Docker) containers sharing the cache volume because the lock lives in the host kernel on the shared inode.

at line 745
protected bool isProcessAlive(string $token)

Determine whether the process owning the given lock token is still alive.

Parameters

string $token

Return Value

bool

at line 775
protected pruneStaleReferences()

Remove reference links left behind by crashed workers.

at line 810
protected int pruneFilesByMaxAge()

Prune canonical files that are older than max_age.

Return Value

int

The total size in bytes of the remaining files.

at line 844
protected pruneFilesByMaxSize(int $currentSize)

Prune oldest canonical files that exceed the max_size.

Parameters

int $currentSize

Current total size of the files.

at line 894
protected pruneOrphanLockFiles()

Remove lock files of processes that have terminated.

at line 936
protected string lockDir()

Get the directory in which process lock files are stored.

Return Value

string

at line 946
protected string refsDir()

Get the directory in which reference links are stored.

Return Value

string

at line 958
protected string lockPath(string $token)

Get the path to the lock file for the given process token.

Parameters

string $token

Return Value

string

at line 973
protected string getDiskFile(File $file, resource $target)

Cache an file from a storage disk and get the path to the cached file. Files from local disks are not cached.

Parameters

File $file

Cloud storage file
resource $target

Target file resource

Return Value

string

Exceptions

Exception

at line 1016
protected string cacheFromResource(File $file, resource $source, resource $target)

Store the file from the given resource to a cached file.

Parameters

File $file
resource $source
resource $target

Return Value

string

Path to the cached file

Exceptions

Exception

at line 1054
protected ensureDirExists(string $dir)

Creates the directory if it doesn't exist yet.

Parameters

string $dir

at line 1068
protected string getCachedPath(File $file)

Get the path to the cached file file.

Parameters

File $file

Return Value

string

at line 1082
protected resource getFileStream(string $url)

Get the stream resource for an file.

Parameters

string $url

Return Value

resource

at line 1098
protected bool isRemote(File $file)

Determine if an file is remote, i.e. served by a public webserver.

Parameters

File $file

Return Value

bool

at line 1110
protected Filesystem getDisk(File $file)

Get the storage disk on which a file is stored.

Parameters

File $file

Return Value

Filesystem

at line 1123
protected ClientInterface makeHttpClient()

Create a new Guzzle HTTP client.

Return Value

ClientInterface