auto.Package

Import path: earth2studio.models.auto.Package

View source on GitHub

Documentation

A generic file system abstraction with local caching, uses Fsspec WholeFileCacheFileSystem to manage files. Designed to be used for accessing remote resources, particularly checkpoint files for pre-trained models. Presently supports public folders on NGC, huggingface repos, s3 and any other built in file system Fsspec supports.

Parameters:

• root (str) –

  Root directory for file system

• fs (AbstractFileSystem | None, default: None ) –

  The target filesystem to run underneath. If none is provided a fsspec filesystem will get initialized based on the protocal of the root url, by default None

• fs_options (dict, default: {} ) –

  Filesystem object keyword arguments to use, by default {}

• cache (bool | None, default: None ) –

  Toggle local caching, typically you want this to be true unless the package is a local file system. If None, will only use caching for remote packages, by default None

• cache_options (dict, default: {} ) –

  Caching options provided to Fsspec. See CachingFileSystem in fsspec for valid options filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.implementations.cached.CachingFileSystem, by default {}

default_cache classmethod

default_cache(path: str = '') -> str

Default cache location for packages located in ~/.cache/earth2studio

Parameters:

• path (str, default: '' ) –

  Sub-path in cache direction, by default ""

Returns:

• str –

  Local cache path

default_timeout classmethod

default_timeout() -> int

Default remote store timeout in seconds

Returns:

• int –

  Time out in seconds

default_blocksize classmethod

default_blocksize() -> int

Default remote store block size

Returns:

• int –

  Download block size in bytes

cache property

cache: str

Cache path

open

open(file_path: str) -> BufferedReader

Open file inside package, caching it to local cache store in the process

Parameters:

• file_path (str) –

  Local file path in package directory

Returns:

• BufferedReader –

  Opened file, can get file path with BufferedReader.name

resolve

resolve(file_path: str) -> str

Resolves current relative file path to absolute path inside Package cache

Parameters:

• file_path (str) –

  Local path of file in package directory

Returns:

• str –

  File path inside cache

get

get(file_path: str) -> str

PhysicsNeMo / backwards compatibility

Parameters:

• file_path (str) –

  Local path of file in package directory

Returns:

• str –

  File path inside cache