Filesystem¶

Filesystem cache backend. For usage details, see Backends: Filesystem.

`FileCache`	Filesystem cache backend.
`FileDict`	A dictionary-like interface to files on the local filesystem.
`LRUDict`	A SQLite db used to track LRU metadata for cached items:
`LRUFileDict`	A size-restricted version of FileDict, using LRU eviction.

class requests_cache.backends.filesystem.FileCache(cache_name='http_cache', use_temp=False, decode_content=True, serializer=None, **kwargs)[source]¶

Bases: BaseCache

Filesystem cache backend.

Parameters:

cache_name (Union[Path, str]) – Base directory for cache files
use_cache_dir – Store database in a user cache directory (e.g., ~/.cache/)
use_temp (bool) – Store cache files in a temp directory (e.g., /tmp/http_cache/). Note: if cache_name is an absolute path, this option will be ignored.
decode_content (bool) – Decode JSON or text response body into a human-readable format
extension – Extension for cache files. If not specified, the serializer default extension will be used.
max_cache_bytes – Enable LRU caching, and set the maximum total size (in bytes) of cached responses on the file system.
max_file_bytes – The maximum size of a single file. By default, this is the same as max_cache_bytes. Only used if max_cache_bytes is set.
block_bytes – The size of a block of data on the file system, which will be used when computing total file size on disk. Only used if max_cache_bytes is set.
sync_index – On startup, sync LRU metadata with any changes on disk since last use. Use this if you intend to modify cache files outside of requests-cache. Leave off to reduce startup time for larger caches. Only used if max_cache_bytes is set.
lock – Replace the default threading.RLock object without your own. Use this if you want to share the lock between multiple cache instances, and/or use a different lock type (such as multiprocessing.RLock or filelock.FileLock).
serializer (Union[str, SerializerPipeline, Stage, None])

property cache_dir: Path¶: Base directory for cache files

clear()[source]¶: Clear the cache

close()¶: Close any open backend connections

contains(key=None, request=None, url=None, verify=True)¶

Check if the specified request is cached

Parameters:

key (Optional[str]) – Check for a specific cache key
request (Union[Request, PreparedRequest, CachedRequest, None]) – Check for a matching request, according to current request matching settings
url (Optional[str]) – Check for a matching GET request with the specified URL
verify (bool) – Check for requests with or without SSL verification

create_key(request, match_headers=None, **kwargs)¶

Create a normalized cache key from a request object

Parameters:

request (Union[Request, PreparedRequest, CachedRequest])
match_headers (Optional[Iterable[str]])

Return type:

str

property db_path: str | Path¶

Path to the cache database file.

Raises:: NotImplementedError – For backends that do not use a file-based database.

delete(*args, **kwargs)[source]¶

Remove responses from the cache according one or more conditions.

Parameters:

keys – Remove responses with these cache keys
expired – Remove all expired responses
invalid – Remove all invalid responses (that can’t be deserialized with current settings)
older_than – Remove responses older than this value, relative to response.created_at
requests – Remove matching responses, according to current request matching settings
urls – Remove matching GET requests for the specified URL(s)
verify – Set to False to remove responses without SSL verification

filter(valid=True, expired=True, invalid=False, older_than=None)¶

Get responses from the cache, with optional filters for which responses to include:

Parameters:

valid (bool) – Include valid and unexpired responses; set to False to get only expired/invalid/old responses
expired (bool) – Include expired responses
invalid (bool) – Include invalid responses (as an empty CachedResponse)
older_than (Union[None, int, float, str, datetime, timedelta]) – Get responses older than this value, relative to response.created_at

Return type:

Iterator[CachedResponse]

get_response(key, default=None)¶

Retrieve a response from the cache, if it exists

Parameters:

key (str) – Cache key for the response
default – Value to return if key is not in the cache

Return type:

Optional[CachedResponse]

property lock: RLock¶: The lock used by the cache.

paths()[source]¶

Get absolute file paths to all cached responses

Return type:: Iterator[Path]

recreate_keys()¶: Recreate cache keys for all previously cached responses

remove_expired_responses(expire_after=None)¶

Parameters:: expire_after (Union[None, int, float, str, datetime, timedelta])

reset_expiration(expire_after=None)¶

Set a new expiration value to set on existing cache items

Parameters:: expire_after (Union[None, int, float, str, datetime, timedelta]) – New expiration value, relative to the current time

save_response(response, cache_key=None, expires=None)¶

Save a response to the cache

Parameters:

cache_key (Optional[str]) – Cache key for this response; will otherwise be generated based on request
response (Response) – Response to save
expires (Optional[datetime]) – Absolute expiration time for this response

update(other)¶

Update this cache with the contents of another cache

Parameters:: other (BaseCache)

urls(**kwargs)¶

Get all unique cached URLs. Optionally takes keyword arguments for filter().

Return type:: List[str]

class requests_cache.backends.filesystem.FileDict(cache_name, use_temp=False, use_cache_dir=False, extension=None, serializer=<requests_cache.serializers.pipeline.SerializerPipeline object>, lock=None, **kwargs)[source]¶

Bases: BaseStorage

A dictionary-like interface to files on the local filesystem.

The cache directory will be created if it doesn’t already exist.

Parameters:

cache_name (Union[Path, str])
use_temp (bool)
use_cache_dir (bool)
extension (Optional[str])
serializer (Union[str, SerializerPipeline, Stage, None])
lock (Optional[RLock])

bulk_delete(keys)¶

Delete multiple keys from the cache, without raising errors for missing keys.

This is a naive, generic implementation that subclasses should override with a more efficient backend-specific implementation, if possible.

Parameters:: keys (Iterable[TypeVar(KT)])

clear()[source]¶

Empty the cache directory.

Return type:: None

close()¶: Close any open backend connections

deserialize(key, value)¶

Deserialize a value, if a serializer is available.

If deserialization fails (usually due to a value saved in an older requests-cache version), None will be returned.

Parameters:: value (TypeVar(VT))

get(k[, d]) → D[k] if k in D, else d. d defaults to None.¶

items() → a set-like object providing a view on D's items¶

keys() → a set-like object providing a view on D's keys[source]¶

property lock: RLock¶: The lock used by the cache.

paths()[source]¶

Get absolute file paths to all cached responses

Return type:: Iterator[Path]

pop(k[, d]) → v, remove specified key and return the corresponding value.¶: If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair¶: as a 2-tuple; but raise KeyError if D is empty.

serialize(value)¶

Serialize a value, if a serializer is available

Parameters:: value (TypeVar(VT))

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶

size()[source]¶

Return the size of the database, in bytes

Return type:: int

update([E, ]**F) → None. Update D from mapping/iterable E and F.¶: If E present and has a .keys() method, does: for k in E.keys(): D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() → an object providing a view on D's values¶

class requests_cache.backends.filesystem.LRUDict(*args, **kwargs)[source]¶

Bases: SQLiteDict

A SQLite db used to track LRU metadata for cached items:

key: The cache key
access_time: The last access time, as a UNIX timestamp in nanoseconds
size: The size of the cached item, in bytes
total_size Combined size of all cache items in bytes, accessed with total_size().

Implementation Notes:

total_size is managed by triggers and stored in a separate single-row table.
As a dict-like interface, size is treated as the main value and access_time is set automatically and updated with update_access_time().
get_lru() Can select multiple keys to evict, up to an arbitrary total size, within a single query using a window function.

bulk_commit()¶

Insert a large number of records within a single transaction

Example

>>> d1 = SQLiteDict('test')
>>> with d1.bulk_commit():
...     for i in range(1000):
...         d1[i] = i * 2

bulk_delete(keys=None, values=None)¶: Delete multiple items from the cache, without raising errors for any missing items. Supports deleting by either key or by value.

clear() → None. Remove all items from D.[source]¶

close()¶: Close any active connections

connection(commit=False)¶

Get a thread-local database connection

Return type:: Iterator[Connection]

count(*args, **kwargs)[source]¶: Count number of responses, optionally excluding expired

deserialize(key, value)¶

Deserialize a value, if a serializer is available.

If deserialization fails (usually due to a value saved in an older requests-cache version), None will be returned.

Parameters:: value (TypeVar(VT))

get(k[, d]) → D[k] if k in D, else d. d defaults to None.¶

get_lru(total_size)[source]¶

Get the least recently used keys with a combined size >= total_size

Parameters:: total_size (int)

init_db()[source]¶: Initialize the database, if it hasn’t already been

items() → a set-like object providing a view on D's items¶

keys() → a set-like object providing a view on D's keys¶

pop(k[, d]) → v, remove specified key and return the corresponding value.¶: If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair¶: as a 2-tuple; but raise KeyError if D is empty.

serialize(value)¶

Serialize a value, if a serializer is available

Parameters:: value (TypeVar(VT))

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶

size()¶

Return the size of the database, in bytes. For an in-memory database, this will be an estimate based on page size.

Return type:: int

sorted(key='access_time', reversed=False, limit=None, **kwargs)[source]¶

Get LRU entries in sorted order, by either access_time or size

Parameters:

key (str)
reversed (bool)
limit (Optional[int])

Return type:

Iterator[str]

total_size()[source]¶

Return type:: int

update([E, ]**F) → None. Update D from mapping/iterable E and F.¶: If E present and has a .keys() method, does: for k in E.keys(): D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

update_access_time(key)[source]¶

Update the given key with the current timestamp

Raises:: KeyError – If the key doesn’t exist in the LRU index
Parameters:: key (str)

vacuum()¶

values() → an object providing a view on D's values¶

class requests_cache.backends.filesystem.LRUFileDict(*args, block_bytes=1, max_cache_bytes=104857600, max_file_bytes=None, sync_index=False, **kwargs)[source]¶

Bases: FileDict

A size-restricted version of FileDict, using LRU eviction.

Parameters:

block_bytes (int) – The size of a block of data on the file system. File sizes will be aligned with this.
max_cache_bytes (int) – The maximum total size of all files in the cache.
max_file_bytes (Optional[int]) – The maximum size of a single file. By default, this is the same as max_cache_bytes.
sync_index (bool) – Check for filesystem changes since last use. Use this if you intend to modify cache files outside of requests-cache. Leave off to reduce startup time for larger caches.

bulk_delete(keys)¶

Delete multiple keys from the cache, without raising errors for missing keys.

This is a naive, generic implementation that subclasses should override with a more efficient backend-specific implementation, if possible.

Parameters:: keys (Iterable[TypeVar(KT)])

clear()[source]¶: Clear the cache directory and LRU index.

close()¶: Close any open backend connections

deserialize(key, value)¶

Deserialize a value, if a serializer is available.

If deserialization fails (usually due to a value saved in an older requests-cache version), None will be returned.

Parameters:: value (TypeVar(VT))

get(k[, d]) → D[k] if k in D, else d. d defaults to None.¶

items() → a set-like object providing a view on D's items¶

keys() → a set-like object providing a view on D's keys¶

property lock: RLock¶: The lock used by the cache.

paths()¶

Get absolute file paths to all cached responses

Return type:: Iterator[Path]

pop(k[, d]) → v, remove specified key and return the corresponding value.¶: If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair¶: as a 2-tuple; but raise KeyError if D is empty.

serialize(value)¶

Serialize a value, if a serializer is available

Parameters:: value (TypeVar(VT))

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶

size()[source]¶

Return the size of the database, in bytes

Return type:: int

update([E, ]**F) → None. Update D from mapping/iterable E and F.¶: If E present and has a .keys() method, does: for k in E.keys(): D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() → an object providing a view on D's values¶