requests_cache.backends package

Classes and functions for cache persistence

requests_cache.backends.get_valid_kwargs(func, kwargs, extras=None)[source]

Get the subset of non-None kwargs that are valid params for func

Return type

Dict

requests_cache.backends.init_backend(backend=None, *args, **kwargs)[source]

Initialize a backend from a name, class, or instance

Return type

BaseCache

Submodules

requests_cache.backends.base module

class requests_cache.backends.base.BaseCache(*args, include_get_headers=False, ignored_parameters=None, **kwargs)[source]

Bases: object

Base class for cache implementations, which can also be used as in-memory cache.

See Custom Backends for details on creating your own implementation.

bulk_delete(keys)[source]

Remove multiple responses and their associated redirects from the cache

clear()[source]

Delete all items from the cache

create_key(request, **kwargs)[source]

Create a normalized cache key from a request object

Return type

str

delete(key)[source]

Delete a response or redirect from the cache, as well any associated redirect history

delete_url(url)[source]

Delete a cached response + redirects for GET <url>

delete_urls(urls)[source]

Delete cached responses + redirects for multiple request URLs (GET requests only)

get_response(key, default=None)[source]

Retrieves response for key if it’s stored in cache, otherwise returns default

Parameters
  • key (str) – Key of resource

  • default – Value to return if key is not in cache

Return type

CachedResponse

has_key(key)[source]

Returns True if cache has key, False otherwise

Return type

bool

has_url(url)[source]

Returns True if cache has url, False otherwise. Works only for GET request urls

Return type

bool

keys(check_expiry=False)[source]

Get all cache keys for redirects and valid responses combined

Return type

Iterator[str]

remove_expired_responses(expire_after=None)[source]

Remove expired and invalid responses from the cache, optionally with revalidation

Parameters

expire_after (Union[None, int, float, str, datetime, timedelta]) – A new expiration time used to revalidate the cache

remove_old_entries(*args, **kwargs)[source]
response_count(check_expiry=False)[source]

Get the number of responses in the cache, excluding invalid (unusable) responses. Can also optionally exclude expired responses.

Return type

int

save_redirect(request, response_key)[source]

Map a redirect request to a response. This makes it possible to associate many keys with a single response.

Parameters
  • request (Union[PreparedRequest, CachedRequest]) – Request object for redirect URL

  • response_key (str) – Cache key which can be found in responses

save_response(response, cache_key=None, expires=None)[source]

Save response to cache

Parameters
  • cache_key (Optional[str]) – Cache key for this response; will otherwise be generated based on request

  • response (Union[Response, CachedResponse]) – response to save

  • expire_after – Time in seconds until this cache item should expire

property urls: Iterator[str]

Get all URLs currently in the cache (excluding redirects)

Return type

Iterator[str]

values(check_expiry=False)[source]

Get all valid response objects from the cache

Return type

Iterator[CachedResponse]

class requests_cache.backends.base.BaseStorage(serializer=None, **kwargs)[source]

Bases: collections.abc.MutableMapping, abc.ABC

Base class for backend storage implementations

Parameters
  • secret_key – Optional secret key used to sign cache items for added security

  • salt – Optional salt used to sign cache items

  • serializer – Custom serializer that provides loads and dumps methods

bulk_delete(keys)[source]

Delete multiple keys from the cache. Does not raise errors for missing keys. This is a basic version that subclasses should override with a more efficient backend-specific version, if possible.

class requests_cache.backends.base.DictStorage(dict=None, /, **kwargs)[source]

Bases: collections.UserDict, requests_cache.backends.base.BaseStorage

A basic dict wrapper class for non-persistent storage

requests_cache.backends.dynamodb module

class requests_cache.backends.dynamodb.DynamoDbCache(table_name='http_cache', connection=None, **kwargs)[source]

Bases: requests_cache.backends.base.BaseCache

DynamoDB cache backend

Parameters
class requests_cache.backends.dynamodb.DynamoDbDict(table_name, namespace='http_cache', connection=None, read_capacity_units=1, write_capacity_units=1, **kwargs)[source]

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface for DynamoDB key-value store

Note: The actual key name on the dynamodb server will be namespace:table_name

In order to deal with how dynamodb stores data/keys, everything, i.e. keys and data, must be pickled.

Parameters
  • table_name – DynamoDb table name

  • namespace – Name of DynamoDb hash map

  • connectionDynamoDb Resource object to use instead of creating a new one

  • kwargs – Additional keyword arguments for resource()

bulk_delete(keys)[source]

Delete multiple keys from the cache. Does not raise errors for missing keys.

clear() None.  Remove all items from D.[source]
composite_key(key)[source]
Return type

Dict[str, str]

requests_cache.backends.filesystem module

class requests_cache.backends.filesystem.FileCache(cache_name='http_cache', use_temp=False, **kwargs)[source]

Bases: requests_cache.backends.base.BaseCache

Backend that stores cached responses as files on the local filesystem. Response paths will be in the format <cache_name>/responses/<cache_key>. Redirects are stored in a SQLite database, located at <cache_name>/redirects.sqlite.

Parameters
  • cache_name (Union[Path, str]) – Base directory for cache files

  • use_temp (bool) – Store cache files in a temp directory (e.g., /tmp/http_cache/). Note: if cache_name is an absolute path, this option will be ignored.

  • extension – Extension for cache files. If not specified, the serializer default extension will be used.

class requests_cache.backends.filesystem.FileDict(cache_name, use_temp=False, extension=None, **kwargs)[source]

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface to files on the local filesystem

clear() None.  Remove all items from D.[source]
paths()[source]

Get file paths to all cached responses

requests_cache.backends.gridfs module

class requests_cache.backends.gridfs.GridFSCache(db_name, **kwargs)[source]

Bases: requests_cache.backends.base.BaseCache

GridFS cache backend. Use this backend to store documents greater than 16MB.

Example

>>> requests_cache.install_cache(backend='gridfs')
>>>
>>> # Or, to provide custom connection settings:
>>> from pymongo import MongoClient
>>> requests_cache.install_cache(backend='gridfs', connection=MongoClient('alternate-host'))
Parameters
  • db_name (str) – Database name

  • connectionpymongo.MongoClient object to reuse instead of creating a new one

  • kwargs – Additional keyword arguments for pymongo.MongoClient

class requests_cache.backends.gridfs.GridFSPickleDict(db_name, collection_name=None, connection=None, **kwargs)[source]

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface for a GridFS database

Parameters
  • db_name – Database name

  • collection_name – Ignored; GridFS internally uses collections ‘fs.files’ and ‘fs.chunks’

  • connectionpymongo.MongoClient object to reuse instead of creating a new one

  • kwargs – Additional keyword arguments for pymongo.MongoClient

clear() None.  Remove all items from D.[source]

requests_cache.backends.mongo module

class requests_cache.backends.mongo.MongoCache(db_name='http_cache', connection=None, **kwargs)[source]

Bases: requests_cache.backends.base.BaseCache

MongoDB cache backend

Parameters
  • db_name (str) – Database name

  • connection (Optional[MongoClient]) – pymongo.MongoClient object to reuse instead of creating a new one

  • kwargs – Additional keyword arguments for pymongo.MongoClient

class requests_cache.backends.mongo.MongoDict(db_name, collection_name='http_cache', connection=None, **kwargs)[source]

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface for a MongoDB collection

Parameters
  • db_name – Database name

  • collection_name – Collection name

  • connectionpymongo.MongoClient object to reuse instead of creating a new one

  • kwargs – Additional keyword arguments for pymongo.MongoClient

bulk_delete(keys)[source]

Delete multiple keys from the cache. Does not raise errors for missing keys.

clear() None.  Remove all items from D.[source]
class requests_cache.backends.mongo.MongoPickleDict(db_name, collection_name='http_cache', connection=None, **kwargs)[source]

Bases: requests_cache.backends.mongo.MongoDict

Same as MongoDict, but pickles values before saving

requests_cache.backends.redis module

class requests_cache.backends.redis.RedisCache(namespace='http_cache', connection=None, **kwargs)[source]

Bases: requests_cache.backends.base.BaseCache

Redis cache backend.

Parameters
  • namespace – redis namespace (default: 'requests-cache')

  • connection (Optional[Redis]) – Redis connection instance to use instead of creating a new one

  • kwargs – Additional keyword arguments for redis.client.Redis

class requests_cache.backends.redis.RedisDict(namespace, collection_name='http_cache', connection=None, **kwargs)[source]

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface for redis key-value store.

Notes

  • In order to deal with how redis stores data/keys, all keys and data are pickled.

  • The actual key name on the redis server will be namespace:collection_name.

Parameters
  • namespace – Redis namespace

  • collection_name – Name of the Redis hash map

  • connection – (optional) Redis connection instance to use instead of creating a new one

  • kwargs – Additional keyword arguments for redis.client.Redis

bulk_delete(keys)[source]

Delete multiple keys from the cache. Does not raise errors for missing keys.

clear() None.  Remove all items from D.[source]

requests_cache.backends.sqlite module

class requests_cache.backends.sqlite.DbCache(db_path='http_cache', use_temp=False, fast_save=False, **kwargs)[source]

Bases: requests_cache.backends.base.BaseCache

SQLite cache backend.

Parameters
  • db_path (Union[Path, str]) – Database file path (expands user paths and creates parent dirs)

  • use_temp (bool) – Store database in a temp directory (e.g., /tmp/http_cache.sqlite). Note: if db_path is an absolute path, this option will be ignored.

  • fast_save (bool) – Speedup cache saving up to 50 times but with possibility of data loss. See DbDict for more info

  • kwargs – Additional keyword arguments for sqlite3.connect()

bulk_delete(keys)[source]

Remove multiple responses and their associated redirects from the cache, with additional cleanup

clear()[source]

Clear the cache. If this fails due to a corrupted cache or other I/O error, this will attempt to delete the cache file and re-initialize.

class requests_cache.backends.sqlite.DbDict(db_path, table_name='http_cache', fast_save=False, use_temp=False, **kwargs)[source]

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface for SQLite.

It’s possible to create multiply DbDict instances, which will be stored as separate tables in one database:

d1 = DbDict('test', 'table1')
d2 = DbDict('test', 'table2')
d3 = DbDict('test', 'table3')

All data will be stored in separate tables in the file test.sqlite.

Parameters
  • db_path – Database file path

  • table_name – Table name

  • fast_save – Use ‘PRAGMA synchronous = 0;’ to speed up cache saving, but with the potential for data loss

  • timeout – Timeout for acquiring a database lock

bulk_commit()[source]

Context manager used to speed up insertion of a large number of records

Example

>>> d1 = DbDict('test')
>>> with d1.bulk_commit():
...     for i in range(1000):
...         d1[i] = i * 2
bulk_delete(keys=None, values=None)[source]

Delete multiple keys from the cache, without raising errors for any missing keys. Also supports deleting by value.

clear() None.  Remove all items from D.[source]
close()[source]

Close any active connections

connection(commit=False)[source]

Get a thread-local database connection

Return type

Iterator[Connection]

init_db()[source]

Initialize the database, if it hasn’t already been. This must be done in shared connection, but all subsequent queries can use thread-local connections.

vacuum()[source]
class requests_cache.backends.sqlite.DbPickleDict(db_path, table_name='http_cache', fast_save=False, use_temp=False, **kwargs)[source]

Bases: requests_cache.backends.sqlite.DbDict

Same as DbDict, but serializes values before saving

requests_cache.backends.sqlite.chunkify(iterable, max_size=999)[source]

Split an iterable into chunks of a max size

Return type

Iterator[List]

requests_cache.backends.sqlite.sqlite_template(timeout=5.0, detect_types=0, isolation_level=None, check_same_thread=True, factory=None, cached_statements=100, uri=False)[source]

Template function to get an accurate signature for the builtin sqlite3.connect()