requests_cache.backends package

Classes and functions for cache persistence

requests_cache.backends.get_valid_kwargs(func, kwargs, extras=None)

Get the subset of non-None kwargs that are valid params for func

Return type

Dict

requests_cache.backends.init_backend(backend=None, *args, **kwargs)

Initialize a backend from a name, class, or instance

Return type

BaseCache

Submodules

requests_cache.backends.base module

class requests_cache.backends.base.BaseCache(*args, include_get_headers=False, ignored_parameters=None, **kwargs)

Bases: object

Base class for cache implementations, which can also be used as in-memory cache.

See Custom Backends for details on creating your own implementation.

bulk_delete(keys)

Remove multiple responses and their associated redirects from the cache

clear()

Delete all items from the cache

create_key(request, **kwargs)

Create a normalized cache key from a request object

Return type

str

delete(key)

Delete a response or redirect from the cache, as well any associated redirect history

delete_url(url)

Delete a cached response + redirects for GET <url>

delete_urls(urls)

Delete cached responses + redirects for multiple request URLs (GET requests only)

get_response(key, default=None)

Retrieves response for key if it’s stored in cache, otherwise returns default

Parameters

• key (str) – Key of resource

• default – Value to return if key is not in cache

Return type

CachedResponse

has_key(key)

Returns True if cache has key, False otherwise

Return type

bool

has_url(url)

Returns True if cache has url, False otherwise. Works only for GET request urls

Return type

bool

keys(check_expiry=False)

Get all cache keys for redirects and valid responses combined

Return type

Iterator[str]

remove_expired_responses(expire_after=None)

Remove expired and invalid responses from the cache, optionally with revalidation

Parameters

expire_after (Union[None, int, float, str, datetime, timedelta]) – A new expiration time used to revalidate the cache

remove_old_entries(*args, **kwargs)

response_count(check_expiry=False)

Get the number of responses in the cache, excluding invalid (unusable) responses. Can also optionally exclude expired responses.

Return type

int

save_redirect(request, response_key)

Map a redirect request to a response. This makes it possible to associate many keys with a single response.

Parameters

• request (Union[PreparedRequest, CachedRequest]) – Request object for redirect URL

• response_key (str) – Cache key which can be found in responses

save_response(response, cache_key=None, expires=None)

Save response to cache

Parameters

• cache_key (Optional[str]) – Cache key for this response; will otherwise be generated based on request

• response (Union[Response, CachedResponse]) – response to save

• expire_after – Time in seconds until this cache item should expire

property urls: Iterator[str]

Get all URLs currently in the cache (excluding redirects)

Return type

Iterator[str]

values(check_expiry=False)

Get all valid response objects from the cache

Return type

Iterator[CachedResponse]

class requests_cache.backends.base.BaseStorage(serializer=None, **kwargs)

Bases: collections.abc.MutableMapping, abc.ABC

Base class for backend storage implementations

Parameters

• secret_key – Optional secret key used to sign cache items for added security

• salt – Optional salt used to sign cache items

• serializer – Custom serializer that provides loads and dumps methods

bulk_delete(keys)

Delete multiple keys from the cache. Does not raise errors for missing keys. This is a basic version that subclasses should override with a more efficient backend-specific version, if possible.

class requests_cache.backends.base.DictStorage(dict=None, /, **kwargs)

Bases: collections.UserDict, requests_cache.backends.base.BaseStorage

A basic dict wrapper class for non-persistent storage

requests_cache.backends.dynamodb module

class requests_cache.backends.dynamodb.DynamoDbCache(table_name='http_cache', connection=None, **kwargs)

Bases: requests_cache.backends.base.BaseCache

DynamoDB cache backend

Parameters

• table_name (str) – DynamoDb table name

• namespace – Name of DynamoDb hash map

• connection (Optional[ServiceResource]) – DynamoDb Resource object to use instead of creating a new one

• kwargs – Additional keyword arguments for resource()

class requests_cache.backends.dynamodb.DynamoDbDict(table_name, namespace='http_cache', connection=None, read_capacity_units=1, write_capacity_units=1, **kwargs)

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface for DynamoDB key-value store

Note: The actual key name on the dynamodb server will be namespace:table_name

In order to deal with how dynamodb stores data/keys, everything, i.e. keys and data, must be pickled.

Parameters

• table_name – DynamoDb table name

• namespace – Name of DynamoDb hash map

• connection – DynamoDb Resource object to use instead of creating a new one

• kwargs – Additional keyword arguments for resource()

bulk_delete(keys)

Delete multiple keys from the cache. Does not raise errors for missing keys.

clear() → None.  Remove all items from D.

composite_key(key)

Return type

Dict[str, str]

requests_cache.backends.filesystem module

class requests_cache.backends.filesystem.FileCache(cache_name='http_cache', use_temp=False, **kwargs)

Bases: requests_cache.backends.base.BaseCache

Backend that stores cached responses as files on the local filesystem. Response paths will be in the format <cache_name>/responses/<cache_key>. Redirects are stored in a SQLite database, located at <cache_name>/redirects.sqlite.

Parameters

• cache_name (Union[Path, str]) – Base directory for cache files

• use_temp (bool) – Store cache files in a temp directory (e.g., /tmp/http_cache/). Note: if cache_name is an absolute path, this option will be ignored.

• extension – Extension for cache files. If not specified, the serializer default extension will be used.

clear()

Clear the cache

class requests_cache.backends.filesystem.FileDict(cache_name, use_temp=False, extension=None, **kwargs)

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface to files on the local filesystem

clear() → None.  Remove all items from D.

keys() → a set-like object providing a view on D’s keys

paths()

Get absolute file paths to all cached responses

Return type

List[str]

requests_cache.backends.gridfs module

class requests_cache.backends.gridfs.GridFSCache(db_name, **kwargs)

Bases: requests_cache.backends.base.BaseCache

GridFS cache backend. Use this backend to store documents greater than 16MB.

Example

>>> requests_cache.install_cache(backend='gridfs')
>>>
>>> # Or, to provide custom connection settings:
>>> from pymongo import MongoClient
>>> requests_cache.install_cache(backend='gridfs', connection=MongoClient('alternate-host'))

Parameters

• db_name (str) – Database name

• connection – pymongo.MongoClient object to reuse instead of creating a new one

• kwargs – Additional keyword arguments for pymongo.MongoClient

class requests_cache.backends.gridfs.GridFSPickleDict(db_name, collection_name=None, connection=None, **kwargs)

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface for a GridFS database

Parameters

• db_name – Database name

• collection_name – Ignored; GridFS internally uses collections ‘fs.files’ and ‘fs.chunks’

• connection – pymongo.MongoClient object to reuse instead of creating a new one

• kwargs – Additional keyword arguments for pymongo.MongoClient

clear() → None.  Remove all items from D.

requests_cache.backends.mongo module

class requests_cache.backends.mongo.MongoCache(db_name='http_cache', connection=None, **kwargs)

Bases: requests_cache.backends.base.BaseCache

MongoDB cache backend

Parameters

• db_name (str) – Database name

• connection (Optional[MongoClient]) – pymongo.MongoClient object to reuse instead of creating a new one

• kwargs – Additional keyword arguments for pymongo.MongoClient

class requests_cache.backends.mongo.MongoDict(db_name, collection_name='http_cache', connection=None, **kwargs)

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface for a MongoDB collection

Parameters

• db_name – Database name

• collection_name – Collection name

• connection – pymongo.MongoClient object to reuse instead of creating a new one

• kwargs – Additional keyword arguments for pymongo.MongoClient

bulk_delete(keys)

Delete multiple keys from the cache. Does not raise errors for missing keys.

clear() → None.  Remove all items from D.

class requests_cache.backends.mongo.MongoPickleDict(db_name, collection_name='http_cache', connection=None, **kwargs)

Bases: requests_cache.backends.mongo.MongoDict

Same as MongoDict, but pickles values before saving

requests_cache.backends.redis module

class requests_cache.backends.redis.RedisCache(namespace='http_cache', connection=None, **kwargs)

Bases: requests_cache.backends.base.BaseCache

Redis cache backend.

Parameters

• namespace – redis namespace (default: 'requests-cache')

• connection (Optional[Redis]) – Redis connection instance to use instead of creating a new one

• kwargs – Additional keyword arguments for redis.client.Redis

class requests_cache.backends.redis.RedisDict(namespace, collection_name='http_cache', connection=None, **kwargs)

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface for redis key-value store.

Notes

• In order to deal with how redis stores data/keys, all keys and data are pickled.

• The actual key name on the redis server will be namespace:collection_name.

Parameters

• namespace – Redis namespace

• collection_name – Name of the Redis hash map

• connection – (optional) Redis connection instance to use instead of creating a new one

• kwargs – Additional keyword arguments for redis.client.Redis

bulk_delete(keys)

Delete multiple keys from the cache. Does not raise errors for missing keys.

clear() → None.  Remove all items from D.

requests_cache.backends.sqlite module

class requests_cache.backends.sqlite.DbCache(db_path='http_cache', use_temp=False, fast_save=False, **kwargs)

Bases: requests_cache.backends.base.BaseCache

SQLite cache backend.

Parameters

• db_path (Union[Path, str]) – Database file path (expands user paths and creates parent dirs)

• use_temp (bool) – Store database in a temp directory (e.g., /tmp/http_cache.sqlite). Note: if db_path is an absolute path, this option will be ignored.

• fast_save (bool) – Speedup cache saving up to 50 times but with possibility of data loss. See DbDict for more info

• kwargs – Additional keyword arguments for sqlite3.connect()

bulk_delete(keys)

Remove multiple responses and their associated redirects from the cache, with additional cleanup

clear()

Clear the cache. If this fails due to a corrupted cache or other I/O error, this will attempt to delete the cache file and re-initialize.

class requests_cache.backends.sqlite.DbDict(db_path, table_name='http_cache', fast_save=False, use_temp=False, **kwargs)

Bases: requests_cache.backends.base.BaseStorage

A dictionary-like interface for SQLite.

It’s possible to create multiply DbDict instances, which will be stored as separate tables in one database:

d1 = DbDict('test', 'table1')
d2 = DbDict('test', 'table2')
d3 = DbDict('test', 'table3')

All data will be stored in separate tables in the file test.sqlite.

Parameters

• db_path – Database file path

• table_name – Table name

• fast_save – Use ‘PRAGMA synchronous = 0;’ to speed up cache saving, but with the potential for data loss

• timeout – Timeout for acquiring a database lock

bulk_commit()

Context manager used to speed up insertion of a large number of records

Example

>>> d1 = DbDict('test')
>>> with d1.bulk_commit():
...     for i in range(1000):
...         d1[i] = i * 2

bulk_delete(keys=None, values=None)

Delete multiple keys from the cache, without raising errors for any missing keys. Also supports deleting by value.

clear() → None.  Remove all items from D.

close()

Close any active connections

connection(commit=False)

Get a thread-local database connection

Return type

Iterator[Connection]

init_db()

Initialize the database, if it hasn’t already been. This must be done in shared connection, but all subsequent queries can use thread-local connections.

vacuum()

class requests_cache.backends.sqlite.DbPickleDict(db_path, table_name='http_cache', fast_save=False, use_temp=False, **kwargs)

Bases: requests_cache.backends.sqlite.DbDict

Same as DbDict, but serializes values before saving

requests_cache.backends.sqlite.chunkify(iterable, max_size=999)

Split an iterable into chunks of a max size

Return type

Iterator[List]

requests_cache.backends.sqlite.sqlite_template(timeout=5.0, detect_types=0, isolation_level=None, check_same_thread=True, factory=None, cached_statements=100, uri=False)

Template function to get an accurate signature for the builtin sqlite3.connect()