requests_cache.backends package¶
Classes and functions for cache persistence
- requests_cache.backends.get_valid_kwargs(func, kwargs, extras=None)[source]¶
Get the subset of non-None
kwargs
that are valid params forfunc
- Return type
- requests_cache.backends.init_backend(backend=None, *args, **kwargs)[source]¶
Initialize a backend from a name, class, or instance
- Return type
Submodules¶
requests_cache.backends.base module¶
- class requests_cache.backends.base.BaseCache(*args, include_get_headers=False, ignored_parameters=None, **kwargs)[source]¶
Bases:
object
Base class for cache implementations, which can also be used as in-memory cache.
See Custom Backends for details on creating your own implementation.
- create_key(request, **kwargs)[source]¶
Create a normalized cache key from a request object
- Return type
- delete(key)[source]¶
Delete a response or redirect from the cache, as well any associated redirect history
- delete_urls(urls)[source]¶
Delete cached responses + redirects for multiple request URLs (
GET
requests only)
- get_response(key, default=None)[source]¶
Retrieves response for key if it’s stored in cache, otherwise returns default
- Parameters
key (
str
) – Key of resourcedefault – Value to return if key is not in cache
- Return type
- has_url(url)[source]¶
Returns True if cache has url, False otherwise. Works only for GET request urls
- Return type
- remove_expired_responses(expire_after=None)[source]¶
Remove expired and invalid responses from the cache, optionally with revalidation
- response_count(check_expiry=False)[source]¶
Get the number of responses in the cache, excluding invalid (unusable) responses. Can also optionally exclude expired responses.
- Return type
- save_redirect(request, response_key)[source]¶
Map a redirect request to a response. This makes it possible to associate many keys with a single response.
- Parameters
request (
Union
[PreparedRequest
,CachedRequest
]) – Request object for redirect URLresponse_key (
str
) – Cache key which can be found inresponses
- save_response(response, cache_key=None, expires=None)[source]¶
Save response to cache
- Parameters
cache_key (
Optional
[str
]) – Cache key for this response; will otherwise be generated based on requestresponse (
Union
[Response
,CachedResponse
]) – response to saveexpire_after – Time in seconds until this cache item should expire
- property urls: Iterator[str]¶
Get all URLs currently in the cache (excluding redirects)
- class requests_cache.backends.base.BaseStorage(serializer=None, **kwargs)[source]¶
Bases:
collections.abc.MutableMapping
,abc.ABC
Base class for backend storage implementations
- Parameters
secret_key – Optional secret key used to sign cache items for added security
salt – Optional salt used to sign cache items
serializer – Custom serializer that provides
loads
anddumps
methods
- class requests_cache.backends.base.DictStorage(dict=None, /, **kwargs)[source]¶
Bases:
collections.UserDict
,requests_cache.backends.base.BaseStorage
A basic dict wrapper class for non-persistent storage
requests_cache.backends.dynamodb module¶
- class requests_cache.backends.dynamodb.DynamoDbCache(table_name='http_cache', connection=None, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseCache
DynamoDB cache backend
- Parameters
table_name (
str
) – DynamoDb table namenamespace – Name of DynamoDb hash map
connection (
Optional
[ServiceResource
]) – DynamoDb Resource object to use instead of creating a new onekwargs – Additional keyword arguments for
resource()
- class requests_cache.backends.dynamodb.DynamoDbDict(table_name, namespace='http_cache', connection=None, read_capacity_units=1, write_capacity_units=1, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseStorage
A dictionary-like interface for DynamoDB key-value store
Note: The actual key name on the dynamodb server will be
namespace
:table_name
In order to deal with how dynamodb stores data/keys, everything, i.e. keys and data, must be pickled.
- Parameters
table_name – DynamoDb table name
namespace – Name of DynamoDb hash map
connection – DynamoDb Resource object to use instead of creating a new one
kwargs – Additional keyword arguments for
resource()
requests_cache.backends.filesystem module¶
- class requests_cache.backends.filesystem.FileCache(cache_name='http_cache', use_temp=False, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseCache
Backend that stores cached responses as files on the local filesystem. Response paths will be in the format
<cache_name>/responses/<cache_key>
. Redirects are stored in a SQLite database, located at<cache_name>/redirects.sqlite
.- Parameters
cache_name (
Union
[Path
,str
]) – Base directory for cache filesuse_temp (
bool
) – Store cache files in a temp directory (e.g.,/tmp/http_cache/
). Note: ifcache_name
is an absolute path, this option will be ignored.extension – Extension for cache files. If not specified, the serializer default extension will be used.
- class requests_cache.backends.filesystem.FileDict(cache_name, use_temp=False, extension=None, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseStorage
A dictionary-like interface to files on the local filesystem
requests_cache.backends.gridfs module¶
- class requests_cache.backends.gridfs.GridFSCache(db_name, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseCache
GridFS cache backend. Use this backend to store documents greater than 16MB.
Example
>>> requests_cache.install_cache(backend='gridfs') >>> >>> # Or, to provide custom connection settings: >>> from pymongo import MongoClient >>> requests_cache.install_cache(backend='gridfs', connection=MongoClient('alternate-host'))
- Parameters
db_name (
str
) – Database nameconnection –
pymongo.MongoClient
object to reuse instead of creating a new onekwargs – Additional keyword arguments for
pymongo.MongoClient
- class requests_cache.backends.gridfs.GridFSPickleDict(db_name, collection_name=None, connection=None, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseStorage
A dictionary-like interface for a GridFS database
- Parameters
db_name – Database name
collection_name – Ignored; GridFS internally uses collections ‘fs.files’ and ‘fs.chunks’
connection –
pymongo.MongoClient
object to reuse instead of creating a new onekwargs – Additional keyword arguments for
pymongo.MongoClient
requests_cache.backends.mongo module¶
- class requests_cache.backends.mongo.MongoCache(db_name='http_cache', connection=None, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseCache
MongoDB cache backend
- Parameters
db_name (
str
) – Database nameconnection (
Optional
[MongoClient
]) –pymongo.MongoClient
object to reuse instead of creating a new onekwargs – Additional keyword arguments for
pymongo.MongoClient
- class requests_cache.backends.mongo.MongoDict(db_name, collection_name='http_cache', connection=None, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseStorage
A dictionary-like interface for a MongoDB collection
- Parameters
db_name – Database name
collection_name – Collection name
connection –
pymongo.MongoClient
object to reuse instead of creating a new onekwargs – Additional keyword arguments for
pymongo.MongoClient
- class requests_cache.backends.mongo.MongoPickleDict(db_name, collection_name='http_cache', connection=None, **kwargs)[source]¶
Bases:
requests_cache.backends.mongo.MongoDict
Same as
MongoDict
, but pickles values before saving
requests_cache.backends.redis module¶
- class requests_cache.backends.redis.RedisCache(namespace='http_cache', connection=None, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseCache
Redis cache backend.
- Parameters
namespace – redis namespace (default:
'requests-cache'
)connection (
Optional
[Redis
]) – Redis connection instance to use instead of creating a new onekwargs – Additional keyword arguments for
redis.client.Redis
- class requests_cache.backends.redis.RedisDict(namespace, collection_name='http_cache', connection=None, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseStorage
A dictionary-like interface for redis key-value store.
Notes
In order to deal with how redis stores data/keys, all keys and data are pickled.
The actual key name on the redis server will be
namespace:collection_name
.
- Parameters
namespace – Redis namespace
collection_name – Name of the Redis hash map
connection – (optional) Redis connection instance to use instead of creating a new one
kwargs – Additional keyword arguments for
redis.client.Redis
requests_cache.backends.sqlite module¶
- class requests_cache.backends.sqlite.DbCache(db_path='http_cache', use_temp=False, fast_save=False, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseCache
SQLite cache backend.
- Parameters
db_path (
Union
[Path
,str
]) – Database file path (expands user paths and creates parent dirs)use_temp (
bool
) – Store database in a temp directory (e.g.,/tmp/http_cache.sqlite
). Note: ifdb_path
is an absolute path, this option will be ignored.fast_save (
bool
) – Speedup cache saving up to 50 times but with possibility of data loss. SeeDbDict
for more infokwargs – Additional keyword arguments for
sqlite3.connect()
- class requests_cache.backends.sqlite.DbDict(db_path, table_name='http_cache', fast_save=False, use_temp=False, **kwargs)[source]¶
Bases:
requests_cache.backends.base.BaseStorage
A dictionary-like interface for SQLite.
It’s possible to create multiply DbDict instances, which will be stored as separate tables in one database:
d1 = DbDict('test', 'table1') d2 = DbDict('test', 'table2') d3 = DbDict('test', 'table3')
All data will be stored in separate tables in the file
test.sqlite
.- Parameters
db_path – Database file path
table_name – Table name
fast_save – Use ‘PRAGMA synchronous = 0;’ to speed up cache saving, but with the potential for data loss
timeout – Timeout for acquiring a database lock
- bulk_commit()[source]¶
Context manager used to speed up insertion of a large number of records
Example
>>> d1 = DbDict('test') >>> with d1.bulk_commit(): ... for i in range(1000): ... d1[i] = i * 2
- bulk_delete(keys=None, values=None)[source]¶
Delete multiple keys from the cache, without raising errors for any missing keys. Also supports deleting by value.
- class requests_cache.backends.sqlite.DbPickleDict(db_path, table_name='http_cache', fast_save=False, use_temp=False, **kwargs)[source]¶
Bases:
requests_cache.backends.sqlite.DbDict
Same as
DbDict
, but serializes values before saving
- requests_cache.backends.sqlite.chunkify(iterable, max_size=999)[source]¶
Split an iterable into chunks of a max size
- requests_cache.backends.sqlite.sqlite_template(timeout=5.0, detect_types=0, isolation_level=None, check_same_thread=True, factory=None, cached_statements=100, uri=False)[source]¶
Template function to get an accurate signature for the builtin
sqlite3.connect()