API Reference

This section covers all the public interfaces of requests-cache.

Sessions

class requests_cache.session.CachedSession(cache_name='http_cache', backend=None, expire_after=- 1, urls_expire_after=None, allowable_codes=(200), allowable_methods=('GET', 'HEAD'), filter_fn=None, old_data_on_error=False, **kwargs)

Bases: requests_cache.session.CacheMixin, requests.sessions.Session

Class that extends requests.Session with caching features.

See individual backend classes for additional backend-specific arguments. Also see Advanced Usage for more details and examples on how the following arguments affect cache behavior.

Parameters

• cache_name (str) – Cache prefix or namespace, depending on backend

• backend (Union[str, BaseCache, Type[BaseCache], None]) – Cache backend name, class, or instance; name may be one of ['sqlite', 'mongodb', 'gridfs', 'redis', 'dynamodb', 'memory'].

• expire_after (Union[None, int, float, datetime, timedelta]) – Time after which cached items will expire

• urls_expire_after (Optional[Dict[str, Union[None, int, float, datetime, timedelta]]]) – Expiration times to apply for different URL patterns

• allowable_codes (Iterable[int]) – Only cache responses with one of these codes

• allowable_methods (Iterable[str]) – Cache only responses for one of these HTTP methods

• include_get_headers – Make request headers part of the cache key

• ignored_parameters – List of request parameters to be excluded from the cache key

• filter_fn (Optional[Callable]) – function that takes a aiohttp.ClientResponse object and returns a boolean indicating whether or not that response should be cached. Will be applied to both new and previously cached responses.

• old_data_on_error (bool) – Return expired cached responses if new request fails

• secret_key – Optional secret key used to sign cache items for added security

cache_disabled()

Context manager for temporary disabling the cache

Warning

This method is not thread-safe.

Example

>>> s = CachedSession()
>>> with s.cache_disabled():
...     s.get('http://httpbin.org/ip')

remove_expired_responses(expire_after=None)

Remove expired responses from the cache, optionally with revalidation

Parameters

expire_after (Union[None, int, float, datetime, timedelta]) – A new expiration time used to revalidate the cache

request(method, url, params=None, data=None, json=None, expire_after=None, **kwargs)

This method prepares and sends a request while automatically performing any necessary caching operations. This will be called by any other method-specific requests functions (get, post, etc.). This does not include prepared requests, which will still be cached via send().

See requests.Session.request() for parameters. Additional parameters:

Parameters

expire_after (Union[None, int, float, datetime, timedelta]) – Expiration time to set only for this request; see details below. Overrides CachedSession.expire_after. Accepts all the same values as CachedSession.expire_after except for None; use -1 to disable expiration on a per-request basis.

Return type

Union[Response, CachedResponse]

Returns

Either a new or cached response

Order of operations: A request will pass through the following methods:

1. requests.get()/requests.Session.get() or other method-specific functions (optional)

2. CachedSession.request()

3. requests.Session.request()

4. CachedSession.send()

5. BaseCache.get_response()

6. requests.Session.send() (if not cached)

send(request, **kwargs)

Send a prepared request, with caching.

Return type

Union[Response, CachedResponse]

class requests_cache.session.CacheMixin(cache_name='http_cache', backend=None, expire_after=- 1, urls_expire_after=None, allowable_codes=(200), allowable_methods=('GET', 'HEAD'), filter_fn=None, old_data_on_error=False, **kwargs)

Mixin class that extends requests.Session with caching features. See CachedSession for usage information.

Patching

Utilities for patching requests.

Warning

These functions are not thread-safe. Use CachedSession directly if you want to use caching in a multi-threaded environment.

requests_cache.patcher.clear()

Clear the currently installed cache (if any)

requests_cache.patcher.disabled()

Context manager for temporarily disabling caching for all requests functions

Example

>>> with requests_cache.disabled():
...     requests.get('http://httpbin.org/get')

requests_cache.patcher.enabled(*args, **kwargs)

Context manager for temporarily enabling caching for all requests functions

Accepts the same arguments as install_cache().

Example

>>> with requests_cache.enabled('cache_db'):
...     requests.get('http://httpbin.org/get')

requests_cache.patcher.get_cache()

Get the internal cache object from the currently installed CachedSession (if any)

Return type

Optional[BaseCache]

requests_cache.patcher.install_cache(cache_name='http_cache', backend=None, expire_after=-1, urls_expire_after=None, allowable_codes=(200, ), allowable_methods=('GET', 'HEAD'), filter_fn=None, old_data_on_error=False, session_factory=<class 'requests_cache.session.CachedSession'>, **kwargs)

Install the cache for all requests functions by monkey-patching requests.Session

Example

>>> requests_cache.install_cache('demo_cache')

Accepts all the same parameters as CachedSession. Additional parameters:

Parameters

session_factory (Type[Session]) – Session class to use. It must inherit from either CachedSession or CacheMixin

requests_cache.patcher.is_installed()

Indicate whether or not requests-cache is currently installed

Return type

bool

requests_cache.patcher.remove_expired_responses(expire_after=None)

Remove expired responses from the cache, optionally with revalidation

Parameters

expire_after (Union[None, int, float, datetime, timedelta]) – A new expiration time used to revalidate the cache

requests_cache.patcher.uninstall_cache()

Disable the cache by restoring the original requests.Session

Responses

class requests_cache.response.CachedResponse(original_response, expire_after=None)

Bases: requests.models.Response

A serializable wrapper for requests.Response. CachedResponse objects will behave the same as the original response, but with some additional cache-related details. This class is responsible for converting and setting cache expiration times, and converting response info into a serializable format.

Parameters

• original_response (Response) – Response object

• expire_after (Union[None, int, float, datetime, timedelta]) – Time after which this cached response will expire

property apparent_encoding

The apparent encoding, provided by the chardet library.

close()

Releases the connection back to the pool. Once this method has been called the underlying raw object must not be accessed again.

Note: Should not normally need to be called explicitly.

property content

Content of the response, in bytes.

cookies

A CookieJar of Cookies the server sent back.

elapsed

The amount of time elapsed between sending the request and the arrival of the response (as a timedelta). This property specifically measures the time taken between sending the first byte of the request and finishing parsing the headers. It is therefore unaffected by consuming the response content or the value of the stream keyword argument.

encoding

Encoding to decode with when accessing r.text.

headers

Case-insensitive Dictionary of Response Headers. For example, headers['content-encoding'] will return the value of a 'Content-Encoding' response header.

history

A list of Response objects from the history of the Request. Any redirect responses will end up here. The list is sorted from the oldest to the most recent request.

property is_expired

Determine if this cached response is expired

Return type

bool

property is_permanent_redirect

True if this Response one of the permanent versions of redirect.

property is_redirect

True if this Response is a well-formed HTTP redirect that could have been processed automatically (by Session.resolve_redirects()).

iter_content(chunk_size=1, decode_unicode=False)

Iterates over the response data. When stream=True is set on the request, this avoids reading the content at once into memory for large responses. The chunk size is the number of bytes it should read into memory. This is not necessarily the length of each item returned as decoding can take place.

chunk_size must be of type int or None. A value of None will function differently depending on the value of stream. stream=True will read data as it arrives in whatever size the chunks are received. If stream=False, data is returned as a single chunk.

If decode_unicode is True, content will be decoded using the best available encoding based on the response.

iter_lines(chunk_size=512, decode_unicode=False, delimiter=None)

Iterates over the response data, one line at a time. When stream=True is set on the request, this avoids reading the content at once into memory for large responses.

Note

This method is not reentrant safe.

json(**kwargs)

Returns the json-encoded content of a response, if any.

Parameters

**kwargs – Optional arguments that json.loads takes.

Raises

ValueError – If the response body does not contain valid json.

property links

Returns the parsed header links of the response, if any.

property next

Returns a PreparedRequest for the next request in a redirect chain, if there is one.

property ok

Returns True if status_code is less than 400, False if not.

This attribute checks if the status code of the response is between 400 and 600 to see if there was a client error or a server error. If the status code is between 200 and 400, this will return True. This is not a check to see if the response code is 200 OK.

raise_for_status()

Raises HTTPError, if one occurred.

property raw

Reconstruct a raw urllib response object from stored attrs

Return type

HTTPResponse

reason

Textual reason of responded HTTP Status, e.g. “Not Found” or “OK”.

request

The PreparedRequest object to which this is a response.

reset()

Reset raw response file handler, if previously initialized

revalidate(expire_after)

Set a new expiration for this response, and determine if it is now expired

Return type

bool

status_code

Integer Code of responded HTTP Status, e.g. 404 or 200.

property text

Content of the response, in unicode.

If Response.encoding is None, encoding will be guessed using chardet.

The encoding of the response content is determined based solely on HTTP headers, following RFC 2616 to the letter. If you can take advantage of non-HTTP knowledge to make a better guess at the encoding, you should set r.encoding appropriately before accessing this property.

url

Final URL location of Response.

class requests_cache.response.CachedHTTPResponse(body=None, **kwargs)

Bases: urllib3.response.HTTPResponse

A wrapper for raw urllib response objects, which wraps cached content with support for streaming requests

close()

Flush and close the IO object.

This method has no effect if the file is already closed.

drain_conn()

Read and discard any remaining HTTP response data in the response connection.

Unread data in the HTTPResponse connection blocks the connection from being released back to the pool.

fileno()

Returns underlying file descriptor if one exists.

OSError is raised if the IO object does not use a file descriptor.

flush()

Flush write buffers, if applicable.

This is not implemented for read-only and non-blocking streams.

classmethod from_httplib(r, **response_kw)

Given an http.client.HTTPResponse instance r, return a corresponding urllib3.response.HTTPResponse object.

Remaining parameters are passed to the HTTPResponse constructor, along with original_response=r.

get_redirect_location()

Should we redirect and where to?

Returns

Truthy redirect location string if we got a redirect status code and valid location. None if redirect status and no location. False if not a redirect status code.

geturl()

Returns the URL that was the source of this response. If the request that generated this response redirected, this method will return the final redirect location.

isatty()

Return whether this is an ‘interactive’ stream.

Return False if it can’t be determined.

read(amt=None, decode_content=None, **kwargs)

Simplified reader for cached content that emulates urllib3.response.HTTPResponse.read()

read_chunked(amt=None, decode_content=None)

Similar to HTTPResponse.read(), but with an additional parameter: decode_content.

Parameters

• amt – How much of the content to read. If specified, caching is skipped because it doesn’t make sense to cache partial content as the full response.

• decode_content – If True, will attempt to decode the body based on the ‘content-encoding’ header.

readable()

Return whether object was opened for reading.

If False, read() will raise OSError.

readline(size=- 1, /)

Read and return a line from the stream.

If size is specified, at most size bytes will be read.

The line terminator is always b’n’ for binary files; for text files, the newlines argument to open can be used to select the line terminator(s) recognized.

readlines(hint=- 1, /)

Return a list of lines from the stream.

hint can be specified to control the number of lines read: no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds hint.

release_conn()

No-op for compatibility

seek()

Change stream position.

Change the stream position to the given byte offset. The offset is interpreted relative to the position indicated by whence. Values for whence are:

• 0 – start of stream (the default); offset should be zero or positive

• 1 – current stream position; offset may be negative

• 2 – end of stream; offset is usually negative

Return the new absolute position.

seekable()

Return whether object supports random access.

If False, seek(), tell() and truncate() will raise OSError. This method may need to do a test seek().

stream(amt=None, **kwargs)

Simplified generator over cached content that emulates urllib3.response.HTTPResponse.stream()

supports_chunked_reads()

Checks if the underlying file-like object looks like a http.client.HTTPResponse object. We do this by testing for the fp attribute. If it is present we assume it returns raw chunks as processed by read_chunked().

tell()

Obtain the number of bytes pulled over the wire so far. May differ from the amount of content returned by :meth:urllib3.response.HTTPResponse.read if bytes are encoded on the wire (e.g, compressed).

truncate()

Truncate file to size bytes.

File pointer is left unchanged. Size defaults to the current IO position as reported by tell(). Returns the new size.

writable()

Return whether object was opened for writing.

If False, write() will raise OSError.

writelines(lines, /)

Write a list of lines to stream.

Line separators are not added, so it is usual for each of the lines provided to have a line separator at the end.

Cache Backends

• requests_cache.backends package
  • Submodules
  • requests_cache.backends.base module
  • requests_cache.backends.dynamodb module
  • requests_cache.backends.gridfs module
  • requests_cache.backends.mongo module
  • requests_cache.backends.redis module
  • requests_cache.backends.sqlite module