Advanced Requests#

Following are some tips on using requests-cache with some of the more advanced features of the requests library.

Event Hooks#

Requests has an event hook system that can be used to add custom behavior into different parts of the request process. It can be used, for example, for request throttling:

Example
>>> import time
>>> import requests
>>> from requests_cache import CachedSession

>>> def make_throttle_hook(timeout=1.0):
>>>     """Make a request hook function that adds a custom delay for non-cached requests"""
>>>     def hook(response, *args, **kwargs):
>>>         if not getattr(response, 'from_cache', False):
>>>             print('sleeping')
>>>             time.sleep(timeout)
>>>         return response
>>>     return hook

>>> session = CachedSession()
>>> session.hooks['response'].append(make_throttle_hook(0.1))
>>> # The first (real) request will have an added delay
>>> session.get('https://httpbin.org/get')
>>> session.get('https://httpbin.org/get')

Streaming Requests#

If you use streaming requests, you can use the same code to iterate over both cached and non-cached requests. Cached response content will have already been read (i.e., consumed), but will be available for re-reading so it behaves like the original streamed response:

Example
>>> from requests_cache import CachedSession

>>> session = CachedSession()
>>> for i in range(2):
...     response = session.get('https://httpbin.org/stream/20', stream=True)
...     for chunk in response.iter_lines():
...         print(chunk.decode('utf-8'))