SQLite is a fast and lightweight SQL database engine that stores data either in memory or in a single file on disk.
Despite its simplicity, SQLite is a powerful tool. For example, it’s the primary storage system for a number of common applications including Firefox, Chrome, and many components of both Android and iOS. It’s well suited for caching, and requires no extra configuration or dependencies, which is why it’s ‘used as the default backend for requests-cache.
SQLite is the default backend, but if you want to pass extra connection options or just want to be
explicit, initialize your session with a
>>> from requests_cache import CachedSession, SQLiteCache >>> session = CachedSession(backend=SQLiteCache())
Or by alias:
>>> session = CachedSession(backend='sqlite')
This backend accepts any keyword arguments for
>>> backend = SQLiteCache('http_cache', timeout=30) >>> session = CachedSession(backend=backend)
See Cache Files for general info on specifying cache paths
If you specify a name without an extension, the default extension
.sqlitewill be used
SQLite also supports in-memory databases.
You can enable this (in “shared” memory mode) with the
>>> session = CachedSession('http_cache', use_memory=True)
Or specify a memory URI with additional options:
>>> session = CachedSession(':file:memdb1?mode=memory')
:memory:, if you are only using the cache from a single thread:
>>> session = CachedSession(':memory:')
When working with average-sized HTTP responses (< 1MB) and using a modern SSD for file storage, you can expect speeds of around:
Of course, this will vary based on hardware specs, response size, and other factors.
SQLite supports concurrent access, so it is safe to use from a multi-threaded and/or multi-process
application. It supports unlimited concurrent reads. Writes, however, are queued and run in serial,
so if you need to make large volumes of concurrent requests, you may want to consider a different
backend that’s specifically made for that kind of workload, like
Hosting Services and Filesystem Compatibility#
There are some caveats to using SQLite with some hosting services, based on what kind of storage is available:
SQLite may be used on a NFS, but is usually only safe to use from a single process at a time. See the SQLite FAQ for details.
PythonAnywhere is one example of a host that uses NFS-backed storage. Using SQLite from a multiprocess application will likely result in
sqlite3.OperationalError: database is locked.
Heroku explicitly disables SQLite on its dynos.
AWS EC2, Lambda (depending on configuration), and some other AWS services use ephemeral storage that only persists for the lifetime of the instance. This is fine for short-term caching. For longer-term persistance, you can use an attached EBS volume.