monty.io module¶
Augments Python’s suite of IO functions with useful transparent support for compressed files.
-
class
FileLock
(file_name, timeout=10, delay=0.05)[source]¶ Bases:
object
A file locking mechanism that has context-manager support so you can use it in a with statement. This should be relatively cross-compatible as it doesn’t rely on msvcrt or fcntl for the locking. Taken from http://www.evanfosmark.com/2009/01/cross-platform-file-locking -support-in-python/
Prepare the file locker. Specify the file to lock and optionally the maximum timeout and the delay between each attempt to lock.
- Parameters
file_name – Name of file to lock.
timeout – Maximum timeout for locking. Defaults to 10.
delay – Delay between each attempt to lock. Defaults to 0.05.
-
Error
¶ alias of
FileLockException
-
reverse_readfile
(filename: Union[str, pathlib.Path]) → Generator[str, str, None][source]¶ A much faster reverse read of file by using Python’s mmap to generate a memory-mapped file. It is slower for very small files than reverse_readline, but at least 2x faster for large files (the primary use of such a method).
- Parameters
filename (str) – Name of file to read.
- Yields
Lines from the file in reverse order.
-
reverse_readline
(m_file, blk_size=4096, max_mem=4000000) → Generator[str, str, None][source]¶ Generator method to read a file line-by-line, but backwards. This allows one to efficiently get data at the end of a file.
Based on code by Peter Astrand <astrand@cendio.se>, using modifications by Raymond Hettinger and Kevin German. http://code.activestate.com/recipes/439045-read-a-text-file-backwards -yet-another-implementat/
Reads file forwards and reverses in memory for files smaller than the max_mem parameter, or for gzip files where reverse seeks are not supported.
Files larger than max_mem are dynamically read backwards.
- Parameters
m_file (File) – File stream to read (backwards)
blk_size (int) – The buffer size. Defaults to 4096.
max_mem (int) – The maximum amount of memory to involve in this operation. This is used to determine when to reverse a file in-memory versus seeking portions of a file. For bz2 files, this sets the maximum block size.
- Returns
Generator that returns lines from the file. Similar behavior to the file.readline() method, except the lines are returned from the back of the file.
-
zopen
(filename: Union[str, pathlib.Path], *args, **kwargs) → IO[source]¶ This function wraps around the bz2, gzip and standard python’s open function to deal intelligently with bzipped, gzipped or standard text files.
- Parameters
filename (str/Path) – filename or pathlib.Path.
*args – Standard args for python open(..). E.g., ‘r’ for read, ‘w’ for write.
**kwargs – Standard kwargs for python open(..).
- Returns
File-like object. Supports with context.