3 d8 @sdZddlmZdZdZdgZddlZddlZddlm Z ddl m Z dd l m Z y ddlZWnek rvdZYnXdd lmZd ZGd dde Zddl Z ee j_dS) a cloghandler.py: A smart replacement for the standard RotatingFileHandler ConcurrentRotatingFileHandler: This class is a log handler which is a drop-in replacement for the python standard log handler 'RotateFileHandler', the primary difference being that this handler will continue to write to the same file if the file cannot be rotated for some reason, whereas the RotatingFileHandler will strictly adhere to the maximum file size. Unfortunately, if you are using the RotatingFileHandler on Windows, you will find that once an attempted rotation fails, all subsequent log messages are dropped. The other major advantage of this module is that multiple processes can safely write to a single log file. To put it another way: This module's top priority is preserving your log records, whereas the standard library attempts to limit disk usage, which can potentially drop log messages. If you are trying to determine which module to use, there are number of considerations: What is most important: strict disk space usage or preservation of log messages? What OSes are you supporting? Can you afford to have processes blocked by file locks? Concurrent access is handled by using file locks, which should ensure that log messages are not dropped or clobbered. This means that a file lock is acquired and released for every log message that is written to disk. (On Windows, you may also run into a temporary situation where the log file must be opened and closed for each log message.) This can have potentially performance implications. In my testing, performance was more than adequate, but if you need a high-volume or low-latency solution, I suggest you look elsewhere. See the README file for an example usage of this module. )rangez6$Id: cloghandler.py 6175 2009-11-02 18:40:35Z lowell $zLowell AllemanConcurrentRotatingFileHandlerN)randint)Handler)BaseRotatingHandler) SoftFileLockFc@sjeZdZdZdddZd d Zd d Zd dZddZddZ ddZ ddZ ddZ ddZ ddZdS)ra  Handler for logging to a set of files, which switches from one file to the next when the current file reaches a certain size. Multiple processes can write to the log file concurrently, but this may mean that the file will exceed the given size. arNTFc Cstjj|sHts"tjj|d r0tjj|}n|sHddlm}|dytj ||||Wn(t k rtj |||||_ YnXd|_ ||_ ||_d||_t|j|_|r|j|_dS)a Open the specified file and use it as the stream for logging. By default, the file grows indefinitely. You can specify particular values of maxBytes and backupCount to allow the file to rollover at a predetermined size. Rollover occurs whenever the current log file is nearly maxBytes in length. If backupCount is >= 1, the system will successively create new files with the same pathname as the base file, but with extensions ".1", ".2" etc. appended to it. For example, with a backupCount of 5 and a base file name of "app.log", you would get "app.log", "app.log.1", "app.log.2", ... through to "app.log.5". The file being written to is always "app.log" - when it gets filled up, it is closed and renamed to "app.log.1", and if files "app.log.1", "app.log.2" etc. exist, then they are renamed to "app.log.2", "app.log.3" etc. respectively. If maxBytes is zero, rollover never occurs. On Windows, it is not possible to rename a file that is currently opened by another process. This means that it is not possible to rotate the log files if multiple processes is using the same log file. In this case, the current log file will continue to grow until the rotation can be completed successfully. In order for rotation to be possible, all of the other processes need to close the file first. A mechanism, called "degraded" mode, has been created for this scenario. In degraded mode, the log file is closed after each log message is written. So once all processes have entered degraded mode, the next rotate log attempt should be successful and then normal logging can be resumed. This log handler assumes that all concurrent processes logging to a single file will are using only this class, and that the exact same parameters are provided to each instance of this class. If, for example, two different processes are using this class, but with different values for 'maxBytes' or 'backupCount', then odd behavior is expected. The same is true if this class is used by one application, but the RotatingFileHandler is used by another. NOTE: You should always provide 'filename' as an absolute path, since this class will need to re-open the file during rotation. If your application call os.chdir() then subsequent log files could be created in the wrong directory. r)warnzThe given 'filename' should be an absolute path. If your application calls os.chdir(), your logs may get messed up. Use 'supress_abs_warn=True' to hide this message.Fz%s.lockN)ospathisabsFORCE_ABSOLUTE_PATHsplitabspathwarningsr r__init__ TypeErrorencoding _rotateFailedmaxBytes backupCount lock_filer stream_lock_degrade_debug_degrade) selffilenamemoderrrdebugZsupress_abs_warnr r=/tmp/pip-build-7vycvbft/nipype/nipype/external/cloghandler.pyrKs&B     z&ConcurrentRotatingFileHandler.__init__cCs.|jrtj|j||j|_nt|j||_dS)N)rcodecsopen baseFilenamestream)rrrrr _openFilesz'ConcurrentRotatingFileHandler._openFilecCs,tj||jj|jjr(|j|jdS)z`Acquire thread and file locks. Also re-opening log file when running in 'degraded' mode.N)racquirerr$closedr%r)rrrr r&s  z%ConcurrentRotatingFileHandler.acquirecCstzPy&|jjs&|jj|jr&|jjWn$tk rL|jrH|jjYnXWdz|jjWdtj|XXdS)zgRelease file and thread locks. Flush stream and take care of closing stream in 'degraded' mode.N) r$r'flushrcloseIOErrorrreleaser)rrrr r+s z%ConcurrentRotatingFileHandler.releasecCs*|jjs|jj|jjtj|dS)z$ Closes the stream. N)r$r'r(r)r)rrrr r)s  z#ConcurrentRotatingFileHandler.closecCsdS)aflush(): Do nothing. Since a flush is issued in release(), we don't do it here. To do a flush here, it would be necessary to re-lock everything, and it is just easier and cleaner to do it all in release(), rather than requiring two lock ops per handle() call. Doing a flush() here would also introduces a window of opportunity for another process to write to the log file in between calling stream.write() and stream.flush(), which seems like a bad thing.Nr)rrrr r(s z#ConcurrentRotatingFileHandler.flushcGs||_~~dS)z%Set degrade mode or not. Ignore msg.N)r)rdegrademsgargsrrr rsz&ConcurrentRotatingFileHandler._degradecGsZ|r.|jsVtjjdtj||fd|_n(|jrVtjjdtj||fd|_dS)zsA more colorful version of _degade(). (This is enabled by passing "debug=True" at initialization). z'Degrade mode - ENTERING - (pid=%d) %s Tz(Degrade mode - EXITING - (pid=%d) %s FN)rsysstderrwriter getpid)rr,r-r.rrr rsz,ConcurrentRotatingFileHandler._degrade_debugcCsb|jdkr"|jj|jddS|jjz d}x*| sHtjj|r^d|jtddf}q6Wytj |j|Wn2t t fk rt j d}|jdd|dSXxjt|jddd D]T}d |j|f}d |j|df}tjj|rtjj|rtj|tj ||qW|jd }tjj|r4tj|tj |||jd d Wd|j|jXdS)z< Do a rollover, as described in __init__(). rwNz%s.rotate.%08diTz*rename failed. File in use? exception=%sz%s.%dz.1FzRotation completed)rr$r)r%r r existsr#rrenamer*OSErrorr/exc_inforrremover)rZtmpname exc_valueisfndfnrrr doRollovers:            z(ConcurrentRotatingFileHandler.doRollovercCs,~|jr(|jj|j|j|jSdS)a  Determine if rollover should occur. For those that are keeping track. This differs from the standard library's RotatingLogHandler class. Because there is no promise to keep the file size under maxBytes we ignore the length of the current record. F)_shouldRolloverr$r)r%r)rrecordrrr shouldRollover*s   z,ConcurrentRotatingFileHandler.shouldRolloverc CsV|jdkrRy|jjddWntk r0dSX|jj|jkrFdS|jdddS)NrTFz(Rotation done or not needed at this time)rr$seekr*tellr)rrrr r@<s  z-ConcurrentRotatingFileHandler._shouldRollover)rrrNTF)__name__ __module__ __qualname____doc__rr%r&r+r)r(rrr?rBr@rrrr rCs" V   2)rIbuiltinsr __version__ __author____all__r r/randomrloggingrlogging.handlersrr! ImportErrorZfilelockrr rhandlersrrrr )s(