[Twisted-Python] Where to start: log reader/analysis

[Yoann Aubineau]

Hi Andrew,

I wrote a class that follows a file (eg. log file) and provides an iterator
to walk through it. Don't know if it may be of any use for you (or others).

class FileFollower(object):
    """Iterate through a file while it is updated.

    >>> file = FileFollower("/tmp/testfile")
    >>> file.interval = 5
    >>> for line in file:
    ...     print line
    """

    interval = 1

    def __init__(self, filename, interval=None):
        self.filename = filename
        self.interval = interval or self.interval
        self.stat = None
        self.offset = 0
        self.lines = []
        self.running = True

    #
    # File following

    def follow(self):
        while self.running:
            if self.hasChanged():
                data = self.readChange()
                if data:
                    self.dataReceived(data)
                    break
            time.sleep(self.interval)

    def hasChanged(self):
        stat = os.stat(self.filename)
        if stat != self.stat:
            self.stat = stat
            return True
        return False

    def readChange(self):
        file = open(self.filename)
        file.seek(self.offset)
        data = file.read()
        self.offset = file.tell()
        file.close()
        return data

    #
    # Data buffering

    def dataReceived(self, data):
        lines = data.split(os.linesep)
        lines = lines[:-1]
        for line in lines:
            self.lineReceived(line)

    def lineReceived(self, line):
        self.lines.append(line)

    #
    # Iterator implementation

    def __iter__(self):
        return self

    def next(self):
        if not self.lines:
            self.follow()
        line = self.lines.pop(0)
        return line


2007/8/5, Andrew E <andrew at ellerton.net>:
>
> Dear Twisted Experts (... meant in a nice way :) )
>
> I'm not sure where to start.
>
> I need to write a small server that:
>
> - reads lines in a log file as they are appended
> - reads input from a socket as it becomes available
> - does an analysis of both (like, what time was input received in the
> log, and the output received via the socket)
> - outputs a summary report
>
> Socket I/O is easy - but I'm not sure how to include file reading ...
> its bound to be easy.
>
> Any tips?
>
> Thanks :)
>
> Andrew
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20070806/05616755/attachment.html>