[Reality] Re: Python question--parsing
Michael Dartt
mad96@hampshire.edu
Sat, 18 Mar 2000 11:22:47 -0500 (EST)
'S cool--I'm using StringIO to create a buffer, which I can then call
readline() on. I love comp.lang.python...:-) (Esp. since EFnet hates
me....)
--Mike
On Sat, 18 Mar 2000, Michael Dartt wrote:
> I'd really like to be able to use parse.py's Tokenizer to parse a given
> text string. The problem, of course, is that it calls tokenize(), which
> wants a readline()-type argument. Is there any function I can use that
> will allow me to do this?
>
> Right now, this is the code I have for __init__() in parse.py:
>
> def __init__(self, string):
> self.tokens=[]
> self._neg=None
> #self.fn=string #not sure why this is here...
> try: fd=open(string) # if it's a file, this will work
> except IOError: # must just be a string
> import time
> x=time.time()
> ###Problem line---> tokenize(string, self.eat)
> x=time.time()-x
> print 'Tokenization took %d seconds.' % x
> self.reset()
> else: # is a file
> import time
> x=time.time()
> tokenize(fd.readline,self.eat)
> x=time.time()-x
> print 'Tokenization took %d seconds.' % x
> fd.close()
> self.reset()
>
>
> I suppose I could use re for this, but I'd really rather not: this is much
> more convenient for my purposes.
>
> Thanks in advance for your help. :-)
>
>
> --Mike/Jedin
>
>
>