twisted.words.xish.xpathparser.Scanner : API documentation

Yapps scanner.

The Yapps scanner can work in context sensitive or context insensitive modes. The token(i) method is used to retrieve the i-th token. It takes a restrict set that limits the set of tokens it is allowed to return. In context sensitive mode, this restrict set guides the scanner. In context insensitive mode, there is no restriction (the set is always the full set of tokens).

Method	__init__	Initialize the scanner.
Method	get_token_pos	Get the current token position in the input text.
Method	get_char_pos	Get the current char position in the input text.
Method	get_prev_char_pos	Get the previous position (one token back) in the input text.
Method	get_line_number	Get the line number of the current position in the input text.
Method	get_column_number	Get the column number of the current position in the input text.
Method	get_input_scanned	Get the portion of the input that has been tokenized.
Method	get_input_unscanned	Get the portion of the input that has not yet been tokenized.
Method	token	Get the i'th token in the input.
Method	__repr__	Print the last 10 tokens that have been scanned in
Method	scan	Should scan another token and add it to the list, self.tokens, and add the restriction to self.restrictions

def __init__(self, patterns, ignore, input): (source)

overridden in twisted.words.xish.xpathparser.XPathParserScanner

Initialize the scanner.

Parameters	patterns	[(terminal, uncompiled regex), ...] or `None`
	ignore	[terminal,...]
	input	string If patterns is `None`, we assume that the subclass has defined `self.patterns` : [(terminal, compiled regex), ...]. Note that the patterns parameter expects uncompiled regexes, whereas the `self.patterns` field expects compiled regexes.

def get_token_pos(self): (source)

Get the current token position in the input text.

def get_char_pos(self): (source)

Get the current char position in the input text.

def get_prev_char_pos(self, i=None): (source)

Get the previous position (one token back) in the input text.

def get_line_number(self): (source)

Get the line number of the current position in the input text.

def get_column_number(self): (source)

Get the column number of the current position in the input text.

def get_input_scanned(self): (source)

Get the portion of the input that has been tokenized.

def get_input_unscanned(self): (source)

Get the portion of the input that has not yet been tokenized.

def token(self, i, restrict=None): (source)

Get the i'th token in the input.

If i is one past the end, then scan for another token.

Parameters	i	token index
	restrict	[token, ...] or `None`; if restrict is `None`, then any token is allowed. You may call token(i) more than once. However, the restrict set may never be larger than what was passed in on the first call to token(i).

t.w.x.x.Scanner : class documentation