Re: [HACKERS] Lex and things...

Ansley, Michael wrote:
> 
> Hi,
> 
> Shot, Leon.  The patch removes the #define YY_USES_REJECT from scan.c, which
> means we now have expandable tokens.  Of course, it also removes the
> scanning of "embedded minuses", which apparently causes the optimizer to
> unoptimize a little. 

Oh, no. Unary minus gets to grammar parser and there is recognized as
such. Then for numeric constants it becomes an *embedded* minus in
function doNegate. So unary minus after parser in numeric constants
is embedded minus, as it was earlier before patch. In other words,
I can see no change in representation of grammar after patching.

> However, the next step is attacking the limit on the
> size of string literals.  These seemed to be wired to YY_BUF_SIZE, or
> something.  Is there any reason for this?

Hmm. There is something going on to remove fixed length limits 
entirely, maybe someone is already doing something to lexer in
that respect? If no, I could look at what can be done there.

-- 
Leon.