Improve lexer performance using struct-array

[msujew]

Performs some minor and a major performance optimizations:

1. (major) The TokenSlice uses the Token struct directly instead of using a pointer. This improves lexing performance by ~40%
2. (minor) Adds a running tokens-per-byte average to the lexer. This ensures (1) that we rarely try to increase the slice capacity and (2) keep the capacity as low as possible. Improves performance by ~3%.
3. (minor) changes the lookup from slices.IndexFunc to a custom loop. Does not influence performance - it seems like the Go compiler was able to inline the function call. However, this still makes everything easier to read.
4. (minor) Do not put the utf8 conversion behind a conditional. Improves performance by 3%.