Lexer Building Options

Have a look at the following example:

import org.kiot.lexer.Lexer

val data = LexerData.buildSimple {
	"\\d" action 1
	"." action 2
}

/*
    org.kiot.automaton.MarksConflictException:
        FunctionMark(\d) conflicts with FunctionMark(.) under this pattern: [0..9]
*/

The lexer above can recognize pattern [0..9] as either "digit" or "any", so we got an error. In kiot-lexer, we promote avoiding creating conflicts in your lexer. However, if you want the rules in your lexer to be checked in the order they are defined, you can use:

import org.kiot.lexer.Lexer

val data = LexerData.buildSimple {
	options.strict = false
	"\\d" action 1
	"." action 2
}

class SimpleLexer(chars: CharSequence) : Lexer<Nothing>(data, chars) {
	override fun onAction(action: Int) {
		when (action) {
			1 -> println("a digit")
			2 -> println("a char")
		}
	}
}

SimpleLexer("1").lex()
// output: a digit

SimpleLexer("2").lex()
// output: a char

Besides strict, multiple options can be manipulated.

Name	Type	Default Value	Description
strict	Boolean	true	Whether the lexer is strict or not.
minimize	Boolean	false	Whether to minimize the generated DFA.
compress	Boolean	true	Whether to compress the generated DFA.

Compared with compressed DFA, general DFA requires an extra time complexity of $O(log(|C|))$ to translate a character into its corresponding CharClass, while in compressed DFA, this process is $O(1)$.