Getting Started

You would like to get kiot-lexer in your project first:

Kotlin DSL

allprojects {
	repositories {
		maven("https://jitpack.io")
	}
}

dependencies {
	implementation("com.github.KiotLand.kiot-lexer:kiot-lexer:1.0.6.1")
	// kiot-lexer-js and kiot-lexer-jvm are also alternatives.
}

Groovy

Great! Now have a look at this example:

import org.kiot.automaton.CharClass
import org.kiot.automaton.NFABuilder
import org.kiot.lexer.Lexer
import org.kiot.lexer.LexerData

val data = LexerData.buildSimple {
	NFABuilder.from(' ') action 1
	NFABuilder.from(CharClass.digit).oneOrMore() action 2
	NFABuilder.from(CharClass.letter).oneOrMore() action 3
}
// The code above might seems hard for you... But it's a good way to get familiar with kiot-lexer.
// The code below is equivalent to the above one, with RegExp as matching pattern.
/*
	val data = LexerData.buildSimple {
		" " action 1
		"\\d+" action 2
		"\\w+" action 3
	}
*/

// ... and here we come to lexing.

class SimpleLexer(chars: CharSequence): Lexer<Int>(data, chars) {
	override fun onAction(action: Int) {
		returnValue(action)
	}
}

SimpleLexer("i have 2 ideas").lexAll()
// ret = [3, 1, 3, 1, 2, 1, 3]

SimpleLexer("!!").lex()
// Crashes: Mismatch in [0, 0]

In kiot-lexer, lexer datas are created from LexerData.buildSimple or LexerData.build. The difference is that LexerData.buildSimple only have one default state while LexerData.build supports multiple states, which is mentioned here.

Here's the general usage of LexerData.buildSimple:

Lexer.simple {
    [PATTERN] action [ACTION]
    // [ACTION] can be "ignore" to declare that you accepct this pattern but do nothing
    // and more stuff like this...
}