Quality gates
Latest stable Based on Text release 0.2.1
The release quality evidence is a deterministic repository gate. It proves selected stable behaviors; it is not a broad statistical claim about NLP accuracy.
Covered evidence
Section titled “Covered evidence”| Area | Stable evidence |
|---|---|
| Korean mixed-text tokenization | KoreanTextProcessorTest surface-token fixtures |
| Japanese mixed-text tokenization | JapaneseProcessorTest fixtures |
| mixed-language detection | LanguageDetectorExtensionsTest |
| request boundary failures | TokenizeMessageTest, BlockMessageTest, and processor facade tests |
| safety adapter behavior | runnable example tests with sanitized failure assertions |
The fixtures emphasize user-visible token surfaces. They intentionally avoid asserting every internal morphological choice, leaving room for model and dictionary improvements without hiding a user-facing regression.
Reproduce targeted evidence
Section titled “Reproduce targeted evidence”./gradlew :tokenizer-korean:test \ --tests "io.bluetape4k.tokenizer.korean.KoreanTextProcessorTest"
./gradlew :tokenizer-japanese:test \ --tests "io.bluetape4k.tokenizer.japanese.JapaneseProcessorTest"
./gradlew :lingua:test \ --tests "io.bluetape4k.lingua.LanguageDetectorExtensionsTest"The report was validated on macOS with JDK 21 or newer and the checked-in Gradle wrapper; Gradle 9.5.1 was observed in the recorded local output.
What it does not prove
Section titled “What it does not prove”The report does not:
- compare against third-party NLP systems;
- publish precision, recall, or F1 over a large external corpus;
- guarantee performance or memory on another host;
- establish that a default blockword dictionary matches every product policy;
- replace application tests for supported languages and input distribution.
State these gaps when using the evidence in a release decision. A deterministic gate is valuable because it is reproducible, not because it answers every quality question.
Extend the gate
Section titled “Extend the gate”Add a fixture when a user-visible token, route, masking result, or sanitized error becomes part of your product contract. Add scored corpus evaluation only with a versioned corpus, explicit metric, reproducible command, and interpretation threshold.
See testing for application coverage and Aho-Corasick benchmarks for performance evidence.