Language engineering · Egothor

Radixor

Fast, deterministic multilingual stemming for Java, built around compact patch-command tries and a runtime path designed to stay small, predictable and easy to deploy.

Open repository ↗Egothor lineage

≈31–32Mtokens/scurrent English benchmark workload

≈4×Porterthroughput comparison

≈6×Porter2throughput comparison

BSD-3licenseopen source

Stemming as a compiled transformation problem

Radixor learns how a word form changes into a stem, encodes that change as a compact patch command and stores the transformation in a trie. The expensive structural work happens when a model is built; live lookup uses an immutable compiled representation.

Dictionaryword → stem pairs→ Mutable triecollect mappings→ Reduced trieshare equivalent subtrees→ Compiled trieimmutable runtime form

The model is data-driven rather than hard-coded around language-specific suffix rules. Compiled dictionaries can remain compact, deterministic and directly loadable while still supporting preferred or multiple stemming candidates when the source data is ambiguous.

Engineering characteristics

Deterministic results

Candidate ordering and compiled artifacts are designed for repeatability, making behaviour easier to test, compare and version.

Build-time reduction

Equivalent trie subtrees can be canonicalized before deployment, reducing duplication without adding runtime reduction work.

Immutable runtime structures

Compiled nodes use read-only array-oriented structures intended for direct, allocation-light lookup.

Operationally explicit

Compilation, persistence, extension and runtime loading are separate steps rather than hidden lifecycle behaviour.

Measured performance

The public JMH comparison currently reports roughly 31–32 million tokens per second for the bundled English workload, around 4× the throughput of Snowball original Porter and 6× Snowball English (Porter2) on the same deterministic token stream.

Benchmark scope matters. This is a throughput comparison for the documented workload, not a claim that the stemmers are linguistically equivalent. Environment, dataset and methodology belong with the result; the repository publishes the benchmark material for review.

Heritage without legacy constraints

Radixor continues the conceptual line from the original Egothor stemming work and the later Stempel packaging, but it is a modern implementation rather than a repackaging of historical code. The durable idea is compact learned transformation; the contemporary engineering adds stronger persistence boundaries, deterministic behaviour, testability, benchmarks and explicit quality evidence.

Project resources

Source repositoryCode, releases and README ↗ Technical documentationArchitecture, usage and quality material ↗ Egothor heritageSearch and stemming lineage →