New Czechoslovak hyphenation patterns, word lists, and workflow
Gespeichert in:
| Titel: | New Czechoslovak hyphenation patterns, word lists, and workflow |
|---|---|
| Autoren: | Sojka Petr, Sojka Ondřej |
| Quelle: | TUGboat: The Communications of the TeX Users Group |
| Verlagsinformationen: | TeX Users Group, 2021. |
| Publikationsjahr: | 2021 |
| Schlagwörter: | dělení slov, generování vzorů, databáze slov, vícejazyčná sazba, slabičné algoritmy, patgen, soutěživé vzory, 05 social sciences, 0202 electrical engineering, electronic engineering, information engineering, 0501 psychology and cognitive sciences, 02 engineering and technology, hyphenation, pattern generation, word list database, multilingual typesetting, syllabification algorithms, competing patterns |
| Beschreibung: | Space- and time-effective segmentation and hyphenation of natural languages stay at the core of every document preparation system, web browser, or mobile rendering system. We use the unreasonable effectiveness of pattern generation with patgen . It is possible to use hyphenation patterns to solve the dictionary problem also for close languages without compromise. In this article, we show how we applied the marvelous effectiveness of patgen for the generation of the new Czechoslovak hyphenation patterns that cover both Czech and Slovak languages. We show that developing universal, up-to-date, high-coverage and high-generalization hyphenation patterns is feasible, generated from semi-automatically prepared word lists from actual language usage. We evaluate the new approach and argue that the new Czechoslovak hyphenation patterns bring significant coverage and generalization improvements, and space savings. We share all the data, word lists, and workflow for reproducibility and usage. |
| Publikationsart: | Article |
| Sprache: | English |
| ISSN: | 0896-3207 |
| DOI: | 10.47397/tb/42-2/tb131sojka-czech |
| Zugangs-URL: | https://tug.org/docs/liang/liang-thesis.pdf https://www.muni.cz/vyzkum/publikace/1788557 |
| Dokumentencode: | edsair.doi.dedup.....0b78990ee26076441c279d08cd9d19a2 |
| Datenbank: | OpenAIRE |
Nájsť tento článok vo Web of Science | Abstract: | Space- and time-effective segmentation and hyphenation of natural languages stay at the core of every document preparation system, web browser, or mobile rendering system. We use the unreasonable effectiveness of pattern generation with patgen . It is possible to use hyphenation patterns to solve the dictionary problem also for close languages without compromise. In this article, we show how we applied the marvelous effectiveness of patgen for the generation of the new Czechoslovak hyphenation patterns that cover both Czech and Slovak languages. We show that developing universal, up-to-date, high-coverage and high-generalization hyphenation patterns is feasible, generated from semi-automatically prepared word lists from actual language usage. We evaluate the new approach and argue that the new Czechoslovak hyphenation patterns bring significant coverage and generalization improvements, and space savings. We share all the data, word lists, and workflow for reproducibility and usage. |
|---|---|
| ISSN: | 08963207 |
| DOI: | 10.47397/tb/42-2/tb131sojka-czech |