論文 ·日本語 ·書誌確認済
fugashi, a Tool for Tokenizing Japanese in Python
McCann, Paul ・ Paul O'Leary McCann
- 刊行年
- 2020
- 収録
- 『Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)』 pp. 44-51
- 言語
- 英語
- OpenAlex
- W3092814655
- DOI
- 10.18653/v1/2020.nlposs-1.7
- MAG
- 3092814655
- URL
- https://aclanthology.org/2020.nlposs-1.7/
要旨
Recent years have seen an increase in the number of large-scale multilingual NLP projects. However, even in such projects, languages with special processing requirements are often excluded. One such language is Japanese. Japanese is written without spaces, tokenization is non-trivial, and while high quality open source tokenizers exist they can be hard to use and lack English documentation. This paper introduces fugashi, a MeCab wrapper for Python, and gives an introduction to tokenizing Japanese.
主題
この書誌の出所
- refs:sino_japanese— McCann2020Fugashi（2026-08-12取得）
- openalex— W3092814655（2026-08-14取得）
引用
McCann, Paul・Paul O'Leary McCann（2020） fugashi, a Tool for Tokenizing Japanese in Python 『Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)』 pp. 44-51
McCann2020Fugashi