I am Sakae Mizuki (水木 栄), Ph.D. graduate in Computer Science (Natural Language Processing) from Institute of Science Tokyo (formerly Tokyo Institute of Technology), where I had the honor of being supervised by Prof. Naoaki Okazaki. Having completed my doctoral studies, I now continue my career in the R&D division at Hotto Link, Inc.

In parallel to my full-time work, I am also a part-time researcher at Institute of Science Tokyo and visiting researcher at National Institute of Advanced Industrial Science and Technology. My research interests focused on representation learning, lexical semantics, with a specific emphasis on integrating lexical knowledge into large language models.

Github: s-mizuki-nlp

Email: sakae.mizuki [aatt] nlp [dot] c.titech.ac.jp

Publications

Journal

• Sakae Mizuki and Naoaki Okazaki. Learning Hierarchical Code Representation for Hypernymy Detection. Journal of Information Processing (TOD), 14(4):8–23, 2021. Paper

International Conference

• Naoaki Okazaki, Kakeru Hattori, Hirai Shota, Hiroki Iida, Masanari Ohi, Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Rio Yokota, Sakae Mizuki. Building a Large Japanese Web Corpus for Large Language Models. In Proceedings of the First Conference on Language Modeling (COLM 2024), pp. (to appear), October 2024. Paper
• Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki Iida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, Naoaki Okazaki. Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities. Proceedings of the First Conference on Language Modeling (COLM 2024), pp. (to appear), October 2024. Paper
• Shogo Matsuno, Sakae Mizuki, and Takeshi Sakaki. Construction of Evaluation Datasets for Trend Forecasting Studies. In Proceedings of the 17th International AAAI Conference on Web and Social Media (ICWSM 2023), pp. 1041-1051, June 2023. Paper, Dataset
• Sakae Mizuki and Naoaki Okazaki. Semantic Specialization for Knowledge-based Word Sense Disambiguation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023), pp. 3457-3470, May 2023. Paper, Code
• Sakae Mizuki and Naoaki Okazaki. Analyzing the Variation Property of Contextualized Word Representations. In AI 2019: Advances in Artificial Intelligence, pp. 393–405, December 2019. Paper

Domestic Conference

• Koshiro Saito, Sakae Mizuki, Masanari Ohi, …, and Naoaki Okazaki. Advantages of Training LLMs on Japanese Text (LLMに日本語テキストを学習させる意義). The 261st Special Interest Group on Natural Language Processing (SIG-NL), September 2024. (in Japanese) Paper, Best Paper Award
• Kazuki Fujii, Taishi Nakamura, Mengsay Loem, …, Sakae Mizuki, Rio Yokota, and Naoaki Okazaki. Developing High-Performance Japanese Large-scale Language Models Using Continual Pre-training (継続事前学習による日本語に強い大規模言語モデルの構築). The 30th Annual Meeting of The Association for Natural Language Processing (NLP2024), March 2024. (in Japanese) Paper, Best Paper Award
• Sakae Mizuki, Hiroki Iida, Kazuki Fujii, …, and Naoaki Okazaki. Efficient Japanese Language Capabilities Enhancement in Large-scale Language Models: Utilizing Vocabulary Expansion and Parallel Corpus in Continual Pre-training (大規模言語モデルの日本語能力の効率的な強化: 継続事前学習における語彙拡張と対訳コーパスの活用). The 30th Annual Meeting of The Association for Natural Language Processing (NLP2024), March 2024. (in Japanese) Paper
• Naoaki Okazaki, Kakeru Hattori, Shota Hirai, …, and Sakae Mizuki. Swallow Corpus: Large-scale Japanese Web Text Corpus (Swallowコーパス: 日本語大規模ウェブコーパス). The 30th Annual Meeting of The Association for Natural Language Processing (NLP2024), March 2024. (in Japanese) Paper, Best Paper Award
• Sakae Mizuki and Naoaki Okazaki. Semantic Specialization for Knowledge-based Word Sense Disambiguation. 29th Annual Meeting of The Association for Natural Language Processing (NLP2023), March 2023. (in Japanese) Paper, Best Paper Award
• Ryogo Ishikawa, Ayana Niwa, Sakae Mizuki, Naoaki Okazaki. Robust Dependency Parsing for the Omission of a Post-positional Particle using Pseudo Training Data. The 28th Annual Meeting of the Association for Natural Language Processing (NLP2022), March 2022. (in Japanese) Paper
• Sakae Mizuki and Naoaki Okazaki. Hyponymy Detection using Hierarchical Code Learning. 27th Annual Meeting of The Association for Natural Language Processing (NLP2021), pp. 1236–1241, March 2021. (in Japanese) Paper, Best Paper Award
• Sakae Mizuki and Takeshi Sakaki. General-Purpose Oriented Extended Named Entity Labeling of Wikipedia Entries. IEICE Technical Report, vol. 117, no. 82, NLC2017-9, pp. 47-52, June 2017. (in Japanese) Paper, Best Paper Award

Articles

• Sakae Mizuki. Behind the Scenes of “Semantic Specilization for Knowledge-based Word Sense Disambiguation” (「埋め込み表現の意味適応による知識ベース語義曖昧性解消」ができるまで). Journal of Natural Language Processing, vol. 30, no. 3, pp. 1105-1109, September 2023. (in Japanese) Article

Others

• For a full list of publications including research papers written as part of office duties, refer to my Google Scholar

Honors and Awards

• Sep. 2024: Best Paper Award (co-authored), the 261st Special Interest Group on Natural Language Processing (SIG-NL)
• Mar. 2024: Best Paper Award (co-authored two papers), the 30th Annual Meeting of The Association for Natural Language Processing (NLP2024)
• Mar. 2023: Best Paper Award, the 29th Annual Meeting of The Association for Natural Language Processing (NLP2023)
• Mar. 2021: Best Paper Award, the 27th Annual Meeting of The Association for Natural Language Processing (NLP2021)
• Jan. 2018: Best Paper Award, IEICE Technical Committee on Natural Language Understanding and Models of Communication (NLC)

Education

• Apr. 2018 - Dec. 2023: Ph.D. of Engineering (Computer Science), Institute of Science Tokyo (formerly Tokyo Institute of Technology), Japan
  • Research field: Natural Language Processing
• Apr. 2007 - Mar. 2009: Master of Engineering, Tokyo University, Japan
  • Research field: Aerospace Engineering
• Apr. 2003 - Mar. 2007: Bachelor of Engineering, Nagoya University, Japan
  • Research field: Aerospace Engineering

Work Experience

• Jul. 2024 - Present: National Institute of Advanced Industrial Science and Technology
  • Visiting Researcher
• Jan. 2024 - Present: Institute of Science Tokyo
  • Part-time Researcher
• Mar. 2015 - Present: Hotto Link Inc.
  • Research Engineer
• Apr. 2009 - Feb. 2015: Mizuho Bank, Ltd. (seconded to Mizuho-DL Financial Technology Co., Ltd.)
  • Financial Engineer

Projects

• Oct. 2023 - Present: Swallow LLM: large-scale language model research and development program

Talks

• Aug. 2024: Paper Reading Seminar. Instruction-tuned LanguagInstruction-tuned Language Models are Better Knowledge Learnerse Models are Better Knowledge Learners. In: 最先端NLP勉強会 2024. Slide
• Jan. 2024: LLM Seminar. 大規模言語モデルSwallow. In: LLM勉強会.
• Sep. 2023: LLM Seminar. Model imitationによるInstruction tuningのサーベイ. In: LLM勉強会.
• Mar. 2023: Invited talk. 埋め込み表現の意味適応による知識ベース語義曖昧性解消. In: NLP Colloquium.
• Mar. 2022: Workshop Lightning Talk. SNSを出典とする言語資源の公開にまつわるノウハウ. In: NLP2022 Co-located Workshop 日本語における評価用データセットの構築と利用性の向上 (JED2022).
• Sep. 2021: Paper Reading Seminar. A Distributional Approach to Controlled Text Generation. In: 最先端NLP勉強会 2021. Slide
• Sep. 2020: Paper Reading Seminar. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. In: 最先端NLP勉強会 2020. Slide
• Sep. 2019: Paper Reading Seminar. Ordered neurons: Integrating tree structures into recurrent neural networks. In: 最先端NLP勉強会 2019. Slide
• Aug. 2018: Paper Reading Seminar. Sequence-to-Action: End-to-End Semantic Graph Generation for Semantic Parsing. In: 最先端NLP勉強会 2018. Slide
• Mar. 2016: Workshop. ソーシャルメディア分析サービスにおけるNLPに関する諸問題について. In: NLP2016 Co-located Workshop 論文に書かない（書けない）自然言語処理.

Membership

• The Association for Natural Language Processing (NLP)
• Information Processing Society of Japan (IPSJ)
• The Securities Analysts Association of Japan (SAAJ)
• LLM-jp: LLM勉強会

Skills

• Japanese (native)
• English (fluent)
• Python, PyTorch (, R and C++)
• Elasticsearch
• Data science