I am author of a CJK language library for python called cihai (https://cihai.git-pull.com https://cihai.git-pull.com). So as part of this, and after years, I e

I am author of a CJK language library for python called cihai (https://cihai.git-pull.com https://cihai.git-pull.com). So as part of this, and after years, I eventually realized the only way to make a scalable tool to lookup Han glyphs is to build upon UNIHAN: The Unicode Consortium's Han Unification effort. I write about Unicode and UNIHAN in my own words here: http://unihan-etl.git-pull.com/en/latest/unihan.html http://unihan-etl.git-pull.com/en/latest/unihan.html The challenge with Unicode and hanzi is there are many historical and regional variants to a single source Han grapheme of the same meaning. So, each glyph or variant gets its own codepoint, or number, reserved. In fact, this years when Unicode 10.0 is cut, the new CJK Extension F will introduce 7,473 characters (http://unicode.org/versions/Unicode10.0.0/ http://unicode.org/versions/Unicode10.0.0/). Thankfully, my only task is to make the database accessible in as friendly a way as possible. Which is actually a mammoth task, see, there are over 90 fields which are used to denote dictionary indices, regional IRG [1] indices (which are national-level workgroups that convene to add new characters), phonetics (mandarin, cantonese jyutping, and more). The fields are dense. They pack in objects that are most easily split up by regular expressions. https://github.com/cihai/unihan-etl/blob/master/unihan_etl/expansion.py https://github.com/cihai/unihan-etl/blob/master/unihan_etl/e... So a UNIHAN field for kHanyuPinyin (http://www.unicode.org/reports/tr38/#kHanyuPinyin http://www.unicode.org/reports/tr38/#kHanyuPinyin): U+5364 kHanyuPinyin 10093.130:xī,lǔ 74609.020:lǔ,xī U+5EFE kHanyuPinyin 10513.110,10514.010,10514.020:gǒng U+5364 is two values (separated by the space), then a list of items either of the colon (:), which are separated by commas. You may wonder where this all comes from. The effort is global, but a good deal of it is thanks to people who took their time to contribute it, organizationally or personally. Take a look in the descriptions of the fields at http://www.unicode.org/reports/tr38/ http://www.unicode.org/reports/tr38/ for bibliographic info. In any event, the hope is to create a successor to cjklib (https://pypi.python.org/pypi/cjklib https://pypi.python.org/pypi/cjklib) and have datasets for CJK available in datapackages (http://frictionlessdata.io/data-packages/ http://frictionlessdata.io/data-packages/). That way, sources of data are sustainable and not tied down to any one library. [1] https://en.wikipedia.org/wiki/Ideographic_Rapporteur_Group https://en.wikipedia.org/wiki/Ideographic_Rapporteur_Group