Unicode fonts and tools for X11

The classic X Window System bitmap fonts are now available in an ISO 10646-1/Unicode extension.

We have extended all the "-misc-fixed-*" fonts:

  5x7     -Misc-Fixed-Medium-R-Normal--7-70-75-75-C-50-ISO10646-1
  5x8     -Misc-Fixed-Medium-R-Normal--8-80-75-75-C-50-ISO10646-1
  6x9     -Misc-Fixed-Medium-R-Normal--9-90-75-75-C-60-ISO10646-1
  6x10    -Misc-Fixed-Medium-R-Normal--10-100-75-75-C-60-ISO10646-1
  6x12    -Misc-Fixed-Medium-R-Semicondensed--12-110-75-75-C-60-ISO10646-1
  6x13    -Misc-Fixed-Medium-R-SemiCondensed--13-120-75-75-C-60-ISO10646-1
  6x13B   -Misc-Fixed-Bold-R-SemiCondensed--13-120-75-75-C-60-ISO10646-1
  7x13    -Misc-Fixed-Medium-R-Normal--13-120-75-75-C-70-ISO10646-1
  7x13B   -Misc-Fixed-Bold-R-Normal--13-120-75-75-C-70-ISO10646-1
  7x14    -Misc-Fixed-Medium-R-Normal--14-130-75-75-C-70-ISO10646-1
  7x14B   -Misc-Fixed-Bold-R-Normal--14-130-75-75-C-70-ISO10646-1
  8x13    -Misc-Fixed-Medium-R-Normal--13-120-75-75-C-80-ISO10646-1
  8x13B   -Misc-Fixed-Bold-R-Normal--13-120-75-75-C-80-ISO10646-1
  9x15    -Misc-Fixed-Medium-R-Normal--15-140-75-75-C-90-ISO10646-1
  9x15B   -Misc-Fixed-Bold-R-Normal--15-140-75-75-C-90-ISO10646-1
  10x20   -Misc-Fixed-Medium-R-Normal--20-200-75-75-C-100-ISO10646-1

Coverage

These fonts contain now all characters found in the following character sets:

• ISO 8859 parts 1-5, 7-10, 13-15 (i.e., all parts except Arabic and Thai)
• ISO 6937 and the CEN MES-1 European Unicode Subset
• IBM/Microsoft code pages CP 437, 850, 1251, 1252, and many others
• Microsoft/Adobe Windows Glyph List 4 (WGL4)
• KOI8-R
• DEC VT100 graphics symbols

The 6x13, 8x13, 9x15, 9x18, and 10x20 fonts cover a much larger repertoire in addition, that covers the comprehensive CEN MES-3A European Unicode 3.2 Subset, the International Phonetic Alphabet, Armenian, Georgian, Thai, Yiddish, all Latin, Greek, and Cyrillic characters, all mathematical symbols (including the entire TeX repertoire), APL, Braille, Runes, and much more. 9x15 and 10x20 also cover Ethiopian.

Newly added fonts

The following new "-misc-fixed-*" fonts were added:

  6x13O   -Misc-Fixed-Medium-O-SemiCondensed--13-120-75-75-C-60-ISO10646-1
  7x13O   -Misc-Fixed-Medium-O-Normal--13-120-75-75-C-70-ISO10646-1
  8x13O   -Misc-Fixed-Medium-O-Normal--13-120-75-75-C-80-ISO10646-1
  9x18    -Misc-Fixed-Medium-R-Normal--18-120-100-100-C-90-ISO10646-1
  9x18B   -Misc-Fixed-Bold-R-Normal--18-120-100-100-C-90-ISO10646-1
  12x13ja -Misc-Fixed-Medium-R-Normal-ja-13-120-75-75-C-120-ISO10646-1
  18x18ja -Misc-Fixed-Medium-R-Normal-ja-18-120-100-100-C-180-ISO10646-1
  18x18ko -Misc-Fixed-Medium-R-Normal-ko-18-120-100-100-C-180-ISO10646-1

6x13O, 7x13O and 8x13O are oblique/italic versions of 6x13, 7x13 and 8x13. 9x18 is an improved version of 9x15 that has more space above and below the base characters to increase readability and to allow overstriking combining characters to work properly. 18x18ja and 18x18ko provide Japanese and Korean doublewidth ideograms for 9x18. 12x13ja provides Japanese doublewidth ideograms for 6x13.

Adobe BDF fonts

I have also created revised ISO10646-1 versions of all the Adobe and B&H pixel fonts that come with X11R6.4. They contained about 30 additional Postscript characters (roughly the CP1252 repertoire) that were present in the old ISO8859-1 BDF files, but were not encoded and therefore not accessible for X clients. The revised ISO10646-1 versions contain not only these but also many more automatically generated accented Latin characters (e.g., all characters from ISO 8859 parts 1-4, 9-10, 13-15), and they also fix a few long-standing bugs with the old fonts (missing NBSP, exchanged multiplication/division sign, etc.).

Status

The fonts are now completed and implement at present version 3.2 of the Unicode standard (ISO 10646-1/Amd.1:2002). I will maintain them to fix bugs and to satisfy any newly reported user requirements. Note that the new fonts fix a problem with the Latin-1 quotation mark and accents.

Download

The fonts are freely available with installation instructions and example UTF-8 text files.

The "-misc-fixed-*" font package:

https://www.cl.cam.ac.uk/~mgk25/download/ucs-fonts.tar.gz

CJK ideographic wide character supplement (unpack into the same subdirectory as the above):

https://www.cl.cam.ac.uk/~mgk25/download/ucs-fonts-asian.tar.gz

The Adobe and B&H font package:

https://www.cl.cam.ac.uk/~mgk25/download/ucs-fonts-75dpi100dpi.tar.gz

There is also a change log file for the "-misc-fixed-*" fonts.

Other character sets

The font packages include the ucs2any.pl Perl script, which converts ISO 10646-1 fonts into any other encoding for which there is a Unicode mapping table available. This way, you can quickly generate ISO 8859-* versions from the above fonts automatically, for the benefit of older software that cannot yet handle ISO 10646-1 fonts directly.

Distribution

I periodically contribute a recent snapshot of all of the above fonts to XFree86 and they have been shipping as part of the XFree86 releases since XFree86 4.1. I have also made them available to X.Org for inclusion into one of the next official X11 distributions as a replacement for the current ISO 8859-1 BDF fonts (hopefully they will be in X11R6.7).

Copyright

Who created the original -misc-fixed-* ASCII fonts or the later ISO 8859-1 extension is not documented. They most likely came from either MIT Project Athena or their industrial partner DEC in the 1980s. Most of these fonts contained in the header of the BDF file the property

COPYRIGHT "Public domain font.  Share and enjoy."

Related information and links

• Read the UTF-8 and Unicode FAQ for Unix/Linux for detailed general information on how to use Unicode and its ASCII-compatible UTF-8 encoding under Unix, Linux, X11, etc.
• To use these ISO10646-1 fonts, you will need applications that support ISO10646-1 fonts (hardly any software released before ~2001 does). These are not simply 8-bit replacement fonts but usually need to be used together with UTF-8 support in an application. For instance, if you want to use these fonts with xterm, you need to use an xterm version that can handle ISO10646-1 fonts (e.g., the one in XFree86 4.x).
• The "-misc-fixed-*" fonts were created and extended using Mark Leisher’s xmbdfed font editor, which later evolved into the gbdfed font editor (using GTK+ instead of Motif). You can use the latter to view and modify these fonts.
• Unicode X11 font names end with -ISO10646-1. This is the value in the official X registry (git) for the X Logical Font Descriptor (XLFD) fields CHARSET_REGISTRY and CHARSET_ENCODING for all Unicode and ISO 10646-1 16-bit fonts. There is no registered XLFD scheme yet for ISO 10646 characters outside the BMP, though some proposals have been discussed.
• Unicode and ISO 10646 merged CJK ideograph repertoires from several groups of national source standards. In order to indicate that an ISO10646-1 font with ideographic characters was designed following the glyph style from one particular group of national source standards, the ADD_STYLE_NAME XLFD field can be used to indicate the corresponding language or region. Examples for such ADD_STYLE_NAME values are:

  ADD_STYLE_NAME	IRG Source	Countries	Standards
  zh	G	China, Hongkong, Singapore	GB2312, GB12345, GB7589, GB7590, GB8565, GB16500
  zh_TW	T	Taiwan	CNS 11643
  ja	J	Japan	JIS X 0208, JIS X 0212
  ko	K	Korea	KS C 5601, KS C 5657, PKS C 5700
  vi	V	Vietnam	TCVN 5773, TCVN 6056

  ISO 639 language and ISO 3166 country/region codes should be preferred in ADD_STYLE_NAME. This way, if an application knows a style preference from, for example, the RFC 1766 code "xx-yy" in a language tag, or from the locale name "xx_YY.ZZZ", it can search for a suitable font by looking for ADD_STYLE_NAME values in the following order of preference: "xx_YY", "xx", "", "*".

• The specification of the BDF X11 pixel font format is available as Technical Note 5005 from Adobe or in a simplified form as part of the X11 documentation.
• The fonts use the Adobe standard glyph names for Unicode fonts.
• There is also a note on why the apostrophe and grave accent look different in the new fonts.
• A bug in old GTK+ libraries is triggered by the presence of ISO10646-1 Helvetica fonts.

Other information relevant to Unicode font projects

• The FreeType Project has implemented an open source version of the Microsoft/Apple TrueType font rendering engine. OpenType is a merger of the TrueType and Type1 font formats, with extensions for better handling of all scripts supported by Unicode.
• ttf2bdf and xmbdfed by Mark Leisher allow you to generate X11 BDF pixel fonts from TrueType fonts and edit them.
• PfaEdit is a free Type1, TrueType, OpenType and BDF font editor by George Williams.
• Two free PostScript Type1 font editors that are currently under development are Spif by Matty Farrow and gfonted by Raph Levien.
• Commercial vector font editors: Macromedia Fontographer, FontLab.
• Taco Hoekwater has created a Unicode math font that can be used with Postscript Times Roman.
• xfsft: TrueType Font Support For X is a set of patches to make X11 support truetype fonts by Juliusz Chroboczek (included with XFree86 4.0).
• Fonts in the X Window System by Nelson Beebe is a useful quick introduction. The XFree86 Font Deuglification Mini HOW-TO by Doug Holland also contains numerous good tips for setting up fonts under Linux/X11.
• Luc Devroye’s Font software page is a very comprehensive collection of pointers to further font resources.
• Ulf Jordan’s Misc-Fixed ISO 10646-1 Outline Font Project is to develop Type1 versions of the BDF font family provided here.
• Nick Reinking prepared TrueType versions of the Latin-1 subsets of 6x13 and 6x13B.
• The Electronic Font Open Laboratory in Japan is also working on a family of Unicode bitmap fonts.

Why are there no Indic or Syriac glyphs in the ucs-fonts package?

In European and East Asian scripts, each Unicode character can be represented by a single graphical shape ("glyph"). The X11 font system is entirely built around the idea that there is a one-to-one relationship between characters and glyphs, which works fine for Latin, Greek, Cyrillic, Hebrew, Han, Hiragana, Katakana, Hangul, etc. However, things are far more complicated for handwritten cursive scripts such as Arabic, Syriac and the various Indic scripts (Devanagari, Bengali, Gurmukhi, Gujarati, Oriya, Tamil, Telugu, Kannada, etc.). For these scripts, the sequence of values ("characters") encoded in a Unicode string (which usually corresponds to the sequence of keystrokes during entry and the sequence of phonemes when speaking) first has to be converted into a sequence of graphical symbols ("glyphs") as they are found in a font, before a string can be displayed. In a given Latin font style, always the same graphical glyph of a font will be used for representing a character on a screen. In an Arabic or Indic font, the shape of the glyph depends not only on the character that it represents, but also on its neighbour characters. Sometimes, different glyphs have to be used depending on the character appearing at the beginning, middle, or end of a word, and often certain entire sequences of characters have to be represented by a special ligature glyph. A very simple form of that is used in Latin fine typography in the form of the "fi" and "fl" ligatures, but in Indic scripts, the situation is far more extreme, and the number of glyphs is often several times the number of characters. For details and examples, read Chapter 9 and Chapter 8 as well as the relevant code charts of The Unicode Standard.

The Unicode standard does contain encoding ranges for a simple scheme of Arabic glyphs, the "Arabic Presentation Forms". This was possible, because for Arabic there is a reasonably good consensus among font designers on how many glyphs are actually necessary for proper rendering of Arabic text, even though some argue that for really high-quality typesetting the Unicode collection of Arabic presentation forms is not sufficient. For Indic scripts on the other hand, there seems no consensus among font designers, which glyphs are actually necessary as this can vary significantly across different font styles. Therefore, an Indic font is always a proprietary non-standardized collection of glyphs together with a mapping table that defines, how sequences of standard Unicode characters have to be transformed into sequences of non-standard Indic glyphs from this particular font, before the text can be displayed.

The OpenType font format developed by Microsoft and Adobe is an outline font format that does include such character/glyph mapping tables. The BDF format used by X11 pixel fonts does not have any standardized way of including a character/glyph mapping table, and neither do current BDF editors such as xmbdfed or X servers. The Pango rendering library developed for the Gnome project can make use of BDF glyph fonts, but it requires the corresponding character/glyph mapping table in a separate client-side file. The X11 standards currently provide no support for transmitting such mapping tables over the X11 protocol. Roman Czyborra’s GNU Unifont does contain a naive representation of the Indic glyphs shown in the Unicode Standard code charts, but that is of no use in practice for displaying Indic strings properly.

Summary: X11 was never designed for Arabic, Syriac, Indic, and special libraries such as Pango have to be used for these scripts. If you want to help getting Indic supported under X11, you have to extend the X11 standards to fix this problem and provide a font mechanism that understands that some scripts need to map characters into glyphs. The solution is unfortunately not as easy as just drawing a few glyphs with a font editor, otherwise we would already have added the Indic scripts long ago to the ucs-fonts package.

Markus Kuhn

created 1998-09-22 – last modified 2022-12-07 – https://www.cl.cam.ac.uk/~mgk25/ucs-fonts.html