MIRACLE LINUX : Samba Internationalisation Project

1. Introduction

In Samba 3.0, the character conversion routines have changed dramatically from that of Samba 2.x. Although the official Samba 3.0.0 is out for a while, only a limited amount of verifications are done for testing its multibyte characters handlings.

This project is intended to perform complete internationalisation testings on Samba, and fixing bugs whenever such bugs found.

Below is the list of brief descriptions of what this project is all about.

Implementing regression tests for internationalisation features in Samba.
Verifying multibyte capabilities of Samba and reporting bugs found by tests.
Fixing bugs that are found involving multibyte characters handling.

Our goal is to merge end products of this project (tests and patches) into mainstream of Samba.

2. Outlines

We are aware that implementing complete multibyte features is a hard task to accompolish as:

Bugs are less likely to be found, as not every developer owns multibyte environment.
Developers require knowledge of multibyte programming skills.
Preparing and setting up multiple patterns of environment for testing is too resource-intensive.

Points (1) and (3) are somehow related. One of the issues to be considered in the development of Samba is, because testing Samba requires a vast amount of knowledge in Linux, Windows network and Samba itself, so testing multibyte features are done by a small number of people.

What we are doing right now is to incorporate automatic testing scripts in the build farm that samba.org provides. Build farm tests codes in CVS everyday, and it would be a feasible solution for problems (1) and (3) that occasional internationalisation tests are carried out in a reasonable time span.

It is not likely that automatic testing will cover the entire functionality of Samba 3.0. However, by incorporating into automatic testing framework, it would at least assure developers that the code achieved a certain level of multibyte capabilities. Future product have to pass the tests meaning the code would be regression-free to a certain extent.

To address point (2), we are preparing the White Paper and a section of internationalisation in Samba Developer's Guide.

iconv issues

Also, it is worth mentioning that the implementation of iconv() functions in glibc and libiconv are not sufficient. Because Samba 3.0 employs iconv() functions provided by external packages for multibyte character conversions, we are working on fixing these packages to work correctly.

For more detailed information on what their problems are, please refer to the following link.

Issues in iconv

3. Output

Patches for internationalisation tests for Samba 3.0.1 will be available soon.

Below is the list of bugs reported to bugzilla.samba.org.

[Table] Current Bugzilla Status
BUG ID	STATUS	DESCRIPTION
412	FIXED	d_printf() cannot handle '\'
456	FIXED	Makefile does not install souce/po/*.msg files automatically
570	REOPEN	configure in samba-3.0.0 ignores LDFLAGS environment variable
587	NEW	smbcontrol close-share doesn't work if PID not specified
589	NEW	smbclient '-N' option force users to logon anonymously
593	NEW	%U macro doesn't take non-ascii chars
617	FIXED	net.8.xml: missing <title> in <refsect3> from net.8 man page
618	FIXED	smbcquotas(1) man page shows wrong QUOTA_SET_COMAND format
619	NEW	smbcacls(1) ignores -n option
620	NEW	smbcacls(1): ACL type can be either ALLOWED or DENIED, not 0 or 1
625	FIXED	winbindd.8.xml: missing sample config of nsswitch.conf and hosts db
631	FIXED	wrepld cannot be made up
761	NEW	printing parameter doesn't override the default values of printer-related commands
763	FIXED	pdbedit command returns wrong status by account delete
770	FIXED	smbd cannot cancel the UNIX print jobs
900	NEW	The symlink, link command always display the usage messages

4. Issues

This section describes the problems observed in current Samba 3.0 release.

4.1. nmbd Structure

nmbd fails when handling some kinds of multibyte characters. This is because it receives data in dos charset but it handles data internally with unix charset, so if unix charset is set to UTF-8, buffer provided for it would overflow.

4.2. File Name Collision

In Samba 3.0, a new mangling method "hash2" is introduced. While this algorithm is less likely to cause harms compare to the "hash" algorithm employed in Samba 2.x, is not a complete solution. All it does is just to make collisions _LESS_ likely to happen.

Operations like "drag and drop" handles file names in SFN, which means that whenever a conflict occurs among multiple files, only one of them would be chosen with graphical file manipulations.

4.3. Samba Macro Variables

Samba prepares macro variables like %U and %G which are substituted with some other strings when requested. These macros only permits limited ascii codes for inputs, as some special codes appear to be security risks.

However, under current implementation, it is throwing away most of harmless multibyte characters.

Contacts

For more information, please send emails to samba-dev@miraclelinux.com.

Samba 3.0 Internationalisation Project

Table of Contents