代码之家 › 专栏 › 技术社区 › James Cooper

Java 1.6 Windows-1252编码在3个字符上失败

windows-1252 codepages java

4

James Cooper · 技术社区 · 16 年前

编辑:我确信这个问题有点不合情理。感谢那些回应的人。我可能会发布一个更具体的后续问题。

int badCount = 0;
for (int i = 1; i < 255; i++) {
    String str = "Hi " + new String(new char[] { (char) i });

    String toLatin1  = new String(str.getBytes("UTF-8"), "latin1");
    assertEquals(str, new String(toLatin1.getBytes("latin1"), "UTF-8"));

    String toWin1252 = new String(str.getBytes("UTF-8"), "Windows-1252");
    String fromWin1252 = new String(toWin1252.getBytes("Windows-1252"), "UTF-8");

    if (!str.equals(fromWin1252)) {
        System.out.println("Can't encode: " + i + " - " + str + 
                           " - encodes as: " + fromWin1252);
        badCount++;
    }
}

System.out.println("Bad count: " + badCount);

    Can't encode: 129 - Hi ? - encodes as: Hi ??
    Can't encode: 141 - Hi ? - encodes as: Hi ??
    Can't encode: 143 - Hi ? - encodes as: Hi ??
    Can't encode: 144 - Hi ? - encodes as: Hi ??
    Can't encode: 157 - Hi ? - encodes as: Hi ??
    Can't encode: 193 - Hi Ã - encodes as: Hi ??
    Can't encode: 205 - Hi Ã - encodes as: Hi ??
    Can't encode: 207 - Hi Ã - encodes as: Hi ??
    Can't encode: 208 - Hi ? - encodes as: Hi ??
    Can't encode: 221 - Hi ? - encodes as: Hi ??
    Bad count: 10

Mac OS 10.6.2上的JDK 1.6.0_07

Latin1对所有254个字符进行对称编码。Windows-1252没有。这三个可打印字符(193205207)在Latin1和Windows-1252中是相同的代码,所以我不希望出现任何问题。

--詹姆斯

2 回复 | 直到 16 年前

1

4

Joachim Sauer 16 年前

在我看来,测试程序有很大的缺陷,因为它在没有语义的字符串之间进行了有效的无用转换。

如果要检查所有字节值是否都是给定编码的有效值,则类似于以下内容:

public static void tryEncoding(final String encoding) throws UnsupportedEncodingException {
    int badCount = 0;
    for (int i = 1; i < 255; i++) {
        byte[] bytes = new byte[] { (byte) i };

        String toString = new String(bytes, encoding);
        byte[] fromString = toString.getBytes(encoding);

        if (!Arrays.equals(bytes, fromString)) {
            System.out.println("Can't encode: " + i + " - in: " + Arrays.toString(bytes) + "/ out: "
                    + Arrays.toString(fromString) + " - result: " + toString);
            badCount++;
        }
    }

    System.out.println("Bad count: " + badCount);
}

字节值 (相当于此范围内的Unicode代码点)从1到255。

尝试打印示例中程序处理的实际字节数组,您会发现实际上并没有检查所有字节值,并且一些“错误”匹配与其他匹配是重复的。

"Windows-1252" 当参数生成此输出时:

Can't encode: 129 - in: [-127]/ out: [63] - result: ï¿½
Can't encode: 141 - in: [-115]/ out: [63] - result: ï¿½
Can't encode: 143 - in: [-113]/ out: [63] - result: ï¿½
Can't encode: 144 - in: [-112]/ out: [63] - result: ï¿½
Can't encode: 157 - in: [-99]/ out: [63] - result: ï¿½
Bad count: 5

这告诉我们 Windows-1252

The Wikipedia article on Windows-1252

根据微软和Unicode联盟网站上的信息,位置81、8D、8F、90和9D未使用

2

BalusC 16 年前

代码的作用是什么( String->byte[]->String 转码,而且毫无意义(事实上,它肯定会丢失数据)。转码手段 byte[]->String->byte[]

public byte[] transcode(byte[] input, String inputEnc, String targetEnc)
{
    return new String(input, inputEnc).getBytes(targetEnc);
}

当然,当输入包含目标编码不支持的字符时,它将丢失数据。