解码 8bit 邮件消息：Content-Transfer-Encoding: 8bit

Posted 2023-03-15

【中文标题】解码 8bit 邮件消息：Content-Transfer-Encoding: 8bit

【英文标题】：Decoding 8bit mail message: Content-Transfer-Encoding: 8bit

【发布时间】：2013-12-25 05:54:36

【问题描述】：

我正在开发一个电子邮件查看器，它可以读取 .eml 文件并在浏览器控件中显示消息。我找到了代码 sn-p，它可以显示 7 位和引用打印消息（内容传输编码：引用打印/内容传输编码：base64）。 我需要的是解码 8 位消息。

    private static AlternateView ImportText(StringReader r, string encoding, System.Net.Mime.ContentType contentType)
    
        string line = string.Empty;
        StringBuilder b = new StringBuilder();
        while ((line = r.ReadLine())!= null)
        
            switch (encoding)
            
                case "quoted-printable":
                    if (line.EndsWith("="))
                    
                        b.Append(DecodeQuotedPrintables(line.TrimEnd('='), contentType.CharSet));
                    
                    else
                    
                        b.Append(DecodeQuotedPrintables(line, contentType.CharSet) + "\n");
                    
                    break;
                case "base64":
                    b.Append(DecodeBase64(line, contentType.CharSet));
                    break;

                case "8bit": // I need an 8bit decoder here!!!
                    b.Append(IneedAn8bitDecoderHere(line, contentType.CharSet));
                    break;
                default:
                    b.Append(line);
                    break;
            
        

        AlternateView returnValue = AlternateView.CreateAlternateViewFromString(b.ToString(), null, contentType.MediaType);
        returnValue.TransferEncoding = TransferEncoding.QuotedPrintable;
        return returnValue;
    

我搜索了一个 8 位解码器，但没有找到。我真的需要一个 8 位解码器吗？你知道有一个好的吗？

更新：

相关标题：

 MIME-Version: 1.0
 Content-Type: text/plain; charset="koi8-r";
 Content-Transfer-Encoding: 8bit

我的代码中的正文消息（字符串行）：

 ����������� �� ����, � �����  ��� � ������        ��������� �������  �   ��������  �������� ��   ������� 

Outlook 在现实世界中显示的内容：

 Фантастично но факт, я снова  как и раньше сделалась статной  и   красивой  примерно за  месяцок 

我认为我不需要 case "8bit": 部分。正如 SLaks 所提到的，我需要在流程开始时将邮件源加载到字节数组而不是字符串中。从字节数组检查邮件标头中的 charset= 将给出适当的代码页。

【问题讨论】：

一旦你把它读成一个字符串，那就太晚了。您需要使用适当的Encoding 从字节解码。

From Exchange Server 2003: Content-Transfer-Encoding: 8bit 8 位编码与 7 位编码具有相同的行长度限制。它允许 8 位字符。 8bit 文件不需要编码或解码。由于并非所有 MTA 都可以处理 8 位数据，因此 8 位编码不是 Internet 邮件的有效编码机制。

说的是二进制附件，根本不是字符串。

【参考方案1】：

这就是我解决问题的方法：

// My previous method:
string file = File.ReadAllText("koi8-r.eml");

// Correct method:    
Encoding efile = detectTextEncoding("koi8-r.eml", out file);

txtRaw.Text = output;

链接：detectEncoding()

// Function to detect the encoding for UTF-7, UTF-8/16/32 (bom, no bom, little
// & big endian), and local default codepage, and potentially other codepages.
// 'taster' = number of bytes to check of the file (to save processing). Higher
// value is slower, but more reliable (especially UTF-8 with special characters
// later on may appear to be ASCII initially). If taster = 0, then taster
// becomes the length of the file (for maximum reliability). 'text' is simply
// the string with the discovered encoding applied to the file.
public Encoding detectTextEncoding(string filename, out String text, int taster = 1000)

byte[] b = File.ReadAllBytes(filename);

//////////////// First check the low hanging fruit by checking if a
//////////////// BOM/signature exists (sourced from http://www.unicode.org/faq/utf_bom.html#bom4)
if (b.Length >= 4 && b[0] == 0x00 && b[1] == 0x00 && b[2] == 0xFE && b[3] == 0xFF)  text = Encoding.GetEncoding("utf-32BE").GetString(b, 4, b.Length - 4); return Encoding.GetEncoding("utf-32BE");   // UTF-32, big-endian 
else if (b.Length >= 4 && b[0] == 0xFF && b[1] == 0xFE && b[2] == 0x00 && b[3] == 0x00)  text = Encoding.UTF32.GetString(b, 4, b.Length - 4); return Encoding.UTF32;     // UTF-32, little-endian
else if (b.Length >= 2 && b[0] == 0xFE && b[1] == 0xFF)  text = Encoding.BigEndianUnicode.GetString(b, 2, b.Length - 2); return Encoding.BigEndianUnicode;      // UTF-16, big-endian
else if (b.Length >= 2 && b[0] == 0xFF && b[1] == 0xFE)  text = Encoding.Unicode.GetString(b, 2, b.Length - 2); return Encoding.Unicode;               // UTF-16, little-endian
else if (b.Length >= 3 && b[0] == 0xEF && b[1] == 0xBB && b[2] == 0xBF)  text = Encoding.UTF8.GetString(b, 3, b.Length - 3); return Encoding.UTF8;  // UTF-8
else if (b.Length >= 3 && b[0] == 0x2b && b[1] == 0x2f && b[2] == 0x76)  text = Encoding.UTF7.GetString(b,3,b.Length-3); return Encoding.UTF7;  // UTF-7


//////////// If the code reaches here, no BOM/signature was found, so now
//////////// we need to 'taste' the file to see if can manually discover
//////////// the encoding. A high taster value is desired for UTF-8
if (taster == 0 || taster > b.Length) taster = b.Length;    // Taster size can't be bigger than the filesize obviously.


// Some text files are encoded in UTF8, but have no BOM/signature. Hence
// the below manually checks for a UTF8 pattern. This code is based off
// the top answer at: https://***.com/questions/6555015/check-for-invalid-utf8
// For our purposes, an unnecessarily strict (and terser/slower)
// implementation is shown at: https://***.com/questions/1031645/how-to-detect-utf-8-in-plain-c
// For the below, false positives should be exceedingly rare (and would
// be either slightly malformed UTF-8 (which would suit our purposes
// anyway) or 8-bit extended ASCII/UTF-16/32 at a vanishingly long shot).
int i = 0;
bool utf8 = false;
while (i < taster - 4)

    if (b[i] <= 0x7F)  i += 1; continue;      // If all characters are below 0x80, then it is valid UTF8, but UTF8 is not 'required' (and therefore the text is more desirable to be treated as the default codepage of the computer). Hence, there's no "utf8 = true;" code unlike the next three checks.
    if (b[i] >= 0xC2 && b[i] <= 0xDF && b[i + 1] >= 0x80 && b[i + 1] < 0xC0)  i += 2; utf8 = true; continue; 
    if (b[i] >= 0xE0 && b[i] <= 0xF0 && b[i + 1] >= 0x80 && b[i + 1] < 0xC0 && b[i + 2] >= 0x80 && b[i + 2] < 0xC0)  i += 3; utf8 = true; continue; 
    if (b[i] >= 0xF0 && b[i] <= 0xF4 && b[i + 1] >= 0x80 && b[i + 1] < 0xC0 && b[i + 2] >= 0x80 && b[i + 2] < 0xC0 && b[i + 3] >= 0x80 && b[i + 3] < 0xC0)  i += 4; utf8 = true; continue; 
    utf8 = false; break;

if (utf8 == true) 
    text = Encoding.UTF8.GetString(b);
    return Encoding.UTF8;



// The next check is a heuristic attempt to detect UTF-16 without a BOM.
// We simply look for zeroes in odd or even byte places, and if a certain
// threshold is reached, the code is 'probably' UF-16.          
double threshold = 0.1; // proportion of chars step 2 which must be zeroed to be diagnosed as utf-16. 0.1 = 10%
int count = 0;
for (int n = 0; n < taster; n += 2) if (b[n] == 0) count++;
if (((double)count) / taster > threshold)  text = Encoding.BigEndianUnicode.GetString(b); return Encoding.BigEndianUnicode; 
count = 0;
for (int n = 1; n < taster; n += 2) if (b[n] == 0) count++;
if (((double)count) / taster > threshold)  text = Encoding.Unicode.GetString(b); return Encoding.Unicode;  // (little-endian)


// Finally, a long shot - let's see if we can find "charset=xyz" or
// "encoding=xyz" to identify the encoding:
for (int n = 0; n < taster-9; n++)

    if (
        ((b[n + 0] == 'c' || b[n + 0] == 'C') && (b[n + 1] == 'h' || b[n + 1] == 'H') && (b[n + 2] == 'a' || b[n + 2] == 'A') && (b[n + 3] == 'r' || b[n + 3] == 'R') && (b[n + 4] == 's' || b[n + 4] == 'S') && (b[n + 5] == 'e' || b[n + 5] == 'E') && (b[n + 6] == 't' || b[n + 6] == 'T') && (b[n + 7] == '=')) ||
        ((b[n + 0] == 'e' || b[n + 0] == 'E') && (b[n + 1] == 'n' || b[n + 1] == 'N') && (b[n + 2] == 'c' || b[n + 2] == 'C') && (b[n + 3] == 'o' || b[n + 3] == 'O') && (b[n + 4] == 'd' || b[n + 4] == 'D') && (b[n + 5] == 'i' || b[n + 5] == 'I') && (b[n + 6] == 'n' || b[n + 6] == 'N') && (b[n + 7] == 'g' || b[n + 7] == 'G') && (b[n + 8] == '='))
        )
    
        if (b[n + 0] == 'c' || b[n + 0] == 'C') n += 8; else n += 9;
        if (b[n] == '"' || b[n] == '\'') n++;
        int oldn = n;
        while (n < taster && (b[n] == '_' || b[n] == '-' || (b[n] >= '0' && b[n] <= '9') || (b[n] >= 'a' && b[n] <= 'z') || (b[n] >= 'A' && b[n] <= 'Z')))
         n++; 
        byte[] nb = new byte[n-oldn];
        Array.Copy(b, oldn, nb, 0, n-oldn);
        try 
            string internalEnc = Encoding.ASCII.GetString(nb);
            text = Encoding.GetEncoding(internalEnc).GetString(b);
            return Encoding.GetEncoding(internalEnc);
        
        catch  break;     // If C# doesn't recognize the name of the encoding, break.
    



// If all else fails, the encoding is probably (though certainly not
// definitely) the user's local codepage! One might present to the user a
// list of alternative encodings as shown here: https://***.com/questions/8509339/what-is-the-most-common-encoding-of-each-language
// A full list can be found using Encoding.GetEncodings();
text = Encoding.Default.GetString(b);
return Encoding.Default;

【参考方案2】：

由于StringReader()，您的实现可能会遇到问题。在某个地方，有人需要将原始字节转换为字符串。除非你在此之前做一些特别的事情，否则 .Net 会为你做这件事，并且通常会使用计算机的默认设置。

8 位时代的问题在于，第 8 位有几十个实现（如果不是更多的话），并且没有真正的方法可以从字节中分辨出要使用哪个实现。如果您使用 ASCII，任何设置了第 8 位的内容都将转换为 ASCII 63 - ?。如果您使用的是 UTF-8，任何设置了第 8 位的内容都将尝试读取接下来的一到五个字符 (see Wikipedia for more info)，如果这不起作用，它将转换为 UTF-8 65533 �就是你所看到的。如果您手动指定编码，例如给您koi8-r 的编码，那么将正确解析第 8 位。下面是显示这一点的示例代码。而不是转储到Console，我正在使用消息拳击，但只要你记得change your console's encoding，你就可以切换它。

var bytes = new byte[]  226 ;
var s1 = System.Text.Encoding.ASCII.GetString(bytes);//Invalid
var s2 = System.Text.Encoding.UTF8.GetString(bytes);//Invalid
var s3 = System.Text.Encoding.GetEncoding("koi8-r").GetString(bytes); //Б

MessageBox.Show(String.Format("0 1 2", s1, s2, s3));

总而言之，如果您获得了 UTF-8 替换字符（您是），这意味着您已经丢失了这些字节的原始值，您需要提前修复它。无论是把字节转换成字符串都需要考虑Content-Type: text/plain; charset="koi8-r";，事后你不能这样做。