Java IO 乱码

Posted 2020-08-06 Lucian

InputStreamReader isr = new InputStreamReader(new FileInputStream("./test/垃圾短信训练集80W条.txt"),"UTF-8");
//FileReader fr = new FileReader(new File("./test/垃圾短信训练集80W条.txt"));
BufferedReader br = new BufferedReader(isr);

//FileWriter fw = new FileWriter(new File("./test/垃圾短信训练集80W条_result.txt"), true);
OutputStreamWriter osw = new OutputStreamWriter(new FileOutputStream("./test/垃圾短信训练集80W条_result.txt"),"UTF-8");
BufferedWriter bw = new BufferedWriter(osw);

问题出在FileReader读取文件的过程中，FileReader继承了InputStreamReader，但并没有实现父类中带字符集参数的构造函数，
所以FileReader只能按系统默认的字符集来解码，然后在UTF-8 -> GBK -> UTF-8的过程中编码出现损失，造成结果不能还原最初的字符。 
原因明确了，这个问题解决起来并不困难，用InputStreamReader代替FileReader，
InputStreamReader isr=new InputStreamReader(new FileInputStream(fileName),"UTF-8");
这样读取文件就会直接用UTF-8解码，不用再做编码转换。