提问者:小点点

如何在R中读取utf-8格式的数据?


我的系统:win7 R-3.0.2。

> Sys.getlocale()
[1] "LC_COLLATE=Chinese (Simplified)_People's Republic of China.936;LC_CTYPE=Chinese 
(Simplified)_People's Republic of China.936;LC_MONETARY=Chinese (Simplified)_People's        
republic of China.936;LC_NUMERIC=C;LC_TIME=Chinese (Simplified)_People's Republic of China.936"

微软记事本中保存了两个内容相同的文件:一个保存为ansi格式,另一个保存为utf8格式。数据是M370马来西亚航空公司的死名。或者您可以这样创建文件。

1)将数据复制到微软记事本中。

乘客姓名,性别,出生日期
HuangTianhui,男,1948/05/28
姜翠云,女,1952/03/27
李红晶,女,1994/12/09

2)在记事本中以ansi格式保存为test. ansi。
3)在记事本中以utf-8格式保存为test.utf8。

read.table("test.ansi",sep=",",header=TRUE)  #can work fine
read.table("test.utf8",sep=",",header=TRUE)  #can't work

然后,我将编码设置为utf-8。

options(encoding="utf-8")
read.table("test.utf8",sep=",",header=TRUE,encoding="utf-8")


 In read.table("test.utf8", sep = ",",header=TRUE,encoding = "utf-8") :
invalid input found on input connection 'test.utf8'

如何读取数据文件(test. utf8)?
在python中,它是如此简单

rfile=open("g:\\test.utf8","r",encoding="utf-8").read()
rfile
'\ufeff乘客姓名,性别,出生日期\n\nHuangTianhui,男,1948/05/28\n\n姜翠云,女,1952/03
/27\n\n李红晶,女,1994/12/09'
rfile.replace("\n\n","\n").replace("\ufeff","").splitlines()
['乘客姓名,性别,出生日期', 'HuangTianhui,男,1948/05/28', '姜翠云,女,1952/03/27',
 '李红晶,女,1994/12/09']

Python能比R做得更好。


我发现当数据在data. frame中时,它无法正常显示,
当数据是data.frame的一列时,它可以正常显示,
奇怪的是,当数据是data.frame的一行时,它不能正常显示。


共2个答案

匿名用户

OS:Windows-7(64位)

R版本:

package_version(R.version)

[1] ‘3.0.2’

将您的语言环境从“中文”更改为“English_United州。1252”

  Sys.setlocale(category="LC_ALL", locale = "English_United States.1252")

  Sys.getlocale(category="LC_ALL")

[1] "LC_COLLATE=English_United States.1252;LC_CTYPE=English_United        States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252"

用中文编码读入数据

 df_ch <- read.table("test.utf8",
                     sep=",",
                     header=FALSE, 
                     encoding="chinese", 
                     stringsAsFactors=FALSE
                    )

用UTF-8编码读入数据

 df_utf8 <- read.table("test.utf8",
                        sep=",",
                        header=FALSE, 
                        encoding="UTF-8", 
                        stringsAsFactors=FALSE 
                      )
 df_ch$V1[1]

[1] "乘客姓å"

 df_utf8$V1[1]

[2] "乘客姓名"

 df_utf8$V1

[1] "乘客姓名"         "HuangTianhui"    "姜翠云"          "李红晶"          "LuiChing"        "宋飞飞"         
[7] "唐旭东"          "YangJiabao"      "买买提江·阿布拉" "安文兰"          "鲍媛华"          "边亮京"         
[13] "边茂勤"          "曹蕊"            "车俊章"          "陈长军"          "陈建设"          "陈昀"           
[19] "戴淑玲"          "丁立军"          "丁莹"            "丁颖"            "董国伟"          "杜文忠"         
[25] "冯栋"            "冯纪新"          "付宝峰"          "甘福祥"          "甘涛"            "高歌"           
[31] "管文杰"          "韩静"            "侯爱琴"          "侯波"            "胡偲婠(婴儿)"    "胡效宁"         

显示数据帧中一行的unicode数据

  df_utf8[1,]
                                    V1               V2                               V3
  1 <U+FEFF><U+4E58><U+5BA2><U+59D3><U+540D> <U+6027><U+522B> <U+51FA><U+751F><U+65E5><U+671F>

显示数据框中一行的中文数据

as.character(df_utf8[1,])

[1] "乘客姓名"  "性别"     "出生日期"

as.character(df_utf8[2,])

[1] "HuangTianhui" "男"           "1948/05/28"

可以通过将数据帧转换为列表并强制将数据转换为字符格式来显示具有国际字符的多列数据。

  df_utf8_ch <- lapply(df_utf8, as.character)

  df_utf8_ch

$V1 1 "乘客姓名" "HuangTianhui" "姜翠云" "李红晶" "LuiChing" "宋飞飞"
7 "唐旭东" "YangJiabao" "买买提江·阿布拉" "安文兰" "鲍媛华" "边亮京"
[13] "边茂勤" "曹蕊" "车俊章" "陈长军" "陈建设" "陈昀"
[19] "戴淑玲" "丁立军" "丁莹" "丁颖" "董国伟" "杜文忠"
[25] "冯栋" "冯纪新" "付宝峰" "甘福祥" "甘涛" "高歌"
[31] "管文杰" "韩静" "侯爱琴" "侯波" "胡偲婠(婴儿)" "胡效宁"
[37] "黄毅" "姜学仁" "姜颖" "焦微微" "焦文学" "鞠坤"
[43] "康旭" "黎明中" "李国辉" "李洁" "李乐" "李文博"
[49] "李燕" "李宇辰" "李志锦" "李志欣" "李智" "栗延林"
[55] "梁路阳" "梁旭阳" "林安南" "林明峰" "刘凤英" "刘金鹏"
[61] "刘强" "刘如生" "刘顺超" "柳忠福" "楼宝棠" "卢先初"
[67] "鹿建华" "罗伟" "马骏" "马文芝" "毛土贵" "么立飞"
[73] "蒙高生" "孟兵" "孟凡余" "欧阳欣" "石贤文" "宋春玲"
[79] "宋坤" "苏强国" "汤雪竹" "田军伟" "田清君" "汪厚彬"
[85] "王春勇" "王纯华" "王丹" "王海涛" "王利军" "王林诗"
[91] "王墨恒(婴儿)" "王守宪" "王淑敏" "王献军" "王永刚"

 View(df_ch)

 View(df_utf8)

查看(df_ch)

查看(df_utf8)

好消息是您拥有utf8格式的所有数据,可用于进一步的数据分析。

分析完成后,您可以将区域设置更改回“中文”

  Sys.setlocale(category="LC_ALL", locale = "chinese")

  Sys.getlocale(category="LC_ALL")

 [1] "LC_COLLATE=Chinese (Simplified)_People's Republic of China.936;LC_CTYPE=Chinese   (Simplified)_People's Republic of China.936;LC_MONETARY=Chinese (Simplified)_People's Republic of China.936;LC_NUMERIC=C;LC_TIME=Chinese (Simplified)_People's Republic of China.936"

在字符串编码之间进行转换可能需要探索一些函数。

编码()

图标()

宏达

匿名用户

尝试read. table的不同参数:fileEncode

 read.table("test.utf8", sep = "," , header=TRUE, fileEncoding = "UTF-8")