Pandas df.to_csv(“file.csv” encode=“utf-8”) still gives trash characters for minus sign
我读过一些关于python 2的限制,关于熊猫对csv的限制。等等…)我打中了吗?我在python 2.7.3上
当它们出现在字符串中时,≥和-就会变成垃圾字符。除此之外,出口是完美的。
1 | df.to_csv("file.csv", encoding="utf-8") |
有什么解决办法吗?
df.head()是:
1 2 3 4 5 6 7 | demography Adults ≥49 yrs Adults 18?49 yrs at high risk|| \ state Alabama 32.7 38.6 Alaska 31.2 33.2 Arizona 22.9 38.8 Arkansas 31.2 34.0 California 29.8 38.8 |
csv输出是这样的
1 2 3 4 5 6 | state, Adults a‰¥49 yrs, Adults 18a?’49 yrs at high risk|| 0, Alabama, 32.7, 38.6 1, Alaska, 31.2, 33.2 2, Arizona, 22.9, 38.8 3, Arkansas,31.2, 34 4, California,29.8, 38.8 |
整个代码是这样的:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | import pandas import xlrd import csv import json df = pandas.DataFrame() dy = pandas.DataFrame() # first merge all this xls together workbook = xlrd.open_workbook('csv_merger/vaccoverage.xls') worksheets = workbook.sheet_names() for i in range(3,len(worksheets)): dy = pandas.io.excel.read_excel(workbook, i, engine='xlrd', index=None) i = i+1 df = df.append(dy) df.index.name ="index" df.columns = ['demography', 'area','state', 'month', 'rate', 'moe'] #Then just grab month = 'May' may_mask = df['month'] =="May" may_df = (df[may_mask]) #then delete some columns we dont need may_df = may_df.drop('area', 1) may_df = may_df.drop('month', 1) may_df = may_df.drop('moe', 1) print may_df.dtypes #uh oh, it sees 'rate' as type 'object', not 'float'. Better change that. may_df = may_df.convert_objects('rate', convert_numeric=True) print may_df.dtypes #that's better res = may_df.pivot_table('rate', 'state', 'demography') print res.head() #and this is going to spit out an array of Objects, each Object a state containing its demographics res.reset_index().to_json("thejson.json", orient='records') #and a .csv for good measure res.reset_index().to_csv("thecsv.csv", orient='records', encoding="utf-8") |
你的"坏"输出是UTF-8,显示为CP1252。
在Windows上,如果文件开头没有字节顺序标记(bom)字符,那么许多编辑器将采用默认的ANSI编码(在美国Windows上为CP1252),而不是UTF-8。虽然bom对utf-8编码毫无意义,但它的utf-8编码状态可以作为某些程序的签名。例如,即使在非Windows操作系统上,Microsoft Office的Excel也需要它。尝试:
1 | df.to_csv('file.csv',encoding='utf-8-sig') |
编码器将添加物料清单。