NumPy排序问题
我写了一个代码能够从CSV文件将值导入Python。我会把数据分成若干阶段处理,一切都很顺利,直到我决定对它进行排序。
原始数据集,a 读取为:
a = ['27622000' '21731000' '15969000' '10795000' '6740700' '3914000' '2119400'
'1105000' '584200' '293420' '7836700' '4577100' '2502800' '1307700'
'662020' '328020' '159110' '76451' '36528' '17000' '1097400' '542300'
'264240' '127210' '60799' '28603' '13239' '5932.7' '2546.4' '1035.4'
'148800' '70291' '32856' '15083' '6772.4' '2912.1' '1198.4' '465.64'
'170.48' '57.957' '23321' '10567' '4669.8' '1980.7' '799.96' '305.02'
'108.46' '35.823' '10.317' '2.6763' '4281.1' '1804.6' '727.56' '277.04'
'98.374' '31.673' '9.0643' '2.3084' '0.56351' '0.15655' '800.37' '330.07'
'119.93' '38.918' '11.377' '2.998' '0.73989' '0.19126' '0.060255'
'0.025337' '167.43' '58.87' '18.492' '4.9108' '1.2305' '0.3694' '0.12245'
'0.048883' '0.018582' '0.012778' '11.358' '5.6482' '1.868' '0.35063'
'0.16371' '0.12595' '0.069311' '0.036627' '0.016214' '0.0083006' '0' '0'
'0' '0' '0.019067' '0.085504' '0.051517' '0.026125' '0.012807'
'0.0059197']
如果我检查a 的类型,输出将显示为:
print(type(a))
接着输出为:
<class 'numpy.ndarray'>
如果我用以下函数对它进行排序:
a_sorted = np.ndarray.sort(a)
输出将变为:
a_sorted = ['0' '0' '0' '0' '0.0059197' '0.0083006' '0.012778' '0.012807' '0.016214'
'0.018582' '0.019067' '0.025337' '0.026125' '0.036627' '0.048883'
'0.051517' '0.060255' '0.069311' '0.085504' '0.12245' '0.12595' '0.15655'
'0.16371' '0.19126' '0.35063' '0.3694' '0.56351' '0.73989' '1.2305'
'1.868' '10.317' '1035.4' '10567' '10795000' '108.46' '1097400' '11.358'
'11.377' '1105000' '119.93' '1198.4' '127210' '1307700' '13239' '148800'
'15083' '159110' '15969000' '167.43' '170.48' '17000' '18.492' '1804.6'
'1980.7' '2.3084' '2.6763' '2.998' '2119400' '21731000' '23321' '2502800'
'2546.4' '264240' '27622000' '277.04' '28603' '2912.1' '293420' '305.02'
'31.673' '328020' '32856' '330.07' '35.823' '36528' '38.918' '3914000'
'4.9108' '4281.1' '4577100' '465.64' '4669.8' '5.6482' '542300' '57.957'
'58.87' '584200' '5932.7' '60799' '662020' '6740700' '6772.4' '70291'
'727.56' '76451' '7836700' '799.96' '800.37' '9.0643' '98.374']
它确实能把其中一些排序得非常好,但有些却出于某些原因没有排序好。看起来排序函数似乎只看前面的数字,却没有把 "." 考虑在内。我到底错在了哪里?
解决方案
正如评论中所提到的,在排序之前把数据从字符串转换为浮点数,以获得数值排序而非字典序排序。
对于NumPy数组,不仅它本身的类型重要(type(a)),它的值的数据类型(a.dtype)也同样关键:如果你不是(或除了)写成 print(type(a)),而改成写成 print(a.dtype),你很可能会看到结果 <U32,这表明你的数组包含字符串(最大长度32的小端Unicode序列,更准确地说)。字符串是按字典序排序的,例如,以 1 开头的值总是在以 2 开头的值之前排序,无论前者是否真正是 100,后者是否真正是 20。
使用 np.asarray(a, dtype=float) 或 a.astype(float),你可以把数组内容转换为浮点数(print(a.dtype) 现在将产生 float64),排序现在就会在真正期望的数值顺序下进行(在这里,我需要使用 np.asarray(),因为在我的情况下,a 是一个列表而不是NumPy数组):
import numpy as np
a = ['27622000','21731000','15969000','10795000','6740700','3914000','2119400',
'1105000','584200','293420','7836700','4577100','2502800','1307700',
'662020','328020','159110','76451','36528','17000','1097400','542300',
'264240','127210','60799','28603','13239','5932.7','2546.4','1035.4',
'148800','70291','32856','15083','6772.4','2912.1','1198.4','465.64',
'170.48','57.957','23321','10567','4669.8','1980.7','799.96','305.02',
'108.46','35.823','10.317','2.6763','4281.1','1804.6','727.56','277.04',
'98.374','31.673','9.0643','2.3084','0.56351','0.15655','800.37','330.07',
'119.93','38.918','11.377','2.998','0.73989','0.19126','0.060255',
'0.025337','167.43','58.87','18.492','4.9108','1.2305','0.3694','0.12245',
'0.048883','0.018582','0.012778','11.358','5.6482','1.868','0.35063',
'0.16371','0.12595','0.069311','0.036627','0.016214','0.0083006','0','0',
'0','0','0.019067','0.085504','0.051517','0.026125','0.012807',
'0.0059197']
a = np.sort(np.asarray(a, dtype=float)) # Use float data type
with np.printoptions(suppress=True): # Suppress scientific notation
print(a)
# [ 0. 0. 0. 0.
# 0.0059197 0.0083006 0.012778 0.012807
# 0.016214 0.018582 0.019067 0.025337
# 0.026125 0.036627 0.048883 0.051517
# 0.060255 0.069311 0.085504 0.12245
# 0.12595 0.15655 0.16371 0.19126
# 0.35063 0.3694 0.56351 0.73989
# 1.2305 1.868 2.3084 2.6763
# 2.998 4.9108 5.6482 9.0643
# 10.317 11.358 11.377 18.492
# 31.673 35.823 38.918 57.957
# 58.87 98.374 108.46 119.93
# 167.43 170.48 277.04 305.02
# 330.07 465.64 727.56 799.96
# 800.37 1035.4 1198.4 1804.6
# 1980.7 2546.4 2912.1 4281.1
# 4669.8 5932.7 6772.4 10567.
# 13239. 15083. 17000. 23321.
# 28603. 32856. 36528. 60799.
# 70291. 76451. 127210. 148800.
# 159110. 264240. 293420. 328020.
# 542300. 584200. 662020. 1097400.
# 1105000. 1307700. 2119400. 2502800.
# 3914000. 4577100. 6740700. 7836700.
# 10795000. 15969000. 21731000. 27622000. ]
或者,你也许已经想在加载CSV时就确保值是浮点数而不是字符串。我不知道你的原始CSV文件长什么样、你使用的CSV加载方式是什么,但无论如何,一些常用的库已经提供了相关功能:
- 两者 [
numpy.genfromtxt()] 以及 [pandas.read_csv()] 提供一个dtype参数,在必要时强制执行某种数据类型的解释。 - 内置的
csv模块提供QUOTE_NONNUMERIC标志([csv.reader(…, quoting=csv.QUOTE_NONNUMERIC)]):从CSV文件读取的每一行都会作为一个字符串列表返回。除非指定QUOTE_NONNUMERIC格式选项,否则不会进行自动数据类型转换(在这种情况下,未带引号的字段将转换为浮点数)