2023年7月26日发(作者：)

python：去重（list，dataframe）1.对列表去重1.1.⽤for或while去重1.2.⽤集合的特性set()>>> l = [1,4,3,3,4,2,3,4,5,6,1]>>> type(l)>>> set(l){1, 2, 3, 4, 5, 6}>>> res = list(set(l))>>> res[1, 2, 3, 4, 5, 6]1.3.使⽤itertools模块的grouby⽅法>>> li2 = [1,4,3,3,4,2,3,4,5,6,1]>>> () # 排序>>> it = y(li2)>>> for k, g in it:... print (k)...1234561.4.使⽤keys()⽅式>>> li4 = [1,0,3,7,7,5]>>> {}.fromkeys(li4){1: None, 0: None, 3: None, 7: None, 5: None}>>> {}.fromkeys(li4).keys()dict_keys([1, 0, 3, 7, 5])>>> list({}.fromkeys(li4).keys())[1, 0, 3, 7, 5]1.5.使⽤unique对于⼀维数组或者列表，unique函数去除其中重复的元素，并按元素由⼤到⼩返回⼀个新的⽆元素重复的元组或者列表return_index=True：返回新列表a=[1 2 3 4 5]中每个元素在原列表A = [1 2 5 3 4 3]中第⼀次出现的索引值return_inverse=True：返回原列表A = [1 2 5 3 4 3]中每个元素在新列表a=[1 2 3 4 5]中的索引值>>> A = [1, 2, 5, 3, 4, 3]>>> a, s, p = (A, return_index=True, return_inverse=True)>>> print ("新列表：",a)新列表： [1 2 3 4 5]>>> print ("return_index", s)return_index [0 1 3 4 2]>>> print ("return_inverse", p)return_inverse [0 1 4 2 3 2]2.对数据框去重2.1.⽤unique()对单属性列去重>>> import pandas as pd>>> data = {'id':['A','B','C','C','C','A','B','C','A'],'age':[18,20,14,10,50,14,65,14,98]}>>> data = ame(data)>>> ()array(['A', 'B', 'C'], dtype=object)###或者>>> ()array(['A', 'B', 'C'], dtype=object)2.2.⽤_duplicates()对单属性列去重>>> _duplicates(['id']) id age0 A 181 B 202 C 142.3.⽤_duplicates()对多属性列去重>>> _duplicates(['id','age']) id age0 A 181 B 202 C 143 C 104 C 505 A 146 B 658 A 982.4.⽤ated()对多属性列去重>>> isduplicated = ated(['id','age'],keep='first')>>> [~isduplicated,:] id age0 A 181 B 202 C 143 C 104 C 505 A 146 B 658 A 98>>> [isduplicated,:] id age7 C 14