• 有一组列表套列表的数据,如果根据指定的多列判断去重?


    排除index的干扰,比较其他列,如果数据重复则删除后面出现的。

    In [1]: new_list_of_list = [
       ...:     ['8834769542251520',  0, '金跃跃', '9860000100001', '董事长、总经理'],
       ...:     ['8834769542251520',  1, '王晓慧', '9860000200001', '董事、副董事长、副总经理'],
       ...:     ['8834769542251520',  2, '戚海平', '9860000300001', '董事、财务总监、董事会秘书'],
       ...:     ['8834769542251520',  3, '陈榴生', '9860000400001', '董事'],
       ...:     ['8834769542251520',  4, '刘浩', '9860000500001', '董事'],
       ...:     ['8834769542251520',  5, '陈榴生', '9860000400001', '董事'],
       ...:     ['8834769542251520',  6, '刘浩', '9860000500001', '董事']
       ...:     ]
    
    # 转换为pandas进行处理
    In [2]: import pandas as pd
    
    # pandas赋予列名
    In [3]: df_new_list_of_list = pd.DataFrame(new_list_of_list, columns=["wenjian_id", "index", "name", "report_type", "duty"])
    
    # 指定去重需要参考的列名
    In [6]: df_new_list_of_list = df_new_list_of_list.drop_duplicates(subset=["wenjian_id", "name", "report_type", "duty"],keep="first")
    
    In [7]: df_new_list_of_list
    Out[7]:
             wenjian_id  index name    report_type           duty
    0  8834769542251520      0  金跃跃  9860000100001        董事长、总经理
    1  8834769542251520      1  王晓慧  9860000200001   董事、副董事长、副总经理
    2  8834769542251520      2  戚海平  9860000300001  董事、财务总监、董事会秘书
    3  8834769542251520      3  陈榴生  9860000400001             董事
    4  8834769542251520      4   刘浩  9860000500001             董事
    
  • 相关阅读:
    [SQL Basics] Indexes
    [DS Basics] Data structures
    [Algorithm Basics] Search
    [Algorithm Basics] Sorting, LinkedList
    [Java Basics] multi-threading
    [Java Basics] Reflection
    [Java Basics] Collection
    SSH basics
    纯css实现三角形
    2015年百度实习生前端笔试题上海卷a
  • 原文地址:https://www.cnblogs.com/everfight/p/drop_duplicates.html
Copyright © 2020-2023  润新知