• 在IDEA上用python来连接集群上的hive


    1.在使用Python连接hive之前需要将hive中的文件拷贝到自己创建python项目中

    cp -r apache-hive-0.14.0-bin/lib/py  /home/jia/Desktop

    2.把hive上的py目录下载到桌面之后,进入py目录,复制里面所有的文件到你创建的python项目下

    3.新建一个myHiveLink.py文件,访问hive的代码如下

    import sys
    from hive_service import ThriftHive
    from hive_service.ttypes import HiveServerException
    from thrift import Thrift
    from thrift.transport import TSocket
    from thrift.transport import TTransport
    from thrift.protocol import TBinaryProtocol
    
    def hiveExe(sql):
        try:
            transport = TSocket.TSocket('121.8.xxx.xx', 10000)
            transport = TTransport.TBufferedTransport(transport)
            protocol = TBinaryProtocol.TBinaryProtocol(transport)
            client = ThriftHive.Client(protocol)
            transport.open()
    
            client.execute(sql)
    
            print "The return value is : "
            resultSets=client.fetchAll()
            for j in range(len(resultSets)):
                print resultSets[j]
            print "............"
            transport.close()
        except Thrift.TException, tx:
            print '%s' % (tx.message)
    
            return resultSets
    
    if __name__ == '__main__':
        hiveExe("select * from tableName limit 10")

    4.访问结果如下:

    /usr/bin/python2.7 /home/sendi/IdeaProjects/hive_python/.idea/HiveLink/myHiveLink.py
    The return value is : 
    460030087488272    1333280xxxx    2433    16881    20150608005859    20150608005859    NULL    113.4092361    23.1117361    20150608
    460037560630758    1812682xxxx    2817    5920    20150608005850    20150608005858    16    113.39436    23.42839    20150608
    460030991691721    1532152xxxx    2817    31314    20150608005851    20150608005858    16    113.34354    23.28444    20150608
    460036360705126    1537219xxxx   2817    22386    20150608005851    20150608005858    16    113.3470139    23.2713194    20150608
    460036360703708    1532570xxxx   2817    3617    20150608005853    20150608005858    16    113.3468056    23.3133333    20150608
    460036810304576    1533673xxxx   2817    322    20150608005851    20150608005858    16    113.359375    23.2908333    20150608
    460036110265159    1530011xxxx   2817    3409    20150608005854    20150608005858    16    113.3260417    23.2946528    20150608
    460030991632527    1532152xxxx    2817    22384    20150608005854    20150608005858    16    113.3470139    23.2713194    20150608
    460036671118650    1895716xxxx    2817    31360    20150608005853    20150608005858    16    113.35415    23.30307    20150608
    460036360702214    1534570xxxx   2817    22386    20150608005851    20150608005858    16    113.3470139    23.2713194    20150608

    5.如果访问不了,可能是没有启动hive,服务,则进入hive的bin目录启动服务

    hive --service hiveserver &
  • 相关阅读:
    自定义Hooks:四个典型的使用场景
    FreeSql的各种工程demo上新啦
    萌新看过来,你还学不懂VScode插件吗?
    项目7加法运算前后端分离AspNetMvcApi&Vue实现
    张高兴的 Entity Framework Core 即学即用:(一)创建第一个 EF Core 应用
    task5
    Day09参数+递归
    K8S原来如此简单(四)Service+Ingress
    别再写一堆的 for 循环了!Java 8 中的 Stream 轻松遍历树形结构,是真的牛逼!
    task23
  • 原文地址:https://www.cnblogs.com/aijianiula/p/4670010.html
Copyright © 2020-2023  润新知