pip install six
pip install bit_array
pip install thriftpy
pip install sasl-0.2.1-cp36-cp36m-win32.whl
pip install thrift_sasl
pip install bitarray-0.8.3-cp36-cp36m-win32.whl
pip install impyla
pip install pandas
测试代码
from impala.dbapi import connect
from impala.util import as_pandas
conn = connect(host='10.86.45.40', port=21050)
cur = conn.cursor()
cur.execute('SHOW TABLES')
data = as_pandas(cur)
print(data)
cur.execute('SELECT * FROM geelylogdatawarehouse limit 10')
cur.execute('SELECT row_number() over (order by appid) as rnum, * FROM geelylogdatawarehouse where ymd = 20171230 order by appid limit 10 offset 1000000')
cur.execute('SELECT count(*) FROM geelylogdatawarehouse where ymd = 20171230')
data = cur.fetchall()`
print(data)
description = cur.description
print(description)
for row in cur:
print(row[4])
data = as_pandas(cur)
print(data)
报错:
thriftpy.parser.exc.ThriftParserError: ThriftPy does not support generating module with path in protocol 'c'
解决:
根据提示找到parser.py文件 488行 if url_scheme == '': 修改为 if len(url_scheme) <= 1:
- whl文件需要到https://www.lfd.uci.edu/~gohlke/pythonlibs/下载
或者使用附件里的版本 - centos下:sudo yum install gcc-c++ python-devel.x86_64 cyrus-sasl-devel.x86_64 然后pip install sasl
- 如果直接安装impyla则会报Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual
C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools
- whl文件需要到https://www.lfd.uci.edu/~gohlke/pythonlibs/下载