IOError: [Errno 2] No such file or directory: 'hdfs:///data/testdata.json'
New to python coding, getting following error
I can view that testdata.json' that this location using
hdfs dfs -ls /data/testdata.json'
Traceback (most recent call last):
File"testdata.json'", line 6, in
with open('hdfs:///data/testdata.json') as data_file:
IOError: [Errno 2] No such file or directory: 'hdfs:///data/testdata.json'
python process_sensor_file.py
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | #!/bin/python import json from pprint import pprint with open('hdfs:///data/testdata.json',"r") as data_file: source_data = json.load(data_file) print(source_data) print(json.dumps(source_data, indent=2)) for item in source_data['CityData']: Longitude = item['Longitude'] TimeStamp = item['TimeStamp'] print(Longitude, TimeStamp) |
您需要一个hdfs驱动程序,以便python能够从hdfs(如
来自文档:
1 2 3 4 | from hdfs3 import HDFileSystem hdfs = HDFileSystem(host='localhost', port=8020) with hdfs.open('/data/testdata.json') as f: data = f.read(1000000) |