关于python:IOError:[Errno 2]没有这样的文件或目录:’hdfs:///data/testdata.json’

IOError: [Errno 2] No such file or directory: 'hdfs:///data/testdata.json'

New to python coding, getting following error

I can view that testdata.json' that this location using

hdfs dfs -ls /data/testdata.json'

Traceback (most recent call last):
File"testdata.json'", line 6, in
with open('hdfs:///data/testdata.json') as data_file:
IOError: [Errno 2] No such file or directory: 'hdfs:///data/testdata.json'
python process_sensor_file.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
#!/bin/python
import json
from pprint import pprint

with open('hdfs:///data/testdata.json',"r") as data_file:
     source_data = json.load(data_file)
print(source_data)

print(json.dumps(source_data, indent=2))

for item in source_data['CityData']:
            Longitude = item['Longitude']
            TimeStamp = item['TimeStamp']
            print(Longitude, TimeStamp)


您需要一个hdfs驱动程序,以便python能够从hdfs(如hdfs3读取数据。

来自文档:

1
2
3
4
from hdfs3 import HDFileSystem
hdfs = HDFileSystem(host='localhost', port=8020)
with hdfs.open('/data/testdata.json') as f:
    data = f.read(1000000)