关于python：用于保存json数据的多个URL


Multiple URLs to save json data

我尝试一次调用多个(超过10个URL)并保存所有这些10个URL的数据，这些数据将采用JSON格式，并尝试保存在我的位置。

下面是我尝试过的代码，使用它我只能获取保存在我的JSON文件中的最后一个URL的数据。如何获取所有URL的数据并存储在单个JSON文件中？

1
2
3
4
5
6
7
8
9
10
11
12
13
import json
import requests

URLs = ['http://httpbin.org/ip',
'http://httpbin.org/user-agent',
'http://httpbin.org/headers']

json_list = []
for url in URLs:
    data = requests.get(url)
    resolvedwo = data.json()
    with open('resolvedworesolution.json', 'w') as f:
         json.dump(resolvedwo, f)
相关讨论

您的问题是，每次在循环中都要覆盖该文件。相反，将循环结果存储在一个列表中，并只将其写入文件一次

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import requests
import json

URLs = ['http://httpbin.org/ip',
        'http://httpbin.org/user-agent',
        'http://httpbin.org/headers']

json_list = []

for url in URLs:
    data = requests.get(url)
    resolvedwo = data.json()
    json_list.append(resolvedwo)

with open('resolvedworesolution.json', 'w+') as f:
    json.dump(json_list, f, sort_keys=True, indent=4)

输出：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[
    {
       "origin":"137.221.143.66, 137.221.143.66"
    },
    {
       "user-agent":"python-requests/2.21.0"
    },
    {
       "headers": {
           "Accept":"*/*",
           "Accept-Encoding":"gzip, deflate",
           "Host":"httpbin.org",
           "User-Agent":"python-requests/2.21.0"
        }
    }
]
相关讨论

您可以将信息存储在可作为整体序列化的对象中：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import json
import requests

URLs = ['http://httpbin.org/ip',
'http://httpbin.org/user-agent',
'http://httpbin.org/headers']

json_list = []
for url in URLs:
    data = requests.get(url)
    resolvedwo = data.json()
    json_list.append(resolvedwo)

with open('resolvedworesolution.json', 'w+') as f:
    json.dump(json_list, f)
相关讨论

在写入文件时，以w模式打开文件将在写入之前擦除/截断内容。

1
with open('resolvedworesolution.json', 'a') as f:

那应该能解决你的问题

相关讨论

写入文件时使用附加模式以"保留"现有数据：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import json
import requests
URLs = ['http://httpbin.org/ip',
'http://httpbin.org/user-agent',
'http://httpbin.org/headers']

json_list = []
for url in URLs:
    data = requests.get(url)
    resolvedwo = data.json()
    with open('resolvedworesolution.json', 'a') as f:   # Using the append mode
        json.dump(resolvedwo, f)
        f.write("
")                                   # new line for readability

输出：

1
2
3
{"origin":"159.122.207.241, 159.122.207.241"}
{"user-agent":"python-requests/2.21.0"}
{"headers": {"Accept":"*/*","Accept-Encoding":"gzip, deflate","Host":"httpbin.org","User-Agent":"python-requests/2.21.0"}}

编辑：

您可以一次性将响应写入文件：

1
2
3
4
with open('resolvedworesolution.json', 'a') as f:
    f.write(str(resolvedwo))
    f.write("
")

或

1
2
3
4
5
6
for url in URLs:
    data = requests.get(url)
    with open('resolvedworesolution.json', 'a') as f:
        f.write(data.text)
        f.write("
")
相关讨论

而不是：

1
resolvedwo = data.json()

你可能想要：

1
resolvedwo += data.json()
相关讨论