关于Python WebSocket，异步，队列：Python WebSocket，异步，队列-具有生产者和消费者方法的自定义服务器类和处理程序类

Python websockets, asyncio, queue - custom server class and handler class with producer and consumer methods

我在理解异步工作流时遇到问题...

编辑-更改代码以合并asyncio.Queue：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

#!/usr/bin/env python

import asyncio
import websockets
import threading

class WSServer:

def serve_forever(self):
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
start_server = websockets.serve(self.handler, '127.0.0.1', 5678)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

async def handler(self, websocket, path):
loop = asyncio.get_event_loop()
master = MyClass(websocket)
while True:
listener_task = asyncio.ensure_future(master.get_message())
producer_task = asyncio.ensure_future(master.produce())
done, pending = await asyncio.wait(
[listener_task, producer_task],
return_when=asyncio.FIRST_COMPLETED)

if listener_task in done:
await master.consume()
else:
listener_task.cancel()

if producer_task in done:
msg_to_send = producer_task.result()
await master.send_message(msg_to_send)
else:
producer_task.cancel()

class MyClass:

incoming = asyncio.Queue()
outgoing = asyncio.Queue()

def __init__(self, websocket):
self.ws = websocket

async def get_message(self):
msg_in = await self.ws.recv()
await self.incoming.put(msg_in)

async def send_message(self, message):
await self.ws.send(message)

async def consume(self):
msg_to_consume = await self.incoming.get()
# do something 'consuming' :)
consume_output = msg_to_consume
await self.outgoing.put(consume_output)

async def produce(self):
msg_out = await self.outgoing.get()
return msg_out

if __name__ == '__main__':
s = WSServer()
t = threading.Thread(target=s.serve_forever)
t.daemon = True
t.start()
while True:
asyncio.sleep(5)

更改MyClass.consume()时，它可以工作(在一台机器上，而不是在另一台机器上)，但是行为异常：

1
2
3
4
5
6
7
8
9

async def consume(self):
msg_to_consume = await self.incoming.get()
# do something 'consuming' :)
consume_output = msg_to_consume
await self.outgoing.put('THIS WILL NOT GET INTO QUEUE???!!!')
print('Outgoing empty 1: ' + str(self.outgoing.empty()))
# And this will get into queue O.o
await self.outgoing.put(consume_output)
print('Outgoing empty 2: ' + str(self.outgoing.empty()))

我有两个等待，因为在第一次调用self.outgoing.put()之后，self.outgoing队列仍然为空！只有当我再次调用它时，似乎才收到该项目……有什么想法吗？

其他机器只是抛出错误：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

Exception in connection handler
Traceback (most recent call last):
File"/usr/lib/python3/dist-packages/websockets/server.py", line 78, in handler
yield from self.ws_handler(self, path)
File"test2.py", line 33, in handler
msg_to_send = producer_task.result()
File"/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File"/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
result = coro.throw(exc)
File"test2.py", line 66, in produce
msg_out = await self.outgoing.get()
File"/usr/lib/python3.5/asyncio/queues.py", line 168, in get
yield from getter
File"/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
yield self # This tells Task to wait for completion.
RuntimeError: Task <Task pending coro=<MyClass.produce() running at test2.py:66> cb=[_wait.<locals>._on_completion() at /usr/lib/python3.5/asyncio/tasks.py:414]> got Future <Future pending> attached to a different loop

原版的：

我有这段代码，显然不能正常使用:)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

#!/usr/bin/env python

import asyncio
import websockets
import threading

class WSServer:

def serve_forever(self):
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
start_server = websockets.serve(self.handler, '127.0.0.1', 5678)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

async def handler(self, websocket, path):
loop = asyncio.get_event_loop()
master = MyClass(websocket)
while True:
listener_task = asyncio.ensure_future(master.get_message())
producer_task = asyncio.ensure_future(master.produce())
done, pending = await asyncio.wait(
[listener_task, producer_task],
return_when=asyncio.FIRST_COMPLETED)

if listener_task in done:
await master.consume()
else:
listener_task.cancel()

if producer_task in done:
if producer_task.result():
await master.send_message()
else:
producer_task.cancel()

class MyClass:

incoming = []
outgoing = []

def __init__(self, websocket):
self.ws = websocket

async def get_message(self):
self.incoming.append(self.ws.recv())

async def send_message(self):
self.ws.send(self.outgoing.pop(0))

async def consume(self):
self.outgoing.append(self.incoming.pop(0))

async def produce(self):
if self.outgoing:
return True

if __name__ == '__main__':
s = WSServer()
t = threading.Thread(target=s.serve_forever)
t.daemon = True
t.start()
while True:
asyncio.sleep(5)

我想要达到的目标：

使WSServer实例在与主线程不同的线程中运行(与WSServer.serve_forever一起正常工作)

对于每个连接的客户端，在WSServer.handler方法中创建MyClass实例，该实例包含两个列表-一个用于传入消息，另一个用于传出消息。

传入应从MyClass.get_message()-基本上是websocket.recv()

可以从MyClass.consume()中填充传出内容-作为响应，但也可以从此代码范围之外填充。

当MyClass.incoming中有东西时，通过MyClass.consume()处理；当Myclass.outgoing中有东西时，通过MyClass.send_message()处理

我不确定MyClass.produce()，因为我真的不需要产生任何东西，只要有消息就向外发送消息。我也看到了一些使用asycnio.Queue()的代码

我在这里找到了类似的线程，但是老实说，我的理解超出了他们的示例和问题：

在python中的生产者和使用者线程上需要帮助

异步队列消费者协程

什么是正确的方法呢？