Open
Description
What Operating System(s) are you seeing this problem on?
macOS (Apple Silicon)
Backend.AI version
main
Describe the bug
DockerError(404, 'network not found') occurs when deleting multi kernel sessions.
2023-04-11 16:52:25.683 DEBUG ai.backend.agent.server [32323] rpc::destroy_local_network(name:bai-singlenode-HASH)
2023-04-11 16:52:25.684 ERROR ai.backend.agent.server [32323] unexpected error
Traceback (most recent call last):
File "/Users/sh/bai-dev/src/ai/backend/agent/server.py", line 157, in _inner
return await meth(
^^^^^^^^^^^^^^^^^
File "/Users/sh/bai-dev/src/ai/backend/agent/server.py", line 133, in _inner
return await meth(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^
File "/Users/sh/bai-dev/src/ai/backend/agent/server.py", line 623, in destroy_local_network
return await self.agent.destroy_local_network(network_name)
^^^^^^^^^^^^^^^^^
File "/Users/sh/bai-dev/src/ai/backend/agent/docker/agent.py", line 1439, in destroy_local_network
network = await docker.networks.get(network_name)
^^^^^^^^^^^^^^^^^
File "/Users/sh/bai-dev/dist/export/python/virtualenvs/python-default/3.11.2/lib/python3.11/site-packages/aiodocker/networks.py", line 21, in get
data = await self.docker._query_json(
^^^^^^^^^^^^^^^^^
File "/Users/sh/bai-dev/dist/export/python/virtualenvs/python-default/3.11.2/lib/python3.11/site-packages/aiodocker/docker.py", line 300, in _query_json
async with self._query(
^^^^^^^^^^^^^^^^^
File "/Users/sh/bai-dev/dist/export/python/virtualenvs/python-default/3.11.2/lib/python3.11/site-packages/aiodocker/utils.py", line 309, in __aenter__
resp = await self._coro
^^^^^^^^^^^^^^^^^
File "/Users/sh/bai-dev/dist/export/python/virtualenvs/python-default/3.11.2/lib/python3.11/site-packages/aiodocker/docker.py", line 275, in _do_query
raise DockerError(response.status, json.loads(what.decode("utf8")))
^^^^^^^^^^^^^^^^^
aiodocker.exceptions.DockerError: DockerError(404, 'network bai-singlenode-HASH not found')To Reproduce
Just simply create a multi-kernel session and destroy it.
Expected Behavior
the number of attempt to delete the docker network should be one, not multiple times.And although such error occurs, we should handle the error.
Anything else?
No response