admin管理员组

文章数量:1028849

Ascend 910b 运行vllm报错 libatb.so: cannot open shared object file

ascend上运行vllm报错, 说找不到 so库OSError: libatb.so: cannot open shared object file: No such file or directory

代码语言：shell复制
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 12.39it/s]

[rank0]: Traceback (most recent call last):
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch_npu/op_plugin/atb/_atb_ops.py", line 33, in _register_atb_extensions
[rank0]:     torch.ops.load_library(atb_so_path)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/_ops.py", line 1350, in load_library
[rank0]:     ctypes.CDLL(path)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/ctypes/__init__.py", line 374, in __init__
[rank0]:     self._handle = _dlopen(self._name, mode)
[rank0]: OSError: libatb.so: cannot open shared object file: No such file or directory

[rank0]: The above exception was the direct cause of the following exception:

[rank0]: Traceback (most recent call last):
[rank0]:   File "./ascend/example_vllm.py", line 13, in <module>
[rank0]:     llm = LLM(model="Qwen/Qwen2.5-0.5B-Instruct")
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/utils.py", line 1022, in inner
[rank0]:     return fn(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/entrypoints/llm.py", line 242, in __init__
[rank0]:     self.llm_engine = self.engine_class.from_engine_args(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 489, in from_engine_args
[rank0]:     engine = cls(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 276, in __init__
[rank0]:     self._initialize_kv_caches()
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 421, in _initialize_kv_caches
[rank0]:     self.model_executor.determine_num_available_blocks())
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/executor/executor_base.py", line 102, in determine_num_available_blocks
[rank0]:     results = self.collective_rpc("determine_num_available_blocks")
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/executor/uniproc_executor.py", line 56, in collective_rpc
[rank0]:     answer = run_method(self.driver_worker, method, args, kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/utils.py", line 2196, in run_method
[rank0]:     return func(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
[rank0]:     return func(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm_ascend/worker/worker.py", line 227, in determine_num_available_blocks
[rank0]:     self.model_runner.profile_run()
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
[rank0]:     return func(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm_ascend/worker/model_runner.py", line 1360, in profile_run
[rank0]:     self.execute_model(model_input, kv_caches, intermediate_tensors)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
[rank0]:     return func(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm_ascend/worker/model_runner.py", line 1140, in execute_model
[rank0]:     hidden_or_intermediate_states = model_executable(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
[rank0]:     return forward_call(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 486, in forward
[rank0]:     hidden_states = self.model(input_ids, positions, kv_caches,
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/compilation/decorators.py", line 172, in __call__
[rank0]:     return self.forward(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 348, in forward
[rank0]:     hidden_states, residual = layer(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
[rank0]:     return forward_call(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 247, in forward
[rank0]:     hidden_states = self.self_attn(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
[rank0]:     return forward_call(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 178, in forward
[rank0]:     q, k = self.rotary_emb(positions, q, k)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
[rank0]:     return forward_call(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/model_executor/custom_op.py", line 25, in forward
[rank0]:     return self._forward_method(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm_ascend/ops/rotary_embedding.py", line 45, in rope_forward_oot
[rank0]:     torch_npu._npu_rotary_embedding(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch_npu/op_plugin/atb/_atb_ops.py", line 53, in wrapper
[rank0]:     _register_atb_extensions()
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch_npu/op_plugin/atb/_atb_ops.py", line 46, in _register_atb_extensions
[rank0]:     raise nnal_ex from e
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch_npu/op_plugin/atb/_atb_ops.py", line 33, in _register_atb_extensions
[rank0]:     torch.ops.load_library(atb_so_path)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/_ops.py", line 1350, in load_library
[rank0]:     ctypes.CDLL(path)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/ctypes/__init__.py", line 374, in __init__
[rank0]:     self._handle = _dlopen(self._name, mode)
[rank0]: OSError: [Errno None] Please check that the nnal package is installed. Please run 'source set_env.sh' in the NNAL installation path.
[ERROR] 2025-04-24-11:09:16 (PID:2461, Device:0, RankID:-1) ERR99999 UNKNOWN applicaiton exception

这个问题是atb相关的环境变量未设置. 可以使用如下语句设置:

source /usr/local/Ascend/nnal/atb/set_env.sh

如果这个目录不存在, 则是必须的依赖未安装. 正常来说下面的四个都是必须的.

代码语言：shell复制
# 安装Ascend-cann-toolkit_8.0.0_linux-x86_64.run
wget -O Ascend-cann-toolkit_8.0.0_linux-x86_64.run .0.0/Ascend-cann-toolkit_8.0.0_linux-x86_64.run?response-content-type=application/octet-stream
bash Ascend-cann-toolkit_8.0.0_linux-x86_64.run --full --install-for-all --quiet
# 安装Ascend-cann-kernels-910b_8.0.0_linux-x86_64.run 
wget -O Ascend-cann-kernels-910b_8.0.0_linux-x86_64.run .0.0/Ascend-cann-kernels-910b_8.0.0_linux-x86_64.run?response-content-type=application/octet-stream
bash Ascend-cann-kernels-910b_8.0.0_linux-x86_64.run --install --quiet
# 安装Ascend-cann-nnal_8.0.0_linux-x86_64.run
wget -O Ascend-cann-nnal_8.0.0_linux-x86_64.run .0.0/Ascend-cann-nnal_8.0.0_linux-x86_64.run?response-content-type=application/octet-stream
source /usr/local/Ascend/ascend-toolkit/set_env.sh
bash Ascend-cann-nnal_8.0.0_linux-x86_64.run --install --quiet
# 安装 Ascend-mindie_1.0.0_linux-x86_64.run
wget -O Ascend-mindie_1.0.0_linux-x86_64.run .0.0/Ascend-mindie_1.0.0_linux-x86_64.run?response-content-type=application/octet-stream 
source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh
bash Ascend-mindie_1.0.0_linux-x86_64.run --install --quiet

Ascend 910b 运行vllm报错 libatb.so: cannot open shared object file

ascend上运行vllm报错, 说找不到 so库OSError: libatb.so: cannot open shared object file: No such file or directory

代码语言：shell复制
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 12.39it/s]

[rank0]: Traceback (most recent call last):
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch_npu/op_plugin/atb/_atb_ops.py", line 33, in _register_atb_extensions
[rank0]:     torch.ops.load_library(atb_so_path)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/_ops.py", line 1350, in load_library
[rank0]:     ctypes.CDLL(path)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/ctypes/__init__.py", line 374, in __init__
[rank0]:     self._handle = _dlopen(self._name, mode)
[rank0]: OSError: libatb.so: cannot open shared object file: No such file or directory

[rank0]: The above exception was the direct cause of the following exception:

[rank0]: Traceback (most recent call last):
[rank0]:   File "./ascend/example_vllm.py", line 13, in <module>
[rank0]:     llm = LLM(model="Qwen/Qwen2.5-0.5B-Instruct")
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/utils.py", line 1022, in inner
[rank0]:     return fn(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/entrypoints/llm.py", line 242, in __init__
[rank0]:     self.llm_engine = self.engine_class.from_engine_args(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 489, in from_engine_args
[rank0]:     engine = cls(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 276, in __init__
[rank0]:     self._initialize_kv_caches()
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 421, in _initialize_kv_caches
[rank0]:     self.model_executor.determine_num_available_blocks())
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/executor/executor_base.py", line 102, in determine_num_available_blocks
[rank0]:     results = self.collective_rpc("determine_num_available_blocks")
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/executor/uniproc_executor.py", line 56, in collective_rpc
[rank0]:     answer = run_method(self.driver_worker, method, args, kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/utils.py", line 2196, in run_method
[rank0]:     return func(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
[rank0]:     return func(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm_ascend/worker/worker.py", line 227, in determine_num_available_blocks
[rank0]:     self.model_runner.profile_run()
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
[rank0]:     return func(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm_ascend/worker/model_runner.py", line 1360, in profile_run
[rank0]:     self.execute_model(model_input, kv_caches, intermediate_tensors)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
[rank0]:     return func(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm_ascend/worker/model_runner.py", line 1140, in execute_model
[rank0]:     hidden_or_intermediate_states = model_executable(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
[rank0]:     return forward_call(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 486, in forward
[rank0]:     hidden_states = self.model(input_ids, positions, kv_caches,
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/compilation/decorators.py", line 172, in __call__
[rank0]:     return self.forward(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 348, in forward
[rank0]:     hidden_states, residual = layer(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
[rank0]:     return forward_call(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 247, in forward
[rank0]:     hidden_states = self.self_attn(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
[rank0]:     return forward_call(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 178, in forward
[rank0]:     q, k = self.rotary_emb(positions, q, k)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
[rank0]:     return self._call_impl(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
[rank0]:     return forward_call(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm/model_executor/custom_op.py", line 25, in forward
[rank0]:     return self._forward_method(*args, **kwargs)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/vllm_ascend/ops/rotary_embedding.py", line 45, in rope_forward_oot
[rank0]:     torch_npu._npu_rotary_embedding(
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch_npu/op_plugin/atb/_atb_ops.py", line 53, in wrapper
[rank0]:     _register_atb_extensions()
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch_npu/op_plugin/atb/_atb_ops.py", line 46, in _register_atb_extensions
[rank0]:     raise nnal_ex from e
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch_npu/op_plugin/atb/_atb_ops.py", line 33, in _register_atb_extensions
[rank0]:     torch.ops.load_library(atb_so_path)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/site-packages/torch/_ops.py", line 1350, in load_library
[rank0]:     ctypes.CDLL(path)
[rank0]:   File "/data/miniconda3/envs/ascend-3.10.14/lib/python3.10/ctypes/__init__.py", line 374, in __init__
[rank0]:     self._handle = _dlopen(self._name, mode)
[rank0]: OSError: [Errno None] Please check that the nnal package is installed. Please run 'source set_env.sh' in the NNAL installation path.
[ERROR] 2025-04-24-11:09:16 (PID:2461, Device:0, RankID:-1) ERR99999 UNKNOWN applicaiton exception

这个问题是atb相关的环境变量未设置. 可以使用如下语句设置:

source /usr/local/Ascend/nnal/atb/set_env.sh

如果这个目录不存在, 则是必须的依赖未安装. 正常来说下面的四个都是必须的.

代码语言：shell复制
# 安装Ascend-cann-toolkit_8.0.0_linux-x86_64.run
wget -O Ascend-cann-toolkit_8.0.0_linux-x86_64.run .0.0/Ascend-cann-toolkit_8.0.0_linux-x86_64.run?response-content-type=application/octet-stream
bash Ascend-cann-toolkit_8.0.0_linux-x86_64.run --full --install-for-all --quiet
# 安装Ascend-cann-kernels-910b_8.0.0_linux-x86_64.run 
wget -O Ascend-cann-kernels-910b_8.0.0_linux-x86_64.run .0.0/Ascend-cann-kernels-910b_8.0.0_linux-x86_64.run?response-content-type=application/octet-stream
bash Ascend-cann-kernels-910b_8.0.0_linux-x86_64.run --install --quiet
# 安装Ascend-cann-nnal_8.0.0_linux-x86_64.run
wget -O Ascend-cann-nnal_8.0.0_linux-x86_64.run .0.0/Ascend-cann-nnal_8.0.0_linux-x86_64.run?response-content-type=application/octet-stream
source /usr/local/Ascend/ascend-toolkit/set_env.sh
bash Ascend-cann-nnal_8.0.0_linux-x86_64.run --install --quiet
# 安装 Ascend-mindie_1.0.0_linux-x86_64.run
wget -O Ascend-mindie_1.0.0_linux-x86_64.run .0.0/Ascend-mindie_1.0.0_linux-x86_64.run?response-content-type=application/octet-stream 
source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh
bash Ascend-mindie_1.0.0_linux-x86_64.run --install --quiet

本文标签： Ascend 910b 运行vllm报错 libatbso cannot open shared object file

版权声明：本文标题：Ascend 910b 运行vllm报错 libatb.so: cannot open shared object file 内容由热心网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：http://it.en369.cn/jiaocheng/1747547260a2174136.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。