Support Knowledge Base

Unable to Run Llama_chat Python* Demo from OpenVINO™ Model Server (OVMS)

Content Type: Error Messages | Article ID: 000097983 | Last Reviewed: 10/04/2024

Description Resolution

Description

Unable to run a Large Language Model (LLM) like LLaMa2 from Meta on OVMS.
Encountered errors when running llama_chat Python* Demo from OVMS GitHub* repository.

Deprecated llama_chat Python* Demo from OVMS GitHub* repository.

Build the llm_text_generation Python* Demo which uses the new MediaPipe Python* calculator feature that makes it easier to save the llama model.

This article applies to 1 products.

OpenVINO™ toolkit

Contact support