Class: LLM::LlamaCpp
Overview
The LlamaCpp class implements a provider for
llama.cpp
through the OpenAI-compatible API provided by the
llama-server binary.
Constant Summary
Constants inherited
from OpenAI
OpenAI::HOST
Instance Method Summary
collapse
Methods inherited from OpenAI
#assistant_role, #complete, #embed, #models
Methods inherited from Provider
#assistant_role, #chat, #chat!, #complete, #embed, #inspect, #models, #respond, #respond!, #schema, #with
Constructor Details
#initialize(host: "localhost", port: 8080, ssl: false) ⇒ LLM::LlamaCpp
15
16
17
|
# File 'lib/llm/providers/llamacpp.rb', line 15
def initialize(host: "localhost", port: 8080, ssl: false, **)
super
end
|
Instance Method Details
33
34
35
|
# File 'lib/llm/providers/llamacpp.rb', line 33
def audio
raise NotImplementedError
end
|
21
22
23
|
# File 'lib/llm/providers/llamacpp.rb', line 21
def files
raise NotImplementedError
end
|
27
28
29
|
# File 'lib/llm/providers/llamacpp.rb', line 27
def images
raise NotImplementedError
end
|
#moderations ⇒ Object
39
40
41
|
# File 'lib/llm/providers/llamacpp.rb', line 39
def moderations
raise NotImplementedError
end
|
#responses ⇒ Object
45
46
47
|
# File 'lib/llm/providers/llamacpp.rb', line 45
def responses
raise NotImplementedError
end
|
#default_model ⇒ String
Returns the default model for chat completions
53
54
55
|
# File 'lib/llm/providers/llamacpp.rb', line 53
def default_model
"qwen3"
end
|