Class: LLM::Buffer

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/llm/buffer.rb

Overview

LLM::Buffer provides an Enumerable object that yields each message in a conversation on-demand, and only sends a request to the LLM when a response is needed.

Instance Method Summary collapse

Constructor Details

#initialize(provider) ⇒ LLM::Buffer

Parameters:



14
15
16
17
18
 
# File 'lib/llm/buffer.rb', line 14

def initialize(provider)
  @provider = provider
  @pending = []
  @completed = []
end

Instance Method Details

#drain ⇒ Array<LLM::Message> Also known as: flush

Note:

This method is especially useful when using the streaming API.

Drains the buffer and returns all messages as an array

Examples:

llm = LLM.openai(key: ENV["KEY"])
bot = LLM::Bot.new(llm, stream: $stdout)
bot.chat "Hello", role: :user
bot.messages.flush

Returns:

See Also:



116
117
118
 
# File 'lib/llm/buffer.rb', line 116

def drain
  to_a
end

#each {|LLM::Message| ... } ⇒ void

This method returns an undefined value.

Yields:

  • (LLM::Message) —

    Yields each message in the conversation thread

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



25
26
27
28
29
30
31
32
 
# File 'lib/llm/buffer.rb', line 25

def each(...)
  if block_given?
    empty! unless @pending.empty?
    @completed.each { yield(_1) }
  else
    enum_for(:each, ...)
  end
end

#unread ⇒ Array<LLM::Message>

Returns an array of unread messages

Returns:

See Also:



39
40
41
 
# File 'lib/llm/buffer.rb', line 39

def unread
  reject(&:read?)
end

#find ⇒ LLM::Message?

Find a message (in descending order)

Returns:



46
47
48
 
# File 'lib/llm/buffer.rb', line 46

def find(...)
  reverse_each.find(...)
end

#last(n = nil) ⇒ LLM::Message, ...

Returns the last message(s) in the buffer

Parameters:

  • n (Integer, nil) (defaults to: nil) —

    The number of messages to return

Returns:



55
56
57
58
59
60
61
 
# File 'lib/llm/buffer.rb', line 55

def last(n = nil)
  if @pending.empty?
    n.nil? ? @completed.last : @completed.last(n)
  else
    n.nil? ? to_a.last : to_a.last(n)
  end
end

#<<(item) ⇒ void Also known as: push

This method returns an undefined value.

Parameters:

  • item ([LLM::Message, Hash, Symbol]) —

    A message and its parameters



67
68
69
70
 
# File 'lib/llm/buffer.rb', line 67

def <<(item)
  @pending << item
  self
end

#[](index) ⇒ LLM::Message?

Returns a message, or nil

Parameters:

  • index (Integer, Range) —

    The message index

Returns:



78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'lib/llm/buffer.rb', line 78

def [](index)
  if @pending.empty?
    if Range === index
      slice = @completed[index]
      (slice.nil? || slice.size < index.size) ? to_a[index] : slice
    else
      @completed[index]
    end
  else
    to_a[index]
  end
end

#inspect ⇒ String

Returns:

  • (String) 


93
94
95
96
 
# File 'lib/llm/buffer.rb', line 93

def inspect
  "#<#{self.class.name}:0x#{object_id.to_s(16)} " \
  "completed_count=#{@completed.size} pending_count=#{@pending.size}>"
end

#empty? ⇒ Boolean

Returns true when the buffer is empty

Returns:

  • (Boolean) 


101
102
103
 
# File 'lib/llm/buffer.rb', line 101

def empty?
  @pending.empty? and @completed.empty?
end