Class: LLM::Tracer

Inherits:
Object
  • Object
show all
Defined in:
lib/llm/tracer.rb

Overview

The LLM::Tracer is the superclass of all LLM tracers. It can be helpful for implementing instrumentation and hooking into the lifecycle of an LLM request. See LLM::Tracer::Telemetry, and LLM::Tracer::Logger for example tracer implementations.

Direct Known Subclasses

Logger, Null, Telemetry

Defined Under Namespace

Classes: Langsmith, Logger, Null, Telemetry

Constant Summary collapse

FINISH_METADATA_PROC_KEY =
:"llm.tracer.finish_metadata_proc"

Instance Method Summary collapse

Constructor Details

#initialize(provider, options = {}) ⇒ Tracer

Returns a new instance of Tracer.

Parameters:

  • provider (LLM::Provider) —

    A provider

  • options (Hash) (defaults to: {}) —

    A hash of options



22
23
24
25
 
# File 'lib/llm/tracer.rb', line 22

def initialize(provider, options = {})
  @provider = provider
  @options = {}
end

Instance Method Details

#consume_request_metadata ⇒ Hash

Consume and clear per-request metadata. Called by the telemetry tracer at span start.

Returns:

  • (Hash)


228
229
230
231
232
233
 
# File 'lib/llm/tracer.rb', line 228

def consume_request_metadata
  key = thread_request_metadata_key
  data = thread[key] || {}
  thread[key] = nil
  data
end

#on_request_start(operation:, model: nil, inputs: nil) ⇒ void

This method returns an undefined value.

Called before an LLM provider request is executed.

Parameters:

  • operation (String)
  • model (String) (defaults to: nil)
  • inputs (Hash, nil) (defaults to: nil) —

    Optional span attributes (e.g. gen_ai.input.messages) from llm.rb or caller.

Raises:

  • (NotImplementedError)


33
34
35
 
# File 'lib/llm/tracer.rb', line 33

def on_request_start(operation:, model: nil, inputs: nil)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_request_finish(operation:, res:, model: nil, span: nil, outputs: nil, metadata: nil) ⇒ void

This method returns an undefined value.

Called after an LLM provider request succeeds.

Parameters:

  • operation (String)
  • res (LLM::Response)
  • span (Object, nil) (defaults to: nil)
  • model (String) (defaults to: nil)
  • outputs (Hash, nil) (defaults to: nil) —

    Optional span attributes (e.g. gen_ai.output.messages) from llm.rb or caller.

  • metadata (Hash, nil) (defaults to: nil) —

    Optional metadata (emitted as langsmith.metadata.*) from llm.rb or caller.

Raises:

  • (NotImplementedError)


46
47
48
 
# File 'lib/llm/tracer.rb', line 46

def on_request_finish(operation:, res:, model: nil, span: nil, outputs: nil, metadata: nil)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_request_error(ex:, span:) ⇒ void

This method returns an undefined value.

Called when an LLM provider request fails.

Parameters:

Raises:

  • (NotImplementedError)


55
56
57
 
# File 'lib/llm/tracer.rb', line 55

def on_request_error(ex:, span:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_tool_start(id:, name:, arguments:, model:) ⇒ void

This method returns an undefined value.

Called before a local tool/function executes.

Parameters:

  • id (String) —

    The tool call ID assigned by the model/provider

  • name (String) —

    The tool (function) name.

  • arguments (Hash) —

    The parsed tool arguments.

  • model (String) —

    The model name

Raises:

  • (NotImplementedError)


70
71
72
 
# File 'lib/llm/tracer.rb', line 70

def on_tool_start(id:, name:, arguments:, model:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_tool_finish(result:, span:) ⇒ void

This method returns an undefined value.

Called after a local tool/function succeeds.

Parameters:

Raises:

  • (NotImplementedError)


81
82
83
 
# File 'lib/llm/tracer.rb', line 81

def on_tool_finish(result:, span:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_tool_error(ex:, span:) ⇒ void

This method returns an undefined value.

Called when a local tool/function raises.

Parameters:

  • ex (Exception) —

    The raised error.

  • span (Object, nil) —

    The span/context object returned by #on_tool_start.

Raises:

  • (NotImplementedError)


92
93
94
 
# File 'lib/llm/tracer.rb', line 92

def on_tool_error(ex:, span:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil) ⇒ self

Opens a trace group so subsequent LLM spans share the same OpenTelemetry trace_id (and appear as one trace in backends like Langfuse). When +trace_group_id+ is a string, it is used to derive the trace_id.

Parameters:

  • trace_group_id (String, nil) (defaults to: nil) —

    Optional. When present, converted to a 16-byte trace_id so all spans created until #stop_trace are grouped in one trace.

  • name (String) (defaults to: "llm") —

    Name for the root span (e.g. "chatbot.turn").

  • attributes (Hash) (defaults to: {}) —

    OpenTelemetry attributes to set on the root span.

  • metadata (Hash, nil) (defaults to: nil) —

    Optional. Trace-level metadata merged into the trace (e.g. langsmith.metadata.*). Only used by tracers that support it (e.g. Langsmith).

Returns:

  • (self)


112
113
114
 
# File 'lib/llm/tracer.rb', line 112

def start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil)
  self
end

#stop_trace ⇒ self

Finishes the trace group started by #start_trace. Safe to call even if no trace is active.

Returns:

  • (self)


120
121
122
 
# File 'lib/llm/tracer.rb', line 120

def stop_trace
  self
end

#inspect ⇒ String

Returns:

  • (String)


126
127
128
 
# File 'lib/llm/tracer.rb', line 126

def inspect
  "#<#{self.class.name}:0x#{object_id.to_s(16)} @provider=#{@provider.class} @tracer=#{@tracer.inspect}>"
end

#spans ⇒ Array

Returns:

  • (Array)


132
133
134
 
# File 'lib/llm/tracer.rb', line 132

def spans
  []
end

#flush! ⇒ nil

Note:

This method is only implemented by the Telemetry tracer. It is a noop for other tracers.

Flush the tracer

Returns:

  • (nil)


142
143
144
 
# File 'lib/llm/tracer.rb', line 142

def flush!
  nil
end

#merge_extra(metadata: nil, inputs: nil, outputs: nil) ⇒ self

Merges extra attributes for the current trace/span. Used by applications (e.g. chatbot) to add metadata, span inputs, or span outputs to the next span or to the trace. No-op by default; Langsmith merges into fiber-local storage and emits them as langsmith/GenAI attributes.

Parameters:

  • metadata (Hash, nil) (defaults to: nil) —

    Key-value pairs merged into trace/span metadata (e.g. langsmith.metadata.*).

  • inputs (Hash, nil) (defaults to: nil) —

    Key-value pairs set on the next span at start (e.g. gen_ai.input.messages). Consumed when the span is created.

  • outputs (Hash, nil) (defaults to: nil) —

    Key-value pairs set on the current span at finish (e.g. gen_ai.output.messages). Must be set before the request finishes (e.g. in a block passed to the provider).

Returns:

  • (self)


161
162
163
 
# File 'lib/llm/tracer.rb', line 161

def merge_extra(metadata: nil, inputs: nil, outputs: nil)
  self
end

#set_finish_metadata_proc(proc) ⇒ self

Optional: set a proc to supply metadata when the next chat span finishes. The proc is called with the response (res) and should return a Hash of metadata (e.g. { intent: "...", confidence: 1.0 }) to merge onto the span as langsmith.metadata.*. Cleared after use. Used by apps to attach routing/intent that is only known after the response.

Parameters:

  • proc (Proc, nil) —

    (res) -> Hash or nil

Returns:

  • (self)


174
175
176
177
 
# File 'lib/llm/tracer.rb', line 174

def set_finish_metadata_proc(proc)
  thread[FINISH_METADATA_PROC_KEY] = proc
  self
end

#current_extra ⇒ Hash

Returns the current extra bag (metadata, inputs, outputs) for the current thread/trace. Used by subclasses; default returns empty hashes.

Returns:

  • (Hash) —

    { metadata: {}, inputs: {}, outputs: {} }



186
187
188
 
# File 'lib/llm/tracer.rb', line 186

def current_extra
  {}
end

#consume_extra_inputs ⇒ Hash

Returns and clears extra inputs for the next span. Called by the telemetry tracer when starting a span. Subclasses (e.g. Langsmith) override to return fiber-local inputs; default returns {}.

Returns:

  • (Hash) —

    Attribute key => value to set on the span at start



196
197
198
 
# File 'lib/llm/tracer.rb', line 196

def consume_extra_inputs
  {}
end

#consume_extra_outputs ⇒ Hash

Returns and clears extra outputs for the current span. Called by the telemetry tracer when finishing a span. Subclasses override to return fiber-local outputs; default returns {}.

Returns:

  • (Hash) —

    Attribute key => value to set on the span at finish



206
207
208
 
# File 'lib/llm/tracer.rb', line 206

def consume_extra_outputs
  {}
end

#set_request_metadata(metadata) ⇒ nil

Store per-request metadata (e.g. user_input) to be consumed by tracers when starting the next span. Used for plain-text input.value / output.value.

Parameters:

  • metadata (Hash) —

    e.g. { user_input: "the user question" }

Returns:

  • (nil)


216
217
218
219
220
221
222
 
# File 'lib/llm/tracer.rb', line 216

def set_request_metadata(metadata)
  return nil unless metadata && !metadata.empty?
  key = thread_request_metadata_key
  current = thread[key] || {}
  thread[key] = current.merge(metadata.compact)
  nil
end