Class: LLM::Tracer::Telemetry

Inherits:
LLM::Tracer show all
Defined in:
lib/llm/tracer/telemetry.rb

Overview

The LLM::Tracer::Telemetry tracer provides telemetry support through the opentelemetry-ruby RubyGem. The gem should be installed separately since this feature is opt-in and disabled by default.

Examples:

InMemory export

#!/usr/bin/env ruby
require "llm"
require "pp"

llm = LLM.openai(key: ENV["KEY"])
llm.tracer = LLM::Tracer::Telemetry.new(llm)

ses = LLM::Session.new(llm)
ses.talk "hello"
ses.talk "how are you?"
ses.tracer.spans.each { |span| pp span }

OTLP export

#!/usr/bin/env ruby
require "llm"
require "opentelemetry-exporter-otlp"

endpoint = "https://api.smith.langchain.com/otel/v1/traces"
exporter = OpenTelemetry::Exporter::OTLP::Exporter.new(endpoint:)

llm = LLM.openai(key: ENV["KEY"])
llm.tracer = LLM::Tracer::Telemetry.new(llm, exporter:)

ses = LLM::Session.new(llm)
ses.talk "hello"
ses.talk "how are you?"

See Also:

Direct Known Subclasses

Langsmith

Constant Summary

Constants inherited from LLM::Tracer

FINISH_METADATA_PROC_KEY

Instance Method Summary collapse

Methods inherited from LLM::Tracer

#consume_extra_inputs, #consume_extra_outputs, #consume_request_metadata, #current_extra, #inspect, #merge_extra, #set_finish_metadata_proc, #set_request_metadata

Constructor Details

#initialize(provider, options = {}) ⇒ LLM::Tracer::Telemetry

param [LLM::Provider] provider An LLM provider



47
48
49
50
51
 
# File 'lib/llm/tracer/telemetry.rb', line 47

def initialize(provider, options = {})
  super
  @exporter = options.delete(:exporter)
  setup!
end

Instance Method Details

#flush! ⇒ nil

Note:

Exports are batched in the background by default. Long-lived processes usually do not need to call this method. Short-lived scripts should call #flush! before exit to reduce the risk of losing spans that are still buffered.

Flushes queued telemetry to the configured exporter.

Returns:

  • (nil)


188
189
190
191
 
# File 'lib/llm/tracer/telemetry.rb', line 188

def flush!
  @tracer_provider.force_flush
  nil
end

#start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil) ⇒ self

When +trace_group_id+ is provided, it is converted to an OpenTelemetry trace_id (via a deterministic 16-byte hash) so all spans until #stop_trace share that trace_id and appear as one trace in OTLP/Langfuse.

Parameters:

  • trace_group_id (String, nil) (defaults to: nil) —

    Optional. When present, converted to a 16-byte trace_id so all spans created until #stop_trace are grouped in one trace.

  • name (String) (defaults to: "llm") —

    Name for the root span (e.g. "chatbot.turn").

  • attributes (Hash) (defaults to: {}) —

    OpenTelemetry attributes to set on the root span.

  • metadata (Hash, nil) (defaults to: nil) —

    Optional. Trace-level metadata merged into the trace (e.g. langsmith.metadata.*). Only used by tracers that support it (e.g. Langsmith).

Returns:

  • (self)


60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/llm/tracer/telemetry.rb', line 60

def start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil)
  return self if trace_group_id.to_s.empty?

  span_context = span_context_from_trace_group_id(trace_group_id.to_s)
  parent_ctx = ::OpenTelemetry::Trace.context_with_span(
    ::OpenTelemetry::Trace.non_recording_span(span_context)
  )
  attrs = attributes.compact
  attrs["llm.trace_group_id"] = trace_group_id.to_s
  root_span = @tracer.start_span(
    name,
    kind: :server,
    attributes: attrs,
    with_parent: parent_ctx
  )
  @root_span = root_span
  @root_context = ::OpenTelemetry::Trace.context_with_span(root_span)
  self
end

#stop_trace ⇒ self

Returns:

  • (self)


82
83
84
85
86
87
 
# File 'lib/llm/tracer/telemetry.rb', line 82

def stop_trace
  @root_span&.finish
  @root_span = nil
  @root_context = nil
  self
end

#on_request_start(operation:, model: nil, inputs: nil) ⇒ Object

Parameters:

  • operation (String)
  • model (String) (defaults to: nil)
  • inputs (Hash, nil) (defaults to: nil) —

    Optional span attributes (e.g. gen_ai.input.messages) from llm.rb or caller.



91
92
93
94
95
96
97
 
# File 'lib/llm/tracer/telemetry.rb', line 91

def on_request_start(operation:, model: nil, inputs: nil)
  case operation
  when "chat" then start_chat(operation:, model:, inputs:)
  when "retrieval" then start_retrieval(operation:)
  else nil
  end
end

#on_request_finish(operation:, res:, model: nil, span: nil, outputs: nil, metadata: nil) ⇒ Object

Parameters:

  • operation (String)
  • res (LLM::Response)
  • span (Object, nil) (defaults to: nil)
  • model (String) (defaults to: nil)
  • outputs (Hash, nil) (defaults to: nil) —

    Optional span attributes (e.g. gen_ai.output.messages) from llm.rb or caller.

  • metadata (Hash, nil) (defaults to: nil) —

    Optional metadata (emitted as langsmith.metadata.*) from llm.rb or caller.



101
102
103
104
105
106
107
108
 
# File 'lib/llm/tracer/telemetry.rb', line 101

def on_request_finish(operation:, res:, model: nil, span: nil, outputs: nil, metadata: nil)
  return nil unless span
  case operation
  when "chat" then finish_chat(operation:, model:, res:, span:, outputs:, metadata:)
  when "retrieval" then finish_retrieval(operation:, res:, span:)
  else nil
  end
end

#on_request_error(ex:, span:) ⇒ Object

Parameters:



112
113
114
115
116
117
118
119
 
# File 'lib/llm/tracer/telemetry.rb', line 112

def on_request_error(ex:, span:)
  return nil unless span
  attributes = {"error.type" => ex.class.to_s}.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.request.finish")
  span.status = ::OpenTelemetry::Trace::Status.error(ex.message)
  span.tap(&:finish)
end

#on_tool_start(id:, name:, arguments:, model:) ⇒ void

This method returns an undefined value.

Parameters:

  • id (String) —

    The tool call ID assigned by the model/provider

  • name (String) —

    The tool (function) name.

  • arguments (Hash) —

    The parsed tool arguments.

  • model (String) —

    The model name



124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/llm/tracer/telemetry.rb', line 124

def on_tool_start(id:, name:, arguments:, model:)
  attributes = {
    "gen_ai.operation.name" => "execute_tool",
    "gen_ai.request.model" => model,
    "gen_ai.tool.call.id" => id,
    "gen_ai.tool.name" => name,
    "gen_ai.tool.call.arguments" => LLM.json.dump(arguments),
    "gen_ai.provider.name" => provider_name,
    "server.address" => provider_host,
    "server.port" => provider_port
  }.merge!(trace_attributes(span_kind: "tool")).compact
  span_name = ["execute_tool", name].compact.join(" ")
  span = create_span(span_name.empty? ? "gen_ai.tool" : span_name, attributes:)
  span.add_event("gen_ai.tool.start")
  span
end

#on_tool_finish(result:, span:) ⇒ void

This method returns an undefined value.

Parameters:



144
145
146
147
148
149
150
151
152
153
154
 
# File 'lib/llm/tracer/telemetry.rb', line 144

def on_tool_finish(result:, span:)
  return nil unless span
  attributes = {
    "gen_ai.tool.call.id" => result.id,
    "gen_ai.tool.name" => result.name,
    "gen_ai.tool.call.result" => LLM.json.dump(result.value)
  }.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.tool.finish")
  span.tap(&:finish)
end

#on_tool_error(ex:, span:) ⇒ void

This method returns an undefined value.

Parameters:

  • ex (Exception) —

    The raised error.

  • span (Object, nil) —

    The span/context object returned by #on_tool_start.



159
160
161
162
163
164
165
166
 
# File 'lib/llm/tracer/telemetry.rb', line 159

def on_tool_error(ex:, span:)
  return nil unless span
  attributes = {"error.type" => ex.class.to_s}.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.tool.finish")
  span.status = ::OpenTelemetry::Trace::Status.error(ex.message)
  span.tap(&:finish)
end

#spans ⇒ Array<OpenTelemetry::SDK::Trace::SpanData>

Note:

This method returns an empty array for exporters that do not implement 'finished_spans' such as the OTLP exporter

Returns:

  • (Array<OpenTelemetry::SDK::Trace::SpanData>)


174
175
176
177
178
 
# File 'lib/llm/tracer/telemetry.rb', line 174

def spans
  return [] unless @exporter.respond_to?(:finished_spans)
  flush!
  @exporter.finished_spans
end