Class: LLM::Tracer::Telemetry

Inherits:
LLM::Tracer show all
Defined in:
lib/llm/tracer/telemetry.rb

Overview

The LLM::Tracer::Telemetry tracer provides telemetry support through the opentelemetry-ruby RubyGem. The gem should be installed separately since this feature is opt-in and disabled by default. This feature exists to support integration with tools like LangSmith.

Examples:

InMemory export

#!/usr/bin/env ruby
require "llm"
require "pp"

llm = LLM.openai(key: ENV["KEY"])
llm.tracer = LLM::Tracer::Telemetry.new(llm)

ses = LLM::Session.new(llm)
ses.talk "hello"
ses.talk "how are you?"
ses.tracer.spans.each { |span| pp span }

OTLP export

#!/usr/bin/env ruby
require "llm"
require "opentelemetry-exporter-otlp"

endpoint = "https://api.smith.langchain.com/otel/v1/traces"
exporter = OpenTelemetry::Exporter::OTLP::Exporter.new(endpoint:)

llm = LLM.openai(key: ENV["KEY"])
llm.tracer = LLM::Tracer::Telemetry.new(llm, exporter:)

ses = LLM::Session.new(llm)
ses.talk "hello"
ses.talk "how are you?"

See Also:

Instance Method Summary collapse

Methods inherited from LLM::Tracer

#inspect

Constructor Details

#initialize(provider, options = {}) ⇒ LLM::Tracer::Telemetry

param [LLM::Provider] provider An LLM provider



46
47
48
49
50
 
# File 'lib/llm/tracer/telemetry.rb', line 46

def initialize(provider, options = {})
  super
  @exporter = options.delete(:exporter)
  setup!
end

Instance Method Details

#flush! ⇒ nil

Note:

Exports are batched in the background by default. Long-lived processes usually do not need to call this method. Short-lived scripts should call #flush! before exit to reduce the risk of losing spans that are still buffered.

Flushes queued telemetry to the configured exporter.

Returns:

  • (nil)


151
152
153
154
 
# File 'lib/llm/tracer/telemetry.rb', line 151

def flush!
  @tracer_provider.force_flush
  nil
end

#on_request_start(operation:, model: nil) ⇒ Object

Parameters:

  • operation (String)
  • model (String) (defaults to: nil)


54
55
56
57
58
59
60
 
# File 'lib/llm/tracer/telemetry.rb', line 54

def on_request_start(operation:, model: nil)
  case operation
  when "chat" then start_chat(operation:, model:)
  when "retrieval" then start_retrieval(operation:)
  else nil
  end
end

#on_request_finish(operation:, res:, model: nil, span: nil) ⇒ Object

Parameters:

  • operation (String)
  • model (String) (defaults to: nil)
  • res (LLM::Response)
  • span (Object, nil) (defaults to: nil)


64
65
66
67
68
69
70
71
 
# File 'lib/llm/tracer/telemetry.rb', line 64

def on_request_finish(operation:, res:, model: nil, span: nil)
  return nil unless span
  case operation
  when "chat" then finish_chat(operation:, model:, res:, span:)
  when "retrieval" then finish_retrieval(operation:, res:, span:)
  else nil
  end
end

#on_request_error(ex:, span:) ⇒ Object

Parameters:



75
76
77
78
79
80
81
82
 
# File 'lib/llm/tracer/telemetry.rb', line 75

def on_request_error(ex:, span:)
  return nil unless span
  attributes = {"error.type" => ex.class.to_s}.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.request.finish")
  span.status = ::OpenTelemetry::Trace::Status.error(ex.message)
  span.tap(&:finish)
end

#on_tool_start(id:, name:, arguments:, model:) ⇒ void

This method returns an undefined value.

Parameters:

  • id (String) —

    The tool call ID assigned by the model/provider

  • name (String) —

    The tool (function) name.

  • arguments (Hash) —

    The parsed tool arguments.

  • model (String) —

    The model name



87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
 
# File 'lib/llm/tracer/telemetry.rb', line 87

def on_tool_start(id:, name:, arguments:, model:)
  attributes = {
    "gen_ai.operation.name" => "execute_tool",
    "gen_ai.request.model" => model,
    "gen_ai.tool.call.id" => id,
    "gen_ai.tool.name" => name,
    "gen_ai.tool.call.arguments" => LLM.json.dump(arguments),
    "gen_ai.provider.name" => provider_name,
    "server.address" => provider_host,
    "server.port" => provider_port
  }.compact
  span_name = ["execute_tool", name].compact.join(" ")
  span = @tracer.start_span(span_name.empty? ? "gen_ai.tool" : span_name, kind: :client, attributes:)
  span.add_event("gen_ai.tool.start")
  span
end

#on_tool_finish(result:, span:) ⇒ void

This method returns an undefined value.

Parameters:



107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/llm/tracer/telemetry.rb', line 107

def on_tool_finish(result:, span:)
  return nil unless span
  attributes = {
    "gen_ai.tool.call.id" => result.id,
    "gen_ai.tool.name" => result.name,
    "gen_ai.tool.call.result" => LLM.json.dump(result.value)
  }.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.tool.finish")
  span.tap(&:finish)
end

#on_tool_error(ex:, span:) ⇒ void

This method returns an undefined value.

Parameters:

  • ex (Exception) —

    The raised error.

  • span (Object, nil) —

    The span/context object returned by #on_tool_start.



122
123
124
125
126
127
128
129
 
# File 'lib/llm/tracer/telemetry.rb', line 122

def on_tool_error(ex:, span:)
  return nil unless span
  attributes = {"error.type" => ex.class.to_s}.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.tool.finish")
  span.status = ::OpenTelemetry::Trace::Status.error(ex.message)
  span.tap(&:finish)
end

#spans ⇒ Array<OpenTelemetry::SDK::Trace::SpanData>

Note:

This method returns an empty array for exporters that do not implement 'finished_spans' such as the OTLP exporter

Returns:

  • (Array<OpenTelemetry::SDK::Trace::SpanData>)


137
138
139
140
141
 
# File 'lib/llm/tracer/telemetry.rb', line 137

def spans
  return [] unless @exporter.respond_to?(:finished_spans)
  flush!
  @exporter.finished_spans
end