Text Embedder

Universal Embeddings

Overview

A versatile text embedding component that supports multiple embedding models and provides standardized output format. Ideal for converting text into vector representations for semantic search, clustering, and similarity analysis.

Key Features

Multi-model support
Batch processing
Dimension consistency
Error handling

Supported Models

OpenAI Embeddings
Hugging Face Models
Custom Embedding Models

Configuration

Required Parameters

embeddingModelModel configuration object
messageText to embed

Model Configuration Options

modelTypeType of embedding model
dimensionsOutput vector dimensions
optionsModel-specific options

Example Usage

const embedder = new TextEmbedder({
  embeddingModel: {
    modelType: "openai",
    dimensions: 1536,
    options: {
      model: "text-embedding-ada-002"
    }
  }
});

// Single text embedding
const result = await embedder.embed({
  message: "Your text to embed"
});

// Batch processing
const batchResult = await embedder.embedBatch({
  messages: [
    "First text to embed",
    "Second text to embed",
    "Third text to embed"
  ]
});

// Custom model configuration
const customEmbedder = new TextEmbedder({
  embeddingModel: {
    modelType: "custom",
    dimensions: 768,
    options: {
      modelPath: "path/to/model",
      normalize: true,
      batchSize: 32
    }
  }
});

Best Practices

Use appropriate model dimensions
Implement caching for repeated texts
Handle long texts appropriately

Tips

Normalize vectors when needed
Use batch processing for efficiency
Monitor embedding dimensions

Response Format

{
  "embedding_data": {
    "vector": number[],
    "dimensions": number,
    "model_info": {
      "type": string,
      "name": string,
      "version": string
    }
  },
  "metadata": {
    "input_length": number,
    "processing_time": number,
    "token_count": number
  },
  "status": {
    "success": boolean,
    "error": string | null
  }
}