Search Agent

The Search Agent is an advanced tool that performs intelligent web searches using Google's Custom Search API. It processes search queries, retrieves relevant results, and chunks content for optimal processing and analysis.

Search Agent workflow and architecture

Configuration Parameters

Required Input Parameters

search_query: The search term or phrase to be processed
google_api_key: Your Google API key for authentication
google_cse_id: Google Custom Search Engine ID

Optional Configuration

llm: Language model configuration (default: GPT-4)
search_links: List of specific URLs to search within (optional)
number_of_results: Maximum number of search results to return (default: 10)
chunk_size: Size of text chunks for processing (default: 1000)
chunk_overlap: Overlap between consecutive chunks (default: 200)

Output Format

{
  "search_results": [
    {
      "title": string,
      "link": string,
      "snippet": string,
      "chunks": [
        {
          "content": string,
          "start_index": number,
          "end_index": number
        }
      ],
      "relevance_score": number
    }
  ],
  "metadata": {
    "total_results": number,
    "processing_time": string,
    "query_timestamp": string
  }
}

Features

Advanced web search capabilities
Intelligent content chunking
Relevance scoring
Customizable search parameters
Rate limiting and error handling
Content preprocessing and cleaning

Note: The Google Custom Search API has usage quotas and rate limits. Make sure to check your quota usage and implement appropriate rate limiting in your application.

Tip: Adjust the chunk_size and chunk_overlap parameters based on your specific use case. Larger chunks may provide more context but require more processing resources, while smaller chunks might be more suitable for specific types of analysis.

Example Usage

const searchAgent = new SearchAgent({
  google_api_key: "your-api-key",
  google_cse_id: "your-cse-id",
  chunk_size: 1000,
  chunk_overlap: 200,
  number_of_results: 5
});

const results = await searchAgent.search({
  query: "latest developments in AI",
  search_links: ["https://example.com/ai-news"]
});