Attachment External Processing in Conversational Bot

The Buzzeasy Conversational Bot provides three different ways to handle attachments: None, Bot (LLM), and External. This guide focuses on the External processing option, which allows you to send attachments to an external service for specialized processing.

Attachment Processing Options

• None: The attachment will not be processed and the bot will reply that the attachments are not supported.
• Bot (LLM): The attachment will be processed by the LLM if the attachment type is image. If the attachment type is not image, the bot will reply that the attachment is not supported.
• External: The attachment will be sent to the external URL specified in the Conversational Bot node configuration. The external service will process the attachment and return a response to the bot, which then replies based on the conversation content and the response from the external service.

Configuration

To set up external attachment processing, you need to configure the following in your Conversational Bot node:

External Service URL

Set the URL for the external service in the Conversational Bot node configuration. This URL will be used to send the attachment for processing via HTTP POST requests.

API Key Authentication

Set an API key in the Conversational Bot node configuration. This API key will be sent in the request header x-api-key: <api_key> to authenticate the request with the external service.

How It Works

1. Attachment Upload: When a customer uploads an attachment during the conversation, and the bot is configured for external processing, Buzzeasy will send the attachment to your external service.

2. Processing Request: Buzzeasy sends a HTTP POST request to your configured endpoint with the BotProcessAttachmentRequest payload containing attachment details and conversation metadata.

3. External Processing: Your external service processes the attachment (e.g., OCR for documents, image analysis, PDF text extraction) and returns the processed results.

4. Bot Response: The bot receives the processed data and uses it to continue the conversation with the customer based on the extracted information.

Request Payload (BotProcessAttachmentRequest)

The BotProcessAttachmentRequest represents the incoming request from Buzzeasy to your endpoint:

public class BotProcessAttachmentRequest
{
    public required string ConversationId { get; set; }
    public string? CustomerIdentifier { get; set; }
    public string? CustomerId { get; set; }
    public MediaGroup MediaGroup { get; set; }
    public Dictionary<string, string> WorkflowData { get; set; } = new();
    public required string AttachmentUrl { get; set; }
    public required string AttachmentType { get; set; }
    public string? AttachmentName { get; set; }
}

Properties:

• ConversationId (required): A unique identifier for the ongoing conversation.
• CustomerIdentifier (optional): A string that may represent the customer's identity (e.g., email, phone, account ID).
• CustomerId (optional): A unique customer ID, if available.
• MediaGroup (required): Indicates the media channel (e.g., Chat, Email, Voice) through which the conversation is happening.
• WorkflowData (required): A dictionary containing current workflow data key-value pairs that have accumulated so far.
• AttachmentUrl (required): The URL where the attachment can be downloaded for processing.
• AttachmentType (required): The MIME type of the attachment (e.g., "application/pdf", "image/jpeg", "image/png").
• AttachmentName (optional): The original filename of the attachment.

Example Request JSON Schema

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "ConversationId": {
      "type": "string"
    },
    "CustomerIdentifier": {
      "type": ["string", "null"]
    },
    "CustomerId": {
      "type": ["string", "null"]
    },
    "MediaGroup": {
      "type": "string"
    },
    "WorkflowData": {
      "type": "object",
      "additionalProperties": {
        "type": "string"
      }
    },
    "AttachmentUrl": {
      "type": "string"
    },
    "AttachmentType": {
      "type": "string"
    },
    "AttachmentName": {
      "type": ["string", "null"]
    }
  },
  "required": ["ConversationId", "MediaGroup", "WorkflowData", "AttachmentUrl", "AttachmentType"]
}

Example Request Payload

{
  "ConversationId": "conv-1234-abc",
  "CustomerIdentifier": "jane.smith@example.com",
  "CustomerId": "cust-456",
  "MediaGroup": "Chat",
  "WorkflowData": {
    "CustomerType": "Premium"
  },
  "AttachmentUrl": "https://buzzeasy.com/attachments/invoice-123.pdf",
  "AttachmentType": "application/pdf",
  "AttachmentName": "invoice-123.pdf"
}

Response Payload (BotProcessAttachmentResponse)

Your endpoint should return a BotProcessAttachmentResponse:

public class BotProcessAttachmentResponse
{
   public Dictionary<string, string>? WorkflowData { get; set; }
   public required JsonObject Result { get; set; }
}

Properties:

• WorkflowData (optional): Any updates or new key-value pairs to be merged back into the Buzzeasy conversation workflow data.
• Result (required): A JSON object that contains the text for the LLM to process. This can be PDF text, OCR text, or any other text that the external service returns after processing the attachment.

Example Response JSON Schema

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "WorkflowData": {
      "type": ["object", "null"],
      "additionalProperties": {
        "type": "string"
      }
    },
    "Result": {
      "type": "object"
    }
  },
  "required": ["Result"]
}

Example Response Payload

{
  "WorkflowData": {
    "DocumentType": "Invoice",
    "InvoiceNumber": "INV-2024-001"
  },
  "Result": {
    "text": "Invoice Number: INV-2024-001\nDate: June 15, 2024\nAmount: $1,250.00\nDue Date: July 15, 2024\nCustomer: Acme Corporation\nItems:\n- Consulting Services: $1,000.00\n- Software License: $250.00",
    "extractedFields": {
      "invoiceNumber": "INV-2024-001",
      "amount": "1250.00",
      "dueDate": "2024-07-15",
      "customer": "Acme Corporation"
    }
  }
}

Use Cases

External attachment processing is particularly useful for:

• Document Analysis: Extract text and data from PDFs, invoices, contracts, or forms
• Image Processing: Perform OCR on scanned documents or analyze image content
• File Validation: Verify document authenticity or check compliance requirements
• Data Extraction: Extract structured data from unstructured attachments
• Content Moderation: Scan attachments for inappropriate content

Error Handling

If your external service encounters an error while processing the attachment, you should return an HTTP 400 (Bad Request) status code and error message. The bot will handle these errors gracefully and inform the customer that the attachment could not be processed.

Summary

External attachment processing extends the Conversational Bot's capabilities by allowing specialized processing of various file types through your custom external services. This enables sophisticated document analysis, data extraction, and content processing that can significantly enhance the bot's ability to assist customers with document-related queries.