Knowledge Sources

Overview

Knowledge Sources are the foundation of your knowledge system - they represent the actual data that your AI agents can search and reference through collections. Sources can be files uploaded from your computer or live data connections to external systems.

1. Sources Dashboard

Knowledge Sources Interface - Main dashboard showing all your data sources

The sources dashboard (SourcesTable component) provides:

Sources Table: Paginated table showing source name, type, last updated date, tags, and vector status
Inline Editing: Click edit button to modify source names directly in the table
Search Functionality: Search sources by name
Source Actions: Upload Files and Add Live Data buttons in the header
Row Actions: View and Delete options in dropdown menu for each source

2. File Upload Sources

File Upload Process

Knowledge Sources File Upload - Upload documents directly from your computer

Click "Upload Files" from the sources dashboard
File Upload Modal: Opens the FilesModal with tabbed interface
Choose Upload Method:
- File Upload Tab: Individual file selection and upload
- Folder Crawler Tab: Upload multiple files from folder structure

File Upload Tab (`FileUploadTab`)

File Selection:

File Input: Select multiple files using browser file picker
File Parsing: Uses useFileParser hook for content extraction
Progress Tracking: Shows upload progress percentage during processing

File Management:

File List: Preview selected files with names and content
Name Editing: Customize source names before upload (defaults to filename)
Custom Tags: Add additional tags beyond default "File Upload" tag
Tag Management: Add/remove custom tags with tag input field

Upload Configuration:

Automatic Tags: Files automatically get "File Upload" tag plus file extension
Team Association: Sources automatically associated with current team
Database Storage: Content stored in sources table with metadata

3. Live Data Sources

Live Data Integration Process

Knowledge Sources Live Data - Configure real-time data feeds and API integrations

Click "Add Live Data" from the sources dashboard
Live Data Modal: Opens the LiveDataModal with form interface
Select Integration Type: Choose from available live data source types

Available Live Data Types

The system supports several live data integration types:

Support Desk Integration:

Subdomain: Your support desk subdomain
Email: Support desk account email
API Token: Support desk API authentication token
Locale: Language setting (default: en-us)

Web Scraping (WebScrapingForm):

URL: Website URL to scrape
Processing Options: Text extraction settings
Include URLs: Option to include URL metadata
Extract Media Content: Process media elements
Retrieve Links: Extract link information

CMS Integration:

URL: CMS instance URL
API Token: CMS API authentication
Map Configuration: Field mapping settings

Prompt Integration (PromptForm):

Prompt: Custom prompt for data generation
Processing Configuration: Token limits and chunk settings

Bot Logs Integration (BotLogsForm):

Bot Log Configuration: Integration with bot logging systems
Data Extraction: Automated log content processing

4. Source Management Features

Sources Table Interface

Table Columns:

Name: Source name with inline editing capability (click edit icon)
Type: Source type (File, Support Desk, Web Scraping, etc.)
Last Updated: Date when source was last modified
Tags: Visual tags showing source category and metadata
Vector: Indicator showing if source has vector embeddings (Yes/No)

Inline Name Editing

Edit Process:

Start Editing: Click the edit icon next to any source name
Edit Mode: Input field appears with current name
Save/Cancel: Use checkmark to save or X to cancel
Keyboard Shortcuts: Enter to save, Escape to cancel
Database Update: Changes saved to sources table immediately

Tag System

Automatic Tags:

File Upload: Added to all uploaded files
Live Data: Added to all live data integrations
File Extensions: File type extensions added automatically
Custom Tags: User-defined tags during upload

Tag Display:

Color Coding: Blue for "File Upload", Purple for "Live Data"
Visual Design: Rounded pill-style tags with appropriate colors
Tag Management: Tags managed through upload/creation process

5. Source Preview and Actions

Source Preview Interface (`SourcePreview`)

Preview Features:

Source Details: View complete source information and content
Content Display: Full text content of the source
Metadata Viewing: Creation date, last updated, tags, and type information
Source Actions: Edit, update, or manage source settings

Access Method:

Table Click: Click on any source name in the table to open preview
Row Selection: Preview opens in modal or side panel
Source Management: Update source information from preview interface

Source Deletion

Delete Process:

Actions Menu: Click the three-dot menu for any source
Select Delete: Choose delete option from dropdown
Confirmation Dialog: DeleteSourceDialog confirms deletion intent
Database Cleanup: Source removed from sources table
Collection Impact: Source automatically removed from any collections

Delete Confirmation:

Warning Message: Clear indication of permanent deletion
Confirmation Required: User must explicitly confirm action
Cascade Effects: Automatic removal from collections and relationships

Database Structure

Sources Table Schema:

Primary Fields: id, name, type, content, team_id
Metadata Fields: tags, created_at, last_updated, vector
Team Isolation: All sources filtered by current team
Vector Storage: Embeddings stored as JSON in vector field

6. Processing Options and Configuration

Common Processing Options (`ProcessingOptions`)

Text Processing Settings:

Chunk Size: Configure how content is split (default varies by source type)
Chunk Overlap: Set overlap between text chunks for better context
Max Token Limit: Maximum tokens per chunk for API compatibility
Processing Quality: Options for content extraction quality

Live Data Configuration (`CommonFields`)

Standard Configuration Fields:

Name: Descriptive name for the data source
Type: Source type (automatically set based on integration)
Enabled: Toggle to enable/disable data source
URL: Primary endpoint or URL for data retrieval

Processing Configuration:

Advanced Options: Chunk size, overlap, and token limit settings
Content Extraction: Options for media content and link extraction
Update Frequency: How often to check for new data

7. Error Handling and User Feedback

Upload Process Feedback

File Upload Status:

Progress Tracking: Real-time progress bar during file upload
Success Indicators: Confirmation when files are successfully added
Error Messages: Clear error reporting for failed uploads
Validation Feedback: Immediate feedback for missing required fields

Live Data Integration Feedback

Configuration Validation:

Field Validation: Real-time validation of required fields
Connection Testing: Immediate feedback on API connectivity
Configuration Errors: Clear error messages for invalid settings
Success Confirmation: Confirmation when live data source is created

Database Integration

Team Isolation:

Automatic Team Assignment: Sources automatically associated with current team
Access Control: Users can only see sources from their team
Data Security: Team-based filtering ensures data privacy

React Query Integration:

Cache Management: Efficient data loading and cache invalidation
Real-time Updates: Automatic refresh when sources are modified
Performance Optimization: Paginated loading for large source lists

On this page