Connectors

Connectors allow you to read documents from and write converted outputs to various storage systems and databases. Use connectors with the Batch API to process documents at scale.

Available Connectors

Databases

• AstraDB - Write document chunks with embeddings to DataStax AstraDB vector database
• OpenSearch - Write document chunks with embeddings to OpenSearch indexes
• Snowflake - Read documents from Snowflake stage and write document data into Snowflake tables
• Spark - Read from and write to Apache Spark

Object Storage

• Amazon S3 - Read from and write to S3-compatible storage
• Azure Blob Storage - Read from and write to Azure Blob Storage containers
• Google Cloud Storage - Read from and write to Google Cloud Storage buckets
• Google Drive - Read from and write to Google Drive

Enterprise Content Management

• FileNet - Read documents from IBM FileNet Content Manager
• SharePoint - Read from and write to SharePoint sites and OneDrive

Streaming Platforms

• Kafka - Stream document chunks to Apache Kafka topics

Connector Types

Source Connectors

Source connectors read documents for conversion. Multiple source connectors can be specified in a single batch job.

Target Connectors

Target connectors write converted document outputs.

Using Connectors

Connectors are configured as part of the sources and target fields in the Batch API request. See Batch API documentation.

Each connector has specific authentication requirements and configuration parameters. See the individual connector pages for complete setup instructions.