SFTP / FTPS
File-based export over a secure legacy transfer protocol, still required by many enterprise partners and financial institutions that don't accept cloud-storage delivery.
High-Level Design
SFTP/FTPS exists for the partners who genuinely cannot receive data any other way.
💼 Business Context
- Many enterprise, banking, and healthcare partners mandate SFTP/FTPS for compliance or legacy-integration reasons — cloud-bucket delivery is simply not an accepted option for them
- Keeping this path well-supported avoids losing otherwise-viable partnerships over a transport-protocol requirement
- Owned by Data Engineering / Partner Integrations
🔌 Technical Overview
The same export pipeline used for S3/GCS/Azure Blob generates the file, but delivery is via SSH.NET-based SFTP (or FTPS where SFTP isn't supported) to a partner-managed server, running as a Docker container job on Azure Container Apps Jobs with credentials rotated through Azure Key Vault. Because SFTP servers vary widely in reliability, this path includes stricter retry-with-backoff and a dead-letter alert if delivery fails after the maximum retry window.
Protocol Support
💾 SFTP Delivery Config
{
"partner": "acme_financial",
"protocol": "sftp",
"host": "sftp.acmefinancial.example",
"remote_path": "/inbound/cxos/",
"encryption": "pgp",
"max_retries": 5
}
🔗 Integration Points
- Query & Analytics Engine — executes the export query, same as other batch destinations
- Azure Key Vault — SFTP credentials and PGP keys
- Azure Container Apps Jobs — runs the scheduled delivery job
- Alerts & Notifications — pages the owning team on repeated delivery failure
🧰 Services Consumed
- Owning microservice —
Cxos.Connectors.BatchExport(see the Full Application Service Map) - No dedicated database — stateless connector (see Platform Connectors above)
⚠️ Non-Functional Considerations
- Scale: per-partner file sizes are typically modest (single-digit GB), sized to the partner's own ingestion capability, not the lakehouse
- Latency: batch by design, on a defined schedule agreed with each partner
- Reliability: stricter retry/backoff than cloud-storage delivery, since partner-managed SFTP servers are outside CXOS's reliability guarantees
- Security/Privacy: PGP file-level encryption is used in addition to transport encryption for the most sensitive partner exports (e.g., financial data)
🎯 Enterprise Example
A banking partner's compliance team only accepts PGP-encrypted files over SFTP. The export pipeline reuses the same query and file-generation logic as every other batch destination, differing only in the delivery and encryption step — avoiding a duplicate pipeline just for one partner's transport requirement.