Overview
The Ingest Salesforce connector reads from your Salesforce org over the REST API using OAuth2 JWT Bearer Flow — a server-to-server authentication pattern that does not require a browser-based user-consent step. You'll create a "connected app" inside Salesforce, upload a public certificate, and provide three values to Ingest: the connected app's consumer key, your matching private key, and the username Ingest should impersonate.
This is the recommended pattern for data integrations because the access token does not rotate the way user OAuth refresh tokens do, and the integration user can be assigned the minimal permission set without affecting any human user's permissions.
Setup guide
Generate a key pair
On any machine with OpenSSL:
openssl genrsa -out salesforce.key 2048 openssl req -new -x509 -days 36500 -key salesforce.key -out salesforce.crt -subj "/CN=ingest"
Keep salesforce.key private. Upload salesforce.crt to Salesforce in the next step.
Create the connected app
- In Salesforce, go to Setup → App Manager → New Connected App.
- Fill in Connected App Name, API Name, and Contact Email.
- Check Enable OAuth Settings.
- Set the Callback URL to
https://login.salesforce.com/services/oauth2/success(it isn't used in the JWT flow but is required by Salesforce). - Check Use digital signatures and upload the
salesforce.crtfile from step 1. - Selected OAuth scopes: Manage user data via APIs (api), Perform requests at any time (refresh_token, offline_access).
- Click Save, then Continue.
- After save, copy the Consumer Key (also called Client ID).
Pre-authorize the connected app
The JWT flow requires the connected app to be pre-authorized for the integration user:
- Setup → Manage Connected Apps → Edit the app you just created.
- Set Permitted Users to Admin approved users are pre-authorized. Save.
- Add the integration user (or a profile / permission set the user has) to the connected app.
Find your instance URL
- Setup → My Domain. Copy the Current My Domain URL without the
https://and trailing slash — e.g.acme.my.salesforce.com. This is thesalesforce_instance_urlvalue the connector needs at runtime.
Hand the values to Ingest
In the Ingest UI under Connectors → Salesforce, fill in:
- Salesforce instance URL — the my-domain host from step 4 (e.g.
acme.my.salesforce.com). - Consumer Key — from step 2 (stored in AWS Secrets Manager as
consumer_key). - Private Key — the contents of
salesforce.key, including the-----BEGIN PRIVATE KEY-----lines (stored asprivate_key). - Username — the Salesforce username of the integration user Ingest should run as (stored as
username).
Mind the limits
Salesforce enforces a per-org daily API request limit that depends on your Edition and licensed user count — Enterprise typically gets 100,000+ calls/day; Professional Edition is 1,000 calls/day per org. The Ingest runtime dispatches at 2 req/sec by default and uses AIMD backoff on 429s. Errors with status 401 (bad/expired token, missing pre-authorization) and 403 (scope missing or feature not licensed) are treated as fatal — fix the cause before retrying.
Pagination is nextRecordsUrl-based: every 2,000 records, the response carries a fully-qualified follow-up URL the runtime calls until the result set is exhausted.
Pick endpoints
The connector uses SOQL queries internally — every endpoint is the same /services/data/v60.0/query URL with a different SOQL body. Most customers want:
- CRM core:
account,contact,lead,opportunity,case - Opportunity detail:
opportunity_line_item,opportunity_contact_role,opportunity_history - Sales catalog:
product,pricebook,pricebook_entry,contract,quote,quote_line_item,sales_order,order_item - Marketing:
campaign,campaign_member - Activity:
task,event - People & permissions:
user,user_role,profile,permission_set,permission_set_assignment,field_permissions,object_permissions,record_type - Asset / org metadata:
asset,organization
Supported streams
30 endpoints are available out of the box. Each endpoint syncs into its own Iceberg table in Snowflake.
| Endpoint | Description | Reference |
|---|---|---|
| account account | – | |
| asset asset | – | |
| campaign campaign | – | |
| campaign_member campaign_member | – | |
| case case | – | |
| contact contact | – | |
| contract contract | – | |
| event event | – | |
| field_permissions field_permissions | – | |
| lead lead | – | |
| object_permissions object_permissions | – | |
| opportunity opportunity | – | |
| opportunity_contact_role opportunity_contact_role | – | |
| opportunity_history opportunity_history | – | |
| opportunity_line_item opportunity_line_item | – | |
| order_item order_item | – | |
| organization organization | – | |
| permission_set permission_set | – | |
| permission_set_assignment permission_set_assignment | – | |
| pricebook pricebook | – | |
| pricebook_entry pricebook_entry | – | |
| product product | – | |
| profile profile | – | |
| quote quote | – | |
| quote_line_item quote_line_item | – | |
| record_type record_type | – | |
| sales_order sales_order | – | |
| task task | – | |
| user user | – | |
| user_role user_role | – |
Authentication
- Auth type
- OAuth 2.0
- Provider docs
- developer.salesforce.com ↗
Performance & limits
- Rate limit
- Per-org daily API call cap (1,000 / 5,000 / 100,000+ depending on edition and license count). Ingest dispatches at 2 req/sec by default; AIMD backoff handles 429s.