Zero-copy segmentation from Databricks overview
Learn how zero-copy segmentation evaluates segment rules in Databricks using profile and interaction data.
Zero-copy segmentation from Databricks extends segmentation in Reltio Intelligent 360 to interaction and transaction data stored in Databricks. Reltio Data Pipeline for Databricks publishes mastered entity data into a target schema in the Databricks catalog configured for the pipeline. The catalog is governed by Unity Catalog. Interaction and transaction data remains in your mapped Databricks tables or views.
During segment evaluation, Databricks evaluates profile attributes, interaction attributes, or both. Reltio uses the matching entity identifiers to update the profiles included in the segment.
The following roles configure zero-copy segmentation from Databricks:
How zero-copy segmentation works
| Sequence | Stage | Action and result |
|---|---|---|
| 1 | Pipeline configuration | Configure Reltio Data Pipeline for Databricks to use Unity Catalog, and specify the catalog that stores the segmentation data. |
| 2 | Entity publication | Run the data pipeline. On its first successful run, the pipeline creates a target schema in the configured catalog and publishes mastered entity data to the entities table in that schema. |
| 3 | Segment definition | Create a segment rule in Reltio Intelligent 360. The rule uses enabled profile attributes and, when required, interaction attributes from a mapped Databricks table or view. |
| 4 | Segment refresh | Refresh the segment manually or through its schedule. Reltio converts the rule into SQL and submits the query to the configured Databricks SQL warehouse. |
| 5 | Query execution | The query reads <catalog>.<target_schema>.entities. When the rule includes interaction attributes, the query joins the mapped Databricks table or view. |
| 6 | Results | Databricks writes the matching entity identifiers to <catalog>.<target_schema>.segment_results. Reltio uses those identifiers to update the segment. |
Attributes available for segment rules
Segment rules use profile attributes, interaction attributes, or both. The following table identifies the source of each attribute type and its configuration requirements.
| Data | Source | Configuration requirement |
|---|---|---|
| Profile attributes | The <catalog>.<target_schema>.entities table published by Reltio Data Pipeline for Databricks | Enable each entity attribute for segmentation in the L3 configuration. |
| Interaction attributes | Interaction and transaction data in a mapped Databricks table or view | Create an interaction mapping and grant the service principal access to the source object. |
Databricks configuration
Zero-copy segmentation requires Databricks to contain the published Reltio entity data, provide the SQL warehouse that evaluates segment rules, and grant the service principal access to source and result objects. Review the required state for each configuration area.
The following table lists the required configuration for each area:
| Configuration area | Required configuration |
|---|---|
| Data pipeline storage | Reltio Data Pipeline for Databricks uses Unity Catalog and a configured target catalog. |
| Published entity data | The pipeline has storageFormat set to JSON and splitTable set to false, and has completed a successful run that created the target schema and entities table. |
| Databricks connection | The connection identifies the workspace, SQL warehouse, and segmentation catalog. The segmentationCatalog value matches the catalog configured for Reltio Data Pipeline for Databricks, and the connection has passed the connection test. |
| Service principal access | The service principal has access to the SQL warehouse and the required Unity Catalog privileges to read source data and create or use segment_results. |
Segment evaluation and refresh
Reltio evaluates zero-copy segments in batch. Define a schedule for periodic evaluation, or start a manual evaluation to refresh results outside the schedule. Each evaluation runs the segment rule in Databricks and updates the segment in Reltio.
Configure and verify the required Databricks access before the first evaluation, as described in Configure Databricks permissions for zero-copy segmentation. If setup or evaluation fails, use Troubleshoot zero-copy segmentation from Databricks to identify the cause.