Data integration for real-time marketing in Customer Insights — Journeys

A journey responds to events. A high-quality journey responds to the right events at the right moments. That depends on data flowing into Customer Insights — Journeys from across the marketing and operational stack: ecommerce platforms, web analytics, mobile apps, transaction systems, support cases. Designing the data integration is half the work of building a marketing programme.

Event types.

• System events — built into Customer Insights — Journeys (email opened, link clicked, form submitted).
• Dataverse events — record changes in any Dataverse table.
• Custom triggers — externally defined events emitted from any source.

Custom triggers are the integration extension point — anything that happens in any system can trigger a journey if it emits a custom trigger.

Custom trigger anatomy.

• Trigger definition — name, schema of properties.
• Properties — typed fields (string, number, datetime, etc.).
• Audience binding — how the trigger identifies the recipient.
• Description — human-readable purpose.

Example: a "Cart Abandoned" trigger with properties cartValue, cartItems, abandonedAt, identified by customerId mapping to a contact in Customer Insights.

Emitting a custom trigger.

• Power Automate — Send a custom event action.
• REST API — direct POST to the journey API.
• Dataverse plug-in — fires from server-side logic.
• Azure Function — fires from custom event source.

The pattern: source system emits trigger → journey processes immediately.

Authentication for trigger emission.

• API key (simpler, less secure).
• Service principal OAuth (preferred for production).
• Power Automate uses connector credentials.

For high-volume emissions, rate limiting applies; design for graceful backoff.

Unified profile from Customer Insights — Data. When CI-Data is paired with CI-Journeys:

• CI-Data unifies customer profiles across multiple sources (CRM, ecommerce, transactional).
• The unified profile is the audience for journeys.
• Profile attributes are usable in personalisation.

Without CI-Data, CI-Journeys uses Dataverse contacts as the audience — simpler but less powerful for cross-system unification.

Common integration patterns.

• Ecommerce → CI-Journeys — order placed, cart abandoned, browsed but not purchased.
• Mobile app → CI-Journeys — app opened, feature used, in-app conversion.
• Support system → CI-Journeys — case opened, case resolved, NPS score.
• CRM (Dynamics or Salesforce) → CI-Journeys — opportunity stage change, contract signed.
• Web analytics → CI-Journeys — page viewed, video watched, content downloaded.

Each integration is a source emitting custom triggers, mapped to the unified profile.

Profile enrichment. Beyond triggers, profile data flows from sources:

• Static attributes — name, email, country.
• Behavioural — engagement scores, recent activity.
• Transactional — total spend, average order value.
• Inferred — segments, propensity scores.

CI-Data computes these; CI-Journeys consumes them.

Real-time vs batch.

• Real-time triggers — event arrives, journey starts within seconds.
• Batch updates — profile attributes updated via daily refresh.

Both are needed: real-time for event-driven engagement; batch for profile maintenance.

Webhooks for ingestion. A common pattern:

• External system POSTs a webhook on event.
• Azure Function receives, validates, transforms.
• Azure Function calls CI-Journeys API with the custom trigger.

The Azure Function provides:

• Authentication abstraction.
• Schema validation.
• Idempotency.
• Error handling and retry.

Building this once and reusing for all integration points is more maintainable than per-source custom code.

Identity resolution. A challenge: matching event-source identity to the unified profile.

• Source identifies by email → email match.
• Source identifies by external ID → mapping table.
• Anonymous events → cookie / device ID → eventual profile association.

CI-Data has identity resolution logic; CI-Journeys benefits from it via the unified profile.

Schema management. Custom triggers have schemas:

• Versioning — when adding properties, version the trigger.
• Required vs optional — required properties block emission if missing.
• Documentation — what each property means, what valid values are.

Without governance, schemas drift; emitters and journeys get out of sync.

Volume and throttling.

• Custom trigger emission has rate limits.
• High-volume scenarios need batching or queuing.
• Marketing-grade volume (millions of events per day) needs careful architecture.

Common pitfalls.

• No deduplication. Same event emitted twice; recipient gets two journey instances.
• Identity mismatch. Event has email but unified profile has phone; no association; journey doesn't fire.
• Schema drift. Emitter changes property; journey breaks silently.
• No monitoring. Events missing; journeys not firing; nobody notices until campaign metrics drop.
• Privacy missing. Event includes personal data without proper consent; compliance issue.
• Source-side failures cascade. Source system down; events lost; backfill complex.

Privacy considerations. Every event flowing into CI-Journeys may contain personal data:

• Consent — recipient consented to processing.
• Retention — events retained per policy.
• Purpose limitation — events used only for the consented purposes.
• Subject access requests — events included in DSARs.

Build privacy into integration from day one; retrofit is hard.

Observability.

• Trigger emission logs — count, success rate, latency.
• Journey entry counts — vs triggers emitted.
• Failure investigation — when journeys don't fire as expected.

Strategic positioning. Marketing automation's value compounds with data quality and integration breadth. A journey responding to one event is fine; a journey responding to a coordinated set of behavioural and transactional events across the customer lifecycle is powerful. The investment in clean, reliable, observable data integration is what makes marketing programmes scale beyond simple email campaigns into orchestrated customer experiences.