Event-driven architecture for Dynamics 365

A traditional integration calls Dynamics 365 API when data is needed; an event-driven integration is notified when Dynamics 365 changes happen. The shift from request-response to event-driven enables loose coupling, scalability, and real-time reactivity. For complex multi-system landscapes, event-driven architecture (EDA) often outperforms point-to-point integration.

Request-response vs event-driven.

• Request-response — caller pulls; tight coupling; synchronous.
• Event-driven — source pushes; loose coupling; asynchronous.

Different problem domains favour different approaches; understanding both helps.

Event-driven components.

• Event source — produces events (Dynamics, F&O, external systems).
• Event broker — routes events (Event Grid, Service Bus).
• Event consumer — reacts to events.
• Event store — historical record.

Each component does one job; clean separation.

Event types from Dynamics 365.

• Record changes — Create, Update, Delete in Dataverse.
• Business events — F&O business events (workflow completed, etc.).
• Custom events — emitted by plug-ins or workflows.
• System events — login, error, etc.

Rich event sources; selecting which to publish is configuration choice.

Brokers and patterns.

• Azure Event Grid — publish-subscribe; many subscribers per event.
• Azure Service Bus — queues and topics; reliable delivery.
• Apache Kafka / Confluent — high-throughput streaming.
• Microsoft Fabric Eventstream — for analytical workloads.

Each has positioning; choose by reliability, throughput, durability needs.

Pub/sub pattern.

• Source publishes event.
• Broker notifies all subscribers.
• Subscribers process independently.
• Adding new subscriber requires no source change.

This loose coupling is the headline benefit.

Queue pattern.

• Source publishes to queue.
• One consumer processes each message.
• Load-balanced if multiple consumers.
• Message preserved until processed.

For ordered, reliable, single-consumer scenarios.

Event schemas.

• Defined schema — known structure.
• Versioned — schemas evolve.
• Documented — consumers understand.

CloudEvents is a common standardised format; preferred for new designs.

Producer responsibilities.

• Emit reliably — even if downstream busy.
• Include context — IDs, timestamps, source identification.
• Maintain schema — version changes communicated.
• Idempotency hints — duplicate detection support.

Consumer responsibilities.

• Subscribe to relevant events.
• Process idempotently — handle duplicates.
• Respect ordering where needed.
• Handle failures gracefully — retry, dead-letter.

Each side has obligations to make the architecture work.

Event sourcing. A specific pattern:

• Store all state changes as events.
• Reconstruct current state by replaying events.
• Time-travel — see state at any past point.

Powerful but heavier; not always needed.

CQRS (Command Query Responsibility Segregation). Related:

• Commands (writes) and queries (reads) separate.
• Write side may use events.
• Read side optimised for queries.

For complex domains with heavy read traffic.

Sagas for distributed transactions.

• Long-running, multi-system process.
• Compensating actions if a step fails.
• Replaces 2PC (two-phase commit) for distributed.

Common pattern for cross-system workflows in Dynamics 365 ecosystem.

Idempotency. Critical:

• Events delivered at-least-once.
• Duplicates inevitable.
• Consumer must handle.

Patterns:

• Idempotency key — same event has same key.
• Inbox table — track processed events.
• State-based logic — set state, not increment.

Without idempotency, retries cause duplicates.

Event ordering.

• Strict ordering — events processed in order published.
• Out-of-order — order doesn't matter.

Strict ordering more expensive; often unnecessary for business logic.

Backpressure.

• Source produces faster than consumer can process.
• Queue grows.
• Consumer falls behind.

Handle via:

• More consumer instances — scale out.
• Consumer optimisation — faster processing.
• Producer throttling — slow source.
• Buffering with TTL — drop old.

Dead-letter handling.

• Failed messages quarantined.
• Investigation and resubmission.
• Critical for reliability.

Covered in [[message-replay-and-poison-queue-handling-for-d365]].

Observability.

• Tracing — request flow across services.
• Event metrics — published, consumed, failed.
• Latency — end-to-end timing.
• Correlation IDs — link related events.

Without observability, debugging event-driven systems is hard.

Common pitfalls.

• Tight coupling sneaking in. Consumer needs producer's internal details.
• Schema drift. Producer changes; consumers break.
• No idempotency. Duplicates cause data corruption.
• Loose ordering assumed. Order-dependent logic; bugs surface randomly.
• No dead-letter strategy. Failed events disappear.
• Synchronous assumptions in async system. Caller expects immediate result; doesn't get one.

When event-driven wins.

• Many systems consume same data.
• Sources are external / hard to call.
• Volume is high; pulling expensive.
• Loose coupling enables independent evolution.
• Real-time matters.

When request-response wins.

• Specific, well-known consumer.
• Strong consistency requirement.
• Simple query patterns.
• Small scale.

Both have place; not religious choice.

Hybrid common. Most real architectures:

• Some integrations event-driven.
• Some request-response.
• Pragmatic per integration.

Strategic positioning. Event-driven architecture is the maturing pattern for modern enterprise integration. Dynamics 365 supports it well via plug-ins, business events, and integration with Azure messaging. For organisations with multi-system landscapes, investing in event-driven patterns produces more scalable, maintainable integrations.

For architects:

• Identify where event-driven fits.
• Standardise on a broker.
• Establish schema discipline.
• Build observability.
• Document patterns for re-use.

The investment is in foundational infrastructure and patterns; the payback compounds as new integrations leverage the platform. Greenfield architectures should consider event-driven from start; existing point-to-point can refactor gradually.