Polling vs push patterns for Dynamics 365 integrations

Integrating Dynamics 365 with another system requires choosing how data flows: polling (consumer asks periodically) or push (source notifies on change). Each pattern has different trade-offs in freshness, complexity, and resource consumption. Most production architectures use both for different scenarios.

Polling pattern.

• Consumer periodically queries source.
• "Has anything new since last check?"
• Source returns delta or nothing.
• Consumer processes any new data.

Simple to implement; well-understood; works with any data source.

Push pattern.

• Source notifies consumer on event.
• Consumer processes immediately.
• No periodic queries.

More efficient at scale; requires source support for notifications.

Polling characteristics.

• Simple — basic HTTP/SQL queries.
• Predictable load on source — fixed query frequency.
• Latency — bounded by poll interval.
• Self-healing — if consumer down, catches up on resumption.
• No source coupling — consumer's choice.

Push characteristics.

• Lower latency — near-instant.
• Lower overhead — no wasted polls when nothing changed.
• Source coupling — source must support notification.
• Recovery complex — if consumer down, missed events need replay.
• Burst-y load — high spikes at busy times.

When polling wins.

• Source doesn't push — no notification API.
• Latency tolerance — minutes acceptable.
• Simple integration — minimise complexity.
• Batch processing — process accumulated data efficiently.
• Predictable resource usage preferred.

When push wins.

• Real-time needs — seconds matter.
• High volume with low change rate — polling wastes resources.
• Source supports webhooks / events.
• Pubsub patterns — multiple consumers.

Polling implementations for Dynamics.

• OData with modifiedon filter — $filter=modifiedon gt '2026-11-01T...'.
• Change tracking — Dynamics's incremental change endpoint.
• Custom delta endpoint — implementation in plug-in.
• Database polling — direct SQL queries (sometimes for F&O).

Push implementations for Dynamics.

• Webhooks — Dataverse webhook step.
• Service Bus — Dataverse → Service Bus → consumer.
• Event Grid — Dataverse → Event Grid → consumers.
• Power Automate — flow triggered on Dataverse change.
• Business events — F&O business events.

Polling parameters.

• Frequency — how often to query.
• Batch size — records per poll.
• Filter — what to retrieve.
• Retry on failure.

Frequency vs freshness trade-off. Faster polling = more current but more load.

Adaptive polling. Smarter polling:

• High activity — poll more frequently.
• Low activity — back off.

Reduces wasted polls during quiet periods.

Long polling. Hybrid:

• Consumer makes request.
• Source holds the connection.
• Source responds when data available or timeout.
• Consumer immediately requests again.

Combines polling simplicity with push-like latency.

Webhooks. A push pattern:

• Source sends HTTP POST to consumer on event.
• Consumer accepts; processes.
• Source typically expects 200 quickly; further processing async on consumer side.

Webhook reliability.

• Retries on failure — limited.
• No replay typically — lost events lost.
• Dead letter if used (Service Bus / Event Grid).

For critical events, webhook-only is risky; broker-mediated push (Service Bus) is more reliable.

Hybrid approaches.

• Push for real-time — immediate.
• Polling for catch-up — periodic reconciliation.
• Both for redundancy — robust against either failing.

Belt-and-suspenders; common for mission-critical integrations.

Reconciliation polling. Combined pattern:

• Push for normal flow.
• Periodic polling to catch any missed events.
• Reconcile differences.

This pattern handles webhook reliability gaps.

Performance considerations.

• Polling load — depends on frequency and source efficiency.
• Push load — depends on event volume.

For high-change-rate systems, push usually more efficient. For low-change-rate, polling can be cheaper.

Throttling and limits.

• Dynamics APIs — rate-limited.
• Polling too frequently — hits throttling.
• Push — limits at broker level.

Respect throttling; honour Retry-After headers.

Monitoring.

• Polling — track query frequency, returned counts.
• Push — track events emitted vs received.
• End-to-end latency — both patterns.
• Lag / backlog — accumulation.

Visibility regardless of pattern.

Choosing in practice.

• Real-time integration? → push.
• Source pushes available? → push.
• Batch-acceptable latency? → polling.
• Mission-critical reliability? → hybrid.
• Simple integration? → polling.
• High volume? → push usually.

Common pitfalls.

• Polling too frequently — throttled; expensive.
• Polling too rarely — stale data.
• No idempotency — duplicates from retries.
• Webhook without retry strategy — events lost.
• No monitoring — pattern silently failing.
• Hard-coded URLs — webhook URLs change; integration breaks.

Migration patterns.

• Polling → push when source adds notification support.
• Push → polling when broker overhead exceeds benefit.

Architecture not religious; adapt as systems evolve.

Strategic positioning. Polling vs push is a core integration architecture decision. Both have valid use; understand trade-offs per integration. Default to push for real-time and high-volume scenarios; polling for simpler, batch-acceptable cases; hybrid for critical reliability.

For architects:

• Make conscious choice per integration.
• Document the rationale.
• Build observability for chosen pattern.
• Re-evaluate as systems evolve.

The pattern choice affects user experience, infrastructure cost, and operational complexity. Choose with care; don't default to one without consideration.