Dynamics 365 and Azure OpenAI

The AI features users see in Dynamics 365 — Copilot in Sales, Service, Finance, Supply Chain — are powered by Azure OpenAI. Behind the friendly Copilot UX, GPT-4 and successor models process requests, generate responses, and reason about business data. Understanding Azure OpenAI's role helps both with using existing AI features and building custom ones.

Azure OpenAI overview.

• Microsoft's hosted version of OpenAI's models.
• Available models: GPT-4, GPT-4 Turbo, GPT-4o, GPT-4.5, embeddings models, image models.
• Runs in Azure regions.
• Enterprise governance — private networking, content filtering, data residency.
• Same underlying models as OpenAI's API but with Microsoft compliance and integration.

How Dynamics uses it.

• Copilot in Sales — opportunity summarisation, email drafting.
• Copilot in Customer Service — case summaries, response drafting, knowledge search.
• Copilot in Finance — variance narratives, exception explanations.
• Copilot in Supply Chain — exception management, recommendation explanations.
• Customer Insights — copilot-assisted segment creation.
• Document AI — invoice and document processing.

Each feature uses Azure OpenAI behind the scenes with appropriate context.

Grounding. Critical for accurate AI in Dynamics:

• AI prompted with relevant Dynamics data.
• "Summarise this opportunity" includes the opportunity data.
• "Draft a response" includes case context.
• Avoids hallucination.

Without grounding, AI generates plausible but wrong responses.

Retrieval-augmented generation (RAG).

• Question received.
• Search relevant context (records, KB articles).
• Pass context + question to LLM.
• LLM generates response from context.

Standard pattern; most Copilot features use it.

Privacy and data handling.

• Customer data NOT used for model training (Microsoft commitment).
• Data stays in tenant for the request.
• Encrypted in transit and at rest.
• Logged briefly for abuse prevention.

The data handling is a key part of why enterprises trust Azure OpenAI over consumer ChatGPT.

Content filtering.

• Built-in safety filters — hate speech, violence, sexual content.
• Adjustable severity.
• Custom filters possible.

Ensures appropriate output for business contexts.

Custom AI scenarios. Beyond built-in Copilot, organisations build custom:

• AI Builder integrates Azure OpenAI for prompts.
• Power Automate + Azure OpenAI for custom AI flows.
• Custom apps call Azure OpenAI directly.
• Plug-ins can invoke AI logic.

Common custom scenarios.

• Summarisation — long documents, meeting transcripts.
• Categorisation — classify support tickets.
• Sentiment analysis — customer feedback.
• Generation — draft proposals, emails, reports.
• Translation — multilingual content.
• Extraction — pull structured data from unstructured.

Prompt engineering. The art of getting useful output:

• Clear instructions.
• Examples (few-shot learning).
• Constraints — format, length, tone.
• Role-setting — "You are an expert at..."
• Step-by-step reasoning for complex tasks.

Production prompts are tested, refined, versioned.

Token economics. Pricing model:

• Per-token — input + output tokens.
• Prices vary by model (GPT-4 more expensive than GPT-4o).
• Volume matters — large enterprises consume significant tokens.

For high-volume AI features, cost modelling essential.

Latency.

• Sub-second for short prompts.
• Seconds for longer.
• Streaming reduces perceived latency.

UX accommodates AI latency through loading states or streamed responses.

Function calling / tool use. Modern feature:

• LLM can invoke defined functions.
• "Tell me about this customer" → LLM calls getCustomer(id) → uses result.
• Enables LLM to access data and take actions.

Copilot for Dynamics uses tool calling extensively.

Model selection.

• GPT-4 — most capable; expensive; slower.
• GPT-4o — capable; faster; cheaper.
• GPT-3.5 — limited reasoning; very cheap; very fast.

Match model to task; not everything needs top model.

Fine-tuning.

• Customise models with your data (limited).
• For specific terminology, style.
• Most Dynamics use cases don't need fine-tuning; prompting suffices.

Embeddings. Separate model type:

• Convert text to vectors.
• Vectors searchable by similarity.
• Foundation for RAG.

Azure OpenAI provides embedding models.

Vector databases. Pair with embeddings:

• Azure AI Search — Microsoft's vector-capable search.
• Cosmos DB with vector support.
• Specialty vector DBs — Pinecone, Weaviate.

For semantic search and RAG patterns.

Common pitfalls.

• Hallucination unchecked. AI outputs not verified; wrong information propagated.
• No prompt versioning. Tweaks made; can't roll back.
• Cost surprises. High volume; unexpected bills.
• Sensitive data in prompts. Compliance issues.
• No human review. AI outputs treated as gospel.
• Prompt injection vulnerable. User input overrides instructions.

Best practices.

• Always verify critical outputs.
• Version prompts with code.
• Monitor token consumption.
• Sanitise inputs.
• Trust user inputs as data, not instructions — prompt injection mitigation.
• Test extensively before production.
• Human-in-the-loop for high-stakes decisions.

Audit and compliance.

• AI-generated content tagged.
• Audit log of AI invocations.
• Output review.
• Compliance with sector regulations.

Healthcare, financial services have specific AI compliance requirements emerging.

Strategic positioning. Azure OpenAI is the AI foundation across Microsoft cloud, including Dynamics 365. For organisations on Microsoft, it's the natural choice for both built-in AI features and custom AI scenarios.

For decision-makers:

• Use built-in Copilot features.
• Build custom AI where business value exists.
• Govern AI use thoughtfully.
• Invest in prompt engineering capability.
• Monitor cost and output quality.

AI is increasingly integrated into business operations. The organisations that use it intentionally — with verification, governance, and quality control — gain advantage. Those that use it carelessly create reputational and operational risk. Treat AI as powerful but fallible; build process around it.